Wednesday, 2015-11-25

*** kzaitsev_mb has joined #openstack-app-catalog00:08
*** rhagarty_ has quit IRC00:09
*** rhagarty has joined #openstack-app-catalog00:11
*** kzaitsev_mb has quit IRC00:14
lifelessdocaedo: hi sorry, unexpected guests turned up00:16
lifelessare you still around?00:16
docaedolifeless: no problem, and yep, I'm still around00:16
*** rhagarty has quit IRC00:17
*** rhagarty has joined #openstack-app-catalog00:17
lifelessdocaedo: cool00:17
lifelessdocaedo: so - is there some documentation around on the things reviewers look for in adding artifacts?00:17
lifelesse.g. license, protocol, security, etc  ?00:18
lifelessor is it all tribal knowledge at the moment?00:18
docaedoIt's tribal knowledge at the moment, and nothing much is set in stone.00:19
lifelessso since that is what I committed to reviewing00:19
lifelessperhaps I should just write down something that makes sense to me00:19
lifelessand let the reviewers tell me what they do differently ?00:19
docaedothat would be excellent00:19
lifelesswhere would be the best place to put that?00:19
lifelessin-tree in docs/ ?00:19
lifelessI'll work something up- cheers00:20
docaedothanks a bunch man!00:20
*** kzaitsev_mb has joined #openstack-app-catalog00:33
docaedoI had a good conversation today with some folks working on TOSCA and the TOSCA-Heat translator.  If schedules line up, I think that topic will be on our agenda for the first meeting in december :)00:36
*** kzaitsev_mb has quit IRC00:38
*** rhagarty_ has joined #openstack-app-catalog00:45
*** rhagarty has quit IRC00:47
*** kzaitsev_mb has joined #openstack-app-catalog00:53
*** kzaitsev_mb has quit IRC01:03
*** kzaitsev_mb has joined #openstack-app-catalog01:30
*** kzaitsev_mb has quit IRC01:36
*** kzaitsev_mb has joined #openstack-app-catalog11:05
*** openstackgerrit has quit IRC14:06
*** openstackgerrit has joined #openstack-app-catalog14:07
*** kebray has joined #openstack-app-catalog16:14
*** kzaitsev_mb has quit IRC17:41
kfox1111docaedo: ideally, the compress will generate everything needed, and apache itself won't have to write anything.18:35
kfox1111which is why I was a little puzzled why it needed write permissions.18:36
*** kebray has quit IRC18:36
docaedokfox1111: ah ok - then I need to confirm that compress is trying to write to the correct place first - maybe if compress has been called first, the wsgi process won't try to write to that directory18:47
docaedoI'll start messing with that right now, suspect today will be a less the full-length day ;)18:49
kfox1111yeah, thats what I'm guessing. somehow horizon's doing it slightly differently though. in horizon, if you set offline compression to true, it will just complain if one of the files don't exist at runtime.18:51
kfox1111yeah, its nice and quiet here too.18:51
docaedois STATICFILES_DIRS generated from whatever STATIC_ROOT is?18:54
kfox1111its built in openstack_catalog/static_settings.py18:55
* docaedo happened to be looking there right this moment :)18:55
kfox1111its stuff to be pulled in to the static dir by the system.18:56
kfox1111so, mostly the dependencies.18:56
kfox1111then, if you look at something like: cat openstack_catalog/templates/_scripts.html18:57
kfox1111the compress js section,18:57
kfox1111manage.compress will read in every file in the section, merge them all to 1, json minify it, and then put in a <script src='somehash.js'> in its place.18:58
docaedoyeah not what I need to look at :) I realize the issue of sorts is that is being run without the overrides the WSGI gets from, so it is trying to write the compress output to the git clone dir18:58
kfox1111so the browser doesn't have to pull in several big files, just one smaller one.18:58
kfox1111so is the getting the openstack_catalog dir in preference to the one in the python path?18:59
kfox1111I guess you could write out into the git checkout instead of the python path. the pip install happens after the is written, it would be copied along to the right place too.19:00
docaedothe problem with that is there's potential for git conflicts later (or maybe the vcsrepo module can get unhappy?) so infra folks say not to touch stuff in that dir to ensure future git pulls work reliably19:01
kfox1111hmm... I'd assert we should never checkin a, but yeah, I can see that argument.19:02
kfox1111you could copy into the installed directory. that might cause it to find the right path too.19:03
kfox1111hmm... no, it would want to be in the parent directory...19:05
docaedobut expects to be one directory above openstack_catalog19:06
kfox1111well, its a venv right?19:06
kfox1111if so, it won't ever conflict with another, so its probably fine to throw down there.19:06
docaedohow can we tell to pull in vars from oh I think can be told to get things from env vars...19:06
docaedoand no, it's not in a venv19:07
kfox1111it will pull in openstack_catalog/, and will pull in local_settings.py19:07
kfox1111how were you trying to call the 'python compress' ? did you include the same pythonpath from the wsgi settings?19:08
kfox1111oh... and this may be part of it... horizon does the following commands outside of apache:19:10
docaedowell openstack_catalog is installed in system, so just using system paths everywhere. But is being called from /opt/apps_site19:10
kfox1111/usr/bin/python /usr/share/openstack-dashboard/ collectstatic --noinput19:10
kfox1111/usr/bin/python /usr/share/openstack-dashboard/ compress --force19:10
kfox1111where /opt/apps_site is the git checkout?19:10
docaedoseems like the collectstatic command is working19:10
docaedoyes - here's what happens:19:11
kfox1111mkdir /opt/apps_mgmt; cp /opt/apps_mgmt?19:11
docaedoclone app-catalog to /opt/apps_site; cd /opt/apps_site; pip install .; the copy from puppet templates dir to installed directory19:12
kfox1111hmm... command name doesn't really matter either, so we could just cp /opt/apps_site_manage.py19:14
kfox1111then run that out of /opt.19:14
docaedolet me try something, we can when calling collectstatic include --pythonpath OR --settings to point to an additional path or specific settings module19:15
kfox1111the problem is, python can only find one version of a module. it can't have subcomponents in the same module in a different place.19:16
kfox1111so by moving somewhere else, it gets it to stop looking at local for the module first.19:17
docaedooh wait, let me see if/how they solved this with askbot, which is a django site hosted by infra :)19:17
* docaedo thinks the answers are here:
* kfox1111 nodes they are using virtualenv's...19:22
docaedoYou sure about the virtual environments?19:23
kfox1111pretty sure. :)19:23
kfox1111I'm not seeing a manage anywhere.19:24
kfox1111in examples though, they have this:19:24
docaedooh haha damn, I am looking at a clone from several months ago locally!19:25
kfox1111which looks like it may be a script hook somwhere.19:25
docaedoBUT the venvs doesn't matter really for what we're running into19:25
docaedothe version I was looking at previously was calling compress, but latest version, nope :/19:26
kfox1111sure. just kind of funny. every openstack project seems to suffer from "no, do it this way" when they do it "wrong" themselves. :/19:27
docaedoyeah I know. I try not to think about that...19:27
kfox1111I'm guessing the call is in that other script it mentions.19:27
docaedothey have a file in templates that gets put into the install dir, so we could do the same, and copy wherever we want, and call it with a pointer at (which would have whatever "production" values we want)19:28
docaedobasically that's the same thing we're trying to do with local_settings.py19:29
docaedowhich is cleaner IMO, and requires fewer updates to the infra puppet module19:29
docaedo(vs. having to remember if we make any changes to, they have to also be reflected in puppet module template copy)19:29
kfox1111yeah. so lets just try copying to /opt/ and call it from there. I think that would probably work. it will find the right openstack_catalog file from the system path then.19:30
docaedowill try that19:34
docaedo642 static files copied to '/usr/local/lib/python2.7/dist-packages/openstack_catalog/web/static'.19:36
kfox1111that looks about right. :)19:36
kfox1111ah... I think compression is not requred if debug is still on.19:38
kfox1111we had forgotten to turn off DEBUG last we talked too.19:38
docaedoyeah that's a new/different issue though19:39
docaedowhen I turn off debug in local_settings, I get bad gateway after the puppet run19:39
kfox1111yeah, but if debug was false, I think we would have noticed the apache trying to do the compression thing earlier.19:39
kfox1111bad gateway?19:39
docaedodid not dig in yet to see why, but kept switching back and forth and was able to reproduce, so I wanted to get this one out of the way first19:39
kfox1111bad gateway.... well, wsgi runs in a seperate process. if the process died for some reason, apache might print that...19:40
docaedolet me put the stuff into the puppet manifest and try running that, confirm it does all the things we expect it should do19:40
docaedorunning compress, I get this "Compressing... No handlers could be found for logger "scss.ast""19:55
docaedois that nothing to worry about? Or does that mean it didn't actually do anything?19:55
kfox1111I see that too. never tracked down why yet. seems benign.19:55
docaedo(and the other thing worked - I copied to the openstack_catalog install directory for the sake of keeping it clean, and not having to include a new path in the manifest - seems to work!19:56
kfox1111I think its a python logging configuration thing.19:56
kfox1111yeah, cause its not going to find openstack_catalog inside that dir, and so then searches the python path and finds it, I think.19:56
kfox1111seems like a good solution though.19:57
docaedoso with debug false in I get "Bad Request (400)" for / .. BUT /api/v1/assets works :)19:59
docaedono errors thrown when hitting / .. just the 40019:59
docaedoso weird19:59
kfox1111is it working now with debug = true and the python dir owned by not www-data?20:00
docaedooh dunno, lemme see ownership20:00
docaedoinstalled dir (/usr/local/lib/python2.7/dist-packages/openstack_catalog) is owned by root, but it's always been like that, the one that needed apache ownership was web/static/CACHE20:01
docaedoLet me try dropping that part, and see if calling collectstatic creates the directory as expected20:01
*** kzaitsev_mb has joined #openstack-app-catalog20:03
kfox1111ah. cause I think compress puts all its stuff in web/static/CACHE20:05
kfox1111so if it tries to rebuild things at runtime, it would do it there.20:05
kfox1111collectstatic I think writes lib/* to static. I think CACHE is only generated by compress.20:06
kzaitsev_mbkfox1111: you seem right
kzaitsev_mbhello btw )20:08
kfox1111hi. :)20:08
kzaitsev_mbsorry for disappearing so sudden =)20:08
kzaitsev_mbmy wife and I — we planned to spend previous week in Paris and I planned to take laptop with me, but then our plans changed =/20:09
kfox1111no worries. :)20:11
kfox1111I've been working on a keystone cluster for work, so I've been kind of awol as well.20:12
docaedokzaitsev_mb: hope the change of plans still included a vacation but without being tied to work :)\20:14
kfox1111+1 :)20:15
docaedooh that was a weird smiley, fatfingered it, should have just been ")20:15
kfox1111ah... systemd vs everyone else flairs up again. :)20:15
docaedodamnit, hands are too cold today! :) :) :)20:15
kfox1111vi vs emacs was getting too old. ;)20:15
kfox1111(xkcd put it to rest too. :)20:16
kzaitsev_mbthere's always relevant xkcd =)20:16
kzaitsev_mband yep, we went to lisbon with no luggage instead, but overall it was a stressfull week )20:17
docaedooh sorry to hear that20:17
docaedoI have been banging on this puppet manifest for a bit - trying to make it work right from a clean start - when we first switched, jim blair had to log on to hosting box and manually run pip install one time before everything worked (then seems to work fine after that)20:19
docaedothing is, I suspect "works fine after" is an illusion that is going to burn us eventually20:19
docaedoso I'm making mods to the puppet manifest, then pip uninstall openstack_catalog and rm -rf the installed directory .. dependencies (in puppet) fail first pass, work second time20:20
docaedojust wanted to let you know I'm poking at that, before I can even move on to compress working right, and disabling debug :)20:20
* docaedo is hating computers again today :)20:21
kfox1111yeah. always better to get the burn down case workign right.20:23
kfox1111one of the things I don't like about config management systems like puppet/chef/etc. they allow you too easily to get into a state where you can keep it working, but you can't easily know its safe to come back up from bare hardware.20:23
kfox1111you can add a lot of testing framework on top to help, but its a lot of effort.20:24
docaedotrue, but what's the alternative? I would rather be pushed regularly to make this work from a clean instance than have it be 99% puppet plus "log in as root and do these two things if that other thing happens"20:32
kfox1111I tend to use heat templates these days. using the cloud 'stand up a new one, then roll it into place' methedology rather then keep patching the pet model.20:37
kfox1111you know you can always stand up a new one, since you do it every time. :)20:37
kfox1111as a bonus, you can autorecover from failed hardware easily. :)20:38
openstackgerritKirill Zaitsev proposed openstack/app-catalog-ui: Add reno for RElease NOtes
kfox1111kzaitsev_mb: interesting patch. is it overkill though? Its generating man pages and tex docs?21:13
kfox1111doesn't pbr generate a changelog already?21:14
kzaitsev_mbkfox1111: everything is moving to reno for release notes. I'm kind of making a bunch of similar patches for murano. They're going to be automatically published on doc.o.o21:15
kzaitsev_mb here's an example =)21:16
kfox1111ah. so they are doing it for html support?21:16
kzaitsev_mbI guess so21:16
kfox1111that makes sense then.21:17
openstackgerritKirill Zaitsev proposed openstack/app-catalog-ui: Add reno for RElease NOtes
kzaitsev_mbWhat do you mean by current release? like nova's liberty?21:22
kfox1111like, app-catalog-ui release 1.0.0 is officially released. so there should be a release note about it.21:24
kfox1111we should be updating things in release notes every time we do a release?21:24
kzaitsev_mboh you mean just inlude a note about the release? =)21:24
kzaitsev_mbno reno doesn't work like that =)21:24
kzaitsev_mbpip install reno21:25
kzaitsev_mbreno new slug21:25
kzaitsev_mbthis would create a new note file releasenotes/note/slug-{uid}.yaml with a bunch of info in it )21:25
kzaitsev_mbyou attach it to the commit and all the files are later combined in a giant release-notes html.21:26
kfox1111so for each nontrivial commit, you add a releasenotes/note/foo-uid with the change info in it?21:27
kzaitsev_mbthis should make cherry-picking important fixes easy as they would also contain notes about what wa cherry-picked, that would be included into the respective branch21:27
kzaitsev_mbkfox1111: yeah, that's how I understood how that would work21:27
kfox1111that makes sense.21:27
kfox1111so we still should maybe add a 1.0.0 note just so it isn't saying everything's unreleased.21:28
kzaitsev_mbI can dig up a couple of letters from release team. =) Nobody really reads those ))21:28
kfox1111maybe we have a 1.0 series, a 1.2 series, etc.21:29
kfox1111I'd read them. :) the upgrade notes in nova looks very useful.21:29
kzaitsev_mboh, Andreas pointed, that there should be some way around, when the repo does not contain a stable branch21:29
kzaitsev_mbI should dig into that )21:30
*** openstackgerrit has quit IRC21:36
*** openstackgerrit has joined #openstack-app-catalog21:37
*** kebray has joined #openstack-app-catalog22:03
*** kebray has quit IRC22:05
*** kebray has joined #openstack-app-catalog22:07
kfox1111I just vented about the instance-user's thing on its spec. hopefully the nova folks will do something. :/23:05
kfox1111I'm kind of fed up with it though. :/23:05
*** kzaitsev_mb has quit IRC23:09
docaedoI could understand your frustration23:09
docaedoI just finally circled back (well, for like the last hour+) to the puppet manifest stuff and sorted out the dependencies and stuff, I think I have things working now except for that debug = false giving me a 400 for /23:10
kfox1111nice. :)23:10
*** kzaitsev_mb has joined #openstack-app-catalog23:10
kfox1111kzaitsev_mb: just saw:
kfox1111which makes sense after seeing your review. :)23:24
docaedoThat's the infra patch - leaves debug on for now because TBH I'm stumped .. will keep picking away at it though over the next few days23:26
kzaitsev_mbI'm not sure if he's doing it right though. They all would end up in unreleased, as far as I understand. Although. maybe if they cherry-pick the change to stable/liberty — that would move the notes there... hm, gotta check this out a bit more23:27
kfox1111docaedo: does the compress need a --force?23:28
kfox1111I'm guessing there will be a new stable/mitaka added on release.23:29
kfox1111docaedo: k. I'm really not sure why debug off would cause that. I'm kind of curious if the wsgi processes are crashing in that mode.23:29
kfox1111maybe do a ps ax | grep wsgi before and after a test to see if any of them go away/get new pids?23:30
docaedoweird thing there is that is IS working, /api/v1/assets serves properly23:31
kfox1111yeah, I'm guessing thats because its going through flask and not django.23:32
kfox1111flask should ignore the debug thing entirely.23:32
docaedoah right, I hadn't considered that23:32
kfox1111docaedo: other then maybe needing a --force, the review looks good to me.23:36
kfox1111hmm... wait..23:37
kfox1111collect static and compress depend only on manage,23:38
kfox1111manage is dependent on ${root_dir}/ for source.23:38
kfox1111I'm not seeing it ever rerun them on js/css changes, just when changes?23:39
docaedoyeah I think .. we probably need to run that every time the repo changes, which means we also need to pip install every time the repo changes otherwise we won't catch an updates23:45
docaedoso pip install should include --update, and "file { "${install_dir}/"" should have subscribe instead of require23:45
docaedothanks for looking at that.  just pushed up a new patch, *seems* like it does the things it's expected to do :)23:50
kfox1111better, but I think there's still an issue... let me puzzle through it.23:53
kfox1111ah. maybe not...23:54
kfox1111so is reinstalled every time install-app_catalog gets done, whic is any time a file changes.23:54
kfox1111then the exec's depend on that...23:54
kfox1111is ordering garenteed in puppet files?23:55
*** kebray has quit IRC23:56
kfox1111If so, I think the last thing is the notify on the install-app_catalog should probably be moved till durring the python-compress23:56
kfox1111otherwise the webserver might reload before the static/compressed files are ready.23:56

Generated by 2.14.0 by Marius Gedminas - find it at!