Wednesday, 2016-01-13

*** tzn has quit IRC00:10
*** tzn has joined #openstack-solar01:10
*** tzn has quit IRC01:16
*** tzn has joined #openstack-solar02:13
*** tzn has quit IRC02:19
*** tzn has joined #openstack-solar04:16
*** tzn has quit IRC04:20
*** tzn has joined #openstack-solar05:17
*** tzn has quit IRC05:22
*** tzn has joined #openstack-solar06:18
*** tzn has quit IRC06:22
*** tzn has joined #openstack-solar07:19
*** tzn has quit IRC07:23
*** tzn has joined #openstack-solar07:47
*** dshulyak_ has joined #openstack-solar08:05
*** tzn has quit IRC08:07
openstackgerritDmitry Shulyak proposed openstack/solar: Implement db based lock mechanism for orchestration  https://review.openstack.org/25912708:23
*** salmon_ has joined #openstack-solar08:30
pigmejhello :)08:35
pigmejdshulyak_: you made config values optional?08:43
dshulyak_hi08:43
dshulyak_yes08:44
pigmejk,08:52
pigmej+2 :)08:52
pigmejsalmon_: would be cool if you can check it, we should switch to it *asap*08:52
openstackgerritDmitry Shulyak proposed openstack/solar: init script for solar-celery  https://review.openstack.org/26632109:03
dshulyak_i think at the same time we need switch to init script09:04
pigmejyeah09:08
dshulyak_should i leave riak in in /.solar_config_override ? or it will be better to use sqlite everywhere?09:14
pigmejhmm, I think our override in vagrant should be "big" by default09:17
pigmejbut we should also test this "small" version too09:18
pigmejmaybe we should have 2 overrides ? _big and _small and then we copy/move/link one to /.solar_config_override ?09:18
openstackgerritDmitry Shulyak proposed openstack/solar: Remove redis and celery container, use init script for celery  https://review.openstack.org/26681009:35
*** tzn has joined #openstack-solar09:38
pigmejdshulyak_: can you also start preparing docs that will describe "small" and "big" celery (sql vs redis or whatever?)09:39
*** tzn has quit IRC09:39
dshulyak_?09:46
dshulyak_:)09:46
dshulyak_what u want to see in this doc?09:46
pigmejhow to switch celery to redis09:47
pigmejor just note that it's possible, isn't it worth it ?09:47
pigmejthe same we should do for solar_db09:47
pigmejor what do you think ?09:55
dshulyak_i can add couple of lines, it will be really short09:59
pigmejyeah09:59
pigmejfor us it's obvious that we support it, but maybe for someone it will be not that obvious :)10:00
*** tzn has joined #openstack-solar10:01
tznfolks, please update https://etherpad.openstack.org/p/solar-release-0.1.010:14
tznwith status10:14
tznDummy tracking for now10:15
pigmejdone10:18
openstackgerritJedrzej Nowak proposed openstack/solar: Added possibility to change computable input func  https://review.openstack.org/26647511:22
pigmejsalmon_: updated11:23
tznthanks!11:24
openstackgerritJedrzej Nowak proposed openstack/solar: Added possibility to change computable input func  https://review.openstack.org/26647511:27
openstackgerritJedrzej Nowak proposed openstack/solar: Improved inputs documentation  https://review.openstack.org/26685511:38
pigmejok I'm starting split then...11:41
openstackgerritDmitry Shulyak proposed openstack/solar: Remove redis and celery container, use init script for celery  https://review.openstack.org/26681011:45
dshulyak_please review this last patch - https://review.openstack.org/#/c/266810/12:05
dshulyak_i think we dont even need to upload new box12:05
dshulyak_vagrant destroy solar-dev / up solar-dev is enough12:05
openstackgerritJedrzej Nowak proposed openstack/solar: Moved examples, resources and templates  https://review.openstack.org/26687112:09
pigmejdshulyak_: salmon_ please review this split https://review.openstack.org/#/c/266871/12:10
dshulyak_https://review.openstack.org/#/c/266871/1/README.md enumeration :)12:11
pigmej?12:12
dshulyak_check basic flow12:12
pigmejah it was there forever, markdown takes care about it12:12
pigmejdshulyak_: for me everythign works, but I may miss something.12:12
dshulyak_yes, i see that it was there forever12:13
dshulyak_i didnt expect that github markdown will take care of that12:13
pigmejI /just/ changed directory name for examples/templates/resources12:13
dshulyak_yesyes12:13
pigmejIt does handle it :)12:14
pigmejhttps://github.com/Mirantis/solar#solar-usage :)12:14
pigmejdshulyak_: I checking your12:19
pigmejsalmon_: hmm isn't vagrant always work from vagrant user?12:29
salmon_it uses sudo there so I'm not sure :)12:31
pigmejfor this roles sudo is not used as I see12:32
pigmejwell, let's add owner=vagrant for additional security ;D12:32
openstackgerritJedrzej Nowak proposed openstack/solar: Moved examples, resources and templates  https://review.openstack.org/26687112:33
pigmejdone :)12:33
pigmejsalmon_: dshulyak_ I added owner=vagrant to that git, just for ensuring,12:33
salmon_this is why was just asking :P12:33
pigmejsalmon_: well for me it was vagrant BUT you added owner to repositories for some reason, and it was there required, so probably we have some use case when it's needed12:34
salmon_repositories were owned by root for sure12:35
pigmejon vbox ?12:37
salmon_yup12:38
salmon_pigmej: dshulyak_merging split12:39
pigmejyeah it conflicts some things, but it will be easier to fix smaller things..12:40
pigmejI hope nothing will explode :D12:40
openstackgerritMerged openstack/solar: Moved examples, resources and templates  https://review.openstack.org/26687112:42
pigmejYaY12:42
pigmej:)12:42
pigmejdshulyak_: your patch is not working for me out of the box12:43
dshulyak_what do you mean?12:44
pigmejforever pending state12:44
dshulyak_i think you are not using .config_overrdie12:45
pigmejI'm using defaults12:45
pigmejand when I spawned celery by hand12:45
pigmejeverything worked12:45
dshulyak_yesyes12:45
dshulyak_there is config_override12:46
dshulyak_variable12:46
pigmej?12:46
pigmejI destroyed env12:46
pigmejI created new12:46
pigmejthen I executed examples and they were forever pending12:46
pigmejwhen I spawned celery by hand they started working12:46
dshulyak_https://github.com/openstack/solar/blob/master/bootstrap/playbooks/solar.yaml#L46-L4712:47
dshulyak_solar-celery is running under vagrant12:47
dshulyak_maybe we should add this line to /root/.bashrc as well12:48
pigmejhmm12:48
pigmejor execute celery from vagrant user12:48
dshulyak_so leave it for vagrant only or add for root also?12:49
pigmejI would prefer to have everythign set under vagrant12:50
pigmejand somehow make celery working from vagrant user / or just exprot this override variable, isn't it good idea ?12:50
pigmejdshulyak_: besides that, it's ok :)12:55
openstackgerritJedrzej Nowak proposed openstack/solar: Added possibility to change computable input func  https://review.openstack.org/26647512:57
dshulyak_pigmej: sorry, i didnt get the last part, so it works for you under vagrant user? or no?12:57
pigmejunder vagrant it works12:58
pigmejwhen I spawn celery by hand12:58
pigmej(except that we don't have host verification turned off for vagrant user)12:58
dshulyak_my irc client prints messages in random order...13:00
pigmej;D13:01
dshulyak_pigmej: i think it should work without spawning celery by hand13:01
dshulyak_with vagrant user13:01
dshulyak_it doesnt?13:01
pigmejyes it works13:01
pigmejBUT we don't have hosts verification turned off for vagrant user13:01
salmon_yup, only for root13:05
dshulyak_i will add same ssh config for vagrant13:06
openstackgerritDmitry Shulyak proposed openstack/solar: Implement db based lock mechanism for orchestration  https://review.openstack.org/25912713:06
openstackgerritDmitry Shulyak proposed openstack/solar: init script for solar-celery  https://review.openstack.org/26632113:06
openstackgerritDmitry Shulyak proposed openstack/solar: Remove redis and celery container, use init script for celery  https://review.openstack.org/26681013:06
dshulyak_thats rebase13:07
pigmejwhy the heck vagrant can't destroy only one box ?13:09
pigmejah typo :[13:13
dshulyak_i think it can13:14
dshulyak_yes, vagrant destroy solar-dev wont affect others13:15
pigmejyeah13:15
pigmejdshulyak_: it's still not working for me (celery is still started as root from init-scripts)13:20
dshulyak_pigmej: yes, sorry, it is started from root, is it a problem?13:21
dshulyak_cli should be used from vagrant, because SOLAR_CONFIG_OVERRIDE is expored there13:22
pigmejdshulyak_: but as I said, I have forever pending state13:22
pigmejby default, when testing your patch13:22
dshulyak_env | grep solar ?13:22
pigmejempty13:22
dshulyak_let me recheck13:22
dshulyak_hm13:23
dshulyak_maybe i exported it by hand, by i think this line should work https://github.com/openstack/solar/blob/master/bootstrap/playbooks/solar.yaml#L46-L4713:23
openstackgerritJedrzej Nowak proposed openstack/solar: Removed wrong git option from solar-resources clone  https://review.openstack.org/26690713:26
pigmejsalmon_: so ^ file is owned by vagrant because it's in shared folder13:26
pigmejand git has no 'owner' parameter in our ansible (newer ansible has, be warned)13:27
dshulyak_msg: unsupported parameter for module: owner13:27
dshulyak_y13:27
pigmejdshulyak_: ^'13:27
pigmejdshulyak_: about that celery, as I said, it's ok for vagrant user, but by default you run it as root, and we don't have it exported there13:27
pigmejso as a result something is bad13:28
dshulyak_for root it is exported in init script13:28
dshulyak_actually im totally confused13:29
pigmejthen I will destroy everything and check again13:29
pigmejme t oo13:29
salmon_pigmej: I thought you tested it :P13:29
dshulyak_u are using vagrant user from cli or root?13:29
pigmejsalmon_: I did, but on my local machine... I have new ansible there it seems13:29
pigmejdshulyak_: so again, from cli I spawn celery as vagrant by hand=> everything works13:29
dshulyak_no13:29
pigmejbut when I use your init scripts I have forever pending state13:29
dshulyak_not celery13:29
dshulyak_cli13:29
dshulyak_soalr cli13:30
pigmejvagrant13:30
pigmejah, vagrant user doesn't have permission to this DB13:30
dshulyak_not sqlite should be used13:31
dshulyak_but riak13:31
pigmejyeah BUT celery uses sqlite13:32
dshulyak_because13:32
dshulyak_there is not SOLAR_CONFIG_OVERRIDE exported13:32
dshulyak_and celery that is started by init script uses riak13:32
pigmejwhich is owned by root, so my vagrant user (CLI) doesn't have access to it ?13:32
pigmejhow the heck celery uses riak ?13:33
pigmejah13:33
dshulyak_thats the part that needs to be fixed13:33
pigmejI think it's other way13:33
dshulyak_i dont know why vagrant user doesnt have this exported - export SOLAR_CONFIG_OVERRIDE="/.solar_config_override"13:33
dshulyak_no13:33
pigmejah13:33
dshulyak_https://review.openstack.org/#/c/266321/3/utils/solar-celery L1313:34
pigmejbecause it's not anymore in bashrc13:34
dshulyak_hm, why?13:34
pigmejprobably something went wrong ;D13:34
pigmejcan you both merge my broken 'owner' ?13:35
pigmejbut I still don't get why I have no export in bashrc...13:35
pigmejok, dshulyak_ I exported by hand13:36
pigmejand the same result13:36
pigmejvagrant@solar-dev:/vagrant$ echo $SOLAR_CONFIG_OVERRIDE13:36
pigmej/.solar_config_override13:36
pigmejI think the reason is: vagrant@solar-dev:/vagrant$ ls -all /tmp/celery.db13:36
pigmej-rw-r--r-- 1 root root 7168 Jan 13 14:18 /tmp/celery.db13:36
salmon_pigmej: your change is waiting for jenkins13:37
pigmejah...13:37
pigmej:(13:37
pigmejdshulyak_: yup, I cahnged ownership of /tmp/celery.db and it's fine now13:38
dshulyak_yeah, it is because of /tmp/celery.db13:38
pigmejso the problem is that, celery silents error messages from sqlite or sth13:38
pigmejbecause vagrant has no access to /tmp/celery.db :)13:38
pigmejdshulyak_: so what we do with it?13:48
dshulyak_i am trying to start celery from vagrant user13:49
pigmejk13:50
openstackgerritMerged openstack/solar: Removed wrong git option from solar-resources clone  https://review.openstack.org/26690713:52
pigmejsalmon_: you need to adjust https://review.openstack.org/#/c/266360/13:59
openstackgerritJedrzej Nowak proposed openstack/solar: Improved inputs documentation  https://review.openstack.org/26685514:00
salmon_yu[14:01
salmon_yup14:01
pigmejthough I have no idea14:05
pigmejhow14:05
salmon_dshulyak_: can you review https://review.openstack.org/#/c/266475/414:43
openstackgerritDmitry Shulyak proposed openstack/solar: Implement db based lock mechanism for orchestration  https://review.openstack.org/25912714:48
openstackgerritDmitry Shulyak proposed openstack/solar: init script for solar-celery  https://review.openstack.org/26632114:48
openstackgerritDmitry Shulyak proposed openstack/solar: Remove redis and celery container, use init script for celery  https://review.openstack.org/26681014:48
dshulyak_updated https://review.openstack.org/#/c/266810/ to use vagrant user14:50
dshulyak_sec, forgot to remove container14:50
dshulyak_from docker-compose14:51
tznGuys, the guys from solar project removed it from github and readthedocs14:51
tznwhat now14:51
dshulyak_i think it doesnt make sense because all of this mounts are vagrant specific14:51
dshulyak_hm, i will remove docker specific things in separate patch, i guess Dockerfile/run.sh and part of docker compose can be rmeoved14:52
openstackgerritMerged openstack/solar: Added possibility to change computable input func  https://review.openstack.org/26647514:52
dshulyak_please recheck this one https://review.openstack.org/#/c/266810/14:56
tznhttp://solar.readthedocs.org/en/latest/15:00
pigmejtzn: cool :)15:02
pigmejthen we can build documentation automaticaly :)15:02
pigmejgithub gives us nothing ;) we're using openstack/solar still15:02
pigmejor you meant pypi tzn ?15:02
tzn?15:03
tznyou meant where?15:03
tznI mean - what you are asking for really?15:03
pigmej"<tzn> Guys, the guys from solar project removed it from github and readthedocs"15:03
tznaaa, let me check pypi15:04
tznit's still there, but 40415:05
tznsearch brings that up15:05
tznbut no content15:05
pigmejthen it should be possible to register that name afair15:06
pigmejbecause search index is updated in async way15:07
pigmejdshulyak_: where od you use vagrant user ? I don't see it15:07
pigmejdo*15:08
tznhttps://pypi.python.org/pypi?%3Aaction=pkg_edit&name=solar15:09
dshulyak_pigmej: you asked me about user?15:10
dshulyak_because irc shows that that messages was from tzn15:10
dshulyak_it is in celery_init.yaml15:10
tzn@dshulyak_ what is your PyPi username?15:11
pigmejdshulyak_: what client do you use ?D:15:11
dshulyak_tzn: dshulyak15:12
dshulyak_pigmej: colloquy, it is osx15:13
pigmejanwnyay, I dislike that we have init script but we will start celery from other script as different user, while init-script starts it as root,15:13
dshulyak_but previously it was ok15:13
pigmejI'm not sure but maybe adding permission to celery.db would be ok?15:13
dshulyak_pigmej: what do you mean?15:13
dshulyak_init script starts it with whatever user you are15:13
pigmejIt will be pita during development15:13
pigmejdshulyak_: almost, but you add it to defaults, so on reboot it will be again owned by root15:14
pigmejisn't it?15:15
dshulyak_i dont know ;)15:15
dshulyak_will it be even started after reboot?15:15
salmon_why not /var/lib/solar/celery.db ?15:15
dshulyak_mainly because solar.db was lready in tmp )15:16
salmon_ah :)15:16
pigmejdshulyak_: you add it to defaults so yeah I suppose it will be ...15:16
dshulyak_pigmej: i checked, after reboot it wont be started15:17
pigmejok, then I don't understand ubuntu init system....15:17
dshulyak_idk, maybe thats another system responsibility15:17
pigmejyeah ubuntu upstart (which is finally dead btw)15:18
pigmejok salmon_ can you check this PR taht dima mentioned ?15:18
pigmejdshulyak_: any ideas wtf https://review.openstack.org/#/c/266855/ ?15:18
pigmejok it's in conflict now15:19
dshulyak_pigmej: it says there is conflict15:19
dshulyak_oh15:19
dshulyak_my client15:19
pigmej:D15:19
salmon_pigmej: which one?15:21
openstackgerritJedrzej Nowak proposed openstack/solar: Improved inputs documentation  https://review.openstack.org/26685515:22
pigmejsalmon_: https://review.openstack.org/#/c/266810/15:23
pigmejdshulyak_: are you doing something more with it ?15:23
dshulyak_pigmej: hm, actually doc says that update.rc should be enough15:24
dshulyak_so it might that i messed up smth15:24
dshulyak_no, i think we can use it as it is15:24
dshulyak_and if i will figure out how to restart it properly15:24
dshulyak_not restart - start on boot properly15:24
dshulyak_i will submit new patch15:24
pigmejk15:25
pigmejdshulyak_: https://review.openstack.org/#/c/266810/4/bootstrap/playbooks/tasks/celery_init.yaml@1015:27
openstackgerritDmitry Shulyak proposed openstack/solar: Remove redis and celery container, use init script for celery  https://review.openstack.org/26681015:29
pigmejdshulyak_: for some reason I expected "crap" there :D15:34
pigmejespecially "oh crap" :P15:34
dshulyak_i found better init script for celery - https://github.com/celery/celery/blob/3.1/extra/generic-init.d/celeryd15:51
dshulyak_it needs to be customized a bit15:52
pigmejyeah15:53
pigmejcan you salmon_ and dshulyak_ check https://review.openstack.org/#/c/266855/ and https://review.openstack.org/#/c/266255/15:54
pigmej?15:54
salmon_pigmej: why do you need a lock there?16:18
pigmejsalmon_: where ?16:20
salmon_https://review.openstack.org/#/c/266255/1/solar/core/handlers/__init__.py16:20
pigmej(hint these are 2 different PR, they differ by 600 :P)16:22
pigmejbecause of gevent16:22
pigmejand theading too16:22
pigmejI don't want to force stevedore to load something in concurrent environment16:23
pigmejit uses __import__ magic + entrypoints..16:23
pigmejalso I wanted to load one handler max one time16:23
*** dshulyak_ has quit IRC16:26
*** dshulyak_ has joined #openstack-solar16:28
dshulyak_i would expect that with gevent it is safe16:40
pigmejit's safe now :)16:42
dshulyak_no, i mean what the purpose of this lock16:42
pigmejyeah16:42
pigmejah16:42
pigmejwell, stevedore takes time to load,16:43
dshulyak_there is no way one gthread will switch control16:43
dshulyak_to another one randomly, e.g wo io/sleep16:43
pigmejit is, stevedore uses stevedore file16:43
dshulyak_hm16:43
pigmejdshulyak_: stevedore uses pkg_resources, and pkg_resources reads entry_points file16:50
pigmejtherefore it *can* switch there (small chances but it's possible)16:50
dshulyak_so this lock should protect from reading/loading same driver twice or more times?16:52
dshulyak_in case if several tasks will be started concurrently and drivers wont be imported16:52
pigmejyup16:54
pigmejthat's the only case for that16:54
pigmejprobably nothing bad would happend if we would load it several times, but ... it's faster this way16:54
pigmejthat's why I have this global _handlers dict16:55
pigmejI'm using the same method there https://github.com/openstack/solar/blob/master/solar/computable_inputs/processor.py#L29-L3616:55
*** dshulyak_ has quit IRC16:58
*** dshulyak_ has joined #openstack-solar17:04
*** dshulyak_ has quit IRC17:10
pigmejok, cu for today :) See you tomorrow :)17:11
openstackgerritLukasz Oles proposed openstack/solar: Add worpdress example from tutorial  https://review.openstack.org/26636017:20
openstackgerritLukasz Oles proposed openstack/solar: Add test for wordpress example  https://review.openstack.org/26636017:23
pigmejsalmon_: how did you solve examples / resources stuff ?17:37
salmon_not yet, just cleaning my git repo :)17:38
pigmejthen set it workflow -1 please :)17:38
*** tzn has quit IRC17:52
*** tzn has joined #openstack-solar17:54
*** tzn has quit IRC17:58
*** ikalnitsky has quit IRC18:08
*** evgenyl has quit IRC18:08
*** evgenyl has joined #openstack-solar18:21
*** ikalnitsky has joined #openstack-solar18:31
*** dshulyak_ has joined #openstack-solar18:56
*** dshulyak_ has quit IRC19:03
*** dshulyak_ has joined #openstack-solar19:15
*** dshulyak_ has quit IRC19:35
openstackgerritMerged openstack/solar: Improved inputs documentation  https://review.openstack.org/26685521:22
openstackgerritLukasz Oles proposed openstack/solar: Allow to modify computable inputs in Composer files  https://review.openstack.org/26719322:02
openstackgerritMerged openstack/solar: Implement db based lock mechanism for orchestration  https://review.openstack.org/25912722:11
openstackgerritMerged openstack/solar: init script for solar-celery  https://review.openstack.org/26632122:13
openstackgerritMerged openstack/solar: Remove redis and celery container, use init script for celery  https://review.openstack.org/26681022:17
*** salmon_ has quit IRC22:28
*** tzn has joined #openstack-solar22:53
*** tzn has quit IRC22:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!