Wednesday, 2015-08-26

sdakerhallisey almost done reviewing your patch00:00
rhalliseyk00:00
sdakejenkins out to lucnh00:01
*** achanda has quit IRC00:01
*** alisonh has joined #kolla00:03
sdakeok comments are done rhallisey00:05
sdakewhen is a new version coming ;)00:05
*** dims_ has joined #kolla00:06
rhalliseytmr maybe00:06
sdakeyou going to commit to doing the whole source base ;)00:07
*** dims has quit IRC00:07
rhalliseyya I can do the whole thing00:08
rhalliseyjust want to have a template done00:08
sdakelets get the job done on one of them00:08
sdakewe can possibly dsitribute the wor around00:08
sdaketo speed things up00:08
sdakethe reference implementation of config.py needs love00:08
sdakei'd like to see the json as a list rather then dictionary00:08
sdakeif that is possible00:09
rhalliseykk00:10
sdakethe goal is to remove files00:10
sdakeand remove cruft00:11
sdakethat is why we are doing this00:11
sdakethe side benefit is multiple customized config files00:11
sdakepleasa try to address all my comments :)00:11
sdakeuse click "done" in the review when done with each one :)00:11
sdakei'd really like our contributors to start responding to reviewers comments00:12
sdakeits hard to know what got fixed00:12
sdakebrb00:12
*** dims has joined #kolla00:12
*** sdake_ has joined #kolla00:13
*** blahRus has quit IRC00:13
*** dims_ has quit IRC00:15
*** sdake has quit IRC00:16
rhalliseysdake_, kk00:17
sdake_amazing kolla queue down to 1 page00:18
rhalliseyya00:18
*** ccrouch1 has quit IRC00:20
*** weiyu_ has joined #kolla00:23
*** diogogmt has joined #kolla00:23
*** openstack has joined #kolla00:35
sdake_which part dont you understand the bindmount ?00:35
sdake_take nova-compute as an eaple00:36
sdake_ /opt/kolla/configs/nova-compute/nova.conf is bindmounted00:36
sdake_with /opt/kolla/config.json00:36
sdake_ /opt/kolla/configs/cnfig.json would have tobe bindmounted00:36
sdake_basically it needs to end in the same directory as nova-compute on start of the nova-compute container on the host00:36
*** alisonh has joined #kolla00:38
rhalliseysdake_, I still don't understand your point.  When will this fail?00:38
sdake_if you remove config.json from the dockerfile where does it com from00:39
sdake_i really don't want config.json in the dockerfile00:39
*** diogogmt has quit IRC00:39
rhalliseyok that's your point00:39
rhalliseygotcha00:39
*** diogogmt has joined #kolla00:40
*** sdake has joined #kolla00:56
*** sdake_ has quit IRC00:59
*** weiyu_ has quit IRC01:08
Kennanhi kolla guys01:27
rhalliseyKennan, hello01:27
sdakesup knenan01:27
KennanI tried found a ubuntu and updated kernel to 3.1901:27
KennanI build from source01:27
Kennantools/build.py --base ubuntu --type source --template -T 3501:28
Kennanas this01:28
Kennanand found01:28
sdakeright01:28
KennanERROR:__main__:cinder-ba Failed with status: parent_error01:28
KennanERROR:__main__:cinder-ap Failed with status: parent_error01:28
KennanERROR:__main__:cinder-vo Failed with status: parent_error01:28
KennanERROR:__main__:cinder-ba Failed with status: error01:28
KennanERROR:__main__:kolla-ans Failed with status: error01:28
KennanERROR:__main__:cinder-sc Failed with status: parent_error01:28
sdakethat works according to SamYaple  :)01:28
Kennanthose images failed to build01:28
sdakeyes i ontice cinder is failing to build as well01:28
sdakei htink someone broke it01:28
sdakejust ignor eit, cinder is off by default01:28
sdakeyou need kolla-ansible01:28
Kennanok. anyway, let me try verify it other containers work or not01:29
sdaketry adding kolla-ansible to your build command01:29
sdakethat should get it done01:29
sdakeyou absolutely must have kolla-ansible01:29
Kennanhi sadke: not sure where to add it ?01:30
*** erkules has joined #kolla01:30
Kennanhi sdake: for kolla-ansible01:31
Kennanwhat's it for build images? not clear about it01:32
Kennanit seems ./tools/kolla-ansible deploy is for deploy containers01:32
sdaketools/build.py --base ubuntu --type source --template -T 35 kolla_ansible01:32
sdakethat will rebuild kolla ansible i think ;)01:32
*** dolpher has joined #kolla01:32
sdakerun with -T 101:32
sdakeas well so you can see what fialed01:32
sdakeand paste to a pastebin the log of the fialure01:32
sdakeupstream may be busted01:33
sdakesam been jamming patches in at a fast rate01:33
sdakei notice our gate i failing now too on from source builds01:33
*** erkules_ has quit IRC01:33
Kennanok sdake: let me try clean old images and build again01:34
Kennanwill try use your instructions here01:34
Kennantools/build.py --base ubuntu --type source --template -T 35 kolla_ansible01:34
sdakedont clean old images01:34
sdakekeep your images01:34
sdakejust reuibd kolla_ansible01:34
sdakethat is one rason we added the regex building feature01:35
Kennanok01:35
KennanINFO:__main__:Successfully built images01:35
KennanINFO:__main__:=========================01:35
KennanINFO:__main__:Images that failed to build01:35
KennanINFO:__main__:===========================01:35
Kennanis kolla_ansible right here? seems not any images have such name01:36
sdaketry \*ansible\*01:37
Kennanoh, seems all invalid expression01:38
Kennan raise error, v # invalid expression01:39
Kennanit seesm kolla-ansible is OK01:42
Kennannot kolla_ansible :001:42
sdakerun sudo docker images | grep ansible and paste results01:42
*** weiyu_ has joined #kolla01:42
sdakei don't know regex very well unfortuantely01:42
sdakemaybe you an google how to do aw ildcard for *ansible* :)01:43
*** zhiwei has joined #kolla01:44
Kennanhi sdake: I have found the issue, seems code bug01:49
KennanI would send bug fix if other not fix it01:49
sdakewe take git reviews :)01:50
sdakeplease file a bug first01:50
sdakeand uses Closes-Bug: #XYZ at end of commit message01:50
Kennansure :)01:50
*** Haomeng has joined #kolla01:56
*** Haomeng|2 has quit IRC01:59
openstackgerritKai Qiang Wu(Kennan) proposed stackforge/kolla: Fix the wrong instruction for pip  https://review.openstack.org/21692402:02
sdakekennan thanks for the contribution bro!02:04
Kennanit is my pleasure sdake:02:04
*** unicell has quit IRC02:08
*** achanda has joined #kolla02:23
*** jasonsb has quit IRC02:28
*** loth has quit IRC02:28
*** ccrouch has joined #kolla02:37
*** achanda has quit IRC02:38
*** dims has quit IRC02:46
*** klint has joined #kolla02:52
SamYaplehey sdake02:55
sdakeyosam02:55
SamYapledid you see my build.py patch?02:55
SamYapleshould make everyone stop complaining02:55
sdakenot sure i did02:56
sdakesay rhallisey  hada good idea for config..json worth a review02:56
sdakehis code is pretty trrible :)02:56
sdakejust be warned02:56
SamYaplehttps://review.openstack.org/#/c/216691/02:56
SamYaplesdake: yea i looked at that review02:56
SamYaplei thought I was handling that issue in RC102:56
SamYaplewe would also need to tweak common.sh to make it compatile until all images are converted02:57
SamYaplegood start though02:57
sdakeyou are samyaple02:57
sdakei think he is concerned it isn't done today right now :)02:57
SamYapleanyway check out the review i linked02:58
sdakeya let me finish with slideware02:58
sdakealmost done02:58
SamYaplealso makes builds run 20% faster on my machine!02:58
sdakei looked at it briefly looked like alot of change02:58
sdakeand went "meh" :)02:58
sdakewhat preisely doe it fix02:58
SamYapleit is, had to rewrite a big chunk of code to remove the build tiering system02:58
SamYaplebasically each parent knows about its children now and will add its children to the work queue once it has been processed02:59
sdakeso how do you uild base ahead of somethig nelse?02:59
SamYaplerather than a tiering system02:59
SamYaple^^02:59
sdakeas long as base builds ahead of osmethin else wfm02:59
sdakebut the gate is failing02:59
sdakethere is a patch from kennan which may or may not fix it02:59
SamYaplei see the queue initially with images who's parent is None02:59
SamYaplewhat do you mean the gate is failing?03:00
sdakecool let me dirnk a red bull and i'll have a look :)03:00
sdakesource gate is not building03:00
SamYapleit failed on cinder for that patch03:00
sdakecinder fials to bulid03:00
sdakefor which patch03:00
SamYapleyea not related to this patch though03:00
SamYaplehttps://review.openstack.org/#/c/216691/03:00
SamYapleremove tiering03:00
sdakei saw that03:00
sdakei mean which cinder patch is busted03:00
SamYapleyea03:01
SamYaplei have teh setup-gate working!03:01
SamYaplegoing to push a final patch and recheck 5-10 times03:01
sdakerecheck doen't work if it passes the gate I dont think03:01
SamYapleit does03:01
*** rhallisey has quit IRC03:02
SamYaplethe build tiering patch also cleans up logs fixes regex and properly fixes the test_build.py03:02
SamYaplerather than ignoring the issue like the regex patch did03:02
openstackgerritMerged stackforge/kolla: Fix the wrong instruction for pip  https://review.openstack.org/21692403:03
sdakeis heat going to somehow magically land on my desk before 8am :)03:03
SamYapleits going to land in the review queue03:04
SamYaplesend me a sinple heat template btw for me to test03:04
sdakein working shape?03:04
SamYaplelike a brownbag of commands03:04
Kennanhi guys. I noticed the change in /etc/kolla/nova-compute/nova.conf manully can not help. as every time run anisble deploy would override it. any suggestions?03:04
SamYapleyea03:04
sdakein the demos directory03:04
SamYapleKennan: update it from the deploy host03:04
sdakekennan that should not be the case03:04
SamYaplesdake: it is03:04
SamYaplehe is talking about editing the target host03:05
sdakeya don't edit the target host03:05
sdakeedit the deploy host03:05
Kennanhi, I used all-in-one case03:05
sdakein that case editing that file should be fine03:05
Kennanand I changed /etc/kolla/  to use qemu03:05
SamYapleno sdake it wont03:05
sdakedid you examine the internals of the container?03:05
SamYapleKennan: if you want a permanat change /etc/kolla/config/nova/nova-compute.conf03:05
sdakei am pretty sure that is what file i said03:05
sdakekennan but if its not my apologies03:05
Kennanno issue. But could you guys tell me how to edit that to use qemu ?03:06
KennanI did not have kvm enabled in VM03:06
sdakeModify /etc/kolla/nova/nova-compute.conf03:06
sdakeAdd the override:03:06
sdake[libvirt]03:06
sdakevirt-type=qemu03:06
sdakeitmay be virt_type03:06
sdakeneed to check the intranets03:06
Kennansdake: it can not work. as rm containers, and deploy again, it would back to old03:07
sdakevirt_type03:07
Kennanso the virt_type can not be kept03:07
sdakekennan that is not how it is supposed to work03:07
sdake /etc/kolla/nova/nova-compute.conf right?03:08
SamYaple/etc/kolla/config/nova/nova-compute.conf03:08
Kennansdake: it is03:08
SamYaplesdake: wrong file03:08
Kennanit is /etc/kolla/nova-compute/nova.conf03:08
SamYaple/etc/kolla/config/nova/nova-compute.conf03:08
sdakei thought /etc/kolla/nova-compute/nova.conf was the default override file?03:08
SamYapleno, never has been03:08
SamYaple/etc/kolla/config/nova/nova-compute.conf03:08
Kennanok SamYaple03:09
Kennanlet me try that03:09
sdakegroan that is confusing03:09
SamYapleonly on AIO03:09
SamYaplewe went over thisw03:09
Kennanhi03:09
Kennan-rw-r--r-- 1 root root 0 Aug 26 02:03 /etc/kolla/config/nova/nova-compute.conf03:09
Kennan?03:09
SamYapleKennan: correct03:09
Kennanit is zero size03:09
SamYapleyou put your override settings in there03:09
Kennanok03:09
sdakekennan my apologies dude03:09
sdakestill learning my way around the syste m;-)03:09
Kennanno issue :) sdake:03:10
openstackgerritSam Yaple proposed stackforge/kolla: Setup the gate for cross-distro support  https://review.openstack.org/21339703:10
*** jasonsb has joined #kolla03:13
openstackgerritKai Qiang Wu(Kennan) proposed stackforge/kolla: Fix doc for sudo case  https://review.openstack.org/21694103:17
SamYaplesdake: both centos and fedora are working for builds in both rax and hpcloud servers!03:18
SamYapleproperly using the dedicated ephemeral disk as well03:19
KennanSamYaple: is it build from source work ?03:19
Kennanfor binary, i am not sure(as I hit issue yesterday)03:20
SamYapleKennan: no this is for the gating system for commits. We are setting up the gates to do ubuntu and centos binary and source installs03:20
KennanOK SamYaple: just want to learn what's gate system like, for example, all is ubuntu OS ? or03:21
SamYapleright now we have 3 build gates, 1 fedora (binary only) 2 centos (binary and source)03:22
SamYapleonce my patch merges we will have another ubuntu source gate03:22
KennanGood to know, what's public cloud os look like ? I mean when hpcloud or rax allocate servers03:22
Kennanis it VM or bare metal?03:23
SamYaplevm03:23
SamYapleyou can go launch them by hand in the differnet clouds03:23
SamYaplethey are pretty vanilla images03:23
KennanOk SamYaple: final question, what's OS for the VM ? and version (I choose ubuntu 14.04 with 3.19 kernel as you suggested :)03:23
KennanI failed on centos 7.1 before03:24
SamYaplefor the gate?03:24
Kennanyes03:24
SamYapleubuntu 14.04 3.13 kernel for the ubuntu builds, centos 7 3.10 kernel for centos builds03:24
SamYapleI cant control the kernel in the gate03:25
Kennanok. SamYaple: I remember ubuntu 14.04 3.13 kernel need with btrfs for docker ?03:25
SamYapleyes03:25
SamYapleor a statically built docker03:26
Kennanseems btrfs needs some manual configuraion in docker side, perhap I need read your code to learn more details03:26
SamYapleKennan: yea, but just normal you can do `docker daemon -s btrfs` and it would work03:26
Kennanoh? I thought it would have a separate disk and resizefs with btrfs03:27
sdakewhat does --base-tag do?03:27
SamYapleKennan: it does03:27
SamYaplejust after yo uset that up03:27
SamYaplesdake: https://github.com/stackforge/kolla/blob/master/docker_templates/base/Dockerfile.j2#L103:28
Kennanok SamYaple: if I understand your point clearly, you need have related disk resizefs with btrfs, and then can run with docker daemon -s btrfs03:28
Kennanright?03:28
SamYapleKennan: yes03:28
SamYapleformat disk to btrfs, mount at /var/lib/docker, start docker with `docker daemon -s btrfs`03:29
Kennanok. I would check your gate code, and try to understand how you did that to make ubuntu 14.04 with 3.13 kernel work03:29
KennanSamYaple: did you have such link for gate code your changes?03:29
SamYaplei do03:30
SamYaplehttps://review.openstack.org/#/c/213397/29/tests/setup_centos.sh03:30
Kennanthanks03:30
*** achanda has joined #kolla03:30
*** achanda has quit IRC03:35
*** diga has joined #kolla03:40
*** unicell has joined #kolla03:52
openstackgerritZhou Zheng Sheng proposed stackforge/kolla: Speedup Git Cloning Ansible Repository  https://review.openstack.org/21694803:54
*** unicell has quit IRC04:07
*** unicell has joined #kolla04:07
*** chandankumar has joined #kolla04:22
*** weiyu_ has quit IRC04:30
*** ashishb has joined #kolla04:30
*** ashishb has quit IRC04:45
*** chandankumar has quit IRC04:54
*** achanda has joined #kolla05:01
*** weiyu_ has joined #kolla05:21
*** dims has joined #kolla05:36
*** dims has quit IRC05:42
*** weiyu_ has quit IRC05:46
openstackgerritMerged stackforge/kolla: Speedup Git Cloning Ansible Repository  https://review.openstack.org/21694806:00
*** weiyu has joined #kolla06:01
openstackgerritSam Yaple proposed stackforge/kolla: Add Ansible support for Heat  https://review.openstack.org/20552806:04
mandreyo kolla06:23
sdakeyo mandre06:23
sdakenice to see all the new contributors ;-)06:24
sdakeand users as well06:24
sdakewe finally made something useful ;)06:24
mandrethe chan got really busy while i was out06:25
mandreamazing06:25
*** jpeeler has quit IRC06:26
mandrethe review queue is almost empty, what's up with SamYaple?06:26
*** vern has left #kolla06:29
*** chandankumar has joined #kolla06:30
SamYapleyea what sup with me?06:37
SamYaplewhat did i do?06:37
sdakemandre you have a whoel new code base to learn - better get busy ;-)06:38
SamYapleI feel like I have done something wrong. What did I do wrong mandre?06:38
*** achanda has quit IRC06:40
mandreSamYaple: you used to fill the review queue with patches06:46
SamYapleoh it was filled yesterday06:46
SamYapleits just empty while i do things for sdake06:47
SamYaplemandre: https://review.openstack.org/#/c/216691/ https://review.openstack.org/21339706:47
SamYapleboth of those are ready to go06:47
SamYapleand could use serious review06:47
SamYaplethey are gate affecting things06:47
*** mandre has quit IRC06:47
sdakewe need unit tests for build.py06:49
SamYaplesdake: agreed, not in that patch though06:50
SamYaplelarge enough laready06:50
sdakeya todo for later06:50
*** jpeeler has joined #kolla06:53
SamYaplesdake: https://review.openstack.org/#/c/216691/ https://review.openstack.org/21339706:55
SamYapleboth need reviews06:55
sdakei just reviewed one o fthem06:55
sdakehte build.py one06:56
sdakeboy that was hard to reiew06:56
sdakeone small nitpick06:56
*** inc0 has joined #kolla06:56
sdakewhy is cinder failing to build from source06:56
* sdake groans06:56
SamYaplesdake: you broke it a while ago06:57
SamYaplei told you that06:57
sdakehrm must have missed that06:57
SamYaplethe sed lines you added in are dangerous _and_ broke source06:58
*** dolpher has quit IRC06:59
inc0good morning07:00
sdakesamyaple reviewed07:00
inc0SamYaple, now I can calmly review your code, sorry for waiting07:00
sdakeso on dangerous side lets table that discussion until tomorrow ;)07:00
sdakeon breaking from source - bummer :(07:01
SamYapleinc0: thanks man!07:01
sdakesamyaple my understanding is kernel is resonsible for device # allocation not udev07:01
SamYaplei added your code suggestion, even though the while loop is way more effiecent, your code is more readable07:01
inc0I'm a wedding planner, so afternoons are everything but calm for me;)07:01
sdakeudev just creates the devices as directed by the kernel after device allocation occurs07:01
SamYaplesdake: i am aware and ill explain what its broken and bad07:02
sdakeinc0 you serously plan weddings?07:02
SamYaplebut not right now07:02
inc0sdake, just one07:02
inc0my own07:02
sdakeya we got more important shit to do SamYaple07:02
sdakelets argue about it later ;)07:02
sdakeinc0 my consolences!07:03
inc0haha07:03
SamYaplesdake: its not an agruement, its a requirement ill point you to the breakdown of the out-of-sync udev stuff07:03
sdakecool well can we discuss it tomorrow07:03
sdakei rally need to stay focused on what i'm doing :)07:03
sdakepromise i'll have all time in world for you tomorrow for this topic07:04
inc0I'm glad you've added tests Sam, thanks07:05
SamYapleinc0: ive also made it so you can use regex proper like "^base$" or "keystone|^cinder-api$"07:07
inc0but normal build.py neutron works?07:08
sdakeinc0 on eway to find out :)07:08
SamYapleinc0: correct07:08
inc0I'll give it a try once I'm done with code review;)07:08
*** mandre has joined #kolla07:10
mandresorry SamYaple, my vps decided to kill itself07:11
mandrei'll do your reviews07:11
SamYaplethe more eyes the better07:11
SamYapleboth of them heavily affect the gate07:11
SamYapleif they are bad our gate is dead07:12
sdakeour gate is nonvoting07:12
sdakeso its not dead ;)07:12
sdakesetup_gate is only run on image and functional jobs07:12
sdakeit just votes -1 ;)07:12
SamYaplewell i mean for image building07:12
SamYaplewe need t osee if that works for commits07:12
SamYaplevoting or not07:12
sdakei could make those jobs eaily voting07:13
sdakebut i dont trust the gate enough to do so07:13
sdakewe tried that once and the gate was busted for several days07:13
sdakebecause delorean ate shit07:13
SamYaplei agree it shouldnt be voting, but if things dont build we cant confidently approve patches07:14
sdakeif we were to make the gating voting infra would need to mirror delorean07:15
inc0when I had my build patch queue, each patchset could have different FAIL/SUCCESS combination07:15
inc0while last one/middle one could have all success07:15
sdakeinc0 that is fixed by retry=307:15
sdakeas the default07:15
inc0ok07:16
*** jmccarthy has joined #kolla07:16
SamYaplesdake: https://review.openstack.org/#/c/213397/07:16
SamYapleim not sure what you want me to document07:16
SamYaplethat is clear as day, im adding a key07:17
SamYapleits not magic, its a key07:17
sdakea key to what?07:17
sdakei dont get it07:17
SamYaplethats the way yo uadd keys07:17
SamYaplesigning key07:17
sdakewhos signing key is it?07:17
SamYaplethats the way you add keys for packages to debian07:17
SamYaplefor the docker packages..07:17
SamYapleadd the repo add the key update07:17
SamYaplethats the debian way07:17
sdakeadd a comment07:18
sdaketo the review pls07:18
SamYaplei did07:19
sdakeok looking07:19
SamYaplewhat would you like me to say and where07:19
inc0SamYaple, do we have situation when we don't build base image?07:19
SamYapleinc0: building only data image i guess07:19
inc0data image is base from code standpoint07:20
SamYapleinc0: data image is based of of the distro, not the base image07:20
inc0base image == image with no parent07:20
inc0that's what I mean07:20
SamYaplethe initial images that can be built without dependacies are base and data07:20
SamYapleyup07:20
inc0I think sooner or later we will have this no-base-rebuild use case07:21
SamYaplei think you mentioned you wanted image building without parents at one point, but the cache takes care of that07:21
inc0for optimizations07:21
SamYaplethe cache handles that07:21
inc0unless parent changes07:21
SamYaplewell the nyou should be rebuilding it all07:21
inc0which happends, and we don't want to force stuff07:21
SamYaplenot just teh child07:21
SamYaplethe child wont be valid if its based off a change base07:22
sdakei am not keen to keep adding cli options to build.py ad infinitum07:22
inc0you should, but I think people should have a choice in this regard anyway07:22
sdakethe more options the more comlex07:22
SamYapleim going to have t odisagree there. changes in the base heavily affect al lthe other images07:22
sdakeneed to keep things simple07:22
SamYaplethe cache wil lallow quick/instant rebuilds07:22
SamYaplesince i did the set_time thing the cache has always hit for me07:22
inc0SamYaple, they will, but power user might want to tailor their base themselves07:23
SamYapleinc0: yea so the rebuild all the images07:23
SamYapleotherwise how would they use the base07:23
sdakehow do you build just one image with regex?07:23
inc0if you rebuild and you have your own base, this will override it right?07:23
SamYaplesdake: it biulds the image and its parents sdake07:24
sdakei get that - i mean whta is the regex operation07:24
inc0sdake, currently you're not, requirements were that this build filtered images AND their parents07:24
SamYaplesdake: build.py cinder-base07:24
sdakei thought so07:24
sdakekennan tried build.py ansible_kolla today and it failed07:24
SamYaplesdake: its kolla-ansible07:24
SamYapleor ansible-kolla07:24
sdakeoh well that would explain it07:24
SamYaplebut - not _07:24
*** dims has joined #kolla07:24
inc0or just ansible07:25
inc0we don't have more of these07:25
SamYaplethat also work07:25
inc0build script won't try to pull stuff right?07:25
sdakesamyaple heat in my future soon ? :)07:25
*** weiyu has quit IRC07:26
sdakerunnign out of midnight oil07:26
inc0so use case: you want to distribute your builds07:26
inc0one server does base, other servers does other07:26
SamYaplesdake: no, created_time is wierd07:26
inc0which would be quite cool for constant rebuilds, which people might actually like07:27
*** jmccarthy has quit IRC07:27
*** weiyu has joined #kolla07:27
SamYapleinc0: in that case we are talking about a custom build script07:27
SamYaplego for it07:27
sdakesamyaple consult asalkeld :)07:27
sdakeitso k to ask for help - god knows I ask 24/7 :)07:27
inc0SamYaple, if we'll add conditional pulling/not rebuilding, it will be doable07:27
SamYaplesdake: yea i know where the issue is, just pdb in a container is... not so good07:27
SamYapleinc0: that breaks the reliability of the image though07:28
sdakewhich part of i am not keen to add cli options to build.py wasn't clear?07:28
SamYapleif the base doesnt match it should be rebuild07:28
SamYapleand we are defintely not adding distributed building07:28
SamYaplewhat a nightmare that would be07:28
inc0ehh, I'm always for new features, sorry07:28
inc0:P07:28
*** dims has quit IRC07:29
sdakebuild.py looks feature complete from my perspective07:29
sdakelets let the community weigh in on what else they need07:29
sdakebefore we gold plate it further07:29
SamYaplesdake: we still need some git tweaks, like refcommit changes and things07:30
SamYaplebut thats options in build.ini not flags07:30
sdakeSamYaple agreed07:30
sdakevbel is working on that07:30
sdakeafter that, and possibly adding something to build stuff for plugins07:30
sdakei dont see much to add07:30
sdakewe have a mountain of work to do without refactoring build.py a million times ;)07:32
sdakealthough i like sam's lsat refactor07:33
SamYaplefinal remove the bash remnants07:34
SamYaplefinally*07:34
SamYaplefinnaly use threads and queue correctly!07:34
SamYapleno more tiers!07:34
sdakeif it came with a fix for heat you would be my hero for a day :)07:34
sdakemaybe even a week07:34
sdakethis could use reviews for ryan in the morning to gleen over https://review.openstack.org/#/c/216874/07:37
sdakeit will affect how the whole project works and our work over the next month07:37
sdakeso would appreciate wide review :)07:37
vbelgood morning/evening :)07:38
SamYaplesdake heat is almost done, just after fixing the first problem, the same issue popped up for different values :/07:39
sdakehot07:39
sdakepaste to #heat would be my next step :)07:39
mandreSamYaple: wrt cross-distro gate setup patch, what's the reason for all these LVM commands for centos and fedora?07:40
sdakemandre he is setting up lvm thin provisioning to speedu p the gating process07:41
sdakeatm lvm iuses loopbackby default on centos07:41
SamYapleyea that07:41
mandremaybe worth putting a comment in the code07:42
SamYaplealso docker 1.8.1 recommends thinprov versus direct disk07:42
SamYaplemandre: would you mind if i did that in the next patch?07:42
SamYaplethis is holding up infra mergeing the ubuntu gate07:42
SamYaplei need that to test the ubuntu code07:42
SamYapleI can add it now if you really want though, or i can promise to add it in the next patch :)07:43
mandreeven if you promise, there's still a risk you forget :)07:43
mandreubuntu doesn't need thin provisioning?07:44
SamYapleoh ye of little faith07:44
SamYaplewell for one mandre i cant test ubuntu right now :/07:44
SamYaplethis patch is holding up the gate07:44
SamYapleso i will have to patch ubuntu right after this merges to test teh ubuntu gate07:44
sdakesamyaple note the gate patches need a rebase07:44
mandresorry, i meant there is still a risk a bus run over you07:44
SamYaplejesus dude07:45
SamYaplewtf07:45
sdakemandre uses th ebus argument ;)07:45
sdakesamyaple i mean my gate patches07:45
SamYapleoh07:45
SamYaplehttps://review.openstack.org/#/c/213531/07:45
SamYaplelooks good to me07:45
mandreSamYaple: hold on a second for that PS, i had another comment07:46
*** weiyu has quit IRC07:46
SamYapleyea im patching heat bugs atm take your time07:46
mandrecommented in the review07:47
sdakeSamYaple i thought i had a docker images gate in there for centos07:47
sdakelooks like i dont07:47
sdakeor whateveryou removed inthatlast patch07:48
* sdake is bleary eyed07:48
sdakethe reason cinder from source doesn't work is lvm isn't instlled07:49
SamYaplesdake: i know07:49
SamYaplei know why the sed failed07:50
*** weiyu_ has joined #kolla07:50
sdakepretty sure cinder from source didn't work previousl y)07:50
SamYaplebut it built07:51
sdakeSamYaple what is the lvm package on ubuntu07:51
SamYaplelvm207:52
*** shardy has joined #kolla07:52
SamYapleapt-get install lvm2 should be enough07:52
SamYaplewell apt-get install -y --no-install-recommends lvm07:52
SamYaplewell apt-get install -y --no-install-recommends lvm2*07:52
sdakewhat doe --no-intall-recommends do07:55
sdakedoes07:55
SamYaplekeeps alot of unneeded dependancies out07:56
SamYapleits why the ubuntu images are less than half the size of centos07:56
*** weiyu_ has quit IRC08:06
*** jmccarthy has joined #kolla08:07
inc0SamYaple, build.py neutron builds data container as wel08:09
inc0well08:09
SamYapleinc0: no it doesnt ;)08:09
inc0Processing: data08:10
inc0I can see it.08:10
inc0right now.08:10
SamYaplethat doesnt say building does it now?08:10
SamYaplelook at the first step in the builder function08:11
SamYaplefor clarity, I can move the log position for the Processed message08:11
inc0I'd say don't log "processing" unless you are actually processing it08:11
SamYapleso it always shows up08:11
SamYaplei am processing it08:12
inc0I personally thin it's misleading08:12
SamYaplei process it and check if its matched, or if its parents are errord, or if it is source builds or not08:12
inc0at least make it debug08:12
inc0rather than info08:12
inc0and "building" would be info08:12
SamYaplethat sfair08:12
sdakeinc0 that sounds good to me08:12
SamYaplethe logging could be switched around in that whole section08:13
SamYapleright now it doesnt show "Proceeced" if it errors08:13
SamYapleonly if it builds correctly08:13
SamYaplenot really accurate08:13
openstackgerritVladislav Belogrudov proposed stackforge/kolla: Add ability to make tarballs from Git clones  https://review.openstack.org/21665508:13
inc0other than that it looks pretty solid08:14
inc0will improve multithreaded performance08:14
SamYapleyea, its about 20% faster since my threads always have things to process08:14
sdakevbel can we call this done ; https://blueprints.launchpad.net/kolla/+spec/add-proxy-to-dockerfiles08:14
SamYaplebut its mostly dependant on disk anyway08:14
inc0depends on your network08:15
inc0it can be network-dependant...trust me, I know...08:15
SamYaplewell its disk bound for me to build in 12 minutes08:15
inc0yeah proxy works with headers08:15
inc0it's something like that for me too...in home08:16
vbelsdake: yes, with small note - I need to put a few unset http_* to kolla-common.sh to clean up it so that openstack services don't go to my proxies :)08:16
inc0in my office its horrible08:16
SamYaplevbel: that sounds fine08:16
sdakevbel please submit a patch - i want to start closing out blueprints that are close08:16
SamYaplewe source that file each time anyway08:16
*** akwasnie has joined #kolla08:16
inc0vbel, add no_proxy env variable08:16
SamYaplebut the issue is that if you ever exec in teh contaienr it will have those values set08:17
SamYaplejust keep that in mind08:17
sdakewe need to document how to build with proxies08:17
sdakevbel if you want to take that on it would be appreciated as well08:17
sdakei struggled with kennan today to help him through it08:17
vbelyes, will do08:17
sdakelots of people will want to know how to do that08:17
vbelcan I use wildcard for no_proxy?08:17
inc0I'm not sure08:18
sdakefrom my experience with magnum, atleast 30% of people end up deploying behind a proxy08:18
vbelit is easier to unset things to be sure no one attempts to read them08:18
vbelno_proxy is not very popular and can be dismissed by some apps08:18
inc0vbel, mind sending a patch that will crate a placeholder?08:19
openstackgerritSteven Dake proposed stackforge/kolla: Install lvm in Cinder from-source containers  https://review.openstack.org/21702508:20
vbelinc: you mean templating kolla-common.sh? Or just unsets?08:21
vbel:)08:21
sdakeif we can get two cr's to review that would appreciate it, gate is busted as is08:21
inc0I don't think we can template kolla-common08:22
inc0but some code for unset would be nice08:22
sdakei had hoped to get rid of kolla-common.sh08:22
inc0custom unset I mean08:22
inc0or custom include08:22
vbelok08:23
sdakevbel just jam it in there for now08:23
SamYaplevbel: cant you override the docker varilables?08:24
sdakelets get a patch in the queue today pls :)08:24
SamYapleENV HTTP_PROXY ''08:24
SamYaple?08:24
SamYapleif so we can add a footer to the end of the final images (cinder-api, glance-registry, etc)08:24
sdakesamyaple he uses the include to set up the proxies08:24
SamYapleI am aware08:24
SamYapleso we can add the footer to clean up08:25
sdakekind of messy but atleast we can get rid of kolla-common.sh that way :)08:25
SamYapleyea but the common.sh thing wouldnt work anyway08:25
SamYapleif you docker exec into the image its gonna get wierd08:25
*** weiyu_ has joined #kolla08:25
sdakesamyaple what hour is give up hour for heat08:25
SamYapleidk man08:26
SamYapleits a new wierd bug now08:26
SamYapleand i dont know the code base08:26
openstackgerritMichal Jastrzebski (inc0) proposed stackforge/kolla: Templates for rsyslog  https://review.openstack.org/21518108:26
sdakevbel if we start removing /docker now without git references, is it going to damage your work?08:27
sdakesamyaple fwiw we tried ENV https_proxy=""08:29
sdakeand I dont think that worked08:29
sdakedidn't work i mean08:29
inc0sdake, where it didn't worked?08:29
sdakethe environment variable needs to be unset08:29
inc0you need several places08:29
sdakein vbel's tseting when we POCed this idea08:29
SamYaplethats not how oyu set docker env variables08:30
SamYapleENV HTTPS_PROXY ""08:30
sdakeyaa that didn't work08:30
inc0sdake, did you add it to /etc/default/docker as well?08:30
SamYapleok08:30
inc0SamYaple, https_proxy="" works well08:30
inc0ENV https...08:30
SamYapleinc0: not in a dockerfile08:30
inc0SamYaple, in dockerfile, I do it all the time08:30
sdakeadd what to e/tc/defualt/docker?08:31
SamYaplesdake just said it didnt inc008:31
sdakevbel tested it08:31
sdakei didn't test it08:31
SamYaplebut if that _does_ whats the problem wit hthe footer?08:31
sdakeso i dont know exactly what he did08:31
inc0add export https_proxy="" there08:31
SamYapleinc0: that wont do it08:31
inc0this way docker will use proxy for example for image pulling08:31
SamYaplethats not how docker does env variables08:31
inc0SamYaple, I'm doing it even now08:31
SamYapleexport will not work in the dockerfile like oyu think then08:32
inc0SamYaple, I'm doing it even now08:32
inc0without that curls and yums doesn't work, with that it does08:32
sdakeinc0 the idea is that the running container will nto use the proxy08:32
sdakebut the build proxy will08:32
sdakethis is vbels requirement08:32
inc0sdake, yeah, you need to either unset it or add hosts to no_proxy08:32
inc0I personally use no_proxy and it helps, openstack understands no_proxy08:33
sdakeunset it how?08:33
inc0let me paste you my include header file08:33
sdakesounds good08:33
sdakethe only thing we teted that worked was to ENV it in, and unset in start.sh08:34
sdakesetting to ="" didn't do the job08:34
sdakebut we didn't test ENV ""08:34
SamYapleinc0: that doesnt work like i said08:34
SamYaplei just tested it08:34
sdakewe tested start.sh export https_proxy=""08:34
inc0http://paste.openstack.org/show/428062/08:34
SamYapleinc0: now try and unset that08:35
SamYaplesdake: ENV HTTPS_PROXY ""08:36
SamYaplethat does work however08:36
SamYapleobviously export doesnt work in the dockerfile08:36
SamYaplethats my point08:36
sdakesamyaple cool that isn't quite what vbel tested08:36
vbel ENV https - if you do sudo or su you are out of luck08:36
sdakevbel tested export https_proxy="'" in _start.sh_08:36
sdakenot in the dockerfile08:37
SamYaplevbel: I beileve it is broken, but you can set it to empty was my point. i did not test if it works. i trust you :)08:37
vbelI will run tests today :)08:37
sdakeya if tis set to empty - the services think proxy_server =""08:38
vbelwill tell what works here08:38
sdakewhat there needs to be is an unset docker operation08:38
SamYaplesdake: or build env variables ;)08:38
vbelyes, people requested UNSET many times. One workaround is to squash images :)08:38
sdakeya then we wouldn't need all this hacking08:38
sdakepbourke you around?08:39
inc0UNSET will be hard as it has to be done on last container08:40
inc0OR we can do include header and include footer in build script08:40
inc0and add this to EVERY dockerfile08:41
SamYapleinc0: yea that was my suggestion08:41
inc0that would work and that would be most clean solution08:41
SamYaplebut apparent that wont work08:41
SamYaplevbel: said it still is broken08:41
inc0let me check plz08:41
pbourkesdake: morning08:42
sdakehey pbourke08:42
sdakewhats the word on swift08:42
pbourkesdake: sorry no patch yesterday08:42
vbelI would include header and footer in all templates, not just base. But for current situation I would just add "unset proxy" in common runtime script (kolla-common.sh or whatever we think appropriate)08:42
sdakedid the solution involve not rquiring ansible 2.0?08:43
pbourkesdake: we're going to leave it as is for this patch08:43
pbourkesdake: as it does work and will need to investigate a soltuion that works for 1.9 and 2.008:43
sdakecool i dont want a hard dep on ansible 2.0 in the base ansible files08:43
sdakei dont mind a hard dep on asinbile in container kolla-ansible08:43
pbourkesdake: I just was making sure the patch still is good when rebased from master08:43
pbourkesdake: and fell over a few things08:44
sdakecool well lets get it up and merged :)08:44
pbourketoday is the day08:44
SamYapleyea sdake we can still have a unique swift.conf per service but i wont hold up the patch anyfurther08:44
SamYapleill write that in later08:44
openstackgerritSteven Dake proposed stackforge/kolla: Install lvm in Cinder from-source containers  https://review.openstack.org/21702508:45
vincent_vdkhmm, can't build gnocchi-api because yum fails08:48
vincent_vdkah, packages not available08:50
sdakegnocchi has been busted forever08:53
vincent_vdk:)08:53
* sdake would +2 +a a patch to remove gnocchi from code base ;)08:53
SamYapleit biulds for me!08:54
vincent_vdkcan you skip that one when building all the containers08:54
vincent_vdknot for me08:54
SamYapleubuntu source for me08:54
vincent_vdkah08:54
vincent_vdkthis is on centos/rdo08:54
SamYapleyup08:54
SamYapledelorean is alwaays busted08:54
openstackgerritMichal Jastrzebski (inc0) proposed stackforge/kolla: Ansible bits to rsyslog  https://review.openstack.org/21506208:55
openstackgerritMichal Jastrzebski (inc0) proposed stackforge/kolla: Templates for rsyslog  https://review.openstack.org/21518108:55
openstackgerritMichal Jastrzebski (inc0) proposed stackforge/kolla: rsyslog container  https://review.openstack.org/21458908:55
sdakevincent_vdk I would support a patch to remove the yum shit from gnocchi08:57
sdakesince we have no ansible work for it08:58
sdakeand all it does atm is fail to buidl and cause confusion08:58
sdakeif someone fixes it later thats good ;)08:58
sdakei dont want non-building containers in the build08:58
sdakeits sloppy and bad for our project08:58
sdakeif we need to keep it for from source so be it08:59
vincent_vdkto be honest, i don't even know what it does08:59
*** weiyu_ has quit IRC08:59
sdakeits a time series database08:59
vincent_vdkwould have to look it up :)08:59
sdakethat is as much as I know ;)08:59
vincent_vdkah08:59
vincent_vdki don't see the use to add all these kind of things to openstack08:59
sdakei think its a backend replacement for ceilometer09:00
sdakebut I am not certain09:00
sdakewe are packaging the big tent long term09:00
sdakeso get used to it :)09:00
sdakethere are 40+ projects in the big tent atm09:01
sdakeatleaset 30 of them are clientserver apps09:01
*** inc0 has quit IRC09:01
sdakeSamYaple I just had an epiphany that json change is rquired for upgrade from liberty to mitaka09:02
*** weiyu has joined #kolla09:02
sdakesamyaple please prove me wrong09:02
SamYaplerequired?09:03
*** inc0 has joined #kolla09:05
sdakeas in how would we do upgrade from liberty to mitaka and bring json with it09:05
sdakewithout it being specified in the docker files in liberty09:06
SamYaplesdake: mitaka has new containers and playbook changes, whats the issue?09:07
*** jmccarthy has quit IRC09:08
*** jmccarthy has joined #kolla09:08
sdakethinking09:09
sdakebeen up long time09:09
*** jmccarthy has quit IRC09:13
inc0SamYaple, about https://review.openstack.org/215062 - common role by default runs on every node? or is it specified somewhere?09:14
sdakesamyaple not sure - we will need to revisit this discussion - need to set up one more centos node09:14
inc0because I'd keep rsyslog as separate logging role, as we'll expand on that - and soon09:14
sdakebrb folks time to setup another centos09:15
*** weiyu has quit IRC09:15
SamYapleinc0: no the rsyslog container goes in the common role09:16
SamYaplethe logging role will be for other things later09:16
SamYaplelike kibana eletic serach etc09:16
inc0oO09:16
inc0why not keeping it as separate role?09:16
inc0it's clean09:17
SamYaplebecause the logging container runs on all nodes...09:18
SamYapleits a requirement of all services09:18
SamYapleit goes in the common role09:18
SamYapleplease dont fight me on the ansible architecture too09:18
inc0you fight me on python, it's only fair09:19
SamYaplei wrote that python and the ansible09:19
inc0well, still, I'll challange it09:19
SamYaplego for it09:20
inc0you can always -2 my patches;)09:20
SamYapleif i cant hold up to challenges i dont have a good point09:20
inc0or ask me politely to go away and never come back;)09:20
SamYaplejeez thats horrible09:20
SamYaplei like you09:20
inc0anyway, I'm ok with common role09:20
inc0just teasing09:20
SamYaplewe may butt heads but that isnt a horrible thing man09:20
*** jmccarthy has joined #kolla09:26
openstackgerritPaul Bourke proposed stackforge/kolla: Implement Ansible role for Swift  https://review.openstack.org/21506509:27
*** akscram1 has joined #kolla09:28
*** akscram has quit IRC09:28
pbourkesdake: ^^09:28
inc0guys btw feedback from ops meetup - monitoring09:30
inc0do we want to create monitoring stack for our containers?09:30
SamYaplelike to monitor if the containers are up inc0?09:30
inc0no, like zabbix09:32
SamYapleim not familiar with that09:32
SamYapleheard the name though09:32
inc0graphite?09:32
inc0splunk?09:33
inc0that sort of stuff09:33
SamYapleyea sure i think that would be good optional addons09:33
*** weiyu_ has joined #kolla09:33
sdakei am wary of adding new blueprints at this time inc009:40
sdakebut if you want to put into discussion phase that is our typical process :)09:40
inc0well I was thinking early M rly09:40
inc0just something I've heard from folks who have been on ops summit09:41
inc0I had meeting yesterday, and monitoring stack was one of big gaps09:41
sdakepbourke does that work09:41
pbourkesdake: yes09:41
sdakepbourke what is the swift "api" port that i need to expose in my firewall for the swift cli to work09:43
pbourkesdake: 808009:44
openstackgerritMerged stackforge/kolla: Install lvm in Cinder from-source containers  https://review.openstack.org/21702509:46
sdakedo directoreis /srv/node/sdb1 and sdb2 and sdb3 need to exist on host pbourke?09:46
pbourkesdake: no that's handled by the data container09:46
pbourkesdake: the only thing you need to supply are the rings09:46
*** dims has joined #kolla09:48
*** weiyu_ has quit IRC09:54
sdakepbourke is the only thing you tried from source or does rdo binary work too?09:54
pbourkesdake: I only tried rdo binary actually09:55
sdakenice09:55
sdakemandre or samyaple can one of you please review https://review.openstack.org/#/c/215065/09:56
sdakewe can fix up problems with followup patches09:56
inc0sdake, do you sleep? like...ever?10:00
sdakeinc0 yes10:00
sdakeall next week - sleeping10:00
openstackgerritVladislav Belogrudov proposed stackforge/kolla: Add ability to make tarballs from Git clones  https://review.openstack.org/21665510:02
vbel^^^ now handles build.py links, kolla-build and tox10:04
*** jmccarthy has quit IRC10:06
*** jmccarthy1 has joined #kolla10:06
SamYaplevbel: tests fine for me10:07
SamYaplewell see what the gate has to say, but im good with it thus far10:07
*** weiyu_ has joined #kolla10:12
*** weiyu_ has quit IRC10:14
pbourkevbel: if 'source' in image and 'source' in image['source']:10:15
pbourkeis that intentional?10:15
pbourkelooks strange :)10:15
vbelyes10:15
vbelwe have source type and sometimes source is missing, e.g. for base and data10:15
vbelso one is type and another is location of source from build.ini10:16
*** Kennan2 has joined #kolla10:16
*** Kennan has quit IRC10:16
openstackgerritMichal Jastrzebski (inc0) proposed stackforge/kolla: Ansible bits to rsyslog  https://review.openstack.org/21506210:19
vbelis there online debug output for registry push in build.py?10:23
SamYaplevbel: no the push doesnt seem to be returning anything we are catching vbel10:23
SamYapleits probably looking for the wrong key in return10:23
SamYaplei dont know10:23
*** weiyu_ has joined #kolla10:24
vbelSamYaple: ok, thanks! For some reason I cannot push it - requests.packages.urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out10:24
SamYaplecan you push from the command line10:24
pbourkepush was working for me yesterday10:25
*** zhiwei has quit IRC10:25
pbourkewith appropriate tag10:25
pbourkevbel: check, port=None10:25
sdakesamyaple10:40
sdakei am ready to try multinode for first  time10:40
sdakeneed bit of handholding10:40
sdakemy AIO deploys perfectly from registry10:40
sdakedo i need to put my hostnames in /etc/hostname?10:41
*** vbel has quit IRC10:44
sdakeanyone that has deployed multinode please throw me a bone re how to configure my system10:45
sdakedo i need /etc/hostname with ips in it ?10:45
*** vbel has joined #kolla10:45
SamYaplesdake: the hosts should be able to resolve each other10:46
SamYapleso if you have server01 if you be able to ping server0210:46
SamYapleby name10:46
SamYapleif you have that setup, that should be enouhg10:46
inc0sdake, I run ansible with ips10:46
sdakeeven ping the deployment from host?10:46
inc0sdake, what would prohibit it? deployment host has to ssh to deployed one10:48
sdakeinc0 i will try ips first10:48
inc0so packages must be able to travel both ways10:48
sdakeyou use ips in the inventory file?10:48
inc0yup10:48
SamYapleinc0: are you doing multinode _kolla_ with ips in the inventory file?10:49
inc0I even done 10.10.10.[12-20] as compute nodes10:49
inc0yup SamYaple10:49
SamYaplecool10:49
inc0any problem with it?10:49
SamYaplei havent tested that glad that works10:49
SamYaplei would have though galera would break because we use hostnames there10:49
sdakegood that takes a load off -d on't have to get my resolver shit working10:49
inc0SamYaple, it might, I didn't run full HA with it10:50
inc0one controller and multiple computes10:50
inc0worth a test tho10:51
* sdake groans10:51
SamYaplesdake: youll need hostnames10:51
sdakesamyaple ack10:51
SamYaplemake sure they can all resolve each others hostnames and youre good10:51
inc0SamYaple, if Galera uses /etc/hostname on each node, that might work10:51
inc0not sure about /etc/hosts10:51
SamYapleinc0: not when server02 cant resolve server0110:51
inc0I think having hostnames are safest approach tho, yeah10:52
SamYapleput the the servers in /etc/hosts on each node and youre golden10:52
SamYapleand I only test with hostnames in the inventory, but you might get away with ips there10:52
inc0yeah, just make one big /etc/hosts and scp it everywhere10:52
SamYapleits only three nodes so i think hell be ok10:52
inc0you can even scp it with ansible;)10:52
inc0if its 1 controller and 2 computes it should work anyway10:53
inc0I didn't do multiple controllers with that10:53
openstackgerritMerged stackforge/kolla: Implement Ansible role for Swift  https://review.openstack.org/21506510:53
vbelrunning tox and Co still not enough to get thru gate. Can I run gate by hand locally?10:56
sdakesamyaple quick setup question10:57
sdakeone of my hosts is not reading /etc/hosts first10:57
SamYaplewhat do you mean?10:57
sdakeinstead its ranodmly creating ip addresses when i ping10:57
SamYaplewtf?10:57
sdakePING minime-02.broked.org (92.242.140.2) 56(84) bytes of data.10:57
sdake^C10:57
sdake--- minime-02.broked.org ping statistics ---10:58
sdake3 packets transmitted, 0 received, 100% packet loss, time 1999ms10:58
sdakeminime-02 is 192.168.1.10210:58
sdakeit works from all machines that are not deployment hosts10:58
sdakerather the hosts to deploy from10:58
SamYaplethats a fqdn name youre trying to ping though10:58
sdakebasically where I am launching ansible from10:58
sdakeping minime-02 is the command10:58
pbourkevbel: tox -e<name-of-failing-test>10:58
sdake[sdake@bigiron etc]$ ping minime-0210:58
sdakePING minime-02.broked.org (92.242.140.2) 56(84) bytes of data.10:58
SamYaplesdake: dude im kinda fried at the moment. i would guess your /etc/hosts is setup wrong10:59
SamYaplethats the first stop in name resolution10:59
sdakesamyaple i copied it directly from my 3 other working boxes..10:59
SamYapleidk man10:59
* sdake hates computers11:00
*** weiyu_ has quit IRC11:00
vbeltox does not fail locally. It just fail while gate-kolla-build-images-centos-binary11:01
sdakecooli put a short name in e/tc/hsots and its working11:01
pbourkesdake: i know that feeling11:01
sdakesamyaple do i put th short name in the inventory file or the fqdn11:02
SamYapleboth if you want11:02
SamYapleshortnames are required11:02
*** weiyu_ has joined #kolla11:02
inc0sdake, it has to be resolvable, that's all afaik11:02
sdakeok here goes nothign ;)11:03
sdakewhy isn't the inventory file in /etc btw11:03
sdakejust curiosity there11:03
inc0it's not technically config11:04
inc0it's more like input to function, same as plays11:04
sdakechange ansible_ssh_user=sam to =root?11:04
inc0if you don't have sam's account;)11:04
inc0remember to either spread ssh keys or give ansible_ssh_pass11:05
sdakei have full ssh for root an sdake accoutns11:05
sdakefrom to all machines11:05
inc0then you can either do user=root or sdake, root is shorter11:06
sdakeand have eto sudo for multinode when running playbok?11:07
inc0ansible knows how to sudo11:08
inc0afair by default, but let me check11:08
pbourkedoes anyone have build.py seem to just hang11:09
pbourkeeven with -T 1, it just sits there with no output seemingly forever11:09
sdakefatal: [minime-one] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute u'ansible_em1'", 'failed': True}11:09
pbourkesdake: did you set your interface in /etc/kolla/globals.yml11:09
sdaketo em111:09
inc0every server has this iface?11:10
inc0pbourke, works for me,11:10
sdakeem1 is present across all nodes in the cluster11:11
inc0and it's up?11:11
inc0hmm11:11
vbelSorry, user root is not allowed to execute '/bin/testr run test_build.BuildTestCentosBinaryDocker' as docker  ????11:11
sdakeyes its my management network11:11
*** jmccarthy1 has quit IRC11:12
vbelI cannot run sudo -g docker testr run test_build.BuildTestCentosBinaryDocker as myself or root, I am in docker group11:12
inc0sdake, paste full log please11:12
pbourkevbel: try without sudo?11:12
pbourkevbel: I had that issue before11:12
pbourkeremoved sudo and it worked, not sure why11:12
vbelI run tox -e11:13
vbelok, will try11:13
pbourkeremove sudo in tox.ini11:13
sdakehttp://ur1.ca/nk725 -> http://paste.fedoraproject.org/259489/4058759411:13
sdakethere was a bug filed about this11:13
sdakeI am running deploy host of f21, running target machines of centos711:13
sdakei bet selinux11:14
sdakelet me turn that shit off11:14
sdakeit is not selinux11:15
inc0hmm...that's strange11:15
inc0seems like ansible fails to gather facts11:15
inc0properly11:15
sdakedependency problem?11:15
sdakeit would be fantastic if ansible told me what it was doing11:16
sdakeis there anything on these machines to look at?11:16
inc0can you manually add task debug to print hostvars?11:16
sdakelike files11:16
inc0http://docs.ansible.com/ansible/debug_module.html11:16
sdakeinc0 I added -vv in kolla-ansible tool11:16
inc0you need to specify task with debug11:16
inc0in play11:17
vbelI can run tox locally but gate fails - http://logs.openstack.org/55/216655/7/check/gate-kolla-build-images-centos-source/ab128b4/console.html11:17
inc0what ansible was doing is it tried to render haproxy.cfg11:17
inc0and failed with KeyError on line 23 (or similar)11:17
*** jmccarthy has joined #kolla11:18
inc0https://github.com/stackforge/kolla/blob/master/ansible/roles/haproxy/templates/haproxy.cfg.j2#L2311:18
inc0it seems ansible doens't see your interfaces11:18
sdakedoes jinja2 need to be installed on the deployed to hosts?11:18
pbourkevbel: I suspect this line: if os.path.basename(script_path) == 'kolla':11:18
pbourkethat is not evaluating true on the CI machine11:19
inc0no, just on deployer11:19
sdakewhat python code is running on the deployed to host?11:19
vbelpbourke: yes, looks like that11:19
inc0shade, MySQL-python11:20
inc0and docker-py11:20
inc0but I'm not sure any more, I think shade is in kolla_ansible11:20
vbelcan one run tox not in kolla folder?11:20
sdakeinc0 I mean what python code11:20
vbelpbourke: can one run tox not in kolla folder?11:20
inc0it's generated by ansible from plays11:20
sdakeSamYaple really looking for debug help here - totally stuck11:20
sdakewhere is it stored on disk11:20
sdakei want to look at it11:20
SamYaplewhats going on?11:20
pbourkevbel: not that I know of, there may be some arg to set base dir11:21
inc0sdake, I don't think this fails on deployed host tho11:21
inc0I think it fails on rendering templates, and that's deployer host thing11:21
vbelpbourke: I can run tox from kolla, from tests ...11:21
sdakesamyaple http://paste.fedoraproject.org/259489/4058759411:21
inc0but I don't know where are these pythons...tmp would be my guess11:21
pbourkevbel: its workspace is /home/jenkins/workspace/gate-kolla-build-images-centos-source11:22
pbourkevbel: is that an issue?11:22
pbourkevbel: try renaming your kolla dir to something else and see if it fails11:22
pbourkesdake: do all hosts have an ip assigned to em111:23
pbourkesdake: and are interfaces up11:23
sdakepbourke all ip assigned to em1, all interfaces are up11:24
vbelpbourke: yes, it is :) Thanks! Will hack it further11:24
*** rhallisey has joined #kolla11:24
pbourkeansible is refusing to gather info about those interfaces for some reason11:25
inc0sdake, add this debug please, we'll see what info it actually gathered?11:26
*** pbourke has quit IRC11:28
*** pbourke has joined #kolla11:28
pbourkehttps://github.com/ansible/ansible/issues/4848#issuecomment-4091217611:29
pbourkesdake: can you try deploying one service, e.g. rabbitmq11:31
pbourkesdake: then try the rest11:31
pbourke(--tags=rabbitmq)11:31
*** weiyu_ has quit IRC11:32
pbourkeactually it tries to access the interface so will probably be same problem11:32
inc0I have different issue with ansible...instead of prints it does cowsay... oO11:32
openstackgerritVladislav Belogrudov proposed stackforge/kolla: Add ability to make tarballs from Git clones  https://review.openstack.org/21665511:34
*** Haomeng|2 has joined #kolla11:37
*** shardy_ has joined #kolla11:38
*** shardy has quit IRC11:39
*** Haomeng has quit IRC11:40
*** shardy_ has quit IRC11:43
*** shardy has joined #kolla11:44
*** dims_ has joined #kolla11:48
*** dims has quit IRC11:52
sdakemariadb busted - please see this log - multinode deploy http://ur1.ca/nk7g1 -> http://paste.fedoraproject.org/259509/8997014411:53
sdakeneed help asap ;)11:53
inc0sdake, hmm...looks like permissions in /var/lib/mysql?11:55
sdakei see that11:56
sdakehowever same exact container starts up on a differnet node11:56
inc0but /var/lib/mysql is a volume11:56
inc0so it will inherit perms11:56
vbelcan be 'data' container change, does it chown it while bootstrapping?11:57
SamYapleremember the mysql folder is empty for new nodes11:59
SamYapleonly the bootstrap node (that geenrates the files) needs to chown12:00
SamYaplethe rest dont initialze the files12:00
SamYapleglaera does that12:00
inc0http://michaelheap.com/cowsay-and-ansible/ really...12:04
SamYapleI HATE IT12:04
inc0lol12:05
inc0I loled when I saw that..12:06
*** jmccarthy has quit IRC12:09
inc0ANSIBLE_COW_SELECTION=random12:11
inc0:D12:11
inc0I think I got wh40k titan now12:11
*** jmccarthy has joined #kolla12:14
openstackgerritMichal Jastrzebski (inc0) proposed stackforge/kolla: Ansible bits to rsyslog  https://review.openstack.org/21506212:16
*** jasonamyers has joined #kolla12:27
sdakeany of this log helpful to people: http://ur1.ca/nk7qg -> http://paste.fedoraproject.org/259523/0591919112:28
sdakepbourke ^^12:28
sdakemariadb is failing to startup multinode12:29
sdakeor galera12:29
sdakeor whatever the hell it is12:29
pbourkesdake: I had a different maria issue12:30
sdakepbourke one issue is the chown is in the wrong place12:30
pbourkesdake: in the end I hacked my multinode to have maria just run one one node :(12:30
pbourkesdake: yeah it was permission related12:30
sdakeit needs to run every time not only on the bootstrap node12:30
sdakei fixed that and rebuilt12:30
pbourkeI haven't seen the above though, let me look more closely12:30
sdakenow having this problem12:30
pbourkehow did you get around the interface problem12:31
inc0SamYaple, voluming /dev/log doesn't seem to be working :( I've ran rsyslog container with /dev/log:/dev/log and another kolla_ansible with /dev/log:/dev/log12:32
inc0logger foo doesn't work12:32
*** shardy_ has joined #kolla12:34
*** shardy has quit IRC12:35
inc0I'm not sure if we can do without syslogd in container12:35
sdakepbourke i put my 3 machines in the network szection12:36
pbourkesdake: yeah I saw you pm, thanks12:36
sdakesamyaple says he understnds problem and will address it12:36
*** diga has quit IRC12:38
*** shardy_ has quit IRC12:39
*** shardy has joined #kolla12:40
vbelyahoo, got thru gate :) https://review.openstack.org/#/c/216655/12:43
*** weiyu has joined #kolla12:43
*** shardy has quit IRC12:44
*** shardy has joined #kolla12:45
*** vbel has quit IRC12:58
*** vbel has joined #kolla13:05
pbourkevbel: added one small question on your review13:05
vbelyes, name in archive should be keystone-xzcs , so source['dest'] is not what we need - it will be keystone.tar. With arcname=os.path.basename(clone_dir) we will get keystone-master or keystone-2015.1.113:08
vbelwe have to have service-xxx because we link it inside of Dockerfiles with ln -s /service-* /service13:11
*** weiyu has quit IRC13:13
pbourkevbel: thanks for clarification on archive name13:24
pbourkevbel: nice job13:24
vbelpbourke: thanks  a million for the review!13:25
*** weiyu has joined #kolla13:26
*** inc0 has quit IRC13:28
*** jtriley has joined #kolla13:37
*** weiyu has quit IRC13:42
*** klint has quit IRC13:47
*** inc0 has joined #kolla13:54
vbelit looks like build.py cannot find base images if I specify 'oracle' as namespace. But any other string works ('vlad', 'oracle123')13:57
vbelCan anyone try that please?13:57
*** Kennan2 has quit IRC14:01
*** diogogmt has quit IRC14:01
SamYaplevbel: what do you mean?14:02
SamYapleim not sure i follow14:02
*** Kennan has joined #kolla14:02
*** ftcpops has joined #kolla14:04
vbeltools/build.py --tag latest --namespace oracle -b oraclelinux -t source --template  --debug -T1 keystone   < does not work, but if namespace is localhost:5000/oracle or vlad or oracle123 it works14:04
*** diogogmt has joined #kolla14:04
SamYapleweird, how does it fail?14:05
*** Haomeng|2 has quit IRC14:05
vbelyes, strange. I see Sorted image keystone with parent oracle123/oraclelinux-source-base:latest  with oracle123 as name, but not with simple oracle14:06
*** unicell1 has joined #kolla14:06
vbelCould not find parent image from some images. Aborting14:06
SamYapleah i think i know why14:06
*** unicell has quit IRC14:06
SamYaplewe check if the namespace is in the image['fullname']14:06
SamYapleit will match everything for you14:07
SamYaplebecause oracle is in oraclelinux14:07
SamYaplei can fix that14:07
mandreSamYaple: no comment about why lvm thin provisioning is necessary in your new patch :(14:07
SamYaplemandre: i had a mental note i was going to take care of it next patch and spaced. I addressed your other comment14:08
SamYaplegive me a minute and ill reposst14:08
mandreyeah i saw that14:09
vbelSamYaple: thanks a lot :)14:09
SamYaplevbel: ill post a patch for you to test in a minute14:09
SamYaplemandre: what would you like it to say?14:09
SamYaplewithout doing this it broke centos14:09
mandreyou sure? the current gate setup script doesn't have it i believe14:10
SamYaplemandre: docker 1.8.1 in this patch14:10
SamYaplethings changed14:10
mandrei see14:10
SamYaplethis is better though anyway14:10
SamYapleits better perforamnce14:10
SamYapleand more expplicit so when we do dind we have a better playground14:10
mandrethen just put a comment in the code, it's not obvious if you don't know the reason14:11
mandreif you can link to docker recommentation that'd be even better14:11
mandreany reason why you only do it for centos/fedora and not ubuntu?14:12
mandreolder docker?14:12
SamYaplemandre: i dont even have an ubuntu gate to test on14:13
SamYaplebut i cant use devicemapper for ubuntu14:13
SamYapleto old linked libraries14:13
*** rhallisey has quit IRC14:17
*** diogogmt has quit IRC14:18
*** rhallisey has joined #kolla14:19
SamYaplenope vbel that wasnt the issue14:19
*** thumpba has joined #kolla14:19
*** rhallisey_ has joined #kolla14:20
*** thumpba has quit IRC14:20
SamYaplevbel: can you file a bug14:20
*** thumpba has joined #kolla14:20
*** inc0 has quit IRC14:21
SamYaplevbel:             if self.namespace not in image['parent_name']:14:22
SamYaple                image['parent'] = None14:22
SamYapleyoure namespace _will_ match your base distro14:22
SamYaplelet me think for a moment14:22
*** rhallisey has quit IRC14:24
SamYaplevbel: can you file a bug with that code with this irc conversation14:24
SamYaplei will have to 'think' later, my brain is fried :)14:24
SamYaplealso feel free to fix it, but i will if you dont14:24
SamYaplethis would also affect me if i used ubun as my namespace, wth base ubuntu14:25
vbelSamYaple: yes, here - https://bugs.launchpad.net/kolla/+bug/148902114:26
openstackLaunchpad bug 1489021 in kolla "cannot use 'oracle' as namespace for build.py" [Medium,New]14:26
SamYapleawesome thanks vbel14:26
SamYapleif you want to assign it to me ill figure it out for you14:26
vbelok, thanks!14:27
SamYaplei suggest you just use orac1e14:27
SamYaple:)14:27
SamYaplefixed!14:27
openstackgerritSteven Dake proposed stackforge/kolla: DO NOT MERGE - Workaround for horizon  https://review.openstack.org/21659214:58
openstackgerritSteven Dake proposed stackforge/kolla: DO NOT MERGE - Fix up galera  https://review.openstack.org/21725114:58
*** sdake has quit IRC14:58
*** daneyon has joined #kolla15:01
*** jmccarthy1 has joined #kolla15:08
*** jmccarthy has quit IRC15:08
*** diogogmt has joined #kolla15:15
*** inc0 has joined #kolla15:24
inc0meeting now or in 1hr?15:24
pbourkedoes it count for stats if you review a patch without voting15:27
inc0nope15:34
mandreinc0: meeting in 1h i think15:34
inc0thanks mandre that's what I though15:35
inc0t15:35
mandrethanks for the reminder btw, i totally forgot we were wednesday15:36
SamYaplepbourke: no it does not15:38
pbourkekk thought as much15:38
*** chandankumar has quit IRC15:40
-openstackstatus- NOTICE: restarting gerrit due to a slow memory leak15:43
*** blakec has joined #kolla15:56
*** dims_ has quit IRC16:00
*** dims has joined #kolla16:01
*** unicell1 has quit IRC16:04
*** akwasnie1 has joined #kolla16:04
*** rhallisey_ is now known as rhallisey16:09
*** erkules has quit IRC16:13
*** erkules has joined #kolla16:13
*** akwasnie1 has left #kolla16:17
*** inc0_ has joined #kolla16:17
*** akwasnie1 has joined #kolla16:17
*** inc0_ has quit IRC16:17
*** sdake has joined #kolla16:20
sdakeso tired16:21
inc0sdake, I bet...you didn't sleep at all did you?16:21
SamYaplesdake: fixed heat16:21
SamYaplesdake https://review.openstack.org/#/c/217058/16:22
SamYapleafter almost 10 hours of pdb16:22
sdakereminder meeting today in 10 mins16:22
SamYaplei was stepping through sqlalchemy to find the issue :/16:22
sdakesamyple hot16:23
harmwmeeting time guys16:29
*** jmccarthy1 has quit IRC16:29
sdakekolla meeting now16:30
openstackgerritRyan Hallisey proposed stackforge/kolla: WIP:Replace config-external with a JSON file for Neutron-OVS  https://review.openstack.org/21687416:40
*** mfalatic has joined #kolla16:44
*** unicell has joined #kolla16:52
*** achanda has joined #kolla16:52
*** jasonamyers has quit IRC16:54
openstackgerritSam Yaple proposed stackforge/kolla: Add Ansible support for Heat  https://review.openstack.org/20552817:02
openstackgerritSam Yaple proposed stackforge/kolla: Remove tiered building in build.py  https://review.openstack.org/21669117:05
*** akwasnie has quit IRC17:19
*** daneyon has quit IRC17:27
pbourkeok im out too guys, will pick up reviews in the morning17:33
pbourkesdake: hope you're still alive after demo ;)17:33
inc0cya pbourke ++;)17:34
sdakepbourke demo failed duringovs bridge bringup17:34
sdakepbourke ciao17:34
inc0I'm off as well, have a good night everyone17:34
*** inc0 has quit IRC17:34
mandresame here, good night all17:34
rhalliseypbourke, you're spamming the etherpad with '+'17:35
akwasnie1bye Michal :)17:35
rhalliseylol17:35
sdakeholly christ undo17:35
*** akwasnie1 has quit IRC17:35
harmwlol17:35
rhalliseyhaha17:35
rhalliseywtf is happening17:36
*** loth has joined #kolla17:38
*** achanda_ has joined #kolla17:39
*** achanda has quit IRC17:39
*** dims_ has joined #kolla17:41
*** dims has quit IRC17:42
rhalliseypbourke, around?17:43
SamYaplewtf guys17:46
SamYaplei levae for 5 minutes17:46
*** achanda_ has quit IRC17:48
rhalliseywhat is going on in the etherpad17:50
SamYaplepbourke went native it seems17:50
*** achanda has joined #kolla17:51
sdakerhallisey the etherpad is beign spammed by someone17:56
rhallisey:/17:56
sdake#openstack-infra is investigating17:56
sdakeafter #openstack-infra investigated, the person stopped17:56
sdakebecause opentackinra was on to them17:56
SamYaplesdake: it wasnt me!17:57
SamYaplei know thats what you were thinking17:57
SamYapleadmit it17:57
sdakeno i wasn't thinking that at all17:57
rhalliseyI thought one of you guys feel asleep on the keyboard17:57
SamYaplei make jokes. its my thing17:57
sdakeI was thinkign it is osmone in openstack-meeting-4 and someone in openstack-infra17:57
SamYapleuhoh17:58
SamYaplelooks like more spam17:58
SamYaplenow its copy pasta17:58
*** jasonsb has quit IRC18:04
*** absubram has joined #kolla18:07
*** jasonamyers has joined #kolla18:12
*** unicell has quit IRC18:14
*** unicell has joined #kolla18:14
*** achanda has quit IRC18:16
*** openstackgerrit has quit IRC18:17
*** openstackgerrit has joined #kolla18:17
openstackgerritRyan Hallisey proposed stackforge/kolla: Need to import gitapi instead of git  https://review.openstack.org/21733618:18
SamYaplerhallisey:18:20
SamYaplei dont have an error with that script18:20
SamYapledid you pip install -r requirements.txt?18:20
rhalliseyoh gitpython18:21
rhalliseythat's the module I was looking for18:21
rhalliseyI was looking for python-git18:22
rhalliseyk abandoned18:22
SamYapleopenstack talks are up guys18:26
SamYapleonly 2 kolla talks got added18:33
SamYapleactually just one18:33
SamYaplethe other just mentions kolla18:33
rhalliseyya just notice18:35
rhalliseyany reason kolla only got 1?18:35
rhalliseyfrom the community18:35
rhalliseythink that's a bit odd18:35
SamYaplewell my subbissions got moved catagories at teh last second, after most had voted18:35
SamYaple:(18:35
SamYapleunlikely the votes were adjusted18:35
rhalliseythat kinda sucks18:36
SamYapleindeed18:36
rhalliseyI'd images there would've been 3-4 for the community up there18:36
rhallisey1!18:36
SamYaplei thought that lab was going to be killer, but it was one of the track changes last second18:36
SamYaplewell this sucks :(18:41
SamYapleits cool though the talk with sdake will be good18:41
*** jasonsb has joined #kolla18:44
rhalliseymaybe kolla is next summit18:46
rhalliseyI really thought I would be a think for this one18:46
rhalliseyit*18:46
SamYapleto be fair, next summit well have a stable branch and people can test without building images18:47
SamYaplemaybe we just drank the kolla kool-aid and it isn't so amazing :)18:48
ftcpopsWell, how's that for an introduction to kolla?18:48
rhalliseySamYaple, I think we're drinking fine whiskey18:48
ftcpopsI went out for a run, came back to see my keyboard spewing "+" all over your etherpad18:48
ftcpopsSORRY ABOUT THAT!18:48
rhalliseylol!18:49
ftcpopsI was like, wow, who's going nuts on the '+'s?  oh...crap...18:49
*** jeblair has joined #kolla18:52
*** dims_ has quit IRC18:53
*** dims has joined #kolla18:54
*** achanda has joined #kolla19:16
*** jasonamyers has quit IRC19:25
*** jasonamyers has joined #kolla19:25
*** achanda has quit IRC19:25
*** daneyon has joined #kolla19:29
*** daneyon has quit IRC19:33
*** daneyon has joined #kolla19:48
*** anteaya has joined #kolla19:51
anteayaftcpops: might I suggest in future you select a different etherpad username19:51
anteayaftcpops: the name you selected combined with a stuck keyboard is an odd combination19:52
*** daneyon has quit IRC19:54
*** thumpba has quit IRC19:55
*** daneyon has joined #kolla19:55
ftcpops"quit being an asshat"?  I didn't type that in, I think someone helped me with that one ;-)19:55
ftcpopsAlthough for the Tokyo Summit, I'll make sure to put that on my name badge19:57
rhalliseylol20:00
*** jasonsb has quit IRC20:01
*** achanda has joined #kolla20:02
*** jasonsb has joined #kolla20:02
*** achanda has quit IRC20:08
*** daneyon has quit IRC20:09
ftcpopssdake: sorry again about the etherpad plus spamming :-(20:11
ftcpopsI just ran, and timed: kolla/cmd/build.py -T 1000 -i include_proxy.dockerfile --template --no-cache20:11
ftcpopsreal    304m5.164s20:11
ftcpopsuser    0m1.970s20:11
ftcpopssys     0m3.291s20:11
ftcpopsSystem is a CentOS 7 VM running on top of Windows via Virtual Box with 4Gb RAM20:12
ftcpops100Mb Ethernet via Proxy20:12
*** jasonamyers has quit IRC20:12
*** jasonamyers has joined #kolla20:23
*** daneyon has joined #kolla20:24
*** achanda has joined #kolla20:26
*** thumpba has joined #kolla20:29
*** jasonsb has quit IRC20:33
*** jasonsb has joined #kolla20:36
anteayaftcpops: yeah, maybe locking the screen, seems like you have some helpful housemates21:01
*** openstackgerrit has quit IRC21:01
*** openstackgerrit has joined #kolla21:02
*** jasonamyers has quit IRC21:03
*** achanda has quit IRC21:07
*** daneyon_ has joined #kolla21:16
*** daneyon has quit IRC21:19
ftcpopsHas anyone successfully run this command (or similar) from behind a corporate proxy?   ansible-playbook -i inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml site.yml21:22
rhalliseyftcpops, I've been having issues myself lately with it21:25
rhalliseyftcpops, what are you running into?21:25
ftcpopsPLAY [haproxy] ****************************************************************21:25
ftcpopsGATHERING FACTS ***************************************************************21:25
ftcpopsok: [localhost]21:25
ftcpopsTASK: [common | Starting Ansible container] ***********************************21:25
ftcpopsfailed: [localhost] => {"changes": ["{\"status\":\"Pulling repository docker.io/kollaglue/centos-binary-kolla-ansible\"}\r\n", "{\"errorDetail\":{\"message\":\"Error: image kollaglue/centos-binary-kolla-ansible:latest not found\"},\"error\":\"Error: image kollaglue/centos-binary-kolla-ansible:latest not found\"}\r\n"], "failed": true, "status": ""}21:25
ftcpopsmsg: Unrecognized status from pull.21:25
ftcpopsFATAL: all hosts have already failed -- aborting21:25
*** absubram has quit IRC21:26
ftcpopsbut when I run: docker run -it kollaglue/centos-binary-kolla-ansible bash it works21:26
rhalliseyftcpops, do $ docker images21:26
rhalliseywhat's the tag for that image21:26
rhalliseyit's looking for latest21:26
jpeelermy guess is that you need to set docker_pull_policy: "missing"21:28
ftcpopssorry, I don't know how to find the tag :(21:28
ftcpopsjpeeler: when I set it to missing, I get this:21:29
ftcpopsTASK: [haproxy | Starting keepalived container] *******************************21:29
ftcpopsfailed: [localhost] => {"changed": true, "failed": true}21:29
ftcpopsmsg: Docker API Error: Cannot start container 62830bd04a38b9bb306649041d0028d1db1b2494d49b8cf8ebfb6d0b2c366db6: [8] System error: exec format error21:29
ftcpopsFATAL: all hosts have already failed -- aborting21:29
jpeelerwhat does "docker ps" show? i don't think i've seen that error before21:30
ftcpops$ docker ps21:31
ftcpopsCONTAINER ID        IMAGE                                          COMMAND                 CREATED             STATUS              PORTS               NAMES21:31
ftcpopsefab539742c5        kollaglue/centos-binary-kolla-ansible:latest   "/bin/sleep infinity"   2 minutes ago       Up 2 minutes                            kolla_ansible21:31
jpeeleruse a pastebin please - that wasn't too bad though21:32
ftcpopsok :)21:32
jpeelerok pastebin docker ps -a, docker logs <failed haproxy id>21:32
jpeeleri mean if this is an all in one install, you may be able to just skip haproxy like rhallisey does, right ryan?21:33
rhalliseyexec format error O.o21:33
ftcpopshttp://paste.openstack.org/show/428763/21:34
rhalliseyI'm guessing it something in /etc/globals.yml21:34
rhalliseyI'm not sure what that iss21:35
jpeeleruh, did that docker ps output get truncated?21:36
jpeelerit doesn't show the keepalived container exited, so maybe kill it and rerun?21:36
ftcpopsokay, I'll kill them all and rm them and will start over21:36
jpeelerthere's a stupid script "cleanup-containers"21:37
jpeelerif you haven't done anything but kolla containers, use that21:37
ftcpopsbeautiful.  Thanks, jpeeler21:38
ftcpopsyeah, it's a VM I'm using just for kolla21:38
*** jtriley has quit IRC21:40
*** thumpba has quit IRC21:40
*** thumpba has joined #kolla21:40
*** shardy has quit IRC21:46
*** thumpba has quit IRC21:47
*** thumpba has joined #kolla21:49
*** thumpba has quit IRC21:51
ftcpopsOkay, here's a clean run with docker_pull_policy: always ==> http://paste.openstack.org/show/428765/21:51
ftcpopsHere's a clean run with docker_pull_policy: missing ==> http://paste.openstack.org/show/428766/21:52
rhalliseyftcpops, have you built the container fairly recently?21:54
ftcpopsthis morning21:54
rhalliseymaster may just be broken21:54
*** dims_ has joined #kolla21:56
rhalliseyftcpops, I'm rebuilding now will try again soon21:58
ftcpopsokay thanks rhallisey!  I have to go coach baseball soon, but I'll check back. Thanks for all your help and sorry again for all the plusses (so embarrassing)21:59
rhalliseyftcpops, no problem have fun!  It was just an accident no worries!21:59
*** dims has quit IRC21:59
*** achanda has joined #kolla22:06
*** ftcpops has quit IRC22:06
*** SourabhP has joined #kolla22:06
*** diogogmt has quit IRC22:08
*** sdake_ has joined #kolla22:15
*** sdake has quit IRC22:18
*** dims_ has quit IRC22:24
*** dims has joined #kolla22:24
*** achanda_ has joined #kolla22:28
*** achanda has quit IRC22:29
sdake_whitepaper on container tech and kolla mentioned quite a bit from foundation: http://www.openstack.org/assets/pdf-downloads/Containers-and-OpenStack.pdf22:30
rhalliseysdake_, except only 1 talk from the community got accepted22:31
rhalliseycool though22:31
sdake_ya not alot of slots22:31
sdake_you sid one other tlak mentioning kolla got accepted that was what?22:32
rhalliseyit was a talk about how ebay was looking to use it22:32
rhalliseyit's a user_story22:32
rhalliseythey mention kolla22:32
sdake_which one22:33
sdake_link plz22:33
*** dims has quit IRC22:34
*** sdake_ is now known as sdake22:41
*** rhallisey has quit IRC22:41
*** achanda has joined #kolla23:18
*** achanda_ has quit IRC23:18
*** achanda has quit IRC23:38
*** blakec has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!