| *** Allen_Gao has joined #kolla | 00:02 | |
| *** vhosakot has quit IRC | 00:03 | |
| *** vhosakot has joined #kolla | 00:04 | |
| vhosakot | back! | 00:04 |
|---|---|---|
| * vhosakot is reading all the replies! | 00:04 | |
| *** tobe has quit IRC | 00:06 | |
| openstackgerrit | Allen Gao proposed openstack/kolla: Keep consistent with others for ironic containers name https://review.openstack.org/280557 | 00:06 |
| *** aginwala has quit IRC | 00:06 | |
| vhosakot | ok, I will delete all the containers and build with less number of threads then.. cool | 00:08 |
| *** aginwala has joined #kolla | 00:08 | |
| *** kproskurin has quit IRC | 00:10 | |
| vhosakot | "sudo docker rm -f `docker ps -aq`" deletes all the containers at once | 00:10 |
| *** Allen_Gao has quit IRC | 00:18 | |
| *** Allen_Gao has joined #kolla | 00:18 | |
| vhosakot | SamYaple: are you there ? | 00:20 |
| vhosakot | SamYaple: here is my "docker info " | 00:20 |
| vhosakot | http://paste.openstack.org/show/487515/ | 00:20 |
| vhosakot | I'm using aufs graph driver.... | 00:20 |
| *** diogogmt has quit IRC | 00:21 | |
| vhosakot | should we use btrfs for kolla ? Doesn't kolla support aufs ? | 00:21 |
| *** mbound has quit IRC | 00:26 | |
| *** tzn has joined #kolla | 00:28 | |
| Allen_Gao | vhosakot: I think no limit, FYI I'm using overlayfs | 00:29 |
| *** salv-orlando has quit IRC | 00:32 | |
| vhosakot | Allen_Gao: cool, thanks! | 00:34 |
| *** tzn has quit IRC | 00:39 | |
| *** iceyao has joined #kolla | 00:58 | |
| SamYaple | vhosakot: this isnt a kolla issue | 00:59 |
| SamYaple | its a docker and your storage backend issue | 00:59 |
| SamYaple | with slower storage docker trips over itself trying to do multiple things | 00:59 |
| SamYaple | its a single threa daemon | 00:59 |
| vhosakot | SamYaple: aufs issue on my Ubuntu ? | 00:59 |
| SamYaple | just lower the number of threads you use to build to like 2 or 4 | 01:00 |
| SamYaple | should help | 01:00 |
| SamYaple | no | 01:00 |
| vhosakot | SamYaple: is the number of thread configurable in globals.yml ? | 01:00 |
| SamYaple | aufs is slow, btrfs would probably work better | 01:00 |
| SamYaple | vhosakot: this is a buildtime issue not a deploy time issue | 01:00 |
| SamYaple | all that cruft is from building | 01:00 |
| *** iceyao has quit IRC | 01:00 | |
| vhosakot | kolla-buid's number of threads.. is this something I can configure in globals ? | 01:01 |
| SamYaple | no globals is for deploying, kolla-ansible | 01:01 |
| SamYaple | kolla-build.conf is for building | 01:01 |
| vhosakot | ah | 01:01 |
| vhosakot | where is kolla-build.conf ? | 01:02 |
| SamYaple | have you generated it? | 01:03 |
| vhosakot | I see kolla/etc/oslo-config-generator/kolla-build.conf | 01:03 |
| SamYaple | yea thats the generated location | 01:03 |
| vhosakot | how can I generate it ? | 01:03 |
| SamYaple | thats it | 01:03 |
| vhosakot | oh ok | 01:03 |
| vhosakot | I see two.. which one ? | 01:04 |
| vhosakot | /home/kolla/kolla/etc/oslo-config-generator/kolla-build.conf | 01:04 |
| vhosakot | /usr/local/share/kolla/etc_examples/oslo-config-generator/kolla-build.conf | 01:04 |
| *** tobe has joined #kolla | 01:04 | |
| vhosakot | which of these two ? | 01:05 |
| *** JoseMello has joined #kolla | 01:06 | |
| *** iceyao has joined #kolla | 01:08 | |
| vhosakot | how can I set conf.threads ? | 01:10 |
| SamYaple | vhosakot: either config. also from the cli | 01:11 |
| vhosakot | how can I lower the number of threads ? | 01:11 |
| vhosakot | what is the command ? :) | 01:11 |
| vhosakot | I'm three days old in kolla :) | 01:12 |
| *** jtriley has joined #kolla | 01:12 | |
| vhosakot | I see it! | 01:12 |
| vhosakot | --threads THREADS, -T THREADS, --kolla-build-threads THREADS | 01:12 |
| vhosakot | :) | 01:12 |
| vhosakot | how many should I use ? | 01:12 |
| vhosakot | like 10 ? | 01:12 |
| *** Allen_Gao has quit IRC | 01:15 | |
| *** Allen_Gao has joined #kolla | 01:15 | |
| *** Allen_Gao has quit IRC | 01:16 | |
| *** allen has joined #kolla | 01:16 | |
| *** allen has quit IRC | 01:17 | |
| *** diogogmt has joined #kolla | 01:18 | |
| SamYaple | the default is 8 | 01:20 |
| SamYaple | you should use 2 or 4 | 01:20 |
| *** iceyao has quit IRC | 01:20 | |
| *** iceyao has joined #kolla | 01:21 | |
| *** vhosakot has quit IRC | 01:24 | |
| *** vhosakot has joined #kolla | 01:25 | |
| *** vhosakot has quit IRC | 01:29 | |
| openstackgerrit | Merged openstack/kolla: Refined documentation regarding registry https://review.openstack.org/282084 | 01:32 |
| *** sdake_ has joined #kolla | 01:34 | |
| *** iceyao has quit IRC | 01:35 | |
| *** sdake has quit IRC | 01:36 | |
| *** iceyao has joined #kolla | 01:36 | |
| *** ssurana has quit IRC | 01:38 | |
| *** sdake has joined #kolla | 01:38 | |
| *** Serlex has left #kolla | 01:39 | |
| *** sdake_ has quit IRC | 01:41 | |
| clayton | SamYaple: have you thought about how to handle keepalived for HA routers now that you can run neutron l3 agent inside its own container? | 01:41 |
| *** Allen_Gao has joined #kolla | 01:41 | |
| clayton | seems like it would be bad to run them inside the l3-agent container, since restarting that container would probably cause gateway failovers | 01:42 |
| *** Allen_Gao has quit IRC | 01:42 | |
| *** Allen_Gao has joined #kolla | 01:42 | |
| SamYaple | clayton: not neccesarily | 01:43 |
| SamYaple | keepalived can keep the vip on the same host even if the process dies | 01:43 |
| SamYaple | look at the -V option | 01:43 |
| *** Liuqing has joined #kolla | 01:44 | |
| *** tfukushima has joined #kolla | 01:44 | |
| clayton | yeah, but if it's the primary and stops heartbeating then the backup is going to take over | 01:44 |
| clayton | seems like you'd want to avoid those sort of spurious failovers if you can | 01:45 |
| SamYaple | its a configurable failover, and its like 10 seconds I think. but you can always stop the other ones or increase the time out | 01:45 |
| SamYaple | havent really looked into it to be honest | 01:45 |
| clayton | nod. I've been thinking about it. my thought was to work out a way to run the keepalived processes in another container that I'd almost never restart | 01:46 |
| clayton | not sure how gross I think that is compared to the other options | 01:46 |
| *** allen has joined #kolla | 01:47 | |
| clayton | but honestly I don't know enough about how l3 agent manages keepalived to know how practical that is | 01:47 |
| clayton | hoping to get some time in the next few weeks to look into it | 01:47 |
| SamYaple | my best guess right now (10 seconds in to thinking about it) is jacking up the heartbeat timeout to like 60 seconds | 01:48 |
| clayton | ideally I'd like my failover to be faster than that :) | 01:49 |
| *** allen has quit IRC | 01:49 | |
| *** allen has joined #kolla | 01:49 | |
| SamYaple | i agree. but again, havent really thought about it :) | 01:50 |
| *** allen has quit IRC | 01:50 | |
| clayton | fair enough, I'll let you know if I come up with anything clever | 01:50 |
| SamYaple | sounds good | 01:50 |
| *** vhosakot has joined #kolla | 01:53 | |
| *** aginwala has quit IRC | 01:54 | |
| vhosakot | SamYaple: ok, will use 4 threads then... thanks! | 01:55 |
| vhosakot | will file a bug for this | 01:55 |
| SamYaple | vhosakot: that is not a bug | 01:56 |
| SamYaple | please dont file one for it | 01:56 |
| SamYaple | or if you do, do it in docker | 01:56 |
| *** tobe has quit IRC | 01:56 | |
| *** aginwala has joined #kolla | 01:56 | |
| vhosakot | SamYaple: don't you think kolla-build must find if images are leaked due to aufs issue.. if, reduce the number of threads and build ? | 01:57 |
| vhosakot | SamYaple: like build throttling | 01:57 |
| SamYaple | vhosakot: it doesnt happen to me | 01:57 |
| SamYaple | or the gate | 01:57 |
| SamYaple | i use AUFS | 01:57 |
| vhosakot | I saw it :) | 01:58 |
| SamYaple | its a docker daemon bug | 01:58 |
| SamYaple | if you want to provide a 'FAQ' type doc, thats fine, but its not a kolla bug | 01:58 |
| vhosakot | so, the images in "docker ps -a" in http://paste.openstack.org/show/487514/... aren't they built by kolla ? | 01:58 |
| vhosakot | tiny_babbage, hopeful_sinoussi, stupefied_golick, cocky_kare, etc etc | 01:59 |
| SamYaple | vhosakot: kolla-build tells docker to build it. after that its in docker engines control | 01:59 |
| SamYaple | docker engine is failing here | 01:59 |
| sdake | SamYaple isn't there a case where partial builds end with a container that is left for debug purpose? | 01:59 |
| SamYaple | sdake: yea but this isnt it | 01:59 |
| vhosakot | ah, so those funny names are genertaed by docker then ? not kolla ? | 01:59 |
| SamYaple | vhosakot: correct | 01:59 |
| vhosakot | ah | 02:00 |
| sdake | vhosakot yes generated by docker | 02:00 |
| sdake | i was thinking it was an intermediate build leak but i'm not certain | 02:00 |
| sdake | lk4d4 (docker guy) says the threading in docker is maddening :) | 02:00 |
| SamYaple | more like it dosnt exist | 02:00 |
| sdake | all kinds of races | 02:00 |
| SamYaple | its very single threaded | 02:00 |
| sdake | go has all kinds of multithreading | 02:01 |
| sean-k-mooney | sdake: esplically if you try to delete images concurrently | 02:01 |
| sdake | i'm not a gopher dude | 02:01 |
| SamYaple | multithreading is very much non-existant in the docker engine | 02:01 |
| sdake | but its got goroutinees built in | 02:01 |
| vhosakot | "docker images built simultaneuosuly in multi-threaded manner cause leaked images with aufs graph driver" is this the right way to write the bug against docker ? | 02:01 |
| SamYaple | its psuedo multithreading | 02:01 |
| sdake | and docker uses that heavily | 02:01 |
| sean-k-mooney | correct me if im wrong but are goroutines like isolate or greenthreads? | 02:02 |
| sdake | they are no coroutines no | 02:02 |
| vhosakot | SamYaple, sdake is the wording right ? | 02:02 |
| sdake | they are full linux threads | 02:03 |
| vhosakot | " docker images built simultaneuosuly in multi-threaded manner cause leaked images with aufs graph driver" | 02:03 |
| vhosakot | i will file a bug against docker | 02:03 |
| vhosakot | I need to know what to write :) | 02:03 |
| sdake | vhosakot that sounds good | 02:03 |
| vhosakot | cool | 02:03 |
| sean-k-mooney | sdake: ah good to know, thanks. | 02:03 |
| sdake | make sure to show all the leaked stuff | 02:03 |
| sdake | docker inspects on each one woud lbe helpful for them as well i suspect | 02:04 |
| vhosakot | sdake, SamYaple how about this... if kolla-build sees this leaked image issue I'm seeing, kolla-build should throw and error and exit | 02:04 |
| sdake | no we don't want exits | 02:04 |
| vhosakot | because, other images may pass ? | 02:04 |
| sdake | yes | 02:04 |
| sdake | and we have retry | 02:04 |
| openstackgerrit | Merged openstack/kolla-mesos: Make Mesos resources configurable https://review.openstack.org/280144 | 02:05 |
| vhosakot | http://paste.openstack.org/show/487514/ if you look at this, none passed for me :) | 02:05 |
| SamYaple | vhosakot: no see thats the point. its docker failing. you can have those leaking containers and build images correctly | 02:05 |
| SamYaple | kolla has no idea | 02:05 |
| SamYaple | it just has what docker tells it | 02:05 |
| openstackgerrit | Merged openstack/kolla-mesos: Add neutron config https://review.openstack.org/265109 | 02:05 |
| rhallisey | we should put that in the docs | 02:05 |
| rhallisey | kolla has no idea | 02:05 |
| vhosakot | I will update the docs, sure | 02:05 |
| SamYaple | rhallisey: ive honestly never seen it outside of the gate | 02:06 |
| SamYaple | and i fixed that in the gate | 02:06 |
| sdake | i've only seen in in vms myself on my laptop | 02:06 |
| sdake | with overlayfs | 02:06 |
| SamYaple | yea its a "your stuff is too slow" problem | 02:06 |
| rhallisey | I was joking | 02:06 |
| rhallisey | I was taking about 'kolla has no idea' | 02:06 |
| SamYaple | What would 'kolla' look like? | 02:07 |
| vhosakot | SamYaple: doesn't kolla need "let me slow down then" mechanism ? :) | 02:07 |
| SamYaple | we need a graphic | 02:07 |
| rhallisey | sdake, has a sticker | 02:07 |
| SamYaple | vhosakot: it has one! threads | 02:07 |
| rhallisey | golden ticket right | 02:07 |
| rhallisey | ? | 02:07 |
| SamYaple | oh jeez that golden ticket stuff | 02:07 |
| sdake | yup golden ticket | 02:07 |
| *** allen has joined #kolla | 02:07 | |
| sdake | but too late to use that | 02:07 |
| SamYaple | ok dinner | 02:08 |
| vhosakot | does kolla-buidl check the output for "docker ps -a" after building each image to check if the image was built fine ? | 02:08 |
| SamYaple | nope | 02:08 |
| vhosakot | SamYaple: shouldn;t it check ? | 02:09 |
| SamYaple | its goign to trust docker for that | 02:09 |
| SamYaple | nope | 02:09 |
| vhosakot | well, we know what docker is doing for me in aufs | 02:09 |
| SamYaple | again, that stuff can stick around and have perfect images | 02:09 |
| vhosakot | ok cool | 02:09 |
| vhosakot | I was thinking, if the built image if missing in "docker ps -a", print a message.. does it make sense | 02:10 |
| sean-k-mooney | you could just use the regex suppot in tools/cleanup-containers to delete the leaked conteiners aferwards and we could document the issue? | 02:10 |
| *** Marga__ has joined #kolla | 02:10 | |
| *** sdake_ has joined #kolla | 02:11 | |
| sean-k-mooney | vhosakot: out of interest what kernel are you using? | 02:11 |
| vhosakot | ubuntu 14.04 4.2.0-27-generic | 02:12 |
| *** sdake has quit IRC | 02:12 | |
| sean-k-mooney | oh ok i recent ly upgreaded to that too. i was noticing some aufs errors in dmesg | 02:12 |
| vhosakot | I'm talking about kolla-build finding built containers that are missing in "docker ps -a" and retry building them | 02:12 |
| vhosakot | if a built image is missing in "docker ps -a", shouldn't it be retried ? | 02:12 |
| sean-k-mooney | vhosakot: after the image is built why would docker keep the conteiner in docker ps -a if it was not running? | 02:14 |
| *** Marga_ has quit IRC | 02:14 | |
| *** Allen_Gao has quit IRC | 02:14 | |
| vhosakot | sorry, I did not say correctly, if funny names are found in "docker ps -a" :) | 02:15 |
| *** allen is now known as Allen_Gao | 02:15 | |
| sean-k-mooney | vhosakot: if you have other container on you plathform that could cause issues though | 02:15 |
| vhosakot | right right | 02:16 |
| vhosakot | ok, I was getting my questions answered! :) | 02:16 |
| vhosakot | cool, I will retry with lesser number of threads! :) | 02:16 |
| vhosakot | thanks! | 02:16 |
| sean-k-mooney | SamYaple: are you still around? i was going to respond to https://review.openstack.org/#/c/275932/ but it could be quicker if you were here? | 02:17 |
| sean-k-mooney | SamYaple: ill take that as a no :) TLDR if source infra containers are not desired for binary disto i can work with that and ill drop the review | 02:20 |
| sean-k-mooney | SamYaple: i did want to get your and everyones input on an intermediate solution until packages of ovs with dpdk are available | 02:21 |
| sdake_ | sean-k-mooney have to say i am not keen on building infra packages from source | 02:22 |
| sdake_ | they are far too complex and hard to get correct | 02:22 |
| sean-k-mooney | yes i know | 02:23 |
| sdake_ | if we permit openvswitch we end up permitting all infra services to be built from source | 02:23 |
| sean-k-mooney | i have been maintain devstack support for ovs with dpdk since icehouse so i was hopeing to enable ovs with dpdk in kolla next cycle | 02:23 |
| sdake_ | is dpdk some type of plugin? | 02:24 |
| sean-k-mooney | binary packages should be available in ubuntu 16.04. it is an optional compile time libary | 02:24 |
| sdake_ | if so we have made a decision that we want plugin support already | 02:24 |
| sean-k-mooney | so its basically a different build target when building ovs | 02:25 |
| sean-k-mooney | it does not use the kenel module and does all forwarding in userspace | 02:25 |
| sdake_ | dpdk is a replacement for the kernel mdoule? | 02:26 |
| sdake_ | i know i could google to find out | 02:26 |
| sdake_ | but been rolling since 5am | 02:26 |
| sdake_ | 8pm now | 02:27 |
| sean-k-mooney | yes kindof. ovs has a netdev datapath. dpdk is an optional extention to that datapath | 02:27 |
| sdake_ | want to understand what use case your after | 02:27 |
| sean-k-mooney | basically dpdk is an opensouce toolkit for developing high performace packet processing applicaitons | 02:28 |
| sean-k-mooney | it was started by intel and 6wind and has beeen integreated with upstream openvswitch | 02:28 |
| *** dims has joined #kolla | 02:28 | |
| sdake_ | which language | 02:28 |
| sdake_ | c? | 02:28 |
| sean-k-mooney | its in c yes | 02:28 |
| sdake_ | cool i like c | 02:28 |
| sdake_ | however being a C master, I understand the complexity of building from source vs doing so for python packages | 02:29 |
| *** dims_ has quit IRC | 02:29 | |
| sean-k-mooney | so basically how dpdk works is it uses a userspace driver to poll the nic instead of being interupt driven. | 02:29 |
| sean-k-mooney | yes the build chain add some challanges but it is relativly stable | 02:30 |
| sdake_ | will you be in austin? | 02:30 |
| *** aginwala has quit IRC | 02:30 | |
| sean-k-mooney | yes probably. i should find out in the next week or 2. so what i wanted to ask was as packages are not yet availabel and since | 02:30 |
| sean-k-mooney | kolla would prefer not to do source build of infra contianers | 02:31 |
| sdake_ | i dont think we ha ereally made a deicision on that point sean-k-mooney | 02:31 |
| sdake_ | we have made a decision to build python from source | 02:32 |
| sdake_ | other things no decision has been made | 02:32 |
| sdake_ | probably worth a design session at summit | 02:32 |
| sean-k-mooney | oh ok . i would the comunity be ok if i added source deployment support for ovs form souce to another repo i maintian for the mitika cycle | 02:33 |
| sdake_ | mitaka is over capacity at the moment | 02:33 |
| sdake_ | we have apprxoimately 100 unsolved bugs | 02:33 |
| sean-k-mooney | oh yes i know | 02:33 |
| sdake_ | and 40 unimplemented blueprints | 02:33 |
| sdake_ | i dont want to add any more stuff to mitaka :) | 02:34 |
| sdake_ | its not a "I dont like your idea thing" it a "holy shit we have too much risk" thing :) | 02:34 |
| sean-k-mooney | sure | 02:34 |
| sean-k-mooney | i was actully going to suggest not doing anything upstream until newton | 02:35 |
| sdake_ | yes you mentioned both above is why I point that out :) | 02:35 |
| sdake_ | [19:33:03] <sean-k-mooney>oh ok . i would the comunity be ok if i added source deployment support for ovs form souce to another repo i maintian for the mitika cycle | 02:35 |
| sean-k-mooney | yes i miss phrase as i started reading your message while typeing, i mainting dpdk support of openstack in https://github.com/openstack/networking-ovs-dpdk | 02:36 |
| sean-k-mooney | would you be ok if i added a kolla sub folder to keep some experamental support for this cycle? | 02:37 |
| sdake_ | sub folder to kolla repository? | 02:37 |
| *** sdake_ is now known as sdake | 02:37 | |
| sdake | we have not made a deicision on a /contrib directory | 02:38 |
| sdake | i think the cores are in favor | 02:38 |
| sean-k-mooney | i would like to submit a blueprint/spec to dicuss for next cycle to see if this is someting that alings with kolla goals. no sub folder to networking-ovs-dpdk | 02:38 |
| sdake | i am not in favor prior during mitaka cycle | 02:38 |
| sdake | people are free to do with their repositories as they please :) | 02:38 |
| sean-k-mooney | well so i just dont want it to be percived as a fork or anything like that as i want to work upstream as much as i can | 02:39 |
| sdake | there was discussion of farming out kolla for the various projects like how devstack and horizon has done (plugins) | 02:39 |
| sean-k-mooney | ya that was something i was hoping would be added to kola at some point | 02:40 |
| sdake | I dont know if plugins are viable in our architecture | 02:40 |
| sdake | not saying they aren't | 02:40 |
| sdake | not saying they are | 02:40 |
| sdake | but its an open question to me at this point | 02:40 |
| sean-k-mooney | i dont think the ansible code as writted is easily plugable currently but it could be if desired | 02:41 |
| sdake | the bottom line is mitaka is overloaded with deliveries for kolla - our bug tracker and feature tracker are off the hook | 02:41 |
| sdake | our review queue is super long | 02:41 |
| sdake | the community was pretty clear that upgrades are mandatory for this cycle | 02:42 |
| sdake | here is our priority discussion from idcycle: | 02:42 |
| sdake | https://etherpad.openstack.org/p/kolla-mitaka-midcycle-priorities | 02:42 |
| *** Allen_Gao has quit IRC | 02:43 | |
| sdake | upgrades scored a 9.2 | 02:43 |
| sean-k-mooney | ok yes so ill close the review and i will submit a blueprint/ spech for dpdk support when the review backlog is less overloaded | 02:43 |
| sdake | there were 9 people in the room | 02:43 |
| sean-k-mooney | yes i would really like to see upgrades done too | 02:43 |
| sdake | diags came in second | 02:44 |
| sdake | stability (mostly of liberty) came in third | 02:44 |
| sean-k-mooney | yes i dont know if i can help much but i would like to get more involved in kolla in general. | 02:45 |
| sdake | if you want to get involed in kolla Ican help you there ;) | 02:45 |
| sdake | adding more to the plate isn't ideal in the 3 weeks we have left before mitaka-3 | 02:46 |
| sdake | lets eat the food we have first :) | 02:46 |
| sean-k-mooney | yes sorry | 02:46 |
| sdake | no reason to apologize | 02:46 |
| sean-k-mooney | i had actuly hoped to do more this cycle on the kolla side but got tied up in neutron/nova. but if there are some low | 02:47 |
| sean-k-mooney | haging fruit bug i might see if i can fix one or two | 02:47 |
| sdake | docs are low hagning fruit | 02:47 |
| sdake | if you want big impact work, fix bugs that are critical/high ;) | 02:48 |
| *** Allen_Gao has joined #kolla | 02:48 | |
| sdake | even just 1 bug would do :) | 02:48 |
| sdake | I could solve all the low hanging fruit in a day (the docs) | 02:48 |
| sdake | things like upgrades take longer | 02:49 |
| sdake | one area where you could immediately help is in drop-root blueprint | 02:49 |
| sdake | its not quite finished yet | 02:49 |
| sdake | the work is easy | 02:49 |
| sdake | and mostly done | 02:49 |
| sean-k-mooney | cool ill take a look. i belive the libvirt container is still not resolved for the drop-root change | 02:50 |
| sean-k-mooney | that was waiting on docker 1.10 right | 02:50 |
| sdake | https://blueprints.launchpad.net/kolla/+spec/drop-root | 02:50 |
| sdake | only a few TODOs left | 02:51 |
| sdake | it is waiting on contributors not docker 1.10 | 02:51 |
| sean-k-mooney | ok cool weel ill put that to the top of my list. but yes ill look at the docs and take a look at the other open bugs too | 02:51 |
| sdake | sean-k-mooney should be pretty easy to copy the patterns used by other ocntainers | 02:52 |
| sdake | if you ahve neutron experience, drop root would e helpful | 02:52 |
| sdake | hae you deployed kolla? | 02:52 |
| sean-k-mooney | yes i have it running at home | 02:52 |
| sean-k-mooney | im also trying to get our team to adopt it | 02:53 |
| sdake | sweet top 3 pain points in order please? | 02:53 |
| sean-k-mooney | our corparte proxies but that is fixed with the inject file. am let me think | 02:54 |
| sdake | pretty sure kolla already works with corporate proxies, although its undocumented | 02:54 |
| sean-k-mooney | oh it does yes | 02:55 |
| sdake | dave-mccowan got it rolling today behind his corporate proxy | 02:55 |
| sean-k-mooney | our corparte proxies is just usally my main pain point | 02:55 |
| sdake | i know the last week has been a charlie foxtrot | 02:56 |
| sean-k-mooney | its fairly simple to do with the kolla-build header and futter | 02:56 |
| sdake | but that is atypical of our development :) | 02:56 |
| sdake | yup that is why the header and footer are there - for proxies | 02:56 |
| sdake | I would document it myself but I have no idea what to document ;) | 02:56 |
| sdake | if you want to document the proxy thing that would be helpful | 02:57 |
| sean-k-mooney | i guess right now for our team to adopt is just the ramp on ansible and docker. i have worked with docker before but ansible was new. overall the experince of using kolla has been good | 02:57 |
| dave-mccowan | sean-k-mooney sdake documented: https://review.openstack.org/282057 | 02:58 |
| dave-mccowan | i didn't get the header/footer to work | 02:58 |
| sean-k-mooney | one sec i can share mine if i sanities them | 02:59 |
| *** jasonsb has joined #kolla | 02:59 | |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Update Instructions for Building Behind HTTP Proxy https://review.openstack.org/282057 | 03:01 |
| *** Marga__ has quit IRC | 03:04 | |
| *** Marga_ has joined #kolla | 03:05 | |
| vhosakot | SamYaple, sdake: done, filed docker bug! :) https://github.com/docker/docker/issues/20477 | 03:06 |
| sean-k-mooney | dave-mccowan: the following is what i use https://etherpad.openstack.org/p/kolla-proxy-header-footer | 03:06 |
| sean-k-mooney | dave-mccowan: the pip and apt mirror are obviously optional but i like to run them locally to speed up the image build | 03:07 |
| *** vhosakot has quit IRC | 03:08 | |
| *** JoseMello has quit IRC | 03:08 | |
| *** salv-orlando has joined #kolla | 03:09 | |
| *** Jeffrey4l__ has joined #kolla | 03:11 | |
| *** salv-orlando has quit IRC | 03:11 | |
| sdake | ok folks i've hit the wall | 03:13 |
| sdake | time to PT FO | 03:13 |
| sdake | night night all :) | 03:13 |
| sean-k-mooney | oh just realised its 3 am... i should go to sleep too night | 03:14 |
| *** tzn has joined #kolla | 03:17 | |
| *** vhosakot has joined #kolla | 03:18 | |
| *** sdake has quit IRC | 03:21 | |
| *** rhallisey has quit IRC | 03:22 | |
| *** tzn has quit IRC | 03:22 | |
| vhosakot | other funny docker names :) | 03:23 |
| vhosakot | https://github.com/docker/docker/blob/630a5a23c73276faefaedd0b639ce1525c2bdc24/pkg/namesgenerator/names-generator.go#L30 | 03:23 |
| *** dims has quit IRC | 03:32 | |
| SamYaple | sean-k-mooney: if you build the package and host it _outside_ kolla in a repo then kolla can install it whith the current code. I don't see why it would need to be compiled as such by Kolla | 03:32 |
| *** dims has joined #kolla | 03:34 | |
| *** dims has quit IRC | 03:37 | |
| *** vhosakot has quit IRC | 03:37 | |
| *** unicell has quit IRC | 03:41 | |
| *** Marga_ has quit IRC | 03:42 | |
| *** Allen_Gao has quit IRC | 03:48 | |
| *** tfukushima has quit IRC | 03:50 | |
| *** tfukushima has joined #kolla | 04:02 | |
| *** Marga_ has joined #kolla | 04:05 | |
| *** Marga_ has quit IRC | 04:06 | |
| *** Marga_ has joined #kolla | 04:06 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Allow cinder-volume to be configured to use nfs https://review.openstack.org/279913 | 04:09 |
| *** yuanying_ has quit IRC | 04:10 | |
| openstackgerrit | Lingfeng Xiong proposed openstack/kolla: Add backward compatibility for liberty (ubuntu binary) https://review.openstack.org/279892 | 04:11 |
| *** rstarmer has joined #kolla | 04:19 | |
| *** Allen_Gao has joined #kolla | 04:19 | |
| *** Allen_Gao has quit IRC | 04:24 | |
| *** Allen_Gao has joined #kolla | 04:29 | |
| *** Allen_Gao has quit IRC | 04:33 | |
| *** Liuqing has quit IRC | 04:47 | |
| *** Allen_Gao has joined #kolla | 04:48 | |
| *** yuanying has joined #kolla | 04:50 | |
| *** salv-orlando has joined #kolla | 04:51 | |
| *** Allen_Gao has quit IRC | 04:53 | |
| *** Liuqing has joined #kolla | 04:56 | |
| *** salv-orlando has quit IRC | 05:07 | |
| *** tzn has joined #kolla | 05:07 | |
| *** tzn has quit IRC | 05:12 | |
| *** daneyon_ has quit IRC | 05:17 | |
| *** daneyon has joined #kolla | 05:17 | |
| *** salv-orlando has joined #kolla | 05:22 | |
| mandre | SamYaple, awesome! let's get your VM in the gate patch merged | 05:26 |
| *** salv-orlando has quit IRC | 05:28 | |
| *** dave-mccowan has quit IRC | 05:28 | |
| SamYaple | mandre: i think it only works on ovh gates.... | 05:28 |
| mandre | "only worked" or "only works"? | 05:28 |
| SamYaple | only worked | 05:30 |
| SamYaple | rax gates it "no valid hosts"'s on me | 05:30 |
| SamYaple | mandre: running it through some rechecks | 05:31 |
| mandre | SamYaple: hmmm, se we will never be completely good with a fixed setting for the hypervisor then? | 05:33 |
| mandre | we need to make it dependent on whre is running the CI node? | 05:33 |
| *** Allen_Gao has joined #kolla | 05:36 | |
| *** Allen_Gao has quit IRC | 05:41 | |
| *** yuanying has quit IRC | 05:43 | |
| SamYaple | mandre: no thats not it | 05:54 |
| SamYaple | its just an issue i cant pinpoint yet | 05:55 |
| *** Marga_ has quit IRC | 06:01 | |
| *** Marga_ has joined #kolla | 06:11 | |
| *** rstarmer has quit IRC | 06:12 | |
| *** Liuqing has quit IRC | 06:22 | |
| *** Liuqing has joined #kolla | 06:24 | |
| *** salv-orlando has joined #kolla | 06:34 | |
| *** kjelly has quit IRC | 06:36 | |
| *** salv-orlando has quit IRC | 06:37 | |
| *** Allen_Gao has joined #kolla | 06:41 | |
| *** Allen_Gao has quit IRC | 06:46 | |
| *** jasonsb has quit IRC | 06:52 | |
| *** tzn has joined #kolla | 06:56 | |
| *** tzn has quit IRC | 07:01 | |
| *** chandankumar has joined #kolla | 07:21 | |
| *** yuanying has joined #kolla | 07:28 | |
| *** unicell has joined #kolla | 07:30 | |
| *** aginwala has joined #kolla | 07:40 | |
| *** unicell has quit IRC | 07:43 | |
| *** unicell has joined #kolla | 07:44 | |
| *** alisonh has quit IRC | 07:48 | |
| *** Liuqing has quit IRC | 07:53 | |
| *** iceyao has quit IRC | 07:54 | |
| *** fgimenez has joined #kolla | 07:58 | |
| *** alisonh has joined #kolla | 08:06 | |
| *** Serlex has joined #kolla | 08:09 | |
| *** shardy has joined #kolla | 08:10 | |
| *** alisonh has quit IRC | 08:12 | |
| *** alisonh has joined #kolla | 08:15 | |
| *** asalkeld has quit IRC | 08:17 | |
| *** salv-orlando has joined #kolla | 08:22 | |
| *** mbound has joined #kolla | 08:30 | |
| *** salv-orlando has quit IRC | 08:31 | |
| -openstackstatus- NOTICE: Gerrit is going to be restarted due to performance problems | 08:32 | |
| *** athomas has quit IRC | 08:35 | |
| *** jmccarthy has joined #kolla | 08:43 | |
| *** liyi has joined #kolla | 08:43 | |
| *** mikelk has joined #kolla | 08:48 | |
| *** yuanying has quit IRC | 08:49 | |
| *** kproskurin has joined #kolla | 08:52 | |
| *** salv-orlando has joined #kolla | 09:03 | |
| *** achanda has quit IRC | 09:07 | |
| *** gfidente has joined #kolla | 09:09 | |
| *** achanda has joined #kolla | 09:12 | |
| *** aginwala has quit IRC | 09:29 | |
| *** achanda has quit IRC | 09:30 | |
| *** macsz has joined #kolla | 09:39 | |
| *** pbourke has joined #kolla | 09:40 | |
| *** akwasnie has joined #kolla | 09:42 | |
| elemoine_ | sdake, I created a Launchpad bug for the extend_start code duplication issue: https://bugs.launchpad.net/kolla/+bug/1546944. We can probably find a solution to that general problem. | 09:44 |
| openstack | Launchpad bug 1546944 in kolla "Tech-Debt: Duplicate code in extend_start.sh" [Wishlist,Triaged] | 09:44 |
| *** pbourke has quit IRC | 09:45 | |
| *** tzn has joined #kolla | 09:50 | |
| *** fgimenez_ has joined #kolla | 09:50 | |
| *** fgimenez has quit IRC | 09:50 | |
| *** tzn has quit IRC | 09:51 | |
| *** tzn has joined #kolla | 09:51 | |
| *** pbourke has joined #kolla | 09:58 | |
| *** liyi has quit IRC | 10:01 | |
| *** pbourke_ has joined #kolla | 10:02 | |
| *** salv-orl_ has joined #kolla | 10:03 | |
| *** pbourke has quit IRC | 10:05 | |
| *** salv-orlando has quit IRC | 10:06 | |
| *** Liuqing has joined #kolla | 10:12 | |
| *** Liuqing has quit IRC | 10:14 | |
| *** tpot has quit IRC | 10:14 | |
| Serlex | why are some docker images showing as none? | 10:18 |
| Serlex | after kolla-build | 10:18 |
| *** chandankumar has quit IRC | 10:19 | |
| *** tfukushima has quit IRC | 10:20 | |
| *** mbound has quit IRC | 10:20 | |
| *** fgimenez_ has left #kolla | 10:21 | |
| *** yuanying has joined #kolla | 10:25 | |
| *** shardy has quit IRC | 10:34 | |
| pbourke_ | Serlex: Im guessing you're doing 'docker images -a' | 10:39 |
| pbourke_ | the None are just intermediate layers afaik | 10:39 |
| pbourke_ | you shouldnt see them without the -a | 10:40 |
| Serlex | hmm ok | 10:40 |
| Serlex | I see them with docker images | 10:41 |
| Serlex | I failed couple failed kolla-builds, last one finished successfully | 10:41 |
| Serlex | I was hoping to see a clean list of docker images, but some of them still showing as none. So i'm not sure whether to deploy | 10:41 |
| pbourke_ | http://www.projectatomic.io/blog/2015/07/what-are-docker-none-none-images/ | 10:44 |
| Serlex | brilliant thank you! | 10:47 |
| *** tzn has quit IRC | 10:48 | |
| nihilifer | guys using ovs - are you sure that ovs-vsctl should point to the /run/openvswitch/db.sock everytime everywhere on every distro? | 10:48 |
| nihilifer | i'm trying to run ovs on centos-source containers right now | 10:48 |
| *** salv-orl_ has quit IRC | 10:48 | |
| nihilifer | and i'm always getting | 10:48 |
| nihilifer | ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) | 10:48 |
| *** Serlex has left #kolla | 10:48 | |
| nihilifer | for every ovs-vsctl command (no matter if that's kolla_ensure_openvswitch_configured or anything else) | 10:49 |
| nihilifer | when i specify the --db argument and provide a full path, it's working fine | 10:49 |
| *** pbourke_ has quit IRC | 10:50 | |
| *** yuanying has quit IRC | 10:51 | |
| *** pbourke has joined #kolla | 10:52 | |
| *** chandankumar has joined #kolla | 10:55 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla: [WIP] Provide ovs-db socket path directly https://review.openstack.org/275621 | 10:59 |
| *** achanda has joined #kolla | 11:01 | |
| *** chandankumar has quit IRC | 11:02 | |
| *** achanda has quit IRC | 11:06 | |
| *** rmart04 has joined #kolla | 11:14 | |
| *** mikelk has quit IRC | 11:22 | |
| *** mikelk has joined #kolla | 11:22 | |
| *** salv-orlando has joined #kolla | 11:29 | |
| *** JoseMello has joined #kolla | 11:29 | |
| *** shardy has joined #kolla | 11:31 | |
| *** Allen_Gao has joined #kolla | 11:31 | |
| *** akwasnie has quit IRC | 11:31 | |
| *** mbound has joined #kolla | 11:37 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Fix Mesos-managed volumes https://review.openstack.org/282293 | 11:39 |
| *** Allen_Ga_ has joined #kolla | 11:40 | |
| *** Allen_Gao has quit IRC | 11:40 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Fix Mesos-managed volumes https://review.openstack.org/282293 | 11:42 |
| *** Allen_Gao has joined #kolla | 11:44 | |
| *** tzn has joined #kolla | 11:44 | |
| *** Allen_Ga_ has quit IRC | 11:46 | |
| *** tzn has quit IRC | 11:48 | |
| *** tzn has joined #kolla | 11:48 | |
| *** dave-mccowan has joined #kolla | 11:49 | |
| *** akwasnie has joined #kolla | 11:50 | |
| *** chandankumar has joined #kolla | 11:50 | |
| *** Allen_Gao has quit IRC | 11:56 | |
| *** bmace has quit IRC | 11:56 | |
| *** bmace has joined #kolla | 11:57 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Fix Mesos-managed volumes https://review.openstack.org/282293 | 11:58 |
| *** Allen_Gao has joined #kolla | 12:00 | |
| *** Allen_Gao has quit IRC | 12:01 | |
| *** Allen_Gao has joined #kolla | 12:02 | |
| *** tzn has quit IRC | 12:05 | |
| *** rmart04 has quit IRC | 12:18 | |
| *** rhallisey has joined #kolla | 12:19 | |
| *** tzn has joined #kolla | 12:26 | |
| *** shardy has quit IRC | 12:29 | |
| *** dims_ has joined #kolla | 12:29 | |
| openstackgerrit | Allen Gao proposed openstack/kolla: Keep consistent with others for ironic containers https://review.openstack.org/280557 | 12:32 |
| *** stvnoyes has quit IRC | 12:33 | |
| *** stvnoyes has joined #kolla | 12:34 | |
| *** sdake has joined #kolla | 12:43 | |
| sdake | morning | 12:45 |
| *** iceyao has joined #kolla | 12:48 | |
| akwasnie | hi sdake | 12:49 |
| sdake | hey akwasnie | 12:49 |
| sdake | did you sync up with elemoine on the merge plan for diags? | 12:49 |
| akwasnie | yes, he prepared a good plan, I just added one point about making sure Kibana is working correctly witk Heka and Elasticsearch | 12:50 |
| sdake | ok, fwiw normally we process patches in order submitted and yours was first, so this is a bit unfair to you | 12:50 |
| sdake | apologies for that, but sometimes technical reasons make a different ordering more appropriate | 12:51 |
| *** tzn has quit IRC | 12:51 | |
| sdake | i think the current ordering plan is the right order | 12:51 |
| sdake | buttom up | 12:51 |
| akwasnie | sdake, np:) | 12:51 |
| akwasnie | I think it is a right order too | 12:52 |
| *** pbourke has quit IRC | 13:01 | |
| elemoine_ | sdake: I hope my comment regarding file_match was clear enough | 13:01 |
| elemoine_ | otherwise I'm happy to discuss this more | 13:01 |
| sdake | elemoine i didn't see it | 13:01 |
| sdake | can you give me the tldr | 13:02 |
| sdake | elemoine note I dindnt -1 your patch :) | 13:02 |
| elemoine_ | https://review.openstack.org/#/c/279489/ | 13:02 |
| sdake | elemoine yes but why is not swift and cinder and heat in there | 13:03 |
| elemoine_ | I have separate patches/commits for cinder and heat | 13:04 |
| sdake | yes but its the same file | 13:04 |
| elemoine_ | and these patches come after that of Neutron | 13:04 |
| sdake | shouldn't that file_match have swift|cinder|heat|glance|neutron? | 13:04 |
| elemoine_ | this is the patch for cinder : https://review.openstack.org/#/c/281259/8/ansible/roles/common/templates/heka-openstack.toml.j2 | 13:04 |
| sdake | what do you expect file_match should be at he conclusion of hte merging | 13:05 |
| *** pbourke has joined #kolla | 13:05 | |
| elemoine_ | the last openstack service patch is magnum: https://review.openstack.org/#/c/281329 | 13:06 |
| elemoine_ | so https://review.openstack.org/#/c/281329/7/ansible/roles/common/templates/heka-openstack.toml.j2 | 13:06 |
| elemoine_ | is the final state | 13:06 |
| elemoine_ | note that Swift is not treated the same way – Swift still uses Syslog | 13:07 |
| elemoine_ | see https://review.openstack.org/#/c/279490/9 (wich you -1'ed) | 13:07 |
| sdake | elemoine must havebeen tired during original review | 13:09 |
| sdake | ignore my comment | 13:09 |
| sdake | i'll fix up the reviews now jus ta moment | 13:09 |
| elemoine_ | no problem at all | 13:10 |
| *** fthiagogv__ has quit IRC | 13:12 | |
| *** Allen_Gao has quit IRC | 13:13 | |
| *** Serlex has joined #kolla | 13:14 | |
| sdake | elemoine I thought gerrit top was first patch gerrit bottom was last patch | 13:15 |
| sdake | elemoine your sayign gerrit top patch is last patch, gerrit bottom is first patch | 13:15 |
| sdake | elemoine this is why I was confused by the comment | 13:15 |
| sdake | err by the change | 13:15 |
| * sdake learning new gerrit ropes yet again | 13:15 | |
| elemoine_ | gerrit is broken :) | 13:15 |
| *** Allen_Gao has joined #kolla | 13:16 | |
| sdake | i like the related changes full view | 13:16 |
| elemoine_ | it depends on the current patch | 13:16 |
| sdake | thats a cool feature | 13:16 |
| elemoine_ | I like it too but I find the order confusing at times | 13:16 |
| sdake | elemoine did you get my request for filing a techdebt bug | 13:18 |
| sdake | about unifying all the commonality you have in extend_start if thats possible | 13:18 |
| elemoine_ | I had done it already | 13:18 |
| elemoine_ | just a sec | 13:18 |
| sdake | you had filed a tech debt bug? | 13:18 |
| elemoine_ | yes | 13:18 |
| elemoine_ | https://bugs.launchpad.net/kolla/+bug/1546944 | 13:19 |
| openstack | Launchpad bug 1546944 in kolla "Tech-Debt: Duplicate code in extend_start.sh" [Wishlist,Triaged] | 13:19 |
| *** salv-orlando has quit IRC | 13:21 | |
| sdake | elemoine can you confirm that all heka work is done with this patch set? | 13:22 |
| *** pbourke has quit IRC | 13:23 | |
| sdake | elemoine what is your launchpad id? | 13:23 |
| elemoine_ | we need to remove Rsyslog but SamYaple suggested that we do that after merging this patchset | 13:23 |
| sdake | agreed | 13:23 |
| elemoine_ | launchpad id: elemoine | 13:23 |
| sdake | elemoine tbh I didn't think you would hit our deadlines :) | 13:24 |
| sdake | any bugs found can be fixed in the rcs | 13:24 |
| elemoine_ | and I'm currently fixing Swift | 13:24 |
| sdake | nice job! | 13:24 |
| elemoine_ | thanks | 13:24 |
| sdake | i want the full stack tho, so work with akwasnie to get her work merged asap | 13:24 |
| *** pbourke has joined #kolla | 13:24 | |
| sdake | whatever it takes | 13:24 |
| sdake | hey pbourke | 13:24 |
| elemoine_ | sure | 13:24 |
| pbourke | sdake: hey | 13:24 |
| sdake | we have 2 weeks left | 13:24 |
| elemoine_ | I've reviewed her ES patch many times and it looks good to me | 13:25 |
| *** Allen_Gao has quit IRC | 13:25 | |
| sdake | elemoine interested in joining the kolla-drivers team? | 13:25 |
| sdake | these folks have ability to flip bits in launchpad | 13:25 |
| sdake | triage bugs and whatnot | 13:25 |
| *** Allen_Gao has joined #kolla | 13:25 | |
| *** akwasnie has quit IRC | 13:26 | |
| sdake | cool | 13:26 |
| sdake | apologies for coming to these reviews late | 13:27 |
| sdake | but I thnk there will need to be some integration work done between the two patchsets | 13:27 |
| *** Allen_Gao has quit IRC | 13:27 | |
| *** Allen_Gao has joined #kolla | 13:28 | |
| elemoine_ | sdake: sure! | 13:31 |
| sdake | elemoine what we need now is rechecks on your patchset when it fials the gate | 13:31 |
| sdake | i have kicked off some | 13:31 |
| sdake | and will finish all the rechecks in this patchset this morning | 13:31 |
| sdake | but I am not around24/7 | 13:31 |
| *** Allen_Ga_ has joined #kolla | 13:31 | |
| sdake | I need to sleep and eat and take care of my children | 13:31 |
| sdake | so if you chould scan the patch set every couple hours for gate failures and recheck the ones that fail ci | 13:32 |
| sdake | every 2 hours should do | 13:32 |
| elemoine_ | ok | 13:32 |
| sdake | i think we are good to merge this entire patch set | 13:32 |
| sdake | from what I've seen soo far | 13:32 |
| elemoine_ | as I said I am working on debugging Swift right now | 13:32 |
| elemoine_ | but that can be a new patch (on top of the rest) | 13:33 |
| elemoine_ | to avoid rebasing the whole thing | 13:33 |
| *** Allen_Gao has quit IRC | 13:35 | |
| sdake | elemoine_ re: | 13:39 |
| sdake | https://review.openstack.org/#/c/275842/22 | 13:39 |
| elemoine_ | yep | 13:40 |
| sdake | it logs to /var/log/kolla/heka | 13:40 |
| sdake | is that correct for keepalived and haproxy? | 13:40 |
| sdake | shouldn't it log to /var/log/kolla/loadbalance? | 13:40 |
| sdake | or something like that | 13:40 |
| sdake | or /var/log/kolla/high-availability | 13:40 |
| elemoine_ | it logs to /var/lib/kolla/heka/log | 13:40 |
| sdake | log is a log device? | 13:41 |
| *** allen has joined #kolla | 13:41 | |
| elemoine_ | log is a Unix socket, created by Heka for Syslog-type logging | 13:41 |
| sdake | ok | 13:41 |
| sdake | thanks | 13:41 |
| elemoine_ | it works the same way as Rsyslog here | 13:41 |
| *** allen has quit IRC | 13:41 | |
| elemoine_ | and I'm trying to do the same for Swift | 13:42 |
| *** dwalsh_ has joined #kolla | 13:42 | |
| *** dwalsh has joined #kolla | 13:42 | |
| *** Allen_Gao_ has joined #kolla | 13:43 | |
| *** Allen_Ga_ has quit IRC | 13:45 | |
| sdake | elemoine_ rpbolem found in https://review.openstack.org/#/c/275841/20 | 13:45 |
| *** Allen_Gao has joined #kolla | 13:45 | |
| sdake | please file a techdebt bug to fix it | 13:45 |
| elemoine_ | rename of the volume? | 13:46 |
| elemoine_ | the "heka" volume | 13:46 |
| sdake | yes | 13:46 |
| sdake | heka could be used by soemone else | 13:46 |
| elemoine_ | what about mariadb? | 13:47 |
| sdake | this would surely cuse problems no? | 13:47 |
| *** Allen_Gao has quit IRC | 13:47 | |
| sdake | what is mariadb's volume name? | 13:47 |
| elemoine_ | and rabbitmq? | 13:47 |
| elemoine_ | run docker volume ls | 13:47 |
| *** Allen_Gao_ is now known as Allen_Gao | 13:47 | |
| sdake | i dont have a deployment that works | 13:47 |
| elemoine_ | and see the volume names used by kolla | 13:47 |
| *** shardy has joined #kolla | 13:47 | |
| sdake | can you paste that for m e please | 13:48 |
| elemoine_ | sdake: http://paste.openstack.org/show/487573/ | 13:48 |
| sdake | my gear is imapired | 13:48 |
| elemoine_ | basically we do not namespace volumes | 13:48 |
| sdake | ok file a bug that all named volumes must be prepended with kolla_ | 13:48 |
| sdake | and we can tackle as one piece of work | 13:48 |
| elemoine_ | yeah, a more general bug | 13:49 |
| elemoine_ | I'll do it | 13:49 |
| sdake | using mariadb as a named volume is sure to cause problems | 13:49 |
| sdake | elemoine_ link bug so i can triage please | 13:51 |
| *** achanda has joined #kolla | 13:52 | |
| *** salv-orlando has joined #kolla | 13:54 | |
| elemoine_ | I haven't created it yet | 13:54 |
| elemoine_ | you want it now? | 13:54 |
| sdake | elemoine https://review.openstack.org/#/c/275840/14 | 13:59 |
| limamauricio | nova uses ceph to boot an image? | 13:59 |
| sdake | yes file all tech debt bugs now | 13:59 |
| sdake | so Ican triage | 13:59 |
| sdake | limamauricio if ceph is configured, yes | 13:59 |
| sdake | if ceph is configured, glance and nova data are stored in ceph | 13:59 |
| sdake | including boot images etc | 14:00 |
| limamauricio | so, what virt_type i should use? | 14:00 |
| limamauricio | qemu? | 14:01 |
| *** diogogmt has quit IRC | 14:01 | |
| sdake | if your in a vm yes | 14:01 |
| *** tzn has joined #kolla | 14:01 | |
| *** tzn has quit IRC | 14:01 | |
| *** tzn has joined #kolla | 14:01 | |
| limamauricio | hmm | 14:01 |
| *** fthiagogv has joined #kolla | 14:02 | |
| limamauricio | i'm failing | 14:02 |
| limamauricio | :/ | 14:02 |
| *** diogogmt has joined #kolla | 14:02 | |
| sdake | ok well i've got the school rush now, so I can't help unfortunately - I'll be back later | 14:03 |
| limamauricio | ok | 14:03 |
| limamauricio | thanks | 14:03 |
| sdake | elemoine your patch set looks good to go, lets get the gate at 100% on all the patch sets | 14:03 |
| sdake | elemoine and get those tech debt bugs filed on the first patch | 14:04 |
| sdake | elemoine and get them fixed in your local repo so they are ready to go | 14:04 |
| sdake | the chown 400 seems most critical - moreso then swift | 14:04 |
| limamauricio | someone already boot an image successfully using centos + ceph? | 14:05 |
| *** jtriley has quit IRC | 14:06 | |
| *** spisarski has quit IRC | 14:08 | |
| elemoine_ | sdake https://bugs.launchpad.net/kolla/+bug/1547533 | 14:12 |
| openstack | Launchpad bug 1547533 in kolla "Tech-Debt: Namespace the Docker volumes" [Undecided,New] | 14:12 |
| *** pbourke has quit IRC | 14:12 | |
| *** pbourke has joined #kolla | 14:13 | |
| sdake | actually I'm not sure namespacing makes sense | 14:14 |
| sdake | since we expect dedicated gear | 14:14 |
| sdake | and the rest of our services are nto namespaced | 14:14 |
| elemoine_ | I was wondering too | 14:14 |
| sdake | well argue more with me then :) | 14:15 |
| sdake | 7am - brain not working at its finest :) | 14:15 |
| limamauricio | someone already boot an image successfully using centos + ceph? | 14:15 |
| sdake | limamauricio yes i have in the past | 14:15 |
| sdake | not after 1.10 tho ;) - my env is not operational enough to do so | 14:16 |
| limamauricio | :/ | 14:16 |
| elemoine_ | sdake: I preferred to create the bug and argue there :) | 14:16 |
| limamauricio | i tried to boot an image, but it failed | 14:16 |
| sdake | ok | 14:17 |
| limamauricio | now i dont know if my deploy is working well | 14:17 |
| sdake | well we do need tracking bugs for the first patch in your series | 14:17 |
| sdake | the chmod 400 | 14:17 |
| sdake | I am retty sure sudoers file must be chmod 400 | 14:17 |
| sdake | and owned by root: | 14:17 |
| sdake | heka (and rsyslog0 are definately different then mariadb | 14:18 |
| sdake | heka or rsyslog may be installed in a container by default installation of os down the road | 14:19 |
| sdake | its a tough call | 14:19 |
| sdake | namespacing everything would be best ;-) | 14:19 |
| sdake | but that makes upgrades a huge pita | 14:19 |
| Serlex | Hi guys, is it safe to reload a vagrant machine after host was shutdown? | 14:20 |
| elemoine_ | sdake: I created bugs for those as well | 14:24 |
| *** achanda has quit IRC | 14:24 | |
| *** achanda has joined #kolla | 14:24 | |
| *** achanda has quit IRC | 14:25 | |
| *** akwasnie has joined #kolla | 14:29 | |
| *** macsz has quit IRC | 14:30 | |
| *** jtriley has joined #kolla | 14:33 | |
| akwasnie | https://review.openstack.org/#/c/267714/ - can anyone take a look and do a review? | 14:33 |
| akwasnie | with elemoine we would like to start integration of heka and elasticsearch | 14:34 |
| akwasnie | and this review blocks us a bit :) | 14:34 |
| *** diogogmt has quit IRC | 14:37 | |
| openstackgerrit | Proskurin Kirill proposed openstack/kolla-mesos: Fix nova group name in nova.conf.j2 https://review.openstack.org/280465 | 14:38 |
| *** openstackgerrit_ has quit IRC | 14:43 | |
| *** chandankumar has quit IRC | 14:43 | |
| *** openstackgerrit_ has joined #kolla | 14:44 | |
| nihilifer | if no "more eyes" will look at that in ~30 mins, i'm just going to give +1w | 14:47 |
| *** diogogmt has joined #kolla | 14:49 | |
| *** openstackgerrit_ has quit IRC | 14:49 | |
| openstackgerrit | Merged openstack/kolla-mesos: Fix nova group name in nova.conf.j2 https://review.openstack.org/280465 | 14:50 |
| akwasnie | nihilifer: nice :P | 14:51 |
| *** spisarski has joined #kolla | 14:57 | |
| *** openstackgerrit_ has joined #kolla | 14:57 | |
| openstackgerrit | Proskurin Kirill proposed openstack/kolla-mesos: Fix glance-registry enable typo https://review.openstack.org/280466 | 14:58 |
| *** openstackgerrit_ has quit IRC | 14:59 | |
| *** openstackgerrit_ has joined #kolla | 15:00 | |
| *** akwasnie has quit IRC | 15:02 | |
| pbourke | SamYaple: ping | 15:02 |
| *** tzn has quit IRC | 15:03 | |
| openstackgerrit | Proskurin Kirill proposed openstack/kolla-mesos: Split glance group into 2 groups https://review.openstack.org/282372 | 15:04 |
| *** unicell has quit IRC | 15:04 | |
| *** Jeffrey4l__ has quit IRC | 15:07 | |
| pbourke | anyone here working on upgrades atm? | 15:07 |
| *** jasonsb has joined #kolla | 15:10 | |
| *** dave-mccowan has quit IRC | 15:11 | |
| openstackgerrit | Merged openstack/kolla-mesos: Fix glance-registry enable typo https://review.openstack.org/280466 | 15:13 |
| openstackgerrit | Proskurin Kirill proposed openstack/kolla-mesos: Split glance group into 2 groups https://review.openstack.org/282372 | 15:15 |
| *** blahRus has joined #kolla | 15:18 | |
| *** d_code has joined #kolla | 15:18 | |
| SamYaple | pbourke: pong | 15:24 |
| pbourke | SamYaple: hey man | 15:24 |
| *** dave-mccowan has joined #kolla | 15:25 | |
| pbourke | SamYaple: I have the following scenario | 15:25 |
| *** achanda has joined #kolla | 15:25 | |
| pbourke | SamYaple: basically you have a running liberty setup, then move to master and run an upgrade task | 15:25 |
| openstackgerrit | Merged openstack/kolla: Added Elasticearch and its deployment. https://review.openstack.org/267714 | 15:25 |
| pbourke | SamYaple: what Im seeing is the common role does not run, so no kolla-toolbox | 15:25 |
| pbourke | SamYaple: any idea why that might be, i.e. it runs for -e action=deploy but not upgrade | 15:26 |
| SamYaple | pbourke: i dont quite follow | 15:27 |
| SamYaple | the common role will run once per play | 15:27 |
| SamYaple | can you give me a way to replicate it? | 15:27 |
| pbourke | sure, one sec | 15:27 |
| *** dims_ has quit IRC | 15:27 | |
| SamYaple | hey sdake mandre I would like to merge this https://review.openstack.org/#/c/280749 | 15:27 |
| limamauricio | I ran docker exec ceph_osd_0 ceph -s and i have it http://pastebin.com/Y3sNveAr | 15:27 |
| pbourke | or more a few mins, I'll roll back my env and try to get a set of steps | 15:27 |
| SamYaple | but I dont want the failure to schedule vm to fail the gate | 15:28 |
| SamYaple | so i suggest we launch the vm but dont track the results | 15:28 |
| SamYaple | it will still fail if, say, it cant create a network or upload a glance image | 15:28 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: [WIP] Add openvswitch support https://review.openstack.org/282384 | 15:28 |
| SamYaple | but for the booting itself, I dont want to fail for that reason | 15:28 |
| limamauricio | ceph is not running very well, right? | 15:29 |
| openstackgerrit | Merged openstack/kolla: Launch VM in gate https://review.openstack.org/280749 | 15:29 |
| *** mbound has quit IRC | 15:29 | |
| elemoine_ | all the Heka patches but the very first one passed the gates | 15:29 |
| elemoine_ | I'm not sure what to do about https://review.openstack.org/#/c/275840/ | 15:30 |
| *** achanda has quit IRC | 15:30 | |
| *** sdake_ has joined #kolla | 15:30 | |
| elemoine_ | should I recheck it? | 15:31 |
| *** sdake has quit IRC | 15:31 | |
| *** mbound has joined #kolla | 15:31 | |
| *** Marga_ has quit IRC | 15:33 | |
| SamYaple | limamauricio: I have typically used ceph exclusively, why do you ask | 15:35 |
| openstackgerrit | Merged openstack/kolla-mesos: Split glance group into 2 groups https://review.openstack.org/282372 | 15:35 |
| limamauricio | I can not boot an image in the nova | 15:36 |
| SamYaple | elemoine_: that is unrelated to teh failures | 15:36 |
| SamYaple | elemoine_: you can clearly see your stuff builds fine. a recheck should not be needed | 15:36 |
| limamauricio | and i do know if i deploy it correctly | 15:37 |
| SamYaple | sdake_: ping ^ no need to recheck that in my opinion | 15:37 |
| *** chandankumar has joined #kolla | 15:37 | |
| elemoine_ | understood | 15:38 |
| elemoine_ | SamYaple: I am currently debugging Swift (and making progress) | 15:38 |
| elemoine_ | but I want to fix it with a new CR, as opposed to doing a rebase | 15:39 |
| elemoine_ | I am pretty sure you'll agree with that :) | 15:39 |
| limamauricio | i'm trying to build and deploy kolla + centos + binary | 15:39 |
| SamYaple | elemoine_: i do. this is why i dont like 15 patches in a stream personally | 15:40 |
| SamYaple | sdake is a big fan of them for some reason | 15:40 |
| *** inc0 has joined #kolla | 15:40 | |
| inc0 | good morning | 15:40 |
| SamYaple | morning inc0 | 15:40 |
| elemoine_ | and it's hard to maintain for me | 15:40 |
| SamYaple | elemoine_: i would personally soon seen larger patchsets for stuff like this, requiring more thorugh review | 15:42 |
| elemoine_ | SamYaple: what do you mean? I did not understand you | 15:43 |
| SamYaple | elemoine_: dont worry about it. the takeaway is i dont like 20 patchset streams for these reasons! | 15:44 |
| sdake_ | the reason I am a fan is so that patches may be reviewed as one unit if people care to do so | 15:46 |
| sdake_ | ok i've got a dr appointment back in few hours | 15:46 |
| sdake_ | id like the rechecks to finish up - only 3-4 patches don't pass the gate | 15:46 |
| sdake_ | we should be able to merge today if we stay on top of the rechecks | 15:47 |
| SamYaple | sdake_: all of them pass now | 15:47 |
| sdake_ | all gates/ | 15:47 |
| sdake_ | i rechecked them a couple hours ago | 15:47 |
| SamYaple | please dont kill infra resources for unrelated failures.... | 15:47 |
| sdake_ | i've really got to jet, i'll remove my -2 | 15:47 |
| sdake_ | infra has load management in their system | 15:48 |
| sdake_ | they can handle large patch sets | 15:48 |
| sdake_ | gotta jet | 15:48 |
| SamYaple | its drags it all down man | 15:48 |
| sdake_ | this is not passing the gate https://review.openstack.org/#/c/275841/ | 15:48 |
| SamYaple | sdake_: i just saw it and issued recheck | 15:49 |
| *** fthiagogv has quit IRC | 15:49 | |
| *** fthiagogv has joined #kolla | 15:49 | |
| sdake_ | this is not passing gate https://review.openstack.org/#/c/279490/9 | 15:49 |
| elemoine_ | SamYaple, sdake_ sorry it was me saying that they were all passing the gate | 15:50 |
| SamYaple | elemoine_: no i looked too | 15:50 |
| SamYaple | this is the problem with 20 patchset streams... | 15:50 |
| elemoine_ | I got confused with something else | 15:50 |
| SamYaple | easy to miss things | 15:50 |
| sdake_ | this is not passing gate: | 15:50 |
| sdake_ | https://review.openstack.org/#/c/275841/20 | 15:50 |
| SamYaple | sdake_: you already listed that one | 15:51 |
| sdake_ | oh sorry | 15:51 |
| sdake_ | ok those 3 need to pass gate | 15:51 |
| SamYaple | 2 | 15:51 |
| sdake_ | then I'll remove -1 | 15:51 |
| SamYaple | 2 | 15:51 |
| sdake_ | rather -2 | 15:51 |
| limamauricio | SamYaple, i want to user qemu, but it not works | 15:51 |
| sdake_ | ok gotta jet back in couple hours | 15:51 |
| SamYaple | sdake_: no theres only 2 | 15:51 |
| sdake_ | by then hopefully the whole stream passing gate | 15:51 |
| sdake_ | bbiaf | 15:51 |
| *** dencaval has joined #kolla | 15:51 | |
| elemoine_ | the latter is a python27 problem, while my code does not include python code :) | 15:51 |
| sdake_ | elemoine_ i understand | 15:51 |
| SamYaple | elemoine_: in this case though that doesnt matter | 15:51 |
| sdake_ | python is a voting gate | 15:52 |
| SamYaple | sdake_: actually no thats a check | 15:52 |
| sdake_ | so it has to pass absolutely or the whoel patch stream wont merge | 15:52 |
| SamYaple | it can merge ithout passing that | 15:52 |
| SamYaple | it just has to pass the check run | 15:52 |
| sdake_ | oh i thought it was gating | 15:52 |
| sdake_ | it hsould be gating ;-) | 15:52 |
| sdake_ | bbiaf | 15:52 |
| sdake_ | seriously ahve to run | 15:52 |
| SamYaple | it is gating | 15:52 |
| SamYaple | thats a check though | 15:52 |
| limamauricio | you do it to use qemu? the default value is kvm... | 15:52 |
| sdake_ | yes i know | 15:52 |
| SamYaple | gating doesnt occur until merge | 15:52 |
| SamYaple | you can merge with failed checks | 15:52 |
| sdake_ | yes I understand how gating works :) | 15:53 |
| sdake_ | ok gotta roll | 15:53 |
| SamYaple | then we dont need to recheck | 15:53 |
| SamYaple | if you understand :0 | 15:53 |
| pbourke | SamYaple: ok - deploy liberty keystone+rabbitmq+mariadb. Switch to master, upgrade just keystone. kolla-toolbox is not started | 15:54 |
| openstackgerrit | Merged openstack/kolla-mesos: Fix Mesos-managed volumes https://review.openstack.org/282293 | 15:54 |
| pbourke | SamYaple: it says 'registering common role has run' but then skips on ahead. whats the deal with that logic | 15:55 |
| pbourke | so it doesn't run for every role? | 15:55 |
| *** tzn has joined #kolla | 15:55 | |
| SamYaple | pbourke: ah look at common/tasks/upgrade.yml | 15:55 |
| pbourke | its empty... | 15:56 |
| SamYaple | yup | 15:56 |
| SamYaple | its a stub upgrade file | 15:56 |
| pbourke | ah | 15:56 |
| pbourke | ok | 15:56 |
| *** fthiagogv has quit IRC | 15:56 | |
| pbourke | makes sense | 15:56 |
| limamauricio | someone already got to create an instance of an image using ceph? | 15:56 |
| *** Liuqing has joined #kolla | 15:57 | |
| SamYaple | limamauricio: all the time | 15:57 |
| limamauricio | aio? | 15:57 |
| openstackgerrit | Derek Ditch proposed openstack/kolla: Added script that can build master binary images on centos. https://review.openstack.org/282399 | 15:57 |
| inc0 | limamauricio, did you make changes to ceph that enables air? | 15:57 |
| d_code | sdake_: https://review.openstack.org/282399 | 15:57 |
| limamauricio | yes | 15:57 |
| limamauricio | i did | 15:58 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Upgrade common role https://review.openstack.org/282400 | 15:58 |
| inc0 | what are errors then? | 15:58 |
| SamYaple | pbourke: https://review.openstack.org/282400 | 15:58 |
| *** fthiagogv has joined #kolla | 15:58 | |
| limamauricio | inc0 http://pastebin.com/TA1Xhdjb docker logs nova_pompute | 16:00 |
| limamauricio | compute* | 16:00 |
| *** pbourke_ has joined #kolla | 16:00 | |
| *** tzn has quit IRC | 16:00 | |
| *** pbourke has quit IRC | 16:01 | |
| *** akwasnie has joined #kolla | 16:02 | |
| inc0 | what kernel do you have on your base system? | 16:03 |
| *** coolsvap|away has quit IRC | 16:03 | |
| inc0 | seems like something that can't talk to rbd | 16:03 |
| d_code | SamYaple: I just read your selinux docs in the repo. Do you know of any specific operations that have caused issues in the past? I’m running selinux in Permissive mode and have yet to get an AVC denial | 16:03 |
| limamauricio | 3.10.0-327.el7.x86_64 | 16:03 |
| limamauricio | centos | 16:03 |
| SamYaple | d_code: there are many many operations that cause problems, basically all sudo commands | 16:04 |
| SamYaple | mariadb extend_start.sh chown /var/lib/mysql | 16:04 |
| d_code | which happens when? Do I need to create new domains / projects / networks? | 16:04 |
| SamYaple | d_code: "A SELinux-enabled system that runs in permissive mode is not protected by SELinux." | 16:05 |
| SamYaple | just fyi | 16:05 |
| d_code | SamYaple: I’m well aware…and I don’t like being in this state. My intention was to help create policies as I’m quite familiar with selinux | 16:05 |
| limamauricio | inc0, I need to change something in the globals file besides enable_ceph: "yes" | 16:05 |
| limamauricio | SamYaple, i have to run setenforce 0 and set selinux=permissive? | 16:06 |
| SamYaple | limamauricio: that should be enough | 16:06 |
| *** salv-orlando has quit IRC | 16:06 | |
| inc0 | SamYaple, looking at limamauricio trace it seems that libvirt doesn't know how to talk to cep | 16:07 |
| inc0 | ceph | 16:07 |
| inc0 | it should be in kernel tho | 16:07 |
| SamYaple | inc0: yea i saw. limamauricio is the first one to use centos-source for ceph it seems, so likely brekage | 16:07 |
| SamYaple | inc0: kernel is not involved here | 16:07 |
| inc0 | ok | 16:08 |
| limamauricio | I may have forgotten some configuration | 16:08 |
| limamauricio | i'm changing to binary | 16:08 |
| limamauricio | :) | 16:08 |
| inc0 | hmm...I don't know if anyone used ceph+centos really | 16:08 |
| SamYaple | limamauricio: i would suggest testing with binary | 16:08 |
| SamYaple | limamauricio: im the major proponent of ceph here, and I use ubuntu | 16:08 |
| limamauricio | i'll do | 16:08 |
| SamYaple | sdake deploys centos-binary ceph | 16:08 |
| inc0 | both me and Sam use ceph, but we're both ubuntu | 16:08 |
| d_code | SamYaple: regarding usefulness of build script, as I was stepping through the quickstart and happened to discover kolla when neither master nor liberty would build for binary or source, I walked through the quickstart and found issues that were not 100% clear on how to proceed. sdake_ was a huge help and I produced this script mainly so I could reproduce my results | 16:08 |
| limamauricio | ok | 16:09 |
| d_code | sdake_ asked me to submit it | 16:09 |
| d_code | I can ansiblize it, but don’t have time just yet | 16:09 |
| SamYaple | d_code: i meant usefulness being non-idempotent | 16:09 |
| SamYaple | d_code: that script breaks if you run it twice | 16:09 |
| *** coolsvap|away has joined #kolla | 16:09 | |
| d_code | well…not 100%… the git clone and the docker start would fail…I could put checks around those | 16:10 |
| d_code | pip doesn’t care if you run the install more than once, and you can kolla-build all day long (which I did) | 16:10 |
| d_code | but, could check the state of those items and quit if it’s already there | 16:11 |
| d_code | or skip | 16:11 |
| SamYaple | if you run that scirpt to kolla build and the build breaks halfway through but you had previous good builds you can easily end up in a state of different base images and not know it | 16:11 |
| d_code | oh…yeah…that’s true | 16:11 |
| SamYaple | this is why we recommend cleaning the environment before rebuilding | 16:11 |
| d_code | so clear out the images? | 16:11 |
| d_code | run cleanup-images | 16:11 |
| d_code | or containers maybe | 16:11 |
| SamYaple | im just saying the script needs a bit of work before merge, as is i feel like it may cause more issues that it solves | 16:11 |
| SamYaple | it shouldnt break on rerun and it should have a bit more error handling | 16:12 |
| d_code | sure thing | 16:12 |
| SamYaple | but the script itself I am all for if it helps which i think it will | 16:12 |
| d_code | fair enough. thanks for the feedback :) | 16:12 |
| SamYaple | anytime | 16:12 |
| *** akwasnie has quit IRC | 16:14 | |
| *** g3ek has quit IRC | 16:14 | |
| *** g3ek has joined #kolla | 16:15 | |
| d_code | SamYaple: I’ve never used get-pip-.py. Does that let us skip the setuptool depenency? | 16:16 |
| d_code | I found the script | 16:16 |
| d_code | ah…nm… it will install it if not there | 16:17 |
| limamauricio | I'm not finding no variable which is defined the virt_type | 16:18 |
| limamauricio | nova default is kvm | 16:19 |
| limamauricio | but i want to use qemu | 16:19 |
| *** unicell has joined #kolla | 16:20 | |
| Serlex | GATHERING FACTS *************************************************************** | 16:20 |
| Serlex | fatal: [control01] => failed to transfer file to /root/.ansible/tmp/ansible-tmp-1455898618.83-24426472018452/setup: | 16:20 |
| openstackgerrit | Merged openstack/kolla: Aodh ubuntu binary container https://review.openstack.org/280515 | 16:21 |
| SamYaple | limamauricio: http://docs.openstack.org/developer/kolla/deployment-philosophy.html#kolla-s-solution-to-customization | 16:23 |
| *** jasonsb has quit IRC | 16:23 | |
| limamauricio | :) | 16:24 |
| limamauricio | SamYaple, i already did it | 16:24 |
| limamauricio | :) | 16:24 |
| limamauricio | but now i'm testing in binary | 16:25 |
| elemoine_ | sdake_ SamYaple I'll soon leave the office for the week-end, feel free to tell me if there's anything I can do for Heka and the gates. I'm happy to make changes if necessary. I have other patches coming up. Have a good WE. | 16:26 |
| SamYaple | thanks elemoine_ ! | 16:27 |
| *** achanda has joined #kolla | 16:27 | |
| *** vhosakot has joined #kolla | 16:31 | |
| vhosakot | SamYaple: morning, I am running kolla-build with 2 threads and see it stuck at "DEBUG:kolla.cmd.build:manila-base:Getting archive from http://tarballs.openstack.org/manila/manila-master.tar.gz" for 12 hours now ? | 16:32 |
| *** achanda has quit IRC | 16:33 | |
| SamYaple | vhosakot: sounds like its your network connection/tarballs.opesntack.org failure | 16:33 |
| SamYaple | i would just restart teh build | 16:33 |
| vhosakot | restarted many times... same issue | 16:34 |
| SamYaple | vhosakot: can you run `http://tarballs.openstack.org/manila/manila-master.tar.gz` | 16:34 |
| SamYaple | `wget http://tarballs.openstack.org/manila/manila-master.tar.gz` | 16:34 |
| sbezverk | SamYaple I built docker container with iscsid and taregtcli, when iscsiadm does discovery from another node, I can see shared by this container targets.. Atthis point everything is manually configured, could you pprovide some guidance how it can be packed as a patch, I have never done it before.. | 16:36 |
| vhosakot | SamYaple: the wget worked! | 16:36 |
| vhosakot | but, it is hung in kolla-build | 16:36 |
| vhosakot | SamYaple: does kolla-build retry hung wget's ? | 16:38 |
| *** salv-orlando has joined #kolla | 16:40 | |
| *** iceyao has quit IRC | 16:42 | |
| *** mikelk has quit IRC | 16:49 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Upgrade task for Swift https://review.openstack.org/276855 | 16:49 |
| *** pbourke_ has quit IRC | 16:49 | |
| *** pbourke has joined #kolla | 16:53 | |
| *** tzn has joined #kolla | 16:55 | |
| vhosakot | SamYaple: I see kolla-build does not timeout if wget fails.. I'm seeing this issue... shall I file a bug for this ? ad send the fix (I will add a tirmeout value and retry).. Let me know ? | 16:57 |
| openstackgerrit | Derek Ditch proposed openstack/kolla: Addressed whitespace and idempotence for several items. https://review.openstack.org/282427 | 16:58 |
| *** tzn has quit IRC | 16:59 | |
| vhosakot | SamYaple: let me know what you think | 17:01 |
| *** pbourke has quit IRC | 17:05 | |
| fthiagogv | Guys, someone already saw this error ""One or more undefined variables: 'dict object' has no attribute 'stdout' | 17:05 |
| fthiagogv | "" in Task: neutron | Reading json from variables ?? | 17:05 |
| *** pbourke has joined #kolla | 17:06 | |
| *** pbourke_ has joined #kolla | 17:08 | |
| *** jtriley has quit IRC | 17:09 | |
| *** sbezverk has quit IRC | 17:09 | |
| *** sbezverk has joined #kolla | 17:10 | |
| *** pbourke has quit IRC | 17:11 | |
| *** inc0 has quit IRC | 17:11 | |
| *** inc0 has joined #kolla | 17:11 | |
| fthiagogv | Do someone deploy in multinode? | 17:16 |
| *** cloudnautique has joined #kolla | 17:16 | |
| *** chandankumar has quit IRC | 17:18 | |
| *** chandankumar has joined #kolla | 17:18 | |
| *** inc0 has quit IRC | 17:20 | |
| pbourke_ | fthiagogv: yes | 17:21 |
| *** inc0 has joined #kolla | 17:22 | |
| *** inc0_ has joined #kolla | 17:24 | |
| inc0_ | ok, back to nova upgrades | 17:24 |
| fthiagogv | pbourke_ Do multinode conf have any rule to set the nodes? I set control, network, storage, compute in differents nodes. Is it okay? | 17:25 |
| inc0_ | fyi sdake_ SamYaple I also think removing upgrade folder makes sense now | 17:25 |
| *** inc0 has quit IRC | 17:26 | |
| *** Serlex has quit IRC | 17:30 | |
| sbezverk | How do you guys connect to mariadb in dockers env. to drop and re-initialize db? | 17:30 |
| *** ssurana has joined #kolla | 17:30 | |
| *** haplo37 has joined #kolla | 17:30 | |
| *** Allen_Gao has quit IRC | 17:30 | |
| pbourke_ | fthiagogv: I dont fully understand | 17:31 |
| *** Allen_Gao has joined #kolla | 17:31 | |
| pbourke_ | fthiagogv: you can configure the node layout in ansible/inventory/multinode if that's what you mean? | 17:31 |
| fthiagogv | pbourke_, yes. | 17:31 |
| d_code | pbourke_: how can I take those changes and put them on the previous review? do a git revert? | 17:32 |
| pbourke_ | d_code: code you will want to use interactive rebase | 17:33 |
| pbourke_ | i think it should be outlined in that link I posted | 17:33 |
| *** spisarski1 has joined #kolla | 17:34 | |
| *** spisarski has quit IRC | 17:34 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add ansible support for Manila https://review.openstack.org/269688 | 17:35 |
| *** dims has joined #kolla | 17:35 | |
| inc0_ | SamYaple, question about https://review.openstack.org/#/c/254395/16/ansible/library/kolla_docker.py@614 | 17:35 |
| inc0_ | I don't really understand what you mean, all of these functions may return different types, and to cast them out to single one will be incredibly inconvinent and possibly breaking | 17:36 |
| inc0_ | especially with pythons type handling | 17:36 |
| *** Marga_ has joined #kolla | 17:38 | |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add ansible support for Manila https://review.openstack.org/269688 | 17:38 |
| *** gfidente has quit IRC | 17:41 | |
| *** cloudnautique has quit IRC | 17:41 | |
| openstackgerrit | Merged openstack/kolla: Keep consistent with others for ironic containers https://review.openstack.org/280557 | 17:43 |
| *** cloudnautique has joined #kolla | 17:44 | |
| *** kproskurin has quit IRC | 17:44 | |
| sdake_ | back in the office | 17:46 |
| sdake_ | inc0_ cool | 17:46 |
| sdake_ | vhosakot if your behind a firewll you need proxy setup | 17:46 |
| sdake_ | elemoine_ enjoy :) | 17:46 |
| sdake_ | d_code agree with samyaple, should not allow run twice | 17:47 |
| sdake_ | inc0_ we used ceph in liberty for our demo on centos-binary | 17:47 |
| vhosakot | sdake_: it worked after retrying 4 times... the code must timeout, print a messgae, and continue, instead of haning... | 17:47 |
| inc0_ | ok | 17:47 |
| inc0_ | good to know | 17:47 |
| vhosakot | sdake_: will submit a patch for this | 17:47 |
| sdake_ | the wget wouldn't timeout if you have your proxy set | 17:47 |
| d_code | sdake_: I made the changes, but I screwed up the amendment when I commited | 17:48 |
| d_code | trying to figure out how to undo that…I’ve used git quite a bit, but new to gerrit | 17:48 |
| *** unicell has quit IRC | 17:49 | |
| sbezverk | sdake I need to reset cinder DB, when I try running mysql -u root -p it does not connect, what is the right way to connect to DB? | 17:49 |
| openstackgerrit | Derek Ditch proposed openstack/kolla: Addressed whitespace and idempotence for several items. https://review.openstack.org/282399 | 17:50 |
| sdake_ | 3 heka patches not passing gate | 17:50 |
| d_code | there we go | 17:50 |
| *** achanda has joined #kolla | 17:50 | |
| sdake_ | sbezverk are you using password to login? | 17:50 |
| sbezverk | sdake_ yes | 17:51 |
| sdake_ | sbezverk by not connect to db, you mean you get a econrefused or you get an invalid password? | 17:51 |
| vhosakot | sdake_: ah, did not know wget wouldn't time if proxy is set | 17:51 |
| sbezverk | ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) | 17:51 |
| sdake_ | sbezverk you need to specify the ip address of mysql | 17:52 |
| vhosakot | sdake_: I still see the same error... no leaked containers, no funny names | 17:52 |
| vhosakot | msg: The requested image does not exist: 192.168.122.41:4000/kollaglue/ubuntu-source-rsyslog:2.0.0 | 17:52 |
| sdake_ | vhosakot docker images | grep rsyslog | 17:52 |
| sdake_ | and paste it not in irc :) | 17:52 |
| vhosakot | kollaglue/ubuntu-source-rsyslog 2.0.0 6b24cc9a7331 3 hours ago 236.7 MB | 17:53 |
| vhosakot | oops, sorry.. its there | 17:53 |
| vhosakot | will send all the output in pastebin like yesterday :) | 17:53 |
| sdake_ | note a recent patch to the quickstart brought the registry documentation up to date | 17:53 |
| sdake_ | vhosakot build.py --registery 192.168.1.122.41:4000 --push | 17:53 |
| sdake_ | vhosakot build.py --registery 192.168.1.122.41:4000 --push rsyslog | 17:53 |
| SamYaple | inc0_: im specifically refering to the variable 'result' | 17:54 |
| inc0_ | yeah, it just passes what it gets | 17:55 |
| inc0_ | it's common practice in python | 17:55 |
| SamYaple | inc0_: im saying dont do that | 17:55 |
| vhosakot | sdake_: cool, I have more updates for quickstart.. will push later | 17:55 |
| SamYaple | it needs to be a dict if you are returning it to ansible | 17:55 |
| inc0_ | casting it all to same type might be breaking | 17:55 |
| *** tzn has joined #kolla | 17:56 | |
| SamYaple | look at the functions, they all return none or True | 17:56 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add ansible support for Manila https://review.openstack.org/269688 | 17:56 |
| inc0_ | return_json makes json out of it | 17:56 |
| inc0_ | so it has to be jsonable, so bool, string, dict, none...all works | 17:56 |
| SamYaple | inc0_: i dont like the idea the variable could be None, True, or a dict with data | 17:56 |
| SamYaple | its being returned to ansible | 17:56 |
| inc0_ | not as-is | 17:56 |
| inc0_ | it goes through jsonification before that | 17:56 |
| SamYaple | so sanitize it to _something_. Wether a dict always, or a bool | 17:57 |
| SamYaple | honestly the way we use it i think a bool works | 17:57 |
| SamYaple | hey guys I acidentally +W this https://review.openstack.org/#/c/280749/ | 17:57 |
| inc0_ | but that will either limit it remendously or be prone to problems | 17:57 |
| SamYaple | soooo im going to submit a patch for this, but its breaking the gate for some hosts sorry | 17:57 |
| SamYaple | i didnt see it til jsut now | 17:57 |
| inc0_ | bool works now, but if you try to reuse it in a different way, it's wasted work | 17:58 |
| inc0_ | and little benefit to it | 17:58 |
| inc0_ | also you -1 on returning bool from method because other methods return bool or none;) | 17:59 |
| inc0_ | I think way to solve this issue is to make methods themselves return consistent type | 17:59 |
| inc0_ | not casting it afterwards | 17:59 |
| inc0_ | and that's beyond scope of this patch | 18:00 |
| inc0_ | I will add todo comment tho, ok? | 18:00 |
| sbezverk | sdake_ what about running this after cinder db is re-created? "cinder-manage db sync" | 18:00 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Quick gate fix https://review.openstack.org/282447 | 18:00 |
| SamYaple | inc0_: sdake_ rhallisey i need a quick review on this https://review.openstack.org/282447 | 18:01 |
| SamYaple | sorry for the trouble, i was trying to -W and +W instead.... | 18:01 |
| sbezverk | sdake_ here is what I get when I try to run db sync http://paste.openstack.org/show/487610/ | 18:01 |
| SamYaple | inc0_: no they all return bool or None | 18:01 |
| SamYaple | inc0_: this was intentional | 18:01 |
| SamYaple | this matches docker-py | 18:01 |
| SamYaple | _however_ all of that is internal ot the kolla_docker module | 18:02 |
| inc0_ | so I'd say let's keep it bool or none | 18:02 |
| SamYaple | if you are returning variables to the host (which is new to this type) make it a bool on return | 18:02 |
| inc0_ | for ansible | 18:02 |
| *** tzn has quit IRC | 18:02 | |
| SamYaple | I say one type for ansible | 18:02 |
| SamYaple | ansible and jinja2 is not as flexible with different type callings | 18:02 |
| inc0_ | ok, but that will limit us afterwards to return dicts | 18:02 |
| SamYaple | so _that_ can be the todo | 18:03 |
| inc0_ | ok, I can do that | 18:03 |
| SamYaple | that works for me | 18:03 |
| SamYaple | so it retursn bool now with a TODO to return dict? | 18:03 |
| inc0_ | yeah | 18:04 |
| sbezverk | SamYaple I built docker container with iscsid and taregtcli, when iscsiadm does discovery from another node, I can see shared by this container targets.. Atthis point everything is manually configured, could you pprovide some guidance how it can be packed as a patch, I have never done it before.. | 18:04 |
| fthiagogv | Do someone know about this error? http://pastebin.com/C5wC0q7R | 18:04 |
| SamYaple | works for me. I think that means the only thing that needs to change is the results var section | 18:04 |
| SamYaple | sbezverk: youve never submitted a patch to openstack? | 18:05 |
| sbezverk | SamYaple nope, I am in integrator not developer ;-) | 18:05 |
| SamYaple | sbezverk: ok, its a bit of work to setup. youll need to do some reading https://wiki.openstack.org/wiki/How_To_Contribute | 18:06 |
| SamYaple | specifically the "if you're a developer" section | 18:06 |
| SamYaple | it has instructions on how to setup to submit a patch | 18:06 |
| sbezverk | SamYaple got it, thank you | 18:06 |
| sdake_ | fthiagogv haven't seen that, file a bug with reproducer please | 18:09 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Update cleanup-containers to remove ceph containers https://review.openstack.org/282451 | 18:11 |
| *** Liuqing has quit IRC | 18:12 | |
| vhosakot | "./build.py --registry 192.168.1.122.41:4000 rsyslog" passed, but, "./build.py --registry 192.168.1.122.41:4000 --push rsyslog" failed with error "ERROR:__main__:unable to ping registry endpoint https://192.168.1.122.41:4000/v0/" | 18:12 |
| sdake_ | limamauricio you can't just remove the eph containers, you also have to modify /etc/fstab | 18:12 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Playbook for nova upgrade https://review.openstack.org/254395 | 18:13 |
| sdake_ | vhosakot docker ps | grep registry | 18:13 |
| vhosakot | is up | 18:13 |
| vhosakot | 9704003bc20c registry:2 "/bin/registry /etc/d" 15 hours ago Up 22 minutes 0.0.0.0:4000->5000/tcp registry | 18:13 |
| limamauricio | hmm... | 18:13 |
| sdake_ | vhosakot the reason rsyslog can't be found is you haven't pushed it to the registry | 18:13 |
| vhosakot | right, I can build rsyslog but cannot push.. | 18:13 |
| fthiagogv | sdake_, can you look if this is correct? http://pastebin.com/7p4dZYMW If yes, I will file a bug. | 18:13 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Playbook for nova upgrade https://review.openstack.org/254395 | 18:14 |
| sdake_ | fthiagogv I don't know what ansible_ssh_pass does | 18:14 |
| sdake_ | is that default or something you added in? | 18:14 |
| sdake_ | vhosakot which os are you on? | 18:15 |
| *** jtriley has joined #kolla | 18:15 | |
| vhosakot | Ubuntu 14.04 | 18:15 |
| sdake_ | did you set the --insecure-registry flag in your docker daemon startup? | 18:15 |
| sdake_ | ps -ef | grep docker | 18:15 |
| fthiagogv | sdake_, set password to authenticate when ansible access via ssh. | 18:15 |
| vhosakot | yes | 18:15 |
| vhosakot | /usr/bin/docker daemon --insecure-registry 192.168.122.41:4000 | 18:16 |
| sdake_ | what machine is .41, your laptop or something? | 18:16 |
| vhosakot | it is the IP of eth0 on Ubuntu 14.04, the same IP I use to SSH into the Ubuntu host | 18:16 |
| sdake_ | vhosakot well that should work, but obiously doesnt | 18:17 |
| sdake_ | i am not an ubuntu wizard unfortunately | 18:17 |
| sdake_ | it is possible you have ipteables rules blocking port 400 | 18:17 |
| sdake_ | try telnet 4000 | 18:17 |
| sdake_ | see if that responds with a connection | 18:17 |
| vhosakot | "curl 192.168.122.41:4000" passes BTW | 18:17 |
| limamauricio | sdake, remove the ceph container manually even after run the script | 18:17 |
| limamauricio | :/ | 18:17 |
| sdake_ | limamauricio what I'm saing is it would make sense to clean up /etc/fstab as well, possibly in cleanup-host | 18:18 |
| sdake_ | grep -v ceph /etc/fstab > /etc/fstab may do it ;) | 18:18 |
| sdake_ | but that could have unintended consequences | 18:18 |
| limamauricio | oh | 18:19 |
| limamauricio | :) | 18:19 |
| sdake_ | you may need to save it to a temporary secure file | 18:19 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Playbook for nova upgrade https://review.openstack.org/254395 | 18:19 |
| sdake_ | and then copy that over | 18:19 |
| sdake_ | /query inc0_ | 18:19 |
| *** unicell has joined #kolla | 18:19 | |
| *** aginwala has joined #kolla | 18:20 | |
| *** unicell has quit IRC | 18:20 | |
| *** unicell has joined #kolla | 18:20 | |
| vhosakot | sdake_: tcp6 0 0 :::4000 :::* LISTEN - | 18:20 |
| vhosakot | from netstat | 18:20 |
| limamauricio | sdake, this is a change a little criticism, I will study a safe way to do this | 18:21 |
| *** mbound has quit IRC | 18:21 | |
| limamauricio | sdake, i'm deploying centos/binary now | 18:22 |
| *** dims_ has joined #kolla | 18:22 | |
| limamauricio | did you already deployed it ? | 18:23 |
| *** achanda has quit IRC | 18:25 | |
| *** dims has quit IRC | 18:25 | |
| *** ccesario has quit IRC | 18:25 | |
| *** larsks has quit IRC | 18:25 | |
| *** ByPasS has quit IRC | 18:25 | |
| *** clayton has quit IRC | 18:25 | |
| *** mkoderer__ has quit IRC | 18:25 | |
| *** aginwala has quit IRC | 18:25 | |
| *** ByPasS has joined #kolla | 18:25 | |
| *** mkoderer__ has joined #kolla | 18:26 | |
| *** larsks has joined #kolla | 18:26 | |
| *** clayton has joined #kolla | 18:26 | |
| *** bradjones has quit IRC | 18:27 | |
| *** SamYaple has quit IRC | 18:27 | |
| *** hparekh has quit IRC | 18:27 | |
| *** nihilifer has quit IRC | 18:27 | |
| *** cfarquhar has quit IRC | 18:27 | |
| *** SamYaple has joined #kolla | 18:27 | |
| *** cfarquhar has joined #kolla | 18:27 | |
| *** cfarquhar has quit IRC | 18:27 | |
| *** cfarquhar has joined #kolla | 18:27 | |
| *** nihilifer has joined #kolla | 18:27 | |
| *** hparekh has joined #kolla | 18:27 | |
| *** ccesario has joined #kolla | 18:28 | |
| *** bradjones has joined #kolla | 18:28 | |
| *** bradjones has quit IRC | 18:28 | |
| *** bradjones has joined #kolla | 18:28 | |
| *** clayton_ has joined #kolla | 18:28 | |
| vhosakot | I see docker-proxy process when I run the registry on 4000 | 18:28 |
| *** clayton has quit IRC | 18:28 | |
| *** clayton_ is now known as clayton | 18:29 | |
| vhosakot | Is docker-proxy listening on port 4000 expected ? | 18:30 |
| SamYaple | hey inc0_ i posted the wrong pep8 quote in the docker-py patch | 18:32 |
| SamYaple | the only issue I really have with kolla_docker in your nova patch is the "if container_info is None" part | 18:33 |
| SamYaple | if that can be changed to "if not container_info:" we will be better off | 18:33 |
| vhosakot | SamYaple: Is docker-proxy listening on port 4000 expected ? | 18:33 |
| SamYaple | vhosakot: sounds like you told your registry to start on port 4000 | 18:34 |
| vhosakot | yeah.. but 4000 is used by docker-proxy in netstat | 18:34 |
| sdake_ | vhosakot that is what is supposed to be doing | 18:34 |
| sdake_ | vhosakot try turning off iptables | 18:35 |
| vhosakot | haaa | 18:35 |
| SamYaple | did you run it with -p 4000:5000? if so thats what its suposed to be doing | 18:35 |
| SamYaple | docker-proxy is proxying 4000 to 5000 | 18:35 |
| vhosakot | yes | 18:35 |
| vhosakot | ran this "docker run -d -p 4000:5000 --restart=always --name registry registry:2" | 18:36 |
| sdake_ | vhosakot that is what you are supposed to run | 18:36 |
| sdake_ | not sure why your push is failing | 18:36 |
| vhosakot | flushed IPt as well "sudo iptables --flush" | 18:36 |
| sdake_ | unless the sematics of --registry has changed | 18:36 |
| vhosakot | so, docker-proxy process is not a problem | 18:37 |
| vhosakot | is it ? | 18:37 |
| *** haplo37 has quit IRC | 18:37 | |
| vhosakot | docker-proxy is working fine.. proxying 4000 to 5000 | 18:38 |
| inc0_ | SamYaple, these are not equivalent | 18:38 |
| SamYaple | inc0_: what is not? | 18:38 |
| SamYaple | oh i know | 18:38 |
| SamYaple | thats teh point | 18:38 |
| SamYaple | they are not equivelant | 18:38 |
| SamYaple | i know what im asking | 18:39 |
| vhosakot | I can build rsyslog, but cannot push | 18:39 |
| SamYaple | the api breakage was docker returning None or [] | 18:39 |
| SamYaple | they change things in docker-py | 18:39 |
| SamYaple | hence why i used 'if not <var>:' everywhere inc0_ | 18:39 |
| limamauricio | kolla supports linuxbridge? | 18:40 |
| SamYaple | limamauricio: indeed | 18:40 |
| SamYaple | though i dont know how tested it is | 18:40 |
| SamYaple | i know ive deployed with it when i added it, but not alot since | 18:40 |
| limamauricio | hmm | 18:41 |
| inc0_ | SamYaple, def compare_image(self, container_info=self._get_container_info()): wont work | 18:41 |
| inc0_ | self is not defined | 18:41 |
| cloudnautique | Hello, is there a workaround for the Kolla-toolbox container? Its failing to build on master because of my.cnf conflicts | 18:41 |
| cloudnautique | between Percona and Maria | 18:41 |
| SamYaple | inc0_: only thing i -1 for was the 'if not <var>:' thing, should have made it more clear | 18:42 |
| SamYaple | though i do think you shouldnt be using an underscore for _get_container_info, i wont push it | 18:42 |
| inc0_ | also, about tasks | 18:42 |
| SamYaple | i think its wrong since <class>.get_container_info is useful | 18:42 |
| inc0_ | I saw things changed during rebase | 18:42 |
| inc0_ | I'll remove underscore, np | 18:43 |
| *** achanda has joined #kolla | 18:43 | |
| inc0_ | SamYaple, do we need to run bootstrap nova compute during upgrade? | 18:44 |
| SamYaple | inc0_: maybe | 18:44 |
| SamYaple | technically the "upgrade" could happen on a fresh host | 18:44 |
| SamYaple | think about it | 18:44 |
| SamYaple | upgrade should also do the same job as deploy in the end | 18:45 |
| inc0_ | no, I don't agree | 18:45 |
| SamYaple | just additional tasks | 18:45 |
| inc0_ | it does less | 18:45 |
| inc0_ | and that might be key | 18:45 |
| inc0_ | also things will break on fresh host anyway | 18:45 |
| SamYaple | ok again, if i run upgrade, then i run deploy it should never change anything | 18:45 |
| SamYaple | and it will not | 18:45 |
| SamYaple | after upgrade, running deploy should change nothing | 18:45 |
| inc0_ | if you add stuff to conductors inventory | 18:45 |
| inc0_ | it will try to stop conductors and there is no such container | 18:46 |
| SamYaple | thats fine | 18:46 |
| SamYaple | it wont break | 18:46 |
| inc0_ | at least for now let's assume upgrade is run on same conf as deploy | 18:46 |
| inc0_ | in this case might not break, but it might elsewhere | 18:46 |
| inc0_ | this is not the use-case I want to tackle right now | 18:47 |
| SamYaple | ok im goign to say this again, after i run upgrade.yml i should be able to run deploy.yml with 0 changed | 18:47 |
| inc0_ | technically upgrade might be as well "rolling deploy" | 18:47 |
| SamYaple | so if thats not the case, upgrades not done | 18:47 |
| SamYaple | rolling disruptive deploy sure | 18:47 |
| inc0_ | if you changed your config in the meantime | 18:47 |
| inc0_ | that means your deploy is not done | 18:47 |
| SamYaple | always arguing about everything huh | 18:48 |
| inc0_ | and prerequisite to upgrade is finished deploy | 18:48 |
| SamYaple | yes if you make a chnage AFTER running upgrades, deploy will change | 18:48 |
| inc0_ | no, I don't want to introduce scope creep | 18:48 |
| SamYaple | but i said run upgrades.yml then deploy.yml no changes should occur | 18:48 |
| SamYaple | its not | 18:48 |
| inc0_ | 2 weeks before mitaka-3 | 18:48 |
| SamYaple | this wass always scope | 18:48 |
| SamYaple | upgrades not done if i cant run deploy | 18:48 |
| inc0_ | if you keep your inventory way it was | 18:48 |
| SamYaple | then your upgrade play will fail | 18:48 |
| inc0_ | no, why? | 18:48 |
| SamYaple | just throw the bootstrap stuff in bootstrap service and were done | 18:49 |
| SamYaple | no need for this | 18:49 |
| inc0_ | I'm not against it, I'm asking | 18:49 |
| inc0_ | I want to understand why it's needed | 18:49 |
| SamYaple | because the compute node will fail if it doesnt have the volume bootstraped | 18:49 |
| SamYaple | there is no harm in bootstraping it if it doesnt exist | 18:49 |
| *** Marga_ has quit IRC | 18:50 | |
| inc0_ | ok, but bootstrapping volume is idempotent? | 18:50 |
| SamYaple | why fight the envitable failure when the "fix" is just move all the bootstrap_service stuff to... bootstrap_service | 18:50 |
| SamYaple | yes | 18:50 |
| inc0_ | if wont break existing one? | 18:50 |
| SamYaple | no | 18:50 |
| SamYaple | we run it every deploy | 18:50 |
| inc0_ | ok | 18:50 |
| inc0_ | fyi I didnt fight it | 18:50 |
| *** sdake_ is now known as sdake | 18:50 | |
| inc0_ | I just say that "upgrade with different inventory than deploy is not supported" | 18:51 |
| inc0_ | it might work, but I won't hold upgrade patches to fix it if it ever become a problem | 18:51 |
| SamYaple | im fine with that, but its likely to happen so if there is a simple change like this that can be done to fix it... well we might as well include it | 18:51 |
| SamYaple | i mean if we catch it now it saves a bug later | 18:51 |
| SamYaple | this isn't a refactor past what you were already doing | 18:51 |
| inc0_ | sure, ofc | 18:52 |
| SamYaple | re the :self._container thing, yea you are right self. is undefined | 18:52 |
| SamYaple | missed that | 18:52 |
| inc0_ | but if it would be more than that then that's my standpoint | 18:52 |
| SamYaple | i mean it just depends on what is more | 18:53 |
| SamYaple | a simple refactor, fine. code changes in extend_start.... maybe a tech-debt bug | 18:53 |
| inc0_ | all depends on case I guess | 18:53 |
| SamYaple | but if we can play with the tasks order until it works for that and prevents bugs I say we should | 18:53 |
| inc0_ | https://review.openstack.org/#/c/254395/20 bot failed -> there you go | 18:54 |
| inc0_ | thing tho, on multinode will database_created be defined? | 18:55 |
| inc0_ | because we include it with "when" | 18:55 |
| inc0_ | I'm not sure if ansible will have this fact set up on non-db nodes | 18:55 |
| SamYaple | inc0_: you copied in the wrong task | 18:56 |
| *** fthiagogv has quit IRC | 18:56 | |
| inc0_ | aaa yeah, well I need do nova_api db as well | 18:56 |
| SamYaple | inc0_: ok three comments up. all needed | 18:57 |
| SamYaple | no you dont need to create teh db again | 18:57 |
| SamYaple | you need to db_sync | 18:57 |
| SamYaple | the db and users are fine | 18:57 |
| inc0_ | don't forget we're doing upgrade from liberty | 18:58 |
| inc0_ | there was no nova-api db then | 18:58 |
| inc0_ | all hail nova | 18:58 |
| inc0_ | so I, in fact, think we need to include it in playus | 18:58 |
| SamYaple | hmmm | 18:59 |
| SamYaple | i think that just means you need to call bootstrap.yml instead of bootstrap_service.yml no? | 18:59 |
| SamYaple | from upgrade.yml | 18:59 |
| inc0_ | hmm...if it won't mess up with original nova database | 19:00 |
| inc0_ | it shouldnt''' | 19:00 |
| SamYaple | deploy never does :) | 19:00 |
| inc0_ | so in fact we might not split to bootstrap_servie at all | 19:00 |
| inc0_ | yeah I'll just do that | 19:00 |
| SamYaple | inc0_: not split it out at all? | 19:03 |
| SamYaple | then do we really need to for the other services? | 19:04 |
| SamYaple | no no inc0_ you still need to | 19:04 |
| inc0_ | why? | 19:04 |
| SamYaple | we did that because the db_sync only runs after database creation | 19:04 |
| SamYaple | so what you need to do is run boostrap.yml and then bootstrap_service.yml from upgrade.yml | 19:05 |
| SamYaple | otherwise the service will never trigger a db_sync | 19:05 |
| inc0_ | yeah, database_created | 19:05 |
| inc0_ | that was the reason for splout | 19:05 |
| inc0_ | split | 19:05 |
| *** Serlex has joined #kolla | 19:06 | |
| SamYaple | so we still need splitout, but for nova (since it has a new db) we need to call boostrap.yml _and_ boostrap_service.yml | 19:06 |
| sdake | where are the docker proxy operation commands that someone put in an etherpad? | 19:06 |
| sdake | I am having trouble locating them. | 19:06 |
| openstackgerrit | Merged openstack/kolla: Quick gate fix https://review.openstack.org/282447 | 19:07 |
| Serlex | Anyone here having problems starting rsyslog container? | 19:08 |
| inc0_ | but then bootstrap_service will be called twice | 19:08 |
| inc0_ | we can run api-db creation in upgrade play | 19:08 |
| *** mbound has joined #kolla | 19:08 | |
| inc0_ | and tag it as todo: remove from next release | 19:08 |
| inc0_ | one of special tasks that will be valid only for L->M | 19:08 |
| SamYaple | inc0_: it will be called twice, but run once | 19:09 |
| inc0_ | how so? | 19:10 |
| SamYaple | because it wont run the first time | 19:10 |
| SamYaple | it triggers on nova database creation, not nova_api database creation | 19:10 |
| sbezverk | SamYaple would it be ok to add iscsid binaries to cinder-volume container? The reason I am asking is because it needs access to volume group in order to create LVM volume if iscsid runs in a separate container it will also need access to volume groups to map to iscsi target. | 19:10 |
| SamYaple | nova is the only case where we have two | 19:10 |
| SamYaple | sbezverk: no thats the reason its not a simple fix | 19:11 |
| inc0_ | yeah all hail nova | 19:11 |
| SamYaple | sbezverk: doing so means we have multiple processes needing to be tracked in cinder-volume turing it into a "fat container" | 19:11 |
| inc0_ | I need to think how to do it well | 19:11 |
| SamYaple | inc0_: and you can't move nova api database creation out of boostrap.yml entirely, since new deploys | 19:11 |
| inc0_ | right now I'm rather thinking about adding this task to upgrade.yml | 19:12 |
| SamYaple | if you just call bootstap.yml then boostrap_service.yml from upgrades.yml it will work fine | 19:12 |
| SamYaple | don't duplicate teh task please | 19:12 |
| inc0_ | and including just bootstap_service | 19:12 |
| SamYaple | im just going to -1 if you dup the task | 19:12 |
| inc0_ | it will not be idempotent | 19:12 |
| SamYaple | it will | 19:12 |
| inc0_ | no | 19:12 |
| SamYaple | yes | 19:13 |
| sbezverk | SamYaple but in the case of separated approach you need to develop Docker specific cinder API so the cinder volume process instead of just creating a lvm volume need to send a create command to iscsi container to create it.. | 19:13 |
| inc0_ | if I include bootstrap | 19:13 |
| inc0_ | itself | 19:13 |
| SamYaple | it will | 19:13 |
| inc0_ | let me finish | 19:13 |
| Serlex | i guess not | 19:13 |
| inc0_ | if I include bootstrap twice | 19:13 |
| SamYaple | sbezverk: yes it is not trivial | 19:13 |
| inc0_ | first one with database_created conditional | 19:13 |
| inc0_ | or database_created and api_database_created rather | 19:13 |
| SamYaple | sbezverk: you can include the _tools_ in the cinder-volume container. you can;t launch the process from there | 19:13 |
| inc0_ | it will not tic because I create just one of them | 19:14 |
| *** Marga_ has joined #kolla | 19:14 | |
| inc0_ | and in upgrades if I add bootrap_service unconditionally later on, it will try to syncdb each time it is run | 19:14 |
| inc0_ | db created or not | 19:14 |
| *** achanda has quit IRC | 19:14 | |
| SamYaple | ok inc0_ the first run of bootstrap.yml will create nova_api_db. but it wont trigger bootstrap_service.yml | 19:14 |
| SamYaple | https://github.com/openstack/kolla/blob/master/ansible/roles/nova/tasks/bootstrap.yml#L90 | 19:14 |
| inc0_ | and that might lead to uncontrolled migrations | 19:14 |
| SamYaple | thats just database_created | 19:15 |
| SamYaple | the second run of bootstrap_service.yml will run the db_sync | 19:15 |
| *** Marga_ has quit IRC | 19:15 | |
| inc0_ | and will run it always, while it should be run once | 19:15 |
| inc0_ | that's why we have database_created in the first placve | 19:15 |
| SamYaple | well no you should aalways run db_sync when you do an upgrade | 19:15 |
| SamYaple | but i dont know what you are arguing for | 19:16 |
| SamYaple | or against for that matter | 19:16 |
| inc0_ | I'm trying to think if this will break stuff later on | 19:16 |
| *** Serlex has quit IRC | 19:16 | |
| sdake | gate is bust | 19:16 |
| inc0_ | don't blame me for being careful in that | 19:16 |
| sdake | these 3 heka reviews are blocked on gate not working | 19:16 |
| SamYaple | sdake: yea its not all from that patch earlier, some of it is | 19:17 |
| SamYaple | sdake: the ubuntu gates are from the early merge but the fix is up | 19:17 |
| SamYaple | the centos gates are bust | 19:17 |
| SamYaple | packaging issues | 19:17 |
| sdake | heka is failing on all the gates | 19:17 |
| sdake | https://review.openstack.org/#/c/275840/14 | 19:17 |
| SamYaple | i know, i just explained why | 19:18 |
| SamYaple | you need to fix centos | 19:18 |
| SamYaple | the ubuntu gates are reporting failues, but there is a fix up gfor that | 19:18 |
| SamYaple | (it actually gets to deploy vm and that fails) | 19:18 |
| *** Marga__ has joined #kolla | 19:18 | |
| SamYaple | actually sdake this just merged https://review.openstack.org/#/c/282447/ | 19:19 |
| SamYaple | so ubuntu gate should be good on out. centos has packaging issues | 19:19 |
| sdake | SamYaple this is what your talking about? http://logs.openstack.org/40/275840/14/check/gate-kolla-dsvm-build-centos-binary/78f470f/console.html#_2016-02-19_18_06_08_639 | 19:20 |
| SamYaple | sdake: yea | 19:22 |
| SamYaple | thast unrelated to any patch, just a enw failure | 19:22 |
| sdake | yup | 19:22 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add a Dockerfile for Heka https://review.openstack.org/275840 | 19:23 |
| sdake | ubuntugate failing on elasticsearch | 19:24 |
| sdake | http://logs.openstack.org/40/275840/14/check/gate-kolla-dsvm-build-ubuntu-source/43a31e1/console.html#_2016-02-19_18_04_44_679 | 19:24 |
| *** diogogmt has quit IRC | 19:24 | |
| sdake | pretty sure thats a new failure too that is unfixed | 19:24 |
| sdake | SamYaple can you fix that elasticsearch problem plz | 19:25 |
| *** vhosakot has quit IRC | 19:25 | |
| sdake | i'll look into the centos problem now | 19:25 |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Implement Option for Two VIPs to separate API traffic https://review.openstack.org/282513 | 19:25 |
| *** diogogmt has joined #kolla | 19:25 | |
| dave-mccowan | sdake ^^ | 19:26 |
| sdake | dave-mccowan yup atm we are blocked onthe gate being busted | 19:26 |
| * sdake groans | 19:26 | |
| SamYaple | sdake: i see no problem with it. i think elasticsearch went down or messed up | 19:26 |
| sdake | i have rechecked it 3 or 4 times | 19:26 |
| sdake | the package has probably been renamed | 19:27 |
| *** vhosakot has joined #kolla | 19:28 | |
| sdake | ok folks, we need to make a decision, to merge heka today, or wait until the gate is ready to go | 19:28 |
| sdake | heka has been tested by atleast 2 CRs | 19:29 |
| SamYaple | dave-mccowan: reviewed. looks good! a few comments | 19:29 |
| sdake | the reason we wouldn't be able to merge heka today is because it requires a rebase on master after these centos percona and ubuntu elastic search problems are fixed | 19:29 |
| SamYaple | sdake: i vote merge it. well sort the bugs as they come | 19:29 |
| SamYaple | mitaka-3 is around the corner and we need heka for logging | 19:30 |
| sdake | ya thats where i stand on it as well | 19:30 |
| sdake | inc0 jpeeler rhallisey mandre | 19:30 |
| SamYaple | pbourke_ | 19:30 |
| sdake | inc0_ | 19:30 |
| sdake | paul is probably asleep | 19:30 |
| SamYaple | paul is probably drinking.... | 19:30 |
| rhallisey | I reviewed a few | 19:30 |
| rhallisey | I thought they were good | 19:30 |
| sdake | rhallisey scroll up a little bit dude ;) | 19:30 |
| SamYaple | rhallisey: the question is, merge them now or wait for gate (which has several issue ATM) | 19:31 |
| rhallisey | oh I see | 19:31 |
| rhallisey | I'm find with merging it. I think we can sort it out | 19:31 |
| * sdake paying 1 us dollar for a working gate ;) | 19:31 | |
| inc0_ | SamYaple, will database_created be present on deploy on compute node? | 19:32 |
| sdake | inc0_ any feedback on above please | 19:32 |
| SamYaple | inc0_: no, but its not checked by those tasks | 19:32 |
| inc0_ | it's checked on include | 19:32 |
| SamYaple | oh i see your point | 19:32 |
| SamYaple | yea i jsut go that | 19:32 |
| SamYaple | got* | 19:32 |
| *** jmccarthy has quit IRC | 19:32 | |
| SamYaple | oh wait wait | 19:32 |
| SamYaple | it will be yes inc0_ | 19:32 |
| SamYaple | inc0_: i worked that out way before liberty | 19:33 |
| SamYaple | https://github.com/openstack/kolla/blob/master/ansible/roles/nova/tasks/bootstrap.yml#L14 | 19:33 |
| SamYaple | the next line registers it on _all_ hosts | 19:33 |
| sdake | ok folks i'm going to pull tlhe trigger on heka | 19:33 |
| dave-mccowan | thanks SamYaple. do you have some context behind why to add a second vrrp_instance, instead of combining them under one? | 19:33 |
| SamYaple | sdake: do it | 19:33 |
| sdake | please please get the gate working today :) | 19:34 |
| dave-mccowan | not doubting, btw, i just want to learn. | 19:34 |
| jpeeler | i say merge it... | 19:34 |
| SamYaple | dave-mccowan: seperate tracking also can possibly provide a bit of unintentional loadbalancing (including intentional load balancing down the road) | 19:34 |
| SamYaple | dave-mccowan: external and internal are on different nodes, and that can be force by policy | 19:34 |
| SamYaple | but mostly because that jinja2 you have is REALLY ugly | 19:35 |
| SamYaple | :P | 19:35 |
| *** dwalsh_ has quit IRC | 19:35 | |
| *** dwalsh has quit IRC | 19:35 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Playbook for nova upgrade https://review.openstack.org/254395 | 19:35 |
| sdake | i can't do anything for it atm, itsi n a recheck and gerrit is blocked from voting | 19:36 |
| SamYaple | sdake: thats not true | 19:37 |
| SamYaple | I can change my vote | 19:37 |
| *** dwalsh has joined #kolla | 19:37 | |
| SamYaple | maybe your auth is screewd up | 19:37 |
| sdake | i cant? | 19:37 |
| sdake | wierd | 19:37 |
| sdake | i can on other reviews | 19:37 |
| sdake | letme logout and login | 19:37 |
| dave-mccowan | SamYaple could the two instances share the same check_alive script without changes? | 19:37 |
| SamYaple | you are on an old patchset | 19:37 |
| SamYaple | remove the patchset number in the url | 19:37 |
| SamYaple | dave-mccowan: yea | 19:38 |
| sdake | ok that problem is solved | 19:40 |
| sdake | heka INC | 19:40 |
| *** diogogmt has quit IRC | 19:40 | |
| *** dwalsh has quit IRC | 19:42 | |
| *** dwalsh has joined #kolla | 19:44 | |
| SamYaple | ok inc0_ i have reviewed it. it looks good, one comment on includes | 19:46 |
| *** diogogmt has joined #kolla | 19:46 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Playbook for nova upgrade https://review.openstack.org/254395 | 19:48 |
| inc0_ | done | 19:48 |
| sdake | SamYaple thoughts on this comment welcome in the review please https://review.openstack.org/#/c/282513/1/tools/openrc-example | 19:49 |
| sdake | inc0_ two very minor nitpicks in your patch | 19:55 |
| sdake | inc0_ but i'd like to keep the code sanitary could you fix and we will merge asap | 19:55 |
| SamYaple | sdake: i replied | 19:56 |
| SamYaple | you should find it funny | 19:56 |
| sdake | SamYaple I am tlaking about this review https://review.openstack.org/#/c/254395/23 | 19:56 |
| sdake | not sure which one your talking abou t;) | 19:56 |
| SamYaple | sdake: https://review.openstack.org/#/c/282513/1/tools/openrc-example | 19:57 |
| sdake | oh that one | 19:57 |
| SamYaple | :) | 19:58 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Playbook for nova upgrade https://review.openstack.org/254395 | 19:58 |
| inc0_ | there you go sdake | 19:58 |
| sdake | inc0 thanks | 19:58 |
| inc0_ | god my head hurts | 19:59 |
| inc0_ | let's get this one done today | 19:59 |
| *** tzn has joined #kolla | 19:59 | |
| SamYaple | inc0_: its already +2+W | 19:59 |
| SamYaple | inc0_: just go to sleep | 19:59 |
| inc0_ | sleep? I'm going to try one of whiskeys we bought from dark corner | 20:00 |
| SamYaple | mmmm | 20:00 |
| sdake | oh right whisky | 20:01 |
| sdake | i need to get my address to sam ;) | 20:01 |
| sdake | inc0_ i think the amount of punishment you have taken today feels like my whole week :) | 20:01 |
| sdake | now heat and neutron in the queue and we are done with openstack upgrades from a "in the queue" perspective ;-) | 20:02 |
| *** limamauricio has quit IRC | 20:03 | |
| inc0_ | yeah I'll review it all later | 20:03 |
| *** tzn has quit IRC | 20:03 | |
| inc0_ | next week - infra upgrades | 20:04 |
| *** dwalsh has quit IRC | 20:04 | |
| *** chandankumar has quit IRC | 20:05 | |
| sdake | i was thinking inc0_ | 20:05 |
| inc0_ | sdake, in our presentation I think we should start from kolla 1.1.0 | 20:06 |
| sdake | upgrade in copy_once case | 20:06 |
| sdake | do config's need to be rebuilt? | 20:06 |
| sdake | inc0_ sounds great, just need someone to do the work ;-) | 20:06 |
| inc0_ | sdake, configs may change between releases | 20:06 |
| inc0_ | (nova is prime example now) | 20:06 |
| sdake | inc0_ yes that is what i am getting at | 20:06 |
| inc0_ | and since we're restarting all the containers, it will copy new one | 20:06 |
| sdake | oh right | 20:07 |
| sdake | ok nm then | 20:07 |
| *** jmccarthy1 has quit IRC | 20:11 | |
| *** jmccarthy has joined #kolla | 20:11 | |
| sdake | t-2 weeks to mitaka-3!! | 20:13 |
| dave-mccowan | SamYaple sdake I have written a bash script that creates a haproxy-ca.crt and haproxy.pem suitable for TLS on the external VIP. I assume I can stick that in the tools/ directory. what is a good destination directory for the generated files (.key, .crt, .pem, .cnf)? | 20:20 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Upgrade cinder playbook https://review.openstack.org/278049 | 20:24 |
| openstackgerrit | Merged openstack/kolla: Use "optional" for ceph confs https://review.openstack.org/280808 | 20:25 |
| openstackgerrit | Merged openstack/kolla: Add a Dockerfile for Heka https://review.openstack.org/275840 | 20:25 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add Heka log decoder infrastructure https://review.openstack.org/276667 | 20:28 |
| SamYaple | oh jeez | 20:29 |
| SamYaple | sdake: all the heka patches are out of date | 20:29 |
| SamYaple | im going to be rebasing them | 20:29 |
| *** cloudnautique has quit IRC | 20:30 | |
| *** dims_ has quit IRC | 20:31 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add Heka to common role https://review.openstack.org/275841 | 20:32 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect HAProxy and Keepalived logs https://review.openstack.org/275842 | 20:32 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add Heka log decoder for RabbitMQ https://review.openstack.org/276668 | 20:32 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect RabbitMQ logs https://review.openstack.org/275844 | 20:32 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add Heka log decoder for OpenStack https://review.openstack.org/275845 | 20:32 |
| rhallisey | rebase! | 20:32 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Nova logs https://review.openstack.org/275846 | 20:32 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add Heka log decoder for MariaDB https://review.openstack.org/275847 | 20:32 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect MariaDB logs https://review.openstack.org/275848 | 20:32 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Glance logs https://review.openstack.org/276760 | 20:33 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Ansible write its logs into "kolla_logs" https://review.openstack.org/279191 | 20:33 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add Heka log decoder for Keystone/Apache https://review.openstack.org/279192 | 20:33 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Keystone logs https://review.openstack.org/279193 | 20:33 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Neutron logs https://review.openstack.org/279489 | 20:33 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Swift logs https://review.openstack.org/279490 | 20:33 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Cinder logs https://review.openstack.org/281259 | 20:33 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Heat logs https://review.openstack.org/281327 | 20:34 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Murano logs https://review.openstack.org/281328 | 20:34 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Magnum logs https://review.openstack.org/281329 | 20:34 |
| SamYaple | rhallisey: they werent even in sync with each other! so i couldnt just cherrypick at the end :) | 20:34 |
| rhallisey | : | 20:35 |
| rhallisey | :D | 20:35 |
| *** achanda has joined #kolla | 20:35 | |
| SamYaple | its ok rhallisey i asked elemoine not to rebase so much while we were reviewing. its all done now. no harm | 20:36 |
| *** haplo37 has joined #kolla | 20:37 | |
| *** tzn has joined #kolla | 20:40 | |
| *** rhallisey has quit IRC | 20:40 | |
| *** sdake has quit IRC | 20:44 | |
| dave-mccowan | SamYaple when you're hacking kolla and have a change to just one container, what's the fastest way to re-deploy? can i kill and start just one container? is that advisable? | 20:45 |
| SamYaple | dave-mccowan: yea thats fine | 20:46 |
| dave-mccowan | what's the command to kill one container? /me is a docker newb | 20:46 |
| *** tzn has quit IRC | 20:47 | |
| SamYaple | docker rm -f <container_name> | 20:47 |
| *** ayoung has quit IRC | 20:52 | |
| *** ssurana has quit IRC | 20:53 | |
| *** tzn has joined #kolla | 20:55 | |
| *** dencaval has quit IRC | 20:56 | |
| *** jasonsb has joined #kolla | 20:58 | |
| *** ssurana has joined #kolla | 20:59 | |
| *** jtriley_ has joined #kolla | 21:00 | |
| openstackgerrit | Merged openstack/kolla: Upgrade common role https://review.openstack.org/282400 | 21:00 |
| *** pbourke_ has quit IRC | 21:00 | |
| *** alyson_ has quit IRC | 21:02 | |
| *** jtriley has quit IRC | 21:02 | |
| *** tzn has quit IRC | 21:05 | |
| *** jtriley_ has quit IRC | 21:11 | |
| openstackgerrit | Merged openstack/kolla: Playbook for nova upgrade https://review.openstack.org/254395 | 21:15 |
| openstackgerrit | Merged openstack/kolla: Add Heka log decoder infrastructure https://review.openstack.org/276667 | 21:15 |
| *** jtriley has joined #kolla | 21:18 | |
| openstackgerrit | Merged openstack/kolla: Add Heka to common role https://review.openstack.org/275841 | 21:18 |
| openstackgerrit | Merged openstack/kolla: Make Heka collect HAProxy and Keepalived logs https://review.openstack.org/275842 | 21:18 |
| *** achanda has quit IRC | 21:18 | |
| openstackgerrit | Merged openstack/kolla: Add Heka log decoder for RabbitMQ https://review.openstack.org/276668 | 21:18 |
| openstackgerrit | Merged openstack/kolla: Make Heka collect RabbitMQ logs https://review.openstack.org/275844 | 21:19 |
| *** achanda has joined #kolla | 21:21 | |
| SamYaple | its merging | 21:22 |
| SamYaple | ITS MERGING | 21:22 |
| *** jasonsb has quit IRC | 21:24 | |
| openstackgerrit | Merged openstack/kolla: Upgrade cinder playbook https://review.openstack.org/278049 | 21:27 |
| *** cloudnautique has joined #kolla | 21:30 | |
| *** shardy has quit IRC | 21:31 | |
| openstackgerrit | Merged openstack/kolla: Add Heka log decoder for OpenStack https://review.openstack.org/275845 | 21:40 |
| elemoine_ | Thanks for the rebases SamYaple! | 21:41 |
| SamYaple | elemoine_: its late for you, ill handle the merge conflict as well unless you say otherwise | 21:43 |
| *** d_code has quit IRC | 21:44 | |
| *** dave-mccowan has quit IRC | 21:44 | |
| *** jtriley has quit IRC | 21:45 | |
| *** cloudnautique has quit IRC | 21:50 | |
| *** jtriley has joined #kolla | 21:53 | |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Murano logs https://review.openstack.org/281328 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Magnum logs https://review.openstack.org/281329 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Nova logs https://review.openstack.org/275846 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add Heka log decoder for MariaDB https://review.openstack.org/275847 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Add Heka log decoder for Keystone/Apache https://review.openstack.org/279192 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Cinder logs https://review.openstack.org/281259 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Heat logs https://review.openstack.org/281327 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Keystone logs https://review.openstack.org/279193 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Swift logs https://review.openstack.org/279490 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Neutron logs https://review.openstack.org/279489 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Ansible write its logs into "kolla_logs" https://review.openstack.org/279191 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect MariaDB logs https://review.openstack.org/275848 | 21:56 |
| openstackgerrit | Sam Yaple proposed openstack/kolla: Make Heka collect Glance logs https://review.openstack.org/276760 | 21:56 |
| *** dave-mccowan has joined #kolla | 21:57 | |
| *** cloudnautique has joined #kolla | 21:59 | |
| *** salv-orl_ has joined #kolla | 22:03 | |
| *** salv-orlando has quit IRC | 22:06 | |
| *** ByPasS has quit IRC | 22:07 | |
| *** ByPasS has joined #kolla | 22:13 | |
| *** jtriley has quit IRC | 22:20 | |
| elemoine_ | SamYaple do as you think is best... | 22:23 |
| SamYaple | elemoine_: i rebased them | 22:29 |
| *** daneyon has quit IRC | 22:30 | |
| elemoine_ | Did the commits pass the gate this time? | 22:30 |
| *** sdake has joined #kolla | 22:34 | |
| SamYaple | elemoine_: some of them | 22:35 |
| sdake | yo | 22:35 |
| SamYaple | about halfway merged | 22:35 |
| *** dims has joined #kolla | 22:36 | |
| *** JoseMello has quit IRC | 22:38 | |
| openstackgerrit | Merged openstack/kolla: Make Heka collect Nova logs https://review.openstack.org/275846 | 22:40 |
| *** achanda has quit IRC | 22:44 | |
| *** tzn has joined #kolla | 22:44 | |
| *** achanda has joined #kolla | 22:45 | |
| *** tzn has quit IRC | 22:48 | |
| *** achanda has quit IRC | 22:50 | |
| elemoine_ | Wondering why merging a few patches takes so much time. I guess I need to better understand OpenStack's gating/ci system. | 22:50 |
| sdake | elemoine_ it goe sthrough a gate job | 22:50 |
| sdake | in order | 22:50 |
| sdake | all openstack patches that enter the repo are serilaized | 22:50 |
| sdake | not per project | 22:50 |
| sdake | but entirely serialized | 22:50 |
| elemoine_ | I see. I'm sure there's a web page explaining the full process | 22:51 |
| sdake | http://zuul.openstack.org/ | 22:51 |
| sdake | part of the issue is I think they are still going through gate checks because of sam's rebase | 22:52 |
| *** tzn has joined #kolla | 22:52 | |
| SamYaple | sdake: i think they are serialized per project... | 22:53 |
| sdake | openstack infra will run check jobs in parallel | 22:53 |
| SamYaple | it actually runs check and gate jobs in parallel | 22:54 |
| sdake | the top of zuul says right what the integrated queue does :) | 22:54 |
| sdake | well whatever it does, there is a visualization of it - enjoy :) | 22:54 |
| sdake | i'm off for a early friday - been rough week home alone with the kids - need some hotdog refillz | 22:54 |
| *** Allen_Gao has quit IRC | 22:55 | |
| *** Allen_Gao has joined #kolla | 22:55 | |
| *** blahRus has quit IRC | 22:55 | |
| sdake | Actually, the issue I am seeing is the kolla_docker ansible library module is trying to index a dict like a list, which is causing the KeyError exception. | 22:56 |
| *** vhosakot has quit IRC | 22:58 | |
| sdake | SamYaple smoenoen complainign about rsyslog not working | 22:58 |
| sdake | fwiw I have heard this compalint here on irc about 4-5 times in the last cuple of days | 22:58 |
| *** vhosakot has joined #kolla | 22:58 | |
| SamYaple | sdake: yea im ignoring it until heka | 22:59 |
| sdake | do you know what the issue is? | 22:59 |
| SamYaple | havent even begun to look into it | 23:00 |
| SamYaple | so much going on | 23:00 |
| sdake | related to keyerror exception? | 23:00 |
| sdake | i'll tryt o get more info | 23:00 |
| sdake | seems like a keyerror shouldn't be hapepning rsyslog or not | 23:00 |
| openstackgerrit | Merged openstack/kolla: Add Heka log decoder for MariaDB https://review.openstack.org/275847 | 23:00 |
| openstackgerrit | Merged openstack/kolla: Make Heka collect MariaDB logs https://review.openstack.org/275848 | 23:01 |
| openstackgerrit | Merged openstack/kolla: Make Heka collect Glance logs https://review.openstack.org/276760 | 23:01 |
| vhosakot | sdake, SamYaple: I debugged entire day today, I can push and pull ubuntu image to 192.168.122.41:4000, but kolla-ansible throws the error | 23:02 |
| vhosakot | msg: The requested image does not exist: 192.168.122.41:4000/kollaglue/ubuntu-source-rsyslog:2.0.0 | 23:03 |
| vhosakot | I have flushed all my iptables rules and chains | 23:03 |
| SamYaple | vhosakot: that image does not exist on your registry then | 23:03 |
| SamYaple | does it exist in `docker images` | 23:03 |
| vhosakot | which image rsyslog ? | 23:03 |
| SamYaple | 192.168.122.41:4000/kollaglue/ubuntu-source-rsyslog:2.0.0 | 23:03 |
| sdake | vhosakot are you sure your using the --registry flag to build? | 23:04 |
| *** achanda has joined #kolla | 23:04 | |
| vhosakot | I see this line in "docker images" | 23:04 |
| vhosakot | kollaglue/ubuntu-source-rsyslog 2.0.0 6b24cc9a7331 9 hours ago 236.7 MB | 23:04 |
| sdake | vhosakot that looks lik you didn't build with the --registry option | 23:04 |
| vhosakot | should it be "192.168.122.41:4000/kollaglue/ubuntu-source-rsyslog" instead of "kollaglue/ubuntu-source-rsyslog" ? | 23:05 |
| sdake | yes | 23:05 |
| vhosakot | no, I did not use the --registry argument for kolla-build... | 23:05 |
| vhosakot | should I ???? !!!!! | 23:05 |
| sdake | I mentioned thtat about 6 hours ago ;-) | 23:05 |
| *** haplo37 has quit IRC | 23:05 | |
| sdake | or yesterday even | 23:05 |
| sdake | and its in our build documentation | 23:05 |
| vhosakot | the quickstart does not mention that :) | 23:05 |
| sdake | your right about that, perhaps it should | 23:06 |
| sdake | but the build docs mention it | 23:06 |
| vhosakot | aah | 23:06 |
| vhosakot | I ran "sudo kolla-build --retries 0 --base ubuntu --type source " | 23:06 |
| vhosakot | I need to add --registry flag to this right ? | 23:06 |
| sdake | use --registry 192.168.122.41:4000 --push | 23:06 |
| vhosakot | for kolla-build ? | 23:06 |
| vhosakot | ./doc/image-building.rst is the build guide ? | 23:07 |
| sdake | http://eavesdrop.openstack.org/irclogs/%23kolla/%23kolla.2016-02-19.log.html#t2016-02-19T18:12:33 | 23:07 |
| sdake | there you said you tried it and push failed | 23:07 |
| sdake | did you fix some problem with your ability to push? | 23:07 |
| vhosakot | I did it with ./build.py in /kolla/cmd.... did not pass it to kolla-build command | 23:08 |
| sdake | that is the same thing | 23:09 |
| vhosakot | yes, "./build.py --registry 192.168.1.122.41:4000 --push --base ubuntu --type source rsyslog" fails | 23:09 |
| sdake | SamYaple can you work with vhosakot on this, i've got to take my children to dinner | 23:09 |
| sdake | or they will starve ;( | 23:09 |
| sdake | bye | 23:09 |
| vhosakot | cool, thanks! good night sdake | 23:09 |
| vhosakot | :) | 23:09 |
| vhosakot | yes, I see kolla-build calls kolla/cmd/build.py | 23:12 |
| *** sbezverk has quit IRC | 23:16 | |
| *** tzn has quit IRC | 23:25 | |
| *** spisarski1 has quit IRC | 23:28 | |
| *** sdake has quit IRC | 23:35 | |
| *** rhallisey has joined #kolla | 23:36 | |
| *** Marga_ has joined #kolla | 23:51 | |
| *** Marga__ has quit IRC | 23:54 | |
| dgonzalez | hi jpeeler, are you on? | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!