Sunday, 2016-10-09

« Saturday, 2016-10-08 Index Monday, 2016-10-10 »
*** inc0 has joined #openstack-kolla00:15
openstackgerritMerged openstack/kolla: Remove unneeded Heka bootstrapping code  https://review.openstack.org/38412200:20
*** mewald has quit IRC00:20
*** zhubingbing has joined #openstack-kolla00:25
*** rhallisey has quit IRC00:38
*** schwicht has quit IRC00:44
*** sdake has joined #openstack-kolla00:50
*** sdake_ has joined #openstack-kolla00:52
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla: [WIP] Fix upgrades for rabbitmq  https://review.openstack.org/38402900:53
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla: [WIP] Fix upgrades for rabbitmq  https://review.openstack.org/38402900:55
*** sdake has quit IRC00:55
*** schwicht has joined #openstack-kolla00:56
*** salv-orlando has joined #openstack-kolla00:57
*** zhubingbing has joined #openstack-kolla01:01
*** salv-orlando has quit IRC01:01
zhubingbinghi01:20
zhubingbingguys01:20
openstackgerritzhubingbing proposed openstack/kolla: fix collectd dockerfile  https://review.openstack.org/38399701:23
*** sdake_ has quit IRC01:27
*** HyperJohnGraham_ has joined #openstack-kolla01:33
*** sdake has joined #openstack-kolla01:49
*** liyifeng has joined #openstack-kolla01:55
*** salv-orlando has joined #openstack-kolla01:58
*** sdake has quit IRC01:58
*** schwicht has quit IRC01:58
*** salv-orlando has quit IRC02:02
*** zhurong has joined #openstack-kolla02:09
*** zhubingbing has quit IRC02:12
*** bmace has quit IRC02:13
*** bmace has joined #openstack-kolla02:14
*** inc0 has quit IRC02:19
*** daneyon has joined #openstack-kolla02:22
*** daneyon has quit IRC02:26
*** zhubingbing has joined #openstack-kolla02:27
*** MarMat has joined #openstack-kolla02:30
*** eaguilar has quit IRC02:35
openstackgerritJack Ning proposed openstack/kolla: Change keystone.log's permit to keystone:keystone  https://review.openstack.org/38414602:53
openstackgerritzhubingbing proposed openstack/kolla: fix mistral requires auth_uri to point to v3 explicitly  https://review.openstack.org/38414702:53
openstackgerritJeffrey Zhang proposed openstack/kolla: Use mariadb.cnf for mariadb custom file name  https://review.openstack.org/38404302:54
*** bmace has quit IRC02:55
*** salv-orlando has joined #openstack-kolla02:59
*** zhubingbing has quit IRC02:59
*** sdake has joined #openstack-kolla02:59
sdakehey folks03:00
*** zhubingbing has joined #openstack-kolla03:01
*** sdake_ has joined #openstack-kolla03:01
*** salv-orlando has quit IRC03:03
openstackgerritJeffrey Zhang proposed openstack/kolla: integrate gnocchi with ceilometer  https://review.openstack.org/37273703:04
*** sdake has quit IRC03:05
*** bmace has joined #openstack-kolla03:08
openstackgerritLi Yingjun proposed openstack/kolla: Support Searchlight Docker container  https://review.openstack.org/37714503:12
*** yingjun has joined #openstack-kolla03:13
*** tonanhngo has joined #openstack-kolla03:36
*** MarMat has quit IRC03:38
*** zhubingbing has quit IRC04:09
*** sdake_ has quit IRC04:09
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: WIP fencing pod  https://review.openstack.org/38392204:28
*** neilus has joined #openstack-kolla04:35
*** neilus has quit IRC04:39
*** unicell1 has joined #openstack-kolla04:48
*** unicell has quit IRC04:48
*** zhurong has quit IRC04:52
*** zhurong has joined #openstack-kolla04:53
*** bmace has quit IRC04:55
*** bmace has joined #openstack-kolla04:56
*** zhurong has quit IRC04:56
*** Guest37605 is now known as kklimonda04:56
*** kklimonda has joined #openstack-kolla04:57
*** zhubingbing has joined #openstack-kolla04:57
*** zhubingbing has quit IRC04:59
*** sdake has joined #openstack-kolla05:17
sdakeyo05:20
openstackgerritMerged openstack/kolla: integrate gnocchi with ceilometer  https://review.openstack.org/37273705:23
*** zhurong has joined #openstack-kolla05:27
*** MarMat has joined #openstack-kolla05:50
*** ayoung has joined #openstack-kolla05:52
*** MarMat has quit IRC05:56
*** salv-orlando has joined #openstack-kolla06:01
*** salv-orlando has quit IRC06:05
*** zhubingbing has joined #openstack-kolla06:15
*** zhubingbing has quit IRC06:31
*** jascott1_tmp has joined #openstack-kolla06:56
*** jascott1_tmp has quit IRC06:56
openstackgerritLi Yingjun proposed openstack/kolla: Enable keystone authentication for Barbican  https://review.openstack.org/38416107:00
*** harlowja has quit IRC07:20
*** msimonin has joined #openstack-kolla07:27
*** msimonin has quit IRC07:27
openstackgerritJack Ning proposed openstack/kolla: Change keystone.log's permit to keystone:keystone  https://review.openstack.org/38414607:28
*** zhurong has quit IRC07:30
*** zhurong has joined #openstack-kolla07:45
openstackgerritJeffrey Zhang proposed openstack/kolla: Choose node with largest seqno number for mariadb recovery  https://review.openstack.org/38417008:01
*** Jeffrey4l has quit IRC08:01
openstackgerritzhubingbing proposed openstack/kolla: fix collectd dockerfile  https://review.openstack.org/38399708:08
*** Jeffrey4l has joined #openstack-kolla08:09
openstackgerrithuwh proposed openstack/kolla-kubernetes: fix 'raw_doc' doesn`t assignment error when show configmap`s template  https://review.openstack.org/38417408:26
openstackgerritJeffrey Zhang proposed openstack/kolla:  apt instead of apt-get to install packages  in Ubuntu  https://review.openstack.org/37947708:27
openstackgerritJeffrey Zhang proposed openstack/kolla: WIP: setting basic structures for iscsi container  https://review.openstack.org/38194208:29
*** mewald has joined #openstack-kolla08:30
openstackgerritzhubingbing proposed openstack/kolla: fix collectd dockerfile  https://review.openstack.org/38399708:34
*** mewald has left #openstack-kolla08:39
*** tonanhngo has quit IRC08:40
*** salv-orlando has joined #openstack-kolla08:46
openstackgerritMerged openstack/kolla: change perm and user wsgi file permission  https://review.openstack.org/37531308:49
openstackgerritMerged openstack/kolla: Deprecate Fedora based Docker images  https://review.openstack.org/36918408:50
openstackgerritzhubingbing proposed openstack/kolla: fix collectd dockerfile  https://review.openstack.org/38399708:53
*** msimonin has joined #openstack-kolla08:59
*** sdake has quit IRC09:01
*** tonanhngo has joined #openstack-kolla09:02
*** tonanhngo has quit IRC09:04
*** zhurong has quit IRC09:07
*** zhurong has joined #openstack-kolla09:08
*** dwalsh has joined #openstack-kolla09:08
*** salv-orlando has quit IRC09:24
*** msimonin has quit IRC09:25
*** msimonin has joined #openstack-kolla09:27
*** daneyon has joined #openstack-kolla09:35
*** daneyon has quit IRC09:40
*** salv-orlando has joined #openstack-kolla09:40
*** tonanhngo has joined #openstack-kolla09:42
*** mewald has joined #openstack-kolla09:43
*** tonanhngo has quit IRC09:44
*** khntumtarm has joined #openstack-kolla09:44
*** khntumtarm has quit IRC09:45
*** kuntumtarm has joined #openstack-kolla09:45
*** caisan_ has joined #openstack-kolla09:58
*** zhurong has quit IRC10:02
*** tonanhngo has joined #openstack-kolla10:04
*** tonanhngo has quit IRC10:06
*** yingjun has quit IRC10:21
mewaldI noticed my keystone database is huge compared to the others. It has around 65MB when backed up and gzip compressed. I noticed the largest part of that data volume comes from the "token" table. Shouldn't keystone be using memcached to storage tokens instead of the DB?10:21
*** cnf has joined #openstack-kolla10:55
cnfgood afternoon10:55
mewaldhi10:56
cnfare the kolla images considered production ready?10:59
cnfand how quickly do they follow releases?10:59
mewaldwhat does production ready mean for you?11:07
cnfput stuff on it you count on11:09
mewaldin a cloud you should never count on anything, which is why applications must be designed for failure ^^ So if you're talking about workloads, I believe it is as save as any other OpenStack distribution11:11
mewaldIn my experience, anything is as production ready as the people operating it. I've seen VMware environments going down because of idiots operating it and VMware is usually considered rock solid.11:12
cnfyeah, but if the images break every 3 other build, and builds are 2 versions behind11:13
cnfthen you can be a rock solid operator, it's not going to be fun11:13
mewaldWe run two environments with Kolla and work with our own fork of it which we test intensively before rollout. That has proven to work well for use. But we ran on master branch and that wasn't stable enough. So we decided to run in stable/mitaka and backport stuff we needed.11:16
*** dwalsh has quit IRC11:16
*** yingjun has joined #openstack-kolla11:45
*** salv-orlando has quit IRC11:58
openstackgerritJeffrey Zhang proposed openstack/kolla: Choose node with largest seqno number for mariadb recovery  https://review.openstack.org/38417012:09
*** zhurong has joined #openstack-kolla12:13
*** tonanhngo has joined #openstack-kolla12:14
*** tonanhngo has quit IRC12:14
*** g3ek has quit IRC12:30
*** haplo37 has quit IRC12:30
*** g3ek has joined #openstack-kolla12:36
*** haplo37 has joined #openstack-kolla12:36
*** liyifeng has quit IRC12:47
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix keystone fernet file exchange via ssh  https://review.openstack.org/36941813:01
*** salv-orlando has joined #openstack-kolla13:29
*** yingjun has quit IRC13:31
*** yingjun has joined #openstack-kolla13:31
*** yingjun_ has joined #openstack-kolla13:33
*** yingjun has quit IRC13:33
*** salv-orlando has quit IRC13:34
*** awiddersheim has joined #openstack-kolla13:49
*** sdake has joined #openstack-kolla13:50
sdakeyo13:51
sdakemorning peeps13:51
sdakebjolo ping14:05
*** msimonin1 has joined #openstack-kolla14:05
*** msimonin has quit IRC14:07
sdakeJeffrey4l mariadb recovery looskgood14:08
*** tonanhngo has joined #openstack-kolla14:11
*** tonanhngo has quit IRC14:12
*** williamcaban has joined #openstack-kolla14:14
*** msimonin1 has quit IRC14:17
*** inc0 has joined #openstack-kolla14:18
*** msimonin has joined #openstack-kolla14:20
*** zhurong has quit IRC14:20
*** msimonin has quit IRC14:21
*** g3ek has quit IRC14:21
*** haplo37 has quit IRC14:21
*** rhallisey has joined #openstack-kolla14:28
*** g3ek has joined #openstack-kolla14:29
sdakegroan election year winding down - time to ditch the gold14:29
kfox1111_did anything change in the last couple weeks for cinder user specificaion in nova?14:30
*** haplo37 has joined #openstack-kolla14:30
sdakekfox1111_ are ou ttaking aobut the dockerfile spec or k8s spec?14:30
sdakekfox1111_ do you know how to use git blame?14:30
kfox1111_genconfig14:30
*** salv-orlando has joined #openstack-kolla14:30
sdakei haven't seen any reviewsgo by - but then again i dont see all reviews14:31
kfox1111_I finally noticed the cinder attach is trying to use user 'cinder'14:31
kfox1111_it use to use 'nova' I think.14:31
sdakegroan14:31
sdakehope that doesn't present an upgrade problem14:31
sdakelet me check the code with git blame14:31
kfox1111_it may be something different in the gate, but it looks pretty similar to the minikube one that worked a few weeks ago.14:31
sdaketry it yourself oto - if you haven't tried git blame it shows you each line in the file and date of change14:32
*** tonanhngo has joined #openstack-kolla14:32
*** msimonin has joined #openstack-kolla14:33
*** msimonin has quit IRC14:33
*** tonanhngo has quit IRC14:33
inc0sdake, btw I pushed some code to your PS14:33
inc0I hope you don't mind14:33
sdakeinc0 thanks dude14:34
sdakeno totally appreciated :)14:34
sdakehelps shorten the implementation cycle14:34
sdakeI wasbusy yesterday14:34
sdakeinc0 i never mind if people push code on my patchsets as longas they dont f' em up ;)14:34
inc0my setup went fubar (you know it's all vms in openstack)14:34
sdakeinc0 thanks for telling me though - i didnt get notified in email14:34
inc0so I need to fully redeploy it14:34
*** msimonin has joined #openstack-kolla14:35
sdakeoh you run vms?14:35
inc0yeah14:35
sdakeI use bare metal - much less painful14:35
inc0but lots of them14:35
*** msimonin has left #openstack-kolla14:35
*** salv-orlando has quit IRC14:35
sdakei have 10 node cluster14:35
inc0I had 5 node for ansible and 5 node for k8s:)14:35
sdakewhich i am going to start using very soon all the time14:35
kfox1111_hmm... no, its still hard coded in trunk:14:35
kfox1111_rbd_user = nova14:35
sdakeya unclear if i will be able to hold onto my hardware too :(14:35
sdakekfox1111_ there is this commit cfe93911dadd710ca5041726d625aa8959bcc7e614:36
kfox1111_oddly: nova logs have:14:36
kfox1111_Generated XML ('<disk type="network" device="disk">\n  <driver name="qemu" type="raw" cache="none" discard="unmap"/>\n  <source protocol="rbd" name="volumes/volume-148e06ef-3a74-4934-8434-eb66d007b7a8">\n    <host name="172.17.0.1" port="6789"/>\n  </source>\n  <auth username="cinder">\n    <secret type="ceph" uuid="6e7fdc4e-2cb6-4577-bffa-8803ffe0b747"/>\n  </auth>\n  <target bus="virtio" dev="14:36
kfox1111_vdb"/>\n  <serial>148e06ef-3a74-4934-8434-eb66d007b7a8</serial>\n</disk>\n14:36
kfox1111_looking...14:36
sdakekfox1111_ what is wrong with that nova log line in your mind14:36
sdakehelp me pick out the problem - just woke up ;)14:37
kfox1111_the "auth username" part maybe?14:37
kfox1111_I'm still trying to figure out the issue rally. I've spent a day and a half on it. :/14:37
sdakethis is when mounting a device14:37
sdakekfox1111_ thanks for asking for help then14:38
kfox1111_everything looks just about fine, but the cinder volume fails to attach in libvirt.14:38
sdakei know cinder worked as of a few weeks ago14:38
kfox1111_most of the issue was trying to get enough logging to actually see it fail.14:38
* sdake WTB BETTER GATING14:38
* kfox1111_ nods. had it working with minikube.14:38
sdakekfox1111_ btw, have you rebuilt all your codebase14:38
sdakekfox1111_ your containers14:38
kfox1111_using 2.0.2 containers.14:39
sdakekfox1111_ oh ya- nm then14:39
kfox1111_so they are identicle. always. :)14:39
sdakekfox1111_ the cool thing about that is - we know its not the dockerfiles :)14:39
* kfox1111_ nods14:39
sdakekfox1111_ i really think you want to add build on demand there14:39
sdakekfox1111_ but we can tackle that later14:39
kfox1111_thats one of the reasons I did that. reduced variability.14:39
sdakerather build per commit14:39
* kfox1111_ nods14:40
sdakeya but you need that variability to make sure the code is good14:40
kfox1111_we need a set of gate jobs with stable containers to test k8s against.14:40
sdakeanyway back t topic at hand14:40
kfox1111_we need a stable set of k8s stuff to test kolla containers against.14:40
sdakeya cros repo gating - will be interresting14:41
kfox1111_we'll get there. :)14:41
sdakei'm frieghtened a bit by the repo split14:41
sdakeit looks easy - but will be challenging14:41
sdakethe mechanics of the split itself - making the repos is easysauce14:41
sdakethe mechanics of git rm is easy14:41
sdakethe gating - groan14:41
kfox1111_yeah. it will have some fallout. gate checks wil lbe important. :)14:41
kfox1111_its so much easier once you have a good set of tests though. getting there really sucks. :/14:42
kfox1111_hmm... no real cinder section...14:47
kfox1111_oh. its libvirt.14:47
kfox1111_ok... so, somethings wrong with generating the config.14:48
kfox1111_and its using default values.14:49
kfox1111_I've got the following overrides set:14:49
kfox1111_enable_ceph: "yes"14:49
kfox1111_cinder_backend_ceph: "yes"14:49
kfox1111_nova_backend_ceph: "no"14:49
HyperJohnGraham_morning all :)14:49
kfox1111_and the conditional in the template is:14:49
kfox1111_{% if enable_ceph | bool and nova_backend == "rbd" %}14:49
kfox1111_and in the defaults...14:50
kfox1111_nova_backend_ceph: "{{ enable_ceph }}"14:50
kfox1111_nova_backend: "{{ 'rbd' if nova_backend_ceph | bool else 'default' }}"14:50
kfox1111_this might be the issue.14:51
kfox1111_I don't want to back nova with cinder, but want cinder volume support anyway.14:51
*** tonanhngo has joined #openstack-kolla14:52
*** tonanhngo has quit IRC14:53
sdakekfox1111_ sorry got in a side convo ;)14:53
kfox1111_no worries14:53
*** kfox1111_ is now known as kfox111114:53
sdakekfox1111 ok read your log14:54
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: External Ceph Tools and Docs  https://review.openstack.org/38104114:56
kfox1111hmm...14:57
kfox1111the rbd_secret_uuid section isn't in libvirt either...14:57
kfox1111this is making me question my sanity a little. I could have swarn I tested cinder volume attachment in minikube.14:58
sdakewhat about qemu.conf?14:59
sdakekfox1111 so - the problem is in nova then?14:59
sdakeyou don't want nova to use ephemeral storage?14:59
kfox1111yeah.14:59
kfox1111I usually configure my clouds with local ephemeral.14:59
kfox1111its more explicit.14:59
kfox1111if the user has a cattle, they can use local ephemeral. if they have a pet, they can back it with a cinder volume.15:00
kfox1111if you force ephemeral in nova, there is no choice.15:00
sdakekfox1111 the only real solutin I have for you is to test out your theory15:01
* kfox1111 nods. that ps does that.15:02
sdakekfox1111 that is - change the config to not back cinder15:02
sdakeoh15:02
sdake:)15:02
sdakesorry - laggy in the mornings a bit15:02
kfox1111https://review.openstack.org/#/c/381041/100..101/tools/setup_gate.sh15:02
kfox1111doh... if I can type it right....15:02
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: External Ceph Tools and Docs  https://review.openstack.org/38104115:02
kfox1111had an extra = in it.15:03
*** yingjun_ has quit IRC15:05
*** yingjun has joined #openstack-kolla15:05
*** yingjun has quit IRC15:10
*** tonanhngo has joined #openstack-kolla15:12
openstackgerritpawnesh kumar proposed openstack/kolla: Use upper constraints for all jobs in tox.ini  https://review.openstack.org/38422115:13
*** liyifeng has joined #openstack-kolla15:16
*** kuntumtarm has quit IRC15:18
sdakeinc0 our gov docs need changning15:26
sdakeinc0 and meeting repo15:26
inc0how so?15:26
sdakeinc0 i am listed as the ptl15:26
sdakemight as  well submit reviews to change that15:26
sdakeand I'll +1 em15:26
sdakenobody is going to submit them for you :)15:27
sdakei mean I could - but sort of busy15:27
sdakethere is also a cross project wiki that has my name all over it15:28
sdakethat needs changing15:28
sdakerather a taem coordination point15:28
sdakeor something15:28
inc0https://github.com/openstack/governance/blob/master/reference/projects.yaml#L2453 <- this is up to date15:28
sdakei forget what its called15:28
inc0which one you meant?15:28
sdakeinc0 oh nice someone did change it for you15:28
sdakeiinc0 ther eis a meeting repo15:29
sdakenot sure if its changed there too15:29
sdakei saw in gov and meetings repo it was not changed a few days ago15:29
sdakei am pretty sure last night actually15:29
sdakebecuase I saw that docs: and thought that was odd15:29
sdakehttps://wiki.openstack.org/wiki/CrossProjectLiaisons15:30
sdakeinc0 i'll handle release management until endof 3.0.0 - then its up to you to decide if you want to do it yourself or delegate it15:30
sdakethe other stuff can be sorted out at your liesure :)15:30
*** ayoung has quit IRC15:31
*** salv-orlando has joined #openstack-kolla15:31
inc0sdake, when we're done with 3.0 I'll put my name there for time being15:31
inc0then gonna distribute;)15:31
sdakesounds good15:32
sdakei always worked on distributing that15:32
sdakebut all that cross project stuff is new15:32
sdakelike 3-4 months15:32
sdakecoolsvap was kind enough to be our cross project liason15:32
sdakenot sure if he is attending those meetings orn ot15:32
sdakeinc0 note please don't change the release liason - as i need to do the tagging for 3.0.015:33
sdakeafter that, we can change it to whoever15:33
sdakethe release team checks the release liason15:33
sdaketo see if they are authorized15:33
sdakenot sure why this stuff is in wiki15:34
sdakeshould be in yaml with web frontend15:34
sdakemaybe justnot enough time15:34
inc0I'll just keep it all like it's now till release15:34
sdakeok sound sgood15:35
sdakesomethign to keep in mind dfor future15:35
*** salv-orlando has quit IRC15:36
sdakeinc0 do you know whwat eta is on binary version of ubuntu newton?15:37
inc0I think they build every 2 weeks or so, let me see current packages15:38
kfox1111hmm... that totally didn't work. :/15:38
sdakethey build master every couple of weeks?15:38
inc0I think they already have it15:39
sdakeinc0 - its super important to get that in if possible :)15:39
sdakeinc0 if you have time ma patch there would be great15:39
sdakei have done centos rdo and source15:39
inc0I'm doint it right now15:39
sdakei'd do deb but dont know how15:39
sdakerather ubuntu15:39
sdakecool15:39
sdakethanks :)15:40
inc0good news, it's already there15:40
sdakecool15:41
inc0https://github.com/openstack/kolla/blob/master/docker/base/sources.list.ubuntu#L1115:41
inc0we'll have to bump it to ocata when it arrives, which is soon15:41
sdakeinc0 where does the oslo.* and othe rpython packages come from for ubuntu binary - surely not that repo?15:42
inc0yeah, pretty sure15:42
sdakeok cool15:42
inc0I'll confirm when I deploy15:42
sdakeso we are good to go15:42
sdakewe should add a binary ubuntu gate15:43
sdakeonce we get these critical/high bugs fixed ;)15:43
sdakewe have a week after the 12th to tag our final15:43
sdakeactually couple weeks i think15:43
sdakebut its going to be done before summit imo15:43
sdakeinc0 just checked out your patch15:48
sdakeinc0 downside of this approach is an extra task for every named volume15:48
sdakeinc0 further downsidde is it doesn't work for nova ;)15:48
sdakenova has stuff owned by multiple users15:48
sdakebut it looks pretty good solution minus those two parts :)15:49
inc0don't forget it's upgrade play15:49
inc0that's going to be removed as soon as we fix all UIDs15:49
inc0but ad nova, what exactly do you mean? which files?15:49
sdakeinc0 nova has qemu files and libvirtfiles and nova files iirc15:51
sdakeits been awhile since i've poked inside a nova container15:51
sdakeit can probably be done with this method15:51
sdakewill require multiple tasks15:51
sdakei like that the work is not inside the containers15:51
sdakethat is my main issue :)15:52
sdake(minus the need to modify base of course)15:52
inc0so I was thinking15:52
inc0I don't think we need to setup uids in base15:52
inc0in fact we can set them up in end containers and reuse same set of uids15:52
sdakemake it harder..15:52
inc0I think it will be easier15:53
inc0and super easy to distribute15:53
sdakewe dont have time to distribute15:53
sdakethis needs to be done today15:53
sdakeby one person15:53
sdakeit could be split in two15:53
sdakethe magic that chowns the ownership of the files15:53
sdakedistribution serves a different purpose in this project15:54
sdakea full map in base will be far easier to maintian15:55
sdakethen spread across 100 container files15:55
sdakeinc0 with your task method, we are looking at ALOT of tasks15:56
sdakeits not just rabbitmq that needs to be chowned, epmd also needs a chown15:56
inc0but we won't need to change the most important dockerfile every time we add something new to Kolla15:56
sdakeinc0 ya - thats a tradeoff15:57
inc0and since we can reuse same uids/gids because of separation15:57
sdakeand the tradeoff is worth it imo15:57
inc0I don't see value in doing it in base really15:57
sdakeit keeps it in one place15:57
inc0but it doesn't have to be in one place imho15:57
inc0in fact, we could do a macro to create users15:58
inc0with gids and whatnot15:58
sdakei understand y you think it doesn't have to be in oone place and agree it doesn't15:58
sdakebut there is significant value thereof15:58
sdakeya macro would be good15:58
sdakein base ;)15:58
sdakethe essential value of base is that the passwd file iand groups file is shared among containers15:59
sdakewe really do want that to happen15:59
sdakethe tradeoff is that base must be modified each time a new container is added16:00
sdakemacro makes that a one-liner16:00
inc0well, that will remove ability for us to make contrib containers - something I want16:01
inc0and I don't see value of shared /etc/passwd tbh16:01
sdakei'm not modifyign 100+ dockerfiles16:01
sdakeespeically when I think its the wrong approach16:01
inc0think long term, right now we need to fix it in few16:02
inc0and let's discuss it properlyu16:02
inc0later16:02
sdakeit needs to be fixedd in anything with a volume16:02
sdakenot just rabbitmq16:02
sdakesystemd was added in ubuntu16:02
sdakesystemd uids conflict with all uids in 2.0.216:02
inc0then you need to add this 100 times in one file or 1 time in 100 files16:02
sdakenova and neutron need to communicate16:03
sdakewhich means they need specific uids that are always the same16:03
sdakerather nova and libvirt16:03
sdakethis means the UID/GID # must be hardcoded16:04
inc0how they communicate tho?16:04
sdakevia files in a named volume16:04
inc0ahh nova libvirt16:04
inc0so +1 to hardcoded gids16:04
inc0done in specific Dockerfiles16:04
inc0but whatever, if you do it today in base16:05
inc0we can always move it out in Ocata and do it properly16:05
inc0doesn't really matter16:05
sdakewell depends on your definition of proper ;)16:05
sdakebut wfm16:05
inc0also how about middle ground? like nova-base?16:05
sdakecan you sort out a macro16:05
sdakethat would definately make life easier16:05
*** Xu_Chao has joined #openstack-kolla16:05
sdakewell it would have to go in nova-base in your popropsal I think16:05
inc0yeah, let's make rabbitmq work tho first16:05
sdakeinc0 ubuntu upgrade is totally borked16:06
inc0I'm ok with nova base16:06
rhalliseysdake, inc0, and spec repo16:06
inc0what I'm reluctant to is base itself16:06
rhalliseywe need one16:06
* rhallisey continues reading back16:06
sdakeonly centos rabbitmq is broken16:06
inc0can't we use dir?:(16:06
sdakerhallisey agree spec repo would be helpful but this isn't a feature so much as a bug fix16:07
inc0btw systemd in xenial is a thing in host right?16:07
sdakeas in upgrade is broken16:07
rhalliseyinc0, all projects have a spec repo16:07
sdakeinc0 right16:07
rhalliseyinfra has tools that renders specs16:07
sdakerhallisey not all proects16:07
inc0so if you have ubuntu14.04 on host and 16.04 in container16:07
inc0that shoulld work right?16:07
rhalliseywell major ones do16:07
inc0keep in mind that we don't support host OS upgrades..16:07
rhalliseynot a requirement but a lot of them do16:07
sdakei got that inc016:07
sdakerhallisey ya its probably time mfor kolla16:08
sdakei dont like specs repos in emerging projects but kolla has emerged16:08
sdakei'd recommend keeping k8s spec free as much as possible :)16:08
rhalliseyinc0, the trick part with OS upgrade is docker16:08
*** Xu_Chao has quit IRC16:08
sdakeok lets not solve that problem right now ;)16:08
inc0rhallisey, but not our problem;)16:08
rhalliseyif we support both version we should b ok16:08
inc0we support given version of docker, apart from that it's different stuff16:09
sdakeinc0 - macro - your the jinja2 macro genius around here16:10
*** janem has quit IRC16:10
*** jtriley has joined #openstack-kolla16:11
inc0first let's get rabbitmq upgrade work please16:11
sdakeinc0 one problem with nova-base, is keeping track of the hardcoded pids16:11
inc0then I'll macro what we ended up doing16:11
sdakeinc0 UBUNTU UPGRADE IS BROKEN16:11
inc0sdake, then let's make it work for rabbitmq before we move on please16:11
sdakeya fair enough16:12
sdakestill going in bae - tracking hundreds of uids acrosss a whoel bunch of files is not my idea of fun :)16:12
sdakebae/base16:12
inc0sdake, you need to keep track only across nova16:12
sdakenow your optimizing16:12
inc0we can re-use pids16:12
sdakethere may be other cases16:12
inc0uids(16:12
sdakewe really shouldn't16:12
rhalliseyhow long have you been arguing about this :)16:12
rhallisey2 days now?16:13
inc0not too long16:13
sdakeabout 2 hrs rhallisey16:13
sdakeya this is the casebook definition of bikeshedding16:13
inc0so the more uids we have the higher chance that we'll run out of them or get into main OS uid space16:13
inc0which is arguably worse16:13
inc0(and if that actually works is a fucking bad docker security)16:13
sdaketiliting at windmills16:13
sdakeinc0 did you test your patch16:17
inc0I did not - noted in comment16:18
inc0my env went fubar16:18
inc0and I was commiting yesterday evening16:18
sdakedid master break? http://logs.openstack.org/29/384029/3/check/gate-kolla-dsvm-deploy-centos-binary-centos-7-nv/67290d3/console.html#_2016-10-09_01_18_05_80581816:20
sdakenah python in your patch looks wrong16:22
sdakei'll fix16:22
inc0sure16:22
sdakei still lprefer to do it in one place rather then all over the various playbooks16:22
sdakebut that comes with a tradeoff too16:23
*** janem has joined #openstack-kolla16:23
openstackgerritprameswar proposed openstack/kolla: Modify QSG to install dependencies needed for ansible  https://review.openstack.org/38407316:23
sdakerhallisey if we had broken apart the repos earlier - our abi wouldn't have been preserved and consumers would be unable to upgrade this container :(16:24
rhalliseyhm16:24
rhalliseywhy wouldn't have been preseverd16:24
inc0hmm16:25
inc0so my build hangs on "setting up open-iscsi"...16:25
inc0anyone seen it?16:25
rhalliseyI'm attempting to deploy standalone neuton node16:29
rhalliseyin triple16:29
rhalliseyo16:29
rhalliseysee if I can get it right this time :/16:29
*** sdake_ has joined #openstack-kolla16:32
*** salv-orlando has joined #openstack-kolla16:32
*** sdake has quit IRC16:34
*** salv-orlando has quit IRC16:37
*** jtriley has quit IRC16:39
*** imcsk8 has quit IRC16:41
*** imcsk8 has joined #openstack-kolla16:42
sdake_inc0 people have reported problems with iscsi16:45
sdake_icn0 check launchpad for bug ref16:45
sdake_inc0 the return type for a dictionary update is none not the dictioinary16:46
sdake_inc0 but  testing your patch with a fix now16:46
inc0yeah, I always forget about this one16:46
sdake_inc0 ya it makes no sense to me why its none16:48
sdake_i think i'd want the updated dictionary returned ;)16:48
openstackgerritMerged openstack/kolla: Choose node with largest seqno number for mariadb recovery  https://review.openstack.org/38417016:48
inc0ok I have no idea what happend to iscsi but it works now16:49
inc0had to reboot machine16:49
inc0ehh I need hardware for this16:49
*** haplo37 has quit IRC17:02
sdake_ok deploy works with my patch17:03
*** jtriley has joined #openstack-kolla17:03
sdake_let me try upgrade from 2.0.2 to 3.0.017:03
sdake_be back while machine implodes17:03
*** g3ek has quit IRC17:05
*** mgoddard has joined #openstack-kolla17:05
*** g3ek has joined #openstack-kolla17:08
*** jtriley has quit IRC17:08
*** haplo37 has joined #openstack-kolla17:11
*** mgoddard has quit IRC17:19
sdake_say inc0 - around?17:22
inc0yup17:23
sdake_re ci/cd sessiosn on wednesday @ 9am17:23
sdake_are you sure you want  to ump rightinto that first thingin the morning? :)17:23
inc0thur 9am17:23
sdake_slots for hard topics are better at 11 and 1-217:23
sdake_also need etherpads for all those sessions linked in the schedule17:24
sdake_each summit we produce one master etherpad with links to each sessions etherpads17:24
sdake_and by we, i mean the openstack foudnation17:24
inc0I know, I'll get on it when we put out release fire;)17:25
sdake_ok17:25
sdake_cool - not sure if your being coached by ttx or not17:25
inc0nope17:25
inc0but I've been to summits:)17:25
sdake_ok - well hes got alot on his plate17:25
sdake_so if you feel lost somewhere reach out17:25
rhalliseyya inc017:25
sdake_we have some unwritten process we use17:25
rhalliseylet's start with something super hype first17:26
sdake_i schedule docs or 9am :)17:26
inc0what is super hype?:)17:26
sdake_that way people are nice and relaxed teh rest of the day17:26
inc0docs are 9am Friday:)17:26
sdake_i saw that17:26
rhalliseyhigh availability17:26
rhalliseykolla-k8s17:26
rhalliseyop experieces17:27
sdake_nah dont want contentious or complex topics at 9am slots17:27
rhalliseybaremetal17:27
sdake_rhallisey op exp is a fishbowl17:27
sdake_baremetal definately needs to not be at 9am :)17:27
rhalliseybm is backup17:27
rhalliseyosic review17:27
rhalliseyhow about that17:27
sdake_winning17:27
sdake_thats a great way to start  off the day :)17:27
rhalliseywe can woot for kolla for the whole session17:27
sdake_rhallisey wins :)17:28
rhalliseycan we have koll-k8s happen on thrusday17:29
rhalliseyone in the morning17:29
rhalliseyin in the afternoon17:29
rhalliseyand through HA in there too17:29
sdake_this is where an etherpad comes in handy17:29
inc0yeah I was thinking of osic review switch17:29
rhalliseyyes17:29
inc0ha is next session and requires lots of brain power17:29
rhalliseyk i'll make one with times17:29
rhalliseyactually lets use the existing one17:29
rhalliseyhttps://etherpad.openstack.org/p/kolla-o-summit-schedule17:30
sdake_rhallisey try follow format of last summits please17:30
sdake_rhallisey alot of people hit that etherpad page we had at last summit17:30
rhalliseyya that was useful17:30
sdake_and the summit prior17:30
sdake_the schedule one ;)17:30
sdake_so we dont have to jekr around and click through to see the session details17:31
sdake_its pretty much an optimization for all the devs :)17:31
*** salv-orlando has joined #openstack-kolla17:33
openstackgerritprameswar proposed openstack/kolla:  apt instead of apt-get to install packages  in Ubuntu  https://review.openstack.org/37947717:33
rhalliseyinc0, did  you sumbit the schedule?17:35
sdake_rhallisey yes its live17:35
rhalliseywhere's the summit link17:35
sdake_rhallisey it can be changed17:35
sdake_(or cold e last cycle right up until last minute)17:35
*** inc0_ has joined #openstack-kolla17:35
sdake_rhallisey its int he topic dude ;)17:35
rhalliseynot the summit page but the one edited last time17:35
sdake_LTMGFY?17:35
rhalliseyhttps://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads17:35
rhalliseynothing there yet17:36
sdake_there ya go17:36
*** inc0 has quit IRC17:36
sdake_change Ocata to Mitaka17:36
sdake_rather Newton17:36
sdake_and copy and paste17:36
sdake_into a new etherpad17:36
rhalliseynewton17:36
sdake_and link to it17:36
rhalliseyI remember writing this17:36
rhalliseyhuge pita17:36
sdake_i wrote a bunch of it too :)17:36
rhalliseyso useful though17:36
sdake_we split the work iirc17:37
*** salv-orlando has quit IRC17:38
sdake_hmm can't build 3.0.017:39
sdake_wonder if someone busted my env17:39
sdake_sweet reinstall of requiremetns seems to have fixed it ;)17:46
sdake_damn spoke too soon it appears17:46
rhalliseyhttps://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads17:49
rhalliseyok17:49
sdake_inc0 if your not busy with something else, could you have a look at jeffrey's patch to fix the amriadb_recovery problem17:53
inc0_sdake_, testing rabbitmq thingy17:53
sdake_inc0_ i got that - bug is assigned to me :)17:53
sdake_i am teesting atm17:53
*** jtriley has joined #openstack-kolla17:57
openstackgerritSteven Dake proposed openstack/kolla: [WIP] Fix upgrades for rabbitmq  https://review.openstack.org/38402918:00
* HyperJohnGraham_ hands sdake_ a hammer18:02
*** jtriley has quit IRC18:02
sdake_actualy I have a funeral to attend today in about 1 hour18:02
*** jtriley has joined #openstack-kolla18:02
sdake_but i should have a basic test of rabbitmq done prior18:03
sdake_rabbitmq upgrade that is18:03
HyperJohnGraham_cool18:04
sdake_atleast the funeral is open bar18:05
sdake_that is in my will atm :)18:05
HyperJohnGraham_irish ?18:05
sdake_nah18:05
HyperJohnGraham_;)18:05
sdake_but should be party rather then big somber thing18:05
HyperJohnGraham_yup18:05
sdake_funeral is celebratoin of someone's life18:05
sdake_HyperJohnGraham_ - I can almost get my panemera in my garage now :)18:06
sdake_HyperJohnGraham_ built 5 sheleves yesterday18:07
sdake_rather 5 sets of shelving18:07
HyperJohnGraham_:)18:07
sdake_59$ at costo per18:07
*** jtriley has quit IRC18:07
sdake_HyperJohnGraham_ have you tried kolla-k8s lately?  or is sbezverk waiting for postsummit to help you get rolling?18:07
HyperJohnGraham_i am ready to try but i didnt want to rush anyone seeing as you all have deadlines looming :)18:08
rhalliseyinc0_, https://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads#Kolla18:10
rhalliseyat some point let's talk about moving around some of these18:11
sdake_anyone seen this : ERROR:kolla.image.build.bifrost-deploy:unable to ping registry endpoint https://192.168.1.103:4000/v0/18:12
rhalliseynope18:12
sdake_HyperJohnGraham_ much appreciated18:12
inc0_rhallisey, thanks!18:12
sdake_i just blew away my registry and made a new one prior to this18:12
*** rhallisey has left #openstack-kolla18:13
*** rhallisey has quit IRC18:13
*** rhallisey has joined #openstack-kolla18:13
rhalliseyinc0_, no problem18:13
sdake_i'm pretty sure i have a conflict at the distro requirements timeslot18:13
sdake_i would like to attend that for sure18:13
sdake_let me doublecheck18:14
rhalliseyya I also have conflicts18:15
rhalliseymaybe I should write triplo's so I can see my conflicts18:15
sdake_i have conflict with the 3rd party plugins slot18:16
sdake_but don't need to be there for that18:16
sdake_inc0_ cool thing about taking on the ptl gig now is you get to see how the ptgs work18:22
sdake_and dont have to jerk around with scheduling midcycles18:22
sdake_that is a serious pain :)18:22
sdake_say inc0_ i see some chatter on the ml about PTGs18:23
sdake_do we have PTG space allocated?18:23
sdake_or has that hit your radar yet18:23
sdake_boy building takes FOREVER18:24
sdake_i guess i'll go get a shave18:24
sdake_bbiaf :)18:24
inc0_sdake_, I was asked if we want to be in PTG and answered "ofcourse18:25
openstackgerritprameswar proposed openstack/kolla:  apt instead of apt-get to install packages  in Ubuntu  https://review.openstack.org/37947718:25
*** rbergeron has quit IRC18:31
*** rbergeron has joined #openstack-kolla18:31
sdake_inc0 roger18:33
*** salv-orlando has joined #openstack-kolla18:34
sdake_my images finally pushing18:36
sdake_someone changed my docker settings18:36
*** daneyon has joined #openstack-kolla18:37
sdake_does the dind image do anything useful for us?18:37
sdake_I know we were planning to use that for gate testing; the gate is capped on ram as it is18:38
sdake_curious if it has any other usage?18:38
*** salv-orlando has quit IRC18:39
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: External Ceph Tools and Docs  https://review.openstack.org/38104118:40
openstackgerritSteven Dake proposed openstack/kolla: Mark performance monitoring stack experimental  https://review.openstack.org/38423718:41
sdake_inc0_ - i think enoug htime has elapsed on operator-list18:42
*** daneyon has quit IRC18:42
inc0_regarding heka?18:42
sdake_inc0_ can you follow that thread up with something like "Since nobody responded, we assume deprecating heka is ok with the  operator community so we are proceeding as indicated."18:42
sdake_and then lets get the deprecation notice merged18:43
sdake_yup regarding heka18:43
inc0_https://review.openstack.org/#/c/381293/ sdake_ review this one18:43
kfox1111sdake_: broke down and reviewed the nova code... it looks like rbd_user doens't kick in until rbd_secret_uuid is set. so maybe that will work....18:44
sdake_inc0_ there ya go18:44
sdake_inc0_ jeffrey's requests aren't unreasonable18:46
sdake_not sure they are needed18:46
sdake_inc0_ up to you if you want them in or not - I have +2'ed it, you just need 1 more :)18:46
inc0_yeah I know, but I like to keep release notes bried18:46
inc0_brief18:46
inc0_if somebody needs details, we can provide18:46
inc0_when someone asks;)18:47
sdake_if you read my review, the details are on the "operator list"18:47
sdake_which is right in the  review18:47
sdake_a link thereof18:47
sdake_this is how deprecations are supposed to work18:47
inc0_and link to thread is in commit message18:47
sdake_right18:47
sdake_everything seems right from my pov18:47
sdake_Jeffrey4l ping18:47
sdake_Jeffrey4l when you are about - have a look at https://review.openstack.org/#/c/381293/3/releasenotes/notes/heka-deprecation-d53e757470b3f7b4.yaml18:48
sdake_i've got to jet18:48
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: External Ceph Tools and Docs  https://review.openstack.org/38104118:49
sdake_i am going to tkae a shower - note upgrade is broken entirely atm18:49
sdake_kuryr.yml is msising18:49
*** berendt has joined #openstack-kolla18:52
*** jtriley has joined #openstack-kolla19:03
*** jtriley has quit IRC19:08
kfox1111YAY! :)19:09
kfox11112016-10-09 19:06:34.023304 | ++ sshpass -p 'cubswin:)' ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no cirros@172.18.0.66 ls -l /dev/vdb19:09
kfox11112016-10-09 19:06:34.247822 | brw-------    1 root     root      253,  16 Oct  9 20:06 /dev/vdb19:09
*** pbourke has quit IRC19:09
*** pbourke has joined #openstack-kolla19:10
*** msimonin has joined #openstack-kolla19:18
rhalliseynice work kfox111119:22
rhalliseykolla will have to copy some of your gate work19:23
rhallisey:)19:23
*** berendt has quit IRC19:23
kfox1111odd.. centos based stuff still failed.19:27
kfox1111some other minior issue though...19:27
kfox1111well, this is one datapoint... should run more...19:31
*** jmccarthy has quit IRC19:34
*** jmccarthy has joined #openstack-kolla19:34
*** janem has quit IRC19:46
*** haplo37 has quit IRC19:54
*** g3ek has quit IRC19:55
*** haplo37 has joined #openstack-kolla19:56
*** g3ek has joined #openstack-kolla19:57
*** jtriley has joined #openstack-kolla19:57
*** matrohon has quit IRC19:57
*** janem has joined #openstack-kolla20:00
kfox1111hmm... one retry so far. both times ubuntu worked, and centos not...20:04
kfox1111evidence is growing.... :)20:05
*** jtriley has quit IRC20:08
*** mewald has quit IRC20:31
*** inc0_ has quit IRC20:34
*** dwalsh has joined #openstack-kolla20:36
*** msimonin has left #openstack-kolla20:46
*** Jeffrey4l has quit IRC20:49
*** Jeffrey4l has joined #openstack-kolla20:55
*** dwalsh has quit IRC21:01
*** jtriley has joined #openstack-kolla21:04
*** jtriley has quit IRC21:08
*** salv-orlando has joined #openstack-kolla21:30
*** neilus has joined #openstack-kolla21:37
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: External Ceph Tools and Docs  https://review.openstack.org/38104121:57
*** jtriley has joined #openstack-kolla21:58
*** bjolo has quit IRC22:01
*** bjolo has joined #openstack-kolla22:01
*** dave-mccowan has joined #openstack-kolla22:02
*** jtriley has quit IRC22:03
*** jtriley has joined #openstack-kolla22:03
*** jtriley has quit IRC22:08
*** neilus has quit IRC22:17
*** neilus has joined #openstack-kolla22:21
*** neilus has quit IRC22:26
*** neilus has joined #openstack-kolla22:27
*** ssurana has joined #openstack-kolla22:30
*** haplo37 has quit IRC22:32
*** janem has quit IRC22:32
*** salv-orlando has quit IRC22:33
*** g3ek has quit IRC22:33
*** haplo37 has joined #openstack-kolla22:38
*** g3ek has joined #openstack-kolla22:39
*** bjolo has quit IRC22:53
*** bjolo has joined #openstack-kolla22:55
*** sdake has joined #openstack-kolla22:57
*** sdake_ has quit IRC22:59
*** sdake has quit IRC23:01
*** lamt has quit IRC23:04
*** neilus_ has joined #openstack-kolla23:13
*** neilus has quit IRC23:14
*** haplo37_ has quit IRC23:40
*** neilus has joined #openstack-kolla23:42
*** haplo37_ has joined #openstack-kolla23:42
*** neilus__ has joined #openstack-kolla23:44
*** neilus_ has quit IRC23:44
*** neilus__ has quit IRC23:44
*** neilus_ has joined #openstack-kolla23:46
*** neilus has quit IRC23:46
*** neilus_ has quit IRC23:50
« Saturday, 2016-10-08 Index Monday, 2016-10-10 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!