Monday, 2016-03-21

sdakeSiRiuS_ cool thanks00:00
sdakeas far as i can tell haproxy keeps going south00:00
SiRiuS_sdake, maybe try and deploy without haproxy00:01
sdake<6>2016-03-20T23:55:13Z groan Keepalived_healthcheckers[15]: Netlink: filter function error00:01
sdakegroan00:02
sdakeselinux00:02
SiRiuS_sdake, yeah, you need to disable that... like the docs say :)00:03
sdakelol00:03
sdakeya rtfm right00:03
sdakei wrote many of that :)00:03
sdakekolla works remarkably well with selinux enabled ;-)00:07
*** bmace has quit IRC00:07
*** bmace has joined #kolla00:07
SamYaplesdake: it does not work at all with selinux enabled when deploying00:08
SamYapleafter deployment, yea00:08
sdakei deployed with selinux enabled just fine00:09
SamYaplei assure you, you did not. either selinux was not actually enabled, or it was already deployed00:09
SamYapleselinux _will_ block sudo that we use if its enabled without the right profiles00:09
sdakei'm in the wheel group00:10
sdakebut it doesn't matter it was definately enabled and i've deployed  about 15 times00:10
sdakemay not work multinode00:10
SamYaplethat has absolutely nothign to do with it00:10
sdakealthough haproxy tanks badly00:11
SamYapleyour user is not the one that runs anything, its all the docker daemon00:11
sdaketry it for yourself00:11
SamYapleI have. i am well aware of where it breaks and why00:11
SamYaplewe talked about it at the midcycle00:11
SamYapleso either you have some super awful policies letting a bunch of stuff through.... or its not enabled00:12
SamYapleboth are possible00:12
sdakeits not enabled now00:12
sdakei have defult enforcing policies00:12
sdakehave/had00:12
SamYaplei did a big writeup on this for kolla. kolla doesnt work with selinux for deployment, it kinda works after deployment00:12
sdakeit may noot eploy correctly but it finishes deploying00:13
sdakeand init-runonce works00:13
sdakebooting vms does not00:13
SamYapleif you say so sdake00:13
openstackgerritAngus Salkeld proposed openstack/kolla-mesos: Fix python 3 unit tests  https://review.openstack.org/29505400:13
*** Jeffrey4l__ has quit IRC00:19
openstackgerritMerged openstack/kolla: Explain how to use kolla-ansible pull  https://review.openstack.org/28499100:21
sdakehttps://bugs.launchpad.net/tripleo/+bug/136774200:23
openstackLaunchpad bug 1367742 in tripleo "keepalived brings the VIP up in multiple places on a controller reboot" [High,Triaged]00:23
*** sdake has quit IRC00:24
openstackgerritMerged openstack/kolla: Update quickstart doc to use drop-in unit files  https://review.openstack.org/29486900:24
*** ccesario has joined #kolla00:26
*** sdake has joined #kolla00:29
SamYaplesdake: whats up with the vip thing00:31
sdakenot sure00:31
sdakelaptop vm who knows such things00:31
SamYapleno i mean why did you post it00:34
sdakebecause i had that netlink error00:34
sdakekeepalived was spewing it00:35
sdakeafter clean reboot still spewing it00:39
sdakeSamYaple what do you make of this http://paste.fedoraproject.org/343188/45852084/ -> http://paste.fedoraproject.org/343188/4585208400:41
sdakekeepalived entered backup state immediatly it appears00:41
sdakenova api still producing 104 errors00:42
SamYaplekeepalived wont be in master state anymore00:42
sdakeit goes straight from master to backup?00:43
SamYaplei meant the config file00:44
sdake2016-03-21 00:36:41.043 24 WARNING nova.image.glance [req-1ce2db9d-da94-4ff6-a509-330d67331884 bf5f7206697d471aab3f95a83ad0cffe 14154fc479e6403ba3ca84fe21a9d8ab - - -] No protocol specified in for api_server 'http://192.168.56.101:9292', please update [glance] api_servers with fully qualified url including scheme (http / https)00:45
sdakethat doesn't seem to make any sense00:45
sdakethat is why nova boot is timing out00:45
sdakethat raises an exception00:46
sdake[glance]00:47
sdakeapi_servers = 192.168.56.101:929200:47
sdakeya thats the problem loos like00:47
SamYapleneeds protocol00:49
SamYaplewierd. are you running ceph?00:49
sdakenope00:49
SamYaplethis error is not in my logs00:49
sdakebut fixing it removed the erorr but didn't fix nova boot00:50
sdakemaybe it was reverted in master00:50
sdakecentos binary is weeks behind00:50
*** vhosakot has quit IRC00:52
SamYaplekeystone releases an 8.1.0?!?00:55
SamYaplesdake: http://lists.openstack.org/pipermail/openstack-dev/2016-March/088219.html00:56
*** iceyao has joined #kolla00:57
sdakeodd no-reply in headers of message00:59
*** iceyao has quit IRC01:02
*** iceyao has joined #kolla01:06
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505201:08
openstackgerritSam Yaple proposed openstack/kolla: Split versions onto new line  https://review.openstack.org/29505101:08
openstackgerritSam Yaple proposed openstack/kolla: Update to latest stable versions  https://review.openstack.org/29505001:08
openstackgerritSam Yaple proposed openstack/kolla: Allow version_check to work on local git repo  https://review.openstack.org/29505601:08
openstackgerritSam Yaple proposed openstack/kolla: Dynamically pull list of projects and versions  https://review.openstack.org/29505701:08
openstackgerritSam Yaple proposed openstack/kolla: Allow version_check to work on local git repo  https://review.openstack.org/29505601:11
openstackgerritSam Yaple proposed openstack/kolla: Dynamically pull list of projects and versions  https://review.openstack.org/29505701:11
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505201:11
openstackgerritSam Yaple proposed openstack/kolla: Split versions onto new line  https://review.openstack.org/29505101:11
openstackgerritSam Yaple proposed openstack/kolla: Update to latest stable versions  https://review.openstack.org/29505001:11
SamYapleugh i hate patchstreams01:11
*** daneyon has joined #kolla01:12
*** daneyon has quit IRC01:20
*** phuongnh has joined #kolla01:21
*** vhosakot has joined #kolla01:22
sdakenova api says cirros image not found01:24
sdakeregistered image via member-create01:25
sdakeand updated to accepted01:25
sdakestill no bueno01:25
sdakeSamYaple thoughts?01:25
SamYaplei dont know what is wrong with your environment, no. i would stil lguess glance personally01:25
SamYapleerrr rabbitmq01:25
SamYaplesdake: this really needs to merge like yesterday01:28
SamYaplehttps://review.openstack.org/#/c/294921/01:28
SamYaplemariadb multinode is bust01:28
SamYapleinfact theres a bunch of patches in the queue it would be cool if you could ack01:29
SamYapleim sitting on ~10 patches to make my environment work01:29
sdakei am actually getting ready to leave to phoenix in about 5-10 minutes01:29
sdakebut if i'm still alive when I get home I'll go over the review queue01:29
*** Liuqing has joined #kolla01:30
SamYapleok01:30
SamYapleplan is still to tag tomorrow, yes?01:30
SamYapleor rather branch01:30
sdakelets get these reviews in01:33
sdakeand branch when kolla is stable01:33
sdakeit may be unstable on my end because of master vs rcs built in my environment01:33
sdakehard to tell01:33
sdakebut i dont have time atm to rebuild, i am packing my computer to roll out01:34
sdake2 hour drive01:34
sdakei should be home in 3 hours or so01:34
SamYaplewell we dont tag after a branch so we still ahve time for "stable"01:34
SamYaplebut youre right sdake it shouldnt be known broken01:35
SamYaplebut we are gating on master code right now so its dangerous01:35
sdakeright i understand the issue clearly01:35
sdakebut if there  are patches we know fix nonmsater brekaage we shoudl mere them01:35
sdakeyou mentioned you have 10 patches01:35
SamYapleyes01:36
SamYaplevarious things from critical to needed01:36
sdakemake sure all the stuff has bug ids please01:36
sdakei've got to roll - wife yelling01:36
sdakelater01:36
SamYaplebye01:36
*** sdake has quit IRC01:36
SamYaplemandre: you around?01:37
mandrenot for long SamYaple01:37
mandrewhat's up?01:37
SamYaplei was going to be for some reviewes01:38
SamYaplebut if youre leaving its fine01:38
mandrepoint me to them, we'll see if I have time :)01:38
SamYapleok its a few in order of please-merge-now01:39
SamYaplehttps://review.openstack.org/29492101:39
mandreBTW, I'll be on and off for the next 2 weeks01:39
SamYaplehttps://review.openstack.org/29486301:39
SamYapleno problem01:39
SamYaplehttps://review.openstack.org/#/c/29486201:39
SamYaplehttps://review.openstack.org/#/c/294937/01:40
SamYaplehttps://review.openstack.org/#/c/294931/01:40
SamYaplehttps://review.openstack.org/#/c/293797/01:40
SamYaplehttps://review.openstack.org/#/c/29372801:40
SamYapleerr, not that last one01:40
mandreyou know what, I'll just search for patches with owner:SamYaple :)01:41
SamYaplehttps://review.openstack.org/#/q/owner:sam%2540yaple.net+status:open01:41
SamYapleanything from mysql up01:41
mandrearghh I had even less time than I though01:43
mandrebbl01:43
SamYaplelol01:43
*** vhosakot has quit IRC01:45
*** weiyu has joined #kolla01:45
*** vhosakot has joined #kolla01:45
*** iceyao_ has joined #kolla01:53
*** iceyao has quit IRC01:53
*** Liuqing has quit IRC01:57
*** SiRiuS_ has quit IRC01:58
*** iceyao has joined #kolla02:02
openstackgerritVikram Hosakote proposed openstack/kolla: Fix Linux bridge reconfig issue  https://review.openstack.org/29505902:02
*** iceyao_ has quit IRC02:06
openstackgerritMerged openstack/kolla: Add check if all the hostnames are resolvable  https://review.openstack.org/29471302:10
*** rhallisey has quit IRC02:11
*** iceyao_ has joined #kolla02:20
openstackgerritMerged openstack/kolla: Attach external NIC to a NAT-Network if on Wi-Fi  https://review.openstack.org/29434002:21
*** vhosakot has quit IRC02:22
SamYaplethanks for the reviews asalkeld02:23
SamYaplelots of little things to merge before we branch02:23
asalkeldnp02:23
*** iceyao has quit IRC02:23
*** vhosakot has joined #kolla02:48
*** klint has joined #kolla02:51
*** mdnadeem has joined #kolla03:00
openstackgerritSam Yaple proposed openstack/kolla: Dynamically pull list of projects and versions  https://review.openstack.org/29505703:01
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505203:01
openstackgerritSam Yaple proposed openstack/kolla: Split versions onto new line  https://review.openstack.org/29505103:01
*** daneyon has joined #kolla03:02
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505203:05
openstackgerritMerged openstack/kolla: Fix Linux bridge reconfig issue  https://review.openstack.org/29505903:08
openstackgerritSam Yaple proposed openstack/kolla: Dynamically pull list of projects and versions  https://review.openstack.org/29505703:10
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505203:10
openstackgerritSam Yaple proposed openstack/kolla: Split versions onto new line  https://review.openstack.org/29505103:10
*** vhosakot has quit IRC03:17
*** iceyao_ has quit IRC03:21
*** iceyao has joined #kolla03:21
openstackgerritSam Yaple proposed openstack/kolla: Fix heat endpoint registration  https://review.openstack.org/29502503:23
*** dave-mccowan has quit IRC03:27
openstackgerritSam Yaple proposed openstack/kolla: Fix rax-ord gate  https://review.openstack.org/29500103:28
SamYapleasalkeld: would you be willing to take on https://bugs.launchpad.net/kolla/+bug/155982803:41
openstackLaunchpad bug 1559828 in kolla "deprecated heat options" [High,Triaged]03:41
SamYaple?03:41
asalkeldlooking03:41
SamYaplebasically we just want to remove those deprecated things and use non-deprecated things03:41
SamYapleshould be config-option only type stuff03:41
asalkeldyeah, just need a line or two in the heat environment03:42
SamYaplethats what im thinking, but i know nothing about heat03:42
SamYapleif you can handle it i would be grateful03:42
asalkeldk03:43
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505203:48
openstackgerritAngus Salkeld proposed openstack/kolla-mesos: Fix auto doc generation  https://review.openstack.org/29507403:52
asalkeldSamYaple: are you more interested in a user using the deprecated plugin, or just the log?04:00
asalkeldi have a way to disable it, but for some reason it first logs the deprecated, then the fact that it's now disabled04:00
openstackgerritAngus Salkeld proposed openstack/kolla: Disable deprecated heat plugins.  https://review.openstack.org/29507804:05
asalkeldSamYaple: ^04:05
asalkeldthat's my quick solution04:05
*** weiyu has quit IRC04:36
*** vishwanathj_away has quit IRC04:41
*** Marga_ has quit IRC04:43
*** mdnadeem has quit IRC04:48
*** Jeffrey4l__ has joined #kolla04:54
openstackgerritJeffrey Zhang proposed openstack/kolla: Install neutron in manila-share group only when enable_manila is true  https://review.openstack.org/29502405:03
*** barra204 has quit IRC05:13
*** jgriffith has quit IRC05:13
*** jgriffith has joined #kolla05:14
*** weiyu has joined #kolla05:14
*** jgriffith is now known as Guest491605:14
*** weiyu has quit IRC05:25
*** sdake has joined #kolla05:29
*** vishwanathj has joined #kolla05:33
*** sdake has quit IRC05:38
*** sdake has joined #kolla05:41
*** daneyon has quit IRC05:42
nihilifero/05:43
*** weiyu has joined #kolla05:47
*** weiyu has quit IRC05:49
*** Liuqing has joined #kolla05:52
*** Jeffrey4l__ has quit IRC05:53
*** sdake has quit IRC05:55
*** weiyu has joined #kolla05:56
*** openstackgerrit has quit IRC06:17
*** openstackgerrit_ is now known as openstackgerrit06:17
*** openstackgerrit_ has joined #kolla06:18
*** openstackgerrit_ is now known as openstackgerrit06:18
*** openstackgerrit_ has joined #kolla06:19
*** Marga_ has joined #kolla06:30
*** achanda has quit IRC06:31
*** achanda has joined #kolla06:34
*** achanda has quit IRC06:36
*** Jeffrey4l__ has joined #kolla06:36
openstackgerritAngus Salkeld proposed openstack/kolla: Disable deprecated heat plugins.  https://review.openstack.org/29507806:42
*** Jeffrey4l__ has quit IRC06:45
*** vishwanathj is now known as vishwanathj_zzz07:15
*** gfidente has joined #kolla07:30
*** achanda has joined #kolla07:36
*** Marga_ has quit IRC07:41
*** achanda has quit IRC07:42
*** mikelk has joined #kolla08:01
*** stvnoyes has quit IRC08:04
*** stvnoyes has joined #kolla08:05
*** jmccarthy has quit IRC08:06
*** jmccarthy has joined #kolla08:07
*** rstarmer has joined #kolla08:12
*** iceyao has quit IRC08:29
*** iceyao has joined #kolla08:30
openstackgerritMerged openstack/kolla-mesos: Fixing run, show, list for interactive mode  https://review.openstack.org/29234308:32
openstackgerritMerged openstack/kolla-mesos: Added custom help for interactive mode  https://review.openstack.org/29238408:32
*** akwasnie has joined #kolla08:38
*** jmccarthy has quit IRC08:45
*** jmccarthy has joined #kolla08:48
*** Serlex has joined #kolla08:50
*** Jeffrey4l__ has joined #kolla08:55
*** achanda has joined #kolla09:00
*** achanda has quit IRC09:01
*** shardy has joined #kolla09:02
*** shardy_ has joined #kolla09:06
*** mgoddard has joined #kolla09:06
*** mbound has joined #kolla09:14
*** Jeffrey4l__ has quit IRC09:21
*** elemoine has quit IRC09:22
*** SiRiuS_ has joined #kolla09:24
*** elemoine has joined #kolla09:25
openstackgerritMerged openstack/kolla: Allow user to choose filesystem for osd  https://review.openstack.org/29486309:26
*** athomas has joined #kolla09:28
*** rstarmer has quit IRC09:35
*** iceyao_ has joined #kolla09:44
*** iceyao has quit IRC09:46
*** Liuqing has quit IRC09:51
openstackgerritMerged openstack/kolla: Add memcached_servers to keystone_auth section  https://review.openstack.org/29493709:52
openstackgerritMerged openstack/kolla-mesos: Fix python 3 unit tests  https://review.openstack.org/29505409:56
*** kproskurin has joined #kolla10:01
openstackgerritMerged openstack/kolla: Change deprecated options throughout Kolla  https://review.openstack.org/29493110:05
openstackgerritMerged openstack/kolla: Move manila variable  https://review.openstack.org/29379710:05
*** kproskurin has quit IRC10:05
*** ccesario has quit IRC10:06
*** pbourke has quit IRC10:14
*** pbourke has joined #kolla10:14
*** iceyao_ has quit IRC10:20
ccesario_morning !10:40
*** weiyu has quit IRC10:46
*** athomas has quit IRC10:55
pbourkeanyone know how to wrap long lines in gerrit diffs?10:57
*** mbound has quit IRC11:00
*** achanda has joined #kolla11:01
*** achanda has quit IRC11:01
*** dims has joined #kolla11:01
*** athomas has joined #kolla11:01
*** phuongnh has quit IRC11:01
*** mbound has joined #kolla11:04
*** rhallisey has joined #kolla11:14
*** Liuqing has joined #kolla11:15
*** shardy has quit IRC11:23
*** shardy_ has quit IRC11:23
*** shardy has joined #kolla11:27
openstackgerritMichal Rostecki proposed openstack/kolla-mesos: [WIP] Fix OpenStack services registration  https://review.openstack.org/29125111:29
openstackgerritMichal Rostecki proposed openstack/kolla-mesos: Adjust profiles to already implemented services  https://review.openstack.org/29522211:29
openstackgerritMichal Rostecki proposed openstack/kolla-mesos: Fix the service conf file path  https://review.openstack.org/29522311:29
*** mlima has joined #kolla11:30
*** dave-mccowan has joined #kolla11:47
*** mbound_ has joined #kolla11:48
*** mbound has quit IRC11:48
*** Liuqing has quit IRC12:01
*** sbezverk has joined #kolla12:01
*** iceyao has joined #kolla12:02
*** dencaval has joined #kolla12:02
*** Liuqing has joined #kolla12:03
*** dwalsh has joined #kolla12:06
openstackgerritSam Yaple proposed openstack/kolla: Fix mysql bootstrap  https://review.openstack.org/29492112:23
*** Liuqing has quit IRC12:23
*** Liuqing has joined #kolla12:24
*** dwalsh has quit IRC12:25
SamYapleasalkeld: yea what you have works for me if you think thats best. Im just trying to undeprecate mitaka before release12:26
*** jpeeler has joined #kolla12:26
SamYaplepbourke: can you reconsider https://review.openstack.org/#/c/294638/12:27
SamYapleim not looking to play with formatting right before we branch12:27
*** Liuqing has quit IRC12:27
openstackgerritSam Yaple proposed openstack/kolla: Dynamically pull list of projects and versions  https://review.openstack.org/29505712:29
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505212:29
openstackgerritSam Yaple proposed openstack/kolla: Split versions onto new line  https://review.openstack.org/29505112:29
SamYaplepbourke: and also https://review.openstack.org/#/c/294206/12:30
SamYaplepbourke: if you just want ot hit the whole review queue, we are branching stable/mitaka today asap12:30
SamYapleneed to make sure queue is empty of mergable patches12:30
SamYaplelike this one pbourke https://review.openstack.org/#/c/294862/ tis got to merge12:32
SamYaplefixes all the osd races12:32
pbourkeSamYaple: the commit msg on the heka log patch doesn't quite make sense to me12:32
pbourke"we dont have to copy all configs over at the same time"12:33
SamYaplethe reason we were copying over all of the heka files was because we didnt have reconfigure task12:33
SamYapleso there was no way to get new files to take affect12:33
SamYaplenow we do so we dont have to copy them12:33
openstackgerritMerged openstack/kolla: Enabled nova->glance snapshots  https://review.openstack.org/29463812:33
SamYaplewe dont have to copy them if they shouldnt be copied*12:33
SamYaplelike swift or elasticsearch12:33
SamYapletheyll get copied _if_ you deplyo swift, not always12:34
SamYaple(notice tehre is no when conditinoal on the current swift file copy)12:34
pbourkeyeah I see it now12:34
pbourkethanks12:35
SamYapleakwasnie: i have kibana installed but..... how to use kibana :)12:38
pbourkeI think ive looked at everything outstanding ping me if not12:38
openstackgerritMerged openstack/kolla: Rework heka log copying  https://review.openstack.org/29420612:39
openstackgerritMerged openstack/kolla: Fix 'optional' config.json values  https://review.openstack.org/29435012:39
openstackgerritMerged openstack/kolla: Install neutron in manila-share group only when enable_manila is true  https://review.openstack.org/29502412:39
*** athomas has quit IRC12:40
SamYaplepbourke: ping https://review.openstack.org/#/c/294862/ that one is landing in mitaka (it fixes the ceph-osd races)12:42
SamYapleoh hey pbourke i do a blog thing now. yaple.net12:45
pbourkenice!12:46
SamYapleive been meaning too for a very long time, but i finally got a really good hosted deal and theree we are12:46
pbourkeyou should get a twitter account to go with, seen as noone uses rss anymore ...12:46
SamYaplei dont use either!12:46
SamYapleyea ayoung was telling me to tweeter yesterday12:47
SamYaplemaybe i should12:47
pbourkewell, wordpress gives you rss12:47
SamYaplei should really start building an online presence so when the robots come checking theyll see im tech friendly and pro-robit12:47
pbourkebut I think a lot gave up on it after google killed reader12:47
pbourkesurprised you're not hosting it yourself though12:48
SamYapleayoung made the right call dissuading me of that12:48
SamYaplethis is hosted, so no bandwidth and they apply all the security updatess12:48
SamYaplebeen VERY happy with dreamhost so far12:48
SamYapleplus i can plug things like cloudflare and letsencrypt in on the fly and they auto configure that stuff12:49
pbourkewell, yeah, but all those arguments could also apply to the openstack cluster sitting in your closet :)12:50
SamYaplenot the "they manage teh security updates" argument12:50
SamYapleits a very public facing thing adn this is what they do as a company12:51
SamYapleif i was hosting it at my house and someone compromised it.... well then all my house is vulerable12:51
SamYaplenot cool12:51
*** athomas has joined #kolla12:51
*** klint has quit IRC13:00
openstackgerritMerged openstack/kolla: Fix mysql bootstrap  https://review.openstack.org/29492113:02
*** kproskurin has joined #kolla13:05
*** kproskurin has quit IRC13:09
openstackgerritMichal Rostecki proposed openstack/kolla-mesos: [WIP] Fix OpenStack services registration  https://review.openstack.org/29125113:11
openstackgerritMichal Rostecki proposed openstack/kolla-mesos: Fix the service conf file path  https://review.openstack.org/29522313:11
openstackgerritMichal Rostecki proposed openstack/kolla-mesos: Adjust profiles to already implemented services  https://review.openstack.org/29522213:11
openstackgerritMichal Rostecki proposed openstack/kolla-mesos: Allow to use "run" method only if the runner is enabled  https://review.openstack.org/29526713:11
openstackgerritMichal Rostecki proposed openstack/kolla-mesos: Fix path to ml2_conf.ini files  https://review.openstack.org/29526813:11
akwasnieSamYaple: hi:) you have to specify default index pattern in settings (log-*) and then you will be able to discover and visualize data. There are still some things that we have to add to kibana: default dashboards and default index pattern to make the whole process automatic13:12
*** dwalsh has joined #kolla13:19
*** dims_ has joined #kolla13:34
*** jtriley has joined #kolla13:36
*** dims has quit IRC13:36
openstackgerritRyan Hallisey proposed openstack/kolla: Improve openstack-base image for centos binary  https://review.openstack.org/29504813:41
*** vinkman has joined #kolla13:45
*** vinkman has left #kolla13:51
SamYapleakwasnie: awesome. and docs... :D13:51
*** dave-mccowan has quit IRC13:55
SamYaplerhallisey: look good! the packages. small ask to sort alhpabetically so the diff looks good next time13:56
rhalliseysure13:56
SamYaplealso if you could review this patchstream before our mitakka branch https://review.openstack.org/#/c/295056/13:57
SamYaplefolks we are cutting the stable/mitaka branch today13:57
SamYapleso if there is anything you dont want to have to backport, get a review up asap13:57
*** sdake has joined #kolla13:58
*** kproskurin has joined #kolla13:59
sdakemorning13:59
SamYaplemorning sdake14:00
SamYaplequeue looking good for mitaka fork14:00
SamYaplewere you going to do passwords radnom thing or do you want me too sdake ?14:00
sdakei said i would14:01
sdakeyou mean mitaka branch14:01
sdakeyou mean th equeue has been merged14:01
sdakei just got up - give me 5-10 mns to wake up14:01
*** kproskurin has quit IRC14:01
SamYaplemost of the queue looks good yes14:01
SamYaplei did not know if you would be up for the passwrods thing today14:01
sdakeremember no caffeiene in my diet anymore14:02
sdakeya i can do it in likke 5 minutes14:02
SamYaplegood! youll live longer14:03
openstackgerritRyan Hallisey proposed openstack/kolla: Improve openstack-base image for centos binary  https://review.openstack.org/29504814:04
SamYaplelooks good rhallisey! im looking forward to the time/size of all of this14:06
SamYapledid that script i posted help?14:06
SamYapleor rather onelner14:06
rhalliseyyes14:06
rhalliseyused it exactly as pasted14:06
SamYapleawesome!14:06
rhalliseythanks14:06
SamYaplethats what i do for source too, modified a bit of course14:06
rhalliseyya I checked centos-source also, but there was nothing above 1014:07
SamYaplecool14:07
*** dave-mccowan has joined #kolla14:12
openstackgerritMerged openstack/kolla: Allow external ceph journals and fix bootstrap  https://review.openstack.org/29486214:17
sdakea more reasonable number is probably 5-714:19
sdakebut 10 wfm :)14:19
sdakethe only real thing we don't want to share is compute14:19
sdakee.g. the compute image and base should be small along with libvirt14:20
*** vishwanathj_zzz is now known as vishwanathj14:25
*** spisarski has joined #kolla14:26
*** haplo37 has joined #kolla14:32
*** Jeffrey4l__ has joined #kolla14:40
akwasniehi sdake! Did you see my msg yesterday? " it turns out that shield requires a licence - it starts with free 30 day trial, but after that additional licence is required. It is easy to install and configure (i tried and it works), but license stuff rather eliminates it from being used in kolla."14:41
sdakeas suspected14:42
akwasnieyes, unfortunately14:42
*** mgoddard_ has joined #kolla14:45
*** mgoddard has quit IRC14:48
*** achanda has joined #kolla14:50
SamYapleakwasnie: help me please14:52
SamYaplehow to view logs in kibana14:52
SamYapleakwasnie: as far as i can tell this isnt working at all (i see no logs anywhere)14:53
SamYaplebut its consuming 1GB of ram!14:54
SamYaplesdake: https://review.openstack.org/#/c/295025/ switched to partial bug for you14:58
sdakeSamYaple this is why we need a proper dashboard integration14:59
*** rmart04 has joined #kolla14:59
SamYaplewhy?14:59
*** mgoddard_ has quit IRC15:01
*** mgoddard has joined #kolla15:01
*** vinkman has joined #kolla15:03
*** inc0 has joined #kolla15:05
inc0good morning15:05
SamYaplehey inc0 !15:06
SamYaplegot kibana deployed15:06
SamYaplecannot into logs. how to do?15:06
inc0let me try15:07
inc0akwasnie is around, ask her;)15:07
SamYapleyea i did but i think she grew bored of me asking questions :)15:07
SamYapleand who can blame her15:07
inc0Ala is very patient person, I doubt that15:07
openstackgerritSteven Dake proposed openstack/kolla: Add generate_passwords.py to generate passwords  https://review.openstack.org/29372815:08
inc0I know you can bring even the most patient people to the brink...15:08
*** mlima has quit IRC15:08
SamYapleha.15:09
SamYapleha.15:09
SamYapleyoure one to talk15:09
sdakewell agree with inc0 here ;)15:09
SamYaple:)15:09
SamYapleif anyone wants some easy reviews, take a crack at this patchstream https://review.openstack.org/#/c/29505615:10
openstackgerritSteven Dake proposed openstack/kolla: Add generate_passwords.py to generate passwords  https://review.openstack.org/29372815:13
*** mlima has joined #kolla15:13
sdakeSamYaple re dashboard integration i am not talking about horzino15:14
sdakei am talkingabout a kibana  dashboard15:14
sdakethat presents an opentack-eque view of the world15:14
*** ssurana has joined #kolla15:14
SamYapleoh yea15:15
SamYapleagree15:15
sdaketo make a dashboard you regiter some json with elasticsearch15:18
*** vhosakot has joined #kolla15:18
SamYapleyea the conversation from the other day makes alot more sense now15:18
*** achanda has quit IRC15:19
*** Guest4916 is now known as jgriffith15:23
openstackgerritSteven Dake proposed openstack/kolla: Add generate_passwords.py to generate passwords  https://review.openstack.org/29372815:24
*** vinkman has left #kolla15:25
sdakeyay stock rsu wire day15:26
*** akwasnie has quit IRC15:36
*** daneyon has joined #kolla15:37
*** ccesario_ has quit IRC15:39
*** athomas has quit IRC15:40
openstackgerritSteven Dake proposed openstack/kolla: Add generate_passwords.py to generate passwords  https://review.openstack.org/29372815:40
sdakeSamYaple it was acutally a 2 liner because it needed a wrap15:41
sdakebut i also tidied up the docs15:41
sdakeand fixed the merge with passwords.yml15:41
sdakegit did some wierd vodo magic and totally blew up the file on its automerge15:41
sdakeunfortunately i just cratered my dev environment so i am unable to test if that password.yml file actually deploys15:42
sdakesudo cp passwords.yml /etc/passwd15:43
sdakegroan15:43
sdake8am work ftl15:43
*** athomas has joined #kolla15:45
*** twm2016 has joined #kolla15:50
sdakeinc0 please review https://review.openstack.org/#/c/293728/15:50
sdakerhallisey please review https://review.openstack.org/#/c/293728/15:50
SamYaplesdake: https://review.openstack.org/#/c/295025/15:51
sdakeacked15:52
sdakeanything elsse needing attention?15:52
SamYaplehttps://review.openstack.org/#/c/295056/15:53
SamYaplethat whole stream15:53
openstackgerritMerged openstack/kolla: Fix heat endpoint registration  https://review.openstack.org/29502515:55
openstackgerritMerged openstack/kolla: Allow version_check to work on local git repo  https://review.openstack.org/29505615:59
openstackgerritMerged openstack/kolla: Dynamically pull list of projects and versions  https://review.openstack.org/29505715:59
vhosakotnice merges!16:01
openstackgerritMerged openstack/kolla: Split versions onto new line  https://review.openstack.org/29505116:01
*** vishwana_ has joined #kolla16:02
SamYapleanyone have anything that needs to go into mitaka that they don't want to backport?16:02
*** iceyao has quit IRC16:03
SamYaplesdake: i think the only thing left is generating the passwords16:03
sdakenote i just sent a mila to the ml re backports16:03
*** vishwana_ is now known as vishwanathj_16:04
sdakewtb +2 reviews on https://review.openstack.org/#/c/293728/16:04
*** ccesario has joined #kolla16:05
*** vishwanathj has quit IRC16:06
sdakeinc0 rhallisey jpeeler asalkeld ^^16:06
*** vhosakot has quit IRC16:13
twm2016SamYaple: Sorry to interrupt but I'm having trouble finding the use of the deprecated options from this bug https://bugs.launchpad.net/kolla/+bug/1559828 in the kolla project. Are they used in a separate project?16:15
openstackLaunchpad bug 1559828 in kolla "deprecated heat options" [High,Triaged] - Assigned to Trevor McCasland (twm2016)16:15
*** banix has joined #kolla16:16
SamYapletwm2016: there is already a review up for that. its not linking in launchpad for some reason, but asalkeld is working on it We start by mapping the rbd to a host16:16
SamYapleoops bad paste16:17
SamYapletwm2016: https://review.openstack.org/#/c/295078/16:17
twm2016ah okay, thanks!16:17
openstackgerritCarlos Cesario proposed openstack/kolla: Added note about nested environment  https://review.openstack.org/29538916:26
*** vhosakot has joined #kolla16:26
sdakethere is a whole lot of deprecated options when i ran last16:28
sdakebut it was master and not mitaka16:29
SamYaplesdake: nope those are all gone16:32
SamYaplenow just the stupid not-true user-name deprecation thing16:32
*** aginwala has joined #kolla16:32
*** twm2016 has quit IRC16:36
*** sdake_ has joined #kolla16:38
*** sdake has quit IRC16:41
*** unicell1 has quit IRC16:44
sdake_[sdake@minime-03 tools]$ ./build.py --registry 192.168.1.103:4000 --tag 3.0.0 --push16:44
*** daneyon_ has joined #kolla16:49
*** achanda has joined #kolla16:49
*** aginwala has quit IRC16:49
SamYaple?16:50
*** aginwala has joined #kolla16:50
*** rmart04 has quit IRC16:52
*** daneyon has quit IRC16:52
*** mikelk has quit IRC16:53
*** aginwala has quit IRC16:54
*** vhosakot has quit IRC16:55
*** aginwala has joined #kolla16:55
*** achanda has quit IRC16:57
openstackgerritCarlos Cesario proposed openstack/kolla: Added note about nested environment  https://review.openstack.org/29538916:57
*** mgoddard_ has joined #kolla16:58
*** aginwala has quit IRC17:00
*** mgoddard has quit IRC17:02
openstackgerritSam Yaple proposed openstack/kolla: Add generate_passwords.py to generate passwords  https://review.openstack.org/29372817:02
*** aginwala has joined #kolla17:03
*** aginwala has quit IRC17:03
sdake_msg: Hostname has to resolve to IP address of api_interface17:04
sdake_it does resolve...17:04
*** aginwala has joined #kolla17:04
sdake_which hostname speicfically?17:04
SamYaplesdake type `hostname`17:04
SamYaplesdake_: i knew you were still having rabbitmq issues :P17:05
sdake_this is not a vm, this is my bare metal hardware17:05
sdake_which was working last week fantasitcally and works if this patch is reverted17:05
*** vishwanathj has joined #kolla17:05
SamYaplei dont think it was working quite right then17:05
sdake_what i was running yesterday was a vm17:05
SamYaplethe patch is needed for sure17:06
SamYapleplease post the failure task17:06
sdake_moment17:07
sdake_http://paste.fedoraproject.org/343402/80050145/ -> http://paste.fedoraproject.org/343402/8005014517:07
SamYapleyour hostname is resolving to an ipv6 address17:08
SamYapleneeds to be ipv4 of api_interface17:08
*** vishwanathj_ has quit IRC17:08
*** Marga_ has joined #kolla17:08
sdake_how do i get it to resolve and ipv4?17:09
SamYaplei have to lunch now, but gaurnteed way? disable ipv617:09
sdake_what you said makes no sense to me, its an ipv4 address17:09
SamYapleill think further on this moving forward17:09
SamYaplee80::215:17ff:fe35:bece minime-03\nfe80::42:20ff:fe4e:16e17:09
SamYaplethat ipv6 dude17:09
sdake_i think there is a bug in  there somewhere17:10
sdake_the code should be smart enough to know about ipv4 and ipv617:10
SamYapleagreed, it can be improved. its still needed17:10
SamYaplesdake_: chage `getent hosts` to `getent ahostsv4` and see what comes up17:13
SamYaplemight need more parsing17:13
sdake_i just turned o ff ipv4 but i'll undo and give it a try17:14
sdake_rather turned off ipv617:14
*** Marga_ has quit IRC17:16
*** Marga_ has joined #kolla17:17
sdake_SamYaple ahostsv4 fixes it17:20
sdake_that needs to make the branch as well17:20
*** unicell has joined #kolla17:21
*** Serlex has left #kolla17:22
*** vhosakot has joined #kolla17:23
openstackgerritSteven Dake proposed openstack/kolla: Fix problem where ipv6 address causes failure  https://review.openstack.org/29542017:24
*** unicell has quit IRC17:24
*** unicell has joined #kolla17:24
*** Marga_ has quit IRC17:25
*** Marga_ has joined #kolla17:26
*** aginwala has quit IRC17:29
*** aginwala has joined #kolla17:32
*** mlima_ has joined #kolla17:34
*** jasonsb has quit IRC17:36
*** mlima has quit IRC17:37
*** vhosakot has quit IRC17:37
*** aginwala has quit IRC17:38
*** aginwala has joined #kolla17:38
*** aginwala has quit IRC17:39
*** mbound_ has quit IRC17:39
*** aginwala has joined #kolla17:40
*** sdake_ has quit IRC17:40
*** achanda has joined #kolla17:58
g3ekhello all, can someone explain me how the VIP is mount on the haproxy container? I checked all the ansible/roles/haproxy files and the docker/haproxy files and nothing there explain how the interface is mounted and how the IP is attached to it.17:59
*** mgoddard__ has joined #kolla18:01
*** salv-orlando has joined #kolla18:02
haplo37I think is ansible/roles/haproxy/tasks/start.yml18:02
SamYapleinc0: nihilifer rhallisey can one of you merge this https://review.openstack.org/#/c/295001/ the gate failure is because of github (look at previous patches for clean gates) this fixes rax-ord issue18:04
g3ekhaplo37, yeah I already see that but this is for wait_for (wait till the IP is mounted). In the same file it said that keepalived is installed... Probably this that mount the IP sorry for disturbing18:05
*** mgoddard_ has quit IRC18:05
SamYapleg3ek: i dont understand "mount the IP" but the haproxy role comes bundled with keepalived and keepalived controls the vip18:06
g3ekSamYample: yeah that was what I was speaking about. English is not my main language , so I just choose the wrong expression ^^18:08
SamYapleg3ek: no problem :) just want to make sure you have the right information18:08
openstackgerritCarlos Cesario proposed openstack/kolla: Added note about nested environment  https://review.openstack.org/29538918:08
*** aginwala has quit IRC18:10
SamYapleinc0: need you to look over https://github.com/ceph/ceph-ansible/blob/etcd-ceph-conf/local.yml#L7-L1218:10
*** mgoddard__ has quit IRC18:11
g3ekwill ceilometer, cinder and barbican  containers will be ready for the mitaka release?18:13
*** sdake has joined #kolla18:13
g3ekll18:13
g3ekand mongodb*18:14
openstackgerritMerged openstack/kolla: Fix rax-ord gate  https://review.openstack.org/29500118:15
SamYapleg3ek: no mitaka is done with new features. although Cinder has been there since Liberty for ceph backends18:15
SamYapleg3ek: newton will have cinder + lvm, and probably ceilometer18:16
g3ekok, we will probably push you the ceilometer version we are working on then18:16
*** aginwala has joined #kolla18:17
inc0SamYaple, I don't get it...it does look horrible18:17
SamYapleinc0: dont get what?18:18
SamYapleinc0: sdakes lab broke because hostname resolved to ipv6, this getent resolves only ipv418:18
inc0SamYaple, you've shown me ceph-ansible code18:19
SamYapleoh im sorry18:19
SamYaplebad paste18:19
SamYaplejpeeler merged it https://review.openstack.org/#/c/295420/18:19
SamYaplesorry for the bad paste inc018:19
inc0fun fact, I was talking about ceph-ansible with Kevin just 5mins ago18:20
inc0yeah my bad, I have everything setup on ipv418:20
SamYapleceph-ansible is a great companion to OSA. both are strong ansible projects18:20
SamYaplei think both are lacking in _what_ they are deploying18:20
SamYaplebut they are strong ansible projects18:20
SamYaplejpeeler out of curiousity, have you had a moment to look at selinux policy things?18:21
jpeelerno :/ though... i might be able to right this second18:22
SamYaplejpeeler: i was curious since sdake insisted it worked fine with selinux. which i saw with my own eyes it did not18:23
SamYapleif you have time it would be greatly apprciated18:23
sdakedidn't work fine18:24
sdakebut it did deploy18:24
sdakenote i always used sudo to run commands18:24
sdakeperhaps that transferred appropriate selinux contexts to the files18:24
*** achanda has quit IRC18:24
SamYaplethats not how that works though18:24
sdakeok18:24
openstackgerritMerged openstack/kolla: Fix problem where ipv6 address causes failure  https://review.openstack.org/29542018:24
SamYaplesince all the code thats broken would haave been run by docker daemon18:24
sdakeok there is one more problem18:24
*** athomas has quit IRC18:25
sdakepython-memcached seems to not be in our centos image18:25
sdakei am not sure if its a master problem or not18:25
sdakebut glance and a few other services don't work18:25
SamYaplesdake: rhallisey https://review.openstack.org/#/c/295048/3/docker/openstack-base/Dockerfile.j218:26
SamYaplecan you add python-memcached here18:26
sdakei am working on it18:26
sdakeoh alrady a review up18:26
sdakecool18:26
rhalliseyya sure18:26
sdakerhallisey put a new review up18:26
sdakeand i'll test it18:26
sdakeif it works, we can cut the branch after that merges18:26
SamYapleim super stoked for mitaka18:26
SamYaplethis is amazing guys18:27
sdakeya mitaka is fantastic18:27
sdakereconfigure and update in particular are gems that nobody else has18:27
sdakeor atleast not in a tidy fashion :)18:27
sdakeupdate/upgrade18:27
SamYaplewell we still need some tidying up too18:27
sdakeya lots of tidy from the bug tracker it apperas18:28
SamYapleelemoine_: ping18:28
sdakeneed heat working need a password on kibana are my two major pain points atm ;)18:28
SamYapleelemoine_: where do we stand on _all_ services logging to heka? mainly libvirt18:28
SamYaplekibana password is eh18:28
openstackgerritRyan Hallisey proposed openstack/kolla: Improve openstack-base image for centos binary  https://review.openstack.org/29504818:28
sdakerhallisey pulling review now and reubilding18:28
rhalliseyk cool18:28
SamYaplethanks rhallisey you da man18:29
rhalliseyI'm also excited for M18:29
SamYapleits just so clean now18:29
SamYapleand there is still a bit of cleaning we can do too which is nice18:29
*** mgoddard has joined #kolla18:30
*** aginwala has quit IRC18:34
*** Marga_ has quit IRC18:34
*** achanda has joined #kolla18:35
*** gfidente has quit IRC18:43
*** mgoddard has quit IRC18:43
sdaketring upgrade with my busted images18:45
*** aginwala has joined #kolla18:49
*** aginwala has quit IRC18:50
sdakeya master not booting with latest centos binary images18:50
sdakesame shit ass with the vm18:50
*** aginwala has joined #kolla18:50
sdakeass/as18:51
SamYaplei cant help you with rdo18:51
SamYapletest source builds please18:51
sdakeunderstood18:51
sdakei am doing that now18:51
SamYaplewe can branch without rdo working, thats not the same as tagging18:51
sdakejust to verify the stuff works18:51
sdakeif rdo is broken that is ok18:51
sdakeagree18:51
sdakeactually i think we need to use a different repo then current-passed-ci18:51
SamYaplewe just want it working for tagged18:52
sdakebecause that isn't the branch repo18:52
SamYaplesdake: you are correct18:52
sdakeright18:52
SamYaplethats what they said in the rdo channel the other day18:52
SamYapletehre is a mitaka branch repo now18:52
SamYaplebut we have to branch for that18:52
SamYapleW00T! my testing branch is now 'noop' after rebase on master18:54
sdakerhallisey please file a bug for that centos binary change18:54
SamYaplethat means my stack of 10 patches has merged!18:54
sdaketrivialfix is off atm18:54
*** aginwala has quit IRC18:56
*** inc0 has quit IRC18:58
openstackgerritSam Yaple proposed openstack/kolla: Fix bad apache2 close  https://review.openstack.org/29545818:59
*** Marga_ has joined #kolla19:00
*** Marga_ has quit IRC19:00
*** Marga_ has joined #kolla19:01
SamYaplesdake: can you deploy with spice please?19:02
SamYaplelooks like it may be busted19:02
SamYaplehttps://bugs.launchpad.net/kolla/+bug/154342919:02
openstackLaunchpad bug 1543429 in kolla "horizon and nova-spicehtml5proxy cannot start" [Critical,In progress] - Assigned to Sam Yaple (s8m)19:02
sdakeSamYaple let me finish building and deploying centos source without spice to make sure centos in general is functional19:03
SamYapleok im testing ubuntu now19:03
SamYapleive been using reconfigure, i like it19:04
SamYaplehey sdake did we say COPY_ONCE was deprecated this cycle?19:04
SamYaplei know we changed the default19:04
sdakeno not deprecated at all19:04
sdakewe are keeping it long term19:04
sdakebut we jsut changed the default19:04
openstackgerritRyan Hallisey proposed openstack/kolla: Improve openstack-base image for centos binary  https://review.openstack.org/29504819:04
sdakepeople wont use our software if it doesn't offer a immutable option19:05
sdaketrust me, i talk to lots of customers about it19:05
sdakewithin the first 3 questions are "is kolla immutable"19:05
SamYaplei dont trust you on that at all. people wont use it _with_ copy_once19:05
SamYapleitll be the odd bird19:05
*** inc0 has joined #kolla19:05
SamYaplebut the default is fine19:05
SamYaplei dont care that it stick arounds19:05
SamYapleitll just be buggy19:05
sdakei think we can fix the bugs19:06
SamYaplesure19:06
SamYaplebut it will _always_ be buggy19:06
sdakei disagree19:06
SamYapleand more downtime involving19:06
sdakei agree19:06
SamYaplethe person that cares more about immunitbility than uptime wont have a job very long19:06
sdaketradeoffs19:06
SamYaplethe thing is, people who really want immutibility want the config baked into the image19:07
SamYaplenow that will probably never happen with kolla19:07
sdakepossibly19:07
sdakebut we can still claim immutability19:07
SamYapleit would really require docker-poxy to make that happen19:07
sdakewith copy_once19:07
SamYaplei agree19:07
SamYaplethe claim can be made with reasonable honesty19:08
sdakeit meets my defintion of immutability which delivers a delcaritve experience19:08
sdakethat is why people want immutable infrastructure19:08
SamYaplemaybe19:09
SamYapleunless that container goes away19:09
SamYapleno way to get it back19:09
SamYaplethat is where teh baked in configs come in19:09
SamYaplebut were saying the same things19:09
*** dave-mccowan has quit IRC19:10
*** vhosakot has joined #kolla19:12
openstackgerritSam Yaple proposed openstack/kolla: Wrong variable name for spicehtml5proxy  https://review.openstack.org/29546219:16
*** spisarski has quit IRC19:24
*** spisarski has joined #kolla19:24
*** inc0 has quit IRC19:27
*** achanda has quit IRC19:28
*** aginwala has joined #kolla19:29
elemoine_SamYaple I basically only ported to Heka the services whose logs were previously collected by Rsyslog. I agree there still is work to do, to collect other logs from other services (libvirt, memcached, etc.)19:30
SamYapleelemoine_: we need at least libvirt for mitaka, is this somethign you can take on before then?19:33
*** mbound has joined #kolla19:35
*** dave-mccowan has joined #kolla19:36
SamYapleneed core reviews for https://review.openstack.org/#/c/293728/19:37
SamYaplejpeeler ? ^19:37
jpeeleri can look at it19:38
vhosakotSamYaple: elemoine_: I'd like implement copying logs form libvirt to heka. I will ask for help. When should this be finished/merged ?19:44
SamYaplevhosakot: as soon as possible19:44
SamYaplesorely needed for mitaka19:45
vhosakotis there a bug for it ? should I open one ?19:45
SamYaplevhosakot: i dont think there is a bug, please do19:46
SamYaplesdake: you agree libvirt logs are critical yes?19:46
SamYaplei feel all logging work is backport potential anyway19:46
vhosakotSamYaple: yes, agreed.. so, the requirement is to collect /var/log/libvirt/libvirtd.log from inside the nova_libvirt container into the heka container.. is this right ?19:50
SamYaplevhosakot: maybe. i dont know whats easier. libvirtd can also log to syslog19:50
elemoine_vhosakot SamYaple I am very busy with other stuff these days, but I'm happy to assist you vhosakot, so feel free to ping me if you need help19:51
SamYapleelemoine_: will this require a custom heka profile?19:51
vhosakotelemoine_: thanks!... as the very first step, I need to know what to collect :)19:51
SamYapleif so.... how to write that?19:51
*** achanda has joined #kolla19:52
sdakeSamYaple absolutely libvirt logs are critical20:03
sdakerhallisey https://review.openstack.org/#/c/293728/20:04
sdakeSamYaple finding the libvirt logs in /var/log/docker is a workaround but its far from ideal20:07
sdake*/var/lib20:08
SamYaplesdake: no horizon logs either20:09
sdakewtb horizon logs20:09
sdakethat was the other thing i saw20:09
SamYaplewe can do both of these20:09
*** inc0 has joined #kolla20:10
inc0o/20:11
SamYaplehey inc020:12
SamYapleremember the logs you were going to do in liberty......20:12
SamYaplecan you do those :D20:12
inc0what do you mean?20:12
SamYaplelibvirt, horizon and others20:12
SamYaplenot loging to central place20:12
inc0heka doesn't deal with it/20:13
inc0I know Eric was going to make it20:13
SamYaplehe said he is too busy20:13
SamYapleso this means we have too20:13
inc0hmm20:14
sdakeinc0 can yu ack this review https://review.openstack.org/#/c/293728/20:16
inc0sdake, one nitpick from Ryan20:17
inc0you want to fix it or want to merge and fix typo later?20:17
SamYapleits docs. i think fix typo after20:18
SamYaplethis is why i dont docs-in-review :D20:18
sdakelets fix it now20:18
sdakeSamYaple you sstill hae the code local?20:18
sdakeyou did the last push20:18
inc0too late20:18
SamYaplewhen do you plan on branching stable sdake20:18
inc0I already +w it:P20:19
SamYapleit should really be like now so we can merge the other stuff20:19
sdakewell i'd like to have a successful deploy20:19
inc0but you can quickly push another patch20:19
sdakeso why dont you give me 30 misn to an hour20:19
sdakecan someone get that doc fixed din the meantime20:19
SamYaplesure. also two patches in the queue, but we can backport20:19
sdakelets get em merged now20:19
sdakelinks please20:19
sdakewe have enoug hcores around to merge things20:19
SamYaplehttps://review.openstack.org/#/c/295458/20:20
SamYaplehttps://review.openstack.org/#/c/295462/20:20
SamYaplealso i bungled a patch for rax-ord gate. but rax-ord gate was broken anyway so eh20:20
sdakenice centos source builds and launches a vm20:21
openstackgerritSam Yaple proposed openstack/kolla: Fix rax-ord commands  https://review.openstack.org/29549520:22
SamYapleawesome20:22
*** dencaval has quit IRC20:24
SiRiuS_sdake, you got virtualbox to work with centos source?20:24
*** Serlex has joined #kolla20:24
openstackgerritMerged openstack/kolla: Add generate_passwords.py to generate passwords  https://review.openstack.org/29372820:25
sdakeSiRiuS_ i cratered my vm by overwriting my /etc/passwd20:25
sdakebut centos binary is dead atm20:25
sdakecentos source works on my bare metal20:26
sdakeso I think the answer wouldl have been yes SiRiuS_ if I hadn't cratered my vm :)20:26
SiRiuS_sdake, :) ok than, you need to do a clean install20:27
vhosakotSamYaple: I don't see any libvirt messages in syslog (/var/log/syslog).. the only libvirt log I see in nova_libvirt container is in /var/log/libvirt/libvirtd.log20:28
*** aginwala has quit IRC20:29
vhosakotSamYaple: sdake: if this (/var/log/libvirt/libvirtd.log) is the log inside nova_libvirt container that needs to be collected into heka, I will start working on it20:29
sdakevhosakot also we need horizon please file bugs for both20:30
vhosakotSamYaple: sdake: https://libvirt.org/logging.html#log_daemon  says that libvirt writes logs to /var/log/libvirt/libvirtd.log20:31
*** mlima_ has quit IRC20:31
vhosakotsdake: ok, I will do one at a time :)  cool, will file bugs for both20:31
sdakevhosakot yes it does20:31
sdakebut heka was never made to collect it20:31
sdakeit should be pretty simple cut and paste20:31
vhosakotsdake: so, I work on hake collect libvirt logs... will ask elemoine_ for heelp20:32
SamYaplevhosakot: awesome!20:32
vhosakotcool...20:32
*** aginwala has joined #kolla20:33
*** haplo37 has quit IRC20:33
openstackgerritSam Yaple proposed openstack/kolla: Enabled spice  https://review.openstack.org/29550420:33
*** spisarski has quit IRC20:35
*** jmccarthy1 has joined #kolla20:37
*** spisarski has joined #kolla20:38
SiRiuS_sdake, try and make use of the clone feature in virtualbox, it will save you a lot of time20:38
SamYaplesdake:20:40
SamYaple[WIP] Deploy multinode in gate20:40
SamYapleThis is very rough but gets the basic idea across for multinode in a20:40
SamYaplesingle node. Some points for reflection:20:40
SamYaple* sysctl breaks in the container due to missing /proc entries for some modules (in this case the bridge module)20:40
SamYaple* --pid=host launches the process in the same pid namespace as the docker daemon resides in, not the true host pid namespace20:40
SamYapleahhhhh stupid paste buffer20:40
SamYaplesorry guys20:40
SamYaplehttps://review.openstack.org/#/c/295504/20:40
SamYaplesdake: ^ fixes spice20:40
SamYapletested20:40
openstackgerritMerged openstack/kolla: Improve openstack-base image for centos binary  https://review.openstack.org/29504820:40
sdakeya i just ran recnfgiure with spice20:40
sdakei'll test yoru patch momoent20:40
SamYaplesdake: reconfig is broken with spice20:41
SamYaplewithout my other patches20:41
SamYaplespice is broken without that patch20:41
sdakeinc0 https://review.openstack.org/#/c/295504/120:43
sdakeSamYaple yes i tested it - i acked it as a result20:43
sdakeonce the stuff in the gate hits the queue and spice i'll branch20:45
sdakethat still leaes alot to backport20:45
sdakebut that is what i tested and that is what i want to branch with20:45
SamYaplei agree20:46
*** jmccarthy1 has quit IRC20:46
SamYaplemore stuff can be backported20:46
openstackgerritMerged openstack/kolla: Wrong variable name for spicehtml5proxy  https://review.openstack.org/29546220:46
SamYaplesdake: https://review.openstack.org/#/c/241500/20:48
sdakecool i thought he pid=host would be a problem point20:49
sdakewhat about memory?20:49
SamYapledangit sdake thats an old patch. i meant https://review.openstack.org/#/c/295495/20:50
SamYaplebad past20:50
SamYaplelook at the review number, its 50,000 reviews old!20:50
ccesarioSamYaple, http://paste.openstack.org/show/491353/    I think that passwords.yml needs to be clean for these options20:50
sdakethis will be our next tag ;) https://github.com/openstack/governance/blob/master/reference/tags/assert_supports-rolling-upgrade.rst20:51
ccesariokolla-genpwd does not replaced these passwords20:51
SamYaplesdake: bad rebase20:51
SamYapleccesario: you want to submit a quick patch for that?20:51
SamYapleill file a bug for you20:51
sdakethis one first tho https://github.com/openstack/governance/blob/master/reference/tags/assert_supports-upgrade.rst20:52
sdakeSamYaple can you fix please20:52
sdakeuse same bug id as genpwd20:53
sdakei dont wnat to branch a busted tree20:53
sdakebut we are close to branching any moment now ;)20:53
SamYapleccesario: will you be submitting that patch, or i?20:54
*** jmccarthy1 has joined #kolla20:56
ccesarioSamYaple, ok20:56
ccesariolet me try :D20:56
*** mbound has quit IRC20:57
*** jmccarthy1 has quit IRC20:57
openstackgerritCarlos Cesario proposed openstack/kolla: Remove static password  https://review.openstack.org/29551321:01
ccesarioSamYaple, ^^21:02
SamYapleccesario: responded. quick change and +221:04
*** aginwala has quit IRC21:09
*** dwalsh has quit IRC21:09
*** jtriley has quit IRC21:11
vhosakotccesario: ping21:12
openstackgerritCarlos Cesario proposed openstack/kolla: Remove static password  https://review.openstack.org/29552221:14
ccesariovhosakot, pong21:15
vhosakotccesario: do you see any error if vity_type=qemu is not set in /etc/kolla/config/nova-compute.conf... ?21:15
SamYapleccesario: you removed th change id and it created a new patch, so youll need to abandon the old one21:15
ccesariovhosakot, in my case, I 'm running nested env... if I did not set  it, the nova-compute services does not UP21:16
ccesarioSamYaple, only click in "abandon" button ?21:17
vhosakotccesario: ah ok, so "kolla-ansible deploy" fails or "creating a nova VM using nova CLI" fails ?21:18
SamYapleyes ccesario21:18
ccesariovhosakot, after put it, the service it works as expected (I don't remember the error generated in log files.... if is this that you need)21:18
vhosakotccesario: cool, the service fails...ok21:19
vhosakotccesario: just wanted to know what happened.. thanks for the info21:19
ccesariovhosakot, the container is deployed, but nova-compute logs show me errors21:19
ccesariovhosakot, ;)21:19
vhosakotccesario: ok, did you run init_runonce ?21:20
ccesariovhosakot, really I dont remember man.....  but I can try reproduce it again (tomorrow)21:21
vhosakotcool, np... :)21:21
ccesarioI need go to my home now... :) end shift21:22
ccesariothank you so much guys by help :)21:22
vhosakotccesario: bye and thanks!21:22
ccesariovhosakot, bye!21:23
*** aginwala has joined #kolla21:25
SamYaplesdake 3 bugs left to merge before branch21:27
*** ccesario_ has joined #kolla21:27
*** banix has quit IRC21:27
sdakehttps://review.openstack.org/29552921:31
sdakei wish i had an end of shift ;)21:32
sdakeneed ack on https://review.openstack.org/#/c/295504/21:33
*** inc0 has quit IRC21:34
sdakeinc0 would you reconsider on https://review.openstack.org/#/c/295458/21:34
*** achanda has quit IRC21:35
*** ccesario_ has quit IRC21:38
openstackgerritMerged openstack/kolla: Enabled spice  https://review.openstack.org/29550421:39
*** ccesario_ has joined #kolla21:39
SamYaplesdake: https://review.openstack.org/#/c/295495/21:39
*** sdake_ has joined #kolla21:39
*** sdake has quit IRC21:41
jpeelerSamYaple: after upgrading to the latest selinux-policy, i haven't seen any problems yet. are there any outstanding bugs for selinux things not working?21:44
jpeeleri am using some outdates images, so i may just rebuild everything tonight21:45
*** aginwala has quit IRC21:45
SamYapleohh nice. so latest selinux policy makes this work?21:46
SamYapledo we have someone from the channel to thank for that/21:46
*** achanda has joined #kolla21:47
jpeeleri haven't stressed it too much. but i was running an ancient policy and saw problems. redeployed after upgrading and things look fine21:47
*** ccesario_ has quit IRC21:48
*** ccesario_ has joined #kolla21:48
rhalliseyjpeeler, are you using openstack-selinux?21:49
SamYapleayoung: ^ selinux stuff *may* be working now?21:49
SamYaplemind repeating your tests and looking for breakage?21:49
rhalliseytbh I think there will be more AVCs popping up as people do specific deployments21:49
jpeelerrhallisey: i don't even see such a package21:49
sdake_it wfm too - but nova was flkaey21:50
sdake_but i was using rdo binary which is busted atm21:50
sdake_it does not deploy vms properly21:50
rhalliseyjpeeler, it's in rdo21:50
sdake_it locks up on nova boot21:50
rhalliseyopenstack-selinux has specific selinux policy that's not in base policy21:51
rhalliseyI run that package so if we can get some AVCs I'll add the policy in there21:51
jpeeleri don't know that much about selinux, but i thought we were looking for selinux problems on the host21:52
*** aginwala has joined #kolla21:52
jpeelerand if you wanted to test openstack policy, you'd have to bake that package into image building right?21:53
openstackgerritMerged openstack/kolla: Remove static password  https://review.openstack.org/29552221:53
SamYaple2 more patches need merging21:54
sdake_sam is like a clock ;-)21:55
sdake_counting down to d day21:55
*** ccesario_ has quit IRC21:55
SamYapleseriously want to merge21:55
rhalliseywe are looking on the host, but openstack-selinux might have some policy to be of use21:55
SamYaplegot some -2's to remove21:55
ayoungSamYaple, oh really?  Cool21:55
*** ccesario_ has joined #kolla21:56
rhalliseykolla might be good to host it's own policy21:56
SamYapleayoung: it seems that way. you be the judge21:56
ayoungSamYaple, I'll give it a try soon..the queue is getting long, though21:56
rhalliseyI was thinking of building it into rdo21:56
ayoungSamYaple,  When I read this, I read "replace the overcloud with Kolla"   http://lists.openstack.org/pipermail/openstack-dev/2016-March/090055.html21:57
sdake_SamYaple believes jeff but not sdake ;)21:57
* sdake_ giggles21:57
ayoungsdake_, is it that, or is it like the Adama rule from Galactica, that he has to be told twice21:57
rhalliseyayoung, also read it the same :)21:58
sdake_ayoung sometimes things need repeating :)21:58
SamYaplesdake_: to be fair, most of the time we do this back and forth you come back with "oh this was working different than i thought"21:58
SamYapleayoung: thats the plan!21:59
SamYapleayoung: rhallisey should know21:59
sdake_yes to be fair i think out loud21:59
sdake_on irc21:59
sdake_and my thinking via typing is slower then my brain :)21:59
SamYapleagreed22:00
*** sdake_ is now known as sdake22:02
sdakewtb merging of the remaining patches!!22:06
sdakecomon cores pull the trigger lets move on22:06
*** aginwala has quit IRC22:07
*** ccesario_ has quit IRC22:07
SamYaplesdake: waiting on you for one of them....22:08
SamYaplehttps://review.openstack.org/#/c/295495/22:09
SamYaplemichal is gone, but could some one ack https://review.openstack.org/#/c/295458/22:09
SamYaplejpeeler: maybe?22:09
SamYaplelast patch people22:09
SamYaplethen we branch22:09
*** aginwala has joined #kolla22:10
jpeeler+222:12
*** sdake_ has joined #kolla22:12
SamYaplejpeeler: us south carolinians have to stick together22:14
SamYaplesdake: we are set to go when these to patches merge22:14
jpeeler:)22:14
*** sdake has quit IRC22:15
openstackgerritMerged openstack/kolla: Fix rax-ord commands  https://review.openstack.org/29549522:16
openstackgerritMerged openstack/kolla: Fix bad apache2 close  https://review.openstack.org/29545822:16
*** achanda has quit IRC22:18
SamYaplebranch sdake_ branch!22:18
*** aginwala has quit IRC22:20
*** aginwala has joined #kolla22:22
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505222:23
SamYaplewoot!22:23
sdake_this needs merging22:25
sdake_https://review.openstack.org/#/c/295546/22:25
sdake_not sure why the irc bot didn't poke it out22:25
sdake_SamYaple you will have to cherrypick that to the stable/liberty branch22:25
*** shardy has quit IRC22:25
SamYaplei dont follow?22:26
SerlexAnyone having problems 'keystone | Creating Keystone database] ' stage?22:29
Serlex"UndefinedError: 'dict object' has no attribute 'stdout'" not sure what to do with that22:29
SamYaplesdake_: how old is your keystone container?22:30
SamYapleSerlex: *22:30
sdake_its about 1 hour old22:31
sdake_both centos source and binary22:31
sdake_binary is currently doa22:31
sdake_this should fix it https://review.openstack.org/29554922:31
*** spisarski has quit IRC22:32
SamYaplesdake_: i cant seem to submit to stable/mitaka22:33
sdake_yes you need to merge my patch above22:33
sdake_https://review.openstack.org/#/c/295546/22:33
SamYapleoh i need to rebase onto it22:33
SamYapleive already +2's22:33
sdake_that will work too22:33
sdake_jpeeler can you ack that pleae22:34
sdake_its super obviously needed ;)22:34
*** bmace has quit IRC22:34
jpeelerdone and then i'm gone22:35
sdake_thanks jeff :)22:35
SamYaplejpeeler: you did go today22:35
sdake_have a nice evening22:35
SamYaplego home soldier22:35
*** pierre_ has joined #kolla22:36
pierre_Hi, I am having an error using kolla-ansible deploy \22:37
pierre_TASK: [keystone | Creating admin project, user, role, service, and endpoint]22:37
pierre_ValueError: Expecting , delimiter: line 1 column 268 (char 267)22:37
pierre_I am using ubuntu base and source22:38
SamYaplepierre_: when did you build these images?22:43
sdake_ok guys the irc bot changes shoudl merge shortly22:44
sdake_groan 58 bugs in rc222:46
pierre_I built the images on Friday22:46
pierre_or this morning maybe22:46
Serlexok tahnks sdake and SamYaple22:47
pierre_should I rebuilt from source?22:47
Serlexthanks*22:47
Serlexsdake_22:47
sdake_Serlex sup22:47
SamYaplepierre_: we _just_ branched stable/mitaka moments ago22:48
SamYaplepierre_: i suggest you rebuild yes22:48
Serlexsorry I was thanking for the reply to my question22:48
Serlexunless I got it completely wrong22:48
pierre_Thanks I will give it another try22:49
Serlex(10:29:29 PM) Serlex: "UndefinedError: 'dict object' has no attribute 'stdout'" not sure what to do with that22:49
Serlex(10:30:30 PM) SamYaple: sdake_: how old is your keystone container?22:49
Serlex(10:30:33 PM) SamYaple: Serlex: *22:49
Serlexis this a known bug?22:50
openstackgerritMerged openstack/kolla-mesos: Adjust profiles to already implemented services  https://review.openstack.org/29522222:50
SamYapleSerlex: this is a know issue for old/bad containers22:50
sdake_Serlex i just deployed master about 1-2 hours ago it worked for me22:50
openstackgerritMerged openstack/kolla-mesos: Fix the service conf file path  https://review.openstack.org/29522322:50
Serlexbasically go for fresh setup?22:51
* SamYaple sighs22:51
SamYaplerax gates are failing with "msg: Hostname has to resolve to IP address of api_interface"22:51
sdake_we dont guarantee master to master upgrades work22:51
SamYaplewhich makes sense because rax screw with thier hostnames22:51
sdake_we may in teh future but dont at present22:51
sdake_the problem is our api changes often unfortunately22:52
sdake_we guarantee 2.0 will upgrade to 3.0 at present22:52
*** Serlex has quit IRC22:52
sdake_and if the liberty bcakport finishes we guarantee 1.0 will upgrade to 2.022:52
sdake_we will also guarantee z streams will upgrade in their respective x.y streams22:52
openstackgerritMerged openstack/kolla-mesos: Allow to use "run" method only if the runner is enabled  https://review.openstack.org/29526722:54
sdake_thanks sam for prodding me to cut a branch - i think that was the right move22:54
openstackgerritMerged openstack/kolla-mesos: Fix path to ml2_conf.ini files  https://review.openstack.org/29526822:54
sdake_not ekolla-mesos has not branched yet22:54
sdake_waiting on asalkeld to do that work22:54
asalkeldsdake_: what's the timeline for that?22:55
sdake_whenever you like22:55
sdake_sooner is better - but itss up to how you wantto handle backports22:55
sdake_i am handling backports for stable/mitaka of kolla22:55
*** aginwala has quit IRC22:58
openstackgerritAngus Salkeld proposed openstack/kolla: Disable deprecated heat plugins.  https://review.openstack.org/29507822:58
asalkeldok22:59
*** vhosakot has quit IRC23:01
*** openstackgerrit has quit IRC23:03
*** openstackgerrit_ is now known as openstackgerrit23:03
*** openstackgerrit has quit IRC23:03
*** openstackgerrit_ has joined #kolla23:03
*** openstackgerrit_ is now known as openstackgerrit23:04
*** openstackgerrit_ has joined #kolla23:08
*** dims_ has quit IRC23:09
*** dims has joined #kolla23:09
sdake_sam remove -2 from this please https://review.openstack.org/#/c/291285/23:09
sdake_SamYaple ^^23:09
*** dwalsh has joined #kolla23:15
*** vhosakot has joined #kolla23:17
sbezverksdake could you give me the link to horizon bug please?23:19
*** achanda has joined #kolla23:20
*** dims has quit IRC23:21
*** dims has joined #kolla23:22
*** sdake has joined #kolla23:25
*** jtriley has joined #kolla23:25
*** sdake_ has quit IRC23:26
*** achanda has quit IRC23:27
*** dims has quit IRC23:27
*** dims has joined #kolla23:28
openstackgerritSerguei Bezverkhi proposed openstack/kolla: Changes horizon log location to kolla_logs volume  https://review.openstack.org/29556823:33
*** dims has quit IRC23:33
*** alisonh has quit IRC23:35
*** pierre_ has quit IRC23:35
openstackgerritAngus Salkeld proposed openstack/kolla-mesos: Fix auto doc generation  https://review.openstack.org/29507423:38
*** jtriley has quit IRC23:40
*** aginwala has joined #kolla23:46
*** achanda has joined #kolla23:49
*** alisonh has joined #kolla23:53
SamYaplesdake: im not making a bug for the normal branching things23:54
SamYapleyou didnt make a but for the gitreview23:54
SamYaplebug*23:54
SamYaplethere is a reason for that23:54
*** aginwala has quit IRC23:54
SamYaplethats the overhead im not ok with23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!