Tuesday, 2016-10-18

*** hogepodge has quit IRC00:02
*** weshay_bbiab is now known as weshay00:05
*** hogepodge has joined #tripleo00:08
*** david-lyle has quit IRC00:09
*** hogepodge has quit IRC00:20
*** hogepodge has joined #tripleo00:21
*** david-lyle has joined #tripleo00:22
*** bana_k has quit IRC00:23
openstackgerritEmilien Macchi proposed openstack/tripleo-common: Refresh DeployIdentifier as well on update  https://review.openstack.org/38753100:26
rbradyEmilienM: it looks like it's hitting the nova config bug we saw in master00:27
EmilienMrbrady: which one?00:28
EmilienMI'm investigating why tripleo multinode jobs are broken00:28
rbradythe *workers values set to 000:28
*** yolanda has quit IRC00:28
EmilienMI see something weird, our stable/newton jobs download packages from master00:29
EmilienMdid I drink something wrong?00:29
EmilienMrbrady: what patch fixed it? let me find it again00:29
*** yolanda has joined #tripleo00:29
EmilienMI thought dprince would backport it00:29
*** david-lyle has quit IRC00:29
EmilienMrbrady: where do you see it in logs?00:29
EmilienMrbrady: how can you tell it's nova workers?00:30
*** david-lyle has joined #tripleo00:31
rbradyEmilien: I'm guessing.  This is the exact same error I saw from stack create in my local env00:31
EmilienMrbrady: ok. Do you have apache logs in your env?00:31
rbradyno, I upgraded after dan's patch00:32
rbradyemilienm: the exact error was nova-api failing to start00:32
EmilienMlet me try to backport this stuff, and probably squash it...00:33
*** ctayal has quit IRC00:33
*** david-lyle has quit IRC00:34
*** david-lyle has joined #tripleo00:36
*** limao has joined #tripleo00:36
EmilienMrbrady: thx btw00:40
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: [newton] Only set NovaWorkers in the non-default case  https://review.openstack.org/38773700:41
*** maticue has quit IRC00:42
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: [newton] Only set NovaWorkers in the non-default case  https://review.openstack.org/38773700:46
*** david-lyle has quit IRC00:47
*** ctayal has joined #tripleo00:47
EmilienMi'm really unsure it will help ^ I see in puppet catalog httpd error first00:48
EmilienMand nova-api is not deploying with httpd in newton00:48
*** david-lyle has joined #tripleo00:48
*** gfidente has quit IRC00:50
*** david-lyle has quit IRC00:51
*** david-lyle has joined #tripleo00:52
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Enable object versioning in Swift proxy  https://review.openstack.org/38633000:54
*** david-lyle has quit IRC00:59
*** ctayal has quit IRC00:59
*** ctayal has joined #tripleo01:00
*** david-lyle has joined #tripleo01:01
*** lifeless has quit IRC01:02
*** lifeless has joined #tripleo01:02
*** ctayal has quit IRC01:03
*** xuao has joined #tripleo01:04
*** david-lyle has quit IRC01:05
*** david-lyle has joined #tripleo01:06
*** rhallisey has quit IRC01:07
*** david-lyle has quit IRC01:07
*** bana_k has joined #tripleo01:10
*** yamahata has quit IRC01:12
*** david-lyle has joined #tripleo01:13
EmilienMI reported https://bugs.launchpad.net/tripleo/+bug/163431901:13
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]01:13
*** david-lyle has quit IRC01:14
*** david-lyle has joined #tripleo01:15
*** hogepodge has quit IRC01:16
*** yolanda has quit IRC01:17
*** david-lyle has quit IRC01:19
*** bana_k has quit IRC01:19
*** david-lyle has joined #tripleo01:20
*** david-lyle has quit IRC01:21
*** dmacpher has quit IRC01:22
*** david-lyle has joined #tripleo01:23
*** yolanda has joined #tripleo01:23
*** david-lyle has quit IRC01:24
*** david-lyle has joined #tripleo01:26
*** yamahata has joined #tripleo01:30
*** fzdarsky_ has joined #tripleo01:31
*** david-lyle has quit IRC01:32
*** david-lyle has joined #tripleo01:34
*** fzdarsky has quit IRC01:35
*** hogepodge has joined #tripleo01:35
*** david-lyle has quit IRC01:37
*** david-lyle has joined #tripleo01:38
*** NobodyCam has quit IRC01:42
*** NobodyCam has joined #tripleo01:42
*** gregwork has quit IRC01:42
*** gregwork has joined #tripleo01:43
*** david-lyle has joined #tripleo01:44
*** david-lyle has quit IRC01:44
*** david-lyle has joined #tripleo01:45
*** david-lyle has quit IRC01:46
*** david-lyle has joined #tripleo01:57
*** david-lyle has quit IRC01:57
*** david-lyle has joined #tripleo01:59
*** david-lyle has quit IRC02:00
*** tzumainn has quit IRC02:00
*** david-lyle has joined #tripleo02:01
*** david-lyle has quit IRC02:02
*** david-lyle has joined #tripleo02:03
*** david-lyle has quit IRC02:03
*** dmacpher has joined #tripleo02:04
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775002:06
*** david-lyle has joined #tripleo02:07
slagleEmilienM: maybe that ^02:07
slaglenot sure though02:07
*** david-lyle has quit IRC02:08
*** ooolpbot has joined #tripleo02:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]02:10
*** david-lyle has joined #tripleo02:10
*** david-lyle has quit IRC02:11
*** david-lyle has joined #tripleo02:12
*** david-lyle has quit IRC02:14
*** david-lyle has joined #tripleo02:17
*** david-lyle has joined #tripleo02:19
*** david-lyle has quit IRC02:19
*** david-lyle has joined #tripleo02:21
*** david-lyle has quit IRC02:21
*** david-lyle has joined #tripleo02:26
openstackgerritJohn Fulton proposed openstack/tripleo-specs: Integrate TripleO with Tendrl for External Storage Deployment/Management  https://review.openstack.org/38763102:29
*** tiswanso has joined #tripleo02:30
*** limao has quit IRC02:34
*** limao has joined #tripleo02:35
*** bana_k has joined #tripleo02:37
*** hogepodge has quit IRC02:40
*** ctayal has joined #tripleo02:47
*** ctayal has quit IRC02:47
*** ctayal has joined #tripleo02:48
*** thrash is now known as thrash|g0ne02:48
openstackbugzilla.redhat.com bug 1385928 in openstack-tripleo "Last poodle breaks the node importation mechanism" [Unspecified,New] - Assigned to jslagle02:53
*** ctayal has quit IRC02:53
openstackLaunchpad bug 1634202 in tripleo "Failure to import new nodes using RDO Newton on RHEL" [Medium,New] - Assigned to wes hayutin (weshayutin)02:53
dhill_I've attached a patch that solves that issue02:53
*** bana_k has quit IRC02:57
*** cwolferh has quit IRC03:02
*** hogepodge has joined #tripleo03:04
*** ooolpbot has joined #tripleo03:10
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]03:10
*** ooolpbot has quit IRC03:10
*** fultonj has quit IRC03:18
*** coolsvap has quit IRC03:20
*** david-lyle has quit IRC03:23
*** david-lyle has joined #tripleo03:24
*** david-lyle has quit IRC03:25
*** david-lyle has joined #tripleo03:26
*** david-lyle has quit IRC03:27
*** david-lyle has joined #tripleo03:28
*** david-lyle has quit IRC03:29
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore  https://review.openstack.org/36544903:30
*** david-lyle has joined #tripleo03:30
*** david-lyle has quit IRC03:33
*** david-lyle has joined #tripleo03:34
*** tiswanso has quit IRC03:35
*** links has joined #tripleo03:35
*** rbrady is now known as rbrady-afk03:35
*** david-lyle has quit IRC03:36
*** david-lyle has joined #tripleo03:50
*** david-lyle has quit IRC03:51
*** david-lyle has joined #tripleo03:53
*** david-lyle has quit IRC03:54
*** david-lyle has joined #tripleo03:57
*** david-lyle has quit IRC03:57
*** yamahata has quit IRC03:58
*** david-lyle has joined #tripleo04:00
*** david-lyle has quit IRC04:01
*** david-lyle has joined #tripleo04:02
*** newmember has joined #tripleo04:02
*** david-lyle has quit IRC04:04
*** david-lyle has joined #tripleo04:05
*** david-lyle has quit IRC04:05
*** david-lyle has joined #tripleo04:06
*** david-lyle has quit IRC04:08
*** bana_k has joined #tripleo04:09
*** ooolpbot has joined #tripleo04:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]04:10
*** david-lyle has joined #tripleo04:10
*** david-lyle has joined #tripleo04:12
*** david-lyle has quit IRC04:13
*** david-lyle has joined #tripleo04:14
*** david-lyle has quit IRC04:15
*** david-lyle has joined #tripleo04:17
*** david-lyle has quit IRC04:18
*** david-lyle has joined #tripleo04:19
*** david-lyle has quit IRC04:20
*** david-lyle has joined #tripleo04:21
*** bana_k has quit IRC04:27
*** spredzy has quit IRC04:33
*** spredzy_ has joined #tripleo04:33
*** spredzy_ is now known as spredzy04:33
*** numans has joined #tripleo04:36
*** rwsu has joined #tripleo04:56
*** ooolpbot has joined #tripleo05:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]05:10
*** jaosorior has joined #tripleo05:18
*** saneax-_-|AFK is now known as saneax05:26
*** penick has joined #tripleo05:26
*** bana_k has joined #tripleo05:30
*** ctayal has joined #tripleo05:32
*** sudipto has joined #tripleo05:39
*** sudipto_ has joined #tripleo05:39
*** newmember has quit IRC05:46
*** chlong has quit IRC05:47
jaosoriordtrainor: still around?05:48
dtrainorsure am, for a few more minutes.  what can i do for ya?05:48
jaosoriorhey dude, I am about to test out your puppet-tripleo change05:49
jaosoriorand I rechecked your commit https://review.openstack.org/#/c/386817/205:49
jaosoriorthere is some puppet-lint stuff that needs to be fixed (easy fixes), but have one question. Why port 3000 for the UI?05:49
dtrainorah i see the updates.  thanks.05:49
*** skramaja has quit IRC05:51
dtrainorback in the day the ui was ran out of node, and that was the default port for .... express I think?05:51
dtrainorso, no good reason, really05:52
jaosoriorand, do we expect someone to try to deploy horizon on the undercloud?05:52
*** tremble has joined #tripleo05:52
dtrainori would not expect that since the undercloud system is not availble to assign a role to05:52
jaosoriorso it's all good05:53
jaosoriordtrainor: but, just one thing to be considered05:53
jaosoriorif we are not expecting to deploy horizon05:53
dtrainori mean we'd expect to deploy horizon on the undercloud05:53
jaosoriormaybe it would be a good thing to consider, switching to port 80 (even if we hope to always redirect users to 443)05:53
dtrainori'm ok with that.  seeing as haproxy terminates all the ssl, and proxies to a non-ssl apache, that would be appropriate.05:54
jaosoriordtrainor: ah, but if horizon is deployed in the undercloud, wouldn't it have an issue with the SSL ports? Since both horizon and the tripleo-ui would use 44305:54
*** lmiccini has joined #tripleo05:54
dtrainori wouldn't expect horizon to be deployed on the *undercloud*, but I would expect it to be deployed on the *overcloud*05:55
jaosoriorok, that make sense and shouldn't be an issue05:55
dtrainorthanks for the comments, i'll be sure to fix them first thing in the A.M.05:55
dtrainorright now i'm trying to get a local Satellite system up and running so i can provision in home labs more quickly05:56
jaosoriordtrainor: sorry for the late comments, just noticed the puppet-lint gate failing today.05:56
dtrainorno that's great, i appreciate the feedback05:56
*** skramaja has joined #tripleo05:57
*** ctayal has quit IRC05:59
*** rcernin has joined #tripleo06:01
*** chlong has joined #tripleo06:01
*** links has quit IRC06:01
*** ccamacho has quit IRC06:04
*** bana_k has quit IRC06:09
*** ooolpbot has joined #tripleo06:10
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]06:10
*** ooolpbot has quit IRC06:10
*** jtomasek_ has joined #tripleo06:13
*** jprovazn has joined #tripleo06:17
openstackgerritDan Trainor proposed openstack/puppet-tripleo: Enable communication between UI and the Undercloud by making HAProxy proxy for the UI  https://review.openstack.org/38681706:18
*** pcaruana has joined #tripleo06:18
jaosoriordtrainor: do you have already the commit in instack-undercloud that depends on this one?06:20
*** sudipto_ has quit IRC06:21
*** sudipto has quit IRC06:21
dtrainori do not06:21
*** dsneddo__ has quit IRC06:21
dtrainori'll take a look at it06:21
jaosoriordtrainor: well, if all you need from the undercloud is to enable this, I can do that one.06:22
dtrainori'd liek to take a stab at it for the experience if you don't mind06:23
jaosoriordtrainor: no biggie06:24
jaosoriorgo ahead06:24
*** penick has quit IRC06:24
*** chandankumar has joined #tripleo06:24
*** rasca has joined #tripleo06:25
dtrainori think we're just talking about defaulting undercloud_service_certificate to True06:26
*** hogepodge has quit IRC06:30
jaosoriordtrainor: so, we need make enable_ui true when the UI is enabled in the undercloud. There is a section in the yaml file that adds the hiera where we enable each service for the undercloud where that needs to be06:31
dtrainorright, enable_ui is True by default06:31
jaosoriordtrainor: but not for haproxy06:31
dtrainori see it.06:32
jaosoriordtrainor: https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L53806:32
*** jtomasek_ has quit IRC06:32
jaosoriord0ugal: are you around?06:32
dtrainorit's in there, under the stable/newton branch06:33
jaosoriorah, I see the enable_ui parameter now06:34
jaosoriorand it matches haproxy.pp06:34
jaosoriormy bad06:34
dtrainorno worries, didn't think to double-check since i've seenit there once06:35
jaosoriorinteresting... so why didn't it enable it in my undercloud06:35
jaosoriorah, I think my undercloud is too old now06:35
jaosoriorgotta update it06:36
dtrainori believe the only thing left is to default generate_service_certificate to True06:37
dtrainoror not, actually.  tell you what, i'm going to get some sleep, i'll be back in the morning06:38
jaosoriordtrainor: have a good one dude, talk to you later06:39
dtrainorthanks for the help and guidance06:39
jaosoriorglad to help06:40
*** sudipto has joined #tripleo06:42
*** sudipto_ has joined #tripleo06:42
*** zaneb has quit IRC06:49
*** zaneb has joined #tripleo06:50
*** waleedm has joined #tripleo06:54
waleedmHi, all Does any one know when i can get python-tripleoclient-5.2.0-1.el7ost.noarch package !for RDO906:56
*** yamahata has joined #tripleo07:02
*** tesseract has joined #tripleo07:03
*** tesseract is now known as Guest8585507:03
*** coolsvap has joined #tripleo07:04
*** ccamacho has joined #tripleo07:05
*** shardy has joined #tripleo07:08
apetrichd0ugal, who else was hitting the badstatusline bug? Do you know if they have bz for it?07:09
*** ooolpbot has joined #tripleo07:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]07:10
jaosoriorapetrich: a bad status line bug coming from heat?07:10
d0ugalapetrich, jaosorior: https://bugs.launchpad.net/tripleo/+bug/163419507:10
openstackLaunchpad bug 1634195 in tripleo "Creating the default plan can fail when communicating with swift. It should be retried" [Medium,Confirmed]07:10
jaosoriorhaven't seen that one07:10
apetrichd0ugal, brilliant thank you07:11
jaosoriord0ugal: currently seeing this though http://paste.openstack.org/show/586113/07:11
*** ebarrera has joined #tripleo07:11
apetrichjaosorior, I hit that during the mixed deployment yesterday07:11
d0ugalapetrich: so weshay hit it and I also seen it mentioned in #rdo, but I forget who that was. trown|outtypewww was in the discussion tho'07:11
d0ugaljaosorior: looking.07:11
*** cylopez has joined #tripleo07:11
*** dmacpher has quit IRC07:12
d0ugaljaosorior: You need: https://review.openstack.org/#/c/386558/07:12
d0ugaljaosorior: sorry, I missed your ping earlier.07:12
* d0ugal isn't sure how07:12
jaosoriord0ugal: no biggie07:12
d0ugaljaosorior: LOL I wrote "rebase" rather than "recheck"07:13
d0ugalon that review07:13
* d0ugal sighs07:13
jaosoriord0ugal: testing07:15
d0ugaljaosorior: thanks.07:15
*** amoralej|off is now known as amoralej07:16
*** aufi has joined #tripleo07:21
*** jlinkes has joined #tripleo07:21
*** shardy has quit IRC07:22
jaosoriorccamacho: hey dude, did the service restart do the trick?07:22
jaosoriord0ugal: your fix worked for me, thanks dude!07:23
*** hjensas has joined #tripleo07:24
*** hjensas has quit IRC07:24
*** hjensas has joined #tripleo07:24
*** milan-afk is now known as milan07:24
d0ugaljaosorior: thanks07:24
*** zoli_gone-proxy is now known as zoliXXL07:24
*** nyechiel has joined #tripleo07:25
ccamachohey jaosorior yeahp It worked but in +-1hour started to crash again.. The issue was that my undercloud was without resources, so I applied this https://review.openstack.org/#/c/386696/ and it worked. Not sure why... but last days have been kinda cumbersome dude and today I have some flu...07:27
zoliXXLgood morning07:27
openstackgerritDougal Matthews proposed openstack/instack-undercloud: Only create the Mistral config env if it doesn't exist  https://review.openstack.org/38783207:28
jaosoriorccamacho: I always set up the number of workers for my undercloud. it's been a while since I've been having issues with the number of workers there07:28
jaosoriord0ugal: was that rebase a clean rebase? (missed the -x flag in the cherry-pick)07:29
jaosoriorccamacho: don't worry about it, hope you get better.07:29
d0ugaljaosorior: gah, gerrit doesn't add the -x unless it has merged already. I guess I'll do it manually or wait for the master change to land07:29
*** ohamada has joined #tripleo07:30
ccamachoYesterday night I applied this one (https://review.openstack.org/#/c/377650/7) but was really tired, so good so far, Ill give you some feedback :) I enabled the flag here tripleo-heat-templates/environments/services/internal-tls-certmonger.yaml  with the puppet-tripleo changes from here https://review.openstack.org/#/c/37764507:30
ccamachojaosorior, but in any case I think they work fine :) I just want to check how gnocchi is working07:31
openstackgerritDougal Matthews proposed openstack/instack-undercloud: Only create the Mistral config env if it doesn't exist  https://review.openstack.org/38783207:31
d0ugaljaosorior: ^ fixed07:32
jaosoriorccamacho: for all the changes, remember these two are needed: t-h-t https://review.openstack.org/#/c/366075/  puppet-tripleo  https://review.openstack.org/#/c/327029/07:32
*** b00tcat has joined #tripleo07:33
ccamachojaosorior, if the services are not enabled using TLS they shouldn't affect upgrades, right? In the case of an upgrade, should we upgrade like M-non-TLS -> O-non-TLS (And then enable TLS?)07:33
jaosoriorccamacho: if you have any issues with the gnocchi patch, let me know. there was an issue cause CentOS hasn't upgraded the openstack-selinux rules, but that fix is in RHEL now, and should be available in CentOS pretty soon.07:33
jaosoriorccamacho: yeah, pretty much.07:33
*** mcornea has joined #tripleo07:33
jaosoriorccamacho: still gotta figure out the issue in HA that I told you about (HAProxy not reloading). But for a new deployment it should be fine.07:34
ccamachojaosorior nice07:34
*** nyechiel has quit IRC07:38
*** jpich has joined #tripleo07:39
*** hogepodge has joined #tripleo07:40
*** jpena|off is now known as jpena07:45
*** mrunge has quit IRC07:50
*** masco has joined #tripleo07:50
*** mrunge has joined #tripleo07:53
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo: Fix broken rabbitmqctl commands when using ipv6  https://review.openstack.org/38692107:56
jpichjtomasek: Yay, I see you fully tackled that processor count issue! Thank you! :)08:01
jtomasekjpich: :)08:02
*** openstackgerrit has quit IRC08:04
*** openstackgerrit has joined #tripleo08:04
jistrjaosorior: hey btw i think we didn't need to recheck this one on the N-1 attempt, as all the voting jobs were green :) https://review.openstack.org/#/c/386669/ i'll +A when i get at least the multinode jobs green via a recheck08:08
jaosoriorjistr: sure08:09
jaosoriorjistr: wanted to see the result of the upgrades gate08:10
jaosoriorjistr: I was sure it should be passing with that commit.08:10
jaosoriorjistr: but anyway, +A when multinode jobs pass is cool with me08:10
*** ooolpbot has joined #tripleo08:11
*** ooolpbot has quit IRC08:11
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]08:11
jistrjaosorior: ah ok. We'll see if it's passing this time, last time the upgrade job failed on http://mirror.centos.org/centos/7/cloud/x86_64/openstack-newton/repodata/repomd.xml: [Errno 14] curl#52 - "Empty reply from server"08:12
jistrso it didn't get far...08:12
jaosoriorjistr: exactly. I'm pretty sure it should be passing08:13
jaosoriorjistr: anyway, don't worry about it. it's cool to +A that one.08:13
jistrhmm is the upgrades job passing now usually? (it's not tracked not in cistatus.html)08:14
jaosoriorjistr: can you check this out https://review.openstack.org/#/c/387189/ ? Seems that the inline migration logic was faulty08:14
jaosoriorjistr: I've seen it passing several times, but it's flaky. Which is why I wanted to check it out for that commit.08:15
*** paramite has joined #tripleo08:16
*** owalsh has joined #tripleo08:17
*** phpcodemonkey has joined #tripleo08:24
*** yamahata has quit IRC08:24
*** jbadiapa_ has quit IRC08:26
openstackgerritFlavio Percoco proposed openstack/tripleo-heat-templates: Don't set nova::wsgi::apache::workers when it is 0  https://review.openstack.org/38730908:27
*** zoliXXL is now known as zoli|brb08:30
*** tobias_fiberdata has joined #tripleo08:38
*** electrofelix has joined #tripleo08:42
openstackgerritFlavio Percoco proposed openstack/tripleo-heat-templates: Re-add undercloud.yaml  https://review.openstack.org/35203708:44
*** athomas has joined #tripleo08:48
*** akrivoka has joined #tripleo08:48
*** derekh has joined #tripleo08:48
*** zoli|brb is now known as zoli08:51
*** zoli is now known as zoliXXL08:51
*** jbadiapa_ has joined #tripleo08:52
*** openstackgerrit has quit IRC09:04
*** openstackgerrit has joined #tripleo09:04
*** tobias-fiberdata has joined #tripleo09:07
*** ooolpbot has joined #tripleo09:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]09:10
*** ealcaniz has joined #tripleo09:10
*** tobias_fiberdata has quit IRC09:10
*** tobias_fiberdata has joined #tripleo09:13
*** shardy has joined #tripleo09:16
*** tobias-fiberdata has quit IRC09:16
*** abregman|afk is now known as abregman09:16
b00tcatcould someone please check https://review.openstack.org/#/c/383057 and https://review.openstack.org/#/c/383056/ ?09:18
*** pmannidi has quit IRC09:19
jaosoriorb00tcat: commented on https://review.openstack.org/#/c/383056/09:20
*** abregman is now known as abregman|afk09:23
*** hjensas has quit IRC09:24
*** cwolferh has joined #tripleo09:25
shardyMorning folks, is anyone looking into https://bugs.launchpad.net/tripleo/+bug/1634319 ?09:26
openstackLaunchpad bug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged]09:26
*** zoliXXL is now known as zoli|lunch09:27
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Add deployment cancellation/deletion  https://review.openstack.org/38735509:27
shardyjpena: thanks for the comments on bug #1634319, are you planning to post a patch?09:27
openstackbug 1634319 in tripleo "tripleo stable/newton CI deploys packages from ocata" [Critical,Triaged] https://launchpad.net/bugs/163431909:27
jpenashardy: I can send a patch to pass the env var. If that still fails, I'll need some help to understand the CI scripts a bit better :)09:28
b00tcatthanks jaosorior09:28
shardyHmm, there is already https://review.openstack.org/#/c/387750/ from slagle but it's not picked up in the launchpad review09:29
*** tremble has quit IRC09:29
shardyjpena: ack, lets see if slagle's patch will work (unclear atm from the CI results)09:29
shardythanks for the feedback on the bug09:29
*** egafford has joined #tripleo09:30
jpenashardy: np :)09:31
jtomasekjpich, akrivoka: ok, so if we are able to still backport after rc3 then we should push for post-deployment workflow09:33
*** tremble has joined #tripleo09:33
jpichjtomasek, akrivoka: This may be helpful in understanding what can be backported after release: http://docs.openstack.org/project-team-guide/stable-branches.html#appropriate-fixes09:33
jpichjtomasek, akrivoka: Also http://docs.openstack.org/project-team-guide/stable-branches.html#support-phases above, for the first while all bug fixes are ok09:34
*** gfidente has joined #tripleo09:34
*** gfidente has quit IRC09:34
*** gfidente has joined #tripleo09:34
jtomasekjpich: thanks09:35
*** oneswig has joined #tripleo09:36
*** oneswig has quit IRC09:37
jtomasekakrivoka, jpich: could you please test this one, so we manage to get it into rc3 today? https://review.openstack.org/38735509:37
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Add deployment cancellation/deletion  https://review.openstack.org/38735509:38
*** hjensas has joined #tripleo09:39
jpichjtomasek: Pulling it down now09:40
*** hewbrocca-afk is now known as hewbrocca09:40
*** mgould|afk is now known as mgould09:41
*** anshul has joined #tripleo09:41
*** jaosorior has quit IRC09:42
*** jaosorior has joined #tripleo09:42
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Add deployment cancellation/deletion  https://review.openstack.org/38735509:43
jtomasekjpich: I've just sent a tiny update to that patch, sorry:)09:43
jpichjtomasek: Ack!09:44
jaosoriorCouple of one-liners if someone has time https://review.openstack.org/#/c/386519/  https://review.openstack.org/#/c/386522/09:45
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add parameters to run cinder over httpd  https://review.openstack.org/38720209:48
*** pmannidi has joined #tripleo09:48
*** dtantsur|afk is now known as dtantsur09:50
openstackgerritMerged openstack/puppet-tripleo: Remove faulty migration logic to stop nova-api  https://review.openstack.org/38718909:51
openstackgerritMerged openstack/python-tripleoclient: Allow overriding CLI parameter defaults with env files  https://review.openstack.org/38666909:51
jaosoriorshadower: hey dude, can you check these two out? https://review.openstack.org/#/c/386519/  https://review.openstack.org/#/c/386522/09:54
shadowerjaosorior: I already did :-)09:55
jaosoriorshadower: aah, thanks dude!09:56
shadowerno problem :-)09:56
akrivokajtomasek: will test that asap09:56
*** abregman|afk is now known as abregman09:56
akrivokajtomasek: is there a post deployment workflow patch already up for review?09:56
*** limao has quit IRC09:57
*** oshvartz has joined #tripleo10:01
*** electrofelix has left #tripleo10:02
shadowerjaosorior: btw these validations are (imho) good to go and just wait for another +2 and +A: https://review.openstack.org/#/c/351248/ https://review.openstack.org/#/c/387210/ and https://review.openstack.org/#/c/353953/10:04
shadowerif you have some time10:04
jaosoriorshadower: I'll check em out10:04
*** florianf has joined #tripleo10:04
jaosoriorshadower: hey dude, seems to me that this one https://review.openstack.org/#/c/353953/3 will fail if we enable zaqar for the overcloud10:09
jaosoriorshadower: zaqar's endpoint in haproxy has different timeouts than those due to the needs of websockets10:09
shadowerjaosorior: can you point me to the zaqar values?10:10
openstackgerritMerged openstack/tripleo-validations: Validation to check the undercloud processes  https://review.openstack.org/35124810:11
jaosoriorshadower: ah, wait up, I thought you were checking for equality10:12
jaosoriorI guess it should be fine10:12
jtomasekflorianf: the last change I did to your patch was that I completely removed the fetchStacks() dispatch and let the polling take care of it10:12
jaosoriorshadower: https://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp#L117110:12
jtomasekflorianf: what is the difference between _hasStackDeletionStarted and optimistic update?10:12
openstackgerritMerged openstack/tripleo-validations: Add the NoopFirewallDriver validation  https://review.openstack.org/38721010:13
jtomasekflorianf: if the fetchStacks you issue from deleteStack brings wrong stack state, then deleteStackSuccess won't get called, but then another fetStacks will never happen10:13
jaosoriorshadower: my bad. It looks good10:14
jpichjtomasek: I left a review on the patch, btw is node assignment working well for you? I updated tripleo-common to the latest stable/newton version but I still can't do assign nodes without getting a mistral error. It's quite possibly my frankeinstein environment though, I'll try and update everything to the latest newton later on10:14
*** phpcodemonkey_ has joined #tripleo10:15
*** phpcodemonkey has quit IRC10:15
*** phpcodemonkey_ is now known as phpcodemonkey10:15
jtomasekjpich: It worked on the setup from yesterday evening I installed10:15
shardyrook: So ref https://bugs.launchpad.net/tripleo/+bug/163417110:15
openstackLaunchpad bug 1634171 in tripleo "[RFE] OVN split northd and plugin composable services" [Undecided,New]10:15
jpichjtomasek: Newton?10:16
*** xuao has quit IRC10:16
shardybasically I want to take advantage of the new composable services interfaces, and avoid any more special-casing for the ovn northd (an in particular to avoid it being hard-coded to only work on the controller)10:16
*** pmannidi has quit IRC10:16
jtomasekjpich: umm, nope, I used standard 'latest' repos10:17
openstackgerritMerged openstack/tripleo-validations: Validate haproxy configuration values  https://review.openstack.org/35395310:17
rookshardy: makes total sense.10:17
shardyrook: the problem is any manifest which has a conditional in it probably needs to be defined as two services, so we may need to break that manifest up10:17
jpichjtomasek: Ok! I'll retry anyway, this isn't a brand-new env so lots of things could have gone wrong in the meantime10:17
openstackgerritMerged openstack/tripleo-heat-templates: Ensure all HostsDeployments finish before puppet  https://review.openstack.org/38671610:17
jtomasekjpich: what is the error you're getting?10:17
rookshardy we could associate the ovndb / ovnnorthd "role" with a controller right?10:17
shardyone deploying the northd and one deploying the plugin10:17
shardyrook: yup, we'd just have a new composable service, e.g OS::TripleO::OVNNorthd or something10:17
shardywhich by default can be deployed on the container10:18
shardysorry Controller10:18
jaosoriorshardy: any more patches?10:18
rookright shardy so i see two, ::OVNNorthd and ::OVNDB10:18
jpichjtomasek: Invalid input [name=tripleo.baremetal.v1.tag_nodes, class=DirectWorkflowSpec, missing=['node_uuids'], unexpected=[u'untag_node_uuids', u'tag_node_uuids']]10:18
shardyjaosorior: sorry, patches for what?10:18
rookwell i guess 3, one for the plugin10:18
*** phpcodemonkey has quit IRC10:19
*** anbu has joined #tripleo10:19
jtomasekjpich: yep, your setup is missing latest updates to nodes assignment workflow10:19
* jtomasek checks stable/newton10:19
jaosoriorshardy: sorry, wrong person10:19
jaosoriorshadower: any more patches?10:20
shardyrook: yeah I was a little unclear about the OVNDB part - what deploys that?10:20
shardye.g there's a conditional which deploys northd, but it's not clear to me atm what DB it's supposed to reference10:20
rookshardy currently OOO installs that on the controllers10:20
rookanbu: ^10:20
rookshardy possibly I am using those interchangably10:20
anbushardy, Right now OVNDB will be deployed along with northd by ovn::northd puppet class10:20
jpichjtomasek: Alright, thanks. I pulled the latest tripleo-common, switched to the newton branch and ran the commands in the README to update it - but I think other folks have had problems where Mistral doesn't always update actions correctly10:20
jtomasekjpich: yep it is there10:21
EmilienMany progresso n the CI problem?10:21
shadowerjaosorior: this one would be neat: https://review.openstack.org/#/c/375477/10:21
jaosoriorshadower: I'm on it10:21
jtomasekjpich: you also need to update workflows10:21
shadowermakes the validation output much nicer to read10:21
shardyEmilienM: slagle posted a patch but I rechecked it trying to see if the problem is fixed10:21
jpichjtomasek: Ah!! That might be it, sorry for the noise and thank you for double-checking10:21
EmilienMlooks like no10:21
jtomasekjpich: mistral workbook-update workbooks/baremetal.py10:21
EmilienMok I'm looking10:21
shardyanbu: ok, and would you ever want to deploy northd and ovndb independently (e.g on different nodes perhaps)10:22
jaosoriorshadower: anything specific I have to do to test it out?10:22
anbushardy, Unless the deployer needs to do that, there is no need for deploying them independently.10:23
shardyI'm thinking if we had e.g dedicated networker node(s), or even dedicated ovn node(s)10:23
*** panda|Zz is now known as panda10:23
shardyanbu: Ok, so for now just decoupling the plugin and the northd configuration would be enough10:23
EmilienMwhy don't we have apache logs10:23
jaosoriorEmilienM: what's up?10:24
jpichjtomasek: That was it indeed, thank you10:24
jtomasekjpich: cool!10:24
jaosoriorEmilienM: apache logs where?10:24
anbushardy, Support for pacemaker profile is on the pipeline (waiting for some OVS patches to get merged). With the pacemaker profile ovn-northd and OVNDB resource will be managed independently by the pacemaker10:24
EmilienMon subnodes 2 of multinodes10:24
shadowerjaosorior: just try a few validations (say ntpstat, undercloud-disk-space and undercloud-process-count) with and without10:24
jaosoriorshadower: I tried the haproxy validation again, looks quite the same, unless I messed something up. Did python setup.py install in the validations repo10:25
shardyanbu: Ok, have you seen that we've been moving away from pacemaker for most services tho?10:25
shardybandini: ^^10:25
anbushardy, I didn't know that.10:25
shadowerjaosorior: how do you run the validation? With the run-validation script or directly with ansible?10:25
anbushardy, FYI - https://review.openstack.org/#/c/372274/ is the patch for your reference.10:25
* bandini looks10:26
rookanbu: pacemaker-lite is what I keep hearing10:26
jaosoriorEmilienM: what patch is it? I can see that the apache logs in another patch for subnode 210:26
jaosoriorshadower: run-validation10:26
shardyanbu: https://blueprints.launchpad.net/tripleo/+spec/ha-lightweight-architecture10:26
shadowerjaosorior: this patch works only with ansible direct, but there's this for run-validation: https://review.openstack.org/#/c/379348/10:26
anbushardy, Thanks for sharing this link Steve10:26
bandinianbu, rook: https://github.com/openstack/tripleo-specs/blob/master/specs/newton/pacemaker-next-generation-architecture.rst10:26
shardyanbu: that was implemented during Newton, so you may want to chat with one of the HA experts such as bandini to ensure your HA model is a good fit for that new architecture10:26
jaosoriorshadower: how do I run it only with ansible then?10:26
anbusure shardy10:27
shadowerjaosorior: if you cd tripleo-validations and then do: ansible-playbook -i tripleo-anisble-inventory validations/undercloud-ram.yaml10:27
shadowerjaosorior: that should work10:27
bandinianbu: happy to help, feel free to ping ;)10:27
*** tremble has quit IRC10:27
anbusure bandini10:27
jaosoriorshadower: now it complaigns about not finding the hosts file :/10:28
shardyrook: let me push a couple of WIP patches showing what I mean, then you can see if they make sense and/or work? :)10:28
anbubandini, I will look at the architecture and will get back. Just a simple question, will there be a need for OCF resource agent in the new architecture?10:28
rookshardy sure, anbu should really take lead, i was trying to help out with the OVN work, but maybe someone closer to the project should? I also wanted to learn a bit more of the inner workings of Puppet and THT10:29
*** tremble has joined #tripleo10:29
openstackgerritBabu Shanmugam proposed openstack/puppet-tripleo: WIP - Fix bug in ovn plugin  https://review.openstack.org/38736610:29
shardyrook: sure, I just feel like I'm maybe not explaining things that well, and some patches may help10:29
shadowerjaosorior: try passing -i scripts/tripleo-ansible-inventory10:29
shardyI've not really got bandwidth to test this atm tho, so I'll throw up some patches and we can see who's willing to test them :)10:30
anbushardy, https://review.openstack.org/387366 is still in WIP. It does not isolate the ovn from the plugin though10:30
EmilienMjaosorior: thx dude https://review.openstack.org/#/c/387189/10:31
shardyanbu: ack, yeah that looks better than hard-coding references to the controller, but it's still not quite what I had in mind10:32
anbuok shardy10:32
shardyanbu: I'll push my idea and you can see which you prefer10:32
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775010:32
jaosoriorshadower: was able to try it out finally. +A10:32
shadowerjaosorior: awesome!10:33
rookanbu shardy alright to not work against each other (conflicts) i have opted to abandon my pathset https://review.openstack.org/#/c/387594/10:33
shadowerjaosorior: thanks a lot10:33
jaosoriorshadower: no biggie10:33
anbuThanks for your help rook10:33
rookanbu hope i helped and didn't confuse things any further :)10:34
bandinianbu: sec (am on a call ;)10:35
shardyanbu: right now you need northd to only run on one controller, right?10:35
openstackgerritMerged openstack/tripleo-validations: Add a validation-specific Ansible output callback  https://review.openstack.org/37547710:36
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add parameters to run cinder over httpd  https://review.openstack.org/38720210:36
*** thrash|g0ne is now known as thrash10:36
*** rhallisey has joined #tripleo10:37
rookshardy I don't think they have a HA solution yet. Waiting on some patches for that10:37
shardyOk so we can't access the northd/ovndb service via a vip and haproxy10:38
anbushardy, we need just one instance of northd running all the time.10:38
openstackgerritJiri Stransky proposed openstack/python-tripleoclient: Allow overriding CLI parameter defaults with env files  https://review.openstack.org/38791910:39
rookshardy not to confuse things but in my mind, i compare the northd to odl (the sdn controller).10:39
*** jtomasek is now known as jtomasek|biab10:40
shardyrook: Yeah odl has this same problem10:41
shardythat's accessing odl via the vip/haproxy tho10:42
* shardy looks to see how we configure haproxy10:42
shardyI don't see how that will work10:43
rookshardy: seems like a common problem. Should there be a set of generic steps that these 3rd party SDN controllers follow to add in their plugin + a SDN controller that _might_ run on the Controller role?10:43
shardytrozet: ^^10:44
anburook, shardy I don't about about ODL. But to put it in simple terms northd is something that gets input from neutron plugin and informs the agents (running in the compute) about the actions in a more detailed manner10:44
jaosoriorCan someone check this out https://review.openstack.org/#/c/387266/ ?10:44
shardyrook: yeah, it's kinda a tricky fit for our composability model, because we expect services to run on all nodes in a cluster, then be balanced via haproxy10:44
rookanbu very similar to SDN Controllers.10:44
anbuok rook10:44
rookanbu most work in that fashion.10:45
shardyrook: this would be fine if we deployed a dedicated single SDNController role, but with a 3 node HA controller deployment it's kinda tricky10:45
rookshardy sure.10:45
rookshardy: seems like most of these controllers are bypassing the 3 node ha anyway10:45
rookand just sticking with a single node.10:46
rook"these controllers" being SDN Controllers10:46
*** chlong has quit IRC10:46
shardyrook: yeah10:46
shardyrook: I'm wondering if we can special-case the haproxy configuration so we just hit the first node in the e.g opendaylight_api_node_ips list10:46
rookshardy If this is a bigger problem then just OVN, I would rather see us implement a generic solution that could be implemented with any SDN/Whatever controller.10:46
shardyrook: Yeah, agreed10:47
rookisn't there like a way to stick connections with haproxy?10:47
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489210:47
shardyrook: pretty sure there is but don't know it off the top of my head10:48
shardyAFAICS that ODL configuration is going to load-balance over nodes where two aren't even running the service10:48
rookugh, we can use the mariadb config as a example in haproxy10:48
rookI am pretty sure we use it there.10:48
* rook reaches deep into the memory banks10:48
jaosoriorshadower: hey dude, can you check this outhttps://review.openstack.org/#/c/387266/ ?10:49
rookyeah, stick on dst : https://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp#L102210:50
rookso it shouldn't move around to the other nodes ^ shardy10:51
EmilienMcan i have a second +2 on https://review.openstack.org/#/c/387750/ please?10:55
EmilienMwe need this one to fix CI (and i'll approve it once jobs are finished)10:55
jaosoriorEmilienM: done10:55
*** numans has quit IRC10:55
EmilienMjaosorior: thx10:56
*** sudipto_ has quit IRC10:58
*** sudipto has quit IRC10:58
d0ugalakrivoka: https://bugs.launchpad.net/tripleo/+bug/163127911:00
openstackLaunchpad bug 1631279 in tripleo "SnmpdReadonlyUserPassword isn't set by the Mistral workflow" [High,Fix released] - Assigned to Dougal Matthews (d0ugal)11:00
d0ugalakrivoka: I'll add a comment with the workaround11:00
akrivokad0ugal: thank you11:00
*** zoli|lunch is now known as zoli11:01
*** zoli is now known as zoliXXL11:01
EmilienMshardy: so I'm going to propose RC3 this morning anyway11:04
EmilienMeven if we miss some backports, we don't have choice now.11:04
EmilienMteam: please continue as fixing rc3 bugs and doing backports; I'll handle the release thing11:05
shardyEmilienM: agreed, we'll have to handle those after the GA11:05
*** spredzy has quit IRC11:06
d0ugalakrivoka: https://bugs.launchpad.net/tripleo/+bug/1631279/comments/1011:06
openstackLaunchpad bug 1631279 in tripleo "SnmpdReadonlyUserPassword isn't set by the Mistral workflow" [High,Fix released] - Assigned to Dougal Matthews (d0ugal)11:06
*** coolsvap has quit IRC11:06
akrivoka thanks d0ugal11:07
EmilienM41 bugs were fixed in RC3. it's more than milestones :P11:07
*** spredzy has joined #tripleo11:07
jpichakrivoka: FWIW in case you hit other issues, I've been documenting some of the workarounds needed to deploy with the UI during the RC period at https://etherpad.openstack.org/p/tripleo-ui-newton-rc - depending on what version of the code you're running you probably won't need most of these though :)11:08
akrivokajpich: thanks, that's helpful11:08
d0ugalEmilienM: Impressive!11:09
d0ugalEmilienM: That is because we introduced all the bugs in milestones >_<11:10
*** pkovar has joined #tripleo11:10
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Modify the constraint to allow single quote for DPDK core list param  https://review.openstack.org/38767611:10
EmilienMd0ugal: :P11:10
d0ugaljpich: That is quite a list!11:10
d0ugalakrivoka: There is a chance the SnmpdReadonlyUserPassword will be fixed, I am just reinstalling now to test11:11
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Use Password Generation provided by the Mistral workflows  https://review.openstack.org/37554411:11
jpichd0ugal: Things got messy for a while :)11:11
florianfjtomasek|biab: What about a situation where you request a delete from heat, but the next fetch still returns a create_in_progress (If they happen in close succession -- I don't think the delete happens immediately, even if the delete API call was successful)? If we do an optimistic update (update the isRequestingStackDelete flag with every fetch), this will re-enable the delete button, even though a delete has been requested.11:11
*** abregman is now known as abregman|afk11:12
*** lucas-afk is now known as lucasagomes11:12
d0ugalEmilienM: Do you think we can get this in? https://review.openstack.org/#/c/386558/11:13
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489211:13
d0ugalEmilienM: it's a real issue that anyone that runs the undercloud install twice will hit. jaosorior hit it earlier today.11:13
EmilienMd0ugal: right now we can get nothing11:13
d0ugal(and manually tested the fix)11:13
EmilienMCI is broken11:13
d0ugalEmilienM: ah well11:13
EmilienMhttps://review.openstack.org/#/c/387750 might be the fix, i'm watching zuul11:13
*** dprince has joined #tripleo11:14
*** dprince has quit IRC11:14
d0ugalEmilienM: k, thanks - let me know if I can help11:14
EmilienMd0ugal: but it's too late for RC311:14
EmilienMd0ugal: we'll backport it but it won't make GA11:14
EmilienMd0ugal: we'll probably tag again after GA11:14
d0ugalEmilienM: Alright, fair enough. Should I document a workaround somewhere?11:14
EmilienMd0ugal: you're already helping by fixing bugs11:14
EmilienMd0ugal: I think it's fine, we'll backport it this week. It just won't make GA11:15
d0ugalI'll add a workaround comment to the bug11:15
EmilienMd0ugal: ok11:15
akrivokad0ugal: I'm not seeing the SnmpdReadonlyUserPassword parameter in the ui parameters form... do you know where I should look for it?11:16
EmilienMd0ugal: be prepared to push a workaround in doc, but not sure. We'll most probably backport it11:16
d0ugalakrivoka: I don't know my way around the GUI - akrivoka ?11:17
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: specify bind host for ceilometer  https://review.openstack.org/38793811:17
d0ugalI mean: jpich ?11:17
openstackgerritSteven Hardy proposed openstack/puppet-tripleo: WIP Split ovn plugin and northd configuration  https://review.openstack.org/38793911:17
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Split OVN northd and ml2 plugin  https://review.openstack.org/38794011:17
jpichakrivoka: It's in that etherpad at #8, does that help?11:18
shardyrook, anbu: ^^ WIP patches showing another approach to wire this up which should remove the need to hard-code the ovn northd IP11:18
shardynot tested, feedback welcome11:18
shardyyou're also welcome to push fixes and take over the patches11:18
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775011:19
*** shardy is now known as shardy_lunch11:19
akrivokajpich: ah, yes. thanks11:19
EmilienMjaosorior: can you +2 again please? ^11:19
EmilienMjaosorior: I rechecked11:19
EmilienMjaosorior: the puppet-tripleo fix was not yet in delorean current11:19
EmilienMand now it is I think11:19
*** pkovar has quit IRC11:21
*** numans has joined #tripleo11:22
anbuthanks shardy_lunch11:22
*** fzdarsky_ is now known as fzdarsky|afk11:22
*** fzdarsky|afk is now known as fzdarsky|lunch11:22
pandaposted this https://bugs.launchpad.net/tripleo/+bug/1634468 for failures to periodic nonha job in the last two days11:26
openstackLaunchpad bug 1634468 in tripleo "periodic nonha job fails to install undercloud" [Undecided,New]11:26
d0ugalThe option is defined here, but where in the code is it generated? https://github.com/openstack/instack-undercloud/blob/master/instack_undercloud/undercloud.py#L390-L39311:26
d0ugalAnyone know?11:26
d0ugalMaybe in puppet?11:27
* d0ugal searches11:27
akrivokajpich: have you seen this one maybe? https://paste.fedoraproject.org/454932/67900481/raw/11:28
akrivokaProperty GnocchiPassword not assigned11:28
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Run ceilometer over httpd  https://review.openstack.org/38795811:28
EmilienMpanda: have you checked keystone logs?11:28
d0ugalakrivoka: What code are you running?11:29
jpichakrivoka: Is this a new env? I thought we recently got all the passwords automatically generated by tripleo-common11:29
d0ugalakrivoka: A new install?11:29
akrivokad0ugal: yes, latest11:29
d0ugalakrivoka: I think I just found a new bug in master11:29
EmilienMpanda: http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/9697c38/logs/undercloud/var/log/keystone/keystone.txt.gz#_2016-10-18_06_35_27_74911:29
d0ugalakrivoka: can you show me: mistral environment-get overcloud11:29
EmilienMCould not find domain: Default11:29
EmilienMmaybe jaosorior can help to debug this keystone thing11:29
akrivokad0ugal: https://paste.fedoraproject.org/454935/79019614/11:30
pandaI've seen this for the mitaka HA bug11:30
d0ugalakrivoka: Yeah, you are hitting the same issue.11:30
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: DO NOT MERGE - TEST  https://review.openstack.org/38795911:30
d0ugalakrivoka: openstack overcloud plan delete overcloud && openstack overcloud play create overcloud11:30
EmilienMslagle: hey11:30
d0ugalakrivoka: That will get you the passwords you are missing11:30
openstackgerritJames Slagle proposed openstack/instack-undercloud: DO NOT MERGE - TEST  https://review.openstack.org/38796111:31
EmilienMslagle: your patch seems to be ok btw, it just master broke too in the meantime11:31
akrivokad0ugal: so the default plan is missing some passwords?11:31
EmilienMslagle: and was fixed by https://github.com/openstack/puppet-tripleo/commit/bd5fd8c1095a6d66e94a6629a3be6bf5f81dc68711:31
slagleEmilienM: yea, i just realized that tripleo-ci patches dont trigger newton multinode jobs11:31
EmilienMslagle: I pushed over your patch to trigger CI jobs again11:31
EmilienMslagle: right, but I did a depends-on on a newton backport11:31
EmilienMslagle: 38775011:32
jaosoriorEmilienM: where is that being called? AFAIK the default domain in keystone has the ID 'default' and name 'Default'. So we would need to see which was used and where11:32
slagleoh, and it passed?11:32
EmilienMslagle: not yet, it's in zuul11:32
EmilienMjaosorior: you tell me11:32
slaglewhere? which one?11:32
d0ugalakrivoka: Yeah, because of an issue in instack-undercloud, but re-creating it should solve it.11:32
jaosoriorEmilienM: which job was it?11:32
jaosoriorah, periodic11:32
d0ugalakrivoka: Something code in instack-undercloud is in the wrong order.11:32
slagleEmilienM: that's what i was trying to do with https://review.openstack.org/38796111:32
*** ccamacho is now known as ccamacho|lunch11:33
d0ugalakrivoka: s/something/some/11:33
akrivokad0ugal: yeah, I see they're there now, thanks11:33
EmilienMslagle: ok, let's see how it works11:34
d0ugalakrivoka: np, this bug isn't in Newton at least, because it hasn't been backported yet, lol11:34
EmilienMslagle: thanks for finding the fix last night btw11:34
*** ayoung has quit IRC11:34
jaosoriorEmilienM: ok, so something clearly is trying to access keystone using the wrong domain ID. It's using the name instead of the ID. Now, I'm not sure if that's happening from a puppet provider, or if we do that in some part of the undercloud install.11:35
openstackgerritJames Slagle proposed openstack/instack-undercloud: DO NOT MERGE - TEST  https://review.openstack.org/38796111:36
EmilienMjaosorior: in puppet-keystone, we set DEFAULT_DOMAIN = 'Default'11:36
EmilienMin provider/keystone.rb11:36
*** pkovar has joined #tripleo11:36
EmilienMI can't recall if it's the ID or name, maybe chem can tell11:37
pandajaosorior: EmilienM could this Warning lead to the error "unable to connect" we're seeing in puppet ?11:37
*** jpena is now known as jpena|lunch11:38
jaosoriorpanda: I don't think so11:38
* chem backtracking11:38
jaosoriorpanda: if it's an "unable to connect" that sounds like it's apache not starting11:38
EmilienMapache is started11:39
EmilienMsee here11:39
EmilienMKeystone_service[glance::image]/ensure: created11:39
EmilienMthe provider worked for other resources11:40
EmilienMbut failed to create the admin user11:40
EmilienMsounds like related to admin endpoint11:40
EmilienMjaosorior: didnt you change something recently?11:40
akrivokajpich: will add this to your troubleshooting etherpad11:40
*** dsariel has quit IRC11:41
chemjaosorior: EmilienM where are the logs for this wrong default domain stuff ?11:41
jaosoriorEmilienM: for keystone's domain? no11:41
jpichakrivoka: Thanks11:41
bandinianbu: back, sorry. So no in general you do not really need a new OCF resource for a service, unless you're solving a very specific problem11:41
pandachem: http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/9697c38/logs/undercloud/var/log/keystone/keystone.txt.gz#_2016-10-18_06_26_49_89711:42
EmilienMjaosorior: no, for admin endpoints.11:42
EmilienMjaosorior: it might be a networking issue11:42
EmilienMchem: you can see the failure in http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/9697c38/logs/undercloud/var/log/undercloud_install.txt.gz11:42
jaosoriorEmilienM: indeed, I made such a change.11:42
EmilienMjaosorior: right ;-)11:43
chempanda: EmilienM thanks11:43
jaosoriorEmilienM: uhm... in CI we do specify a no_proxy for the haproxy public VIP and for the undercloud IP that the services bind on, but I'm not sure if we do the same for the admin VIP that is now listening for the services.11:43
pandaI see only tnpd listening to that IP on host-info log11:43
EmilienMthis one https://review.openstack.org/#/c/360366/11:43
anbubandini, For ovn we are going with pacemaker for managing the OVN DB servers which are active/passive servers.11:44
anbubandini, Only instance of the DB server will rw and all the other will be just backing up and cannot be accessed by the client. Pacemaker will be used promote one of the backup servers in case the active server failed11:45
anbubandini, Since the IP address on which the active server listens will be used by the ovn agent running in all the compute nodes, the DB server's active instance will have to be colocated with the IPAddr2 resource11:46
jaosoriorEmilienM: nevermind, we are setting no_proxy for that VIP too.11:47
bandinianbu: I see, yes if we are talking master/slave with promotion and demotion we likely need an ad-hoc resource agent that manages that11:47
anbubandini, I learnt the same from the architecture doc that you shared earlier.11:48
jaosoriorEmilienM: so, the address that it's failing on is not the admin, seems to be failing on the public one.11:49
anbubandini, We also need northd to be managed by pacemaker. It is just a primitive resource. In case the node running northd fails, it has to be restarted in other node.11:49
akrivokad0ugal: did you get a successful deploy? it failed for me11:49
pandais haproxy configured to listen to that IP/port correctly ?11:49
anbubandini, northd is just a systemd resource. It is independent of the OVN DB servers.11:50
bandinianbu: do you have more info about what you are trying to do? i.e. what is northd etc.11:50
anbubandini, northd is just a simple process which reads from one database (North database) and write onto another database (South database)11:50
anbuI have a WIP patch for these changes. https://review.openstack.org/#/c/372274/11:51
*** rbrady-afk is now known as rbrady11:51
anbubandini, How can the northd fit in the new architecture?11:52
d0ugalakrivoka: I've not tried that far yet :)11:52
d0ugalakrivoka: working on fixing these issues11:53
jpichakrivoka: What's the failure?11:53
akrivokaResource CREATE failed: Error: resources.AllNodesDeploySteps.resources.ComputeDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 111:53
bandinianbu: let me read up on that a bit11:53
anbuok bandini11:53
jpichakrivoka: openstack stack failures list overcloud?11:53
jaosoriorakrivoka: that should have been fixed recently. Not sure if the fix merged already11:54
*** jtomasek|biab is now known as jtomasek11:54
*** dprince has joined #tripleo11:55
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489211:55
chempanda: jaosorior EmilienM hum so this undercloud has two ips one for db and one for keystone, is that expected ?11:55
jaosoriorakrivoka: yeah, you need this https://review.openstack.org/#/c/386716/11:55
jaosoriorchem, yes11:55
jpichakrivoka: Huh, that one I haven't personally seen, but jaosorior knows all the things :)11:55
chemjaosorior: oki11:55
pandachem: yes, I think because pacemaker is enabled11:55
jaosoriorchem, the undercloud services run on, and haproxy listens on and (public and admin respectively)11:56
akrivokahah, thiis merged an hour ago11:56
pandajaosorior: I don't see anything listening on 13000 on that IP11:56
akrivokathanks jaosorior :)11:56
jaosoriorpanda: wasn't the issue in the undercloud?11:56
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775011:57
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: DO NOT MERGE - TEST  https://review.openstack.org/38795911:57
jaosorioror am I getting really confused here?11:57
pandajaosorior: hmm, no you're right, I got confused11:57
EmilienMslagle: how it broke CI jobs11:57
*** flepied has quit IRC11:57
EmilienMslagle: I guess it was needed11:57
openstackgerritDan Prince proposed openstack/puppet-tripleo: Set memcached_servers for nova API  https://review.openstack.org/38762411:58
EmilienMdprince: https://review.openstack.org/#/c/387974/ FYI11:58
dprinceEmilienM: cool, so backport mine but use your for ocata?11:59
EmilienMdprince: yes11:59
jaosoriorpanda: so in the undercloud nothing is running on ?11:59
chemjaosorior: panda I cannot find haproxy conf and log ?11:59
EmilienMdprince: because I'll have to do it for all modules, it will takes me some days or weeks11:59
openstackgerritJames Slagle proposed openstack/instack-undercloud: DO NOT MERGE - TEST  https://review.openstack.org/38796111:59
jaosoriorchem: http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/9697c38/logs/ you gotta download undercloud.tar.xz and there it is in /etc/haproxy/haproxy.cfg12:00
dprinceEmilienM: yep, gotcha12:00
EmilienMdprince: well, for other modules it's easier as we use this Define12:00
EmilienMbut still some work12:00
pandajaosorior: I don't see it on the host-info log in the netstat section12:00
chemjaosorior: ah, thanks12:00
* EmilienM brb12:00
pandajaosorior: let me check again12:00
jaosoriorpanda: where can I find that log?12:01
pandajaosorior: nope, nothing, the only reference is on the firewall configuration12:01
pandajaosorior: http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/9697c38/logs/undercloud/var/log/host_info.txt.gz12:01
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489212:01
jaosoriorpanda: so.. haproxy is not running O_O12:02
pandajaosorior: should it run on the undercloud, yes ?12:03
*** phpcodemonkey has joined #tripleo12:03
jaosoriorpanda: yes12:03
*** jayg|g0n3 is now known as jayg12:03
chemjaosorior:  panda -> haproxy loaded    inactive dead12:05
chemjaosorior:  panda in var/log/host_info.txt12:05
openstackgerritMerged openstack/tripleo-ui: Add deployment cancellation/deletion  https://review.openstack.org/38735512:06
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Add deployment cancellation/deletion  https://review.openstack.org/38797912:07
chemjaosorior:  panda there is this in audit.log http://paste.openstack.org/show/586171/ but not sure it's related12:07
*** ealcaniz has quit IRC12:07
*** ealcaniz has joined #tripleo12:07
openstackgerritiswarya vakati proposed openstack/python-tripleoclient: Update .coveragerc after the removal of respective directory  https://review.openstack.org/38798012:08
pandachem: jaosorior I see puppet is starting it correctly12:09
jaosoriorchem: While that's relevant and needs to be fixed (it will cause issues with certificates at some point) I don't think it's related since it just issued a reload of the haproxy configuration12:09
pandawhooops. Here http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/9697c38/logs/undercloud/var/log/undercloud_install.txt.gz#_2016-10-18_06_25_28_00012:09
pandaso it must die after12:09
*** kjw3 has joined #tripleo12:10
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Ensure all HostsDeployments finish before puppet  https://review.openstack.org/38798212:10
jaosoriorrhallisey: hey dude can you check this out from tue audit log? http://paste.openstack.org/show/586171/12:10
*** tzumainn has joined #tripleo12:11
jaosoriorchem: actually, now I'm not sure, that might be it.12:11
rhalliseyjaosorior, I doubt that's causing an issue12:12
rhalliseywhat issue are you seeing?12:12
jaosoriorrhallisey: uhm... so what's going on in that log?12:13
jaosoriorrhallisey: well, haproxy not starting... or starting but dying at some point. pretty weird12:14
jaosoriorrhallisey: you're right12:14
rhalliseyjaosorior, can you gather the logs from permissive12:14
jaosoriorthat's not the issue12:14
pandaEmilienM: triaging question, I'd put this bug in critical since it's blocking CI and with ocata-1 milestone, but doesn't seem that the gates are failing with that error .. so maybe High is enough12:14
jaosoriorpanda, chem, I found the issue12:14
*** hewbrocca is now known as hewbrocca-afk12:14
pandajaosorior: \o/12:14
*** kjw3 has quit IRC12:14
jaosoriorfor some reason, puppet prints that haproxy started. but that's not true12:14
jtomasekakrivoka: I am hitting the same syntax error issue today12:15
jaosoriorpanda, chem look http://paste.openstack.org/show/586174/12:15
chemjaosorior: hum ... still strange :)12:15
*** fzdarsky|lunch is now known as fzdarsky12:15
jaosoriorit's the same ceilometer issue that we were looking at with EmilienM and amoralej12:15
akrivokajtomasek: this is supposed to fix it, I'm testing now https://review.openstack.org/#/c/38671612:16
jaosoriorchem, panda so haproxy fails to start cause it can't bind to that IP with ceilometer12:16
*** flepied has joined #tripleo12:16
jtomasekakrivoka: ack12:16
jaosoriorrhallisey: sorry for the noise... either way, I'm curious what that selinux log means12:16
rhalliseyno problem12:17
pandausually bind failure give an reason ... but I suspect it's because the IP is not there yet ? The alternative is that it's already in use, so ceilometer is started on the same IP/port12:17
pandajaosorior: ^12:17
*** ealcaniz has quit IRC12:19
*** trown|outtypewww is now known as trown12:19
jaosoriorthe ceilometer configuration doesn't contain the host we set in hiera in the undercloud12:19
jaosoriorpanda, chem this is my attempt at fixing it https://review.openstack.org/#/c/387958/ , at least we do know that setting the bind host in apache works12:19
*** maticue has joined #tripleo12:20
openstackgerritMerged openstack/tripleo-ui: Add deployment cancellation/deletion  https://review.openstack.org/38797912:20
chemjaosorior: panda I'm concern about puppet missing the failed start, maybe there is a problem with haproxy-systemd-wrapper not returning an error12:21
*** shardy_lunch is now known as shardy12:21
jaosoriorchem: well, according to the log I showed you http://paste.openstack.org/show/586174/ it returns 256, from what I understood12:22
chemjaosorior: panda, well it may just capture that haproxy exited with rc=256, but not sure that it pass down the status "haproxy-systemd-wrapper: exit, haproxy RC=256"12:23
jaosoriorgood point12:24
jaosoriorand if that's the case, that's messed up12:24
jaosoriorchem: any idea where's the code for that?12:24
openstackgerritLukas Bezdicka proposed openstack/tripleo-common: Refresh DeployIdentifier as well on update  https://review.openstack.org/38753112:25
*** bfournie has quit IRC12:25
chemjaosorior: /usr/lib/systemd/system/haproxy-systemd-wrapper ?12:25
pandajaosorior: here ? https://github.com/horms/haproxy/blob/master/src/haproxy-systemd-wrapper.c12:26
*** kjw3 has joined #tripleo12:26
pandajaosorior: it does return status here at the end12:26
pandachem: ^12:26
jaosoriorpanda, chem https://github.com/horms/haproxy/blob/master/src/haproxy-systemd-wrapper.c#L211 yep12:27
pandaand that's a change from 201412:27
*** michapma_alt has joined #tripleo12:28
jaosoriorEmilienM, shardy an we merge this? https://review.openstack.org/#/c/385488/212:30
chemjaosorior: panda are we sure we have that version ? https://github.com/horms/haproxy/commit/e8ea598721ee7e7c093853cc1b8c1f5abe717f9012:30
jaosoriorchem: that's messed up12:30
jaosoriorby the looks of it12:30
jaosoriorI thinkw e don't12:30
EmilienMjaosorior: things are fairly broken now, not sure we need to merge things12:31
pandait's a change from 2014 ...how's that possible ?12:31
jaosoriorEmilienM: that is pretty much the same commit I had done (which I abandoned), that will fix ceilometer and the issue panda was seeing12:32
EmilienMjaosorior: moreover, it's not managing upgrade from non wsgi to wsgi12:32
chemjaosorior: panda well so that's the whole story then :) this wrapper is broken and let puppet continue while it shouldn't12:32
jaosoriorEmilienM: I see12:32
openstackgerritDougal Matthews proposed openstack/instack-undercloud: Only create the Mistral config env if it doesn't exist  https://review.openstack.org/38655812:32
EmilienMjaosorior: ah no12:32
jaosoriorEmilienM: isn't the upgraded handled by puppet-ceilometer https://github.com/openstack/puppet-ceilometer/blob/master/manifests/api.pp#L159 ?12:32
pandachem: well'that's part of the story, we still have to find out why haproxy fails in the first place12:32
EmilienMwe stop services during upgrade.12:33
chemjaosorior: the upgrade to wsgi ceilometer is handled in osp9/osp10 upgrade12:33
EmilienMok +A12:33
*** oshvartz has quit IRC12:33
pradkEmilienM, as i mentioned in the same commit.. puppet a;ready does this12:33
jaosoriorchem: so, how do we get the newer code for the haproxy wrapper?12:33
pradkEmilienM, see bnemec and my comments in the commit12:33
jaosoriorEmilienM: thanks dude12:33
chemjaosorior: you upgrade haproxy I guess12:34
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Use Password Generation provided by the Mistral workflows  https://review.openstack.org/37554412:34
chemjaosorior: but it looks like 1.6 ...12:34
pradkEmilienM, thx12:34
jaosoriorchem: yeah, and in CentOS we have 1.5.X12:34
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775012:34
chemjaosorior: or more than 1.5-dev23 ?12:34
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: DO NOT MERGE - TEST  https://review.openstack.org/38795912:34
EmilienMslagle: still not working?12:35
openstackgerritJames Slagle proposed openstack/instack-undercloud: DO NOT MERGE - TEST  https://review.openstack.org/38796112:35
chemjaosorior: panda but why it cannot bind to  ?12:35
jaosoriorchem: cause ceilometer was binding on
jaosoriorwhich was wrong12:36
chemjaosorior: I see, so let's fix that.12:37
pradkthe issue is the ceilo-api script upstream in newton ignores the host info in ceilo.conf .. and defaults to whatever pbr gives12:37
pradkhence the issue12:37
jaosoriorchem: well, we want to run ceilometer over httpd, and making that switch will fix this issue too. The patch is already +Aed and will merge soon.12:37
pradkits really an upstream bug imo but we have to work around it for now.. its better to move to wsgi anyway12:37
pandachem: I see that commit in 1.5.0 too12:37
*** jpena|lunch is now known as jpena12:38
openstackgerritJames Slagle proposed openstack/instack-undercloud: DO NOT MERGE - TEST  https://review.openstack.org/38796112:39
*** fultonj has joined #tripleo12:39
chemjaosorior: and sorry, the switch to wsgi I talked about earlier is irrelevent, that's overcloud...12:41
*** fzdarsky_ has joined #tripleo12:42
pabelangerwanted to get some feedback on https://review.openstack.org/#/c/385608/ which adds ansible-playbook support to diskimage-builder.  Now, if an ansible.d directory is found in your element, disk-image-create, will run ansible-playbook against it.  However, question I had, would people be okay with using ansible.d/site.yaml as the entry point or something more like ansible.d/install.d/site.yaml,12:42
pabelangeransible.d/finalise.d/site.yaml to better control when ansible-playbook would be run?  The current patch only runs ansible-playbook after the finalise phase12:42
chempanda: I've got 1.5.14-3.el7 and it always exits 012:43
*** amoralej is now known as amoralej|lunch12:43
chempanda: I think 1.5.23 (or 24) is needed (by the look of the tag)12:43
jaosoriorsame here12:44
chemjaosorior: panda well so this version is buggy12:44
pandachem: curious ... do we have the spec file ?12:44
pandachem: I dont' understand why taht old commit is not pulled ...12:45
pandajaosorior: I updated the bug with the findings .. if you want to modify the commit message to close it ..12:46
pandajaosorior: too late ?12:46
jaosoriorpanda: too late12:47
*** sudipto has joined #tripleo12:47
*** sudipto_ has joined #tripleo12:47
chempanda: I don't have the source rpm, where is the bug.  I think it would be good to have downstream as well ...12:47
chempanda: (sorry multiple question sentence ...)12:48
pandajaosorior: /me ragequits12:48
jaosoriorpanda: well, it can be closed without the commit12:49
jaosoriorpanda: I'll close it manually once it merges12:49
openstackgerritMerged openstack/tripleo-heat-templates: Add apache workers to nova-api conditional  https://review.openstack.org/38759812:50
jaosoriorpanda: can you pass the bug url again?12:50
*** tiswanso has joined #tripleo12:50
*** tiswanso has quit IRC12:50
EmilienMjaosorior: really it would be great to stabilize a bit the things now and stop merging stuffs12:51
EmilienMCI has been very broken for 4 days now12:51
pandajaosorior: so, just to undestand, you patch tells ceilometer to use existing apache with wsgi module to not bind the port ?12:51
*** tiswanso has joined #tripleo12:51
EmilienMi'm proposing the RC3 now because we won't land anything else this morning12:51
jaosoriorEmilienM: what I'm trying to merge now is the fixes dude.12:52
*** mgrohar has joined #tripleo12:52
EmilienMjaosorior: cool.12:52
*** dsariel has joined #tripleo12:53
pradkif ci is broken all the more reaosn to wait and get things stable and working?12:53
jaosoriorpanda: yeah, so instead of running the ceilometer-api script that was binding to port we will run in in a vhost over httpd. that vhost is configured to listen on a specific address, which will allow haproxy to run (since it was conflicting before)12:54
pradki really think we should backport my wsgi fix to undercloud -> https://review.openstack.org/#/c/385488/12:54
jaosoriorI don't understand that concept of "wait for things to get stable", thought we were "working to fix things"12:54
EmilienMI didn't say to wait, I just said we need to be conservative now12:54
EmilienMyes, we need this wsgi stuff, we approved it12:55
*** paramite has quit IRC12:55
pandajaosorior: but you didn'r remove any line containing ceilomter-api12:55
EmilienMand yes we can backport it. CI will just not pass, all our multinode jobs are failing12:55
*** paramite has joined #tripleo12:55
jaosoriorpanda: right, that's done in puppet-ceilometer https://github.com/openstack/puppet-ceilometer/blob/master/manifests/api.pp#L15912:56
pradkpanda, puppet ceilo handles turning it off before starting httpd12:56
slagleEmilienM: I think https://review.openstack.org/#/c/387750/ is right now12:56
EmilienMslagle: ok, thx12:56
pandammmhh, this is bad, a periodic job fails on something gate is not failing on, and gates are failing on something that the periodic job is not failing on ...12:56
*** penick has joined #tripleo12:57
jaosoriorshardy, slagle are we by any chance disabling telemetry in the undercloud from the jobs?12:57
*** cylopez has quit IRC12:58
shardyjaosorior: yes, it's been disabled by default for some time, but pradk has a patch which re-enables it12:58
*** ccamacho|lunch is now known as ccamacho12:58
shardywe don't use those services for anything12:58
jaosoriorpanda: ^^12:58
jaosoriorshardy: thanks, now it makes sense12:58
*** bfournie has joined #tripleo12:58
slaglejaosorior: i dont think we do it via tripleo-ci though12:59
pandajaosorior: sorry for being dense, if the stopping is done int ceilometer-api, and the module was already included before your last change, shy do you need to add the wsgi stuff ?12:59
shardyNo, there's an enable_telemetry option in instack-undercloud12:59
pandain* why*12:59
jaosoriorpanda: even if that module was included, the service_name triggers a conditional. Before, it was running ceilometer-api via a script (so it runs over eventlet I think), and by changing the service_name, which it goes to the other part of the conditional, which stops the service, and expects us to run it over apache.13:01
*** ayoung has joined #tripleo13:01
jaosoriorpanda: this is the conditional https://github.com/openstack/puppet-ceilometer/blob/master/manifests/api.pp#L14713:01
*** cylopez has joined #tripleo13:01
jaosoriorayoung: are you there?13:02
jaosoriorayoung: didn't see you in #sec-interop13:02
*** tobias_fiberdata has quit IRC13:02
ayoungjaosorior, i'm here..looks like VPN is down13:02
jaosoriorayoung: PMed you13:03
*** jaosorior is now known as jaosorior_mtg13:04
pandajaosorior: ah, yes, almost there .. there, I understand now :) the order of inclusion does not matter, you specified service name in value on the template. Thanks13:04
jaosorior_mtgpanda: no biggie13:04
*** lblanchard has joined #tripleo13:05
*** jcoufal has joined #tripleo13:06
soc_offshardy: I'm here13:07
shardysoc_off: Aha, was confused by your nick :)13:07
shardysoc_off, mcornea: So I'd like to chat about https://bugs.launchpad.net/tripleo/+bug/163309013:07
openstackLaunchpad bug 1633090 in tripleo "composable roles and network isolation" [Medium,Triaged] - Assigned to Steven Hardy (shardy)13:07
mcorneashardy: yep13:08
shardyI'm not sure if you saw my latest question, but what do you expect to happen when you define a custom role, then enable network isolation including e,.g single-nic-vlans?13:08
shardyright now we wire the networks in differently based on knowledge of the primary purpose of the roles13:08
shardywhich can be customized by the deployer, but we know the default services that are deployed13:08
shardywith a user defined role, that's not the case13:09
shardyso what's a reasonable default?  Enable all isolated networks on custom roles?13:09
*** fzdarsky_ has joined #tripleo13:09
shardyOr just allow the default as it is now, which is to wire everything in via the ctlplane and let the deploy decide what isolation is appropriate?13:09
trozetshardy, rook, anbu: I'm trying to understand the eariler discussion.  What is the problem?13:09
shardytrozet: the haproxy config for odl doesn't seem to consider that there's only one node running the odl api13:10
shardytrozet: has it been tested with more than one controller?13:10
soc_offshardy: hard to say, it's simple for usecase of ipv6 but for network iso it's much more though13:10
shardymaybe I'm missing something13:10
openstackgerritKeith Schincke proposed openstack/tripleo-specs: Integrate TripleO with Tendrl for External Storage Deployment/Management  https://review.openstack.org/38763113:10
mcorneashardy: I think enabling all the isolated network could work, but we need nic templates for usual roles like networker node where we exemplify what networks are required for a node running the neutron agents for example13:10
trozetshardy: why would haproxy care how many nodes there are?13:10
soc_offshardy: I'd expect us to split the network iso and let the user to add the correct templates to the role13:10
trozetshardy: yeah we had it working in OPNFV, just buggy because ODL HA is buggy13:10
shardytrozet: because it's going to try to load balance accross three nodes, but only one has the API running?13:11
shardythat was my concern at least13:11
shardywill it just see the port isn't open and ignore the two inactive nodes?13:11
trozetshardy: yeah it should13:11
*** hjensas has quit IRC13:11
shardytrozet: ok, sounds like we're good then, thanks for clarifying13:12
shardysoc_off: so, that's already possible, the user has to make a copy of one of the example templates, and enable whatever networks they want13:12
shardyso I'm trying to figure out how to make the interface easier, without making unsafe assumptions13:12
mcorneashardy: I think ideally would be for the user not having to pass manually what networks it needs enabled but generate this based on which networks the services assinged to the custom role are binding to13:13
trozetshardy: I think its fine becuase we test that exact scenario 24/7 in OPNFV (one ODL instance on controller) and use haproxy13:13
pandachem: spec in https://git.centos.org/blob/rpms!haproxy.git/7463da74923be7492605bf742540ca6abe44d86e/SPECS!haproxy.spec pulls http://www.haproxy.org/download/1.5/src/haproxy-1.5.14.tar.gz, and that rtarball cintains the right code. None of the patches mess with that file13:13
soc_offmcornea: you think sevices should define network requirements?13:14
pandatarball* contains*13:14
shardymcornea: Yeah I think you mentioned that before, it's a nice idea13:14
shardybut the problem is that the ServiceNetMap parameter isn't fixed at the time we create the plan13:14
shardywhich is when the j2 templating happens13:14
shardyso it's going to be hard to generate the per-role nic configs at that point13:14
mcorneasoc_off: they already do it if I'm not mistaken13:14
shardysoc_off: we already define the networks for each service in network/service_net_map.j2.yaml13:15
trozetshardy: did you get any time to look at the refactor patches for ODL?13:15
shardymcornea: Ok I'll see if there's some way to generate the config dynamically13:15
chempanda: arghh ... so it's something else13:16
pandachem: I'm trying to find out what RC 256 means for systemd ...13:17
mcorneashardy: thanks13:17
chempanda: ack13:17
shardytrozet: Sorry not yet, will look today13:17
*** Goneri has joined #tripleo13:17
trozetshardy: thanks13:17
shardytrozet: FYI I've proposed we take a similar approach to your odl implementation for ovn13:18
shardyas that has a similar requirement to run a service on only the bootstrap controller13:18
shardyfeedback on those patches welcome :)13:18
trozetshardy: sure.  link me them or just add me as  a reviewer and I will take a look13:19
*** fzdarsky_ has quit IRC13:19
shardytrozet: thanks, added13:20
shardythey're wip but I'm trying to define a pattern we can reuse for all services with this requirement13:20
*** kjw3 has quit IRC13:20
shardythe odl implementation (with your fixes) is pretty close to what we want I think13:20
*** kjw3 has joined #tripleo13:21
*** jrist has quit IRC13:22
*** jrist has joined #tripleo13:22
*** rbowen has joined #tripleo13:23
*** jrist has quit IRC13:24
*** jcoufal has quit IRC13:26
*** jrist has joined #tripleo13:30
*** tremble has quit IRC13:31
*** abregman|afk is now known as abregman13:32
*** penick has quit IRC13:33
*** nyechiel has joined #tripleo13:33
*** tremble has joined #tripleo13:33
*** tremble has quit IRC13:33
*** tremble has joined #tripleo13:33
*** jtomasek is now known as jtomasek|afk13:33
chempanda: jaosorior_mtg I think I've got it :) we have a int overflow.  Return code are unsigned int, and 256 is outside the scope, the overflow cut the higher bit and what is left are 0s, which make the exit code 013:33
*** ramishra has joined #tripleo13:35
*** jcoufal has joined #tripleo13:36
EmilienMslagle: it looks like it pass in stable/newton but fail in master?13:37
*** morazi has joined #tripleo13:38
jaosorior_mtgchem: are you sure? Thought an int in C would have more range than that. 'char' would be the one with that range.13:38
slagleEmilienM: what failed in master?13:39
chempanda: jaosorior_mtg http://www.faqs.org/docs/abs/HTML/exit-status.html13:39
slagleEmilienM: oh, the ci patch. looking13:39
slagleEmilienM: it failed during the oc deploy, so i assume is the same issue as everything else for master?13:40
pandachem: you're right, I created a simple c test that returns 25613:40
pandagcerami@redpanda:~/Downloads/haproxy-1.5.14$ echo $?13:40
pandagcerami@redpanda:~/Downloads/haproxy-1.5.14$ cat test.c13:41
pandaint main(void) {13:41
panda    return 256;13:41
EmilienMslagle: checking13:41
chempanda: well I did the same :)13:41
slagleEmilienM: looks like newton is fixed though13:41
EmilienMslagle: yeah :)13:41
slagleEmilienM: will have to get master fixed before we can merge it13:41
pandachem: I cant' believe it /o\13:41
EmilienMbut master was fixed iirc13:41
EmilienMslagle: see http://tripleo.org/cistatus.html13:41
chempanda: jaosorior_mtg  it feels good to think again to those computer science things :)13:42
EmilienMgate-tripleo-ci-centos-7-nonha-multinode passed 1h ago13:42
chempanda: jaosorior_mtg so it's a haproxy bug13:42
EmilienMshardy: on which role heat engine tries to start?13:43
shardyEmilienM: I had two slightly different failures, I'm going to re-test now13:43
shardyI may have got myself confused13:43
shardythanks for the help working through it13:43
slagleEmilienM: 2016-10-18 13:20:26.000 |     mError: Could not start Service[httpd]: Execution of '/usr/bin/systemctl start httpd' returned 1: Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.m13:43
slaglethat is how it failed13:44
EmilienMshardy: yeah I'm very surprised it doens't work13:44
EmilienMslagle: ok this problem is fixed13:44
EmilienMslagle: it was a fix in puppet-tripleo13:44
slaglei'll recheck i guess13:44
EmilienMthat is in consistent repo now,13:44
*** amoralej|lunch is now known as amoralej13:44
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489213:44
*** morazi has quit IRC13:45
jaosorior_mtgchem: now I got to read it13:46
*** jaosorior_mtg is now known as jaosorior13:46
jaosoriorwas on a meeting13:46
*** ramishra has quit IRC13:47
jaosoriorchem: haha duuude, that's some cool find right there13:47
openstackgerritMerged openstack/instack-undercloud: Run ceilometer api under wsgi on undercloud  https://review.openstack.org/38548813:47
openstackgerritMerged openstack/tripleo-heat-templates: Enable proxy headers parsing for Neutron  https://review.openstack.org/38726613:47
pandadamn, I wass triggered by 255 + 1 at first, but not triggered enough :)13:47
chemjaosorior: yes, it feels like fresh air after hour of puppet or heat debugging :)13:48
EmilienMchem: you love debugging, say it :P13:48
chemEmilienM: hehe ... more than love, it's passion all the way :)13:50
EmilienMteam: weekly meeting in 10 min13:51
beaglesit's not so much an overflow, but the only the 8 lsb are used for exit status13:53
chempanda: jaosorior I didn't find quickly how to submit a bug on upstream haproxy and I have to go for 1 hour now.  If you could give me update of where you filled that bug so that I can follow it downstream for osp, that would be appreciated.13:53
pandachem: the tripleo bug ?13:54
pandachem: https://bugs.launchpad.net/tripleo/+bug/163446813:54
openstackLaunchpad bug 1634468 in tripleo "periodic nonha job fails to install undercloud" [High,Confirmed]13:54
chembeagles: yep right it just chop of the leftmost bit :)13:55
EmilienMplease add off items here https://etherpad.openstack.org/p/tripleo-meeting-items13:55
beagleschem fun and games.. it's what we get for using higher level languages :)13:55
pandabeagles: since bind returns -1 on error, next thign is trying to find what returns that 256 ... :)13:55
*** cdearborn has joined #tripleo13:56
pandaanyway, this was fun :)13:56
openstackgerritJohn Fulton proposed openstack/tripleo-specs: Integrate TripleO with Tendrl for External Storage Deployment/Management  https://review.openstack.org/38763113:57
chempanda: which shouldn't happen by the way :)13:57
chempanda: so I guess that the signed -1 int is casted to 256 and then returned as 013:58
*** morazi has joined #tripleo13:58
*** tiswanso has quit IRC13:58
pandachem: whoa, crazy jumps :)13:58
akrivokajtomasek|afk: did you resolve the deployment syntax error? I'm getting the same error even with https://review.openstack.org/#/c/386716/13:58
chempanda: yeah ... I'm about to fill a paper with 0 and 1 ... but I have to leave.  See you in 1 hour13:59
pandachem: enjoy13:59
*** ramishra has joined #tripleo14:01
mwhahahalooks like something broken in the python world, http://logs.openstack.org/23/387523/1/check/gate-instack-undercloud-docs-ubuntu-xenial/b9e8e90/console.html#_2016-10-18_13_56_16_98656414:03
openstackLaunchpad bug 1634505 in tripleo "Post-deployment config for UI" [Critical,Triaged] - Assigned to Ana Krivokapić (akrivoka)14:03
slagleEmilienM: now the delorean setup is failing14:03
slagleon https://review.openstack.org/#/c/387750/14:03
slagleprobably bitten by some new release on pip14:04
EmilienMslagle: :( damn it14:04
*** akuznetsov has joined #tripleo14:04
EmilienMslagle: after tripleo meeting I go to airport, I'll catch-up later.14:05
*** limao has joined #tripleo14:06
-openstackstatus- NOTICE: We are away of pycparser failures in the gate and working to address the issue.14:06
EmilienMjaosorior: please backport https://review.openstack.org/#/c/385488/214:07
EmilienMand tag rc3 in gerrit topic14:07
EmilienMi'll review it later14:07
slagleEmilienM: see openstackstatus ^^14:07
slagleit's killing everyone14:07
EmilienMslagle: ah cool14:07
*** jprovazn has quit IRC14:07
slaglei think you meant "uncool" :)14:07
slaglebut yea14:08
*** tiswanso has joined #tripleo14:08
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Run ceilometer api under wsgi on undercloud  https://review.openstack.org/38803514:09
jaosoriorEmilienM: done14:09
*** lucasagomes is now known as lucas-hungry14:12
*** florianf has quit IRC14:14
*** jaosorior has quit IRC14:16
openstackgerritJohn Fulton proposed openstack/tripleo-specs: Integrate TripleO with Tendrl for External Storage Deployment/Management  https://review.openstack.org/38763114:19
*** akuznetsov has quit IRC14:20
*** abregman is now known as abregman|afk14:22
*** mgrohar has quit IRC14:22
ansiwenshardy, EmilienM: so, when I wanna add the ec2api profile to puppet-tripleo, I have to add a section to profile/base/keystone.pp as well, right? are there any other places, where I usually have to add something along with a new service?14:25
*** tiswanso has quit IRC14:27
*** ramishra has quit IRC14:28
*** ramishra_ has joined #tripleo14:28
*** limao has quit IRC14:28
*** zoliXXL is now known as zoli|mtg14:29
shardyansiwen: you may also need to add it to the haproxy manifest if you want it to be accessible via the VIPs14:30
fultonjshardy, EmilienM: i'm hoping to get https://review.openstack.org/#/c/387631/ discussed at summit. Any feedback you have is appreicated. thanks.14:30
*** paramite has quit IRC14:30
shardyfultonj: FYI the tripleo meeting is happening right now in #openstack-meeting-alt14:31
shardyyou're welcome to mention it there (I'll check it out also)14:31
fultonjshardy: thanks. joining14:31
fultonjgfidente: ^14:32
*** abregman|afk is now known as abregman14:33
openstackgerritayoung proposed openstack/puppet-tripleo: Optionally include mod auth mellon for apache  https://review.openstack.org/35994214:34
*** flepied1 has joined #tripleo14:34
*** flepied has quit IRC14:36
openstackgerritGuil Barros proposed openstack/tripleo-heat-templates: Fix api_extensions_path in neutron-opencontrail environment  https://review.openstack.org/38804714:38
jpichhonza: could you pop up on #openstack-release? We're having issues with the upload to npm thing again, could use your input :-)14:41
*** rajinir has joined #tripleo14:44
*** phpcodemonkey has quit IRC14:46
*** anshul has quit IRC14:49
*** yamahata has joined #tripleo14:50
*** apetrich has quit IRC14:53
jpichhonza: Thanks for jumping in so quickly, that was super helpful as usual :)14:54
pradktrown, in quickstart, i see newton and stable/newton .. is there a difference between the two? wrt packages?14:55
honzajpich: np!  i might bug you later for some high level info on how releases are done so i can do this right :)14:56
*** waleedm has quit IRC14:57
trownpradk: ya, stable/newton refers to RDO stable packages14:57
jpichhonza: Sure, no problems! It's changed a lot since the last time I was around, we can piece together our broken pieces of knowledge and figure it out :-)14:57
*** tremble has quit IRC15:02
*** zoli|mtg is now known as zoli|brb15:02
*** paramite has joined #tripleo15:07
*** penick has joined #tripleo15:11
*** sudipto_ has quit IRC15:12
*** sudipto has quit IRC15:12
*** lucas-hungry is now known as lucasagomes15:12
*** dtrainor has quit IRC15:14
*** dtrainor has joined #tripleo15:15
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add IPv6 coverage using existing jobs  https://review.openstack.org/38251515:15
*** abregman has quit IRC15:15
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add IPv6 coverage using existing jobs  https://review.openstack.org/38251515:16
*** mcornea has quit IRC15:21
*** skramaja has quit IRC15:24
*** aufi has quit IRC15:25
*** jlinkes has quit IRC15:27
*** zoli|brb is now known as zoli15:27
*** zoli is now known as zoliXXL15:27
*** tiswanso has joined #tripleo15:28
*** saneax is now known as saneax-_-|AFK15:33
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart: Pin pycparser  https://review.openstack.org/38808215:39
akrivokadtrainor: so I'm still getting this error when trying a deploy https://paste.fedoraproject.org/454944/76791684/raw/15:42
dtrainori saw this somewhere, hold on15:42
openstackLaunchpad bug 1633713 in tripleo "puppet-ntp is breaking ci" [Critical,Fix released]15:43
*** chandankumar has quit IRC15:44
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart: Pin pycparser  https://review.openstack.org/38808215:44
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Include ceilometer in swift proxy pipeline  https://review.openstack.org/37195015:45
openstackgerritDan Trainor proposed openstack/instack-undercloud: Enable SSL on the Undercloud by default  https://review.openstack.org/38808415:46
dtrainorakrivoka, I have not actually ran in to that (yet) but I saw it being talked about15:47
akrivokaI used DIB_INSTALLTYPE_puppet_modules=source, could it be caused by that...?15:48
openstackgerritDan Trainor proposed openstack/instack-undercloud: Enable SSL on the Undercloud by default  https://review.openstack.org/38808415:49
dtrainorIt could, sure, depending on the contents of those modules.  but EmilienM had fixed this yesterday... hrm15:50
*** ohamada has quit IRC15:52
*** b00tcat has quit IRC15:53
*** jlinkes has joined #tripleo15:53
openstackgerritDan Trainor proposed openstack/instack-undercloud: Enable SSL on the Undercloud by default  https://review.openstack.org/38808415:54
dtrainor(sorry for the noise... revising my processes)15:54
*** ebarrera has quit IRC15:56
*** panda is now known as panda|bbl15:58
*** penick has quit IRC15:59
*** numans has quit IRC16:01
*** zoliXXL is now known as zoli|gone16:02
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Disable Neutron DHCP Agent with OVN  https://review.openstack.org/38808816:02
*** zoli|gone is now known as zoli_gone-proxy16:04
*** michapma_alt has quit IRC16:06
*** fzdarsky is now known as fzdarsky|afk16:10
-openstackstatus- NOTICE: pycparser 2.16 released to fix assertion error from today.16:12
*** penick has joined #tripleo16:12
openstackgerritBen Nemec proposed openstack/tripleo-specs: Add promotion-blocker tag  https://review.openstack.org/38809416:13
openstackgerritBen Nemec proposed openstack/tripleo-specs: Remove spec-lite tag  https://review.openstack.org/38809516:13
*** cylopez has quit IRC16:19
*** numans has joined #tripleo16:20
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Add barbican profile rspec testing  https://review.openstack.org/38809916:21
*** jtomasek|afk is now known as jtomasek16:22
openstackgerritDan Trainor proposed openstack/tripleo-ui: Make sample file use %(tenant_id)s macro for heat and swift endpoints instead of suggesting to use hard-coded values  https://review.openstack.org/38810116:23
openstackgerritMerged openstack/diskimage-builder: Add zypper-minimal element  https://review.openstack.org/38157516:24
*** Guest85855 has quit IRC16:25
openstackgerritMerged openstack/diskimage-builder: Add opensuse-minimal element  https://review.openstack.org/38157616:27
*** gfidente is now known as gfidente|afk16:28
*** lmiccini has quit IRC16:28
*** ebarrera has joined #tripleo16:30
*** yamahata has quit IRC16:34
*** myoung is now known as myoung|alt16:35
jtomasekakrivoka: I didn't try to resolve it yet16:35
*** egafford has quit IRC16:39
rookshould tht bugs go against tripleo?16:41
*** ayoung has quit IRC16:42
jpichrook: Yep, with the tripleo-heat-templates tag if possible16:43
*** jpich has quit IRC16:44
EmilienMpanda|bbl: any progress on the haproxy error?16:44
openstackgerritChristopher Brown proposed openstack/tripleo-docs: Clarify stable branch image build exports  https://review.openstack.org/38811216:44
panda|bblchem: ^ were you able to open a bug to haproxy, or should I look into it ?16:45
*** abregman has joined #tripleo16:47
*** ctayal has joined #tripleo16:49
panda|bblEmilienM: the haproxy part should be worked around with the ceilo patch, which is not passing for the multinode job erroor, but I heard that pycparser 2.16 was released and that should work now. The error in haproxy itself, may take longer to fix. So, not many progress. I'll launch a recheck to the ceilo patch16:50
*** rcernin has quit IRC16:51
EmilienMslagle, rbrady, d0ugal : have you seen this mistral error when upgrading an undercloud from newton to ocata? http://logs.openstack.org/50/387750/5/check/gate-tripleo-ci-centos-7-undercloud-upgrades-nv/d186ef5/console.html#_2016-10-18_16_49_14_57533116:51
d0ugalEmilienM: Yeah16:51
EmilienMpanda|bbl: yes we need James's patch first16:51
d0ugalEmilienM: fixed by https://review.openstack.org/#/c/386558/16:52
*** cylopez has joined #tripleo16:52
EmilienMpanda|bbl: when https://review.openstack.org/#/c/387750/ is merged, we can hug slagle and do recheck16:52
EmilienMd0ugal: excellent new. you have a fix for everything dude.16:52
d0ugalEmilienM: lol, I wish16:52
EmilienMd0ugal: you also break everything.16:52
EmilienMbut that's ok.16:52
d0ugalEmilienM: :(16:52
EmilienMd0ugal: :P16:52
d0ugalIf I didn't break it, what would I fix?16:52
panda|bblEmilienM: ack, request for a hug to slagle at the summit queued.16:53
EmilienMd0ugal: :)16:53
EmilienMpanda|bbl: well, you can try :P16:53
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Disable Neutron DHCP Agent with OVN  https://review.openstack.org/38808816:55
*** masco has quit IRC16:57
*** bana_k has joined #tripleo16:59
*** dsneddon_ is now known as dsneddon17:01
rookbeagles: so OS::TripleO::Services::NeutronDhcpAgent: OS::Heat::None would replace the dhcp line i have?17:01
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Fix api_extensions_path in neutron-opencontrail environment  https://review.openstack.org/38804717:01
*** dtantsur is now known as dtantsur|afk17:01
*** derekh has quit IRC17:01
trozetdsneddon: so everry recheck fails on gate-tripleo-ci-centos-7nonha-multinode17:02
*** chandankumar has joined #tripleo17:02
trozetdsneddon, EmilienM: i see no patch has been merged into stable/newton since Friday - i think the branch is broken17:02
dsneddontrozet, Was afraid of that, although I didn't think that a recheck would hurt anything.17:03
beaglesrook: it would go in the resource_registry section actually - sorry should've mentioned it :)17:03
trozetdsneddon, EmilienM: whatever was fixed for master CI, are we sure that was backported into stable/newton?17:03
panda|bblchem: I think the only way is to write to the mailing list17:03
leifmadsenso question around TripleO documentation -- should I be attempting to keep information on previous versions (and if so, how far back), or just update things to match what is current?17:04
*** pkovar has quit IRC17:05
mwhahahatrozet: i thought they identified that the overcloud was getting the wrong packages (ocata vs newton) and that it should be fixed17:05
slagleleifmadsen: we've used the annotations to distinguish differences like that in the past17:05
jistroneliner with green CI waiting for another +2 :) https://review.openstack.org/#/c/387702/17:06
slagleleifmadsen: if those become too much maintenance, we could always consider branching the docs17:06
*** jpena is now known as jpena|off17:06
trozetmwhahaha: EmilienM sent an email with a patch to RDO that specified a pkg version.  But I don't think that was the only fix.  After that was merged i rechecked and it still failed.  Then yesterday everything was magically working and I'm not sure if something else was changed17:06
EmilienMtrozet: we need https://review.openstack.org/#/c/387750/ and all will be fixed.17:06
*** chandankumar has quit IRC17:07
Slowerhow come I'm getting:17:07
EmilienMslagle: passing multinode job :)17:08
SlowerException creating plan: Failed to run action [action_ex_id=d9dbab58-cfd1-4d8f-9640-9b11c174d3d7, action_cls='<class 'mistral.actions.action_factory.GeneratePasswordsAction'>', attributes='{}', params='{u'container': u'overcloud'}'] Environment not found [name=tripleo.undercloud-config]17:08
slagleEmilienM: cool17:08
EmilienMslagle: once it pass, do you mind if I approve it?17:08
slagleEmilienM: go ahead17:08
leifmadsenslagle: ok thanks... basically I'm trying to update the docs to match Newton on the basic deployment CLI screen, but I don't really have the ability to test across multiple versions (I could... but taking an hour each time to spin up an undercloud is a bit burdensome :))17:08
leifmadsenI'd prefer maybe the docs to just match whatever is most current?17:09
leifmadsenI see there are annotations, but it kind of makes the docs a little messy...17:09
trozetEmilienM:  it's failing almost everything17:09
EmilienMtrozet: branch is not broken, it's an infra problem. We're about to fix it.17:09
Sloweranyone seen that before?17:09
EmilienMtrozet: you might have missed some episodes17:09
slagleleifmadsen: yea, i think matching latest is the minimum we should strive for17:09
leifmadsenwonder if it's maybe better to just keep branches and then update each release, and then at least you just have to update master?17:09
trozetEmilienM: ok can you drop me a line when it is done?17:09
EmilienMtrozet: a lot of outages today, mainly infra related17:09
slagleleifmadsen: yep, branches are another option17:09
leifmadsenslagle: for example, docs still use the neutron CLI command, when you can do all the same stuff in the unified CLI (which I think is the preferred)17:09
EmilienMtrozet: just follow https://review.openstack.org/#/c/387750/ and it merges, all can be rechecked17:10
trozetEmilienM: kk17:10
leifmadsenI mean I guess i could technically try to spin up multiple underclouds and then test and update everything in one fell swoop, then at least each time after that it's just updating it to "latest release"17:10
leifmadsenactually, I should be able to do that relatively easily if I just create multiple local VMs... hrmmm... ok leave it with me17:11
*** chandankumar has joined #tripleo17:12
*** chandankumar has quit IRC17:14
*** chandankumar has joined #tripleo17:14
*** trown is now known as trown|lunch17:15
*** rasca has quit IRC17:16
*** yamahata has joined #tripleo17:18
*** bana_k has quit IRC17:22
akrivokaso hmmm... is everybody installing undercloud with DIB_INSTALLTYPE_puppet_modules=source? cause it just failed for me without it.17:23
*** numans has quit IRC17:26
leifmadsenI didn't yesterday...17:29
leifmadsenand it seemed to work?17:29
*** penick has quit IRC17:32
*** mgould is now known as mgould|afk17:32
*** yamahata has quit IRC17:32
*** lucasagomes has quit IRC17:33
*** jistr has quit IRC17:33
*** flaper87 has quit IRC17:33
*** igorbelikov has quit IRC17:33
*** dobson has quit IRC17:33
*** EmilienM has quit IRC17:33
*** SpamapS has quit IRC17:33
*** Jokke_ has quit IRC17:33
*** weshay has quit IRC17:33
*** patrickeast has quit IRC17:33
*** greghaynes has quit IRC17:33
*** Ng has quit IRC17:33
*** jpena|off has quit IRC17:33
*** jroll has quit IRC17:33
*** zoli_gone-proxy has quit IRC17:33
*** Jokke_ has joined #tripleo17:33
*** Jokke_ has quit IRC17:33
*** Jokke_ has joined #tripleo17:33
*** greghaynes has joined #tripleo17:33
*** weshay has joined #tripleo17:34
*** SpamapS has joined #tripleo17:34
*** zoli_gone-proxy has joined #tripleo17:34
*** lucasagomes has joined #tripleo17:34
*** jistr has joined #tripleo17:34
*** dobson has joined #tripleo17:34
*** jpena|off has joined #tripleo17:34
*** EmilienM has joined #tripleo17:35
*** EmilienM has quit IRC17:35
*** EmilienM has joined #tripleo17:35
*** jroll has joined #tripleo17:35
*** athomas has quit IRC17:38
*** bana_k has joined #tripleo17:38
*** SpamapS has quit IRC17:39
*** SpamapS has joined #tripleo17:39
*** patrickeast has joined #tripleo17:39
*** igorbelikov has joined #tripleo17:41
*** akrivoka has quit IRC17:44
*** Ng has joined #tripleo17:45
*** ChanServ sets mode: +v Ng17:45
*** apetrich has joined #tripleo17:49
EmilienMslagle, shardy: I'm moving bugs from rc3 to ocata-1 https://launchpad.net/tripleo/+milestone/newton-rc317:50
EmilienMwith backport tag17:51
*** dprince has quit IRC17:52
*** egafford has joined #tripleo17:52
*** ebarrera has quit IRC17:54
EmilienMplease use this tag (with ocata-1 target) if you have bugfixes that needs backports17:54
*** cylopez has quit IRC17:55
*** penick has joined #tripleo17:55
*** jprovazn has joined #tripleo17:57
*** athomas has joined #tripleo18:00
*** yamahata has joined #tripleo18:00
openstackgerritMarkos Chandras proposed openstack/diskimage-builder: elements: simple-init: Remove SUSE interfaces  https://review.openstack.org/38815818:03
*** dprince has joined #tripleo18:07
gregworkfultonj: hi john, was wondering if you had any further thoughts on those files I sent you yesterday18:08
*** ctayal has quit IRC18:10
openstackgerritOliver Walsh proposed openstack/tripleo-specs: Real-time compute nodes  https://review.openstack.org/38816218:15
*** ctayal has joined #tripleo18:18
*** abregman is now known as abregman|afk18:19
*** jcoufal_ has joined #tripleo18:20
*** jcoufal has quit IRC18:21
fultonjgregwork: let me look at that. in a minute18:21
gregworkfultonj: much appreciated18:21
fultonjgregwork: at first blush it looks like your THT18:22
larsksI've have a deployment in which it looks as if my parameter_defaults was ignored, but I'm not sure.  Can I interrogate a heat stack to verify that the parameter_defaults are what I expect them to be?18:23
gregworkfultonj: ?18:23
fultonjgregwork: your templates might be the problem18:23
*** trown|lunch is now known as trown18:23
gregworkwhat do you think is the issue18:24
gregworklike an issue with my configuration of the templates, or the templates themselves18:24
fultonjyour config of the templates. let me verify first18:24
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Don't use ssh-keygen -A for init scripts  https://review.openstack.org/37898518:25
*** tiswanso has quit IRC18:26
*** tiswanso has joined #tripleo18:26
*** tiswanso has quit IRC18:27
*** milan has quit IRC18:32
fultonjgregwork: have any variations of this install worked?18:34
fultonjgregwork: e.g. if you install without ceph does everything else work?18:35
gregworkeven this install deploys correctly18:35
gregworkits just the ceph clients are not configured to use the ceph server storage18:35
*** amoralej is now known as amoralej|off18:36
gregworkso compute/control have default/swift configuration18:36
gregworkthe issue here is that the overcloud deploy fully prepares and deploys the cephstorage-X nodes, configures the controler nodes as monitors18:36
gregworkhowever the cinder/glance configs are never done18:36
gregworkso the storage is inaccessible / unused18:36
gregworkwhen i do a volume create in horizon it create it via swift18:37
gregworkwell it creates it as an lvm18:37
*** jlinkes has quit IRC18:37
gregworkif i grep 'rbd' in /etc/cinder/cinder.conf i get nothing18:37
gregworkwell i get commented lines18:37
*** jlinkes has joined #tripleo18:38
fultonjgregwork: i understand, i guess tripleo didn't suceed though heat reported the stack is ok18:38
gregworkim at a loss on what i can do here18:38
*** nyechiel has quit IRC18:38
fultonjgregwork: version18:40
fultonjgregwork: sorry what version are you using?18:40
*** flaper87 has joined #tripleo18:46
*** flaper87 has joined #tripleo18:46
jtomasekrbrady: I've updated the bug with hopefully relevant info on the status of deployment created by GUI https://bugs.launchpad.net/tripleo/+bug/163450518:47
openstackLaunchpad bug 1634505 in tripleo "Post-deployment config for UI" [Critical,Triaged] - Assigned to Ana Krivokapić (akrivoka)18:47
*** jobewan has joined #tripleo18:50
rbradyjtomasek: thanks.18:51
*** dsariel has quit IRC18:52
*** jcoufal_ has quit IRC18:55
trownlarsks: `openstack stack environment show overcloud`18:56
larskstrown, thanks! I had finally found my way there, and confirmed that indeed my parameter defaults looked screwy...but not for obvious reasons.18:57
*** jcoufal has joined #tripleo18:57
*** jlinkes has quit IRC18:59
fultonjgregwork: i don't know the version you have. looks like mitka judging by your storage-environment.yaml. i see you changed those files a lot which is probably where the issue is. i recommend using the shipped templates and only passing your own overrides. here's an example of what i think you want.19:01
fultonjin the above ~/templates is just a symlink to /usr/share/openstack-tripleo-heat-templates19:02
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: We no longer need to copy old instack.qcow2 on image promote  https://review.openstack.org/38818019:02
*** newmember has joined #tripleo19:10
*** jcoufal has quit IRC19:12
*** jcoufal has joined #tripleo19:13
*** fultonj has quit IRC19:22
*** ctayal has quit IRC19:23
*** phpcodemonkey has joined #tripleo19:29
*** fultonj has joined #tripleo19:31
*** lucasagomes is now known as lucas-afk19:32
*** dsariel has joined #tripleo19:37
*** Goneri has quit IRC19:38
*** chandankumar has quit IRC19:39
*** abregman|afk is now known as abregman19:39
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart: Remove images/publish role  https://review.openstack.org/38818919:41
panda|bblchem: https://www.mail-archive.com/haproxy@formilux.org/msg23896.html19:42
*** panda|bbl is now known as panda19:42
*** jcoufal has quit IRC19:47
openstackgerritMerged openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775019:47
*** jcoufal has joined #tripleo19:48
*** snecklifter has joined #tripleo19:49
*** egafford has quit IRC19:50
snecklifterHello, small docs change, review appreciated, thanks: https://review.openstack.org/#/c/388112/19:51
*** gfidente|afk is now known as gfidente19:52
openstackgerritMerged openstack/tripleo-heat-templates: Modify the constraint to allow single quote for DPDK core list param  https://review.openstack.org/38767619:52
openstackgerritMerged openstack/tripleo-heat-templates: Fix api_extensions_path in neutron-opencontrail environment  https://review.openstack.org/38770219:53
openstackgerritMerged openstack/tripleo-heat-templates: Set nova service_name via t-h-t  https://review.openstack.org/38651919:55
openstackgerritMerged openstack/puppet-tripleo: Remove explicit service_name setting from nova manifest  https://review.openstack.org/38652219:55
*** ramishra_ has quit IRC19:57
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add IPv6 coverage using existing jobs  https://review.openstack.org/38251519:58
*** ramishra has joined #tripleo19:59
*** michapma_alt has joined #tripleo20:01
*** cdearborn has quit IRC20:02
*** cylopez has joined #tripleo20:03
*** cdearborn has joined #tripleo20:05
*** ctayal has joined #tripleo20:13
tdasilvatrown: hello, i'm trying to deploy triple-quickstart and ran into an error "Unable to get valid context for heat-admin" when trying to ssh to overcloud node from undercloud20:19
tdasilvawas wondering if you have any tips?20:19
trownhmm never seen that20:19
*** ramishra has quit IRC20:20
trowntdasilva: how are you doing ssh to overcloud from the undercloud?20:20
*** jayg is now known as jayg|g0n320:20
tdasilvatrown: ssh heat-admin@
*** jeckersb is now known as jeckersb_gone20:20
trowntdasilva: hmm and is what shows up in `nova list`?20:21
tdasilvatrown: correct20:21
tdasilvatrown: http://paste.fedoraproject.org/455176/14768221/20:22
*** ramishra has joined #tripleo20:22
tdasilvatrown: i was trying to follow directions here: http://tripleo.org/troubleshooting/troubleshooting.html#debugging-using-heat and got stuck on that20:23
trowntdasilva: so the deploy failed?20:25
tdasilvatrown: yeah :(20:25
tdasilvatrown: i was trying to figure out what happened.20:26
tdasilvait fails on AllNodesDeploySteps20:26
*** Goneri has joined #tripleo20:27
trowntdasilva: hmm, nothing useful in `openstack stack failures list overcloud`?20:27
trownlooks like that command needs to make its way into the troubleshooting doc20:27
tdasilvanot sure if this is any helpful, but: https://da.gd/ZNDa20:27
tdasilvalet me try that20:27
trownshould at least give you puppet output, and failure is puppet "Deployment exited with non-zero status code: 6"20:28
tdasilvatrown: https://da.gd/zcMPx20:29
trownlooks like keystone never came up20:30
trownodd that you wouldnt be able to ssh in... that is pretty late in the deploy (in the middle but well after provisioning)20:31
tdasilvatrown: yeah, google seems to point to a selinux issue ???20:32
trowntdasilva: hmm, what images are you using?20:33
tdasilvatrown: i'm new to tripleo, so i'll try to answer this correctly. basically i can quickstart with "--release master-tripleo-ci"20:34
tdasilvatrown: does that answer your question?20:34
tdasilvas/i can/i ran20:35
*** akuznetsov has joined #tripleo20:35
trowntdasilva: indeed, that means it runs with images from tripleo-ci periodic job20:35
tdasilvaok, that's how I understood20:35
trowntdasilva: which should not have selinux enforcing20:35
trownpanda: was that haproxy issue you were seeing manifesting in keystone not starting (because httpd did not start)?20:36
*** akuznetsov has quit IRC20:39
*** akrzos has quit IRC20:40
*** jprovazn has quit IRC20:41
pandatrown: it was not keystone, but trying to get a token from haproxy port of ceilometer20:41
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: DO NOT MERGE fake periodic job to update cache to Ocata  https://review.openstack.org/38252120:42
*** akrzos has joined #tripleo20:42
trowntdasilva: I have to run, but could you try a run with "--release master" to isolate whether the issue is with tripleo-ci image?20:44
tdasilvatrown: absolutely, i need to go to, but will try that and report later20:45
tdasilvatrown: thanks for your time20:45
openstackgerritwes hayutin proposed openstack/tripleo-specs: update the index for ocata  https://review.openstack.org/38820820:46
*** trown is now known as trown|outtypewww20:46
*** kjw3 has quit IRC20:48
*** kjw3 has joined #tripleo20:48
*** ayoung has joined #tripleo20:51
*** mcornea has joined #tripleo20:52
openstackgerritmathieu bultel proposed openstack/tripleo-quickstart: Upgrade job dont need the overcloud workers settings  https://review.openstack.org/38821120:54
EmilienMslagle: can you review https://review.openstack.org/#/c/386042/ please?20:55
slagleEmilienM: yes. i've been meaning to give it a try, but not sure when I will get to it at this point20:59
openstackgerritMerged openstack/diskimage-builder: Add option to be able to run_functests.sh in parallel  https://review.openstack.org/34506521:03
*** lblanchard has quit IRC21:06
*** ramishra has quit IRC21:07
*** ramishra has joined #tripleo21:09
openstackgerritTim Rozet proposed openstack/puppet-tripleo: Fixes missing ODL ML2 Authentication info  https://review.openstack.org/38682021:09
EmilienMcan I have a +2 on this backport please? https://review.openstack.org/#/c/387678/ I'll approve it once CI pass on OVB too21:10
openstackgerritTim Rozet proposed openstack/puppet-tripleo: Fixes missing ODL ML2 Authentication info  https://review.openstack.org/38682021:11
EmilienMalso please look https://review.openstack.org/#/c/386558/21:12
*** mcornea has quit IRC21:15
*** saneax-_-|AFK has quit IRC21:17
*** snecklifter has quit IRC21:18
*** r-mibu has quit IRC21:19
*** r-mibu has joined #tripleo21:19
*** saneax-_-|AFK has joined #tripleo21:19
thrashEmilienM: done21:23
*** maticue has quit IRC21:24
*** Goneri has quit IRC21:25
*** hjensas has joined #tripleo21:27
*** hjensas has quit IRC21:27
*** hjensas has joined #tripleo21:27
*** ramishra has quit IRC21:27
*** nyechiel has joined #tripleo21:27
*** abregman is now known as abregman|afk21:28
*** dprince has quit IRC21:28
*** ramishra has joined #tripleo21:29
*** itlinux has joined #tripleo21:30
openstackgerritSven Anderson proposed openstack/puppet-tripleo: Implement Nova ec2api profile  https://review.openstack.org/33656221:31
*** jcoufal_ has joined #tripleo21:32
*** jcoufal has quit IRC21:33
*** akuznetsov has joined #tripleo21:36
*** jcoufal has joined #tripleo21:37
*** jcoufal_ has quit IRC21:38
*** ccamacho is now known as ccamacho|out21:39
*** akuznetsov has quit IRC21:40
*** thrash is now known as thrash|g0ne21:41
*** nyechiel has quit IRC21:42
*** cylopez has quit IRC21:51
*** kjw3 has quit IRC21:52
*** hjensas has quit IRC21:56
*** jcoufal has quit IRC22:00
*** cdearborn has quit IRC22:03
*** fultonj has quit IRC22:05
*** abregman|afk is now known as abregman22:07
*** ctayal has quit IRC22:08
*** morazi has quit IRC22:13
openstackgerritMerged openstack/python-tripleoclient: Allow overriding CLI parameter defaults with env files  https://review.openstack.org/38791922:15
openstackgerritMerged openstack/tripleo-heat-templates: Move trunk service plugin to the proper list  https://review.openstack.org/38709622:15
openstackgerritMerged openstack/tripleo-common: Refresh DeployIdentifier as well on update  https://review.openstack.org/38753122:15
openstackgerritMerged openstack/tripleo-heat-templates: Pass heat domain admin password to keystone  https://review.openstack.org/38728922:15
openstackgerritMerged openstack/instack-undercloud: Tune number of workers for undercloud  https://review.openstack.org/38752322:16
openstackgerritMerged openstack/tripleo-heat-templates: heat-api-cfn endpoint is created to RegionOne instead of regionOne  https://review.openstack.org/38669722:16
openstackgerritMerged openstack/instack-undercloud: Run ceilometer api under wsgi on undercloud  https://review.openstack.org/38803522:16
openstackgerritMerged openstack/instack-undercloud: Only create the Mistral config env if it doesn't exist  https://review.openstack.org/38655822:16
*** apetrich has quit IRC22:17
*** apetrich has joined #tripleo22:19
openstackgerritgreghaynes proposed openstack/diskimage-builder: WIP nodepool element  https://review.openstack.org/38824122:22
*** penick has quit IRC22:25
*** ctayal has joined #tripleo22:25
openstackgerritgreghaynes proposed openstack/diskimage-builder: WIP nodepool element  https://review.openstack.org/38824122:26
openstackgerritMarkos Chandras proposed openstack/diskimage-builder: elements: runtime-ssh-host-keys: Add openssh-client mapping for SUSE  https://review.openstack.org/38824222:26
*** newmember has quit IRC22:32
*** bfournie has quit IRC22:33
openstackgerritOpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements  https://review.openstack.org/37512522:36
*** akuznetsov has joined #tripleo22:36
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements  https://review.openstack.org/38114822:37
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Ensure all HostsDeployments finish before puppet  https://review.openstack.org/38798222:37
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Split out hosts config deployment  https://review.openstack.org/38824522:37
*** rhallisey has quit IRC22:39
*** akuznetsov has quit IRC22:41
*** gfidente has quit IRC22:47
*** pmannidi has joined #tripleo22:54
openstackgerritMerged openstack/tripleo-heat-templates: Enable object versioning in Swift proxy  https://review.openstack.org/38633022:59
*** yamahata has quit IRC23:04
*** yamahata has joined #tripleo23:05
*** bfournie has joined #tripleo23:05
rookbeagles: thanks, fixed that patch23:08
rookbeagles: i suppose I can do the same thing with l3 and metadata if i need to23:14
*** abregman is now known as abregman|afk23:14
*** dsariel has quit IRC23:14
*** rajinir has quit IRC23:16
*** ctayal has quit IRC23:19
*** jobewan has quit IRC23:23
openstackgerritMerged openstack/python-tripleoclient: Save the result of direct action calls in Mistral  https://review.openstack.org/38173923:23
openstackgerritMerged openstack/tripleo-common: Port Scale Down Functionality into Workflow  https://review.openstack.org/38767823:24
openstackgerritMerged openstack/puppet-tripleo: Set memcached_servers for nova API  https://review.openstack.org/38762423:27
*** newmember has joined #tripleo23:29
*** phpcodemonkey has quit IRC23:29
*** phpcodemonkey has joined #tripleo23:30
*** phpcodemonkey has quit IRC23:30
*** phpcodemonkey has joined #tripleo23:30
*** phpcodemonkey has joined #tripleo23:31
*** phpcodemonkey has quit IRC23:32
*** phpcodemonkey has joined #tripleo23:32
*** phpcodemonkey has quit IRC23:32
*** ctayal has joined #tripleo23:32
*** phpcodemonkey has joined #tripleo23:33
*** phpcodemonkey has quit IRC23:33
*** phpcodemonkey has joined #tripleo23:34
*** phpcodemonkey has quit IRC23:34
*** phpcodemonkey has joined #tripleo23:34
*** phpcodemonkey has quit IRC23:35
*** phpcodemonkey has joined #tripleo23:35
*** phpcodemonkey has quit IRC23:36
*** phpcodemonkey has joined #tripleo23:36
*** phpcodemonkey has quit IRC23:36
*** akuznetsov has joined #tripleo23:39
*** akuznetsov has quit IRC23:43
*** yamahata has quit IRC23:48
*** ramishra has quit IRC23:53
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Disable Neutron DHCP Agent with OVN  https://review.openstack.org/38808823:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!