Tuesday, 2017-10-31

*** bobh has quit IRC00:02
openstackgerritMerged openstack/python-tripleoclient stable/pike: Do not return the config download path but only print it  https://review.openstack.org/51460900:07
openstackgerritBrent Eagles proposed openstack/tripleo-common master: Add ansible playbook for configuring octavia  https://review.openstack.org/48745000:10
openstackgerritBrent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper  https://review.openstack.org/51540200:10
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798500:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]00:10
*** akrivoka has quit IRC00:16
*** newmember has joined #tripleo00:21
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: persist memcached logs in /var/log/containers/memcached/memcached.log  https://review.openstack.org/51627100:28
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Set verbosity by default for memcached  https://review.openstack.org/51627000:29
*** akrivoka has joined #tripleo00:31
*** newmember has quit IRC00:32
*** newmember has joined #tripleo00:33
*** bobh has joined #tripleo00:33
*** owalsh_pto has quit IRC00:36
*** noslzzp has quit IRC00:36
openstackgerritMerged openstack/puppet-tripleo stable/ocata: [ocata] Add Puppet package to bindep, for module build  https://review.openstack.org/51594700:38
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Prepare puppet-tripleo 6.5.4  https://review.openstack.org/51513400:38
*** achadha has joined #tripleo00:39
*** bobh has quit IRC00:42
*** ecerquei_ has quit IRC00:43
*** achadha has quit IRC00:45
*** psahoo has joined #tripleo00:45
*** bfournie has joined #tripleo00:46
*** liverpooler has quit IRC00:47
*** bfournie has quit IRC00:48
*** owalsh_ has joined #tripleo00:49
*** newmember has quit IRC00:51
*** owalsh has joined #tripleo00:51
*** newmember has joined #tripleo00:51
*** owalsh- has joined #tripleo00:53
*** owalsh_ has quit IRC00:54
*** owalsh_ has joined #tripleo00:55
*** newmember has quit IRC00:55
*** owalsh has quit IRC00:56
*** newmember has joined #tripleo00:56
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/51163300:58
*** owalsh has joined #tripleo00:58
*** owalsh- has quit IRC00:59
*** newmember has quit IRC01:01
*** owalsh- has joined #tripleo01:01
*** owalsh_ has quit IRC01:01
*** newmember has joined #tripleo01:02
*** yamahata has quit IRC01:04
*** owalsh_ has joined #tripleo01:04
*** owalsh has quit IRC01:04
*** salmankhan has joined #tripleo01:05
*** rhallisey has quit IRC01:07
*** owalsh has joined #tripleo01:07
*** owalsh- has quit IRC01:08
*** itlinux has quit IRC01:09
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798501:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]01:10
*** salmankhan has quit IRC01:10
*** owalsh- has joined #tripleo01:10
*** owalsh_ has quit IRC01:11
EmilienMpabelanger: hey you around?01:14
*** owalsh has quit IRC01:14
EmilienMmaybe late01:14
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Fix standalone ControllerOpenstack vars  https://review.openstack.org/51246601:15
pabelangerEmilienM: I am01:17
pabelangerjust writing emails to mgmt :)01:17
*** bobh has joined #tripleo01:17
*** rlandy has quit IRC01:20
EmilienMpabelanger: wondering why https://review.openstack.org/#/c/515132/ didn't pass01:21
*** psachin has joined #tripleo01:21
*** bfournie has joined #tripleo01:21
EmilienMhttp://logs.openstack.org/32/515132/3/check/puppet-openstack-module-build/c6bfc51/job-output.txt.gz#_2017-10-31_01_13_42_05428601:21
*** psachin has quit IRC01:22
EmilienMwe have bindep in place now, I'm confused01:22
*** psachin has joined #tripleo01:22
pabelangerlooking01:23
pabelangerhttp://logs.openstack.org/32/515132/3/check/puppet-openstack-module-build/c6bfc51/ara/result/12f066a7-3c20-4353-9a82-03680c52dba8/01:24
pabelangersudo: no tty present and no askpass program specified01:24
pabelangerEmilienM: ^01:24
pabelangernow, to see why that is01:24
pabelangerhttp://logs.openstack.org/32/515132/3/check/puppet-openstack-module-build/c6bfc51/job-output.txt.gz#_2017-10-31_01_13_38_24529701:25
pabelangerlooks like revoke-sudo is being used01:25
pabelangerwhich happen just before error01:25
pabelangerhttp://logs.openstack.org/32/515132/3/check/puppet-openstack-module-build/c6bfc51/job-output.txt.gz#_2017-10-31_01_13_42_05428601:25
pabelangeroh, hm01:26
pabelangerthat is also the old zuulv2 build job01:26
*** bobh has quit IRC01:26
pabelangerlet me fine new shiny zuulv3 one01:26
EmilienMyeah it's weird heh01:26
*** amodi has quit IRC01:33
pabelangerEmilienM: okay, we need to land https://review.openstack.org/515980/01:38
pabelangerthen update puppet jobs to use it01:38
*** fragatina has quit IRC02:01
*** apetrich has quit IRC02:03
*** apetrich has joined #tripleo02:04
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798502:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]02:10
*** ooolpbot has quit IRC02:10
*** threestrands has joined #tripleo02:11
*** threestrands has joined #tripleo02:11
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/newton: Prepare puppet-tripleo 5.6.5  https://review.openstack.org/51513202:14
*** rwsu has joined #tripleo02:22
*** catintheroof has joined #tripleo02:24
*** bobh has joined #tripleo02:30
*** gvrangan has joined #tripleo02:31
*** gvrangan has quit IRC02:33
*** gvrangan has joined #tripleo02:33
*** gvrangan has quit IRC02:38
*** akrivoka has quit IRC02:40
*** catintheroof has quit IRC02:43
*** axisys has joined #tripleo02:43
*** axisys has left #tripleo02:45
*** shreshtha has joined #tripleo02:50
*** cshastri has joined #tripleo02:57
*** gvrangan has joined #tripleo03:04
*** bobh has quit IRC03:08
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798503:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]03:10
*** bkopilov has quit IRC03:12
*** bobh has joined #tripleo03:14
*** achadha has joined #tripleo03:16
*** achadha_ has joined #tripleo03:16
*** gvrangan has quit IRC03:18
*** ramishra has joined #tripleo03:19
*** achadha has quit IRC03:20
*** newmember has quit IRC03:22
*** vkmc has quit IRC03:29
openstackgerritSteve Baker proposed openstack/python-tripleoclient master: Use common defaults for image prepare CLI  https://review.openstack.org/51613903:32
*** vkmc has joined #tripleo03:35
*** shreshtha has quit IRC03:36
*** bobh has quit IRC03:37
tonybHow do I work out what overcloud.AllNodesDeploySteps.ComputeDeployment_Step4 is doing and why it appears to "hang"03:38
*** fragatina has joined #tripleo03:49
*** fragatina has quit IRC03:49
*** fragatina has joined #tripleo03:50
*** fragatina has quit IRC03:50
*** fragatina has joined #tripleo03:51
*** achadha_ has quit IRC03:52
*** ykarel has joined #tripleo04:06
*** udesale has joined #tripleo04:06
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172798504:10
openstackLaunchpad bug 1727985 in tripleo "yum-plugin-priorities is not installed in tripleo image when overcloud packages are installed" [Critical,Triaged]04:10
*** ooolpbot has quit IRC04:10
*** tzumainn has quit IRC04:11
*** jidar has quit IRC04:22
*** jidar has joined #tripleo04:22
*** fragatina has quit IRC04:22
*** stee_3 has quit IRC04:23
*** jpena|off has quit IRC04:23
*** pliu has quit IRC04:23
*** lhinds has quit IRC04:23
*** stee_3 has joined #tripleo04:24
*** pliu has joined #tripleo04:25
*** lhinds has joined #tripleo04:25
*** jpena|off has joined #tripleo04:25
*** fragatina has joined #tripleo04:27
*** cshastri has quit IRC04:31
*** bkopilov has joined #tripleo04:38
*** anilvenkata has joined #tripleo04:39
*** skramaja has joined #tripleo04:45
*** ratailor has joined #tripleo04:47
*** janki has joined #tripleo05:06
*** pdeore has joined #tripleo05:09
jaosoriortonyb: that runs puppet/ansible steps; but it could be doing many things05:20
jaosoriortonyb: the best way to figure out exactly what it's doing is to go to a controller node, and check what os-collect-config is doing via journalctl05:21
*** janki has quit IRC05:27
*** janki has joined #tripleo05:27
openstackgerritMerged openstack/tripleo-heat-templates master: nova-placement: switch auth_uri to keystone versionless endpoint  https://review.openstack.org/51473905:28
*** iranzo has joined #tripleo05:32
*** yamahata has joined #tripleo05:35
*** threestrands has quit IRC05:46
*** fragatina has quit IRC05:47
*** achadha has joined #tripleo05:48
*** achadha has quit IRC05:53
*** achadha has joined #tripleo05:56
*** achadha_ has joined #tripleo05:57
*** gkadam has joined #tripleo05:57
*** cshastri has joined #tripleo05:58
*** threestrands has joined #tripleo05:58
*** threestrands has quit IRC05:58
*** threestrands has joined #tripleo05:58
*** jfrancoa has joined #tripleo05:59
*** achadha has quit IRC06:00
*** threestrands has quit IRC06:03
*** dparkes has quit IRC06:14
*** abregman has joined #tripleo06:16
*** yprokule has joined #tripleo06:26
*** agurenko has quit IRC06:27
*** suuuper has joined #tripleo06:28
*** iranzo has quit IRC06:42
*** moshele has joined #tripleo06:49
ratailorjaosorior, you around ?06:52
jaosorioryeah, sup?06:52
*** tosky has joined #tripleo06:53
*** achadha_ has quit IRC06:54
*** iranzo has joined #tripleo06:55
*** iranzo has quit IRC06:55
*** iranzo has joined #tripleo06:55
*** jtomasek has joined #tripleo06:58
tonybjaosorior: Okay, do you mean the undercloud or the node I tagged control and then deployed?06:58
*** rcernin has quit IRC06:59
jaosoriortonyb: the node you tagged control (which is a node in the overcloud)06:59
tonybjaosorior: Thanks07:00
jaosoriortonyb: so, the undercloud has no idea what's actually going on in the overcloud; all it gets is heat events (which is the steps that you see) and further you can see some logs once a certain step completes.07:00
*** jtomasek has quit IRC07:00
jaosoriortonyb: however, if you want to see what's going on in real time in a certain node07:00
tonybjaosorior: it eventually tied out so I'll delete the stack and try again tomorrow07:00
jaosorioryou can ssh into that node, and check the output of journalctl07:00
ratailorshould tls_proxy be set in step4 as done here.07:00
ratailorjaosorior,  ^^ https://review.openstack.org/#/c/509392/6/manifests/profile/base/nova/ec2api.pp07:01
*** dparkes has joined #tripleo07:01
tonybjaosorior: Thanks07:01
jaosoriorratailor: I suggest you start the TLS proxy in step 3 in the bootstrap node and step 4 for every other node.07:01
ratailorWhenever I tried to deploy overcloud with my changes for enabling TLS for ec2api, It got timed out after completing step3.07:01
jaosoriorratailor: this avoids restarts due to the change in the httpd vhost configurations07:01
*** gkadam has quit IRC07:01
*** brault has joined #tripleo07:02
ratailorjaosorior, same as done here https://review.openstack.org/#/c/494657/1/manifests/profile/base/nova/api.pp07:02
jaosoriorratailor: something like this https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/neutron/server.pp#L11607:02
*** marios has joined #tripleo07:02
jaosoriorratailor: yep07:02
ratailorjaosorior, Thanks!07:02
jaosoriorratailor: if you want help further help, I could take a look at your deployment to try to figure out why it's timing out07:03
ratailorsure,07:03
ratailorCan I try this, if it fails again, I will provide you access to my system. Does it sounds good ?07:04
jaosoriorratailor: sure! let me know07:04
ratailorjaosorior, sure07:05
*** spectr has joined #tripleo07:08
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Force memcached container log to file  https://review.openstack.org/51620607:08
jaosoriormandre: what about this? ^^07:08
*** agurenko has joined #tripleo07:09
*** yamahata has quit IRC07:14
*** anshul has joined #tripleo07:17
*** jprovazn has joined #tripleo07:17
*** pcaruana has joined #tripleo07:17
mandrejaosorior: très bien07:18
jaosoriormerci beaucoup07:19
*** tosky has quit IRC07:19
mandre;-)07:19
jaosoriormandre: who else reviews adding containers to tripleo-common ?07:20
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/51599107:20
*** dsneddon has joined #tripleo07:20
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui stable/pike: Imported Translations from Zanata  https://review.openstack.org/51599007:20
jaosoriormandre: so I can poke them about https://review.openstack.org/#/c/467074/ :P07:20
mandrejaosorior: I'm not a snitch!07:24
jaosoriorlol07:25
mandrejaosorior: you can give a try with bandini or jistr|mtgs if he's around, any tripleo core who understands well enough the containers07:25
*** paramite has joined #tripleo07:26
jaosoriorthanks bandini !07:29
bandinio/07:29
bandinijaosorior: can I pick your brain later this morning for a) a completely weird heat issue and b) the tls pcmk upgrade stuff?07:30
jaosoriorbandini: sure07:31
jaosoriorbandini: let me know when so I start brewing coffee :P07:31
bandinijaosorior: merci. In an hour or so?07:31
bandini:)07:31
jaosoriorack07:31
bandiniI am moderately sure I hit a heat bug, but before filing it I'd like to be sure I am not on crack :)07:32
*** jtomasek has joined #tripleo07:33
*** adarazs|ruck is now known as adarazs|rover07:40
rascamorning everyone!07:41
jfrancoagood morning!07:42
jfrancoadoes anybody know who's the CI ranger today? I need some help debugging one issue in the upgrade CI jobs07:43
*** rcernin has joined #tripleo07:44
thervebandini, Around if you need me to have a look07:44
*** shardy has joined #tripleo07:45
*** shardy has quit IRC07:45
bandinitherve: oh thanks! I'll write my thoughts down and then you can hit me with a cluebat ;)07:46
*** shardy has joined #tripleo07:49
jaosoriormandre, bandini: Could you check this out https://review.openstack.org/#/c/516378/ ?07:57
*** aufi has joined #tripleo07:58
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Add missing less file added by latest patternfly version  https://review.openstack.org/50561007:59
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Use ListView from patternfly-react for NodesListView  https://review.openstack.org/50561107:59
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Use ListView from patternfly-react for NodeDrives list  https://review.openstack.org/50561208:00
*** ykarel is now known as ykarel|lunch08:03
*** ebarrera has joined #tripleo08:04
jaosoriormandre: got 10 min?08:05
*** gkadam has joined #tripleo08:05
*** ebarrera has quit IRC08:09
*** ebarrera has joined #tripleo08:09
*** tesseract has joined #tripleo08:16
*** ccamacho has joined #tripleo08:17
*** ffiore has joined #tripleo08:17
jfrancoaadarazs|rover: o/ do you have a minute to give me a hand with an issue in the stable/pike upgrades job (multinode-containers-upgrade) ?08:18
adarazs|roverjfrancoa: hi; what's the problem?08:19
jfrancoaadarazs|rover: it seems the script used for updating overcloud repos is not complete in stable/pike legacy-tripleo-ci-centos-7-containers-multinode-upgrades job08:20
jfrancoaadarazs|rover: http://logs.openstack.org/46/510846/1/check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades/5759822/logs/undercloud/home/zuul/overcloud-repo.yaml.txt.gz08:20
jfrancoaadarazs|rover: so the jobs fail because the overcloud repo can't be updated : http://logs.openstack.org/46/510846/1/check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades/5759822/logs/undercloud/home/zuul/failed_upgrade.log.txt.gz08:21
*** etingof has quit IRC08:21
jfrancoaadarazs|rover: It seems that for some reason (I could not find where), we are setting the variable upgrade_overcloud_repo_template to overcloud-repo.yaml.j2 instead of overcloud-repo-tripleo-ci.yaml.j208:22
* adarazs|rover looks into it in a bit.08:23
*** gfidente has joined #tripleo08:24
*** gfidente has quit IRC08:24
*** gfidente has joined #tripleo08:24
jfrancoaadarazs|rover: thanks08:26
*** pgadiya has joined #tripleo08:27
*** d0ugal has quit IRC08:29
*** d0ugal has joined #tripleo08:29
bandinitherve, jaosorior: https://etherpad.openstack.org/p/heat-tripleo-oddity-pike this is the problem I am observing (I have the live env available where this is reproduceable)08:32
*** gfidente has quit IRC08:32
*** jpena|off is now known as jpena08:33
*** pdeore has quit IRC08:34
*** gfidente has joined #tripleo08:34
*** gfidente has quit IRC08:34
*** gfidente has joined #tripleo08:34
openstackgerritMerged openstack/tripleo-validations master: Use OrderedDict to sort the hostname before the alias  https://review.openstack.org/51623708:36
jaosoriorbandini: bluejeans?08:36
*** hewbrocca_afk is now known as hewbrocca08:37
*** ffiore has quit IRC08:38
bandinijaosorior: that'd be awesome. https://redhat.bluejeans.com/4884650097/08:38
mandrebandini: do you have a service running on BM that could overwrite the hieradata or everything runs in containers?08:41
*** etingof has joined #tripleo08:41
*** DJ_ has joined #tripleo08:42
mandrebandini: IIRC the current state of things, the hieradata are global to the host, so changing it somewhere may affect all the services and containers running on the node08:42
*** ykarel|lunch is now known as ykarel08:42
jaosoriormandre: he doesn't08:42
jaosoriortherve: you around?08:43
DJ_Hi08:44
*** amoralej|off is now known as amoralej08:44
*** pdeore has joined #tripleo08:44
*** jpich has joined #tripleo08:44
thervejaosorior, Yep08:45
jaosoriortherve: wanna join the bluejeans? :D08:45
therveSure :)08:45
DJ_I want to do an upgrade on RHOSP10, with some new environment file. Is it possible ? or should I pass the exact same environment parameters that were used at deployment time.08:46
*** oidgar has joined #tripleo08:46
openstackgerritmathieu bultel proposed openstack/tripleo-validations stable/pike: Use OrderedDict to sort the hostname before the alias  https://review.openstack.org/51655908:46
*** ffiore has joined #tripleo08:48
*** bogdando has joined #tripleo08:48
*** gfidente has quit IRC08:49
*** achadha has joined #tripleo08:54
*** pdeore_ has joined #tripleo08:54
*** pdeore has quit IRC08:55
adarazs|roverjfrancoa: I have no clue about the upgrades role so it takes a while to unravel how that is used...08:58
*** achadha has quit IRC08:58
adarazs|roverjfrancoa: FWIW it looks to me that the correct template is used: http://logs.openstack.org/46/510846/1/check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades/5759822/job-output.txt.gz#_2017-10-24_05_09_32_00474208:59
adarazs|rover'src': u'overcloud-repo-tripleo-ci.yaml.j2'09:00
jfrancoaadarazs|rover: yes, I saw it exactly after I wrote you before...but I can't understand then why it is so short09:01
jfrancoaadarazs|rover: it's seems like the openstack-tripleo-quickstart-extras package its using didn't have this patch https://github.com/openstack/tripleo-quickstart-extras/commit/cf15dafa4b3752b56b76767db175c800e31266a509:04
* jfrancoa is going to check tripleo-quickstart-extras version used09:04
adarazs|roverit should be used from master...09:05
adarazs|rovernot packaged09:05
*** akane has joined #tripleo09:08
jfrancoaadarazs|rover: http://logs.openstack.org/46/510846/1/check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades/5759822/job-output.txt.gz#_2017-10-24_04_05_03_078224 do you know the way to know what does this version (tripleo-quickstart-extras-2.1.1.dev204) include?09:11
*** gfidente has joined #tripleo09:13
*** mcornea has joined #tripleo09:14
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable keystone db sync to log to stdout  https://review.openstack.org/51656409:19
*** lucas-afk is now known as lucasagomes09:19
*** aufi has quit IRC09:20
jfrancoaadarazs|rover: ok...this is strange...I took a job from master to compare to and the content in overcloud-repo.yaml is the same. But the job passed this time: http://logs.openstack.org/95/506595/4/check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades/5272b87/logs/undercloud/home/zuul/overcloud-repo.yaml.txt.gz09:20
jfrancoaadarazs|rover: do you think it's worth to open a launchpad bug?09:22
jaosoriorbogdando: does kolla have docs per container?09:25
jaosoriorbogdando: not sure where I would add these docs :/09:25
bogdandojaosorior: I'm not sure what docs it has09:25
jaosoriormandre: ^^09:25
openstackgerritJuan Badia Payno proposed openstack/puppet-tripleo master: [WIP] logging: support service_config_settings configuration mechanism  https://review.openstack.org/50093509:26
bogdandojaosorior: https://docs.openstack.org/kolla/latest/ seems like there is no docs for env vars09:26
*** ffiore has quit IRC09:28
*** jistr|mtgs is now known as jistr09:29
*** ffiore has joined #tripleo09:29
jaosoriorbogdando, mandre: When you get some time, can you also check this one https://review.openstack.org/#/c/516378/ ?09:29
mandrebogdando, jaosorior: right, there is no such thing for now... I got a local change that added some of them that's been sitting on my drive for way too long09:30
bogdandono worries09:30
openstackgerrityatin proposed openstack/tripleo-quickstart master: Fix delorean_hash for newton and ocata promotion  https://review.openstack.org/51656709:32
ykareladarazs|rover, ^^09:32
adarazs|roverjfrancoa: I don't see what connection does https://github.com/openstack/tripleo-quickstart-extras/commit/cf15dafa4b3752b56b76767db175c800e31266a5 have with your problem.09:33
adarazs|roverjfrancoa: also -- quickstart-extras is *not* from any package, it get cloned from source from master during the job run.09:33
bogdandojaosorior: yay, it's all green, good to go09:33
adarazs|roverunless the upgrades jobs do something different09:33
*** aufi has joined #tripleo09:34
*** psahoo has quit IRC09:35
jaosoriorbogdando, mandre: thanks09:35
jfrancoaadarazs|rover: if you see the patch, there was a conditional before in line 6. If target_upgrade_version was not master (which is this case, as the target_upgrade_version in stable/pike is 'pike') then we would end up having only this exact content in overcloud-repo.yaml http://logs.openstack.org/46/510846/1/check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades/5759822/logs/undercloud/home/zuul/overcloud-repo.yaml.txt.gz09:37
jfrancoang all content from lines 6 to 10509:37
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Extract all errors from tripleo deployment  https://review.openstack.org/51640209:38
*** shardy has quit IRC09:42
*** shardy has joined #tripleo09:42
ccamachofolks mind to do a quick review here https://review.openstack.org/#/c/513405/ ??09:42
ccamachoalready +2 and passing CI09:43
*** dsariel__ has joined #tripleo09:44
adarazs|roverjfrancoa: which change are these logs from?09:45
*** owalsh- is now known as owalsh09:47
jfrancoaadarazs|rover: this one: https://review.openstack.org/#/c/510846/09:49
jfrancoaadarazs|rover: and the same job (multinode-containers-upgrade) does fail for the same reason in this other stable/pike change https://review.openstack.org/#/c/512466/ (which does not modify anything related to upgrades).09:51
*** pgadiya has quit IRC09:52
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Refactor Register nodes dialog to use redux-form  https://review.openstack.org/50490309:53
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Add NetIpMap to hieradata for *ExtraConfig overrides  https://review.openstack.org/51470709:54
*** mandre has quit IRC09:55
*** mandre_ has joined #tripleo09:56
*** mandre_ is now known as mandre09:56
*** brault has quit IRC09:56
adarazs|roverjfrancoa: so yeah, please open a but, not sure what's going on.09:56
*** salmankhan has joined #tripleo09:57
thervebandini, Found it!09:57
thervebandini, https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/clustercheck.yaml#L6209:57
jfrancoaadarazs|rover: ok,  thanks for the help. I'll try to dig up a little bit more, this overcloud-upgrade role it's a bit painful09:57
adarazs|roveryep.09:58
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Remove str_replace for bootstrap_server_id in deploy_steps.j2  https://review.openstack.org/51344909:58
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Select first node as bootstrap node not using name  https://review.openstack.org/51345009:58
bandinitherve: holy crap, batman! not sure why I don't see it in my upstream envs, but I'll investigate more09:59
bandinithanks!09:59
thervebandini, I think there is a catch, it depends on the service order10:00
bandinitherve: aah that would explain it, yes!10:00
*** leseb_ has joined #tripleo10:00
therveI don't think it's stable? It seems to be a dictionary10:00
*** rickflare has joined #tripleo10:00
bandiniah righto10:00
bandiniyeah this would explain this issue quite well10:00
bandiniI'll experiment a bit10:00
*** jistr_ has joined #tripleo10:00
*** agurenko_ has joined #tripleo10:00
*** isq_ has joined #tripleo10:01
*** leseb has quit IRC10:01
*** jistr has quit IRC10:01
*** jschlueter has quit IRC10:01
*** agurenko has quit IRC10:01
*** thrash|g0ne has quit IRC10:01
*** rwsu has quit IRC10:01
*** migi has quit IRC10:01
*** rickflare2 has quit IRC10:01
*** rwsu has joined #tripleo10:02
*** migi has joined #tripleo10:02
*** isq has quit IRC10:03
lvdombrkrhello folks, i use controllers HA and have problems with galera cluster..after i restarted all controllers, mariadb ar down on all controllers and in logs i see: 2017-10-31  9:58:32 139730676390080 [ERROR] WSREP: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out)10:03
lvdombrkr at gcomm/src/pc.cpp:connect():16210:03
lvdombrkr10:03
lvdombrkr017-10-31  9:58:32 139730676390080 [ERROR] Aborting10:03
*** thrash has joined #tripleo10:03
*** thrash has quit IRC10:03
*** thrash has joined #tripleo10:03
lvdombrkrany ideas?10:03
*** zoli has quit IRC10:04
*** gfidente has quit IRC10:05
*** pgadiya has joined #tripleo10:06
*** pblaho has joined #tripleo10:06
*** zoli has joined #tripleo10:06
*** iranzo has quit IRC10:08
*** gfidente has joined #tripleo10:08
*** gfidente has quit IRC10:08
*** gfidente has joined #tripleo10:08
*** pblaho has quit IRC10:09
*** brault has joined #tripleo10:09
*** pblaho has joined #tripleo10:09
*** agurenko_ is now known as agurenko10:10
*** ratailor has quit IRC10:11
*** ipsecguy has quit IRC10:12
*** gfidente has quit IRC10:13
*** gfidente has joined #tripleo10:16
*** gfidente has quit IRC10:16
*** gfidente has joined #tripleo10:16
*** ipsecguy has joined #tripleo10:18
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: DNM - WIP testing OVB job speed improvements  https://review.openstack.org/50094210:21
*** akane has quit IRC10:21
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: DNM - WIP testing OVB job speed improvements  https://review.openstack.org/50094210:23
*** pgadiya has quit IRC10:24
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: DNM - WIP testing OVB job speed improvements  https://review.openstack.org/50094210:24
*** brault has quit IRC10:24
*** udesale has quit IRC10:26
ccamachojaosorior thanks man! how are you doing?10:27
*** iranzo has joined #tripleo10:27
*** gfidente has quit IRC10:29
*** gfidente has joined #tripleo10:32
*** gfidente has quit IRC10:32
*** gfidente has joined #tripleo10:32
*** aufi has quit IRC10:32
*** jschlueter|znc has joined #tripleo10:35
*** brault has joined #tripleo10:38
*** pgadiya has joined #tripleo10:38
*** brault has quit IRC10:43
openstackgerritCarlos Camacho proposed openstack/python-tripleoclient master: Add openstack undercloud backup  https://review.openstack.org/46621310:44
*** aufi has joined #tripleo10:46
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Tripleoclient unified queue name.  https://review.openstack.org/50990210:53
openstackgerritMarios Andreou proposed openstack/tripleo-specs master: Fix link in Queens major upgrade spec  https://review.openstack.org/51661110:59
d0ugaljpich: Hey, since you opened the bug - would you mind taking a look at https://review.openstack.org/#/c/511249/ ?10:59
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Neutron]  https://review.openstack.org/51661210:59
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Heat]  https://review.openstack.org/51661311:00
*** pdeore_ has quit IRC11:00
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: os-net-config not run when conf is empty and re-introduce exception.  https://review.openstack.org/51661411:01
*** pdeore_ has joined #tripleo11:01
jpichd0ugal: Sure, thanks for the heads-up!11:07
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Aodh]  https://review.openstack.org/51661611:07
d0ugaljpich: thank you!11:08
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Gnocchi]  https://review.openstack.org/51661711:08
*** atoth has joined #tripleo11:08
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Mistral]  https://review.openstack.org/51661811:08
gfidenteshardy back to ceph multiple clusters, do you have pointers to the code which implements the spine/leaf scenario to push different configs on different nodes with same role?11:11
shardygfidente: Hey, sure it's https://review.openstack.org/#/c/514707/ but it really only works if you can use hiera interpolation to set the per-node bind IPs for the different leaf networks11:12
gfidenteshardy ah yeah okay11:13
gfidenteI saw that one11:13
gfidentewell no in ceph scenario it's not just the binding ip11:14
*** ansmith has quit IRC11:14
shardygfidente: I don't think we have a general solution to this yet, other than using RoleNameExtraConfig to override specfic settings per role11:14
shardygfidente: yeah I figured it'd be more difficult in the ceph case, happy to try to figure out a solution but I don't have one right now11:14
gfidenteshardy yeah I don't have one either11:14
gfidentewas trying to figure how to approach it11:15
*** salmankhan has quit IRC11:15
shardygfidente: can we put the per-node data needed into the dynamic inventory, then map that into the vars needed for ceph-ansible?11:15
shardygfidente: any examples of the data we need?11:15
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: os-net-config not run when conf is empty and re-introduce exception.  https://review.openstack.org/51661411:15
gfidenteshardy so I think we'll have to run ceph-ansible multiple times with a different set of IPs for each role11:17
gfidenteit is literally deploying multiple clusters11:18
*** rcernin has quit IRC11:18
gfidenteit's not a customization for some of the nodes11:18
shardygfidente: would it be easier to deploy the clusters first and wire in multiple external clusters to the openstack environment?11:18
gfidenteshardy oh they aren't external11:18
shardyor are there requirements around hyperconvergence/co-location with the openstack stuff?11:18
*** salmankhan has joined #tripleo11:18
gfidenteyeah the ultimate goal is to have an independent ceph cluster with each openstack AZ11:19
gfidenteso the name of the AZ could be the scaling factor for the entire set of settings11:19
gfidenteincluding service name so that we trigger ceph-ansible multiple times with a different inventoryu11:20
shardygfidente: Hum, Ok so the idea is to have a custom role per az?11:20
gfidentewell it's the only thing I could think of at leat11:20
*** pdeore has joined #tripleo11:20
shardyyeah we don't really have a strong abstraction for az's in tripleo yet11:20
fultonjbugzy: regarding http://paste.openstack.org/show/625022/, please check if you have this patch https://review.openstack.org/#/c/51458411:20
openstackgerritMerged openstack/tripleo-validations stable/pike: Use OrderedDict to sort the hostname before the alias  https://review.openstack.org/51655911:21
shardygfidente: so custom roles seems reasonable, but we're missing a way to associate the different groups of osd/mon services I guess11:22
openstackgerritMerged openstack/tripleo-specs master: Fix link in Queens major upgrade spec  https://review.openstack.org/51661111:22
gfidenteshardy right unless with mixin that with actual roles_data ?11:22
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: os-net-config not run when conf is empty and re-introduce exception.  https://review.openstack.org/51661411:22
*** pdeore_ has quit IRC11:22
shardygfidente: well we do have the concept of tags in roles_data, maybe we could use that11:22
shardygfidente: we'd need to expose those tags from heat I guess (j2 render them into some outputs) then we could set per role tags that ansible could use to group things in the inventory?11:24
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/roles_data.yaml#L1311:24
*** panda|ruck|off is now known as panda|ruck11:24
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: DO NOT MERGE - Testing specific DLRN hash tag  https://review.openstack.org/51662411:24
*** agurenko is now known as agurenko_11:25
*** ratailor has joined #tripleo11:27
fultonjshardy gfidente so assuming you had tags like ceph-cluster1 and ceph-cluster2, for different custom roles and had assigned diff hosts to those roles, then the ceph-ansible workflow in tripleo-common could make different sets of nodes and the workflow could run more than once per group?11:27
*** gfidente has quit IRC11:27
shardyfultonj: yes that's what I was thinking11:27
fultonjso some yaql in heat just needs to get the tags into the mistral environment so it can act that way11:28
fultonjseems good in theory11:28
fultonjit's my first time seeing the tags in roles_data11:29
shardyfultonj: I think we just need to dump the data via j2 when the template is rendered, sec I'll push a patch11:29
*** gfidente has joined #tripleo11:30
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Expose role tags via heat RoleTags output  https://review.openstack.org/51662611:32
shardyfultonj, gfidente: ^^ I guess you may need to pass the same data into the workflows11:33
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Mixed-1]  https://review.openstack.org/51662711:33
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Add validation task in docker services [Nova]  https://review.openstack.org/51662811:34
*** noslzzp has joined #tripleo11:38
fultonjshardy: thanks. how the data is passed isn't clear to me, but i see it getting collected per role in your patch.11:38
fultonjgfidente probably knows (but i shouldn't speak for him :)11:39
shardyfultonj: we'd do the same trick e.g here:11:40
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/common/deploy-steps.j2#L16611:40
shardyfultonj: the output I added would enable the data to be added to the dynamic tripleo-ansible-inventory though, which is the main way I personally drive ansible things with tripleo11:40
*** agurenko_ is now known as agurenko11:41
shardyand I know that's the direction we're headed e.g the "split stack" model, but I'm not quite clear how that relates to the worflow steps for ceph atm11:41
shardyI assume at some point we'll rework the ceph integration to use external_workflow_tasks?11:41
gfidenteshardy yep11:42
gfidentethat was the idea from ptg11:42
shardygfidente: ack, Ok thanks for confirming11:42
shardythat's working quite well for driving the kubespray ansible installer11:42
openstackgerritMerged openstack/python-tripleoclient stable/pike: Autodetect insecure registry in container image prepare  https://review.openstack.org/51645211:45
openstackgerritJohn Fulton proposed openstack/tripleo-docs master: DockerCephDaemonImage is now optional for Ceph deployments  https://review.openstack.org/51544611:46
openstackgerritAlexey Morlang proposed openstack/tripleo-heat-templates master: Merge from from Juniper THT for opencontrail  https://review.openstack.org/51663011:48
*** iranzo has quit IRC11:48
*** brault has joined #tripleo11:49
*** pchavva has joined #tripleo11:49
*** iranzo has joined #tripleo11:50
*** iranzo has quit IRC11:50
*** iranzo has joined #tripleo11:50
*** ratailor has quit IRC11:50
*** pgadiya has quit IRC11:51
*** liverpooler has joined #tripleo11:52
*** jprovazn has quit IRC11:52
*** brault has quit IRC11:53
*** udesale has joined #tripleo11:54
*** jprovazn has joined #tripleo11:55
beaglesgfidente, fultonj: can you give https://review.openstack.org/#/c/508195/ a quick once over to see if there is something major off-track there. I figure I'll might need a couple of revisions to adapt to functional changes on the patches it depends on but I'd like to get an early check for general correctness in approach11:58
fultonjbeagles: sure, i'll look at it this morning11:59
beaglesfultonj, thanks!11:59
*** dougbtv has joined #tripleo12:00
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Update roles listing  https://review.openstack.org/50941512:01
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Refactor ImportPlanCard into ActionCard  https://review.openstack.org/51226512:01
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: [WIP] Select Roles feature  https://review.openstack.org/51226612:01
beaglesshardy, jaosorior: I addressed your feedback on https://review.openstack.org/#/c/490082/ but missed you on the question of whether the server_names line should have the default value for the hiera lookup removed as well12:02
*** abregman has quit IRC12:03
beaglesshardy, jaosorior: I updated as if it should be since it seemed a similar thing12:03
shardybeagles: ack thanks for the update - lgtm!12:07
beaglesshardy, thx!12:07
*** shardy is now known as shardy_lunch12:07
sri_beagles: Hi, I am getting this ssl error while creating lb with Octavia any ideas ?  worker.log Could not connect to instance. Retrying.: SSLError: [SSL] PEM lib (_ssl.c:2535)12:07
*** pdeore has quit IRC12:08
sri_beagles: amphora vm logs , amphora-agent.log Invalid request from ip=::ffff:192.168.199.62: [SSL: SSL_HANDSHAKE_FAILURE] ssl handshake failure (_ssl.c:1783)12:10
beaglessri_, 1s12:11
*** raildo has joined #tripleo12:11
sri_beagles: ok, amphora vm  messages logs : ERROR octavia.amphorae.backends.health_daemon.health_sender [-] No controller address found. Unable to send heartbeat.: Empty12:13
*** dprince has joined #tripleo12:13
beaglessri_, so something happened with the post deployment where the controller_ip_list wasn't updated properly12:14
*** leitan has joined #tripleo12:15
*** agurenko is now known as agurenko_12:17
*** akrivoka has joined #tripleo12:18
sri_beagles: ok, so i can debug the post deployment script to see if it's getting correct IP address of controllers12:21
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart-extras master: Include 'queens' as target version for upgrade.  https://review.openstack.org/51664712:23
*** DJ_ has quit IRC12:23
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: Fix iptables rules override bug in clustercheck docker service  https://review.openstack.org/51664912:25
*** masco has joined #tripleo12:26
*** rhallisey has joined #tripleo12:28
openstackgerritAlexey Morlang proposed openstack/puppet-tripleo master: Merge from from Juniper repo for opencontrail  https://review.openstack.org/51665112:30
*** pdeore has joined #tripleo12:30
sri_beagles: it seems script get the controller ip address right ! http://paste.openstack.org/show/625085/12:31
*** catintheroof has joined #tripleo12:31
bandinijaosorior, therve, mandre: ^ is the tested fix for this morning's oddity (therve++)12:31
*** catintheroof has quit IRC12:32
*** catintheroof has joined #tripleo12:32
*** bfournie has quit IRC12:32
*** bfournie has joined #tripleo12:32
*** morazi has joined #tripleo12:32
*** ansmith has joined #tripleo12:34
*** rlandy has joined #tripleo12:34
*** trown|outtypewww is now known as trown12:34
*** bfournie has quit IRC12:36
*** gkadam has quit IRC12:37
*** ecerquei_ has joined #tripleo12:38
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Run Octavia configuration on the overcloud  https://review.openstack.org/50819512:39
*** jcoufal has joined #tripleo12:39
*** tosky has joined #tripleo12:44
*** jmelvin has joined #tripleo12:45
mandrebandini: have you ever consider a career in book publishing? you're commit messages read like novels :)12:46
*** lucasagomes is now known as lucas-hungry12:47
*** rbowen has joined #tripleo12:47
*** toure_biab is now known as toure12:47
*** Dinesh_Bhor has quit IRC12:47
*** udesale has quit IRC12:49
bandinimandre: ahahah it is because I am going senile. So I can look at an older commit and know wth is going on ;)12:49
*** udesale has joined #tripleo12:49
*** shardy_lunch is now known as shardy12:49
*** jpena is now known as jpena|lunch12:51
*** lblanchard has joined #tripleo12:52
*** ecerquei_ has quit IRC12:56
*** udesale has quit IRC12:56
*** ecerquei has joined #tripleo12:56
*** udesale has joined #tripleo12:56
*** gfidente has quit IRC12:56
*** mandre is now known as mandre_afk12:56
*** bkopilov has quit IRC12:57
*** janki has quit IRC12:58
*** ffiore has quit IRC13:02
*** gfidente has joined #tripleo13:04
jaosoriorccamacho: all good here. you?13:04
jaosoriorbandini: looking13:05
jaosoriorbeagles: what was the question again?13:06
*** bfournie has joined #tripleo13:07
beaglesjaosorior, the feedback on https://review.openstack.org/#/c/490082/4/manifests/haproxy.pp was that the $controller_hosts_real wasn't needed any longer13:08
jaosoriorright13:08
beaglesjaosorior, I was checking to see if the same applied to the following line - ie. that $controller_hosts_names_real was also no longer necessary13:09
jaosorioryep13:09
jaosoriorit does13:09
beaglesjaosorior, ok.. I removed it also anyways assuming that was the case13:09
*** tzumainn has joined #tripleo13:10
*** jprovazn has quit IRC13:11
*** aputtur_ has joined #tripleo13:11
*** jprovazn has joined #tripleo13:12
jaosoriorshardy: what about the controller_virtual_ip?13:12
*** pkovar has joined #tripleo13:14
*** ffiore has joined #tripleo13:15
shardyjaosorior: yeah that has a similar problem but it's not such a bad default, because it'll point to the ctlplane e.g internal connectivity would still work13:16
shardyjaosorior: so I didn't block on it13:16
shardyjaosorior: the issue with *_ips is the list of ips would potentially be completely wrong13:16
*** jcoufal_ has joined #tripleo13:16
jaosoriorright13:18
*** jcoufal has quit IRC13:18
jaosoriorbeagles: well, should octavia be always listening on both the octavia VIP (e.g. the internal api network) and the ctlplane network?13:18
*** aputtur_ has quit IRC13:19
*** newmember has joined #tripleo13:19
*** aputtur_ has joined #tripleo13:20
*** bobh has joined #tripleo13:23
jtomasekd0ugal: hi, I am trying to test new workflow so I pulled the patch and updated workbook using mistral workbook-update but the new workflow is not in workflow-list, any Idea what I might be doing wrong? https://review.openstack.org/#/c/509832/7/workbooks/plan_management.yaml13:23
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Improve node count indicator in roles step  https://review.openstack.org/51667413:27
EmilienMhello13:28
EmilienMpanda|ruck, adarazs|rover: what's up?13:28
abishopmatbu: pls take another look at https://review.openstack.org/513202 (gfidente and I commented)13:28
adarazs|roverEmilienM: if you have a few minutes we're on https://bluejeans.com/u/gcerami/ discussing the things that are up.13:28
EmilienMadarazs|rover: on my way13:29
beaglesjaosorior, I don't think it would run on the ctrlplane as I understand it13:30
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: [Newton only] Fix double sed backup during upgrade.  https://review.openstack.org/51667613:31
openstackgerritArx Cruz proposed openstack-infra/tripleo-ci master: DO NOT MERGE - Testing specific DLRN hash tag  https://review.openstack.org/51662413:32
jaosoriorbeagles: alright, could you change the hiera to not include the ctlplane vip in the unique statement? only leave the hiera call and the default value13:33
jaosoriorbrb13:33
beaglesjaosorior, yup13:33
*** jaosorior has quit IRC13:33
*** skramaja has quit IRC13:34
jpichjtomasek: Perhaps you found a new bug, I wonder if the indentation should only be two spaces before the new workflow name13:37
openstackgerritEmilien Macchi proposed openstack/instack-undercloud stable/pike: heat-dbsync: increase timeout to 15 minutes  https://review.openstack.org/51668013:37
*** abregman_ has joined #tripleo13:38
openstackgerritEmilien Macchi proposed openstack/instack-undercloud stable/ocata: heat-dbsync: increase timeout to 15 minutes  https://review.openstack.org/51668213:40
*** lucas-hungry is now known as lucasagomes13:41
openstackgerritEmilien Macchi proposed openstack/instack-undercloud stable/newton: heat-dbsync: increase timeout to 15 minutes  https://review.openstack.org/51668313:41
jtomasekjpich: hmm good point13:42
openstackgerritBrent Eagles proposed openstack/puppet-tripleo master: Add Octavia API endpoint to haproxy  https://review.openstack.org/49008213:42
openstackgerritEmilien Macchi proposed openstack/instack-undercloud stable/ocata: heat-dbsync: increase timeout to 15 minutes  https://review.openstack.org/51668213:42
*** lvdombrkr has quit IRC13:43
*** lvdombrkr has joined #tripleo13:43
*** ratailor has joined #tripleo13:45
d0ugaljtomasek, jpich - looking.13:46
jtomasekd0ugal, jpich: the indentation on line 983 seems to be off13:47
d0ugaljtomasek: I don't even have that line number :)13:47
jtomasek*89313:47
jtomasekd0ugal: it's in that patch https://review.openstack.org/#/c/509832/7/workbooks/plan_management.yaml13:47
d0ugaljtomasek: got it, I do see a 893 :)13:48
d0ugaljtomasek, jpich - it was totally broken when I reviewed it the time before last, so I guess it still hasn't been tested at all13:48
*** amoralej is now known as amoralej|lunch13:48
jpichhm workbook-update seems to say "invalid DSL" for me here, maybe a couple more things to fix up yeah13:49
jtomasekd0ugal: I see, thing is that when I try to update the same workbook on master, I still see only 5 plan_management workflows, but that workbook contains much more, I have no idea what am I missing here13:49
*** ratailor is now known as rtailor13:49
d0ugaljtomasek: after that indentation change it still failed?13:50
d0ugalI can try updating it.13:50
EmilienMtripleo meeting in 10 min, feel free to add topics on https://etherpad.openstack.org/p/tripleo-meeting-items13:50
jtomasekd0ugal: It 'failed' on master too13:51
matbuabishop: yep i saw your comment13:51
d0ugaljtomasek: oh, that is interesting. No errors?13:51
*** brault has joined #tripleo13:51
*** jpena|lunch is now known as jpena13:52
jtomasekd0ugal: none, passes just fine13:52
d0ugaljtomasek: ooft, that patch has a nasty conflict.13:53
rtailorshardy, you around ?13:53
openstackgerritSteven Hardy proposed openstack/python-tripleoclient master: Refactor undercloud deploy to use Ansible.  https://review.openstack.org/51596713:54
rtailorshardy, I am working on enabling TLS for ec2api and I have made changes available here https://review.openstack.org/#/c/509392/ and https://review.openstack.org/#/c/509393/13:55
d0ugaljtomasek: What do you use to load the workbooks?13:55
d0ugaljtomasek: I have 8 on my fairly recent quickstart env13:56
gfidenteEmilienM are you running the tripleo meeting today/13:56
gfidente?13:56
rtailorshardy, To test it, I want to have those changes in my overcloud, for that I am re-deploying my overcloud with above changes on overcloud.13:56
jtomasekd0ugal: 'openstack workbook update workbooks/plan_management.yaml'13:56
gfidenteEmilienM or mwhahaha?13:56
EmilienMgfidente: nope, mwhahaha is back13:56
gfidenteack13:56
EmilienMgfidente: good reminder though13:56
EmilienMgfidente: please don't tell mwhahaha I ran it late last week :P13:56
jtomasekd0ugal: can you 'openstack workflow list | grep plan_management' please?13:56
rtailorshardy, For t-h-t related chagnes, I have done at /usr/share/opentack-tripleo-heat-templates and for puppet-tripleo, I have followed http://hardysteven.blogspot.in/2016/08/tripleo-deploy-artifacts-and-puppet.html13:57
*** oanson has quit IRC13:57
d0ugaljtomasek: http://paste.openstack.org/show/625110/13:57
d0ugaljtomasek: my env is 11 days old13:57
*** oanson has joined #tripleo13:57
rtailorshardy, I am getting Timedout everytime, for overcloud.AllNodesDeploySteps.ComputeDeployment_Step4.0: resource creation.13:58
*** pdeore has quit IRC13:58
jtomasekd0ugal: ok, those are exactly the ones I am missing13:58
jtomasekd0ugal: and I see only the newly added ones (already on master)13:58
rtailorshardy, I cant see any service running on overcloud-controller.13:58
d0ugaljtomasek: oh, that is strange. I guess I'll try updating to the version in master.13:59
jtomasekd0ugal: that would help, yeah13:59
mwhahaha#startmeeting tripleo14:00
mwhahaha#topic agenda14:00
mwhahaha* Review past action items14:00
mwhahaha* One off agenda items14:00
mwhahaha* Squad status14:00
mwhahaha* Bugs & Blueprints14:00
openstackMeeting started Tue Oct 31 14:00:11 2017 UTC and is due to finish in 60 minutes.  The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
mwhahaha* Projects releases or stable backports14:00
mwhahaha* Specs14:00
mwhahaha* open discussion14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
mwhahahaAnyone can use the #link, #action and #info commands, not just the moderatorǃ14:00
mwhahahaHi everyone! who is around today?14:00
*** openstack changes topic to " (Meeting topic: tripleo)"14:00
openstackThe meeting name has been set to 'tripleo'14:00
*** agurenko_ is now known as agurenko14:00
*** openstack changes topic to "agenda (Meeting topic: tripleo)"14:00
fultonjo/14:00
akrivokao/14:00
ccamacho\o/14:00
rtailorshardy, even rabbitmq is also not running.14:00
*** jokke_ has joined #tripleo14:00
jtomaseko/14:00
beagleso-14:00
atotho/14:00
EmilienMo/14:01
dpeacocko/14:01
jfrancoao/14:01
d0ugalo/14:01
panda|rucko/14:01
owalsho/14:01
trowno/14:01
*** janki has joined #tripleo14:01
matbuo/14:01
gfidenteo/14:01
chemo/14:02
jpicho/14:02
saneaxo/14:02
mwhahaha#topic review past action items14:02
*** openstack changes topic to "review past action items (Meeting topic: tripleo)"14:02
mwhahahaEmilienM to prepare an etherpad for tripleo onboarding session in Sydney14:03
EmilienMoops14:03
EmilienMlet me prepare it today14:03
mwhahaha#action EmilienM to prepare an etherpad for tripleo onboarding session in Sydney14:03
sshnaidmo/14:03
mwhahaha(postponed) merge https://review.openstack.org/#/c/512569 (gfidente)14:03
jristo/14:03
mwhahahanot merged yet14:03
EmilienM#link https://etherpad.openstack.org/p/SYD-forum-tripleo-onboarding14:03
mwhahahabut +A so it should merge today, i'll keep an eye on it14:03
gfidentemwhahaha thanks14:03
gfidenteme too14:04
mwhahahateam to review https://review.openstack.org/#/c/511509/ asap14:04
mwhahahais pending merge14:04
mwhahaha(still)14:04
mwhahahaugh the ci gate queue is 20 hours+14:04
* mwhahaha makes note for later14:04
rtailorowalsh, you around ?14:04
mwhahaha(postponed) marios and matbu to provide an upgrade workflow status14:04
mwhahahamarios, matbu: update?14:05
shardyo/ sorry late14:05
*** mandre_afk is now known as mandre14:05
mariosmwhahaha: o/ no, we discussed the overall workflow last week14:05
matbumwhahaha: hey, so we have been in a upgrade meetup last week14:05
owalshrtailor: yea, what's up?14:06
matbumwhahaha: we have a lot of action items to adressed, i think the next week we can have some good input to share with the team14:06
mariosmwhahaha: to be honest i'm not sure what the task is for.. is this for tracking major upgrade workflow tasks (in which case this is a good start https://bugs.launchpad.net/tripleo/+bug/1728905 )14:06
openstackLaunchpad bug 1728905 in tripleo "RFE: Feature parity for CLI and UI via mistral/tripleo-common for Queens major upgrade" [High,Triaged] - Assigned to Marios Andreou (marios-b)14:06
*** ykarel has quit IRC14:06
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: Gate test - do not merged  https://review.openstack.org/49459314:06
mwhahahamarios: the original ask was to provide current status of workflow since we wanted to make sure that it's established for folks to start helping out with ASAP14:06
rtailorowalsh, The tls stuff, I couldn't deploy overcloud with my changes, its failing everytime due to timeout issue, And I couldn't even get any service running on overcloud-controller14:07
chemmwhahaha: we have some documentation comming about the expected workflow, a dev doc kind of thing14:07
rtailorowalsh, the rabbitmq is also not running and on overcloud compute, I could see rabbitmq not reachable error messages.14:07
mwhahahak so maybe next week provide a summary for everyone14:07
mariosmwhahaha: ack ok this makes sense... and as others are pointing out it was one of the main things that came up last week too so we'll have action on it soon. will likely be combination of things, docs for starters14:07
chemmwhahaha: how to for main course14:08
chemmwhahaha: and ci for desert14:08
mwhahaha#action marios, matbu, chem provide doc/status of upgrade workflow14:08
marioschem: :D14:08
rtailorowalsh, If you want to look at my setup, which I tried to deploy today.14:08
mwhahahak we'll revisit next week :)14:08
matbumwhahaha: thx :)14:08
mwhahaha(postponed) document that upgrades on baremetal aren't tested in Pike and Queens - I think done in https://review.openstack.org/#/c/496223/14:08
owalshrtailor: yea, I can take a look14:08
mwhahahaok that's all on the past action items14:09
mwhahaha#topic one off agenda items14:09
mwhahaha#link https://etherpad.openstack.org/p/tripleo-meeting-items14:09
*** openstack changes topic to "one off agenda items (Meeting topic: tripleo)"14:09
mwhahaha(gfidente) is anybody else interested into deploying multiple instances of a service configured differently on different nodes?14:09
*** amodi has joined #tripleo14:09
mwhahaha#link https://blueprints.launchpad.net/tripleo/+spec/deploy-multiple-ceph-clusters14:09
gfidentemwhahaha so let me introduce a bit better the topic14:09
shardywe discussed this earlier, and there are similarities to the requirements for spine/leaf deployments14:10
EmilienMah, we had this discussion when we started containers14:10
mwhahahaI ran across a similar request for multiple rabbitmq clusters14:10
gfidentefor ceph this ends up being a way of deploying multiple instances of the ceph services, with different configurations14:10
mwhahahaie for notification/mesaging14:10
shardyYeah, also e.g per service DB instances to enable scaling beyond a single DB cluster14:11
gfidenteI would like to team up with somebody else so that we can come up with a better implementation and idea14:11
EmilienMcan ceph run configuration from something else than a config file? (etcd, etc)14:11
shardythere are a few use-cases I think14:11
gfidenteEmilienM I don't think14:11
EmilienMI also think we can pass multiple --config-file arguments to the OpenStack services14:11
EmilienMand there must be some sort of override14:11
EmilienMbut that's kind of an hack14:12
gfidenteEmilienM yah that's possible with ceph as well14:12
shardywell I think role specific parameters already solves part of this14:12
*** catintheroof has quit IRC14:12
shardybut we're missing part for overriding per-node parameters (e.g things like the bind IPs)14:12
gfidenteshardy and also we make up the list of monitors or osds out of the service name14:12
gfidentebut in this case, for each cluster, the list should be made up from the node with a given role14:12
gfidentenot just those hosting the service14:13
shardygfidente: yup, but I think we agreed roles_data tags could be used to subcategorize where a service spans multiple roles?14:13
*** noslzzp has quit IRC14:13
gfidenteshardy yeah I haven't looked into how to use those yet, but yes, tags are a good idea14:13
shardybut yeah there's a few things we'll need to address, probably need a spec or at least some more prototyping14:13
shardyI like the idea of using the dynamic inventory to group the nodes14:14
gfidenteso I wonder if it might be useful organize a weekly call with interested people?14:14
*** catintheroof has joined #tripleo14:14
shardywhich should be possible if we go ahead with the split stack model and drive all configuration via ansible14:14
EmilienM++14:15
shardygfidente: should we start with an etherpad to enumerate the issues, and then set up a call when we know what there is to discuss?14:15
shardyalso we can link WIP patches etc for each issue14:15
gfidenteshardy ok great yes, will do14:15
EmilienMshardy: good idea14:15
fultonj+114:16
gfidentemaybe I can share that during next meeting with some dradts14:16
gfidente*drafts14:16
mwhahahagfidente: do you want an action item reminder or will you do another one-off agenda item?14:17
gfidenteaction item14:17
mwhahaha#action gfidente put together issues around multiple service instances14:18
mwhahahak done14:18
owalshgfidente: /me interested too14:18
mwhahahaany other one off items anyone want to bring up?14:18
gfidenteowalsh great14:18
mwhahahamoving on to status14:19
mwhahaha#topic Squad status14:19
mwhahahaci14:19
mwhahaha#link https://etherpad.openstack.org/p/tripleo-ci-squad-scrum14:19
mwhahahaupgrade14:19
mwhahaha#link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status14:19
mwhahahacontainers14:19
mwhahaha#link https://etherpad.openstack.org/p/tripleo-containers-squad-status14:19
mwhahahaintegration14:19
mwhahaha#link https://etherpad.openstack.org/p/tripleo-integration-squad-status14:19
*** openstack changes topic to "Squad status (Meeting topic: tripleo)"14:19
mwhahahaui/cli14:19
mwhahaha#link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status14:19
mwhahahavalidations14:19
mwhahaha#link https://etherpad.openstack.org/p/tripleo-validations-squad-status14:19
mwhahahanetworking14:19
mwhahaha#link https://etherpad.openstack.org/p/tripleo-networking-squad-status14:19
mwhahahamandre, dprince: looks like contatiners status is not updated for this week14:20
*** jaosorior has joined #tripleo14:20
mwhahahagfidente: integration status is also stale14:20
EmilienMto which squad is the config-download work being assigned?14:20
mandrejeez, screw the DST14:20
gfidentemwhahaha yeah we didn't do much work this week14:20
mandremwhahaha: will do it ASAP14:20
mwhahahagfidente: k14:20
mwhahahamandre: thanks14:20
mwhahahaworkflows: thrash any pressing issues or status related items?14:21
gfidente(we converted scenario004 to ceph-ansible though)14:21
dprincemwhahaha: I can send an update to the list today14:21
dprincewould that work?14:21
gfidente(so we now have rgw and mds via ceph-ansible tested in CI)14:21
mwhahahadprince: that or just update the etherpad14:21
mwhahahadprince: your preference :)14:22
dprincemwhahaha: ack, perhaps both14:22
openstackgerritMerged openstack/tripleo-docs master: Stop using deprecated options for the prepare cmd  https://review.openstack.org/51638814:23
mwhahahaok moving on14:23
mwhahaha#topic bugs & blueprints14:23
mwhahaha#link https://launchpad.net/tripleo/+milestone/queens-214:23
mwhahahaFor Queens we currently have 69 (+0) blueprints and about 497 (+0) open bugs. 490 queens-2 and 7 queens-3.14:23
*** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)"14:23
mwhahaha#action mwhahaha to move medium bugs to queens-314:24
mwhahahaforgot to do that before i went on pto, i'll get to that today14:24
EmilienM(I'll follow-up with my unanswered question later)14:24
*** fzdarsky has joined #tripleo14:24
EmilienMmwhahaha: I moved all blueprints to queens-2 FYI (last week)14:24
mwhahahaEmilienM: oh that's probably integration14:24
EmilienMsounds good14:24
mwhahahaany bugs/blueprints that people would like to talk about or highlight?14:25
mwhahahasounds like nope14:26
mwhahaha#topic projects releases or stable backports14:26
*** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)"14:26
EmilienMso14:26
EmilienMwe were (and still are?) stuck by some zuul v3 jobs14:26
EmilienMto produce tarballs14:26
EmilienMit should be sorted out today or tomorrow14:26
EmilienMonce we have that, I'll re-propose a release for newton/ocata/pike14:27
EmilienMqueens-1 has been sorted out, and pushed14:27
*** amoralej|lunch is now known as amoralej14:27
EmilienMso we hope to have new stable releases by end of this week14:27
EmilienMsecond update:14:27
EmilienMthe topic we've discuss about extending the stable-policy for installers in paused now, we'll discuss about that during the Summit and try to make progress in person14:28
EmilienMI'll keep everyone posted on the mailing list14:28
EmilienMmwhahaha: done14:28
mwhahahaEmilienM: thanks14:28
mwhahaha#topic specs14:28
mwhahaha#link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open14:28
*** openstack changes topic to "specs (Meeting topic: tripleo)"14:28
mwhahahareminder, we should try and get all the specs merged by the end of queens-214:29
mwhahahaI sent a ML note about that a while back14:29
mwhahahaplease take some time to review the open specs14:29
mwhahahabesides that, does anyone want to talk about a specific spec?14:29
mwhahahasounds like nope14:30
mwhahaha#topic open discussion14:30
*** openstack changes topic to "open discussion (Meeting topic: tripleo)"14:30
mwhahahasoooo where are we at on master promotions14:30
jristthanks everyone with your hard work on ci and promotions14:30
*** catintheroof has quit IRC14:30
EmilienMafiu we have some issues with the build container job14:30
EmilienM(in RDO CI)14:30
EmilienMbut other than that, all jobs seem to pass well14:30
mwhahahado we have bugs created for the container build failures?14:31
EmilienMadarazs|rover, panda|ruck can give more details14:31
EmilienMamoralej and mandre as well14:31
amoralejEmilienM, https://bugs.launchpad.net/kolla/+bug/172891514:32
openstackLaunchpad bug 1728915 in kolla "kolla tries to install python2-cffi but latest version is python-cffi in base" [Undecided,In progress] - Assigned to Alfredo Moralejo (amoralej)14:32
mandrereview at https://review.openstack.org/#/c/516643/14:32
mandreit's about to merge14:32
amoralejmandre, oracle jobs timed out, i had to recheck14:32
jfrancoaEmilienM: I just wanted to raise a concern, as the upgrade jobs seem to be passing but that's because they are not running the upgrade step https://bugs.launchpad.net/tripleo/+bug/172891714:32
openstackLaunchpad bug 1728917 in tripleo "Upgrades CI jobs in master not performing upgrade" [Undecided,In progress] - Assigned to Jose Luis Franco (jfrancoa)14:32
EmilienMmandre: nice. Is that the only known blocker?14:32
EmilienMjfrancoa: on master?14:32
jfrancoaEmilienM: yes14:32
EmilienMjfrancoa: I wouldn't look at them now14:32
amoralejEmilienM, yes, the only known one14:33
EmilienMjfrancoa: containers are from Pike14:33
EmilienMjfrancoa: we don't even have QUeens containers yet14:33
amoralejbut one of the ovb jobs in last periodic run seems hang in deploy overcloud14:33
EmilienMjfrancoa: I would wait until we have a promotion and then try out the upgrades again14:33
jfrancoaEmilienM: ok, so it's not so worrying then14:33
EmilienMjfrancoa: nope.14:33
EmilienMamoralej: any logs?14:33
amoralejEmilienM, still running https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/282/console14:33
amoralejit's more that 3hr in that step14:34
EmilienMmhh ok14:34
amoralejthe other two ovb passed14:34
EmilienMcould it be i/o in the VM?14:34
amoralejwhich looks like a good signal14:34
amoralejand there is an intermitent issue with pushing containers14:35
amoralejlast containers build job in pike failed with it, and we've hit it in master in previous runs also14:35
*** catintheroof has joined #tripleo14:35
*** ykarel has joined #tripleo14:36
mwhahahaok so sounds like we're just waiting on some CI and we'll keep an eye on that14:36
EmilienMamoralej: the one where it fails to push on localhost?14:36
amoralejyeah14:37
*** spectr has quit IRC14:37
EmilienMhave we checked i/o on the machine?14:37
EmilienMpushing on a container on localhost shouldn't be a problem really14:37
EmilienMunless the VM is just dying14:37
amoralejlet me check if we have metrics14:38
EmilienMwe can check that offline14:38
EmilienMon #rdo probably14:39
mwhahahaso the other thing to talk about, what is the current status of the gate pipeline? do we know why we consistenly keep resetting the gate?14:39
*** spectr has joined #tripleo14:39
* EmilienM hasn't checked yet this morning - looks at adarazs|rover / panda|ruck14:39
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates stable/pike: Gate test - do not merge  https://review.openstack.org/51670014:40
adarazs|rovermwhahaha: I was wondering about that too, I was looking at one gate yesterday and it doesn't make much sense what zuul does with the gate jobs, killing them after 2 hours and starting them again.14:40
mwhahahait was 20hours earlier in the meeting now everything says 0 so i wonder if they reset zuul14:40
mwhahahaadarazs|rover: usually that happens if a job fails in teh queue, the reset get reset14:40
EmilienMadarazs|rover: yes it does make sense, we talked about it last week14:40
mwhahahado we have a way to track & research all gate failures?14:40
EmilienMzuul makes sure patches about to merge don't break another patch also about to merge14:40
EmilienMthat's why we have the shared queue14:41
EmilienMand that's why we need to look at every failure14:41
EmilienM1) make sure they are reported in Launchpad14:41
mwhahahai've asked infra and they don't have anything, but do we have that information available to us via the ci status pages or the extend status tool?14:41
EmilienM2) make sure they are in elastic recheck14:41
adarazs|roverEmilienM: ah, okay, I didn't follow that discussion.14:41
EmilienMwe should really do 1) and 2) before any recheck14:41
EmilienMadarazs|rover: it's how zuul works14:42
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Add pre-deployment negative tests for validations  https://review.openstack.org/48849514:42
EmilienMadarazs|rover: I can find the doc if you need14:42
EmilienMadarazs|rover: https://docs.openstack.org/infra/zuul/user/gating.html14:43
mwhahahaadarazs|rover, panda|ruck: so one of the things I would like to see from the CI squad is the review/metrics around these gate failures. Do you think you folks can look into getting this info14:43
adarazs|roverEmilienM: I think I just haven't looked closely at how it works until now. :)14:44
adarazs|roverEmilienM: thanks for the docs.14:44
EmilienMcool!14:44
*** cshastri has quit IRC14:44
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Add post-deployment negative tests for validations  https://review.openstack.org/50401414:45
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Add timestamper_cmd for tripleo-validations  https://review.openstack.org/49001314:46
adarazs|rovermwhahaha: hm, so you want some stats on why gate jobs fail; I don't think we have anything like that as they are the same jobs as the check jobs, so I think their stats gets merged in the rest of the jobs on SOVA.14:46
adarazs|rovermwhahaha: we would need to name them differently to discern the failures of gates vs. check jobs.14:46
mwhahahaadarazs|rover: I assume this info is in SOVA but yes we need to surface the gate failures14:47
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Fix ansible-lint.sh to check playbooks  https://review.openstack.org/44652514:47
*** udesale has quit IRC14:47
EmilienMadarazs|rover: I think what is really needed *now* is to track failures in *gate* and report them correctly in Launchpad and elastic recheck (queries)14:47
mwhahahaadarazs|rover: it's very important that we actively look at those immediately14:47
mwhahaha#action ci squad to start gathering gate failure metrics and information14:47
*** psachin has quit IRC14:48
*** noslzzp has joined #tripleo14:49
mwhahahaany other topics or things folks would like to discuss?14:49
mwhahahasounds like nope14:50
mwhahahathanks everyone14:50
mwhahaha#endmeeting14:50
*** openstack changes topic to "CI status: Orange - gate unstable - promotion 51 days | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/"14:50
openstackMeeting ended Tue Oct 31 14:50:31 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:50
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-10-31-14.00.html14:50
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-10-31-14.00.txt14:50
openstackLog:            http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-10-31-14.00.log.html14:50
jristthanks mwhahaha14:50
*** achadha has joined #tripleo14:51
bnemecOof, zuul restart.  Guess we'll find out how rh1 is really performing now...14:53
*** achadha has quit IRC14:55
*** abregman_ is now known as abregman|mtg14:58
*** dtantsur|afk is now known as dtantsur14:58
*** dmarlin has joined #tripleo15:00
*** stendulker has joined #tripleo15:00
mariosmandre: /win 1615:00
mariosmandre: sorry15:00
*** abregman|mtg has quit IRC15:04
openstackgerritPradeep Kilambi proposed openstack/python-tripleoclient master: Support to drive undercloud deploy via undercloud.conf  https://review.openstack.org/51135015:05
*** agurenko is now known as agurenko_15:07
*** oidgar has quit IRC15:09
*** achadha has joined #tripleo15:11
*** ykarel has quit IRC15:11
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo master: Drop the old skip-metering-database flag  https://review.openstack.org/51579615:12
*** achadha has quit IRC15:12
*** achadha has joined #tripleo15:12
bnemecAnd zuul restart #2 for the day.15:12
mwhahahait's going to be like that i guess15:13
*** achadha has quit IRC15:13
*** jprovazn has quit IRC15:14
*** achadha has joined #tripleo15:14
*** anshul has quit IRC15:15
*** dtantsur is now known as dtantsur|afk15:15
*** newmember has quit IRC15:16
*** jbadiapa has quit IRC15:18
*** achadha has quit IRC15:19
*** jcoufal_ has quit IRC15:19
*** jcoufal has joined #tripleo15:20
*** achadha has joined #tripleo15:20
*** stendulker has quit IRC15:21
*** stendulker has joined #tripleo15:22
mandreEmilienM: the backport for the DockerInsecureRegistry param autodetect merged, I'll let you do the honor of pushing the button for https://review.openstack.org/#/c/514477/15:23
EmilienMmandre: what honor :P15:24
mandreEmilienM: really nice catch BTW, I always forget that we don't do branches in oooq15:24
EmilienMcool!15:24
mandrethx ;-)15:24
*** noslzzp has quit IRC15:25
*** noslzzp has joined #tripleo15:26
openstackgerritBogdan Dobrelya proposed openstack/python-tripleoclient master: Add missing puppet configuration to undercloud deploy  https://review.openstack.org/51377415:26
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart master: WIP: Use tripleo-upgrade role in CI  https://review.openstack.org/51338215:27
*** spectr has quit IRC15:29
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564315:30
*** akrivoka has quit IRC15:31
*** akrivoka has joined #tripleo15:31
*** catintheroof has quit IRC15:34
*** PhilSliderS has quit IRC15:37
*** brault has quit IRC15:38
*** brault has joined #tripleo15:38
*** bogdando has quit IRC15:39
*** itlinux has joined #tripleo15:39
*** pblaho has quit IRC15:40
*** catintheroof has joined #tripleo15:40
*** pblaho has joined #tripleo15:40
*** spectr has joined #tripleo15:41
*** brault has quit IRC15:41
openstackgerritJulie Pichon proposed openstack/tripleo-common master: Remove Pike-only migration steps  https://review.openstack.org/51258515:41
openstackgerritJulie Pichon proposed openstack/tripleo-common master: Fix typos in container_images_prepare failure task names  https://review.openstack.org/51672015:41
*** catintheroof has quit IRC15:42
openstackgerritMerged openstack/tripleo-docs master: DockerCephDaemonImage is now optional for Ceph deployments  https://review.openstack.org/51544615:42
*** PhilSliderS has joined #tripleo15:43
*** moshele has quit IRC15:43
*** suuuper has quit IRC15:46
*** ffiore has quit IRC15:48
*** bkopilov has joined #tripleo15:50
mcorneashardy: hi o/ I'm trying to upgrade an env and major upgrade composable step fails with The Resource Type (OS::TripleO::Network::Management) could not be found; as a side note OS::TripleO::Network::Management: OS::Heat::None is set in an environment file.15:51
sshnaidmplease, can somebody help with nested heat templates? I create this stack with 3 files: http://paste.openstack.org/show/625126/  but I get error: "ERROR: The Resource Type (/tmp/subnode.yml) could not be found"15:52
mcorneashardy: any hint on what I can do to move forward?15:52
*** jbadiapa has joined #tripleo15:55
*** rtailor has quit IRC15:55
*** brault has joined #tripleo15:56
*** salmankhan has quit IRC15:57
*** aputtur_ has quit IRC15:58
*** aputtur_ has joined #tripleo15:58
shardymcornea: Hmm, do you have the fix from https://review.openstack.org/#/c/506806/ ?15:59
mcorneashardy: checking15:59
*** janki has quit IRC16:00
*** bnemec has quit IRC16:00
mcorneashardy: yep, it's there16:00
*** salmankhan has joined #tripleo16:01
*** stendulker has quit IRC16:02
*** jaosorior has quit IRC16:03
shardysshnaidm: I suspect the error may be misleading, /tmp/subnode has invalid syntax because there aren't any parameters defined to match the various get_param calls16:05
shardysshnaidm: maybe try creating just that stack first to test16:05
*** salmankhan has quit IRC16:05
shardymcornea: Ok could you pls raise a bug with more details?16:06
mcorneashardy: sure, in a minute16:07
*** anilvenkata has quit IRC16:07
*** salmankhan has joined #tripleo16:11
*** ffiore has joined #tripleo16:14
*** bnemec has joined #tripleo16:15
bnemecWoo, only one failure so far in spite of the fact that we had 32 stacks creating at once.16:16
*** trown is now known as trown|lunch16:16
bnemecNot bad for a creaky old cloud that is due to be retired.16:16
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add option for barbican API container to log to stdout/stderr  https://review.openstack.org/51673016:17
*** masco has quit IRC16:17
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DO NOT MERGE - Testing specific DLRN hash tag Also testing dlrn_hash_tag_newest set to specific hash.  https://review.openstack.org/51662416:17
*** panda|ruck is now known as panda|ruck|bbl16:18
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add ability to specify KEK for simple crypto plugin  https://review.openstack.org/51673216:20
openstackgerritAde Lee proposed openstack/tripleo-heat-templates master: Add ability to specify multiple barbican backends  https://review.openstack.org/51673316:20
*** fragatina has joined #tripleo16:20
*** fragatina has quit IRC16:21
*** fragatina has joined #tripleo16:22
*** gkadam has joined #tripleo16:22
*** shardy is now known as shardy_afk16:24
*** aufi has quit IRC16:25
*** brault has quit IRC16:26
*** brault has joined #tripleo16:26
*** brault has quit IRC16:27
*** brault has joined #tripleo16:28
*** iranzo has quit IRC16:30
mcorneashardy_afk: https://bugs.launchpad.net/tripleo/+bug/172903916:32
openstackLaunchpad bug 1729039 in tripleo "Ocata -> Pike overcloud upgrade: major-upgrade-composable-steps-docker fails with The Resource Type (OS::TripleO::Network::Management) could not be found." [Undecided,New]16:32
*** ramishra has quit IRC16:33
*** anilvenkata has joined #tripleo16:34
*** rbowen has quit IRC16:36
*** oidgar has joined #tripleo16:39
*** catintheroof has joined #tripleo16:40
*** ebarrera_ has joined #tripleo16:41
*** anilvenkata has quit IRC16:44
*** marios has quit IRC16:45
*** catintheroof has quit IRC16:45
*** ffiore has quit IRC16:46
mwhahahawe just had a dlrn failure in the gate, http://logs.openstack.org/78/498978/2/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/e219437/job-output.txt.gz#_2017-10-31_16_41_32_66960416:47
mwhahahalooks like it couldn't find openstack-macros16:47
*** yamahata has joined #tripleo16:47
mwhahahahttp://logs.openstack.org/78/498978/2/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/e219437/logs/delorean_logs/df/45/df45125423b6368026d393bb3f166946c501ad18_dev/root.log.txt.gz16:48
mwhahahaadarazs|rover, -^16:48
*** iranzo has joined #tripleo16:49
*** dparkes has quit IRC16:49
*** iranzo has quit IRC16:49
*** iranzo has joined #tripleo16:49
adarazs|rovermwhahaha: looking.16:51
*** mcornea has quit IRC16:52
hamzyI tried a small change and still received a hang... do you know what I might be doing wrong? http://paste.openstack.org/show/625006/16:53
adarazs|roverjpena, amoralej: you might have a faster time figuring out why this DLRN build failed. I don't think it's related to the "build-test-packages" role directly.16:53
adarazs|rover^16:53
adarazs|roverjpena, amoralej: can you take a look at the logs mwhahaha linked?16:53
* jpena checks16:53
*** rbowen has joined #tripleo16:54
*** gkadam has quit IRC16:54
amoralejDEBUG util.py:417:  Error: No Package found for openstack-macros ?16:54
*** ebarrera_ has quit IRC16:54
adarazs|roveramoralej: hm, so that's not a real package I suppose.16:54
*** iranzo has quit IRC16:54
amoralejit is16:54
adarazs|roverI see, it's in rdoinfo... so it's not that we need to add it to https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/build-test-packages/defaults/main.yml#L12-L2016:56
jpenaamoralej: we have an interesting situation. It's not in the new deps repo, but also not in the current-passed-ci one (because it's quite old)16:57
jpenawe need it in the deps repo16:57
amoralejoh16:57
*** bnemec has quit IRC16:57
*** moshele has joined #tripleo16:58
adarazs|rovermwhahaha: great idea w/ looking at the failing gate job.16:58
adarazs|rovermwhahaha++16:58
adarazs|roverprobably we hit this a couple of times.16:59
amoralejthis should be happening in other jobs16:59
*** rbowen has quit IRC17:00
adarazs|roveramoralej: maybe, but I think the gate job puts together a bunch of changes to be merged so it makes it more probably we hit it.17:00
*** jcoufal has quit IRC17:01
adarazs|roverthen again I might not know what I'm talking about ;)17:02
adarazs|roverwhich happens quite often around DLRN :)17:02
mwhahahathat failure is not part of the zuul info17:02
mwhahahathat's part of the dlrn setup17:02
*** jcoufal has joined #tripleo17:03
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Remove 'Refresh results' button on the nodes page  https://review.openstack.org/51547417:03
mwhahahahttps://github.com/softwarefactory-project/DLRN/blob/9a8529f75f3b1496475be615b37751ae64101b60/scripts/centos.cfg#L417:04
mwhahahaso maybe repo problems?17:04
*** raildo has quit IRC17:05
*** raildo has joined #tripleo17:05
amoralejadarazs|rover, mwhahaha jpena it should be fixed now17:05
amoralejmetadata may take a while to be refreshed in the proxy cache17:06
*** moshele has quit IRC17:06
mwhahahaso i guess the quesiton I have is what happened?17:06
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Improve node count indicator in roles step  https://review.openstack.org/51667417:06
mwhahahathis is old code so i'm assuming env changed?17:06
* adarazs|rover doesn't understand. if this is part of the chroot setup, I assume it should have failed every single time.17:07
amoralejmwhahaha, today we switched to new dependencies repo17:07
amoralejwhich is *almost* the same content17:07
mwhahahaamoralej: did you announce it or test it?17:07
amoralejmwhahaha, yes17:07
jpenaadarazs|rover: when the chroot setup is done, missing packages are ignored (that's why it doesn't complain about a missing python3-pbr)17:07
jpenaadarazs|rover: but for some other packages, openstack-macros was a proper build requirement17:08
amoralejbut we missed some issues17:08
amoralejas one in kolla and this one17:08
mwhahahaamoralej: is this tracked somewhere for RCA later?17:08
adarazs|roverjpena: I see.17:08
amoralejproblem is that some time ago we switched openstack-macros from deps repo to delorean one17:08
amoralejbut17:08
adarazs|roverso that's why this just appeared now.17:08
amoralejas we haven't promoted the repo in last 50 days17:08
amoralejand you use current-passed-ci to build packages17:09
amoralejopenstack-macros is not in delorean repo17:09
amoralejso, short version. it'll be fixed by next promotion17:09
openstackgerritLuke Hinds proposed openstack/puppet-tripleo master: [WIP] Implements AIDE Intrusion Detection System  https://review.openstack.org/48302517:09
mwhahahaif we ever get a promotion at this rate17:09
amoraleji've applied a workaround by adding it to deps repo17:09
adarazs|roverthanks for the explanation.17:10
openstackgerritLuke Hinds proposed openstack/tripleo-heat-templates master: [WIP] Implements AIDE Intrusion Detection System  https://review.openstack.org/48302817:10
*** tosky has quit IRC17:11
*** lucasagomes is now known as lucas-afk17:12
*** catintheroof has joined #tripleo17:12
hamzyif I make a change to tripleo_common/actions/validations.py what do I need to do to get mistral to reload that class? "sudo mistral-db-manage --config-file /etc/mistral/mistral.conf populate" doesn't seem to do it17:12
*** bnemec has joined #tripleo17:12
sshnaidmcan anybody say what I do wrong in this template http://paste.openstack.org/show/625140/ , when I get error: ERROR: Failed to validate: Failed to validate: resources[0]: The specified reference "private_subnet" (in subnode_port.Properties.fixed_ips[0].subnet) is incorrect.17:14
openstackgerritOr Idgar proposed openstack/puppet-tripleo stable/pike: Remove erroneous octavia code related to sync_db  https://review.openstack.org/51337617:15
*** jprovazn has joined #tripleo17:15
bnemecsshnaidm: There's no private_subnet resource in that template.17:17
bnemecDid you mean get_param?17:17
sshnaidmbnemec, I pass it from parent template17:18
sshnaidmbnemec, at least I intent to do...17:18
bnemecsshnaidm: You don't pass resources, you pass parameters.  Try get_param.17:18
sshnaidmbnemec, thanks! seems like started to work *phew*17:20
sshnaidmseems like I understand why we move from heat to ansible17:20
bnemecReferencing something that doesn't exist in Ansible isn't going to work any better. ;-)17:22
* adarazs|rover going to log off for today, maybe checks back later on the promotions.17:23
adarazs|roversee ya!17:23
sshnaidmbnemec, yeah, I'm kidding..17:24
*** fragatina has quit IRC17:24
sshnaidmbnemec, but this stack was completed and nothing was created..17:25
*** fragatina has joined #tripleo17:25
sshnaidmbnemec, could be my problem is with output? http://paste.openstack.org/show/625141/17:25
bnemecsshnaidm: It does look like there's a problem with that output.17:27
sshnaidmbnemec, do you know maybe how do I output all IPs of resource group?17:28
*** rmascena has joined #tripleo17:28
*** jpich has quit IRC17:29
*** pcaruana has quit IRC17:29
*** fragatina has quit IRC17:29
*** trown|lunch is now known as trown17:30
*** raildo has quit IRC17:31
bnemecsshnaidm: I'm not positive how it works with a resource group, but you have no outputs defined on the subnode template so there's nothing for the parent template to return.17:36
*** tesseract has quit IRC17:36
bnemecI expect that's what the current error is saying.17:36
bnemecSo the first step will be to add outputs to subnode.yaml with whatever data you want exposed.17:37
sshnaidmbnemec, thanks, trying..17:37
openstackgerritMerged openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/51599117:39
*** d0ugal has quit IRC17:43
*** salmankhan has quit IRC17:43
sshnaidmbnemec, I did like that, and still have "null" in output.. http://paste.openstack.org/show/625146/17:49
*** fragatina has joined #tripleo17:50
EmilienMmwhahaha: https://review.openstack.org/#/c/516683/17:50
EmilienMmwhahaha: where is multinode-oooq17:50
mwhahahaEmilienM: click Toggle CI17:51
mwhahahalegacy-tripleo-ci-centos-7-nonha-multinode-oooq legacy-tripleo-ci-centos-7-nonha-multinode-oooq : ERROR Project openstack/requirements does not have the default branch master17:51
EmilienMmwhahaha: perfect17:51
*** ccamacho has quit IRC17:53
*** mcornea has joined #tripleo17:53
*** jfrancoa has quit IRC17:54
*** itlinux has quit IRC17:56
*** fzdarsky has quit IRC18:02
*** panda|ruck|bbl is now known as panda|ruck18:02
*** d0ugal has joined #tripleo18:03
*** itlinux has joined #tripleo18:06
*** dsariel__ has quit IRC18:09
*** zzzeek has quit IRC18:10
sshnaidmtrown, do you know maybe - how can I get outputs of resource group? I have "null"s with this configuration: http://paste.openstack.org/show/625148/18:10
*** jpena is now known as jpena|off18:10
sshnaidmany heat expert ^^18:10
TenguEmilienM mwhahaha darn, that might explain the CI failures I have on some review… didn't check further the error18:11
*** zzzeek has joined #tripleo18:12
trownsshnaidm: I am far from heat expert, but looking18:12
sshnaidmtrown, you have a magic look :) just worked now :D18:13
trownsshnaidm: nicest thing anyone said to me all month :)18:15
*** moshele has joined #tripleo18:15
*** agurenko_ has quit IRC18:16
*** oidgar has quit IRC18:17
*** zzzeek has quit IRC18:17
*** zzzeek has joined #tripleo18:18
*** dparkes has joined #tripleo18:19
*** zzzeek has quit IRC18:20
*** pchavva has quit IRC18:21
*** d0ugal_ has joined #tripleo18:22
*** d0ugal has quit IRC18:23
*** zzzeek has joined #tripleo18:25
*** hewbrocca is now known as hewbrocca_afk18:26
*** yprokule has quit IRC18:27
*** zzzeek_ has joined #tripleo18:27
*** anshul has joined #tripleo18:27
*** dparkes has quit IRC18:29
*** chlong has joined #tripleo18:30
*** zzzeek has quit IRC18:30
fpandsneddon: ping. Could you take a look at https://review.openstack.org/#/c/513972/ when you get a chance, thanks18:31
*** zzzeek_ has quit IRC18:31
*** toure is now known as toure_biab18:37
*** bnemec has quit IRC18:38
openstackgerritJames Slagle proposed openstack/tripleo-validations master: Specify vars after children sections  https://review.openstack.org/51676418:38
*** zzzeek has joined #tripleo18:39
openstackgerritJames Slagle proposed openstack/tripleo-quickstart master: fs10: deploy steps with ansible  https://review.openstack.org/50830718:40
slagleEmilienM: i think i have a more proper fix for the inventory issue ^^18:41
* EmilienM clicks18:41
*** newmember has joined #tripleo18:42
EmilienMslagle: wow, well done18:42
*** zzzeek has quit IRC18:42
openstackgerritAlfredo Moralejo proposed openstack/tripleo-puppet-elements master: Install yum-plugin-priorities in pre-install phase  https://review.openstack.org/51676518:43
EmilienMslagle: probably not urgent at this point but if you can take a look at https://review.openstack.org/#/c/514745/ and https://review.openstack.org/#/c/514736/ when you have time, and give feedback. I haven't followed config-download work over the last 7 days I think18:43
*** oidgar has joined #tripleo18:44
slagleEmilienM: yea i'll have a look. i need to update the workflows as well based on some comments18:44
*** zzzeek has joined #tripleo18:44
EmilienMslagle: ok18:45
*** dprince has quit IRC18:49
*** achadha_ has joined #tripleo18:53
*** achadha has quit IRC18:56
*** achadha_ has quit IRC18:58
*** dparkes has joined #tripleo18:59
*** yamahata has quit IRC18:59
panda|ruckall master OVB passed again.18:59
*** rbowen has joined #tripleo19:01
*** oidgar has quit IRC19:01
*** yamahata has joined #tripleo19:07
*** ebarrera_ has joined #tripleo19:07
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci master: Default $NODEPOOL_PROVIDER  https://review.openstack.org/49003719:08
openstackgerritBrent Eagles proposed openstack/tripleo-common master: Add ansible playbook for configuring octavia  https://review.openstack.org/48745019:08
openstackgerritBrent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper  https://review.openstack.org/51540219:08
*** dparkes has quit IRC19:10
*** salmankhan has joined #tripleo19:12
*** ipsecguy_ has joined #tripleo19:13
*** catintheroof has quit IRC19:14
*** yamahata has quit IRC19:14
*** ebarrera has quit IRC19:14
*** pcaruana has joined #tripleo19:15
*** dprince has joined #tripleo19:15
*** ipsecguy has quit IRC19:15
*** rbowen has quit IRC19:15
*** moshele has quit IRC19:15
*** yamahata has joined #tripleo19:17
*** moshele has joined #tripleo19:17
*** oidgar has joined #tripleo19:18
*** newmember has quit IRC19:23
*** dparkes has joined #tripleo19:24
*** pkovar has quit IRC19:25
*** cylopez has joined #tripleo19:26
*** adac has joined #tripleo19:26
adacHi there! With tripleo, can I setup a single node that contains all the openstack components?19:27
mwhahahaadac: well tripleo still requires 2 nodes. One for the undercloud and one for the overcloud, but yea we do it in CI19:28
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Increase the size of the Mistral output limit  https://review.openstack.org/51677119:29
*** mcornea has quit IRC19:32
adacmwhahaha, hmm ok I see. No clue yet what under and overcloud is, but will look it up thank!19:33
adac*thanks!19:33
*** cylopez has quit IRC19:34
adacmwhahaha, is there a good video tutorial that shows on how to setup tripleo/openstack btw?19:34
*** cylopez has joined #tripleo19:35
mwhahahaadac: I think we have some older ones. http://tripleo.org/index.html is a good set of resources/documentation19:35
adacmwhahaha, ok thanks I will have a look at it thanks19:38
adacmwhahaha, how long do you think I need to get familiar with all this from scratch? :)19:38
*** myoung is now known as myoung|afk19:39
panda|ruckcontainers build passed19:39
mwhahahaadac: a while :D it's a big learning curve19:39
adacmwhahaha, oh yes it looks like that :)19:40
*** oidgar has quit IRC19:42
homeskicurious if anyone knows what effect increasing api_workers parameter would have in ironic.conf19:44
*** salmankhan has quit IRC19:44
*** moshele has quit IRC19:46
*** amoralej is now known as amoralej|off19:48
*** pcaruana has quit IRC19:49
*** rbowen has joined #tripleo19:51
*** catintheroof has joined #tripleo19:55
*** rbowen has quit IRC19:56
*** rbowen has joined #tripleo19:56
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Add undercloud cinder roles and environment files  https://review.openstack.org/51480919:56
*** bfournie has quit IRC19:57
*** cylopez has quit IRC20:05
*** ccamacho has joined #tripleo20:10
*** gfidente is now known as gfidente|afk20:12
*** achadha has joined #tripleo20:13
panda|ruckI can almost smell the promotion20:15
*** jtomasek has quit IRC20:17
*** achadha has quit IRC20:17
*** mrunge has quit IRC20:18
*** etingof has quit IRC20:21
*** mrunge has joined #tripleo20:22
*** jmelvin has quit IRC20:23
*** ebarrera_ has quit IRC20:31
*** achadha has joined #tripleo20:35
*** achadha has quit IRC20:35
*** bfournie has joined #tripleo20:36
*** achadha has joined #tripleo20:36
*** bfournie has quit IRC20:37
*** sshnaidm is now known as sshnaidm|afk20:39
*** aputtur_ has quit IRC20:39
*** etingof has joined #tripleo20:42
openstackgerritSteve Baker proposed openstack/tripleo-docs master: Document skopeo as the tool used for image uploads  https://review.openstack.org/51645720:44
*** trown is now known as trown|outtypewww20:45
openstackgerritDavid Moreau Simard proposed openstack-infra/tripleo-ci master: Don't replace /root/.ssh/authorized_keys, append to it  https://review.openstack.org/51678520:45
panda|ruckomg20:49
panda|ruckomg20:49
*** ansmith has quit IRC20:49
EmilienMonly https://review.rdoproject.org/jenkins/job/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-master/119/console remains20:50
dmsimardyou'll jinx it20:51
panda|ruckdmsimard: the only thing left to do was staring at it20:53
EmilienMno timeout please20:56
panda|ruck....20:56
panda|ruckI think I'm going to tableflip20:57
panda|ruckI can't believe it20:58
EmilienMit passed21:02
panda|ruck\o/21:02
*** leitan has quit IRC21:02
panda|rucknobody moves21:02
panda|ruckstay exaclty where you are21:02
*** jcoufal_ has joined #tripleo21:02
panda|ruckI'll monitor the promotion part21:03
*** jcoufal has quit IRC21:05
*** toure_biab is now known as toure21:05
*** toure is now known as toure_biab21:05
EmilienMpanda|ruck: is it a post job?21:06
panda|ruckEmilienM: no, it's in a different server21:07
EmilienMpanda|ruck: where can we follow that?21:08
panda|ruckdmsimard: another periodic triggered right after this one21:08
panda|ruckEmilienM: http://38.145.33.13/master.log it's a bit unreadable at the moment21:08
*** catintheroof has quit IRC21:09
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: Get DLRN hash before setting up repos and images  https://review.openstack.org/51285521:09
*** rhallisey has quit IRC21:10
panda|ruckEmilienM: next run is in 1 minute21:10
panda|ruckdmsimard: why another pipeline triggered right after the last one ?21:11
panda|ruckpromotion is failing, because the tripleo-ci-testing link has already changed21:12
*** rbowen has quit IRC21:13
bugzyfultonj: I did not have that patch, just applied it and attempted deploy again, but it failed with the same error. I will delete the failed environment along with the nodes and start afresh with the patch to see if it works21:14
EmilienMpanda|ruck: not sure what that means, tbh21:14
panda|ruckEmilienM: no, it's another thing21:18
panda|ruckEmilienM: we are missing reports from two jobs21:18
panda|ruckperiodic-ovb-3ctlr_1comp-featureset002 and periodic-ovb-1ctlr_1comp-featureset02021:19
panda|ruckcheciing if they reported correctly21:19
panda|rucksuccess to DLRN21:19
EmilienM:-/21:20
adacwhat happens if the undercloud node fails?21:21
panda|ruckEmilienM: no it's first problem I mentioned21:24
panda|ruckEmilienM: so, every periodic pieline trigger starts with the promotion of the latest consistent to tripleo-ci-testing21:24
dmsimardpanda|ruck: I don't know ? I think the one that just finished was triggered manually by amoralej|off21:25
dmsimardmaybe it was due to run according to schedule21:26
panda|ruckthe second trigger changed the tripleo-ci-testing link, so the promoter script says it has not passed, because now it refers to a different hash21:27
panda|ruckI'll try to run with a fixed hash21:27
*** jmelvin has joined #tripleo21:27
EmilienMtbh, I have no idea what you're talking about, all of this area is unclear to me21:27
EmilienMwhere is this code?21:27
panda|ruckEmilienM: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/dlrnapi_promoter.py21:30
amoralej|offit should promote now21:31
amoralej|offpanda|ruck, i think not all jobs passed with the same hash21:31
amoralej|offwhen the promoter scripts run?21:31
*** mrunge has quit IRC21:31
amoralej|offperiodically?21:32
*** rcernin has joined #tripleo21:32
panda|ruckamoralej|off: yes21:33
*** anshul has quit IRC21:34
panda|ruckamoralej|off: it will not promote because tripleo-ci-testing changed in the meantime21:34
*** amoralej|off is now known as amoralej21:34
panda|ruckI'm hardocidng the hash values and running manually21:34
amoralejpanda|ruck, but in current execution missing jobs already passed21:34
panda|ruckIf not all jobs passed with the same hash we won't promote21:34
amoralejso, in first execution, last for jobs passed with second hash21:35
*** leitan has joined #tripleo21:35
amoralejand in current execution all but those four passed already21:35
*** threestrands has joined #tripleo21:36
amoralejpanda|ruck, EmilienM it has promoted \o/21:36
amoralejit's just the dashboard didn't refresh yet21:36
slagleEmilienM: boom. https://review.openstack.org/#/c/508307/21:37
slagleEmilienM: we better merge it before ansible does another release!21:37
EmilienMslagle: lol21:37
EmilienMamoralej: cool!21:37
panda|ruckwhy did we have two periodic runs at the same time ?21:38
panda|ruckamoralej:21:38
panda|ruckno no, this is bad21:38
amoralejoverlaping21:38
panda|ruckI don't know what promoted21:38
panda|ruckthis is not good, should never have happened21:38
*** mrunge has joined #tripleo21:39
panda|ruckcontainers images upload failed21:39
amoralejit didn't21:39
*** akrivoka has quit IRC21:41
panda|ruckI don't know what promoted21:41
*** bobh has quit IRC21:41
EmilienMslagle: we'll approve when https://review.openstack.org/#/c/516764/1 get +A :D good work21:42
panda|ruckhalf the jobs on the first trigger were working on 3b718f3fecc866332ec0663fa77e758f8346ab9321:44
panda|ruckthe other half on 3b718f3fecc866332ec0663fa77e758f8346ab9321:44
*** rmascena has quit IRC21:45
panda|ruckand we don't have container images in dockerhub21:45
*** jprovazn has quit IRC21:45
*** dprince has quit IRC21:46
panda|ruckI meant the other half on 94ad4df4585c86761c4eb4406080d2ede5e13e4821:47
panda|ruckthis is what promoted in the end 94ad4df4585c86761c4eb4406080d2ede5e13e4821:48
panda|ruckbut we're missing the container images21:48
*** dmarlin has quit IRC21:48
*** ansmith has joined #tripleo21:48
amoralejpanda|ruck, yeah21:48
*** jcoufal has joined #tripleo21:49
panda|ruckfailed: [localhost] (item=aodh-api) => {"failed": true, "item": "aodh-api", "msg": "Error pulling trunk.registry.rdoproject.org/master/centos-binary-aodh-api - code: None message: Error: image master/centos-binary-aodh-api:94ad4df4585c86761c4eb4406080d2ede5e13e48_b1259967 not found"21:49
panda|ruckare the images even there ?21:49
amoralejpanda|ruck, where do you see that?21:49
amoraleji see it's still running in https://review.rdoproject.org/jenkins/job/periodic-tripleo-centos-7-master-containers-build/247/console21:49
amoralejfor the promoted hash21:50
panda|ruckamoralej: the promoter script, the part that is supposed to reaupload the images to dockerhub21:50
amoralejoh21:50
*** jcoufal_ has quit IRC21:50
amoralejthe container-build job is not in the list of jobs to promote21:51
panda|ruckno, because the test on that hash count21:51
panda|rucknot the build itself21:51
panda|ruckand the test run only when the build succeded21:52
amoralejbut it seems the child jobs doesn't force to use the same hash21:52
amoralejdo they pull hash tag for containers?21:52
amoralejor something like tripleo-ci-testing?21:52
dsneddonfpan, It took me a while with distractions, but I reviewed https://review.openstack.org/#/c/513972/121:52
amoralejpanda|ruck, so, i'm not concerned about the content of the repo21:53
dmsimardamoralej, panda|ruck: periodic jobs typically don't overlap, maybe the fact that this one was triggered manually didn't prevent the overlap21:53
panda|ruckamoralej: it uses tripleo-ci-testing, because the jobs are supposed to run one at a time21:53
amoralejthe difference between the two hashes is just a commit21:53
amoralejbut we need to fix the containers thing21:53
panda|ruckdmsimard: that's what I suspect21:53
*** amodi has quit IRC21:53
fpandsneddon: thanks!21:53
amoralejdmsimard, yes, that's my guess too21:53
dmsimardamoralej: now, to tell if that's a bug or not... I have no clue21:54
amoralejpanda|ruck, can we manually upload containers if https://review.rdoproject.org/jenkins/job/periodic-tripleo-centos-7-master-containers-build/247/console finishes ok?21:54
panda|ruckamoralej: yes21:54
dsneddonfpan, Have you tested this change with actual hardware?21:54
amoralejok, then i think it's not that bad, just let's hope we don't hit any issue in the job21:54
dsneddonfpan, I would, but I don't have access to the right hardware.21:55
amoralejok it failed21:55
amoralej:(21:55
fpandsneddon: yeah, I did test it on my server, and had cisco test it too21:55
*** bnemec has joined #tripleo21:55
dsneddonfpan, Yeah, all the functional bits looked good to me, just a couple of questions about unit tests and sanity checking.21:55
panda|ruckwe have to revert the promotion21:56
amoralejhttps://logs.rdoproject.org/openstack-periodic/periodic-tripleo-centos-7-master-containers-build/14bf5a5/kolla/logs/opendaylight.log21:56
amoralejdmsimard, could we re-run only that job with sfmanager ?21:56
dmsimardamoralej: re-run that periodic job ? I'd rather stay away from enqueuing periodic jobs now :)21:57
fpandsneddon: yeah good catch, I used members[0] for both of the interfaces, that's why it passed with the wrong names eth1, will fix21:58
amoralejdmsimard, panda|ruck i need to leave now21:59
*** paramite has quit IRC21:59
amoralejwhat if we promote 3b718f3fecc866332ec0663fa77e758f8346ab9321:59
amoralejpanda|ruck, ^21:59
amoralejwe have containers for that21:59
amoralejin fact that's the hash actually used for the jobs that passed22:00
dmsimardI have to leave soon too, need to go out with the kids for halloween22:00
panda|ruckamoralej: because half of the jobs did not test that hash22:00
amoralejwhich ones?22:01
dsneddonfpan, As far as I'm concerned, it's a +2 with that fixed.22:01
amoralejpanda|ruck, look at https://review.rdoproject.org/jenkins/job/periodic-tripleo-centos-7-master-containers-build/246/22:01
dmsimardpanda|ruck, amoralej: if the jobs passed without too much of a hiccup, we can likely expect the next run to pass22:01
fpandsneddon: great, thanks again for the review22:01
dsneddonfpan, And thanks for the patch22:01
dmsimardif the jobs running right now are likely to fail, we should abort them to make sure the next run starts clean22:01
*** jcoufal has quit IRC22:02
amoralejdmsimard, yeah, hopefully but containers-build job is unreliable22:02
dmsimardregarding reverting the promotion.. I can help with that later if need be22:02
panda|ruckamoralej: featureset016-018 tested the other hash22:02
dmsimardnot sure what's that imply, but I have to go dress up as pikachu now22:02
amoralejpanda|ruck, those jobs are based on containers?22:02
* dmsimard afk22:02
panda|ruckamoralej: yes22:03
amoralejthen they used 3b718f3fecc866332ec0663fa77e758f8346ab93 although reported to the other one22:03
amoralejin fact, there are not containers for 94ad4df4585c86761c4eb4406080d2ede5e13e48_b1259967, so they couldn't use that hash22:03
panda|ruckamoralej: probably the used the containers from 3b718f3fecc866332ec0663fa77e758f8346ab93 but tested inside overcloud based on the other22:04
amoralejpanda|ruck, in any case, the different between the two is not tested22:04
amoralejit's just a commit in tripleo-ui22:04
amoralejso i'd say it's pretty safe to promote 3b7...22:05
amoralejbut anyway22:05
amoralejwe can wait22:05
amoralejfor next run22:05
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Multinode provision role  https://review.openstack.org/51679522:05
amoraleji'm leaving22:05
*** jmelvin has quit IRC22:05
*** amoralej is now known as amoralej|off22:05
panda|ruckall the gates are going to use the the images, and the containers jobs are going to fail without images22:05
panda|ruckI think I'll revert22:06
*** leitan has quit IRC22:08
*** bobh has joined #tripleo22:13
*** chlong has quit IRC22:14
*** moshele has joined #tripleo22:17
*** gfidente|afk has quit IRC22:19
*** bobh has quit IRC22:20
openstackgerritMerged openstack/tripleo-validations master: Specify vars after children sections  https://review.openstack.org/51676422:27
*** itlinux has quit IRC22:41
*** moshele has quit IRC22:42
*** cylopez has joined #tripleo22:45
*** cylopez has quit IRC22:46
*** rlandy has quit IRC22:49
panda|ruckthe dashboard did not reset22:52
*** jlabarre has quit IRC22:57
*** adac has quit IRC22:57
mwhahahashardy_afk, dsneddon: when you guys get a chance cna you re-review https://review.openstack.org/#/c/512395/ It had fallen into merge conflict and i cleaned it up since trozet isn't around23:16
dsneddonmwhahaha, Is the rdo-kolla-build-integration job broken, or did that patch not pass the test?23:17
mwhahahadsneddon: i don't think i've ever seen that pass.23:17
*** brault_ has joined #tripleo23:18
*** brault has quit IRC23:18
mwhahahadsneddon: seems broken on opendaylight23:18
*** leitan has joined #tripleo23:19
dsneddonmwhahaha, OK, I attempted to merge it.23:20
dsneddonmwhahaha, Thanks23:20
dsneddonmwhahaha, Since it had already received the necessary +2s and I tried to workflow +1 once already.23:20
openstackgerritMerged openstack/tripleo-heat-templates stable/newton: Bump fs.inotify.max_user_instances for scale  https://review.openstack.org/50952123:20
*** yamahata has quit IRC23:21
*** yamahata has joined #tripleo23:21
openstackgerritMerged openstack/instack-undercloud master: Remove legacy ceilometer api from undercloud  https://review.openstack.org/50793623:22
*** moshele has joined #tripleo23:23
*** chem has quit IRC23:25
mwhahahadsneddon: ok thanks23:27
mwhahahaHey look things are finally merging23:27
*** leitan has quit IRC23:29
*** leitan has joined #tripleo23:30
*** daidv has quit IRC23:35
openstackgerritMerged openstack/tripleo-heat-templates master: Don't open unused nova ports in iptables  https://review.openstack.org/51433023:35
openstackgerritMerged openstack/tripleo-quickstart master: Update images location to new ones  https://review.openstack.org/51036323:35
*** daidv has joined #tripleo23:35
*** moshele has quit IRC23:40
*** leitan has quit IRC23:40
*** moshele has joined #tripleo23:40
*** leitan has joined #tripleo23:44
*** bfournie has joined #tripleo23:47
openstackgerritJames Slagle proposed openstack/tripleo-common master: Additional tests for config-download  https://review.openstack.org/51150623:49
openstackgerritJames Slagle proposed openstack/tripleo-common master: Skip hidden files in test_tags_are_set  https://review.openstack.org/51287523:49
openstackgerritJames Slagle proposed openstack/tripleo-common master: Add workflow for deployment via config-download  https://review.openstack.org/51287623:49
openstackgerritJames Slagle proposed openstack/tripleo-common master: Log Ansible actions to ansible.log  https://review.openstack.org/51478323:49
openstackgerritJames Slagle proposed openstack/tripleo-common master: Add work_dir for some actions  https://review.openstack.org/51478423:50
*** moshele has quit IRC23:50
openstackgerritJames Slagle proposed openstack/python-tripleoclient master: Add --config-download  https://review.openstack.org/51470123:50
*** moshele has joined #tripleo23:54
*** achadha has quit IRC23:56
*** achadha has joined #tripleo23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!