Friday, 2017-08-04

fultonj...which was.... (/me getting an example)00:00
fultonje.g. http://git.openstack.org/cgit/openstack/tripleo-heat-templates/plain/docker/docker-toool00:00
fultonjpython tool.py -c $ID00:00
fultonjthe above shows '/bin/docker run --name ...  --volume=/foo:/bar ...'00:01
fultonji am basically looking for evidence that a change did what i expected00:01
fultonjeven the ouput of 'docker inspect $ID' would be fine00:02
larsksfultonj: I am not looking at anything that would aid in this sort of inspection.00:02
fultonjlarsks: ok, thanks for the info00:03
*** Goneri has joined #tripleo00:03
fultonji was thinking of just scrapping something together to grab that00:03
fultonj(or reproduce locally and wait for ci with fingers crossed)00:04
stevebakerfultonj: when python-paunch ffe is approved then you'll be able to use "paunch debug" to collect some info00:09
fultonjstevebaker: cool00:09
*** rbowen has joined #tripleo00:10
fultonjanyway to cheat and try it today in CI?00:10
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855400:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,Triaged]00:10
stevebakerfultonj: it replaces docker-toool, so you00:10
fultonjnot via a real commit, but putting up a depends-on to a WIP which hacks it in00:10
stevebaker'll need to use that for now00:10
fultonjstevebaker: ok00:10
fultonjthanks for connecting those two items for me00:11
stevebakerfultonj: "paunch debug" has landed for a while now, but it missed the release window00:11
fultonjstevebaker: is paunch going to get integrated with our CI?00:11
fultonji can say that better00:11
fultonjare there plans for our CI to leverage it?00:11
stevebakerfultonj: not that I know of - there is potential I guess00:12
*** catintheroof has quit IRC00:14
fultonjok00:14
*** itlinux has joined #tripleo00:16
*** gbarros has quit IRC00:23
*** nyechiel has quit IRC00:31
*** psahoo has joined #tripleo00:32
*** dmacpher has joined #tripleo00:36
*** myoung is now known as myoung|gone00:37
*** myoung|gone is now known as myoung00:37
*** nyechiel has joined #tripleo00:40
*** psahoo has quit IRC00:40
*** dixiaoli has joined #tripleo00:46
*** gbarros has joined #tripleo00:48
*** dixiaoli has quit IRC00:49
*** dixiaoli has joined #tripleo00:49
*** nyechiel has quit IRC00:50
*** rbowen has quit IRC00:52
*** psahoo has joined #tripleo00:52
*** slagle has quit IRC00:53
*** slagle has joined #tripleo00:54
*** itlinux has quit IRC00:57
*** limao has joined #tripleo01:00
*** limao has quit IRC01:01
*** limao has joined #tripleo01:01
*** itlinux has joined #tripleo01:02
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855401:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,Triaged]01:10
*** ooolpbot has quit IRC01:10
*** Goneri has quit IRC01:16
*** psahoo has quit IRC01:25
*** limao has quit IRC01:29
*** trozet has joined #tripleo01:32
stevebakerianw: hey, ramishra just pointed me at this dib-utils retirement note https://github.com/openstack/diskimage-builder/blob/958ea8a3372157fe62c3a2bb1fcd55a9cadedab1/releasenotes/notes/remove-dib-utils-37f70dfad54900a0.yaml01:34
ianwstevebaker: yes, we moved it into a private call for dib01:35
stevebakerianw: os-refresh-config depends on dib-run-parts, preferably without installing diskimage-builder. I'm just wondering what we should do about that01:35
ianwit still exists, just dib doesn't depend on it01:35
ramishraianw: so the project is not retired as mentioned in the releasenote?01:36
ianwramishra: well, if someone said "i want to use dib-run-parts for XYZ" i'd suggest to them not to01:36
ramishraianw: I understand diskimage-builder uses an internal one01:37
*** trozet has quit IRC01:37
ianwbut, it's not like we're going to delete what's there01:37
stevebakerok, as long as dib-utils isn't going away without a plan for users of it01:37
*** limao has joined #tripleo01:38
*** psahoo has joined #tripleo01:38
ramishrastevebaker: but os-refresh-config should possibly remove the dependency in the future?01:38
ramishramay be use something internal like dib01:39
stevebakerramishra: yeah, we could just copy our own internal one, orc-run-parts01:40
*** yamahata has quit IRC01:41
stevebakerramishra: you would still need this symlink though, for source installs https://review.openstack.org/#/c/49050001:41
ianwfor dib, there was some thought we may re-implement the runner in python01:42
ramishrastevebaker: yeah01:42
*** pmannidi has joined #tripleo01:43
stevebakerianw: shame we can't share that now that it is inside dib ;)01:43
ramishraianw: any specific reason dib has it's own dib-run-parts?01:44
ianwramishra: so that we can fiddle with it as we like, and not break os-run-parts :)01:45
ianwos-refresh-config i mean01:45
ianwi was going to say, that the os's "run-parts" might work just as well too01:46
stevebakerianw: now we're really getting into a history lesson :) dib-run-parts was written because the fedora run-parts was ancient and lacked features, even though the debian one was decent01:47
*** tzumainn has quit IRC01:52
ianwstevebaker: yeah, we went through that with spamaps at the time, and decided that the idea of "dib-utils" hadn't quite worked out as nothing else was in there, or looked like being in there01:54
stevebakerianw: sure, it was literally just for os-refresh-config to use dib-run-parts01:55
openstackgerritMerged openstack/python-tripleoclient stable/newton: Install policycoreutils-python  https://review.openstack.org/49064601:58
*** _mmethot_ has quit IRC02:02
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855402:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,Triaged]02:10
*** ooolpbot has quit IRC02:10
*** http_GK1wmSU has joined #tripleo02:13
*** http_GK1wmSU has left #tripleo02:14
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Convert scenario001-multinode-containers job to ceph-ansible  https://review.openstack.org/47928802:16
*** mmethot has joined #tripleo02:32
*** gbarros has quit IRC02:32
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262302:36
*** mmethot has quit IRC02:43
*** ramishra has quit IRC02:48
*** ramishra has joined #tripleo02:50
*** udesale has joined #tripleo02:51
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/48739802:53
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814802:53
openstackgerritMerged openstack/tripleo-heat-templates master: Update EventPipelinePublisher param description to include zaqar  https://review.openstack.org/48786103:08
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855403:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,Triaged]03:10
*** ooolpbot has quit IRC03:10
*** ykarel has joined #tripleo03:22
*** ykarel has quit IRC03:29
*** mmethot has joined #tripleo03:30
*** mmethot has quit IRC03:31
*** mmethot has joined #tripleo03:31
EmilienMthrash, mwhahaha : self approving https://review.openstack.org/#/c/49067703:38
EmilienMto unblock ovb03:38
*** gkadam has joined #tripleo03:43
EmilienMfultonj: around?03:43
EmilienMmichapma: http://logs.openstack.org/05/486905/15/experimental/gate-tripleo-ci-centos-7-scenario008-multinode-oooq-nv/9aeb87a/logs/subnode-2/var/log/neutron/server.log.txt.gz#_2017-08-03_18_26_18_43703:45
EmilienMAssertionError: Wrong port binding controller is specified03:45
EmilienMsounds like a real error in config03:45
*** dbecker has quit IRC03:52
*** shreshtha has joined #tripleo04:00
*** links has joined #tripleo04:03
*** ykarel has joined #tripleo04:07
*** dbecker has joined #tripleo04:08
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855404:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,Triaged]04:10
*** ooolpbot has quit IRC04:10
*** limao has quit IRC04:17
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Bump fedora/fedora-minimal DIB_RELEASE 26  https://review.openstack.org/48257004:18
*** prateek has joined #tripleo04:19
*** skramaja has joined #tripleo04:20
*** Vijayendra has joined #tripleo04:32
openstackgerritMerged openstack/puppet-pacemaker master: Pin puppet-stdlib to 4.16.0  https://review.openstack.org/49050704:34
*** pdeore has joined #tripleo04:37
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814804:44
*** limao has joined #tripleo04:58
openstackgerritMerged openstack/diskimage-builder master: Updated from global requirements  https://review.openstack.org/48798905:01
*** ratailor has joined #tripleo05:07
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855405:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)05:10
*** ooolpbot has quit IRC05:10
*** janki has joined #tripleo05:12
openstackgerritMerged openstack/diskimage-builder master: Allow users to specify partition type in the MBR PTE  https://review.openstack.org/48239605:19
*** dpawar has joined #tripleo05:19
*** yamahata has joined #tripleo05:21
openstackgerritJanki Chhatbar proposed openstack/tripleo-heat-templates master: Containarise Barbican API  https://review.openstack.org/48145105:29
*** psachin has joined #tripleo05:29
*** jchhatbar has joined #tripleo05:34
*** janki has quit IRC05:36
*** paramite has joined #tripleo05:38
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Use number for KeystoneCronTokenFlushMaxDelay instead of string  https://review.openstack.org/49074805:38
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Use number for KeystoneCronTokenFlushMaxDelay instead of string  https://review.openstack.org/49074805:42
*** pgadiya has joined #tripleo05:48
*** masco has joined #tripleo05:51
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-common master: Use nova-api image for configuration as well  https://review.openstack.org/49038505:53
*** marios has joined #tripleo05:58
*** yprokule has joined #tripleo06:00
*** cshastri has joined #tripleo06:03
*** jfrancoa has joined #tripleo06:04
*** lmiccini has joined #tripleo06:06
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855406:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)06:10
*** ooolpbot has quit IRC06:10
*** pgadiya has quit IRC06:17
*** pgadiya has joined #tripleo06:18
*** Nakato has left #tripleo06:19
jaosoriormarios: can you check these two when you get the chance https://review.openstack.org/#/c/489891/  https://review.openstack.org/#/c/489896/ ?06:22
jaosoriorbandini: ^^06:22
jaosoriordciabrin_: around yet?06:22
mariosjaosorior: ack i have it in my notes from last night will do at some point :)06:27
bandinijaosorior: on it (sorry yesterday was the whole day on the phone)06:30
*** rcernin has joined #tripleo06:30
*** itlinux has quit IRC06:30
jaosoriorbandini: no worries06:31
jaosoriormarios, bandini thanks guys06:31
openstackgerritMerged openstack/tripleo-quickstart-extras master: Use prepare command for overcloud_containers.yaml  https://review.openstack.org/47597506:33
openstackgerritMerged openstack/tripleo-heat-templates master: Changing the default port-binding configuration  https://review.openstack.org/48557306:33
*** dmacpher has quit IRC06:35
*** anshul has joined #tripleo06:38
jaosoriormandre: how does the promotion of images work? does that happen automatically or is that something that one still does manually?06:41
jaosorior*container images06:41
openstackgerritlidong proposed openstack/tripleo-quickstart-extras master: Fix a typo  https://review.openstack.org/49075706:42
*** ebarrera has joined #tripleo06:48
*** nyechiel has joined #tripleo06:48
*** jpena|off is now known as jpena06:52
openstackgerritlidong proposed openstack/tripleo-quickstart-extras master: Update links in README  https://review.openstack.org/49076106:52
*** jlinkes has joined #tripleo06:54
*** milan has joined #tripleo07:00
*** bogdando has joined #tripleo07:04
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855407:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)07:10
*** ooolpbot has quit IRC07:10
*** mcornea has joined #tripleo07:13
*** tesseract has joined #tripleo07:16
*** nyechiel has quit IRC07:16
d0ugalapetrich: ^ you looked at this?07:18
dciabrin_jaosorior, hey o/07:18
apetrichd0ugal, nope. looking now07:18
d0ugalapetrich: cool, me too - just need to get coffee etc. - back in 5-1007:18
*** amoralej|off is now known as amoralej07:23
jaosoriordciabrin_: yooo, lets fix all the shits :D07:26
dciabrin_jaosorior, absolutely!07:27
*** jpena is now known as jpena|brb07:30
*** Vijayendra has quit IRC07:31
*** ccamacho has joined #tripleo07:34
*** pcaruana has joined #tripleo07:38
*** http_GK1wmSU has joined #tripleo07:43
*** http_GK1wmSU has left #tripleo07:43
d0ugalapetrich: http://logs.openstack.org/72/480272/26/check-tripleo/gate-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/4ce0ea6/logs/undercloud/var/log/mistral/mistral-db-manage.log.txt.gz#_2017-08-03_20_46_47_59207:44
d0ugalapetrich: I think that mistral patch broke things again.07:44
*** dparkes has quit IRC07:44
d0ugalapetrich: but I see EmilienM has some patches in flight, so maybe they will sort it out.07:45
apetrichd0ugal, aye and EmilienM patch seems to work but I'm not seeing where "Failed to find action" at that point would be keystone related07:47
*** Vijayendra has joined #tripleo07:47
*** jtomasek has joined #tripleo07:48
*** ecerquei has quit IRC07:48
*** egonzalez has joined #tripleo07:49
apetrichd0ugal, oh I see07:49
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Add 'discover and enroll nodes' workflow  https://review.openstack.org/48852607:49
*** arxcruz is now known as arxcruz|sick07:49
apetrichd0ugal, so happy that my new debug db-manage was able to capture that :)07:50
apetrich"new" as in this release07:51
d0ugalapetrich: I am very happy with it too!07:51
d0ugalapetrich: what did you find?07:51
*** ecerquei has joined #tripleo07:51
*** cylopez has joined #tripleo07:52
apetrichd0ugal, just saw the logs that you pointed out to the fail to create action07:52
*** ecerquei has quit IRC07:53
d0ugalapetrich: ah07:53
*** ecerquei has joined #tripleo07:53
d0ugalRevert "Revert "Revert "Add support for Keystone authtoken"""07:54
d0ugal^ best commit message.07:54
d0ugalTrying to figure out if that adds or removes it!07:54
*** jpena|brb is now known as jpena07:55
*** ffiore has joined #tripleo07:55
*** Vijayendra has quit IRC07:57
*** ckyriakidou has joined #tripleo08:04
*** panda|4g has joined #tripleo08:07
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855408:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)08:10
*** ooolpbot has quit IRC08:10
*** Vijayendra has joined #tripleo08:10
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Modify resource dependencies of certmonger_user resources  https://review.openstack.org/49079108:17
jaosoriordciabrin_: lets see how this works ^^08:17
dciabrin_jaosorior, w00t08:21
*** tvignaud has quit IRC08:23
*** dobson` has quit IRC08:24
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test  https://review.openstack.org/47574708:25
*** derekh has joined #tripleo08:25
rnoriegajaosorior, Buenos días! :-) Could you take a look at this env file? https://review.openstack.org/#/c/486187/08:25
rnoriegaccamacho, ^^08:25
*** paramite has quit IRC08:27
jaosoriorrnoriega: no08:28
jaosoriorrnoriega: nah, just kidding. +A08:28
rnoriegajaosorior, hahahah08:28
rnoriegajaosorior, gracias amigo!08:28
jaosoriorno hay problema. Ahí me avisas si necesitas ayuda con mas parches08:29
*** dciabrin_ is now known as dciabrin08:29
rnoriegajaosorior, yay! :-)08:30
*** iranzo has joined #tripleo08:30
*** iranzo has joined #tripleo08:30
*** dobson has joined #tripleo08:31
*** pmannidi has quit IRC08:32
*** tvignaud has joined #tripleo08:34
*** yamahata has quit IRC08:39
*** ffiore has quit IRC08:39
*** social has joined #tripleo08:40
ccamachornoriega jaosorior you are too fast for me on a Friday08:42
rnoriegaccamacho, jaosorior is too fast, too furious!08:42
jaosoriorbeing too furious is part of working with openstack08:45
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-common master: Use nova-api image for configuration as well  https://review.openstack.org/49038508:46
*** salmankhan has joined #tripleo08:51
*** hewbrocca_afk is now known as hewbrocca08:51
*** lucas-afk is now known as lucasagomes08:55
bogdandohi folks. Could you please assist with debugging a cryptic "Error rendering template network/network.network.yaml : 'network' is undefined" message from overcloud deploy mistral workflow?08:55
bogdandothe tripleo.plan_management.v1.update_deployment_plan08:56
bogdandosomething bad I have in my templates for sure, but have no ideas where to start debugging it08:56
*** ecerquei has quit IRC09:01
jistrsounds like a jinja error, so i think the undefined var is this one09:02
jistrhttps://github.com/openstack/tripleo-heat-templates/blob/master/network/network.network.j2.yaml#L409:02
jistrand i bet the solution would be to update tripleo-common, likely this is missing in your env09:02
jistrhttps://github.com/openstack/tripleo-common/commit/e48a73cb4ef2d40cfdd322e7cc19f615aa4ad1c709:02
jistrbogdando: ^09:02
bogdandojistr: thanks!09:02
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Add validation task in docker services [cinder]  https://review.openstack.org/48994009:05
*** iranzo has quit IRC09:07
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855409:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)09:10
*** ooolpbot has quit IRC09:10
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates master: Add IronicPxe to the default controller  https://review.openstack.org/49051209:11
*** ansmith has quit IRC09:11
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service  https://review.openstack.org/49005009:20
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service  https://review.openstack.org/49005009:22
*** ansmith has joined #tripleo09:24
*** tosky has joined #tripleo09:25
*** limao has quit IRC09:26
*** paramite has joined #tripleo09:26
*** limao has joined #tripleo09:26
*** limao has quit IRC09:31
*** dixiaoli has quit IRC09:34
*** psahoo has quit IRC09:35
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates master: WIP: Adds pacemaker update_tasks for Pike minor update workflow  https://review.openstack.org/48656709:39
*** tosky has quit IRC09:44
*** pgadiya has quit IRC09:45
*** nyechiel has joined #tripleo09:48
jaosoriorbandini: you around?09:49
*** iranzo has joined #tripleo09:51
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Add timestamper_cmd for tripleo-validations  https://review.openstack.org/49001309:52
*** dtantsur|afk is now known as dtantsur09:55
*** stendulker has joined #tripleo09:55
*** paramite has quit IRC09:59
*** iranzo has quit IRC10:01
*** pgadiya has joined #tripleo10:02
*** achadha has joined #tripleo10:02
*** akrivoka has joined #tripleo10:03
achadhahey guys, seeing this weird error after doing a 'yum update' on the undercloud. Problem wasn't seen before update, so flavor-vs-ironic-capabilities are solid: deploy keeps failing with 'no hosts found error'. There are enough available ironic nodes with the correct capability. Ironic conductor says something like:10:04
achadha2017-08-04 02:59:07.662 30457 ERROR ironic.drivers.modules.agent_base_vendor Command: sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 192.0.2.11:3260 -T iqn.2008-10.org.openstack:a4d38a10:04
achadhaf8-fa76-42ad-8a41-c9449d454acc --login10:04
*** akrivoka has quit IRC10:04
achadhaAnyone seen something like this before?10:05
*** akrivoka has joined #tripleo10:06
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test  https://review.openstack.org/47574710:07
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855410:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)10:10
*** ooolpbot has quit IRC10:10
*** janki has joined #tripleo10:11
*** jchhatbar has quit IRC10:12
openstackgerritMerged openstack/tripleo-heat-templates master: Adds environment file for ODL + SRIOV  https://review.openstack.org/48618710:16
*** openstackgerrit has quit IRC10:18
*** achadha has quit IRC10:20
*** achadha has joined #tripleo10:23
bandinijaosorior: kind of (will go to lunch in a bit). what's up?10:25
*** jchhatbar has joined #tripleo10:27
jaosoriorbandini: aaah no biggie, let me know when you;'re back from lunch10:28
*** janki has quit IRC10:29
*** openstackgerrit has joined #tripleo10:30
openstackgerritRaoul Scarazzini proposed openstack/tripleo-docs master: Add name and comment fields desc for instackenv  https://review.openstack.org/48958710:30
*** gcerami has joined #tripleo10:36
bogdandofolks, could we merge https://review.openstack.org/#/c/486135 ?10:38
*** panda|4g has quit IRC10:39
jaosoriorbogdando: what's that --heat-native parameter you removed?10:40
*** achadha_ has joined #tripleo10:40
*** achadha has quit IRC10:40
*** CaptTofu___ is now known as CaptTofu10:41
bogdandojaosorior: it uses a heat-all container instead of the heat host services10:41
bogdandoUC10:41
jaosorioraah ok10:42
bogdandoand the container comes from the tripleo containers source, which can be custom by that patch10:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Change the directory for haproxy certs/keys to be service-specific  https://review.openstack.org/48989610:43
*** achadha_ has quit IRC10:47
*** achadha has joined #tripleo10:47
*** udesale has quit IRC10:48
dtantsurmorning folks! may I please bother you with two simple patches again? https://review.openstack.org/#/c/489228/ and https://review.openstack.org/#/c/487175/10:48
dtantsurthe former is a blocker for ironic in the overcloud10:48
dtantsurI could also use some puppet review on https://review.openstack.org/#/c/490026/ it's also a blocker for us10:49
dtantsurcheers jaosorior :)10:50
jaosoriorglad to help10:50
*** snecklifter has joined #tripleo10:51
jaosoriordciabrin: can you check this out https://review.openstack.org/#/c/489132/3/puppet/services/pacemaker.yaml ? not sure what's missing here.10:51
*** lmiccini has quit IRC10:52
dciabrinjaosorior, ha right, there are two keys in fact. one for the cluster and one for the remotes, I had it wrong last time we discussed that10:53
*** jtomasek has quit IRC10:54
jaosoriordciabrin: ok. so, were we setting before the other key?10:54
dciabrinjaosorior, the remote key is created by puppet-tripleo, the cluster key is created by pcs when it sets up a new cluster if i understand correctly10:55
bogdandojistr: that was not enough to fix the undefined 'network'... I wonder how to trace the values down10:55
bogdandosomething goes wrong, but?..10:55
jaosoriordciabrin: wait...is /etc/corosync/authkey autogenerated by pacemaker?10:56
dciabrinjaosorior, yes I think it is. but only when you "pcs cluster setup". I have to double check that with bandini probably10:56
bogdandojistr: here is my env file https://pastebin.com/JChgdPBM oooq creates for me10:56
jaosoriordciabrin: ok, lets wait til he gets back from lunch... What bothers me is: How does pacemaker communicate the "secret" key to the other nodes in the cluster?10:57
dciabrinjaosorior, iirc there's one key in /etc/pacemaker/authkey and one is /etc/corosync/authkey10:57
dciabrinjaosorior, it pushes it via pcsd, every node has a daemon to replicate the pcs actions10:58
*** ansmith has quit IRC10:58
*** ykarel is now known as ykarel|away10:59
jaosoriordciabrin: right, but that goes in plaintext10:59
jaosoriordciabrin: it shouldn't be trying to replicate a "secret" key that it'll use for communications10:59
dciabrinjaosorior, let's hope not :)10:59
jaosoriorthat's a terrible idea11:00
dciabrinjaosorior, right, I see what you mean obviously I don't know how the internal works11:00
jaosoriordciabrin: however, there seems to be a way to set it via puppet11:00
jaosoriordciabrin: https://github.com/openstack/puppet-pacemaker/blob/master/manifests/new/setup/auth_key.pp11:00
dciabrinjaosorior, oh wow i didn't know that.11:01
*** jkilpatr has joined #tripleo11:01
*** nyechiel has quit IRC11:02
jaosoriordciabrin: I'm not sure if we ever call that though11:02
*** tosky has joined #tripleo11:02
jistrbogdando: hmm strange... not sure how to trace mistral+tripleo-common issues but maybe i'd first try restarting all mistral services. It may be that they didn't pick up the change.11:03
dciabrinjaosorior, I think we never did that, I don't recall having used encrypted corosync cluster.11:03
jaosoriordciabrin: so, the default value is undef. So that's not really used by default either11:03
openstackgerritMerged openstack/puppet-tripleo master: Ensure directory exists for certificates for haproxy  https://review.openstack.org/48989111:04
jaosoriorbut, I think we donp't call any resource that would call that11:04
dciabrinjaosorior, ah, ack11:05
*** jchhatbar has quit IRC11:06
*** Vijayendra has quit IRC11:06
*** pkovar has joined #tripleo11:08
*** rdopiera has left #tripleo11:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855411:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)11:10
*** achadha has quit IRC11:13
d0ugalbogdando: I can try and help you debug a workflow if that is what you need.11:13
*** athomas has quit IRC11:13
*** achadha has joined #tripleo11:14
*** athomas has joined #tripleo11:14
openstackgerritDerek Higgins proposed openstack/tripleo-common master: Truncate command lenth in healthcheck_port  https://review.openstack.org/49083211:18
openstackgerritDerek Higgins proposed openstack/tripleo-common master: Truncate command length in healthcheck_port  https://review.openstack.org/49083211:19
*** achadha has quit IRC11:30
*** lucasagomes is now known as lucas-hungry11:31
*** achadha has joined #tripleo11:35
*** pchavva has joined #tripleo11:36
openstackgerritDerek Higgins proposed openstack/tripleo-common master: Add ironic-api healthcheck  https://review.openstack.org/49083611:37
*** http_GK1wmSU has joined #tripleo11:38
*** http_GK1wmSU has left #tripleo11:41
*** achadha has quit IRC11:44
*** mdnadeem has joined #tripleo11:44
*** janki has joined #tripleo11:44
*** achadha has joined #tripleo11:45
*** stendulker has quit IRC11:46
*** achadha has quit IRC11:50
openstackgerritPranali Deore proposed openstack/tripleo-heat-templates master: Mount NFS volume to docker container.  https://review.openstack.org/49083911:51
*** ratailor has quit IRC11:51
*** MrChao has joined #tripleo11:51
*** shreshtha has quit IRC11:54
*** jpena is now known as jpena|lunch11:54
*** morazi has joined #tripleo11:54
*** skramaja has quit IRC11:55
openstackgerritJames Slagle proposed openstack/python-tripleoclient stable/newton: Add --skip-deploy-identifier  https://review.openstack.org/48981211:57
*** masco has quit IRC12:00
*** dprince has joined #tripleo12:00
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Enhance completion message when upgrading non controller nodes  https://review.openstack.org/48286812:00
*** jlabarre has joined #tripleo12:06
*** mmethot has quit IRC12:09
*** ykarel|away is now known as ykarel12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855412:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)12:10
*** ooolpbot has quit IRC12:10
*** atoth has quit IRC12:11
bandinidciabrin, jaosorior: yes I believe authkey is generated automagically. I need to verify that12:12
jaosoriorbandini: and how is that communicated to the other nodes in the cluster?12:12
bandinijaosorior: I *think* it is pcs doing the magic12:12
bandinibut need to doublecheck tbh12:12
jaosoriorbandini: right, but that means then that it is initially passed in cleartext :/12:12
jaosoriorwhich is no good12:13
bandinijaosorior: not necessarily. pcs creates a secure channel on the pcsd and passes files through that. the secure channel is created using the hacluster credentials which have to be set on each node12:13
jaosoriorbandini: which credentials?12:14
bandinijaosorior: pcs to work needs all nodes to have hacluster linux user to exist and have the same password on all nodes. it uses that (via ssh) to setup the secure pcsd communication12:15
bandiniiirc12:15
jaosoriorbandini: ooooh12:15
jaosoriorbandini: well, that's another story then :D12:15
bandiniyeah :)12:15
jaosoriorbandini: thanks for the details.12:15
bandininp12:15
jaosoriorbandini: soo... it is still not clear what else needs to be done here, right? https://review.openstack.org/#/c/489132/3/puppet/services/pacemaker.yaml12:16
bandinijaosorior: I need to double check what manual steps would be needed to "upgrade" an existing cluster to an encrypted one, yes12:16
bandiniit's on my list12:16
*** mdnadeem has quit IRC12:17
*** pdeore has quit IRC12:18
*** dougbtv_ has joined #tripleo12:20
*** sbrzozow has joined #tripleo12:20
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras master: WIP: Support for TripleO centralized logging  https://review.openstack.org/49062312:21
*** dougbtv_ is now known as dougbtv_laptop12:23
*** dprince has quit IRC12:24
jaosoriorbandini: also, could you check this out https://review.openstack.org/#/c/490748/ :D12:24
thrashEmilienM: I think I *finally* have an idea of why this is failing.12:25
*** akrivoka has quit IRC12:26
*** weshay_pto is now known as weshay12:26
*** liverpooler has joined #tripleo12:26
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart master: enable container periodic job  https://review.openstack.org/47574312:27
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814812:28
openstackgerritLee Yarwood proposed openstack/tripleo-docs master: Fix typo in containers image prep command  https://review.openstack.org/49084212:31
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart master: Add featureset023, tests up to overcloud images build  https://review.openstack.org/48780112:31
*** gkadam has quit IRC12:31
openstackgerritMerged openstack/tripleo-heat-templates master: Fix up multipath docker indentation  https://review.openstack.org/48995212:31
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test  https://review.openstack.org/47574712:32
*** jtomasek has joined #tripleo12:33
*** ansmith has joined #tripleo12:34
*** pdeore has joined #tripleo12:34
*** prateek has quit IRC12:35
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Modify resource dependencies of certmonger_user resources  https://review.openstack.org/49079112:35
*** lblanchard has joined #tripleo12:36
*** atoth has joined #tripleo12:36
*** dprince has joined #tripleo12:37
*** rhallisey has joined #tripleo12:37
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Modify resource dependencies of certmonger_user resources  https://review.openstack.org/49079112:37
*** akrivoka has joined #tripleo12:38
*** gbarros has joined #tripleo12:38
*** pdeore_ has joined #tripleo12:38
*** pdeore_ has quit IRC12:39
*** pdeore_ has joined #tripleo12:39
*** ebarrera has quit IRC12:40
*** pdeore has quit IRC12:40
*** ebarrera has joined #tripleo12:42
*** Goneri has joined #tripleo12:43
*** amoralej is now known as amoralej|lunch12:43
*** rbowen has joined #tripleo12:45
openstackgerritMarios Andreou proposed openstack/tripleo-common master: Adds download of config and execution of the deploy_steps_playbook  https://review.openstack.org/49084712:47
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: WIP: also write an upgrade_tasks playbook  https://review.openstack.org/49084812:47
*** jkilpatr has quit IRC12:49
openstackgerritDmitry Tantsur proposed openstack/instack-undercloud master: [WIP] Set all nodes resource class to "baremetal" on undercloud install  https://review.openstack.org/49085112:49
openstackgerritMarios Andreou proposed openstack/python-tripleoclient master: Add --config-path to specify a named directory for stack config  https://review.openstack.org/49085212:49
*** lucas-hungry is now known as lucasagomes12:50
openstackgerritMarios Andreou proposed openstack/tripleo-common master: Adds download of config and execution of the deploy_steps_playbook  https://review.openstack.org/49084712:50
*** lblanchard has quit IRC12:51
*** paramite has joined #tripleo12:52
*** pgadiya has quit IRC12:54
*** jpena|lunch is now known as jpena12:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: EnvironmentConfiguration changes  https://review.openstack.org/48389612:56
honzajtomasek: new error handling patches are finally succeeding in ci :)12:58
jtomasekhonza: I just wanted to ask about it. is it due to updated rdo packages?12:58
*** jmelvin has joined #tripleo12:58
honzajtomasek: yes, there was some issue but amoralej|lunch fixed all the things :)12:59
honzajtomasek: new deps are now available so everything should work again12:59
jtomasekhonza: awesome news! thanks amoralej|lunch :)12:59
*** atoth has quit IRC13:00
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814813:01
*** sbrzozow has quit IRC13:03
*** jkilpatr has joined #tripleo13:05
*** pdeore_ has quit IRC13:06
*** tzumainn has joined #tripleo13:06
*** pgadiya has joined #tripleo13:07
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855413:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)13:10
*** ooolpbot has quit IRC13:10
*** Goneri has quit IRC13:11
*** gbarros has quit IRC13:11
*** lblanchard has joined #tripleo13:12
*** maeca has joined #tripleo13:13
*** lblanchard has quit IRC13:13
*** lblanchard has joined #tripleo13:13
*** achadha has joined #tripleo13:16
*** sbrzozow has joined #tripleo13:16
*** catintheroof has joined #tripleo13:17
*** gbarros has joined #tripleo13:17
*** jcoufal has joined #tripleo13:17
openstackgerritMerged openstack/tripleo-quickstart-extras master: Undercloud deploy custom sources of container images  https://review.openstack.org/48613513:19
openstackgerritMerged openstack/tripleo-heat-templates master: Copy scheduler configuration from service/ironic to services-docker/ironic  https://review.openstack.org/48922813:20
*** achadha has quit IRC13:21
chandankumarmwhahaha: please have a look on this https://review.openstack.org/#/c/486965/13:23
*** amoralej|lunch is now known as amoralej13:24
*** Goneri has joined #tripleo13:26
*** cdearborn has joined #tripleo13:28
*** fzdarsky has joined #tripleo13:34
*** ykarel has quit IRC13:34
openstackgerritMichael Chapman proposed openstack/tripleo-heat-templates master: Add OPNFV scenario environment  https://review.openstack.org/48690513:34
honzaakrivoka: any ideas on how to unittest the swift-related actions?  seems like it's gonna take a lot of mocking :(13:38
*** liverpooler has quit IRC13:39
*** liverpooler has joined #tripleo13:39
akrivokahonza: yeah, I don't think there's a way around it13:39
akrivokahonza: you can take inspiration from plan export tests13:39
honzaakrivoka: alright, let's do this :)13:40
akrivoka:D13:40
honzaakrivoka: (as in me, just to be clear)13:40
akrivokaI assumed as much ;)13:40
honzadidn't want to offend13:41
*** jkilpatr_ has joined #tripleo13:41
*** pgadiya has quit IRC13:41
*** jkilpatr has quit IRC13:42
openstackgerritBob Fournier proposed openstack/tripleo-heat-templates master: Render VIPs dynamically based on network_data.yaml  https://review.openstack.org/48626013:44
thrashweshay: https://review.openstack.org/49087113:46
weshaythrash, you da man13:46
weshaythanks13:46
*** janki has quit IRC13:46
*** jistr is now known as jistr|mtg13:46
thrashweshay: I can see it being a gate eventually, but for now, experimental should do.13:46
weshaythrash, toure, and Adriano, do you guys mind just announcing that in your next meeting so folks have the information they need as to when it's appropriate to run the ovb job.. e.g. introspection13:48
thrashweshay: it's not really a very clear case of when from a mistral perspective13:48
*** egonzalez has quit IRC13:49
thrashweshay: one thing we might be able to do is to add ironic inspector to the devstack jobs..13:49
thrashand then test that the appropriate actions were actually created.13:50
pabelangerEmilienM: bnemec: mwhahaha: So, looking at the most recent uncatagorized failed jobs in elastic-recheck, I can see an issue with DLRN builds that should be straightforwrad to fix.13:50
pabelangerhttp://logs.openstack.org/96/489896/2/gate/gate-tripleo-ci-centos-7-containers-multinode/69b1379/logs/delorean_logs/e0/66/e066b99ec21e5aaedaaa73387d6316993ea67fc4_dev/root.log.txt.gz13:50
pabelangerDLRN should be configured to also use our AFS mirrors and not directly hit upstream13:50
pabelangerthat should help mitigate some networking failures13:50
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test  https://review.openstack.org/47574713:51
*** agopi has joined #tripleo13:52
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Convert scenario001-multinode-containers job to ceph-ansible  https://review.openstack.org/47928813:58
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260714:00
*** ratailor has joined #tripleo14:01
pabelangerEmilienM: bnemec: mwhahaha: I propose we create a generic bug in launchpad to start tracking DLRN failures, I have a logstash query ready that we can start to use to better track them.14:02
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814814:03
*** marios has quit IRC14:03
*** marios has joined #tripleo14:04
openstackgerritMerged openstack/tripleo-heat-templates master: Change the directory for haproxy certs/keys to be service-specific  https://review.openstack.org/48989614:04
openstackgerritMerged openstack/tripleo-ui master: Add warning state to validation status icons  https://review.openstack.org/48734614:04
*** atoth has joined #tripleo14:04
bnemecpabelanger: Sounds good, thanks.14:07
*** bnemec is now known as beekneemech14:07
beekneemechHappy Friday!14:07
*** fzdarsky has quit IRC14:08
openstackgerritNuman Siddique proposed openstack/tripleo-quickstart master: Add fset030 for OVN with containerized oc  https://review.openstack.org/49062414:09
mwhahahaor something14:09
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras master: WIP: Support for TripleO centralized logging  https://review.openstack.org/49062314:09
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855414:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)14:10
*** ooolpbot has quit IRC14:10
*** links has quit IRC14:12
*** achadha has joined #tripleo14:17
dmsimarddprince, Slower: is there logic in tripleo to authenticate to a registry before trying to pull from it ?14:19
dmsimardi.e, docker login14:19
dprincedmsimard: not that I know of. Not yet14:20
dmsimardmburned: ^^14:20
dmsimarddprince: it sounds like Red Hat official registry requires authentication (downstream), is it on the roadmap to implement it ?14:21
*** achadha has quit IRC14:22
dprincedmsimard: I'm not aware of it getting tracked yet. as far as upstream tripleo is concerned this would be an optional feature much like we handle similar things w/ instack-undercloud14:23
adarazsEmilienM, amoralej: so we rekicked the periodic jobs to get another promotion but of course in a busy time they didn't manage to start on the same consistent hash. 3 ran on 491d9ca9c4681b937e3e80cc09ab85a4b4903671, two on a hash one newer (a change to python-reno), and 1 on a change with 3 more changes, to python-reno, python-oslo-policy and python-django-horizon.14:26
*** anshul has quit IRC14:26
*** ppradhan has joined #tripleo14:27
adarazsEmilienM, amoralej: I propose to manually promote 491d9ca9c4681b937e3e80cc09ab85a4b4903671 if everything finishes successfully.14:27
adarazsas probably those changes wouldn't have made the jobs fail/succeed.14:27
amoralejreno is not a problem, django-horizon is not tested (i think we don't run any automated tests on horizon, right?)14:28
adarazsyeah.14:28
amoralejand policy is from 1.25.0 to 1.25.1 which was released in https://review.openstack.org/#/c/489599/14:30
amoralejwhich seems to fix "the show-policy directive. This helps projects leverage that directive for rendering the policy they maintain in their configuration guide."14:30
amoralejso seems safe14:30
adarazsthat's what I thought, cool.14:30
*** ratailor has quit IRC14:33
weshayrasca, PING A LING, re: ha containers14:33
*** ratailor has joined #tripleo14:33
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates master: WIP: Add scenario007-multinode-container CI templates for OVN containers  https://review.openstack.org/48909714:35
rascaweshay, was just about to pack my baggages :D :D14:35
weshayrasca, here's what I need.. just an email to all your code.. I can take it from there14:35
rascaweshay, http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/hardware_environments/ha-labs/containers/osp12-prep-ha-containers.sh.j214:36
rascaweshay, and... http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/hardware_environments/had00-lab/network_configs/float_nic_with_vlans/config_files/config-containers-ha-osp12.yml14:37
rascaweshay, here's everything I made so far14:37
rascaweshay, well I just realized the public chat, but hey, it's too late now14:37
openstackgerritIan Main proposed openstack/tripleo-common master: Add health checks during kolla build  https://review.openstack.org/48841914:38
openstackgerritDerek Higgins proposed openstack/tripleo-common master: Add health check command for ironic-pxe image  https://review.openstack.org/49089214:39
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Convert scenario001-multinode-containers job to ceph-ansible  https://review.openstack.org/47928814:41
*** MrChao has quit IRC14:44
*** ratailor has quit IRC14:44
weshayrasca, you better pack up your office too ;)14:45
rascaweshay, I'm sending you the photo of my new office soon14:46
*** mcornea has quit IRC14:49
*** shreshtha has joined #tripleo14:57
*** ramishra has quit IRC14:58
*** jistr|mtg is now known as jistr15:01
*** mdnadeem has joined #tripleo15:04
*** achadha has joined #tripleo15:04
*** isq_ has quit IRC15:07
*** sbrzozow has quit IRC15:07
*** isq_ has joined #tripleo15:07
*** jtomasek has quit IRC15:07
*** pchavva has quit IRC15:08
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170855415:10
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)15:10
*** ooolpbot has quit IRC15:10
*** jtomasek has joined #tripleo15:11
*** jtomasek has quit IRC15:12
*** shreshtha has quit IRC15:12
jistrfolks if you have some bandwidth please review the patches that allow the Mistral-Ansible workflows to connect to the overcloud on split stack envs15:13
jistrhttps://review.openstack.org/#/q/topic:bug/170818015:13
*** gbarros has quit IRC15:13
jistr(they unblock the ceph-ansible integration on that issue)15:13
*** dprince has quit IRC15:13
jistrthey've been proven to work in CI on the ceph-ansible patch15:14
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC VNX Manila Driver  https://review.openstack.org/49064215:14
*** rcernin has quit IRC15:14
EmilienMadarazs: ok15:14
adarazsEmilienM, amoralej: I think ovb-ha-oooq just timed out instead of passing...15:15
amoralejmmm15:15
*** jfrancoa has quit IRC15:16
EmilienMthrash: yeah?15:16
thrashEmilienM: it looks like the auth session is not being created properly. I'm running it down now.15:17
thrashso, it's unable to create the introspection actions.15:17
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test  https://review.openstack.org/47574715:17
*** ckyriakidou has quit IRC15:18
adarazsjpena: is it possible to pause DLRN building new hashes for a bit to let all the upstream periodic jobs pick up the same hash? things happen so fast during the day that it's unlikely that it happens :/15:18
*** mmethot has joined #tripleo15:19
jpenaadarazs: doing so would create a large backlog, and we'll surely need a fix to be built (remember yesterday with Heat). The jobs should be able to coordinate and use the same hash, which is what we do with pipelines or with the dlrn API15:20
adarazsjpena: I agree that's what we should do, and I want to implement that using DLRN API, but currently it's not :/15:20
*** nyechiel has joined #tripleo15:21
adarazsand we have a couple of upstream gates failing currently :/15:21
*** gbarros has joined #tripleo15:21
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP - use infra mirror for buildlogs repo  https://review.openstack.org/49062915:21
*** jtomasek has joined #tripleo15:22
jpenaadarazs: I can pause the builder once it finishes the current run for a few hours. I guess it's master that you need to stop, right?15:22
adarazsjpena: I don't mean a lot of pausing, just about ~10 minutes while the consistent link will stay the same.15:22
*** sbrzozow has joined #tripleo15:22
adarazsjpena: yes.15:22
adarazsmaybe a bit more than 10 minutes15:22
jpenaadarazs: ok then. I'm monitoring the current run, will let you know once it's stopped15:23
*** jaosorior has quit IRC15:23
adarazsjpena: thanks!15:23
michapmaEmilienM: the odl ci job is failing validation now, but overcloud deploy completes, so we have some progress.15:24
EmilienMmichapma: neutron works?15:24
michapmaEmilienM: well it starts. There was a patch merged yesterday to tht to fix odl that we needed to rebase on.15:25
EmilienMhttp://logs.openstack.org/05/486905/16/experimental/gate-tripleo-ci-centos-7-scenario008-multinode-oooq-nv/c492c4a/logs/undercloud/home/jenkins/overcloud_validate.log.txt.gz#_2017-08-04_15_16_0215:25
EmilienMpingtest fails, let me look15:25
EmilienMmichapma: sounds like the VM can't be reached by the floating IP15:26
EmilienMhttp://logs.openstack.org/05/486905/16/experimental/gate-tripleo-ci-centos-7-scenario008-multinode-oooq-nv/c492c4a/logs/undercloud/home/jenkins/overcloud_validate.log.txt.gz#_2017-08-04_15_15_4315:26
*** jtomasek has quit IRC15:26
EmilienMmichapma: we are close ;-)15:26
adarazsjpena: the queue should be empty now.15:26
jpenaadarazs: yes, it just finished15:27
jpenago ahead15:27
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921515:27
michapmaEmilienM: yeah we are. trozet is on pto today but back monday. It might just be an issue with master branch+odl so I'll see if I can replicate it in the lab.15:27
*** jtomasek has joined #tripleo15:28
EmilienMmichapma: ack. What says neutron logs?15:28
adarazsjpena: I triggered them, it's going to take a bit until they get started and fetch the repo link.15:28
jpenaadarazs: no worries. I thought you wanted the worker to stop for a day or so :)15:29
adarazsah no :)15:29
michapmaEmilienM: some errors but they don't look serious http://logs.openstack.org/05/486905/16/experimental/gate-tripleo-ci-centos-7-scenario008-multinode-oooq-nv/c492c4a/logs/subnode-2/var/log/neutron/server.log.txt.gz15:29
EmilienMmichapma: does l3 and dhcp works?15:30
michapmaEmilienM: they might be disabled and handled by odl15:30
openstackgerritMerged openstack/tripleo-quickstart-extras master: Move subunit test results generation pieces in tempest-results  https://review.openstack.org/48696515:30
michapmaEmilienM: is all of /var/log collected or only some files?15:31
EmilienMmichapma: only some files probably, you'll need to patch quickstart I think15:32
michapmaEmilienM: will do15:32
*** jtomasek has quit IRC15:32
*** achadha has quit IRC15:33
*** ebarrera has quit IRC15:35
*** iranzo has joined #tripleo15:38
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC VMAX Manila Driver  https://review.openstack.org/49094915:40
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC VMAX Manila Driver  https://review.openstack.org/49094915:41
*** suuuper has joined #tripleo15:41
*** achadha has joined #tripleo15:41
*** nyechiel has quit IRC15:42
*** dtantsur is now known as dtantsur|afk15:46
*** jlinkes has quit IRC15:46
*** iranzo has quit IRC15:47
adarazsjpena: there's a bit of a queue for the OVB check gate, it's going to take a bit until they get scheduled.15:48
jpenaadarazs: ack15:48
*** apetrich has quit IRC15:52
*** apetrich has joined #tripleo15:53
*** yamahata has joined #tripleo15:53
openstackgerritMerged openstack/puppet-pacemaker master: Switch requires for wait-for-settle into resource collectors  https://review.openstack.org/48953415:54
openstackgerritMerged openstack/tripleo-docs master: Fix typo in containers image prep command  https://review.openstack.org/49084215:54
dpawarEmilienM: hi15:55
*** snecklifter has quit IRC15:56
EmilienMdpawar: what's up - just go ahead and ask things here15:57
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC VMAX ISCSI Cinder Driver  https://review.openstack.org/48962415:58
dpawarEmilienM: i am trying quickstart, on overcloud controller it does not install os package "createrepo-0.9.9-26.el7.noarch" is there any way i can specify this package to be installed on overcloud controller ?15:59
*** cshastri has quit IRC16:00
thrashEmilienM: it works locally!!! WTH??16:03
openstackgerritSanjay Upadhyay proposed openstack/tripleo-quickstart master: This adds a variable to add stack user to the undercloud image  https://review.openstack.org/49040716:03
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC VMAX ISCSI Cinder Driver  https://review.openstack.org/48962416:03
thrashEmilienM: oh, wait... I may not have ironic-inspector fully set up... One sec.16:04
*** sbrzozow has quit IRC16:04
dpawarEmilienM: basically, i am making changes in https://github.com/vtas-hyperscale-ci/puppet-veritas_hyperscale so that, it creates a package repo using "createrepo" command. http://paste.openstack.org/show/617552/16:05
openstackgerritAlfredo Moralejo proposed openstack/tripleo-quickstart master: Create configurations for pike jobs in ci.centos.org  https://review.openstack.org/49096016:05
mwhahahadpawar: the packages would need to be in the image, the repo stuff won't work because we actually disable package installs except for on updates16:06
*** psachin has quit IRC16:07
*** marios has quit IRC16:07
dpawarmwhahaha: you mean only for quickstart, package installs is disabled ?16:08
mwhahahano downstream16:09
mwhahahaactually16:09
adarazsjpena: okay, all job finally started, now we just need to wait until the last one grabs the repo file. :)16:09
* adarazs really wants to redesign this system to use symlink...16:09
adarazsjust not enough time.16:09
dpawarbut here https://github.com/vtas-hyperscale-ci/puppet-veritas_hyperscale we do install veritas packages16:10
*** pcaruana has quit IRC16:10
dpawarhttps://github.com/vtas-hyperscale-ci/puppet-veritas_hyperscale/blob/master/manifests/controller_pkg_inst.pp#L32-L3616:11
mwhahahadpawar: this is one of those lesser known things which is why they actually need to be added to the images16:11
openstackgerritJulia Kreger proposed openstack/diskimage-builder master: Fix cylical systemd config for dhcp-all-interfaces  https://review.openstack.org/49096216:11
mwhahahadpawar: we actually noop the package command in puppet via OS::TripleO::Services::TripleoPackages https://github.com/openstack/tripleo-heat-templates/blob/master/roles_data.yaml#L13216:11
mwhahahadpawar: so that service effectively disables package installs on the overcloud puppet runs. which is why you need to have the packages preloaded in the overcloud images16:12
mwhahahadpawar: so back to your original question how do you inject those in quickstart? i'm not sure but that'll be a problem for folks trying to consume your changes downstream as well16:12
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Isilon Manila  backend  https://review.openstack.org/48889616:13
*** itlinux has joined #tripleo16:16
dpawarmwhahaha: we tried "puppet-upload" followed by 'openstack overcloud deploy" on a OSP11 setup, it works fine. but to test current master(rdo) i am trying quickstart. i modifed overcloud-depoly.sh to use puppet-upload and then run overcloud deploy. it did work. now wanted to use repo instead of manyally copying the pacakges16:17
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Isilon Manila  backend  https://review.openstack.org/48889616:17
*** cylopez has quit IRC16:18
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: scenario001: run tempest only in Pike and beyond  https://review.openstack.org/49096416:18
*** milan has quit IRC16:18
EmilienMadarazs, weshay ^ need review on that please - it's blocking stable/ocata CI16:19
adarazslooking16:19
mwhahahadpawar: you could use virt-customize to add them (the rpms) to the overcloud16:20
adarazsjpena: you can reenable the master DLRN worker. thank you for pausing it!16:21
jpenaadarazs: great, re-enabling16:21
dpawarmwhahaha: we are not planing to have customized overcloud image. It is assumed that, overcloud is already deployed and then veritas software will be installed using 'overcloud deploy'.16:22
*** hewbrocca is now known as hewbrocca_afk16:23
adarazsEmilienM: I have a question, commented on the change16:23
mwhahahadpawar: I dont' think that's going to work16:24
EmilienMadarazs: let me double check the tempest test but I think it does more than pingtest16:24
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC VMAX ISCSI Backend  https://review.openstack.org/48939316:26
*** derekh has quit IRC16:26
EmilienMadarazs: see my answer!16:27
EmilienMadarazs: thanks for double checking really nice one ;-)16:27
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Unity Cinder backend  https://review.openstack.org/48759916:27
EmilienMadarazs: is my jinja syntax correct?16:28
*** lucasagomes is now known as lucas-afk16:28
*** achadha has quit IRC16:29
adarazsEmilienM: the pingtest template that is used actually does a couple of more stuff: http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/ci/pingtests/scenario001-multinode.yaml#n13116:33
adarazsEmilienM: not sure if it's important, but it does exercise those extra parts (that is part of the scenario), also heat.16:34
adarazsI'm not insisting, but it should be conscious decision to reduce the coverage on that job.16:34
EmilienMadarazs: we already exercise Heat in other jobs16:34
EmilienMadarazs: it's a very conscious decision, we're increasing the coverage16:34
EmilienMadarazs: though having both pingtest + tempest is too heavy16:35
EmilienMwe're increasing the job runtime16:35
EmilienMadarazs: i'm working with arxcruz to have telemetry tempest autoscaling test16:35
EmilienMadarazs: which will test even more than the pingtest16:35
*** gcerami1 has joined #tripleo16:35
EmilienMadarazs: and again we're experimenting here, we'll probably adjust16:36
openstackgerritMerged openstack/tripleo-common master: Allow tripleo-admin creation both for Nova-managed and split-stack  https://review.openstack.org/48961316:36
adarazsEmilienM: so we don't care about losing that gnocchi resource alarm, heat autoscaling and aodh alarm bit? not that we do much with it or check anything.16:36
adarazsit's just there :)16:36
EmilienMadarazs: ok let me repeat :-)16:36
openstackgerritMerged openstack/puppet-tripleo master: Configure dockerd with --iptables=false  https://review.openstack.org/49020116:36
adarazsoh sorry.16:36
EmilienMadarazs: https://review.openstack.org/#/c/490376/3/config/general_config/featureset005.yml16:37
EmilienMwe're working with arx on adding autoscaling test16:37
adarazsEmilienM: cool, I don't have further questions. :)16:37
EmilienMadarazs: I hope we can merge it early next week16:37
adarazsack16:37
*** achadha has joined #tripleo16:38
EmilienMadarazs: if you want, I can add a TODO16:38
EmilienMadarazs: keep pingtest for now until https://review.openstack.org/#/c/490376/3 is merged16:38
adarazsEmilienM: it's okay.16:38
EmilienMadarazs: sounds like a good compromise16:38
EmilienMadarazs: actually, let me do it so we dont have regression in the middle16:38
adarazsEmilienM: I added a reference to that review in my +2 comment. submitted.16:38
*** gcerami has quit IRC16:39
adarazsoops :)16:39
EmilienMadarazs: ok, well, let's keep like this now16:39
EmilienMadarazs: we'll see how CI pass on this one and merge it16:39
EmilienMadarazs: the main blocker for us now is the tempest-config generator which doesn't enable telemetry in tempest.conf16:40
openstackgerritVictoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Removes manila-generic-config from TripleO  https://review.openstack.org/49097416:41
EmilienMadarazs: see in http://logs.openstack.org/76/490376/3/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/2027cef/logs/undercloud/home/jenkins/tempest/etc/tempest.conf.txt.gz16:41
EmilienMweshay: where is the tool that generate tempest config?16:41
adarazsEmilienM: maybe add a comment similar to what's in the commit message so that we know why this difference is in the featureset file.16:41
weshayEmilienM, sef16:42
weshaysec16:42
adarazsEmilienM: i mean put it also inside the file where you have the if condition16:42
weshayadarazs, so we've gotten past the overcloud image builds in sf.. We'll need the builder to include the call to post status to the delorean api integrated16:43
EmilienMadarazs: ok16:43
*** ebarrera has joined #tripleo16:43
openstackgerritGabriele Cerami proposed openstack/tripleo-quickstart master: enable container periodic job  https://review.openstack.org/47574316:43
*** achadha has quit IRC16:43
weshayadarazs, let me know if there are any problems are blockers there16:43
*** dprince has joined #tripleo16:43
adarazsweshay: I understand, but I had zero time to work on it this week :(16:43
weshaythis coming week, or next16:43
weshaysorry.. thus far, or next week?16:44
toskyEmilienM: https://github.com/redhat-openstack/python-tempestconf/16:44
adarazswell, I'm not sure how much time I will have next week with the TC/UA meetup either. I was planning to finish this week, but was chasing promotions instead.16:44
EmilienMtosky: thanks16:45
adarazsI'll try to finish it up in some downtime, but I can't promise it.16:45
adarazsweshay: ^16:45
EmilienMtosky: maybe you can help, but ceilometer is not enabled16:45
openstackgerritAna Krivokapic proposed openstack/tripleo-common master: Replaces uuid.uuid4 with uuidutils.generate_uuid()  https://review.openstack.org/48057116:45
toskyEmilienM: let me check16:45
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: scenario001: run tempest only in Pike and beyond  https://review.openstack.org/49096416:46
EmilienMadarazs: ^ done16:46
toskybefore I check, there are two possibilies: a) there could be no code to autodetect ceilometer, so you need to specify it manually in the command line or b) the code is there, but it does not work16:46
EmilienMtosky: see the config generated: http://logs.openstack.org/76/490376/3/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/2027cef/logs/undercloud/home/jenkins/tempest/etc/tempest.conf.txt.gz16:46
EmilienMtosky: https://github.com/redhat-openstack/python-tempestconf/blob/95fe370848f0132f470d04995749695e7b827608/config_tempest/config_tempest.py#L95416:47
adarazsEmilienM: added one more comment16:47
weshayjpena, ping re: container builds and promotion16:47
*** ykarel has joined #tripleo16:47
jpenaweshay: pong16:48
EmilienMtosky: the service is "metric" for gnocchi16:48
toskyEmilienM: whats is the current proper name16:48
EmilienMtosky: "event" for panko16:48
EmilienMtosky: "alarming" for aodh16:48
toskyEmilienM: I think that no one touched the part regarding telemetry stuff in recent times16:48
EmilienMand "metering" for ceilometer16:48
tosky(and no one requested it)16:49
*** ecerquei has joined #tripleo16:49
weshayjpena, we're pretty close to closing out the master promotion jobs in rdo software factory16:50
*** dpawar has quit IRC16:50
weshayjpena, we'll have a series of test jobs that will send pass/fail status to the delorean api16:50
toskyEmilienM: if I read the code correctly, the availability of "metering" should have lead to ceilometer = True16:50
EmilienMtosky: so we need ceilometer = True when metering service is available in Keystone.16:51
weshayjpena, now for the promotion part, we were told there can be some cron job on the delorean api server that reads the job status and can promote16:51
weshayjpena, is that the case?16:51
toskyEmilienM: but isn't it the case already with that code? Lines 955 and 95616:51
toskyis there some log that contains the list of enabled services/entry points on keystone for that job?16:51
jpenaweshay: well, this is first news :). Actually, it doesn't have to be on the dlrn server, you can get the ci reports from the dlrn api, so it could run anywhere16:52
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/48739816:52
*** ykarel has quit IRC16:52
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814816:52
jpenaweshay: gcerami1 was testing promotions16:52
weshayadarazs, fyi ^16:52
weshayjpena, ok16:53
EmilienMtosky: https://github.com/redhat-openstack/python-tempestconf/blob/95fe370848f0132f470d04995749695e7b827608/config_tempest/config_tempest.py#L90 should be metering16:53
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Consolidate puppet/docker deployments with one deploy steps workflow  https://review.openstack.org/48329316:53
EmilienMtosky: it would make things easier16:53
adarazsweshay: yeah, we don't need to have it there.16:53
adarazsjust wherever.16:53
toskyEmilienM: fine, but line 955 should account also for "metering"16:53
adarazsAlan suggested it there.16:53
weshayEmilienM, did you get the info you needed16:53
*** ecerquei has quit IRC16:54
adarazsEmilienM, weshay: I have to log off for today, see you guys next week from EST timezone, probably only on Tuesday, I'm traveling on Monday.16:54
weshayk16:54
weshaysafe travels16:54
EmilienMtosky: I don't know why L955 is here, too complex really16:54
EmilienMadarazs: see you16:54
EmilienMweshay: yes and I'm crying now.16:54
toskyEmilienM: it's there because once upon a time the entry point was different16:54
EmilienMtosky: i'm sending a PR16:54
toskyEmilienM: you may and it may solve the issue, or it may not; the current code should work16:55
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Move deploy-steps-playbook to deploy-steps-tasks  https://review.openstack.org/48568616:55
EmilienMtosky: look: http://logs.openstack.org/76/490376/3/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/2027cef/logs/undercloud/home/jenkins/tempest/etc/tempest.conf.txt.gz16:56
EmilienMceilometer = False16:56
toskyEmilienM: I've seen it16:56
EmilienMtosky: and ceilometer runs fine: http://logs.openstack.org/76/490376/3/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/2027cef/logs/subnode-2/var/log/ceilometer/16:56
toskyagain, I understand it16:56
toskybut I would like to understand where is the issue16:56
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Default docker_puppet_debug to false  https://review.openstack.org/48610316:56
toskymy point is that changing line 90 should be the same and won't solve the real issue16:57
EmilienMtosky: let me see if tripleo creates ceilometer keystone resources16:57
*** gkadam has joined #tripleo16:58
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Add RoleConfig output  https://review.openstack.org/48573116:58
EmilienMNotice: /Stage[main]/Ceilometer::Keystone::Auth/Keystone::Resource::Service_identity[ceilometer]/Keystone_service[ceilometer::metering]/ensure: created16:59
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Add support for update_tasks  https://review.openstack.org/48610416:59
EmilienMhttp://logs.openstack.org/76/490376/3/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/2027cef/logs/subnode-2/var/log/messages.txt.gz#_Aug__3_22_42_2716:59
EmilienMtosky: ^16:59
EmilienMso the "metering" service is in keystone17:00
*** ppradhan has left #tripleo17:00
toskyEmilienM: I'd like to test live on a system, but my only deployment apparently has no ceilometer17:01
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Add environment to disable deploy steps  https://review.openstack.org/48618017:01
*** mdnadeem has quit IRC17:04
*** yamahata has quit IRC17:04
*** ebarrera has quit IRC17:04
thrashEmilienM: even stanger... I have a oooq devmode ovb on rdocloud that also works.17:04
*** gcerami1 is now known as gcerami017:08
*** achadha has joined #tripleo17:09
toskyEmilienM: btw, no PR for python-tempestconf; we use review.gerrithub.io17:10
EmilienMtosky: yep17:10
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Modifying Cisco templates to support composable roles  https://review.openstack.org/48714617:11
EmilienMtosky: https://review.gerrithub.io/37272117:12
*** suuuper has quit IRC17:13
toskyEmilienM: as mentioned, I'd like to see it working17:13
toskythe gates should help a bit, but only for packstack17:14
*** jpena is now known as jpena|off17:14
*** achadha has quit IRC17:14
EmilienMtosky: I'm pretty sure this code was never tested17:14
EmilienMtosky: not a lot of folks test telemetry17:14
EmilienM(we do)17:14
EmilienManyway, this patch is a cleanup, please consider it17:14
toskyEmilienM: yes, it is a cleanup, but I don't think will change anything17:14
*** achadha has joined #tripleo17:15
*** achadha has quit IRC17:16
*** achadha has joined #tripleo17:16
*** pchavva has joined #tripleo17:18
EmilienMtosky: can you test it with ceilometer enabled?17:19
toskyEmilienM: I should reprovision, so not sure I can do it now17:19
*** achadha has quit IRC17:19
EmilienMtosky: can you merge my patch? I don't think we should block it17:20
*** achadha has joined #tripleo17:20
*** achadha has quit IRC17:20
toskyEmilienM: what is your goal? Have the scenario001 passing?17:21
EmilienMtosky: yes, we're now running tempest on scenario00117:21
EmilienMtosky: but we don't test Telemetry17:21
EmilienMtosky: and my goal is to do so17:22
toskyEmilienM: and I think that this patch won't change anything17:22
EmilienMtosky: any help on that front is welcome17:22
toskyit does not fix the issue17:22
toskysure17:22
toskygive me a machine with telemetry enabled and I will debug ti17:22
toskyit*17:22
EmilienMtosky: we're running tempest in tripleo, isn't awesome? :)17:22
EmilienMtosky: I don't have a machine17:22
weshaylolz17:22
weshayEmilienM, you need a box?17:22
toskyit is, but it does not make me feel pushing a patch which I think it's not useful17:22
weshaymaster w/ telemetry17:23
EmilienMweshay: tosky needs it apparently.17:23
EmilienMand I'm having hard time to convaince tempest config mantainer to merge a patch that cleans things up17:23
toskya patch which is useless for the goal17:23
toskyI can merge it, it won't improve the situation and back to point 017:24
toskycan you please explain me why it should magically solve the problem of discovering telemetry, when that condition is already checked and fails?17:24
toskyI may be wrong, but so far you didn't explain why it should work17:24
tosky"cleanup" is good, but it does not explain why it should work17:24
toskyweshay: and yes, if you have a machine I can use it17:25
*** bogdando has quit IRC17:25
weshayI have to set it up17:26
toskyyou know what? It's 19.30, I'm relocating, ping me when it's ready17:27
*** tosky has quit IRC17:27
*** openstackstatus has quit IRC17:27
*** openstack has joined #tripleo17:28
*** amoralej is now known as amoralej|off17:30
*** openstackstatus has joined #tripleo17:30
*** ChanServ sets mode: +v openstackstatus17:30
*** yprokule has quit IRC17:32
openstackgerritSwaminathan Vasudevan proposed openstack/diskimage-builder master: Failed to open parameter YAML error while trying to unmount imagedir  https://review.openstack.org/49063717:33
*** sbrzozow has joined #tripleo17:34
*** yamahata has joined #tripleo17:39
weshaygcerami0, FYI.. http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-scenario001-multinode-oooq/1da6891/console.html#_2017-08-04_06_17_14_82235717:44
gcerami0weshay: it's a fake image17:45
weshayyup17:46
gcerami0weshay: empty, created only so that that deploy doesn't complain17:46
weshayya17:46
weshaygcerami0, makes sense17:46
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/newton: Release 5.6.1  https://review.openstack.org/49099917:47
EmilienMmburned: ^17:47
mburnedEmilienM: ack, thanks17:47
EmilienMmwhahaha: ^ please review17:47
jkilpatr_What role/method does Tripleo CI use to collect logs and ship them to logstash>?17:48
jkilpatr_there seem to be a lot of different half finished efforts for that lying around, but this is the only one I see in prod and I would like to use it myself down in RDO17:48
*** pchavva has quit IRC17:49
*** tosky has joined #tripleo17:54
*** maeca has quit IRC17:56
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262317:56
*** salmankhan has quit IRC17:57
*** ecerquei has joined #tripleo17:57
toskyweshay: if you didn't provision the system yet, and you can set tempest_config to true, that would help; if not, and the provisioning is ongoing, no need to interrupt it17:57
*** mmethot has quit IRC17:58
weshaytosky, ok17:58
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates master: Deploy Mistral with Keystone v3 options (authtoken)  https://review.openstack.org/46104018:00
*** itlinux has quit IRC18:03
*** itlinux has joined #tripleo18:05
*** athomas has quit IRC18:06
*** pkovar has quit IRC18:07
*** paramite has quit IRC18:08
*** paramite has joined #tripleo18:12
jkilpatr_weshay, ^18:13
weshayjkilpatr_, ?18:13
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Remove TripleO CI mitaka environment  https://review.openstack.org/49101118:14
*** ddeja has quit IRC18:14
*** gbarros has quit IRC18:16
jkilpatr_do you know where this data comes from? http://logstash.openstack.org/#/dashboard/file/logstash.json?query=build_name:%2018:17
*** slagle has quit IRC18:19
*** openstackstatus has quit IRC18:26
*** openstack has joined #tripleo18:27
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: use infra mirror for buildlogs repo  https://review.openstack.org/49062918:27
EmilienMpabelanger: ^ final version of the patch, covering all tripleo ci jobs18:28
EmilienMpabelanger: fyi, puppet ci switched -patch merged- so we're good on this side18:28
*** dpawar has joined #tripleo18:28
*** openstackstatus has joined #tripleo18:29
*** ChanServ sets mode: +v openstackstatus18:29
pabelangerEmilienM: cool! I have elastic-recheck queries up to better track places where we still need to make the changes.18:30
pabelangerEmilienM: did you see my comments this morning about adding our AFS mirrors to mock (in DLRN)?18:30
*** gcerami0 has quit IRC18:33
*** dpawar has quit IRC18:35
EmilienMpabelanger: I didn't sorry, let me read logs now18:35
*** gcerami0 has joined #tripleo18:36
EmilienMweshay, mwhahaha: if you can review https://review.openstack.org/#/c/490629/ today, we would reduce buildlogs issues in CI18:37
EmilienMweshay, mwhahaha : which is still quite there: http://status.openstack.org/elastic-recheck/#167468118:37
EmilienMit's not a lot though18:37
*** gvranga__ has joined #tripleo18:38
weshayEmilienM++18:38
EmilienMpabelanger: the query is incomplete18:38
EmilienMpabelanger: we miss newton & ocata18:38
*** gvranga__ has quit IRC18:38
weshayEmilienM, I'm fixing https://review.openstack.org/#/c/490964/2/config/general_config/featureset005.yml atm will look in a hot second18:38
EmilienMweshay: ok removing +W18:39
EmilienMweshay: please push over18:39
weshayya.. will have it up in a few18:40
*** rlandy has joined #tripleo18:40
*** sbrzozow has quit IRC18:41
pabelangerEmilienM: sorry, which query?18:42
pabelangerEmilienM: Oh, I see18:43
EmilienMhttp://status.openstack.org/elastic-recheck/#167468118:43
pabelangerEmilienM: ya, in fact, I am going to remove that query in favor of https://review.openstack.org/#/c/490998/18:43
EmilienMpabelanger: i'll modify it and propose a change in e-r18:43
EmilienMah18:43
pabelangerEmilienM: that will infact better track yum client timeouts18:43
EmilienMpabelanger: this one is more general18:44
EmilienMbut ok18:44
EmilienMthis one will make it hard to know which servers went down18:44
pabelangerEmilienM: right, catch all for the moment18:44
pabelangerEmilienM: what is should tell us that, A job is not using AFS mirrors, B, AFS mirror is down, C, cloud netowrking issue18:44
EmilienMyeah18:46
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: scenario001: run tempest only in Pike and beyond  https://review.openstack.org/49096418:50
weshayEmilienM, https://review.openstack.org/49096418:50
EmilienMweshay: ok I see18:51
EmilienMweshay: we need confitional for each parameter18:51
weshayand to pass {%-  -%} to chomp18:51
EmilienMweshay: validate_template is only needed when you have a pingtest, right?18:51
*** slagle has joined #tripleo18:51
EmilienMsame for test_regex (when you run tempest) right?18:51
weshayya.. but nothing will change even if that is set18:51
EmilienMmhh ok18:52
weshaywe're just worried about the execution18:52
weshayright?18:52
EmilienMweshay: yes18:52
EmilienMweshay: +2, let's see how ci works18:52
EmilienMmwhahaha: will also ask for a review when you have time18:52
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/ocata: Remove docker references from stable/ocata scenarios.  https://review.openstack.org/48987418:53
*** sbrzozow has joined #tripleo18:57
*** cdearborn_ has joined #tripleo18:58
*** cdearborn__ has joined #tripleo19:07
*** gbarros has joined #tripleo19:07
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262319:11
*** cdearborn has quit IRC19:14
*** gkadam has quit IRC19:16
*** ppradhan has joined #tripleo19:17
*** cdearborn_ has quit IRC19:18
*** cdearborn has joined #tripleo19:18
*** salmankhan has joined #tripleo19:29
*** salmankhan has quit IRC19:33
EmilienMtosky: I also found out that cinder was always enabled in tempest.conf, which is problematic when you don't want to test cinder, are you aware about this problem?19:34
*** cdearborn__ has quit IRC19:34
toskyEmilienM: cinder or cinder backup?19:35
toskyno, it was not reported19:35
toskyor better, we spoke with arxcruz|sick about cinder backup or so19:35
EmilienMtosky: cinder19:35
EmilienMwhat isn't supported?19:36
*** akrivoka has quit IRC19:36
toskyuhm, I didn't say anything about supported or not19:37
toskyArx fixed few stuff about cinder detection: https://github.com/redhat-openstack/python-tempestconf/commit/49a0950814b7feb6a4be2e0a53d82351535150cf19:38
EmilienMerr reported19:38
toskyno problems with cinder detection have been reported19:39
EmilienMok19:39
* tosky looks at weshay for the system19:39
EmilienMtosky: let me show you, give me a sec19:39
EmilienMtosky: I have a case where cinder is not enabled in openstack but in tempest19:39
weshaytosky, it's deploying, should be done soon19:40
weshayrdo-cloud is slow :)19:41
EmilienMtosky: see here http://logs.openstack.org/72/490172/4/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-container/a3efcae/logs/undercloud/home/jenkins/tempest_output.log.txt.gz#_2017-08-03_22_27_2819:42
EmilienMsee the trace at the end19:43
toskyso the problem is not that cinder was always enabled, but that disabling it discover-tempest-config fails19:43
toskyand it's volume backup (check_volume_backup_service)19:44
toskyso I think this is something that Arx mentioned19:44
toskycheck_volume_backup_service is a new function introduced recently by Arx to fix another issue19:45
*** mmethot has joined #tripleo19:48
toskyby recently I mean 2/3 weeks ago19:48
*** itlinux has quit IRC19:49
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: scenario001: run tempest only in Pike and beyond  https://review.openstack.org/49096419:51
EmilienMweshay, mwhahaha ^ comments addressed19:51
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/ocata: Remove docker references from stable/ocata scenarios.  https://review.openstack.org/48987420:00
*** dougbtv_laptop has quit IRC20:01
*** ansmith has quit IRC20:08
EmilienMadarazs: ack your comments - thanks, we can make it better later20:11
EmilienMadarazs: for the commit message I mean. The jinja thing is afik required (cc weshay)20:15
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Unity Manila Backend  https://review.openstack.org/49107820:21
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Unity Manila Backend  https://review.openstack.org/49107820:24
-openstackstatus- NOTICE: Gerrit is being restarted to pick up CSS changes and should be back momentarily20:37
mwhahahaanyone have the cmd line to rerun the ansible execution on a node?20:38
mwhahahaie it failed and i want to rerun the playbook, i've got to pass the step but i can't remember the syntax20:38
*** jkilpatr_ has quit IRC20:41
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC VNX Manila Backend  https://review.openstack.org/49108820:46
openstackgerritMerged openstack/tripleo-heat-templates master: Run gnocchi upgrade with sacks in docker template  https://review.openstack.org/48879620:47
openstackgerritMerged openstack/tripleo-heat-templates master: Stop and disable openstack-nova-compute service on compute nodes  https://review.openstack.org/49022620:47
openstackgerritMerged openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/48835720:47
openstackgerritMerged openstack/tripleo-common master: Prompt to clear breakpoints when using deployed-server  https://review.openstack.org/49012520:47
openstackgerritMerged openstack/puppet-tripleo stable/newton: Release 5.6.1  https://review.openstack.org/49099920:47
openstackgerritMerged openstack/os-net-config master: There is no documentation for os-net-config  https://review.openstack.org/48517420:47
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Unity Manila Backend  https://review.openstack.org/49107820:48
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Enable pingtest on scenarios container jobs  https://review.openstack.org/49012920:48
fultonjEmilienM: why abaondon https://review.openstack.org/#/c/489710 ?20:55
EmilienMfultonj: because jobs run on https://review.openstack.org/#/c/490129/ now20:56
EmilienMfultonj: so we don't need depends-on :)20:56
EmilienMfultonj: see 490129 in zuul, scenarios container job run fine!20:57
fultonj?! cool20:57
* fultonj looking20:57
EmilienMfultonj: it's because of https://review.openstack.org/#/c/490163/ :-)20:57
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC VMAX Manila Backend  https://review.openstack.org/49109421:01
fultonj Did glance start working in scenario001-multinode-oooq-container?21:01
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC VMAX Manila Backend  https://review.openstack.org/49109421:02
fultonjin zuul i see lint and docs ok so far, others running21:02
*** rbowen has quit IRC21:02
EmilienMfultonj: it's running let's see results21:03
fultonjok, i'm telnet'd into it21:03
*** ecerquei has quit IRC21:04
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC VMAX Manila Backend  https://review.openstack.org/49109421:04
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC VNX Manila Backend  https://review.openstack.org/49108821:06
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Unity Manila Backend  https://review.openstack.org/49107821:07
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Isilon Manila  backend  https://review.openstack.org/48889621:08
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: use infra mirror for buildlogs repo  https://review.openstack.org/49062921:09
*** itlinux has joined #tripleo21:15
*** ansmith has joined #tripleo21:16
*** agopi has quit IRC21:16
*** sbrzozow has quit IRC21:17
mwhahahadprince: where did the logs go for the deployment process under containers? /var/log/messages is blank on the deployed system so things that used to show up with os-collect-config seem to be going no where21:19
mwhahahadprince: i'm trying to troubleshoot why the ovb-containers job is failing for https://review.openstack.org/#/c/489618/ but i cant seem to get any useful logs21:20
*** paramite has quit IRC21:21
fultonji didn't know the CI logs had /var/log/extra/docker/docker_allinfo21:23
fultonjhttp://logs.openstack.org/88/479288/23/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fecc289/logs/subnode-2/var/log/extra/docker/docker_allinfo.log.txt.gz21:23
fultonjnice ! ^21:23
fultonjthat could be expanded to have the output of docker-toool for each container id so we could see what exact volumes they were started with21:25
dprincemwhahaha: /var/log/containers is the directory you want21:25
mwhahahadprince: no the part that executes the ansible from heat21:26
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: DNM - run tempest in scenario-container jobs  https://review.openstack.org/49016721:26
mwhahahadprince: not the logs for services running in containers21:26
dprincemwhahaha: oh, so it would be in the heat "software deployments" output21:26
mwhahahaie i don't think host prep is configuring syslog correctly21:26
mwhahahabecause it's no longer writing it out locally on the system21:26
dprincehmmm. could be21:27
mwhahahaso it's next to impossible to see what's actually running on the system as part of the deployment21:27
*** jkilpatr_ has joined #tripleo21:29
*** Goneri has quit IRC21:30
*** cdearborn has quit IRC21:30
slaglemwhahaha: maybe related to this? https://review.openstack.org/#/c/487964/21:32
*** morazi has quit IRC21:32
slaglemwhahaha: however, we needed that so that we actually saw the error in heat21:32
mwhahahaslagle: no os-collect-config is no longer showing up in any local logs21:32
mwhahahaslagle: so that would be in the heat logs but not on the local system21:32
mwhahahabut i'm still poking at it :/21:33
mwhahahaso like this stuff, http://logs.openstack.org/18/489618/1/check/gate-tripleo-ci-centos-7-nonha-multinode-oooq/12dcdb9/logs/subnode-2/var/log/messages.txt.gz#_Aug__3_14_28_2421:34
mwhahahai'm trying to figure out where that's going21:34
*** dprince has quit IRC21:35
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262321:43
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027221:43
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert objectstorage-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262421:43
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262321:44
*** rlandy has quit IRC21:45
*** dparkes has joined #tripleo21:46
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027221:46
*** jcoufal has quit IRC21:50
*** thrash is now known as thrash|g0ne21:52
*** jmelvin has quit IRC21:52
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements master: Set remove-machine-id to executable  https://review.openstack.org/48961821:55
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Convert scenario001-multinode-containers job to ceph-ansible  https://review.openstack.org/47928821:58
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260722:00
weshaymandre, you there? I was going to start poking at a ha deployment with containers.. I have the downstream setup. Just wondering if there were any upstream efforts around this already22:00
*** tzumainn has quit IRC22:03
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Switch scenario002 to run Tempest  https://review.openstack.org/49110222:07
*** ppradhan has left #tripleo22:07
*** noslzzp has quit IRC22:17
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Switch scenario004 to run Tempest  https://review.openstack.org/49111322:29
openstackgerritJames Slagle proposed openstack/tripleo-common stable/ocata: Prompt to clear breakpoints when using deployed-server  https://review.openstack.org/49111522:32
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: use AFS mirror for buildlogs repo  https://review.openstack.org/49062922:33
*** gcerami0 has quit IRC22:34
openstackgerritMerged openstack/puppet-pacemaker master: Update openstackdocstheme>=1.16.0  https://review.openstack.org/48906222:42
*** Goneri has joined #tripleo22:52
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: CI test - never merge  https://review.openstack.org/49112522:55
*** catintheroof has quit IRC22:56
*** lblanchard has quit IRC22:57
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Replace references to deprecated controllerExtraConfig  https://review.openstack.org/48039523:05
openstackgerritTony Breeds proposed openstack/tripleo-common master: Default to host CPU architecture if none is supplied  https://review.openstack.org/48947623:12
openstackgerritMerged openstack/tripleo-docs master: Add name and comment fields desc for instackenv  https://review.openstack.org/48958723:15
openstackgerritMerged openstack/tripleo-docs master: Document using current-tripleo  https://review.openstack.org/49017023:16
*** dparkes has quit IRC23:24
*** toure is now known as toure|gone23:43
*** mmethot has quit IRC23:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!