Wednesday, 2018-09-19

openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/ocata: GATE CHECK for TripleO  https://review.openstack.org/56429100:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/pike: GATE CHECK for TripleO  https://review.openstack.org/60224800:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60327500:00
*** dhill_ has joined #tripleo00:03
*** dmacpher has joined #tripleo00:05
*** hamzy has joined #tripleo00:05
*** janki has quit IRC00:09
*** janki has joined #tripleo00:09
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179229600:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179234300:10
openstackLaunchpad bug 1792296 in tripleo "Overcloude deploy error:Timed out waiting for messages from Execution" [Critical,In progress] - Assigned to Quique Llorente (quiquell)00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256000:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179286200:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179287000:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179287200:10
openstackLaunchpad bug 1792343 in tripleo "[tripleo] rocky baremetal deployment fails with jq: error: Could not open file /var/lib/heat-config/deployed/<id>.notify.json" [Critical,Triaged]00:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr)00:10
openstackLaunchpad bug 1792862 in tripleo "[master] Telemetry Tempest integration tests failed giving Unable to complete operation on subnet b13c76ec-c851-48d0-91a9-245a2fdcad9b: One or more ports have an IP allocation from this subnet." [Critical,Fix committed] - Assigned to Mehdi Abaakouk (sileht)00:10
openstackLaunchpad bug 1792870 in tripleo "[Rocky] FS01 periodic job failed at overcloud prepare image giving Error: Unable to establish IPMI v2 / RMCP+ session\n'.: ProcessExecutionError: Unexpected error while running command" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)00:10
openstackLaunchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)00:10
*** janki has quit IRC00:10
*** janki has joined #tripleo00:11
openstackgerritMerged openstack/os-collect-config stable/queens: import zuul job settings from project-config  https://review.openstack.org/59782800:20
openstackgerritMerged openstack/os-collect-config stable/rocky: import zuul job settings from project-config  https://review.openstack.org/59784500:24
openstackgerritMerged openstack/os-refresh-config stable/rocky: import zuul job settings from project-config  https://review.openstack.org/59784700:24
openstackgerritMerged openstack/tripleo-upgrade stable/pike: import zuul job settings from project-config  https://review.openstack.org/59782300:24
openstackgerritMerged openstack/tripleo-upgrade stable/queens: import zuul job settings from project-config  https://review.openstack.org/59784000:24
openstackgerritMerged openstack/tripleo-heat-templates master: switch documentation job to new PTI  https://review.openstack.org/59606100:24
openstackgerritMerged openstack/tripleo-upgrade stable/rocky: Enhance undercloud reboot procedure for baremetal  https://review.openstack.org/60336500:31
*** dmacpher_ has joined #tripleo00:34
*** dmacpher has quit IRC00:36
openstackgerritMerged openstack/tripleo-common master: Add bashate in gate jobs and fix existing errors  https://review.openstack.org/60305700:55
openstackgerritMerged openstack/tripleo-quickstart master: Enable yamllint to avoid future key duplications  https://review.openstack.org/59758100:55
openstackgerritMerged openstack/tripleo-heat-templates master: Add CephOSD service to roles/Standalone.yaml  https://review.openstack.org/60322700:55
openstackgerritMerged openstack/tripleo-quickstart master: use a more stable centos repo for k8s and container rpms  https://review.openstack.org/60327300:56
*** Chaserjim has joined #tripleo00:59
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: provider: Add vexxhost  https://review.openstack.org/59643201:08
*** Chaserjim has quit IRC01:08
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179229601:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179234301:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256001:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179286201:10
openstackLaunchpad bug 1792296 in tripleo "Overcloude deploy error:Timed out waiting for messages from Execution" [Critical,In progress] - Assigned to Quique Llorente (quiquell)01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179287001:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179287201:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1792343 in tripleo "[tripleo] rocky baremetal deployment fails with jq: error: Could not open file /var/lib/heat-config/deployed/<id>.notify.json" [Critical,Triaged]01:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr)01:10
openstackLaunchpad bug 1792862 in tripleo "[master] Telemetry Tempest integration tests failed giving Unable to complete operation on subnet b13c76ec-c851-48d0-91a9-245a2fdcad9b: One or more ports have an IP allocation from this subnet." [Critical,Fix committed] - Assigned to Mehdi Abaakouk (sileht)01:10
openstackLaunchpad bug 1792870 in tripleo "[Rocky] FS01 periodic job failed at overcloud prepare image giving Error: Unable to establish IPMI v2 / RMCP+ session\n'.: ProcessExecutionError: Unexpected error while running command" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)01:10
openstackLaunchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)01:10
*** Chaserjim has joined #tripleo01:12
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Ensure some directories are created with the correct SELinux label  https://review.openstack.org/60053201:19
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Allow to run bootstrap containers in privileged mode.  https://review.openstack.org/60053301:19
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Set proper setype for service directories  https://review.openstack.org/60053401:19
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Allow to deactivate SELinux separation for selected containers  https://review.openstack.org/60053501:19
*** Chaserjim has quit IRC01:21
*** mschuppert has joined #tripleo01:26
*** mrsoul has quit IRC01:29
*** assassin has joined #tripleo01:30
*** Chaserjim has joined #tripleo01:33
*** slong has quit IRC01:39
EmilienMjaosorior_: hey01:48
EmilienMjaosorior_: I'm testing https://review.openstack.org/#/c/603390 - I'm not sure I got it working so far01:49
openstackgerritJanki Chhatbar proposed openstack/tripleo-heat-templates master: Allow overlay tunnel endpoints on IPv6 address  https://review.openstack.org/60353201:52
*** ykarel|away has joined #tripleo01:52
*** morazi has quit IRC01:57
EmilienMjaosorior_: I think I found why01:58
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Only set ca_bundle for HAProxy if internal TLS enabled & fix mounts  https://review.openstack.org/60339002:05
EmilienMjaosorior_: ^ see02:05
EmilienMjaosorior_: I'm not sure. I'm actually testing it in my env now. Feel free to push over if needed. /me off now02:08
*** Chaserjim has quit IRC02:09
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179286202:10
openstackLaunchpad bug 1792862 in tripleo "[master] Telemetry Tempest integration tests failed giving Unable to complete operation on subnet b13c76ec-c851-48d0-91a9-245a2fdcad9b: One or more ports have an IP allocation from this subnet." [Critical,Fix committed] - Assigned to Mehdi Abaakouk (sileht)02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179287002:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179287202:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1792870 in tripleo "[Rocky] FS01 periodic job failed at overcloud prepare image giving Error: Unable to establish IPMI v2 / RMCP+ session\n'.: ProcessExecutionError: Unexpected error while running command" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)02:10
openstackLaunchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Incomplete] - Assigned to Quique Llorente (quiquell)02:10
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman  https://review.openstack.org/60051702:12
*** Chaserjim has joined #tripleo02:13
weshay_ptochkumar|off, quique|rover|off just a friendly reminder.. use the alert tag when jobs are failing in the gate or patches can't land for some reason02:15
weshay_ptouse promotion-blocker for issues w/ the periodic jobs02:15
weshay_ptonot every promotion-blocker is an alert02:15
*** thrash is now known as thrash|g0ne02:17
*** saneax has joined #tripleo02:20
*** Chaserjim has quit IRC02:22
*** Chaserjim has joined #tripleo02:25
*** Chaserjim has quit IRC02:30
*** Chaserjim has joined #tripleo02:34
*** ykarel|away has quit IRC02:43
*** rnoriega has quit IRC02:45
*** rascasoft has quit IRC02:45
*** Chaserjim has quit IRC02:45
*** rnoriega has joined #tripleo02:46
*** apetrich has quit IRC02:51
*** Chaserjim has joined #tripleo02:52
*** jistr has quit IRC03:00
*** jistr has joined #tripleo03:00
*** Chaserjim has quit IRC03:05
*** ramishra has joined #tripleo03:08
*** Chaserjim has joined #tripleo03:08
*** Chaserjim has quit IRC03:13
*** Chaserjim has joined #tripleo03:13
*** cgoncalves|pto has quit IRC03:16
*** cgoncalves has joined #tripleo03:17
*** pvc has quit IRC03:28
*** Chaserjim has quit IRC03:29
*** saneax has quit IRC03:44
*** saneax has joined #tripleo03:46
*** dxiri has joined #tripleo03:47
*** Chaserjim has joined #tripleo03:48
*** assassin has quit IRC03:52
*** udesale has joined #tripleo03:52
*** ykarel|away has joined #tripleo03:53
*** Chaserjim has quit IRC03:57
openstackgerritMerged openstack/tripleo-heat-templates master: Delete FreeIPA enroll script  https://review.openstack.org/59134704:01
*** rfolco has quit IRC04:08
*** Chaserjim has joined #tripleo04:09
jaosorior_EmilienM: thanks for the fix04:12
*** jaosorior_ is now known as jaosorior04:13
*** skramaja has joined #tripleo04:19
*** dxiri has quit IRC04:20
*** Chaserjim has quit IRC04:21
*** Chaserjim has joined #tripleo04:22
*** Chaserjim has quit IRC04:34
*** Chaserjim has joined #tripleo04:35
jaosoriormandre: let me know when you're around.04:44
*** Petersingh has joined #tripleo04:53
*** Chaserjim has quit IRC04:56
*** Chaserjim has joined #tripleo05:00
*** hamzy_ has joined #tripleo05:01
*** hamzy has quit IRC05:02
*** Petersingh is now known as Petersingh|afk05:04
*** ykarel|away has quit IRC05:04
*** iranzo has joined #tripleo05:05
*** Chaserjim has quit IRC05:09
*** hamzy has joined #tripleo05:09
*** sshnaidm has joined #tripleo05:10
*** hamzy_ has quit IRC05:11
*** sshnaidm has quit IRC05:11
*** slaweq has joined #tripleo05:11
*** Chaserjim has joined #tripleo05:13
openstackgerritMerged openstack/tripleo-heat-templates master: fix tox python3 overrides  https://review.openstack.org/57401905:14
*** dxiri has joined #tripleo05:15
*** slaweq has quit IRC05:15
*** hamzy has quit IRC05:17
*** hamzy has joined #tripleo05:18
*** hamzy has quit IRC05:23
*** hamzy has joined #tripleo05:24
*** shyamb has joined #tripleo05:24
*** Chaserjim has quit IRC05:25
*** Chaserjim has joined #tripleo05:26
*** Petersingh|afk is now known as Petersingh05:27
*** shyamb has quit IRC05:29
*** rcernin has quit IRC05:30
*** rcernin has joined #tripleo05:30
*** ratailor has joined #tripleo05:31
*** quique|rover|off is now known as quiquell|rover05:33
*** Chaserjim has quit IRC05:35
*** Chaserjim has joined #tripleo05:36
quiquell|roverGood morning05:42
jaosoriormorning!05:43
*** aufi has joined #tripleo05:45
*** Chaserjim has quit IRC05:45
*** dxiri has quit IRC05:46
*** ykarel|away has joined #tripleo05:47
*** Chaserjim has joined #tripleo05:49
*** shyamb has joined #tripleo05:50
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-heat-templates master: ceilometer: Use new archive policy  https://review.openstack.org/60355605:51
openstackgerritRajesh Tailor proposed openstack/instack-undercloud stable/queens: Expose nova.conf config lines to be used as hiera overrides  https://review.openstack.org/60355705:53
*** phuongnh has joined #tripleo05:54
*** hjensas has quit IRC05:54
*** Chaserjim has quit IRC05:57
*** Chaserjim has joined #tripleo05:57
*** jtomasek has quit IRC06:00
*** jtomasek has joined #tripleo06:01
*** saneax has quit IRC06:02
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: WIP: Enabling CI for OVS-DPDK deployment  https://review.openstack.org/58696906:04
*** rcernin_ has joined #tripleo06:04
*** jtomasek has quit IRC06:06
*** rcernin has quit IRC06:06
*** ksambor has joined #tripleo06:07
*** jfrancoa has joined #tripleo06:08
*** holser_ has joined #tripleo06:12
*** abishop has quit IRC06:13
mandrejaosorior: here06:13
*** abishop has joined #tripleo06:16
*** janki has quit IRC06:17
*** holser_ has quit IRC06:18
jaosoriormandre: hey! just a question about your haproxy/openshift patch06:18
jaosoriormandre: so... TLS :D... is that handled then by openshift?06:18
*** slaweq has joined #tripleo06:18
openstackgerritQuique Llorente proposed openstack/tripleo-common stable/queens: Upgrade docker ceph container  https://review.openstack.org/60336906:19
mandrejaosorior: hey, yeah the 443 port is to serve apps that will be running over HTTPS, the certs and all are handled by openshift06:21
*** ykarel|away is now known as ykarel06:21
*** holser_ has joined #tripleo06:21
jaosoriorah cool! so this is more for apps, and not the openshift API itself?06:21
*** Chaserjim has quit IRC06:22
mandreright, this is for the apps running on top of openshift and accessed by the users... all your pods06:22
jaosoriorgot it now06:22
jaosoriormandre: thanks06:22
*** Chaserjim has joined #tripleo06:23
*** dxiri has joined #tripleo06:23
*** slaweq has quit IRC06:23
mandrein my understanding it's only for web apps, we only need endpoints for port 80 and 44306:24
mandreshadower: do you confirm? ^^06:24
jaosoriormandre: let me know which patches are ready for reviews (if you need any)06:24
openstackgerritSaravanan KR proposed openstack/tripleo-quickstart master: WIP: Enable OVS-DPDK in CI on featureset034  https://review.openstack.org/53788606:26
mandrejaosorior: will do, we're getting there06:26
*** yprokule has joined #tripleo06:29
*** hamdyk has joined #tripleo06:30
*** Chaserjim has quit IRC06:33
*** rcernin has joined #tripleo06:34
*** Chaserjim has joined #tripleo06:34
*** Petersingh is now known as Petersingh|afk06:36
*** rcernin_ has quit IRC06:36
*** jtomasek has joined #tripleo06:37
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: Disable vxlan_networking check for upgrades jobs.  https://review.openstack.org/60357006:38
*** saneax has joined #tripleo06:40
*** matbu has quit IRC06:43
*** hjensas has joined #tripleo06:43
*** matbu has joined #tripleo06:46
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: DNM: test undercloud with enable_metadata_network/enable_isolated_metadata  https://review.openstack.org/60357106:51
owalshjaosorior: ^^^ wonder if we need this for metadata to work06:51
owalshjaosorior: or NeutronForceMetadata06:52
*** chkumar|off is now known as chkumar|ruck06:53
*** cylopez has joined #tripleo06:54
*** janki has joined #tripleo06:55
openstackgerritMartin André proposed openstack/tripleo-quickstart master: Revert "Pin openshift-ansible to last known good version"  https://review.openstack.org/60357206:56
*** apetrich has joined #tripleo06:57
*** florianf|afk is now known as florianf06:57
*** pvc has joined #tripleo06:59
pvchi06:59
pvcykarel06:59
chkumar|ruckmandre: jaosorior needs +w on this https://review.openstack.org/#/c/596618/ to unblock promotion07:01
*** dxiri has quit IRC07:01
*** Chaserjim has quit IRC07:03
*** shyamb has quit IRC07:04
*** rcernin has quit IRC07:06
*** Chaserjim has joined #tripleo07:07
ykarelpvc, hi07:09
pvcim deploying now the overcloud07:10
pvcwait07:10
*** shyamb has joined #tripleo07:10
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Refactor openshift services for composable roles  https://review.openstack.org/59961807:10
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Fix openshift new node detection  https://review.openstack.org/60001207:10
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Clean up previous osa inventory dir before deployment  https://review.openstack.org/60002807:10
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Configure haproxy for openshift infra  https://review.openstack.org/60124107:10
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Use openshift-ansible container instead of RPMs  https://review.openstack.org/58386807:10
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Fix inventory files for newer openshift-ansible  https://review.openstack.org/60357707:10
ykarelpvc, so introspection completed?07:10
mandrechkumar|ruck: looking07:11
pvcnot yet :(07:11
ykarelchkumar|ruck, may be that's wrong patch u asked for +w, afaik it's not blocking promotion, although we should get that also07:11
pvcykarel can i ask?07:11
ykarelyes07:12
pvcabout the FFU from ocata to rocky07:12
pvcit's possible right?07:12
ykarelpvc, i don't know much there, but u can ask, someone else here can have answer07:12
ykarelafair jistr said that it's possible , but currently we are not testing it in CI07:12
ykarelpvc, ^^07:13
pvcbut newton to rocky is the one you test right?07:13
pvcso ocata to rocky is possible07:13
ykarelpvc, newton to queens07:14
ykarelrelease to release +307:14
*** assassin has joined #tripleo07:16
*** slaweq has joined #tripleo07:16
*** amoralej|off is now known as amoralej07:17
pvcnoted on this. :) wait im running the introspect07:17
openstackgerritQuique Llorente proposed openstack/tripleo-common stable/queens: Upgrade docker ceph container  https://review.openstack.org/60336907:17
*** rdopiera has joined #tripleo07:17
*** shyamb has quit IRC07:17
openstackgerritQuique Llorente proposed openstack/tripleo-common stable/rocky: Upgrade docker ceph container  https://review.openstack.org/60336707:18
*** Petersingh|afk is now known as Petersingh07:18
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: L3 routed networks - data + env (1/3)  https://review.openstack.org/58218007:18
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: L3 routed networks - network.j2 (2/3)  https://review.openstack.org/58218107:19
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: L3 routed networks - subnet fixed_ips (3/3)  https://review.openstack.org/58230107:19
pvcykarel unexpected boot error cant boot on pxe over ipv407:19
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add host_routes to port_from_pool.j2  https://review.openstack.org/60317207:20
ykarelpvc, no idea on this07:21
*** Chaserjim has quit IRC07:24
*** ykarel is now known as ykarel|lunch07:25
*** shyamb has joined #tripleo07:27
*** phuongnh has quit IRC07:28
*** Petersingh is now known as Petersingh|lunch07:29
*** rdopiera has quit IRC07:29
shadowermandre: yeah if you're talking about the ports for the router (i.e. infra node) than 80 and 443 should be enough07:29
*** rdopiera has joined #tripleo07:30
mandreshadower: perfect, thanks for the confirmation07:30
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Refactor openshift services for composable roles  https://review.openstack.org/59961807:31
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Fix openshift new node detection  https://review.openstack.org/60001207:31
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Clean up previous osa inventory dir before deployment  https://review.openstack.org/60002807:31
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Configure haproxy for openshift infra  https://review.openstack.org/60124107:31
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Fix inventory files for newer openshift-ansible  https://review.openstack.org/60357707:31
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Use openshift-ansible container instead of RPMs  https://review.openstack.org/58386807:31
quiquell|roverchkumar|ruck: This is new ? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz07:35
*** gkadam has joined #tripleo07:36
*** tosky has joined #tripleo07:37
*** jtomasek has quit IRC07:37
*** kopecmartin has joined #tripleo07:37
*** jtomasek has joined #tripleo07:39
*** Chaserjim has joined #tripleo07:41
jaosoriorowalsh: well, I tried that and it didn't work :/07:42
jaosoriordo you want me to give you access to the environment? if you wanna test it out07:42
owalshjaosorior: afk for a while but sure I guess... looking like a neutron issue AFAICT though07:43
jaosoriorowalsh: sure, let me know when you're around07:43
jaosoriorlater I'll poke beagles about this too :P07:43
pvchi07:45
pvcanyone?07:45
pvchttpboot for ocata right?07:45
*** shyamb has quit IRC07:45
pvccan review this?07:48
pvchi anyone07:48
pvchttp://paste.openstack.org/show/730329/07:48
pvcykarel http://paste.openstack.org/show/730329/07:50
mariosquiquell|rover: so lets merge  https://review.openstack.org/#/c/603322/3 ?07:52
mariosquiquell|rover: but please fixup the cherrypicks07:52
mariosquiquell|rover: and i couldn't quickly find pike07:52
mariosquiquell|rover: but -1 @ rocky/quieens07:52
*** cylopez has left #tripleo07:52
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: Check if openstack-glance-registry is enabled before stopping it.  https://review.openstack.org/60358107:53
quiquell|rovermarios: Pike is not needed, it has the correct one in their release files (I think )07:53
quiquell|rovermarios: Also already added the cherry-picks07:53
pvcquiquell can you review this? our introspection is failing :(07:53
mariosquiquell|rover: no please check the comments07:54
pvchttp://paste.openstack.org/show/730329/07:54
mariosquiquell|rover: you have to use the sytax like "cherry picked from xyz"07:54
mariosquiquell|rover: so gerrit will understand it please07:54
openstackgerritMerged openstack/tripleo-heat-templates master: Remove External network from DVR related configuration  https://review.openstack.org/59265207:54
openstackgerritMerged openstack/tripleo-heat-templates master: Add scenario010 to check job list  https://review.openstack.org/59903507:54
mariosquiquell|rover: makes it easier to track what has merged already07:54
mariosquiquell|rover: ack on pike though i don't fully understand why it is special/don't have the details07:55
quiquell|roverpvc: You have missing a parameter provisioning that node ipmi_terminal_port07:55
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: test running upgrade in standalone run.  https://review.openstack.org/60241607:55
quiquell|roverpvc: I think you needed some kind of json to provision it, that json is missin the parameter07:55
pvci have an instackenv.json07:56
quiquell|rovermarios: the git review -x was not adding it, beacuase it has conflicts, so I did it manually didn't remember the correct syntax07:56
quiquell|rovermarios: xyz is Change-Id, Commit-Id ?07:56
quiquell|roverpvc: This one is missing the parameter07:56
mariosquiquell|rover: cherry picked from commit 0c62b6cd8d696befb1c0c31bb6e206199ce1edac07:57
*** gkadam has quit IRC07:57
mariosquiquell|rover: yeah, so apparently git review -x will only add it automatically if the change is already merged. otherwise yeah yhou have to add it manually07:57
quiquell|rovermarios: ack07:57
mariosquiquell|rover: it is the COMMIT id07:57
mariosquiquell|rover: not the change id07:57
*** Chaserjim has quit IRC07:58
mariosquiquell|rover: oh you said git review -x , i mean the ui /click to cherrypick. hm i don't know why git review -x didn't add it07:58
openstackgerritQuique Llorente proposed openstack/tripleo-common stable/rocky: Upgrade docker ceph container  https://review.openstack.org/60336707:58
mariosquiquell|rover: sorry :) /me runs for more coffee07:58
pvcis there a default for quiquell07:58
openstackgerritQuique Llorente proposed openstack/tripleo-common stable/queens: Upgrade docker ceph container  https://review.openstack.org/60358307:59
mariosquiquell|rover: so no idea why git review -x didnt work for you. when you click to cherrypick from the gerrit UI, if the change is merged already you will get the nice cherry picked from in the commit message for you07:59
quiquell|rovershit07:59
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60327508:00
openstackgerritQuique Llorente proposed openstack/tripleo-common stable/queens: Upgrade docker ceph container  https://review.openstack.org/60336908:00
*** akrivoka has joined #tripleo08:00
quiquell|rovermarios: done08:00
quiquell|rovermarios: Don't know my theory is that if it has conflicts it does not add the line08:01
openstackgerritMerged openstack/tripleo-heat-templates master: Remove references to logging_group  https://review.openstack.org/59744508:01
openstackgerritMerged openstack/tripleo-heat-templates master: Enable fluentd on scenario002-multinode-containers  https://review.openstack.org/59701708:01
openstackgerritMerged openstack/tripleo-heat-templates master: heat-api-cfn fix log path  https://review.openstack.org/59702608:01
openstackgerritMerged openstack/tripleo-heat-templates master: heat-api fix log path  https://review.openstack.org/59705008:01
openstackgerritMerged openstack/tripleo-heat-templates master: Remove the duplicated word  https://review.openstack.org/59523408:01
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Stop hardcoding baremetal API version in stackrc  https://review.openstack.org/59745708:01
*** bogdando has joined #tripleo08:02
mariosquiquell|rover: so anyway, back to the original question what is the holdup on merging master? if any?08:05
mariosquiquell|rover: /me clicking... in 3 .. .08:05
quiquell|rovermarios: Don't know it already has zuul +1 and +w08:06
mariosquiquell|rover: heh yeah i just +w it08:06
quiquell|rovermarios: Zuul has just started the gates08:06
marioszuul +1 was from recheck alex did yesterday08:06
mariosquiquell|rover: my question was, y did you not merge it already what was the holdup08:06
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Undercloud - Restart keepalived on update  https://review.openstack.org/60358708:06
quiquell|rovermarios: What holdup ?08:07
mariosquiquell|rover: it has 4 +2s . no holdup, perhaps i should have said what are we waiting for in order to merge it.08:08
*** jpich has joined #tripleo08:08
mariosquiquell|rover: but np, thanks my question was answered ! :)08:08
quiquell|rovermarios: I suppose we were waitting for CI08:08
quiquell|rovermarios: Main one is the tqe08:09
quiquell|rovermarios: It's the one that unblocks mater/rocky/queens fs01908:09
*** dmacpher_ has quit IRC08:09
mariosquiquell|rover: looks like that is already in the gate?08:09
*** fhubik has joined #tripleo08:10
quiquell|rovermarios: Yep, timing issue, zuul is very busy now08:10
pvcquiquell08:10
pvcmy server always boot on tftp08:11
pvchow can i chnage thaT?08:11
quiquell|roverpvc: no clue man08:11
chkumar|ruckquiquell|rover: yes, I checked in the morning,08:12
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-quickstart master: ceilometer: Don't use deprecated meter  https://review.openstack.org/60359008:12
quiquell|roverchkumar|ruck: Created the bug for it08:12
quiquell|roverchkumar|ruck: It's in the etherpad08:12
*** dxiri has joined #tripleo08:13
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-quickstart master: ceilometer: Don't use deprecated meter  https://review.openstack.org/60359008:13
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-quickstart master: ceilometer: Don't use deprecated metric  https://review.openstack.org/60359008:13
*** sri_ has joined #tripleo08:14
openstackgerritMarios Andreou proposed openstack/tripleo-common stable/rocky: Upgrade docker ceph container  https://review.openstack.org/60336708:15
mariosquiquell|rover: trying to get the cherrypick to show up on gerrit ^^^08:15
mariosquiquell|rover: but it STILL isn't working i have no idea why08:16
mariosquiquell|rover: one example is https://review.openstack.org/#/c/583001/08:16
mariosquiquell|rover: you can see the Cherry-picks on the right hand side?08:16
quiquell|rovermarios: parentesis maybe08:16
quiquell|rovermarios: And after Change-Id08:17
mariosquiquell|rover: i included it08:17
quiquell|rovermarios: Argg...08:17
jaosoriormandre, marios: could you check this out https://review.openstack.org/#/c/603390/ ?08:17
chkumar|ruckquiquell|rover: you can do git cherry-pick -x <commit -id>08:17
chkumar|ruckit does the job08:17
quiquell|roverchkumar|ruck: It didn't add the line... I think it was because it had conflicts08:17
quiquell|rovermarios: Can you do the same for queens please ?08:18
marioschkumar|ruck: yeah that isn't the issue here but rather why gerrit isn't showing the related cherypicks at https://review.openstack.org/#/c/603367/08:18
marioschkumar|ruck: even when we added it08:18
quiquell|rovermarios: try to put it in the last line08:18
mariosquiquell|rover: you do it i mean so you can get your review back08:19
chkumar|ruckmarios: you mean this https://review.openstack.org/#/q/c7945101b360c01d988c5e991926389037a8ff7e?08:19
mariosquiquell|rover: just edit directly via gerrit its only commit message ;)08:19
mariosjaosorior: ack checking08:19
marioschkumar|ruck: not quite... more like https://up1-vrutkovs.int.open.paas.redhat.com/#YvD82G39QYUSd7HCzJS-wg08:21
marioschkumar|ruck: i.e. it doesn't pickup that this is cherrypick and show you the related branches in the gerrit front page08:21
openstackgerritQuique Llorente proposed openstack/tripleo-common stable/rocky: Upgrade docker ceph container  https://review.openstack.org/60336708:22
mariosquiquell|rover: perhaps master needs to merge? but that is strange08:22
openstackgerritQuique Llorente proposed openstack/tripleo-common stable/queens: Upgrade docker ceph container  https://review.openstack.org/60336908:22
quiquell|rovermarios: done08:23
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-heat-templates master: ceilometer: Use new archive policy  https://review.openstack.org/60355608:24
quiquell|rovermschuppert: We need a promotion to get the latest ironic rootwrap08:24
mschuppertquiquell|rover: ack, thx08:25
mariosjaosorior: ack please +a it if you are ready lgtm from quick visual review08:25
pvchi08:25
pvcanyone can help me on httpboot ?08:25
pvcit seems my node doesnt get the connectio non the pxe08:25
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: Removed the deprecated RamFilter from NFV environment files  https://review.openstack.org/60359508:26
jaosoriormarios: thanks08:26
*** Petersingh|lunch is now known as Petersingh08:26
quiquell|rovermschuppert: We can try the job with a Depends-On in the change to see if it passes08:26
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Allow enabling routed networks on undercloud update|upgrade  https://review.openstack.org/60136908:26
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: Remove the deprecated RamFilter from NFV environment files  https://review.openstack.org/60359508:26
mschuppertquiquell|rover: yes will do that in a bit08:27
quiquell|rovermschuppert: ok cool, let me know08:28
*** Chaserjim has joined #tripleo08:28
quiquell|roverchkumar|ruck: fs019 at queens promotion timed out :-/08:29
quiquell|roverchkumar|ruck: I think it's not related to cpeh08:29
quiquell|roverceph08:29
quiquell|roverchkumar|ruck: timeout in the gates too upstream http://logs.openstack.org/24/567224/109/check/tripleo-ci-centos-7-scenario004-multinode-oooq-container/c1361c5/08:31
*** jpena|off is now known as jpena08:31
openstackgerritMartin Schuppert proposed openstack/puppet-tripleo stable/queens: Revert "Revert "SSL support for haproxy -> novnc proxy connection""  https://review.openstack.org/59414508:31
chkumar|ruckquiquell|rover: scenario4 is timing out most of the time, sometime it passes and timedout08:32
jistrykarel|lunch, pvc: re which migrations of fast forward are possible, i don't think ocata->rocky would work currently, but ocata->queens might08:32
quiquell|roverchkumar|ruck: ack08:32
pvcjistr thank you so much are you familiar with pxelinux.0 0 bytes?08:35
chkumar|ruckquiquell|rover: in check jobs fs019 rocky passing https://review.rdoproject.org/zuul/builds.html?job_name=legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-rocky08:36
quiquell|roverchkumar|ruck: It's only periodic that fail, some DLRN rpm update breaks it08:37
quiquell|roverchkumar|ruck: It's not just the ceph thing, was not able to find the real rpm that breaks it08:37
quiquell|roverchkumar|ruck: wait... is periodic the one  you show me...08:37
chkumar|ruckquiquell|rover: if rpm is the issue, let me do adiff from a pass and fail08:37
quiquell|roverchkumar|ruck: That's interesting the ceph change is not merged08:38
quiquell|roverchkumar|ruck: is failing08:38
chkumar|ruckquiquell|rover: https://review.rdoproject.org/r/#/c/13943/33 on this one it passed08:38
quiquell|roverchkumar|ruck: the passing one is the one with the Depends-On08:38
*** shyamb has joined #tripleo08:38
chkumar|ruckpatchset 3308:38
quiquell|roverchkumar|ruck: That was with the Depepends-On tqe change, that it's not merged yet08:39
quiquell|roverchkumar|ruck: Last periodic has fail bacuase tqe is not merged yet08:39
quiquell|rover:-(08:39
quiquell|roverbig zuul queue08:39
*** ykarel|lunch is now known as ykarel08:41
ykareljistr, ack08:41
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade master: Configure undercloud for containerized upgrade.  https://review.openstack.org/60315108:42
*** shyamb has quit IRC08:43
jistrpvc: not familiar with that, sorry08:43
*** shyamb has joined #tripleo08:43
chkumar|ruckquiquell|rover: regarding rocky fs020 issue yesterday I was speaking with panda about the same08:45
chkumar|ruckquiquell|rover: http://paste.openstack.org/show/730331/08:45
chkumar|ruckquiquell|rover: we are doing a false positive guess08:45
bandiniany volunteers for a nice performance rabbit perf boost? https://review.openstack.org/#/c/602082/08:45
quiquell|roverchkumar|ruck: It's not a false possitive http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/common/deploy-steps-tasks.yaml#n22408:47
quiquell|roverchkumar|ruck: I would say false negative08:47
*** derekh has joined #tripleo08:49
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: enable ansible-lint as pre-commit hook  https://review.openstack.org/58216308:49
*** Chaserjim has quit IRC08:49
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Use python3 for tox linting  https://review.openstack.org/58858708:50
quiquell|roverchkumar|ruck: the name is missleading, it always print08:51
chkumar|ruckquiquell|rover: yes,08:51
*** owalsh has quit IRC08:51
openstackgerritJuan Badia Payno proposed openstack/tripleo-heat-templates master: Pacemaker-cinder-volume & pacemaker-cinder-backup log path fix  https://review.openstack.org/59746208:52
quiquell|roverchkumar|ruck: so, ansible goes well, but overcloud status is not ok08:53
*** chem has joined #tripleo08:55
chkumar|ruckquiquell|rover: do we need to do something here http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/overcloud-deploy/tasks/overcloud-status.yml ?08:55
*** owalsh has joined #tripleo08:56
openstackgerritJanki Chhatbar proposed openstack/tripleo-heat-templates master: Allow overlay tunnel endpoints on IPv6 address  https://review.openstack.org/60353208:56
*** dtantsur|afk is now known as dtantsur08:57
*** Chaserjim has joined #tripleo08:57
*** dmacpher_ has joined #tripleo08:58
quiquell|roverchkumar|ruck: the command openstack overcloud status is not returning DEPLOY_SUCCESS08:58
quiquell|roverchkumar|ruck: We need to know what's happening there08:59
chkumar|ruckquiquell|rover: it is coming from here http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/overcloud-deploy/templates/overcloud-deploy.sh.j2#n7409:00
quiquell|roverchkumar|ruck: Andd this iss good09:00
quiquell|roverchkumar|ruck: But we need to know why is not DEPLOY_SUCCESS09:01
quiquell|roverchkumar|ruck: The tasks has run ok09:01
chkumar|ruckquiquell|rover: is it because the message the connection has already closed?09:01
quiquell|roverchkumar|ruck:  or Error attempting to run <function create_incomplete_consumers at 0x7fd4c6e08d70>"09:02
quiquell|roverchkumar|ruck: We have to find09:02
chkumar|ruckquiquell|rover: I am updating the bug description09:03
quiquell|roverchkumar|ruck: Cool09:03
*** dxiri has quit IRC09:04
chkumar|ruck2018-09-19 03:12:38 |     "failed_when_result": false,09:05
chkumar|ruck2018-09-19 03:12:38 |     "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | default([]))": []09:05
chkumar|ruckquiquell|rover: ^6 something fishy is happening here that's why it is coming09:05
jankimarios, hey. Can you please workflow https://review.openstack.org/#/c/585015/?09:05
*** salmankhan has joined #tripleo09:06
openstackgerritDamien Ciabrini proposed openstack/puppet-tripleo master: mysql: use clustercheck credentials to poll galera state  https://review.openstack.org/60361209:07
quiquell|roverchkumar|ruck: Do you know where we can check the openstack status in the logs ?09:08
mariosjanki: ack09:09
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256009:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)09:10
chkumar|ruckquiquell|rover: nope ykarel ^^09:10
*** mhenkel_ has joined #tripleo09:11
quiquell|roverykarel: Do you know where to find in logs the openstack overcloud status call result ?09:11
quiquell|roverchkumar|ruck: Let's reproduce it, and call it online09:12
*** gfidente has joined #tripleo09:12
ykarelquiquell|rover, nope09:12
ykarelbut this is strange it's happening only in fs02009:12
quiquell|roverykarel: yep, ansible is passing, but openstack overcloud status does not return SUCCESSS but we don't know what return, going to try to reproduce09:13
jankimarios, thank you :)09:13
chkumar|ruckykarel: quiquell|rover do we capture /home/zuul/failed_deployments.log somewhere in CI?09:13
chkumar|ruckif not then I am going to captue it09:13
ykarelquiquell|rover, ack09:14
ykarelchkumar|ruck, yes it's captured09:14
ykareli have seen earlier, but in this case it didn't failed so not captured09:14
jankimarios, this also please https://review.openstack.org/#/c/596987/09:15
chkumar|ruckhttp://codesearch.openstack.org/?q=failed_deployments.log&i=nope&files=&repos=09:15
*** Chaserjim has quit IRC09:15
*** Chaserjim has joined #tripleo09:16
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Switch previous release of master from 'queens' to 'rocky'  https://review.openstack.org/59077409:17
pvcykarel09:24
dtantsurmorning folks! need your opinion on https://bugzilla.redhat.com/show_bug.cgi?id=1627949 don't we update things post-install?09:25
openstackbugzilla.redhat.com bug 1627949 in openstack-ironic "auto-discovery of nodes feature not enabling when configuring post undercloud install" [Unspecified,New] - Assigned to rhos-maint09:25
*** Chaserjim has quit IRC09:25
gchamoulakrivoka: hi Ana, could you please take a look at this one? https://review.openstack.org/#/c/513333/09:25
akrivokagchamoul: ack09:27
gchamoulthx09:27
*** Chaserjim has joined #tripleo09:29
*** shyamb has quit IRC09:29
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-heat-templates master: ceilometer: sync default change  https://review.openstack.org/60362109:29
*** shyamb has joined #tripleo09:30
bandinijaosorior: https://review.openstack.org/#/c/602082/ if you have a min09:30
dtantsuralso an easy backport if somebody has time: https://review.openstack.org/#/c/595765/09:31
quiquell|roverchkumar|ruck: Why do you think overcloud depploy script return false possitive ?09:31
quiquell|roverchkumar|ruck: grep -Eq DEPLOY_SUCCESS is legit09:32
chkumar|ruckquiquell|rover: https://logs.rdoproject.org/43/13943/34/check/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/ddd3b1e/logs/undercloud/home/zuul/tempest.log.txt.gz09:32
quiquell|roverchkumar|ruck: the command openstack overcloud status is returning somet hing different09:32
chkumar|ruckquiquell|rover: here it is passing09:32
chkumar|ruckquiquell|rover: failed in tempest09:32
quiquell|roverchkumar|ruck: So we go beyond overcloud deploy even if we don't have a SUCCESS_DEPLOY ?09:33
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Display errors when validations failed  https://review.openstack.org/60200509:34
openstackgerritGiulio Fidente proposed openstack/tripleo-common stable/rocky: Upgrade docker ceph container  https://review.openstack.org/60336709:34
openstackgerritGiulio Fidente proposed openstack/tripleo-common stable/queens: Upgrade docker ceph container  https://review.openstack.org/60336909:35
chkumar|ruckquiquell|rover: 2018-09-19 03:12:38 |     "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | default([]))": [] I was saying based on this output09:35
quiquell|roverchkumar|ruck: Latest from rocky is failing at overcloud deploy https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/job-output.txt.gz#_2018-09-19_02_34_02_84479609:36
quiquell|roverchkumar|ruck: Check is working fine09:36
quiquell|roverchkumar|ruck: Tempest is not executed09:37
chkumar|ruckquiquell|rover: one min, https://review.rdoproject.org/zuul/builds.html?job_name=legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky09:37
mariosack janki added comment please check i think it might be better to include the ""09:37
*** Chaserjim has quit IRC09:38
chkumar|ruckquiquell|rover: if we take a look at recent run from testproject, there is passing there and from earlier failure, soemehting was fishy there at the output during openstack status command09:38
quiquell|roverchkumar|ruck: Why is different the testrun and the periodic ?09:39
chkumar|ruckquiquell|rover: it is the same periodic job executed under https://review.rdoproject.org/r/#/c/13943/09:40
quiquell|roverchkumar|ruck: periodic fails at overcloud deploy but testproject at tempest :-/09:40
quiquell|roverchkumar|ruck: Yep, I know...09:40
quiquell|roverchkumar|ruck: weird09:40
*** shyamb has quit IRC09:41
quiquell|roverchkumar|ruck: grep -Eq DEPLOY_SUCCESS is ok at testproject09:41
quiquell|roverchkumar|ruck: It also has the Error attempting to run <function create_incomplete_consumers at 0x7fcf065ebd70>09:41
quiquell|roverchkumar|ruck: Maybe it was DEPLOY_IN_PROGRESS or the like, and now it takes a little more09:42
chkumar|ruckmy assumptions is wrong, is it issue with infra?09:42
quiquell|roverchkumar|ruck: Don't know two different run same job, different overcloud status :-/09:42
quiquell|roverchkumar|ruck: Do you have the path to the script ?09:43
quiquell|roverThe one that check the status09:43
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart-extras master: Remove validations in undercloud reinstall.  https://review.openstack.org/60362509:43
ykarelquiquell|rover, chkumar|ruck i don't think grep -Eq DEPLOY_SUCCESS has anything to do with the difference09:43
ykarelthe thing to deal is status_code09:44
quiquell|roverykarel: grep is ok, the overcloud status is not09:44
quiquell|roverykarel: maybe it wass IN_PROGRESS or the like, we want to know what it was09:44
jfrancoaquiquell|rover:  ^^^ if you want you can link that patch to check if it now re-runs without the validations https://review.openstack.org/60362509:45
*** shyamb has joined #tripleo09:45
quiquell|roverjfrancoa: This is rocky->master only isssue or it's general ?09:45
ykarelquiquell|rover, openstack overcloud deploy returned 1 somehow09:45
*** Chaserjim has joined #tripleo09:45
jfrancoaquiquell|rover: I'm not sure...seems like it's appearing only in rocky->master. Anyway, it will be applied to all branches as the patch is in tripleo-quickstart-extras09:46
quiquell|roverjfrancoa: ack, thanks so much09:46
*** owalsh has quit IRC09:46
jfrancoaquiquell|rover: you're welcome09:46
openstackgerritQuique Llorente proposed openstack-infra/tripleo-ci master: Switch previous release of master from 'queens' to 'rocky'  https://review.openstack.org/59077409:46
quiquell|roverchkumar|ruck: Can you put back the bug with the original comment, it's not a false positive or negative09:47
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: [DNM] testing overcloud deploy ststaus  https://review.openstack.org/60362709:48
chkumar|ruckquiquell|rover: ^^ checking here overcloud status09:49
quiquell|roverchkumar|ruck: wait status is ok09:51
ykarelchkumar|ruck, good to remove the pipe for above command, that has the issue09:51
ykarelnot the status one09:51
quiquell|roverykarel, chkumar|ruck: We are not running openstack overcloud failures09:51
ykareldue to some reason overcloud deploy failed09:51
quiquell|roverykarel, chkumar|ruck: so sucess is ok09:51
*** owalsh has joined #tripleo09:52
openstackgerritJanki Chhatbar proposed openstack/tripleo-common master: Make ODL healthcheck IPv6 compatible  https://review.openstack.org/59698709:52
jankimarios, ^09:52
quiquell|roverreturning value is exit_code not the output not grep09:52
ykarelthat's what i am trying to say for some time :)09:52
quiquell|roverykarel: sorry :-)09:53
mariosthank you janki done09:53
quiquell|roverykarel: so, we have a DEPLOY_SUCCESS but the status_code is 109:53
quiquell|roverykarel: that's it ?09:53
ykarelquiquell|rover, yes exactly, and that's because openstack overcloud deploy command failed for some reason, may be some issue with tripleoclient09:54
ykarelbut not sure09:54
jankimarios, thanks :)09:54
quiquell|roverykarel: But then why do we have DEPLOY_SUCCESS ?09:54
ykarelthough stack created successfully but not the command09:54
quiquell|roverykarel: ahh ok, minor issue with client but stack is there09:55
*** mhenkel_ has quit IRC09:55
ykarelquiquell|rover, yes that only need to find, tripleoclient is wrapper, it can have it's own exit codes09:55
ykarelso need to find09:55
quiquell|roverykarel: Damn it always with the wrapping, looks like we are working at a presents shop09:55
ykarel:)09:55
ykarelat present shop, shopkeeper knows what he is packing, but we don't :)09:56
quiquell|roverykarel: so, heap, mistral logs re going to be ok, since the stack is there09:56
quiquell|roverykarel: we are just the wrapping monkeis09:56
*** shyamb has quit IRC09:57
ykarelquiquell|rover, if it's reproduced without || status_code=$? in the deploy command we might get some output09:58
pvcykarel NO DHCP or proxyDHCP offers were received09:58
quiquell|roverchkumar|ruck: You are right, maybe is just the "Connection is already closed."09:59
ykarelpvc, what's ur environment, baremetal, vms ?09:59
quiquell|roverand tripleoclient is returning 1 there09:59
pvcim using baremetal server09:59
*** shyamb has joined #tripleo10:00
ykarelpvc, so multiple baremetals, 1 for undercloud and different for overcloud10:00
chkumar|ruckquiquell|rover: let see10:01
pvcyes10:01
pvci have 3 baremetasl10:01
pvc1 for udnerclound 1 for overcloud controller nad 1 for overcloud compute10:01
ykarelyes that Connection is Closed seems relevant10:02
ykarelpvc, ack10:02
quiquell|roverIt's around here http://git.openstack.org/cgit/openstack/python-tripleoclient/tree/tripleoclient/plugin.py#n14210:02
pvcack10:03
ykarelpvc, is dnsmasq running, systemctl status openstack-ironic-inspector-dnsmasq10:03
ykareland can u share ironic-inspector logs10:04
openstackgerritJanki Chhatbar proposed openstack/tripleo-heat-templates master: Fix files conditions for scenario 007 and 008  https://review.openstack.org/60219610:04
ykareli think there some info would be available10:04
pvchow can i boot it on httpboot?10:04
pvcit seems it booting on tfpt10:04
*** Chaserjim has quit IRC10:04
*** dxiri has joined #tripleo10:04
*** shyamb has quit IRC10:05
pvcdnasmasq running fine ykarel10:05
ykarelpvc, don't know about that httpboot switch10:06
ykarelpvc, have u tried troubleshooting defined here: https://docs.openstack.org/ironic-inspector/latest/user/troubleshooting.html10:09
ykarelgood to start with that10:09
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256010:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)10:10
*** Chaserjim has joined #tripleo10:12
openstackgerritMartin Schuppert proposed openstack/tripleo-heat-templates master: Move nova-metadata api to httpd wsgi  https://review.openstack.org/58262310:14
openstackgerritQuique Llorente proposed openstack/python-tripleoclient stable/rocky: Add a guard to break if no connection  https://review.openstack.org/60363310:15
quiquell|roverykarel, chkumar|ruck: ^10:15
*** fhubik is now known as fhubik|brb10:15
quiquell|roverchkumar|ruck: Going to change the bug name and description10:16
*** paramite|clone has joined #tripleo10:20
pvccan i use pxe_ssh ykarel10:21
quiquell|roverpvc: I think pxe_ssh is deprectated10:22
pvcso i cnat use that?10:22
quiquell|roverpvc: You have to use pxe_ipmitool10:22
quiquell|roverI think...10:22
pvcipmitool connection is fine10:22
pvcbut it cant boot over the pxe10:22
pvcdo i need to add an mac address?10:22
quiquell|roverpvc I think it discovers it10:23
quiquell|roverpvc only the ip address and port is needed10:23
pvcport?10:23
pvcwhat port shoud i add10:23
pvci dont have port on my json10:23
quiquell|roverWait you are deploy in real baremetal with real ipmi ?10:23
pvcyes with real ipmi10:24
pvcbaremetal with real ipmi10:24
pvcwhat port can i use?10:24
quiquell|roverpvc then the type is just ipmi and you don't need a port10:25
pvc"pm_type":"pxe_ipmitool",10:25
quiquell|roverjust pm_user, pm_password, pm_addr10:25
pvcthat "pm_type":"pxe_ipmitool",?10:25
quiquell|rover"pm_type": "ipmi"10:25
pvcokay wait10:25
dtantsurand pm_user (or is it pm_username?)10:26
pvcshould i also add ipmi on enabled drivers?10:26
pvcpm_user10:26
quiquell|roverpvc: what release ocata ?10:26
pvcocata10:26
pvcenabled_drivers = pxe_ssh,pxe_drac,pxe_ilo,pxe_ipmitool10:27
pvcno ipmi :(10:27
dtantsuruse pxe_ipmitool then10:27
pvcwait10:27
quiquell|roveryep, the params are the same10:28
pvchere http://paste.openstack.org/show/730336/10:28
quiquell|roverboot_otion:local10:28
quiquell|roverfeels wrong10:28
dtantsuryep, a typo there10:29
pvci'll delete it?10:29
dtantsuryou can delete it, boot_option:local is the default10:29
quiquell|roverboot_option:local <- pvc10:29
pvcokay wait thank you10:29
pvci will run the baremetal import and introspection and monitor the ipmi console of that node10:29
quiquell|roverchkumar|ruck: At your run neutron_tempest_plugin.api.admin.test_networks.NetworksTestAdmin.test_create_network_with_project  is failing10:31
quiquell|roverpvc: yw let us know10:31
pvcdo i need to share my dnsmasq.conf?10:32
*** shyamb has joined #tripleo10:33
pvchere quiquell|rover10:34
pvchttp://paste.openstack.org/show/730337/10:34
pvcquiquell|rover PXE-MOF : Exiting Intel Boot Agent10:35
pvcon ipmi console10:35
pvcquiquell|rover NO DHCP or proxyDHCP offers were received10:35
dtantsurthis DHCP business has no relationship with ironic drivers largely10:36
dtantsurpvc: chances are non-zero you're hitting https://docs.openstack.org/ironic/latest/admin/troubleshooting.html#dhcp-during-pxe-or-ipxe-is-inconsistent-or-unreliable10:36
pvcwhere should i run this config terminal?10:37
*** leanderthal has joined #tripleo10:37
pvcit can get an IP address then afterward NO DHCP or proxyDHCP offers were received10:37
*** Chaserjim has quit IRC10:38
pvcshould i add an mac address dtantsur? i have 3 interface for my overcloud baremetal ndoe10:38
*** Petersingh is now known as Petersingh|afk10:39
*** dciabrin has joined #tripleo10:39
dtantsurpvc: it should not matter10:39
pvcdtantsur Enabled processing hooks: ['ramdisk_error', 'root_disk_selection', 'scheduler', 'validate_interfaces', 'capabilities', 'pci_devices', 'extra_hardware', 'lldp_basic', 'local_link_connection']10:41
dtantsurpvc: you're looking at the wrong place. ironic is not even in play yet, you're probably facing networking issues.10:41
dtantsurhave you checked the link I posed above?10:41
pvcyes10:41
dtantsurhave you tried tcpdump on the undercloud?10:41
pvcwait10:41
*** Chaserjim has joined #tripleo10:42
pvctcpdump -i any port 67 or port 68 or port 6910:42
pvcis that right dtantsur10:43
dtantsurlooks correct to me10:43
dtantsuralso check logs from openstack-ironic-inspector-dnsmasq service10:44
*** Chaserjim has quit IRC10:46
*** Chaserjim has joined #tripleo10:46
pvcdtantsur http://paste.openstack.org/show/730338/10:47
dtantsurokay, then check the logs of that service above. maybe it rejects requests from some reason10:49
*** shyamb has quit IRC10:49
*** shyamb has joined #tripleo10:49
pvcjournalctl -u openstack-ironic-inspector-dnsmasq?10:51
dtantsuryep10:52
quiquell|roverchkumar|ruck: pike promotion issue https://bugs.launchpad.net/tripleo/+bug/179329310:52
openstackLaunchpad bug 1793293 in tripleo "pike promotion failing trying to use IPv6 to get buildlogs" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)10:52
quiquell|roverdtantsur: who are the experts on python-tripleoclient ?10:53
dtantsurquiquell|rover: everyone here? :)10:53
dtantsurwell, except for me, I only know ironic :)10:53
quiquell|roverdtantsur: Hehe like asking a foreigner about directions :-)10:54
*** dxiri has quit IRC10:55
pvcdtantsur http://paste.openstack.org/show/730340/10:55
chkumar|ruckquiquell|rover: that is the mirror timed out issue10:55
pvctcpdump dtantsur http://paste.openstack.org/show/730339/10:55
*** Chaserjim has quit IRC10:56
pvcdtantsur PXE-E32: TFTP open timeout10:56
chkumar|ruckquiquell|rover: https://review.openstack.org/603633 review go against master and then backported to stable/rocky10:56
quiquell|roverchkumar|ruck: This is different, it's using IPv610:56
*** Petersingh|afk is now known as Petersingh10:56
dtantsurah, TFTP timeout. that's a different story, I guess10:57
quiquell|roverchkumar|ruck: It's not a 404 it cannot access IPv6 address10:57
dtantsurpvc: anything curious in inetd logs (probably in journald)?10:57
pvchere is the journalctl http://paste.openstack.org/show/730340/10:57
chemweshay_pto: omg wes is on pto10:57
chemweshay_pto: marios: super good news, the standalone upgrade jobs passed!!10:58
chemweshay_pto: marios: jfrancoa found a legitimate upgrade bug and fixed it in https://review.openstack.org/#/c/603581/10:58
quiquell|roverchkumar|ruck: Will do the review to master and backport it10:58
chemweshay_pto: marios: then added the depends-on in https://review.openstack.org/#/c/602416/10:58
marioschem: nice :) noted for checking in bit10:59
chemweshay_pto: marios: and we can see in zuul that it worked10:59
*** Chaserjim has joined #tripleo10:59
chemmarios: jfrancoa I want to cry, it's the most beautiful day in my life10:59
chemmarios: jfrancoa upgrade tested in 1h15 !10:59
*** panda has joined #tripleo11:00
pvccan i use httpboot than tftpboot dtantsur?11:00
chemEmilienM: mwhahaha thanks a lot for the standalone work, we found an issue in upgrade thanks to it, how amazing is that!  And it tooks 1h15m11:00
dtantsurpvc: sorry, I don't get the question11:01
* chem is overreacting 11:01
openstackgerritQuique Llorente proposed openstack/python-tripleoclient master: Add a guard to break if no connection  https://review.openstack.org/60364111:01
* chem celebrating for the rest of the week11:01
pvcit seems that my pxe is booting on tftp not on http11:01
quiquell|roverbandini: cherry-picks need to share Change-Id from cherry-picked ?11:02
pvcnothing wrong on the journalctl dtantsur11:02
quiquell|rovergfidente: ^11:02
dtantsurpvc: it's because do use ipxe, we need to get the ipxe ROM to the node (if your node does not have it already)11:02
chkumar|ruckquiquell|rover: a nit edit in commit msg and rest is ok for python-tripleoclient11:02
dtantsurfor that we need tftp11:02
quiquell|roverchkumar|ruck: argg I am redoing it11:02
gfidentequiquell|rover yes11:02
gfidenteI updated both the cherry picks11:03
quiquell|rovergfidente: ack11:03
pvci see to tftp is fine i just having the PXE-E32: TFTP open timeout11:03
quiquell|rovergfidente: Have new one about other stuff didn't know I have tu respect Change-Id11:03
quiquell|rovergfidente: Thanks for the update11:03
quiquell|rovergfidente: btw, review -x didn't add the cherry pick line, maybe because there were conflicts ?11:04
dtantsurpvc: if the logs are fine, my only guess is STP on the port (the link I posted some time ago)11:04
openstackgerritQuique Llorente proposed openstack/python-tripleoclient stable/rocky: Add a guard to break if no connection  https://review.openstack.org/60364511:05
quiquell|rovergfidente: Do the cherry picked have to be merged ?11:05
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove references to logging_source  https://review.openstack.org/60364911:06
jaosoriorjbadiapa: ^^11:06
openstackgerritQuique Llorente proposed openstack/python-tripleoclient stable/queens: Add a guard to break if no connection  https://review.openstack.org/60365011:06
pvcdtantsur but it can get an IP address from the dhcp server11:06
pvccan i use also the uefi boot instead of local?11:06
*** shyamb has quit IRC11:06
chkumar|ruckquiquell|rover: cherry picking is beahving weired11:07
dtantsurpvc: you can, but it may not solve networking problems you experience..11:07
quiquell|roverchkumar|ruck: Do I have to backport to pike too ?11:07
dtantsurpvc: as to STP, I've seen cases where it kicked in after initial DHCP11:07
chkumar|ruckquiquell|rover: i donot think so11:07
quiquell|roverchkumar|ruck: Don't to rocky and queens11:07
quiquell|rovers/Don't/Done/g11:07
quiquell|roverchkumar|ruck: I think https://bugs.launchpad.net/tripleo/+bug/1793293 is different than the mirror one11:09
openstackLaunchpad bug 1793293 in tripleo "pike promotion failing trying to use IPv6 to get buildlogs" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)11:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256011:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)11:10
*** ooolpbot has quit IRC11:10
pvcdtantsur >> Started Pxe over ipv4  pxelinux.0 0 bytes11:10
chkumar|ruckquiquell|rover: where can i look for IPV6 addrees is used?11:12
chkumar|ruckquiquell|rover: oh sorry got that11:12
chkumar|ruckquiquell|rover: you are correct11:12
*** dtrainor has quit IRC11:13
*** dtrainor has joined #tripleo11:13
pvcwe'll try to switch on boot mode uefi dtantsur11:14
pvcwhere can i do this on undercloud dtantsur DHCP during PXE or iPXE is inconsistent or unreliable?11:14
*** sri_ has quit IRC11:15
bandiniquiquell|rover: yes11:15
*** Chaserjim has quit IRC11:16
mschuppertquiquell|rover: adding depends on did not help. we still have the old rootwrap filter e.g. http://logs.openstack.org/45/594145/3/check/tripleo-ci-centos-7-3nodes-multinode/a4078c7/logs/undercloud/etc/ironic-inspector/rootwrap.d/11:17
quiquell|rovermschuppert: that's weird... maybe build-test-packages is not working fine11:19
quiquell|rovermschuppert: do you have the review ?11:19
mschuppertquiquell|rover: https://review.openstack.org/#/c/594145/ .tests are still running. the above is from 1st failed one11:19
*** jpena is now known as jpena|lunch11:20
*** Chaserjim has joined #tripleo11:20
quiquell|rovermschuppert: Humm points to multiple branches, let me check if it can handle that11:21
jaosoriorbeagles: around?11:21
mschuppertquiquell|rover: ack. will be back after lunch11:22
quiquell|rovermschuppert: ok11:22
openstackgerritQuique Llorente proposed openstack/python-tripleoclient master: Add a guard to break if no connection  https://review.openstack.org/60364111:25
*** dhill_ has quit IRC11:25
openstackgerritQuique Llorente proposed openstack/python-tripleoclient stable/rocky: Add a guard to break if no connection  https://review.openstack.org/60364511:26
hjensasmschuppert: I don't think depends on works for merged patches ... I think it expects stuff just to be there in that case, but with our promotion being 2-3 weeks behind master ? ... *sigh*11:26
openstackgerritQuique Llorente proposed openstack/python-tripleoclient stable/queens: Add a guard to break if no connection  https://review.openstack.org/60365011:26
quiquell|rovermschuppert: I think you need to depend on https://review.openstack.org/#/c/598229/11:29
*** Chaserjim has quit IRC11:33
*** Chaserjim has joined #tripleo11:34
beaglesjaosorior: yep11:38
jaosoriorbeagles: hey, got some time to help me out debugging the TLS everywhere issue we talked about in Denver?11:39
beaglesjaosorior: the metadata thing?11:39
*** ssbarnea has quit IRC11:39
jaosoriorbeagles: yeah11:39
beaglesjaosorior: yep sure11:40
beaglesjaosorior: so this is metadata from overcloud node to the metadata agent on the undercloud/11:40
beagles?11:40
jaosoriorbeagles: correct11:40
jaosoriorbeagles: sent you a PM with the tmate session11:41
beaglesjaosorior: gotcha11:41
*** med_ has joined #tripleo11:42
*** owalsh has quit IRC11:42
*** rh-jelabarre has joined #tripleo11:46
*** morazi has joined #tripleo11:46
pvchi11:49
pvcykarel dtantsur11:49
pvcocata to rocky is possible right11:49
*** ansmith has quit IRC11:49
quiquell|rovermschuppert: confirmed build-test-packages is not reconstructing the ironic-inspector with the changes, can I update the patch ?11:50
quiquell|rovermschuppert: http://logs.openstack.org/45/594145/3/check/tripleo-ci-centos-7-3nodes-multinode/a4078c7/logs/undercloud/home/zuul/install_built_repo.log.txt.gz11:51
*** Petersingh is now known as Petersingh|afk11:51
quiquell|rovermschuppert: ZUUL_CHANGES does not have the Depends-On on it :-/ ZUUL_CHANGES=openstack/puppet-tripleo:stable/queens:refs/changes/45/594145/311:52
jaosoriormandre: what scenario was running openshift?11:55
mandrescenario00911:55
openstackgerritMarios Andreou proposed openstack/tripleo-specs master: Improve upgrades_tasks CI coverage with standalone for Stein  https://review.openstack.org/57985411:55
quiquell|roverjaosorior: It's was failing on something about the controllers11:55
*** ssbarnea has joined #tripleo11:56
*** udesale has quit IRC11:56
*** Petersingh|afk has quit IRC11:56
*** raildo has joined #tripleo11:56
dtantsurpvc: you mean, upgrade? you need to upgrade release by release (except for FFU newton -> queens)11:58
pvcFFU from ocata to rocky11:59
pvcis that possible?11:59
mschuppertquiquell|rover: sure you can update the patch12:00
quiquell|rovermschuppert: Just try to do Depends-On: https://review.openstack.org/#/c/598229/12:01
EmilienMchem: 1h15 is still long imho, but cool if it was helpful12:01
quiquell|rovermschuppert: The changes are not at zuul_changes so build-test-package is not building it12:01
mschuppertquiquell|rover: sure, will do that12:02
openstackgerritMartin Schuppert proposed openstack/puppet-tripleo stable/queens: Revert "Revert "SSL support for haproxy -> novnc proxy connection""  https://review.openstack.org/59414512:03
quiquell|rovermschuppert: you can check logs/undercloud/home/zuul/install_built_repo.log.txt.gz to see if the instack-undercloud rpm is building12:03
*** Chaserjim has quit IRC12:03
mschuppertquiquell|rover: perfect. will let you know12:03
*** amoralej is now known as amoralej|lunch12:04
dtantsurpvc: not supported iirc12:06
dtantsurbut I'm not the biggest upgrade expert here12:07
*** Chaserjim has joined #tripleo12:07
*** mhenkel_ has joined #tripleo12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256012:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)12:10
pvcso if ever i will use ocata to queens12:10
pvcthen upgrade queens to rocky?12:11
jistrpvc: you'd probably be the first to try this, but i think it has a chance of working fine :)12:13
*** dprince has joined #tripleo12:13
pvcyou mean ocata to rocky jistr?12:13
jistrpvc: i mean ocata to queens FFU, then queens to rocky normal. I think FFU direct from ocata to rocky doesn't have much chance of working now.12:14
*** shyamb has joined #tripleo12:15
*** rfolco has joined #tripleo12:17
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart-extras master: Update subnode repos for mixed_upgrades jobs.  https://review.openstack.org/60370912:17
pvci see jistr we'll get back to you if our FFU will be successful :) im on the part of deploying the overcloud now12:17
*** thrash|g0ne is now known as thrash12:19
*** Chaserjim has quit IRC12:20
*** med_ has quit IRC12:20
*** Chaserjim has joined #tripleo12:24
*** jpena|lunch is now known as jpena12:24
mandretrown|outtypewww: in case you didn't notice, I've fixed our issue with newest openshift-ansible, https://review.openstack.org/#/c/603577/12:28
*** toure|gone is now known as toure12:29
mandretrown|outtypewww: I'm thinking about re-ordering https://review.openstack.org/#/c/600067/ after https://review.openstack.org/#/c/601241/ so we don't have to depend on https://review.openstack.org/#/c/602112/12:29
quiquell|roverTo close promotion-blocker at rocky for tripleoclient https://review.openstack.org/#/c/603641/12:30
*** Chaserjim has quit IRC12:30
mandretrown|outtypewww: we'll make your test patch depend on https://review.openstack.org/#/c/603050/ and add the infra service to scenario00912:32
*** quiquell|rover is now known as quique|rover|lch12:33
*** Chaserjim has joined #tripleo12:35
*** rbrady has joined #tripleo12:36
*** Chaserjim has quit IRC12:40
pvchi jistr12:40
*** Petersingh|afk has joined #tripleo12:41
gfidentemwhahaha not sure if you know anything about this but ironic-conductor container in undercloud is trying to deploy an image saving it in /tmp first and it's running out of space12:42
gfidentemwhahaha I suppose this has been fixed already but I am not sure what it the right fix?12:42
pvchi can i ask guys if i can set a default size on overcloud node? so if i have an 1TB i will just use 500GB12:43
*** Chaserjim has joined #tripleo12:44
mwhahahagfidente: no idea. Ask dtantsur12:44
dtantsurgfidente: ironic does cache images locally, not sure what the question is12:45
gfidentedtantsur yeah in the container it's running out of space when caching12:45
gfidentebecause tmp is only 4k size12:45
gfidentelooks like a config issue, probably fixed in a version newer than what I deployed12:46
*** leanderthal has quit IRC12:46
*** jfrancoa has quit IRC12:46
jankicgoncalves, hi12:46
dtantsurI suspect so, /tmp is not the right path12:47
*** Petersingh|afk has quit IRC12:48
*** Petersingh|afk has joined #tripleo12:48
EmilienMmwhahaha: is today a good day to release rocky rc2?12:52
*** dhill_ has joined #tripleo12:52
*** Chaserjim has quit IRC12:52
mwhahahaEmilienM: probably as good a day as any. Let me check the state of things in a bit and get back to you12:53
EmilienMk12:53
EmilienMthe gate is pretty backed up12:54
EmilienM13 hours12:54
EmilienMonly one alert though12:54
openstackgerritMartin André proposed openstack/tripleo-quickstart master: Bump openshift-ansible to latest released rpm  https://review.openstack.org/60357212:55
*** trown|outtypewww is now known as trown|brb12:55
*** trown|brb is now known as trown12:56
trownmandre: that reorder will be a bit painful since the composable role refactor is in between12:56
*** Chaserjim has joined #tripleo12:56
trownmandre: trying out your tht patch by itself with latest openshift-ansible package12:57
jistrpvc: you mean partitioning? hmm i don't know, that would probably be a question for Ironic12:57
trownmandre: I am totally stuck on the container permission thing... selinux is permissive and I still hit that issue12:58
EmilienMchkumar|ruck, quique|rover|lch: any patch that we need landed asap to resolve the timeout issues?12:58
EmilienMI'm asking in case we need to purge the gate so we can land faster blocker patches12:59
mandretrown: alright, I'll https://review.openstack.org/#/c/599618/ on top of https://review.openstack.org/#/c/602112/ and update scenario009 to add the infra service in a later patch12:59
mandretrown: rebooting the undercloud did the trick for me12:59
*** tzumainn has joined #tripleo13:00
mandrewindows 98 style13:00
EmilienMs/98/all versions/13:00
trownmandre: ya I tried that... CI networking setup doesnt persist reboot though13:00
mandretrown: hmm it would be good to figure out what causes mistral-executor not to be able to read from the docker socket13:01
mandretrown: what was really odd it that when I shelled in the container and executed the same docker command it worked13:02
mandreas the mistral user13:02
trownhmm I wonder if just restarting mistral-executor would have the same effect as a reboot13:02
*** nodoz has joined #tripleo13:03
mandregive it a try, I suppose it should work13:03
*** jfrancoa has joined #tripleo13:03
chkumar|ruckEmilienM: https://review.openstack.org/#/c/603323/ already in gates and https://review.openstack.org/60364113:03
*** ratailor has quit IRC13:03
*** gbarros has joined #tripleo13:03
*** ansmith has joined #tripleo13:04
*** holser_ has quit IRC13:04
*** holser__ has joined #tripleo13:04
EmilienM603323 is in check and 603641 too and no review yet13:05
EmilienMso should we clear the gate to avoid timeouts?13:05
chkumar|ruckEmilienM: https://review.openstack.org/603621 this one also need13:05
chkumar|ruckEmilienM: syes, sure,13:06
EmilienMsileht: could we squash https://review.openstack.org/#/c/603556/ and https://review.openstack.org/#/c/603621/ ?13:07
openstackgerritMartin Schuppert proposed openstack/tripleo-heat-templates master: Move nova-metadata api to httpd wsgi  https://review.openstack.org/58262313:07
*** jfrancoa has quit IRC13:07
*** sri_ has joined #tripleo13:08
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256013:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)13:10
*** ooolpbot has quit IRC13:10
openstackgerritMartin André proposed openstack/tripleo-common master: Switch to origin-docker-build  https://review.openstack.org/59930713:10
openstackgerritMartin André proposed openstack/tripleo-common master: Add container-registry image to openshift master role  https://review.openstack.org/60211213:10
openstackgerritMartin André proposed openstack/tripleo-common master: Tag openshift images for Infra service  https://review.openstack.org/60305013:10
*** jfrancoa has joined #tripleo13:11
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates stable/rocky: Add CephOSD service to roles/Standalone.yaml  https://review.openstack.org/60375813:11
*** Chaserjim has quit IRC13:14
*** janki has quit IRC13:16
mandretrown: stupid question warning - we're only deploying a master node in CI, correct? no worker?13:17
trownmandre: ya... we are only deploying one node13:17
trownmandre: all in one13:17
mandretrown: ok, I'll change it to be all in one then13:18
mandretrown: well... we're only deploying the master atm13:18
mandreunless i'm mistaken13:18
*** Chaserjim has joined #tripleo13:19
*** quique|rover|lch is now known as quiquell|rover13:21
trownmandre: hmm... but we can launch an app I think..13:21
*** amoralej|lunch is now known as amoralej13:22
*** skramaja has quit IRC13:22
mandretrown: I don't think we can, we don't have a worker13:23
quiquell|roverEmilienM: The main fix that have to land to unblock rocky is this https://review.openstack.org/#/c/603323/13:24
*** aufi has quit IRC13:24
quiquell|roverEmilienM: And this can fix fs020 tripleoclient issues https://review.openstack.org/#/c/603645/13:24
trownmandre: hmm... I havent tried that I guess13:25
*** rdopiera has quit IRC13:26
*** Chaserjim has quit IRC13:26
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Refactor openshift services for composable roles  https://review.openstack.org/59961813:26
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Fix openshift new node detection  https://review.openstack.org/60001213:26
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Clean up previous osa inventory dir before deployment  https://review.openstack.org/60002813:26
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Configure haproxy for openshift infra  https://review.openstack.org/60124113:26
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Fix inventory files for newer openshift-ansible  https://review.openstack.org/60357713:26
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Use openshift-ansible container instead of RPMs  https://review.openstack.org/58386813:26
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Deploy openshift all in one in scenario009  https://review.openstack.org/60378013:26
*** Chaserjim has joined #tripleo13:26
mandretrown: https://review.openstack.org/#/c/603780/ should do it13:27
jfrancoaquiquell|rover: hey, have you observed in the undercloud upgrades an issue with ironic-inspector? Sep 19 09:40:17 centos-7-limestone-regionone-0002118624 ironic-inspector[24550]: Stderr: u'/usr/bin/ironic-inspector-rootwrap: Unauthorized command: systemctl start openstack-ironic-inspector-dnsmasq.service (no filter matched)\n'13:29
*** Petersingh|afk has quit IRC13:31
EmilienMquiquell|rover: see comment on https://review.openstack.org/#/c/603645/13:32
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-heat-templates master: ceilometer: Use new archive policies  https://review.openstack.org/60362113:36
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart-extras master: Remove validations in undercloud reinstall.  https://review.openstack.org/60362513:37
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates master: Expose IronicImageDownloadSource as a parameter  https://review.openstack.org/60379613:37
*** udesale has joined #tripleo13:38
quiquell|roverjfrancoa: yep, there is a fix, but we need a promotion to get the proper rpm.13:39
quiquell|roverjfrancoa: https://bugs.launchpad.net/tripleo/+bug/179307313:39
openstackLaunchpad bug 1793073 in tripleo "[queens] fs01 noop job failed with Stderr: u'/usr/bin/ironic-inspector-rootwrap: Unauthorized command: systemctl start openstack-ironic-inspector-dnsmasq.service (no filter matched)" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)13:39
jfrancoaquiquell|rover: ack, thanks13:41
mwhahahajaosorior: plz +1 https://review.openstack.org/#/c/603489/13:42
*** Chaserjim has quit IRC13:43
jaosoriormwhahaha: ack13:44
chemEmilienM: as a side note the 1h15 include a full ansible-lint of the generated ansible playbook13:44
chemEmilienM: I'm kind of unsure what's the next step here, should we make another Standalone jobs with the upgrade or keep that that one and include the lint+upgrade in it ?13:45
chemEmilienM: I'm in favor of 2 (add lint and upgrade to the current one)13:45
openstackgerritQuique Llorente proposed openstack/python-tripleoclient master: Add a guard to break if no connection  https://review.openstack.org/60380213:46
gfidentedtantsur where is conductor supposedly caching the local copy of the image?13:46
gfidentedtantsur I am not sure why it's trying to cache it in /tmp for me13:47
gfidentedtantsur and if that is expected at all?13:47
dtantsurgfidente: somewhere under /var/lib/ironic if the memory serves me right (this is actually configured by tripleo)13:47
openstackgerritQuique Llorente proposed openstack/python-tripleoclient stable/rocky: Add a guard to break if no connection  https://review.openstack.org/60380413:48
quiquell|roverEmilienM: ^ did a git review -x but the cherry pick line does not appear13:49
*** aufi has joined #tripleo13:49
EmilienMquiquell|rover: what if I try?13:49
EmilienMdo you send me a beer in Quebec?13:49
gfidentedtantsur ok that is great I have some images there already indeed13:49
quiquell|roverEmilienM: A manual to craft beer would be better13:50
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/rocky: Add a guard to break if no connection  https://review.openstack.org/60380413:50
EmilienMquiquell|rover: done ^13:50
EmilienMI'll send you my address13:50
quiquell|roverEmilienM: git review -x ... something else ?13:50
EmilienMpython-tripleoclient git:(master) git checkout -b bug/1793264 origin/stable/rocky13:51
EmilienMpython-tripleoclient git:(bug/1793264) git fetch https://git.openstack.org/openstack/python-tripleoclient refs/changes/02/603802/1 && git cherry-pick -x FETCH_HEAD13:51
*** rascasoft has joined #tripleo13:51
EmilienMpython-tripleoclient git:(bug/1793264) git review13:51
EmilienMthat's all13:51
gfidentedtantsur oh I think what I am seeing in the conductor logs is the error message from nova13:51
gfidentedtantsur is the ramdisk trying to save the image in /tmp ?13:51
quiquell|roverEmilienM: not git review, crafting manual in the way13:52
dtantsurgfidente: ah, do you have https://github.com/openstack/tripleo-heat-templates/commit/42c118244a3b6dceb1a766be332c566cb0f6bdb9#diff-3a86bd3fe6563a2c93e44be96a8cdd4c ?13:54
dtantsurthis may be the cause (in that case, booo at people using so little RAM :)13:54
openstackgerritQuique Llorente proposed openstack/python-tripleoclient stable/queens: Add a guard to break if no connection  https://review.openstack.org/60380613:54
gfidentedtantsur so the ram should be good enough to contain a raw copy of the image right?13:54
gfidentedtantsur you know this is important for edge deployments because we were thinking to use 'direct' to avoid iscsi13:55
dtantsurgfidente: there is another option: to use whole disk images and stream raw images from conductor13:55
dtantsurbut the default image is a partition one, so yes13:55
dtantsurunfortunately, it's hard to get everything at once :)13:56
openstackgerritQuique Llorente proposed openstack/python-tripleoclient stable/queens: Add a guard to break if no connection  https://review.openstack.org/60380613:56
gfidentedtantsur sure yes13:56
gfidenteslagle ^^ good topic for the edge squad13:56
dtantsurgfidente: related: https://review.openstack.org/60379613:57
EmilienMslagle: hey, when I tried to run an overcloud deploy and it failed, ansible logs were available in /var/lib/mistral but when I ran again, new config-download directory wasn't created, so no logs to inspect after second falure13:57
*** rdopiera has joined #tripleo13:57
dtantsur++ to discussing this topic13:57
EmilienMI'm wondering if that's a bug or I missed a step before trying to redeploy13:57
quiquell|roverLet's try to merge this first https://review.openstack.org/#/c/603804/ to ensure rocky promotions13:59
quiquell|roverWe can later merge master and queens13:59
*** janki has joined #tripleo13:59
*** hjensas has quit IRC14:00
quiquell|rovermschuppert: Did it work ?14:00
mschuppertquiquell|rover: not started yet. still in the queue14:01
quiquell|rovermschuppert: ack14:01
*** errantekarmico has joined #tripleo14:03
*** shyamb has quit IRC14:03
*** janki has quit IRC14:04
*** ykarel is now known as ykarel|away14:04
*** ade_lee has joined #tripleo14:05
pvcdtantsur14:05
pvchow can i set a lvm to be a root device for overcloud14:05
pvc?14:05
*** jtomasek has quit IRC14:08
dtantsurpvc: oh, that's a can of warms (/me is afk for some time sorry)14:08
dtantsurpvc: ask yolanda on #openstack-ironic14:09
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256014:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)14:10
*** ooolpbot has quit IRC14:10
*** errantekarmico has quit IRC14:10
fultonjcschwede: if you're happy w/ https://review.openstack.org/#/c/582811 would you mind +1'ing it?14:11
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: provider: Add vexxhost  https://review.openstack.org/59643214:12
fultonjjaosorior: do you hav a sec to talk about https://review.openstack.org/#/c/597221 ?14:12
cschwedefultonj: done, thx for the clarification!14:12
*** Chaserjim has joined #tripleo14:13
fultonjthanks cschwede14:13
slagleEmilienM: no idea, need context. maybe the second failure didn't get as far as the config-download step14:13
jaosoriorfultonj: sure14:13
EmilienMslagle: I'll try to reproduce. It reached config download, as I saw ansible steps running14:13
jaosoriorfultonj: if that's the name used in queens, and it was kept since that's what was done before, then that's fine by me. it was merely a suggestion.14:14
fultonjjaosorior: ok, great14:14
fultonjjaosorior: are you ok with +2'ing it?14:14
jaosoriorfultonj: look again14:14
fultonjthanks jaosorior14:15
jaosoriornp14:15
*** errantekarmico has joined #tripleo14:16
fultonjjaosorior: not sure if you have cycles for the tht patch on the same topic which depends on the tripleo-common patch14:17
fultonjanyone want to review https://review.openstack.org/#/c/582811 ?14:17
fultonjhoping to get it into final rocky release14:17
fultonjso we can scale up ceph monitors on day 114:18
jaosoriorfultonj: about to have a meeting :/ remind me in an hour or so if you haven't gotten reviewers yet.14:18
fultonjthanks jaosorior14:18
*** mcornea has joined #tripleo14:24
*** salmankhan1 has joined #tripleo14:24
*** salmankhan has quit IRC14:26
*** salmankhan1 is now known as salmankhan14:26
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: Add a script to compare reviews CI artifacts  https://review.openstack.org/58847514:29
*** Chaserjim has quit IRC14:32
*** Chaserjim has joined #tripleo14:33
*** salmankhan has quit IRC14:34
*** salmankhan has joined #tripleo14:34
*** salmankhan1 has joined #tripleo14:35
*** gouthamr_ is now known as gouthamr14:35
*** udesale has quit IRC14:37
*** hamdyk has quit IRC14:38
*** Chaserjim has quit IRC14:38
*** salmankhan has quit IRC14:38
*** salmankhan1 is now known as salmankhan14:38
*** ksambor has quit IRC14:40
*** thrash is now known as thrash|biab14:40
*** quiquell|rover is now known as quique|rover|off14:43
*** dprince has quit IRC14:43
*** jtomasek has joined #tripleo14:44
*** salmankhan has quit IRC14:44
*** errantekarmico has quit IRC14:46
*** salmankhan has joined #tripleo14:50
*** errantekarmico has joined #tripleo14:51
*** pvc has quit IRC14:55
*** Chaserjim has joined #tripleo14:56
*** nodoz has quit IRC15:01
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-quickstart master: ceilometer: Don't use deprecated metric  https://review.openstack.org/60359015:03
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Run online migrations via "external-upgrade run"  https://review.openstack.org/60382615:04
*** Chaserjim has quit IRC15:05
*** ykarel|away is now known as ykarel15:07
*** dxiri has joined #tripleo15:07
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Conditionally mount the TLS-related certs on the haproxy container  https://review.openstack.org/60382715:08
jaosoriorEmilienM: ^^15:08
EmilienMjaosorior: ++ thx15:08
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256015:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)15:10
*** ooolpbot has quit IRC15:10
*** dxiri has quit IRC15:12
*** hjensas has joined #tripleo15:16
*** udesale has joined #tripleo15:17
*** salmankhan1 has joined #tripleo15:26
*** salmankhan1 has quit IRC15:27
*** salmankhan has quit IRC15:28
*** dtantsur is now known as dtantsur|brb15:28
*** dprince has joined #tripleo15:31
*** abishop has quit IRC15:32
*** chkumar|ruck is now known as chkumar|off15:34
*** dxiri has joined #tripleo15:36
jaosoriorfultonj: is https://review.openstack.org/#/c/582811/ tested in CI?15:41
*** fhubik|brb has quit IRC15:41
fultonjjaosorior: yes15:41
fultonjscenario001 and sceario00415:41
*** panda has quit IRC15:42
fultonjlogs from scenario001 [1] indicate that the playbook was able to back up the tarball to swift [1]15:42
fultonj[1] http://logs.openstack.org/11/582811/25/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/277c0a0/logs/undercloud/var/lib/mistral/config-download-latest/ansible.log.txt.gz#_2018-09-07_18_36_00_12315:42
*** iranzo has quit IRC15:43
*** dtantsur|brb is now known as dtantsur15:46
*** mhenkel_ has quit IRC15:46
*** rdopiera has quit IRC15:46
openstackgerritHonza Pokorny proposed openstack/tempest-tripleo-ui master: Add basic project structure  https://review.openstack.org/57573015:46
*** pcaruana has joined #tripleo15:47
openstackgerritSam Betts proposed openstack/tripleo-heat-templates master: Add helper resource for converting string to map  https://review.openstack.org/60071315:48
*** moguimar has quit IRC15:49
*** bdodd has joined #tripleo15:50
*** jfrancoa has quit IRC15:54
*** rajinir has joined #tripleo15:55
*** ykarel is now known as ykarel|away15:56
*** kopecmartin has quit IRC15:56
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60327516:00
*** salmankhan has joined #tripleo16:01
*** ykarel|away has quit IRC16:01
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements stable/queens: Update test-requirements.txt  https://review.openstack.org/60383716:02
*** yprokule has quit IRC16:03
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements stable/queens: import zuul job settings from project-config  https://review.openstack.org/59783816:03
*** abishop has joined #tripleo16:04
*** florianf is now known as florianf|afk16:05
*** pvc has joined #tripleo16:05
pvc anyone here experience in enabling a command in ipmi?16:05
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements stable/pike: Update test-requirements.txt  https://review.openstack.org/60383916:06
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements stable/pike: import zuul job settings from project-config  https://review.openstack.org/59782016:06
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements stable/ocata: Update test-requirements.txt  https://review.openstack.org/60384016:08
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements stable/ocata: import zuul job settings from project-config  https://review.openstack.org/59780516:08
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256016:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)16:10
*** ooolpbot has quit IRC16:10
*** agopi has quit IRC16:12
mwhahahaslagle: is https://bugs.launchpad.net/tripleo/+bug/1792343 a blocker for rocky GA?16:20
openstackLaunchpad bug 1792343 in tripleo "[tripleo] rocky baremetal deployment fails with jq: error: Could not open file /var/lib/heat-config/deployed/<id>.notify.json" [Critical,Triaged]16:20
mwhahahaor rc216:20
slaglemwhahaha: i don't think so, as it doesn't reproduce. quique|rover|off weshay_pto rascasoft would know for sure.16:23
*** dtantsur is now known as dtantsur|afk16:23
mwhahahak16:24
slagledo we have any ovb jobs passing on rocky?16:24
mwhahahai don't know if we have rocky ovb jobs16:25
*** aufi has quit IRC16:25
slaglesome are passing on master. i'm not aware of any config-download differences between rocky and master that might cause that issue16:26
mwhahahapromotions have some passing ovb jobs16:27
mwhahahalegacy-periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-rocky seems ok16:27
chkumar|offmwhahaha: slagle fs01 was failing earlier inRDO CI16:27
*** pcaruana has quit IRC16:27
slaglechkumar|off: with this issue?16:28
chkumar|offbut last night we do some clean up on rdo cloud side we are not seeing this issue16:28
mwhahahalast fs001 periodoic passed16:28
*** udesale has quit IRC16:28
mwhahahaEmilienM: ok so i think we're ok to go for rc216:28
chkumar|offslagle: rascasoft is aactually facingthis issue16:28
EmilienMmwhahaha: ok I'll propose today16:29
EmilienMalso stable tags16:29
*** phuongnh has joined #tripleo16:29
slaglechkumar|off: ok, if he could update the trello card with those details, it would be great. and i'll take a look at the environment16:30
slaglechkumar|off: b/c the latest update on the trello card is that it doesn't reproduce16:30
slaglechkumar|off: https://trello.com/c/BgBpry6v16:30
chkumar|offEmilienM: is there a way to get josn output from  openstack overcloud status command?16:30
chkumar|offslagle: I will poke in the morning , let them know16:31
*** jpich has quit IRC16:32
*** holser__ has quit IRC16:36
*** ykarel has joined #tripleo16:36
*** panda has joined #tripleo16:38
slaglechkumar|off: there's no json output for the status command. file a bug if you need it16:40
chkumar|offslagle: sure will do that16:40
*** ramishra has quit IRC16:42
toskyslagle: uh, is the -W with +2 an oversight here? https://review.openstack.org/#/c/594980/16:45
*** phuongnh has quit IRC16:47
slagletosky: yea sorry16:50
toskynp :)16:50
*** agopi has joined #tripleo16:50
*** derekh has quit IRC16:51
*** rbrady has quit IRC16:51
*** rbrady has joined #tripleo16:52
*** gbarros has quit IRC16:53
*** salmankhan has quit IRC16:57
*** sri_ has left #tripleo16:58
EmilienMquique|rover|off: why did you abandon https://review.openstack.org/#/c/603641/ ??17:02
openstackgerritJohn Fulton proposed openstack/tripleo-docs master: Update standalone documentation to include steps for Ceph  https://review.openstack.org/60271017:05
*** radeks has joined #tripleo17:07
*** dsneddon has quit IRC17:08
*** dsneddon has joined #tripleo17:08
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: break out release config by distro type  https://review.openstack.org/60238717:09
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154017:09
*** trown is now known as trown|lunch17:09
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256017:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)17:10
*** errantekarmico has quit IRC17:11
openstackgerritSorin Sbarnea proposed openstack-infra/tripleo-ci master: fedora28 standalone job definition  https://review.openstack.org/59537417:13
*** vinaykns has joined #tripleo17:16
*** radeks has quit IRC17:18
*** amoralej is now known as amoralej|off17:18
*** jpena is now known as jpena|off17:30
*** Chaserjim has joined #tripleo17:35
*** thrash|biab is now known as thrash17:36
openstackgerritMerged openstack/tripleo-quickstart master: use fs config to assign standalone role, tests  https://review.openstack.org/57283017:38
*** vinaykns has quit IRC17:40
*** mcornea has quit IRC17:40
*** Chaserjim has quit IRC17:40
*** mcornea has joined #tripleo17:41
openstackgerritMerged openstack/tripleo-puppet-elements master: Add python3 heat agent for Fedora  https://review.openstack.org/59649817:41
openstackgerritMerged openstack/os-collect-config stable/pike: import zuul job settings from project-config  https://review.openstack.org/59781117:41
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: break out release config by distro type  https://review.openstack.org/60238717:47
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154017:47
openstackgerritMerged openstack/tripleo-quickstart master: tripleo-ci/master: add virt7-container-common-candidate  https://review.openstack.org/60297317:49
*** bogdando has quit IRC17:54
*** errantekarmico has joined #tripleo17:59
*** gbarros has joined #tripleo18:00
*** gbarros has quit IRC18:01
*** morazi has quit IRC18:04
*** trown|lunch is now known as trown18:08
openstackgerritAlex Schultz proposed openstack/instack-undercloud master: Include missing config classes  https://review.openstack.org/60385918:09
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256018:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)18:10
*** ooolpbot has quit IRC18:10
trownanyone able to review https://review.openstack.org/60043818:13
*** gbarros has joined #tripleo18:16
*** ykarel has quit IRC18:26
*** gbarros has quit IRC18:36
trownthanks weshay_pto18:36
openstackgerritAlex Schultz proposed openstack/puppet-tripleo master: Include missing config classes  https://review.openstack.org/60386718:45
EmilienMthe gate is almost 16h backed up18:46
EmilienMshould I purge it?18:46
mwhahahano18:46
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Tag step plays  https://review.openstack.org/59907218:46
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Remove "when failed" from debug task names  https://review.openstack.org/59822118:46
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Tag tasks in in common tasks  https://review.openstack.org/60325018:46
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Convert with_dict tasks to use loop and be less chatty  https://review.openstack.org/60325118:46
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: puppet host  https://review.openstack.org/60325218:46
mwhahahayou leave the gate alone18:46
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet  https://review.openstack.org/60325318:46
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks  https://review.openstack.org/60325418:46
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs  https://review.openstack.org/60325518:46
EmilienMfine for me !18:46
mwhahahathings are merging, it's just realllllly slow18:47
mwhahahaand the occasional timeout caused by container downloads is not helping18:49
*** vkmc is now known as vkmc|afk18:49
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/rocky: Release 9.4.0 (Rocky RC2)  https://review.openstack.org/60387118:55
*** gbarros has joined #tripleo19:04
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: L3 routed networks - data + env (1/3)  https://review.openstack.org/58218019:05
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: L3 routed networks - network.j2 (2/3)  https://review.openstack.org/58218119:05
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: L3 routed networks - subnet fixed_ips (3/3)  https://review.openstack.org/58230119:05
*** lblanchard has joined #tripleo19:08
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256019:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)19:10
mwhahahawe meed to fix the basic install in tripleo-docs, it doesn't mention any of the container stuff19:14
mwhahahaprobably need to replace some of it with the items from the containers deployment/overcloud19:14
*** assassin has quit IRC19:16
*** morazi has joined #tripleo19:18
*** Chaserjim has joined #tripleo19:18
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/queens: Prepare 8.3.6 (Queens) release  https://review.openstack.org/60387919:18
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo stable/pike: Prepare 7.4.16 (Pike) release  https://review.openstack.org/60388019:19
EmilienMmwhahaha: I guess I can give up ocata19:19
mwhahahaprobably19:19
mwhahahaso got a new one, DockerException: Error while fetching server API version: ('Connection aborted.', error(13, 'Permission denied')) when trying to inspect the rsyslog-base container19:20
mwhahahastevebaker: you around yet?19:20
EmilienMI had it once19:20
EmilienMcan you paste full trace plz?19:20
EmilienMalso where?19:20
mwhahahaagainst tripleo master19:21
mwhahahafollowing the docs19:21
EmilienMI saw it in CI a few time ago19:21
EmilienMlet me logstash it again19:21
mwhahahahttp://paste.openstack.org/show/730382/19:21
EmilienMk19:22
mwhahaharoles_data.yaml = Standalone.yaml19:22
mwhahahaoh so it's trying to inspect my registry version19:23
EmilienMlet's see if I get something from CI : build_name: *tripleo-ci* AND build_status: FAILURE AND message: "Error while fetching server API version"19:23
*** akrivoka has quit IRC19:24
EmilienMgot nothing in CI logs19:26
EmilienMI swear I saw it but maybe more than 30 days ago19:26
mwhahahaso yea i don't think the tripleo container iamge prepare is working right19:26
EmilienMI know log retention isn't far19:26
*** errantekarmico has quit IRC19:26
mwhahahafigured it out19:28
mwhahahawell kinda19:28
mwhahahai didn't logout/login after installing19:28
mwhahahaso i didn't have the docker group19:28
mwhahahaso when it goes to push to the registry, permission denied19:29
openstackgerritAakarsh proposed openstack/tripleo-quickstart master: Adding browbeat_workload file to fs053  https://review.openstack.org/60388419:34
EmilienMmwhahaha: it looks like something we need to document at least19:41
mwhahahahttps://bugs.launchpad.net/tripleo/+bug/179337519:41
openstackLaunchpad bug 1793375 in tripleo "attempting the tripleo container image prepare workflow without logging out after install results in an error" [Medium,Triaged]19:41
mwhahahaprobably should put a note in the deploy success message19:42
*** cmyster has quit IRC19:42
*** cmyster has joined #tripleo19:42
*** salmankhan has joined #tripleo19:51
*** rbrady has quit IRC19:59
*** abishop has quit IRC20:00
*** errantekarmico has joined #tripleo20:02
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256020:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)20:10
stevebakermwhahaha: hey20:16
*** errantekarmico has quit IRC20:16
mwhahahastevebaker: hi2u see ttps://bugs.launchpad.net/tripleo/+bug/179337520:16
mwhahahaalso with an h20:16
stevebakermwhahaha: I'd like to document using sudo, since that is in our podman using future anyway20:17
mwhahahastevebaker: yea that would also work20:18
stevebakermwhahaha: should I add that to this change or can you review it now? https://review.openstack.org/#/c/599462/20:19
mwhahahastevebaker: i'll review that now20:20
mwhahahastevebaker: feel free to propose a sudo version to resolve that bug20:20
stevebakerok20:20
*** paramite|clone has quit IRC20:24
*** jtomasek has quit IRC20:29
openstackgerritMerged openstack/tripleo-docs master: Document container prepare workflow  https://review.openstack.org/59946220:30
*** lblanchard has quit IRC20:35
*** Chaserjim has quit IRC20:39
*** ansmith has quit IRC20:45
*** Chaserjim has joined #tripleo20:47
openstackgerritVictor Pickard proposed openstack/tripleo-heat-templates master: Change log directory for OpenDaylight  https://review.openstack.org/60390720:48
*** gbarros has quit IRC20:53
*** raildo has quit IRC20:54
fultonjtoday's oooq master-tripleo-ci defaults to trying to downloading the neutron-server-opendaylight container though it tries to install the non-ODL version (centos-binary-neutron-server) http://paste.openstack.org/show/730384/21:00
openstackgerritDamien Ciabrini proposed openstack/tripleo-heat-templates master: WIP Do not overwrite password file during docker-puppet  https://review.openstack.org/60296921:01
*** Chaserjim has quit IRC21:01
fultonjhow might I easily add centos-binary-neutron-server to my local registry?21:01
*** trown is now known as trown|outtypewww21:04
*** Chaserjim has joined #tripleo21:05
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256021:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)21:10
silehtEmilienM, can you re review https://review.openstack.org/#/c/603621/ ?21:12
*** Chaserjim has quit IRC21:13
*** errantekarmico has joined #tripleo21:15
openstackgerritSteve Baker proposed openstack/tripleo-docs master: Make every reference to the prepare command use sudo  https://review.openstack.org/60391421:16
openstackgerritGoutham Pacha Ravi proposed openstack/tripleo-quickstart master: Disable manila tests on ocata and pike  https://review.openstack.org/60391521:17
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tripleo-quickstart master: ceilometer: Don't use deprecated metric  https://review.openstack.org/60359021:20
fultonjstevebaker: ^ ?21:20
* fultonj was going to try passing a param to override it...21:21
fultonjDockerNeutronServerImage: docker.io/tripleomaster/centos-binary-neutron-server:35414701c176a6288fc2ad141dad0f73624dcb94_4352748521:21
fultonje.g. ^21:22
stevebakerfultonj: how are you doing your image preperation?21:23
openstackgerritMerged openstack/tripleo-docs master: Document --override-ansible-cfg  https://review.openstack.org/58412521:24
fultonjstevebaker: quickstart came with a containers-prepare-parameter.yaml21:24
fultonjhttp://paste.openstack.org/show/730385/21:24
fultonjand the default deploy command included it21:24
fultonjquickstart used to provide a script to prep the containers21:25
stevebakerfultonj: yes, prepare is done during deploy now21:25
fultonjright21:25
fultonjso  overcloud-prep-containers.sh is gone21:25
fultonjwhich is fine, i'm just trying to get used to the new way21:26
fultonjthat and oooq seems to have set up a deployment confused about which neturon-server to use21:26
stevebakerfultonj: this is the logic which should be doing the right thing, but apparently isn't https://github.com/openstack/tripleo-common/blob/master/tripleo_common/image/kolla_builder.py#L235-L23921:26
fultonjso now that it's done during deploy and i can see that it's trying to access contianer i don't have...21:27
fultonjhttp://paste.openstack.org/show/730384/21:27
*** agopi has quit IRC21:28
fultonjstevebaker: can i drop in an override hard coding DockerNeutronServerImage ?21:28
fultonjstevebaker: so it looks at the services in the composable role...21:29
stevebakerfultonj: rather than that, you could try hardcoding neutron_driver:odl in your container_image_prepare_defaults.yaml. See the neutron_driver comment in this change https://review.openstack.org/#/c/602104/3/container-images/container_image_prepare_defaults.yaml21:29
openstackgerritGoutham Pacha Ravi proposed openstack/tripleo-quickstart master: DNM: Debug scenario004 cleanup  https://review.openstack.org/60055621:30
stevebakerfultonj: just to clarify, you want the odl images in this case?21:30
fultonjstevebaker: actually i don't want ODL21:30
stevebakerfultonj: huh, so the image parameter is set correctly, but the prepare image copying is copying the odl image to the undercloud instead of the vanilla one? interesting21:32
openstackgerritGoutham Pacha Ravi proposed openstack/tripleo-heat-templates stable/queens: DNM: Debugging scenario004  https://review.openstack.org/60064221:32
*** radeks has joined #tripleo21:33
fultonjso i could try neutron_driver:ovn21:33
fultonjor null acutally21:34
fultonji want the default simple network21:34
stevebakerfultonj: it will be null by default, so I don't think setting it will help21:35
*** bdodd has quit IRC21:35
fultonjso the question is, why is neutron-server missing?21:36
fultonjhttp://paste.openstack.org/show/730384/21:36
fultonjstevebaker: sorry i need to go21:37
fultonjthanks for your help i'll keep poking at it21:37
fultonji have a better idea what's going on now21:37
stevebakerfultonj: is there a CI job showing this failure?21:37
fultonjno21:37
fultonji basically did my own quickstart install21:38
fultonjwith defaults21:38
* fultonj needs to step out for now21:38
fultonji'll let you know if i find anything interesting21:38
stevebakerfultonj: ok, look at the logs for this call, it may show what is being tranferred https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/container-image-prepare.j2.yaml#L10121:38
*** Chaserjim has joined #tripleo21:44
*** dsneddon has quit IRC21:44
*** dsneddon has joined #tripleo21:45
*** dprince has quit IRC21:53
*** salmankhan has quit IRC21:53
*** mcornea has quit IRC21:56
*** owalsh has joined #tripleo21:57
*** owalsh has quit IRC21:58
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256022:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)22:10
*** ooolpbot has quit IRC22:10
*** Chaserjim has quit IRC22:28
*** Chaserjim has joined #tripleo22:30
*** agopi has joined #tripleo22:30
*** Chaserjim has quit IRC22:34
*** Chaserjim has joined #tripleo22:38
openstackgerritHonza Pokorny proposed openstack/tempest-tripleo-ui master: Add basic project structure  https://review.openstack.org/57573022:39
*** mrunge_ has joined #tripleo22:44
*** mrunge has quit IRC22:44
*** rcernin has joined #tripleo22:44
*** Chaserjim has quit IRC22:47
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Allow enabling routed networks on undercloud update|upgrade  https://review.openstack.org/60136923:01
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add MTU to neutron networks and nic-config templates  https://review.openstack.org/59959323:04
*** tzumainn has quit IRC23:08
*** Chaserjim has joined #tripleo23:08
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256023:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)23:10
*** gfidente has quit IRC23:15
*** ansmith has joined #tripleo23:15
*** Chaserjim has quit IRC23:16
*** slaweq has quit IRC23:19
*** Chaserjim has joined #tripleo23:38
*** errantekarmico has quit IRC23:39
*** errantekarmico has joined #tripleo23:39
*** thrash is now known as thrash|g0ne23:41
*** radeks has quit IRC23:42
*** Chaserjim has quit IRC23:46
*** Chaserjim has joined #tripleo23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!