| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 00:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 00:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 00:10 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Add ctlplane interface routes https://review.openstack.org/604943 | 00:12 |
| *** sai_p has quit IRC | 00:24 | |
| *** hjensas has quit IRC | 00:47 | |
| *** tzumainn has quit IRC | 00:47 | |
| *** hjensas has joined #tripleo | 00:55 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 01:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 01:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 01:10 |
| *** pmannidi has joined #tripleo | 01:13 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet https://review.openstack.org/603253 | 01:15 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks https://review.openstack.org/603254 | 01:15 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs https://review.openstack.org/603255 | 01:15 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Name unnamed tasks https://review.openstack.org/604460 | 01:15 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Fix tasks in check mode https://review.openstack.org/604461 | 01:15 |
| *** phuongnh has joined #tripleo | 01:17 | |
| *** mschuppert has quit IRC | 01:23 | |
| *** mrsoul has quit IRC | 01:24 | |
| *** mmethot has quit IRC | 01:41 | |
| *** dciabrin has quit IRC | 01:41 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: Pass NeutronMechanismDrivers parameter to prepare https://review.openstack.org/604952 | 01:55 |
| *** weshay has joined #tripleo | 01:57 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Set prepare neutron_driver from NeutronMechanismDrivers https://review.openstack.org/604953 | 02:02 |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 02:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 02:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 02:10 |
| *** psachin has joined #tripleo | 02:44 | |
| *** skramaja has joined #tripleo | 02:52 | |
| *** jrist has joined #tripleo | 03:07 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 03:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 03:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 03:10 |
| *** ramishra has joined #tripleo | 03:18 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: ensure tripleo-validations are turned off upstream https://review.openstack.org/604878 | 03:21 |
| weshay | jrist, hey man | 03:27 |
| weshay | jrist, validations were never meant for upstream | 03:27 |
| weshay | only 3rd party | 03:27 |
| weshay | they take TOOOO long | 03:27 |
| weshay | jrist, see the bug? | 03:28 |
| *** ykarel|away has joined #tripleo | 03:39 | |
| *** ykarel|away is now known as ykarel | 03:49 | |
| *** udesale has joined #tripleo | 03:57 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 04:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 04:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 04:10 |
| *** rcernin has quit IRC | 04:24 | |
| *** ratailor has joined #tripleo | 04:25 | |
| *** mcarden has joined #tripleo | 04:36 | |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-quickstart master: Re-enable manila tests in scenario004/pike https://review.openstack.org/600556 | 04:36 |
| *** Petersingh has joined #tripleo | 04:38 | |
| *** rcernin has joined #tripleo | 04:38 | |
| *** Petersingh is now known as Petersingh|afk | 04:39 | |
| openstackgerrit | Steve Baker proposed openstack/paunch master: Implement cap_add, cap_drop https://review.openstack.org/604659 | 04:44 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Pin openshift-ansible to last known good version https://review.openstack.org/600438 | 04:49 |
| *** agopi_ has joined #tripleo | 04:50 | |
| *** ykarel has quit IRC | 04:51 | |
| *** agopi|training has quit IRC | 04:53 | |
| *** sanjayu_ has quit IRC | 04:53 | |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/puppet-tripleo master: Add httpd and tls support to manila-api https://review.openstack.org/592188 | 04:54 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Set prepare neutron_driver from NeutronMechanismDrivers https://review.openstack.org/604953 | 05:01 |
| *** ykarel has joined #tripleo | 05:07 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 05:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 05:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 05:10 |
| *** shyamb has joined #tripleo | 05:14 | |
| *** rcernin_ has joined #tripleo | 05:17 | |
| *** rcernin has quit IRC | 05:19 | |
| *** Petersingh|afk is now known as Petersingh | 05:20 | |
| *** jaosorior has joined #tripleo | 05:20 | |
| Tengu | hello there :) | 05:24 |
| *** shyamb has quit IRC | 05:26 | |
| jaosorior | good morning! | 05:27 |
| *** shyamb has joined #tripleo | 05:31 | |
| *** ksambor has joined #tripleo | 05:34 | |
| *** pcaruana has joined #tripleo | 05:41 | |
| *** chkumar|off is now known as chkumar|ruck | 05:42 | |
| chkumar|ruck | Good morning | 05:42 |
| *** quique|rover|off is now known as quiquell|rover | 05:45 | |
| quiquell|rover | chkumar|ruck: o/ | 05:45 |
| chkumar|ruck | quiquell|rover: \o | 05:45 |
| quiquell|rover | jaosorior, Tengu: o/ | 05:45 |
| chkumar|ruck | to diable validation, weshay has patch up https://review.openstack.org/#/c/604878/3 zuul jobs has just started running | 05:46 |
| quiquell|rover | chkumar|ruck: I tries to do it yesterday changing featuresets, but I think it was not correct | 05:52 |
| quiquell|rover | chkumar|ruck: Comparing it give us around 15 minutes | 05:53 |
| quiquell|rover | chkumar|ruck: At least it can reduce timeouts if they are not needed | 05:53 |
| quiquell|rover | jrist: You there ? | 05:54 |
| *** iranzo has joined #tripleo | 05:54 | |
| jaosorior | quiquell|rover: 15 mins is not a lot. Is that the main suspect of the timeouts right now? Thought at some point it was the containerized undercloud that was causing the timeouts. | 05:56 |
| quiquell|rover | jaosorior: next step is tu ensure that all timeout jobs have long validation times (I am not sure of that) | 05:57 |
| quiquell|rover | jaosorior: It's clear that sometimes we have even 23 minutes of validation, and that's no good | 05:57 |
| jaosorior | wow | 05:59 |
| jaosorior | that is a long time for validations | 05:59 |
| * Tengu is not really sure pushing the validations on swift and run them through mistral is a good thing | 06:02 | |
| Tengu | that makes two possibly slow services that can eat time like we eat icecream in summer. | 06:02 |
| *** jfrancoa has joined #tripleo | 06:02 | |
| jaosorior | Tengu: well, for swift it all depends on how it's all pushed. if you tar it all up and push it as one blob, it's not too bad | 06:02 |
| jaosorior | which is what dprince and imain were proposing for the plans as well | 06:03 |
| Tengu | jaosorior: not sure how it's done, but I really suspect it's a file-per-file. | 06:03 |
| jaosorior | Tengu: I suspect that too | 06:03 |
| Tengu | and that would make sense since we can run only part of the validations through dedicated workflow in mistral | 06:03 |
| Tengu | so yeah, that can take a hell of a time fetching all the validations, one after the other. | 06:04 |
| Tengu | with the consistency checks done by swift clients and auth and blah. | 06:04 |
| jaosorior | Tengu: talking about validations, has florianf added the feedback from the validations spec that came on the PTG to the spec? | 06:04 |
| Tengu | jaosorior: hmm nope | 06:05 |
| Tengu | jaosorior: but I had a quick talk with him yesterday | 06:05 |
| *** shyamb has quit IRC | 06:05 | |
| Tengu | jaosorior: my team won't be able to produce the framework nor the in-flight for the next release, but florianf and his team seem to be wanting to do it - so I'll just kind of manage and draw the link between his team and mine. | 06:06 |
| Tengu | so we finish the spec, and I hand over the baby while still supervising it in some way ^^' | 06:06 |
| jaosorior | Tengu: that sounds good | 06:06 |
| Tengu | that makes me a project manager. | 06:07 |
| Tengu | bam. | 06:07 |
| jaosorior | lol | 06:07 |
| *** Petersingh is now known as Petersingh|afk | 06:08 | |
| *** shyamb has joined #tripleo | 06:09 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 06:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 06:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 06:10 |
| *** bandini has joined #tripleo | 06:10 | |
| *** shyamb has quit IRC | 06:12 | |
| *** shyamb has joined #tripleo | 06:13 | |
| quiquell|rover | Tengu: btw weshay sais in its disable validations patch that they were not designed for upstream ci | 06:14 |
| quiquell|rover | Tengu: What does he means ? Is he talking about time constraints of upstream ci ? | 06:14 |
| Tengu | quiquell|rover: the CI doesn't provide the minimal resources | 06:14 |
| Tengu | i.e. when I activated the first version of the disk space validation, we didn't get the wanted 60G, more something like 20. | 06:15 |
| Tengu | quiquell|rover: so I've changed the validation playbook in order to check by path instead, with some smart discovery ability. and now it works. | 06:15 |
| quiquell|rover | Tengu: then validation is like a WIP for upstream ci ? | 06:16 |
| Tengu | no idea for the overcloud validations in fact :/. I was working on the undercloud preflights | 06:16 |
| quiquell|rover | Tengu: ack, what do you think about merging https://review.openstack.org/#/c/604878 to give us some air ? | 06:17 |
| Tengu | well, if we don't have validations running, how do we ensure all is deployed as expected? | 06:18 |
| *** slaweq has joined #tripleo | 06:18 | |
| Tengu | we should instead push timers in order to understand what's taking so long. Validations are a must imho. | 06:19 |
| Tengu | especially in a CI | 06:19 |
| quiquell|rover | Tengu: pushing timers is not very infra friendly, all the proviers are affected, we cannot do that | 06:20 |
| Tengu | :( | 06:20 |
| quiquell|rover | Tengu: It's not the case with RDO, we are the provider | 06:20 |
| Tengu | timer are life when it comes to timeouts. | 06:20 |
| quiquell|rover | Tengu: Let's reproduce a timeout without validations this can help | 06:21 |
| Tengu | quiquell|rover: what about pushing timers into the code itself? | 06:21 |
| quiquell|rover | Tengu: timers at a zuul, nothing you can do at the code | 06:21 |
| Tengu | so that we don't change anything at the provider level, but in the code, using things like statd (I think there's that service somewhere) | 06:21 |
| Tengu | well, a playbook task is finished, notify statd, a new one starts, notify statd, and so on | 06:22 |
| quiquell|rover | Tengu: We already has ARA for that | 06:22 |
| quiquell|rover | kind of | 06:22 |
| Tengu | and what does ARA show about that? | 06:22 |
| chkumar|ruck | quiquell|rover: Tengu What about moving validation to a seperate fs if they want to test it? | 06:23 |
| quiquell|rover | Tengu: the playbook and taks time, but is not that simple | 06:23 |
| Tengu | any way to pinpoint what's triggering the timeout? Of course if we drop the 25-minutes validation run, that won't timeout anymore with some chance. | 06:23 |
| chkumar|ruck | ykarel: https://bugs.launchpad.net/tripleo/+bug/1794228 please have a look at this | 06:23 |
| openstack | Launchpad bug 1794228 in tripleo "[queens][pike] No package uwsgi-plugin-python2 available during barbican api container build" [Critical,Triaged] | 06:23 |
| quiquell|rover | Tengu: we are creating tool to comapre ARA between builds in a span of moths or so. | 06:23 |
| quiquell|rover | chkumar|ruck: Good idea, we can make it not voting in case it procues timeouts | 06:24 |
| Tengu | quiquell|rover: added a comment on weshay change proposal - we'll see. I'm not all "yeah-drop-it-now", but if it's the right thing and we can still ensure we actually test the full deploy.... | 06:24 |
| Tengu | still, commit message might lack some precision. | 06:25 |
| quiquell|rover | Tengu: what do you think about chkumar|ruck idea ? we can have non voting job with validations like we do with WIPs | 06:25 |
| ykarel | chkumar|ruck, https://review.openstack.org/#/q/If9b43dbf3fab6a76f47a0e4fb9213c5efa655c0d caused it | 06:25 |
| Tengu | quiquell|rover: why not, but how would you validate the deploy then? | 06:26 |
| Tengu | having two ways of validating stuff doesn't sound right to me. | 06:26 |
| ykarel | chkumar|ruck, to fix https://github.com/openstack/tripleo-common/blob/stable/pike/container-images/tripleo_kolla_template_overrides.j2#L32 should include python2- | 06:26 |
| chkumar|ruck | ykarel: thanks send ina patch | 06:26 |
| Tengu | CI should rely on the tripleo-validations - and if zuul runs some other validation, they should be embeded in the tripleo-validations | 06:26 |
| *** mschuppert has joined #tripleo | 06:27 | |
| Tengu | at least, that would help converging, provide a solid base for validations. | 06:27 |
| *** jtomasek has joined #tripleo | 06:27 | |
| Tengu | brb, breakfast. | 06:27 |
| quiquell|rover | Tengu: brb = breakfast right back | 06:28 |
| *** yprokule has joined #tripleo | 06:31 | |
| *** Petersingh|afk is now known as Petersingh | 06:31 | |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-common stable/queens: Rename uwsgi-plugin-python to uwsgi-plugin-python2 in template overrides https://review.openstack.org/604976 | 06:32 |
| chkumar|ruck | ykarel: quiquell|rover ^^ | 06:32 |
| quiquell|rover | chkumar|ruck: it fixes queens ? | 06:32 |
| chkumar|ruck | ykarel: yes, | 06:33 |
| chkumar|ruck | for pike i am backporting it | 06:33 |
| quiquell|rover | mschuppert: ^ fix for queens, let's see if we have a promotion tomorrow | 06:34 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-common stable/pike: Rename uwsgi-plugin-python to uwsgi-plugin-python2 in template overrides https://review.openstack.org/604977 | 06:35 |
| chkumar|ruck | quiquell|rover: ^6 for pike | 06:35 |
| chkumar|ruck | quiquell|rover: I am looking at rest of the master/rocky periodic failed jobs | 06:35 |
| quiquell|rover | chkumar|ruck: ok, I am still digging in timeout, | 06:36 |
| chkumar|ruck | we need some help from rdo infra also http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/36d37e4/job-output.txt.gz periodic jobs are alsotiming out | 06:37 |
| quiquell|rover | chkumar|ruck: no kidding... | 06:37 |
| chkumar|ruck | quiquell|rover: the job got terminated after deploy the overcloud no idea | 06:38 |
| mschuppert | quiquell|rover: \o/ cool! | 06:38 |
| Tengu | quiquell|rover: :D | 06:39 |
| ykarel | chkumar|ruck, ack | 06:44 |
| openstackgerrit | Thomas Herve proposed openstack/tripleo-quickstart-extras master: Remove duplicate entry in params.yaml https://review.openstack.org/604979 | 06:46 |
| ykarel | chkumar|ruck, i think for stable/branch only patches there is some convention followed | 06:46 |
| ykarel | so good to follow that | 06:46 |
| ykarel | chkumar|ruck, similar to https://review.openstack.org/#/c/599897/ | 06:47 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-common stable/queens: [Queens Only]Rename uwsgi-plugin-python to uwsgi-plugin-python2 for barbican https://review.openstack.org/604976 | 06:48 |
| chkumar|ruck | ykarel: ^^ you mean like this? | 06:48 |
| ykarel | chkumar|ruck, looks good | 06:49 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-common stable/pike: [Pike Only] Rename uwsgi-plugin-python to uwsgi-plugin-python2 in barbican https://review.openstack.org/604977 | 06:49 |
| ykarel | chkumar|ruck, the link u shared for 002-master-upload timing out at overcloud deploy, not after | 06:49 |
| ykarel | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/36d37e4/job-output.txt.gz#_2018-09-24_14_28_59_539855 | 06:50 |
| *** shyamb has quit IRC | 06:50 | |
| chkumar|ruck | ykarel: yes, at overcloud deploy | 06:50 |
| ykarel | hmm i was just correcting, the job got terminated after deploy the overcloud no idea | 06:51 |
| therve | Does anyone else have an issue with reproducer? | 06:52 |
| therve | It seems to be broken because there are 2 centos latest images | 06:52 |
| Tengu | therve: have one, due to weird rights on qcow2 images.... | 06:53 |
| ramishra | therve: yeah it seems broken , I made it to work with --extra-params image_id=<one of the ids> | 06:53 |
| Tengu | well, not weird, just wrong. | 06:53 |
| therve | ramishra: Thanks :) | 06:54 |
| chkumar|ruck | ykarel: quiquell|rover I am not sure this will affect us https://lists.centos.org/pipermail/centos-devel/2018-September/016949.html | 06:54 |
| chkumar|ruck | or not | 06:54 |
| chkumar|ruck | renaming the mirrors to mirrorlist | 06:54 |
| *** Petersingh is now known as Petersingh|afk | 06:55 | |
| openstackgerrit | Thomas Herve proposed openstack/tripleo-quickstart-extras master: Fix multinode image list https://review.openstack.org/604982 | 06:56 |
| therve | ramishra ^^ | 06:56 |
| quiquell|rover | chkumar|ruck: Let's not that in the etherpad | 06:57 |
| quiquell|rover | s/not/note/ | 06:57 |
| *** Petersingh|afk is now known as Petersingh | 06:58 | |
| *** slaweq has quit IRC | 06:58 | |
| *** rdopiera has joined #tripleo | 06:58 | |
| ykarel | chkumar|ruck, looks like that's a feature added but not switched yet, but good to note it | 06:59 |
| ykarel | and seems if that's done it might affect | 06:59 |
| ykarel | as we translate/rename urls to nodepool ones | 07:00 |
| *** shyamb has joined #tripleo | 07:04 | |
| *** sanjayu_ has joined #tripleo | 07:05 | |
| *** rcernin_ has quit IRC | 07:05 | |
| chkumar|ruck | quiquell|rover: I am still seeing conneciton close issue here http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/bed073f/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz | 07:09 |
| *** gfidente has joined #tripleo | 07:09 | |
| chkumar|ruck | above job contains your tripleoclient fixes | 07:09 |
| chkumar|ruck | 2018-09-25 01:59:59.777 ERROR /var/log/containers/ironic/ironic-conductor.log: 8 ERROR ironic.conductor.manager [req-e85bc31d-1934-4e65-9370-cc51ef20f422 6a75c2e401854467b9487ff3149d87cc 6937b21b157d482ea2b03e2364db7031 - default default] Failed to prepare node 42f062f6-de21-4a9a-9599-fbd39b007b68 for cleaning: Failed to set node power state to power on.: PowerStateFailure: Failed to set node power state to | 07:09 |
| chkumar|ruck | power on. | 07:09 |
| quiquell|rover | That's different | 07:09 |
| quiquell|rover | Let me check after I am back | 07:10 |
| *** quiquell|rover is now known as quique|rover|brb | 07:10 | |
| chkumar|ruck | quiquell|rover: sure | 07:10 |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 07:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 07:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 07:10 |
| *** holser_ has joined #tripleo | 07:10 | |
| *** Petersingh_ has joined #tripleo | 07:10 | |
| *** Petersingh has quit IRC | 07:13 | |
| openstackgerrit | Thomas Herve proposed openstack/tripleo-quickstart-extras master: Fix script typo https://review.openstack.org/604987 | 07:16 |
| *** gvrangan has joined #tripleo | 07:22 | |
| *** ykarel is now known as ykarel|lunch | 07:25 | |
| bandini | jaosorior: is the channel subject current? aka no workflows yet? | 07:27 |
| jaosorior | bandini: still dealing with timeouts. | 07:28 |
| bandini | ack | 07:28 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: Don't change user during ping check. https://review.openstack.org/604990 | 07:31 |
| *** shyamb has quit IRC | 07:32 | |
| *** amoralej|off is now known as amoralej | 07:33 | |
| *** psachin has quit IRC | 07:41 | |
| *** Petersingh_ is now known as Petersingh|lunch | 07:42 | |
| *** quique|rover|brb is now known as quiquell|rover | 07:42 | |
| *** shyamb has joined #tripleo | 07:45 | |
| *** dciabrin has joined #tripleo | 07:46 | |
| *** agopi__ has joined #tripleo | 07:46 | |
| *** Petersingh_ has joined #tripleo | 07:47 | |
| *** agopi_ has quit IRC | 07:49 | |
| *** aufi has joined #tripleo | 07:49 | |
| *** Petersingh|lunch has quit IRC | 07:50 | |
| *** psachin has joined #tripleo | 07:50 | |
| quiquell|rover | chkumar|ruck: we don't have timeouts at noop | 07:50 |
| *** shardy has joined #tripleo | 07:52 | |
| *** jpena|off is now known as jpena | 07:53 | |
| *** tosky has joined #tripleo | 07:54 | |
| *** waleedm has joined #tripleo | 07:54 | |
| quiquell|rover | chkumar|ruck: This is new ? http://logs.openstack.org/45/560445/146/check/tripleo-ci-centos-7-containers-multinode/41340e0/logs/undercloud/home/zuul/undercloud_install.log.txt.gz | 07:56 |
| *** shyamb has quit IRC | 07:56 | |
| *** bugzy_ has joined #tripleo | 07:58 | |
| *** bogdando has joined #tripleo | 07:58 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates stable/queens: [Queens/Pike] Run online data migrations on converge, allow disabling them https://review.openstack.org/604342 | 07:58 |
| chkumar|ruck | quiquell|rover: https://bugs.launchpad.net/tripleo/+bug/1794251 | 08:01 |
| openstack | Launchpad bug 1794251 in tripleo "[master] undercloud reinstall failed with invalid selinux context: [Errno 95] Operation not supported" [Critical,Triaged] | 08:01 |
| chkumar|ruck | quiquell|rover: let me check | 08:01 |
| chkumar|ruck | quiquell|rover: fs020 is failing from 22nd with same issue | 08:01 |
| *** bugzy has quit IRC | 08:01 | |
| quiquell|rover | chkumar|ruck: noop upstream change is enqueued in one of the jobs, that's why it does not appear at dashboard-ci | 08:02 |
| chkumar|ruck | quiquell|rover: above container multinode is new | 08:03 |
| chkumar|ruck | I am not sure it is legit to report ntp issue is seen again | 08:08 |
| chkumar|ruck | http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-rocky/c3dcd08/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 08:08 |
| chkumar|ruck | atal: [overcloud-controller-0]: FAILED! => {"changed": true, "cmd": ["ntpdate", "-u", "pool.ntp.org"], "delta": "0:00:56.067516", "end": "2018-09-25 03:48:23.470952", "msg": "non-zero return code", "rc": 1, "start": "2018-09-25 03:47:27.403436", "stderr": "Error resolving pool.ntp.org: Name or service not known (-2)\n25 Sep 03:48:23 ntpdate[15052]: Can't find host pool.ntp.org: Name or service not known | 08:08 |
| chkumar|ruck | (-2)\n25 Sep 03:48:23 ntpdate[15052]: no servers can be used, exiting", "stderr_lines": ["Error resolving pool.ntp.org: Name or service not known (-2)", "25 Sep 03:48:23 ntpdate[15052]: Can't find host pool.ntp.org: Name or service not known (-2)", "25 Sep 03:48:23 ntpdate[15052]: no servers can be used, exiting"], "stdout": "", "stdout_Overcloud configuration failed. | 08:08 |
| chkumar|ruck | 2018-09-25 03:48:25 | lines": []} | 08:08 |
| chkumar|ruck | ykarel|lunch: Tengu ^^ | 08:08 |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 08:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 08:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 08:10 |
| Tengu | broken dns | 08:11 |
| Tengu | once more | 08:11 |
| Tengu | chkumar|ruck: overcloud uses undercloud as dns - at least non-controller nodes. Not sure about controllers for that specific part. | 08:12 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates stable/pike: [Queens/Pike] Run online data migrations on converge, allow disabling them https://review.openstack.org/604997 | 08:13 |
| Tengu | (and undercloud is used as gateway for non-controller nodes) | 08:13 |
| *** cylopez has joined #tripleo | 08:14 | |
| *** fhubik has joined #tripleo | 08:15 | |
| *** AJaeger_ has joined #tripleo | 08:20 | |
| AJaeger_ | tripleo cores, two changes ready for OpenStack wide python3-first goal: please review https://review.openstack.org/#/c/597798/ and one cleanup: https://review.openstack.org/#/c/604531/ | 08:21 |
| *** paramite has joined #tripleo | 08:21 | |
| AJaeger_ | and one more: https://review.openstack.org/#/c/597801/ | 08:21 |
| AJaeger_ | EmilienM, jaosorior , mwhahaha, could you help, please? ^ | 08:21 |
| *** jcoufal has joined #tripleo | 08:22 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-specs master: [WIP] Bare metal provisioning without Nova and Glance https://review.openstack.org/604761 | 08:23 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-quickstart master: DO-NOT-MERGE Enable the centos testing repo for ceph https://review.openstack.org/501810 | 08:24 |
| shardy | Tengu: we don't use the undercloud as a DNS AFAIK, but we do set up optional masquerade/forwarding rules to enable access to the external network when no network-isolation (and explicit external gateway) | 08:25 |
| shardy | Tengu, chkumar|ruck: see https://bugs.launchpad.net/tripleo/+bug/1794038 which may be related - some changes may have broken those rules in the switch to containerized undercloud, depending on your specific undercloud.conf | 08:26 |
| openstack | Launchpad bug 1794038 in tripleo "undercloud masquerade_networks is now silently ignored" [High,In progress] - Assigned to Harald Jensås (harald-jensas) | 08:26 |
| shardy | we didn't see that until recently when the pool.ntp.org default was added, because previously we didn't require any external access by default | 08:27 |
| *** bdodd has quit IRC | 08:30 | |
| Tengu | shardy: hmm by default the undercloud is used as DNS - maybe not on the CI though | 08:31 |
| Tengu | but of course we can update the DNS in the neutron config for the overcloud network | 08:31 |
| shardy | Tengu: I didn't think we set any --dns-nameserver by default, but perhaps I'm mistaken | 08:32 |
| *** bdodd has joined #tripleo | 08:32 | |
| shardy | we use static hosts files everywhere inside the cluster, to avoid any runtime dependency on the undercloud | 08:32 |
| Tengu | ok | 08:33 |
| *** pmannidi has quit IRC | 08:35 | |
| *** ykarel|lunch is now known as ykarel | 08:35 | |
| chkumar|ruck | shardy: Tengu thanks , Checking | 08:36 |
| ykarel | chkumar|ruck, yes i remember this, i shared the fs021 issue with weshay few days back, it's broken since long | 08:36 |
| *** akrivoka has joined #tripleo | 08:37 | |
| chkumar|ruck | ykarel: do we need a bug or any bug already there? | 08:37 |
| ykarel | chkumar|ruck, don't know about that | 08:37 |
| *** dtantsur|afk is now known as dtantsur | 08:38 | |
| *** derekh has joined #tripleo | 08:38 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Cleanup zuul.yaml https://review.openstack.org/604531 | 08:40 |
| chkumar|ruck | ykarel: quiquell|rover on fs021 master it is failing with different issue 2018-09-24 12:04:02.802817 | tripleo-ovb-centos-7 | cat: /etc/nodepool/primary_node_private: No such file or directory | 08:42 |
| chkumar|ruck | http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-master/c9b7d92/job-output.txt.gz | 08:43 |
| chkumar|ruck | two failures related to it on master | 08:43 |
| *** pvc has joined #tripleo | 08:45 | |
| pvc | hi may i know what is the default password of overcloud? | 08:45 |
| pvc | and username? | 08:45 |
| shardy | pvc: there is no default password, but we create a "heat-admin" user by default, which uses the "default" ssh keypair in nova | 08:46 |
| shardy | if you need a default user/password you can add one to the image via virt-customize, but it's against most security/compliance rules so we don't do it by default | 08:47 |
| rdo | pvc: I was about to say the above.. customise the image first before you upload it, then you always know what it is.. when we build the overcloud image it's scrambled :) | 08:48 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Remove unused bootstrap-config.yaml https://review.openstack.org/605009 | 08:49 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: WIP Add hiera keys for SERVICE_bootstrap_nodeid https://review.openstack.org/605010 | 08:49 |
| *** slaweq has joined #tripleo | 08:49 | |
| pvc | there is a problem that my overcloud deploy stacked in some process | 08:50 |
| pvc | kindly see this logs http://paste.openstack.org/show/730688/ | 08:51 |
| pvc | rdo and shardy | 08:51 |
| *** Petersingh_ is now known as Petersingh | 08:51 | |
| pvc | and if i check the nova list, all servers are on active state. I have 2 servers one for controller and one for compute. I can only ssh the one server | 08:52 |
| shardy | pvc: you need to get access to the nodes to check the logs - I suspect your network configuration is bad if it's hanging here, as NetworkDeployment for both roles is IN_PROGRESS | 08:53 |
| *** shyamb has joined #tripleo | 08:53 | |
| shardy | that should only take a few seconds to complete, unless it takes the network down and it never comes back up ;) | 08:53 |
| pvc | thats the problem shardy i cannot login to the instance via IPMI because i dont know what username and password should i input | 08:54 |
| pvc | but its hostname is already change in overcloud.controller plus overcloud.compute | 08:54 |
| shardy | pvc: yup, rdo and I just explained how to fix that | 08:54 |
| rdo | pvc: it might not be best practice, but it can help you troubleshoot... you could reboot and ' | 08:54 |
| rdo | 'root' the VM so you can see what os-net-config it wrote | 08:54 |
| rdo | and you can figure out why that's not able to be applied | 08:54 |
| pvc | if i type the root user they want me to input a password | 08:55 |
| rdo | that's right.. but I'm saying you can, for troubleshooting, reboot the hypervisor (not 'VM' as I said above) and reset the root password | 08:56 |
| shardy | http://libguestfs.org/virt-customize.1.html | 08:56 |
| pvc | but the VM itself is my hypervisor? | 08:57 |
| pvc | im using a baremetal overcloud | 08:57 |
| rdo | pvc: yes, it was my mistake above to use the term 'VM | 08:57 |
| rdo | I should have said physical host | 08:57 |
| shardy | I don't think the image will enable single user mode to do that, but worth trying - AFAIK you'll need to modify the image or use cloud-init to set a root password | 08:58 |
| rdo | given that your deployment hasn't succeeded, I would delete the overcloud (openstack stack delete overcloud -y), modify the image with virt-customize, re-upload the disk image, update the boot images for your nodes, and re-attempt the deployment... at least if it fails again you can login | 08:58 |
| shardy | +1 | 08:58 |
| pvc | this image shardy and rdo overcloud-full? | 08:59 |
| rdo | pvc: yes, overcloud-full qcow2 (latest) | 08:59 |
| shardy | yes you modify the overcloud-full.qcow the upload it again | 08:59 |
| pvc | i see so i need to delete it first in glance | 08:59 |
| rdo | no need to delete in glance | 09:00 |
| rdo | when you upload it, it will replace it | 09:00 |
| rdo | let' | 09:00 |
| shardy | no you can just do openstack overcloud image upload | 09:00 |
| pvc | i see thank you wait i will just edit first the image | 09:00 |
| pvc | Btw this is the command i run "openstack overcloud deploy" | 09:00 |
| pvc | am i missing sometehing? | 09:00 |
| chkumar|ruck | quiquell|rover: jaosorior: marios https://review.openstack.org/#/c/604878/ diable validation what to do with that? merge or wait till was comes | 09:00 |
| shardy | bandini: Hey I've been thinking more about the bootstrap issue, sent http://lists.openstack.org/pipermail/openstack-dev/2018-September/135045.html with some ideas, feedback welcome! | 09:00 |
| chkumar|ruck | *wes comes online | 09:01 |
| rdo | pvc: well, that will be the final step | 09:01 |
| *** panda|off is now known as panda | 09:01 | |
| AJaeger_ | jaosorior: thanks for reviewing - I see now you don't have separate stable team, so let me ask here everybody... | 09:01 |
| bandini | shardy: ack, thanks. I will take a look! | 09:01 |
| shardy | bandini: basically I started generating per-service bootstrap IDs, which works, but I'm wondering if we can simplify things by just using the short_bootstrap_hostname and comparing with the ansible hostname instead | 09:01 |
| quiquell|rover | chkumar|ruck: Looks very controversial, I still don't understand this thing about validations not designed for upstream ci | 09:02 |
| chkumar|ruck | quiquell|rover: same here | 09:02 |
| shardy | AJaeger_: ack looking | 09:02 |
| AJaeger_ | tripleo cores, could you review https://review.openstack.org/#/c/597801/ and https://review.openstack.org/597798 to import zuul changes to move forward with python3-first goal, please? | 09:02 |
| chkumar|ruck | quiquell|rover: is rocky got promoted today? | 09:02 |
| AJaeger_ | thanks, shardy! | 09:03 |
| AJaeger_ | shardy: those two above are the ones open... | 09:03 |
| pvc | virt-customize -a overcloud.full.qco2 --root-password <password> | 09:03 |
| quiquell|rover | chkumar|ruck: yep | 09:03 |
| rdo | pvc: personally, I do... 'mkdir -p ~/images; cd ~/images; tar xvf /usr/share/rhosp-director-images/ironic-python-agent.tar -C .; tar xvf /usr/share/rhosp-director-images/overcloud-full.tar -C .; virt-customize -a ~/images/overcloud-full.qcow2 --root-password password:redhat' | 09:04 |
| chkumar|ruck | quiquell|rover: need some modification in dashboard rocky is missing Tripleo CI promotion alerts | 09:04 |
| AJaeger_ | The two final changes are https://review.openstack.org/597831 and https://review.openstack.org/597848 - both paunch stable updates that fail. Who can look into those to finish python3-first? | 09:04 |
| chkumar|ruck | quiquell|rover: great, finally something is green today | 09:04 |
| shardy | pvc: yeah check the docs I linked, you need a selector to --root-password not just the password | 09:04 |
| quiquell|rover | chkumar|ruck: move the slider, is down | 09:04 |
| shardy | as it supports e.g file:somefile or password:plaintextpassword | 09:05 |
| pvc | thank you so much rdo and shardy | 09:05 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Use local ansible connection for libvirt repro https://review.openstack.org/605013 | 09:06 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-specs master: [WIP] Bare metal provisioning without Nova and Glance https://review.openstack.org/604761 | 09:07 |
| rdo | pvc: no worries.. once that's done you'll need to do an 'openstack overcloud image upload --image-path ~/images --update-existing' | 09:07 |
| *** sshnaidm has joined #tripleo | 09:07 | |
| rdo | shardy: do you know if the image upload command updates the baremetal boot image id's? | 09:08 |
| shardy | rdo: yes I believe it does | 09:08 |
| shardy | whereas direct upload to glance wont | 09:09 |
| rdo | great.. historically we had to run an 'openstack baremetal configure boot' | 09:09 |
| shardy | Yeah, IIRC that was combined into a workflow but I don't recall exactly when | 09:09 |
| rdo | great thx | 09:09 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 09:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 09:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 09:10 |
| *** owalsh_ is now known as owalsh | 09:11 | |
| marios | chkumar|ruck: yeah i don't think we should merge that with all the -1 currently on there seems there needs to be some discussino first | 09:13 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: WIP: Add necessary bits for N-1->N standalone upgrade. https://review.openstack.org/604736 | 09:13 |
| marios | chkumar|ruck: i think tripleo weekly this afternoon might be a place for some of that | 09:13 |
| marios | quiquell|rover: & jaosorior (they were on chkumar|ruck original ping) ^ | 09:15 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: WIP: New workflow for standalone upgrade https://review.openstack.org/604706 | 09:15 |
| *** salmankhan has joined #tripleo | 09:15 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart master: Fix environment vars not working with commands https://review.openstack.org/603360 | 09:15 |
| quiquell|rover | marios: yep | 09:18 |
| *** janki has joined #tripleo | 09:20 | |
| *** AJaeger_ has left #tripleo | 09:20 | |
| chkumar|ruck | marios: yes | 09:22 |
| quiquell|rover | chkumar|ruck, ykarel: To browser ansible tasks http://dashboard-ci.tripleo.org/d/poOr-d0mk/ansible-exploration?orgId=1 | 09:23 |
| quiquell|rover | unsing the influxdb_filter you can filter | 09:24 |
| chkumar|ruck | quiquell|rover: great, looks awesome | 09:24 |
| chkumar|ruck | quiquell|rover++ | 09:24 |
| quiquell|rover | chkumar|ruck: we can filter fi task run on timedout jobs | 09:25 |
| quiquell|rover | So we have timedout jobs that have good times at validations | 09:26 |
| ykarel | quiquell|rover, nice | 09:26 |
| *** jmelis has joined #tripleo | 09:26 | |
| *** dxiri has quit IRC | 09:29 | |
| *** sshnaidm is now known as sshnaidm|afk | 09:29 | |
| chkumar|ruck | quiquell|rover: you can ignore fs016 master failure it was just a timedout in scenario test, i re-runned the job on test project, it is working fine | 09:30 |
| quiquell|rover | chkumar|ruck: you mean in promotions ? | 09:31 |
| chkumar|ruck | quiquell|rover: yes in promotion | 09:31 |
| *** nawar has joined #tripleo | 09:33 | |
| nawar | hi | 09:33 |
| nawar | i need some help plz | 09:33 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart master: Sudo virt-resize for libvirt reproducer https://review.openstack.org/605021 | 09:35 |
| rdo | nawar: what's the question? | 09:35 |
| openstackgerrit | Sergii Golovatiuk proposed openstack/python-tripleoclient master: Add 2h timeout when waiting for websocket messages on package_update https://review.openstack.org/604325 | 09:36 |
| *** sshnaidm|afk is now known as sshnaidm|off | 09:37 | |
| *** shyamb has quit IRC | 09:38 | |
| nawar | ok i installed openstack queens with tripleo using basic configuration provided by the documentation: 2 nodes controller and compute everything went great | 09:40 |
| nawar | now I'm trying to scale up and adding another compute node, I provided the new node and when I execute the command openstack overcloud deploy --compute-scale 2 --templates | 09:41 |
| nawar | I'm getting this message Started Mistral Workflow tripleo.validations.v1.check_pre_deployment_validations. Execution ID: ea50c159-67da-4474-a7ca-e2c1acb331c5 | 09:41 |
| nawar | Waiting for messages on queue 'tripleo' with no timeout. | 09:41 |
| nawar | 'passwords' | 09:41 |
| nawar | and nothing is happening | 09:41 |
| nawar | I checked all logs and I couldn't find any errors | 09:42 |
| *** sshnaidm|off has quit IRC | 09:42 | |
| pvc | rdo question if i deploy my baremetal as an undercloud it's network will not be lost right? because i cannot ping right now so i cant install any package | 09:42 |
| rdo | nawar: where are you executing your command from? | 09:43 |
| nawar | stack user with stackrc env | 09:43 |
| rdo | pvc: Can you rephrase your question? I'm not sure I understand it | 09:43 |
| *** jfrancoa has quit IRC | 09:43 | |
| rdo | nawar: from which directory? | 09:44 |
| jaosorior | nawar: might want to run the same command, but add the --debug option to get relevant logs. | 09:44 |
| nawar | /home/stack | 09:44 |
| pvc | It seems that i cant ping google.com from my undercloud node. I want to install the libguestfs-tools for the virt-customize. I have an dns on my resolv.conf. | 09:44 |
| rdo | pvc: well, you're going to have to fix your network routing :) can you ping your nameserver? | 09:45 |
| rdo | nawar: as above, get the debug output | 09:45 |
| nawar | ok | 09:45 |
| pvc | wait | 09:45 |
| pvc | i cant also :( | 09:45 |
| pvc | Destination Host Unreachable | 09:46 |
| *** jfrancoa has joined #tripleo | 09:46 | |
| rdo | nawar: I have seen this before though, just can't remember why and what I did to fix it :) | 09:46 |
| rdo | pvc: yup, that's your problem.. you'll need to fix your network routing to ensure that your undercloud machine also has a route out to the internet | 09:46 |
| nawar | ok i got an exception return plan_env['passwords'] | 09:48 |
| nawar | KeyError: 'passwords' | 09:48 |
| jaosorior | nawar: did you delete the plan by any chance before running that command? | 09:48 |
| nawar | i don't think so | 09:48 |
| nawar | plan still there I run openstack overcloud plan list | 09:50 |
| quiquell|rover | chkumar|ruck, jaosorior: I think we are suffering from the variability of undercloud install and validations time | 09:52 |
| quiquell|rover | undercloud install at success jobs have variabibility between 1 hours and 23 minutes... | 09:53 |
| chkumar|ruck | quiquell|rover: do we need to inspect overcloud deploy also? | 09:54 |
| chkumar|ruck | may be something comes up there also | 09:54 |
| quiquell|rover | chkumar|ruck: Feels constant | 09:54 |
| quiquell|rover | jaosorior, chkumar|ruck: And validations have ranges between 3 and 18 minutes | 09:54 |
| rdo | nawar: have you tried to download the plan? | 09:54 |
| quiquell|rover | This is all at success jobs | 09:54 |
| rdo | nawar: to check for passwords/ | 09:54 |
| quiquell|rover | so a mix of slow validations + slow undercloud install give as a pretty timeout | 09:55 |
| nawar | nope since the first deployment went well | 09:56 |
| chkumar|ruck | slow undercloiud install -> tripleo image prepare part is taking time or anything else? | 09:56 |
| chkumar|ruck | undercloud-setup : Run the package installation script this step I think is constant 6-8 mins | 09:57 |
| quiquell|rover | chkumar|ruck: this is 22 minutes undercloud install http://logs.openstack.org/42/600642/3/check/tripleo-ci-centos-7-containers-multinode/b89c8aa/ | 09:59 |
| quiquell|rover | chkumar|ruck: this is 1 hour http://logs.openstack.org/78/604878/3/check/tripleo-ci-centos-7-containers-multinode/7c42a0c/ | 09:59 |
| quiquell|rover | chkumar|ruck: all of the success | 09:59 |
| quiquell|rover | chkumar|ruck: Why do we have so much difference there ? | 09:59 |
| quiquell|rover | overcloud deploy is similar | 10:01 |
| ykarel | chkumar|ruck, that should be differnet relases | 10:02 |
| quiquell|rover | Humm... we have to filter by branch | 10:02 |
| quiquell|rover | ykarel: yep... missed that part | 10:02 |
| *** moshele has joined #tripleo | 10:02 | |
| quiquell|rover | ykarel: 22 minutes in queens... | 10:02 |
| quiquell|rover | ykarel: why it takes so much now in master? containers ? | 10:02 |
| ykarel | yes non containerized undercloud | 10:02 |
| quiquell|rover | ykarel: Containers are not suppose to be faster :-) | 10:02 |
| ykarel | container pull/updates are heavy | 10:03 |
| chkumar|ruck | we moved to containers so that installation time should decrease | 10:03 |
| ykarel | only if containers are ready | 10:04 |
| ykarel | and there are lot other reasons for switch | 10:04 |
| quiquell|rover | upgrading should be easier too | 10:05 |
| quiquell|rover | But it's not very CI friendly :-/ | 10:06 |
| *** janki has quit IRC | 10:07 | |
| nawar | I downloaded the plan and checked it nothing weird, in default_passwords.yaml there is a outputs resource call passwords | 10:07 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 10:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 10:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 10:10 |
| openstackgerrit | Thomas Herve proposed openstack/tripleo-common master: Optimize Heat stack check when processing roles https://review.openstack.org/605029 | 10:11 |
| *** sanjayu__ has joined #tripleo | 10:15 | |
| nawar | any advice | 10:18 |
| *** sanjayu_ has quit IRC | 10:18 | |
| *** numans has quit IRC | 10:18 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-docs master: Document standalone/UC update Heat stack specifics https://review.openstack.org/578799 | 10:21 |
| ykarel | nawar, u used same templates while scaling too | 10:22 |
| ykarel | iirc update should include all the templates passed at deployment time | 10:23 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-docs master: Document standalone/UC update Heat stack specifics https://review.openstack.org/578799 | 10:23 |
| *** cylopez has quit IRC | 10:25 | |
| *** Petersingh is now known as Petersingh|afk | 10:27 | |
| *** shyamb has joined #tripleo | 10:29 | |
| nawar | yes | 10:29 |
| nawar | this is my first time using tripleo so I tried the defaults e.g --templates | 10:30 |
| nawar | /usr/share/openstack-tripleo-heat-templates | 10:31 |
| *** dxiri has joined #tripleo | 10:31 | |
| *** phuongnh has quit IRC | 10:33 | |
| dtantsur | slagle: hi! could you please check my response on https://review.openstack.org/#/c/576471/ ? | 10:34 |
| chkumar|ruck | can someone look at undercloud reinstall issue https://bugs.launchpad.net/tripleo/+bug/1794251 | 10:35 |
| openstack | Launchpad bug 1794251 in tripleo "[master] undercloud reinstall failed with invalid selinux context: [Errno 95] Operation not supported" [Critical,Triaged] | 10:35 |
| openstackgerrit | Kamil Sambor proposed openstack/tripleo-heat-templates master: Add posibilities to set tunnel_csum in ovs agent https://review.openstack.org/605033 | 10:35 |
| bogdando | dtantsur: thanks for docs inputs! :) | 10:38 |
| dtantsur | you're welcome | 10:38 |
| dtantsur | reading tripleo-docs is my hobby :D | 10:38 |
| nawar | dtantsur is the tripleo-docs updated ?? | 10:40 |
| dtantsur | nawar: well, it is (best effort, of course) | 10:40 |
| bogdando | ofc it does. As much as we do update it | 10:41 |
| nawar | why I'm asking, I tried openstack tripleo commands and it doesn't exist ? | 10:41 |
| bogdando | nawar: looks like a bug then... | 10:42 |
| quiquell|rover | bogdando: Do you know why this can happend ? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/8ee317f/logs/undercloud/home/zuul/undercloud_reinstall.log.txt.gz#_2018-09-24_12_48_21 | 10:42 |
| quiquell|rover | bogdando: invalid selinux context | 10:42 |
| *** ratailor_ has joined #tripleo | 10:43 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Make every reference to the prepare command use sudo https://review.openstack.org/603914 | 10:43 |
| bogdando | quiquell|rover, Tengu: yes I do | 10:43 |
| bogdando | :) | 10:43 |
| quiquell|rover | bogdando: Also, can you share it with me :-), have some beer here | 10:43 |
| bogdando | quiquell|rover, Tengu: https://review.openstack.org/#/c/600532/ should be the root cause | 10:44 |
| *** ratailor has quit IRC | 10:44 | |
| chkumar|ruck | quiquell|rover: tripleoclient patch broke rpmbuild | 10:44 |
| chkumar|ruck | quiquell|rover: http://logs.openstack.org/04/603804/2/gate/tripleo-ci-centos-7-undercloud-oooq/1266255/logs/delorean_logs/43/76/4376f8379acd62ada80e4ecfa39ffa08bc34a472_dev/rpmbuild.log.txt.gz | 10:44 |
| *** rcernin_ has joined #tripleo | 10:44 | |
| chkumar|ruck | or may be broken earlier | 10:44 |
| chkumar|ruck | it is on rocky rdo | 10:44 |
| quiquell|rover | bogdando: Ahh root cause | 10:45 |
| bogdando | yeah | 10:45 |
| bogdando | almost a fix, just an opposite | 10:45 |
| bogdando | :) | 10:45 |
| quiquell|rover | bogdando: It's revertible ? | 10:46 |
| bogdando | quiquell|rover: I think it is | 10:46 |
| quiquell|rover | bogdando: let's try it | 10:46 |
| quiquell|rover | bogdando: thanks so much | 10:46 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Revert "Ensure some directories are created with the correct SELinux label" https://review.openstack.org/605037 | 10:47 |
| quiquell|rover | chkumar|ruck: ^ Maybe this fix it | 10:47 |
| chkumar|ruck | jaosorior: marios https://review.openstack.org/#/c/604976/ and https://review.openstack.org/604977 needed for pike and queens promotion | 10:50 |
| *** janki has joined #tripleo | 10:50 | |
| *** rcernin_ has quit IRC | 10:52 | |
| ykarel | chkumar|ruck, that rpmbuild.log is seen earlier also, it's intermittent and i think there is bug as well for using mock | 10:54 |
| *** dhill_ has quit IRC | 10:55 | |
| chkumar|ruck | ykarel: doing recheck then | 10:55 |
| *** florianf has joined #tripleo | 10:56 | |
| ykarel | chkumar|ruck, seems https://bugs.launchpad.net/tripleo/+bug/1789506 and is in master | 10:57 |
| openstack | Launchpad bug 1789506 in tripleo "tripleoclient package build fails trying to unlink a /tmp file during testing" [High,Fix released] - Assigned to Sergii Golovatiuk (sgolovatiuk) | 10:57 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Use rdo mirror for q,p,o buildlogs at promotions https://review.openstack.org/604324 | 10:57 |
| chkumar|ruck | ykarel: is this one cherry picked for rocky? | 10:58 |
| ykarel | chkumar|ruck, not checked | 10:58 |
| openstackgerrit | Chandan Kumar proposed openstack/python-tripleoclient stable/rocky: Properly mock unlink https://review.openstack.org/605038 | 10:58 |
| shyamb | Hi | 10:58 |
| chkumar|ruck | ykarel: ^^ | 10:58 |
| ykarel | chkumar|ruck, if the issues are not seen in master now, then good candidate for rocky, but good to check with mwhahaha | 10:58 |
| shyamb | For any third party containers, to support openstack HA, do we need to add new service as pacemaker resource + haproxy or just haproxy is enough? | 10:59 |
| shyamb | shardy: Tengu: jaosorior: Let me know your thoughts here | 11:00 |
| *** artom has quit IRC | 11:00 | |
| *** jcoufal has quit IRC | 11:01 | |
| nawar | so any suggestions ?? | 11:02 |
| *** Petersingh|afk is now known as Petersingh | 11:03 | |
| quiquell|rover | chkumar|ruck, ykarel: Aftewr filtering by branch, at timedout_job, uc, oc times are similar to success jobs | 11:04 |
| chkumar|ruck | quiquell|rover: do we flag or alert if oc > UC is more than 5-10 mins? | 11:05 |
| chkumar|ruck | *we want | 11:05 |
| quiquell|rover | chkumar|ruck: not yet, now we can with the change at dashboard-ci | 11:06 |
| *** janki has quit IRC | 11:07 | |
| chkumar|ruck | quiquell|rover: dsure | 11:07 |
| quiquell|rover | chkumar|ruck: humm wait... sometimes ARA is broken at timeouts | 11:08 |
| quiquell|rover | Look here http://logs.openstack.org/91/587491/4/check/tripleo-ci-centos-7-containers-multinode/f618803/job-output.txt.gz#_2018-09-25_09_05_36_843220 | 11:09 |
| chkumar|ruck | is it post failure or something? | 11:09 |
| quiquell|rover | 1:25 at deploying overcloud | 11:09 |
| Tengu | quiquell|rover: care to explain a bit what's the task? I didn't get any issue while deploying with the patch (and others) for both docker and podman engines. | 11:09 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 11:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 11:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 11:10 |
| quiquell|rover | Tengu: what patch ? | 11:10 |
| Tengu | quiquell|rover: https://bugs.launchpad.net/tripleo/+bug/1794251 | 11:10 |
| openstack | Launchpad bug 1794251 in tripleo "[master] undercloud reinstall failed with invalid selinux context: [Errno 95] Operation not supported" [Critical,In progress] - Assigned to Quique Llorente (quiquell) | 11:10 |
| quiquell|rover | Tengu: Have just revert a clue that bogdando has give us, could be totally wrong | 11:11 |
| Tengu | i.e. https://review.openstack.org/#/c/600532/ - it did pass the CI all the way, so i'm a bit surprised seeing it failing later on. | 11:11 |
| quiquell|rover | Tengu: we are just debugging, maybe it's too premature to add a Closes-Bug there | 11:11 |
| *** shyamb has quit IRC | 11:12 | |
| Tengu | quiquell|rover: care to explain a bit the "undercloud_reinstall" thingy? | 11:12 |
| quiquell|rover | Tengu: I don't see RDO fs020 running in the review | 11:12 |
| *** amoralej is now known as amoralej|lunch | 11:12 | |
| quiquell|rover | Tengu: It's to theck idempotency I think | 11:13 |
| quiquell|rover | Tengu: it's run again and it has to be a noop | 11:13 |
| quiquell|rover | Tengu: but looks like now it's not a noop | 11:13 |
| rdo | nawar: your error is because the CLI can't find the passwords in the plan when stored in Swift | 11:13 |
| Tengu | quiquell|rover: probably due to the "recurse" | 11:14 |
| *** fhubik is now known as fhubik|brb | 11:14 | |
| rdo | nawar: we could do with better debug logging there, but basically it's unable to return the passwords as there's a key error | 11:14 |
| rdo | i.e. no passwords | 11:14 |
| Tengu | that one MIGHT be dropped, have to test again a deploy with podman + non-recurse on that config-data directory. | 11:14 |
| Tengu | quiquell|rover: will update the LP | 11:14 |
| quiquell|rover | Tengu: we can prepare a RDO review to run exaclty the same job | 11:14 |
| Tengu | quiquell|rover: that would be great. | 11:14 |
| quiquell|rover | Tengu: cool, thanks man, all help is welcome | 11:14 |
| *** pcaruana has quit IRC | 11:15 | |
| quiquell|rover | Tengu: will leave in a few, let's do it at come back | 11:15 |
| Tengu | I'd like to understand before that one is reverted, it's basically the root of all the podman/selinux stuff I've worked for 4 weeks ^^' | 11:15 |
| *** udesale has quit IRC | 11:17 | |
| nawar | rdo where can i paste my debug? | 11:19 |
| *** panda is now known as panda|afk | 11:19 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Dropped "recurse" for idempotency https://review.openstack.org/605039 | 11:21 |
| rdo | nawar: pastebin or fpaste? | 11:21 |
| shardy | nawar: or http://paste.openstack.org/ | 11:25 |
| *** dtantsur is now known as dtantsur|bbl | 11:25 | |
| *** leanderthal has joined #tripleo | 11:26 | |
| jaosorior | chkumar|ruck: merging the patches | 11:27 |
| *** moshele has quit IRC | 11:27 | |
| Tengu | quiquell|rover: proposed another patch avoiding the revert - will test it locally as well. | 11:27 |
| jaosorior | chkumar|ruck: could probably ask infr to prioritize those. | 11:29 |
| quiquell|rover | Tengu: pass me te review | 11:31 |
| nawar | rdo plz check http://paste.openstack.org/show/730692/ | 11:32 |
| Tengu | quiquell|rover: https://review.openstack.org/605039 | 11:32 |
| Tengu | it's also linked in the LP ;) | 11:32 |
| *** moshele has joined #tripleo | 11:33 | |
| quiquell|rover | Tengu: there is nothing like a revert to find help :-P | 11:34 |
| quiquell|rover | Tengu: testing review https://review.rdoproject.org/r/16354 | 11:34 |
| Tengu | quiquell|rover: well, bogdando sent me a ping on another channel ;). I was busy with something else, didn't check in here. | 11:35 |
| quiquell|rover | Tengu: going to abandon the revert | 11:36 |
| Tengu | quiquell|rover: thanks :) | 11:36 |
| quiquell|rover | Tengu: bogdando has +2 the revert jo jo jo :-) | 11:36 |
| Tengu | quiquell|rover: well... of course ;) | 11:38 |
| quiquell|rover | Tengu: just kidding | 11:38 |
| *** jpena is now known as jpena|lunch | 11:38 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-common master: Add AllNodesConfig to dynamic inventory vars https://review.openstack.org/605046 | 11:39 |
| *** jfrancoa has quit IRC | 11:42 | |
| *** jfrancoa has joined #tripleo | 11:43 | |
| weshay | jaosorior, I need to speak with you immediately | 11:46 |
| jaosorior | weshay: sup | 11:46 |
| jaosorior | weshay: bluejeans? | 11:47 |
| weshay | jaosorior, sure | 11:47 |
| *** shyamb has joined #tripleo | 11:48 | |
| chkumar|ruck | jaosorior: yes, that would be great to handle the queue | 11:49 |
| *** quiquell|rover is now known as quique|rover|lch | 11:49 | |
| nawar | rdo: ping | 11:55 |
| Tengu | quique|rover|lch: apparently my patch is OK for the CI. | 11:59 |
| Tengu | quique|rover|lch: checking it with my whole podman+selinux process :). | 11:59 |
| *** dprince has joined #tripleo | 12:00 | |
| *** dhill_ has joined #tripleo | 12:01 | |
| *** rh-jelabarre has joined #tripleo | 12:05 | |
| *** psachin has quit IRC | 12:07 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 12:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 12:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 12:10 |
| jaosorior | chkumar|ruck: are you gonna ask the infra folks to prioritze the patches? or will should I? | 12:10 |
| chkumar|ruck | jaosorior: in a call | 12:10 |
| chkumar|ruck | feel free to go ahead | 12:11 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: WIP: Add necessary bits for N-1->N standalone upgrade. https://review.openstack.org/604736 | 12:12 |
| *** raildo has joined #tripleo | 12:12 | |
| *** Petersingh is now known as Petersingh|afk | 12:13 | |
| openstackgerrit | Honza Pokorny proposed openstack/tempest-tripleo-ui master: Add basic project structure https://review.openstack.org/575730 | 12:16 |
| *** Petersingh|afk is now known as Petersingh | 12:16 | |
| weshay | bogdando, Tengu can you both read through my comments on https://review.openstack.org/#/c/604878 | 12:23 |
| weshay | jrist, ^ | 12:23 |
| *** trown|outtypewww is now known as trown | 12:23 | |
| *** thrash|g0ne is now known as thrash | 12:24 | |
| *** ade_lee has quit IRC | 12:24 | |
| Tengu | weshay: thanks for the details, +1 after reading :) | 12:24 |
| jaosorior | Tengu: thanks | 12:24 |
| weshay | thanks | 12:25 |
| jaosorior | I'll wait for bogdando to check out the comment before +2ing | 12:25 |
| weshay | heh | 12:25 |
| jaosorior | after that, I'll purge the gate in order to get that patch to merge faster. | 12:25 |
| *** amoralej|lunch is now known as amoralej | 12:26 | |
| nawar | so rdo any good news | 12:27 |
| jaosorior | weshay, panda|afk what's this patch for https://review.openstack.org/#/c/602673/8 ? | 12:27 |
| Tengu | quique|rover|lch: reinstalling undercloud - the part that failed. Crossing fingers :) | 12:28 |
| weshay | jaosorior, that is to ensure we have the zuulv3 inventory dumped to file for debug | 12:28 |
| Tengu | apparently I still don't have any crash with the podman deploy I run in //. | 12:28 |
| Tengu | so that might be the right thing to do. | 12:28 |
| openstackgerrit | Deepika Khairnar proposed openstack/tripleo-heat-templates master: Add config files/templates to integrate nsx plugin with container https://review.openstack.org/605056 | 12:29 |
| *** shyamb has quit IRC | 12:31 | |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Use sync action get_deployment_failures https://review.openstack.org/605058 | 12:31 |
| chkumar|ruck | jaosorior: weshay we need this https://review.openstack.org/605038 in order to merge this https://review.openstack.org/603804 for rocky | 12:32 |
| chkumar|ruck | fix unit test in order to fix connection close issue | 12:32 |
| *** jcoufal has joined #tripleo | 12:32 | |
| *** agopi__ has quit IRC | 12:34 | |
| *** ratailor_ has quit IRC | 12:37 | |
| *** jpena|lunch is now known as jpena | 12:40 | |
| shardy | chem, jistr: Hey I'm looking at the use of bootstrap_nodeid in the upgrade_tasks, and I don't see how this will work: | 12:40 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/nova-api.yaml#L253 | 12:40 |
| shardy | any idea? | 12:40 |
| jistr | shardy: i'm replying to the list right now, gimme a sec :) | 12:41 |
| shardy | it seems like we'll compare a node uuid with ansible_hostname, which will always be false? | 12:41 |
| shardy | jistr: thanks! | 12:41 |
| * chem needs to work on its speed | 12:41 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer https://review.openstack.org/602492 | 12:41 |
| weshay | bogdando, where you at brotha :) | 12:43 |
| beagles | jaosorior: if you get a sec https://review.openstack.org/#/c/597038/ | 12:43 |
| beagles | jaosorior: btw haven't forgotten the metadata thing - just juggling. should finish the rest of the patches here shortly | 12:44 |
| jaosorior | beagles: the patch looks fine. I don't know where the TLS proxy is being started in t-h-t though | 12:45 |
| jaosorior | I looked at the t-h-t patch and it seems to be missing that. | 12:45 |
| openstackgerrit | Nicolas Hicher proposed openstack-infra/tripleo-ci master: provider: Add vexxhost https://review.openstack.org/596432 | 12:45 |
| bogdando | weshay: here | 12:45 |
| bogdando | sup | 12:45 |
| weshay | bogdando, so you know ci.. you think validations on multinode are worth the cost? | 12:46 |
| weshay | https://review.openstack.org/#/c/604878/ | 12:46 |
| weshay | bogdando, /me is suprised | 12:46 |
| weshay | bogdando, I'd rather catch that in 3rd party ovb | 12:46 |
| bogdando | weshay: well, do we have upstream coverage for validations otherwise? | 12:46 |
| weshay | where requirements can be met | 12:46 |
| bogdando | weshay: ok, that makes sense | 12:46 |
| weshay | bogdando, unit test.. the upstream resources will NEVER pass | 12:46 |
| bogdando | my -1 was not really strict :) | 12:46 |
| weshay | bogdando, heh | 12:47 |
| weshay | jaosorior, ^ | 12:47 |
| beagles | jaosorior: isn't that in docker/services/octavia-api.yaml already? | 12:47 |
| jaosorior | I'll check in a bit | 12:48 |
| jaosorior | purging the gate now | 12:48 |
| beagles | jaosorior: ack - do you have the launchpad bug for the undercloud metadata thing handy? | 12:48 |
| *** panda|afk is now known as panda | 12:49 | |
| *** ramishra has quit IRC | 12:50 | |
| weshay | jaosorior, http://dashboard-ci.tripleo.org/d/poOr-d0mk/ansible-exploration?orgId=1 | 12:50 |
| weshay | fyi | 12:50 |
| *** ramishra has joined #tripleo | 12:54 | |
| nawar | can I get your help guys | 12:55 |
| *** ssbarnea has quit IRC | 12:56 | |
| openstackgerrit | Honza Pokorny proposed openstack/tempest-tripleo-ui master: Add basic project structure https://review.openstack.org/575730 | 12:57 |
| *** lblanchard has joined #tripleo | 12:59 | |
| *** moguimar has quit IRC | 13:00 | |
| *** gvrangan has quit IRC | 13:03 | |
| *** agopi__ has joined #tripleo | 13:05 | |
| *** ssbarnea has joined #tripleo | 13:06 | |
| jistr | shardy: sent a reply | 13:06 |
| nawar | :'(:'( | 13:07 |
| *** numans_ has joined #tripleo | 13:07 | |
| *** quique|rover|lch is now known as quiquell|rover | 13:09 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 13:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 13:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 13:10 |
| *** lblanchard has quit IRC | 13:10 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-common master: Remove non-voting jobs from the gate https://review.openstack.org/603419 | 13:10 |
| openstackgerrit | Dan Macpherson proposed openstack/ansible-role-openstack-operations master: Adding Backup and Restore Operations https://review.openstack.org/604439 | 13:11 |
| quiquell|rover | Tengu: did it pass ? | 13:11 |
| *** nawar has left #tripleo | 13:12 | |
| *** skramaja has quit IRC | 13:12 | |
| openstackgerrit | Honza Pokorny proposed openstack/tempest-tripleo-ui master: Add basic project structure https://review.openstack.org/575730 | 13:13 |
| *** moguimar has joined #tripleo | 13:13 | |
| Tengu | quiquell|rover: still running apparently. | 13:15 |
| Tengu | quiquell|rover: so that's probably a good sign. | 13:16 |
| Tengu | quiquell|rover: had to restart the deploy on my side with podman, but I think it should be OK | 13:16 |
| *** vinaykns has joined #tripleo | 13:18 | |
| shardy | jistr: thanks, and argh - nodeid is interchangable with *node_name | 13:20 |
| shardy | I didn't realize that, but I'm not sure it's per role, we rely on the {{primary_role_name}} e.g Controller by default | 13:20 |
| *** zaneb has quit IRC | 13:21 | |
| *** nawar has joined #tripleo | 13:22 | |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.j2.yaml#L515 | 13:22 |
| jistr | shardy: hmm but when i run it on compute, the bootstrap_nodeid points to compute-0 (as shown in the e-mail) | 13:22 |
| shardy | oh wait you're right | 13:22 |
| *** nawar has left #tripleo | 13:22 | |
| shardy | jistr: yeah I just realized that, but the logic for e.g short_service_bootstrap_node considers all roles | 13:23 |
| *** zaneb has joined #tripleo | 13:23 | |
| shardy | because if a service spans more than one role, you still only want to bootstrap things once | 13:23 |
| jistr | shardy: yea, the (3) problem from e-mail. We should switch to short_service_bootstrap_node then | 13:23 |
| jistr | shardy: i'm curious what connection does this have to the BZ you mentioned though. If the only problem is that we run the bootstrap tasks on 2 roles, could the BZ be a race condition perhaps? | 13:25 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Add a lookup plugin for Mistral action executions https://review.openstack.org/540863 | 13:25 |
| shardy | jistr: Ok thanks, I think the issue is that we have another (different) definition of bootstrap_nodeid here: | 13:25 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/common/deploy-steps.j2#L283 | 13:26 |
| shardy | that is global, which means the docker bootstrap tasks will only ever run on overcloud-controller-0, regardless of the per role "id" | 13:26 |
| shardy | sigh | 13:26 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Port the network env validation to use parameters https://review.openstack.org/547518 | 13:26 |
| jistr | shardy: ah so the docker bootstrap tasks use *that* one. All makes sense then. | 13:27 |
| shardy | jistr: yeah, it's a bit of a mess - hence wondering if we could replace all of it with short_service_bootstrap_node etc | 13:27 |
| shardy | although then we can't use ansible conditionals, docker-puppet will need to filter the tasks based on the hostname | 13:27 |
| shardy | unless we jsonquery filter it or something | 13:28 |
| jistr | shardy: yea we probably should. Both of the current approaches are incorrect, and short_service_bootstrap_node would be the right thing to use for both, i think. | 13:28 |
| shardy | jistr: Ok thanks for the discussion, I'll see if I can get something working which does that | 13:29 |
| jistr | shardy: thanks :) | 13:29 |
| shardy | I'll reply with a summary to the list so folks can keep track :) | 13:29 |
| weshay | mwhahaha, quiquell|rover take a look at http://dashboard-ci.tripleo.org/d/poOr-d0mk/ansible-exploration?orgId=1&var-influxdb_filter=job_result%7C%3D%7CTIMED_OUT | 13:29 |
| jistr | cool :) | 13:29 |
| weshay | mwhahaha, looking for your input over time | 13:29 |
| weshay | mwhahaha, that data is from the containers-multinode job | 13:30 |
| *** nawar has joined #tripleo | 13:32 | |
| *** nawar has left #tripleo | 13:33 | |
| *** ade_lee has joined #tripleo | 13:35 | |
| jrist | weshay: that is a good example of a better commit message needed | 13:36 |
| weshay | jrist, did you read the bug? | 13:36 |
| jrist | yes | 13:38 |
| jrist | weshay: the reason I asked why is because the commit message was terrible :) | 13:38 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Add a lookup plugin to get mistral workflow results https://review.openstack.org/575121 | 13:39 |
| weshay | ensure tripleo-validations are turned off upstream | 13:39 |
| weshay | Let's be explicit about ensuring these tasks are off | 13:39 |
| *** cylopez has joined #tripleo | 13:39 | |
| tosky | honza: I really think that it was my last comment on the review for tempest-tripleo-ui, unless the gates disagree | 13:39 |
| honza | tosky: thank you for your patience! | 13:40 |
| jrist | haha | 13:40 |
| jrist | weshay: copying and pasting it here doesn't make it better | 13:40 |
| mwhahaha | weshay: is that the top 20 tasks? | 13:40 |
| mwhahaha | weshay: can we shrunk that to top 10 maybe? | 13:41 |
| weshay | jrist, just getting out in irc | 13:41 |
| weshay | for comment | 13:41 |
| openstackgerrit | Honza Pokorny proposed openstack/tempest-tripleo-ui master: Add basic project structure https://review.openstack.org/575730 | 13:41 |
| jrist | weshay: doesn't seem to matter that I -1'd anyway :) got pushed through | 13:41 |
| weshay | jrist, I don't think you really understood what was going on | 13:41 |
| jrist | no I do, it's nasty | 13:41 |
| weshay | jrist, k.. do you understand what the validations were running? | 13:42 |
| jrist | I'm not -1 on that anymore weshay ! | 13:42 |
| jrist | look at my last comment | 13:42 |
| weshay | jrist, just retrospecting | 13:43 |
| *** Petersingh is now known as Petersingh|away | 13:43 | |
| *** sanjayu__ has quit IRC | 13:43 | |
| *** Petersingh|away has quit IRC | 13:43 | |
| * weshay looking for bugs based on the validation tasks in ci | 13:43 | |
| *** saneax has joined #tripleo | 13:43 | |
| weshay | low value and high cost | 13:43 |
| jrist | medium value | 13:44 |
| jrist | but I agree | 13:44 |
| weshay | no value | 13:44 |
| weshay | NONE | 13:44 |
| jrist | ouch | 13:44 |
| weshay | they were never meant to run upstream | 13:44 |
| * jrist notes this under weshay's file | 13:44 | |
| weshay | they will NEVER fail | 13:44 |
| weshay | fail a job | 13:44 |
| weshay | once the margins in job run time are not razor thin, we can reconsider tasks like this | 13:45 |
| weshay | however atm.. w/ container downloads, updates etc.. it's just tooo hard to keep the lights on and run low value tasks | 13:46 |
| mwhahaha | move it to a periodic job | 13:47 |
| mwhahaha | like we mentioned for the password rotation bits | 13:47 |
| mwhahaha | at the ptg | 13:47 |
| weshay | mwhahaha, agree it was meant for 3rd party ovb and ci.centos | 13:48 |
| *** Vorrtex has joined #tripleo | 13:52 | |
| jrist | horse -> dead | 13:55 |
| *** rbrady has joined #tripleo | 13:55 | |
| * mwhahaha gets the paddles | 13:56 | |
| mwhahaha | CLEAR | 13:56 |
| florianf | weshay, jrist: I understand validations are costly, but maybe we can come up with a better way to test validation runs in CI. | 13:56 |
| jrist | aye | 13:56 |
| mwhahaha | so if it didn't require mistral, we could totally test it in standalone | 13:56 |
| jaosorior | florianf, jrist: Right the proposal was to test them out in OVB, or have a specific scneario for them. | 13:56 |
| jaosorior | * scenario | 13:56 |
| * jrist giggles at ovb taking on more | 13:56 | |
| florianf | jaosorior: yeah, maybe a job that only runs in tripleo-validations? | 13:57 |
| weshay | florianf, there are several ways we can improve the way they are run | 13:57 |
| weshay | florianf, we'd also have to customize the values that are being validated | 13:57 |
| weshay | florianf, and the job should fail if there is an error | 13:58 |
| jrist | I think we should rewrite them in rust | 13:58 |
| florianf | weshay: I agree with all three statements :) | 13:58 |
| jrist | that ought to do it | 13:58 |
| jaosorior | lol | 13:58 |
| *** fhubik|brb is now known as fhubik | 13:59 | |
| florianf | jrist, weshay : We should come up with a list of priorities of what we want to test in CI (inventory, mistral runs, specific validations, ...) and than consider cost vs value | 13:59 |
| quiquell|rover | jistr++ rust-it! | 13:59 |
| jaosorior | TripleO weekly meeting starting in a minute or so | 13:59 |
| weshay | wow.. I've never seen this before .. the tripleo gate queue.. #1 job is 1hr 11min, #2 job is 24 hr 26min | 13:59 |
| jaosorior | weshay: folks from infra can prioritize patches | 14:00 |
| jaosorior | I asked them to do that | 14:00 |
| jrist | quiquell|rover: :) | 14:00 |
| weshay | florianf, agree.. and like mwhahaha said, the standalone job is going to offer a greater degree of flexibility to what we can do in ci | 14:00 |
| jaosorior | #startmeeting tripleo | 14:00 |
| openstack | Meeting started Tue Sep 25 14:00:41 2018 UTC and is due to finish in 60 minutes. The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: tripleo)" | 14:00 | |
| openstack | The meeting name has been set to 'tripleo' | 14:00 |
| jaosorior | #topic agenda | 14:00 |
| *** openstack changes topic to "agenda (Meeting topic: tripleo)" | 14:00 | |
| jaosorior | * Review past action items | 14:00 |
| jaosorior | * One off agenda items | 14:00 |
| jaosorior | * Squad status | 14:01 |
| jaosorior | * Bugs & Blueprints | 14:01 |
| jaosorior | * Projects releases or stable backports | 14:01 |
| jaosorior | * Specs | 14:01 |
| jaosorior | * open discussion | 14:01 |
| jaosorior | Anyone can use the #link, #action and #info commands, not just the moderatorǃ | 14:01 |
| jaosorior | Hey folks! who's around? | 14:01 |
| marios | o/ hi | 14:01 |
| shardy | o/ | 14:01 |
| weshay | 0/ | 14:01 |
| jfrancoa | o/ hola | 14:01 |
| trown | o/ | 14:01 |
| ksambor | o/ | 14:01 |
| chem | o/ | 14:01 |
| fultonj | o/ | 14:01 |
| rfolco | o/ | 14:01 |
| mwhahaha | hi2u | 14:02 |
| panda | o/ | 14:02 |
| Vorrtex | o/ | 14:02 |
| beagles | o/ | 14:02 |
| bogdando | o/ | 14:02 |
| redrobot | o/ | 14:02 |
| jtomasek | o/ | 14:02 |
| jaosorior | #topic review past action items | 14:03 |
| *** openstack changes topic to "review past action items (Meeting topic: tripleo)" | 14:03 | |
| jaosorior | None! :D | 14:03 |
| jaosorior | #topic one off agenda items | 14:03 |
| *** openstack changes topic to "one off agenda items (Meeting topic: tripleo)" | 14:03 | |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:03 |
| florianf | o/ | 14:04 |
| jaosorior | gfidente: the first one-off agenda item is yours. | 14:04 |
| gfidente | jaosorior hi guys | 14:04 |
| jaosorior | \o | 14:04 |
| gfidente | yeah I am not sure if the one-line sentence is obvious enough but in the UC containerization | 14:04 |
| *** dtantsur|bbl is now known as dtantsur | 14:04 | |
| gfidente | ceph-ansible got installed in the mistral executor container which we never wanted to | 14:04 |
| jaosorior | For reference, this is the topic gfidente added: we want to consume ceph-ansible from the container host, not from within the container | 14:05 |
| shardy | we had to do the same for openshift - it's unfortunate there's not a common install path we can bind in for $ansible roles? | 14:05 |
| gfidente | instead we want to consume whatever version of the playbook is found on the container host | 14:05 |
| gfidente | shardy well we just go mount in undercloud.yaml the dir where ceph-ansible installs its playbooks from baremetal | 14:05 |
| bogdando | I wonder if there is an upgrade impact | 14:06 |
| gfidente | see https://review.openstack.org/#/c/604361/ | 14:06 |
| gfidente | https://review.openstack.org/#/c/604357/ | 14:06 |
| bogdando | like having validations executed against different versions of the package installed | 14:06 |
| matbu | o/ | 14:06 |
| gfidente | bogdando an upgrade impact to what? | 14:06 |
| jrist | o/ | 14:06 |
| jaosorior | Tengu: won't those mounts have issues with SELinux? ^^ | 14:06 |
| gfidente | in queens we didn't have UC containers | 14:06 |
| gfidente | and rocky is not out yet | 14:07 |
| Tengu | jaosorior: what mounts? | 14:07 |
| bogdando | gfidente: makes sense :) | 14:07 |
| Tengu | sorry, was on a patch | 14:07 |
| jaosorior | Tengu: read up. | 14:07 |
| fultonj | Tengu: https://review.openstack.org/#/c/604357/2/environments/undercloud.yaml | 14:07 |
| Tengu | duh. probably, yeah. | 14:07 |
| jaosorior | thought so, that won't work with the policy | 14:08 |
| jaosorior | since it's from /usr/share/ | 14:08 |
| Tengu | hmm we already mount stuff from there. | 14:08 |
| Tengu | it's ro, might work. | 14:08 |
| jaosorior | uhm... | 14:08 |
| Tengu | i.e. the puppet modules are mounted from that location. | 14:08 |
| jaosorior | if we do :Z that would be fine though | 14:08 |
| fultonj | good | 14:08 |
| Tengu | ro,Z you mean? | 14:09 |
| jaosorior | Tengu: yeah | 14:09 |
| Tengu | might be good as well, if nothing else mount that location. | 14:09 |
| Tengu | but I think we can't use :Z in tripleo. | 14:09 |
| jaosorior | Tengu: why not? | 14:09 |
| Tengu | as multiple container are launched | 14:09 |
| Tengu | we already tested that. | 14:09 |
| Tengu | only :z is OK. | 14:09 |
| Tengu | don't you remember? | 14:09 |
| jaosorior | Tengu: you're right. It would have issues with the init containers | 14:10 |
| Tengu | if we had only ONE container, or keep the s0-sublevel-thing that would be OK with :Z | 14:10 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 14:10 |
| Tengu | but that's not the case. so :ro,z only. | 14:10 |
| bogdando | Tengu: we use :Z to share with other containers | 14:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 14:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 14:10 |
| Tengu | bogdando: minus z ;) | 14:10 |
| bogdando | I think libvirt needs that? | 14:10 |
| jaosorior | bogdando: it's lowercase z to share | 14:10 |
| Tengu | bogdando: :Z makes it private for only one container, :z makes it sharable. | 14:10 |
| jaosorior | capital case Z is private | 14:10 |
| gfidente | bogdando Tengu jaosorior if it's about how we mount probably better to discuss in the review | 14:10 |
| Tengu | :) | 14:10 |
| jaosorior | got my selinux notes right here http://jaormx.github.io/2018/selinux-and-docker-notes/ | 14:10 |
| bogdando | ok :) | 14:10 |
| gfidente | I mostly wanted to bring up the issue and why we want a different behavior | 14:11 |
| jaosorior | gfidente: anyway... just wanted to double check. It should be fine then :D | 14:11 |
| * Tengu has his selinux stuff in his head for 3 weeks now | 14:11 | |
| fultonj | it's perfectly valid for the mistral-executor container to run external ansible packages OR for the user to run them too via "manual config download" so we should try to have consistent versions. | 14:11 |
| fultonj | https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ansible_config_download.html#manual-config-download | 14:11 |
| fultonj | so it's good we can have a way to do this | 14:11 |
| fultonj | any external installer will have this issue | 14:11 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade master: Assert services/containers are started after reboot. https://review.openstack.org/605084 | 14:12 |
| Tengu | gfidente: added a comment. | 14:12 |
| gfidente | right so reviews welcomed, thanks Tengu | 14:12 |
| jaosorior | gfidente: thanks for bringing it up | 14:12 |
| Tengu | (so yeah I'm here ;)) | 14:12 |
| jaosorior | Anybody else want to shime in on gfidente's topic? | 14:13 |
| * gfidente left already | 14:13 | |
| * gfidente joking | 14:13 | |
| Tengu | don't you dare gfidente ;) | 14:13 |
| fultonj | shardy: are you going to do something similar? | 14:13 |
| shardy | fultonj: we already added something IIRC but I've not found the patch | 14:14 |
| shardy | will find it and ping you a link, the patches from gfidente seem fine as a workaround to me | 14:14 |
| gfidente | I saw skydive there | 14:14 |
| fultonj | beagles: ^ | 14:14 |
| gfidente | I think that has to go away from the container as well | 14:14 |
| shardy | yeah, we end up bind mounting everything into the executor container, regardless of what services are enabled | 14:14 |
| shardy | not very composable, but I can't think of a better approach atm | 14:14 |
| gfidente | shardy well it's better than preinstalling everything in the image though | 14:15 |
| gfidente | I mean | 14:15 |
| shardy | yeah | 14:15 |
| gfidente | but should be composable yeah | 14:15 |
| shardy | containers ftw! ;) | 14:15 |
| EmilienM | it's not friday :P | 14:15 |
| gfidente | that was if containers aren't solving your problem yet, you're not using containers enough | 14:16 |
| * mwhahaha puts gfidente in a container and ships him off someplace nice | 14:16 | |
| shardy | yup, use more containers (that actually is how openshift solves this, ansible runs inside a container) ;) | 14:16 |
| gfidente | so mwhahaha was rtight! | 14:16 |
| gfidente | sorry I'll never bring up a topic to the meeting again | 14:17 |
| *** fhubik is now known as fhubik|brb | 14:17 | |
| gfidente | thanks all for helping | 14:17 |
| mwhahaha | :o | 14:17 |
| EmilienM | shardy: maybe out of topic but do you mind sharing how it works? we had this discussion last week | 14:17 |
| EmilienM | sdoran / mwhahaha / dprince prototyped things lately, it would be interesting to comapre | 14:18 |
| shardy | EmilienM: see https://review.openstack.org/#/c/583868/ - its WIP but flaper87 probably has more details | 14:18 |
| fultonj | nice! | 14:19 |
| shardy | Ok sounds like we're done on this topic so I'll mention https://bugs.launchpad.net/tripleo/+bug/1792613 | 14:19 |
| openstack | Launchpad bug 1792613 in tripleo "docker-puppet tasks can only run on the primary controller role" [High,In progress] - Assigned to Steven Hardy (shardy) | 14:19 |
| jaosorior | shardy: next topic is yours | 14:19 |
| shardy | I chatted with bandini at the PTG about this, and offered to take a look at how to solve it | 14:20 |
| shardy | turns out it's harder than I expected ;) | 14:20 |
| trown | shardy: EmilienM https://review.openstack.org/#/c/583868/ is working too ... some patches in that series dont work, but I tested the container patch by itself and it works | 14:20 |
| shardy | If folks can check http://lists.openstack.org/pipermail/openstack-dev/2018-September/135045.html and provide feedback I'll spend some more time on it | 14:20 |
| jaosorior | I remember we had this issue at some point when we tried to run keystone on it's own role, separate from the controller role. | 14:20 |
| *** quiquell|rover is now known as quique|rover|off | 14:20 | |
| jaosorior | I didn't know how to fix it. | 14:20 |
| shardy | basically all our bootstrap node stuff is wrong ;D | 14:20 |
| gfidente | shardy I am not sure but in theory people can build their list of what needs to be mounted as it is a param | 14:21 |
| * flaper87 clicks | 14:21 | |
| shardy | gfidente: yeah but it's not easy to say if ceph is enabled add this mount | 14:21 |
| shardy | the manual approach is fine, but it's not super-great for the standalone use-case | 14:22 |
| shardy | Ok for undercloud though | 14:22 |
| *** fhubik|brb is now known as fhubik | 14:22 | |
| mwhahaha | it's fine for standalone cause we don't use mistral | 14:22 |
| mwhahaha | so we'd be using the local version anwyay | 14:22 |
| jaosorior | shardy: is there a proposed solution or approach already for the bootstrap node issue? or is that still up for discussion/solving? | 14:23 |
| shardy | mwhahaha: yeah but you still have to manually build MistralExecutorExtraVolumes? | 14:23 |
| shardy | oh wait | 14:23 |
| mwhahaha | shardy: no mitral | 14:23 |
| * shardy needs more coffee :) | 14:23 | |
| mwhahaha | mistral | 14:23 |
| mwhahaha | :D | 14:23 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-quickstart-extras master: Added podman support for log collection https://review.openstack.org/605090 | 14:24 |
| Tengu | EmilienM: -^^ | 14:24 |
| shardy | Ok I'm done, feedback on the ML would be welcome, and I may have to ask for help if the implementation gets too time consuming | 14:24 |
| jaosorior | shardy: thanks | 14:24 |
| EmilienM | Tengu: ack will look after mtg | 14:24 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-quickstart-extras master: Add podman support for log collection https://review.openstack.org/605090 | 14:24 |
| Tengu | sorry for the noise -.-'. | 14:24 |
| jaosorior | Alright! one last reminder that this is pretty much the last day to submit topics for the Summit | 14:24 |
| jaosorior | #link Stein forum topics: | 14:25 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-forum-stein | 14:25 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Set prepare neutron_driver from NeutronMechanismDrivers https://review.openstack.org/604953 | 14:25 |
| jaosorior | that's the link ^^ | 14:25 |
| *** mjturek has joined #tripleo | 14:25 | |
| jaosorior | although, feel free to make the proposal yourself and add it to the etherpad, like gfidente did. That was a nice idea :D | 14:25 |
| gfidente | but don't mention containers in the topic | 14:26 |
| jaosorior | #containers #kubernetes #cloudnative | 14:26 |
| jaosorior | #servicemesh | 14:27 |
| jaosorior | that'll do the trick :D | 14:27 |
| jaosorior | Anyway | 14:27 |
| jaosorior | Finally, a reminder from the CI team | 14:27 |
| jaosorior | CI Community Meeting starts immediately upon this meeting closing in #oooq. This is our team's weekly "open office hours." All are welcome! Ask/discuss anything, we don't bite. Agenda (add items freely) --> https://etherpad.openstack.org/p/tripleo-ci-squad-meeting. | 14:27 |
| jaosorior | #topic Squad status | 14:28 |
| *** openstack changes topic to "Squad status (Meeting topic: tripleo)" | 14:28 | |
| jaosorior | ci | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 14:28 |
| jaosorior | upgrade | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status | 14:28 |
| jaosorior | containers | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-containers-squad-status | 14:28 |
| jaosorior | integration | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-integration-squad-status | 14:28 |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: WIP Removes EXTRA_TAGS from toci_gate_test&quickstart.sh.j2 https://review.openstack.org/604768 | 14:28 |
| jaosorior | ui/cli | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status | 14:28 |
| jaosorior | validations | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-validations-squad-status | 14:28 |
| jaosorior | networking | 14:28 |
| *** lblanchard has joined #tripleo | 14:28 | |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-networking-squad-status | 14:28 |
| jaosorior | workflows | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status | 14:28 |
| jaosorior | security | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-security-squad | 14:28 |
| jaosorior | edge | 14:28 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-edge-squad-status | 14:28 |
| jaosorior | Anything any squad wants to bring up to the weekly meeting? | 14:28 |
| *** waleedm has quit IRC | 14:29 | |
| EmilienM | nothing outstanding in the container squad, we're still working on podman support | 14:29 |
| * Vorrtex is new around here... does this mean we're at Open Floor? | 14:30 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-quickstart-extras master: Add podman support for log collection https://review.openstack.org/605090 | 14:30 |
| * Vorrtex nope, sorry, retreating. | 14:31 | |
| rfolco | CI squad status https://etherpad.openstack.org/p/tripleo-ci-squad-meeting - L72 | 14:31 |
| marios | Vorrtex: open floor is usuall at the end see https://wiki.openstack.org/wiki/Meetings/TripleO | 14:32 |
| jaosorior | #topic bugs & blueprints | 14:32 |
| *** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)" | 14:32 | |
| jaosorior | #link https://launchpad.net/tripleo/+milestone/rocky-rc2 | 14:32 |
| jaosorior | #link https://launchpad.net/tripleo/+milestone/stein-1 | 14:32 |
| jaosorior | For Stein we currently have 28 blueprints open in Launchpad. | 14:32 |
| jaosorior | Bugs: 6 (-6) rocky-rc2, 783 (+11) stein-1. 102 (-1) open Storyboard bugs. | 14:32 |
| jaosorior | #link https://storyboard.openstack.org/#!/project_group/76 | 14:32 |
| EmilienM | rocky-rc2 was released | 14:32 |
| jaosorior | :D | 14:32 |
| mwhahaha | i'll go through and move bugs out that may still be open | 14:32 |
| EmilienM | mwhahaha: I guess we can do the launchpad dance | 14:32 |
| mwhahaha | #action mwhahaha to move rocky-rc2 bugs out since it's been released | 14:32 |
| *** jmelis has left #tripleo | 14:32 | |
| EmilienM | I think we'll call final rocky at this point | 14:32 |
| mwhahaha | it's likely that rc2 will be the ga release unless someone has serious objections | 14:33 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet https://review.openstack.org/603253 | 14:33 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks https://review.openstack.org/603254 | 14:33 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs https://review.openstack.org/603255 | 14:33 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Name unnamed tasks https://review.openstack.org/604460 | 14:33 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Fix tasks in check mode https://review.openstack.org/604461 | 14:33 |
| *** agopi__ is now known as agopi | 14:33 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Switch fs027 to deploy with podman https://review.openstack.org/600517 | 14:34 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Switch fs027 to deploy with podman https://review.openstack.org/600517 | 14:34 |
| *** geneliu has quit IRC | 14:34 | |
| jaosorior | #topic projects releases or stable backports | 14:34 |
| *** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)" | 14:34 | |
| EmilienM | we released pike & queens yesterday | 14:34 |
| *** moshele has quit IRC | 14:35 | |
| EmilienM | nothing else to report | 14:35 |
| jaosorior | thanks EmilienM | 14:35 |
| jaosorior | #topic specs | 14:35 |
| *** openstack changes topic to "specs (Meeting topic: tripleo)" | 14:35 | |
| jaosorior | #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open | 14:35 |
| EmilienM | the one about podman has 3 +2 https://review.openstack.org/#/c/602480/ | 14:35 |
| EmilienM | maybe we can move forward and iterate eventually? | 14:36 |
| *** cylopez has quit IRC | 14:36 | |
| jaosorior | EmilienM: I can give it a read too. But I'm fine with that going in as well, given the 3 +2s from folks on your squad; who have better visibility on that work. | 14:37 |
| EmilienM | ok. and again, happy to make changes if needed | 14:37 |
| bogdando | note, we'll need one +2 from upgrades team IMO | 14:37 |
| bogdando | before merging | 14:37 |
| EmilienM | good idea | 14:38 |
| jaosorior | bogdando: got anybody in mind that could give it a read? that is a good idea | 14:38 |
| EmilienM | marios / chem | 14:38 |
| EmilienM | at least | 14:38 |
| EmilienM | jistr if you have time too | 14:38 |
| chem | EmilienM: bogdando will look | 14:39 |
| bogdando | thanks | 14:39 |
| EmilienM | thx | 14:39 |
| jistr | ack me too, i already skimmed through but didn't read in full yet | 14:39 |
| EmilienM | ok np | 14:39 |
| jaosorior | EmilienM: OK, I'll give it a read and wait for chem or jistr to read it too before merging. | 14:39 |
| jaosorior | EmilienM: thanks for bringing it up. | 14:39 |
| marios | EmilienM: will check | 14:40 |
| EmilienM | sure, thx all for your time | 14:40 |
| dtantsur | I started formally writing up the proposal of removing nova: https://review.openstack.org/604761 | 14:40 |
| dtantsur | WIP, but a lot of stuff to read already | 14:40 |
| marios | jaosorior: this one is ready for merge i think https://review.openstack.org/579854 | 14:40 |
| Tengu | small note: if this review (https://review.rdoproject.org/r/#/c/16354/ ) is OK, it would be great to push https://review.openstack.org/#/c/605039/ asap - promotion blocker :). | 14:40 |
| openstackgerrit | Rajini Karthik proposed openstack/puppet-tripleo master: Dell EMC SC: Add support for excluded_domain_ips https://review.openstack.org/604444 | 14:40 |
| jaosorior | marios: will take a look. Thanks marios | 14:41 |
| chkumar|ruck | EmilienM: Hello | 14:41 |
| chkumar|ruck | EmilienM: Fs027 changes, since we are going to use podman but in fs027 tempest is runned from container which is dependent on docker that also needs to be updated otherwise tempest will fail | 14:42 |
| jaosorior | Any other specs folks want to bring up? | 14:42 |
| EmilienM | chkumar|ruck: right I'll work on it next no problem | 14:43 |
| EmilienM | chkumar|ruck: fwiw it didn't fail since we still have docker deployed for now | 14:43 |
| jaosorior | #topic open discussion | 14:44 |
| *** openstack changes topic to "open discussion (Meeting topic: tripleo)" | 14:44 | |
| Vorrtex | Hey as I mentioned earlier, I'm new around here, and have been working to get RDO multiarch. I've made the following patch (which at first glance is ugly, but I included a comment to explain its purpose) that will help the RDO multiarch effort, if you guys could take a look. Its quite small and digestible. | 14:44 |
| Vorrtex | https://review.openstack.org/#/c/604401/ | 14:44 |
| *** jfrancoa has quit IRC | 14:45 | |
| chkumar|ruck | EmilienM: ok. I will take a look on podman stuff to update on tempest side also | 14:45 |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates stable/pike: Improve nova statedir ownership logic https://review.openstack.org/587066 | 14:45 |
| florianf | Could I kindly ask for someone to give this backport patch a final push: https://review.openstack.org/#/c/603031/ | 14:45 |
| dtantsur | jaosorior: did we discuss the Forum proposals already? sorry, I had a conflicting meeting | 14:45 |
| EmilienM | chkumar|ruck: we have a new oooq param undercloud_container_cli | 14:45 |
| EmilienM | we could use that | 14:45 |
| chkumar|ruck | EmilienM: yes | 14:45 |
| jaosorior | dtantsur: I briefly did a reminder of that. | 14:45 |
| dtantsur | jaosorior: do you plan to propose topics or should we propose them after talking to you? | 14:46 |
| *** fhubik is now known as fhubik|brb | 14:46 | |
| jaosorior | dtantsur: I was planning to propose them. But if you want to do it (and I know you ahve a topic in there), feel free. | 14:46 |
| dtantsur | jaosorior: I'm fine with leaving it up to you :) | 14:46 |
| shardy | Vorrtex: welcome, thanks for the link to the patch! | 14:47 |
| jaosorior | dtantsur: if you have time already, you can submit the proposal, else I'll do it early in my time tomorrow. | 14:47 |
| Vorrtex | Sure thing! shardy thanks for the welcome :) | 14:47 |
| jaosorior | Vorrtex: thanks for the patch. +2ed it. | 14:47 |
| Vorrtex | jaosorior thanks much! | 14:47 |
| chkumar|ruck | EmilienM: I will take a look and propose the patch for tempest tomorrow | 14:48 |
| weshay | jaosorior, I should probably note that vexxhost is being investigated for 3rd party tripleo ci | 14:48 |
| weshay | jaosorior, we probably want to invite nhicher and others to comment | 14:49 |
| weshay | reviews on https://review.openstack.org/#/c/596432/ | 14:49 |
| jaosorior | weshay: will we have the same capabilities we did in RDO cloud for OVB? | 14:49 |
| EmilienM | chkumar|ruck: ok | 14:49 |
| weshay | jaosorior, I am poking at that | 14:49 |
| openstackgerrit | Rajini Karthik proposed openstack/tripleo-heat-templates master: Dell EMC Sc: Add support for excluded_domain_ips https://review.openstack.org/604447 | 14:50 |
| weshay | asking for more details to be sent out | 14:50 |
| jaosorior | weshay: thanks for the info. | 14:50 |
| openstackgerrit | Victor Pickard proposed openstack/tripleo-heat-templates master: Change log directory for OpenDaylight https://review.openstack.org/603907 | 14:50 |
| *** dxiri has quit IRC | 14:50 | |
| weshay | also as usual.. CI open door immediately following this mtg | 14:50 |
| openstackgerrit | Victor Pickard proposed openstack/tripleo-heat-templates master: Change log directory for OpenDaylight https://review.openstack.org/603907 | 14:51 |
| *** dxiri has joined #tripleo | 14:51 | |
| jaosorior | Anything else folks want to bring up to the weekly meeting? | 14:53 |
| jaosorior | #endmeeting | 14:54 |
| *** openstack changes topic to "Welcome to Rocky | CI Status: RED, DO NOT WORKFLOW https://docs.openstack.org/tripleo-docs/latest/" | 14:54 | |
| openstack | Meeting ended Tue Sep 25 14:54:33 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:54 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-09-25-14.00.html | 14:54 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-09-25-14.00.txt | 14:54 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo/2018/tripleo.2018-09-25-14.00.log.html | 14:54 |
| jaosorior | Thanks for joining folks! | 14:54 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder master: enable caching for gentoo builds https://review.openstack.org/604268 | 14:55 |
| Vorrtex | Thanks for the reviews, guys! | 14:56 |
| *** ykarel is now known as ykarel|away | 14:57 | |
| *** geneliu1 has joined #tripleo | 15:04 | |
| florianf | thanks bogdando ! | 15:08 |
| *** ksambor has quit IRC | 15:10 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 15:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 15:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 15:10 |
| *** kopecmartin is now known as kopecmartin|off | 15:11 | |
| *** udesale has joined #tripleo | 15:12 | |
| *** chkumar|ruck is now known as chkumar|off | 15:17 | |
| *** geneliu1 has left #tripleo | 15:17 | |
| *** dprince has quit IRC | 15:21 | |
| *** ykarel|away has quit IRC | 15:22 | |
| *** geneliu has joined #tripleo | 15:22 | |
| geneliu | hi | 15:22 |
| *** fhubik|brb is now known as fhubik | 15:23 | |
| thrash | is there a reason we are running healthcheck every 2 seconds? | 15:24 |
| *** jtomasek has quit IRC | 15:26 | |
| *** sanjayu_ has joined #tripleo | 15:33 | |
| *** trozet has joined #tripleo | 15:33 | |
| *** saneax has quit IRC | 15:35 | |
| *** numans_ has quit IRC | 15:35 | |
| *** leanderthal has quit IRC | 15:37 | |
| *** dxiri has quit IRC | 15:37 | |
| *** rh-jelabarre has quit IRC | 15:39 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci master: [WIP] Testing ansible-ml2 job https://review.openstack.org/582294 | 15:41 |
| *** rh-jelabarre has joined #tripleo | 15:42 | |
| derekh | if anybody has time to review ^^ I'd appreciate it, trying to get a CI job running that runs ironic in overcloud | 15:43 |
| bogdando | bandini: hi. PTAL https://review.openstack.org/#/c/600849/21 | 15:44 |
| *** artom has joined #tripleo | 15:49 | |
| shardy | derekh: nice, will check it out | 15:52 |
| Tengu | so. see you tomorrow guys! | 15:54 |
| *** yprokule has quit IRC | 15:54 | |
| derekh | shardy: thanks, Once the 3 depended patches have merged I'll update the top level patch to add back in the other jobs (/me is trying to save ci resources) | 15:54 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder master: enable caching for gentoo builds https://review.openstack.org/604268 | 15:58 |
| *** ykarel|away has joined #tripleo | 15:59 | |
| *** ykarel|away is now known as ykarel | 15:59 | |
| *** sshnaidm|off has joined #tripleo | 16:02 | |
| *** dprince has joined #tripleo | 16:03 | |
| *** dprince has quit IRC | 16:03 | |
| *** dprince has joined #tripleo | 16:03 | |
| *** fhubik has quit IRC | 16:06 | |
| *** sshnaidm|off has quit IRC | 16:06 | |
| *** iranzo has quit IRC | 16:06 | |
| *** sshnaidm has joined #tripleo | 16:06 | |
| weshay | mwhahaha, do you have a pointer to the upstream mirror config | 16:06 |
| *** pcaruana has joined #tripleo | 16:07 | |
| *** rdopiera has quit IRC | 16:08 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 16:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 16:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 16:10 |
| *** shyamb has joined #tripleo | 16:11 | |
| mwhahaha | weshay: what are you looking for? | 16:13 |
| weshay | the puppet that sets up infra mirrors | 16:13 |
| mwhahaha | weshay: riiiiight could you be more specific? | 16:16 |
| *** jcoufal has quit IRC | 16:16 | |
| mwhahaha | weshay: do you mean, https://github.com/openstack-infra/system-config/blob/master/modules/openstack_project/manifests/mirror.pp#L3 | 16:17 |
| weshay | thanks | 16:18 |
| weshay | good enough pointer | 16:19 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: ensure tripleo-validations are turned off upstream https://review.openstack.org/604878 | 16:19 |
| jistr | EmilienM: finished review on the podman spec -- well written! Most important unknowns and risks with enough specificity and solution suggestions, while keeping good clarity and info density in the text. ++ thanks | 16:20 |
| EmilienM | cool thanks for this feedback | 16:21 |
| *** jfrancoa has joined #tripleo | 16:22 | |
| *** florianf has quit IRC | 16:23 | |
| *** shyamb has quit IRC | 16:25 | |
| *** Petersingh has joined #tripleo | 16:25 | |
| *** Petersingh has quit IRC | 16:25 | |
| *** aufi has quit IRC | 16:26 | |
| *** udesale has quit IRC | 16:27 | |
| *** phuongnh has joined #tripleo | 16:29 | |
| *** phuongnh has quit IRC | 16:33 | |
| *** bogdando has quit IRC | 16:36 | |
| openstackgerrit | Honza Pokorny proposed openstack/tempest-tripleo-ui master: Add basic project structure https://review.openstack.org/575730 | 16:37 |
| *** sanjayu_ has quit IRC | 16:40 | |
| *** jtomasek has joined #tripleo | 16:43 | |
| *** dtantsur is now known as dtantsur|afk | 16:45 | |
| mandre | is now a good time to push a rebased series of 14 commits? | 16:45 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Dump zuul variables to file https://review.openstack.org/602673 | 16:45 |
| *** zaneb has quit IRC | 16:46 | |
| *** zaneb has joined #tripleo | 16:46 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Add simple validation that OpenShift is deployed https://review.openstack.org/600067 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Refactor openshift services for composable roles https://review.openstack.org/599618 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Fix openshift new node detection https://review.openstack.org/600012 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Clean up previous osa inventory dir before deployment https://review.openstack.org/600028 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Configure haproxy for openshift infra https://review.openstack.org/601241 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Fix inventory files for newer openshift-ansible https://review.openstack.org/603577 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Introduce OpenShiftGlusterNodeVars heat param https://review.openstack.org/604724 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Make glusterfs the default sc when deploying with CNS https://review.openstack.org/604725 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Consolidate openshift-ansible global variables https://review.openstack.org/604726 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Add heat param for openshift prerequisites playbook https://review.openstack.org/604338 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Remove unused networks from OpenShift roles https://review.openstack.org/604727 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Deploy openshift all in one in scenario009 https://review.openstack.org/603780 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Use openshift-ansible container instead of RPMs https://review.openstack.org/583868 | 16:47 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Do not wipe disks on OpenShift gluster nodes https://review.openstack.org/605127 | 16:47 |
| * mandre runs away | 16:47 | |
| *** dtantsur|afk has quit IRC | 16:51 | |
| *** panda is now known as panda|off | 16:52 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder master: enable caching for gentoo builds https://review.openstack.org/604268 | 16:56 |
| *** dtantsur has joined #tripleo | 16:57 | |
| *** dtantsur is now known as dtantsur|afk | 16:58 | |
| *** derekh has quit IRC | 17:00 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP WES: Add necessary bits for N-1->N standalone upgrade. https://review.openstack.org/605149 | 17:00 |
| *** ykarel is now known as ykarel|away | 17:01 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: WIP: New workflow for standalone upgrade https://review.openstack.org/604706 | 17:02 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP WES: Add necessary bits for N-1->N standalone upgrade. https://review.openstack.org/605149 | 17:04 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: WIP: New workflow for standalone upgrade https://review.openstack.org/604706 | 17:04 |
| *** trown is now known as trown|lunch | 17:08 | |
| *** dhellmann has joined #tripleo | 17:09 | |
| *** akrivoka has quit IRC | 17:09 | |
| dhellmann | jaosorior: hey, I noticed you abandoned a few of the zuul import patches to clear up the gate. What's the plan for those? Are you going to restore them at some point later? | 17:09 |
| EmilienM | yes | 17:10 |
| EmilienM | the gate is broken for us | 17:10 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 17:10 |
| EmilienM | we are trying to figure out timeouts, see #topic | 17:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 17:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 17:10 |
| EmilienM | dhellmann: ^ | 17:10 |
| EmilienM | so we've purged the gate to avoid unnecessary timeouts | 17:10 |
| dhellmann | EmilienM : ok, that's fine, I just want to make sure I wasn't supposed to be keeping track of those | 17:10 |
| EmilienM | but we'll restore everything | 17:10 |
| EmilienM | dhellmann: no you don't have to, we do | 17:11 |
| dhellmann | would it be possible to restore them but remove the workflow flag (or even have one of you mark them WIP)? my tracking tool considers abandoned patches as "done" | 17:11 |
| *** ramishra has quit IRC | 17:16 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Update standalone role https://review.openstack.org/605156 | 17:20 |
| EmilienM | jaosorior: see dhellmann's request when you're back | 17:26 |
| *** vinaykns has quit IRC | 17:27 | |
| *** jpena is now known as jpena|off | 17:29 | |
| *** salmankhan has quit IRC | 17:33 | |
| *** shardy has quit IRC | 17:34 | |
| *** nawar has joined #tripleo | 17:34 | |
| *** nawar has quit IRC | 17:35 | |
| *** gfidente has quit IRC | 17:36 | |
| *** nawar has joined #tripleo | 17:36 | |
| *** nawar has quit IRC | 17:37 | |
| *** dxiri has joined #tripleo | 17:38 | |
| *** panda|off has quit IRC | 17:41 | |
| *** holser_ has quit IRC | 17:43 | |
| *** jfrancoa has quit IRC | 17:44 | |
| *** panda has joined #tripleo | 17:45 | |
| *** bugzy_ is now known as bugzy | 17:46 | |
| *** panda is now known as panda|off | 17:47 | |
| *** nawar has joined #tripleo | 17:48 | |
| *** ykarel has joined #tripleo | 17:49 | |
| *** ykarel|away has quit IRC | 17:49 | |
| *** amoralej is now known as amoralej|off | 17:52 | |
| *** mcornea has joined #tripleo | 17:55 | |
| *** nawar has quit IRC | 17:56 | |
| openstackgerrit | Nicolas Hicher proposed openstack-infra/tripleo-ci master: provider: Add vexxhost https://review.openstack.org/596432 | 18:09 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 18:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 18:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 18:10 |
| Tengu | hey there, can anyone push the neded w+1 on that one? https://review.openstack.org/#/c/605039/ | 18:11 |
| Tengu | would be really nice :). | 18:11 |
| Tengu | apparently the test was successful: https://review.rdoproject.org/r/#/c/16354/ (cc quique|rover|off ) | 18:11 |
| Tengu | damn. timeout. | 18:12 |
| slagle | pretty sure we're holding off on approving new patches given the status of the gat | 18:12 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: fedora28 standalone job definition https://review.openstack.org/595374 | 18:12 |
| slagle | gate | 18:12 |
| Tengu | slagle: yeah, but that one is a promotion blocker. there's a patch provided by weshay that should help the gate, not sure about its status | 18:12 |
| Tengu | anyway, can't recheck it now, not on my work laptop, no access :) | 18:13 |
| weshay | slagle, Tengu the changes required for the gate are merged | 18:14 |
| Tengu | ah :) | 18:14 |
| Tengu | weshay: and does it improve? | 18:14 |
| weshay | Tengu, so far | 18:15 |
| weshay | it's still a pretty large queue | 18:15 |
| Tengu | weshay: ok :). so will wait tomorrow for a recheck on the said patch | 18:15 |
| Tengu | as there were a time_out for a voting check | 18:15 |
| Tengu | hopefully the queue will shrink by then. | 18:16 |
| *** trown|lunch is now known as trown | 18:36 | |
| *** hjensas has quit IRC | 18:39 | |
| *** gouthamr has quit IRC | 18:52 | |
| *** dmellado has quit IRC | 18:53 | |
| *** raildo_ has joined #tripleo | 18:53 | |
| *** stevebaker has quit IRC | 18:53 | |
| *** raildo has quit IRC | 18:55 | |
| *** dxiri has quit IRC | 18:56 | |
| *** ykarel has quit IRC | 19:05 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 19:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 19:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 19:10 |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-heat-templates master: Switch Manila API to httpd and support TLS https://review.openstack.org/594566 | 19:11 |
| *** gouthamr has joined #tripleo | 19:12 | |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: podman: create/delete systemd unit files when restart policy is used https://review.openstack.org/600849 | 19:13 |
| *** ssbarnea|bkp has quit IRC | 19:14 | |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: Stop hardcoding 'docker' and make it more generic https://review.openstack.org/601290 | 19:14 |
| *** ssbarnea|bkp has joined #tripleo | 19:14 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Increase post-timeout to 1 hour https://review.openstack.org/605185 | 19:24 |
| mwhahaha | sshnaidm, weshay, panda|off -^ fyi | 19:24 |
| weshay | I saw | 19:27 |
| weshay | rdo went ppooooof | 19:29 |
| * weshay blames thrash | 19:29 | |
| mwhahaha | weshay: so pre+run share time | 19:39 |
| mwhahaha | weshay: tldr, we're screwed unless we can get everything < 3 hours consistently | 19:40 |
| mwhahaha | that includes infra setup time | 19:40 |
| mwhahaha | so i think we need to only have one or two multinode jobs with underclouds | 19:40 |
| mwhahaha | and everything else needs to be a single node standalone | 19:40 |
| openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: Remove timeout logic https://review.openstack.org/589068 | 19:41 |
| *** dxiri has joined #tripleo | 19:44 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet https://review.openstack.org/603253 | 19:48 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks https://review.openstack.org/603254 | 19:48 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs https://review.openstack.org/603255 | 19:48 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Name unnamed tasks https://review.openstack.org/604460 | 19:48 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Fix tasks in check mode https://review.openstack.org/604461 | 19:48 |
| *** salmankhan has joined #tripleo | 19:48 | |
| *** salmankhan has quit IRC | 19:53 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: WIP: Handle LP openvswitch meta-package on upgrade https://review.openstack.org/605200 | 19:54 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: WIP: Handle LP openvswitch meta-package on upgrade https://review.openstack.org/605200 | 19:55 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Introduce OS::TripleO::Services::Podman https://review.openstack.org/604235 | 20:00 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Introduce OS::TripleO::Services::Podman https://review.openstack.org/604235 | 20:00 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud: deploy podman https://review.openstack.org/605221 | 20:02 |
| EmilienM | dprince: ^ see https://review.openstack.org/604235 based on our chat earlier | 20:02 |
| *** trown is now known as trown|outtypewww | 20:03 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Switch fs027 to deploy with podman https://review.openstack.org/600517 | 20:04 |
| *** gouthamr_ has joined #tripleo | 20:05 | |
| mwhahaha | EmilienM: can we do something like https://review.openstack.org/#/c/586679/ so we don't have to have both docker/podman defined in the roles? | 20:06 |
| *** dmellado has joined #tripleo | 20:07 | |
| EmilienM | mwhahaha: no | 20:07 |
| EmilienM | :P | 20:07 |
| mwhahaha | k | 20:07 |
| mwhahaha | -2 | 20:07 |
| EmilienM | fine ! | 20:07 |
| EmilienM | mwhahaha: just fyi I did this way because right now we need docker AND podman together | 20:08 |
| EmilienM | but yes we should do something like this | 20:08 |
| mwhahaha | i mean i guess it's ok for testing | 20:08 |
| EmilienM | mwhahaha: my problem is I need both services now | 20:09 |
| EmilienM | while in TimeSync you need one | 20:09 |
| mwhahaha | yea | 20:09 |
| *** nawar has joined #tripleo | 20:09 | |
| EmilienM | mwhahaha: once we solve the rootwrap thing, I think yes we need to do like TimeSync | 20:09 |
| EmilienM | for now we still need docker | 20:10 |
| mwhahaha | k | 20:10 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 20:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 20:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 20:10 |
| EmilienM | mwhahaha: let me update commit msg | 20:10 |
| EmilienM | you made a goodp oint | 20:10 |
| mwhahaha | at least someone likes my points :( | 20:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Introduce OS::TripleO::Services::Podman https://review.openstack.org/604235 | 20:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud: deploy podman https://review.openstack.org/605221 | 20:11 |
| EmilienM | mwhahaha: I avoided -2 :P | 20:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Switch fs027 to deploy with podman https://review.openstack.org/600517 | 20:12 |
| dprince | EmilienM: one inline comment for you | 20:12 |
| EmilienM | dprince: ack looking | 20:12 |
| *** janki has joined #tripleo | 20:12 | |
| EmilienM | dprince: yeah I guess my point is if we have crane, we can have an insecure crane? | 20:13 |
| *** chem has quit IRC | 20:26 | |
| *** chem has joined #tripleo | 20:26 | |
| *** gouthamr has quit IRC | 20:38 | |
| *** lblanchard has quit IRC | 20:38 | |
| *** gouthamr_ is now known as gouthamr | 20:39 | |
| *** pcaruana has quit IRC | 20:43 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Prepare Docker Registry + Containers in pre-run https://review.openstack.org/580037 | 20:45 |
| dprince | EmilienM: Crane calls itself a lightweight 'docker pull' API. | 20:48 |
| dprince | EmilienM: so I don't necissarily think we need to rename that param | 20:48 |
| *** chem has quit IRC | 20:48 | |
| openstackgerrit | Sam Doran proposed openstack/tripleo-ansible master: Create Ansible module for gathering podman images facts https://review.openstack.org/605229 | 20:50 |
| openstackgerrit | Sam Doran proposed openstack/tripleo-ansible master: Create Ansible module for gathering podman images facts https://review.openstack.org/605229 | 20:51 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet https://review.openstack.org/603253 | 20:54 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks https://review.openstack.org/603254 | 20:54 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs https://review.openstack.org/603255 | 20:54 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Name unnamed tasks https://review.openstack.org/604460 | 20:54 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Fix tasks in check mode https://review.openstack.org/604461 | 20:54 |
| *** jtomasek has quit IRC | 21:00 | |
| openstackgerrit | Merged openstack/tripleo-common stable/queens: Revert "Remove ceph-osd after deployment succeeded in containers" https://review.openstack.org/604779 | 21:05 |
| *** janki has quit IRC | 21:08 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 21:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 21:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 21:10 |
| *** stevebaker has joined #tripleo | 21:12 | |
| *** raildo_ has quit IRC | 21:17 | |
| *** Vorrtex has quit IRC | 21:20 | |
| weshay | so is zuul down too? | 21:23 |
| *** slaweq has quit IRC | 21:27 | |
| *** slaweq has joined #tripleo | 21:27 | |
| *** ade_lee has quit IRC | 21:40 | |
| mwhahaha | just slow | 21:41 |
| *** lblanchard has joined #tripleo | 21:47 | |
| *** nawar has quit IRC | 21:50 | |
| *** agopi has quit IRC | 21:52 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt https://review.openstack.org/591540 | 21:56 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: break out release config by distro type https://review.openstack.org/602387 | 21:56 |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Prepare Docker Registry + Containers in pre-run https://review.openstack.org/580037 | 22:02 |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 22:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1794034 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 22:10 |
| openstack | Launchpad bug 1794034 in tripleo "Validation tasks have increase their duration" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 22:10 |
| *** lblanchard has quit IRC | 22:12 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt https://review.openstack.org/591540 | 22:26 |
| *** dxiri has quit IRC | 22:28 | |
| *** agopi has joined #tripleo | 22:32 | |
| *** agopi has quit IRC | 22:37 | |
| *** thrash is now known as thrash|g0ne | 22:44 | |
| mwhahaha | stevebaker, EmilienM, weshay: so I don't think we adjust --max-concurrent-downloads in docker (default is 3), do we know if that might improve CI times and does podman have an similar config? | 22:49 |
| stevebaker | mwhahaha: interesting. we also have concurrent pulls happening in prepare, I don't know if --max-concurrent-downloads is per image or docker global | 22:51 |
| mwhahaha | stevebaker: it's a dockerd global | 22:51 |
| mwhahaha | not completely sure if it's layers or images tho | 22:52 |
| mwhahaha | sounded like it was images | 22:52 |
| mwhahaha | because layers depend on each other | 22:52 |
| stevebaker | mwhahaha: podman transfers will happen with skopeo, I don't see a way of tweaking concurrency | 22:53 |
| mwhahaha | oh it's 3 layers at a time | 22:53 |
| mwhahaha | so it's layers | 22:53 |
| stevebaker | mwhahaha: I'll have a play with --max-concurrent-downloads | 22:53 |
| mwhahaha | https://docs.docker.com/edge/engine/reference/commandline/pull/#parent-command | 22:53 |
| stevebaker | mwhahaha: I suspect there is not much time to save there, its mostly in modifying the images | 22:54 |
| mwhahaha | stevebaker: i don't know about that, i'm trying to pull them down and it's taking a bunch of time. of course what i'm doing is serial | 22:54 |
| mwhahaha | just a 5pm thought that will annoy me until tomorrow | 22:55 |
| stevebaker | :} | 22:55 |
| mwhahaha | looks like it might? https://github.com/containers/skopeo/pull/354/commits/73c774ecc573c184d06fafc10c7042cfac33e701 | 22:58 |
| mwhahaha | magic go stuff | 22:59 |
| stevebaker | mwhahaha: oh it definitely transfers layers concurrently, but I don't think it exposes a way to say how much | 22:59 |
| *** rcernin has joined #tripleo | 23:07 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1715374 | 23:10 |
| openstack | Launchpad bug 1715374 in tripleo "Reloading compute with SIGHUP prenvents instances to boot" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 23:10 |
| *** tosky has quit IRC | 23:11 | |
| *** dprince has quit IRC | 23:14 | |
| *** agopi has joined #tripleo | 23:22 | |
| openstackgerrit | Merged openstack/tripleo-common stable/queens: Upgrade docker ceph container https://review.openstack.org/603369 | 23:23 |
| openstackgerrit | Merged openstack/tripleo-common stable/queens: [Queens Only]Rename uwsgi-plugin-python to uwsgi-plugin-python2 for barbican https://review.openstack.org/604976 | 23:24 |
| openstackgerrit | Merged openstack/tripleo-common stable/pike: [Pike Only] Rename uwsgi-plugin-python to uwsgi-plugin-python2 in barbican https://review.openstack.org/604977 | 23:24 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Only validate admin and public host in local_subnet cidr https://review.openstack.org/600439 | 23:24 |
| *** mjturek has quit IRC | 23:42 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!