| *** Chaserjim has quit IRC | 00:04 | |
| *** Chaserjim has joined #tripleo | 00:04 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/diskimage-builder master: Move common ubuntu environment setting to ubuntu-common element https://review.openstack.org/563731 | 00:04 |
|---|---|---|
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/diskimage-builder master: Remove redundant sources change/update https://review.openstack.org/563739 | 00:05 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/diskimage-builder master: Add ubuntu-systemd-container operating-system element https://review.openstack.org/563748 | 00:05 |
| *** tosky has quit IRC | 00:09 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 00:10 |
| *** Chaserjim has quit IRC | 00:11 | |
| *** Chaserjim has joined #tripleo | 00:12 | |
| EmilienM | sileht: yes | 00:17 |
| *** lblanchard has joined #tripleo | 00:21 | |
| *** Chaserjim has quit IRC | 00:21 | |
| *** Chaserjim has joined #tripleo | 00:25 | |
| *** Chaserjim has quit IRC | 00:33 | |
| *** Chaserjim has joined #tripleo | 00:37 | |
| *** Chaserjim has quit IRC | 00:41 | |
| *** Chaserjim has joined #tripleo | 00:42 | |
| *** dxiri has quit IRC | 00:43 | |
| *** Chaserjim has quit IRC | 00:58 | |
| *** Chaserjim has joined #tripleo | 00:59 | |
| *** dalvarez has quit IRC | 01:00 | |
| *** tobias-urdin has quit IRC | 01:00 | |
| *** Chaserjim has quit IRC | 01:04 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/diskimage-builder master: Add ubuntu-systemd-container operating-system element https://review.openstack.org/563748 | 01:04 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/diskimage-builder master: ubuntu-common: Update default DIB_RELEASE to bionic https://review.openstack.org/603940 | 01:08 |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 01:10 |
| *** vkmc|afk is now known as vkmc | 01:15 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Introduce alternative to get ip_addresses and server_names https://review.openstack.org/603291 | 01:23 |
| *** mrsoul has joined #tripleo | 01:25 | |
| *** mschuppert has quit IRC | 01:29 | |
| *** pvc has quit IRC | 01:32 | |
| *** cameron_chuang has quit IRC | 01:32 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: tripleo-container-image-prepare add --log-file https://review.openstack.org/603944 | 01:50 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: docker-puppet/podman: export DOCKER env to podman https://review.openstack.org/603945 | 01:52 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman https://review.openstack.org/600517 | 01:52 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman https://review.openstack.org/600517 | 01:52 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: Standardize path to prepare log file https://review.openstack.org/599273 | 01:53 |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 02:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| *** vinaykns has joined #tripleo | 02:20 | |
| vinaykns | anyone around..?? | 02:20 |
| vinaykns | I need a quick help...I'm trying to deploy tripleo quickstart but it is getting failed as it couldn't start virtual bmc, as the configured resource limit was exceeded...any workaround..? | 02:22 |
| vinaykns | http://pastebin.test.redhat.com/649195 | 02:23 |
| *** lblanchard has quit IRC | 02:28 | |
| *** sshnaidm has joined #tripleo | 02:42 | |
| *** sshnaidm is now known as sshnaidm|pto | 02:42 | |
| *** skramaja has joined #tripleo | 02:51 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | ooolpbot: Error: Could not gather data from Launchpad for bug #1792560 (https://launchpad.net/bugs/1792560). The error has been logged | 03:10 |
| *** slaweq has joined #tripleo | 03:11 | |
| *** slaweq has quit IRC | 03:16 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Support ARA statistics in InfluxDB for longest tasks https://review.openstack.org/580238 | 03:18 |
| *** radeks has joined #tripleo | 03:19 | |
| *** radeks has quit IRC | 03:27 | |
| openstackgerrit | David Vallee Delisle proposed openstack/tripleo-puppet-elements stable/queens: Remove Openstack and Ceph packages https://review.openstack.org/603965 | 03:28 |
| *** radeks has joined #tripleo | 03:41 | |
| *** dciabrin has quit IRC | 03:44 | |
| *** udesale has joined #tripleo | 03:53 | |
| *** Chaserjim has joined #tripleo | 03:55 | |
| *** jaganathan_ has quit IRC | 03:57 | |
| *** Chaserjim has quit IRC | 04:02 | |
| *** ykarel has joined #tripleo | 04:05 | |
| *** Chaserjim has joined #tripleo | 04:06 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 04:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| *** radeks has quit IRC | 04:11 | |
| *** ramishra has joined #tripleo | 04:15 | |
| *** Chaserjim has quit IRC | 04:15 | |
| *** yprokule has joined #tripleo | 04:25 | |
| *** med_ has joined #tripleo | 04:29 | |
| *** Petersingh has joined #tripleo | 04:36 | |
| *** Petersingh is now known as Petersingh|afk | 04:42 | |
| *** vinaykns has quit IRC | 04:43 | |
| *** cmyster has quit IRC | 04:49 | |
| *** med_ has quit IRC | 04:51 | |
| *** med_ has joined #tripleo | 04:52 | |
| *** Chaserjim has joined #tripleo | 04:52 | |
| *** Chaserjim has quit IRC | 04:57 | |
| *** Chaserjim has joined #tripleo | 04:59 | |
| *** janki has joined #tripleo | 05:02 | |
| *** jaosorior has quit IRC | 05:03 | |
| *** Petersingh|afk is now known as Petersingh | 05:09 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 05:10 |
| *** ratailor has joined #tripleo | 05:18 | |
| *** ykarel has quit IRC | 05:20 | |
| *** Chaserjim has quit IRC | 05:24 | |
| *** Petersingh has quit IRC | 05:26 | |
| *** Petersingh has joined #tripleo | 05:26 | |
| *** Petersingh is now known as Petersingh|away | 05:31 | |
| *** jtomasek has joined #tripleo | 05:32 | |
| *** Petersingh has joined #tripleo | 05:32 | |
| *** shyamb has joined #tripleo | 05:34 | |
| *** waleedm has joined #tripleo | 05:35 | |
| openstackgerrit | waleed mousa proposed openstack/tripleo-heat-templates master: Removing the deprecated environment files for hw offloading https://review.openstack.org/573102 | 05:35 |
| *** med_ has quit IRC | 05:38 | |
| *** shyamb has quit IRC | 05:39 | |
| *** aufi has joined #tripleo | 05:40 | |
| *** ykarel has joined #tripleo | 05:40 | |
| *** quique|rover|off is now known as quiquell|rover | 05:45 | |
| *** shyamb has joined #tripleo | 05:48 | |
| *** gfidente has joined #tripleo | 05:49 | |
| *** Chaserjim has joined #tripleo | 05:50 | |
| *** cylopez has joined #tripleo | 05:50 | |
| *** cylopez has left #tripleo | 05:51 | |
| *** chkumar|off is now known as chkumar|ruck | 05:53 | |
| quiquell|rover | chkumar|ruck: best here | 05:53 |
| chkumar|ruck | yes | 05:54 |
| quiquell|rover | chkumar|ruck: Paste it again | 05:54 |
| *** Chaserjim has quit IRC | 05:54 | |
| *** iranzo has joined #tripleo | 05:55 | |
| *** Chaserjim has joined #tripleo | 05:58 | |
| *** ksambor has joined #tripleo | 05:59 | |
| quiquell|rover | chkumar|ruck: what you were saying about timeouts ? | 06:00 |
| *** pvc has joined #tripleo | 06:02 | |
| pvc | hi | 06:02 |
| *** shyamb has quit IRC | 06:03 | |
| *** shyamb has joined #tripleo | 06:03 | |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-quickstart master: Disable manila tests on ocata and pike https://review.openstack.org/603915 | 06:04 |
| *** mschuppert has joined #tripleo | 06:07 | |
| *** kopecmartin has joined #tripleo | 06:08 | |
| *** Chaserjim has quit IRC | 06:08 | |
| *** Chaserjim has joined #tripleo | 06:09 | |
| gouthamr | chkumar|ruck quiquell|rover: ^ should fix scenario004 on pike, syntax issue with the last patchset | 06:09 |
| *** holser_ has joined #tripleo | 06:09 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 06:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| pvc | gouthamr | 06:12 |
| pvc | can i skip the node cleaning? | 06:12 |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/tripleo-quickstart master: ceilometer: Don't use deprecated metric https://review.openstack.org/603590 | 06:14 |
| mschuppert | quiquell|rover: the depends-on did not work. mwhahaha mentioned it will not work when the dependent patch is already merged. | 06:18 |
| chkumar|ruck | gouthamr: thanks for the fix, checking | 06:21 |
| *** agurenko has joined #tripleo | 06:22 | |
| quiquell|rover | mschuppert: didn't know... the only thing we can do is to force a version at yum install with a patch | 06:22 |
| quiquell|rover | mschuppert: or wait for promotions | 06:23 |
| mschuppert | quiquell|rover: when is the promition planned ? | 06:24 |
| quiquell|rover | mschuppert: It's for queens ? | 06:24 |
| *** jaosorior has joined #tripleo | 06:25 | |
| mschuppert | quiquell|rover: yes | 06:25 |
| quiquell|rover | mschuppert: The fix for queens promotion is in the gates now | 06:25 |
| mschuppert | quiquell|rover: ok, hope it passes :) | 06:25 |
| quiquell|rover | mschuppert: But we don't know what else can appear... as soon as it merge we will try to force run promotions | 06:25 |
| *** agurenko has quit IRC | 06:26 | |
| mschuppert | quiquell|rover: cool thx | 06:26 |
| *** tahini has quit IRC | 06:28 | |
| *** paramite has joined #tripleo | 06:28 | |
| quiquell|rover | chkumar|ruck: Did we have something new for queens ? | 06:29 |
| chkumar|ruck | gouthamr: can we make manila tempest tests passing aleast for pike | 06:30 |
| chkumar|ruck | quiquell|rover: till now no | 06:30 |
| gouthamr | chkumar|ruck: i was going to send you an email | 06:30 |
| gouthamr | need some help | 06:30 |
| chkumar|ruck | quiquell|rover: just one noop failure http://logs.openstack.org/45/560445/144/check/tripleo-ci-centos-7-containers-multinode-queens/dcb313c/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2018-09-19_23_11_16 | 06:30 |
| *** rdopiera has joined #tripleo | 06:30 | |
| quiquell|rover | chkumar|ruck: master fs002 has another "Connection is already closed." | 06:31 |
| *** mrunge has joined #tripleo | 06:31 | |
| gouthamr | chkumar|ruck: you're aware that manila-tempest-plugin is within openstack/manila repo in stable/pike - and this is a different package in rdo (python-manila-tests) | 06:31 |
| chkumar|ruck | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/8828b1f/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz#_2018-09-20_03_11_51 | 06:32 |
| quiquell|rover | chkumar|ruck: added to the bug | 06:32 |
| quiquell|rover | we need to merge this too | 06:32 |
| gouthamr | chkumar|ruck: backporting a huge change to fix manila-tempest-plugin in stable/pike is a sketchy effort | 06:32 |
| chkumar|ruck | gouthamr: ok, then let's keep it in skip phase | 06:33 |
| *** mrunge_ has quit IRC | 06:33 | |
| gouthamr | chkumar|ruck: i wanted to check with you, isn't it sane to spin a rdo-pike version of python2-manila-tests-tempest | 06:33 |
| chkumar|ruck | quiquell|rover: we cannot create a new branch for manil-tempest-plugin for pike, it is too late | 06:33 |
| chkumar|ruck | sorry | 06:34 |
| chkumar|ruck | gouthamr: ^^ | 06:34 |
| chkumar|ruck | gouthamr: and pike is going to be EOL soon after 6 months from now | 06:34 |
| *** sanjayu_ has joined #tripleo | 06:35 | |
| gouthamr | chkumar|ruck: oh, too late to introduce new packages to pike? or something specific with manila-tempest-plugin? | 06:35 |
| chkumar|ruck | quiquell|rover: yes it is too late to bring a new package to pike | 06:35 |
| chkumar|ruck | which is specific to manila-tempest-plugin or any other project | 06:36 |
| chkumar|ruck | gouthamr: downstream will also not like that it will requires lots of paper work also | 06:36 |
| *** quiquell|rover is now known as quique|rover|brb | 06:36 | |
| gouthamr | chkumar|ruck: hmmm, makes sense... We'll attempt the "backport" route | 06:37 |
| chkumar|ruck | gouthamr: so best case solution is to backport that patch there or better , it is a kind of risky if it works, it will be win win situation | 06:37 |
| gouthamr | chkumar|ruck: and fix python-manila-tests | 06:37 |
| *** jfrancoa has joined #tripleo | 06:37 | |
| chkumar|ruck | gouthamr: on package side, we donot need any package fix for manila-tests | 06:37 |
| gouthamr | chkumar|ruck: thanks for the feedback (arxcruz: fyi :) ^^) | 06:37 |
| gouthamr | chkumar|ruck: ack | 06:38 |
| *** saneax has quit IRC | 06:38 | |
| gouthamr | s/fyi/i tried :) | 06:38 |
| chkumar|ruck | gouthamr: any change in manila intree tempest plugin will directly come here https://github.com/rdo-packages/manila-distgit/blob/pike-rdo/openstack-manila.spec#L372 through usual process | 06:39 |
| gouthamr | chkumar|ruck: we're on the original plan then; let's merge the test skips, fix promotions and then i'll ensure we have the fix in python-manila-tests and re-enable the tests | 06:39 |
| chkumar|ruck | gouthamr: once the backport is up, we will take test it in rdo then merge upstream | 06:39 |
| chkumar|ruck | gouthamr: yup | 06:40 |
| gouthamr | chkumar|ruck: it is up: https://review.openstack.org/#/c/603171/ - will alert when we merge it | 06:40 |
| *** pvc has left #tripleo | 06:42 | |
| *** pvc has joined #tripleo | 06:42 | |
| openstackgerrit | Luca Miccini proposed openstack/puppet-tripleo master: This introduces additional mariadb tuning options, namely: https://review.openstack.org/603996 | 06:42 |
| pvc | is there a way that i can skip this step ipmitool chassis bootdev pxe | 06:44 |
| openstackgerrit | Luca Miccini proposed openstack/puppet-tripleo master: Add additional tuning options for mariadb/galera https://review.openstack.org/603996 | 06:45 |
| openstackgerrit | Luca Miccini proposed openstack/puppet-tripleo master: Add additional tuning options for mariadb/galera https://review.openstack.org/603996 | 06:46 |
| *** gkadam has joined #tripleo | 06:46 | |
| *** slaweq has joined #tripleo | 06:47 | |
| openstackgerrit | Luca Miccini proposed openstack/puppet-tripleo master: Add additional tuning options for mariadb/galera https://review.openstack.org/603996 | 06:48 |
| chkumar|ruck | quique|rover|brb: https://bugs.launchpad.net/tripleo/+bug/1793452 there is some issue with nova compute | 06:51 |
| openstack | Launchpad bug 1793452 in tripleo "[master] fs002 periodic job showing connection closed during overcloud prep image step" [Critical,Triaged] | 06:51 |
| jaosorior | pvc: your questions seem quite ironic specific. You might want to ask in #openstack-ironic as wel. | 06:51 |
| chkumar|ruck | added the logs there | 06:51 |
| *** sanjayu_ has quit IRC | 06:51 | |
| chkumar|ruck | quique|rover|brb: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/8828b1f/logs/undercloud/var/log/containers/nova/nova-compute.log.txt.gz#_2018-09-20_01_52_33_388 | 06:51 |
| *** paramite has quit IRC | 06:51 | |
| *** paramite has joined #tripleo | 06:52 | |
| chkumar|ruck | ykarel: where can i find yum update log for this job http://logs.openstack.org/45/560445/144/check/tripleo-ci-centos-7-containers-multinode-queens/dcb313c/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2018-09-19_23_11_16 ? | 06:52 |
| *** kopecmartin has quit IRC | 06:54 | |
| *** shyamb has quit IRC | 06:55 | |
| chkumar|ruck | gouthamr: I am testing your patch here with rdo periodic job https://review.rdoproject.org/r/#/c/13943/ | 07:00 |
| *** rcernin has quit IRC | 07:02 | |
| ykarel | chkumar|ruck, that might not work ^^ patch is in stable/pike, so if it fails good to check if correct version is instaleld | 07:06 |
| *** shyamb has joined #tripleo | 07:08 | |
| chkumar|ruck | ykarel: ack | 07:08 |
| *** Chaserjim has quit IRC | 07:09 | |
| *** quique|rover|brb is now known as quiquell|rover | 07:09 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 07:10 |
| *** pvc has quit IRC | 07:11 | |
| *** Petersingh is now known as Petersingh|afk | 07:11 | |
| quiquell|rover | chkumar|ruck: This is just a checking that it's not found, later on it's created | 07:12 |
| quiquell|rover | chkumar|ruck: Also it's at the beginning | 07:12 |
| chkumar|ruck | quiquell|rover: so there might be another reason forgetting connection closed then | 07:13 |
| *** ykarel is now known as ykarel|lunch | 07:13 | |
| quiquell|rover | chkumar|ruck: checking working one | 07:14 |
| quiquell|rover | It's a zaqar connection | 07:14 |
| *** jtomasek has quit IRC | 07:15 | |
| *** Chaserjim has joined #tripleo | 07:15 | |
| quiquell|rover | chkumar|ruck: It's same at workgin jobs https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/69856eb/logs/undercloud/var/log/containers/nova/nova-compute.log.txt.gz?level=ERROR | 07:15 |
| *** Petersingh|afk is now known as Petersingh | 07:15 | |
| quiquell|rover | chkumar|ruck: I would say to mark this bug as duplicated of the other about showing connections, and add failing jobs there | 07:16 |
| chkumar|ruck | quiquell|rover: ack | 07:16 |
| arxcruz | gouthamr: ack | 07:21 |
| quiquell|rover | marios: Can we +w this https://review.openstack.org/#/c/603802/ ? | 07:23 |
| quiquell|rover | marios: is the guard for break connection and we are suffering it at master/rocky promotions :-( | 07:23 |
| quiquell|rover | jaosorior: ^ | 07:24 |
| quiquell|rover | ssbarnea: Do you remember why do we fail here ? https://ci.centos.org/job/tripleo-quickstart-gate-master-delorean-quick-basic/6964/console | 07:27 |
| openstackgerrit | Luca Miccini proposed openstack/puppet-tripleo master: Add additional tuning options for mariadb/galera https://review.openstack.org/603996 | 07:27 |
| chkumar|ruck | quiquell|rover: I think https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/8a335fb/logs/undercloud/home/zuul/tempest.log.txt.gz#_2018-09-20_05_03_31 telemetry tests failure is causing the 503 errors for neutron tests failure | 07:28 |
| chkumar|ruck | quiquell|rover: but I need to cross check with telemetry fixes | 07:28 |
| ssbarnea | quiquell|rover: yep, it was because code with fix was not promoted. | 07:28 |
| openstackgerrit | Luca Miccini proposed openstack/puppet-tripleo master: Add additional tuning options for mariadb/galera https://review.openstack.org/603996 | 07:29 |
| *** Chaserjim has quit IRC | 07:29 | |
| quiquell|rover | ssbarnea: Do you have the review ? | 07:29 |
| quiquell|rover | chkumar|ruck: ack | 07:29 |
| chkumar|ruck | no no my assumption is wrong, telemtry is working fine, something else is wrong | 07:29 |
| *** assassin has joined #tripleo | 07:31 | |
| *** bogdando has joined #tripleo | 07:32 | |
| *** two_oes has joined #tripleo | 07:33 | |
| openstackgerrit | Martin André proposed openstack-infra/tripleo-ci master: Remove dead code https://review.openstack.org/578720 | 07:33 |
| jaosorior | mandre: yay for removing code :D | 07:34 |
| quiquell|rover | jfrancoa: Can you help merge this https://review.openstack.org/#/c/588475/ ? | 07:34 |
| quiquell|rover | jfrancoa: It's a script to compare playbook executioons between reviews | 07:34 |
| quiquell|rover | jaosorior: ` | 07:34 |
| quiquell|rover | jaosorior: ^ | 07:34 |
| *** florianf|afk is now known as florianf | 07:35 | |
| mandre | jaosorior: haha, definitely one of the most enjoyable part of my job :) | 07:36 |
| quiquell|rover | mandre: Just a new tool to debug CI issues https://review.openstack.org/#/c/588475/ | 07:36 |
| quiquell|rover | mandre: new script does not affect workflow | 07:36 |
| *** Chaserjim has joined #tripleo | 07:36 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Fix delorean-deps url for releases >= rocky https://review.openstack.org/596618 | 07:36 |
| mandre | quiquell|rover: oh cool, do you have an example output of your tool? | 07:38 |
| *** amoralej|off is now known as amoralej | 07:39 | |
| *** shyamb has quit IRC | 07:41 | |
| *** shyamb has joined #tripleo | 07:41 | |
| quiquell|rover | mandre: Running ./compare-reviews.py 597303 589448 it returns https://paste.fedoraproject.org/paste/8BeuuZCaa-2MUY3DLvHDHQ | 07:44 |
| mandre | quiquell|rover: I've gave it a try, it's pretty nice, I'll try to remember to use it next time I'll be debugging CI | 07:45 |
| quiquell|rover | mandre: At least let get it merged, was super useful for us refactoring CI last sprints, to ensure we were not screwing up ansible playbook calls | 07:45 |
| quiquell|rover | mandre: But you can compare all sort of logs.o.o files between reviews in the last check pipeline | 07:46 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Allow a containerized logrotate to access docker https://review.openstack.org/596274 | 07:46 |
| quiquell|rover | jaosorior: You are my hero today :-) | 07:46 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-common stable/queens: Add container runtime packages for cron image https://review.openstack.org/601558 | 07:46 |
| mandre | quiquell|rover: yeah it can highlight all sorts of small diffs you might otherwise miss | 07:46 |
| *** jpena|off is now known as jpena | 07:47 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-common stable/rocky: Add container runtime packages for cron image https://review.openstack.org/604006 | 07:47 |
| *** jpich has joined #tripleo | 07:48 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Allow a containerized logrotate to access docker https://review.openstack.org/596274 | 07:49 |
| *** dciabrin has joined #tripleo | 07:49 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Fix used paths to match custom working dir https://review.openstack.org/594055 | 07:50 |
| *** mrunge has quit IRC | 07:52 | |
| gfidente | marios can you remove -2 from https://review.openstack.org/#/c/603369/ ? | 07:52 |
| *** mrunge has joined #tripleo | 07:53 | |
| *** Chaserjim has quit IRC | 07:54 | |
| *** Petersingh is now known as Petersingh|lunch | 07:55 | |
| marios | gfidente: yes sec | 07:56 |
| marios | gfidente: well no since master/rocky didn't merge | 07:56 |
| marios | gfidente: https://review.openstack.org/#/q/c7945101b360c01d988c5e991926389037a8ff7e ? | 07:56 |
| marios | gfidente: removed | 07:57 |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates master: WIP-Telemetry Framework https://review.openstack.org/603137 | 07:58 |
| *** shyamb has quit IRC | 07:59 | |
| *** boazel_ has joined #tripleo | 08:01 | |
| *** jfrancoa has quit IRC | 08:02 | |
| *** Chaserjim has joined #tripleo | 08:02 | |
| *** boazel has quit IRC | 08:03 | |
| openstackgerrit | Luca Miccini proposed openstack/puppet-tripleo master: Add additional tuning options for mariadb/galera https://review.openstack.org/603996 | 08:04 |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 08:10 |
| *** jaosorior has quit IRC | 08:12 | |
| *** janki has quit IRC | 08:13 | |
| *** akrivoka has joined #tripleo | 08:15 | |
| *** jfrancoa has joined #tripleo | 08:18 | |
| *** leanderthal has joined #tripleo | 08:19 | |
| *** ykarel|lunch is now known as ykarel | 08:20 | |
| *** Chaserjim has quit IRC | 08:20 | |
| *** janki has joined #tripleo | 08:20 | |
| *** tosky has joined #tripleo | 08:21 | |
| ykarel | bogdando, i remember u mentioned some issues in fedora 28 related to virtualenv, is this the same mentioned here:- https://review.openstack.org/#/c/591652/13/roles/build-test-packages/tasks/main.yml@13 | 08:22 |
| bogdando | ykarel: that was the issue with the preinstalled package in OS infra image | 08:23 |
| ykarel | bogdando, ack, and what was the issue? | 08:23 |
| ykarel | if it's already installed then that install should just skip in general scenario | 08:24 |
| ykarel | i think it might be installed from pip and added to dnf exclude, do u have some env where u can check that | 08:25 |
| *** jchhatbar has joined #tripleo | 08:27 | |
| *** janki has quit IRC | 08:28 | |
| bogdando | ykarel: not really for now | 08:28 |
| bogdando | I swithched to the cloud image w/o anything preinstalled | 08:29 |
| bogdando | and running it on libvirt atm | 08:29 |
| *** jchhatba_ has joined #tripleo | 08:29 | |
| *** jchhatbar has quit IRC | 08:32 | |
| *** shyamb has joined #tripleo | 08:34 | |
| *** Aelia has joined #tripleo | 08:34 | |
| chkumar|ruck | arxcruz: Hello | 08:35 |
| arxcruz | chkumar|ruck: hey | 08:35 |
| chkumar|ruck | arxcruz: There is a problem with validate-tempest role when we use tempest format container and tempest tests fails | 08:35 |
| chkumar|ruck | it fails to generate html report | 08:35 |
| chkumar|ruck | for example https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/8a335fb/logs/undercloud/home/zuul/tempest.log.txt.gz | 08:36 |
| chkumar|ruck | gouthamr: Hello | 08:38 |
| chkumar|ruck | gouthamr: There is some problem with manila tempest tests from the patch which is backported https://logs.rdoproject.org/43/13943/36/check/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-pike/00693ec/logs/tempest.html.gz | 08:38 |
| chkumar|ruck | gouthamr: might be it requires some changes on microversion side not sure | 08:39 |
| *** kopecmartin has joined #tripleo | 08:41 | |
| *** Petersingh|lunch is now known as Petersingh | 08:43 | |
| *** Chaserjim has joined #tripleo | 08:44 | |
| arxcruz | chkumar|ruck: i'm running reproducer to debug | 08:46 |
| arxcruz | chkumar|ruck: have a lp? | 08:46 |
| ykarel | chkumar|ruck, did u checked what i asked earlier about manila stable/pike | 08:47 |
| ykarel | it failed even with the manila stable/pike patch installed | 08:47 |
| ykarel | ? | 08:47 |
| jistr | quiquell|rover, ykarel: just checked what's the latest issue in https://review.openstack.org/#/c/590774/ and commented -- the upgrade to master doesn't seem to be upgrade to latest master | 08:54 |
| jistr | quiquell|rover, ykarel: so i guess we're upgrading to current-tripleo snapshot containers instead of containers with latest content? | 08:55 |
| ykarel | jistr, not checked that for a while, checking what's the issue | 08:55 |
| jistr | it's seems like it's not running with https://review.openstack.org/#/c/602550/ even though the patch has merged to master before the job was triggered | 08:57 |
| *** jaosorior has joined #tripleo | 08:58 | |
| ykarel | chkumar|ruck, openstack-manila.noarch 1:5.0.2-0.20180917234258.45bc48d.el7 @delorean is installed but not from patch | 08:59 |
| *** Chaserjim has quit IRC | 09:00 | |
| *** Petersingh is now known as Petersingh|mtg | 09:04 | |
| chkumar|ruck | ykarel: umm | 09:04 |
| *** Chaserjim has joined #tripleo | 09:04 | |
| chkumar|ruck | ykarel: looking | 09:07 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 09:10 |
| chkumar|ruck | quiquell|rover: opened the bug for neutron tests failure on rocky https://bugs.launchpad.net/tripleo/+bug/1793482 | 09:11 |
| openstack | Launchpad bug 1793482 in tripleo "[Rocky] fs020 periodic job neutron tempest tests are failing by giving 503 Service Unavailable" [Critical,Triaged] | 09:11 |
| ykarel | jistr, i think earlier also it was not testing https://review.openstack.org/#/c/602550/ as there was no Depends-on on it and my dlrn build patch | 09:12 |
| ykarel | jistr, to test the merged tripleo patches, containers should be updated with delorean-current repo | 09:13 |
| *** Chaserjim has quit IRC | 09:13 | |
| ykarel | if that's not done, then that issue is expected until the containers are promoted | 09:13 |
| jistr | ykarel: yea, i guess we're not doing that in the undercloud upgrade job | 09:13 |
| *** Chaserjim has joined #tripleo | 09:13 | |
| jistr | so it is kinda expected, but it's also bad state :) if we ever have to do some fix to master to fix a broken UC upgrade job, we'd have to wait for promote until the fix takes effect | 09:14 |
| jistr | that suggests the UC upgrade job probably can't go back to voting until we fix this ^ | 09:15 |
| jistr | FYI EmilienM mwhahaha ^ | 09:15 |
| chkumar|ruck | ykarel: https://logs.rdoproject.org/43/13943/36/check/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-pike/00693ec/logs/undercloud/var/log/extra/ where is package build log? | 09:15 |
| ykarel | chkumar|ruck, that's not even build i think as that was for stable/pike | 09:16 |
| *** salmankhan has joined #tripleo | 09:16 | |
| chkumar|ruck | ykarel: it was for stable/pike, since package built is not done, so that chnage is not picked | 09:18 |
| chkumar|ruck | slaweq: Hello | 09:21 |
| chkumar|ruck | slaweq: please have a look at this bug https://bugs.launchpad.net/tripleo/+bug/1793482 when you are free, it is a rocky promotion block | 09:21 |
| openstack | Launchpad bug 1793482 in tripleo "[Rocky] fs020 periodic job neutron tempest tests are failing by giving 503 Service Unavailable" [Critical,Triaged] | 09:21 |
| quiquell|rover | chkumar|ruck: ack, mtg now | 09:22 |
| ykarel | chkumar|ruck, that's picked, u need to dig further why it's not build, probably because zuul.branch is not equal to stable/pike | 09:23 |
| rascasoft | mwhahaha, slagle, quiquell|rover I'm still not able to reproduce the problem because deployments are failing during undercloud install, because of a ntp issue | 09:24 |
| rascasoft | mwhahaha, slagle, quiquell|rover specifically "no server suitable for synchronization found" , TASK [Ensure system is NTP time synced] | 09:25 |
| *** Chaserjim has quit IRC | 09:25 | |
| ykarel | rascasoft, did u tried changing the ntp server, u just need to pass the correct ntp server | 09:26 |
| rascasoft | ykarel, well at the moment the configure ntp server IS working, and so it IS correct, in addition to that job is the same since always, so I didn't change anything in that sense | 09:27 |
| rascasoft | ykarel, is there any specific way to pass ntp to the openstack undercloud install command? | 09:28 |
| ykarel | i think in undercloud.conf | 09:28 |
| ykarel | see undercloud_ntp_servers in undercloud.conf | 09:28 |
| ykarel | and try: sudo ntpdate -u <ntpserver>, | 09:29 |
| rascasoft | ykarel, yeah I think I'm hitting a quickstart parameter issue here, so maybe I know what I have to change | 09:29 |
| ykarel | undercloud_undercloud_ntp_servers | 09:29 |
| ykarel | ^^ is quickstart param | 09:30 |
| rascasoft | ykarel, I'm surprised by the way that even if I have a specific ntp already configured on the machine it tries to reconfigure it in any case | 09:30 |
| rascasoft | ykarel, I would expect that if the parameter is not overridden then nothing changes | 09:30 |
| *** jchhatba_ has quit IRC | 09:31 | |
| *** janki has joined #tripleo | 09:31 | |
| ykarel | what u have in undercloud.conf, i guess it's pool.ntp.org | 09:32 |
| rascasoft | ykarel, nope, it is empty: | 09:32 |
| rascasoft | # List of ntp servers to use. (list value) | 09:32 |
| rascasoft | #undercloud_ntp_servers = | 09:32 |
| ykarel | hmm correct, i looked wrong | 09:32 |
| ykarel | that is only set when undercloud_undercloud_ntp_servers is defined | 09:33 |
| ykarel | http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/undercloud-deploy/templates/undercloud.conf.j2#n96 | 09:33 |
| ykarel | so it looks like it tries to run ntpdate -u pool.ntp.org as that's default for NtpServer | 09:35 |
| jistr | i hit the same problem recently, indeed setting undercloud_undercloud_ntp_servers properly fixed it | 09:35 |
| *** etingof has quit IRC | 09:35 | |
| * jistr grateful for the NTP sync check failing early. NTP desync has been cause of insane issues. 100s of person-hours sank into debugging of that | 09:36 | |
| *** derekh has joined #tripleo | 09:37 | |
| rascasoft | jistr, ykarel, do you think this is worth a bug or ntp MUST be set and so the error and the behavior are correct? | 09:37 |
| *** moshele has joined #tripleo | 09:38 | |
| *** Chaserjim has joined #tripleo | 09:39 | |
| ykarel | rascasoft, i think there is already a bug on it, and discussion was done there about it's existance and behavior | 09:39 |
| ykarel | rascasoft, yes as what jistr said that was the rational around having a strict failure | 09:40 |
| jistr | i think wanting to leave the UC nameserver alone kinda makes sense, but it's not critical issue (you can always set it to the same server you're already using), and at the same time it makes things much easier when we can just assume that the t-h-t is in control of NTP server setting always. Easier to do that strict check too. | 09:40 |
| rascasoft | ykarel, jistr, fair enough, will (re)add that parameter | 09:40 |
| *** sshnaidm|pto has quit IRC | 09:41 | |
| rascasoft | jistr, why are you talking about nameserver? | 09:41 |
| jistr | rascasoft: mental typo, i meant NTP :) | 09:41 |
| rascasoft | ok ok | 09:42 |
| rascasoft | :) | 09:42 |
| chkumar|ruck | ykarel: https://logs.rdoproject.org/43/13943/36/check/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-pike/00693ec/logs/ara_oooq/result/066281ca-b6fe-4174-8c4c-2e05cd48a12b/ | 09:42 |
| chkumar|ruck | because of this changes are not picked | 09:42 |
| chkumar|ruck | ykarel: I need to take a look zuul_deps.py code for the same | 09:43 |
| *** dtantsur|afk is now known as dtantsur | 09:43 | |
| rascasoft | jistr, ok BUT, to me: if we want tht to manage it then validation on undercloud.conf, if the parameter is not set, should fail. Otherwise if we want to leave the ability to the user to set it, then if it is not valued in the conf, everything should remain the same, not touched, not pointing any ntp server by default | 09:44 |
| ykarel | chkumar|ruck, afaik that list is generated on the fly base on ZUUL_CHANGES, so don't think u need to go to zuul code | 09:44 |
| ykarel | some jobs configuration can fix that, but i can't check right now | 09:44 |
| ykarel | u can try to fix , if u still see issue, let me know i can look later | 09:45 |
| quiquell|rover | chkumar|ruck, ykarel: What's the issues with ZUUL_CHANGES I have some problems yessterday | 09:45 |
| *** moguimar has joined #tripleo | 09:45 | |
| chkumar|ruck | ykarel: na na that changes comes https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/build-test-packages/library/zuul_deps.py#L44 | 09:45 |
| *** janki has quit IRC | 09:45 | |
| chkumar|ruck | ykarel: "zuul_changes": "testproject:master:refs/changes/42/13942/1^openstack/manila:stable/pike:refs/changes/71/603171/3^testproject:master:refs/changes/43/13943/36" | 09:46 |
| chkumar|ruck | is passed as argument | 09:46 |
| ykarel | yes that what i said | 09:46 |
| ykarel | it needs zuul.branch | 09:46 |
| chkumar|ruck | { | 09:46 |
| ykarel | which might be differnt | 09:46 |
| chkumar|ruck | "artg_change_list": [ | 09:46 |
| chkumar|ruck | { | 09:46 |
| chkumar|ruck | "branch": "master", | 09:46 |
| chkumar|ruck | "host": "https://git.openstack.org", | 09:46 |
| chkumar|ruck | "project": "testproject", | 09:46 |
| chkumar|ruck | "refspec": "refs/changes/43/13943/36" | 09:46 |
| chkumar|ruck | } | 09:46 |
| chkumar|ruck | ] | 09:46 |
| chkumar|ruck | } | 09:46 |
| chkumar|ruck | this is what received and there is not test project in rdo distgit so no dlrn build | 09:47 |
| chkumar|ruck | quiquell|rover: right zuul chanes is not pciked by build test packages | 09:47 |
| jistr | rascasoft: yea meakes sense, i'm not sure how much it's worth fixing though (?). In any case i think if the undercloud does not have any NTP sync set up, we should simply fail, that scenario should be unsupported. In the end people always hit some issue when they didn't set up NTP, we spent 2*10 hours debugging it together what's happening, and only then we realize that the issue was caused by missing NTP | 09:48 |
| jistr | and wrong times. Historically missing NTP has been such a waste of time that i've developed alergy to non-synced computers now :D | 09:48 |
| chkumar|ruck | ykarel: I am digging, if found issue, will propose the fix | 09:48 |
| ykarel | ack | 09:48 |
| quiquell|rover | chkumar|ruck: This is regarding same change diffferent branches ? | 09:49 |
| ykarel | yes | 09:49 |
| rascasoft | jistr, but wait, my problem in this specific case was that everything was setup on my side in terms of ntp (previous undercloud installation) and then it failed because value was not set inside the undercloud.conf | 09:49 |
| quiquell|rover | Looks like we have to use the Depends-On with absolute URL | 09:49 |
| quiquell|rover | they told me yesterday at #zuul channel | 09:50 |
| quiquell|rover | using Change-Id is not a good idea | 09:50 |
| *** Petersingh|mtg is now known as Petersingh | 09:50 | |
| jistr | rascasoft: yea we should probably require that being set i guess, or autodetect the NTP server already in use, and then enforce that it's synced and if not, fail | 09:50 |
| rascasoft | jistr, ok, I'll open a tracker on this, maybe we can continue the discussion there | 09:51 |
| jistr | rascasoft: ack | 09:51 |
| *** janki has joined #tripleo | 09:51 | |
| *** Chaserjim has quit IRC | 09:52 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Make sure rhel-plugin-push.service is stopped after pacemaker stops https://review.openstack.org/602828 | 09:53 |
| *** jaosorior has quit IRC | 09:53 | |
| quiquell|rover | chkumar|ruck: fs019 fix failed at gates with timeout :-(((( | 09:56 |
| quiquell|rover | chkumar|ruck: Going to recheck and cry at my corner | 09:56 |
| chkumar|ruck | quiquell|rover: is it scenario 004 ? | 09:56 |
| quiquell|rover | chkumar|ruck: yep | 09:56 |
| quiquell|rover | chkumar|ruck: Wait you have a fix tor tempest for it ? | 09:56 |
| chkumar|ruck | quiquell|rover: need this https://review.openstack.org/603915 as ap | 09:57 |
| chkumar|ruck | it is waiting on gates | 09:57 |
| chkumar|ruck | quiquell|rover: good news is zuul is green on thus patch | 09:58 |
| chkumar|ruck | Any cores around , we need this patch https://review.openstack.org/#/c/603915/ to fix timedout issue in gates | 09:58 |
| chkumar|ruck | bogdando: mandre EmilienM panda ^^ | 09:59 |
| quiquell|rover | chkumar|ruck: this is for master too ? | 09:59 |
| *** shyamb has quit IRC | 09:59 | |
| *** jchhatbar has joined #tripleo | 10:00 | |
| chkumar|ruck | quiquell|rover: yup | 10:02 |
| quiquell|rover | ok | 10:02 |
| chkumar|ruck | it will fix manila issue till pike | 10:02 |
| *** janki has quit IRC | 10:03 | |
| quiquell|rover | marios: We need to remove timeouts from scenario004 at gates https://review.openstack.org/#/c/603915/ | 10:03 |
| marios | quiquell|rover: chkumar|ruck ack i remember this one (its why i was asking about pike in oooq... is the same issue for queens and pike? | 10:04 |
| *** pcaruana has joined #tripleo | 10:04 | |
| marios | chkumar|ruck: oh no, on queens it is timeout not failed tempest (here mean scen4 https://review.openstack.org/#/c/567224 ) | 10:05 |
| *** jtomasek has joined #tripleo | 10:09 | |
| quiquell|rover | marios: Looks like queens timeout is because temptest, is that right chkumar|ruck ? | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 10:10 |
| quiquell|rover | marios: skipping test suite test_create_get_delete_share removes the timeouts | 10:10 |
| quiquell|rover | jistr: What's the issue with https://review.openstack.org/#/c/590774 ? | 10:15 |
| *** shyamb has joined #tripleo | 10:17 | |
| slaweq | chkumar|ruck: hi, sorry. I will take a look at it now | 10:19 |
| openstackgerrit | Pranali Deore proposed openstack/tripleo-quickstart master: [DNM] For testing glance default glance-image-import.conf file https://review.openstack.org/604030 | 10:20 |
| *** thrash|g0ne is now known as thrash | 10:21 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: WIP: Don't use mixed_upgrade special release files https://review.openstack.org/604032 | 10:26 |
| openstackgerrit | Pranali Deore proposed openstack/tripleo-quickstart master: [DNM] For testing glance default glance-image-import.conf file https://review.openstack.org/604030 | 10:26 |
| *** morazi has quit IRC | 10:26 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: WIP: Don't use mixed_upgrade special release files https://review.openstack.org/604032 | 10:28 |
| openstackgerrit | Juan Badia Payno proposed openstack/puppet-tripleo master: Fix fluentd multifile problem. https://review.openstack.org/601634 | 10:29 |
| *** kopecmartin has quit IRC | 10:30 | |
| quiquell|rover | ykarel: Have you found this issue ? https://bugs.launchpad.net/tripleo/+bug/1793468 ? | 10:31 |
| openstack | Launchpad bug 1793468 in tripleo "master gates fails at overcloud prep containers: ERROR running yum update in container" [High,Triaged] - Assigned to Quique Llorente (quiquell) | 10:31 |
| ykarel | quiquell|rover, nope i didn't looked at that, is it consistently failing? | 10:33 |
| quiquell|rover | ykarel: Nah it's intermitent... | 10:33 |
| *** jfrancoa has quit IRC | 10:33 | |
| ykarel | quiquell|rover, ack | 10:34 |
| *** jfrancoa has joined #tripleo | 10:35 | |
| slaweq | chkumar|ruck: do You know why this SIGHUP was send to neutron: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/8a335fb/logs/overcloud-controller-0/var/log/containers/neutron/server.log.txt.gz#_2018-09-20_05_00_50_878 ? | 10:36 |
| ykarel | jistr, do u know why containers not updated in undercloud-upgrades job? | 10:37 |
| quiquell|rover | chkumar|ruck: overcloud-prep-images issue is new ? https://logs.rdoproject.org/24/567224/109/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens/239b2dd/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz | 10:38 |
| ykarel | jistr, i saw ur comment now: if we ever have to do some fix to master to fix a broken UC upgrade job, we'd have to wait for promote until the fix take, so asked | 10:39 |
| chkumar|ruck | slaweq: I have seen this issue previously | 10:39 |
| chkumar|ruck | sorry | 10:39 |
| chkumar|ruck | quiquell|rover: I have seen this issue previously | 10:39 |
| chkumar|ruck | quiquell|rover: it was coming before running cleanup script on rdo cloud | 10:40 |
| chkumar|ruck | let me find the bug | 10:40 |
| quiquell|rover | chkumar|ruck: if it's featureset001 on queens then is blocking-promotion issue | 10:40 |
| quiquell|rover | chkumar|ruck: It's continuously failing :-( | 10:41 |
| chkumar|ruck | quiquell|rover: https://bugs.launchpad.net/tripleo/+bug/1792872 | 10:41 |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Incomplete] - Assigned to Quique Llorente (quiquell) | 10:41 |
| quiquell|rover | chkumar|ruck: this is different... I think | 10:42 |
| chkumar|ruck | quiquell|rover: https://bugs.launchpad.net/tripleo/+bug/1792872/comments/2 as per weshay_pto he seen it previous | 10:43 |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Incomplete] - Assigned to Quique Llorente (quiquell) | 10:43 |
| chkumar|ruck | slaweq: no idea | 10:44 |
| chkumar|ruck | marios: please have a look at slaweq query | 10:44 |
| quiquell|rover | chkumar|ruck hummm yep is this... maybe it's related to the 5 stacks that are DELETE_IN_PROGRESS | 10:47 |
| chkumar|ruck | quiquell|rover: need to check with kforde may be | 10:47 |
| quiquell|rover | chkumar|ruck: on it | 10:48 |
| jistr | quiquell|rover: re what's the issue with https://review.openstack.org/#/c/590774 -- see my last comment. I think we're using tripleo-current container images from master but we're not updating them to dlrn-current probably, which is wrong. | 10:49 |
| jistr | ykarel: i haven't looked at the details yet, doing a few things in parallel... | 10:49 |
| *** sanjayu_ has joined #tripleo | 10:49 | |
| ykarel | jistr, ack | 10:50 |
| *** abishop has joined #tripleo | 10:51 | |
| quiquell|rover | jistr: what job ? | 10:52 |
| jistr | quiquell|rover: containerized-undercloud-upgrades | 10:53 |
| slaweq | chkumar|ruck: do You know if there is bugzilla for that issue already? | 10:53 |
| slaweq | if not, I will open one and raise this issue on our today squad call | 10:53 |
| rascasoft | jistr, ykarel https://bugs.launchpad.net/tripleo/+bug/1793500 | 10:54 |
| openstack | Launchpad bug 1793500 in tripleo "[tripleo] If undercloud_ntp_servers is not declared then NTP should not be configured" [Undecided,Triaged] | 10:54 |
| chkumar|ruck | slaweq: nope, we are seeing this issue from yesterday | 10:54 |
| slaweq | chkumar|ruck: ok, I will open bz for it then | 10:55 |
| chkumar|ruck | slaweq: thanks for looking into that | 10:56 |
| jchhatbar | Hi. Can I please get some reviews for https://review.openstack.org/#/c/596987/ | 10:56 |
| *** jchhatbar is now known as janki | 10:56 | |
| chkumar|ruck | slaweq: please update the lp also with your thoughts | 10:57 |
| *** jpena is now known as jpena|lunch | 10:57 | |
| *** jaosorior has joined #tripleo | 10:58 | |
| *** udesale has quit IRC | 10:58 | |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Set different branch in zuul_changes https://review.openstack.org/604035 | 10:59 |
| chkumar|ruck | ykarel: quiquell|rover ^^ it will fix the build test package zuul changes ignore issue | 11:00 |
| ykarel | chkumar|ruck, import pdb; pdb.set_trace() | 11:01 |
| ykarel | chkumar|ruck, and it looks like ur change is wrong seeing the continue logic | 11:01 |
| ykarel | so u should fix the way u using it | 11:01 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Set different branch in zuul_changes https://review.openstack.org/604035 | 11:01 |
| quiquell|rover | chkumar|ruck: what's the failing review ? | 11:02 |
| chkumar|ruck | ykarel: yup | 11:02 |
| ykarel | chkumar|ruck, and it's never good to mix branches as far as that logic is concerned | 11:02 |
| quiquell|rover | chkumar|ruck: point to full review url | 11:03 |
| ykarel | the patch just fixes the particular use case for u but could break the entire login | 11:03 |
| quiquell|rover | chkumar|ruck: so it has clear point | 11:03 |
| chkumar|ruck | quiquell|rover: https://review.openstack.org/604035 | 11:04 |
| chkumar|ruck | ykarel: what if we want to test multiple branches changes then | 11:04 |
| ykarel | chkumar|ruck, any example? | 11:04 |
| *** ansmith has quit IRC | 11:05 | |
| quiquell|rover | chkumar|ruck: I mean the review that was not getting the correct changes | 11:05 |
| openstackgerrit | Pranali Deore proposed openstack/tripleo-heat-templates master: [DNM] Pass image import conf file to command which runs glance server https://review.openstack.org/598560 | 11:06 |
| chkumar|ruck | quiquell|rover: https://review.rdoproject.org/r/#/c/13943/35 | 11:06 |
| quiquell|rover | chkumar|ruck: the Depends-On are all in the same branch master | 11:06 |
| quiquell|rover | chkumar|ruck: the issue is "This change depends on a change that failed to merge." | 11:07 |
| quiquell|rover | A wait ps35 | 11:07 |
| chkumar|ruck | ykarel: for example foo project has master (not other branches) branch but bar is consumed in that and tested at different branch level that time it is hard to test the change | 11:09 |
| ykarel | chkumar|ruck, any real example :) | 11:10 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 11:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| chkumar|ruck | ykarel: above review is real exmaple then currently tested in testproject | 11:10 |
| ykarel | chkumar|ruck, any case what i told earlier u need to adjust job definition | 11:11 |
| quiquell|rover | chkumar|ruck: ps35 has not reference to any branch apart from master ? | 11:11 |
| *** boazel_ has quit IRC | 11:11 | |
| quiquell|rover | chkumar|ruck: Do you have the specific patch set with another branch ? | 11:11 |
| ykarel | chkumar|ruck, there are ways to do that, like ZUUL_BRANCH, ZUUL_OVERRIDE_BRANCH etc | 11:11 |
| ykarel | so just need to set those correctly for ur use case | 11:11 |
| *** shyamb has quit IRC | 11:12 | |
| ykarel | but not change the logic of code | 11:12 |
| *** shyamb has joined #tripleo | 11:12 | |
| ykarel | chkumar|ruck, am i explaining correct? | 11:12 |
| quiquell|rover | build-test-package is ignoring other branchges than the one from the main chage by design | 11:12 |
| chkumar|ruck | ykarel: yes I need to figure out override | 11:13 |
| *** yprokule_ has joined #tripleo | 11:13 | |
| quiquell|rover | chkumar|ruck: You will need to have a change on THT on the specific branch to test it | 11:13 |
| ykarel | chkumar|ruck, yes i told u need to dig :) | 11:13 |
| chkumar|ruck | quiquell|rover: https://review.rdoproject.org/r/#/c/13943/36//COMMIT_MSG | 11:13 |
| chkumar|ruck | where it is getting ignored | 11:13 |
| quiquell|rover | chkumar|ruck: Ok the issue is we need noop thing for the different branches | 11:14 |
| quiquell|rover | chkumar|ruck: We are not going to change build-test-package only for that | 11:14 |
| *** pcaruana has quit IRC | 11:15 | |
| quiquell|rover | chkumar|ruck: as ykarel say, you can force the branch in the zuul config | 11:15 |
| chkumar|ruck | quiquell|rover: let me try that | 11:15 |
| *** etingof has joined #tripleo | 11:16 | |
| *** shyamb has quit IRC | 11:16 | |
| quiquell|rover | chkumar|ruck: We can also create a branch in test project | 11:16 |
| *** ratailor has quit IRC | 11:16 | |
| EmilienM | jistr: what's the context exactly? | 11:19 |
| slaweq | chkumar|ruck: I reported bz for this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1631335 | 11:19 |
| openstack | bugzilla.redhat.com bug 1631335 in openstack-neutron "WSGI workers not started again after SIGHUP signal" [Unspecified,New] - Assigned to amuller | 11:19 |
| slaweq | and I will discuss it today with our squad | 11:19 |
| *** pcaruana has joined #tripleo | 11:20 | |
| *** etingof has quit IRC | 11:20 | |
| chkumar|ruck | slaweq: thanks :-) | 11:20 |
| *** Petersingh is now known as Petersingh|afk | 11:21 | |
| *** etingof has joined #tripleo | 11:23 | |
| EmilienM | jaosorior, quiquell|rover, chkumar|ruck: the gate is 25h backed up; what's your plan guys? | 11:26 |
| chkumar|ruck | EmilienM: https://review.openstack.org/603915 this will solve scenario 004 timeout issue | 11:26 |
| jaosorior | damn | 11:27 |
| quiquell|rover | EmilienM: ^ this should help | 11:27 |
| jaosorior | that's quite far in the end of the queue | 11:27 |
| EmilienM | 603915 is at the bottom of gate | 11:27 |
| jaosorior | we need to start abandoning patches then | 11:27 |
| EmilienM | 603915 will land in 2 days probably | 11:27 |
| EmilienM | so no 603915 won't help guys | 11:27 |
| chkumar|ruck | and https://review.openstack.org/#/c/603802/ | 11:27 |
| EmilienM | yes I proposed it yesterday but alex said no | 11:27 |
| jaosorior | EmilienM: any reason he said no? | 11:28 |
| *** kopecmartin has joined #tripleo | 11:28 | |
| EmilienM | he said stuffs were merging | 11:28 |
| quiquell|rover | EmilienM: what do you mean ? | 11:28 |
| jaosorior | wel... stuff IS merging, it's just taking a VERY long time. | 11:28 |
| EmilienM | quiquell|rover: what do I mean by what? | 11:28 |
| jaosorior | and one of the reasons is the timeouts from scenario 004, right? | 11:29 |
| EmilienM | the zuul queue works from the top to the bottom | 11:29 |
| jaosorior | chkumar|ruck: | 11:29 |
| EmilienM | your patches are at bottom | 11:29 |
| EmilienM | and will never land because every time a patch on top fails (timeout), the whole queue resets | 11:29 |
| EmilienM | and it starts again | 11:29 |
| EmilienM | i'm in meeting all day almost, I have no time to look | 11:29 |
| quiquell|rover | EmilienM: to https://review.openstack.org/#/c/603802/ Alex said "stuff is merging" <- stuff is always merging | 11:29 |
| chkumar|ruck | jaosorior: full tempest tests getting executed in sceanrio 4 due to problem in test regex | 11:29 |
| EmilienM | but you should purge the gate | 11:29 |
| EmilienM | and land what you need first; | 11:29 |
| EmilienM | imho | 11:30 |
| * EmilienM disappears | 11:30 | |
| quiquell|rover | jaosorior: Do you know how to purge ? | 11:30 |
| jaosorior | quiquell|rover: I do, let me fetch some coffee frist :D | 11:30 |
| EmilienM | you need to abandon patches | 11:30 |
| quiquell|rover | jaosorior: Send me some | 11:30 |
| EmilienM | and to move them again, you need to restore | 11:30 |
| quiquell|rover | EmilienM: any script for that ? | 11:31 |
| EmilienM | no | 11:31 |
| EmilienM | your hands | 11:31 |
| EmilienM | I'm in a call now | 11:31 |
| quiquell|rover | EmilienM: ack | 11:31 |
| ssbarnea | does tripleo has a styleguide that covers ansible code? if so, where I can find it? i tried tripleo-docs and tripleo-incoubator but found no section on this yet. | 11:31 |
| *** boazel has joined #tripleo | 11:31 | |
| quiquell|rover | jaosorior: Maybe we have to send an email to openstack-dev about not gating shit | 11:31 |
| quiquell|rover | chkumar|ruck: jaosorior going to help purging queue, then we will try to merge first https://review.openstack.org/603915 | 11:32 |
| chkumar|ruck | sure | 11:32 |
| *** pcaruana has quit IRC | 11:32 | |
| quiquell|rover | then https://review.openstack.org/#/c/603323/ (If we are lucky with connection issues) | 11:34 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Improve output of Verify Sphinx build task https://review.openstack.org/600403 | 11:38 |
| *** pcaruana has joined #tripleo | 11:39 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Improve output of vxlan networking tasks https://review.openstack.org/600039 | 11:39 |
| openstackgerrit | Yael Azulay proposed openstack/puppet-tripleo master: Changes are needed for performance and stability at large intervals https://review.openstack.org/603183 | 11:40 |
| jaosorior | quiquell|rover, chkumar|ruck: So, what where the patches that are needed? | 11:43 |
| jistr | EmilienM: re context previously, it's containerized-undercloud-upgrades job. It's not upgrading to latest master, it's upgrading to promoted master. If we make a fix, we don't run with it in containerized undercloud upgrade job until promotion. | 11:45 |
| *** bfournie has quit IRC | 11:45 | |
| jistr | EmilienM: specifically, e.g. this patch was meant to fix R->master undercloud upgrade. https://review.openstack.org/#/c/602550/ | 11:46 |
| jistr | EmilienM: but even though it's merged, the job still doesn't have it, because it upgrades to promoted, not to master | 11:46 |
| quiquell|rover | jaosorior: first | 11:46 |
| quiquell|rover | https://review.openstack.org/#/c/603915/ | 11:46 |
| quiquell|rover | jaosorior: second | 11:47 |
| quiquell|rover | https://review.openstack.org/#/c/603323/ | 11:47 |
| quiquell|rover | chkumar|ruck: anything else ? | 11:47 |
| jistr | EmilienM: or at least we *think* it's upgrading to promoted, to be precise. It certainly looks like that. | 11:47 |
| ykarel | quiquell|rover, chkumar|ruck any idea what's the reason for growing zuul queue? i was trying to understand but no luck, the patches u shared seems to unblock pike/ocata promotions, are there too many patches being approved for pike/ocata? | 11:48 |
| ykarel | and due to those failures, zuul resets? | 11:48 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Only set ca_bundle for HAProxy if internal TLS enabled & fix mounts https://review.openstack.org/603390 | 11:49 |
| quiquell|rover | ykarel: the tempest patch disable a testsuite for queens/rock/master that reduces chances of timeouts | 11:49 |
| quiquell|rover | ykarel: we re having a lot of timeouts because of this | 11:49 |
| quiquell|rover | chkumar|ruck: ^ is that correct ? | 11:49 |
| jaosorior | quiquell|rover: waiting for chkumar|ruck before starting the purge | 11:49 |
| quiquell|rover | jaosorior: ack agree | 11:49 |
| *** morazi has joined #tripleo | 11:50 | |
| quiquell|rover | jaosorior: Don't know what to do with https://review.openstack.org/#/c/603802/ | 11:50 |
| quiquell|rover | jaosorior: "Add a guard..." | 11:50 |
| *** pcaruana has quit IRC | 11:50 | |
| jaosorior | quiquell|rover: if it doesn't help with the current zuul queue issue, I'll abandon it (temporarily) | 11:51 |
| chkumar|ruck | quiquell|rover: that is one of the reason and another is promotion | 11:51 |
| ykarel | quiquell|rover, u mean https://review.openstack.org/#/c/603915/ tempest patch? if yes it looks it was corret earlier | 11:51 |
| ykarel | only issue was for pike/ocata where it needs to be skipped | 11:51 |
| chkumar|ruck | ykarel: https://review.openstack.org/#/c/603915/ | 11:52 |
| quiquell|rover | ykarel: you are right, then I don't know | 11:52 |
| chkumar|ruck | ykarel: test regex was mis behaving | 11:52 |
| jaosorior | chkumar|ruck: I'll start purging patches, except for https://review.openstack.org/#/c/603915/ https://review.openstack.org/#/c/603323/ is that OK, or is there anything else needed? | 11:52 |
| ykarel | chkumar|ruck, how? | 11:52 |
| ykarel | chkumar|ruck, i think that was misbehaving only on ur patch | 11:53 |
| ykarel | not in generic | 11:53 |
| quiquell|rover | chkumar|ruck: ok with jaosorior keeping and purging ? | 11:54 |
| ykarel | jaosorior, i think it's good to know the reason what led to grow zuul queue, if u know that it's good to purge | 11:54 |
| ykarel | are that intermittent timeout or anything else | 11:54 |
| *** kopecmartin has left #tripleo | 11:54 | |
| ykarel | or is it due to delay in promotions | 11:54 |
| jaosorior | ykarel: as far as I've been told, it's the issue that quiquell|rover was talking about, the tempest regex being wrong. If that's not the reason, then yeah, we gotta figure it out. | 11:55 |
| chkumar|ruck | http://logs.openstack.org/22/601522/4/check/tripleo-ci-centos-7-scenario004-multinode-oooq-container/56e440c/logs/undercloud/home/zuul/tempest.log.txt.gz | 11:56 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Disable OVN related services in ODL env file https://review.openstack.org/603437 | 11:56 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Use ansible_host variable in wait_for task. https://review.openstack.org/603382 | 11:56 |
| chkumar|ruck | ykarel: here it is timing out multiple times | 11:56 |
| *** lblanchard has joined #tripleo | 11:56 | |
| *** shyamb has joined #tripleo | 11:57 | |
| ykarel | chkumar|ruck, that is ur patch | 11:57 |
| ykarel | and possibly patch has issues | 11:57 |
| ykarel | but that can't be reason for growing zuul queue | 11:57 |
| chkumar|ruck | ykarel: antoher is delayed promotion | 11:58 |
| *** Petersingh|afk is now known as Petersingh | 11:58 | |
| ykarel | chkumar|ruck, yes exactly, what issue is being caused by promotion delay, any link? | 11:58 |
| ykarel | like jobs timeout or anything else? | 11:58 |
| chkumar|ruck | ykarel: https://review.openstack.org/#/c/601522/ this was my patch where it seen multiple timedout | 11:58 |
| quiquell|rover | ykarel: timeout, produced by container images being updated | 11:58 |
| *** dprince has joined #tripleo | 11:59 | |
| quiquell|rover | ykarel: promotions help with that by pushing newer docker images | 11:59 |
| quiquell|rover | ykarel: so updates are not needed | 11:59 |
| ykarel | quiquell|rover, i know that part :), i just wanted to get the reasoning here, as earlier what was being said seemed wrong to me | 12:00 |
| quiquell|rover | ykarel: What I said before was wrong | 12:00 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates stable/rocky: Disable OVN related services in ODL env file https://review.openstack.org/604069 | 12:00 |
| *** kopecmartin has joined #tripleo | 12:00 | |
| quiquell|rover | ykarel: if statement was the opposite | 12:00 |
| quiquell|rover | chkumar|ruck: CIX | 12:01 |
| *** kopecmartin has quit IRC | 12:01 | |
| *** weshay_pto is now known as weshay | 12:01 | |
| chkumar|ruck | earlier undercloud container upgrade was failing and another was scenario004, in most case execute tempest timed out | 12:02 |
| quiquell|rover | chkumar|ruck: Timeout at tempest is because is the last stuff we do usually | 12:02 |
| quiquell|rover | chkumar|ruck: But timeout could be procued by oc/uc deploy time increasing | 12:03 |
| chkumar|ruck | or during deployt the overcloud as undercloud installation was taking too much | 12:03 |
| ykarel | quiquell|rover, chkumar|ruck iirc we ususally have alert for such issues | 12:05 |
| ykarel | and this helps in prioritizing reviews etc | 12:05 |
| ykarel | and it's always good to know the reason for what's wrong going on | 12:06 |
| ykarel | and i guess we also has some dashboard which can be used to know what's going on with the job in gate queue | 12:07 |
| *** kopecmartin has joined #tripleo | 12:07 | |
| openstackgerrit | Ana Krivokapic proposed openstack/tripleo-validations master: Remove a debug task https://review.openstack.org/604072 | 12:08 |
| EmilienM | jistr: sorry. Was in a call. Let me read backlog. | 12:09 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 12:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| EmilienM | jistr: ok so we want to change to ugrade to latest consistent master I guess? | 12:10 |
| *** aevoo has joined #tripleo | 12:10 | |
| EmilienM | jaosorior: back from meeting. Didn't you guys purge the gate? | 12:11 |
| *** aevoo is now known as aedc | 12:11 | |
| akrivoka | florianf: gchamoul: have you run into problem with running validations on recent master? they are failing for me when run via mistral: https://paste.fedoraproject.org/paste/SZZSMM7zOCW8NEs93m-8Tw | 12:12 |
| jistr | EmilienM: yea same thing the deploy jobs do. We need to run the modify-image Ansible role on the images we're upgrading to. | 12:12 |
| jaosorior | EmilienM: we didn't, cause we couldn't figure out which exactly are the patches needed to fix this issue. It seems (from what I understood), that still the root cause of this delay is not yet known. | 12:12 |
| akrivoka | florianf: gchamoul: works well when run directly with ansible-playbook though | 12:12 |
| EmilienM | jaosorior: then we should stop merging things now | 12:12 |
| akrivoka | florianf: gchamoul: just wondering if it's a known issue or should I open a bug for it | 12:12 |
| EmilienM | if the gate timeout it affects the whole openstack gate and other projects | 12:12 |
| quiquell|rover | EmilienM: How much affects timeouts not promoting for long time | 12:14 |
| quiquell|rover | ? | 12:14 |
| jaosorior | ykarel, chkumar|ruck, quiquell|rover: I'll start purging the gate now to release resources. From what I gathered there are no patches that will surely help this issue yet, right? | 12:14 |
| *** aedc is now known as aedc_ | 12:15 | |
| *** aedc_ is now known as aedc | 12:15 | |
| gchamoul | akrivoka: I've the same issue, that's the one I filed last week | 12:15 |
| quiquell|rover | jaosorior: We know for sure reviews needed to get promotions, and maybe with promotions we will reduce timeouts | 12:15 |
| ykarel | jaosorior, ok for clearing the queue | 12:15 |
| gchamoul | akrivoka, florianf: https://bugs.launchpad.net/tripleo/+bug/1792113 | 12:15 |
| openstack | Launchpad bug 1792113 in tripleo "All tripleo-validations fail while running through Mistral with a stderr "which: no tripleo-ansible-inventory in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)"" [High,New] | 12:15 |
| jaosorior | quiquell|rover: which are those reviews? | 12:15 |
| EmilienM | quiquell|rover: I don't understand the question | 12:15 |
| akrivoka | gchamoul: looks like the send_message task is failing, but I can't really find anything relevant in mistral logs | 12:15 |
| *** rlandy has joined #tripleo | 12:16 | |
| akrivoka | gchamoul: yes, that's exactly what I'm hitting | 12:17 |
| quiquell|rover | We have fixes for promotions: https://review.openstack.org/#/c/603323/, https://review.openstack.org/#/c/603915/ | 12:18 |
| quiquell|rover | Don't know if promoting will help with timeouts <- EmilienM this is my question | 12:19 |
| gchamoul | akrivoka: the title of the bug is not relevant, I didn't pay attention to the result message but the stderr | 12:19 |
| *** bfournie has joined #tripleo | 12:19 | |
| weshay | jaosorior, can you please update the channel to say no workflow +1 atm | 12:19 |
| weshay | jaosorior, gate queue is at 27hr | 12:20 |
| weshay | jaosorior, nothing is landing.. we need to resolve the gate issues | 12:20 |
| jaosorior | uh... how do I update the topic? I don't have the rights in this channel I think. | 12:20 |
| EmilienM | quiquell|rover: we're not talking about the same problem here | 12:20 |
| weshay | mwhahaha, can you grant ops to jaosorior | 12:20 |
| EmilienM | I'm talking about tripleo jobs timeouting in gate and sucking infra resources for nothing | 12:20 |
| weshay | EmilienM, let's merge sagi's ara patches immediately | 12:21 |
| akrivoka | gchamoul: so it's complaining that it cannot find tripleo-ansible-inventory in $PATH, but it's right there in /usr/bin | 12:21 |
| weshay | 1] https://review.openstack.org/#/c/597222 | 12:21 |
| weshay | [2] https://review.openstack.org/#/c/580238 | 12:21 |
| *** mwhahaha sets mode: +o jaosorior | 12:21 | |
| *** mwhahaha sets mode: +o weshay | 12:22 | |
| gchamoul | akrivoka: yes manually it works | 12:22 |
| weshay | jaosorior, we may want to send an email to the list.. or jump on a quick chat on how to proceed | 12:22 |
| *** mwhahaha sets mode: +o EmilienM | 12:22 | |
| weshay | jaosorior, http://zuul.openstack.org/ | 12:22 |
| * mwhahaha is off today | 12:22 | |
| gchamoul | akrivoka: but if you change something in the run-validation script, you will get the same result whatever the change you've made | 12:23 |
| *** tzumainn has joined #tripleo | 12:23 | |
| mwhahaha | Good luck (it was container downloads last I checked) | 12:23 |
| weshay | mwhahaha, k | 12:23 |
| *** trown|outtypewww is now known as trown | 12:23 | |
| quiquell|rover | EmilienM: If we are talking about what's causing the timeouts, I think it can be related to not having promotions | 12:24 |
| weshay | jaosorior, mwhahaha quiquell|rover I think if we land the ceph patches.. we can promote master | 12:24 |
| weshay | scen01/02 are failing | 12:24 |
| openstackgerrit | Merged openstack/tripleo-docs master: Update standalone documentation to include steps for Ceph https://review.openstack.org/602710 | 12:24 |
| weshay | I believe due to that | 12:24 |
| quiquell|rover | weshay: job link ? | 12:25 |
| jaosorior | weshay: which patches? | 12:25 |
| weshay | quiquell|rover, jaosorior chkumar|ruck can we chat for a minute? | 12:25 |
| jaosorior | weshay: sure. Gimme some minutes to get off the current meeting I'm at | 12:25 |
| *** yprokule_ has quit IRC | 12:25 | |
| * jaosorior sucks at multitasking | 12:26 | |
| weshay | https://bluejeans.com/u/whayutin/ | 12:26 |
| trown | mandre: did you see my note about relative success with the container openshift-ansible? restarting the mistral executor container gets past the permission issue | 12:26 |
| quiquell|rover | weshay: ack | 12:26 |
| weshay | dam.. I have a mtg in 4 min w/ the rdo cloud admins | 12:26 |
| quiquell|rover | weshay: we have stuck stacks too, to add to the mix :-( | 12:26 |
| quiquell|rover | weshay: nodepool ones I mean | 12:27 |
| weshay | quiquell|rover, jaosorior https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-master/2723047/ | 12:27 |
| *** two_oes has quit IRC | 12:27 | |
| *** cylopez has joined #tripleo | 12:27 | |
| quiquell|rover | weshay: this can be telemtry issue | 12:27 |
| trown | mandre: maybe we need to do that as part of undercloud install? kind of weird that it only affects the openshift case... | 12:27 |
| mandre | trown: yes, it's great, at least we've confirmed the issue goes away when restarting the mistral-executor container | 12:28 |
| *** mcornea has joined #tripleo | 12:28 | |
| chkumar|ruck | quiquell|rover: f16 and fs17 telemetry issue | 12:28 |
| chkumar|ruck | master only | 12:28 |
| mandre | trown: the odd thing was that when shelling in the container I could run the docker command, but for some reason the mistral process (pid 1) couldn't | 12:29 |
| quiquell|rover | chkumar|ruck, weshay: This is the lp for the issue https://bugs.launchpad.net/tripleo/+bug/1792862 | 12:29 |
| openstack | Launchpad bug 1792862 in tripleo "[master] Telemetry Tempest integration tests failed giving Unable to complete operation on subnet b13c76ec-c851-48d0-91a9-245a2fdcad9b: One or more ports have an IP allocation from this subnet." [Critical,Fix committed] - Assigned to Mehdi Abaakouk (sileht) | 12:29 |
| jaosorior | chkumar|ruck, quiquell|rover: What patches are needed for promotion? | 12:29 |
| *** weshay changes topic to "Welcome to Rocky | CI Status: RED, DO NOT WORKFLOW https://docs.openstack.org/tripleo-docs/latest/" | 12:30 | |
| mandre | trown: yeah maybe bring it up to stevebaker, he might have an idea why it is failing | 12:30 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade stable/rocky: Use ansible_host variable in wait_for task. https://review.openstack.org/604073 | 12:30 |
| *** salmankhan1 has joined #tripleo | 12:30 | |
| trown | mandre: ya I am struggling to come up with a reason reboot would help... docker group is all I can think of, but it seems like it would start up with that the first time | 12:30 |
| chkumar|ruck | sileht: please update this one https://review.openstack.org/#/c/603590/ | 12:30 |
| quiquell|rover | jaosorior: https://review.openstack.org/#/c/603323/, https://review.openstack.org/#/c/603915/ | 12:31 |
| trown | mandre: ya I will send stevebaker an email about it and try to catch him at the end of my day | 12:31 |
| trown | mandre: will probably put up a DNM patch so we can show it working in CI | 12:31 |
| mandre | trown: thanks! I'm deferring to you since stevebaker and I have opposite hours | 12:32 |
| trown | mandre: hehe ya | 12:32 |
| quiquell|rover | jaosorior: But I doubt thi sis enough, fs020 is failing now and we don't have a fix | 12:32 |
| *** ade_lee has quit IRC | 12:32 | |
| quiquell|rover | jaosorior: But we need those to get merged, to reduce the stuff failing at promotions | 12:32 |
| *** salmankhan has quit IRC | 12:33 | |
| *** sri_ has joined #tripleo | 12:33 | |
| *** sanjayu_ has quit IRC | 12:34 | |
| mandre | trown: https://review.openstack.org/#/c/603780/ seems to be working too, what's left to do beside the running openshift-ansible from the container? | 12:35 |
| *** salmankhan1 has quit IRC | 12:35 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Stop cap granting to empty pool when telemetry disabled https://review.openstack.org/585911 | 12:36 |
| trown | mandre: we need to figure out how we will pin the container and update the pin | 12:36 |
| quiquell|rover | chkumar|ruck: Make sense to add a review, to skipp the fs020 test or do we look into the issue ? | 12:36 |
| trown | mandre: to start we can do it manually, but we need to figure out how to automate that | 12:36 |
| chkumar|ruck | quiquell|rover: yes we can skip it for now and do a promotion | 12:36 |
| trown | mandre: after that, I think we are in a good spot to make the job voting | 12:37 |
| ykarel | jaosorior, seems https://review.openstack.org/#/c/603621/ also necessary | 12:38 |
| mandre | woot! | 12:38 |
| ykarel | quiquell|rover, ^^ | 12:38 |
| trown | mandre: then I think we can hand that job over to tripleo-ci squad (after giving some training on how to troubleshoot it) | 12:38 |
| quiquell|rover | ykarel: this will fix master fs016 I suppose | 12:39 |
| chkumar|ruck | ykarel: quiquell|rover for fs016 and fs017 https://review.openstack.org/#/c/603590/ this one is also need | 12:39 |
| chkumar|ruck | ed | 12:39 |
| ykarel | quiquell|rover, hopefully but good to test in paraller with master promtion | 12:39 |
| ykarel | chkumar|ruck, the patch i shared was required for ^^ | 12:40 |
| chkumar|ruck | ykarel: yes | 12:40 |
| mandre | trown: we may also want to put workload on the deployed openshift but it might be a bit trickier because of dns and additional time it takes to build deploy a pod | 12:40 |
| mandre | deploy an app, sorry - i.e build the container and deploy the pod | 12:40 |
| trown | mandre: ya, it is worth trying though, if it fits in CI, definitely worth testing it | 12:41 |
| jaosorior | ykarel: restored it. | 12:41 |
| *** etingof has quit IRC | 12:41 | |
| quiquell|rover | chkumar|ruck: Can yyou prepare the tempest reivew to ignore fs020 ones ? | 12:41 |
| quiquell|rover | chkumar|ruck: It's going to be faster | 12:41 |
| ykarel | jaosorior, Thanks | 12:42 |
| chkumar|ruck | quiquell|rover: yup one min | 12:42 |
| quiquell|rover | chkumar|ruck: thanks os much | 12:42 |
| ykarel | chkumar|ruck, good to test master fixes in the promotion job after getting a ack from sileht | 12:42 |
| *** radeks has joined #tripleo | 12:42 | |
| *** ansmith has joined #tripleo | 12:44 | |
| *** leanderthal has quit IRC | 12:46 | |
| *** gkadam has quit IRC | 12:46 | |
| lyarwood | EmilienM: morning, did you want to push the cookiecutter review for puppet-placement? Happy to work on that this afternoon if you don't have time. | 12:47 |
| EmilienM | lyarwood: I don't have time, please go ahead :) | 12:48 |
| *** gkadam has joined #tripleo | 12:48 | |
| *** gkadam has quit IRC | 12:48 | |
| jrist | jaosorior: what does "BR" mean in your email? | 12:49 |
| jaosorior | Best Regards | 12:49 |
| lyarwood | EmilienM: ack will do, thanks | 12:49 |
| jrist | jaosorior: ah cool thanks | 12:49 |
| *** paramite has quit IRC | 12:49 | |
| jaosorior | jrist: unfortunately it wasn't BeeR | 12:50 |
| *** janki has quit IRC | 12:50 | |
| jrist | :) | 12:50 |
| EmilienM | TripleO Edge meeting in 10 minutes | 12:50 |
| jaosorior | EmilienM: here? | 12:51 |
| florianf | akrivoka, gchamoul: https://bugs.launchpad.net/tripleo/+bug/1789886 | 12:51 |
| openstack | Launchpad bug 1789886 in tripleo "Validation runs through mistral are breaking" [High,Fix released] - Assigned to Florian Fuchs (flo-fuchs) | 12:51 |
| *** salmankhan has joined #tripleo | 12:51 | |
| *** med_ has joined #tripleo | 12:51 | |
| florianf | akrivoka, gchamoul: The two patches attached to the bug have been merged in master. One rocky backport is still up for review though | 12:52 |
| akrivoka | florianf: I'm running master (ran oooq yesterday) and still hitting the bug gchamoul linked above | 12:53 |
| *** rbrady has joined #tripleo | 12:53 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Workflow for provisioning bare metals without Nova https://review.openstack.org/576856 | 12:53 |
| EmilienM | jaosorior: yes, see slagle's email | 12:54 |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/tripleo-quickstart master: ceilometer: Don't use deprecated metric https://review.openstack.org/603590 | 12:54 |
| jaosorior | EmilienM: cool cool | 12:55 |
| sileht | chkumar|ruck, you mean rebased it ? | 12:55 |
| sileht | ykarel, ack about what ? | 12:55 |
| ykarel | sileht, telemetry fixes are ready for test | 12:55 |
| slagle | EmilienM: jaosorior : in 1h10m right? | 12:55 |
| slagle | though I said 1400UTC | 12:55 |
| slagle | *thought | 12:56 |
| florianf | akrivoka: I hit something similar with another fix recently that I didn't see in a fresh oooq install immediately. can I access your env to check if the fix is included? | 12:56 |
| ykarel | sileht, i saw that tripleo-quickstart patch has -1 and u have some PS | 12:56 |
| weshay | jaosorior, ok back | 12:56 |
| sileht | ykarel, I have 3 depending patches to fixes master, one is approved, other failed for unrealted reason | 12:56 |
| ykarel | sileht, so i asked to get ack from u to avoid wasting CI resources before testing | 12:56 |
| ykarel | sileht, okk cool | 12:56 |
| *** cylopez has quit IRC | 12:57 | |
| ykarel | sileht, can u share link for three patches please | 12:57 |
| sileht | it's such a pain to test change in tripleo ... | 12:57 |
| weshay | chkumar|ruck, quiquell|rover so scen01/02 fs16/17 are blocked on telemetry right? | 12:57 |
| sileht | ykarel, https://review.openstack.org/#/c/603329/ | 12:57 |
| sileht | ykarel, then follow Depend-On: | 12:57 |
| *** lblanchard has quit IRC | 12:57 | |
| EmilienM | slagle: ah. Yes 1400UTC is in 1h, sorry | 12:57 |
| weshay | jaosorior, chkumar|ruck quiquell|rover if we need to we have the option of moving the tests to the skip list | 12:57 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: DNM: Testing openshift-ansible container https://review.openstack.org/604076 | 12:57 |
| weshay | so we can get a master promotion | 12:57 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: nova-less-deploy: update to metalsmith 0.7.0 https://review.openstack.org/604077 | 12:58 |
| weshay | I don't see prad online | 12:58 |
| *** raildo has joined #tripleo | 12:58 | |
| EmilienM | slagle: wait | 12:58 |
| jaosorior | weshay: chkumar|ruck is on it. | 12:58 |
| chkumar|ruck | weshay: fs16/fs17/fs020 telemetery issue | 12:58 |
| EmilienM | 2pm UTC is 9am EST no? | 12:59 |
| *** leanderthal has joined #tripleo | 12:59 | |
| weshay | chkumar|ruck, so jaosorior clear the queue for what ever chkumar|ruck puts up | 12:59 |
| quiquell|rover | chkumar|ruck: fs020 It's not neutron issue also affecting rocky ? | 12:59 |
| weshay | quiquell|rover, focus on master for now | 13:00 |
| chkumar|ruck | quiquell|rover: yes, fs020 -> telemetry + neutroin for master | 13:00 |
| chkumar|ruck | fs020 -> neutorn only for rocky | 13:00 |
| quiquell|rover | chkumar|ruck: so we skip the test o we skip the jobs at master.ini ? | 13:00 |
| EmilienM | meh I hate timezones | 13:01 |
| chkumar|ruck | quiquell|rover: if we skip the tests if neutron tempest tests will be at other jobs, it will get skipped | 13:01 |
| dtantsur | stevebaker: hi! do you by chance have any updates re https://bugs.launchpad.net/tripleo/+bug/1788414 ? | 13:01 |
| openstack | Launchpad bug 1788414 in tripleo "No easy way for local debugging on the containerized undercloud/standalone" [High,In progress] - Assigned to Steve Baker (steve-stevebaker) | 13:01 |
| EmilienM | nevermind, edge meeting is indeed in 1h | 13:02 |
| quiquell|rover | chkumar|ruck: Then we disable fs016, fs017, fs020 at master ? | 13:02 |
| ykarel | sileht, i think u can remove depends-on from puppet-openstack-integration, as delay in tripleo patches will delay poi too | 13:02 |
| ykarel | sileht, they both can be fixed parallely | 13:02 |
| chkumar|ruck | quiquell|rover: i think so | 13:02 |
| *** etingof has joined #tripleo | 13:02 | |
| chkumar|ruck | let's do a final call | 13:03 |
| quiquell|rover | chkumar|ruck: Let's do that, and hope we don't hit the connection issue | 13:03 |
| chkumar|ruck | quiquell|rover: ykarel disable fs 16, 17, 19, 20 on master | 13:03 |
| quiquell|rover | chkumar|ruck: fs019 no... we are going to have ceph version | 13:03 |
| chkumar|ruck | quiquell|rover: ykarel 19 and 20 for rocky | 13:03 |
| ykarel | chkumar|ruck, what disable? | 13:03 |
| *** cylopez has joined #tripleo | 13:04 | |
| chkumar|ruck | ykarel: disable jobs to get promotion | 13:05 |
| ykarel | chkumar|ruck, no | 13:05 |
| quiquell|rover | chkumar|ruck: Can't we skip the temptest tests that are failing ? | 13:05 |
| ykarel | chkumar|ruck, if we do that upstream zuul would go in bad state as those fs are tested in check/gate jobs | 13:06 |
| ykarel | quiquell|rover, yes that can be an option if things getting time to be ready | 13:06 |
| quiquell|rover | ykarel: so we have fixes for telemetry but we have to test them that's right ? | 13:06 |
| ykarel | quiquell|rover, yes | 13:06 |
| ykarel | quiquell|rover, https://review.openstack.org/#/c/603590/ | 13:06 |
| quiquell|rover | ykarel: so we doo a noop change with those | 13:06 |
| chkumar|ruck | quiquell|rover: so just disable neutron tests? | 13:06 |
| ykarel | quiquell|rover, yes | 13:06 |
| chkumar|ruck | for master and rocky? | 13:07 |
| quiquell|rover | chkumar|ruck: wait | 13:07 |
| ykarel | quiquell|rover, in testproject u can do that | 13:07 |
| *** shyamb has quit IRC | 13:07 | |
| *** shyamb has joined #tripleo | 13:07 | |
| quiquell|rover | ykarel: this is also needed https://review.openstack.org/#/c/603590/ ? | 13:07 |
| quiquell|rover | Ahh wait this is the msae | 13:07 |
| ykarel | i shared that only | 13:08 |
| openstackgerrit | David Vallee Delisle proposed openstack/tripleo-puppet-elements stable/queens: Remove Openstack and Ceph packages https://review.openstack.org/603965 | 13:08 |
| quiquell|rover | ykarel: this https://review.openstack.org/#/c/603621/ | 13:08 |
| sileht | ykarel, puppet-openstack-integration gate is not really broken, but it will in the future without this change | 13:08 |
| ykarel | sileht, mmm u forgot i think | 13:08 |
| ykarel | few days back we shared a trello | 13:09 |
| ykarel | let me grab link | 13:09 |
| sileht | ykarel, I have test it here: https://review.openstack.org/#/c/603559/ | 13:09 |
| sileht | ykarel, the initial issue was fixed on Ceilometer side | 13:09 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 13:10 |
| ykarel | sileht, https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario001/6283/weirdo-project/logs/testr_results.html.gz | 13:10 |
| ykarel | the patch u shared will fix ^^? | 13:10 |
| ykarel | the more latest https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario001/6333/weirdo-project/logs/testr_results.html.gz | 13:11 |
| ykarel | sileht, see ^^ the previous one is little old | 13:11 |
| sileht | ykarel, what he difference with the jobs ran in puppet-openstack-integration ? | 13:11 |
| sileht | ykarel, the packages version ? | 13:12 |
| quiquell|rover | ykarel, chkumar|ruck: going on https://review.rdoproject.org/r/#/c/13943/ | 13:12 |
| ykarel | sileht, yes the one i shared uses consistent repo, the puppet-openstack-integration upstream jobs runs on puppe-passed-ci repo | 13:12 |
| quiquell|rover | chkumar|ruck: At the same time you can prepare a review, disabling failing tempest for fs016, fs017, fs020 for master | 13:13 |
| quiquell|rover | chkumar|ruck: and fs020 failing tempest for rocky | 13:13 |
| ykarel | sileht, https://trunk.rdoproject.org/centos7-master/puppet-passed-ci/ vs https://trunk.rdoproject.org/centos7-master/consistent | 13:13 |
| sileht | ykarel, so just updating ceilometer packages should fix the issue | 13:13 |
| ykarel | sileht, https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario001/6333/weirdo-project/logs/testr_results.html.gz | 13:13 |
| ykarel | sileht, i think ceilometer fixes are alrday there | 13:13 |
| ykarel | can u share exact commit then i can confirm | 13:14 |
| sileht | ykarel, https://review.openstack.org/#/c/603321/ | 13:14 |
| quiquell|rover | going for lunch now | 13:15 |
| *** quiquell|rover is now known as quique|rover|lch | 13:15 | |
| sileht | ykarel, this fixes if for fixing this bt: https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario001/6333/weirdo-project/logs/ceilometer/agent-notification.txt.gz#_2018-09-20_12_31_00_076 | 13:15 |
| ykarel | sileht, openstack-ceilometer-common-11.0.1-0.20180919095301.af18f2b.el7.noarch is already there | 13:16 |
| ykarel | but still failed | 13:16 |
| sileht | ykarel, sad the fix still not work ... | 13:16 |
| ykarel | :( | 13:16 |
| sileht | ykarel, keystone raises us that we have "service" project twice | 13:16 |
| sileht | ykarel, I have thought it was because we do not pass the domain, so I added the domain, but it still not works... | 13:17 |
| rascasoft | quique|rover|lch, slagle, mwhahaha, I have a reproducer | 13:17 |
| ykarel | sileht, that's bad | 13:18 |
| ykarel | sileht, do u need a reproducer? | 13:18 |
| rascasoft | quique|rover|lch, slagle, mwhahaha, job is finishing to collect the logs, but the error was reproduced, still jq: error: Could not open file ...json | 13:18 |
| chkumar|ruck | quique|rover|lch: on it | 13:18 |
| sileht | ykarel, no but I need some help to understand how tempest is ran | 13:18 |
| *** jfrancoa has quit IRC | 13:19 | |
| slagle | rascasoft: ok, thanks. send me the details on how to access the environment | 13:19 |
| ykarel | sileht, i think chkumar|ruck can help on that ^^ | 13:19 |
| *** jfrancoa has joined #tripleo | 13:19 | |
| chkumar|ruck | sileht: give me some time, I am updating the skip list to fix prmotipn | 13:20 |
| sileht | ykarel, we have two issues currently, this bt and some deprecated was removed but still used by ROD/puppet/... | 13:20 |
| *** gfidente has quit IRC | 13:21 | |
| ykarel | sileht, so https://review.openstack.org/#/c/603329/ should fix the deprecated part removal? | 13:23 |
| ykarel | and for other u need help from chkumar|ruck | 13:23 |
| sileht | ykarel, yes | 13:23 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Display output of package install in case of failure https://review.openstack.org/595576 | 13:23 |
| sileht | ykarel, but it also need some tempest changes | 13:23 |
| *** shyamb has quit IRC | 13:23 | |
| sileht | ykarel, I don't known how to configure tempest for this repo | 13:23 |
| ykarel | sileht, what u want to configure | 13:24 |
| *** paramite has joined #tripleo | 13:24 | |
| sileht | ykarel, The same thing I does here: https://review.openstack.org/#/c/603590/ | 13:24 |
| sileht | ykarel, setting some conf in tempest.conf | 13:24 |
| *** gfidente has joined #tripleo | 13:25 | |
| ykarel | sileht, ok | 13:25 |
| ykarel | sileht, so can u add these details at some place, i think we can keep tripleo and poi stuffs seperate | 13:25 |
| ykarel | starting by removing that Depends-ON | 13:26 |
| ykarel | sileht, may be u can add https://trello.com/c/AsJeUA6I/922-poiscenario-001-1-telemetry-tempest-test-failing | 13:26 |
| ykarel | i can check how those confs are set, generally those are set by puppet-* modules, or the deployment, poi in this case | 13:27 |
| ykarel | i think puppet-tempest provides a way to configure it, and that's doable | 13:27 |
| sileht | ykarel, I have tried to set them in poi but the generated configuration was wrong | 13:27 |
| *** udesale has joined #tripleo | 13:27 | |
| ykarel | sileht, looking ur patch now | 13:28 |
| sileht | in PS3 I have tried that: https://review.openstack.org/#/c/603329/3/manifests/tempest.pp | 13:28 |
| sileht | then I see http://logs.openstack.org/29/603329/3/check/puppet-openstack-integration-5-scenario001-tempest-centos-7-luminous/f39ec7a/logs/tempest.conf.txt.gz updated with me conf | 13:29 |
| *** jpena|lunch is now known as jpena | 13:29 | |
| sileht | but according the gnocchi_wsgi_access_ssl.txt.gz, this is not the used tempest configuration | 13:30 |
| sileht | gnocchi_wsgi_access_ssl.txt.gz still show cpu_util instead cpu | 13:30 |
| ykarel | sileht, okk tempest configuration updated, but not used somehow | 13:30 |
| sileht | ykarel, yes | 13:30 |
| sileht | ykarel, this tempest configuration options are new, maybe the tempest plugin is not up2date ? | 13:31 |
| sileht | ykarel, how can I check that ? | 13:31 |
| *** jistr is now known as jistr|call | 13:32 | |
| ykarel | sileht, rdoinfo, or delorean repo | 13:32 |
| *** hjensas has quit IRC | 13:32 | |
| sileht | maybe this commit is missing: https://github.com/openstack/telemetry-tempest-plugin/commit/3b79e460e2ee183dd70e6dfcd4ef47a0a93db90a | 13:32 |
| ykarel | sileht, python2-telemetry-tests-tempest-0.1.1-0.20180910161257.2dcc6ec.el7.noarch.rpm is in puppet-passed-ci | 13:33 |
| weshay | apetrich, 1 sec | 13:34 |
| sileht | ykarel, damn, I also need this package updated ... | 13:35 |
| sileht | ykarel, the one in centos7-master-head looks OK | 13:35 |
| *** aedc_ has joined #tripleo | 13:35 | |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Add networks to IronicConductor role. https://review.openstack.org/604089 | 13:36 |
| sileht | 0:0.1.1-0.20180913122527.49ed45e.el7 | 13:36 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: libvirt standalone deployment https://review.openstack.org/591077 | 13:36 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: f28 support for quickstart https://review.openstack.org/591652 | 13:36 |
| sileht | ykarel, anyways I will restore the my PS3 changes | 13:36 |
| sileht | ykarel, there is so many repos to update the tempest and ceilometer conf, that I this you should exclude temporary the telemetry job | 13:37 |
| sileht | ykarel, and I will be in PTO in 3 hours, and back in Wednesday | 13:38 |
| *** aedc has quit IRC | 13:38 | |
| ykarel | sileht, to promote that we need promotion of consistent --> puppet-passed-ci | 13:38 |
| ykarel | then only it can be available for test in check jobs for poi | 13:39 |
| sileht | ykarel, what are the jobs that does the promotion ? | 13:39 |
| sileht | ykarel, quickstart + tht ? | 13:39 |
| weshay | marios, https://trello.com/c/Ble66eRY/431-list-of-upstream-reviews-ruck-rover | 13:39 |
| ykarel | sileht, nope | 13:39 |
| ykarel | here it's differnt story | 13:39 |
| ykarel | sileht, https://ci.centos.org/view/rdo/view/weirdo-pipelines/view/weirdo-promote-puppet/ | 13:39 |
| ykarel | if all four scenarios pass , consistent promotes to puppet-passed-ci | 13:40 |
| sileht | ykarel, yeah but to make them pass, you need the tempest configuration update in poi, that can't be merge because the packages are too old | 13:41 |
| *** cylopez has quit IRC | 13:41 | |
| sileht | looks like a chicken-egg problem | 13:41 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Skip some neutron_tempest_plugin tests for master and rocky and telemetry for master https://review.openstack.org/604091 | 13:42 |
| dtantsur | thrash: hi! have you seen mistral just stopping doing anything without any traces in logs? | 13:42 |
| thrash | dtantsur: not to my knowledge | 13:42 |
| chkumar|ruck | arxcruz: quique|rover|lch: ykarel weshay jaosorior ^^ it is needed to skip tempest in order to get promotion | 13:42 |
| sileht | (mainly due because we miss to remove deprecated config from RDO/puppet suffs) | 13:42 |
| * dtantsur ragequits | 13:43 | |
| ykarel | sileht, thinking about it, sileht about ur PTO part, can anyone else help in moving the things | 13:43 |
| ykarel | in ur absense | 13:43 |
| toure | dtantsur I am working on it | 13:43 |
| sileht | ykarel, I see another way to solve it, changing defaults in tempest telemetry project, but that will break upstream stable branches | 13:43 |
| *** cylopez has joined #tripleo | 13:44 | |
| arxcruz | chkumar|ruck: is this already o cix board? | 13:44 |
| dtantsur | toure: oh, is it something known? | 13:44 |
| toure | dtantsu: it is something odd, and only happens after about 48hrs | 13:44 |
| sileht | ykarel, I see only pradk known tripleo+ceilometer, but I don't known/think he have time to dig into that | 13:44 |
| toure | just discovered | 13:44 |
| dtantsur | toure: ah, it's another problem. the 48hrs is slightly different (I also faced it). | 13:45 |
| toure | oh | 13:45 |
| chkumar|ruck | arxcruz: long story short, zuul queue is high, neutornt empest tests are broken and telemetry for that we need to disable telemetry in fs016, fs17 anf fs020 | 13:45 |
| dtantsur | toure: I have a fresh environment where I've rebuilt a container. | 13:45 |
| chkumar|ruck | arxcruz: https://review.rdoproject.org/etherpad/p/ruckrover-sprint19 | 13:45 |
| dtantsur | toure: after long wait I get YaqlEvaluationException: Can not evaluate YAQL expression [expression=task().result, error=Task execution not found [id=c861612e-badb-4d9c-9953-aaf1a638a3e8], data={}] | 13:45 |
| chkumar|ruck | arxcruz: https://trello.com/c/la6Dkifb/755-cixlp1793482tripleociproa-rocky-fs020-periodic-job-neutron-tempest-tests-are-failing-by-giving-503-service-unavailable and https://trello.com/c/OpWmOkAZ/742-cixlp1792862tripleociproa-master-telemetry-tempest-integration-tests-failed-giving-unable-to-complete-operation-on-subnet-b13c76 | 13:45 |
| *** mjturek has joined #tripleo | 13:46 | |
| toure | hrm | 13:46 |
| dtantsur | which is... super weird. mistral is telling me it lost the task it was executing | 13:46 |
| dtantsur | it's really frustrating. nothing has worked for me since the switch to containerized undercloud. just nothing. | 13:46 |
| toure | dtantsur just out of curiosity if you disable selinux and try again does the problem go away | 13:46 |
| dtantsur | lemme try, why not... | 13:47 |
| toure | yeah all types of things broke | 13:47 |
| chkumar|ruck | sileht: ykarel I think best we can add the tempest.conf generaiton part in temepstconf it will solve updating the config at multiple place related to telemetry pieces | 13:47 |
| dtantsur | toure: I assume you mean on the undercloud, not in a container? | 13:47 |
| toure | correct | 13:48 |
| dtantsur | looks stuck as well | 13:48 |
| ykarel | chkumar|ruck, i guess u already working on that part, right? | 13:48 |
| ykarel | i saw some patches, but don't remember | 13:48 |
| toure | dtantsur the setenforce command? | 13:48 |
| chkumar|ruck | ykarel: nope, i donot get cycles, if someone wants to work, we can help | 13:49 |
| dtantsur | toure: nope, the action execution | 13:49 |
| chkumar|ruck | ykarel: only support is there to discover services | 13:49 |
| toure | dtantsur so I think both of our issues are related and I have been trying to narrow it down | 13:49 |
| chkumar|ruck | not specific config for telemetry | 13:49 |
| toure | in my setting I can still poll mistral for information but I can not execute new workflows | 13:50 |
| sileht | chkumar|ruck, ykarel I think I will change the telemery plugins defaults | 13:50 |
| sileht | chkumar|ruck, ykarel do on stable branches keep the tempest telemetry plugins in sync with master ? | 13:50 |
| *** radeks has quit IRC | 13:50 | |
| chkumar|ruck | sileht: yes | 13:50 |
| toure | dtantsur which leads me to believe something is preventing writes from the containers | 13:51 |
| *** cylopez has quit IRC | 13:51 | |
| ykarel | sileht, https://github.com/redhat-openstack/rdoinfo/blob/master/rdo.yml#L8281-L8290 | 13:51 |
| sileht | chkumar|ruck, ok, so if I change the default I will break N-2 :( | 13:51 |
| dtantsur | toure: yeah, I know. in my case one specific action seems to get stuck: GetPubKey | 13:51 |
| ykarel | sileht, rocky and earlier releases are pinned: https://github.com/redhat-openstack/rdoinfo/blob/master/rdo.yml#L8281-L8290 | 13:51 |
| dtantsur | I can run e.g. enrollment workflow | 13:51 |
| chkumar|ruck | sileht: in theory, tempest plugin is branchless, a particular tag should support stbale branches | 13:51 |
| ykarel | sileht, but where RDO packages are not used, it can break there older releases | 13:52 |
| ykarel | the one i shared applies to RDO packages | 13:52 |
| chkumar|ruck | ykarel: https://github.com/openstack/telemetry-tempest-plugin/tree/0.1.0 is working fine with rocky | 13:53 |
| toure | dtantsur I will let you know what I find on my side still digging through logs from mistral -> swift | 13:53 |
| chkumar|ruck | if things got updated in master , it will work with master,but rocky 0.0.1 is fine | 13:53 |
| chkumar|ruck | stable branches can use the help of requriements.txt | 13:53 |
| sileht | chkumar|ruck, ykarel ah ok perfect, I will update our defaults then | 13:54 |
| toure | dtantsur not sure on your system but logs aren't being captured on execution failures | 13:54 |
| sileht | chkumar|ruck, so you will get the good configuration out of the box | 13:54 |
| chkumar|ruck | sileht: thanks :-) | 13:54 |
| ykarel | sileht, Thanks | 13:54 |
| sileht | chkumar|ruck, that may break our upstream queens branch, but we can figure out this later | 13:54 |
| ykarel | sileht, ack | 13:55 |
| *** quique|rover|lch is now known as quiquell|rover | 13:55 | |
| sileht | ykarel, chkumar|ruck https://review.openstack.org/#/c/604095/ | 13:55 |
| sileht | I will update the poi change and drop the quickstart patch | 13:55 |
| sileht | ykarel, the tht patch must land before the promotion, but since it's already +A, that should be good | 13:56 |
| quiquell|rover | rascasoft: That's awesome maybe slagle can take a look now | 13:56 |
| slagle | TripleO Edge squad meeting in 5 minutes | 13:56 |
| EmilienM | tripleo edge meeting here in a few min ! | 13:56 |
| rascasoft | quiquell|rover, already did :) | 13:56 |
| *** chkumar|ruck is now known as chkumar|off | 13:56 | |
| rascasoft | quiquell|rover, investigations are ongoing | 13:57 |
| ykarel | sileht, ack, quikcstart patch not needed? | 13:57 |
| quiquell|rover | rascasoft: Cool | 13:57 |
| sileht | ykarel, just some part of it | 13:57 |
| *** waleedm has quit IRC | 13:57 | |
| ykarel | sileht, ack will keep an eye there | 13:58 |
| *** bfournie has left #tripleo | 13:58 | |
| *** bfournie has joined #tripleo | 13:59 | |
| sileht | ykarel, sorry the quickstart one is not needed, indeed :) | 13:59 |
| ykarel | sileht, ack | 14:00 |
| slagle | meeting time! | 14:00 |
| slagle | #startmeeting TripleO Edge Squad Meeting | 14:00 |
| slagle | #startmeeting TripleO Edge Squad Meeting | 14:01 |
| openstack | Meeting started Thu Sep 20 14:01:01 2018 UTC and is due to finish in 60 minutes. The chair is slagle. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| quiquell|rover | ykarel: Let's disable the temptest test for neutron and telemtry, testing RDO job for telemetry is still queued | 14:01 |
| *** openstack changes topic to " (Meeting topic: TripleO Edge Squad Meeting)" | 14:01 | |
| openstack | The meeting name has been set to 'tripleo_edge_squad_meeting' | 14:01 |
| quiquell|rover | ykarel: Is more important to ensure promotions | 14:01 |
| slagle | ping slagle, emilien, csatari, jaosorior, owalsh, fultonj, gfidente, hjensas, jtomasek,thrash, bogdando | 14:01 |
| jaosorior | o/ | 14:01 |
| thrash | o/ | 14:01 |
| fultonj | o/ | 14:01 |
| bogdando | o/ | 14:01 |
| slagle | #info remove or update your nick from the Meeting Template on the etherpad if you want (or don't want) to be ping'd for the start of the meeting | 14:01 |
| jtomasek | o/ | 14:01 |
| jillr | o/ | 14:01 |
| slagle | #link https://etherpad.openstack.org/p/tripleo-edge-squad-status | 14:02 |
| slagle | Anyone can use the #link, #action, #help, #idea and #info commands, not just the moderatorǃ | 14:02 |
| slagle | #topic Agenda | 14:02 |
| *** openstack changes topic to "Agenda (Meeting topic: TripleO Edge Squad Meeting)" | 14:02 | |
| slagle | * One off items | 14:02 |
| slagle | * Ongoing Work | 14:02 |
| slagle | * New Work/Specs | 14:02 |
| slagle | * Review Requests | 14:02 |
| slagle | * Goals for the week | 14:02 |
| *** ade_lee has joined #tripleo | 14:02 | |
| slagle | hi everyone | 14:02 |
| slagle | i put together an arbitrary agenda, but we can adjust it if needed | 14:03 |
| slagle | #topic One off items | 14:03 |
| *** openstack changes topic to "One off items (Meeting topic: TripleO Edge Squad Meeting)" | 14:03 | |
| slagle | i thought it could be useful to start with some goals of this squad | 14:03 |
| slagle | Highlight what we're working on to increase visiblity | 14:04 |
| slagle | Ask for or offer help to increase collaboration | 14:04 |
| slagle | Align our efforts around Edge focused priorities | 14:04 |
| gfidente | o/ | 14:04 |
| slagle | that's what I came up with. any other thoughts? | 14:04 |
| jaosorior | sounds good to me | 14:04 |
| slagle | yea it seemed like we have a lot of different projects wanting to work on tripleo edge related stuff | 14:04 |
| slagle | it would be good to have some cross communication and alignment | 14:05 |
| dtantsur | o/ | 14:05 |
| fultonj | slagle: sounds good | 14:06 |
| slagle | alright, do we have any followup discussions from the PTG? | 14:06 |
| slagle | #link https://etherpad.openstack.org/p/tripleo-ptg-stein-edge | 14:06 |
| bogdando | we should be clear with the use cases we want to target and omit | 14:06 |
| bogdando | there is too much of use cases, when folks saying Edge | 14:06 |
| bogdando | then we could Align our efforts around Edge focused priorities upstream | 14:06 |
| slagle | bogdando: what use cases do you have in mind? | 14:07 |
| fultonj | bogdando: yes, i think the "telco 5g" usecase as identified by edge working group is the usecase | 14:07 |
| fultonj | bogdando: if there are advocates for other usecases they can come forward, but it seems at PTG we converged on a specific architecture | 14:07 |
| bogdando | fultonj: that's great. So could we probably set the tripleo use cases in spec? | 14:08 |
| dtantsur | a link handy? | 14:08 |
| fultonj | said usecase has an architecture w/ diagrams | 14:08 |
| fultonj | split control plane spec has a diagram and we need to make it more specific | 14:08 |
| * fultonj looks for link | 14:09 | |
| slagle | fultonj: is that what you're referring to? https://www.dropbox.com/s/255x1cao14taer3/MVP-Architecture_edge-computing_PTG.pptx?dl=0 | 14:09 |
| *** Petersingh is now known as Petersingh|away | 14:09 | |
| fultonj | slagle: yes | 14:10 |
| slagle | that was sent to edge-computing@openstack.org | 14:10 |
| *** Petersingh|away has quit IRC | 14:10 | |
| slagle | #link https://www.dropbox.com/s/255x1cao14taer3/MVP-Architecture_edge-computing_PTG.pptx?dl=0 | 14:10 |
| *** aevoo has joined #tripleo | 14:10 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 14:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| gfidente | fultonj slagle but tripleo case is probably closer to the diagram we had in the spec | 14:10 |
| fultonj | we have a variation w/ slightly diff terms | 14:10 |
| gfidente | fultonj slagle though that probably needs some updates as well | 14:10 |
| slagle | ok, so can we take an action here to refine the tripleo split-controlplane use case and see how it lines up with that^^? | 14:10 |
| gfidente | https://specs.openstack.org/openstack/tripleo-specs/specs/rocky/split-controlplane.html | 14:10 |
| gfidente | slagle yeah I can take the item | 14:11 |
| fultonj | slagle: yes, i'll help on it too | 14:11 |
| slagle | see where we differ, and if should adjust at all? | 14:11 |
| fultonj | gfidente: how about a new diagram w/ comments on diffs | 14:11 |
| fultonj | e.g. far-edge is more like our edge | 14:11 |
| gfidente | fultonj now you're signing up for too much work :D | 14:11 |
| *** moshele has quit IRC | 14:11 | |
| gfidente | fultonj but yeah far-edge there matches more or less our edge, agreed | 14:12 |
| gfidente | would be nice to have an updated diagram for this same meeting next week | 14:12 |
| fultonj | +1 | 14:12 |
| slagle | bogdando: would you like to be involved with this? does it match what you're asking for? | 14:12 |
| *** apevec has joined #tripleo | 14:12 | |
| bogdando | slagle: seems so | 14:12 |
| apevec | quiquell|rover, re. https://bugs.launchpad.net/tripleo/+bug/1793293 - where is that failing exactly? | 14:13 |
| openstack | Launchpad bug 1793293 in tripleo "pike promotion failing trying to use IPv6 to get buildlogs" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 14:13 |
| fultonj | i have proposed milestones starting on line 78 of https://etherpad.openstack.org/p/tripleo-ptg-stein-edge | 14:13 |
| slagle | bogdando: ok, cool, since you weren't at the PTG, it'd be good for you to be involved, asking ?'s that we might be overlooking | 14:13 |
| apevec | see comment (original from PaulB) | 14:13 |
| *** Chaserjim has joined #tripleo | 14:13 | |
| apevec | repourl should be using mirror | 14:13 |
| bogdando | basically we have a few more RFEs to align | 14:13 |
| fultonj | by all means suggest changes to milestones, i just wanted to start with something | 14:13 |
| bogdando | not only control plane split | 14:13 |
| slagle | #action bogdando fultonj gfidente refine tripleo split-controlplane use case and see how it lines up with proposed Edge architecture from PTG | 14:13 |
| bogdando | it's mostly related to plans management | 14:13 |
| *** aedc_ has quit IRC | 14:13 | |
| apevec | oops meeting | 14:14 |
| bogdando | so we need to map these onto that upstream architecture picture | 14:14 |
| quiquell|rover | apevec: pike's fs019 periodic job at RDO | 14:14 |
| slagle | bogdando: right, we don't have specs for many of those things. they are ideas at this point | 14:14 |
| bogdando | and see the caveats and limitations for tripleo | 14:14 |
| quiquell|rover | apevec: at repo setup, didn't have time to find why is not using IPv4 | 14:14 |
| *** cylopez has joined #tripleo | 14:14 | |
| bogdando | slagle: ack, wrt being involved | 14:14 |
| slagle | #topic Ongoing Work | 14:15 |
| *** openstack changes topic to "Ongoing Work (Meeting topic: TripleO Edge Squad Meeting)" | 14:15 | |
| slagle | besides split-controlplane, is there other related work that folks would like to highlight? | 14:15 |
| dtantsur | any undercloud work? | 14:17 |
| slagle | i'll mention there was some talk at the PTG about creating a squad around metalsmith integration | 14:17 |
| slagle | dtantsur: oh hi :) | 14:17 |
| dtantsur | :) | 14:17 |
| dtantsur | I'm curious about potential distributed undercloud (and ironic issues around it) | 14:17 |
| bogdando | that's what I meant talking about use case | 14:18 |
| slagle | dtantsur: it's a possibility. what we talked about for a first pass with split-controlplane was assume you can provision anything as we do today, or rely on deployed-server | 14:19 |
| bogdando | like having AIO undercloud managing edges, or having it 1:1 + 1 central, or distributed... | 14:19 |
| gfidente | do I understand correctly we're talking about decoupling not only the heat stacks but deploy from different underclouds? | 14:19 |
| dtantsur | we probably start with undercloud as we have it, but I guess cerntralized undercloud is not where we really want to go | 14:19 |
| slagle | probably not | 14:19 |
| bogdando | dtantsur: may be, just not sure yet how this aligns with upstream Edge group architectures aforementioned | 14:20 |
| bogdando | so we need to be clear for tripleo scope | 14:20 |
| dtantsur | bogdando: they don't talk much about the way to deploy things, do they? | 14:20 |
| bogdando | as we want it | 14:20 |
| slagle | there's both ideas around distributed undercloud and a more portable distributed deployment via config-download export | 14:20 |
| fultonj | AFAICT split control plane and distribuged UC are orthogonal in that i can deploy what's in the diagram using alread-deployed-servers or w/ distributed uc so ironic can stretch to the edge | 14:20 |
| dtantsur | right | 14:20 |
| fultonj | they will work well together but neither should block progress of the other | 14:20 |
| slagle | fultonj: yes exactly, for step 0, we should focus on what we can do today already | 14:21 |
| dtantsur | but since I cannot help with things outside of ironic, I wonder what I can do here and now :) | 14:21 |
| gfidente | fultonj slagle except for the networks | 14:21 |
| bogdando | they are not very orthogonal, as undercloud shares architecture with clouds it manges ;) | 14:21 |
| fultonj | dtantsur: i assume you'd focus on the ironic stuff and if orthogonality breaks down we'd talk | 14:22 |
| *** Chaserjim has quit IRC | 14:22 | |
| fultonj | wasn't trying to say it shouldn't be included | 14:22 |
| slagle | bogdando: that's not always the case, and i think we should be looking at decoupling where it is the case | 14:22 |
| dtantsur | I mean, PXE over WAN is not the best idea | 14:22 |
| gfidente | slagle I agree we should be looking at decoupling, maybe not stein but next :D | 14:22 |
| fultonj | dtantsur: toally agree | 14:22 |
| dtantsur | so we're left with deployed-servers essentially | 14:22 |
| fultonj | more like we have no alternative to deployed-servers until dtantsur's part is ready | 14:23 |
| fultonj | but at least we can use deployed-servers to get started in the meantime | 14:23 |
| *** radeks has joined #tripleo | 14:23 | |
| slagle | should we look at filing specs for some of these things, even if it's just a described use case? | 14:23 |
| fultonj | (to get started split control plane) | 14:23 |
| dtantsur | my part won't be ready if it's not a part of the/a use case | 14:24 |
| slagle | rest of the spec can be empty, but if the issue/case is described, we'll have it captured | 14:24 |
| beagles | mwhahaha: I need to inject an iptables rule on the undercloud, where would I look to do that | 14:24 |
| fultonj | dtantsur: i think it should be part of the usecase (wasn't trying to exlclude it) | 14:24 |
| mwhahaha | beagles: new versions it goes in the THT | 14:25 |
| mwhahaha | beagles: old versions go in instack-undercloud stuff | 14:25 |
| bogdando | 4:22:51 PM GMT+2 - dtantsur: I mean, PXE over WAN is not the best idea. | 14:25 |
| bogdando | Exactly | 14:25 |
| beagles | mwhahaha: ack | 14:25 |
| bogdando | so we gonna need keep undercloud close to the edge cloud cuz of that | 14:25 |
| bogdando | and that means we need 1:N mappings | 14:25 |
| fultonj | bogdando: you mean they need to be tightly coupled? | 14:26 |
| dtantsur | bogdando: or distributed undercloud | 14:26 |
| bogdando | 1 UC per N edge clouds grouped by LAN segments, w/o going to WAN | 14:26 |
| bogdando | whatever we call it | 14:26 |
| dtantsur | well | 14:26 |
| gfidente | dtantsur but to make sure, when you say distributed you mean decoupled, not undercloud services distributed on different nodes right? | 14:26 |
| dtantsur | we can have on undercloud with ironic-conductros close to the nodes | 14:26 |
| gfidente | dtantsur or are you thinking distributing ironic-conductor? | 14:26 |
| dtantsur | yes, I'm thinking actually distributing | 14:26 |
| gfidente | dtantsur ah ok so that's one more scenario | 14:27 |
| bogdando | that is that Edge group calling Federated IIUC | 14:27 |
| slagle | dtantsur: isn't ironic just looking at that in stein? | 14:27 |
| *** Chaserjim has joined #tripleo | 14:27 | |
| dtantsur | like, central undercloud is a control plane with ironic's down there at the edge | 14:27 |
| gfidente | dtantsur ok ok great | 14:27 |
| bogdando | there is a lot of tricky places | 14:27 |
| gfidente | so in that case networks are less of an issue | 14:27 |
| bogdando | with state sync | 14:27 |
| dtantsur | slagle: we have options already | 14:27 |
| *** cylopez has quit IRC | 14:27 | |
| dtantsur | what we discuss is more about full federation, with edges being completely independent | 14:27 |
| bogdando | like sharing images for nodes | 14:27 |
| slagle | dtantsur: ok. would you like to write something up about how it might work? | 14:27 |
| bogdando | across those distributed clusters | 14:27 |
| slagle | i think it can stand separately from split-controlplane initially | 14:28 |
| dtantsur | slagle: I had an email, but I can wrap something more formal. you mean, a spec? | 14:28 |
| dtantsur | bogdando: ironic has local cache for images. the first time it can pull them from central glance. | 14:28 |
| slagle | dtantsur: it could be a spec, just describing the use case, or dump it in the squad etherpad | 14:28 |
| fultonj | https://specs.openstack.org/openstack/tripleo-specs/specs/rocky/split-controlplane.html can still proceed w/ deployed servers regardless and then can be modified w/ distributed uc comes in | 14:28 |
| dtantsur | OR ironic can deploy from files that you push via ansible :) | 14:28 |
| bogdando | dtantsur: so it is very similar scope of problems to solve as they have it for glance | 14:28 |
| slagle | fultonj: yes. we should aim to start simple | 14:28 |
| slagle | or we'll never get off the ground | 14:28 |
| dtantsur | slagle: yeah, I'll start with the etherpad | 14:29 |
| slagle | ok cool | 14:29 |
| slagle | bogdando: please add your ideas too | 14:29 |
| bogdando | slagle: will do. | 14:29 |
| gfidente | dtantsur slagle and do we care about isolated underclouds , or only about distributing undercloud services like conductor closer to the edge zone? | 14:29 |
| slagle | #action dtantsur bogdando describe/document distributed undercloud/ironic in squad etherpad | 14:29 |
| dtantsur | gfidente: isolated underclouds can be easy (if they're completely independent) or hard (if they're federated) | 14:30 |
| bogdando | basically, I'm elaborating on the concern initially raised - defining use cases limitations. For undercloud et al | 14:30 |
| dtantsur | the former requires no work, the latter - a lot of work :) | 14:30 |
| dtantsur | so I'm talking about distributing some services | 14:30 |
| bogdando | deferated Ironic is a perfect example | 14:30 |
| bogdando | federated | 14:30 |
| dtantsur | (which has a problem with rabbit, I guess....) | 14:30 |
| slagle | bogdando: please define it :) | 14:30 |
| bogdando | yeah, right | 14:30 |
| gfidente | dtantsur so how much distributing conductor buys us compared to isolated underclouds? | 14:31 |
| dtantsur | gfidente: PXE, IPMI isolated within Edge | 14:31 |
| dtantsur | also less traffic for images | 14:31 |
| dtantsur | ah, sorry | 14:31 |
| dtantsur | I misread your question | 14:31 |
| gfidente | dtantsur yeah but we get that with isolated underclouds as well | 14:31 |
| dtantsur | right | 14:31 |
| dtantsur | having one undercloud is a good thing on its own, no? | 14:32 |
| dtantsur | like, one installer for 50 clouds, not 50 installers | 14:32 |
| dtantsur | remember, each installer is an operation cost of its own | 14:32 |
| bogdando | so I expect we to define the isolated vs federated and what is synced / cached from central UC et al | 14:32 |
| bogdando | that really matters for complexity of the solution | 14:32 |
| gfidente | dtantsur right so a benefit is that it descreases maintenance | 14:33 |
| dtantsur | yeah, we probably should have a break-down of >=3 cases | 14:33 |
| jaosorior | dtantsur: if you're planning to distribute ironic services on to the edge sites; couldn't that work piggy back from the split-control plane work? lest the distributed ironic services be deployed as a separate stack which is dependant on the central undercloud stack. | 14:33 |
| dtantsur | gfidente: also you can have a lighter node on the edge if you have only ironic stuff there | 14:33 |
| slagle | bogdando: agreed, but remember we don't necessarily have to define everything before we get started. we can start small | 14:33 |
| fultonj | https://specs.openstack.org/openstack/tripleo-specs/specs/rocky/split-controlplane.html | 14:33 |
| dtantsur | jaosorior: I guess? I'm not really familiar with the split control place work, I must admit | 14:33 |
| fultonj | ^ describes a way to start small | 14:33 |
| slagle | fultonj: yes. whether it's one undercloud or many underclouds, we can illustrate that case working | 14:34 |
| *** jistr|call is now known as jistr | 14:34 | |
| jaosorior | dtantsur: might want to give it a read; you might get the distributed ironic work for free if done generically enough that we could also use it for the undercloud. | 14:34 |
| *** Chaserjim has quit IRC | 14:35 | |
| *** rascasoft has quit IRC | 14:35 | |
| dtantsur | yep. there is some specific configuration to get done for ironic to understand which nodes belong to which site | 14:35 |
| dtantsur | but that should not be hard | 14:36 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Workflow for provisioning bare metals without Nova https://review.openstack.org/576856 | 14:36 |
| sileht | ykarel, chkumar|off I have a good new about the ceilometer bt issue, it should not block your CI, we have the same bug in our upstream CI and the job is green, so only the tempest options default change is needed :) | 14:36 |
| fultonj | wow you got that done quickly :) (jk) | 14:37 |
| dtantsur | heh | 14:37 |
| ykarel | sileht, nice | 14:37 |
| bogdando | slagle: yes, but I'm talking about very high level requirements | 14:37 |
| bogdando | for distributed vs shared-nothing choices | 14:37 |
| bogdando | that really matters to know how we want it | 14:38 |
| dtantsur | shared-nothing is possible right now, right? | 14:38 |
| bogdando | cuz the associated complexity is exponentially different | 14:38 |
| fultonj | please define "shared-nothing" | 14:38 |
| slagle | dtantsur: let's hope so :) | 14:38 |
| fultonj | a link would be fine for me | 14:38 |
| *** jfrancoa has quit IRC | 14:38 | |
| bogdando | isolated underclouds , nothing gets syncronized | 14:38 |
| dtantsur | fultonj: complete isolation? | 14:38 |
| bogdando | nor state, not images | 14:38 |
| fultonj | i see | 14:38 |
| bogdando | nor performance/monitoring metrics | 14:39 |
| bogdando | other solutions is one way sunc | 14:39 |
| bogdando | sync | 14:39 |
| dtantsur | somewhat related: are any sessions planned at the Forum? | 14:39 |
| bogdando | from edge to central and not back | 14:39 |
| bogdando | there are more | 14:39 |
| fultonj | it looks like we have two subsquads: distributed_uc + split-control plane and both should implement same usecase and click together | 14:39 |
| *** Chaserjim has joined #tripleo | 14:39 | |
| bogdando | exactly | 14:39 |
| slagle | fultonj: we don't have 2 subsquads | 14:39 |
| bogdando | :) | 14:40 |
| jaosorior | dtantsur: not about this, but it's a good time to propose it https://etherpad.openstack.org/p/tripleo-forum-stein | 14:40 |
| slagle | b/c we have nothing defined for distributed | 14:40 |
| gfidente | I liked the click together part though | 14:40 |
| slagle | it sounds like we have some folks who want to investigate in that direction, and that is good | 14:40 |
| jaosorior | We have until next week to propose topics. | 14:40 |
| dtantsur | jaosorior: done | 14:40 |
| fultonj | we have nothing defined for distributed | 14:40 |
| jaosorior | dtantsur++ | 14:41 |
| fultonj | we have https://www.dropbox.com/s/255x1cao14taer3/MVP-Architecture_edge-computing_PTG.pptx?dl=0# | 14:41 |
| fultonj | and the AI to define it better | 14:41 |
| fultonj | slagle: assuming AI of diagram + write up is implemented, will that be sufficient? | 14:41 |
| slagle | fultonj: that's the end architecture of the result. i think what we're talking about here is distribution of the deployment tool | 14:41 |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add OVB nodeset for v3 run https://review.openstack.org/604109 | 14:42 |
| bogdando | another high level choice for architecture to align it with upstream track, is how much of autonomity we want to give to control plane (UC) | 14:42 |
| bogdando | and local management capabilities | 14:43 |
| bogdando | vs central | 14:43 |
| dtantsur | this is a very good question | 14:43 |
| bogdando | indeed | 14:43 |
| bogdando | and really matters, as it also may bring some *complexity* | 14:43 |
| dtantsur | some ironic contributors want local management + central control plane. but it is hard. | 14:43 |
| fultonj | bogdando: agree, at PTG we talked about this some | 14:44 |
| bogdando | right, so we need to state all these very clear for tripleo and outline how that differs from upstream use cases | 14:44 |
| bogdando | just to not set expectations too high :) | 14:44 |
| fultonj | outcome from my perspective... edge nodes do not need to be autonomous meaning, if they loose connection to control plane they can continue to run current workload but new workloads and getting status of workload is unavailable until connection restored | 14:45 |
| fultonj | for the telco 5g usecase | 14:45 |
| gfidente | bogdando fultonj I think that is the purpose of the updated diagram for next week? at least regarding overcloud plans? | 14:45 |
| fultonj | gfidente: yes | 14:46 |
| slagle | fultonj: you're talking overcloud, bogdando asked about undercloud | 14:46 |
| slagle | but yes, it's something we'd have to figure out for a distributed undercloud as well | 14:46 |
| bogdando | My point is that we have architecture shared | 14:47 |
| bogdando | its tht you know | 14:47 |
| bogdando | and ansible | 14:47 |
| *** skramaja has quit IRC | 14:47 | |
| bogdando | so we need to have it aligned and not going diverged | 14:47 |
| slagle | yes sounds good | 14:47 |
| slagle | we have about 10 minutes left | 14:48 |
| slagle | do we feel we have enough captured as to what to look at for this week? | 14:48 |
| *** Chaserjim has quit IRC | 14:48 | |
| slagle | we took a few actions, are there any others? | 14:48 |
| fultonj | i'd like to get milestones | 14:49 |
| fultonj | if it's too early i guess we can wait | 14:49 |
| *** ratailor has joined #tripleo | 14:49 | |
| slagle | fultonj: what did you mean by that? | 14:49 |
| slagle | (saw the ? in the etherpad) | 14:49 |
| fultonj | 1. POC | 14:50 |
| fultonj | 2. CI to test POC | 14:50 |
| fultonj | 3. add features | 14:50 |
| slagle | oh ok, i thought you were asking for the upstream milestone dates or something :) | 14:50 |
| fultonj | basically i'm happy to start working on these things (i'm still on #1) | 14:50 |
| fultonj | i'll certainly need help for #2 | 14:51 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: nova-less-deploy: update to metalsmith 0.7.0 https://review.openstack.org/604077 | 14:51 |
| slagle | fultonj: i'd say file some blueprints as deps of the main split-controlplane blueprint | 14:51 |
| fultonj | slagle: ah, ok | 14:51 |
| fultonj | sounds good | 14:51 |
| openstackgerrit | Nicolas Hicher proposed openstack-infra/tripleo-ci master: provider: Add vexxhost https://review.openstack.org/596432 | 14:51 |
| slagle | fultonj: align them against the upstream milestones for stein, then we'll have a good idea of what we want to knock out for stein and when | 14:51 |
| fultonj | excellent | 14:52 |
| *** Chaserjim has joined #tripleo | 14:52 | |
| slagle | fultonj: i like the overview in the etherpad | 14:52 |
| slagle | we could have blueprints for most of that stuff | 14:52 |
| fultonj | ok, great | 14:52 |
| slagle | so then folks can come in and try it out | 14:52 |
| slagle | e.g., if i had a day or so to try network iso with split-controlplane, etc | 14:53 |
| slagle | fultonj: i can help file some of these | 14:53 |
| fultonj | thanks | 14:53 |
| slagle | #action fultonj slagle file blueprints for split-controlplane work in stein | 14:53 |
| openstackgerrit | Merged openstack/tripleo-docs master: Amend minor update docs for Rocky https://review.openstack.org/597109 | 14:54 |
| slagle | #topic Review Requests | 14:54 |
| *** openstack changes topic to "Review Requests (Meeting topic: TripleO Edge Squad Meeting)" | 14:54 | |
| slagle | are there any outstanding related reviews folks would like some eyes on? | 14:54 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Update default value for ntp servers https://review.openstack.org/604113 | 14:55 |
| * gfidente adding items in that list as well | 14:55 | |
| fultonj | owalsh isn't here but i think he has one: https://review.openstack.org/#/c/600042 | 14:56 |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/tripleo-heat-templates stable/pike: Add a way to override base path when file driver is used https://review.openstack.org/601286 | 14:56 |
| dtantsur | the metalsmith work is not a priority, but is ready for review | 14:56 |
| fultonj | i will review it and vote after i play with it more | 14:56 |
| slagle | #link https://review.openstack.org/#/c/600042 | 14:56 |
| gfidente | slagle fultonj I think most of the features in 3) are necessary for the CI job at 2) to pass though | 14:56 |
| slagle | #link https://review.openstack.org/#/c/576856 | 14:56 |
| openstackgerrit | Sorin Sbarnea proposed openstack/python-tripleoclient master: Update default value for ntp servers https://review.openstack.org/604113 | 14:56 |
| slagle | gfidente: it's not. let's start very simple | 14:57 |
| fultonj | gfidente: i didn't think that | 14:57 |
| slagle | i've already deployed a compute node in a separate stack, and it works fine w/o any of that stuff | 14:57 |
| fultonj | gfidente: i thought it would work at least the POC passed the ping test (but without ceph) | 14:57 |
| *** Chaserjim has quit IRC | 14:57 | |
| slagle | that should be the first thing the CI job does | 14:58 |
| fultonj | yeah | 14:58 |
| fultonj | it will work, just not be architected as we want it | 14:58 |
| slagle | ++ | 14:58 |
| fultonj | so we distribute parts out as we go | 14:58 |
| gfidente | and what about the tirpleoclient changes? | 14:59 |
| fultonj | gfidente: could the ci be a script until the client catches up? | 14:59 |
| slagle | we can keep discussin post meeting | 14:59 |
| fultonj | the ci would run 'openstack overcloud deploy' twice | 14:59 |
| fultonj | ok | 14:59 |
| slagle | thanks for the good discussion today and for joining folks! | 15:00 |
| slagle | #endmeeting | 15:00 |
| *** openstack changes topic to "Welcome to Rocky | CI Status: RED, DO NOT WORKFLOW https://docs.openstack.org/tripleo-docs/latest/" | 15:00 | |
| openstack | Meeting ended Thu Sep 20 15:00:13 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2018/tripleo_edge_squad_meeting.2018-09-20-14.01.html | 15:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2018/tripleo_edge_squad_meeting.2018-09-20-14.01.txt | 15:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo_edge_squad_meeting/2018/tripleo_edge_squad_meeting.2018-09-20-14.01.log.html | 15:00 |
| gfidente | slagle fultonj so you guys are basically signing up for lots of CI work too :D | 15:00 |
| gfidente | I'll stick with the services :P | 15:01 |
| fultonj | dah! | 15:01 |
| fultonj | :) | 15:01 |
| slagle | gfidente: yea maybe :) | 15:01 |
| fultonj | we will need CI help | 15:01 |
| slagle | it really wasn't that bad though when i did it manually | 15:01 |
| fultonj | at least i will | 15:01 |
| *** Chaserjim has joined #tripleo | 15:02 | |
| gfidente | slagle fultonj and also having CI sooner helps | 15:02 |
| gfidente | so there is a tradeoff anyway | 15:02 |
| fultonj | https://github.com/olliewalsh/split/tree/fix_computes | 15:02 |
| *** zaneb has quit IRC | 15:02 | |
| slagle | i deployed a compute node on my desktop, and the control plane was in rdo cloud | 15:02 |
| fultonj | nice | 15:02 |
| slagle | you just create the 2nd stack, config-download the ansible, then run it against the compute node | 15:02 |
| gfidente | slagle yeah so you have the list of params necessary for it to finish? | 15:02 |
| slagle | yep | 15:02 |
| gfidente | slagle when I tried I didn't have your patch and didn't pass in galera/rabbit vip | 15:03 |
| gfidente | slagle that is awesome :D | 15:03 |
| *** zaneb has joined #tripleo | 15:03 | |
| slagle | well that would be a problem :) | 15:03 |
| slagle | but yea i think we've landed all the patches to make it work now | 15:03 |
| fultonj | i gotta try this | 15:03 |
| gfidente | slagle right so you have the list of params we need to get from ctlplane to feed edge? | 15:03 |
| slagle | owalsh had a few extra things, like if you want live migration to work, etc | 15:03 |
| gfidente | ah so owalsh did all the work | 15:04 |
| gfidente | I see | 15:04 |
| gfidente | slagle ahahhaa | 15:04 |
| slagle | gfidente: do I even work on TripleO? | 15:04 |
| gfidente | figure me | 15:04 |
| *** quiquell|rover is now known as quique|rover|off | 15:05 | |
| *** ykarel is now known as ykarel|away | 15:05 | |
| *** gfidente has quit IRC | 15:05 | |
| *** gfidente has joined #tripleo | 15:07 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 15:10 |
| *** jtomasek has quit IRC | 15:10 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Add subnode-setup.yml playbook. https://review.openstack.org/604120 | 15:10 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Don't use mixed_upgrade special release files https://review.openstack.org/604032 | 15:11 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Link subnodes-setup into multinode jobs. https://review.openstack.org/604121 | 15:11 |
| *** aufi has quit IRC | 15:12 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Add extra tasks to run lint and upgrade after deployment. https://review.openstack.org/602416 | 15:12 |
| *** Chaserjim has quit IRC | 15:15 | |
| *** ksambor has quit IRC | 15:15 | |
| *** dxiri has joined #tripleo | 15:16 | |
| *** ykarel|away has quit IRC | 15:18 | |
| *** cylopez has joined #tripleo | 15:18 | |
| *** kopecmartin has quit IRC | 15:18 | |
| *** Chaserjim has joined #tripleo | 15:19 | |
| *** jfrancoa has joined #tripleo | 15:20 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack-infra/tripleo-ci master: Adding new standalone-upgrade jobs and lint playbook in standalone. https://review.openstack.org/604126 | 15:21 |
| *** kopecmartin has joined #tripleo | 15:22 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/python-tripleoclient master: DNM: testing new standalone upgrade job. https://review.openstack.org/604127 | 15:24 |
| chem | jfrancoa: EmilienM weshay: so I've added a test jobs for the new standalone-upgrade job in there https://review.openstack.org/604127, follow the depends on to see the tripleo-ci/quickstart-extra related review | 15:26 |
| *** hjensas has joined #tripleo | 15:26 | |
| EmilienM | chem: wow nice. I'll take a look asap | 15:27 |
| EmilienM | chem: don't we need a new featureset? I guess not? | 15:28 |
| chem | jfrancoa: ha crap I somehow lost the depends on the tht patch that makes it work | 15:28 |
| EmilienM | I don't even think you need the patch in tripleoclient | 15:28 |
| EmilienM | it'll run the jobs in tripleo-ci, iiuc | 15:28 |
| chem | EmilienM: ah ... right that may be, I'll remove it after double checking, thanks for the tip | 15:29 |
| openstackgerrit | Sorin Sbarnea proposed openstack/python-tripleoclient master: Update default value for ntp servers https://review.openstack.org/604113 | 15:29 |
| chem | jfrancoa: didn't pull in your change ... | 15:29 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Add extra tasks to run lint and upgrade after deployment. https://review.openstack.org/602416 | 15:31 |
| chem | crap, crap all ci are going to restart ... will get the result tomorrow :) | 15:31 |
| *** Chaserjim has quit IRC | 15:32 | |
| dtantsur | folks, is it by design that log files are world-readable in the containerized undercloud? | 15:33 |
| dtantsur | EmilienM, jaosorior ^^ | 15:33 |
| chem | EmilienM: you were definitively rigth, removing the last useless patch, thanks. | 15:33 |
| EmilienM | dtantsur: I don't know | 15:33 |
| EmilienM | dtantsur: bogdando & jaosorior might know better, they worked on logging | 15:33 |
| dtantsur | I'm asking because we had bug reports previously about /var/log/ironic world-readable | 15:34 |
| jaosorior | dtantsur: that is not by design. | 15:34 |
| jaosorior | thought the file permissions were handled by the container's rpm or the process | 15:34 |
| jaosorior | anyway, that's surely a bug. | 15:34 |
| *** hjensas has quit IRC | 15:34 | |
| dtantsur | okie | 15:35 |
| *** Chaserjim has joined #tripleo | 15:35 | |
| *** dxiri has quit IRC | 15:35 | |
| bogdando | that's nice we caught it so early, as of Pike had been released recently :D | 15:38 |
| *** agopi has quit IRC | 15:39 | |
| *** sshnaidm|pto has joined #tripleo | 15:44 | |
| *** Chaserjim has quit IRC | 15:47 | |
| dtantsur | heh | 15:47 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: undercloud/stackrc: unset OS_* variables https://review.openstack.org/602367 | 15:48 |
| dtantsur | oh, it's funnier than I thought | 15:48 |
| dtantsur | bogdando, jaosorior, Access: (0644/-rw-r--r--) Uid: (42422/ UNKNOWN) Gid: (42422/ UNKNOWN) | 15:48 |
| bogdando | that's kolla users | 15:49 |
| bogdando | that else but world read? | 15:49 |
| bogdando | what* | 15:49 |
| *** Chaserjim has joined #tripleo | 15:50 | |
| dtantsur | I guess we need 0600, I hope root will still be able to read them :) | 15:50 |
| bogdando | right, just didn't get what is worse than world read | 15:52 |
| bogdando | so the same issue, right | 15:52 |
| *** iranzo has quit IRC | 15:54 | |
| Tengu | dpeacock: hey :). the o7 was a typo in fact XD | 15:56 |
| * Tengu back from is 5 days motorbike roadtrip \o/ | 15:56 | |
| Tengu | and I'm alive | 15:56 |
| *** Chaserjim has quit IRC | 15:57 | |
| dpeacock | Tengu: hah - well - people like it! | 15:58 |
| dpeacock | Tengu: welcome back :-) How was the riding? | 15:58 |
| *** Chaserjim has joined #tripleo | 15:59 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO https://review.openstack.org/603275 | 16:00 |
| chkumar|off | EmilienM: Hello | 16:00 |
| EmilienM | chkumar|off: what's up | 16:01 |
| chkumar|off | EmilienM: I need your feedback on this patch https://review.openstack.org/604035 | 16:02 |
| chkumar|off | current we test stuff on same branch in TQE | 16:02 |
| chkumar|off | but if we want to test on different branch, how to do that? | 16:02 |
| *** cylopez has quit IRC | 16:02 | |
| *** leanderthal has quit IRC | 16:03 | |
| *** rascasoft has joined #tripleo | 16:03 | |
| *** iranzo has joined #tripleo | 16:04 | |
| *** cylopez has joined #tripleo | 16:07 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Check if openstack-glance-registry is enabled before stopping it. https://review.openstack.org/603581 | 16:08 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 16:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| *** jfrancoa has quit IRC | 16:10 | |
| *** iranzo has quit IRC | 16:12 | |
| *** yprokule has quit IRC | 16:13 | |
| *** Chaserjim has quit IRC | 16:14 | |
| *** aedc_ has joined #tripleo | 16:14 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add openstack-zuul-jobs as a required project https://review.openstack.org/604150 | 16:15 |
| *** aevoo has quit IRC | 16:17 | |
| *** ratailor has quit IRC | 16:17 | |
| *** derekh has quit IRC | 16:18 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add openstack-zuul-jobs role path https://review.openstack.org/604150 | 16:19 |
| *** Chaserjim has joined #tripleo | 16:21 | |
| *** kopecmartin is now known as kopecmartin|away | 16:21 | |
| *** akhila has joined #tripleo | 16:24 | |
| *** ykarel|away has joined #tripleo | 16:25 | |
| *** agopi has joined #tripleo | 16:33 | |
| *** holser_ has quit IRC | 16:33 | |
| *** Chaserjim has quit IRC | 16:34 | |
| *** Chaserjim has joined #tripleo | 16:34 | |
| *** bogdando has quit IRC | 16:35 | |
| lyarwood | bandini: https://review.openstack.org/#/c/603372/ - Would you mind if I ported this across to the new openstack/placement repo? | 16:36 |
| bandini | lyarwood: I don't mind, but cdent wanted a different approach anyway? | 16:36 |
| lyarwood | ops, missed that comment | 16:36 |
| lyarwood | bandini: ah cool, yeah I'll give that approach a go, thanks. | 16:37 |
| *** akrivoka has quit IRC | 16:39 | |
| *** akrivoka has joined #tripleo | 16:39 | |
| *** trown is now known as trown|lunch | 16:42 | |
| *** radez has quit IRC | 16:42 | |
| *** Chaserjim has quit IRC | 16:42 | |
| *** amoralej is now known as amoralej|off | 16:43 | |
| *** radez has joined #tripleo | 16:43 | |
| *** sshnaidm|pto has quit IRC | 16:44 | |
| *** jpich has quit IRC | 16:48 | |
| *** sanjayu_ has joined #tripleo | 16:50 | |
| *** udesale has quit IRC | 16:52 | |
| openstackgerrit | Victor Pickard proposed openstack/tripleo-heat-templates master: Change log directory for OpenDaylight https://review.openstack.org/603907 | 16:54 |
| openstackgerrit | Victor Pickard proposed openstack/tripleo-heat-templates master: Change log directory for OpenDaylight https://review.openstack.org/603907 | 16:56 |
| *** salmankhan has quit IRC | 17:01 | |
| *** cylopez has quit IRC | 17:02 | |
| *** ramishra has quit IRC | 17:02 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/queens: Avoid getting one-empty-element-list in blacklisted_hostnames. https://review.openstack.org/604165 | 17:03 |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add openstack-zuul-jobs role path https://review.openstack.org/604150 | 17:07 |
| *** jpena is now known as jpena|off | 17:09 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 17:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| *** gkadam has joined #tripleo | 17:14 | |
| *** florianf is now known as florianf|afk | 17:25 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer https://review.openstack.org/602492 | 17:27 |
| *** mjturek has quit IRC | 17:28 | |
| *** aevoo has joined #tripleo | 17:37 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Set SSH server keep alive options https://review.openstack.org/604171 | 17:38 |
| *** aedc_ has quit IRC | 17:39 | |
| *** dtantsur is now known as dtantsur|afk | 17:40 | |
| *** rdopiera has quit IRC | 17:44 | |
| *** mjturek has joined #tripleo | 17:50 | |
| *** akrivoka_ has joined #tripleo | 17:50 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: Upgrade docker ceph container https://review.openstack.org/603323 | 17:51 |
| *** mcornea has quit IRC | 17:51 | |
| *** akrivoka has quit IRC | 17:53 | |
| *** akrivoka_ has quit IRC | 17:57 | |
| *** mcornea has joined #tripleo | 17:58 | |
| *** gfidente has quit IRC | 18:00 | |
| weshay | jaosorior, hold off on restoring please | 18:01 |
| weshay | jaosorior, having trouble getting https://review.openstack.org/603323 to land | 18:01 |
| *** slaweq has quit IRC | 18:03 | |
| *** trown|lunch is now known as trown | 18:06 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add openstack-zuul-jobs role path https://review.openstack.org/604150 | 18:07 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 18:10 |
| jaosorior | weshay: I'll restore the patches tomorrow anyway | 18:19 |
| jaosorior | I'm off work now | 18:19 |
| jaosorior | 9pm here :D | 18:19 |
| jaosorior | ... and the patches did seem to take too long to merge | 18:19 |
| jaosorior | so maybe they'll be merged tomorrow morning | 18:19 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 18:23 |
| EmilienM | bandini, dciabrin : this it how I see it, wdyt? ^ | 18:23 |
| EmilienM | bandini, dciabrin : I'm wondering if I can make the ansible fact general for all the playbooks | 18:24 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 18:30 |
| *** tosky has quit IRC | 18:32 | |
| *** akhila has quit IRC | 18:39 | |
| *** ykarel|away has quit IRC | 18:41 | |
| *** salmankhan has joined #tripleo | 18:42 | |
| *** radeks has quit IRC | 18:46 | |
| *** radeks has joined #tripleo | 18:46 | |
| *** salmankhan has quit IRC | 18:47 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 18:59 |
| *** marios has quit IRC | 19:01 | |
| *** dtantsur|afk has quit IRC | 19:01 | |
| *** moguimar has quit IRC | 19:01 | |
| *** dtantsur has joined #tripleo | 19:01 | |
| *** med_ has quit IRC | 19:02 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 19:02 |
| *** moguimar has joined #tripleo | 19:03 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 19:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| *** etingof has quit IRC | 19:12 | |
| *** dtantsur has quit IRC | 19:13 | |
| *** moguimar has quit IRC | 19:13 | |
| *** moguimar has joined #tripleo | 19:13 | |
| *** dtantsur has joined #tripleo | 19:17 | |
| *** radeks has quit IRC | 19:22 | |
| *** med_ has joined #tripleo | 19:33 | |
| *** panda is now known as panda|dinner | 19:34 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: use the yaml stdout callback https://review.openstack.org/604194 | 19:36 |
| *** Chaserjim has joined #tripleo | 19:37 | |
| *** ssbarnea|bkp has joined #tripleo | 19:41 | |
| *** jtomasek has joined #tripleo | 19:41 | |
| *** rrubins__ has joined #tripleo | 19:45 | |
| *** jtomasek has quit IRC | 19:47 | |
| *** Chaserjim has quit IRC | 19:50 | |
| *** Chaserjim has joined #tripleo | 19:52 | |
| *** slaweq has joined #tripleo | 19:53 | |
| *** Chaserjim has quit IRC | 19:56 | |
| *** Chaserjim has joined #tripleo | 20:00 | |
| *** dsneddon has quit IRC | 20:02 | |
| *** dsneddon has joined #tripleo | 20:02 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer https://review.openstack.org/602492 | 20:03 |
| *** hjensas has joined #tripleo | 20:03 | |
| *** slaweq has quit IRC | 20:08 | |
| *** etingof has joined #tripleo | 20:09 | |
| *** Chaserjim has quit IRC | 20:09 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 20:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| *** gkadam has quit IRC | 20:10 | |
| *** ssbarnea|bkp has quit IRC | 20:15 | |
| *** jtomasek has joined #tripleo | 20:18 | |
| *** panda|dinner is now known as panda | 20:24 | |
| *** jtomasek has quit IRC | 20:27 | |
| *** owalsh has joined #tripleo | 20:27 | |
| *** ansmith has quit IRC | 20:29 | |
| dciabrin | EmilienM, commented on the review, I like the approach, thx! | 20:31 |
| bandini | EmilienM: nice yeah (the tabs made it fail in CI) | 20:32 |
| EmilienM | stupid editor | 20:32 |
| trown | EmilienM: are we still waiting on setting workflow on patches? | 20:38 |
| EmilienM | trown: I don't know, is gate stable and fixed? are we merging stuffs? | 20:39 |
| EmilienM | I see 13h in gate | 20:40 |
| EmilienM | :-O | 20:40 |
| EmilienM | thrash, dciabrin, bandini : thx for the comments, will address | 20:40 |
| trown | EmilienM: ya, but not much there... I was more asking if we merged something that was supposed to help :P | 20:40 |
| EmilienM | trown: I don't think so | 20:40 |
| trown | ack | 20:41 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: break out release config by distro type https://review.openstack.org/602387 | 20:41 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt https://review.openstack.org/591540 | 20:41 |
| openstackgerrit | Merged openstack/tripleo-common stable/rocky: Upgrade docker ceph container https://review.openstack.org/603367 | 20:41 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Disable manila tests on ocata and pike https://review.openstack.org/603915 | 20:41 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Use openshift_cockpit_deployer_image directly https://review.openstack.org/602074 | 20:41 |
| openstackgerrit | Merged openstack/tripleo-ui master: Use swiftUrl as a base for exported plan download URL https://review.openstack.org/599974 | 20:41 |
| EmilienM | none of these stuff will fix the gate ^ | 20:43 |
| weshay | EmilienM, one patch timed out | 20:45 |
| *** holser_ has joined #tripleo | 20:49 | |
| *** waleedm has joined #tripleo | 20:50 | |
| weshay | EmilienM, 603323 is needed right? | 20:51 |
| weshay | https://review.openstack.org/#/c/603329/ | 20:53 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer https://review.openstack.org/602492 | 20:55 |
| *** trown is now known as trown|outtypewww | 20:56 | |
| *** cylopez has joined #tripleo | 20:57 | |
| *** raildo has quit IRC | 21:02 | |
| *** dprince has quit IRC | 21:04 | |
| *** morazi has quit IRC | 21:06 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 21:10 |
| *** cylopez has quit IRC | 21:11 | |
| *** agopi is now known as agopi|brb | 21:12 | |
| *** agopi|brb has quit IRC | 21:18 | |
| *** ansmith has joined #tripleo | 21:19 | |
| *** waleedm has quit IRC | 21:22 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: udpate reproducer to install required deps https://review.openstack.org/600836 | 21:27 |
| *** mjturek has quit IRC | 21:27 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer https://review.openstack.org/602492 | 21:27 |
| *** thrash is now known as thrash|g0ne | 21:30 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer https://review.openstack.org/602492 | 21:37 |
| *** radeks has joined #tripleo | 21:38 | |
| *** ssbarnea|bkp2 has joined #tripleo | 21:38 | |
| *** akhila has joined #tripleo | 21:41 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Convert with_dict tasks to use loop and be less chatty https://review.openstack.org/603251 | 21:44 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: puppet host https://review.openstack.org/603252 | 21:44 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet https://review.openstack.org/603253 | 21:44 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks https://review.openstack.org/603254 | 21:44 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs https://review.openstack.org/603255 | 21:44 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer https://review.openstack.org/602492 | 21:54 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Ping default gateways before controllers https://review.openstack.org/604229 | 21:55 |
| *** ssbarnea|bkp2 has quit IRC | 21:59 | |
| *** bfournie has quit IRC | 22:00 | |
| *** holser_ has quit IRC | 22:00 | |
| *** toure is now known as toure|gone | 22:06 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 22:10 |
| *** radeks has quit IRC | 22:10 | |
| *** radeks has joined #tripleo | 22:11 | |
| *** ade_lee has quit IRC | 22:23 | |
| *** rh-jelabarre has quit IRC | 22:28 | |
| *** radeks_ has joined #tripleo | 22:28 | |
| *** radeks has quit IRC | 22:31 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] add a job to test the reproducer https://review.openstack.org/604232 | 22:35 |
| *** rlandy is now known as rlandy|bbl | 22:37 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] add a job to test the reproducer https://review.openstack.org/604232 | 22:39 |
| openstackgerrit | Goutham Pacha Ravi proposed openstack/tripleo-quickstart master: Re-enable manila tests in scenario004/pike https://review.openstack.org/600556 | 22:45 |
| *** spsurya has quit IRC | 22:48 | |
| *** spsurya has joined #tripleo | 22:50 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 22:51 |
| *** rcernin has joined #tripleo | 22:53 | |
| *** ade_lee has joined #tripleo | 22:57 | |
| stevebaker | dtantsur: hey, I've commented on that bug. https://bugs.launchpad.net/tripleo/+bug/1788414/comments/9 | 23:01 |
| openstack | Launchpad bug 1788414 in tripleo "No easy way for local debugging on the containerized undercloud/standalone" [High,In progress] - Assigned to Steve Baker (steve-stevebaker) | 23:01 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 23:01 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 23:02 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker https://review.openstack.org/604180 | 23:02 |
| *** tzumainn has quit IRC | 23:04 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr) | 23:10 |
| *** boazel has quit IRC | 23:19 | |
| *** radeks_ has quit IRC | 23:20 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: Split base job in multinode and non multinode https://review.openstack.org/604234 | 23:20 |
| *** sanjayu_ has quit IRC | 23:23 | |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-container-registry master: Install & configure podman in a new task https://review.openstack.org/602879 | 23:25 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: docker: deploy podman until we get podman working without docker https://review.openstack.org/604235 | 23:36 |
| *** rcernin has quit IRC | 23:36 | |
| *** rcernin has joined #tripleo | 23:36 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman https://review.openstack.org/600517 | 23:36 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman https://review.openstack.org/600517 | 23:36 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman https://review.openstack.org/600517 | 23:37 |
| EmilienM | stevebaker: https://review.openstack.org/602879 and https://review.openstack.org/604235 is what I came up with | 23:38 |
| EmilienM | let's see how it works in CI : https://review.openstack.org/600517 | 23:38 |
| *** panda is now known as panda|off | 23:39 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Set proper setype for service directories https://review.openstack.org/600534 | 23:45 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Allow to deactivate SELinux separation for selected containers https://review.openstack.org/600535 | 23:45 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!