| *** ysandeep|out is now known as ysandeep|rover | 03:24 | |
| *** ysandeep|rover is now known as ysandeep|afk | 04:24 | |
| *** ysandeep|afk is now known as ysandeep|rover | 04:45 | |
| *** ysandeep|rover is now known as ysandeep|rover|lunch | 08:02 | |
| *** ysandeep|rover|lunch is now known as ysandeep|rover | 08:38 | |
| marios | food biab | 09:05 |
|---|---|---|
| ysandeep|rover | reviewbot: please add in reviewlist: https://review.opendev.org/c/openstack/tripleo-ci/+/843111 | 09:57 |
| reviewbot | I have added your review to the Review list | 09:57 |
| rlandy | ysandeep|rover: frenzyfriday: hello | 10:29 |
| rlandy | ugh - bunch of node failures | 10:29 |
| rlandy | ysandeep|rover: frenzyfriday: let me know when you are ready to sync | 10:30 |
| frenzyfriday|ruck | rlandy, hello. Lets sync in a few mins, finishing up writing a bug | 10:32 |
| rlandy | I didn't merge rhel 9 | 10:32 |
| rlandy | we were expecting another breaking change | 10:32 |
| ysandeep|rover | I am around | 10:32 |
| rlandy | openvswitch | 10:33 |
| frenzyfriday|ruck | ysandeep|rover, hey, the tripleo_nodes_validation packet loss issue is in all the jobs of openstack-periodic-integration-rhos-17-rhel9 0. What should I add to Steps to reproduce in the bug? | 10:34 |
| rlandy | frenzyfriday|ruck: ysandeep|rover: https://meet.google.com/atz-dnqq-mxj?pli=1&authuser=0 | 10:35 |
| rlandy | when ready | 10:35 |
| rlandy | frenzyfriday|ruck: see breaking change notice above | 10:35 |
| ysandeep|rover | frenzyfriday|ruck, * Install standalone | 10:35 |
| ysandeep|rover | rlandy, https://review.opendev.org/c/openstack/tripleo-ci/+/843111 | 11:04 |
| ysandeep|rover | frenzyfriday|ruck, rlandy we may have a new issue here: https://logserver.rdoproject.org/11/843111/2/openstack-check/tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001/fe51b67/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 11:08 |
| ysandeep|rover | frenzyfriday|ruck, let's keep that under radar after we fix image build job | 11:08 |
| ysandeep|rover | rlandy, wallaby c9 image build - same issue as master | 11:19 |
| *** dviroel|out is now known as dviroel | 11:21 | |
| ysandeep|rover | dviroel, hey o/ at what stage we lay quickstart repos on overcloud nodes: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master/d7b6b2f/logs/overcloud-controller-0/etc/yum.repos.d/ | 11:47 |
| rlandy | chandankumar: dviroel: ysandeep|rover: pls join podified meeting today in 1 hr | 11:59 |
| rlandy | let's meet on friday when ysandeep|rover is off rr | 11:59 |
| rlandy | to discuss where to start here | 11:59 |
| rlandy | marios: ^^ fyi | 11:59 |
| marios | rlandy: looking | 12:00 |
| ysandeep|rover | rlandy: ack, sounds good | 12:00 |
| marios | ysandeep|rover: what you mean? like where in our code (it happens when we call repo-setup in the tripleo-quickstarrt-extras plays) | 12:00 |
| dviroel | rlandy: ack | 12:00 |
| chandankumar | rlandy: ack | 12:00 |
| marios | rlandy: ok don't follow th econtext but sure add me to invite | 12:01 |
| ysandeep|rover | marios: for ovb jobs - I mean before overcloud deploy or after overcloud deploy and if before or after tempest? | 12:01 |
| dviroel | ysandeep|rover: marios did the change to use repo-setup on image-build and container-build, we use repo-setup + release files | 12:01 |
| marios | ah for image build? yeah that happens in the tripleo-ci repo now (by calling repo-setup role) ysandeep|rover | 12:01 |
| marios | but you pointed to fs1 job not image build | 12:02 |
| ysandeep|rover | https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master/d7b6b2f/logs/overcloud-controller-0/etc/yum.repos.d/ -> so the repos here are coming from overcloud images ? | 12:02 |
| ysandeep|rover | marios, dviroel ^^ | 12:04 |
| marios | no ysandeep|rover those repos are coming from tqe plays where we have repo-setup | 12:04 |
| marios | ysandeep|rover: i.e. check these https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/ovb.yml must be one of those | 12:05 |
| marios | with include_role repo-setup | 12:05 |
| ysandeep|rover | marios: thanks for pointer, will check that | 12:07 |
| ysandeep|rover | I was trying to understand do we need repos on overcloud nodes incase of ovb jobs | 12:08 |
| marios | ysandeep|rover: so perhaps ovb is a speical case i cannot quickly find repo-setup in those plays | 12:08 |
| ysandeep|rover | yeah, I also couldn't find in quick search in logs and suspected if they are coming from our overcloud image itself. | 12:11 |
| marios | ysandeep|rover: i see repo-setup for virthost and undercloud there but not overcloud nodes | 12:11 |
| marios | ysandeep|rover: k so is this what we'll be discussing on friday? | 12:12 |
| marios | or is something related | 12:12 |
| ysandeep|rover | marios: nope.. that's different | 12:12 |
| dviroel | marios: no, podified control plane | 12:12 |
| dviroel | iiuc | 12:12 |
| ysandeep|rover | dviroel: correct | 12:13 |
| marios | ysandeep|rover: ah ok thanks | 12:13 |
| marios | thanks dviroel | 12:13 |
| * frenzyfriday|ruck lunch | 12:28 | |
| rlandy | frenzyfriday|ruck: rechecking the donwstream failed components | 12:28 |
| chandankumar | rlandy: ysandeep|rover ubi9 container image is publically available from registry.access.redhat.com for consumption | 12:50 |
| ysandeep|rover | chandankumar, without login? | 12:50 |
| rlandy | chandankumar: can you testproject run with it and see if it's worth switching? | 12:50 |
| chandankumar | ysandeep|rover: yes without login | 12:51 |
| chandankumar | rlandy: sure | 12:51 |
| rlandy | chandankumar++ | 12:51 |
| rlandy | thanks | 12:51 |
| chandankumar | https://paste.centos.org/view/raw/a0dfdea5 | 12:52 |
| rlandy | frenzyfriday|ruck: checked downstream components - compute 16.2 blocked by known bug | 12:53 |
| rlandy | tripleo - qe job rerun | 12:53 |
| rlandy | tripleo on 17 - in rerun now | 12:53 |
| rlandy | ga patches merging | 12:54 |
| rlandy | ysandeep|rover: ^^ fyi - all above | 12:54 |
| ysandeep|rover | rlandy++ thanks! | 12:57 |
| ysandeep|rover | rlandy, will this fix the deployment (network related validation) issue we were seeing today? | 12:58 |
| rlandy | ysandeep|rover: idk | 13:00 |
| ysandeep|rover | okay | 13:00 |
| chandankumar | ysandeep|rover: marios dviroel podified control plane meeting time | 13:01 |
| Tengu | heya! is this one known? https://zuul.opendev.org/t/openstack/build/70943091f6bc44ce94f7fbb8b3b1c95d apparently something changed in pylint, or a sova related patch was merged without proper linting: ERROR: plugins/modules/sova.py:184:15: use-a-generator: Use a generator instead 'any(i in found.group(1) for i in exclude)' | 13:21 |
| dasm | o/ | 13:21 |
| Tengu | it's making https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/843135 red | 13:22 |
| Tengu | unless it's a false flag and the issue is somewhere else. | 13:22 |
| Tengu | everything's possible in that repo :) | 13:22 |
| bhagyashris | arxcruz, rlandy, marios, ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat, chandankumar, frenzy_friday, anbanerj, dviroel, rcastillo, dasm, jm1, marios, frenzyfriday|ruck ysandeep|rover | 13:23 |
| bhagyashris | Tripleo Ci community meeting in 7 mins | 13:23 |
| bhagyashris | https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both#2022-05-24-Community-Call @ line 28 | 13:23 |
| bhagyashris | https://meet.google.com/bqx-xwht-wky?authuser=0 | 13:23 |
| jm1 | bhagyashris: "Podified control plane weekly sync" still going, some might join late | 13:28 |
| jpodivin | frenzyfriday|ruck: I'm seeing some constraints related failure in the tox-py36 job https://zuul.opendev.org/t/openstack/build/a5f6a5cb3e234950bb9168f29be50a46 is there a bug for it yet? | 13:30 |
| frenzyfriday|ruck | jpodivin, checking | 13:30 |
| jpodivin | So far it seems to be just couple of packages, and it started fairly recently, so it could just be some sort of coincidence. | 13:31 |
| bhagyashris | jm1, ack | 13:32 |
| jpodivin | frenzyfriday|ruck: 4 days ago the glanceclient constraint changed https://opendev.org/openstack/requirements/commit/8c864eb1814fc29dbd8c1623776117ad4c346273 | 13:32 |
| jpodivin | still unless there is conflict with something else, or the version is unavailable ... it should work. | 13:33 |
| frenzyfriday|ruck | hm.. I see a gate failure conflicting in python-novaclient and the check is conflicting oslo-log | 13:36 |
| ysandeep|rover | frenzyfriday|ruck, rlandy fyi.. new bug on master https://bugs.launchpad.net/tripleo/+bug/1975591 | 13:40 |
| frenzyfriday|ruck | jpodivin, it maybe related to https://bugs.launchpad.net/tempest/+bug/1975036 ? | 13:41 |
| jm1 | marios: thanks for the update about mixed rhel mode! | 13:52 |
| rlandy | marios++ | 13:59 |
| marios | o/ jm1 rlandy | 13:59 |
| ysandeep|rover | rlandy, frenzyfriday|ruck chandankumar can you stay back.. want to discuss mirror issue | 14:03 |
| rlandy | ysandeep|rover: ack | 14:03 |
| chandankumar | joining back | 14:03 |
| jm1 | chandankumar: thanks for this tempest intro into skiplist etc. .) | 14:04 |
| bhagyashris | marios, chandankumar, thanks ++ | 14:04 |
| jm1 | marios: you were asking about the streaming thingy in our ci community call. did you see the popup window at the end about someone stopping the recording? | 14:04 |
| marios | jm1: no i did not | 14:06 |
| marios | jm1: perhaps we need to change the google meet id on that | 14:06 |
| marios | but i think we advertise it on the hackmd :) so... | 14:07 |
| marios | :/ | 14:07 |
| marios | well i mean i guess no secrets there but still its a bit weird and creepy | 14:07 |
| ysandeep|rover | rlandy, chandankumar https://review.opendev.org/c/openstack/tripleo-ci/+/843111 | 14:09 |
| chandankumar | thanks everyone! | 14:10 |
| ysandeep|rover | chandankumar++ thank you! | 14:12 |
| ysandeep|rover | rlandy, https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001&branch=master&result=SUCCESS&skip=0 | 14:14 |
| Tengu | folks, apparently there's an update in pylint module that breaks on the ansible-role-collect-logs "sanity" tox check. | 14:14 |
| chandankumar | #oooq, I will be on PTO on May 25 & 26. | 14:15 |
| Tengu | it now complains about a line that was added back last year, and for "missing GPLv3 headers" where it's Apache 2.0... | 14:15 |
| Tengu | 16 | 14:15 |
| jm1 | marios: agree, that was weird and we should change our meet id. bhagyashris would you mind changing the id? | 14:16 |
| jpodivin | frenzyfriday|ruck: seems like it. Thank you | 14:18 |
| ysandeep|rover | rlandy, frenzyfriday|ruck I am out soon - please see https://review.opendev.org/c/openstack/tripleo-ci/+/843111 merges. | 14:38 |
| rlandy | failing zuu; | 14:39 |
| rlandy | ysandeep|rover: ^^ | 14:39 |
| rlandy | changes to be made? | 14:39 |
| ysandeep|rover | rlandy, checking | 14:39 |
| ysandeep|rover | hmmm. Error: Unable to find a match: epel-release | 14:41 |
| ysandeep|rover | ahh, patchset 2 caused this.. was working in patchset 1 | 14:50 |
| ysandeep|rover | epel-release noarch 9-2.el9 extras-common 17 k | 14:50 |
| ysandeep|rover | ykarel, ^^ fyi.. we do pull some stuff from centos-addons.repo | 14:50 |
| ysandeep|rover | dviroel, https://logserver.rdoproject.org/57/42657/2/check/periodic-tripleo-centos-9-buildimage-ironic-python-agent-master/8a87c07/yum.repos.d/centos-addons.repo , We don't have any quickstart-* repo alternative for [extras-common] repo right? | 14:52 |
| ykarel | ysandeep|rover, okk, good to check why epel-release is needed at all, and avoid if possible | 14:52 |
| ykarel | we shouldn't use epel | 14:53 |
| ysandeep|rover | ykarel: thanks for suggestion, I am reverting back to previous patchset#1 for now to fix the immediate issue, and I will investigate about epel-release removal in a followup. | 14:54 |
| dviroel | ysandeep|rover: dont think so, extras is commented out on release files | 14:54 |
| ysandeep|rover | dviroel, thanks! | 14:55 |
| ykarel | ysandeep|rover, ack | 14:55 |
| dviroel | ysandeep|rover: http://mirror.regionone.vexxhost-nodepool-tripleo.rdoproject.org/centos-stream/SIGs/9-stream/extras/x86_64/extras-common/ | 14:58 |
| dviroel | but we dont set it up | 14:58 |
| ysandeep|rover | dviroel: thanks! | 15:00 |
| ysandeep|rover | dviroel, ykarel, rlandy fyi.. updated https://review.opendev.org/c/openstack/tripleo-ci/+/843111 | 15:01 |
| rlandy | ysandeep|rover: ok - will watch | 15:01 |
| rlandy | thank you | 15:01 |
| ysandeep|rover | ykarel++ thanks for your help today | 15:06 |
| ysandeep|rover | rlandy, fyi.. I have pinged harald and steve about https://bugs.launchpad.net/tripleo/+bug/1975588 wallaby node provision bug. | 15:06 |
| rlandy | ysandeep|rover: thanks - working on rhel-9 stuff | 15:06 |
| ysandeep|rover | rlandy, frenzyfriday|ruck I am out, Will investigate ntp sync issue in my morning | 15:07 |
| rlandy | k - great | 15:07 |
| frenzyfriday|ruck | ysandeep|rover, thanks :) have a good night | 15:07 |
| rlandy | frenzyfriday|ruck: is this the same error you were investigating: https://sf.hosted.upshift.rdu2.redhat.com/logs/8f/8f7363ec17f415ba57fe75cbae9c8b07a9ca5d71/openstack-periodic-integration-rhos-17-rhel9/periodic-tripleo-ci-rhel-9-scenario007-standalone-rhos-17/85b8dee/logs/undercloud/home/zuul/standalone_deploy.log | 15:12 |
| frenzyfriday|ruck | rlandy, yep. Everything in rhos17 rhel9 is hitting this | 15:12 |
| rlandy | ok | 15:12 |
| rlandy | trying to fix image build | 15:12 |
| rlandy | then will report the rest of the errors are the same | 15:13 |
| ysandeep|rover | >> then will report the rest of the errors are the same - in program call? | 15:15 |
| *** ysandeep|rover is now known as ysandeep|out | 15:17 | |
| rlandy | frenzyfriday|ruck" what's the train situation? | 15:19 |
| rlandy | can we promote that? | 15:19 |
| ysandeep|out | rlandy, fyi.. train fs020 passed - fs035/001 failed in transient issues again - i have rekicked them here : https://review.rdoproject.org/r/c/testproject/+/36256 | 15:19 |
| rlandy | ysandeep|out: great | 15:19 |
| rlandy | was just asking | 15:19 |
| rlandy | ysandeep|out: if they fail with diff tempest tests should I skip and promo? | 15:20 |
| ysandeep|out | rlandy, yes please.. | 15:20 |
| rlandy | k | 15:20 |
| ysandeep|out | will be good for program call status - good to have some green | 15:20 |
| rlandy | ysandeep|out: ok | 15:20 |
| rlandy | ysandeep|out: we can discuss 16.2 tomorrow | 15:21 |
| ysandeep|out | ack | 15:21 |
| rlandy | I'm trying to clear rhel 9 image builds | 15:21 |
| rlandy | new errors there | 15:21 |
| ysandeep|out | rlandy, ykarel is onto something for those tempest related issues, hopefully we will have something to share tomorrow | 15:22 |
| rlandy | ykarel++ | 15:22 |
| rlandy | great | 15:22 |
| bhagyashris | jm1, sure will change that | 15:56 |
| *** dviroel is now known as dviroel|lunch | 15:58 | |
| *** marios is now known as marios|out | 15:58 | |
| rlandy | lunch - brb | 16:19 |
| *** jpena is now known as jpena|off | 16:51 | |
| frenzyfriday|ruck | rlandy, I added 2 testproj fro master components - one which has random failures and the other to be run after fence-agent is fixed. Wallaby c9 might have real tempest issues, checking | 16:51 |
| rlandy | frenzyfriday|ruck: ok | 16:51 |
| rlandy | I'm interested in getting train promo'ed now | 16:52 |
| rlandy | frenzyfriday|ruck: all downstream components should have promoted other than rhel-9 | 16:53 |
| rlandy | and compute on 16.2 | 16:53 |
| rlandy | legit issues | 16:53 |
| frenzyfriday|ruck | \O/ thanks! | 16:53 |
| rlandy | working on rhel 9 now | 16:53 |
| rlandy | for image builds | 16:54 |
| *** dviroel|lunch is now known as dviroel | 16:58 | |
| dviroel | \o/ | 16:58 |
| frenzyfriday|ruck | rlandy, I am leaving for the day in ~15 mins. Could you pls recheck https://review.rdoproject.org/r/c/testproject/+/43113 if the fence-agent patch merges? | 17:12 |
| rlandy | yep | 17:13 |
| frenzyfriday|ruck | I have updated the rest on hackmd - will check the tempest errors tomorrow morning | 17:13 |
| rlandy | frenzyfriday|ruck: pls remind me which one is the fencing patch | 17:13 |
| rlandy | https://review.opendev.org/c/openstack/tripleo-ci/+/843111? | 17:14 |
| frenzyfriday|ruck | rlandy, https://review.opendev.org/c/openstack/tripleo-ci/+/843111 | 17:14 |
| rlandy | ok - got it | 17:14 |
| rlandy | watching the same one for sandeeo | 17:14 |
| frenzyfriday|ruck | this should block the repo and fix it | 17:14 |
| frenzyfriday|ruck | thanks :) | 17:14 |
| rlandy | cool | 17:14 |
| rlandy | have a good night | 17:15 |
| frenzyfriday|ruck | have a nice rest of the day :) | 17:15 |
| rlandy | cannot install both NetworkManager-1:1.39.2-1.el9.x86_64 and NetworkManager-1:1.39.5-1.el9.x86_64\n - cannot install both NetworkManager-1:1.39.3-1.el9.x86_64 and NetworkManager-1:1.39.5-1.el9.x86_64\n - cannot install the best update candidate for package os-net-config-14.2.1-0.20220419122345.19307e0.el9.noarch\n - problem with installed package NetworkManager- | 17:57 |
| rlandy | ugh - here we go again | 17:57 |
| rlandy | rcastillo: hey | 20:32 |
| rlandy | rcastillo: did we update something in openstacksdk? | 20:32 |
| rlandy | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_c49/841188/1/gate/tripleo-ci-centos-9-standalone/c49cb31/job-output.txt | 20:32 |
| rlandy | ft). | 20:33 |
| rlandy | 2022-05-24 20:15:38.850604 | primary | failed: [undercloud -> undercloud] (item=tempest) => {"ansible_loop_var": "item", "attempts": 5, "changed": false, "item": "tempest", "msg": "To utilize this module, the installed version of the openstacksdk library MUST be >=0.99."} | 20:33 |
| * dasm disappears | 21:05 | |
| dasm | Take care Team! | 21:05 |
| rcastillo | rlandy: that error must be related to https://review.opendev.org/c/openstack/ansible-collections-openstack/+/839691 | 21:06 |
| rcastillo | I don't think we should be using master in our jobs yet so something must be wrong | 21:06 |
| *** dviroel is now known as dviroel|out | 21:11 | |
| rlandy | rcastillo: wrong on whose side? | 22:08 |
| rlandy | rcastillo: ^^ this is killing everything | 22:27 |
| rcastillo | rlandy: tripleo-ci I believe | 22:27 |
| rlandy | rcastillo: can you meet for 5? | 22:28 |
| rlandy | can we revert https://review.opendev.org/c/openstack/ansible-collections-openstack/+/839691? | 22:28 |
| rcastillo | rlandy: yes and yes | 22:29 |
| rcastillo | I _may_ have a proper fix though | 22:29 |
| rlandy | https://meet.google.com/jsf-mpjq-bjc?pli=1&authuser=0 | 22:30 |
| rlandy | rcastillo^^ | 22:30 |
| rcastillo | https://review.opendev.org/c/openstack/tripleo-ci/+/843220 | 22:30 |
| rlandy | rcastillo: https://bugs.launchpad.net/tripleo/+bug/1975646 | 22:37 |
| rlandy | rcastillo: pls add to your suggested fix | 22:38 |
| rcastillo | rlandy: ack, updated patch | 22:38 |
| rcastillo | let's see how this goes | 22:38 |
| rlandy | ysandeep|out: frenzyfriday: ^^ fyi - two new bugs - added to rr hackmd | 22:42 |
| rlandy | rcastillo: thanks - if that passes standalone - should be ok | 22:43 |
| rcastillo | I'll keep an eye on it | 22:46 |
| rlandy | rcastillo: thanks - sent email to mailing list | 22:49 |
| rlandy | rcastillo: taking out downstream as well - just fyi | 23:40 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!