| *** ysandeep|out is now known as ysandeep|holiday | 01:28 | |
| *** rlandy|bbl is now known as rlandy | 02:02 | |
| *** rlandy is now known as rlandy|out | 02:15 | |
| marios | \o | 06:12 |
|---|---|---|
| chandankumar | marios: o/ | 07:02 |
| marios | \o | 07:14 |
| *** amoralej|off is now known as amoralej | 07:32 | |
| Tengu|rover | hello there | 07:35 |
| Tengu|rover | last day as rover :). | 07:37 |
| Tengu|rover | time to squash issues, so that handover will be smooth | 07:37 |
| Tengu|rover | bhagyashris|ruck: guess we'll set some call with Pooja and Doug for the passover? | 07:51 |
| Tengu|rover | *handover. whatever. | 07:51 |
| amoralej | hi, we are hitting issues related to pyroute2 version with master jobs in neutron | 07:53 |
| amoralej | dunno if you are seeing similar ones, i'll update pyroute2 but it may take some time as we need to get it in fedora too | 07:53 |
| Tengu|rover | amoralej: ah? maybe linked to the neutron component line being a bit red (with some green)? | 07:59 |
| Tengu|rover | I was wanting to investigate it, it fails on tempest. | 07:59 |
| Tengu|rover | folks, care to add https://review.opendev.org/c/openstack/tripleo-quickstart/+/871730 to your review list? | 07:59 |
| reviewbot | I have added your review to the Review list | 07:59 |
| amoralej | Tengu|rover, you have logs that i can check? | 07:59 |
| Tengu|rover | amoralej: gimme a few, I'm switching from tabs to tabs | 08:00 |
| amoralej | https://logserver.rdoproject.org/openstack-periodic-integration-main/review.rdoproject.org/rdoinfo/master/weirdo-antelope-promote-puppet-scenario002-centos-stream-9/4588687/logs/weirdo-project/logs/neutron/dhcp-agent.txt.gz | 08:00 |
| amoralej | that's the kind of logs i've seen | 08:00 |
| amoralej | note that's ml2/ovs, not ovn | 08:00 |
| Tengu|rover | amoralej: https://bugs.launchpad.net/tripleo/+bug/2003884 is the LP I opened in order to track things, didn't really investigate yet | 08:01 |
| amoralej | but i think ovn also fails | 08:01 |
| amoralej | https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario007-standalone-network-master/64f1e09/logs/undercloud/var/log/containers/neutron/dhcp-agent.log.txt.gz | 08:02 |
| amoralej | yeah, same error | 08:02 |
| Tengu|rover | wow, if that wasn't a good timing :) | 08:02 |
| Tengu|rover | amoralej: care to comment in the LP? | 08:02 |
| amoralej | sure | 08:02 |
| Tengu|rover | ++ thanks!! | 08:02 |
| Tengu|rover | I don't think I'd find it alone | 08:03 |
| Tengu|rover | and you even have an LP for the pinning :) | 08:03 |
| Tengu|rover | or update. or whatever patch is needed :) | 08:03 |
| marios | Tengu|rover: probably only doug around today bhagyashris|ruck and pooja are out for public holiday (regarding handoff) | 08:05 |
| marios | Tengu|rover: do you want to send a meeting invite for half hour before scrum? | 08:05 |
| amoralej | Tengu|rover, i'm working to update pyroute2, it it takes too long we can pin neutron but i prefer to avoid it if possible | 08:05 |
| Tengu|rover | marios: ah, right, I just saw Rabi talking about public holidays. | 08:05 |
| Tengu|rover | marios: sure | 08:05 |
| Tengu|rover | amoralej: sure - at least we have a cause, a way out, so things are under control :) | 08:06 |
| Tengu|rover | this is lovely | 08:06 |
| Tengu|rover | marios: stupid question.... which doug again? ^^' | 08:07 |
| Tengu|rover | got it | 08:09 |
| Tengu|rover | invite sent | 08:09 |
| marios | Tengu|rover: also add rlandy_ for that meeting | 08:18 |
| Tengu|rover | oh, right | 08:20 |
| Tengu|rover | ok you added her already | 08:21 |
| *** jpena|off is now known as jpena | 08:32 | |
| Tengu|rover | rlandy_: heya! I'm working on the tripleo-ci change allowing to get the latest image - I introduced a stupid thing (tabs instead of spaces, due to some wild copy-paste) | 11:02 |
| Tengu|rover | rlandy_: would get that change green first, then we can concentrate on the downstream testproject. | 11:02 |
| Tengu|rover | since the change triggers some image-build, we'll at least know a bit more. | 11:03 |
| Tengu|rover | marios: should I kick the openstack-component-tripleo in order to trigger a promotion of that component? | 11:09 |
| Tengu|rover | that would make things clearer. iirc it's running once a day now? | 11:09 |
| marios | o/ Tengu|rover did the haproxy pin help us? (or is the line red for something else? | 11:15 |
| marios | looking | 11:15 |
| Tengu|rover | marios: well, it didn't re-kick, soooo | 11:15 |
| marios | ah | 11:15 |
| Tengu|rover | I just launched one periodic in testproject to check the state. | 11:15 |
| Tengu|rover | https://review.rdoproject.org/r/c/testproject/+/46701 | 11:15 |
| Tengu|rover | should help seeing how things are now. | 11:15 |
| Tengu|rover | I'd rather avoid passing a red line to the next rr shift :) | 11:16 |
| Tengu|rover | it's running periodic-tripleo-ci-centos-9-scenario004-standalone-tripleo-master right now | 11:16 |
| *** rlandy|out is now known as rlandy | 11:16 | |
| Tengu|rover | SHOULD help hopefully. | 11:16 |
| marios | Tengu|rover: just concerned about rekicking the whole line because resources | 11:17 |
| Tengu|rover | marios: that's why I kick just one job | 11:17 |
| Tengu|rover | :) | 11:17 |
| marios | Tengu|rover: but we should verify with a test at least | 11:17 |
| marios | right | 11:17 |
| Tengu|rover | if it's green, the, we can hope either next run will be fine, or maybe consider a re-kick. | 11:18 |
| Tengu|rover | if it's red.... #digging | 11:18 |
| marios | so in fact... it didn't run yet today right? Tengu|rover https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/upstream.yaml#L455 | 11:18 |
| Tengu|rover | marios: well, if it ran, it failed, because pin wasn't in place. | 11:19 |
| Tengu|rover | as said, it merged earlier today | 11:19 |
| Tengu|rover | and... lemme check | 11:19 |
| marios | Tengu|rover: so no need to kick it will run this evening | 11:19 |
| Tengu|rover | 2023-01-25 20:30 it did run yesterday | 11:19 |
| marios | Tengu|rover: - time: '30 19 * * *' | 11:19 |
| marios | from 13:18 < marios> so in fact... it didn't run yet today right? Tengu|rover https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/upstream.yaml#L455 | 11:19 |
| rlandy | Tengu|rover: hi - hang in there | 11:20 |
| Tengu|rover | marios: it did run, took about 1h, so it matches. | 11:20 |
| Tengu|rover | rlandy: I'll go for lunch in a few | 11:20 |
| rlandy | bhagyashris|ruck I think is on holiday | 11:20 |
| Tengu|rover | rlandy: yep, she is | 11:20 |
| rlandy | Tengu|rover: np - will check in with you later | 11:20 |
| Tengu|rover | marios: I'll wait for the testproject result then, and that's about it. | 11:20 |
| rlandy | Tengu|rover: ok - I'm still working on 17.1 | 11:20 |
| Tengu|rover | I'll update the hackmd accordingly. | 11:20 |
| Tengu|rover | rlandy: same, with the tripleo-ci patch | 11:21 |
| Tengu|rover | rlandy: had to update it for some stupid tabs vs space - and job is running upstream. It should be greener - and if it's ok, we'll be able to re-kick downstream testproject. | 11:21 |
| Tengu|rover | ok, have to go - lunch is ready. Back in ~1h | 11:21 |
| *** dviroel|out is now known as dviroel | 11:32 | |
| *** dviroel_ is now known as dviroel | 11:45 | |
| rlandy | Tengu|rover: I think I know what the issue is | 11:49 |
| rlandy | trying another approach | 11:49 |
| Tengu|rover | rlandy: back. | 12:06 |
| Tengu|rover | rlandy: I think the tripleo-ci patch is working now. | 12:06 |
| Tengu|rover | at least it still didn't crash the checks. | 12:06 |
| Tengu|rover | marios: dang.. either pin isn't in, or there's something still crashing tripleo component. | 12:08 |
| Tengu|rover | lemme inestigate. | 12:08 |
| Tengu|rover | (or the pin didn't help) | 12:08 |
| Tengu|rover | meh. same error. | 12:09 |
| Tengu|rover | amoralej: heya! just a stupid question: is the rdoinfo pin for puppet-haproxy really correct? Or is there anything to do in order to get it actually applied? I launched a job ~1h ago (so rdoinfo patch was properly merged by then), and it seems to still install the wrong puppet-haproxy version... | 12:13 |
| Tengu|rover | rlandy: did I do anything wrong with the way I pass the vars down? | 12:13 |
| marios | Tengu|rover: o/ link to logs? | 12:13 |
| Tengu|rover | marios: https://logserver.rdoproject.org/01/46701/3/check/periodic-tripleo-ci-centos-9-scenario004-standalone-tripleo-master/d58a725/logs/undercloud/var/log/extra/package-list-installed.txt.gz | 12:14 |
| Tengu|rover | here we see the puppet-haproxy 6.4.1-0.20230124153846.95ca912.el9 - it's the wrong version | 12:14 |
| marios | not sure lets wait for amoralej to double check ( amoralej do we need to wait for something else i.e. https://review.rdoproject.org/r/c/rdoinfo/+/46709 should be available as soon as it merges no?) | 12:20 |
| Tengu|rover | marios: ah :). basically the same question. good, I'm not alone | 12:21 |
| marios | Tengu|rover: yeah in fact that 95ca912 is the breaking version (one we want to avoid) https://github.com/puppetlabs/puppetlabs-haproxy/commit/95ca912f77966ed0da7a5bec9265cdacbf09da4c | 12:21 |
| marios | Tengu|rover: cant see something else in logs 14:21 < Tengu|rover> marios: ah :). basically the same question. good, I'm not alone | 12:21 |
| Tengu|rover | marios: exactly. we want the path just before | 12:21 |
| Tengu|rover | marios: 13:13 CET - at least I have it here. | 12:22 |
| marios | Tengu|rover: then... re-add vote there please :) https://review.opendev.org/c/openstack/puppet-tripleo/+/871807 and lets start also pushing that direction | 12:24 |
| marios | Tengu|rover: can you please add info into the cix about the pin not helping us with link to logs etc | 12:25 |
| Tengu|rover | sure. | 12:25 |
| Tengu|rover | done. | 12:27 |
| Tengu|rover | ok, Takashi's patch seems all good, jobs are running tempest, meaning they could deploy haproxy. | 12:32 |
| Tengu|rover | so we may just get the fix before the pin is effective/applied/whatever | 12:32 |
| Tengu|rover | rlandy: afaik tripleo-ci patch is good now. Lemme know if you want to sync on the 17.1 side. | 12:33 |
| rlandy | Tengu|rover: yep - sorry was in meeting - can chat now | 12:33 |
| Tengu|rover | rlandy: :) so, what's your idea? why would the param not be properly passed down? :/ | 12:34 |
| rlandy | Tengu|rover: https://meet.google.com/qdu-jqkb-phe?pli=1&authuser=0 | 12:34 |
| Tengu|rover | joining | 12:35 |
| Tengu|rover | rlandy: tripleo-ci-internal-jobs is up-to-date now. | 12:52 |
| Tengu|rover | no more merge conflict in addition. | 12:52 |
| Tengu|rover | we can re-kick the internal testproject. | 12:52 |
| Tengu|rover | wait... what... arrg | 12:53 |
| Tengu|rover | NOW it's rebased. forgot the git review -d doesn't rebase.... | 12:53 |
| Tengu|rover | rlandy: kicked the testproject. | 12:54 |
| amoralej | Tengu|rover, marios sorry i need to force rebuild as we are reverting to previous version | 12:55 |
| amoralej | gimme a while | 12:55 |
| Tengu|rover | amoralej: ah, that's what you said yesterday - I didn't get there was something manual to do. | 12:55 |
| amoralej | it's done via dummit commit to the distgit | 12:55 |
| Tengu|rover | 'k | 12:56 |
| Tengu|rover | rdoinfo distgit? | 12:56 |
| Tengu|rover | or puppet-haproxy... ? | 12:56 |
| amoralej | Tengu|rover, https://review.rdoproject.org/r/c/puppet/puppet-haproxy-distgit/+/46724 | 12:57 |
| amoralej | that one | 12:57 |
| Tengu|rover | ok! | 12:58 |
| Tengu|rover | ah, yeah. it's really, really a dummy commit XD | 12:58 |
| marios | thanks amoralej /me vaguely remembers the dummy commit now... | 13:03 |
| amoralej | it's tricky, i know :) | 13:03 |
| marios | (and it was not even *that* long ago from 2022 https://review.rdoproject.org/r/c/openstack/openstacksdk-distgit/+/43214 ) | 13:05 |
| Tengu|rover | amoralej: anddd... is it also needed when we remove a pin? "asking for a friend" (truth: proper fix will be nudged to gate in the following 30 minutes or so, so we'll get it before EOD) | 13:08 |
| amoralej | Tengu|rover, nop | 13:09 |
| Tengu|rover | good. so just plain revert on rdoinfo, and once it's merge, next «whatever» will rebuild latest | 13:09 |
| amoralej | as removing the pin is moging ahead | 13:09 |
| amoralej | Tengu|rover, exactly | 13:09 |
| Tengu|rover | hmm, I could propose the revert now, with a depends-on the proper fix? | 13:09 |
| amoralej | Tengu|rover, whichç's the fix? | 13:12 |
| Tengu|rover | amoralej: https://review.opendev.org/c/openstack/puppet-tripleo/+/871807 | 13:12 |
| Tengu|rover | amoralej: https://review.rdoproject.org/r/c/rdoinfo/+/46725 revert's ready. | 13:15 |
| amoralej | i think that should be fine | 13:17 |
| Tengu|rover | since I wasn't really sure about the delay for the proper fix.... | 13:17 |
| amoralej | the only point is to make sure the new commit of tripleo is rebuilt in rdo before revert is merged, but that should be just minutes | 13:17 |
| Tengu|rover | in the end the pin may have been useless, but... | 13:17 |
| Tengu|rover | amoralej: so that's why I added the depends-on | 13:17 |
| Tengu|rover | I can -2 it for the time being if needed. | 13:18 |
| Tengu|rover | marking as WIP for now. | 13:18 |
| rlandy | Tengu|rover: ugh - templating | 13:18 |
| amoralej | that's good, just in case | 13:18 |
| Tengu|rover | rlandy: uh? | 13:19 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/e821b6e9b4724b2d87f91d9fa721d796?logfile=console.log | 13:19 |
| *** amoralej is now known as amoralej|lunch | 13:19 | |
| Tengu|rover | rlandy: "end of stream". | 13:19 |
| rlandy | Tengu|rover: TASK [discover-latest-image : Fetch publication page] | 13:19 |
| rlandy | 2023-01-26 13:06:52.259054 | primary | ERROR | 13:19 |
| rlandy | 2023-01-26 13:06:52.259485 | primary | { | 13:19 |
| rlandy | 2023-01-26 13:06:52.259541 | primary | "msg": "An unhandled exception occurred while templating '{{ base_url }}' | 13:19 |
| Tengu|rover | guess it didn't like something | 13:19 |
| Tengu|rover | fuuuun | 13:19 |
| Tengu|rover | would be good to see what block it hit... | 13:20 |
| * Tengu|rover has far, far too many tabs opened | 13:21 | |
| Tengu|rover | rlandy: lemme do another check.. | 13:24 |
| rlandy | marios: dviroel: Tengu|rover: I'm on a meeting - will miss RR sync | 13:31 |
| marios | Tengu|rover:++ | 13:41 |
| marios | np rlandy | 13:41 |
| Tengu|rover | rlandy: gimme a moment to update the downstream things, I've renamed parameters to match things in a better "ansible" way. | 13:42 |
| Tengu|rover | rlandy: hmmm.... not sure if you kicked the recheck just too soon | 13:48 |
| Tengu|rover | good | 13:49 |
| Tengu|rover | dviroel: fyi, the testproject for tripleo component: https://review.rdoproject.org/r/c/testproject/+/46701 | 13:49 |
| Tengu|rover | (added to the rr hackmd) | 13:50 |
| dviroel | Tengu|rover: ack, tks | 14:00 |
| dviroel | marios: rlandy: tomorrow I will start some hours late, I have a doc appt in my morning. I should be around for or podified meeting | 14:02 |
| *** dasm|off is now known as dasm | 14:03 | |
| dasm | o/ | 14:03 |
| rlandy | Tengu|rover: passed??? | 14:14 |
| rlandy | dviroel: no worries - will cover you | 14:14 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/90/438890/14/check/tripleo-rhel-9-buildimage-ironic-python-agent-rhos-17.1/f147b74/logs/build.log.txt.gz | 14:15 |
| Tengu|rover | rlandy: yeeeahhh | 14:17 |
| Tengu|rover | \o/ | 14:17 |
| rlandy | Tengu|rover: looks like it took 9.1 image | 14:17 |
| rlandy | as base | 14:17 |
| Tengu|rover | don't tell me that. | 14:17 |
| *** amoralej|lunch is now known as amoralej | 14:37 | |
| chandankumar | marios: rlandy we are adding crc rhel/centos-8 based label in downstream zuul | 15:08 |
| chandankumar | in case you need for Adoption ci work | 15:08 |
| marios | thanks chandankumar | 15:12 |
| chandankumar | Do let me know when you start wiring up the job | 15:13 |
| marios | chandankumar: you mean c9 though right? "adding crc rhel/centos-8 " | 15:13 |
| marios | chandankumar: ok we can talk next week about that | 15:13 |
| chandankumar | crc is based on rhel/centos-8 | 15:13 |
| chandankumar | cs/rhel-9 have some issue | 15:13 |
| chandankumar | daniel is already on it | 15:13 |
| chandankumar | sure | 15:14 |
| marios | chandankumar: ah ok (you mean in zuul/label has issues ) ... cos my dev box is 9-stream with crc | 15:14 |
| chandankumar | marios: nah, the issue is related to openstack cloud with cs/rhel-9 crc node | 15:15 |
| chandankumar | it will not block your testing | 15:15 |
| marios | ack | 15:15 |
| * dviroel lunch | 15:40 | |
| chandankumar | see ya people tomorrow o/ | 15:51 |
| *** marios is now known as marios|out | 17:02 | |
| Tengu|rover | OK Team - signing off. See you tomorrow | 17:05 |
| *** Tengu|rover is now known as Tengu | 17:05 | |
| *** amoralej is now known as amoralej|off | 17:27 | |
| *** jpena is now known as jpena|off | 17:32 | |
| *** dviroel is now known as dviroel|out | 20:58 | |
| *** dasm is now known as dasm|off | 23:13 | |
| dasm|off | o/ | 23:13 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!