ysandeep | hello oooci and good morning o/ | 06:05 |
---|---|---|
akahat|ruck | ysandeep, o/ hello Good Morning. | 06:07 |
marios | \o | 06:09 |
akahat|ruck | arxcruz|rover, o/ | 06:51 |
arxcruz|rover | akahat|ruck hey | 06:51 |
arxcruz|rover | good morning | 06:51 |
akahat|ruck | arxcruz|rover, good morning!! | 06:51 |
akahat|ruck | arxcruz|rover, let's have sync. | 06:51 |
akahat|ruck | possible now? or after some time? | 06:52 |
arxcruz|rover | akahat|ruck give me 10 minutes, i'm puting some jogs to re-run | 06:52 |
arxcruz|rover | sounds good for you? | 06:52 |
akahat|ruck | arxcruz|rover, i've already put some master /train /wallaby job in re-run | 06:53 |
akahat|ruck | https://review.rdoproject.org/zuul/status#41469 | 06:53 |
arxcruz|rover | akahat|ruck ouch xD | 06:58 |
arxcruz|rover | akahat|ruck ok, let's sync | 06:58 |
arxcruz|rover | wait, let me grab another coffee | 06:58 |
akahat|ruck | arxcruz|rover, ok | 06:59 |
akahat|ruck | arxcruz|rover, https://meet.google.com/ssw-rukz-acx?pli=1 | 06:59 |
jm1 | moin folks :) | 07:00 |
akahat|ruck | arxcruz|rover, https://review.rdoproject.org/r/c/testproject/+/41465 | 07:04 |
akahat|ruck | arxcruz|rover, http://pastebin.test.redhat.com/1067204 | 07:25 |
arxcruz|rover | akahat|ruck https://code.engineering.redhat.com/gerrit/c/testproject/+/421363 | 07:29 |
akahat|ruck | arxcruz|rover, thank you !! :) | 07:30 |
arxcruz|rover | reviewbot | 07:36 |
arxcruz|rover | reviewbot please add https://review.opendev.org/c/openstack/tripleo-quickstart/+/850568 | 07:37 |
arxcruz|rover | reviewbot please add https://review.opendev.org/c/openstack/tripleo-quickstart/+/850568 to review list | 07:37 |
reviewbot | I have added your review to the Review list | 07:37 |
*** ysandeep is now known as ysandeep|afk | 07:49 | |
* akahat|ruck lunch | 07:55 | |
arxcruz|rover | akahat|ruck https://code.engineering.redhat.com/gerrit/c/testproject/+/421363 failed | 07:56 |
jm1 | akahat|ruck: 3 passes in a row, wanna close this one? https://bugzilla.redhat.com/show_bug.cgi?id=2109807 | 08:46 |
* bhagyashris lunch | 08:49 | |
marios | need review please https://review.opendev.org/c/openstack/tripleo-ci/+/841583 thank you | 08:49 |
reviewbot | I have added your review to the Review list | 08:49 |
marios | chandankumar: thanks for voting, sorry i noticed something just now and will update (moving thje job to minimal template because https://review.opendev.org/c/openstack/tripleo-ci/+/841583/26/zuul.d/multinode-jobs.yaml#161 | 08:55 |
marios | (its going to wipe your vote chandankumar sorry and thanks again) | 08:56 |
chandankumar | ack sir | 08:57 |
marios | thanks chandankumar | 09:04 |
akahat|ruck | jm1, we have green run. You can close that. | 09:36 |
jm1 | akahat|ruck: please go ahead and close it :) | 09:38 |
*** ysandeep|afk is now known as ysandeep | 09:43 | |
rlandy | arxcruz|rover: akahat|ruck: hey - how are things? | 10:35 |
rlandy | want to sync? | 10:35 |
arxcruz|rover | rlandy everything is in control, i'm fine to sync if you want | 10:35 |
akahat|ruck | rlandy, o/ | 10:36 |
akahat|ruck | rlandy, yes. | 10:36 |
rlandy | arxcruz|rover: akahat|ruck - yeah ... it will be quick ... https://meet.google.com/hwe-umtv-gii?pli=1&authuser=0 | 10:36 |
*** jpodivin_ is now known as jpodivin | 11:01 | |
arxcruz|rover | rlandy | 11:06 |
arxcruz|rover | rlandy https://bugs.launchpad.net/tripleo/+bug/1982464 is a tracking but only | 11:06 |
rlandy | frenzy_friday: dviroel_: cockpit downstream is not collecting data | 11:11 |
rlandy | probably from the weekend outage | 11:12 |
rlandy | frenzy_friday: dviroel_: can you check that? maybe needs a restart? | 11:12 |
rlandy | akahat|ruck; jm1: https://bugs.launchpad.net/tripleo/+bug/1982552 - is unvalid the right state here? | 11:13 |
frenzy_friday | rlandy, checking | 11:13 |
akahat|ruck | rlandy, yes. that was invalid.. but Rabi has comment to change version to 1.0.0 | 11:14 |
akahat|ruck | may be chandankumar, jm1 ysandeep can comment on this. | 11:14 |
*** jgilaber is now known as jgilaber|lunch | 11:16 | |
marios | review time o/ join us | 11:17 |
chandankumar | akahat|ruck: https://github.com/ansible/ansible/issues/68950 | 11:17 |
chandankumar | akahat|ruck: Do you have the full error log? | 11:29 |
chandankumar | of the above failure | 11:29 |
akahat|ruck | chandankumar, https://bugzilla.redhat.com/show_bug.cgi?id=2109807 | 11:30 |
ysandeep | marios, https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/850506/1#message-6d5c31f3efeaeceece69705e7825d6e2086bcfe9 | 11:33 |
*** dviroel_ is now known as dviroel | 11:43 | |
dviroel | o/ | 11:45 |
frenzy_friday | the downstream cockpit containers have connection error to sf.hosted.upshift.rdu2.redhat.com restarting now | 11:47 |
jm1 | akahat|ruck, chandankumar, rlandy: https://bugs.launchpad.net/tripleo/+bug/1982552 is invalid because only downstream is affected | 11:50 |
jm1 | akahat|ruck, chandankumar, rlandy: this is the downtream bug https://bugzilla.redhat.com/show_bug.cgi?id=2109807 | 11:50 |
jm1 | akahat|ruck, chandankumar, rlandy: it occured once. the last three runs it passed. we have no idea why it failed. logs dont help here, see bug report comments | 11:51 |
frenzy_friday | rlandy, I have restarted the containers in incockpit. The data should show up in some time | 12:05 |
rlandy | frenzy_friday: thank you | 12:07 |
rlandy | jm1: thanks for confirming | 12:07 |
*** ysandeep is now known as ysandeep|afk | 12:13 | |
dviroel | frenzy_friday: hi, rebooting fixed the issue? | 12:24 |
frenzy_friday | dviroel, I see almost all the data now but http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/RMjmDrEWk/downstream-dashboard?orgId=1 -baremetal, rhos 17.0 trunk-patches, 16.0 are missing | 12:26 |
frenzy_friday | http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/gHXbeNp7z/centos-9-and-centos-8-comparison-master?orgId=1 some data is missing here as well | 12:27 |
frenzy_friday | In the logs I can still see influxdb doing the queries, so maybe it needs more time? | 12:28 |
dviroel | frenzy_friday: yeah, i think that trunk-patches wasn't showing nothing before, if you switch to display data from 30 days ago, still don't have anything | 12:30 |
frenzy_friday | yep | 12:31 |
* dviroel coffee | 12:32 | |
marios | 15:35 #tripleo: < opendevreview> Marios Andreou proposed openstack/tripleo-quickstart master: Remove undercloud_enable_paunch from fs and tidy up fs 66 https://review.opendev.org/c/openstack/tripleo-quickstart/+/850881 | 12:36 |
marios | oops | 12:36 |
bhagyashris | rlandy, https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/411458/19/zuul.d/standalone-jobs.yaml | 12:37 |
chandankumar | rlandy++ frenzy_friday++ thanks, learnt to generate zuul secrets :-) | 12:37 |
frenzy_friday | chandankumar, Cool :D | 12:38 |
marios | jm1: o/ fyi https://review.opendev.org/c/openstack/tripleo-upgrade/+/850509/1#message-62f7c453d270192fd6b054c5c5db408ec30a9f25 (discussed in today's reviews call) | 12:40 |
reviewbot | Do you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks. | 12:40 |
* akahat|ruck brb | 12:41 | |
chandankumar | jm1: thank you for the links :-) | 12:41 |
jm1 | marios: ack, thanks! | 12:42 |
bhagyashris | https://review.opendev.org/c/openstack/tripleo-ci/+/842973/5/zuul.d/standalone-jobs.yaml | 12:42 |
marios | jm1: just keeping that one fyi https://review.opendev.org/c/openstack/tripleo-quickstart/+/850507 | 12:46 |
rlandy | arxcruz|rover: akahat|ruck: what time is CIX on your calendar? | 13:24 |
arxcruz|rover | rlandy in one hour | 13:25 |
arxcruz|rover | is it now ? | 13:25 |
rlandy | arxcruz|rover: it did move - talking with apevec now | 13:26 |
akahat|ruck | rlandy, same | 13:26 |
rlandy | on rhos-ops | 13:26 |
rlandy | clashes with our scrum meeting | 13:26 |
rlandy | arxcruz|rover: akahat|ruck: guess you will have to miss second half of scrum to attend | 13:28 |
rlandy | will chat with alan at prod chain about this | 13:28 |
arxcruz|rover | ok | 13:28 |
akahat|ruck | rlandy, ack | 13:28 |
*** ysandeep|afk is now known as ysandeep | 13:30 | |
jpodivin | akahat|ruck: there appears to be a dependency issue in the tox-docs job for openstack/release. https://bugs.launchpad.net/tripleo/+bug/1982757 | 13:37 |
jpodivin | must have been a recent change | 13:37 |
akahat|ruck | johnsom, looking | 13:39 |
jpodivin | here it is https://review.opendev.org/c/openstack/requirements/+/850459/3/upper-constraints.txt#541 | 13:40 |
akahat|ruck | jpodivin, yeah.. definitely that patch caused this issue. | 13:42 |
jpodivin | akahat|ruck: the version cap is from https://review.opendev.org/c/openstack/releases/+/811941 But there is no bug linked so I have no idea if it's still necessary | 13:43 |
jpodivin | I'm going to run some tests on a potential fix and post it as WIP if it works out. | 13:44 |
* jpodivin it didn't | 13:45 | |
akahat|ruck | jpodivin, changing jsonschema version to 4.7.2 might resolve issue. | 13:45 |
jpodivin | akahat|ruck: unfortunately the original bug is still present | 13:46 |
jpodivin | so if we just remove the 4.0.0 cap it's going to cause other failures. | 13:46 |
akahat|ruck | jpodivin, I'm referring changing version 4.7.2 in openstack/releases/requirements.txt file | 13:46 |
jpodivin | akahat|ruck: me too | 13:47 |
akahat|ruck | jpodivin, yeah.. let's try it.. if we don't face any issues then we are good with it. | 13:47 |
jpodivin | I've just tried that. | 13:47 |
akahat|ruck | :) | 13:47 |
jpodivin | The 'validate' environment fails on ypeError: create.<locals>.Validator.__init__() got an unexpected keyword argument 'types' | 13:48 |
jpodivin | so ... we'll have to patch the 'Validator' class. | 13:48 |
dasm|off | o/ | 13:50 |
*** dasm|off is now known as dasm | 13:51 | |
jpodivin | or rather call | 13:51 |
bhagyashris | akahat|ruck, arxcruz|rover rlandy rcastillo chandankumar | 14:00 |
bhagyashris | scrum time | 14:00 |
bhagyashris | jm1, ^ | 14:00 |
bhagyashris | ysandeep, ^ | 14:01 |
ysandeep | I am there :D | 14:01 |
bhagyashris | dviroel, ^ | 14:01 |
dviroel | joinning | 14:01 |
arxcruz|rover | rlandy akahat|ruck wallaby c9 promoted, train should promoter as well, all jobs passed | 14:33 |
arxcruz|rover | wallaby c8 i'm running the two missing jjobs to promote | 14:34 |
akahat|ruck | \o/ | 14:34 |
rlandy | arxcruz|rover++ | 14:34 |
rlandy | arxcruz|rover: can you dequeue and enqueue w c8? | 14:34 |
rlandy | failed image build | 14:35 |
arxcruz|rover | rlandy train or wallaby? | 14:35 |
arxcruz|rover | oh, w as in wallaby | 14:35 |
rlandy | ack | 14:36 |
arxcruz|rover | rlandy i'm running the image-builder right now | 14:36 |
marios | frenzy_friday: that one https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/402889 | 14:36 |
marios | rlandy: ^^ | 14:36 |
rlandy | akahat|ruck: pls look at failures on tripleo component | 14:40 |
rlandy | 17 on rhel-8 still failing | 14:41 |
rlandy | akahat|ruck: we may be missing a review there | 14:41 |
rlandy | pls compare with what is in train /16.2 | 14:41 |
akahat|ruck | rlandy, yes.. filling bug | 14:42 |
rlandy | akahat|ruck: it's the same issue | 14:42 |
rlandy | I think | 14:42 |
akahat|ruck | rlandy, no. we hit now nova issue. | 14:42 |
akahat|ruck | rlandy, https://sf.hosted.upshift.rdu2.redhat.com/logs/2f/2f5112cd57be0e1f4aa7177473c47f86787ea11b/openstack-component-tripleo/periodic-tripleo-ci-rhel-8-scenario004-standalone-tripleo-rhos-17/1fc433c/logs/undercloud/var/log/containers/nova/nova-compute.log | 14:42 |
rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/2f/2f5112cd57be0e1f4aa7177473c47f86787ea11b/openstack-component-tripleo/periodic-tripleo-ci-rhel-8-standalone-tripleo-rhos-17/98306af/logs/undercloud/var/log/tempest/stestr_results.html | 14:43 |
rlandy | raise libvirtError('virDomainDefineXML() failed') | 14:43 |
rlandy | 2022-07-25 13:32:02.505 7 ERROR nova.compute.manager [instance: 8fb4a195-beeb-4b94-950e-0575c1357aca] libvirt.libvirtError: unsupported configuration: Emulator '/usr/libexec/qemu-kvm' does not support machine type 'pc-q35-rhel9.0.0' | 14:43 |
rlandy | ok that | 14:43 |
chandankumar | rlandy: dviroel: can we get https://review.opendev.org/c/openstack/tripleo-ci/+/850893 +w ? | 14:44 |
akahat|ruck | rlandy, https://bugzilla.redhat.com/show_bug.cgi?id=2110535 | 14:46 |
akahat|ruck | rlandy, one more job hit with different error, https://sf.hosted.upshift.rdu2.redhat.com/logs/2f/2f5112cd57be0e1f4aa7177473c47f86787ea11b/openstack-component-tripleo/periodic-tripleo-ci-rhel-8-bm_envE-3ctlr_1comp-featureset001-tripleo-rhos-17/4126c43/job-output.txt | 14:47 |
akahat|ruck | rerunning it to confirm the issue. | 14:47 |
dviroel | https://review.opendev.org/c/openstack/tripleo-ci/+/850689 | 14:50 |
ysandeep | rlandy, fyi.. baremetal jobs is getting stable for master line: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-centos-9-bm_envC-3ctlr_1comp-featureset001-master&skip=0 (6/10 green - not bad) | 14:51 |
akahat|ruck | rlandy, ysandeep https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-bm_envE-3ctlr_1comp-featureset001-tripleo-rhos-17&skip=0 | 14:54 |
akahat|ruck | this job has failure history since last month. | 14:55 |
akahat|ruck | do we need to consider it or skip? | 14:55 |
marios | jm1: you didn't like the suggestion? | 14:59 |
marios | :D | 14:59 |
marios | or is it cos i always say 'just send an email and say '... ;) | 14:59 |
* marios coffee brb | 14:59 | |
ysandeep | akahat|ruck, you can skip that for now its not in criteria | 15:01 |
jm1 | marios: all is fine! i will look for the last osp all hands / info shareing recordings and if i cannot find it there i will just ping maria. i might leave out that we discussed layoffs in our team though ;) | 15:01 |
akahat|ruck | ysandeep, ok. Thanks for info. | 15:01 |
ysandeep | akahat|ruck, we have to move rhel-8 job to rhel-9, once 17.1 line is up | 15:01 |
ysandeep | less baremetal env more lines :D | 15:02 |
akahat|ruck | :D | 15:03 |
*** ysandeep is now known as ysandeep|dinner | 15:04 | |
marios | jm1: :) | 15:05 |
*** dviroel is now known as dviroel|lunch | 15:08 | |
* akahat|ruck back in hour | 15:09 | |
chandankumar | rlandy: dasm dviroel|lunch https://review.rdoproject.org/r/q/topic:ibm_cloud merge please | 15:10 |
bhagyashris | rlandy, frenzy_friday ysandeep|dinner marios 17.1 criteria file patch https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/420519 | 15:10 |
dasm | chandankumar++ | 15:13 |
arxcruz|rover | rlandy train seems to be promoted | 15:17 |
arxcruz|rover | testing to current-tripleo: (subhash aggregate: 8b105f1c9d61037014914c5d4ad09e4c, commit: b90ee9e79c82658c4b7c9764d16b8c46abbe90e6, distro: a9b76eb5574f9d38cb0dc56619f5e1840621d135, extended: None, component: validation, timestamp: None) API returned different promoted hash: 'aggregate: 8b105f1c9d61037014914c5d4ad09e4c, commit: 4a9dbcbb4fc2851d503c30419783d033db285bad, distro: c9cbe2502bb180b40f28b15401b8e1a9e19a653a, extended: None, | 15:17 |
arxcruz|rover | component: common, timestamp: None' | 15:17 |
arxcruz|rover | 2022-07-25 15:04:29,738 3266993 INFO promoter Candidate hash 'aggregate: 8b105f1c9d61037014914c5d4ad09e4c, commit: b90ee9e79c82658c4b7c9764d16b8c46abbe90e6, distro: a9b76eb5574f9d38cb0dc56619f5e1840621d135, extended: None, component: validation, timestamp: 1658686002': SUCCESSFUL promotion to current-tripleo | 15:17 |
arxcruz|rover | although i'm not seeing it in the dashboard | 15:17 |
arxcruz|rover | the current-tripleo is pointing to this hash, that is from today | 15:17 |
arxcruz|rover | maybe the script on dashboard didn't run yet | 15:18 |
chandankumar | rlandy: once this https://review.rdoproject.org/r/c/config/+/44148 merges, please recheck this patch https://review.rdoproject.org/r/c/testproject/+/44151 to trigger fs01 , feel free to update the patch | 15:21 |
chandankumar | dasm: thanks :-) | 15:21 |
chandankumar | see ya \o | 15:23 |
rlandy | bhagyashris: hey - going to merge that | 15:28 |
rlandy | let's see if we get a promo | 15:29 |
*** ysandeep|dinner is now known as ysandeep|out | 15:45 | |
rlandy | bhagyashris: patch merged | 15:50 |
rlandy | arxcruz|rover++ nice | 15:50 |
rlandy | chandankumar: w+'ed https://review.rdoproject.org/r/c/config/+/4414 | 15:56 |
rlandy | lunch - brb | 15:58 |
* akahat|ruck going offline in 20 mins. | 16:24 | |
rlandy | akahat|ruck: periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-tripleo-rhos-16.2 needs arerun | 16:25 |
akahat|ruck | looking | 16:26 |
*** dviroel|lunch is now known as dviroel | 16:26 | |
dviroel | rlandy: fyi, I updated rr schedule, me and frenzy_friday swapped our shifts | 16:28 |
dviroel | rlandy: but i will cover her on frinday, since she is out | 16:28 |
rlandy | dviroel: sure | 16:29 |
akahat|ruck | Running here: https://code.engineering.redhat.com/gerrit/c/testproject/+/300913 | 16:34 |
akahat|ruck | rlandy, do you need anything from me before i go offline? | 16:37 |
arxcruz|rover | rlandy akahat|ruck master fails only on fs035 what do you guys want to do? | 16:38 |
dasm | brb | 16:38 |
arxcruz|rover | re-run or skip it ? | 16:38 |
akahat|ruck | arxcruz|rover, sqlalchemy error still hitting this? | 16:41 |
arxcruz|rover | akahat|ruck from the revert patch yes | 16:41 |
akahat|ruck | oh.. | 16:42 |
akahat|ruck | arxcruz|rover, revert of reverted patch needs to create? :D | 16:42 |
arxcruz|rover | akahat|ruck no, since the revert patch wasn't merged yet | 16:43 |
akahat|ruck | oh.. then it's okay.. I thought that patch got merged. | 16:44 |
dasm | back | 16:45 |
* akahat|ruck offline.. see you tomorrow. | 16:47 | |
rlandy | arxcruz|rover: promote of it's not consistent | 16:52 |
rlandy | akahat|ruck: I rekicked the 16.2 tripleo fs001 failure | 16:53 |
rlandy | arxcruz|rover: looking at fs035 | 16:53 |
arxcruz|rover | i rekick fs035 now, i'll see if it will pass or not first | 16:54 |
rlandy | arxcruz|rover: looks like it's a deploy failure | 16:56 |
rlandy | not a tempest thing | 16:56 |
rlandy | FATAL | Check Keystone user assignment to roles status | undercloud | item=swift | error={"ansible_job_id": "480130905223.220827", "ansible_loop_var": "tripleo_keystone_resources_user_role_async_result_item", "attempts": 1, "changed": false, "extra_data": {"data": null, "details": "None", "response": "None"}, | 16:57 |
arxcruz|rover | rlandy ok... | 16:57 |
rlandy | arxcruz|rover: ^^ legit | 16:57 |
rlandy | failure | 16:57 |
arxcruz|rover | ok | 16:57 |
arxcruz|rover | checking | 16:57 |
rlandy | arxcruz|rover: probably bug/CIX this | 16:58 |
rlandy | something with keystone and ipv6 | 16:58 |
rlandy | arxcruz|rover: ping security | 16:59 |
rlandy | arxcruz|rover: sisn;t you recently promote security component? | 17:00 |
arxcruz|rover | but it wasn't in master | 17:00 |
rlandy | ok - checking latest commits then | 17:01 |
rlandy | arxcruz|rover: deploy failures look like they started today | 17:09 |
rlandy | arxcruz|rover: possible impact patch ... https://github.com/openstack/tripleo-heat-templates/commit/906991f099469f88c7cecf61877c52e12527ad5c - merged on 07/21 | 17:11 |
arxcruz|rover | hmmm | 17:12 |
arxcruz|rover | ok, i'll take a look | 17:12 |
rlandy | arxcruz|rover: you can try ping ade_lee_ on #tripleo | 17:16 |
rlandy | he's around | 17:16 |
rlandy | might know off hand | 17:16 |
rlandy | if not, you can bug it | 17:16 |
arxcruz|rover | ok | 17:20 |
* dviroel brb | 18:22 | |
rlandy | arxcruz|rover: late for you | 18:54 |
rlandy | can sort through this in your mroning | 18:55 |
*** dviroel is now known as dviroel|afk | 20:17 | |
arxcruz|rover | rlandy wallaby c8 is promoting | 20:31 |
arxcruz|rover | rlandy promoted, now only master missing | 20:33 |
arxcruz|rover | gonna sleep now | 20:33 |
rlandy | arxcruz|rover++ thanks | 21:04 |
* dasm => offline | 21:25 | |
dasm | o/ | 21:25 |
*** dasm is now known as dasm|off | 21:25 | |
*** undefined is now known as Guest6022 | 22:41 | |
*** rlandy is now known as rlandy|out | 23:57 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!