Friday, 2022-01-14

« Thursday, 2022-01-13 Index Saturday, 2022-01-15 »
*** ysandeep|out is now known as ysandeep00:39
ysandeeprlandy|ruck|bbl, thanks!00:39
ysandeep16.2 scheduled run kicked, killing that so that we can rekick after glance and validation promotion00:40
ysandeeprlandy|ruck|bbl, thanks for keeping an eye ++00:41
ysandeeprlandy|ruck|bbl, wohooo glance promoted 01:36
*** rlandy|ruck|bbl is now known as rlandy|ruck01:51
rlandy|ruckysandeep; hey01:51
ysandeeprlandy|ruck, hi o/01:51
rlandy|ruckysandeep: you all set?01:55
ysandeeprlandy|ruck, pm01:55
*** rlandy|ruck is now known as rlandy|out01:57
*** ysandeep is now known as ysandeep|afk02:05
ysandeep|afkfrenzyfriday|ruck, fyi.. I have retriggered c7 train line so that fix an be promoted to current-tripleo for https://bugs.launchpad.net/tripleo/+bug/1957805 06:35
frenzyfriday|ruckAwesome! thanks06:35
*** jpena|off is now known as jpena08:10
*** ysandeep|afk is now known as ysandeep08:19
frenzyfriday|ruckhey folks 0/ pls add to your review list when you get some time: https://review.rdoproject.org/r/c/rdo-jobs/+/38120 (adds fs020), https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/824677 (skiplist fs020), https://review.rdoproject.org/r/c/rdo-jobs/+/37326 (adds fs035)08:22
frenzyfriday|ruckand https://review.opendev.org/c/x/tripleo-ipa/+/82436808:34
*** ysandeep is now known as ysandeep|afk09:51
*** chem is now known as Guest55010:20
*** ysandeep|afk is now known as ysandeep10:42
*** arxcruz|ruck is now known as arxcruz11:13
ysandeepfrenzyfriday|ruck, reviewed/commented ^^11:20
*** dviroel|out is now known as dviroel11:21
*** rlandy|out is now known as rlandy|ruck11:25
rlandy|ruckarxcruz: hey - you around?11:26
rlandy|ruckysandeep: how are things for 16.2?11:27
rlandy|ruckfrenzyfriday|ruck: hey - will sync with you in a few11:27
ysandeeprlandy|ruck, awaiting on 2 jobs fs020/fs035 - failed on tempest in last run - rerun running right now, fingers crossed11:27
rlandy|ruckjuts wanted to touch base with arxcruz11:28
rlandy|ruckysandeep: k - let me know when you are EoD and what to pick up11:28
ysandeeprlandy|ruck, ack11:28
arxcruzrlandy|ruck: i need 30 min 11:30
rlandy|ruckarxcruz: k - please ping when you are back11:30
rlandy|ruckanbanerj: let's sync11:30
rlandy|ruckanbanerj: https://meet.google.com/ipm-ctxw-nhq?pli=1&authuser=011:31
* ysandeep back in ~half an hour, going for a walk11:40
*** ysandeep is now known as ysandeep|afk11:40
*** anbanerj is now known as frenzyfriday|ruck12:02
arxcruzrlandy|ruck: i'm ready 12:06
rlandy|ruckarxcruz:  give me 10 minutes12:07
arxcruzok12:07
rlandy|ruckarxcruz: pls join  https://meet.google.com/ipm-ctxw-nhq?pli=1&authuser=012:17
frenzyfriday|ruckamoralej: Hey, Deploy IPA is failing on centos 9 with Error: Unable to find a match: epel-release. This is where it is installed: https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/freeipa-setup/templates/deploy_freeipa.sh.j2#L39 Is there an alternate approach for epel for Centos9 ? (Bug: https://bugs.launchpad.net/tripleo/+bug/1956563)12:24
amoralejfrenzyfriday|ruck, why is epel required?12:24
amoralejmaybe because of haveged? not it's in rdo repos too12:26
amoralejyou shouldn't rely in epel in tripleo12:26
frenzyfriday|ruckamoralej, yes I think. I see after {{ ansible_pkg_mgr }} install -y haveged we are disabling epel12:26
amoralejright12:27
amoralejthat shouldn't be needed12:27
arxcruzjpena: amoralej hey, where do we keep the pinned version of the rpms ?12:31
frenzyfriday|ruckamoralej, you mean  install -y haveged can be removed?12:31
amoralejfrenzyfriday|ruck, i meant epel-release can be removed12:32
amoralejhaveged can be installed from the RDO dependencies repos which are added by tripleo12:33
amoralejarxcruz, what rpms?12:33
arxcruzamoralej: for example, keystone-tempest-tests we build the rpm for that 12:33
frenzyfriday|ruckamoralej, cool, lemme remove epel and try12:33
arxcruzi forgot the repo where we setup it, i need to check if it's pinned or not 12:34
amoralejarxcruz, in rdoinfo12:34
amoralejyou can use rdopkg12:34
*** holser is now known as holser_12:35
amoralejarxcruz, "rdopkg info keystone-tests-tempest"12:35
chandankumaramoralej: regarding keystone tempest plugin rpm build can you open a bug against tripleo12:40
chandankumarsorry amoralej it is for arxcruz 12:40
amoralejah, ok :)12:40
*** ysandeep|afk is now known as ysandeep12:41
rlandy|ruckchandankumar: you're on vacation!!!12:42
chandankumarmarios: please add it to review list when free https://review.opendev.org/c/openstack/diskimage-builder/+/824660 and https://review.opendev.org/c/openstack/diskimage-builder/+/82465912:42
chandankumarrlandy|ruck: yes, will go offline very soon!12:43
rlandy|ruckchandankumar: have a good weekend12:43
chandankumarrlandy|ruck: you too, thanks :-)12:43
rlandy|ruckysandeep; hey - did you get what you needed from hardware prov?12:45
marioschandankumar: ack 12:45
ysandeeprlandy|ruck: yes, I got the review from steve, I have updated the patch with his suggestions.12:46
rlandy|ruckmarios: hey - have a few minutes to chat with me about c9 tempest compute failures - didn't have time to look at it yesterday but would like to today12:46
mariosrlandy|ruck: sure thing12:47
rlandy|ruckysandeep: k - great12:47
rlandy|ruckysandeep: I can reach steve again on sunday night if you need anything12:47
rlandy|ruckmarios:  https://meet.google.com/ipm-ctxw-nhq?pli=1&authuser=012:47
ysandeepI think they are already on his review list so but It was for this one: https://review.opendev.org/c/openstack/tripleo-common/+/824598 and proposed an improvement for rhel8 https://review.opendev.org/c/openstack/tripleo-common/+/824662 as well.12:52
ysandeeptriage for el9 image build cix cards - done12:53
ysandeeprlandy|ruck, frenzyfriday|ruck ovb debug done? Can we enable back the cleanup script? 12:58
rlandy|ruckysandeep: yes - pls re-enable clean up - thank you12:59
mariosrlandy|ruck: waiting for test https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/824689/1#message-2eca4df57fe281de19df8393023fa002fe95a27812:59
ysandeeprlandy|ruck, ack12:59
frenzyfriday|ruckrlandy|ruck, should I delete the held node fro ovb debug?13:02
rlandy|ruckfrenzyfriday|ruck: you can13:02
rlandy|ruckit should disappear in 24 hours - but good to clean up13:03
frenzyfriday|ruckrlandy|ruck, cool. I rechecked your testproj https://review.rdoproject.org/r/c/testproject/+/36254 with new depends on13:05
rlandy|ruckfrenzyfriday|ruck: great - let me know how you go there13:06
*** amoralej is now known as amoralej|lunch13:06
frenzyfriday|rucknode removed. ++ ysandeep for the script!13:10
ysandeepfrenzyfriday|ruck, glad its coming of use :)13:12
ysandeepthere are some corner cases improvement still needed for node_holder.py, found one issue yesterday - will fix soon13:13
dviroelfrenzyfriday|ruck: hi o/ - https://review.rdoproject.org/r/c/rdo-jobs/+/38120/2#message-4eccd00447746b680a9fc7b7f23973de5fd7714613:15
dviroelfrenzyfriday|ruck: I can update for you if you want13:15
frenzyfriday|ruckdviroel, thanks, updating13:20
dviroelok, tks13:21
*** rcastillo|rover is now known as rcastillo13:23
rlandy|ruckrcastillo: hey - good morning13:24
rlandy|ruckrcastillo: happy no ruck/rover to you13:24
rcastillogood morning13:25
rcastillothx lol13:25
dpeacockDoes anyone know who maintains access to review.rdoproject.org/r ?13:26
dpeacockI haven't been able to login for a long time.13:26
ysandeepdpeacock, may be someone on #rdo can guide you.13:27
rlandy|ruckmatbu++ thanks for the fix yesterday!!13:27
dpeacockysandeep: right - ya - thank you!13:28
rlandy|ruckdpeacock: or ping apevec on internal IRC on #rhos-ops13:29
rlandy|ruckhe should be able to direct you13:29
ysandeeprlandy|ruck, c7 train promoted, i triggered that line few hours ago, check zuul should get green now13:30
ysandeeprlandy|ruck, I think worth a recheck for one patch from https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-7-containers-multinode 13:30
rlandy|ruckysandeep++ thanks for doing that13:30
dpeacockthanks rlandy|ruck 13:30
rlandy|ruckysandeep: yep asked frenzyfriday|ruck  to recheck that test  on opendev zuul once we had merge13:30
ysandeeprlandy|ruck, i see marios rechecked this https://review.opendev.org/c/openstack/puppet-tripleo/+/824543/ , lets see how this goes13:31
ysandeepmarios, sry it was for matbu13:31
mariosysandeep: np13:31
*** dasm|off is now known as dasm13:32
dasmo/13:32
rlandy|ruckdasm: hey - good morning13:32
dasmrlandy|ruck: \o13:33
rlandy|ruckdasm: arxcruz is following up on the wallaby c9 tempes failure13:33
dasmoh, nice!13:33
arxcruzdasm: ;)13:33
rlandy|ruckdasm: pls see conversation above re:  keystone-tempest-tests13:34
rlandy|ruckhopefully that will unblock you13:34
matburlandy|ruck: no pb, it fixed pathlib2 but i think we will need another one13:35
dasmarxcruz: i see something about rdopkg, but i don't understand that. can you give me some pointers?13:36
rlandy|ruckmatbu: ok - feel free to link any other changes to the bug so we can track them13:39
arxcruzdasm: https://meet.google.com/wpm-nhbd-oox?authuser=113:40
matburlandy|ruck: yep I will do13:40
rlandy|ruckthanks13:40
matburlandy|ruck: I will harass ysandeep to re-run a promotion to get the new change available :)13:40
rlandy|ruckmatbu: any of us can rekick the line for you  when ready13:41
ysandeeprcastillo, I wonder why gate jobs are not triggering for https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/823367 even after +w13:47
ysandeeprcastillo, ah parent child patch, https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/823365/4 still not merged13:48
rcastilloysandeep: yeah, I based them off each other to be able to Depends-On both13:49
rcastillomaybe should've done it the other way13:49
ysandeeprcastillo, I think you did right, If patches belong to same project - good to create parent child relationship instead13:50
ysandeepi think depends-on sometimes don't work if patches belongs to same project.13:51
rcastilloysandeep: k, I'll look into getting the dib c9 jobs working so we can get that merged13:54
ysandeeprcastillo, fyi.. https://review.opendev.org/c/openstack/diskimage-builder/+/82465913:55
ysandeephttps://review.opendev.org/c/openstack/diskimage-builder/+/82466013:55
rlandy|ruckysandeep: hey can you send jm1 an example of a ticket you logged to IT about hardware machines that need attention?13:55
ysandeeprlandy|ruck, sure13:56
rlandy|ruckysandeep: so he has the contacts and the group responsible info13:56
rlandy|ruckysandeep: thank you13:56
rlandy|ruckdasm: soniya is out today so we'll skip RBAC meeting13:57
rlandy|ruckdasm: we can meet about other topics if you want13:58
frenzyfriday|ruckhey rlandy|ruck I am leaving for the day in ~30mins https://code.engineering.redhat.com/gerrit/c/testproject/+/301027 (increased time limit still running) I have updated the rest of the patches in tripleo-ci room13:58
dasmrlandy|ruck: i'm with arxcruz, talking about rpm dependencies right now. 13:59
rlandy|ruckdasm: please carry on then :)14:01
*** amoralej|lunch is now known as amoralej14:01
ysandeepjm1: please check email, rlandy|ruck ccied you.. 14:01
ysandeepjm1 fyi.. same info is also available on our infra-doc, I just copied from there.14:01
rlandy|ruckysandeep: thanks14:01
rlandy|ruckfrenzyfriday|ruck: k - great14:01
rlandy|ruckwill watch those14:01
jm1ysandeep: awesome, thanks :)14:01
rlandy|ruckfrenzyfriday|ruck: ok with fs039 so far?14:02
jm1ysandeep: infra doc?14:02
ysandeepjm1, pm14:02
frenzyfriday|ruckrunning14:02
rlandy|ruckgreat14:03
ysandeeprlandy|ruck, could you please extend infra-doc access to jakob if you are owner.14:05
rlandy|ruckysandeep: weshay|ruck needs to do that14:05
rlandy|ruckysandeep: emailed him14:05
ysandeeprlandy|ruck, yeah i just saw weshay|ruck is owner14:05
rlandy|ruckwaiting for him to switch ownership14:05
ysandeeprlandy|ruck, weshay|ruck is back as ruck :)14:05
rlandy|ruckysandeep: it's just IRC bouncer14:05
rlandy|ruckhe has not fixed one job or bug :)14:06
jm1rlandy|ruck: any progress towards getting owner of the infra doc?14:06
ysandeeprlandy|ruck, hahaha lol!14:06
*** ysandeep is now known as ysandeep|brb14:07
rlandy|ruckjm1: ^^ feel free to harass weshay|ruck14:08
rlandy|ruckrcastillo dasm also need the access14:09
rcastilloit this tripleo-ci-infra? I have access14:10
rlandy|ruckrcastillo: ack14:11
rlandy|rucksomehow14:11
*** holser_ is now known as holser14:12
* dasm brb14:14
*** ysandeep|brb is now known as ysandeep14:16
*** holser is now known as holser_14:24
* rlandy|ruck needs to pick up delivery - brb14:28
*** rlandy|ruck is now known as rlandy|ruck|brb14:28
* frenzyfriday|ruck afk14:30
ysandeeprlandy|ruck|brb, lets chat when you are back 14:37
matbuysandeep: can you re-kick the promotion for validations in stable/train centos7 plz ? (for this patch: https://review.rdoproject.org/r/c/openstack/validations-libs-distgit/+/38134)14:38
ysandeepmatbu, sure14:38
matbuysandeep: thx14:39
* dasm here14:40
ysandeeprlandy|ruck|brb, 16.2 promoted with new glance 14:42
ysandeepmatbu, https://review.rdoproject.org/zuul/status#openstack-periodic-integration-stable4-centos7 triggered, thanks for the fix.14:45
*** rlandy|ruck|brb is now known as rlandy|ruck14:45
rlandy|ruckysandeep: awesome14:45
rlandy|ruckysandeep: what's left there?14:46
matbuysandeep: great thank you, watching it :)14:46
ysandeeprlandy|ruck, want to meet on gmeet for 5 mins?14:46
rlandy|ruckysandeep: ack14:46
ysandeeprlandy|ruck, meet.google.com/och-cgwj-uhf14:47
rlandy|ruckhttps://logserver.rdoproject.org/54/36254/61/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/3aecfe5/logs/supplemental/home/cloud-user/deploy_freeipa.log.txt.gz14:48
rlandy|ruckcloser14:48
rlandy|ruckysandeep: joining14:48
jm1rlandy|ruck: wes gave me access to the infra doc :)14:57
rlandy|ruckjm1: great14:58
rlandy|ruckdasm: did you get an invite from me to share the tripleo-ci-infra doc?15:09
dasmchecking15:10
dasmrlandy|ruck: yes,i got it.15:10
rlandy|ruckwoohoo - I have the power15:10
*** ysandeep is now known as ysandeep|out15:25
jm1rlandy|ruck: resetting idracs via chassis mgmt worked, both remaining idracs have been updated now :)15:28
rlandy|ruckjm1: awesome!!15:29
jm1rlandy|ruck: chassis mgmt allows to do firmware updates in a batch so in case we wanna do this in future in should be faster15:32
*** dviroel is now known as dviroel|lunch15:33
* ysandeep|out out, folks see you on tuesday o/ 15:45
marioshave a good one ysandeep|out 15:45
ysandeep|outmarios: thanks, happy weekend :D15:46
*** holser_ is now known as holser15:58
rlandy|ruckarxcruz: dasm: are you guys all set or still need help with keystone merge?16:01
arxcruzrlandy|ruck: i pinged lance and gmann but no answer yet 16:01
mariosrlandy|ruck: i had to recheck the fs1/ovb test for the skiplist - failed unrelated provisioning issue fyi https://review.rdoproject.org/r/c/testproject/+/36267/47#message-f6f5bd4427754057d4f27a201aa401255a7e1efe 16:09
rlandy|ruckdasm: if you are blcked you can start working on containers-multinode16:10
rlandy|ruckmarios: ok16:10
dasmrlandy|ruck: i'll do that. i'm waiting for merge on keystone plugin16:10
rlandy|ruckwill work on tempest-api job16:10
rlandy|ruckdasm: yep - you can find the master job16:10
rlandy|ruckcreate the wallaby overrides16:10
rlandy|rucktestproject 16:11
rlandy|rucklet me know16:11
dasmsure16:11
*** dviroel|lunch is now known as dviroel|16:46
*** dviroel| is now known as dviroel16:46
marioshave a good weekend folks bai16:46
*** marios is now known as marios|out16:47
dviroelo/16:47
dasm\o17:00
dviroelade_lee: o/17:23
dviroelade_lee: are you aware of this error on CS9+FIPS? https://48c35ceb9349148aa916-45dd8d096407cabbe1019de430c921aa.ssl.cf1.rackcdn.com/824479/1/check/tripleo-ci-centos-9-standalone/4c208e8/job-output.txt17:24
*** jpena is now known as jpena|off17:24
*** amoralej is now known as amoralej|off17:33
rlandy|ruckussuri promoting17:41
ade_leedviroel, yeah - I saw that the other day -- still need to troubleshoot17:47
ade_leedviroel, was going to look into it today17:47
dviroelade_lee: oh ok, i was wondering if is a known issue, which you have hacked for c817:48
dviroelade_lee: like https://review.opendev.org/c/zuul/zuul-jobs/+/807031/33/roles/enable-fips/tasks/main.yaml17:48
ade_leedviroel, no - new issue17:48
dviroeloh ok, weird, right after reboot17:49
rlandy|ruckhttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-content-provider19:05
rlandy|ruckwe got trouble19:05
rlandy|ruck2022-01-14 18:42:47.453347 | primary |  Problem: cannot install the best update candidate for package gcc-11.2.1-6.1.el9.x86_6419:06
rlandy|ruck2022-01-14 18:42:47.453362 | primary |   - nothing provides libgomp = 11.2.1-7.4.el9 needed by gcc-11.2.1-7.4.el9.x86_6419:06
rlandy|ruck2022-01-14 18:42:47.453375 | primary |   - nothing provides libgcc >= 11.2.1-7.4.el9 needed by gcc-11.2.1-7.4.el9.x86_6419:06
dasm:/19:08
rlandy|ruck2022-01-14 18:43:01.921597 | primary |  Problem: cannot install the best candidate for the job19:08
rlandy|ruck2022-01-14 18:43:01.921612 | primary |   - nothing provides openssl-libs(x86-64) = 1:3.0.0-6.el9 needed by openssl-devel-1:3.0.0-6.el9.x86_6419:08
rlandy|ruck2022-01-14 18:43:01.921720 | primary | (try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)19:08
rlandy|ruckmirror issues?19:08
rlandy|ruckDownloading https://mirror-int.dfw.rax.opendev.org/pypifiles/packages/65/11/67955ff6fbd5aedee3b4d47847c24510e7de970aa352cacac8444ef33059/selinux-0.2.1-py2.py3-none-any.whl19:09
dviroeloh, let me check if i hit same issue19:11
dviroelsame issue here: https://logserver.rdoproject.org/38/37738/9/check/periodic-tripleo-centos-9-buildimage-overcloud-hardened-uefi-full-master/58c79b9/overcloud-hardened-uefi-full.log19:12
rlandy|ruckdviroel: failing all over gate and check19:12
rlandy|rucklooks like a mirror issue19:12
rlandy|ruckwas going to bug19:12
rlandy|ruckdviroel: is there already a bug open on this19:12
dviroeldont think so19:13
rlandy|ruckok - opening bug19:14
rlandy|ruckdviroel: https://bugs.launchpad.net/tripleo/+bug/195795019:19
dviroellast successfull job was installing openssl-devel.x86_64 - 1:3.0.0-5.el919:23
dviroelopenssl-devel-1:3.0.0-6.el9.x86_64 needs openssl-libs(x86-64) = 1:3.0.0-6.el9, which is missing atm19:23
dviroelbaseos19:24
dviroellibs is baseos, openssl-devel is appstream19:24
rlandy|ruckdviroel: we can try remove these latest versions from appstream19:26
rlandy|ruckand baseos19:26
dviroelhttp://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/ -  openssl-libs-3.0.0-6.el9.x86_64.rpm is there19:26
rlandy|ruckchecking if the earlier version is there19:26
rlandy|ruckearlier version is there19:28
dviroelout of sync mirror then?19:28
dviroelNODEPOOL_MIRROR_HOST=mirror-int.iad.rax.opendev.org19:29
rlandy|ruckdviroel: ack19:31
rlandy|ruckhttp://mirror.iad3.inmotion.opendev.org/centos-stream/9-stream/AppStream/x86_64/os/Packages/19:31
rlandy|ruckmissing from there19:31
rlandy|ruckmirror sync issue19:31
rlandy|ruckdviroel: let19:31
dviroelopenssl-devel-3.0.0-6.el9.x86_64.rpm2022-01-11 12:0619:31
dviroelin http://mirror.stream.centos.org/9-stream19:32
rlandy|ruckhope it magically disappears shortly19:32
dviroel3 days ago19:32
rlandy|ruck[DIR]Packages/2022-01-07 15:38 -19:32
rlandy|ruckhttp://mirror.iad3.inmotion.opendev.org/centos-stream/9-stream/AppStream/x86_64/os/19:32
rlandy|ruckopenssl-3.0.0-6.el9.x86_64.rpm2022-01-11 12:06 19:32
rlandy|ruckprobably got pushed recently19:32
dviroel@team, have a great weekend o/20:54
*** dviroel is now known as dviroel|out20:57
dasmdviroel|out: \o21:01
rlandy|ruck16.2 will promote yaya21:25
rlandy|ruckout21:25
dasm\o/21:27
dasmi just got hit by mirror sync issues in latest test run. i'm gonna check that one on Monday21:46
dasmhave a wonderful weekend! 21:46
*** dasm is now known as dasm|off21:46
*** holser is now known as holser_22:29
*** holser_ is now known as holser23:05
*** holser is now known as holser_23:15
*** holser_ is now known as holser23:52
*** holser is now known as holser_23:52
« Thursday, 2022-01-13 Index Saturday, 2022-01-15 »

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!