*** rlandy|ruck is now known as rlandy|out | 00:20 | |
ysandeep | pojadhav, good morning, when you are around lets sync about 17 on rhel9 current status. | 05:42 |
---|---|---|
pojadhav | ysandeep, hello.. good morning. lets sync now. I am available. | 06:06 |
ysandeep | pojadhav, meet.google.com/bqi-ayfc-ymu | 06:08 |
*** ysandeep is now known as ysandeep|afk | 06:29 | |
dpawlik | arxcruz, marios: hey, could you check https://review.opendev.org/c/openstack/ci-log-processing/+/830941 , https://review.opendev.org/c/openstack/ci-log-processing/+/830169 | 07:15 |
marios | dpawlik: k adding to reviews | 07:18 |
marios | ysandeep|afk: o/ needs rebase fyi https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/830060 | 07:20 |
soniya29 | kopecmartin, any idea why is this patch blocked and what can we do get this merged:- https://review.opendev.org/c/openstack/requirements/+/820378 | 07:34 |
*** ysandeep|afk is now known as ysandeep | 07:41 | |
kopecmartin | soniya29: there is a merge conflict, but mainly the problem is that we weren't sure whether is safe to use that dependency - therefore -1 on workflow | 07:52 |
kopecmartin | i wouldn't merge that, in the comments we can see that there is a concern regarding the stability of the lib , it also lacks testing | 07:52 |
kopecmartin | we can figure out a different way hot to test ping , without that lib | 07:53 |
ysandeep | marios: I thought this was merged, thanks for notifying - I have rebased the patch. | 07:55 |
marios | ack ysandeep | 08:05 |
soniya29 | kopecmartin, let's discuss this today in office hours, and let's take this discussion to the patch itself | 08:05 |
*** jpena|off is now known as jpena | 08:33 | |
jm1 | soniya29: while adding my ssh keys to ci-config repo I saw two old commits from 2020 where you tried to add you key, too. do you still need/want SSH access to cockpit etc.? | 11:05 |
soniya29 | jm1: yeah, i will need them | 11:06 |
jm1 | soniya29: still the correct SSH key? https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/24727 | 11:07 |
soniya29 | jm1, i will check and correct those | 11:13 |
soniya29 | thanks for pointing, i forgot about them :) | 11:13 |
jm1 | soniya29: they have not been merged yet and they wont because i changed the format 🙈 If those keys are still correct i will create a patch for you | 11:14 |
soniya29 | jm1, please go ahead | 11:15 |
jm1 | soniya29: ack, will ping you when its done | 11:15 |
soniya29 | jm1, sure | 11:16 |
*** rlandy|out is now known as rlandy|ruck | 11:18 | |
rlandy|ruck | chandankumar: hello ... how are you with merging parenting patches? | 11:21 |
rlandy|ruck | do you want the review meeting now? | 11:21 |
chandankumar | rlandy|ruck: I have not yet updated the patch | 11:21 |
chandankumar | rlandy|ruck: today not needed | 11:21 |
rlandy|ruck | ok | 11:21 |
chandankumar | rlandy|ruck: but I need one help | 11:21 |
chandankumar | to clear out cs9 patches | 11:21 |
chandankumar | otherwise rdo reviews are ending with merge conflicts | 11:22 |
rlandy|ruck | ysandeep: is downstream all set now? | 11:22 |
rlandy|ruck | jobs starting ok? | 11:22 |
rlandy|ruck | arxcruz: hey ... | 11:24 |
arxcruz | rlandy|ruck hi | 11:24 |
chandankumar | rlandy|ruck: review below | 11:24 |
arxcruz | rlandy|ruck still working on the openssl | 11:24 |
chandankumar | https://review.opendev.org/c/openstack/tripleo-ci/+/830836/ | 11:24 |
chandankumar | https://review.opendev.org/c/openstack/tripleo-ci/+/829655 | 11:24 |
chandankumar | https://review.rdoproject.org/r/c/rdo-jobs/+/39981 | 11:24 |
chandankumar | https://review.rdoproject.org/r/c/rdo-jobs/+/39934 | 11:24 |
chandankumar | https://review.rdoproject.org/r/c/rdo-jobs/+/39793 | 11:24 |
rlandy|ruck | arxcruz: let's touch base | 11:24 |
arxcruz | ok | 11:25 |
rlandy|ruck | we need the key change | 11:25 |
rlandy|ruck | https://meet.google.com/odn-unmu-vam?pli=1&authuser=0 | 11:25 |
rlandy|ruck | arxcruz: ^^ | 11:25 |
chandankumar | it will finish the audit of both master and wallaby | 11:25 |
ysandeep | rlandy|ruck, no same situation but we know what's the cause, There are 11 instances which are stuck in deletion and nodepool is trying to delete them before creating new instances. | 11:29 |
ysandeep | rlandy|ruck, I have opened a ticket with PSI, you are in cc. | 11:29 |
rlandy|ruck | bhagyashris: pls nick | 11:30 |
*** bhagyashris is now known as bhagyashris|ruck | 11:30 | |
rlandy|ruck | bhagyashris|ruck: will meet with you when done with arxcruz | 11:30 |
arxcruz | rlandy|ruck https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/tasks/tempest.yml#L248 | 11:30 |
bhagyashris|ruck | rlandy|ruck, sure | 11:30 |
chandankumar | marios: please have a look when free https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/831198/1#message-d198642e2ff42822a9c247baafc4b36e8d6c5f07 | 11:31 |
chandankumar | thanks! | 11:31 |
chandankumar | ysandeep: hello can you +w https://review.opendev.org/c/openstack/tripleo-ci/+/830836 it ? Thanks! | 11:32 |
rlandy|ruck | arxcruz: https://review.rdoproject.org/r/c/testproject/+/39960/1/.zuul.yaml#10 | 11:33 |
ysandeep | yes | 11:33 |
marios | chandankumar: ack thanks | 11:34 |
rlandy|ruck | arxcruz: https://review.opendev.org/c/openstack/tripleo-quickstart/+/831184 | 11:34 |
rlandy|ruck | manually rebase ^^ | 11:34 |
rlandy|ruck | bhagyashris|ruck: can you join https://meet.google.com/odn-unmu-vam?pli=1&authuser=0 | 11:34 |
* marios relocate biab | 11:40 | |
arxcruz | rlandy|ruck https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/831362 | 11:43 |
rlandy|ruck | ysandeep: hey - can you join https://meet.google.com/odn-unmu-vam?pli=1&authuser=0 | 11:43 |
ysandeep | rlandy|ruck, ack | 11:44 |
rlandy|ruck | - create-scenario010-trigger-for-downstream: | 11:50 |
rlandy|ruck | vars: | 11:50 |
rlandy|ruck | trigger_branch: master | 11:50 |
rlandy|ruck | trigger_os_distro: centos_9 | 11:50 |
rlandy|ruck | arxcruz: thanks - checking what jobs kick there | 11:55 |
rlandy|ruck | arxcruz: so no c8 jobs kicked on that change | 11:55 |
arxcruz | rlandy|ruck i'm updating testproject | 11:56 |
rlandy|ruck | arxcruz: thanks - we will need c7 and c8 | 11:56 |
arxcruz | rlandy|ruck https://review.rdoproject.org/r/c/testproject/+/39944 | 11:56 |
bhagyashris|ruck | rlandy|ruck, fyi https://review.rdoproject.org/r/c/testproject/+/39932 [DNM] Test cs9 master failing jobs | 12:00 |
* bhagyashris|ruck tea brb | 12:02 | |
rlandy|ruck | bhagyashris|ruck: thanks - making small edit | 12:02 |
rlandy|ruck | arxcruz: thanks ... standalone 8 master does not exist anymore - and we'll need force_periodic: true on those | 12:03 |
rlandy|ruck | under the vars | 12:03 |
arxcruz | rlandy|ruck if i use force_periodic it might affect the promotion no ? | 12:04 |
ysandeep | pojadhav, around? | 12:05 |
rlandy|ruck | arxcruz: report_dlrn: false | 12:06 |
rlandy|ruck | in vars | 12:06 |
rlandy|ruck | arxcruz: ^^ make sense? | 12:06 |
pojadhav | ysandeep, yes | 12:06 |
arxcruz | rlandy|ruck done | 12:07 |
arxcruz | rlandy|ruck yes, it does | 12:07 |
rlandy|ruck | thank you | 12:07 |
ysandeep | pojadhav, lets discuss parenting on that your patch: meet.google.com/dxd-wbke-ccu , I see some issues there. | 12:08 |
pojadhav | ysandeep, ack | 12:08 |
chandankumar | ysandeep: https://review.opendev.org/c/openstack/tripleo-ci/+/829655 and this also, thanks! | 12:11 |
ysandeep | chandankumar: sure will check in sometime, In a mtg with pooja | 12:16 |
chandankumar | ysandeep: aye, no problem :-) | 12:17 |
chandankumar | ysandeep: Do you know where we use this job https://code.engineering.redhat.com/gerrit/plugins/gitiles/openstack/tripleo-ci-internal-jobs/+/refs/heads/master/zuul.d/required-projects-overrides.yaml#4 [tripleo-ci-base-required-projects] as a parent in downstream? | 12:18 |
rlandy|ruck | hmmm ... problem on the wallaby line | 12:19 |
rlandy|ruck | create has no dependencies | 12:19 |
ysandeep | chandankumar, yes.. will explain in few mins | 12:20 |
ysandeep | chandankumar, want to chat on gmeet? | 12:30 |
soniya29 | rlandy|ruck, meeting? | 12:30 |
rlandy|ruck | joining 1 mi | 12:31 |
chandankumar | ysandeep: yes | 12:31 |
ysandeep | chandankumar, meet.google.com/nzz-hyxf-stm | 12:31 |
rlandy|ruck | chandankumar: https://review.rdoproject.org/r/c/rdo-jobs/+/39990 - is there a better fix here? | 12:32 |
chandankumar | rlandy|ruck: let me update the patch | 12:37 |
*** ysandeep is now known as ysandeep|break | 12:38 | |
jm1 | soniya29: ssh key merged https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/39982 | 12:41 |
chandankumar | rlandy|ruck: https://review.rdoproject.org/r/c/rdo-jobs/+/39990/3/zuul.d/integration-pipeline-wallaby-centos9.yaml can you check now | 12:42 |
chandankumar | bhagyashris|ruck: rlandy|ruck I might skip community call, Need to drop my brother to station | 12:44 |
bhagyashris|ruck | chandankumar, ack | 12:45 |
rcastillo | o/ | 13:02 |
rlandy|ruck | chandankumar: looks good on https://review.rdoproject.org/r/c/rdo-jobs/+/39990/ | 13:08 |
frenzy_friday | hey folks and reviewbot, pls add to your review list https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/39994 | 13:09 |
reviewbot | I have added your review the to Review list | 13:09 |
marios | frenzy_friday: nice ;) ^ | 13:12 |
rlandy|ruck | marios: hey - had to move 1-1 meeting - clash with prod chain | 13:13 |
marios | rlandy|ruck: k np | 13:13 |
rlandy|ruck | rcastillo: arxcruz: akahat: chandankumar: marios: dasm|rover: soniya29: frenzy_friday: jm1: pojadhav: ysandeep|break: community call in 9 mins | 13:21 |
rlandy|ruck | https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both to add agenda items | 13:21 |
rlandy|ruck | marios: adding the updates/upgrades item | 13:21 |
marios | rlandy|ruck: thank you :) i just fetched the mailing list link to do the same | 13:23 |
*** ysandeep|break is now known as ysandeep | 13:29 | |
rcastillo | need help figuring out an overcloud node provision issue we're hitting on centos ci for the c9 libvirt job | 13:51 |
rcastillo | https://artifacts.ci.centos.org/rdo/jenkins-tripleo-quickstart-promote-master-centos9-current-tripleo-delorean-minimal-29/undercloud/home/stack/overcloud_node_provision.log | 13:51 |
ade_lee | rlandy|ruck, thats great! we should do that then! | 13:57 |
rlandy|ruck | rcastillo: pls reach out to hjensas on #tripleo from hardware prov team | 14:00 |
rlandy|ruck | rcastillo: you can share the logs and setup | 14:00 |
rcastillo | rlandy|ruck: ack | 14:00 |
rlandy|ruck | if you is unavailable, try steve in afternoon hours | 14:00 |
rlandy|ruck | rcastillo: ^^ | 14:01 |
rlandy|ruck | you can use hardware prov chat to leave steve notes | 14:01 |
dasm|rover | o/ | 14:05 |
rlandy|ruck | dasm|rover: we're looking for master and wallaby c9 to promote | 14:07 |
rlandy|ruck | also to see if that cleans up the node provisioning errors in check | 14:08 |
dasm|rover | ack | 14:08 |
rlandy|ruck | dasm|rover: so you have patches from yesterday skiplist reverts that need to be merged? | 14:08 |
dasm|rover | checking on them now | 14:08 |
dasm|rover | rlandy|ruck: all of them failed tests. Nothing to merge | 14:09 |
rlandy|ruck | dasm|rover: ok ... | 14:12 |
rlandy|ruck | bhagyashris_,: pls pass your work to dasm|rover | 14:12 |
rlandy|ruck | pls also nick | 14:12 |
*** pojadhav is now known as pojadhav|afk | 14:19 | |
*** pojadhav|afk is now known as pojadhav | 14:57 | |
pojadhav | rlandy|ruck, chandankumar : kindly review rhel9 standalone component jobs patches https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/312030/ and https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/314912 | 15:10 |
ysandeep | chandankumar: +wed https://review.opendev.org/c/openstack/tripleo-ci/+/829655 lgtm | 15:18 |
*** pojadhav is now known as pojadhav|afk | 15:18 | |
rlandy|ruck | dasm|rover: rekicking failed master periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-master test | 15:29 |
rlandy|ruck | waiting on that one and internal kvm | 15:29 |
rlandy|ruck | can meet with you later re: what's still outstanding on CIX board | 15:29 |
rlandy|ruck | also we need to check the component lines | 15:29 |
rlandy|ruck | upstream pls | 15:29 |
chandankumar | ysandeep: thanks! | 15:32 |
*** ysandeep is now known as ysandeep|dinner | 15:34 | |
dasm|rover | ack | 15:35 |
chandankumar | rlandy|ruck: ysandeep|dinner: please +w these patches https://review.rdoproject.org/r/c/rdo-jobs/+/39934 https://review.rdoproject.org/r/c/rdo-jobs/+/39981 https://review.rdoproject.org/r/c/rdo-jobs/+/39793 | 15:44 |
*** ysandeep|dinner is now known as ysandeep | 15:46 | |
ysandeep | chandankumar, looking | 15:47 |
dasm|rover | rlandy|ruck: when you're free, can I meet with you for a sec? | 16:14 |
rlandy|ruck | dasm|rover: ack - can meet now | 16:17 |
dasm|rover | k | 16:17 |
dasm|rover | https://meet.google.com/iro-jmng-nyz | 16:17 |
dasm|rover | rlandy|ruck: ^ | 16:17 |
*** ysandeep is now known as ysandeep|out | 16:21 | |
rlandy|ruck | arxcruz: can https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/831362/1/playbooks/tasks/tempest.yml merge? | 16:36 |
rlandy|ruck | looks like your testproject passes | 16:36 |
arxcruz | checking | 16:37 |
arxcruz | rlandy|ruck yes, it can be merged, but it has a depends-on on the revert patch, so needs to merge both | 16:38 |
arxcruz | i can remove the depends-on to merge it first | 16:38 |
rlandy|ruck | in the right order | 16:38 |
arxcruz | then the revert, since it will not affect | 16:39 |
rlandy|ruck | arxcruz: ack - pls remove the depends on | 16:39 |
arxcruz | ok | 16:39 |
rlandy|ruck | we will merge https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/831362 first | 16:39 |
rlandy|ruck | then the revert | 16:39 |
rlandy|ruck | dasm|rover: openstack-periodic-integration-stable4-centos7 | 16:40 |
arxcruz | rlandy|ruck done | 16:42 |
*** bhagyashris_ is now known as bhagyashris|ruck | 16:43 | |
bhagyashris|ruck | rlandy|ruck, hey sorry due to network disconnection nick got changed | 16:44 |
rlandy|ruck | bhagyashris|ruck: just met with dasm|rover | 16:47 |
rlandy|ruck | bhagyashris|ruck: pls pass anything you are working on to him since it's late for you | 16:47 |
rlandy|ruck | have 1 failed master c9 test in rerun | 16:47 |
rlandy|ruck | watching the two kvm tests (wallaby and master) in downstrea, | 16:48 |
rlandy|ruck | bhagyashris|ruck: you ok to report on the program call tomorrow? | 16:48 |
rlandy|ruck | bhagyashris|ruck: dasm|rover: we will need to check on component and integration lines in downstream early tomorrow | 16:53 |
rlandy|ruck | and later today | 16:53 |
dasm|rover | ack | 16:53 |
bhagyashris|ruck | rlandy|ruck, ok | 16:53 |
bhagyashris|ruck | i can attend tomorrows program call | 16:54 |
bhagyashris|ruck | np | 16:54 |
bhagyashris|ruck | rlandy|ruck, done handover | 16:55 |
bhagyashris|ruck | rlandy|ruck, dasm|rover looking at the promoter logs http://promoter.rdoproject.org/promoter_logs/centos9_wallaby_2022-03-01T16:51.log centos-9-scenario010-kvm-internal-standalone-wallaby job is missing so will need to recheck that job for promotion | 16:58 |
bhagyashris|ruck | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby | 16:58 |
bhagyashris|ruck | here all jobs are green but centos-9-scenario010-kvm-internal-standalone-wallaby failed due to NODE_FAILURE | 17:00 |
bhagyashris|ruck | https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable1-cs9 | 17:00 |
* bhagyashris|ruck out | 17:02 | |
dasm|rover | ack, thanks | 17:08 |
rlandy|ruck | bhagyashris|ruck: it's running now | 17:09 |
rlandy|ruck | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/27a60f57a6344df5b775c5253c656757?logfile=console.log | 17:09 |
rlandy|ruck | no need to recheck | 17:09 |
rlandy|ruck | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-master | 17:12 |
rlandy|ruck | real failure on ipa master | 17:12 |
rlandy|ruck | will need a bug there | 17:12 |
rlandy|ruck | lunch brb | 17:12 |
rlandy|ruck | maybe skip | 17:12 |
rlandy|ruck | bhagyashris|ruck: dasm|rover: ^^ rerun won;t help | 17:13 |
rlandy|ruck | lunch brb | 17:15 |
dasm|rover | ack | 17:18 |
*** jpena is now known as jpena|off | 17:38 | |
rlandy|ruck | tripleowallabycentos9 promoting :) | 17:49 |
dasm|rover | nice | 17:50 |
dasm|rover | checking issue with multinode ipa master | 17:53 |
*** rlandy|ruck is now known as rlandy|ruck|biab | 18:35 | |
*** rlandy|ruck|biab is now known as rlandy|ruck | 18:53 | |
*** jparker__ is now known as jparker | 19:06 | |
jm1 | rlandy|ruck: who is maintaining our promoter instance on vexxhost? | 19:34 |
rlandy|ruck | jm1: we are | 19:34 |
jm1 | rlandy|ruck: there seem to be mismatches between promoter in our infra-setup and the live instance. i will add a Jira card to fix that | 19:36 |
rlandy|ruck | jm1: ack - pls raise that Thursday's call | 19:37 |
jm1 | rlandy|ruck: today we discovered that multiple instances on vexxhost are based on centos8, not stream. since c8 mirrors are down, our infra-setup playbooks are broken because yum repos cannot be updated. this has to be fixed as well. | 19:38 |
jm1 | atm we cannot apply any updates to those c8 hosts except manually, which would then again break our infra-as-code setup | 19:39 |
jm1 | with "updates" i mean config changes | 19:39 |
jm1 | rlandy|ruck: i will add jira cards and then you can reevaluate priorities, e.g. if i should fix this or skip it for now and continue working on reproducer | 19:41 |
rlandy|ruck | https://review.opendev.org/q/topic:c9_wallaby_gates | 19:41 |
rlandy|ruck | jm1: rcastillo is already on the centos-8 to stream update | 19:42 |
jm1 | rlandy|ruck: did not know, will ping him | 19:45 |
rcastillo | jm1: I'm working on upgrading the promoter server to stream atm | 19:49 |
jm1 | rcastillo: awesome :) | 19:49 |
rlandy|ruck | dasm|rover: hello | 20:53 |
rlandy|ruck | dasm|rover: you busy atm? | 20:53 |
rlandy|ruck | tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallabyopenstack/tripleo-cimasteropenstack-regular2 hrs 5 mins 31 secs2022-03-01 18:23:28SUCCESS | 20:58 |
rlandy|ruck | first clear run on check | 20:59 |
dasm|rover | rlandy|ruck: 'sup? | 20:59 |
dasm|rover | welp | 21:03 |
rlandy|ruck | dasm|rover: no worries - just put the wallaby c8 tests in rerun | 21:03 |
dasm|rover | ack | 21:03 |
rlandy|ruck | dasm|rover: something wrong? | 21:04 |
dasm|rover | no, everything is fine | 21:04 |
rlandy|ruck | ok - need to do some review prep | 21:04 |
dasm|rover | just checking bhagyashris|ruck patches | 21:04 |
dasm|rover | ack | 21:04 |
rlandy|ruck | checking back in a bit | 21:04 |
dasm|rover | k | 21:05 |
dasm|rover | rlandy|ruck: fyi, openstack-periodic-integration-stable4-centos7 succeeded | 22:11 |
rlandy|ruck | dasm|rover: yeah | 22:12 |
rlandy|ruck | so you can close the bug and the card | 22:12 |
rlandy|ruck | thank you | 22:12 |
dasm|rover | done | 22:19 |
rlandy|ruck | great | 22:20 |
rlandy|ruck | rcastillo: any luck with wallaby? | 22:20 |
rcastillo | rlandy|ruck: sadly no | 22:24 |
rcastillo | overcloud doesn't seem to be coming up | 22:25 |
rcastillo | I'll double-check my makeshift wallaby release file | 22:25 |
rlandy|ruck | rcastillo: k - no worries | 22:30 |
rlandy|ruck | hopefully tomorrow we will have a better master build for you to try | 22:31 |
dasm|rover | bhagyashris|ruck: i recheck your testproject patches. One or two jobs passed, but others are still red. Sorry, but I didn't move forward with those. But at least we had a green cs7 train run \o/ | 22:37 |
* dasm|rover is slowly departing... | 22:38 | |
*** dasm|rover is now known as dasm|off | 22:39 | |
rcastillo | rlandy|ruck: yeah. Fingers crossed for master c9 promotion | 22:41 |
rcastillo | rlandy|ruck: also added a couple patches for the centosci c9 wallaby jobs to reviewlist | 22:42 |
* rcastillo out | 22:42 | |
dasm|off | o/ | 22:44 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!