Monday, 2022-04-11

*** amoralej|off is now known as amoralej06:06
jpodivinchkumar|ruck: I'm having trouble with DLRN-rpmbuild for tripleoclient in RDO. Do you know who I should ping about that? 06:22
chkumar|ruckjpodivin: you can amoralej on #rdo channel (he also hangs here)06:26
chkumar|ruck*can ping06:26
jpodivinchkumar|ruck: thanks, will do06:26
*** bhagyashris is now known as bhagyashris|out06:26
marioschkumar|ruck: o/ when you next have time please add to your review queue https://review.opendev.org/c/openstack/tripleo-ci/+/834320  09:38
chkumar|ruckmarios: \o, please have a look when free https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/836988 thanks!09:41
marioschkumar|ruck: ack thanks for checking will revisit 09:42
marioschkumar|ruck: o/ https://review.rdoproject.org/r/c/rdo-jobs/+/40842/2#message-c20afa3058f45f16d66fe0df34aabb70f5d92d3d in case you see anything strange in scen1-4 standalone (it was tested but just in case we miss something else)10:19
chkumar|ruckmarios: sure, thanks for the heads up!10:19
chkumar|ruckjm1: please vote on this https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/41764 when back!10:27
*** chem is now known as Guest167210:29
rlandychkumar|ruck: jm1: how are things today?10:29
rlandylet's sync if you're around10:29
rlandychkumar|ruck: downstream still down?10:30
chkumar|ruckrlandy: yes, issue with new zuul itself10:30
rlandygee10:30
rlandychkumar|ruck: https://meet.google.com/tes-zkyc-qhw?pli=1&authuser=010:31
chkumar|ruckrlandy: ovb jobs will fail during stack create10:31
chkumar|ruckdue to baremetal issue on vexxhost10:31
chkumar|ruckrlandy:  https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-centos-8-scenario010-kvm-internal-standalone-train&skip=010:34
chkumar|ruckrlandy: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4123810:35
*** rlandy_ is now known as rlandy10:50
chkumar|ruckjm1: rlandy https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/41784 for master promotion10:50
rlandyakahat: hi - the promoter keep failing on promoting train c7 again10:50
rlandycan you debug that10:50
rlandyit's still trying10:50
frenzy_fridaybot pls add to review list https://review.rdoproject.org/r/c/rdo-jobs/+/4055310:57
reviewbotI have added your review to the Review list10:58
chkumar|ruckrlandy: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4178411:08
*** dviroel|out is now known as dviroel11:15
marioschkumar|ruck: rlandy: anyone else want to join us with review time?11:16
jm1rlandy: hey :) i guess you already synced with chkumar|ruck?11:17
rlandyjm1: on review time - will sync with you again after review time11:18
rlandyjm1: pls nick11:18
rlandyyou need to be |rover11:18
*** jm1|ruck is now known as jm1|rover11:18
jm1rlandy: upps, o/c, changed it to jm1|rover11:19
jm1|roverrlandy: here i am..11:19
jm1rlandy: split personality 🙊11:20
rlandyjm1|rover: pls join https://meet.google.com/vzm-nrah-qqf?authuser=011:23
chkumar|ruckrlandy: I think we lost you11:26
rlandyykarel: hello ... is there still work on https://bugs.launchpad.net/tripleo/+bug/1964940? not blocking  us but the CIX is still open11:35
ykarelrlandy, i got back from PTO today, will check latest status on it11:35
rlandyykarel: thanks  - no rush - just cleaning CIX board11:36
chkumar|ruckhttps://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers11:44
Guest1672and it crashed12:03
jpodivinchkumar|ruck: I'm running into trouble with cross-swift-py36 and cross-swift-py27 in stable/train of the requirements project.12:22
jpodivinthere seems to be some sort of encoding issue. But I don't know enough about swift properly diagnose it. 12:23
rlandyjpodivin: storage team12:26
rlandychristian?12:26
*** amoralej is now known as amoralej|lunch12:28
jpodivinrlandy: ack, Is there a tag I should mark it with? Or will the ci/promotion blocker suffice? 12:30
rlandypromotion blocker will get it to CIX12:31
rlandythat is all you need12:31
jpodivinrlandy: ok done12:35
rlandydasm: can you review https://review.opendev.org/c/openstack/tripleo-docs/+/835782 - let us know if you are still -1 there12:41
chkumar|ruckrlandy: I have not addressed the comment12:42
chkumar|ruckon this patch12:42
jpodivinrlandy: regarding the https://bugs.launchpad.net/tripleo/+bug/1968184 I'm wondering if this sort of modification could have helped https://review.opendev.org/c/openstack/tripleo-ci/+/834631/1/roles/run-test/templates/common_vars.bash.j2 12:42
chkumar|ruckgot updated due to testing of this https://review.opendev.org/c/opendev/git-review/+/823413 (know fix for fedora 36)12:42
jpodivinit's derived from what some of the other projects with nonstandard branch names are doing12:43
rlandyjpodivin: chkumar|ruck12:50
rlandys patch fixes the general problem12:50
rlandywhich is probably needed anyways12:51
chkumar|ruckrlandy: validation-common is a special case as their branch name got changed12:51
chkumar|ruckI think https://review.opendev.org/c/openstack/tripleo-ci/+/834631 would be a good approach12:51
chkumar|ruckthen we can remove validation-common from branchless project12:52
jpodivinchkumar|ruck: ok. I'm gooing to update the message on the review and move it out of the WIP12:57
jpodivinchkumar|ruck: done, review is open https://review.opendev.org/c/openstack/tripleo-ci/+/83463113:00
rcastilloo/13:00
chkumar|ruckjpodivin: please add closes-bug https://bugs.launchpad.net/bugs/1968184 in the review13:02
jpodivinchkumar|ruck: Ok, I'll do it in a minute, we're having a meeting13:02
*** amoralej|lunch is now known as amoralej13:14
rlandychkumar|ruck: jm1|rover: periodic-tripleo-ci-build-containers-ubi-8-push-wallaby passed13:17
rlandywere you investigating that one?13:18
jm1|roverrlandy: yes13:23
jm1|roverrlandy: reran successfully13:23
jm1|roverrlandy: details in ruck rover notes13:24
dasm|offchkumar|ruck: qq, are you OK with my feedback here? https://review.opendev.org/c/openstack/tripleo-docs/+/835782 If not, I can remove -1 and submit these changes in a separate chakge13:25
dasm|off*change13:25
chkumar|ruckdasm|off: I can submit those changes in a seperate patch13:26
chkumar|ruckcurrently on rr shift13:26
*** dasm|off is now known as dasm13:27
dasmchkumar|ruck: would you like me to do that for you?13:27
chkumar|ruckdasm: go ahead :-)13:27
dasmack13:27
chkumar|ruckrlandy: cix13:31
mariosfolks please have a look when you get a chance thank you -> https://review.rdoproject.org/r/c/rdo-jobs/+/41663 13:37
mariosrlandy: chkumar|ruck: dviroel: arxcruz: o/ please check when you have time thanks https://review.opendev.org/c/openstack/tripleo-ci/+/837343  &&  https://review.rdoproject.org/r/c/rdo-jobs/+/41663 13:41
akahatrlandy, c7_train promoted: http://promoter.rdoproject.org/promoter_logs/centos7_train_2022-04-11T13:15.log13:52
* dasm will join scrum after srbac13:59
mariosjm1|rover: https://hackmd.io/FGMoCiRfSNa8puA1BpTQ-Q?edit thanks14:27
chkumar|ruckrlandy: hello14:57
rlandychkumar|ruck: hi14:57
chkumar|ruckrlandy: Do we want to open a bug for ovb retry limit? I have added info here: https://hackmd.io/1lRiCS-oS3-J1ZYHZH3Xzg?view#11th-Apr-202214:57
chkumar|ruckrlandy: we are still seeing few14:57
rlandychkumar|ruck: then yes  - for tracking - pls14:58
rlandywe can close it when we no longer see the issue14:58
chkumar|ruckrlandy: can you help jm1 on that as a exercise14:58
rlandychkumar|ruck: jm1|rover: going into prod chain cal - pls ping when you are EoD and what you want watched14:58
rlandydownstream should be back now14:59
chkumar|ruckrlandy: I am logging out now14:59
rlandyperiodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-wallabytestprojectmastercheck36255,4614:59
rlandy1 hr 50 mins 16 secs2022-04-11 12:23:51SUCCESS14:59
rlandychkumar|ruck: ^^ passed14:59
chkumar|ruckrlandy: awesome14:59
rlandyshould promote network14:59
chkumar|ruckrlandy: for new master promotion, we just need 3915:02
rlandychkumar|ruck: do you want a new testproject?15:02
chkumar|ruckrlandy: https://review.rdoproject.org/r/c/testproject/+/4016915:02
rlandyah - thanks15:02
chkumar|ruckrlandy: for cs9 wallaby we need only fs064 running here jm1|rover 's patch https://review.rdoproject.org/r/c/testproject/+/4179415:05
rlandygreat15:05
* jm1|rover bbl15:06
chkumar|rucksee ya tomorrow jm1|rover rlandy \o15:08
rlandyhave a good night15:08
jm1|roverchkumar|ruck: thanks for your help :) bye15:08
dasmchkumar|ruck: your docs patch https://review.opendev.org/c/openstack/tripleo-docs/+/835782 got +W 15:16
rlandyjm1|rover: when you are back - can you rerun failed jobs from wallaby c815:24
rlandymay be some legit failures 15:24
rlandynot sure15:24
* dviroel going lunch + take car to repair shop - brb15:32
*** dviroel is now known as dviroel|lunch|afk15:32
*** marios is now known as marios|out16:08
rlandycurrent-tripleo/2022-04-11 13:21 rhos-17 on rhel-9 promoted17:22
*** amoralej is now known as amoralej|off17:30
rlandylunch - brb18:07
* dviroel ice cream - biab18:19
jm1|roverrlandy: all wallaby c8 jobs? its 6 new jobs since my morning 😳18:32
jm1|roverrlandy: https://review.rdoproject.org/r/c/testproject/+/4180818:34
*** pojadhav is now known as pojadhav|out18:40
jm1|rovercs9 wallaby fs64 is done, how to check promotion?18:40
dviroeljm1|rover: hey, you using ruck_rover.py to check jobs missing for promotion?18:49
rlandysorry - bacj18:49
rlandyback18:49
rlandyjm1|rover: no worries - testproject added already18:50
jm1|roverdviroel: yep, why?18:53
jm1|roverdviroel: ah i see18:53
dviroeljm1|rover: if all jobs are green for a specific hash, you can check promoter logs18:54
dviroeljm1|rover: i.e. if you expect a promotion18:54
rlandyjm1|rover: https://review.rdoproject.org/r/c/testproject/+/41808/1/.zuul.yaml looks right18:55
dviroeljm1|rover: OR, you can quickly check updates on tags here: https://trunk.rdoproject.org/centos9-wallaby/ (e.g. for c9 wallaby) - current-tripleo should update after a promotion18:56
rlandyjm1|rover: if you want to meet, I can take you through component promotion18:57
rlandyin downstream18:57
jm1|roverrlandy: can we do tomorrow? want to submit some patches for chandan and then go eod19:00
rlandyjm1|rover; sure19:01
rlandysee you tomorrow19:01
rlandylogging a bug for https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa&job_name=tripleo-ci-centos-9-standalone-on-multinode-ipa&skip=019:01
frenzy_fridayHey rlandy, could you pls merge https://review.rdoproject.org/r/c/config/+/41697 towards your eod today? I'll start the testprojects then tomorrow in my morning19:02
* dviroel looks too19:03
rlandyfrenzy_friday: will do19:03
frenzy_fridaythanks :)19:03
rlandydviroel: +2'ed https://review.rdoproject.org/r/c/config/+/4169719:04
rlandyfeel free to w+ if you are ok with it19:04
rlandyshould only impact quay container19:04
dviroelack19:05
rlandyneed to reboot here19:07
dviroelfrenzy_friday: still around?19:08
dviroelfrenzy_friday: rlandy: i see 4 places to fix c9 secret https://codesearch.rdoproject.org/codesearch/?q=quay_rdoproject_tripleomastercentos9_secret&i=nope&files=&repos=19:11
dviroel2 fixed in the same patch19:12
dviroelanother 1 here: https://review.rdoproject.org/r/c/config/+/41787/619:12
dviroelmissing one19:12
jm1|roverdviroel: thanks for the tip, will check it :)19:13
*** rlandy_ is now known as rlandy19:28
rlandyade_lee: hello ... https://bugs.launchpad.net/tripleo/+bug/1968615 - multinode-ipa job started failing today19:32
rlandyin check and periodic19:32
rlandyade_lee: any recent changes  you are aware of that may have triggered this?19:33
rlandyjm1|rover: chkumar|ruck: ^^ fyi - gate and promo blocker19:33
jm1|roverrlandy: did you already add it to hackmd?19:45
jm1|roverrlandy: cannot find it, will add it19:46
rlandyjm1|rover: not yet - waiting to chat with ade_lee about fix19:48
* dviroel pick up car in repair shop - brb19:49
rlandyjm1|rover; no worries - will add it by my EoD19:49
jm1|roverrlandy: shouldn't this job show up in ./ruck_rover.py --release master --distro centos-9 ?19:49
rlandyjm1|rover: probably passed in a previous run19:50
rlandyof the same hash19:50
rlandyhttps://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_agg_detail.html?ci_name=periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-master19:54
rlandyhttps://logserver.rdoproject.org/69/40169/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/f5a498a/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz19:56
rlandymay be related issue19:56
jm1|roverrlandy: this ipa job fails the ping test. actually i cannot ping redhat.com from my system as well.20:02
jm1|roverredhat.com resolves to two ip addresses for me, 34.235.198.240 and 52.200.142.250. i cannot ping both?!20:03
jm1|roverrlandy: in the jobs which did not fail, redhat.com was resolved to 209.132.183.105 which is pingable from my system20:04
rlandyjm1|rover: hmmm .. maybe something on their side20:04
rlandyI can't ping it either20:04
rlandyI can ping ibm.com20:05
jm1|roverrlandy: its probably some kind of "protection"/firewall. i can open https connections to all of those ips20:05
jm1|roverrlandy: only icmp is blocked20:06
rlandyjm1|rover: must be new20:06
rlandyjm1|rover: let me try change the test ping20:07
rlandyand see if that works20:07
jm1|roverrlandy: older jobs all resolved to 209.132.183.10520:08
jm1|roverrlandy: so redhat.com has to fix its firewall. let me open a bug ;)20:08
* dasm stepping away for a bit20:10
jm1|roverrlandy: opened a bug in service-now, because i have no idea who is responsible for redhat.com..20:16
rlandyjm1|rover: do you have a ticket number I can add to a bug?20:17
jm1|roverrlandy: INC217542220:19
jm1|roverrlandy: added it to ruck rover hackmd as well20:19
jm1|roverrlandy: i am now EOD20:19
rlandyjm1|rover: thanks - have a good night20:20
ade_leerlandy, hey - just saw your ping, been in meetings21:19
ade_leerlandy, are we really failing to resolve redhat.com?21:19
rlandyade_lee: yep - jm1 thinks it's a problem on redhat.com's side21:23
rlandyI changed the test site21:23
rlandytrying that noe21:23
rlandynow21:23
ade_leeok21:23
dviroelrlandy: sorry, missed https://review.rdoproject.org/r/c/config/+/4169721:37
rlandydviroel: no worries21:37
rlandymerged it21:37
rlandythought you were EoD21:37
dviroelrlandy: i was reviewing the following one, which I think that is needed now that we merged this one21:37
rlandydviroel: yeah - that one frenzy_friday can merge in her morning21:38
dviroelthis one https://review.rdoproject.org/r/c/config/+/41787/621:38
rlandythe one with impact21:38
dviroelack21:38
* dviroel going out o/22:02
*** dviroel is now known as dviroel|out22:02
dasmo/22:15
*** dasm is now known as dasm|off23:57

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!