Tuesday, 2022-11-01

dviroel|rover|dinnerrlandy|out: ok00:33
*** dviroel|rover|dinner is now known as dviroel|rover00:33
dviroel|rover"success": true - BM job passed, 17-rhel9 can promote00:35
* dviroel|rover out /o/00:36
*** dviroel|rover is now known as dviroel|out00:36
marioso/ 05:57
ysandeep\o06:11
*** ysandeep is now known as ysandeep|afk06:42
mariosreview opendev org down?07:08
chandankumaryes it is not working for me07:13
marioschandankumar: couple folks reporting in opendev chan so seems to be a thing07:17
-opendevstatus- NOTICE: review.opendev.org (Gerrit) is currently down, we are working to restore service as soon as possible07:30
*** ysandeep|afk is now known as ysandeep07:51
mariosarxcruz|ruck: o/ morning anything you want to discuss just let me know08:18
mariosdviroel|out: arxcruz|ruck: i am still looking at https://bugzilla.redhat.com/show_bug.cgi?id=2138436 something further needed there (will work on it today )08:19
arxcruz|ruckis only for me or review.o.o is out?08:42
mariosarxcruz|ruck: yeah its out 09:30 -opendevstatus:#oooq- NOTICE: review.opendev.org (Gerrit) is currently down, we are working to restore service as soon as possible08:45
arxcruz|ruckhmmm, i did not get this notice... need to check my hexchat configuration :/08:46
mariosarxcruz|ruck: fyi seeing something strange in ci-config (working on https://bugzilla.redhat.com/show_bug.cgi?id=2138436 ) ... perhaps related to upstream outage? all ci-config reviews fail like https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45897/1#message-f46308ae621cfaaaafb3d7a669beb95a522b4a7008:53
reviewbotDo you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks.08:53
mariosarxcruz|ruck: i posted that one as a sanity check https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45897/1#message-f46308ae621cfaaaafb3d7a669beb95a522b4a70 08:54
mariosThis change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset08:54
arxcruz|ruckmarios, probably, since some repos depends on upstream, and inherit from zuul upstream right08:54
mariosyeah might be related08:57
arxcruz|ruckyeah, my rechecks are not going through as well 08:59
mariosarxcruz|ruck: just asking in rhos-ops in case (join please? )09:51
mariosarxcruz|ruck: cos upstream zuul is OK/responding, and for something like required-projects it would try to talk to git not gerrit right? 09:51
marioswondering if somethign else going on 09:52
mariosarxcruz|ruck: fyi also hitting rdo-jobs another sanity check there https://review.rdoproject.org/r/c/rdo-jobs/+/45900/1#message-25c66a7dcd85fa081a26b7a61499924a1eaa6e1109:52
arxcruz|ruckmarios, zuul is responding, but i think when you get some recheck or depends-on, it pull from review.o.o instead of directly from git09:53
mariosarxcruz|ruck: sure but my tests don't have any depends-on and still hitting it. might be related but dno't get how09:55
mariosarxcruz|ruck: https://review.rdoproject.org/r/c/rdo-jobs/+/45900/1#message-25c66a7dcd85fa081a26b7a61499924a1eaa6e11 https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45897/1#message-f46308ae621cfaaaafb3d7a669beb95a522b4a70 09:55
mariosno depends ^^ 09:55
mariosfooooooood 09:55
mariosbrb09:55
*** rlandy|out is now known as rlandy10:38
rlandyarxcruz|ruck: hey - how are things?10:39
rlandyinfra issue?10:39
arxcruz|ruckrlandy, i need 50 min, lunch, but upstream gerrit is down, which is impacting our side as well 10:39
arxcruz|ruckso basically not much to do 10:39
marioso/10:39
rlandymarios: hey10:40
rlandymarios: doug and I went around a bit on your dln reporting patch10:40
mariosrlandy: yeah i caught up but am stuck because of gerrit outage 10:40
rlandymarios: I thought we should be doing https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/43355110:40
mariosrlandy: i updated/summary of current situation at https://bugzilla.redhat.com/show_bug.cgi?id=2138436#c2 10:41
mariosrlandy: no i think what we need now is https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45896 to force re-fetch of the repo file (didn't refetch it by default)10:42
mariosrlandy: but we are stuck/cant test that 10:42
mariosrlandy: we want to avoid the pre-dlrn (re tripleo-ci-internal-config/+/433551) because that was the problem we had with the original fix at https://review.rdoproject.org/r/c/config/+/45865 it was during the pre so too early 10:43
rlandyok - looking at patch 710:44
mariosrlandy: we should be able to test that one (ci-config) is the good news on this 10:45
mariosrlandy: but blocked on https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45896/3#message-a0c36c201c7d10ae3dddfafa42eee295f216137110:45
mariosrlandy: we pinged rhos-ops about those and we assume it is to do with upstream outage but don't know how/why yet10:45
rlandyprobaably the dependecy chain10:50
rlandyrcastillo: hey - happy  year anniversary10:56
rlandyarxcruz|ruck: (when you get back) scn003 - what is the last word there?11:00
ysandeeprcastillo: Happy work anniversary! :)11:01
marioscongrats rcastillo o/ 11:15
arxcruz|ruckrlandy, back, regarding sc003 i put a patch yesterday to fix it 11:24
rlandyarxcruz|ruck: so we are all set for all scenario003 runs in check and gate?11:25
arxcruz|ruckrlandy, the patch did not merge yet, and it's out so... hard to say 11:25
arxcruz|ruckbut once patch merge we will be good everywhere 11:25
rlandygood11:25
chandankumarrlandy: Happy work anniversary :-)11:26
*** dviroel|out is now known as dviroel|rover11:28
dviroel|rovero/11:28
rlandyarxcruz|ruck: network on train component needs some help pls - when gerrit returns11:31
rlandydviroel|rover: hello11:31
rlandywe are without gerrit11:31
dviroel|rovernooo11:31
*** ysandeep is now known as ysandeep|afk11:36
rlandyugh gerrir11:37
rlandygerrit11:37
rlandychandankumar: dviroel|rover: ysandeep|afk: we have to do a 10 minute lightning "current state of next gen CI"11:41
rlandyin addition to other topics11:41
rlandyputting together some slides for your review and additions11:41
dviroel|roverrlandy: ok11:41
mariosdviroel|rover: o/ fyi https://bugzilla.redhat.com/show_bug.cgi?id=2138436#c2 (working on https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45896 but blocked on testing that due to gerrit outage)11:41
dviroel|rovermarios: ack, tks11:42
dviroel|roverrlandy: marios: osp17-1 rhel-8 is 6 days out due to mixed os, we should skip it11:51
dviroel|roverrlandy: marios: https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/43351211:51
chandankumarrlandy: ok11:53
arxcruz|ruckrdo is also out now 11:55
arxcruz|ruckAn error occurred during a connection to review.rdoproject.org:29418. PR_END_OF_FILE_ERROR11:55
* soniya29 stepping out for 1 hr11:56
*** soniya29 is now known as soniya29|afk11:57
dviroel|rover(╯°□°)╯︵ ┻━┻11:57
mariosdviroel|rover: sure makes sense11:59
rlandydviroel|rover: ack - pls promo that12:01
dviroel|roverrlandy: i don't have the powers there12:03
rlandydviroel|rover: merged12:05
dviroel|rovertks12:06
mariostoday is a bit of a disaster then 12:10
marios:/12:10
rlandychandankumar: dviroel|rover: ysandeep|afk: sent you all the link to the lighting talk slides I am working on12:12
rlandychandankumar: dviroel|rover: ysandeep|afk: the headings are all there12:13
rlandychandankumar: dviroel|rover: ysandeep|afk: please edit/add/comment if you think I have left out a major topic12:13
rlandychandankumar: dviroel|rover: ysandeep|afk: if you're in agreement - pls add notes to the slides with your names on them12:14
rlandysee speaker notes12:14
*** ysandeep|afk is now known as ysandeep12:15
* ysandeep looking12:15
rlandychandankumar: dviroel|rover: ysandeep|afk: pls note - this is just on overrriew12:16
rlandyso a few points and a link or two is all we need12:16
ysandeeprlandy, it looks great for high level overview12:17
rlandyysandeep: thanks - but I am still writing there12:18
dviroel|roverrlandy: i will take a look12:24
ysandeepyes, I mean you covered all the major topics, can't think of another topic for high level overview.12:24
rlandythanks - no rush12:24
* ysandeep looking to add notes to individual slides12:24
rlandyI can add as I go but probably more efficient if the person working on the topic adds the points12:25
rlandyjust a few high level12:25
chandankumarrlandy: looks good12:26
chandankumarrlandy: in the speaker notes sections, individual names are written, can we write those slides or you are going to add that? and later we can review and update it12:27
rlandychandankumar: pls add the slide content there12:28
rlandyI can do it but probably better if the person most familr does12:28
rlandyfamiliar12:28
rlandychandankumar: I'll take a cut at what is not added12:29
rlandyand we can review12:29
rlandybut yeah - better if each does the area they know best12:29
ysandeepI will add notes on my slides after podified mtgs12:29
chandankumarwill do that, thank you :-)12:30
dviroel|roverosp17-1 rhel8 promoted12:55
dviroel|roverat least one promotion :P12:56
rlandyakahat: arxcruz|ruck: chandankumar: dviroel|rover: dasm|off: rcastillo: soniya29|afk: ysandeep: jm1[m]: marios: frenzy_friday; community call in about 30 mins: pls add agenda items on https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg12:57
rlandyline 2912:57
rlandydviroel|rover: should we revert that now?13:10
dviroel|roverrlandy: yeah13:15
rlandydviroel|rover: ok - I'll take care of that13:16
rlandydone13:17
dviroel|rovertks13:17
*** dasm|off is now known as dasm13:29
dasmo/13:29
dasmmarios: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4520613:39
mariosdasm: i see thx 13:41
dasmmarios: like i said: i did too much and it made reviewers unhappy. however docker-compose was the latest one working for me13:41
mariosgerrit back? 14:15
rlandypls say yes14:16
rlandywoohoo14:16
rlandymy commit went through14:16
*** ysandeep is now known as ysandeep|dinner14:22
-opendevstatus- NOTICE: review.opendev.org (Gerrit) is back online14:25
arxcruz|ruckyay, opendev is back14:38
rlandymarios: the jobs of concern before look green now ... https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4561114:54
rlandyok if I w+?14:54
rlandyakahat: rechecking your patches for https://review.rdoproject.org/r/q/topic:rdo_registry_decomission14:59
arxcruz|ruckis the all hands now or in one hour? (summer time ends here in germany, my calendar is a bit weird)15:02
rcastilloshows up in an hour for me too15:05
mariosrlandy: yeah thanks i reran the network fs1 and it was green so ready 15:07
* dviroel|rover lunhc15:09
*** dviroel|rover is now known as dviroel|rover|lunch15:09
mariosrlandy: dviroel|rover|lunch: arxcruz|ruck: fyi workflowed the last zed components criteria patch https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4561115:16
marioscoffee brb15:16
rlandyok15:23
rlandydasm: dviroel|rover|lunch: rcastillo: only two patches on review list today15:29
rlandypls check them in your time15:30
rlandyand we can skip review time later15:30
rlandyrest are all crossed our15:30
rlandyout15:30
dasmk15:30
*** ysandeep|dinner is now known as ysandeep15:40
rlandymarios: rcastillo: just fyi - since you guys are taking next week's RR shift - will likely be out on Friday - most of the day to prep for Brno trip15:54
mariosrlandy: sure np15:54
rcastillorlandy: ack that's fine15:55
rlandyadded PTO to  calendar - will check in at some point15:55
rlandydviroel|rover|lunch: ysandeep: chandankumar: moving our friday meeting as well ... will be out then15:56
ysandeepack o/15:57
rlandymoved to thursday15:57
rlandywe can review presentations then15:57
rlandyall: reminder all hands now16:00
*** dviroel|rover|lunch is now known as dviroel|rover16:12
arxcruz|ruckrlandy, dviroel|rover please review https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/86315916:40
arxcruz|ruckmy mistake, i'm sorry, zed still using the old tempest release, and i unskip yesterday 16:41
rlandyarxcruz|ruck: can't we rev tempest in zed?16:41
rlandyeasier than changing walaby16:41
arxcruz|ruckrlandy, we can, but we need it skipped first to get promotions 16:42
arxcruz|ruckcuz change wallaby needs also patch get merged, build, promte the component 16:42
arxcruz|ruckskipping now will unblock us 16:42
rlandyarxcruz|ruck: ok - but pls keep the CIX card open until we clear this16:43
arxcruz|ruckyes16:43
rlandyvoted16:43
mariosrlandy: dviroel|rover: re mixed-rhel BZ -  fyi - looks like the depends-on was not included https://code.engineering.redhat.com/gerrit/c/testproject/+/427472/9#message-f5dcc3183ccf5ad576ba2a1db076ccbf9698d663 - perhaps you can have a look if you get some time, otherwise I will continue tomorrow 17:00
mariostoday was a bit of a disaster with gerrit :/17:00
mariosi still think https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45896 can help us - maybe we can merge that tomorrow and try it if we can't work out how to get it with depends-on 17:01
mariosrlandy: dviroel|rover: ^^ 17:01
dviroel|rovermarios: ack, weird17:03
rlandymarios: dviroel|rover: config playbook?17:04
rlandywon;t take changes17:04
rlandyunmerged17:04
dviroel|roverrlandy: rdo ci-config role change17:06
rlandycalled from where?17:06
dviroel|roverinternal17:07
dviroel|roverhttps://code.engineering.redhat.com/gerrit/c/testproject/+/427472/17:07
* ysandeep out, see everyone tomorrow o/17:07
*** ysandeep is now known as ysandeep|out17:07
rlandydviroel|rover: so the change is called from within in a config based playbook?17:08
rlandyie: the role being changed17:08
dviroel|roverhumm17:08
dviroel|roverprobably17:08
mariosrlandy: yeah might be cos of that (it is called from config play)17:10
dviroel|rover"internal-config/playbooks/dlrn/post-dlrn.yaml"17:11
mariosright17:11
* rlandy could be wrong but I think I remember sagi pointing that out17:11
mariosrlandy: dviroel|rover: if you folks work something out leave me a note (eg on the testproject) otherwise i may try and merge ci-config/+/45896 in my morning and test with that 17:12
rlandyysandeep|out: chandankumar: dviroel|rover: thanks for filling in your slides17:13
rlandywill read through17:13
rlandydviroel|rover: maybe we try the merge and revert?17:13
rlandyin our afternoon17:13
rlandyquiter now17:13
rlandyquieter17:13
rlandythan european morning17:13
dviroel|roverrlandy: yeah, we can do that17:13
* rlandy looks at patch again17:14
rlandydviroel|rover: alright ... merging https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4589617:14
rlandydviroel|rover: lmk if I should revert17:14
mariosrlandy: dviroel|rover: thanks, basically the get_hash role fetches the delorean.repo.md5 during pre and creates hash_info. we are trying to re-run that just before dlrn reporting to fetch the 8 stuff for report but it doesn't fetch it because delorean.repo.md5 already exists (from the pre) that is why i added force: true to get_url 17:15
rlandylunch - brb17:15
dviroel|rovermarios: ack17:15
mariosthanks rlandy dviroel|rover 17:15
*** marios is now known as marios|out17:22
rlandydviroel|rover: need help with anything?18:45
dviroel|roverrlandy: just saw that https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-containers-multinode&skip=018:49
rlandyno logs?18:49
dviroel|roverno logs18:49
dviroel|roverrlandy: we also need to merge arxcruz|ruck fix, now affecting check/gates too18:53
dviroel|roveror propose a different one if needed18:53
dviroel|roverhttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-scenario003-standalone&skip=018:54
rlandydviroel|rover: merged https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/86315918:57
rlandyugh18:57
rlandythat is what I asked18:57
rlandyif it was going o kill check18:57
rlandywe merged zed18:57
rlandyand he removed master18:57
rlandydviroel|rover: pls put master check in18:57
rlandyso we can resolve it18:57
rlandyin skip for now18:57
rlandyto unblock gate18:57
rlandyugh - I didn't want to promote master if if will block check18:58
arxcruz|ruckrlandy, dviroel|rover the fix for scen003 is https://review.opendev.org/c/openstack/tripleo-ci/+/86309218:59
dviroel|roverarxcruz|ruck: rlandy voted19:00
rlandyarxcruz|ruck: dviroel|rover: my guess it this will break downstream19:00
dviroel|roverrlandy: then we can override on downstream?19:01
rlandyarxcruz|ruck: dviroel|rover: we'll need to override it there19:01
rlandyarxcruz|ruck: dviroel|rover: pls put in an override patch for downstream19:02
rlandyor some change that can refer to custom_nameserver19:02
rlandyand then we can merge this19:02
rlandyotherwise it will hose downstream as soon as we merge19:03
dviroel|roverrlandy: i will work on that19:03
rlandydviroel|rover++19:04
arxcruz|ruckit should not affect check rn, sorry about that :(19:05
dviroel|roverrlandy: didn't find any internal job that inherits from scenario003 or scenario003-options19:10
rlandyit should inherit from options19:11
rlandychecking19:11
dviroel|roverregex .*scenario003.* gives nothing19:11
rlandyha - maybe we don;t run that in downstream19:11
rlandydviroel|rover: arxcruz|ruck: we don;t19:12
rlandysorry about that19:12
rlandyidk why we don;t19:12
rlandybut ok19:12
rlandylet's go then19:12
dviroel|roverack19:12
rlandyw+19:13
rlandysorry arxcruz|ruck 19:13
dviroel|roverrlandy: waiting network component on ds, i kicked earlier, after that promote components and kick one line that we missed19:16
marios|outdviroel|rover: i think it is OK now but i'll have to sanity check logs in the morning https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/3a2d6ba8f0f249d289980ae03462a7d6?logfile=console.log 19:16
marios|outgoing to report in a minute ^ 19:16
* dviroel|rover checks19:17
marios|out2022-11-01 19:15:48.937481 | TASK [get_hash : get md5 file]19:17
marios|out2022-11-01 19:15:49.557743 | primary | changed: OK (32 bytes)19:17
dviroel|roverhash 8938272e192d9849efc2711e2941df3819:18
marios|outyea looks different right? and no error from dlrn report this time 19:18
dviroel|roverrhel8-osp17-1/tripleo-ci-testing/89/38/8938272e192d9849efc2711e2941df38/19:18
dviroel|roveryeah19:18
marios|outcool 21:18 < dviroel|rover> rhel8-osp17-1/tripleo-ci-testing/89/38/8938272e192d9849efc2711e2941df38/19:18
dviroel|rovermarios|out: tks, pls check in details on your morning, have a good night o/19:19
marios|outo/ dinner time thanks for checking 19:19
rlandydviroel|rover: looks right https://sf.hosted.upshift.rdu2.redhat.com/logs/72/427472/9/check/periodic-tripleo-ci-rhel-8-9-multinode-mixed-os-rhos-17.1/3a2d6ba/job-output.txt19:49
rlandyhash is 8 hash19:49
dviroel|roveryeah19:51
rlandyfinally19:54
rlandystill battling fs03919:54
rlandydviroel|rover: stepping away for an hour or so - pls leave notes if you want me to do/checks something later21:03
dviroel|roverrlandy: ack21:04
rlandyenjoy your day off21:04
dviroel|roverrlandy: thanks o/21:05
arxcruz|ruckdviroel|rover, rlandy leave comments on the hackmd if you need me to take some action tomorrow morning my time 21:24
dviroel|roverarxcruz|ruck: k21:26
dviroel|rover16-2 started21:32
* dviroel|rover bbl21:32
*** dviroel|rover is now known as dviroel|rover|bbl21:32
rcastilloI'm EOD, I'll see you on thursday22:49
rlandyarxcruz|ruck: https://review.opendev.org/c/openstack/tripleo-ci/+/863092 merged so we're good22:53
rlandyarxcruz|ruck: pls prep the program call doc22:53
rlandyI'll check in with you on that in my morning22:53
* rlandy out22:53
*** rlandy is now known as rlandy|out22:54
*** dasm is now known as dasm|off23:09
*** dasm|off is now known as Guest20223:37

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!