*** ysandeep|out is now known as ysandeep|ruck | 04:02 | |
*** ysandeep|ruck is now known as ysandeep|ruck|afk | 04:38 | |
*** ysandeep|ruck|afk is now known as ysandeep|ruck | 05:08 | |
jm1 | hello team :) | 05:56 |
---|---|---|
ysandeep|ruck | o/ hello everyone | 05:58 |
ysandeep|ruck | arxcruz: could you please add in your review list: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/848472 | 06:11 |
*** ysandeep|ruck is now known as ysandeep|ruck|afk | 06:18 | |
*** amoralej|off is now known as amoralej | 06:19 | |
*** ysandeep|ruck|afk is now known as ysandeep|ruck | 07:11 | |
bhagyashris | marios, hi, i am facing freeze graph issue here https://review.opendev.org/c/openstack/tripleo-ci/+/848260 could you plz help me when you have free time. | 07:30 |
marios | bhagyashris: k will have a look in a bit | 07:34 |
*** kopecmartin_ is now known as kopecmartin | 07:39 | |
bhagyashris | thanks | 07:49 |
marios | bhagyashris: commented @ https://review.opendev.org/c/openstack/tripleo-ci/+/848260/11#message-4907e167356e5c0361cddd7356a5096cc5640106 one suggestion to try first (branches:) | 08:11 |
*** ysandeep|ruck is now known as ysandeep|lunch | 08:15 | |
*** ysandeep|lunch is now known as ysandeep|ruck | 10:16 | |
*** rlandy|out is now known as rlandy | 10:24 | |
rlandy | ysandeep|ruck: hello | 10:24 |
rlandy | ysandeep|ruck: let's sync when you are free | 10:25 |
ysandeep|ruck | rlandy: good morning o/ meet.google.com/nct-ibka-ccq | 10:26 |
rlandy | joining | 10:27 |
arxcruz | Hey guys, I have a reunion in my daugther school, i'll be back in 2 hours +- | 10:57 |
*** arxcruz is now known as arxcruz|brb | 10:59 | |
rlandy | amoralej: sorry for the delay - sending response to ci.centos thread | 11:17 |
rlandy | marios: pls see email - signed updated | 11:17 |
amoralej | ack, thanks rlandy | 11:17 |
marios | rlandy: thank you | 11:21 |
rlandy | frenzy_friday: bhagyashris: need to chat with you about 17.1 component line | 11:24 |
frenzy_friday | rlandy, sure | 11:24 |
rlandy | frenzy_friday: bhagyashris: ok to in 15 mins? | 11:24 |
rlandy | just answering a mail | 11:24 |
frenzy_friday | yep, sure | 11:25 |
rlandy | rcastillo|rover: when you get in - we need to chat re; virt jobs in PSI/vexxhost | 11:27 |
*** dviroel|biab is now known as dviroel | 11:29 | |
rlandy | frenzy_friday: bhagyashris: https://meet.google.com/ijn-jius-owf?pli=1&authuser=0 | 11:36 |
*** amoralej is now known as amoralej|lunch | 11:54 | |
*** arxcruz|brb is now known as arxcruz | 12:03 | |
rlandy | ysandeep|ruck: putting in the compute component reruns | 12:09 |
ysandeep|ruck | ack.. fyi.. tripleo/network/tempest is already in rerun | 12:10 |
ysandeep|ruck | rlandy, remove fs035 from wallaby c8 criteria: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43872 , I will drop the job run in a separate patch. | 12:22 |
ysandeep|ruck | reviewbot: please add in reviewlist: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43872 | 12:24 |
reviewbot | I have added your review to the Review list | 12:24 |
rlandy | marios: dviroel: rcastillo|rover: ^^ fyi | 12:30 |
rlandy | suggesting to remove fs035 from wallaby c8 | 12:30 |
rlandy | we usually skip promote this anyways | 12:30 |
rlandy | in next rr shift, we need to do an evaluation of fs035 | 12:30 |
rlandy | to see if we can get it stable | 12:30 |
rlandy | maybe with fewer tempest tests moved to another ipv6 job | 12:31 |
ysandeep|ruck | rlandy, Question - 17.1 - are we doing only on rhel 9 or rhel-8 also? | 12:31 |
rlandy | marios: do you need that fs035 job for mixed-rhel? | 12:31 |
*** amoralej|lunch is now known as amoralej | 12:31 | |
rlandy | ysandeep|ruck: assume only rhel-9 | 12:31 |
rlandy | good question though | 12:31 |
rlandy | we may need 8 for mixed rhel | 12:32 |
rlandy | but starting with 9 | 12:32 |
rlandy | bhagyashris: pojadhav: is soniya around today? | 12:33 |
rlandy | akahat: ^^ anyone seen her? | 12:33 |
pojadhav | rlandy, soniya is on pto | 12:33 |
akahat | rlandy, no. | 12:33 |
rlandy | oh oops yes | 12:33 |
rlandy | pojadhav++ thanks for reminder | 12:33 |
rlandy | I see calendar | 12:33 |
pojadhav | rlandy, :) | 12:33 |
ysandeep|ruck | rlandy: okay, I think we should reduce our footprint for el8 for wallaby/17. | 12:33 |
ysandeep|ruck | we are running complete line on 17/8 | 12:33 |
dviroel | rlandy: ack | 12:34 |
pojadhav | rlandy, i think she will be available by friday this week. | 12:34 |
ysandeep|ruck | and if 17.1 is not on rhel-8, continuing running complete line for 17.0 make less sense | 12:34 |
rlandy | ysandeep|ruck: if we need rhel-8 for 17.1, my suggestion would be to run one rhel-8 line | 12:34 |
rlandy | ack | 12:34 |
rlandy | ysandeep|ruck: delivery is a bit slammed now with imports | 12:35 |
rlandy | will confirm that on monday's prod chain call when jason is back | 12:35 |
rlandy | good question | 12:35 |
ysandeep|ruck | yeah so are we tracking promotions :) | 12:35 |
rlandy | ysandeep|ruck: but agree we are not keeping up two full rhel-8 lines | 12:35 |
rlandy | only one | 12:35 |
rlandy | or some smaller version | 12:36 |
ysandeep|ruck | ack | 12:36 |
rlandy | my guess is we whole sale switch rhel-8 to 17.2 | 12:36 |
rlandy | 17.1 | 12:36 |
rlandy | once 9 is stable | 12:36 |
marios | rlandy: o/ no i have a new fs for mixed rhe | 12:37 |
rlandy | marios: awesome - any objection to https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43872? | 12:38 |
marios | rlandy: nope voted | 12:41 |
ysandeep|ruck | rlandy, marios could one of you please +w ^^ | 12:43 |
marios | ysandeep|ruck: k | 12:43 |
ysandeep|ruck | thanks I was tracking c8 wallaby promotion | 12:43 |
rlandy | mario beat me to it :) | 12:44 |
ysandeep|ruck | rlandy, https://review.rdoproject.org/r/c/rdo-jobs/+/43835 could you review - remove ipa job from train | 12:45 |
ysandeep|ruck | and https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43860 | 12:45 |
rlandy | checking | 12:49 |
rlandy | done | 12:50 |
ysandeep|ruck | rlandy: thanks! | 12:51 |
*** ysandeep|ruck is now known as ysandeep|brb | 12:52 | |
*** ysandeep|brb is now known as ysandeep|ruck | 13:01 | |
rlandy | ysandeep|ruck: rcastillo|rover: atm there are two wallaby patches in the gate | 13:03 |
rlandy | rest are master | 13:03 |
ysandeep|ruck | last time rush :) | 13:04 |
ysandeep|ruck | we can rekick the tripleo component one more time before eod | 13:04 |
ysandeep|ruck | once those patch which are currently in gate merges | 13:05 |
ysandeep|ruck | patches* | 13:06 |
rlandy | yep in our afternoon time | 13:14 |
*** dasm|off is now known as dasm | 13:15 | |
dasm | o/ | 13:15 |
rlandy | now there are a bunch | 13:15 |
rlandy | all tripleo | 13:15 |
bhagyashris | arxcruz, rlandy, marios, ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat, chandankumar, frenzy_friday, anbanerj, dviroel, rcastillo, dasm, jm1, marios, ysandeep|ruck rcastillo|rover | 13:25 |
bhagyashris | Tripleo CI community meeting in 5 mins | 13:25 |
bhagyashris | https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both#2022-07-05-Community-Call @line 29 | 13:25 |
dasm | arxcruz: o/ have you had an opportunity to review https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43815 ? | 13:26 |
dasm | i don't see any comments from you | 13:26 |
dasm | arxcruz: unfortunately you're the only one who chose golang, so you're the only one who can review this ^^ | 13:27 |
dasm | arxcruz: btw. i like how golang works with coroutines | 13:27 |
arxcruz | dasm shoot, forgot that, i'll take a look | 13:30 |
bhagyashris | rlandy, pojadhav ^ | 13:31 |
bhagyashris | rcastillo|rover, | 13:31 |
arxcruz | dasm is nice for parallelism, thaat's the reason it's faster than python :) | 13:31 |
arxcruz | work with threads in python is a pita | 13:31 |
dasm | arxcruz: you could use async too :) | 13:31 |
dasm | it's similar between python and golang | 13:32 |
arxcruz | dasm yup indeed, but i think at that time requests did not have async | 13:32 |
bhagyashris | Tengu, ^ | 13:32 |
dasm | ack | 13:32 |
arxcruz | i don't recall exactly why i did not use async | 13:32 |
Tengu | uho | 13:32 |
Tengu | joining, sorry. | 13:32 |
rlandy | rcastillo|rover: ysandeep|ruck: want to hold on and sync before ysandeep|ruck EoD? | 14:03 |
ysandeep|ruck | yes.. I will be eod shortly so lets sync | 14:03 |
rcastillo|rover | rlandy: yeah | 14:04 |
arxcruz | dasm wow, looks nice the output, and very quick! | 14:08 |
dasm | \( ゚ヮ゚)/ | 14:10 |
dasm | arxcruz: can i get +2 +W from you? | 14:11 |
dasm | like i said -- you're the only one knowing golang here ^^ | 14:11 |
dasm | or admitting to that :P | 14:12 |
Tengu | golang? | 14:12 |
* Tengu hides and wants to forget he actually did an app in golang | 14:12 | |
dasm | Tengu: we have a tool written in golang which fetches some results: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43815 | 14:12 |
dasm | Tengu: if you want to :) | 14:13 |
Tengu | who had such an idea ^^' | 14:13 |
dasm | arxcruz ;) | 14:13 |
arxcruz | dasm sure | 14:13 |
arxcruz | Tengu was that a bad idea ? | 14:14 |
* Tengu renames arxcruz to Horcrux | 14:14 | |
dasm | arxcruz: right now we have a bus factor 1 ;) | 14:14 |
dasm | Tengu: xD | 14:14 |
arxcruz | dasm we need to update the mariadb script | 14:15 |
Tengu | more seriously, I can't review it. my golang is really limited and it was about a year ago -.-'. | 14:15 |
ysandeep|ruck | rcastillo|rover, https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/component-pipeline-wallaby-centos9.yaml#L43-L52 | 14:20 |
rcastillo|rover | ysandeep|ruck: rlandy https://review.rdoproject.org/r/c/testproject/+/43881 | 14:25 |
arxcruz | jm1 frenzy_friday https://ustraveldocs.com/de/de-niv-visafeeinfo.asp | 14:27 |
dasm | arxcruz: you mean this update? https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43780 | 14:28 |
arxcruz | dasm yes | 14:28 |
ysandeep|ruck | rcastillo|rover, rlandy https://logserver.rdoproject.org/53/21853/93/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby/3977293/logs/undercloud/var/log/tempest/failing_tests.log.txt.gz | 14:35 |
rcastillo|rover | rlandy: did you want to talk about libvirt on vexx/psi? | 14:39 |
*** ysandeep|ruck is now known as ysandeep|out | 14:39 | |
rlandy | rcastillo|rover: yeah - let's chat for when you get off rr | 14:39 |
rcastillo|rover | alright | 14:40 |
rlandy | https://meet.google.com/htc-yquz-kap?pli=1&authuser=0 | 14:40 |
rlandy | rcastillo|rover: ^^ | 14:40 |
rlandy | rcastillo|rover: nodeset: single-centos-9-big-node | 14:45 |
rlandy | cloud-centos-9-stream-big | 14:46 |
dasm | arxcruz: thanks for +2. Any objections before +W? I don't think anyone else is gonna review this code | 15:00 |
Tengu | marios: fyi: https://review.opendev.org/c/openstack/tripleo-ansible/+/848754 will replace the t-h-t patch. No big change on your side though I think, unless CI is using tripleo_bootstrap to install needed packages. | 15:03 |
marios | Tengu: ack noted | 15:03 |
* dviroel lunch | 15:07 | |
*** dviroel is now known as dviroel|lunch | 15:07 | |
pojadhav | rlandy, rcastillo|rover, dviroel|lunch: please review when free https://review.rdoproject.org/r/q/topic:ooo_c7_teardown+owner:pojadhav | 15:36 |
pojadhav | need reviews : https://review.opendev.org/c/openstack/tripleo-quickstart/+/843837 | 15:36 |
reviewbot | I have added your review to the Review list | 15:36 |
pojadhav | need reviews : https://review.rdoproject.org/r/q/topic:ooo_c7_teardown+owner:pojadhav | 15:37 |
reviewbot | I have added your review to the Review list | 15:37 |
arxcruz | dasm no, it's fine do you want me to +w ? | 15:55 |
dasm | arxcruz: yeah, go for it | 15:56 |
dasm | arxcruz: btw, there is another one, which requires your attention: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/848472 | 15:57 |
rlandy | rcastillo|rover: ugh - gate reset | 16:02 |
*** marios is now known as marios|out | 16:03 | |
rcastillo|rover | rlandy: T-7 hrs, maybe we get that through? | 16:04 |
rlandy | gate is getting longer | 16:05 |
rlandy | whatever is in when we kick the last tripleo line promo is in | 16:05 |
rlandy | ugh baremetal just updated | 16:06 |
rlandy | kicking that | 16:06 |
rcastillo|rover | ok we can juggle one more | 16:07 |
dasm | o/ i see this: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/843870/4 has +2 +W but it's not merged. Should we rekick zuul on that? | 16:12 |
rlandy | lunch brb | 16:18 |
*** dviroel|lunch is now known as dviroel | 16:22 | |
*** amoralej is now known as amoralej|off | 16:46 | |
rcastillo|rover | wallaby common should promote | 16:50 |
dviroel | rlandy, rcastillo|rover: hey, we need manilaclient promotion for wallaby. It is ok to kick a clients component line now, instead of waiting until tomorrow 3:00 UTC? | 16:52 |
dviroel | or maybe testproject wallaby ones | 16:52 |
rcastillo|rover | better to tp the wallaby ones I think | 16:52 |
dviroel | yeah, the rest can wait until tomorrow | 16:53 |
rlandy | dviroel: ack - sure | 16:58 |
rcastillo|rover | wallaby tempest component seems to have passed, just needs to report | 16:58 |
rlandy | dviroel: juts don;t kick the whole line | 16:59 |
* rlandy looks | 16:59 | |
dviroel | rcastillo|rover: rdoinfo patch just merged, we have a new release 2.6.4 - so need to promote from consistent to component-ci-testing | 16:59 |
dviroel | rlandy: rcastillo|rover: running wallaby only | 16:59 |
dviroel | https://review.rdoproject.org/zuul/status#36356 | 16:59 |
rlandy | tempest should also promote | 17:01 |
rcastillo|rover | coffee brb | 17:05 |
rlandy | compute promoted | 17:05 |
rlandy | ugh - scenario001 failure on 17 | 17:08 |
rcastillo|rover | wallaby network job passed as well | 17:54 |
rlandy | rcastillo|rover: ack - happy dance | 17:59 |
rlandy | going to run the promo job there | 17:59 |
rlandy | rcastillo|rover: kicking the rhel-9 tripleo component | 18:21 |
rlandy | rcastillo|rover: rekciked common and tempest | 18:41 |
rcastillo|rover | huh, why? I'm lost there | 18:41 |
rcastillo|rover | oh wait, you mean downstream? | 18:42 |
rlandy | tempest is running now | 18:43 |
rlandy | just common | 18:43 |
rlandy | rev'ed since we promoted | 18:43 |
rlandy | downstream - only tripleo | 18:43 |
rcastillo|rover | ah, I see now | 18:44 |
dviroel | hum, fs001-clients-wallaby hitting mirror issues (2nd time already) | 19:52 |
dviroel | Error: Failed to download metadata for repo 'baseos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried | 19:52 |
rlandy | dviroel; ack ... rcastillo|rover will rerun | 20:08 |
rcastillo|rover | dviroel: not sure if it's just clients or all image builds, we'll know when tripleo runs | 20:08 |
rlandy | baremetal will also need a rerun | 20:08 |
rlandy | fs001 | 20:08 |
rlandy | rekicked and common sc 03 | 20:10 |
dviroel | ack, i have a testproject for fs001-clients ready to recheck - just waiting a little bit | 20:10 |
rcastillo|rover | rlandy: build image info showing up on tripleo component as well | 20:27 |
dviroel | rcastillo|rover: mirrors too? | 20:28 |
rcastillo|rover | dviroel: yes, baseos | 20:28 |
dviroel | yeah, same repo too | 20:28 |
dviroel | carloss: ^ seems to be happening more often | 20:29 |
carloss | oh | 20:30 |
dviroel | rcastillo|rover: Downloading successful, but checksum doesn't match. Calculated: d4e650ef6b39755ab61cc4410cd2d445ae215bb42a8d73737deb6fbcfcfcd9e1120d51a0a2d6ad520b9526aa8dff04b71ba395a6f0ee3d7a759182ad1c2bd719(sha512) Expected: 839d326a640e52e1e2ccc47dfcf8ea15ceb1254afba21e7aac706fff4def27c6936eb477f16fb3065b3433abfb87ff20b26b39ebec76b85a434588a85e240ac4(sha512) | 20:31 |
dviroel | rcastillo|rover: maybe the repos is on sync process | 20:31 |
dviroel | checksum not matching | 20:31 |
rcastillo|rover | http://mirror.regionone.vexxhost-nodepool-tripleo.rdoproject.org/centos-stream/9-stream/BaseOS/x86_64/os/repodata/ | 20:38 |
rcastillo|rover | hasn't updated since the 27th, unless I'm looking at the wrong mirror | 20:39 |
dviroel | yeah, there is no new packages too | 20:47 |
rlandy | ugh - tripleo rhel-9 failed node provision | 21:01 |
rlandy | rerunning | 21:01 |
rlandy | tempest will promote | 21:04 |
dviroel | yey | 21:04 |
rlandy | rcastillo|rover: does the dashboard work for you? | 21:05 |
rlandy | dashboard-ci.tripleo.org failing to connect | 21:06 |
rlandy | dviroel: ^^? | 21:06 |
rlandy | not reachable | 21:06 |
dviroel | same here | 21:06 |
rcastillo|rover | same here | 21:06 |
rcastillo|rover | review bot went offline as well | 21:07 |
rcastillo|rover | same node? | 21:07 |
rcastillo|rover | https://rdo.vexxhost.ca/ down for anyone else? | 21:12 |
rlandy | ugh | 21:14 |
rlandy | perfect timing :) | 21:14 |
rlandy | looks like nodes are starting to regenerate | 21:15 |
dviroel | :( | 21:19 |
rcastillo|rover | oh boy | 21:19 |
rlandy | we just have to wit it out | 21:23 |
rlandy | wait | 21:23 |
rlandy | or flat promote | 21:24 |
rcastillo|rover | rlandy: ack let's see if it comes back quickly | 21:25 |
rlandy | rcastillo|rover: should - rechecked my one job | 21:28 |
rlandy | anything you see missing? | 21:28 |
rlandy | dashboard is back | 21:28 |
rcastillo|rover | rlandy: nothing yet, just the node_failures | 21:28 |
rlandy | we have 1.5 hours | 21:33 |
rlandy | we may need to just skip promote in a few | 21:35 |
rcastillo|rover | yeah especially if we're still getting this build image issue | 21:40 |
dviroel | featureset001-clients-wallaby got a node | 21:40 |
rlandy | stack create failed | 21:43 |
rlandy | rcastillo|rover: if we can't get these jobs to pass OVB, I'm just skip promoting them | 21:49 |
rlandy | we | 21:49 |
rlandy | we will catch the issues on the donwstream side | 21:50 |
rcastillo|rover | rlandy: ack | 21:50 |
rlandy | prep'ing the skip patch | 21:52 |
dviroel | clients component seems to be safe to skip fs001, the difference between last promoted content and ci-testing is only python-manilaclient rpm. And we don't have manila on ovb jobs | 21:52 |
dviroel | rlandy: ^ | 21:52 |
rlandy | dviroel: ack - thanks ... common is missing scen 003 | 21:53 |
rlandy | that should pass | 21:53 |
rlandy | so it's baremetal and tripleo | 21:53 |
* dviroel biab + dinner | 21:58 | |
*** dviroel is now known as dviroel|afk | 21:59 | |
* dasm leaves. ttyt | 22:02 | |
*** dasm is now known as dasm|off | 22:02 | |
rcastillo|rover | back in 15 | 22:03 |
rlandy | rcastillo|rover: dviroel|afk: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43888 DNM: Temp remove criteria to promote wallaby | 22:23 |
rlandy | we have half an hour | 22:24 |
rcastillo|rover | rlandy: LGTM, do we just need to run the promote jobs with depends-on? | 22:29 |
rlandy | in progress now | 22:29 |
rlandy | rcastillo|rover: I didn;t force promote network | 22:29 |
rlandy | it promotde 5 hours ago | 22:29 |
rlandy | and there is a failure in dlrn | 22:29 |
rlandy | can try that one later if need be | 22:30 |
rcastillo|rover | rlandy: ack, these four should be fine, they were passing | 22:30 |
rlandy | baremetal, clients, tripleo, common | 22:30 |
rlandy | let's try for those | 22:30 |
rlandy | damage is done | 22:31 |
rcastillo|rover | yup | 22:33 |
rcastillo|rover | everything's still going through downstream still right? so it ain't all that bad | 22:33 |
rlandy | rcastillo|rover: yeah - we'll catch it there if need be | 22:35 |
rlandy | but we have to promote rhel-9 | 22:35 |
rlandy | waiting for tripleo component to promote | 22:36 |
rlandy | so we can rekick that line | 22:36 |
rcastillo|rover | rlandy: anything need watching? heading out otherwise | 23:23 |
rlandy | rcastillo|rover: think it's ok - updating rr notes | 23:23 |
rlandy | just waiting for rhel-9 tripleo component to complete | 23:23 |
rlandy | so I can rekick integration line | 23:23 |
*** dviroel|afk is now known as dviroel | 23:27 | |
dviroel | rlandy: anything that needs attention? | 23:46 |
dviroel | rlandy: thanks for promoting the remaining components | 23:47 |
rlandy | dviroel: think we are ok | 23:47 |
rlandy | just rekicking rhel-9 line now | 23:47 |
rlandy | tripleo component promoted | 23:47 |
rlandy | all wallaby promoted | 23:47 |
dviroel | great | 23:48 |
rlandy | dviroel: should be fine | 23:48 |
rlandy | ovbs are in rerun now | 23:48 |
rlandy | I skip promoted them | 23:48 |
rlandy | to get in under the wire | 23:48 |
dviroel | rlandy: ok, have a good night, see you tomorrow o/ | 23:49 |
rlandy | good night | 23:49 |
*** dviroel is now known as dviroel|out | 23:50 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!