Wednesday, 2022-10-05

marios\o05:05
mariosrlandy|out: nice you got the train promotion too with https://code.engineering.redhat.com/gerrit/c/testproject/+/430167/5#message-af2609dfa2de88c4d8940a87ea957dad9a344987 \o/06:23
*** marios is now known as marios|ruck06:31
jm1good morning folks :)06:48
*** amoralej|off is now known as amoralej07:27
*** jpena|off is now known as jpena07:37
marios|ruckarxcruz: o/ are you still working on adding 'zed' into the tempest skiplist?08:59
arxcruzmarios|ruck yes give me 20 min and i'll have a patch08:59
arxcruzjust finishing to write the documentation and unit tests 08:59
arxcruzmarios|ruck it's working, i can add a patch now as wip you can use it to create the zed meanwhile 09:00
marios|ruckarxcruz: ack thank you - mentioned this here https://review.rdoproject.org/r/c/rdo-jobs/+/45450/2#message-49fa04e62a4c5fd745ce6f3e9287891fed8a8d5e09:00
marios|ruckarxcruz: don't rush with the patch would be great if we get it later today anyway o/ thanks09:05
arxcruzreviewbot please add https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/860390 to review list 09:47
reviewbotI have added your review to the Review list09:47
arxcruzreviewbot please add https://review.opendev.org/c/openstack/tripleo-ci/+/859526 to review list 09:52
reviewbotI have added your review to the Review list09:52
marios|ruckarxcruz: please check when you have time - tidy up if we are going to merge like that? https://review.opendev.org/c/openstack/tripleo-ci/+/859830/4#message-b1ce2b41d75a87c68c7f9f704ba3e74b3fc16c1e ( we still have cix for it)10:14
arxcruzmarios|ruck 10:33
arxcruzmarios|ruck ack, any solution is fine, skipping or changing the protocol will lead to the same that is the test be skipped 10:34
*** amoralej is now known as amoralej|lunch11:03
arxcruzmarios|ruck what's the topic you want?11:34
marios|ruckarxcruz: https://review.rdoproject.org/r/q/topic:oooci_zed https://review.opendev.org/q/topic:oooci_zed11:36
arxcruzmarios|ruck https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/86040311:39
marios|ruckthanks arxcruz 11:48
*** jpodivin_ is now known as jpodivin12:12
*** amoralej|lunch is now known as amoralej12:31
amoralejmarios|ruck, component and integrated promotion pipelines for zed are finished?12:54
*** kopecmartin is now known as kopecmartin|sick12:56
marios|ruckamoralej: no just basic/bootstrap - mainly containers/images built with standalones and multinode 13:03
marios|ruckamoralej: will use this sprint to build it out fully ... should line up to when we get actual zed tripleo 13:03
amoralejack13:04
amoralejso, current promoted-components is manual promotion?13:04
marios|ruckamoralej: eg components here still https://review.rdoproject.org/r/c/rdo-jobs/+/45450 13:04
marios|ruckamoralej: topics https://review.rdoproject.org/r/q/topic:oooci_zed  https://review.opendev.org/q/topic:oooci_zed fyi 13:04
amoralejhttps://trunk.rdoproject.org/centos9-zed/promoted-components/13:04
marios|ruckamoralej: yeah but they aren't running anything yet just promotes everything through 13:07
marios|ruckamoralej: so this is the 'bootstrap' done the lines are up running/containers/dlrn/ceph all those things sorted and we build it this sprint13:07
amoralejack, thanks for the info!13:07
rcastillo|rucko/13:29
marios|ruck\o 13:31
*** marios|ruck is now known as marios13:31
marios\o/13:31
mariosfreedom 13:31
* marios runs away 13:31
rcastillo|ruckhahaha13:31
rcastillo|ruckthank you, marios13:31
mariosrcastillo|ruck: i'll be on cix can handle that so no worries... i updated the things in blocked at least latest status etc. promotions are good we got train. gates have been fine13:32
mariosrcastillo|ruck: otherwise i didn't do something was working on my own stuff so not much to handoff13:32
mariosrcastillo|ruck: if you like we can have a quick call otherwise just ping me for anything?13:32
mariosup to you 13:32
mariosno problem 13:32
rcastillo|ruckmarios sure let's sync real quick13:33
mariosrcastillo|ruck: sure please start meet gimme link13:33
rcastillo|ruckmarios: https://meet.google.com/hju-quct-kuo13:34
*** dasm|off is now known as dasm13:43
dasmo/13:43
marios\o13:46
jm1dasm: \o/13:58
dasmjm1: \o14:00
mariosrcastillo|ruck: off in few need something before i do?15:06
rcastillo|ruckmarios: I think I'm good thanks15:09
rcastillo|ruckhave a good evening15:09
mariosthanks have a good day americas o/ 15:17
*** marios is now known as marios|out15:17
*** jpena is now known as jpena|off16:38
*** amoralej is now known as amoralej|off16:39
dviroelrcastillo|ruck: hey, let me know if you need help with something17:31
rcastillo|ruckdviroel: sure, thanks. Everything's quiet for now looks like17:34
rcastillo|ruckjust looking at components17:34
dviroelrcastillo|ruck:++17:34
rcastillo|ruckoh no18:33
rcastillo|ruckgate blew up18:33
dasmrcastillo|ruck: do you need a help?18:50
rcastillo|ruckdasm: just waiting to see if it's a blip since the jobs all failed at different spots18:57
dasmack18:58
dasmlmk if you need a help, rcastillo|ruck 18:58
dasmi'll try keeping an eye on the chat18:58
rcastillo|ruckappreciate it18:58
rcastillo|ruckyeah there's definitely an issue20:29
rcastillo|ruckCommand '['sudo', 'podman', 'play', 'kube', '/home/zuul/overcloud-deploy/overcloud/heat-launcher/heat-pod.yaml']' returned non-zero exit status 12520:29
rcastillo|ruckdviroel dasm: you guys know where I can find the logs for that command or how to debug this in general?20:30
rcastillo|rucklooking for useful information for logging the bug20:30
rcastillo|ruckI'm seeing this in containers-multinode https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-containers-multinode&skip=020:31
dasmrcastillo|ruck: not from the top of my head. checking20:54
dasmrcastillo|ruck: seems like heat container might be broken: 2022-10-05 20:02:49 | Error: /home/zuul/overcloud-deploy/overcloud/heat-launcher/heat-pod.yaml: copying system image from manifest list: Source image rejected: parsing signature21:00
dasmhttps://access.redhat.com/webassets/docker/content/sigstore/ubi8/pause@sha256=d63b205526b91d8bc05deec98b222d4dcb61289952aab534903d80ade665232f/signature-5: unrecognized signature format, starting with binary 0x3c21:00
dasmhttps://b452c7de58df0e34dc61-55c346d8a0a53f06bfa3e84321f61ba5.ssl.cf5.rackcdn.com/860483/1/check/tripleo-ci-centos-9-containers-multinode/55344e8/logs/undercloud/home/zuul/overcloud_deploy.log21:00
dviroelrcastillo|ruck: no idea atm, please open a bug to raise a cix21:02
rcastillo|ruckdasm: ah, interesting21:03
dasmrcastillo|ruck: either image is broken, or signature incorrect.21:03
dasmbut it definitely is a bug :)21:03
dasmI'm not finding anything in https://github.com/containers/podman/blob/main/troubleshooting.md21:04
rcastillo|ruckhttps://bugs.launchpad.net/tripleo/+bug/199184621:11
dviroelrcastillo|ruck: maybe someone from #tripleo can help21:14
dasmacn21:14
dasmrcastillo|ruck: when did we promote heat component?21:14
dasmwait... if that's affecting both master and wallaby, it's probably something different then21:15
rcastillo|ruckyeah hitting both21:15
* dviroel need to go af21:17
*** dviroel is now known as dviroel|afk21:17
rcastillo|rucklatest ubi8 image got tagged 22 days ago so not that either21:19
dasmi don't know how signature file is being generated21:21
dasmactually, i don't know what *is* signature file :*21:21
dasm:(21:21
dasmsomething like that? https://developers.redhat.com/blog/2019/10/29/verifying-signatures-of-red-hat-container-images#mirroring_red_hat_container_images_to_a_private_registry21:22
rcastillo|ruckdon't know why it's trying to verify that signature if it's not using ubi8 at all according to the pod file21:27
rcastillo|ruckI'll try running with build_containers to see if that helps21:28
dasmk21:28
*** dasm is now known as dasm|off21:34
* dasm|off => offline21:34
dasm|offrcastillo|ruck: o/ good luck. Don't sit too long on that.21:34
rcastillo|ruckdasm|off: o/ see you tomorrow21:34
rcastillo|ruckrekicked master to rebuild images. let's see if that helps23:08
rcastillo|rucktp with build_images passed23:09

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!