| marios | anyone know if there is an issue with rdo gerrit? | 07:42 |
|---|---|---|
| marios | cant vote on rdo-jobs 'internal server error' | 07:42 |
| marios | o/ morning | 07:42 |
| Tengu | hello there! can we get some +W on https://review.opendev.org/c/openstack/tripleo-ci/+/825437 ? there are 3 +2 already... | 07:53 |
| *** amoralej|off is now known as amoralej | 08:05 | |
| *** anbanerj is now known as frenzyfriday | 08:35 | |
| *** jpena|off is now known as jpena | 08:38 | |
| amoralej | how may i only create the dockerfiles from tripleo-common tcib? | 09:42 |
| amoralej | is there any cli that i can use? | 09:42 |
| Tengu | amoralej: hmmm I don't think it's possible. but you want to ask DF - especially Kevin or Alex | 10:02 |
| Tengu | they are at the core of tcib | 10:02 |
| Tengu | they should be available a bit later today (they are both NA, Denver and Houston [more or less]) | 10:03 |
| amoralej | thanks Tengu, no problem, i can wait for ci results :) | 10:11 |
| marios | o/ folks i think most of india is out today right? | 10:35 |
| marios | arxcruz: dviroel|afk: jm1: dasm|off: rlandy|out: sshnaidm: please vote there when you have a minute https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/825508/5#message-f3aa8c53c818c458c57f14ea14a16883b0691878 | 10:35 |
| Tengu | hmm. fun. the rollback in DIB when building the hardened-uefi isn't cleaning everything.... | 10:41 |
| Tengu | i.e. the vg/pv are still present, leading to a second issue when we re-run on the same node, because "vg" volumegroup already exists :). | 10:41 |
| Tengu | guess it's a good candidate for a bug. | 10:42 |
| dpawlik | hello folks. I'm doing a cleanup on images.rdoproject.org because we are each hour out of space | 10:46 |
| dpawlik | I moved directories like: fedora28 fedora-stable ocata queens rocky stein to "old" directory | 10:46 |
| dpawlik | if all will be working on your side, I will remove the old directory after few days | 10:46 |
| dpawlik | also I got another question: some symlink images like CentOS-8-Stream-x86_64-GenericCloud.qcow2 or CentOS-8-x86_64-GenericCloud.qcow2 or CentOS-9-Stream-x86_64-GenericCloud.qcow2 are connected to images that are old | 10:47 |
| dpawlik | should I download new images and change symlink? | 10:48 |
| marios | dpawlik: o/ fine for 'old' directory don't change the symlinks... let me look | 10:52 |
| marios | urgh looks like we have multiple references eg https://codesearch.opendev.org/?q=CentOS-8-GenericCloud-8.3.2011-20201204.2.x86_64.qcow2&i=nope&literal=nope&files=&excludeFiles=&repos= vs https://codesearch.opendev.org/?q=CentOS-Stream-GenericCloud-8-20201217.0.x86_64.qcow2&i=nope&literal=nope&files=&excludeFiles=&repos= | 10:53 |
| marios | dpawlik: i think remove that one too http://images.rdoproject.org/ooo-snap/ | 10:54 |
| marios | looks old. newton :D timestamp 2017 | 10:54 |
| dpawlik | I will move to old | 10:54 |
| dpawlik | CentOS-7-x86_64-GenericCloud.qcow2 can be also updated to new version :P | 10:55 |
| dpawlik | timestamp from 2017 | 10:55 |
| marios | sure but i thought main goal was delete stuff :) | 10:56 |
| dpawlik | xD | 10:57 |
| marios | dpawlik: fix all the things :) | 10:58 |
| dpawlik | I can do an image update + cleanup :D | 10:58 |
| dpawlik | marios: centos7 updated | 11:05 |
| marios | thanks dpawlik | 11:10 |
| *** rlandy|out is now known as rlandy|ruck | 11:12 | |
| rlandy|ruck | slaweq: hello ... could you review https://bugs.launchpad.net/tripleo/+bug/1958955 | 11:20 |
| rlandy|ruck | blocking all c9 ovb | 11:20 |
| slaweq | rlandy|ruck: sure | 11:21 |
| *** dviroel|afk is now known as dviroel | 11:21 | |
| slaweq | rlandy|ruck: I think I know what the issue is, give me 15 minutes and I will propose patch, should be easy hopefully :) | 11:26 |
| dviroel | \o/ | 11:29 |
| marios | sshnaidm: can we please keep the ussuri jobs | 11:33 |
| marios | and then we'll merge the rest of the series at https://review.rdoproject.org/r/c/config/+/38554/3 | 11:33 |
| marios | sshnaidm: talking with rlandy now if you want to join | 11:33 |
| marios | https://meet.google.com/vzm-nrah-qqf | 11:33 |
| dviroel | oh, i will join | 11:33 |
| rlandy|ruck | slaweq++ thank you | 11:44 |
| rlandy|ruck | arxcruz: you around? | 11:44 |
| rlandy|ruck | arxcruz: can you join https://meet.google.com/vzm-nrah-qqf?authuser=0 | 11:44 |
| rlandy|ruck | sshnaidm: hi - where are our keys added t internal nodes | 11:49 |
| rlandy|ruck | a bunch of keys are missing | 11:49 |
| rlandy|ruck | the rdo ones are updated | 11:49 |
| sshnaidm | rlandy|ruck, which nodes? | 11:49 |
| rlandy|ruck | sshnaidm: internal | 11:49 |
| rlandy|ruck | my keys are there | 11:49 |
| arxcruz | https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/copy_container/tasks/cron.yml | 11:50 |
| arxcruz | marios rlandy|ruck dviroel | 11:50 |
| sshnaidm | rlandy|ruck, internal zuul you mean? | 11:50 |
| rlandy|ruck | sshnaidm: ack | 11:50 |
| sshnaidm | rlandy|ruck, idk, I don't think configured it there.. | 11:50 |
| rlandy|ruck | sshnaidm: oh - maybe infra did | 11:53 |
| slaweq | rlandy|ruck: patch proposed https://review.opendev.org/c/openstack/neutron/+/826449 | 11:58 |
| slaweq | and test for it in https://review.rdoproject.org/r/c/testproject/+/38582 | 11:59 |
| slaweq | lets see what Zuul will tell about it :) | 11:59 |
| rlandy|ruck | slaweq: that was quick :) ... will watch that | 12:01 |
| rlandy|ruck | https://review.rdoproject.org/r/c/testproject/+/38582/1/.zuul.yaml great - marios we'll have a check there | 12:02 |
| slaweq | rlandy|ruck: because it was easy fix :) but lets see if that was the only issue in those jobs | 12:03 |
| slaweq | as I didn't deploy tripleo to test it locally | 12:03 |
| rlandy|ruck | ok - we'll see | 12:14 |
| frenzyfriday | hey folks, I have an appointment today, will leave for the day in ~1hr | 12:16 |
| rlandy|ruck | frenzyfriday: sshnaidm: here ... https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/tripleo-ci-internal-config.git;a=blob;f=playbooks/tripleo-rdo-base/pre-keys.yaml;h=58c71eea9dafd5e13a318cbe1954f0bf8e7294fb;hb=HEAD | 12:50 |
| rlandy|ruck | frenzyfriday: can you add everyone's keys here | 12:50 |
| sshnaidm | rlandy|ruck, I see | 12:52 |
| frenzyfriday | rlandy|ruck, ack, adding | 12:52 |
| marios | Tengu: i think your background needs something a bit more 'red hat' not sure.. | 13:03 |
| marios | :D | 13:03 |
| Tengu | marios: I can probably try to paint my cat in red and black to match the color :D | 13:04 |
| marios | haha also a little cat size red hat for her | 13:04 |
| Tengu | :) | 13:04 |
| frenzyfriday | rlandy|ruck, pl review when you get time: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/306656 | 13:05 |
| frenzyfriday | ^ ssh keys | 13:05 |
| rlandy|ruck | frenzyfriday: thank you | 13:07 |
| rlandy|ruck | will merge this when done with work on https | 13:07 |
| sshnaidm | frenzyfriday, can you please add people mails in comments so we know exactly whose these keys of? | 13:09 |
| *** amoralej is now known as amoralej|lunch | 13:09 | |
| rlandy|ruck | rcastillo|rover: can you look into c9 network component | 13:14 |
| rlandy|ruck | all red | 13:14 |
| frenzyfriday | dviroel, sshnaidm, ack, updated | 13:15 |
| dviroel | frenzyfriday: tks | 13:16 |
| * frenzyfriday appointment | afk | 13:18 | |
| rcastillo|rover | o/ | 13:24 |
| rcastillo|rover | rlandy|ruck: looking | 13:24 |
| rlandy|ruck | rcastillo|rover: hey - pls also see https://bugs.launchpad.net/tripleo/+bug/1959100 | 13:26 |
| rlandy|ruck | not strictly our job | 13:26 |
| rlandy|ruck | but good to know how | 13:26 |
| rcastillo|rover | just saw the discussion in #tripleo | 13:26 |
| rlandy|ruck | thanks | 13:26 |
| * dviroel coffee | 13:38 | |
| sshnaidm | do we have a problem with c9 on ovb like undercloud failing? | 13:42 |
| sshnaidm | "Not found image: https://trunk.registry.rdoproject.org/v2/tripleomastercentos9/openstack-heat-api/manifests/6e7f2e3581c311900e20c27e2e530695" | 13:42 |
| rcastillo|rover | rlandy|ruck: network component jobs are also failing to find images in the rdoproject registry | 13:47 |
| *** dasm|off is now known as dasm | 13:48 | |
| rlandy|ruck | sshnaidm: c9 ovb has node provision issue | 13:48 |
| dasm | o/ | 13:48 |
| rlandy|ruck | rcastillo|rover: one sec - working on downstream | 13:48 |
| rcastillo|rover | no worries | 13:49 |
| rlandy|ruck | is current-tripleo hash still there? | 13:49 |
| rlandy|ruck | we may need to promote | 13:49 |
| sshnaidm | rlandy|ruck, it's containers problem, not provisioning, seems like rcastillo|rover case.. | 13:49 |
| rlandy|ruck | we have multiple issues going down | 13:49 |
| sshnaidm | rcastillo|rover, what is current-tripleo in c9 now? | 13:49 |
| rlandy|ruck | but yet - it's old | 13:50 |
| rcastillo|rover | current-tripleo is from the twentieth | 13:57 |
| rcastillo|rover | don't know how to interact with the repository to check if it's there | 13:57 |
| *** amoralej|lunch is now known as amoralej | 13:59 | |
| dviroel | slaweq: rcastillo|rover: neutron fix seems to fix that lot of errors that we saw yesterday - but ovb is still failing in node provision | 14:47 |
| dviroel | rlandy|ruck: ^ | 14:47 |
| dviroel | slaweq: rlandy|ruck: https://logserver.rdoproject.org/82/38582/1/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master/82c14e0/logs/undercloud/var/log/extra/errors.txt.gz | 14:47 |
| rlandy|ruck | yep - one pass one fail | 14:47 |
| dviroel | slaweq: rlandy|ruck: nodes that are failing are still having hard times to get an IP - iiuc | 14:48 |
| slaweq | dviroel: rlandy|ruck that error with Failed to bind port are because of the cloud config I guess, see: | 14:49 |
| slaweq | https://paste.opendev.org/show/812372/ | 14:49 |
| slaweq | that's neutron server log about it | 14:49 |
| slaweq | it has enabled 2 mech_drivers: "openvswitch" and "baremetal" | 14:50 |
| slaweq | openvswitch can't bound port as there is no neutron-ovs-agent on the node | 14:50 |
| slaweq | and "baremetal" can't bound port with vnic_type=normal | 14:50 |
| rlandy|ruck | adding to bug info | 14:55 |
| slaweq | rlandy|ruck: I will check it more deeply | 14:59 |
| dviroel | slaweq: ack, i see these errors in centos-8 jobs too, but they are not failing as in centos-9 jobs (not sure why) | 15:00 |
| dviroel | here: https://logserver.rdoproject.org/openstack-periodic-integration-main-centos8/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master/68b2100/logs/undercloud/var/log/extra/errors.txt.gz | 15:00 |
| slaweq | dviroel: ok, let me check that | 15:00 |
| dviroel | slaweq: thanks | 15:00 |
| * dviroel lunch | 15:00 | |
| *** dviroel is now known as dviroel|lunch | 15:01 | |
| slaweq | dviroel: I think that those errors may be just a red herring | 15:05 |
| slaweq | I noticed that this jobs passed yesterday on other testpatch https://review.rdoproject.org/r/c/testproject/+/36140/34 | 15:05 |
| slaweq | I will update my patch with those depends on to see if that will maybe help | 15:05 |
| rlandy|ruck | rcastillo|rover: hey - let's meet to go through cix cards | 15:16 |
| rcastillo|rover | rlandy|ruck: ok | 15:16 |
| rlandy|ruck | rcastillo|rover: hey https://meet.google.com/rdc-mwpe-bwf?pli=1&authuser=0 | 15:17 |
| rcastillo|rover | rlandy|ruck: https://review.opendev.org/c/openstack/tripleo-ci/+/826124 | 15:23 |
| rlandy|ruck | marios: all revert of skiplist are merged, right? | 15:25 |
| rlandy|ruck | https://bugs.launchpad.net/tripleo/+bug/1957031 | 15:25 |
| rlandy|ruck | OVB | 15:25 |
| marios | rlandy|ruck: checking | 15:25 |
| rlandy|ruck | k - np | 15:25 |
| marios | rlandy|ruck: the one tht was ready merged looks like let me check the testproject see what hapened for the other outstanding patch | 15:26 |
| marios | rlandy|ruck: fs35 node provisioning in testproject https://logserver.rdoproject.org/57/38357/1/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/9f6b596/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz | 15:27 |
| marios | rlandy|ruck: so we still have outstanding https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/825506/3 - gonna update that one and rerun the components | 15:37 |
| rlandy|ruck | marios: k - np | 15:38 |
| rlandy|ruck | rcastillo|rover: hey - want to meet again noww? | 16:01 |
| rcastillo|rover | give me 2 min | 16:01 |
| rlandy|ruck | sure | 16:01 |
| rlandy|ruck | when you are ready | 16:01 |
| rlandy|ruck | https://meet.google.com/xte-umii-fua?pli=1&authuser=0 | 16:02 |
| marios | rlandy|ruck: add to your list https://review.rdoproject.org/r/c/config/+/38601 needed for https://review.rdoproject.org/r/c/rdo-jobs/+/38602 | 16:05 |
| rlandy|ruck | marios: hey - we need to build/push an older version of c9 containers | 16:12 |
| rlandy|ruck | as I think they got pruned | 16:12 |
| rlandy|ruck | do you remember the job config for that? | 16:12 |
| rlandy|ruck | is it juts a testproject run with dlrn_hash_tag? | 16:13 |
| *** dviroel|lunch is now known as dviroel | 16:23 | |
| rlandy|ruck | dasm; hello | 16:31 |
| rlandy|ruck | aren't you on PTO? | 16:31 |
| rlandy|ruck | slaweq: dviroel: just reading back on node provision issue ... | 16:36 |
| rlandy|ruck | watching currently running testproject | 16:37 |
| rlandy|ruck | we can decide tomorrow if it's worth promoting c9 with this failure or not | 16:37 |
| dviroel | ack | 16:39 |
| sshnaidm | rlandy|ruck, jm1 started hackmd: https://hackmd.io/szgyWa5qSUOWw3JJBXLmOQ | 16:53 |
| rlandy|ruck | sshnaidm++ great - will add that to card for planning | 16:54 |
| rlandy|ruck | and sprint board | 16:54 |
| dasm | rlandy|ruck: i was on a pto yesterday | 17:02 |
| rlandy|ruck | np | 17:02 |
| dviroel | @team - what about give some help to jungleboyj's choice for Z cycle naming: https://twitter.com/jungleboyj/status/1486344910693441541 | 17:12 |
| dviroel | please - vote for Zorilla =) | 17:13 |
| rlandy|ruck | dviroel: I guess zulu was a no show | 17:14 |
| dviroel | yeah | 17:15 |
| rlandy|ruck | the south african in me cies | 17:15 |
| *** jpena is now known as jpena|off | 17:44 | |
| *** sshnaidm is now known as sshnaidm|afk | 17:49 | |
| *** amoralej is now known as amoralej|off | 17:53 | |
| rlandy|ruck | rcastillo|rover: hmmm ... https://logserver.rdoproject.org/91/38491/3/check/periodic-tripleo-ci-build-containers-centos-9-push-master/994cbb6/logs/containers-built.log | 18:18 |
| rlandy|ruck | didn't work | 18:18 |
| rlandy|ruck | need to define hash elsewhere | 18:18 |
| rlandy|ruck | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/build-containers.yaml | 18:19 |
| rlandy|ruck | try current-tripleo | 18:20 |
| rlandy|ruck | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/integration-jobs-cs9-master.yaml#L15 | 18:20 |
| rlandy|ruck | correct | 18:20 |
| rcastillo|rover | rlandy|ruck: so set promote_source to current-tripleo? | 18:25 |
| rlandy|ruck | rcastillo|rover: https://review.rdoproject.org/r/c/rdo-jobs/+/36327/1/zuul.d/layout.yaml | 18:29 |
| rlandy|ruck | but the c9 version | 18:29 |
| rlandy|ruck | I think those vars work | 18:30 |
| rlandy|ruck | rcastillo|rover: or ask arxcruz to copy the containers back to rdo registry | 18:31 |
| rlandy|ruck | rcastillo|rover: sorry - almost done with downstream | 18:46 |
| rlandy|ruck | will look back here in a bit | 18:46 |
| dviroel | did the container-push worked for current-tripleo? | 20:23 |
| dviroel | no | 20:29 |
| rlandy|ruck | k - back with upstream now | 20:32 |
| rlandy|ruck | rcastillo|rover: hey - you around? | 20:33 |
| rlandy|ruck | dviroel: will look at all that now | 20:33 |
| rlandy|ruck | sorry - was fixing downstream | 20:33 |
| rcastillo|rover | hi, yeah, looking into molecule issues | 20:33 |
| rlandy|ruck | rcastillo|rover: no problem - carry on | 20:33 |
| sshnaidm|afk | seems like zombie is more accurate.. | 20:57 |
| dviroel | rlandy|ruck: https://opendev.org/openstack/tripleo-ci/src/branch/master/playbooks/tripleo-buildcontainers/run.yaml#L23 | 21:02 |
| dviroel | rlandy|ruck: build-containers for C9 is differente since we started using release files instead of tripleo-repos | 21:03 |
| rlandy|ruck | running now | 21:03 |
| rlandy|ruck | now working? | 21:03 |
| dviroel | oh, lets see | 21:04 |
| *** dviroel is now known as dviroel|afk | 21:06 | |
| rlandy|ruck | dviroel|afk: no show | 21:15 |
| rcastillo|rover | let's see how this goes... https://review.opendev.org/c/openstack/tripleo-ansible/+/826535 | 21:44 |
| rlandy|ruck | cool | 21:57 |
| rcastillo|rover | ok so changing the distro to cs might have been a bit misguided | 22:48 |
| rcastillo|rover | the repo in the var file doesn't ever get used afaik | 22:49 |
| rcastillo|rover | the role grabs the current centos repo: https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/test_deps/tasks/main.yml#L47 | 22:50 |
| rcastillo|rover | So I don't see why ubi would be an issue, or why tripleo-repos is failing for that matter | 22:51 |
| *** dasm is now known as dasm|off | 23:24 | |
| dasm|off | \o | 23:24 |
| *** rlandy|ruck is now known as rlandy|out | 23:41 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!