*** ysandeep|out is now known as ysandeep | 02:04 | |
*** ysandeep is now known as ysandeep|afk | 05:15 | |
*** ysandeep|afk is now known as ysandeep | 05:56 | |
rdogerrit | Merged rdoinfo master: Promote CBS tags update for xena-8s-testing https://review.rdoproject.org/r/c/rdoinfo/+/43740 | 06:45 |
---|---|---|
rdogerrit | Merged rdoinfo master: Bump rdoinfo wallaby tags to latest upper-constraints https://review.rdoproject.org/r/c/rdoinfo/+/43736 | 06:47 |
rdogerrit | Merged rdoinfo master: Bump rdoinfo victoria tags to latest upper-constraints https://review.rdoproject.org/r/c/rdoinfo/+/43735 | 06:48 |
rdogerrit | Merged rdoinfo master: Bump rdoinfo ussuri tags to latest upper-constraints https://review.rdoproject.org/r/c/rdoinfo/+/43734 | 06:48 |
rdogerrit | rdo-trunk created openstack/gnocchiclient-distgit wallaby-rdo: python-gnocchiclient-7.0.6-1 https://review.rdoproject.org/r/c/openstack/gnocchiclient-distgit/+/43745 | 06:49 |
rdogerrit | Merged rdoinfo master: Promote CBS tags update for xena-8s-release https://review.rdoproject.org/r/c/rdoinfo/+/43743 | 06:49 |
rdogerrit | rdo-trunk created openstack/gnocchiclient-distgit ussuri-rdo: python-gnocchiclient-7.0.6-1 https://review.rdoproject.org/r/c/openstack/gnocchiclient-distgit/+/43746 | 06:50 |
rdogerrit | rdo-trunk created openstack/gnocchiclient-distgit victoria-rdo: python-gnocchiclient-7.0.6-1 https://review.rdoproject.org/r/c/openstack/gnocchiclient-distgit/+/43747 | 06:50 |
rdogerrit | Merged rdoinfo master: Promote CBS tags update for wallaby-8s-release https://review.rdoproject.org/r/c/rdoinfo/+/43744 | 06:50 |
rdogerrit | Merged nfvinfo master: Promote CBS tags update for nfvsig-2.16-testing https://review.rdoproject.org/r/c/nfvinfo/+/43733 | 06:53 |
rdogerrit | Merged rdoinfo master: Bump rdoinfo xena tags to latest upper-constraints https://review.rdoproject.org/r/c/rdoinfo/+/43737 | 06:54 |
rdogerrit | rdo-trunk created openstack/gnocchiclient-distgit xena-rdo: python-gnocchiclient-7.0.6-1 https://review.rdoproject.org/r/c/openstack/gnocchiclient-distgit/+/43748 | 06:56 |
rdogerrit | rdo-trunk created openstack/oslo-messaging-distgit xena-rdo: python-oslo-messaging-12.9.4-1 https://review.rdoproject.org/r/c/openstack/oslo-messaging-distgit/+/43749 | 06:56 |
*** amoralej|off is now known as amoralej | 07:05 | |
*** frenzyfriday|PTO is now known as frenzy_friday | 07:13 | |
*** jpena|off is now known as jpena | 07:43 | |
rdogerrit | Matthias Runge created openstack/gnocchiclient-distgit wallaby-rdo: python-gnocchiclient-7.0.7-1 https://review.rdoproject.org/r/c/openstack/gnocchiclient-distgit/+/43676 | 07:43 |
rdogerrit | Joel Capitao proposed openstack/cinderlib-distgit rpm-master: Fix ambiguous shebang https://review.rdoproject.org/r/c/openstack/cinderlib-distgit/+/43730 | 07:50 |
rdogerrit | User bshewale proposed rdo-jobs master: Reparenting sc010 jobs using -options job https://review.rdoproject.org/r/c/rdo-jobs/+/43146 | 08:03 |
rdogerrit | Merged openstack/gnocchiclient-distgit wallaby-rdo: python-gnocchiclient-7.0.7-1 https://review.rdoproject.org/r/c/openstack/gnocchiclient-distgit/+/43676 | 08:13 |
rdogerrit | Merged openstack/octavia-distgit rpm-master: Add octavia-driver-agent package https://review.rdoproject.org/r/c/openstack/octavia-distgit/+/32779 | 08:37 |
rdogerrit | Merged openstack/oslo-messaging-distgit xena-rdo: python-oslo-messaging-12.9.4-1 https://review.rdoproject.org/r/c/openstack/oslo-messaging-distgit/+/43749 | 08:45 |
rdogerrit | Takashi Kajinami created openstack/octavia-distgit yoga-rdo: Add octavia-driver-agent package https://review.rdoproject.org/r/c/openstack/octavia-distgit/+/43750 | 08:47 |
rdogerrit | rdo-trunk created openstack/cinderlib-distgit yoga-rdo: python-cinderlib-4.2.0-1 https://review.rdoproject.org/r/c/openstack/cinderlib-distgit/+/43751 | 09:01 |
*** vkmc_ is now known as vkmc | 09:10 | |
rdogerrit | Jakob Meng proposed rdo-infra/ci-config master: Added new ShellCheck job https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/42701 | 09:19 |
rdogerrit | User pojadhav proposed rdo-jobs master: Remove train c7 jobs https://review.rdoproject.org/r/c/rdo-jobs/+/43172 | 09:22 |
rdogerrit | User pojadhav proposed rdo-jobs master: Remove train c7 jobs https://review.rdoproject.org/r/c/rdo-jobs/+/43172 | 09:26 |
rdogerrit | User pojadhav proposed rdo-jobs master: Remove train c7 jobs https://review.rdoproject.org/r/c/rdo-jobs/+/43172 | 09:28 |
*** rlandy|out is now known as rlandy | 09:29 | |
rdogerrit | User pojadhav proposed config master: Remove train c7 jobs https://review.rdoproject.org/r/c/config/+/43585 | 09:29 |
*** ysandeep is now known as ysandeep|afk | 09:50 | |
amoralej | ykarel, https://review.rdoproject.org/r/c/rdoinfo/+/43728 | 10:03 |
amoralej | what would be needed to test the weird issue? | 10:03 |
rdogerrit | Merged rdoinfo master: Update puppet and facter to last versions for CS9 in Zed https://review.rdoproject.org/r/c/rdoinfo/+/42273 | 10:05 |
amoralej | rlandy, ^ also for your, i've started testing update to 2.17 | 10:06 |
jcapitao[m] | amoralej: https://review.rdoproject.org/r/q/topic:remove-mox3 when you have a chance | 10:14 |
rdogerrit | Jakob Meng proposed rdo-infra/ci-config master: Added new ShellCheck job https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/42701 | 10:18 |
rdogerrit | Merged openstack/oslo-messaging-distgit rpm-master: Remove mox3 https://review.rdoproject.org/r/c/openstack/oslo-messaging-distgit/+/43697 | 10:23 |
rdogerrit | Merged openstack/ironic-ui-distgit rpm-master: Remove mox3 https://review.rdoproject.org/r/c/openstack/ironic-ui-distgit/+/43693 | 10:23 |
rdogerrit | Merged openstack/placement-distgit rpm-master: Remove mox3 https://review.rdoproject.org/r/c/openstack/placement-distgit/+/43699 | 10:27 |
rdogerrit | Merged openstack/horizon-distgit rpm-master: Remove mox3 https://review.rdoproject.org/r/c/openstack/horizon-distgit/+/43694 | 10:28 |
rdogerrit | Merged openstack/heat-distgit rpm-master: Remove mox3 https://review.rdoproject.org/r/c/openstack/heat-distgit/+/43672 | 10:29 |
*** ysandeep|afk is now known as ysandeep | 10:38 | |
rdogerrit | Sandeep Yadav proposed rdo-jobs master: Remove train c7 jobs https://review.rdoproject.org/r/c/rdo-jobs/+/43172 | 10:41 |
rdogerrit | Sandeep Yadav proposed config master: Remove train c7 jobs https://review.rdoproject.org/r/c/config/+/43585 | 10:41 |
rdogerrit | rdo-trunk created openstack/cinderlib-distgit xena-rdo: python-cinderlib: failed to build 4fc56c8 https://review.rdoproject.org/r/c/openstack/cinderlib-distgit/+/43752 | 10:46 |
rdogerrit | rdo-trunk created openstack/cinderlib-distgit xena-rdo: python-cinderlib: failed to build 4fc56c8 https://review.rdoproject.org/r/c/openstack/cinderlib-distgit/+/43753 | 10:47 |
rdogerrit | Chandan Kumar created rdo-jobs master: Rename centos-9-quay job to ubi-9-quay job https://review.rdoproject.org/r/c/rdo-jobs/+/43754 | 10:54 |
ykarel | amoralej, that day i also tested ovn22.03 and that's also impacted with cpu issue | 11:06 |
ykarel | so we need the fix to be included before updating | 11:06 |
amoralej | ykarel, did you get any feedback from ovn team? | 11:07 |
ykarel | amoralej, yes they have pushed a patch for review in ovs master | 11:08 |
amoralej | good | 11:08 |
ykarel | i did some test too with that, and looked good to me | 11:08 |
amoralej | apevec, ^ fyi | 11:10 |
amoralej | ykarel, you have the link to the upstream patch somewhere? | 11:17 |
ykarel | amoralej, https://patchwork.ozlabs.org/project/openvswitch/patch/20220624072959.240183-1-david.marchand@redhat.com/ | 11:17 |
amoralej | jcapitao[m], it seems we have old tackerclient in fedora | 11:38 |
amoralej | from wallaby, didn't we include it in the update to yoga? | 11:38 |
jcapitao[m] | hmm | 11:38 |
jcapitao[m] | I see it's listed in result of https://github.com/rdo-infra/releng/blob/master/scripts/fedora_sync/list_openstack_packages.sh script | 11:44 |
kkula | jcapitao, we have 1.6 version and now missing patch to fix from 1.8 | 11:44 |
rdogerrit | Merged rdo-infra/ci-config master: Enable usage of a user-provided CA bundle for downstream access https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43714 | 11:48 |
jcapitao[m] | kkula: I'm rerunning https://github.com/rdo-infra/releng/blob/master/scripts/fedora_sync/sync_fedora_from_rdo.sh with tackerckient | 11:49 |
jcapitao[m] | to find out the issue | 11:49 |
kkula | ack | 11:50 |
jcapitao[m] | no specific issue | 12:00 |
*** dviroel|out is now known as dviroel | 12:01 | |
jcapitao[m] | my guess is that I got network issues when updating it during script execution | 12:01 |
jcapitao[m] | kkula: I'm gonna push commit | 12:02 |
kkula | jcapitao, thanks! | 12:02 |
*** amoralej is now known as amoralej|lunch | 12:06 | |
rlandy | jcapitao[m]: amoralej|lunch: fyi - new gate blocker https://bugs.launchpad.net/tripleo/+bug/1979985 | 12:29 |
rlandy | from https://review.rdoproject.org/r/c/rdoinfo/+/42273 | 12:29 |
*** arxcruz is now known as arxcruz|brb | 12:42 | |
rdogerrit | Yatin Karel proposed openstack/neutron-distgit rpm-master: Requirement Sync https://review.rdoproject.org/r/c/openstack/neutron-distgit/+/42235 | 12:55 |
*** amoralej|lunch is now known as amoralej | 13:04 | |
amoralej | rlandy, damn, i ran gate jobs | 13:05 |
amoralej | lemme check | 13:05 |
rdogerrit | Alfredo Moralejo created rdoinfo master: Revert "Update puppet and facter to last versions for CS9 in Zed" https://review.rdoproject.org/r/c/rdoinfo/+/43677 | 13:07 |
amoralej | rlandy, ^ reverting it | 13:07 |
tkajinam | amoralej, rlandy maybe we can try this https://review.opendev.org/c/openstack/puppet-openstacklib/+/847783 | 13:08 |
rlandy | checking what runs on that | 13:08 |
amoralej | tkajinam, where do you install facter from in upstream puppet jobs? | 13:09 |
rlandy | let's kick some multinode jobs with that | 13:09 |
rlandy | so we know we're not breaking anything | 13:09 |
amoralej | rlandy, i ran https://review.rdoproject.org/r/c/rdoinfo/+/42273 | 13:09 |
amoralej | i didn't include tripleo-ci-centos-9-containers-multinode | 13:10 |
amoralej | what's different in containers-multinode to the other jobs? | 13:10 |
rdogerrit | Joel Capitao proposed rdoinfo master: Retire mox3 package https://review.rdoproject.org/r/c/rdoinfo/+/43700 | 13:12 |
tkajinam | amoralej, we install puppet/facter from rdo. our script supports usage of rpms by puppetlabs but we are not using it integration jobs | 13:12 |
tkajinam | * in integration job | 13:12 |
amoralej | ok | 13:12 |
amoralej | rlandy, you prefer me to do the revert or test with tkajinam patch? | 13:13 |
rlandy | amoralej: if tkajinam patch works and can be merged - it's ok | 13:13 |
rlandy | if not, we can revert | 13:13 |
amoralej | ok | 13:13 |
rlandy | let me try testproject | 13:13 |
tkajinam | it's still strange that the issue was not caught in the tripleo jobs we tested before merging the bump | 13:18 |
rlandy | sorry - following multiple failures here | 13:20 |
rlandy | back to testproject | 13:20 |
rlandy | https://review.rdoproject.org/r/c/testproject/+/39960 | 13:25 |
rlandy | amoralej: tkajinam: we need to decide before you both are EoD | 13:35 |
rlandy | gate is at risk for wallaby | 13:35 |
amoralej | sure | 13:35 |
amoralej | rlandy, note my patch only updated it in master | 13:35 |
amoralej | not wallaby | 13:36 |
rlandy | atm- yeah | 13:38 |
tkajinam | rlandy, I can merge that puppet change once we get result of ci | 13:41 |
tkajinam | in 1 ~ 2 hours | 13:41 |
rlandy | ok - let's see if it gives us a clean undercloud install | 13:41 |
tkajinam | rlandy, is there any way to check packages installed in each containers ? | 13:41 |
rlandy | tkajinam: yes ... | 13:42 |
rlandy | on the provider jobs | 13:42 |
tkajinam | strange thing is that puppet did not fail when being executed at host | 13:42 |
rlandy | you should see the bild logs | 13:42 |
tkajinam | while it failed when it was executed inside containers | 13:42 |
rlandy | build | 13:42 |
tkajinam | https://review.opendev.org/c/openstack/tripleo-heat-templates/+/841609/ | 13:44 |
tkajinam | https://zuul.opendev.org/t/openstack/build/d19022021a4543a18ecbbaa3ccf8b660 | 13:44 |
tkajinam | facter x86_64 1:3.14.19-1.el9s delorean-master-testing 517 k | 13:47 |
tkajinam | bingo | 13:47 |
tkajinam | amoralej, rlandy, I see the old facter was pulled while building container images | 13:48 |
*** arxcruz|brb is now known as arxcruz | 13:48 | |
amoralej | what time was that? | 13:49 |
amoralej | maybe before the package was in the afs mirror? | 13:49 |
tkajinam | while the host has the new version installed facter-4.2.10-1.el9s.noarch | 13:49 |
tkajinam | 2022-06-27 11:25:58.677449 | 13:50 |
tkajinam | this is when the job failed | 13:50 |
amoralej | so, different version in containers and in host? | 13:50 |
amoralej | that's weird | 13:50 |
tkajinam | 3 in container and 4 in host | 13:50 |
tkajinam | we use facter in host while generating cache, while the cache is parsed by facter in container during container puppet tasks | 13:51 |
amoralej | i see 3 in the host too | 13:51 |
amoralej | https://d42defc27bbe357868ac-cbda4aa1c62aa706d6caf520e6cec7d3.ssl.cf2.rackcdn.com/841609/8/check/tripleo-ci-centos-9-content-provider/d190220/logs/undercloud/var/log/extra/rpm-list.txt | 13:51 |
tkajinam | that's content provider job | 13:51 |
tkajinam | if you look at the build log of multi node job, which is pulling container images built by that content provider job, the node has facter 4 | 13:51 |
amoralej | ah, so the content provider built with 3 | 13:51 |
tkajinam | yeah | 13:51 |
amoralej | but nested jobs with 4? | 13:51 |
tkajinam | yes | 13:51 |
amoralej | ah, so it was synced in the middle | 13:52 |
amoralej | tkajinam, but that's causing the issue? | 13:52 |
tkajinam | fact cache was generated using facter 4 | 13:52 |
amoralej | ahhhhhhhh | 13:52 |
amoralej | damn | 13:52 |
amoralej | that's tricky | 13:52 |
tkajinam | and facter 3 in containers can't parse the cache correctly | 13:52 |
amoralej | so current jobs should go fine? | 13:52 |
tkajinam | let me chec | 13:52 |
tkajinam | check | 13:52 |
tkajinam | that observation would explain why we didn't hit the issue in rdo | 13:53 |
tkajinam | we did(or do) not use separate content provider there | 13:53 |
amoralej | yep | 13:54 |
tkajinam | https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-content-provider&project=openstack/tripleo-heat-templates | 13:54 |
tkajinam | hmm.. we don't the job passed recently but I confirmed facter 3 is pulled in recent 3 successful run | 13:55 |
tkajinam | https://zuul.opendev.org/t/openstack/build/7143bead7f2944fb98ffed70ac147164 | 13:56 |
tkajinam | thus standalone job just passed an I see facter 4 is pulled | 13:56 |
tkajinam | in host as well as in container | 13:57 |
tkajinam | so likely the mirror issue and old version was present during content provider job | 13:57 |
tkajinam | rlandy, https://bugs.launchpad.net/tripleo/+bug/1979985/comments/5 | 14:00 |
rlandy | https://review.rdoproject.org/zuul/stream/e36e2e15f53744e590c981287da3a3f1?logfile=console.log - failed | 14:03 |
rdogerrit | User karolinku created rdoinfo master: Pin tap-as-a-service https://review.rdoproject.org/r/c/rdoinfo/+/43756 | 14:15 |
rdogerrit | rdo-trunk created openstack/cinderlib-distgit wallaby-rdo: python-cinderlib: failed to build 1b58a5f https://review.rdoproject.org/r/c/openstack/cinderlib-distgit/+/43757 | 14:21 |
rdogerrit | Merged rdoinfo master: Pin tap-as-a-service https://review.rdoproject.org/r/c/rdoinfo/+/43756 | 14:41 |
*** pojadhav is now known as pojadhav|afk | 15:04 | |
tkajinam | amoralej, do you care to force-merge this ? https://review.rdoproject.org/r/c/openstack/octavia-distgit/+/37804 | 15:20 |
rdogerrit | Merged openstack/octavia-distgit rpm-master: Fix content of dist.conf https://review.rdoproject.org/r/c/openstack/octavia-distgit/+/37804 | 15:21 |
amoralej | tkajinam, done, i guess we should remove cs8 job from rpm-master already | 15:21 |
tkajinam | amoralej, thanks ! | 15:22 |
tkajinam | amoralej, yeah, we no longer need c9s packages now | 15:22 |
amoralej | you mean c8s | 15:22 |
tkajinam | at least from TripleO's PoV and Puppet's PoV | 15:22 |
amoralej | right? :) | 15:22 |
tkajinam | yeas | 15:22 |
tkajinam | yes | 15:22 |
tkajinam | c8s packages | 15:22 |
amoralej | in tripleo there are certain jobs on cs8, iiuc | 15:22 |
amoralej | the molecule ones | 15:23 |
amoralej | that's what i'm working for | 15:23 |
tkajinam | https://review.opendev.org/c/openstack/tripleo-ansible/+/839688/ | 15:23 |
tkajinam | maybe we need this in master | 15:24 |
amoralej | yep, that one | 15:24 |
tkajinam | ohk | 15:25 |
rdogerrit | Merged openstack/neutron-distgit rpm-master: Remove [DEFAULT] allow_overlapping_ips https://review.rdoproject.org/r/c/openstack/neutron-distgit/+/39024 | 15:26 |
*** dviroel is now known as dviroel|lunch | 15:27 | |
tkajinam | or we can put all cs8 jobs non-voting | 15:27 |
tkajinam | though it might hide actual problems | 15:27 |
amoralej | tkajinam, you mean the molecule ones? | 15:28 |
tkajinam | I mean the jobs in rdo | 15:28 |
tkajinam | we can wait for that tripleo-ansible patch for now. but we are likely to face the same problem more often | 15:28 |
tkajinam | since more and more projects are dropping py36/7 support during this cycle | 15:28 |
amoralej | that's the other option | 15:29 |
amoralej | making non-voting the cs8 ones in rpm-master | 15:29 |
amoralej | but given that we are about to remove it | 15:29 |
amoralej | i'm not sure if it's worthy | 15:29 |
amoralej | let me check how are created the templates | 15:30 |
tkajinam | yeah, as I said it's not urgent, as long as we are on track to remove c8s things | 15:33 |
*** marios is now known as marios|out | 15:39 | |
*** ysandeep is now known as ysandeep|PTO | 16:09 | |
rdogerrit | Merged openstack/neutron-distgit rpm-master: Requirement Sync https://review.rdoproject.org/r/c/openstack/neutron-distgit/+/42235 | 16:13 |
*** jlibosva is now known as Guest3430 | 16:17 | |
*** jlibosva is now known as Guest3431 | 16:23 | |
*** dviroel|lunch is now known as dviroel | 16:37 | |
*** amoralej is now known as amoralej|off | 16:51 | |
rdogerrit | Dariusz proposed rdo-infra/ci-config master: Added new ShellCheck job https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/42701 | 17:12 |
*** Guest3279 is now known as rcastillo | 17:20 | |
rdogerrit | Dariusz created rdo-infra/ci-config master: Rename centos-9-quay job to ubi-9-quay job https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43758 | 17:21 |
*** pojadhav|afk is now known as pojadhav | 17:36 | |
*** rlandy is now known as rlandy|afk | 18:40 | |
*** jpena is now known as jpena|off | 19:06 | |
*** rlandy|afk is now known as rlandy | 19:23 | |
*** dviroel is now known as dviroel|out | 21:40 | |
*** dasm is now known as dasm|off | 22:16 | |
rdogerrit | rdo-trunk created openstack/gnocchi-distgit rpm-master: gnocchi: failed to build a07d82e3 https://review.rdoproject.org/r/c/openstack/gnocchi-distgit/+/43759 | 22:34 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!