*** tosky has quit IRC | 00:12 | |
*** holser has joined #oooq | 00:18 | |
*** holser has quit IRC | 01:14 | |
*** raukadah is now known as chandankumar | 03:04 | |
*** skramaja has joined #oooq | 03:28 | |
*** jmasud has joined #oooq | 03:42 | |
*** ratailor has joined #oooq | 03:56 | |
*** bhagyashris has joined #oooq | 04:19 | |
*** udesale has joined #oooq | 04:32 | |
bhagyashris | ysandeep, can you please check this link https://review.rdoproject.org/zuul/status is it working for you ? | 04:58 |
---|---|---|
ysandeep | bhagyashris, no i faced same issue.. some proxy error | 04:59 |
bhagyashris | ysandeep, yeah me too facing same issue :( | 04:59 |
ysandeep | bhagyashris, i have pinged on internal ops channel.. | 05:01 |
bhagyashris | yeah i see | 05:01 |
bhagyashris | thank you :) | 05:01 |
*** ykarel has joined #oooq | 05:13 | |
*** ykarel_ has joined #oooq | 05:32 | |
*** ykarel has quit IRC | 05:34 | |
*** ykarel_ is now known as ykarel | 05:35 | |
*** marios has joined #oooq | 06:12 | |
*** jmasud has quit IRC | 06:38 | |
*** matbu has joined #oooq | 06:48 | |
*** tosky has joined #oooq | 08:35 | |
*** frenzy_friday has joined #oooq | 08:36 | |
*** arxcruz|2021 is now known as arxcruz | 08:58 | |
arxcruz | and i'm back :D | 08:58 |
marios | \o/ | 09:00 |
marios | me too | 09:00 |
arxcruz | marios: happy new year! | 09:07 |
arxcruz | bhagyashris: ChanServ dmellado soniya29 sshnaidm ysandeep ysandeep kopecmartin happy new year! | 09:08 |
marios | arxcruz: you 2 | 09:08 |
soniya29 | arxcruz: Happy new year | 09:08 |
ysandeep | hey folks o/ Happy new year :D | 09:08 |
bhagyashris | Happy new year all ! :) | 09:09 |
zbr | Happy new year! | 09:21 |
arxcruz | zbr: happy new year | 09:22 |
arxcruz | spotz: happy new year :) | 09:35 |
*** derekh has joined #oooq | 09:50 | |
*** ysandeep is now known as ysandeep|afk | 10:16 | |
*** saneax has joined #oooq | 10:26 | |
*** sanjayu_ has joined #oooq | 10:28 | |
*** saneax has quit IRC | 10:31 | |
*** slaweq has joined #oooq | 10:36 | |
slaweq | marios: hi, can You take a look at https://review.opendev.org/c/openstack/tripleo-heat-templates/+/764095 ? All other backports of this patch are already merged or at least approved (Train) | 10:37 |
marios | slaweq: ack | 10:37 |
slaweq | marios++ thx | 10:38 |
*** matbu has quit IRC | 10:40 | |
*** rfolco has joined #oooq | 10:41 | |
*** udesale_ has joined #oooq | 10:41 | |
*** matbu has joined #oooq | 10:42 | |
marios | np slaweq | 10:43 |
*** udesale has quit IRC | 10:44 | |
rfolco | happy new year folks | 10:48 |
rfolco | chandankumar, marios thanks for reviewing https://review.opendev.org/c/openstack/tripleo-ci/+/767591 | 10:48 |
rfolco | \o/ | 10:48 |
*** sanjayu_ has quit IRC | 10:49 | |
marios | rfolco: o/ | 10:50 |
*** matbu has quit IRC | 11:18 | |
*** matbu has joined #oooq | 11:18 | |
*** ysandeep|afk is now known as ysandeep | 11:38 | |
*** holser has joined #oooq | 11:59 | |
chandankumar | rfolco: marios ysandeep bhagyashris soniya29 arxcruz kopecmartin zbr frenzy_friday Happy New year ! | 12:06 |
ysandeep | Happy new year to all :D | 12:06 |
rfolco | happy 2021 everyone | 12:09 |
frenzy_friday | Happy new year everyone! | 12:14 |
*** slaweq has quit IRC | 12:18 | |
*** chem has joined #oooq | 12:19 | |
chandankumar | arxcruz: marios quick merge https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/769137 | 12:32 |
*** ykarel has quit IRC | 12:33 | |
*** ykarel has joined #oooq | 12:33 | |
*** ykarel_ has joined #oooq | 12:37 | |
*** rlandy has joined #oooq | 12:37 | |
*** ykarel has quit IRC | 12:37 | |
rlandy | happy new year everyone!! | 12:39 |
marios | chandankumar: ack | 12:40 |
marios | rlandy: o/ | 12:40 |
chandankumar | rlandy: \o, welcome back after an year :-) | 12:41 |
rlandy | good to back! | 12:41 |
rlandy | to be | 12:41 |
weshay|ruck | hellooooooooooooooooo :) happy new year folks | 12:45 |
rlandy | weshay|ruck: hey - how was the grand canyon? | 12:47 |
weshay|ruck | lolz.. will tell you later | 12:47 |
*** holser has quit IRC | 12:51 | |
weshay|ruck | sshnaidm, ysandeep can you jump on a call real quick and do a ruck/rover handover? | 12:51 |
sshnaidm | weshay|ruck, sure | 12:51 |
ysandeep | weshay|ruck, ack | 12:51 |
weshay|ruck | k.. | 12:51 |
weshay|ruck | https://meet.google.com/psj-ymrs-jqp | 12:52 |
ysandeep | weshay|ruck, https://code.engineering.redhat.com/gerrit/login/#/c/222438/ | 12:59 |
bhagyashris | arxcruz, rfolco, zbr, sshnaidm, rlandy, marios, ysandeep, bhagyashris, svyas, pojadhav, akahat, weshay, chandankumar, frenzy_friday | 12:59 |
sshnaidm | rlandy, hi, can we take a look at 16.1 today? I think there is a problem with openvswitch that breaks overcloud networking.. maybe need a different version | 12:59 |
bhagyashris | scrum in one min | 12:59 |
rlandy | sshnaidm: sure | 12:59 |
sshnaidm | rlandy, thanks | 12:59 |
rlandy | will check out the versions and job logs | 13:00 |
sshnaidm | soniya29, meet.google.com/oiv-geho-mai | 13:02 |
weshay|ruck | arxcruz, rfolco, zbr, sshnaidm, rlandy, marios, ysandeep, bhagyashris, svyas, pojadhav, akahat, weshay, chandankumar, frenzy_friday scrum | 13:03 |
soniya29 | sshnaidm, in scrum meeting | 13:03 |
rlandy | weshay|ruck: we're on scrum | 13:03 |
bhagyashris | weshay|ruck, https://meet.google.com/oiv-geho-mai | 13:04 |
sshnaidm | soniya29, the scrum is here ^ | 13:04 |
sshnaidm | bhagyashris, need to fix scrum meeting, it leads me to meet.google.com/xnf-tvdh-pmk | 13:04 |
*** pojadhav has joined #oooq | 13:04 | |
bhagyashris | sshnaidm, ohh i will check that | 13:05 |
ysandeep | sshnaidm, hey o/ have we decided to cix https://bugs.launchpad.net/tripleo/+bug/1909574 this one? | 13:10 |
openstack | Launchpad bug 1909574 in tripleo "periodic-tripleo-ci-centos-8-scenario010-kvm-standalone-octavia-master/victoria/ussuri failing on tempest tests with SSHTimeout issue with error message "socket.timeout: timed out"" [High,Triaged] | 13:10 |
ysandeep | where we noticed segfault in ovn. | 13:11 |
sshnaidm | ysandeep, well, it's not blocking prodchain, so maybe better to have it in bugzilla | 13:11 |
sshnaidm | weshay|ruck, wdyt? ^ | 13:11 |
ysandeep | yeah otherwise I will try seek help from neutron guys, they are generally very helpful :D | 13:12 |
*** matbu has quit IRC | 13:16 | |
*** matbu has joined #oooq | 13:16 | |
ysandeep | sshnaidm, i will do that.. thanks! | 13:18 |
sshnaidm | ysandeep++ | 13:18 |
sshnaidm | weshay|ruck, need to look at https://bugzilla.redhat.com/show_bug.cgi?id=1911891 | 13:21 |
openstack | bugzilla.redhat.com bug 1911891 in ansible-openstack-modules "[RFE] Improvement in ansible-playbook" [Medium,New] - Assigned to sshnaidm | 13:21 |
weshay|ruck | sshnaidm, ysandeep can one of you folks join me in the cix call in 9 min? | 13:21 |
ysandeep | weshay|ruck, i will join | 13:21 |
*** ratailor has quit IRC | 13:23 | |
*** tosky has quit IRC | 13:30 | |
*** tosky has joined #oooq | 13:30 | |
spotz | Happy New Years arxcruz | 13:37 |
sshnaidm | weshay|ruck, have 2 min? | 13:58 |
weshay|ruck | aye | 13:58 |
sshnaidm | https://meet.google.com/xod-cdvi-bfb | 13:58 |
rlandy | sshnaidm: ok - so I think I know where the openvswitch is coming in - just not sure how we fix it yet - looking | 14:11 |
rlandy | before the job was throwing an error deleting the openvswitch that was installed from zuul | 14:12 |
rlandy | we need a lower version initially | 14:12 |
*** TrevorV has joined #oooq | 14:18 | |
marios | sshnaidm: can you please review when you have time https://review.opendev.org/c/openstack/tripleo-ci/+/761188 - I split the original, this one just adds the new template. plan is to wire it all up, then at the end we can merge https://review.opendev.org/c/openstack/tripleo-ci/+/769131/ to remove excess | 14:28 |
marios | sshnaidm: topic branch there i will add more reviews https://review.opendev.org/q/topic:reduce-content-providers | 14:29 |
marios | rlandy: sshnaidm: weshay|ruck: arxcruz: zbr: ysandeep: rfolco: chandankumar: please review when you have time https://review.opendev.org/c/openstack/tripleo-ci/+/761188 thank you | 14:36 |
ysandeep | weshay|ruck, hey looks like Attila fixed the infrared job to trigger with extended hash, infrared job triggered today.. if all jobs passes we will promote today.. In today's morning run all tripleo-ci jobs passed.. | 14:36 |
weshay|ruck | marios, thanks for your doc work.. https://f267574198a517c003d1-a40b9478ae1bd073dc32f331038fe6d7.ssl.cf5.rackcdn.com/767375/7/check/openstack-tox-docs/8252ffe/docs/ci/stages-overview.html#the-component-promotion-pipeline | 14:36 |
weshay|ruck | :) | 14:36 |
marios | weshay|ruck: ack i think its almost ready to merge thanks for checking | 14:36 |
weshay|ruck | rlandy, rfolco you two should read through this https://f267574198a517c003d1-a40b9478ae1bd073dc32f331038fe6d7.ssl.cf5.rackcdn.com/767375/7/check/openstack-tox-docs/8252ffe/docs/ci/stages-overview.html#the-integration-promotion-pipeline | 14:37 |
* rlandy looks | 14:37 | |
marios | rlandy: https://review.opendev.org/c/openstack/tripleo-docs/+/767375 thanks | 14:37 |
rlandy | marios: ack - shortly | 14:38 |
marios | rlandy: thx | 14:38 |
rlandy | just fixing ansible repo link | 14:38 |
marios | rlandy: no rush in your next reviews time thank you | 14:38 |
weshay|ruck | marios, re: https://review.opendev.org/q/topic:reduce-content-providers is this something you'd like to present to the team in planning or work through ad-hoc | 14:46 |
marios | weshay|ruck: sure yes i think it is worth showing the order/thinking behind that. i have some notes at https://projects.engineering.redhat.com/browse/TRIPLEOCI-249 | 14:47 |
weshay|ruck | k.. thanks.. will get it on the agenda | 14:47 |
marios | weshay|ruck: but for now we need to agree and merge the first patch which is https://review.opendev.org/761188 | 14:47 |
marios | weshay|ruck: thanks | 14:48 |
weshay|ruck | chandankumar, sshnaidm as you have time.. ^ your review is important here | 14:50 |
marios | weshay|ruck: wrt your comment there ^^^ about test/dnm jobs with the patches needed see http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019522.html [tripleo] next meeting Tuesday Dec 22 @ 1400 UTC in #tripleo Please consider adding your news/blockers/plans in https://etherpad.opendev.org/p/tripleo-meeting-items even if you cannot attend. | 15:10 |
weshay|ruck | aye.. thanks | 15:10 |
marios | weshay|ruck: sorry wrong paste... :/ see see comment there https://review.opendev.org/c/openstack/tripleo-ci/+/761188/7/zuul.d/upgrades-jobs-templates.yaml#97 | 15:11 |
*** jmasud has joined #oooq | 15:22 | |
*** ykarel_ has quit IRC | 15:26 | |
marios | weshay|ruck: 1-1 time? we can push if you busy np | 15:30 |
weshay|ruck | ya.. let's chat | 15:30 |
*** rfolco has quit IRC | 15:31 | |
ysandeep | weshay|ruck, fyi.. periodic-tripleo-ci-rhel-8-standalone-full-tempest-api-rhos-16.2 in current run failed (Looks like random tempest failure) I reran it with https://code.engineering.redhat.com/gerrit/#/c/211643/ so that we can hopefully promote 16.2 today. | 15:31 |
*** jmasud has quit IRC | 15:40 | |
rlandy | sshnaidm: weshay|ruck: so I think we need http://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-config.git/tree/playbooks/base/pre.yaml#n37 for 16.1 as well | 15:52 |
rlandy | it's a config change | 15:52 |
rlandy | so merge and pray | 15:52 |
rlandy | http://git.host.prod.eng.bos.redhat.com/git/openstack/sf-config.git/tree/roles/openvswitch/vars/RedHat.yaml?id=60a1dbdb60e685cddd2202793f642f1ee12b10c3 | 15:52 |
*** ykarel_ has joined #oooq | 15:55 | |
*** ykarel_ has quit IRC | 16:03 | |
weshay|ruck | anyone know how ens3 got back into the workflow? https://bugs.launchpad.net/tripleo/+bug/1910107 | 16:11 |
openstack | Launchpad bug 1910107 in tripleo "master/wallaby undercloud install failure unable to start network service" [Critical,Triaged] | 16:11 |
rlandy | weshay|ruck: yep - 8.3 | 16:18 |
weshay|ruck | ya.. figures | 16:18 |
rlandy | sshnaidm: ping re: openvswitch versions | 16:18 |
*** rfolco has joined #oooq | 16:19 | |
rlandy | weshay|ruck: john fulton was talking about it before break | 16:19 |
rlandy | we have 2.13 which should be correct according to rhos-delivery | 16:20 |
rlandy | only thing I can think of is the bridge is installed with diff version | 16:20 |
weshay|ruck | k.. I'm not sure.. but the issue I'm seeing in centos ci.. is not related to ovs.. | 16:21 |
weshay|ruck | ens3 is a native interface | 16:21 |
rlandy | weshay|ruck: sshnaidm: not sure if this make sense to do :) ... but https://code.engineering.redhat.com/gerrit/222498 Use the openvswitch role for 16.1 | 16:21 |
rlandy | weshay|ruck: looking at error | 16:22 |
*** udesale_ has quit IRC | 16:24 | |
sshnaidm | rlandy, thanks, looking | 16:27 |
rlandy | sshnaidm: we can merge that and try | 16:28 |
sshnaidm | rlandy, ack, merging | 16:28 |
rlandy | sshnaidm: if it does nothing, we have a legit issue | 16:28 |
sshnaidm | rlandy, ok, will rerun 16.1 job | 16:28 |
rlandy | weshay|ruck: so .. ens3 ... | 16:30 |
rlandy | we did a lot of work there when we started working with rhel 8.3 | 16:30 |
rlandy | or I guess rhel 8 in general | 16:30 |
rlandy | on baremetal | 16:30 |
rlandy | depends what the ci.centso nodes look like | 16:31 |
rlandy | I had that role included had to remove it as it was failing elsewhere | 16:31 |
* rlandy traces back | 16:32 | |
rlandy | <fultonj> Dec 10 13:19:42 undercloud.mydomain.tld network[45261]: Bringing up interface ens3-1: Error: Connection activation failed: No suitable device found for this connection (device eth0 not ava> | 16:32 |
weshay|ruck | ya.. | 16:33 |
weshay|ruck | not sure why the undercloud is trying that now | 16:33 |
rlandy | weshay|ruck: pvt sent you chat conversation | 16:34 |
weshay|ruck | rlandy, k.. I'll check to make sure we're nuking it | 16:35 |
rlandy | weshay|ruck: just be careful where you do that work - I got yelled at for adding it in a general place | 16:36 |
rlandy | revert fiasco | 16:36 |
sshnaidm | chandankumar, arxcruz maybe you know, do we have somewhere ansible playbook that prepares devstack for tempest? Like creating users, roles, networks etc etc? | 16:42 |
chandankumar | sshnaidm: there is a script which takes care of that https://opendev.org/openstack/devstack/src/branch/master/lib/tempest | 16:44 |
chandankumar | embedded with in devstack itself | 16:45 |
sshnaidm | chandankumar, yeah, I hoped there is ansible stuff.. | 16:45 |
*** marios is now known as marios|out | 17:24 | |
*** marios|out has quit IRC | 17:54 | |
*** derekh has quit IRC | 18:02 | |
rlandy | weshay|ruck: can I merge this? https://code.engineering.redhat.com/gerrit/#/c/221454/ rhel8 container tools | 18:12 |
rlandy | last successful run was dec 23 | 18:12 |
rlandy | the I can test rhel 8.4 over that | 18:12 |
weshay|ruck | rlandy, can we chat about what you found w/ 8.4? | 18:13 |
rlandy | weshay|ruck: nothing yet ... we can only really test 8.4 once https://code.engineering.redhat.com/gerrit/#/c/222354/ merges | 18:13 |
rlandy | and we switch rhos-release params | 18:13 |
rlandy | either way container-tools should be unlocked | 18:14 |
weshay|ruck | rlandy, k.. are we adding 8.4 to internal dep pipeline or prod jobs? | 18:14 |
rlandy | weshay|ruck: going to ask fuzzball81 to merge https://code.engineering.redhat.com/gerrit/#/c/222354/ then I was going to runs some tests | 18:15 |
rlandy | if it was seemless, I would switch prod | 18:15 |
rlandy | if not, add a dep pipeline | 18:15 |
rlandy | a dep pipeline is just another thing to watch | 18:15 |
rlandy | if the real deployment is going out on 8.4 | 18:16 |
rlandy | we already ran the container build job there | 18:16 |
rlandy | for rhos-release | 18:16 |
weshay|ruck | rlandy, can we chat? 8.4 is not prod | 18:16 |
weshay|ruck | https://access.redhat.com/articles/3078 | 18:16 |
rlandy | weshay|ruck: sure | 18:16 |
rlandy | either way we need the rhos-release stuff merged up | 18:17 |
weshay|ruck | aye.. | 18:17 |
weshay|ruck | https://meet.google.com/axq-trne-kjy?authuser=1 | 18:18 |
rlandy | sshnaidm: did you use an ansible rpm to test ansible 2.10? | 18:42 |
weshay|ruck | rlandy, https://anonbadger.wordpress.com/2020/08/25/why-upstream-ansible-stopped-shipping-rpms/ | 18:50 |
rlandy | https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#upgrading-ansible-from-version-2-9-and-older-to-version-2-10-or-later | 18:52 |
*** jmasud has joined #oooq | 18:56 | |
rlandy | weshay|ruck: https://github.com/rdo-infra/ci-config/tree/master/ci-scripts/infra-setup/roles/rrcockpit/files/compare_rpms/diff_tripleo_builds/rpm_skip_list | 18:59 |
sshnaidm | rlandy, no, from pip | 19:00 |
rlandy | yeah - seems to be the consensus | 19:00 |
rlandy | sshnaidm: looks like we still have the openvswitch tempest fail | 19:11 |
sshnaidm | rlandy, yeah, though I'm not 100% sure it's openvswitch problem.. | 19:13 |
sshnaidm | I see some selinux errors as well.. | 19:13 |
rlandy | sshnaidm: we can try set permissive on the deploy | 19:13 |
rlandy | and see if that helps | 19:13 |
sshnaidm | rlandy, yeah | 19:14 |
sshnaidm | rlandy, how do we do it? | 19:14 |
rlandy | sshnaidm: standalone params, I think - let me check | 19:14 |
rlandy | I'll add that change | 19:14 |
sshnaidm | rlandy, thanks | 19:15 |
sshnaidm | rlandy, if it doesn't work, seems like broken.. | 19:15 |
rlandy | sshnaidm: yeah - then we can BZ with confidence | 19:15 |
weshay|ruck | rlandy, fyi https://opendev.org/openstack/tripleo-image-elements/src/branch/master/elements/interface-names/install.d/71-clean-stale-interface#L8 | 19:29 |
rlandy | weshay|ruck: yeah - that's been there since we first started working with rhel8/centos8 | 19:35 |
rlandy | not sure what went wrong with 8.3 images | 19:35 |
weshay|ruck | aye | 19:35 |
rlandy | I know the stream images are old | 19:35 |
weshay|ruck | rlandy, seems like w/ 8.3 at boot the interface is recreated | 19:35 |
weshay|ruck | it's our overcloud images | 19:35 |
rlandy | weshay|ruck: way to backslide! | 19:36 |
rlandy | ours? | 19:36 |
weshay|ruck | rlandy, check out the last two comments in https://bugs.launchpad.net/tripleo/+bug/1910107 | 19:36 |
openstack | Launchpad bug 1910107 in tripleo "master/wallaby undercloud install failure unable to start network service" [Critical,Triaged] | 19:36 |
rlandy | then we should see it on OVB, no? | 19:36 |
weshay|ruck | yes | 19:37 |
weshay|ruck | rlandy, well.. sorry.. it's created at run time | 19:37 |
weshay|ruck | hrm | 19:38 |
rlandy | sshnaidm: https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/overcloud-deploy/templates/overcloud-selinux-config.yaml.j2 | 19:45 |
rlandy | think we just set overcloud_selinux_enabled to permissive | 19:46 |
rlandy | https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/overcloud-deploy/defaults/main.yml#L192 | 19:46 |
rlandy | override that | 19:46 |
* rlandy tries | 19:46 | |
*** rlandy is now known as rlandy|drappt | 20:18 | |
*** holser has joined #oooq | 20:39 | |
*** holser has quit IRC | 21:53 | |
*** jmasud has quit IRC | 21:56 | |
*** holser has joined #oooq | 22:12 | |
*** jmasud has joined #oooq | 22:18 | |
*** TrevorV has quit IRC | 22:37 | |
*** rlandy|drappt is now known as rlandy | 22:46 | |
rlandy | sshnaidm: hmmm ... maybe we need to run this by the tempest team ... https://sf.hosted.upshift.rdu2.redhat.com/logs/34/221234/7/check/tripleo-ci-rhel-8-multinode-1ctlr-featureset010-rhos-16.1/de5b5e7/logs/undercloud/home/zuul/overcloud-selinux-config.yaml | 22:51 |
rlandy | should be permissive now | 22:52 |
sshnaidm | rlandy, heh.. I think maybe network team | 22:52 |
sshnaidm | this doesn't look good https://sf.hosted.upshift.rdu2.redhat.com/logs/34/221234/7/check/tripleo-ci-rhel-8-multinode-1ctlr-featureset010-rhos-16.1/de5b5e7/logs/undercloud/var/log/openvswitch/ovsdb-server.log | 22:53 |
sshnaidm | I'll hold a node | 22:56 |
rlandy | sshnaidm: yep - possibly legit? | 22:56 |
sshnaidm | seems like that | 22:56 |
*** jmasud has quit IRC | 23:46 | |
*** jmasud has joined #oooq | 23:57 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!