| rlandy|ruck | just need to finish up here | 00:00 |
|---|---|---|
| rlandy|ruck | sec | 00:00 |
| rlandy|ruck | rcastillo|rover: what's the problem with c9 you are seeing? | 00:03 |
| rcastillo|rover | just the thing that's making the ci.centos job fail | 00:04 |
| rcastillo|rover | rlandy|ruck: I think I see the fix for the c7 failure | 00:08 |
| rcastillo|rover | not sure why it's failing all of a sudden | 00:08 |
| rlandy|ruck | rcastillo|rover: not sure why now either | 00:10 |
| rlandy|ruck | if you have a fix, awesome | 00:10 |
| rcastillo|rover | something happened to the git version | 00:10 |
| rlandy|ruck | pls bug the error and submit your fix | 00:10 |
| rlandy|ruck | with closes-bug:#<bug number> | 00:10 |
| rlandy|ruck | thanks | 00:10 |
| rlandy|ruck | https://jenkins-cloudsig-ci.apps.ocp.ci.centos.org/job/tripleo-quickstart-promote-master-centos9-current-tripleo-delorean-minimal/20/console | 00:11 |
| rlandy|ruck | oh - I see the c9 error | 00:11 |
| rlandy|ruck | sorry | 00:11 |
| rlandy|ruck | forgot I asked you about ci.centos | 00:11 |
| rcastillo|rover | no worries, gate blocker is more urgent anyway | 00:17 |
| rcastillo|rover | https://bugs.launchpad.net/tripleo/+bug/1960973 | 00:20 |
| rcastillo|rover | fix https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/829421 | 00:20 |
| rcastillo|rover | rlandy|ruck: ^ | 00:20 |
| rlandy|ruck | looking | 00:45 |
| rlandy|ruck | rcastillo|rover++ | 00:46 |
| rlandy|ruck | nicely done | 00:46 |
| rlandy|ruck | added the promotion-blocker tag | 00:46 |
| rlandy|ruck | rcastillo|rover: you probably want a DNM patch - tripleo-heat-template on stable/train branch | 00:46 |
| rlandy|ruck | and depends-on your change | 00:47 |
| rlandy|ruck | to check the c7 content provider | 00:47 |
| rlandy|ruck | make sense? | 00:47 |
| rcastillo|rover | ok, wasn't sure if we wanted that | 00:47 |
| rcastillo|rover | rlandy|ruck: ack, I can do that | 00:47 |
| rlandy|ruck | yes please | 00:47 |
| rlandy|ruck | updating the program doc | 00:47 |
| rlandy|ruck | rcastillo|rover: ok - I'm out | 00:52 |
| rlandy|ruck | have a good night | 00:52 |
| rlandy|ruck | see you tomorrow | 00:52 |
| rlandy|ruck | thanks for fixing ^^ | 00:52 |
| rcastillo|rover | ok, good night, see you tomorrow | 00:52 |
| *** rlandy|ruck is now known as rlandy|out | 00:52 | |
| *** ysandeep|out is now known as ysandeep | 05:36 | |
| *** ykarel_ is now known as ykarel | 06:49 | |
| marios | akahat: o/ hey do we still need this? can you rebase for the conflict if so https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/827954 | 07:05 |
| ysandeep | pojadhav, rechecked your sc010 testproject to check the latest patchset works https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/829239 | 07:20 |
| marios | chandankumar: please see pvt | 07:26 |
| marios | phew thanks chandankumar ;) | 07:26 |
| pojadhav | ysandeep, ack, i was about to recheck only. thanks. | 07:48 |
| *** amoralej|off is now known as amoralej | 08:03 | |
| *** jpena|off is now known as jpena | 08:33 | |
| akahat | marios, o/ above review is related to this: https://trello.com/c/2CAk8C4p/2330-cixlp1959328tripleociproa-tempestscenariotestnetworkbasicopstestnetworkbasicopstestnetworkbasicops-failing-on-periodic-tripleo-c | 08:34 |
| akahat | looks like we don | 08:34 |
| akahat | don't have any updates on this. | 08:34 |
| marios | akahat: ah ok so you posted the skip revert for testing, not because we are ready to revert it yet | 08:35 |
| marios | akahat: k thanks for checking | 08:35 |
| akahat | marios, ack. | 08:36 |
| akahat | marios, chandankumar arxcruz|ruck rlandy|out promoter server is started now. | 08:51 |
| marios | thx akahat | 08:52 |
| *** ysandeep is now known as ysandeep|lunch | 08:59 | |
| arxcruz|ruck | all, have you guys see this problem before? | 09:03 |
| arxcruz|ruck | https://6aef0912e1e7e067f94c-3957ec525977d9a3771da3eb17c87a1d.ssl.cf5.rackcdn.com/829309/3/check/tripleo-tox-molecule/59333e1/job-output.txt | 09:03 |
| arxcruz|ruck | 2022-02-16 06:36:19.620370 | primary | [31m Problem: package python3-tripleo-common-16.3.1-0.20220126035321.8b0e020.el8.noarch requires python3-ansible-runner >= 1.4.4, but none of the providers can be installed[0m | 09:03 |
| marios | not seen it arxcruz|ruck | 09:04 |
| marios | arxcruz|ruck: so new gate blocker? | 09:05 |
| arxcruz|ruck | marios yes | 09:05 |
| arxcruz|ruck | marios what do you think? put it as non voting so you patch can go through ? | 09:05 |
| marios | arxcruz|ruck: hmm only triploe-ci i think ? | 09:06 |
| marios | https://zuul.openstack.org/builds?job_name=tripleo-tox-molecule | 09:06 |
| marios | looks green @ quickstart and upgrade | 09:06 |
| marios | but | 09:06 |
| marios | how... i mean we merged things last night so... is it new this morning ? | 09:06 |
| arxcruz|ruck | marios seems to be | 09:07 |
| arxcruz|ruck | marios i just notice your patch failed twice on gate on this | 09:07 |
| marios | arxcruz|ruck: so not total gate blocker at least i mean all the tripleo things | 09:07 |
| marios | arxcruz|ruck: i think it needs bug first anyway if you don't have one ... then if we cant find a solution before this afternoon we can make it non voting pending fix | 09:07 |
| arxcruz|ruck | marios ok, i'm creating a bug | 09:08 |
| marios | arxcruz|ruck: my patch in https://review.opendev.org/c/openstack/tripleo-ci/+/829309/ is importnat but not critical (only sc12 job there) lucky we got the parent patch in before this hit | 09:08 |
| marios | phew ! | 09:08 |
| arxcruz|ruck | that's why i'm worried :) | 09:08 |
| marios | arxcruz|ruck: worth reaching out to rdo folks 'cos package conflict' they might be able to help/already know | 09:13 |
| marios | amoralej: any idea about "python3-tripleo-common-16.3.1-0.20220126035321.8b0e020.el8.noarch requires python3-ansible-runner >= 1.4.4, but none of the providers can be installed" have you seen it? | 09:13 |
| marios | arxcruz|ruck: ^ | 09:13 |
| amoralej | mmm | 09:13 |
| amoralej | where is the log? | 09:13 |
| marios | amoralej: https://6aef0912e1e7e067f94c-3957ec525977d9a3771da3eb17c87a1d.ssl.cf5.rackcdn.com/829309/3/check/tripleo-tox-molecule/59333e1/job-output.txt | 09:13 |
| amoralej | i think i know what's the problem | 09:14 |
| amoralej | gimme a while | 09:14 |
| marios | amoralej: ;) thank you :D | 09:15 |
| arxcruz|ruck | https://bugs.launchpad.net/tripleo/+bug/1961035 | 09:16 |
| arxcruz|ruck | didn't mark as promotion blocker for now | 09:16 |
| amoralej | mmm that's centos8 | 09:17 |
| amoralej | it's not what i was thinking... | 09:17 |
| marios | k amoralej thanks for checking anyway | 09:17 |
| amoralej | still checking | 09:17 |
| marios | for some reason it seems to hit tripleo-ci only don't know why | 09:17 |
| marios | we have ./test-requirements.txt:4:ansible-core==2.11.7 | 09:17 |
| amoralej | why we have ansible-core in centos8? | 09:18 |
| marios | amoralej: sshnaidm: made the switch so we use smaller ansible-core package (I believe it is the recommended way now?) | 09:18 |
| marios | amoralej: (and install collections as needed) | 09:18 |
| amoralej | not in centos8 | 09:19 |
| amoralej | lemme check where is coming from | 09:19 |
| marios | so maybe we just need to swtich that job to use centos9 stream nodeset? | 09:19 |
| marios | amoralej: ? ^ | 09:19 |
| amoralej | dunno, honestly i'm not sure what that job is doing | 09:21 |
| amoralej | but deps in cs8 are not ready for ansible-core | 09:21 |
| marios | amoralej: testing that https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/tripleo-repos/molecule/default/converge.yml | 09:25 |
| marios | amoralej: (what that job is doing ^^) | 09:25 |
| marios | amoralej: arxcruz|ruck: lets see if the nodest helps us with https://review.opendev.org/c/openstack/tripleo-ci/+/829503 | 09:25 |
| amoralej | yes, that may help | 09:26 |
| sshnaidm | marios, this job runs on master, right? | 09:26 |
| amoralej | anyway, i'm not sure how that ansible-core package ended up in the container | 09:26 |
| marios | sshnaidm: yeah seems to be the ci repos only? and tripleo-upgrade? https://zuul.openstack.org/builds?job_name=tripleo-tox-molecule | 09:26 |
| sshnaidm | marios, if so, then yes, need to move it to c9 | 09:27 |
| arxcruz|ruck | marios i don't think it will work, because the node create docker containers that run the ansible, so the problem is the container we are using | 09:27 |
| marios | ack sshnaidm yeah its the right thing to do too - hope it also solves the conflict let's see | 09:27 |
| marios | arxcruz|ruck: ah right... | 09:27 |
| marios | arxcruz|ruck: didn't we update that already ? | 09:27 |
| arxcruz|ruck | marios it's using stream8 | 09:28 |
| arxcruz|ruck | https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/tripleo-repos/molecule/default/molecule.yml#L10 | 09:29 |
| marios | arxcruz|ruck: k post the patch? | 09:29 |
| arxcruz|ruck | marios yes, i will | 09:29 |
| marios | arxcruz|ruck: i think we will need my one anywa cos 11:27 < marios> ack sshnaidm yeah its the right thing to do too - hope it also solves the conflict let's see | 09:29 |
| arxcruz|ruck | hmmm, quay doesn't have stream 9 | 09:29 |
| marios | arxcruz|ruck: i.e. even if it doesn't address the issue | 09:29 |
| arxcruz|ruck | marios https://review.opendev.org/c/openstack/tripleo-ci/+/829509 | 09:34 |
| arxcruz|ruck | i hope the python not be an issue | 09:35 |
| marios | arxcruz|ruck: ack | 09:35 |
| arxcruz|ruck | there was a comment by sorin there | 09:35 |
| arxcruz|ruck | brb | 09:35 |
| jm1 | sshnaidm: o/ iiuc Zuul still requires some kind of auth for gerrit connections. Without a password you wont get a self.session and then Zuuls falls back to SSH https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/gerrit/gerritconnection.py#L617 | 09:47 |
| jm1 | sshnaidm: so we have to continue using passwords or ssh keys in reproducer for connections to e.g. rdoproject | 09:49 |
| sshnaidm | jm1, I see :( | 09:55 |
| jm1 | sshnaidm: but since we are reading only, maybe we could patch in anonymous access? I have an idea how to do that but i am not sure whether upstream would accept that ^^ | 09:57 |
| sshnaidm | jm1, I'm not sure how it can be with ssh, it will still required keys | 09:58 |
| jm1 | sshnaidm: no ssh, we use http without auth instead | 09:58 |
| sshnaidm | jm1, I doubt it's possible, but worth to ask | 09:59 |
| jm1 | sshnaidm: will give it a shot | 10:00 |
| *** ysandeep|lunch is now known as ysandeep | 10:03 | |
| arxcruz|ruck | marios hmmm, i think we have more issues related to package dependences | 10:11 |
| arxcruz|ruck | https://logserver.rdoproject.org/19/37919/9/check/periodic-tripleo-centos-8-buildimage-ironic-python-agent-victoria/74b71ed/job-output.txt | 10:11 |
| arxcruz|ruck | 2022-02-16 09:36:21.877672 | primary | 2022-02-16 09:36:21 | - cannot install the best update candidate for package ansible-collections-openstack-1.6.1-0.20220213121331.9c28af7.el8.noarch | 10:12 |
| arxcruz|ruck | amoralej ^ | 10:12 |
| arxcruz|ruck | not the same package though | 10:12 |
| amoralej | arxcruz|ruck, it's the same issue | 10:16 |
| amoralej | the problem is that there is ansible-core installed in the centos8 | 10:17 |
| arxcruz|ruck | then it's a promotion blocker, i'm updating the bug and add info that is blocking other jobs as well | 10:17 |
| arxcruz|ruck | amoralej so, do you have any sugestion? update the package dependence? replace ansible-core ? | 10:20 |
| amoralej | yes, remove ansible-core | 10:21 |
| amoralej | and install regular ansible package | 10:21 |
| amoralej | that's what is in rdo deps repo | 10:21 |
| amoralej | that's running in a container? | 10:21 |
| amoralej | what's the base image? | 10:21 |
| * pojadhav- going out.. will be back in an hour. | 10:25 | |
| *** pojadhav- is now known as pojadhav | 10:25 | |
| *** ykarel_ is now known as ykarel | 10:31 | |
| arxcruz|ruck | need to check | 10:33 |
| *** rlandy|out is now known as rlandy|ruck | 11:08 | |
| rlandy|ruck | arxcruz|ruck: hey | 11:12 |
| rlandy|ruck | arxcruz|ruck: anything other than https://review.opendev.org/c/openstack/tripleo-ci/+/829509 burning? | 11:13 |
| arxcruz|ruck | rlandy|ruck yes, we have a promotion blocker because of this ansible dependence | 11:14 |
| arxcruz|ruck | i update the bug | 11:14 |
| arxcruz|ruck | https://bugs.launchpad.net/tripleo/+bug/1961035 | 11:14 |
| rlandy|ruck | arxcruz|ruck: thank you - will check in with that shortly | 11:14 |
| arxcruz|ruck | i'm thinking add a {pkg} remove -y ansible-core on repo_cmd_before in setup-repos role | 11:14 |
| arxcruz|ruck | but i'm not sure if that's the best option | 11:15 |
| *** dviroel|out is now known as dviroel | 11:21 | |
| soniya29 | arxcruz|ruck, kopecmartin, chandankumar, ysandeep, rlandy|ruck, please add/edit today's agenda for tempest meeting - https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ | 11:23 |
| arxcruz|ruck | soniya29 i might not be able to participate | 11:24 |
| soniya29 | arxcruz|ruck, no problem :) | 11:24 |
| arxcruz|ruck | oh, nevermind, i might participate | 11:24 |
| marios | frenzy_friday: o/ hey can you please check comments and update https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/39424 | 11:36 |
| frenzy_friday | marios, ack, updating | 11:37 |
| chandankumar | rlandy|ruck: https://review.opendev.org/c/openstack/diskimage-builder/+/828617 for ovb check job | 11:38 |
| marios | thanks frenzy_friday | 11:40 |
| rlandy|ruck | arxcruz|ruck: https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001 | 11:42 |
| rlandy|ruck | can you bug that? | 11:42 |
| rlandy|ruck | rcastillo|rover can pick it up when he comes in | 11:42 |
| arxcruz|ruck | rlandy|ruck ack, checking the logs to see what's the issue | 11:44 |
| arxcruz|ruck | seems to be ssl certificate issue | 11:44 |
| ysandeep | marios, this concern was resolved right? https://review.opendev.org/c/openstack/tripleo-quickstart/+/828126/3#message-b06f5fc9dae8a1bfc75403b48dfdec26b74474cc - I see its marked as resolved but no justification in comments. | 11:46 |
| rlandy|ruck | arxcruz|ruck: sync quickly? https://meet.google.com/rof-arfj-mha?pli=1&authuser=0 | 11:46 |
| marios | ysandeep: yeah | 11:46 |
| marios | ysandeep: it is now fully available and in fact we are taking it from dlrn now | 11:47 |
| marios | per the parent patch | 11:47 |
| marios | ysandeep: ([6] https://review.opendev.org/c/openstack/tripleo-quickstart/+/828522 | 11:47 |
| ysandeep | marios, ack, thanks for confirming , looking | 11:47 |
| marios | ysandeep: thanks | 11:48 |
| rlandy|ruck | arxcruz|ruck: can you hear me now? | 11:48 |
| soniya29 | arxcruz|ruck, kopecmartin, chandankumar, rlandy|ruck, tempest meeting in 1 minute | 11:59 |
| arxcruz|ruck | soniya29 i'll be late , syncing with rlandy|ruck | 11:59 |
| soniya29 | sure | 11:59 |
| arxcruz|ruck | rlandy|ruck https://bugs.launchpad.net/tripleo/+bug/1961056 | 12:01 |
| soniya29 | kopecmartin, tempest meeting? | 12:02 |
| arxcruz|ruck | tosky around? can you take a look in the https://review.opendev.org/c/openinfra/python-tempestconf/+/820169 ? | 12:05 |
| rlandy|ruck | dviroel: hey - https://trello.com/c/nrogdbAt/2311-cixlp1958531tripleociproa-octavia-testbasichttptraffic-on-c9-times-out-as-it-asks-for-password-while-connecting-to-the-vm | 12:05 |
| arxcruz|ruck | and see Abishek comment | 12:05 |
| rlandy|ruck | still waiting for wallaby on that?? | 12:05 |
| rlandy|ruck | rcastillo|rover: hey: what's that status of https://trello.com/c/F00VnyBm/2336-cixlp1956475tripleociproa-centos9-nova-migration-breaks-because-sftp-server-is-not-part-of-the-migration-wrapper? | 12:06 |
| dviroel | rlandy|ruck: yes - we are missing backports yet (https://review.opendev.org/c/openstack/octavia/+/829032) - chandan added core reviews to it | 12:06 |
| rlandy|ruck | ysandeep: is this blocking? https://trello.com/c/H7RfkSBu/2358-cixbz2053527osp17rhel9node-provisioning-failing-timeout-waiting-for-provisioned-nodes-to-become-available? should it be added to program call? | 12:08 |
| ysandeep | rlandy|ruck, yes definitely blocking ovb jobs.. steve is working on fix but worth to add in prgm call as blocker. | 12:09 |
| ysandeep | rlandy|ruck: when you have time, review request https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/311566 , needed when we run job with build_container_images: true for rhosp17 on rhel9 | 12:22 |
| ysandeep | pojadhav, hey o/ your sc010 passed, please propose a similiar patch for octavia in upstream for rhel case. | 12:25 |
| pojadhav | ysandeep, ack | 12:28 |
| ysandeep | pojadhav, thanks, please give reference of https://review.opendev.org/c/openstack/octavia/+/816370 in the commit message. | 12:29 |
| pojadhav | ysandeep, sure | 12:30 |
| soniya29 | rlandy|ruck, meeting? | 12:30 |
| frenzy_friday | hey folks, http://10.0.111.235:8080/?orgId=1 - development cockpit | 12:57 |
| ysandeep | marios, rlandy|ruck chandankumar marios Please add https://review.opendev.org/q/topic:undercloud_ffu_wallaby in your next review list regarding undercloud_ffu, I think would be good to merge/review those before jose moves to new team. | 13:00 |
| tosky | arxcruz|ruck: I've seen it once, I'm still not sure about it | 13:01 |
| tosky | so either please wait until I understand it, or not | 13:02 |
| *** amoralej is now known as amoralej|lunch | 13:02 | |
| marios | ysandeep: ack been watching those will do | 13:02 |
| arxcruz|ruck | tosky did you saw Abishek comment why > 0? | 13:02 |
| ysandeep | marios: ++ thanks | 13:03 |
| rlandy|ruck | ysandeep: https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/311566 - +2 - feel free to merge whenever | 13:06 |
| ysandeep | rlandy|ruck, ++ thanks! | 13:11 |
| * dviroel coffee | 13:14 | |
| *** pojadhav is now known as pojadhav|brb | 13:18 | |
| rlandy|ruck | arxcruz|ruck: rcastillo|rover: let's sync after retro | 13:31 |
| arxcruz|ruck | ok | 13:31 |
| dasm | o/ | 13:58 |
| *** amoralej|lunch is now known as amoralej | 14:01 | |
| rlandy|ruck | ysandeep: will touch base with you after rrsync | 14:36 |
| arxcruz|ruck | rlandy|ruck https://review.rdoproject.org/r/c/testproject/+/37919 and https://review.opendev.org/c/openstack/tripleo-quickstart/+/829523 | 14:39 |
| rlandy|ruck | arxcruz|ruck: rcastillo|rover: [TripleO] Gate blocker - opstools repo openstack-discuss | 14:42 |
| marios | rlandy|ruck: oh dear... related to https://review.opendev.org/c/openstack/tripleo-quickstart/+/828126 ? | 14:44 |
| rlandy|ruck | marios: ??? | 14:44 |
| marios | 16:42 < rlandy|ruck> arxcruz|ruck: rcastillo|rover: [TripleO] Gate blocker - opstools repo openstack-discuss | 14:44 |
| marios | rlandy|ruck: ^^ | 14:44 |
| rlandy|ruck | marios: sorry - no | 14:44 |
| rlandy|ruck | just copied an example email of gate blocker | 14:45 |
| rlandy|ruck | for the ansible failure now | 14:45 |
| marios | rlandy|ruck: k sorry... dno't think i can handle another "you broke all the things" sorry for the noise | 14:45 |
| marios | ;) | 14:45 |
| rlandy|ruck | https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/CentOS-8/promotion-testing-hash-victoria.yml | 14:45 |
| rlandy|ruck | arxcruz|ruck: ^^ | 14:45 |
| rlandy|ruck | https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/CentOS-8/wallaby.yml | 14:47 |
| *** pojadhav|brb is now known as pojadhav | 14:51 | |
| rlandy|ruck | ysandeep: https://trello.com/c/Rm90mCHN/2350-cixbz2053378osp162rhel8periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-162-is-failing-on-tempest-test-testcreat - FTBS are cleared now | 15:02 |
| rlandy|ruck | is this still an issue? | 15:02 |
| ysandeep | rlandy|ruck, ftbfs cleared today - next run of tripleo component will promote the fix. | 15:03 |
| rlandy|ruck | sshnaidm: hey - can you join us on rr sync - re: ansible | 15:06 |
| rlandy|ruck | https://meet.google.com/nft-kdnz-ivn?authuser=0 | 15:06 |
| sshnaidm | (ansible >= 2.8.0 or ansible-core >= 2.11) | 15:09 |
| rlandy|ruck | amoralej: hello ... can you help us out with ansible/repo versions? | 15:17 |
| rlandy|ruck | we're on https://meet.google.com/nft-kdnz-ivn?authuser=0 | 15:17 |
| * ysandeep exploring reparenting content-provider job: https://review.opendev.org/c/openstack/tripleo-ci/+/829553/3/zuul.d/content-provider.yaml#12 , We need a job to reuse in downstream without branches and without redefining everything in downstream. | 15:18 | |
| rlandy|ruck | ysandeep: ^^ parent one level up | 15:18 |
| rlandy|ruck | on rr sync | 15:18 |
| rlandy|ruck | can help afterwards | 15:18 |
| ysandeep | ack, I have a hard stop in 20 mins for today, can chat tomorrow if you need longer. | 15:19 |
| rlandy|ruck | ysandeep: have to join cix | 15:19 |
| rlandy|ruck | pls go ahead with fix | 15:19 |
| *** pojadhav is now known as pojadhavdinner | 15:21 | |
| *** pojadhavdinner is now known as pojadhav|dinner | 15:21 | |
| tosky | arxcruz|ruck: no, I still have a question, it's not clear to me and my previous question wasn't fully answered, hopefully now my question is clear | 15:25 |
| chandankumar | see ya! | 15:25 |
| arxcruz|ruck | tosky ok | 15:25 |
| arxcruz|ruck | checking | 15:25 |
| *** ysandeep is now known as ysandeep|out | 15:26 | |
| ysandeep|out | rlandy|ruck, will try to join later today to discuss.. downstream gerrit ultimate slow- my fix is stuck on push | 15:27 |
| *** dviroel is now known as dviroel|lunch | 15:31 | |
| jm1 | rlandy|ruck: who will create the Sprint 58 board? | 15:41 |
| jm1 | rlandy|ruck: can i add a task (with sub-tasks) for reproducer? | 15:42 |
| rlandy|ruck | jm1: usually bhagyashri | 15:42 |
| rlandy|ruck | but she is out | 15:42 |
| rlandy|ruck | so ysandeep|out and I will | 15:42 |
| rlandy|ruck | jm1: at planning meeting | 15:43 |
| rlandy|ruck | we will ask for new items | 15:43 |
| rlandy|ruck | add them to the planning hackmd | 15:43 |
| jm1 | ok | 15:45 |
| amoralej | rlandy|ruck, sorry i was in other call | 15:50 |
| jm1 | rlandy|ruck: we don't use Jira's backlog feature, do we? | 15:50 |
| rlandy|ruck | amoralej: np - we're all on CIX | 15:51 |
| rlandy|ruck | jm1: we do | 15:51 |
| rlandy|ruck | amoralej: we spoke about it on the CIX call | 15:51 |
| rlandy|ruck | apevec pinged you on the card | 15:51 |
| amoralej | lemme check | 15:51 |
| amoralej | omg ansible-core is pushed to centos | 15:54 |
| amoralej | i got it now | 15:54 |
| amoralej | rlandy|ruck, sshnaidm should tripleo be compatible with ansible 2.12? | 15:55 |
| rlandy|ruck | amoralej: "omg ansible-core is pushed to centos" :) and breaking the world | 15:56 |
| sshnaidm | amoralej, in general I don't see why not in wallaby, but train is no-go for 2.12 | 15:56 |
| amoralej | so from wallaby, it should be fine to go with ansible-core | 15:56 |
| tosky | arxcruz|ruck: this is never going to end >.< another question added | 15:56 |
| sshnaidm | amoralej, although I don't understand why this error happens: https://logserver.rdoproject.org/19/37919/11/check/periodic-tripleo-centos-8-buildimage-ironic-python-agent-victoria/d5973ab/job-output.txt | 15:56 |
| amoralej | but for previous versions we should avoid using it | 15:56 |
| arxcruz|ruck | tosky lol | 15:56 |
| sshnaidm | amoralej, in spec we have: (ansible >= 2.8.0 or ansible-core >= 2.11) https://review.rdoproject.org/r/plugins/gitiles/openstack/ansible-collections-openstack-distgit/+/refs/heads/rpm-master/ansible-collections-openstack.spec | 15:57 |
| amoralej | sshnaidm, not in that build | 15:57 |
| amoralej | that's victoria branch | 15:57 |
| amoralej | lemme check | 15:57 |
| arxcruz|ruck | amoralej although it's victoria branch, it also affects other branches | 15:58 |
| amoralej | https://github.com/rdo-packages/ansible-collections-openstack-distgit/blob/victoria-rdo/ansible-collections-openstack.spec#L22 | 15:58 |
| amoralej | sshnaidm, ^ | 15:58 |
| sshnaidm | amoralej, aha! I looked at wrong spec.. | 15:58 |
| amoralej | if we want to support that we need to cherry-pick patches | 15:58 |
| amoralej | but the question is if we want that | 15:59 |
| amoralej | or we want to ignore ansible-core | 15:59 |
| amoralej | mmm | 15:59 |
| amoralej | this will be a mess... | 15:59 |
| sshnaidm | amoralej, for train we must ignore it | 15:59 |
| amoralej | i guess train ussuri and victoria? | 16:00 |
| amoralej | and get it working from wallaby on? | 16:00 |
| amoralej | wdyt? | 16:00 |
| sshnaidm | amoralej, for wallaby tbh I'd keep compatible with ansible-core, it's easier to have it one ansible version on both c8 and c9 | 16:00 |
| amoralej | yeah, i know you wanted that :) | 16:00 |
| *** ysandeep|out is now known as ysandeep | 16:00 | |
| amoralej | i'm thinking in creating a version of ansible that obsoletes ansible-core | 16:00 |
| amoralej | and push it for train, u, v | 16:00 |
| ysandeep | rlandy|ruck, hey o/ are you free for discussion after cix call? | 16:01 |
| amoralej | and from wallaby, we'll need to unbreak deps | 16:01 |
| rlandy|ruck | ysandeep: on 1-1 | 16:01 |
| ysandeep | rlandy|ruck, fyi.. https://review.opendev.org/c/openstack/tripleo-ci/+/829553/ and https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/311760/1/zuul.d/jobs.yaml | 16:01 |
| rlandy|ruck | can meet afterwards | 16:01 |
| amoralej | mmm | 16:01 |
| sshnaidm | amoralej, version of ansible 2.9 you mean? that obsoletes ansible-core | 16:01 |
| amoralej | yes | 16:01 |
| amoralej | bumping epoch | 16:02 |
| amoralej | dunno if it will work | 16:02 |
| amoralej | as ansible-core has obsoletes on ansible | 16:02 |
| amoralej | iirc | 16:02 |
| amoralej | i need to test | 16:02 |
| sshnaidm | well, whatever works is fine | 16:02 |
| amoralej | sshnaidm, rlandy|ruck what's the impact of the issue? | 16:02 |
| amoralej | what is currently broken? | 16:03 |
| amoralej | check/gate for all tripleo? or just specific jobs? | 16:03 |
| sshnaidm | most of c8 | 16:03 |
| jm1 | rlandy|ruck: @backlog: was confused because our backlog only has old tasks and then again we have this ci planning doc on hackmd.io. | 16:03 |
| amoralej | at least master is fine, right? | 16:03 |
| sshnaidm | yeah, check, gate, promotions | 16:03 |
| sshnaidm | yeah, master is no longer c8 | 16:03 |
| ysandeep | rlandy|ruck: Its really late for me, Lets discuss tomorrow's morning, ^^ patches are up - https://review.opendev.org/c/openstack/tripleo-ci/+/829553 and https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/311760 , Please feel free to go ahead with alternate approaches. | 16:04 |
| *** ysandeep is now known as ysandeep|out | 16:04 | |
| sshnaidm | as a hack we can exclude ansible-core in dnf.conf for t, u, v releases | 16:06 |
| sshnaidm | but more robust solution would be better.. | 16:06 |
| rlandy|ruck | amoralej: sshnaidm: sorry - meeting ... wrt what is broken | 16:18 |
| rlandy|ruck | check/gate on tox molecule and all images builds | 16:19 |
| rlandy|ruck | and all c8 promotion lines | 16:19 |
| rlandy|ruck | we can go down the exclude route | 16:19 |
| rlandy|ruck | but it's not pretty | 16:19 |
| amoralej | i'm working in a new ansible build | 16:20 |
| amoralej | that obsoletes ansible-core | 16:20 |
| amoralej | i'm not sure if it will work, tbh | 16:20 |
| frenzy_friday | hey rlandy|ruck, marios, chandankumar, ysandeep|out : Updated some stuff from the comments in https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/39424 . Pls take a look when you get some time. Thanks! | 16:21 |
| marios | frenzy_friday: ack | 16:21 |
| rlandy|ruck | amoralej: we can start working on the excludes patches | 16:21 |
| frenzy_friday | ^ re: cockpit c9 | 16:21 |
| *** rlandy|ruck is now known as rlandy|ruck|mtg | 16:22 | |
| amoralej | sshnaidm, may i see logs for some issue in train? | 16:24 |
| amoralej | i just reproduced.. | 16:25 |
| sshnaidm | arxcruz|ruck, do we have same for train? ^^ | 16:26 |
| arxcruz|ruck | sshnaidm checking | 16:26 |
| amoralej | sshnaidm, don't worry, i think i got it | 16:26 |
| sshnaidm | great | 16:27 |
| sshnaidm | same issue? | 16:27 |
| amoralej | i could do a minimal reproducer | 16:27 |
| amoralej | now i'm trying to fix | 16:27 |
| arxcruz|ruck | sshnaidm we don't, last run was yesterday and it passes, but it might be before the package shows up | 16:27 |
| sshnaidm | arxcruz|ruck, ack, I think amoralej is good now.. | 16:28 |
| arxcruz|ruck | sshnaidm https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_dd6/829182/1/check/tripleo-buildimage-overcloud-full-centos-8-train/dd6a630/rpm_qa.txt | 16:28 |
| arxcruz|ruck | it was before ansible-core be available | 16:28 |
| amoralej | what would be useful is to prepare a patch to run those jobs with an additional repo | 16:28 |
| amoralej | ah, i think in tripleo-ci with a delorean-deps-ansible.repo and a depends-on somewhere? | 16:29 |
| arxcruz|ruck | amoralej tripleo-common trigger these jobs | 16:31 |
| *** rlandy|ruck|mtg is now known as rlandy|ruck | 16:37 | |
| rlandy|ruck | amoralej: triple-common that touches image-yaml example: https://review.opendev.org/c/openstack/tripleo-common/+/829000/ | 16:38 |
| rlandy|ruck | amoralej: ^^ | 16:38 |
| rlandy|ruck | so wallaby branch on tripleo-common | 16:38 |
| rlandy|ruck | we can hack up a test patch if that helps | 16:38 |
| amoralej | rlandy|ruck, yep, i'm preparing a test patch in testproject | 16:39 |
| amoralej | with python-pyvmomi-6.7.1-7.el8s | 16:39 |
| amoralej | i meant with periodic-tripleo-centos-8-buildimage-ironic-python-agent-victoria FAILURE | 16:39 |
| *** dviroel|lunch is now known as dviroel | 16:39 | |
| rlandy|ruck | arxcruz|ruck had a testproject | 16:42 |
| rlandy|ruck | getting | 16:42 |
| rlandy|ruck | https://review.rdoproject.org/r/c/testproject/+/37919 | 16:42 |
| amoralej | yeah, i'm using that as reference | 16:43 |
| amoralej | it's fine to run that periodic job, right? | 16:43 |
| rlandy|ruck | yep | 16:46 |
| rlandy|ruck | ysandeep|out: thank you - looking at patches | 16:46 |
| rlandy|ruck | marios: hey ... we hit a branch issue in downstream ... | 16:51 |
| rlandy|ruck | ysandeep|out has these two patches | 16:51 |
| rlandy|ruck | I think this is a reasonable approach ... | 16:51 |
| rlandy|ruck | can you take a look at https://review.opendev.org/c/openstack/tripleo-ci/+/829553/3/zuul.d/content-provider.yaml | 16:51 |
| rlandy|ruck | and https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/311760 | 16:52 |
| rlandy|ruck | should be ok | 16:52 |
| marios | rlandy|ruck: ok will do (have it in my list already sandeep pinged earlier about it) | 16:52 |
| marios | rlandy|ruck: but not immediatly | 16:52 |
| marios | rlandy|ruck: will be for breakfast ;) | 16:52 |
| marios | rlandy|ruck: unless there is an emergency | 16:52 |
| marios | rlandy|ruck: in which case i'll fetch espresso | 16:52 |
| marios | rlandy|ruck: ack i mean that is the pattern we currently have with the scenario standalones more/less | 16:53 |
| rlandy|ruck | marios: ack - we can't merge anything right now anyways | 16:54 |
| rlandy|ruck | tomorrow is fine | 16:55 |
| marios | rlandy|ruck: k might be worth making tox-molecule non voting for a bit? it is only for ci repos anyway | 16:55 |
| marios | rlandy|ruck: but anyway will check those in my morning | 16:55 |
| rlandy|ruck | marios: waiting on amoralej's test results | 16:55 |
| rlandy|ruck | it won't help image build failures | 16:56 |
| rlandy|ruck | would rather address the actual issue | 16:56 |
| marios | ah didn't realise it hits that too... :/ | 16:56 |
| amoralej | rlandy|ruck, arxcruz|ruck https://review.rdoproject.org/r/c/testproject/+/39501 with https://review.opendev.org/c/openstack/tripleo-quickstart/+/829579 | 17:02 |
| amoralej | just for testing | 17:02 |
| amoralej | in my local test, it worked | 17:03 |
| arxcruz|ruck | amoralej if it fails, we will do this: | 17:03 |
| arxcruz|ruck | https://lescinskas.lt/assets/img/posts/2019/docker-works-on-my-machine.jpg | 17:04 |
| *** marios is now known as marios|out | 17:04 | |
| amoralej | lol | 17:06 |
| *** pojadhav|dinner is now known as pojadhav | 17:08 | |
| rlandy|ruck | lol | 17:13 |
| rlandy|ruck | https://review.rdoproject.org/zuul/stream/8816c934c97947b59f4f495093cef0be?logfile=console.log looking good | 17:24 |
| rlandy|ruck | rcastillo|rover: you busy atm? | 17:25 |
| rcastillo|rover | rlandy|ruck: not particularly, what's up? | 17:26 |
| rlandy|ruck | rcastillo|rover: can you check into fs001 failures in check? https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001 | 17:26 |
| rlandy|ruck | arxcruz|ruck created a LP bug | 17:27 |
| rlandy|ruck | failure is in deployment | 17:27 |
| rcastillo|rover | yeah I'll look at it | 17:27 |
| rlandy|ruck | and looks like it's hitting all OVB in check | 17:27 |
| * rlandy|ruck gets bug | 17:27 | |
| rcastillo|rover | it's the ssl issue? | 17:28 |
| rlandy|ruck | rcastillo|rover: https://bugs.launchpad.net/tripleo/+bug/1961056 | 17:28 |
| rlandy|ruck | tderr_lines": ["C = US, ST = NC, L = Raleigh, O = Red Hat, OU = OOOQ, CN = 10.0.0.5", "error 20 at 0 depth lookup: unable to get local issuer certificate", "error stdin: verification failed"], "stdout": "", | 17:29 |
| rlandy|ruck | ack | 17:29 |
| dviroel | ah, got this error in my compose testing, was wondering if was my fault :P | 17:29 |
| rlandy|ruck | rcastillo|rover: dviroel: https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/deployment/haproxy/haproxy-pacemaker-puppet.yaml#L301 | 17:30 |
| rlandy|ruck | checking blame here | 17:30 |
| rlandy|ruck | https://opendev.org/openstack/tripleo-heat-templates/commit/d5701e6ceb216618077374a6297a99af31776ed7 | 17:30 |
| dviroel | 2w ago - busted | 17:31 |
| rlandy|ruck | lol | 17:31 |
| rlandy|ruck | let's check the merge date that | 17:31 |
| rlandy|ruck | then ping dvd | 17:31 |
| rlandy|ruck | sorry dhill | 17:31 |
| rcastillo|rover | 16 hours ago | 17:32 |
| *** jpena is now known as jpena|off | 17:32 | |
| rlandy|ruck | https://opendev.org/openstack/tripleo-heat-templates/commit/a445fbd02b7ddc583b8a88d4d595edb67fac9fd5 | 17:32 |
| rlandy|ruck | yep - as dviroel says - busted | 17:32 |
| rlandy|ruck | rcastillo|rover: dviroel: chatting on #rhos-dev | 17:33 |
| rlandy|ruck | arxcruz|ruck: fyi | 17:36 |
| rlandy|ruck | we got this one | 17:36 |
| amoralej | it's going to take us some time to fix all the deps for cs8 in wallaby to support ansible-core | 17:37 |
| rlandy|ruck | amoralej: k - should we start with the excludes? | 17:38 |
| amoralej | i'm thinking in applying the same fix for wallaby as for previous to work with ansible-2.9, then update the deps and finally remove the fixed ansible | 17:38 |
| amoralej | no, i think the obsoletes is working fine | 17:38 |
| amoralej | at least it passed the point where it failed before | 17:38 |
| amoralej | but for wallaby i was thinking if we may be able to just fix the issues with ansible-core | 17:38 |
| amoralej | but needs time | 17:38 |
| rlandy|ruck | understand | 17:38 |
| amoralej | rlandy|ruck, https://review.rdoproject.org/r/c/testproject/+/39501 | 17:39 |
| amoralej | that passed | 17:39 |
| rlandy|ruck | amoralej++ | 17:39 |
| amoralej | so, i'll push that new ansible | 17:39 |
| amoralej | to the deps repo | 17:39 |
| amoralej | i'm doing a final build, not scratch | 17:39 |
| rlandy|ruck | amoralej: ok - so we don't need any change on our side? | 17:39 |
| amoralej | hopefully not | 17:39 |
| rlandy|ruck | yay = so excited | 17:43 |
| rlandy|ruck | arxcruz|ruck: rcastillo|rover: anything come out of the CIX call? | 18:04 |
| rcastillo|rover | rlandy|ruck: nothing major | 18:08 |
| rlandy|ruck | k - looking through cards missed | 18:09 |
| rlandy|ruck | lunch brb | 18:13 |
| pojadhav | rlandy|ruck, could you please help me with this patch's issue when you free : https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/311314 | 18:25 |
| ade_lee | dviroel, Is zuul hanging for you too? I was trying to get details about the cs9 job .. | 18:26 |
| pojadhav | it could not able to locate tripleo-ipa role in downstream, tried some changes but no luck :( | 18:26 |
| ade_lee | dviroel, seems like it got further .. but .. | 18:26 |
| pojadhav | rlandy|ruck, please leave some notes, I will pickup from that.. thanks! | 18:27 |
| *** pojadhav is now known as pojadhav|out | 18:27 | |
| dviroel | ade_lee: hey, yeah, i saw at least one standalone passing :) | 18:31 |
| rlandy|ruck | pojadhav|out: will look | 18:43 |
| ade_lee | dviroel, looks like its back | 18:44 |
| ade_lee | dviroel, I'm poking through the various scenarios looking for cause of failure -- nothing obvious yet - feel free to look if you can and see if you see anything intersting | 18:52 |
| dviroel | ade_lee: ack, i will | 18:53 |
| ade_lee | thanks | 18:56 |
| *** rlandy|ruck is now known as rlandy|ruck|mtg | 19:00 | |
| ade_lee | dviroel, so - I'm looking at scenario 3, and it seems like the deployment succeeded, but some thing failed in tempest tests, and no file to determine which tests failed .. | 19:12 |
| ade_lee | https://zuul.opendev.org/t/openstack/build/c13add79d8484c3abecf9ec17b43f830/logs | 19:12 |
| ade_lee | dviroel, maybe you'll see something I don't | 19:12 |
| dviroel | ade_lee: https://fc48307980eaad02895a-97c7700d8ad654a1c78ac6d1f50b424d.ssl.cf5.rackcdn.com/824479/8/check/tripleo-ci-centos-9-scenario003-standalone/c13add7/logs/undercloud/var/log/tempest/stestr_results.html | 19:13 |
| dviroel | "cryptography.exceptions.UnsupportedAlgorithm: sha1 is not supported by this backend for RSA signing. | 19:14 |
| dviroel | " | 19:14 |
| ade_lee | dviroel, ah, gotta know where to look :) | 19:14 |
| ade_lee | dviroel, cool - I can look into this -- interesting - we should be using ecdsa instead of sha-1 here | 19:16 |
| dviroel | ade_lee: scenario001 tests failing in the same issue (tempest/lib/common/ssh.py", line 240, in test_connection_auth) | 19:18 |
| ade_lee | dviroel, interesting - coz ecdsa is set -- https://zuul.opendev.org/t/openstack/build/c13add79d8484c3abecf9ec17b43f830/log/logs/undercloud/home/zuul/tempest/etc/tempest.conf | 19:19 |
| dviroel | ade_lee: hum, are we missing any validations patch with depends-on? | 19:21 |
| dviroel | ade_lee: that didn't merge yet? | 19:21 |
| ade_lee | dviroel, no - that should have merged .. | 19:22 |
| ade_lee | is the version of tempest in the build old? | 19:22 |
| ade_lee | ah .. typo maybe .. | 19:24 |
| ade_lee | ssh_key_type | 19:24 |
| ade_lee | not ssh_keytype | 19:25 |
| dviroel | lol | 19:25 |
| ade_lee | dviroel, https://opendev.org/openstack/tempest/src/branch/master/tempest/config.py#L973-L976 | 19:25 |
| ade_lee | dviroel, ok - fixing -- maybe this will fix a bunch of them .. | 19:26 |
| dviroel | ade_lee: yes. Scenario-010 failures: | 19:28 |
| dviroel | FATAL | Generating certificate authority certificate | standalone | error={\"changed\": true, \"cmd\": \"openssl req -x509 -passin env:CA_PASSPHRASE -new -nodes -key /tmp/octavia-ssl/private/cakey.pem -subj \\\"/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com\\\" -days 18250 -config /tmp/octavia-ssl/openssl.cnf -out /tmp/octavia-ssl/ca_01.pem\\n\", \"delta\": \"0:00:00.027038\", | 19:28 |
| dviroel | \"end\": \"2022-02-15 20:43:59.411864\", \"msg\": \"non-zero return code\", \"rc\": 1, \"start\": \"2022-02-15 20:43:59.384826\", \"stderr\": \"Error configuring OpenSSL modules\\n002CA458B67F0000:error:0700006D:configuration file routines:module_run:module initialization error:crypto/conf/conf_mod.c:243:module=providers, value=provider_sect retcode=-1 \", \"stderr_lines\": [\"Error configuring OpenSSL modules\", | 19:28 |
| dviroel | \"002CA458B67F0000:error:0700006D:configuration file routines:module_run:module initialization error:crypto/conf/conf_mod.c:243:module=providers, value=provider_sect retcode=-1 \"], \"stdout\": \"\", \"stdout_lines\": []}\n\u001b[Ke30=\u001b[4D\u001b[K\u001b[KeyJ1dWlkIjogIjg3N2E2YzNiLWQxODQtNDZmZS04NmRkLTAyNjViNWJi | 19:28 |
| ade_lee | dviroel, looking .. | 19:29 |
| johnsom | ade_lee It kills me that tripleo is still using the old devstack script setup for those certs (it's an open BZ). I bet there is some scripting in tripleo that needs updated like you did for the devstack scripts. | 19:32 |
| johnsom | ade_lee I.e. https://review.opendev.org/c/openstack/octavia/+/827898, I think they copied one of those scripts. | 19:33 |
| ade_lee | johnsom, yeah I suspect its something like that | 19:34 |
| ade_lee | dviroel, where are the failures? | 19:35 |
| dviroel | ade_lee: standalone_deploy.log | 19:35 |
| * dviroel gets link | 19:35 | |
| dviroel | https://4c52b112c1c334d74fb7-de57e41b0b6434f33a0d8e3ede716241.ssl.cf1.rackcdn.com/824479/8/check/tripleo-ci-centos-9-scenario010-ovn-provider-standalone/e8c5afd/logs/undercloud/home/zuul/standalone_deploy.log | 19:35 |
| ade_lee | thanks | 19:35 |
| ade_lee | johnsom, https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/octavia_overcloud_config/tasks/certs_gen.yml#L86 | 19:38 |
| ade_lee | dviroel, johnsom cool - I'll put together a patch for this. In the meantime, the job should be rerunning with ecdsa set properly (hopefully) | 19:43 |
| dviroel | scenario004 - rabbitmq | 19:43 |
| dviroel | ade_lee: & | 19:43 |
| dviroel | ade_lee: ^^] | 19:43 |
| dviroel | ade_lee: 2022-02-15 21:33:42.481857 | | WARNING | ERROR: Can't run container rabbitmq_wait_bundle | 19:44 |
| dviroel | you remember something with rabbitmq before? | 19:44 |
| ade_lee | dviroel, link? | 19:44 |
| dviroel | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_92b/824479/8/check/tripleo-ci-centos-9-scenario004-standalone/92bc7ce/logs/undercloud/home/zuul/standalone_deploy.log | 19:44 |
| ade_lee | dviroel, yeah -- we need the right version to make sure we can enable fips .. | 19:44 |
| ade_lee | dviroel, whats the version of rabbit there? | 19:44 |
| * dviroel looks | 19:45 | |
| dviroel | ade_lee: rabbitmq-server-3.9.10-1.el9s.x86_64 | 19:47 |
| dviroel | https://faf2bfa5943954d933b7-43bef2b011f595598cf89d565fc6c894.ssl.cf1.rackcdn.com/824479/8/check/tripleo-ci-centos-9-content-provider/737da2e/logs/container-builds/4802728d-d962-4efa-9169-0c00c3349dae/base/rabbitmq/rabbitmq-build.log | 19:47 |
| ade_lee | dviroel, the version that was needed on centos-8 was 3.9.8-3 | 19:49 |
| ade_lee | dviroel, of course that was a special build -- what needed to be ported there may or may not be in there. | 19:50 |
| ade_lee | dviroel, gotta check the container log for that container | 19:50 |
| dviroel | ade_lee: ack - yeah, for these services we need to build the containers with packages that we want | 19:51 |
| ade_lee | dviroel, lets see if the ecdsa thing works and fixes this -- if not, we can pull ecks in | 20:00 |
| ade_lee | dviroel, getting kids from school - back in a bit | 20:01 |
| dviroel | ade_lee: ok | 20:06 |
| *** rlandy|ruck|mtg is now known as rlandy|ruck | 20:14 | |
| rlandy|ruck | pojadhav|out: left notes on review | 20:26 |
| *** rlandy|ruck is now known as rlandy|ruck|mtg | 20:43 | |
| *** rlandy|ruck|mtg is now known as rlandy|ruck | 21:04 | |
| amoralej | rlandy|ruck, commits in train and ussuri are merged | 21:05 |
| rlandy|ruck | amoralej++ thank you | 21:05 |
| amoralej | is it working? | 21:05 |
| amoralej | do we have any example to recheck? | 21:06 |
| amoralej | i'm also merging wallaby and victoria | 21:08 |
| amoralej | oooq job in wallaby gate passed, so i guess it's safe | 21:08 |
| *** amoralej is now known as amoralej|off | 21:10 | |
| rcastillo|rover | thanks amoralej|off | 21:11 |
| rcastillo|rover | rlandy|ruck: I'm rechecking https://review.opendev.org/c/openstack/tripleo-ci/+/829309/ to see if it clears gate | 21:13 |
| rcastillo|rover | also the fs020 wallaby job is still failing :( | 21:13 |
| rlandy|ruck | ok - not fixed | 21:15 |
| * dviroel is out /o/ | 21:26 | |
| *** dviroel is now known as dviroel|out | 21:26 | |
| dasm | o/ | 21:27 |
| rlandy|ruck | dasm: sorry missed you | 21:56 |
| rlandy|ruck | chat tomorrow | 21:56 |
| dasm | rlandy|ruck: i'm still around, but just about to leave. I war waiving Doug :) | 21:56 |
| dasm | rlandy|ruck: rbac is running: https://review.rdoproject.org/zuul/status/change/39366,3 | 21:57 |
| dasm | https://review.rdoproject.org/r/c/rdo-jobs/+/39365 | 21:57 |
| dasm | for now i'm keeping it non-voting | 21:57 |
| rlandy|ruck | dasm++ will review | 21:58 |
| rlandy|ruck | sorry - long day | 21:58 |
| rlandy|ruck | lots burning | 21:58 |
| dasm | it identified some places without secure connection | 21:58 |
| dasm | No worries! | 21:58 |
| dasm | Thank you for all what you're doing rlandy|ruck | 21:58 |
| rlandy|ruck | :) | 22:00 |
| * dasm is leaving now. | 22:01 | |
| dasm | Take care rlandy|ruck \o | 22:01 |
| rcastillo|rover | \o dasm | 22:02 |
| rcastillo|rover | rlandy|ruck: should I prepare the exclude patch for the ansible version? | 22:08 |
| rlandy|ruck | rcastillo|rover: amoralej> rlandy|ruck, commits in train and ussuri are merged | 22:08 |
| rlandy|ruck | so some are already merged | 22:08 |
| rlandy|ruck | I think we wait it out | 22:09 |
| amoralej|off | rcastillo|rover, https://review.opendev.org/c/openstack/tripleo-ci/+/829309/ it runs with master? | 22:09 |
| rcastillo|rover | oh yeah, oops | 22:09 |
| amoralej|off | we didn't push the fix to master | 22:09 |
| amoralej|off | lemme send it | 22:09 |
| rlandy|ruck | master yes | 22:09 |
| rlandy|ruck | master is 9 only though | 22:09 |
| rcastillo|rover | yeah tox molecule runs on cs8 on master | 22:09 |
| amoralej|off | i thought master was 9 only | 22:09 |
| rlandy|ruck | except molecule | 22:10 |
| rlandy|ruck | ha | 22:10 |
| rlandy|ruck | if we push that to master, will we mess with 9? | 22:10 |
| rlandy|ruck | in which case we need to rethink where we run molecule tests | 22:11 |
| rlandy|ruck | and switch the node based on branch | 22:11 |
| amoralej|off | rlandy|ruck, no, it will not affect cs9 | 22:11 |
| rlandy|ruck | ok | 22:11 |
| rlandy|ruck | so then yeah, pls push to master | 22:11 |
| rlandy|ruck | our ci repos are branchless | 22:11 |
| rlandy|ruck | so lalways master | 22:12 |
| rlandy|ruck | always | 22:12 |
| amoralej|off | just sent | 22:13 |
| amoralej|off | it will take some time to pass ci | 22:13 |
| rlandy|ruck | ok | 22:13 |
| rlandy|ruck | I think there is an issue with c9 nodes in upstream infra | 22:14 |
| rcastillo|rover | thx | 22:14 |
| rlandy|ruck | content-provider for c9 in retry_limit | 22:14 |
| rlandy|ruck | watching those runs | 22:14 |
| rcastillo|rover | I saw | 22:14 |
| rcastillo|rover | in #rhos-dev | 22:14 |
| rlandy|ruck | rcastillo|rover: https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-content-provider&skip=0 - | 22:16 |
| rcastillo|rover | I'm looking for a running job but none of them are printing logs to stream | 22:17 |
| rlandy|ruck | rcastillo|rover: think we need to ping our friends in #opendev ... no logs | 22:17 |
| rlandy|ruck | rcastillo|rover: can you do that? | 22:17 |
| rlandy|ruck | impossible to tell | 22:17 |
| rlandy|ruck | we can't see any output | 22:17 |
| rcastillo|rover | yeah | 22:18 |
| rlandy|ruck | you can give them the link above | 22:19 |
| rcastillo|rover | yup, dome | 22:20 |
| rcastillo|rover | done* | 22:20 |
| rlandy|ruck | rcastillo|rover++ thank you | 22:21 |
| ade_lee | dviroel|out, things looking better with the ssh_key_type change -- standalone,1,3,7 now passing | 22:35 |
| rlandy|ruck | rcastillo|rover: hmmm ... we need to leave some notes for arxcruz|ruck and rest of the team | 23:03 |
| rlandy|ruck | maybe LP bug | 23:03 |
| rlandy|ruck | with chat details? | 23:03 |
| rlandy|ruck | so they know why we have no jobs starting | 23:03 |
| rcastillo|rover | yeah this doesn't seem like a quick fix | 23:03 |
| rlandy|ruck | for c9 | 23:03 |
| rlandy|ruck | yeah - trouble | 23:03 |
| rlandy|ruck | would you mind opening that bug? | 23:04 |
| rcastillo|rover | sure, I'll write a note in the gchat and file the bug | 23:04 |
| rlandy|ruck | yeah | 23:04 |
| rlandy|ruck | sounds good | 23:04 |
| rlandy|ruck | so we have some trace | 23:04 |
| rlandy|ruck | DF devs watch our bugs so they will know | 23:04 |
| rcastillo|rover | bug: https://bugs.launchpad.net/tripleo/+bug/1961122 | 23:21 |
| rcastillo|rover | rlandy|ruck: left a note in gchat, lmk if I missed anything | 23:21 |
| rcastillo|rover | rlandy|ruck: seems like the node issue is fixed now | 23:29 |
| rlandy|ruck | rcastillo|rover: thank you | 23:43 |
| rcastillo|rover | rlandy|ruck: anyway, leaving for today. Thanks for all the help today. Have a good night | 23:43 |
| rcastillo|rover | looking forward to friday :) | 23:44 |
| rlandy|ruck | I'm sure | 23:44 |
| rlandy|ruck | good night | 23:44 |
| *** rlandy|ruck is now known as rlandy|out | 23:56 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
