| akahat|ruck | o/ | 03:11 |
|---|---|---|
| *** yadnesh|away is now known as yadnesh | 04:37 | |
| marios | o/ | 06:00 |
| soniya29|rover | marios, hello | 06:49 |
| soniya29|rover | 17 rhel9 jobs are getting skipped, i think we need to rekick the line | 06:49 |
| marios | soniya29|rover: looking | 06:50 |
| marios | soniya29|rover: which are you referring to? | 06:52 |
| marios | soniya29|rover: can you give me link pleas | 06:52 |
| marios | soniya29|rover: this? https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=openstack-periodic-integration-rhos-17-rhel9 | 06:52 |
| soniya29|rover | marios, wait | 06:53 |
| soniya29|rover | marios, https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=openstack-periodic-integration-rhos-17-rhel9&result=SKIPPED&skip=0 | 06:55 |
| soniya29|rover | marios, build push container has failed | 06:55 |
| soniya29|rover | if you see today's run only for 17 rhel9 there are just 5 jobs ran among which build push failed | 06:56 |
| marios | soniya29|rover: please dig/check why container failed there https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/buildset/e25b2193690a4ff1adc7a37de4de2898 | 07:01 |
| * soniya29|rover looking into it | 07:01 | |
| marios | soniya29|rover: then rekick line if you think it will help/not a bug ^^ | 07:01 |
| marios | soniya29|rover: let me know if you need help rekick we can jump on phone in a bit | 07:01 |
| marios | soniya29|rover: but first lets see why it failed | 07:01 |
| soniya29|rover | marios, yupp..just a min | 07:02 |
| soniya29|rover | marios: testing here - https://code.engineering.redhat.com/gerrit/c/testproject/+/435331 to see if the failure is consistent | 07:05 |
| soniya29|rover | so while executing 'build-containers' task the build_report.py throws assertionerror for all the containers | 07:16 |
| marios | soniya29|rover: looking | 07:16 |
| soniya29|rover | marios, http://pastebin.test.redhat.com/1081735 | 07:16 |
| marios | /bin/sh: error while loading shared libraries: libc.so.6: cannot change memory protections | 07:17 |
| marios | hmmm that is the error ^^ | 07:17 |
| marios | soniya29|rover: join me? https://meet.google.com/ydq-xxhd-xjp | 07:20 |
| *** ysandeep|out is now known as ysandeep | 07:25 | |
| marios | 09:17 < marios> /bin/sh: error while loading shared libraries: libc.so.6: cannot change memory protections | 07:25 |
| marios | soniya29|rover: /bin/sh: error while loading shared libraries: libc.so.6: cannot change memory protections https://sf.hosted.upshift.rdu2.redhat.com/logs/31/435331/3/check/periodic-tripleo-build-containers-ubi-9-internal-rhel-9-build-push-upload-rhos-17/62d0f75/logs/container-builds/0b23ff42-187a-4425-abd6-647a263ba79a/base/base-build.log | 07:29 |
| marios | same again | 07:29 |
| *** akahat|ruck is now known as akahat|ruck|lunch | 08:14 | |
| *** jpena|off is now known as jpena | 08:29 | |
| *** yadnesh is now known as yadnesh|afk | 08:34 | |
| *** yadnesh|afk is now known as yadnesh | 08:53 | |
| *** amoralej|off is now known as amoralej | 09:10 | |
| *** akahat|ruck|lunch is now known as akahat|ruck | 09:22 | |
| *** soniya29|rover is now known as soniya29|rover|lunch | 09:29 | |
| *** ysandeep is now known as ysandeep|afk | 10:08 | |
| marios | akahat|ruck: added tags to bug we need it on cix board | 10:11 |
| akahat|ruck | marios, added. You might need refresh. | 10:12 |
| *** soniya29|rover|lunch is now known as soniya29|rover | 10:16 | |
| marios | akahat|ruck: ack | 10:22 |
| soniya29|rover | marios, akahat|ruck: i have doctor's appointment now. I will be back by an hour | 10:27 |
| akahat|ruck | soniya29|rover, okay. | 10:28 |
| akahat|ruck | soniya29|rover, is everything okay in downstream? | 10:28 |
| soniya29|rover | i have reported two new bugzilla'a:- https://bugzilla.redhat.com/show_bug.cgi?id=2143572, https://bugzilla.redhat.com/show_bug.cgi?id=2143572 | 10:28 |
| soniya29|rover | we have issue in 17-1/17/16.2 components for which i have filed the bug and raised cix | 10:28 |
| soniya29|rover | fyi akahat|ruck | 10:29 |
| *** soniya29|rover is now known as soniya29|rover|afk | 10:29 | |
| akahat|ruck | soniya29|rover|afk, okay. | 10:29 |
| * soniya29|rover|afk cancelled the appointment | 10:35 | |
| *** soniya29|rover|afk is now known as soniya29|rover | 10:35 | |
| frenzy_friday | hey pojadhav , hackmd for 17.1 on 9.1 (i am still editting) https://hackmd.io/0ZqodGQsTACAG7ptetSrIg?edit | 10:56 |
| *** dviroel_ is now known as dviroel | 11:09 | |
| pojadhav | frenzy_friday, checking | 11:10 |
| * dviroel o/ | 11:10 | |
| marios | bhagyashris: please help me sanity check when you have time https://review.rdoproject.org/r/c/config/+/46130 | 11:16 |
| marios | bhagyashris: did you post patches for the 8 removal? perhaps we can track them in https://issues.redhat.com/browse/TRIPLEOCI-1271 - or if you prefer create a new task for d/stream cleanup 8 line | 11:17 |
| *** rlandy|out is now known as rlandy | 11:18 | |
| rlandy | marios: akahat|ruck: soniya29|rover: hello - how are things? anything I can help with? | 11:18 |
| rlandy | FreeIPA failed to connect LDAP server | 11:19 |
| rlandy | hmmm ... | 11:19 |
| rlandy | same thing in d/s | 11:19 |
| rlandy | pojadhav: hi - you around? | 11:20 |
| rlandy | pojadhav: need to meet about meetings times | 11:20 |
| bhagyashris | marios, yes i have patches i will add in card | 11:20 |
| pojadhav | rlandy, hey.. i am around | 11:20 |
| soniya29|rover | rlandy, we have two new issues reported today in downstream | 11:20 |
| rlandy | pojadhav: https://meet.google.com/qet-orwv-wcv?pli=1&authuser=0 | 11:21 |
| marios | rlandy: container builds failing both upstream and d/stream | 11:23 |
| marios | rlandy: unrelated though/coincidence | 11:24 |
| marios | rlandy: we have bz/lp/cix akahat|ruck and soniya29|rover on those | 11:24 |
| rlandy | k - I'll check | 11:24 |
| pojadhav | rlandy, done | 11:25 |
| rlandy | pojadhav: the community call? | 11:33 |
| rlandy | that will need to shift as well | 11:33 |
| pojadhav | rlandy, sorry i forgot the new timing for community call.. can you please remind me once. I will shift that also | 11:34 |
| rlandy | pojadhav: hour or hour and a half later - whatever looks like less of a clash | 11:35 |
| rlandy | FIPS meeting will clash with 1 hr | 11:36 |
| rlandy | you can ask doug if he attends | 11:36 |
| pojadhav | rlandy, sure | 11:36 |
| rlandy | pojadhav: another reason to action things ASAP - we forget the details :) | 11:36 |
| rlandy | pojadhav: so 1 hr later unless it's a no show for dviroel | 11:37 |
| pojadhav | rlandy, true.. thanks for reminder.. | 11:37 |
| pojadhav | rlandy, there is also podified call | 11:37 |
| rlandy | pojadhav: podified call is a half hour later | 11:37 |
| pojadhav | rlandy, yep | 11:38 |
| rlandy | soniya29|rover: the meeting invite you sent me for friday clashes with a 1-1 | 11:38 |
| rlandy | pls check my calendar and book another time | 11:38 |
| *** ysandeep|afk is now known as ysandeep | 11:38 | |
| rlandy | I can meet for the second half hour | 11:38 |
| soniya29|rover | ookay | 11:39 |
| pojadhav | dviroel, hey can you give me confirmation about tuesday Fips meeting (whether you attend or not).. I was about to shift community call to half/1 hr later. | 11:39 |
| soniya29|rover | rlandy, i have rescheduled the meeting invite | 11:40 |
| soniya29|rover | rlandy, can we meet for a few? | 11:40 |
| dviroel | pojadhav: yes, I do attend every fips meeting, which is now biweekly | 11:43 |
| rlandy | soniya29|rover: ok - accepted that | 11:43 |
| rlandy | soniya29|rover: ack | 11:44 |
| soniya29|rover | rlandy, meet.google.com/wxs-insd-ccz | 11:44 |
| rlandy | one sec | 11:44 |
| bhagyashris | marios, hey looks good to me both the patches. for sanity check i need this patch https://review.rdoproject.org/r/c/config/+/46130 merged | 11:45 |
| marios | bhagyashris: yeah there is a third patch too i will post a test once we merge that config thanks for looking | 11:46 |
| pojadhav | dviroel, thanks! | 11:47 |
| ysandeep | folks o/ looks like we added c8 upgrade job in C9 master line by mistake: https://github.com/rdo-infra/rdo-jobs/blame/e4bf2ac8f2e449b3ff2bb367250e39eabde08614/zuul.d/component-pipeline-main-centos9.yaml#L134 | 11:48 |
| soniya29|rover | rlandy, https://sf.hosted.upshift.rdu2.redhat.com/logs/31/435331/3/check/periodic-tripleo-build-containers-ubi-9-internal-rhel-9-build-push-upload-rhos-17/62d0f75/logs/container-builds/0b23ff42-187a-4425-abd6-647a263ba79a/base/base-build.log | 11:52 |
| soniya29|rover | rlandy, https://sf.hosted.upshift.rdu2.redhat.com/logs/25/435525/1/check/periodic-tripleo-build-containers-ubi-9-internal-rhel-9-build-push-upload-rhos-17.1/8f2c4e0/logs/container-builds/33fc6781-cfe2-40b9-ad7f-cb433240cfd5/base/base-build.log | 11:52 |
| ysandeep | reviewbot, please add in review list: https://review.rdoproject.org/r/c/rdo-jobs/+/46131 | 11:53 |
| reviewbot | I have added your review to the Review list | 11:53 |
| marios | soniya29|rover: i see you confirmed 17.1 also affected as we discussed (checking bz now) | 12:04 |
| soniya29|rover | marios, yeah | 12:04 |
| marios | weird that both up and d/stream break on the same day for containers :/ | 12:04 |
| marios | rlandy: ysandeep: chandankumar: dviroel: please check when you have time https://review.rdoproject.org/r/c/config/+/46130 need to merge for testing - should only affect mixed-os jobs thanks | 12:09 |
| rlandy | marios: k - just following down the container build issue on doenstream now | 12:10 |
| marios | rlandy: ack in your own time thanks | 12:11 |
| ysandeep | marios, looking | 12:12 |
| ysandeep | reviewbot, please add to review list: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46047 | 12:25 |
| reviewbot | I have added your review to the Review list | 12:25 |
| *** pojadhav is now known as pojadhav|afk | 12:27 | |
| bhagyashris | marios, ack | 12:30 |
| rlandy | akahat|ruck: is there progress on https://bugs.launchpad.net/tripleo/+bug/1996874? | 12:32 |
| * akahat|ruck checking. | 12:33 | |
| marios | rlandy: akahat|ruck: was digging there a bit... not sure that https://review.opendev.org/c/openstack/tripleo-ci/+/864852 included in https://logserver.rdoproject.org/69/41469/37/check/periodic-tripleo-ci-build-containers-ubi-9-quay-push-zed/6b813e3/job-output.txt | 12:37 |
| marios | will look again in a minute | 12:37 |
| marios | but bigger question is did something change yesterday - ask in rhos-ops please akahat|ruck if you did not already ? | 12:37 |
| akahat|ruck | marios, ack | 12:38 |
| *** amoralej is now known as amoralej|lunch | 12:41 | |
| soniya29|rover | rlandy, https://code.engineering.redhat.com/gerrit/c/testproject/+/430523 | 12:46 |
| jm1 | rlandy, pojadhav: the new ci community meeting time is conflicting with aoc meeting. cant we move the community meeting half an hour earlier like we had in the past? | 12:54 |
| * soniya29|rover need a break | 12:55 | |
| * soniya29|rover will be back in 10-15 mins | 12:55 | |
| *** soniya29|rover is now known as soniya29|rover|afk | 12:55 | |
| rlandy | jm1: clash with FIPS meeting | 12:59 |
| *** pojadhav|afk is now known as pojadhav | 13:07 | |
| rlandy | jm1: pojadhav will float the timing proposals at the planning call | 13:11 |
| jm1 | rlandy: its okay, we will move our aoc meeting | 13:12 |
| rlandy | pojadhav: ^^ | 13:12 |
| pojadhav | rlandy, jm1 : ack | 13:13 |
| rlandy | marios: akahat|ruck: k - the two downstream issues are in action - reading back on upstream container issue | 13:13 |
| rlandy | akahat|ruck: soniya29|rover|afk: is there a gate issue? openstack/tripleo-ansible | 13:20 |
| rlandy | that we are tracking? | 13:20 |
| rlandy | marios: ^^? | 13:20 |
| *** soniya29|rover|afk is now known as soniya29|rover | 13:21 | |
| soniya29|rover | rlandy, i see only this https://bugs.launchpad.net/tripleo/+bug/1996612 with upstream | 13:22 |
| soniya29|rover | i am not sure wether it is related | 13:22 |
| rlandy | soniya29|rover: no - that one was pinned | 13:22 |
| marios | rlandy: not aware of one? looking | 13:23 |
| soniya29|rover | rlandy, looking | 13:23 |
| marios | ERROR: could not install deps [-c/home/zuul/src/opendev.org/openstack/requirements/upper-constraints.txt, -r/home/zuul/src/opendev.org/openstack/tripleo-ansible/requirements.txt, -r/home/zuul/src/opendev.org/openstack/tripleo-ansible/test-requirements.txt, -r/home/zuul/src/opendev.org/openstack/tripleo-ansible/ansible-requirements.txt]; v = | 13:23 |
| jm1 | pojadhav: looks like there is an invalid user on our ci community call: jsilvahe@gapps.redhat.com | 13:24 |
| marios | InvocationError('/home/zuul/src/opendev.org/openstack/tripleo-ansible/.tox/py38/bin/python -m pip install -c/home/zuul/src/opendev.org/openstack/requirements/upper-constraints.txt -r/home/zuul/src/opendev.org/openstack/tripleo-ansible/requirements.txt -r/home/zuul/src/opendev.org/openstack/tripleo-ansible/test-requirements.txt -r/home/zuul/src/opendev.org/openstack/tripleo-ansible/ansible-requirements.txt', | 13:24 |
| marios | 1) | 13:24 |
| marios | oops | 13:24 |
| marios | looks like a new issue rlandy akahat|ruck can you pleas check? but it seems only once so far | 13:24 |
| akahat|ruck | marios, ok. | 13:24 |
| marios | akahat|ruck: rlandy: for 863876,2 | 13:24 |
| rlandy | marios: ok - will watch to see if it continues | 13:25 |
| pojadhav | jm1, i can see this email jsilvahe@redhat.com | 13:25 |
| jm1 | pojadhav: maybe just drop it? postmaster sent me this mail: The email account that you tried to reach is disabled | 13:26 |
| rlandy | soniya29|rover: pls see #rhos-delivery | 13:26 |
| soniya29|rover | rlandy, yupp | 13:26 |
| pojadhav | jm1, me too got this mail from postmaster :D | 13:28 |
| *** yadnesh is now known as yadnesh|away | 13:30 | |
| *** ysandeep is now known as ysandeep|afk | 13:41 | |
| akahat|ruck | marios, this 863876,2 causing might be cryptography version pin.. i've saw patches fly by.. | 13:45 |
| * akahat|ruck looking for patches | 13:45 | |
| marios | akahat|ruck: but i don't think the pin merged yesterday | 13:50 |
| *** ysandeep|afk is now known as ysandeep | 14:06 | |
| *** amoralej|lunch is now known as amoralej | 14:22 | |
| *** dasm|off is now known as dasm | 14:22 | |
| dasm | o/ | 14:22 |
| jm1 | dasm: o/ | 14:23 |
| dasm | jm1: o/ i didn't run cli on first patch. i ran on a chain of patches, with all 5, pointing ref_specs as the last patch. It gave me cloud="vexxhost" instead of cloud="rhos_dev_stage" | 14:24 |
| dasm | jm1: i pointed on the first patch what is changed, because other patches don't have this file. | 14:25 |
| dasm | probably i should leave a comment on the very last patch. | 14:25 |
| dasm | jm1: i'm gonna give it a try one more time to see how it behaves. yesterday it was too late for me to deep dive into that. | 14:26 |
| pojadhav | bhagyashris, ysandeep, rcastillo : planning meeting | 14:31 |
| pojadhav | https://hackmd.io/P5sTPMSBREe5kBS_4jDhIg | 14:32 |
| bhagyashris | pojadhav, joining | 14:32 |
| jm1 | dasm: although it worked for me i changed patches 2+ to might fix your issue | 14:51 |
| jm1 | dasm: this is what i changed: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46050/17..18/ci-scripts/infra-setup/inventory/group_vars/all.yml | 14:51 |
| dasm | i would assume it's going to fix that. lemme respin | 14:53 |
| dasm | i just reran playbooks and i again ended up with cloud=vexxhost | 14:53 |
| jm1 | dasm: your setup is messed up ;) | 14:53 |
| dasm | (i forgot to jump on the host immediately after it got provisioned to check content of hosts) | 14:54 |
| dasm | jm1: whatever :P | 14:54 |
| dasm | lemme respin one more time | 14:54 |
| frenzy_friday | reviewbot, pls add to review list https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45823 | 15:05 |
| reviewbot | I have added your review to the Review list | 15:05 |
| dasm | jm1: this time it worked | 15:11 |
| dasm | vars: | 15:11 |
| dasm | cloud: "rhos_dev_stage" | 15:11 |
| * dasm likes that | 15:11 | |
| jm1 | dasm: hehe :D | 15:12 |
| dasm | i would love to learn why cli vars was not populated correctly. but maybe next time | 15:13 |
| dasm | jm1: 5x +1 from me | 15:21 |
| jm1 | dasm: hehe :D thank you | 15:26 |
| jm1 | dasm: now lets hope it wont break our infra | 15:26 |
| dasm | jm1: it can be only better :) | 15:27 |
| dasm | jm1: i need to pick up your brain. i have ideas what to do next: split base role into subroles. But maybe you have another idea, which would need to be tackled first. | 15:28 |
| dasm | i thought about splitting services config from servers provisioning. but i might need to revisit my idea with all the knowledge i gained recently, with all the work on staging env. | 15:31 |
| jm1 | dasm: i am happy to chat with you about that, but please lets do it after patches have been merged. /me dead after this week 🙈 | 15:39 |
| dasm | jm1: sure | 15:40 |
| jm1 | dviroel, rlandy, ysandeep: thank you for this podified overview! really nice progress and cool that this is now kind of official for 18 :D | 15:45 |
| * jm1 bbl | 15:49 | |
| *** pojadhav is now known as pojadhav|out | 15:49 | |
| *** dviroel is now known as dviroel|brb|lunch | 16:00 | |
| akahat|ruck | marios, rlandy ysandeep https://opendev.org/openstack/tripleo-ansible/commit/c0bc395ab494ee18e77fba608cdd40762c7b8c51 | 16:06 |
| akahat|ruck | that might be hitting in tripleo-ansible only | 16:06 |
| akahat|ruck | https://review.opendev.org/q/I646cd49b614a9f494e77f9893521d3c070900ed5 | 16:07 |
| ysandeep | marios++ rlandy++ akahat++ soniya29++ thanks for the nice summary | 16:07 |
| akahat|ruck | may be we just revert it and resolve the pep8 failures: https://review.opendev.org/c/openstack/tripleo-ansible/+/864010 | 16:07 |
| akahat|ruck | https://bugs.launchpad.net/tripleo/+bug/1996917 | 16:11 |
| ysandeep | jm1 :) | 16:13 |
| * ysandeep out | 16:15 | |
| *** ysandeep is now known as ysandeep|out | 16:15 | |
| marios | dasm: o/ off in a few. can we schedule somethign for tomorrow to discuss upgrades? | 16:18 |
| marios | dasm: want to send me a meeting invite ? perhaps 1600 UTC? | 16:18 |
| marios | dasm: or just ping when you get int | 16:19 |
| marios | in | 16:19 |
| * akahat|ruck leaving for the day | 16:20 | |
| dasm | marios: sure | 16:29 |
| dasm | marios: 1530utc | 16:31 |
| * dasm is stepping away for doc's appt. | 16:31 | |
| *** marios is now known as marios|out | 16:31 | |
| *** rlandy is now known as rlandy|ruck | 17:04 | |
| *** dviroel|brb|lunch is now known as dviroel | 17:12 | |
| *** jpena is now known as jpena|off | 17:36 | |
| *** amoralej is now known as amoralej|off | 18:12 | |
| *** dviroel_ is now known as dviroel | 18:16 | |
| dviroel | rlandy|ruck: hey - i added tripleo-ci-team keys to both ibm2 and ibm3 | 18:19 |
| dviroel | I may try to create a c9 vm to test it | 18:20 |
| dviroel | let me see | 18:20 |
| * dasm is back | 18:25 | |
| rlandy|ruck | dviroel: thanks | 18:30 |
| * jm1 out for today | 19:15 | |
| * jm1 have a nice evening folks :) | 19:15 | |
| dasm | dviroel: rlandy|ruck fyi this one zuul worked: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/435450 | 19:31 |
| dviroel | dasm: ack, will merge then | 19:34 |
| dasm | thx | 19:34 |
| dasm | dviroel: this one should be good to go as well: https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/435456 | 20:01 |
| rlandy|ruck | thanks | 20:08 |
| dviroel | dasm: ack, voted | 20:09 |
| dasm | thx | 20:10 |
| *** dviroel is now known as dviroel|out | 21:15 | |
| rlandy|ruck | ysandeep|out: new RR hackmd: https://hackmd.io/zkMzq69DTHudvreF37EgLQ | 21:32 |
| * dasm => offline | 22:47 | |
| dasm | o/ | 22:47 |
| *** dasm is now known as dasm|off | 22:47 | |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 23:30 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!