| *** jmasud has joined #oooq | 00:10 | |
| *** dsneddon has joined #oooq | 00:20 | |
| *** jmasud has quit IRC | 00:40 | |
| *** jmasud has joined #oooq | 00:41 | |
| *** dsneddon has quit IRC | 00:45 | |
| *** ysandeep is now known as ysandeep|afk | 00:48 | |
| *** jmasud has quit IRC | 00:50 | |
| *** ysandeep|afk is now known as ysandeep | 00:52 | |
| *** jmasud has joined #oooq | 00:53 | |
| *** dsneddon has joined #oooq | 01:18 | |
| *** jmasud has quit IRC | 01:19 | |
| *** jmasud has joined #oooq | 01:26 | |
| *** jmasud has quit IRC | 01:38 | |
| *** tosky has quit IRC | 01:39 | |
| *** owalsh has quit IRC | 01:58 | |
| *** owalsh has joined #oooq | 02:07 | |
| *** Goneri has quit IRC | 02:18 | |
| *** jmasud has joined #oooq | 03:02 | |
| *** jmasud has quit IRC | 03:03 | |
| *** rlandy has quit IRC | 03:56 | |
| *** apetrich has quit IRC | 04:00 | |
| *** jmasud has joined #oooq | 04:20 | |
| *** ykarel|away is now known as ykarel | 04:31 | |
| *** jmasud has quit IRC | 04:54 | |
| *** udesale has joined #oooq | 04:55 | |
| *** jmasud has joined #oooq | 04:55 | |
| *** weshay has quit IRC | 05:01 | |
| *** weshay has joined #oooq | 05:05 | |
| *** saneax has joined #oooq | 05:08 | |
| *** raukadah is now known as chandankumar | 05:24 | |
| *** akahat has quit IRC | 06:03 | |
| *** udesale_ has joined #oooq | 06:21 | |
| *** udesale has quit IRC | 06:23 | |
| *** jbadiapa has joined #oooq | 06:26 | |
| *** udesale has joined #oooq | 06:28 | |
| *** udesale_ has quit IRC | 06:29 | |
| *** marios has joined #oooq | 06:31 | |
| *** udesale_ has joined #oooq | 06:33 | |
| *** udesale has quit IRC | 06:35 | |
| *** ratailor has joined #oooq | 06:37 | |
| *** jfrancoa has joined #oooq | 06:59 | |
| *** marios has quit IRC | 07:26 | |
| *** marios has joined #oooq | 07:30 | |
| *** marios has quit IRC | 07:32 | |
| *** dsneddon has quit IRC | 07:38 | |
| *** apetrich has joined #oooq | 07:40 | |
| *** marios has joined #oooq | 07:42 | |
| *** ykarel is now known as ykarel|lunch | 07:43 | |
| *** tesseract has joined #oooq | 07:56 | |
| *** sshnaidm_ has joined #oooq | 07:57 | |
| *** sshnaidm|afk has quit IRC | 07:58 | |
| *** ccamacho has quit IRC | 08:09 | |
| *** ccamacho has joined #oooq | 08:09 | |
| *** ccamacho has quit IRC | 08:10 | |
| *** ccamacho has joined #oooq | 08:10 | |
| *** bogdando has joined #oooq | 08:38 | |
| *** amoralej|off is now known as amoralej | 08:39 | |
| *** tosky has joined #oooq | 08:40 | |
| *** jpena|off is now known as jpena | 08:47 | |
| *** jmasud has quit IRC | 08:54 | |
| *** dtantsur|afk is now known as dtantsur | 08:55 | |
| arxcruz|rover | ykarel|lunch: hey, can you confirm that containers were promoted regarding https://bugs.launchpad.net/tripleo/+bug/1865754 ? | 08:55 |
|---|---|---|
| openstack | Launchpad bug 1865754 in tripleo "tripleo-ci-centos-8-scenario001-standalone tempest-conf fails 500 PUT http://192.168.24.1:9292/v2/images/ RADOS invalid argument" [Critical,Triaged] | 08:55 |
| *** jmasud has joined #oooq | 08:56 | |
| *** ykarel|lunch is now known as ykarel | 08:58 | |
| ykarel | arxcruz|rover, i think not | 08:58 |
| arxcruz|rover | ykarel: ack, thanks | 08:59 |
| arxcruz|rover | I'm getting a failure regarding oslo.serialization now investigating | 08:59 |
| ykarel | arxcruz|rover, in centos7? | 08:59 |
| ykarel | or centos8 | 08:59 |
| arxcruz|rover | ykarel: yes, centos7, but those are promotion jobs | 08:59 |
| ykarel | arxcruz|rover, ack btw which job | 08:59 |
| arxcruz|rover | ykarel: https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/f792bd0/job-output.txt for example | 09:00 |
| ykarel | btw scenario001 centos8 is red http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-scenario001-standalone | 09:00 |
| ykarel | no promotion since 26th feb for c8 | 09:00 |
| ykarel | arxcruz|rover, ack | 09:00 |
| arxcruz|rover | ykarel: this is related to https://bugs.launchpad.net/tripleo/+bug/1865754 | 09:02 |
| openstack | Launchpad bug 1865754 in tripleo "tripleo-ci-centos-8-scenario001-standalone tempest-conf fails 500 PUT http://192.168.24.1:9292/v2/images/ RADOS invalid argument" [Critical,Triaged] | 09:02 |
| ykarel | secnario001 standalone c8 is related to ^^ | 09:02 |
| *** danpawlik has joined #oooq | 09:09 | |
| arxcruz|rover | ykarel: yes, that's what I said :) | 09:15 |
| ykarel | ack | 09:15 |
| *** yolanda has quit IRC | 09:29 | |
| *** yolanda has joined #oooq | 09:33 | |
| chandankumar | marios, hello | 09:36 |
| marios | o/ chandankumar | 09:36 |
| chandankumar | marios, you are working on scenario12 job? | 09:36 |
| marios | chandankumar: no i was looking at scen10 ovn didn't look at 12 so go ahead if that's what you're asking | 09:37 |
| marios | chandankumar: otherwise if you need me to then i can look at that one too | 09:37 |
| chandankumar | marios, aye on it | 09:37 |
| marios | chandankumar: ack | 09:39 |
| panda | marios: there wa no button to add a meet, can we use bluejeans ? | 09:59 |
| marios | panda: sure | 09:59 |
| marios | panda: gimme number | 09:59 |
| panda | your room | 09:59 |
| marios | panda: plugging headphones | 09:59 |
| panda | marios: your bj room | 10:00 |
| *** marios is now known as marios|call | 10:01 | |
| *** derekh has joined #oooq | 10:13 | |
| *** holser has joined #oooq | 10:14 | |
| *** skramaja has quit IRC | 10:15 | |
| *** holser has quit IRC | 10:15 | |
| *** holser has joined #oooq | 10:16 | |
| chandankumar | arxcruz|rover, what is the status of c7 master promotion? | 10:18 |
| arxcruz|rover | chandankumar: didn't promote because of oslo.serialization dependence failing | 10:18 |
| chandankumar | arxcruz|rover, ah | 10:18 |
| arxcruz|rover | chandankumar: https://bugs.launchpad.net/tripleo/+bug/1866316 | 10:18 |
| openstack | Launchpad bug 1866316 in tripleo "ara < 1.0.0 is failing on overcloud-prep-image" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 10:18 |
| arxcruz|rover | I add a patch to install ara 1.0.0 | 10:19 |
| arxcruz|rover | that doesn't have the oslo.serialization dependence | 10:19 |
| arxcruz|rover | I hope it works | 10:19 |
| arxcruz|rover | otherwise, we can switch the order and use pip3 first | 10:19 |
| *** ratailor_ has joined #oooq | 10:22 | |
| *** ratailor has quit IRC | 10:23 | |
| ykarel | mmm but is pip3 available at that place | 10:33 |
| ykarel | iirc ara was pinned as 1.0.0 had some issue | 10:34 |
| ykarel | or was due to major ara bump that time | 10:34 |
| ykarel | but if jobs are happy, it can be moved to 1.0.0 | 10:35 |
| arxcruz|rover | ykarel: yeah, so, it has a pip install ara || pip3 install ara | 10:46 |
| arxcruz|rover | and ara dependens on oslo.serialization >= 1.10 | 10:46 |
| arxcruz|rover | which means, newer version that requires python3 arrives and we are screwed | 10:46 |
| ykarel | u can pin deps to < | 10:46 |
| arxcruz|rover | if doesn't work, we can switch the order | 10:47 |
| arxcruz|rover | ykarel: the oslo >= 1.10 comes from ara 0.16 | 10:47 |
| ykarel | i think switching order will not help | 10:47 |
| ykarel | it will help only if pip3 is there in centos7 | 10:47 |
| arxcruz|rover | ykarel: the ara < 1.0 is 7 months old, let's see if the zuul god will be happy with the change, if so, we are good | 10:48 |
| arxcruz|rover | but of course, we can keep thinking other ways | 10:48 |
| ykarel | arxcruz|rover, yes that's the best try | 10:48 |
| ykarel | if works all good | 10:48 |
| ykarel | else u can pin to working oslo.serialization | 10:49 |
| arxcruz|rover | ykarel: can i do a pip install 'ara<1.0.0' 'oslo.serialization==1.10' ? | 10:49 |
| ykarel | arxcruz|rover, i think should work | 10:49 |
| ykarel | u can try it in venv | 10:49 |
| ykarel | py2 | 10:49 |
| ykarel | we use it many times such pins | 10:50 |
| ykarel | better would be to use <'broken version' instead of == for oslo.serialization | 10:50 |
| ykarel | or pin to last working one | 10:50 |
| arxcruz|rover | that would be pip install 'ara<1.0.0' 'oslo.serialization==2.29.2' | 10:53 |
| arxcruz|rover | ykarel: i'll check if the ara 1.0.0 will work, otherwise, add the oslo.serialization==2.29.2 | 10:53 |
| *** marios|call is now known as marios | 10:54 | |
| arxcruz|rover | marios: what's the status of https://bugs.launchpad.net/tripleo/+bug/1865832 ? | 10:54 |
| openstack | Launchpad bug 1865832 in tripleo "tox py3 virtualenv jobs failing ImportError cannot import name 'ContextManager'" [Critical,Triaged] | 10:54 |
| ykarel | arxcruz|rover, ack | 10:54 |
| marios | arxcruz|rover: did you read the comments se comment #6 and #7 i think its ok now should be done unless you are seing it any more | 10:55 |
| arxcruz|rover | marios: yes i did, that's why i'm asking, shall i keep it open in the ruck rover notes, or move it to done | 10:55 |
| marios | arxcruz|rover: 12:55 < marios> arxcruz|rover: did you read the comments se comment #6 and #7 i think its ok now should be done unless you are seing it | 10:56 |
| marios | arxcruz|rover: sorry was getting food... so, unless you are seeing it its done | 10:56 |
| arxcruz|rover | marios: just wanted to double check with you :) | 10:57 |
| chandankumar | arxcruz|rover, kopecmartin soniya29 temepst agenda added https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view#2020-03-06 please add or remove | 10:57 |
| arxcruz|rover | chandankumar: ack | 10:59 |
| *** sshnaidm_ is now known as sshnaidm|afk | 11:13 | |
| *** holser has quit IRC | 11:15 | |
| *** holser has joined #oooq | 11:19 | |
| *** holser has quit IRC | 11:21 | |
| *** holser has joined #oooq | 11:23 | |
| *** holser has quit IRC | 11:24 | |
| *** holser has joined #oooq | 11:25 | |
| *** jbadiapa has quit IRC | 12:02 | |
| *** jbadiapa has joined #oooq | 12:03 | |
| *** baha has quit IRC | 12:04 | |
| *** irclogbot_2 has quit IRC | 12:05 | |
| *** irclogbot_0 has joined #oooq | 12:05 | |
| *** baha has joined #oooq | 12:06 | |
| *** irclogbot_0 has quit IRC | 12:07 | |
| *** irclogbot_3 has joined #oooq | 12:07 | |
| *** jbadiapa has quit IRC | 12:10 | |
| *** ratailor_ has quit IRC | 12:28 | |
| soniya29 | weshay, ping. Meeting time | 12:31 |
| *** dtantsur is now known as dtantsur|afk | 12:33 | |
| *** jpena is now known as jpena|lunch | 12:37 | |
| chandankumar | arxcruz|rover, kopecmartin https://hackmd.io/HrQd03c9SxOMtFPFrq50tg#Second-Phase---Prepare-for-the-shinnin--unpinning- | 12:39 |
| *** jmasud has quit IRC | 12:53 | |
| *** rlandy has joined #oooq | 12:55 | |
| *** udesale has joined #oooq | 12:55 | |
| *** jmasud has joined #oooq | 12:56 | |
| *** udesale_ has quit IRC | 12:57 | |
| *** marios is now known as marios|call | 13:00 | |
| rlandy | panda: chandankumar: weshay: sync call https://meet.google.com/bqx-xwht-wky | 13:00 |
| weshay | soniya29, ugh.. this did not show up on my phone | 13:06 |
| weshay | I need to do something | 13:06 |
| weshay | about this | 13:06 |
| weshay | soniya29, arxcruz|rover kopecmartin sorry again.. /me fixes calendar | 13:06 |
| arxcruz|rover | ykarel: https://trunk.rdoproject.org/centos8/component/tripleo/8f/35/8f356c89db9bcc15a60406d1385a9d8b48045dc4_b135288d/rpmbuild.log | 13:10 |
| arxcruz|rover | the rpmbuild is failing | 13:11 |
| arxcruz|rover | ykarel: so even in my ara patch, i'm still geting the dictionary error... | 13:11 |
| ykarel | arxcruz|rover, it should be fixed now | 13:11 |
| ykarel | how much ago u seen that failure? | 13:12 |
| ykarel | rpmbuild one | 13:12 |
| arxcruz|rover | ykarel: https://review.opendev.org/#/c/711597/ | 13:12 |
| arxcruz|rover | ykarel: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_855/711597/1/check/tripleo-ci-centos-8-containers-multinode/855e169/logs/undercloud/home/zuul/overcloud_deploy.log | 13:13 |
| ykarel | arxcruz|rover, okk tripleoclient built some time back | 13:14 |
| ykarel | arxcruz|rover, see https://trunk.rdoproject.org/centos8-master/report.html | 13:14 |
| ykarel | and from timing of build u can relate | 13:14 |
| ykarel | the issue should not be seen after the build time | 13:15 |
| arxcruz|rover | ykarel: so, a recheck should be ok right ? | 13:15 |
| ykarel | arxcruz|rover, yes | 13:15 |
| arxcruz|rover | ykarel: thanks | 13:15 |
| ykarel | uhh that job is voting | 13:15 |
| ykarel | too quick :) | 13:16 |
| arxcruz|rover | yeah | 13:16 |
| arxcruz|rover | ykarel: you can blame weshay i guess :P | 13:16 |
| arxcruz|rover | brb real quick | 13:18 |
| weshay | arxcruz|rover, would rather have it fail.. tbh | 13:20 |
| dmellado | weshay: are you to blame, man? | 13:24 |
| dmellado | xD | 13:24 |
| weshay | dmellado, to blame.. if that job doesn't vote.. we're all fucked :) | 13:24 |
| dmellado | xD | 13:24 |
| weshay | better to have it fail | 13:24 |
| weshay | it was passing last night when I left it.. | 13:25 |
| dmellado | maybe it has gotten the covid19, who knows! | 13:25 |
| weshay | I didn't get the job to voting in time | 13:25 |
| * dmellado totally understands that | 13:25 | |
| *** derekh has quit IRC | 13:25 | |
| chandankumar | rlandy, https://images.rdoproject.org/master/rdo_trunk/ | 13:25 |
| weshay | I was an hour off | 13:25 |
| weshay | 2020-03-06 00:35:30.764151 | primary | + rsync -av tripleo-ci-testing uploader@images.rdoproject.org:/var/www/html/images/centos8/master/rdo_trunk | 13:27 |
| weshay | 2020-03-06 00:35:32.194573 | primary | cp: cannot stat '/home/zuul/*repo_setup*log': No such file or directory | 13:28 |
| rlandy | ipleo-ci-testing -> 710bba610ab497c3c68b1eda1287a1be | 13:28 |
| chandankumar | https://images.rdoproject.org/centos8/master/rdo_trunk/710bba610ab497c3c68b1eda1287a1be/overcloud-full.tar | 13:30 |
| chandankumar | https://review.opendev.org/#/c/711238/2/toci-quickstart/config/testenv/ovb-rdocloud.yml | 13:30 |
| chandankumar | https://images.rdoproject.org/centos8/master/rdo_trunk/423cb54e84da6ddf5c163d1a5c1d512b/overcloud-full.tar | 13:30 |
| arxcruz|rover | weshay: actually, problem was some failures in the rpm build | 13:32 |
| arxcruz|rover | should be ok now according ykarel | 13:32 |
| weshay | hrm.. ykarel is containers-multinode not used in rdo-info? | 13:32 |
| ysandeep | weshay, o/ reminder out 1:1 | 13:32 |
| ysandeep | our* | 13:32 |
| weshay | ysandeep, have to use irc today | 13:32 |
| ysandeep | weshay, ack pm | 13:32 |
| ykarel | weshay, issues was in dlrn build | 13:32 |
| ykarel | weshay, https://trunk.rdoproject.org/centos8-master/report.html | 13:33 |
| arxcruz|rover | weshay: remindr our 1:1 | 13:33 |
| ykarel | too many failures there, | 13:33 |
| arxcruz|rover | lol | 13:33 |
| ykarel | weshay, so required package was not build, so not available in current repo | 13:33 |
| *** jpena|lunch is now known as jpena | 13:33 | |
| ykarel | thus not available in CI | 13:33 |
| weshay | ykarel, k.. 9pm last night it was passing | 13:34 |
| ykarel | weshay, because of depends-on | 13:34 |
| ykarel | the package got build in CI | 13:34 |
| ykarel | so passing | 13:34 |
| ykarel | for subsequent jobs package needs to available in repo | 13:34 |
| *** holser has quit IRC | 13:40 | |
| *** holser has joined #oooq | 13:41 | |
| chandankumar | weshay, https://review.opendev.org/#/c/707338/ | 13:42 |
| chandankumar | weshay, https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-master-containers-build-push/d9d5540/conf/kolla-build.conf | 13:48 |
| chandankumar | weshay, i need to leave now | 13:57 |
| chandankumar | see ya from home | 13:58 |
| chandankumar | please put the patches here | 13:58 |
| weshay | https://review.opendev.org/#/c/710493/ | 14:00 |
| *** jmasud has quit IRC | 14:03 | |
| panda | rlandy: FYI you're still sharing | 14:04 |
| weshay | panda, sshnaidm|afk I see train running in vexx.. how close are we to moving the other non-master branches? | 14:07 |
| sshnaidm|afk | weshay, all non-masters are there | 14:08 |
| *** derekh has joined #oooq | 14:08 | |
| weshay | sshnaidm|afk, \0/ | 14:09 |
| *** owalsh has quit IRC | 14:10 | |
| *** weshay is now known as weshay|ruck | 14:12 | |
| *** marios|call is now known as marios | 14:17 | |
| panda | marios|call: sorry about the step by step thing. What I did in the past sprint, was refactor the code completely, Then look at a path to get there gradually. I tried to do the same, but the code has changed so much, that I could not see an easy path from the start to the end taht could be done in a reasonable amount of time. | 14:17 |
| panda | marios: ^ | 14:17 |
| marios | ack panda also time is a main factor | 14:18 |
| *** owalsh has joined #oooq | 14:18 | |
| *** rascasoft has joined #oooq | 14:18 | |
| panda | marios: yes, and I had to choose between attempting a split and adding unit tests. Unit test will certainly make any change to the code easier | 14:21 |
| *** holser has quit IRC | 14:27 | |
| *** cgoncalves has quit IRC | 14:32 | |
| *** cgoncalves has joined #oooq | 14:34 | |
| weshay|ruck | kopecmartin, thanks for the email... I'll send a planning meeting to discuss ur results and next steps | 14:38 |
| *** cgoncalves has quit IRC | 14:39 | |
| *** amoralej is now known as amoralej|lunch | 14:41 | |
| *** cgoncalves has joined #oooq | 14:41 | |
| chandankumar | rlandy, I am going to spend time on fixing image role to consume in ovb job for c8 | 14:43 |
| *** udesale has quit IRC | 14:45 | |
| panda | weshay|ruck: you still want the new promoter to promote to something different than current-tripleo ? | 14:52 |
| *** ratailor has joined #oooq | 14:53 | |
| *** ykarel is now known as ykarel|away | 14:53 | |
| *** owalsh has quit IRC | 15:00 | |
| *** amoralej|lunch is now known as amoralej | 15:02 | |
| rlandy | chandankumar: weshay|ruck: rekicked fs010 and fs027 from master pipelines | 15:04 |
| rlandy | pipeline is now centos7 free and 3x daily | 15:04 |
| chandankumar | rlandy, weshay|ruck so from now, no content can be promoted using c7 for master? | 15:06 |
| rlandy | chandankumar: ack | 15:06 |
| rlandy | marios: if you're looking fr more centos-8 work ... | 15:06 |
| rlandy | https://review.opendev.org/710876 Add centos-8 scenario000-multinode update job - FAILING | 15:06 |
| rlandy | needs debug | 15:06 |
| rlandy | I work there sporadically but would be cool if we could close that out | 15:07 |
| chandankumar | rlandy, ^^ recheck it will work, we fixed the issue fs010 issue, it was same there | 15:08 |
| chandankumar | last time I checked | 15:08 |
| chandankumar | rebased | 15:08 |
| weshay|ruck | chandankumar, doesn't matter | 15:08 |
| weshay|ruck | it's all pinned | 15:09 |
| rlandy | chandankumar: k - rebased - recheck | 15:09 |
| chandankumar | rlandy, https://review.opendev.org/#/c/710441/11 and https://review.opendev.org/#/c/711305/3 good to go | 15:12 |
| marios | rlandy: looking | 15:15 |
| rlandy | marios: I rechecked | 15:15 |
| rlandy | so no worries until we get new results there | 15:15 |
| marios | rlandy: k just read the chat | 15:15 |
| rlandy | marios: was just saying *if* it fails, may need more debug - may not get to it | 15:16 |
| chandankumar | weshay|ruck, when we kick build_images role in ovb? | 15:16 |
| chandankumar | there are so many things there build not to build | 15:16 |
| rlandy | chandankumar: lol | 15:17 |
| rlandy | it's a maze if stuff | 15:17 |
| chandankumar | I just need the patch where it is called I can make that working | 15:17 |
| rlandy | chandankumar: want to join me on the image server to fix permissions? | 15:17 |
| rlandy | we need that done | 15:18 |
| rlandy | ovb fails on getting images | 15:18 |
| marios | rlandy: ok noting the chat i can check the status my next morning | 15:18 |
| rlandy | marios: thanks - might be ok - there is always hope :) | 15:18 |
| chandankumar | rlandy, ok | 15:19 |
| chandankumar | sure | 15:19 |
| rlandy | chandankumar: ssh to the promoter - tmux a | 15:21 |
| panda | weshay|ruck: ready to launch, you still want the new promoter to promote to something different than current-tripleo ? | 15:22 |
| weshay|ruck | panda, YES | 15:23 |
| rlandy | chandankumar: you there? | 15:23 |
| weshay|ruck | panda, promote to panda_rulz | 15:23 |
| *** Goneri has joined #oooq | 15:33 | |
| *** owalsh has joined #oooq | 15:34 | |
| *** Goneri has quit IRC | 15:38 | |
| *** owalsh has quit IRC | 15:40 | |
| *** ratailor has quit IRC | 15:41 | |
| rlandy | image jobs rekicked | 15:44 |
| panda | marios: guess where the promoter fails ? :) | 15:48 |
| *** owalsh has joined #oooq | 15:50 | |
| marios | panda: well not sure if relevant but i added comment there https://review.rdoproject.org/r/#/c/24774/56/ci-scripts/container-push/roles/containers-promote/tasks/check_set_ppc_manifests.yml | 15:50 |
| panda | marios: nope | 15:50 |
| marios | panda: but what is it failed pulling ppc/? creating manifest command? what | 15:50 |
| panda | marios: File "/home/centos/ci-config-refactored/ci-scripts/dlrnapi_promoter/dlrn_client.py", line 113, in fetch_current_named_hashes | 15:51 |
| marios | panda: nice | 15:51 |
| panda | yyyyep | 15:51 |
| marios | panda: ack yeah i will spend some time with that on monday | 15:51 |
| marios | panda: i mean the tests part anyway | 15:51 |
| panda | marios: ok | 15:51 |
| marios | panda: trying to get through the patchset a bit more seriously now | 15:51 |
| panda | marios: ok | 15:52 |
| *** owalsh_ has joined #oooq | 15:53 | |
| *** owalsh- has joined #oooq | 15:54 | |
| panda | marios: I will not touch it, this is a very good chance to learn a bit more how the code works. Maybe you should even drive this testing effort. | 15:56 |
| *** owalsh has quit IRC | 15:57 | |
| *** owalsh_ has quit IRC | 15:58 | |
| *** bogdando has quit IRC | 16:03 | |
| chandankumar | wooh something got broken again | 16:07 |
| chandankumar | https://logserver.rdoproject.org/12/25612/14/check/distgit-tripleo-master-testing-centos-8-multinode-1ctlr-featureset010/ff9ab3e/logs/undercloud/home/zuul/undercloud_install.log.txt.gz | 16:07 |
| chandankumar | usage: mistral-db-manage [-h] [--config-dir DIR] [--config-file PATH] | 16:07 |
| chandankumar | https://logserver.rdoproject.org/12/25612/14/check/distgit-tripleo-master-testing-centos-8-singlenode-featureset027/f4b2262/job-output.txt | 16:07 |
| chandankumar | https://logserver.rdoproject.org/12/25612/14/check/distgit-tripleo-master-testing-centos-8-standalone-scenario003/70e2875/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 16:08 |
| chandankumar | rlandy, weshay|ruck ^^ real bug | 16:08 |
| chandankumar | 3 failures | 16:08 |
| weshay|ruck | chandankumar, hrm.. do your jobs pull in current repo for tripleo? | 16:09 |
| weshay|ruck | w/ include_pkgs | 16:09 |
| chandankumar | weshay|ruck, yes | 16:09 |
| rlandy | hmm ... | 16:09 |
| rlandy | log it | 16:09 |
| chandankumar | weshay|ruck, it is running in check | 16:09 |
| weshay|ruck | hrm. k | 16:10 |
| weshay|ruck | https://logserver.rdoproject.org/12/25612/14/check/distgit-tripleo-master-testing-centos-8-standalone-scenario003/70e2875/logs/undercloud/etc/yum.repos.d/delorean-current.repo.txt.gz | 16:10 |
| chandankumar | rlandy, need your comments on this https://review.rdoproject.org/r/#/c/25659/5/zuul.d/distgit-jobs.yaml parenting | 16:11 |
| chandankumar | the goal is avoid repetition | 16:11 |
| rlandy | k - sec | 16:11 |
| *** ykarel|away is now known as ykarel | 16:11 | |
| *** jaosorior has quit IRC | 16:14 | |
| *** TrevorV has joined #oooq | 16:16 | |
| rlandy | chandankumar: weird ... https://images.rdoproject.org/centos8/master/rdo_trunk/ | 16:18 |
| rlandy | gone again | 16:18 |
| chandankumar | rlandy, may be rsync issue? | 16:18 |
| rlandy | how is that even possible - maybe that was that l permission | 16:19 |
| rlandy | not changeable | 16:19 |
| rlandy | checking the image server again | 16:19 |
| rlandy | weshay|ruck: ^^ some weirdness going on here | 16:19 |
| rlandy | chandankumar: wrt parenting in https://review.rdoproject.org/r/#/c/25659/5/zuul.d/distgit-jobs.yaml - what hash do you want them to run? | 16:20 |
| rlandy | tripleo-ci-testing or current-tripleo? | 16:20 |
| chandankumar | rlandy, current-tripleo | 16:20 |
| chandankumar | since periodic is runned in check it will auto go to check | 16:20 |
| chandankumar | *current-tripleo | 16:20 |
| rlandy | should be ok - because no force_promote | 16:21 |
| chandankumar | rlandy, the main goal is to avoid periodic related call | 16:21 |
| chandankumar | like podman registry login | 16:21 |
| rlandy | parenting is good then | 16:21 |
| chandankumar | ykarel, ^^ | 16:21 |
| rlandy | drwxr-xr-x 2 uploader uploader 112 Oct 3 09:20 centos7 | 16:22 |
| rlandy | drwxr-xr-x 3 uploader apache 20 Mar 3 19:09 centos8 | 16:22 |
| rlandy | ha | 16:22 |
| ykarel | rlandy, chandankumar, those jobs do too many things, which are not relevant for distgit jobs | 16:22 |
| ykarel | like dlrn prep, registry login, dlrn reporting | 16:22 |
| ykarel | etc | 16:22 |
| ykarel | couple of more things they do | 16:22 |
| rlandy | chandankumar: you could make non-periodic versions | 16:22 |
| rlandy | that would kick the dlnr reporting | 16:22 |
| chandankumar | rlandy, then we need to do repeatition | 16:23 |
| chandankumar | I cannot use upstream toci base as it is meant for upstream ci | 16:24 |
| chandankumar | rlandy, https://review.rdoproject.org/r/#/c/25659/1/zuul.d/distgit-jobs.yaml then I need to follow this version | 16:24 |
| chandankumar | rlandy, what can i modify here? | 16:25 |
| rlandy | chandankumar: sec - back on images server | 16:25 |
| chandankumar | ok | 16:26 |
| weshay|ruck | rlandy, /me going to add criteria for components... | 16:28 |
| weshay|ruck | don't see any reviews from you right? you have not started? | 16:28 |
| * weshay|ruck just trying to help | 16:28 | |
| rlandy | weshay|ruck: sure - still fighting with image server | 16:28 |
| weshay|ruck | k | 16:28 |
| weshay|ruck | thanks | 16:28 |
| weshay|ruck | seemless | 16:29 |
| rlandy | weshay|ruck: stupid permissions changed again | 16:29 |
| *** kopecmartin is now known as kopecmartin|afk | 16:29 | |
| rlandy | weshay|ruck: somehow the group owner there is apache | 16:29 |
| ykarel | chandankumar, any issue in using scneario jobs directly as parent and modify nodeset and adding additional required vars like https://github.com/openstack/tripleo-ci/blob/master/zuul.d/standalone-jobs.yaml#L677 | 16:29 |
| rlandy | and I can't chown it | 16:29 |
| weshay|ruck | rlandy, ya.. we don't have access to do that afaik | 16:29 |
| weshay|ruck | we can only make dirs and files | 16:30 |
| rlandy | weshay|ruck: I changed it once | 16:30 |
| weshay|ruck | k | 16:30 |
| weshay|ruck | maybe I'm wrong | 16:30 |
| rlandy | I can change perms but not owners | 16:30 |
| rlandy | why???? | 16:30 |
| rlandy | infra can? | 16:30 |
| weshay|ruck | rlandy, ping rhosops | 16:30 |
| rlandy | yep | 16:30 |
| chandankumar | rlandy, dpwalik can | 16:31 |
| *** owalsh has joined #oooq | 16:32 | |
| rlandy | chandankumar: yep - pinged him but he's afk | 16:32 |
| chandankumar | ah mhu is also about to off | 16:32 |
| rlandy | he's gone as well | 16:33 |
| rlandy | chandankumar: anyone else? | 16:33 |
| chandankumar | rlandy, may be jpena | 16:33 |
| *** owalsh_ has joined #oooq | 16:33 | |
| chandankumar | or tristanC | 16:33 |
| chandankumar | or alan | 16:34 |
| *** owalsh| has joined #oooq | 16:34 | |
| *** owalsh^ has joined #oooq | 16:35 | |
| *** owalsh- has quit IRC | 16:35 | |
| rlandy | talking with alan | 16:36 |
| *** owalsh has quit IRC | 16:37 | |
| *** migi has joined #oooq | 16:37 | |
| rlandy | periodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-mastertestprojectmastercheck25411,1333942020-03-06T15:04:38FAILURE | 16:37 |
| rlandy | periodic-tripleo-ci-centos-8-singlenode-featureset027-mastertestprojectmastercheck25411,1340602020-03-06T15:04:37SUCCESS | 16:37 |
| chandankumar | rlandy, weshay|ruck dropping now, see ya on monday | 16:37 |
| rlandy | mixed result there | 16:37 |
| chandankumar | have a nice weekend ahead | 16:38 |
| weshay|ruck | chandankumar, 0/ thank you .. have a good weekend | 16:38 |
| *** chandankumar is now known as raukadah | 16:38 | |
| *** owalsh_ has quit IRC | 16:38 | |
| *** owalsh| has quit IRC | 16:39 | |
| * marios out | 16:40 | |
| weshay|ruck | marios, 0/ | 16:40 |
| marios | \o | 16:40 |
| *** marios is now known as marios|out | 16:40 | |
| *** danpawlik has quit IRC | 16:41 | |
| *** marios|out has quit IRC | 16:55 | |
| rlandy | weshay|ruck: you ok with the criteria change? | 16:58 |
| weshay|ruck | yup.. sorry miggi rang.. almost done | 16:59 |
| rlandy | weshay|ruck: last ovb run got to overcloud deploy -collecting logs | 16:59 |
| rlandy | will see how far it got | 16:59 |
| weshay|ruck | SHUT UP! | 16:59 |
| weshay|ruck | where | 16:59 |
| rlandy | it didn't pass overcloud deploy | 16:59 |
| rlandy | it started | 16:59 |
| weshay|ruck | oh | 16:59 |
| rlandy | https://review.rdoproject.org/zuul/stream/c71a064e089743baa48526b262998a5d?logfile=console.log | 16:59 |
| rlandy | will ping you when there are logs | 16:59 |
| rlandy | weshay|ruck: jpena fixed the ownership ... I didn't run again - will see the image upload when the pipeline kicks in a few hours | 17:00 |
| weshay|ruck | k | 17:01 |
| weshay|ruck | fresh https://review.rdoproject.org/r/25760 | 17:03 |
| *** TrevorV has quit IRC | 17:03 | |
| *** jmasud has joined #oooq | 17:06 | |
| rlandy | weshay|ruck: ^^ looks good - thanks for taking that over | 17:11 |
| rlandy | https://logserver.rdoproject.org/32/25332/41/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/c71a064/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 17:12 |
| rlandy | weshay|ruck: ^^ same deal ... investigating | 17:12 |
| rlandy | 2020-03-06 16:56:54 | content = yaml.load(f) | 17:14 |
| rlandy | https://bugs.launchpad.net/tripleo/+bug/1845927 | 17:15 |
| openstack | Launchpad bug 1845927 in tripleo "Still observing close file warnings with overcloud commands" [Medium,Fix released] - Assigned to Michele Baldessari (michele) | 17:15 |
| weshay|ruck | rlandy, we fixed that last night | 17:18 |
| rlandy | weshay|ruck: reading bug - but still failing | 17:19 |
| weshay|ruck | rlandy, you have current? | 17:19 |
| rlandy | weshay|ruck: current what? | 17:19 |
| rlandy | rekicked https://review.rdoproject.org/r/#/c/25332/ | 17:19 |
| weshay|ruck | we saw it in fs010 | 17:21 |
| rlandy | weshay|ruck: you talking with alan? | 17:22 |
| *** rascasoft has quit IRC | 17:35 | |
| raukadah | rlandy, weshay|ruck https://review.opendev.org/#/c/703972/ is the culprit for mistral extras | 17:36 |
| *** dsneddon has joined #oooq | 17:40 | |
| rlandy | weshay|ruck: want to catch me up where we are with ovb now? | 17:50 |
| rlandy | read the bug | 17:50 |
| *** ykarel is now known as ykarel|away | 17:54 | |
| *** amoralej is now known as amoralej|off | 17:54 | |
| *** jpena is now known as jpena|off | 17:57 | |
| raukadah | rlandy, weshay|ruck I am running ovb in check now due to python tripleoclient dict issue | 17:59 |
| *** derekh has quit IRC | 18:00 | |
| rlandy | raukadah: you might clash with weshay|ruck's run | 18:01 |
| raukadah | taking different approach | 18:01 |
| raukadah | rlandy, https://review.rdoproject.org/r/#/c/25332/ including tripleo current in periodic and see | 18:03 |
| weshay|ruck | I have not rechecked | 18:03 |
| raukadah | weshay|ruck, ^6 watch above review we might get some fun results | 18:04 |
| weshay|ruck | raukadah, k | 18:04 |
| raukadah | rlandy, did we fixed image perm issue? | 18:04 |
| rlandy | raukadah: I hope so | 18:05 |
| raukadah | rlandy, hope for best | 18:05 |
| rlandy | jpena changed the ownership | 18:05 |
| weshay|ruck | rlandy, let's chat later.. and let raukadah finish fixing the images | 18:05 |
| weshay|ruck | we need to be in the overcloud deployment for this issue | 18:05 |
| rlandy | I didn't rerun - we will see when the pipeline kicks again | 18:05 |
| raukadah | rlandy, ovb running https://review.rdoproject.org/r/#/c/25332/ here with toci testing repo | 18:06 |
| weshay|ruck | rlandy, let's rekick an image build job | 18:06 |
| raukadah | and current tripleo component | 18:06 |
| rlandy | weshay|ruck: ok | 18:06 |
| rlandy | weshay|ruck: raukadah: https://review.rdoproject.org/r/#/c/24995/ image build rekicked | 18:07 |
| weshay|ruck | thanks | 18:07 |
| rlandy | we will see how fixed the images rae | 18:07 |
| rlandy | weshay|ruck: going back to component promter if we are not talking ovb right now | 18:07 |
| rlandy | let we know before you rekick and I will add setting to stop dlete | 18:08 |
| rlandy | if I do that now, it will kill raukadah's job | 18:08 |
| *** rlandy is now known as rlandy|brb | 18:13 | |
| *** jmasud has quit IRC | 18:15 | |
| *** saneax has quit IRC | 18:15 | |
| *** tesseract has quit IRC | 18:15 | |
| *** rlandy|brb is now known as rlandy | 18:33 | |
| rlandy | ugh - image permissions changed again :( | 18:40 |
| rlandy | weshay|ruck: ^^ | 18:40 |
| weshay|ruck | seemless! | 18:44 |
| rlandy | changing rync command to see if that helps | 18:46 |
| rlandy | weshay|ruck: see diff between https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/modify-image/tasks/manual.yml#L52 and https://github.com/openstack/tripleo-ci/blob/master/roles/oooci-build-images/tasks/main.yaml#L14 | 19:13 |
| rlandy | iiuc we build images in the workspace | 19:13 |
| rlandy | could we test by making the workspace chmod 0777 | 19:13 |
| rlandy | weshay|ruck: either that or we change the rsync command remove --archive and set to --no-perms | 19:15 |
| * rlandy tries the first | 19:15 | |
| rlandy | oh and master just kicked | 19:16 |
| weshay|ruck | rlandy, have you ever run ovb-manage by hand | 19:25 |
| weshay|ruck | just to create a stack? | 19:25 |
| rlandy | weshay|ruck: no | 19:25 |
| rlandy | but if you kick a job - add the var to not delete stack and kill the job, you'll have a stack | 19:26 |
| rlandy | weshay|ruck: ^^ | 19:26 |
| rlandy | get to the stack via the tenant | 19:26 |
| rlandy | also if you create the stack by hand, you will loose the undercloud | 19:27 |
| rlandy | weshay|ruck: closest is the old reproducer | 19:27 |
| rlandy | see there how to create a stack | 19:27 |
| rlandy | need help? | 19:27 |
| weshay|ruck | depends.. if ur busy no | 19:27 |
| rlandy | weshay|ruck: working on promote components which I can finish on sunday | 19:28 |
| rlandy | waiting for image test | 19:28 |
| rlandy | so I can help | 19:28 |
| rlandy | chat? | 19:28 |
| weshay|ruck | https://meet.google.com/fpr-avri-gbg?authuser=1 | 19:28 |
| *** jmasud has joined #oooq | 19:29 | |
| rlandy | remove_ovb_after_job: false | 19:34 |
| rlandy | weshay|ruck: ^^ | 19:34 |
| *** dsneddon has quit IRC | 19:35 | |
| *** jmasud has quit IRC | 19:40 | |
| *** dsneddon has joined #oooq | 19:44 | |
| *** jfrancoa has quit IRC | 20:07 | |
| weshay|ruck | rlandy, baremetal_25332_46_80608 | 20:07 |
| *** jmasud has joined #oooq | 20:21 | |
| *** jmasud has quit IRC | 20:23 | |
| *** weshay|ruck is now known as weshay | 20:41 | |
| rlandy | it's complicated :) | 20:57 |
| rlandy | ok to merge https://review.rdoproject.org/r/#/c/25762/? | 20:57 |
| rlandy | weshay: ^^? | 20:57 |
| * weshay looks | 20:57 | |
| rlandy | thanks - here goes | 20:58 |
| rlandy | weshay: ironic logs on overcloud | 21:09 |
| *** ChanServ has quit IRC | 21:13 | |
| *** dsneddon has quit IRC | 21:13 | |
| *** pojadhav has quit IRC | 21:13 | |
| *** bhagyashris has quit IRC | 21:13 | |
| *** amoralej|off has quit IRC | 21:13 | |
| *** arxcruz|rover has quit IRC | 21:13 | |
| *** jpena|off has quit IRC | 21:13 | |
| *** zbr|pto has quit IRC | 21:13 | |
| *** tristanC has quit IRC | 21:13 | |
| *** cgoncalves has quit IRC | 21:13 | |
| *** yolanda has quit IRC | 21:13 | |
| *** weshay has quit IRC | 21:13 | |
| *** soniya29 has quit IRC | 21:13 | |
| *** dmsimard|off has quit IRC | 21:13 | |
| *** gchamoul has quit IRC | 21:13 | |
| *** mjturek has quit IRC | 21:13 | |
| *** EmilienM has quit IRC | 21:13 | |
| *** owalsh^ has quit IRC | 21:13 | |
| *** irclogbot_3 has quit IRC | 21:13 | |
| *** ccamacho has quit IRC | 21:13 | |
| *** apetrich has quit IRC | 21:13 | |
| *** dmellado has quit IRC | 21:13 | |
| *** jjoyce has quit IRC | 21:13 | |
| *** Tengu has quit IRC | 21:13 | |
| *** panda has quit IRC | 21:13 | |
| *** gouthamr has quit IRC | 21:13 | |
| *** dtantsur|afk has quit IRC | 21:13 | |
| *** tosky has quit IRC | 21:13 | |
| *** sshnaidm|afk has quit IRC | 21:13 | |
| *** matbu has quit IRC | 21:13 | |
| *** ysandeep has quit IRC | 21:13 | |
| *** raukadah has quit IRC | 21:13 | |
| *** jschlueter has quit IRC | 21:13 | |
| *** baha has quit IRC | 21:13 | |
| *** migi has quit IRC | 21:13 | |
| *** dsneddon has joined #oooq | 21:13 | |
| *** migi has joined #oooq | 21:13 | |
| *** owalsh^ has joined #oooq | 21:13 | |
| *** cgoncalves has joined #oooq | 21:13 | |
| *** irclogbot_3 has joined #oooq | 21:13 | |
| *** baha has joined #oooq | 21:13 | |
| *** yolanda has joined #oooq | 21:13 | |
| *** tosky has joined #oooq | 21:13 | |
| *** ccamacho has joined #oooq | 21:13 | |
| *** sshnaidm|afk has joined #oooq | 21:13 | |
| *** apetrich has joined #oooq | 21:13 | |
| *** weshay has joined #oooq | 21:13 | |
| *** Tengu has joined #oooq | 21:13 | |
| *** dmellado has joined #oooq | 21:13 | |
| *** matbu has joined #oooq | 21:13 | |
| *** ChanServ has joined #oooq | 21:13 | |
| *** jschlueter has joined #oooq | 21:13 | |
| *** ysandeep has joined #oooq | 21:13 | |
| *** raukadah has joined #oooq | 21:13 | |
| *** dtantsur|afk has joined #oooq | 21:13 | |
| *** gouthamr has joined #oooq | 21:13 | |
| *** tristanC has joined #oooq | 21:13 | |
| *** soniya29 has joined #oooq | 21:13 | |
| *** dmsimard|off has joined #oooq | 21:13 | |
| *** gchamoul has joined #oooq | 21:13 | |
| *** mjturek has joined #oooq | 21:13 | |
| *** EmilienM has joined #oooq | 21:13 | |
| *** arxcruz|rover has joined #oooq | 21:13 | |
| *** jpena|off has joined #oooq | 21:13 | |
| *** pojadhav has joined #oooq | 21:13 | |
| *** bhagyashris has joined #oooq | 21:13 | |
| *** amoralej|off has joined #oooq | 21:13 | |
| *** zbr|pto has joined #oooq | 21:13 | |
| *** jjoyce has joined #oooq | 21:13 | |
| *** panda has joined #oooq | 21:13 | |
| *** orwell.freenode.net sets mode: +ooo ChanServ arxcruz|rover zbr|pto | 21:13 | |
| weshay | rlandy, https://logserver.rdoproject.org/32/25332/46/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/80f6933/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 21:18 |
| weshay | 2020-03-06 21:01:23 | - '[2020/03/06 09:00:53 PM] [WARNING] Error in ''ip route add 0.0.0.0/0 via 10.0.0.1 dev br-ex'', restarting br-ex:' | 21:18 |
| rlandy | weshay: let me check a centos7 deploy | 21:19 |
| rlandy | 10: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1350 qdisc noqueue state UNKNOWN group default qlen 1000 | 21:21 |
| rlandy | inet 10.0.0.26/24 brd 10.0.0.255 scope global br-ex | 21:21 |
| rlandy | no - it's right | 21:21 |
| rlandy | 10.0.0.0/24 dev br-ex proto kernel scope link src 10.0.0.26 | 21:21 |
| rlandy | 2020-03-06 21:22:30.917969 | primary | + rsync --verbose --delay-updates --relative --delete --include 'tripleo-ci-testing**' --exclude '*' ./ uploader@images.rdoproject.org:/var/www/html/images/centos8/master/rdo_trunk/ | 21:27 |
| rlandy | 2020-03-06 21:22:30.919298 | primary | rsync: --delete does not work without --recursive (-r) or --dirs (-d). | 21:27 |
| rlandy | ugh | 21:27 |
| rlandy | weshay: ^^ have to revert that change | 21:27 |
| rlandy | will try again over weekend | 21:28 |
| weshay | k | 21:37 |
| weshay | rlandy, don't work the weekend :) come back so FRESH and so CLEAN CLEAN | 22:01 |
| rlandy | weshay: lol don't worry - I'm fine | 22:03 |
| *** rlandy has quit IRC | 22:33 | |
| *** Tengu has quit IRC | 22:39 | |
| *** Tengu has joined #oooq | 22:39 | |
| *** Tengu has quit IRC | 22:43 | |
| *** Tengu has joined #oooq | 22:44 | |
| *** tosky has quit IRC | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!