| *** Goneri has quit IRC | 00:09 | |
| *** jmasud has joined #oooq | 01:30 | |
| *** jmasud has quit IRC | 01:40 | |
| *** rfolco has joined #oooq | 02:24 | |
| *** rfolco has quit IRC | 02:29 | |
| *** ykarel|away is now known as ykarel | 03:55 | |
| *** skramaja has joined #oooq | 03:58 | |
| *** ratailor has joined #oooq | 04:42 | |
| *** udesale has joined #oooq | 04:44 | |
| *** jtomasek has joined #oooq | 04:49 | |
| *** pojadhav|away is now known as pojadhav|ruck | 04:53 | |
| *** jmasud has joined #oooq | 05:10 | |
| *** saneax has joined #oooq | 05:18 | |
| *** jmasud has quit IRC | 05:36 | |
| *** jmasud has joined #oooq | 05:44 | |
| *** jmasud has quit IRC | 06:00 | |
| *** irclogbot_1 has quit IRC | 06:05 | |
| *** irclogbot_0 has joined #oooq | 06:06 | |
| *** marios has joined #oooq | 06:08 | |
| *** jmasud has joined #oooq | 06:24 | |
| *** ysandeep is now known as ysandeep|afk | 06:31 | |
| *** apetrich has joined #oooq | 06:40 | |
| *** jfrancoa has joined #oooq | 06:41 | |
| *** apetrich has quit IRC | 06:45 | |
| *** apetrich has joined #oooq | 06:49 | |
| *** ykarel is now known as ykarel|afk | 06:56 | |
| *** ykarel|afk is now known as ykarel | 07:00 | |
| *** jbadiapa has joined #oooq | 07:18 | |
| chandankumar | cgoncalves, Hello | 07:27 |
|---|---|---|
| chandankumar | cgoncalves, I am running full octavia tempest here https://logserver.rdoproject.org/12/27912/3/check/periodic-tripleo-ci-centos-8-scenario010-standalone-master/baeb73f/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 07:27 |
| chandankumar | cgoncalves, anything we can do on tripleo side to fix Details: b'{"faultcode": "Server", "faultstring": "Provider \'octavia\' reports error: Nova availability zone my_compute_zone not found.", "debuginfo": null}' | 07:28 |
| *** tosky has joined #oooq | 07:28 | |
| chandankumar | it will cover most of the tests | 07:28 |
| *** amoralej|off is now known as amoralej | 07:30 | |
| cgoncalves | chandankumar, hey. sorry I did not reply to your messages on Thursday, I was out of office | 07:39 |
| *** yolanda has joined #oooq | 07:40 | |
| cgoncalves | chandankumar, those are issues in the test code when run against a provider driver that is not the noop. given those are API tests, what do you think about just running the scenario set? | 07:40 |
| chandankumar | cgoncalves, no problem, yes sure I can do that | 07:41 |
| chandankumar | let me update the patch with scenario tests | 07:41 |
| -openstackstatus- NOTICE: uWSGI made a new release that breaks devstack, please refrain from rechecking until a devstack fix is merged. | 07:41 | |
| cgoncalves | thanks! | 07:42 |
| *** jmasud has quit IRC | 07:49 | |
| *** dtantsur|afk is now known as dtantsur | 07:51 | |
| *** ykarel is now known as ykarel|lunch | 08:04 | |
| *** chem has joined #oooq | 08:17 | |
| *** ccamacho has joined #oooq | 08:36 | |
| *** ysandeep|afk is now known as ysandeep | 08:41 | |
| *** sanjayu_ has joined #oooq | 08:47 | |
| *** ykarel|lunch is now known as ykarel | 08:49 | |
| *** saneax has quit IRC | 08:50 | |
| *** sanjayu_ has quit IRC | 08:53 | |
| *** jmasud has joined #oooq | 09:16 | |
| chandankumar | marios, https://tree.taiga.io/project/tripleo-ci-board/us/1779 it will help | 10:00 |
| marios | chandankumar: thanks replied, but we already have files: there | 10:02 |
| chandankumar | marios, ah, need to wait for weshay|ruck to come back and then discuss | 10:09 |
| marios | chandankumar: i think we can just use the task to 'tweak' files/add irrelevant-files if useful etc | 10:10 |
| chandankumar | marios, you are looking at periodic version of the jobs card? | 10:10 |
| marios | chandankumar: to restrict | 10:10 |
| marios | chandankumar: yeah i started digging there | 10:10 |
| chandankumar | marios, ok | 10:10 |
| *** ccamacho has quit IRC | 10:20 | |
| *** saneax has joined #oooq | 10:22 | |
| *** jmasud has quit IRC | 10:25 | |
| chandankumar | marios, ykarel can we https://review.rdoproject.org/r/#/c/28069/ merge this? | 10:32 |
| ykarel | chandankumar, i had a question there, can u confirm that | 10:33 |
| chandankumar | ykarel, yes it | 10:34 |
| chandankumar | ykarel, commented back | 10:34 |
| ykarel | ack +@ | 10:35 |
| ykarel | +2 | 10:35 |
| chandankumar | pojadhav|ruck, fyi https://review.rdoproject.org/r/28069 will fix sc10 c7 issue | 10:35 |
| *** derekh has joined #oooq | 10:35 | |
| chandankumar | brb for 30 mins | 10:36 |
| pojadhav|ruck | chandankumar, ack | 10:36 |
| marios | chandankumar: will check in a sec | 10:49 |
| marios | arxcruz: omnom nom perfect grill time ;) | 10:49 |
| ysandeep | chandankumar, marios In downstream jobs are failing while trying to pull container.. I noticed our container build push failed with - unauthorized: authentication required. Any pointers what can I look to troubleshoot this authentication issue. | 10:53 |
| ysandeep | https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-rhos-17/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-containers-rhel-8-rhos-17-build-push/1af7fd2/job-output.txt | 10:53 |
| *** ccamacho has joined #oooq | 10:56 | |
| marios | ysandeep: could be that rlandy changed the secret? | 10:57 |
| marios | ysandeep: it was added last sprint for the containers push jobs | 10:57 |
| marios | ysandeep: 2020-06-15 00:51:52.691686 | primary | Login Succeeded! | 10:58 |
| marios | hmm strange one | 10:58 |
| ysandeep | marios, yes initial login was successfull | 10:58 |
| marios | ysandeep: is it happening consistently? | 10:58 |
| marios | ysandeep: possible there is something wrong with the registry (might ask in rhos-ops ) | 10:58 |
| marios | ysandeep: and all previous image above it were pushed OK 2020-06-15 00:51:49.590252 | primary | Writing manifest to image destination | 10:59 |
| ysandeep | marios, thank you, let me trigger a test run and if it failed with same error - i will ping the infra guys | 10:59 |
| marios | ysandeep: so definitly not something wrong with the secret anyway | 11:00 |
| ysandeep | marios, okay, many other job failing trying to pull images 404 not found - https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-cloudops/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-scenario001-standalone-cloudops-rhos-17/aba7fb7/logs/undercloud/home/zuul/standalone_deploy.log | 11:02 |
| arxcruz | marios: ohhhh how i hate you now... | 11:03 |
| ysandeep | marios, triggered test run - https://code.engineering.redhat.com/gerrit/#/c/195884/ | 11:05 |
| ysandeep | thank you for pointers :) | 11:05 |
| marios | ysandeep: ack maybe registry issue then? | 11:05 |
| marios | ysandeep: you can check the registry for the image sec leme find url | 11:06 |
| marios | ysandeep: check pvt | 11:06 |
| marios | ysandeep: worth asking about that ^^ too ;) | 11:09 |
| ysandeep | ack | 11:09 |
| *** rlandy has joined #oooq | 11:25 | |
| *** ykarel is now known as ykarel|afk | 11:30 | |
| chandankumar | marios, https://review.rdoproject.org/r/#/c/28069/ replied thanks! | 11:30 |
| *** derekh has quit IRC | 11:31 | |
| marios | chandankumar: thank you | 11:32 |
| rlandy | weshay|ruck: pojadhav|ruck: https://review.opendev.org/#/c/735253/ | 11:34 |
| rlandy | we were triggering off the wring role name | 11:34 |
| rlandy | ^^ patch is needed to clear the IPA job in downstream | 11:35 |
| *** derekh has joined #oooq | 11:37 | |
| ysandeep | rlandy, very good morning , fyi.. container build push failed with "unauthorized: authentication required" for a image , it worked some other images. I retrigger testproject run "periodic-tripleo-containers-rhel-8-rhos-17-build-push" to confirm if failure is consistent. | 11:38 |
| ysandeep | https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-rhos-17/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-containers-rhel-8-rhos-17-build-push/1af7fd2/job-output.txt | 11:38 |
| rlandy | marios: chandankumar: ^^ could I get one more vote on https://review.opendev.org/#/c/735253/? | 11:38 |
| *** rfolco has joined #oooq | 11:38 | |
| rlandy | ysandeep: it happens from time to time. Th rerun should fix it | 11:39 |
| ysandeep | rlandy, ack .. Some other jobs were failing trying to pull images too - 404 not found | 11:40 |
| rlandy | panda: hi - when do you want to try a downstream promotion? | 11:40 |
| marios | rlandy: merge?# | 11:41 |
| rlandy | ysandeep: jobs in the integration line or component lines? | 11:41 |
| rlandy | marios: I w'ed - thanks | 11:42 |
| marios | rlandy: ack | 11:42 |
| ysandeep | rlandy, i faced for component pipeline - https://sf.hosted.upshift.rdu2.redhat.com/logs/30/201630/8/check/periodic-tripleo-ci-rhel-8-bm_envA-3ctlr_1comp-featureset001-baremetal-rhos-17/2879086/logs/undercloud/home/zuul/undercloud_install.log | 11:44 |
| rlandy | ysandeep: I see that - checking connection | 11:44 |
| rlandy | ysandeep: yep - registry issue is already reported in outages | 11:45 |
| rlandy | ysandeep: https://one.redhat.com/outages/list/details/5ee7535cfdce98a796245b7a | 11:46 |
| ysandeep | rlandy, ahh.. okay i will bookmark this link.. thanks! | 11:46 |
| rlandy | rfolco: there is scrum today? | 11:48 |
| *** pojadhav|ruck is now known as pojadhav|brb | 11:49 | |
| rfolco | rlandy, I don't know, there is one left in calendar, thursday is planning | 11:49 |
| *** rfolco is now known as rfolco|rover | 11:50 | |
| rlandy | ysandeep: you've also tested https://review.opendev.org/#/c/732787/, right? | 11:55 |
| rlandy | if so, we can merge that one | 11:55 |
| rlandy | I merged the tripleo-environments patch | 11:55 |
| rlandy | pls vote on ^^ . thanks | 11:55 |
| weshay|ruck | zbr, can you please read through https://bugs.launchpad.net/tripleo/+bug/1883430 | 11:56 |
| openstack | Launchpad bug 1883430 in tripleo "All CentOS upstream and rdo third party jobs broken with Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried." [Critical,Triaged] | 11:56 |
| zbr | looking | 11:57 |
| ysandeep | rlandy, https://review.opendev.org/#/c/732787/ +1 | 11:57 |
| rlandy | chandankumar: could I get another vote on ^^ | 11:57 |
| rlandy | ysandeep: thanks - let me know when you confirm the last patch | 11:57 |
| zbr | weshay|ruck: this needs to be posted to #openstack-infra i think. | 11:57 |
| rlandy | we'll pick one then | 11:57 |
| zbr | counts as broken mirror | 11:58 |
| chandankumar | rlandy, done | 11:58 |
| weshay|ruck | yup | 11:58 |
| chandankumar | rlandy, marios https://code.engineering.redhat.com/gerrit/#/c/201152/ | 11:58 |
| rlandy | chandankumar: thanks | 11:58 |
| weshay|ruck | they are aware of it, maybe just not that it came back a bit | 11:58 |
| ysandeep | rlandy, i will let you know as soon as registry issue solves | 11:58 |
| rlandy | ysandeep: cool - thanks | 11:58 |
| rlandy | let's close this out | 11:58 |
| rlandy | chandankumar: wrt - this is a good idea for general patches - not just build test | 11:59 |
| rlandy | the only reason we are still using the compose it to get the mock deps | 11:59 |
| rlandy | if we can use your patch we can go back to using rhos-release ceph -r 8.2 | 12:00 |
| zbr | weshay|ruck: there was notifce on openstack-infra, solution recheck. | 12:00 |
| chandankumar | rlandy, ok, I will check with jpena|off tomorrow to include mock under deps repo as we do in rdo | 12:01 |
| chandankumar | rlandy, using rhos-release ceph -r 8.2 will cause any issue | 12:01 |
| chandankumar | ? | 12:01 |
| rlandy | chandankumar: I asked jpena about including mock in osp-trunk | 12:01 |
| rlandy | he said only rdo so far | 12:01 |
| zbr | apparently this was yesterday and fixed yesterday. | 12:02 |
| rlandy | chandankumar: atm yes - although we have to get rid of that compose step | 12:02 |
| rlandy | because it will fail on mock | 12:02 |
| chandankumar | zbr, issue was seen today as per ykarel|afk | 12:02 |
| zbr | i added "openstackstatus" to irc client keywords, i think we all need to know about these. | 12:02 |
| chandankumar | rlandy, ok | 12:02 |
| *** amoralej is now known as amoralej|lunch | 12:03 | |
| rlandy | chandankumar: am I making sense here? see ... | 12:03 |
| rlandy | line 119 | 12:04 |
| rlandy | dnf -y install mock | 12:04 |
| chandankumar | rlandy, curl -L -O https://download.devel.redhat.com/rel-eng/RCMTOOLS/rcm-tools-rhel-8-baseos.repo | 12:04 |
| rlandy | that line fails w/o rhos_release_args: "-P 17 -p latest-RHOS_TRUNK-17.0-RHEL-8/ -r 8.2" | 12:04 |
| chandankumar | yes making sense | 12:04 |
| rlandy | chandankumar: so you're saying if we add rcm tools we can get rid of compose | 12:05 |
| rlandy | let me try that | 12:05 |
| rlandy | chandankumar: +2 on your patch | 12:05 |
| chandankumar | rlandy, sorry confused | 12:05 |
| chandankumar | rlandy, gmeet? | 12:06 |
| *** beagles has joined #oooq | 12:06 | |
| rlandy | yeah | 12:06 |
| chandankumar | rlandy, https://meet.google.com/_meet/avw-vjdx-bxm?hs=197&pli=1&authuser=0&ijlm=1592222795672&adhoc=1 | 12:06 |
| *** ysandeep is now known as ysandeep|afk | 12:07 | |
| chandankumar | https://code.engineering.redhat.com/gerrit/200084 | 12:10 |
| rlandy | chandankumar: https://code.engineering.redhat.com/gerrit/#/c/200707/ | 12:13 |
| rfolco|rover | weshay|ruck, hi o/ | 12:14 |
| weshay|ruck | hey rfolco|rover | 12:14 |
| rfolco|rover | weshay|ruck, people are pinging me about scrum... I left it in calendar because this week we have retro/planning, so today is the only day for scrum | 12:15 |
| rfolco|rover | weshay|ruck, do you think we should keep scrum ? | 12:15 |
| weshay|ruck | rfolco|rover, I think we can be quick, shouldn't be much new news since thrs | 12:15 |
| rfolco|rover | k | 12:15 |
| rfolco|rover | thx weshay|ruck | 12:15 |
| *** pojadhav|brb is now known as pojadhav|ruck | 12:23 | |
| weshay|ruck | rfolco|rover, pojadhav|ruck sorry.. wrong channel | 12:27 |
| weshay|ruck | http://rhos-release.virt.bos.redhat.com:3030/rhosp | 12:27 |
| weshay|ruck | let's get train diagnosed | 12:27 |
| weshay|ruck | then master | 12:27 |
| weshay|ruck | then the others | 12:27 |
| rfolco|rover | ack weshay|ruck | 12:28 |
| pojadhav|ruck | weshay|ruck, ack | 12:28 |
| *** derekh has quit IRC | 12:28 | |
| rfolco|rover | apparently same issues ... scen10, fs020... | 12:28 |
| *** derekh has joined #oooq | 12:29 | |
| rlandy | ysandeep|afk: awesome new son the bm success!! | 12:29 |
| rlandy | news | 12:29 |
| weshay|ruck | rfolco|rover, pojadhav|ruck one of you need to join cix :) | 12:31 |
| rfolco|rover | weshay|ruck, o/ | 12:31 |
| pojadhav|ruck | weshay|ruck, https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-scenario010-standalone-train (failing consistently with load balancer test which will be fix using chandan's patch https://review.rdoproject.org/r/#/c/28069/) | 12:37 |
| chandankumar | cgoncalves, Hello | 12:39 |
| cgoncalves | o/ | 12:39 |
| chandankumar | cgoncalves, based on scenario testing https://logserver.rdoproject.org/12/27912/5/check/periodic-tripleo-ci-centos-8-scenario010-standalone-master/6029d95/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 12:39 |
| chandankumar | cgoncalves, {0} setUpClass (octavia_tempest_plugin.tests.scenario.v2.test_traffic_ops.TrafficOperationsScenarioTest) [0.000000s] ... FAILED except this one | 12:39 |
| chandankumar | we are good with running octavia scenario | 12:39 |
| chandankumar | cgoncalves, I will file a bug and move it to skip list | 12:40 |
| cgoncalves | chandankumar, cool! | 12:40 |
| cgoncalves | chandankumar, file a bug for those failed tests? | 12:40 |
| chandankumar | cgoncalves, yes | 12:40 |
| chandankumar | so that we can track in skip list | 12:40 |
| chandankumar | cgoncalves, for each octavia jobs, we will be running network basic ops + octavia scenario + neutorn scenario | 12:41 |
| cgoncalves | chandankumar, ok. I still think something is fishy on the network side | 12:41 |
| chandankumar | in periodic | 12:41 |
| chandankumar | and in new kvm job | 12:41 |
| cgoncalves | why network basic ops + neutron scenario? | 12:41 |
| weshay|ruck | kopecmartin, wf? https://review.opendev.org/#/c/733154/1 | 12:42 |
| *** udesale_ has joined #oooq | 12:42 | |
| chandankumar | cgoncalves, last time we runned those test to check FIP issue, but those are working fine | 12:42 |
| chandankumar | so I think we can add those tests also there | 12:42 |
| chandankumar | if not then I can go with octavia scenario only | 12:43 |
| cgoncalves | chandankumar, octavia test_traffic_ops tests should cover FIP traffic too | 12:43 |
| chandankumar | cgoncalves, ok then | 12:43 |
| chandankumar | cgoncalves, thanks! | 12:43 |
| cgoncalves | ... but they are failing... | 12:43 |
| cgoncalves | neutron FIP tests probably do not cover tempest<->cloud traffic, just vm<->vm | 12:44 |
| chandankumar | cgoncalves, may be having a bug will help to diagnois this issue | 12:44 |
| *** udesale has quit IRC | 12:44 | |
| cgoncalves | ok | 12:44 |
| kopecmartin | weshay|ruck: i would but i don't want to ignore the masayuki comment about single vendor approval .. I'll get another core vote | 12:44 |
| *** ysandeep|afk is now known as ysandeep | 12:45 | |
| chandankumar | cgoncalves, once this https://review.rdoproject.org/r/#/c/27636/ merges, I will create a check job which will triggered against octavia upstream patches | 12:45 |
| chandankumar | as a third party | 12:46 |
| *** ykarel|afk is now known as ykarel | 12:46 | |
| cgoncalves | chandankumar, meaning RDO will run scn010 job for each octavia-tempest-plugin patch set and report there? | 12:48 |
| chandankumar | cgoncalves, yes | 12:48 |
| weshay|ruck | rlandy, how's psi | 12:48 |
| chandankumar | rdo third party jobs will leave votes +1 or -1 | 12:49 |
| cgoncalves | chandankumar, hmm, maybe it would be best to approach the (upstream) octavia team first. | 12:50 |
| chandankumar | cgoncalves, sure | 12:51 |
| rlandy | weshay|ruck: I was going to report back at scrum - if we are still having that ... | 12:54 |
| rlandy | weshay|ruck: I edited the BZ ... the job times are back down to regular times after http://git.app.eng.bos.redhat.com/git/openstack/sf-config.git/commit/?id=e881edfbf2d94cdaff60065730a4a4abfd37d40e merged | 12:54 |
| rlandy | I am making the same change on the centos8 image | 12:55 |
| rlandy | I checked the CIX - no card there - so it must have moved? | 12:55 |
| rlandy | anyways, the registry is having an outage today | 12:55 |
| rlandy | so something is always out | 12:55 |
| *** ratailor has quit IRC | 12:56 | |
| rlandy | weshay|ruck: https://one.redhat.com/outages/list/details/5ee7535cfdce98a796245b7a is our problem now | 12:56 |
| marios | rfolco|rover: so we scrum today or not? only thursdays now? | 13:00 |
| rlandy | marios: would like to touch base with you about image upload after scrum | 13:03 |
| marios | arxcruz: there is scrum :D | 13:08 |
| marios | in the end... | 13:08 |
| chandankumar | kopecmartin, good to go ahead with tempest bump | 13:09 |
| weshay|ruck | panda, talking about ci-config tests | 13:11 |
| weshay|ruck | in scrum | 13:11 |
| panda | scrum ? | 13:12 |
| *** amoralej|lunch is now known as amoralej | 13:20 | |
| marios | rfolco|rover: :D | 13:44 |
| marios | rfolco|rover: obviously i can only do that cos you are in a different continent right now | 13:44 |
| rlandy | marios: hey - have a few to chat with me re: image/dockerfile upload? at the top of the hour? | 13:45 |
| rfolco|rover | marios, have you heard about voodoo ? | 13:45 |
| marios | rfolco|rover: ack yes | 13:45 |
| marios | rlandy: ^^^ ack | 13:45 |
| * marios retrieves the rfolco|rover doll | 13:45 | |
| *** ysandeep is now known as ysandeep|afk | 13:46 | |
| marios | rlandy: can we push it back to half passed in 40 mins ok? | 13:50 |
| chandankumar | weshay|ruck, https://github.com/pycontribs/jenkinsapi | 13:51 |
| sshnaidm_ | weshay|ruck, rfolco|rover arxcruz reproducer if functional again, thanks to bogdando | 13:54 |
| marios | rlandy: actually nm top of hour is fine | 13:54 |
| rfolco|rover | sshnaidm_, ack thanks | 13:54 |
| *** Goneri has joined #oooq | 13:57 | |
| rlandy | marios: 40 mins is fine - whatever works for you | 14:00 |
| marios | rlandy: ready now let's go | 14:01 |
| rlandy | akc | 14:01 |
| rlandy | ack | 14:01 |
| rlandy | marios: https://meet.google.com/cqp-dzib-brp | 14:02 |
| marios | rlandy: ack joining | 14:02 |
| weshay|ruck | kopecmartin, 0/ https://review.opendev.org/#/c/733154/ | 14:18 |
| chandankumar | rlandy, Hello | 14:22 |
| chandankumar | rlandy, https://sf.hosted.upshift.rdu2.redhat.com/logs/84/200084/32/check/periodic-tripleo-ci-rhel-8-standalone-rhos-17/ac81363/job-output.txt | 14:22 |
| chandankumar | rlandy, with mock we are hitting issue | 14:23 |
| chandankumar | rlandy, are you seeing the same issue there? | 14:23 |
| chandankumar | - nothing provides python3-distro needed by mock-1.4.14-2.el8eng.noarch | 14:23 |
| kopecmartin | weshay|ruck: done | 14:23 |
| rlandy | marios: https://code.engineering.redhat.com/gerrit/#/c/202161/ | 14:23 |
| rlandy | https://code.engineering.redhat.com/gerrit/#/c/203146/ | 14:23 |
| marios | rlandy: ack | 14:24 |
| rlandy | chandankumar: ack | 14:24 |
| rlandy | chandankumar: mock and it's dep | 14:24 |
| rlandy | its | 14:24 |
| ykarel | rlandy, can u review https://review.opendev.org/#/c/735210/ | 14:24 |
| chandankumar | rlandy, will try ti fix htat | 14:25 |
| chandankumar | ykarel, seen this issue https://95088f20884767491109-1637f39d4df174e5d3ef0356ebf65767.ssl.cf2.rackcdn.com/735111/1/check/tripleo-ci-centos-8-containers-multinode/cd99956/job-output.txt | 14:26 |
| chandankumar | Cannot download Packages/python3-chardet-3.0.4-7.el8.noarch.rpm: All mirrors were tried | 14:26 |
| rlandy | ykarel: looks ok - I see your answer to the stein/queens questions | 14:27 |
| ykarel | chandankumar, this is the infra issue centos repo is not there, | 14:27 |
| ykarel | chandankumar, this is same i mentioned u some time back | 14:27 |
| rlandy | chandankumar++ thanks | 14:27 |
| chandankumar | ykarel, ok thanks! | 14:27 |
| ykarel | chandankumar, centos missing in https://mirror.ca-ymq-1.vexxhost.opendev.org/ | 14:27 |
| *** pojadhav|ruck is now known as pojadhav|brb | 14:28 | |
| chandankumar | ykarel, ah they are working on that | 14:28 |
| ykarel | yes | 14:28 |
| *** marios has quit IRC | 14:30 | |
| *** marios has joined #oooq | 14:37 | |
| weshay|ruck | pojadhav++ | 14:38 |
| chandankumar | weshay|ruck, rfolco|rover pojadhav|brb we are bumping tempest to 24.0.0 in train https://review.rdoproject.org/r/#/c/26746/ if you see any hiccups let me and kopecmartin know | 14:38 |
| chandankumar | kopecmartin++ nice work sir :-) | 14:38 |
| weshay|ruck | thanks for the heads up | 14:39 |
| rlandy | weshay|ruck: CIX email sent - left the BZ assigned to me as I am watching the outage updates | 14:44 |
| weshay|ruck | ack thanks | 14:45 |
| chandankumar | weshay|ruck, https://review.rdoproject.org/r/#/c/27636/ let me update this then merge | 14:48 |
| pojadhav|brb | chandankumar, ack | 14:49 |
| chandankumar | weshay|ruck, done | 14:51 |
| chandankumar | rlandy, ysandeep|afk you might interested in adding these jobs to downstream octavia pipeline also https://review.rdoproject.org/r/27636 | 14:52 |
| rlandy | chandankumar: ack | 14:52 |
| rlandy | chandankumar: would we have to wait for the depends jobs to be merged? | 14:53 |
| chandankumar | rlandy, those are in gates | 14:53 |
| chandankumar | will be merged soon | 14:53 |
| *** pojadhav|brb is now known as pojadhav|ruck | 14:54 | |
| rlandy | chandankumar: ack -so tomorrow then | 14:54 |
| *** ysandeep|afk is now known as ysandeep | 14:56 | |
| *** ykarel is now known as ykarel|away | 14:56 | |
| chandankumar | rlandy, https://review.rdoproject.org/r/#/c/28068/ | 14:58 |
| ysandeep | chandankumar, ack i will add job in downstream , will try them tomorrow | 14:58 |
| *** ratailor has joined #oooq | 15:01 | |
| chandankumar | ysandeep, good | 15:01 |
| ysandeep | chandankumar, only in integraion pipeline or it will go in component pipeline too? | 15:02 |
| chandankumar | ysandeep, it will go to component pipeline also | 15:02 |
| rlandy | chandankumar: ok to merge https://review.rdoproject.org/r/#/c/28068/? | 15:03 |
| chandankumar | rlandy, yes | 15:04 |
| rlandy | done | 15:05 |
| amoralej | i'm rebuilding openvswitch for centos 8.2 which has been just pushed | 15:16 |
| chandankumar | rlandy, weshay|ruck https://review.opendev.org/#/c/727200/ this one is also good to go | 15:16 |
| amoralej | weshay|ruck, ^ | 15:16 |
| chandankumar | amoralej, thanks! | 15:16 |
| chandankumar | amoralej, let pojadhav|ruck and rfolco|rover know once done | 15:17 |
| chandankumar | rlandy, weshay|ruck https://review.opendev.org/#/c/727200/ is good to go | 15:24 |
| rfolco|rover | chandankumar, can you help me understand where else I need to change to override cirros ? https://review.opendev.org/#/c/733676/ plus https://review.opendev.org/#/c/733862/ did not make it yet. | 15:25 |
| chandankumar | rfolco|rover, let me check | 15:26 |
| *** pojadhav|ruck is now known as pojadhav|dinner | 15:29 | |
| rlandy | will look in a bit | 15:32 |
| chandankumar | rfolco|rover, can we put a depends-on https://review.opendev.org/#/c/733862 on this patch https://review.opendev.org/#/c/733676 | 15:33 |
| rfolco|rover | chandankumar, did not work | 15:33 |
| rfolco|rover | https://review.rdoproject.org/r/#/c/27878/ | 15:33 |
| rfolco|rover | chandankumar, even with these 2 patches, cirros is still 0.4 | 15:34 |
| rfolco|rover | chandankumar, need to change os_tempest? featureset file ? what? | 15:34 |
| chandankumar | rfolco|rover, checking which one is overriding it | 15:34 |
| chandankumar | rfolco|rover, here https://review.opendev.org/#/c/733676/1/playbooks/tempest.yml | 15:38 |
| chandankumar | rfolco|rover, for train you need 0.4? | 15:38 |
| chandankumar | rfolco|rover, https://review.opendev.org/#/c/733676/1/playbooks/tempest.yml@65 https://logserver.rdoproject.org/78/27878/7/check/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-train-new-cirros-3/27973eb/logs/undercloud/home/zuul/tempest/etc/profile.yaml.txt.gz | 15:39 |
| chandankumar | for train 0.4 is used | 15:39 |
| chandankumar | rfolco|rover, if we need 5 we need to modify that condition | 15:39 |
| rlandy | marios: how was this file working for downstream images?? https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/ci-scripts/tripleo-upstream/upload-cloud-images.sh | 15:39 |
| rfolco|rover | chandankumar, oooops I tested in the wrong testproject | 15:40 |
| rfolco|rover | chandankumar, I am so sorry | 15:40 |
| rlandy | #If set by downstream/internal SF jobs don't override UPLOAD_URL? | 15:40 |
| rfolco|rover | chandankumar, I will add both on this one https://review.rdoproject.org/r/#/c/27902/ | 15:40 |
| rlandy | oh - the override | 15:43 |
| marios | rlandy: wdym | 15:48 |
| marios | rlandy: what do you mean? is thre some bug? | 15:48 |
| marios | there | 15:48 |
| ysandeep | chandankumar, fyi.. Posted patch definitions and component jobs - https://code.engineering.redhat.com/gerrit/#/c/203233/ , will test once PSI outage resolves and in the meantime your depends-on patch will merge too | 15:49 |
| ysandeep | job* definations | 15:49 |
| rfolco|rover | chandankumar, wait... it should have worked, the default is 0.4 for train, but I am overriding it | 15:49 |
| rfolco|rover | chandankumar, https://review.rdoproject.org/r/#/c/27878/7/.zuul.yaml | 15:50 |
| rlandy | marios: np - I see the override | 15:50 |
| marios | rlandy: https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/tripleo-ci-internal-config.git;a=blob;f=playbooks/tripleo-ci-periodic-base/upload-images.yaml;h=2f87d0b88d6459362851fddc13ca9eef268f5b0b;hb=c8fe82a5c7f18ec9d8b3182a79613818615d5545#l4 | 15:50 |
| marios | rlandy: wired up there ^^ | 15:50 |
| marios | rlandy: ack | 15:52 |
| *** ratailor has quit IRC | 15:53 | |
| chandankumar | rfolco|rover, but that condition is wining | 15:55 |
| chandankumar | rfolco|rover, set_fact | 15:55 |
| chandankumar | better remove that condition then run ut | 15:55 |
| rfolco|rover | chandankumar, | 15:56 |
| rfolco|rover | - name: set cirros image facts for periodic (stable branches) | 15:56 |
| rfolco|rover | vars: | 15:56 |
| rfolco|rover | default_cirros_image: http://images.rdoproject.org/cirros/cirros-0.4.0-x86_64-disk.img | 15:56 |
| rfolco|rover | set_fact: | 15:56 |
| rfolco|rover | tempest_tempestconf_profile_extras: | 15:56 |
| rfolco|rover | image: "{{ featureset_override['cirros_image'] | default(default_cirros_image) }}" | 15:56 |
| rfolco|rover | when: release != 'master' | 15:56 |
| chandankumar | rfolco|rover, but jobs defined for train na | 15:57 |
| chandankumar | need to take a look deeply what is causing that | 15:58 |
| rfolco|rover | I am still confused, let me switch to other more urgent matter then we discuss this again later | 16:01 |
| marios | sshnaidm_: o/ i wanted to catch you tomorrow at some point if you have time. i can't see why the role is ignoring the params at https://7114429c7841114b6ab2-a5faeab2c736b5ae3f187165e14e12eb.ssl.cf5.rackcdn.com/733818/15/check/tripleo-buildimage-overcloud-full-centos-8/9fecf00/job-output.txt " ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path" | 16:04 |
| sshnaidm_ | marios, ack, ping me tomorrow | 16:05 |
| marios | sshnaidm_: for https://review.opendev.org/#/c/733818/ ... i got it to eventually find the role but now it runs it looks like it ignores params and uses module defaults. anyway grateful if something obvious pops out at you please comment, or we can discuss tomorrow | 16:05 |
| *** sshnaidm_ is now known as sshnaidm | 16:05 | |
| marios | sshnaidm_: thanks | 16:05 |
| *** dsneddon has joined #oooq | 16:11 | |
| *** marios is now known as marios|out | 16:12 | |
| amoralej | rfolco|rover, chandankumar i've applied a workaround by shipping dpdk from centos 8.1 in deps repo until we have ready the new ovs for 8.2 | 16:12 |
| amoralej | that should prevent any issue with 8.2 | 16:13 |
| *** ysandeep is now known as ysandeep|away | 16:13 | |
| amoralej | but let me know if you notice some other issue | 16:13 |
| amoralej | that may be related to 8.2 | 16:13 |
| *** marios|out has quit IRC | 16:20 | |
| chandankumar | amoralej, sure | 16:21 |
| *** dtantsur is now known as dtantsur|afk | 16:22 | |
| chandankumar | weshay|ruck, fix for blacklisted test is also up https://review.opendev.org/735631 | 16:24 |
| *** udesale_ has quit IRC | 16:26 | |
| chandankumar | rlandy, weshay|ruck https://review.opendev.org/#/c/727200/ please +2 and +w it | 16:26 |
| rfolco|rover | amoralej, ack | 16:29 |
| *** jmasud has joined #oooq | 16:32 | |
| *** rlandy is now known as rlandy|brb | 16:32 | |
| rfolco|rover | chandankumar, where is your scen10 experiment w/ octavia plugin ? | 16:41 |
| rfolco|rover | I think I found it https://review.opendev.org/#/c/735512/1 | 16:47 |
| *** skramaja has quit IRC | 16:55 | |
| *** rlandy|brb is now known as rlandy | 17:03 | |
| *** derekh has quit IRC | 17:08 | |
| rlandy | weshay|ruck: downstream registry is accessible again - closed out BZ and CIX | 17:09 |
| weshay|ruck | ++ | 17:09 |
| weshay|ruck | rlandy, keep the bug handy.. so we can just reopen ;) | 17:09 |
| weshay|ruck | when needed | 17:09 |
| rlandy | weshay|ruck: https://bugzilla.redhat.com/show_bug.cgi?id=1847057 - rerunning container build/push now | 17:10 |
| openstack | bugzilla.redhat.com bug 1847057 in releng "Container push jobs are not able to login into https://registry-console-default.cloud.registry.upshift.redhat.com/registry" [Unspecified,Closed: notabug] - Assigned to rlandy | 17:10 |
| *** jmasud has quit IRC | 17:20 | |
| *** jmasud has joined #oooq | 17:21 | |
| *** amoralej is now known as amoralej|off | 17:32 | |
| chandankumar | rfolco|rover, passing | 17:42 |
| chandankumar | you got it | 17:42 |
| rfolco|rover | chandankumar, the 2 patches merged, not sure what else is missing https://logserver.rdoproject.org/91/28091/1/check/periodic-tripleo-ci-centos-8-scenario010-kvm-standalone-train/4b0740c/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 17:44 |
| rfolco|rover | FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/tripleo-heat-installer-templates/ci/environments/octavia-kvm.yaml' | 17:44 |
| weshay|ruck | rfolco|rover, any skip list action on https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/7a4c2dc/logs/undercloud/var/log/tempest/stestr_results.html.gz ? | 17:57 |
| * weshay|ruck checks hackmd | 17:57 | |
| weshay|ruck | chandankumar, fyi https://hackmd.io/yjfkCbnZTgGQOikRdmvhrA | 17:57 |
| rfolco|rover | weshay|ruck, fs020 is so inconsistent | 17:57 |
| rfolco|rover | weshay|ruck, not failing the same tests | 17:58 |
| weshay|ruck | k | 17:58 |
| weshay|ruck | rfolco|rover, reviewing for promotion https://trunk.rdoproject.org/api-centos8-master-uc/api/civotes_agg_detail.html?ref_hash=091acfaaffdad2d7471aeaf0759f3fbd | 17:58 |
| rfolco|rover | weshay|ruck, oh wait, you're right, last 3 failures are exactly the same tests | 18:01 |
| rfolco|rover | https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/fafba23/logs/undercloud/var/log/tempest/stestr_results.html.gz | 18:01 |
| rfolco|rover | https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/3a8abbe/logs/undercloud/var/log/tempest/stestr_results.html.gz | 18:01 |
| rfolco|rover | and | 18:01 |
| rfolco|rover | https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/7a4c2dc/logs/undercloud/var/log/tempest/stestr_results.html.gz | 18:01 |
| rfolco|rover | weshay|ruck, its fs020-train that I can't draw a pattern | 18:02 |
| rfolco|rover | which I believe its a resource problem, since most of my check jobs ran fine. | 18:03 |
| weshay|ruck | rfolco|rover, master is promoting now | 18:09 |
| weshay|ruck | pojadhav|dinner, ^ | 18:10 |
| rfolco|rover | weshay|ruck, ignorning fs020 and scen10 ? | 18:10 |
| weshay|ruck | scen010 passed | 18:10 |
| rfolco|rover | ovn | 18:10 |
| rfolco|rover | ah out of criteria already | 18:10 |
| rfolco|rover | ok | 18:10 |
| weshay|ruck | https://trunk.rdoproject.org/api-centos8-master-uc/api/civotes_agg_detail.html?ref_hash=091acfaaffdad2d7471aeaf0759f3fbd | 18:10 |
| rfolco|rover | great | 18:10 |
| rfolco|rover | what we do about scen10 and fs020 on train side? | 18:11 |
| rfolco|rover | fs020 mostly pass on my testproject so just triggered one here https://review.rdoproject.org/r/#/c/28092/ | 18:11 |
| weshay|ruck | actually this is the hash that is promoting atm https://trunk.rdoproject.org/api-centos8-master-uc/api/civotes_agg_detail.html?ref_hash=0b6cab1e64b0ff91c26420a2b6a25c3d | 18:12 |
| rfolco|rover | and scen10 is failing on loadbalancer | 18:12 |
| weshay|ruck | scen010 ovn or scenario010 | 18:12 |
| rfolco|rover | scen10 | 18:12 |
| weshay|ruck | in check? | 18:12 |
| rfolco|rover | https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-24hr&job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-train&job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-train-upload&job_name=periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-train&job_name=periodic-tripleo-ci-centos-7-scenario001-standalone-train&job_name=periodic-tripleo-ci-centos-7-scenario002-standalone-train&job_name=periodic-t | 18:12 |
| rfolco|rover | ripleo-ci-centos-7-scenario003-standalone-train&job_name=periodic-tripleo-ci-centos-7-scenario004-standalone-train&job_name=periodic-tripleo-ci-centos-7-scenario010-standalone-train&job_name=periodic-tripleo-ci-centos-7-standalone-full-tempest-api-train&job_name=periodic-tripleo-ci-centos-7-standalone-full-tempest-scenario-train&job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-train&job_name=periodic-tripleo-ci-centos-7-undercloud-con | 18:12 |
| rfolco|rover | tainers-train&job_name=periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-train&job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-train&job_name=periodic-tripleo-ci-centos-7-singlenode-featureset050-upgrades-train&job_name=periodic-tripleo-ci-centos-7-standalone-train&job_name=periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-train | 18:12 |
| rfolco|rover | sorry, I have a tiny one for this https://tinyurl.com/ybcv8wp9 | 18:13 |
| weshay|ruck | that's train | 18:13 |
| weshay|ruck | k.. /me looks at train next | 18:13 |
| rfolco|rover | ;) | 18:14 |
| weshay|ruck | rfolco|rover, is there a boog on train scenario010? | 18:20 |
| rfolco|rover | don't think so, I was hoping chandankumar's patches would finally fix it | 18:20 |
| rfolco|rover | oh wait there is yes | 18:20 |
| rfolco|rover | scen10 train | 18:20 |
| rfolco|rover | sec | 18:20 |
| weshay|ruck | k.. mark in on-going https://hackmd.io/YAqFJrKMThGghTW4P2tabA | 18:21 |
| weshay|ruck | rfolco|rover, that's the only blocker in train | 18:21 |
| rfolco|rover | https://bugs.launchpad.net/tripleo/+bug/1881584 | 18:21 |
| openstack | Launchpad bug 1881584 in tripleo "SC10 train periodic job failing on LoadBalancerScenarioTest consistently" [Critical,Triaged] | 18:21 |
| rfolco|rover | weshay|ruck, fix merged today... | 18:22 |
| rfolco|rover | weshay|ruck, let me respin this | 18:22 |
| rlandy | chandankumar: still around? | 18:23 |
| weshay|ruck | rfolco|rover, thanks.. and fs035 is not triggering in periodic for train | 18:24 |
| rlandy | 0:26:48.313 *********** | 18:24 |
| rlandy | 2020-06-15 14:20:37.966681 | primary | fatal: [undercloud]: FAILED! => { | 18:24 |
| rlandy | 2020-06-15 14:20:37.966728 | primary | "changed": true, | 18:24 |
| rlandy | 2020-06-15 14:20:37.966740 | primary | "cmd": "source /home/zuul/dlrn-venv/bin/activate; pip install -r requirements.txt; python setup.py install;\n" | 18:24 |
| rlandy | anyone seeing ^^? | 18:24 |
| rlandy | rfolco|rover: ^^ that hitting upstream at all? | 18:25 |
| weshay|ruck | wait see it.. https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-train | 18:25 |
| rlandy | 2020-06-15 14:20:37.966800 | primary | ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt' | 18:26 |
| rlandy | 2020-06-15 14:20:37.966810 | primary | python: can't open file 'setup.py': [Errno 2] No such file or directory | 18:26 |
| weshay|ruck | rlandy, did it clone correctly? | 18:26 |
| rlandy | weshay|ruck: could be out of date | 18:26 |
| rlandy | probably need to promote hash | 18:26 |
| * rlandy finds the latest working hash | 18:27 | |
| rlandy | killed BM run | 18:27 |
| weshay|ruck | rlandy, osp? | 18:27 |
| rlandy | yep | 18:27 |
| rfolco|rover | weshay|ruck, if nothing changed, fs035 timed out from time to time in the last 6 weeks ruckering | 18:28 |
| weshay|ruck | it worked | 18:30 |
| weshay|ruck | rfolco|rover, periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-trainS U C C E S S | 18:30 |
| weshay|ruck | rfolco|rover, stein is only missing fs001 btw.. which hit infra | 18:30 |
| rfolco|rover | weshay|ruck, yes last one worked | 18:31 |
| weshay|ruck | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-train%09 | 18:31 |
| weshay|ruck | rfolco|rover, would advice a test project on stein fs001 periodic | 18:32 |
| rfolco|rover | weshay|ruck, ok let me see fs001 stein | 18:33 |
| rfolco|rover | https://review.rdoproject.org/r/28093 fs001 stein | 18:37 |
| chandankumar | rlandy, yes sorry got lost in old songs | 18:38 |
| rlandy | chandankumar: hey - https://sf.hosted.upshift.rdu2.redhat.com/logs/52/201352/14/check/periodic-tripleo-ci-rhel-8-bm_envE-3ctlr_1comp-featureset001-baremetal-rhos-17/d4fc613/job-output.txt | 18:39 |
| rlandy | that error familiar to you for osp? | 18:39 |
| rlandy | it could be because the package are very out of date | 18:39 |
| chandankumar | checking | 18:39 |
| rlandy | trying to get the containers to push so we can promote | 18:40 |
| chandankumar | rlandy, my patch will fix it | 18:41 |
| rlandy | chandankumar: which patch is this one? | 18:41 |
| chandankumar | rlandy, https://review.opendev.org/#/c/727200/ | 18:41 |
| *** jmasud has quit IRC | 18:41 | |
| rlandy | k - will add as depends-on | 18:41 |
| chandankumar | rlandy, in env file i added vars related to build test packagfes | 18:41 |
| rlandy | weshay|ruck: rfolco|rover: ok if we merge https://review.opendev.org/#/c/727200/? | 18:42 |
| rlandy | chandankumar's patch | 18:42 |
| *** jmasud has joined #oooq | 18:42 | |
| chandankumar | it will not break anything | 18:42 |
| chandankumar | all places tested | 18:42 |
| weshay|ruck | rfolco|rover, same tests failing in master and ussuri https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-ussuri/82fe9f7/logs/undercloud/var/log/tempest/stestr_results.html.gz | 18:42 |
| * weshay|ruck double checks that | 18:42 | |
| chandankumar | weshay|ruck, please merge this https://review.opendev.org/#/c/727200/ | 18:43 |
| weshay|ruck | yup.. same | 18:44 |
| weshay|ruck | https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/fafba23/logs/undercloud/var/log/tempest/stestr_results.html.gz | 18:44 |
| weshay|ruck | https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-ussuri/82fe9f7/logs/undercloud/var/log/tempest/stestr_results.html.gz | 18:44 |
| weshay|ruck | skip list | 18:44 |
| chandankumar | weshay|ruck, check errors.txt file on compute nodes for more info | 18:44 |
| rlandy | weshay|ruck: hmmm ... think I closed that BZ too soon: | 18:45 |
| rlandy | 2020-06-15 18:44:05.534694 | primary | Error: error authenticating creds for "docker-registry.upshift.redhat.com": error pinging docker registry docker-registry.upshift.redhat.com: Get https://docker-registry.upshift.redhat.com/v2/: x509: certificate signed by unknown authority | 18:45 |
| rfolco|rover | weshay|ruck, ok, I can add to skip list for ussuri and master | 18:45 |
| chandankumar | rlandy, let me know if see any issue due to build test packages | 18:45 |
| rfolco|rover | weshay|ruck, gimme a few min | 18:45 |
| rlandy | I can log in with the token though | 18:45 |
| weshay|ruck | nova.exception.ResizeError: Resize error: not able to execute ssh command: Unexpected error while running command. | 18:45 |
| chandankumar | drop a message here | 18:46 |
| rlandy | chandankumar: thanks | 18:46 |
| chandankumar | rlandy, do we have working rhos-16 job? | 18:46 |
| weshay|ruck | rfolco|rover, fyi.. for that future bug for skip https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/fafba23/logs/overcloud-novacompute-1/var/log/extra/errors.txt.txt.gz | 18:46 |
| chandankumar | I will make build test packafes working for rhos-16 | 18:46 |
| rlandy | chandankumar: no - was just chatting with weshay|ruck about rhos-16 | 18:46 |
| chandankumar | rlandy, aye | 18:47 |
| rlandy | afaik, we're only going to start working with 16.2 | 18:47 |
| chandankumar | I am going offline concentrating on old songs | 18:47 |
| chandankumar | see ya | 18:47 |
| rlandy | bye thanks | 18:48 |
| rfolco|rover | weshay|ruck, ah I found my own bug https://bugs.launchpad.net/tripleo/+bug/1881642 | 18:49 |
| openstack | Launchpad bug 1881642 in tripleo "ussuri tempest failures on fs020" [Critical,Triaged] | 18:49 |
| rfolco|rover | weshay|ruck, will just update w/ master info | 18:49 |
| weshay|ruck | rfolco|rover, promotion-blocker | 18:55 |
| rfolco|rover | weshay|ruck, chandankumar when we should use skip list from https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/roles/validate-tempest/vars/tempest_skip_master.yml | 18:55 |
| rfolco|rover | and not from tqe | 18:55 |
| rfolco|rover | vs this one | 18:55 |
| rfolco|rover | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/validate-tempest/vars/tempest_skip_ussuri.yml | 18:55 |
| rfolco|rover | weshay|ruck, ok | 18:55 |
| weshay|ruck | I think it's still the old one | 18:55 |
| weshay|ruck | in tqe | 18:55 |
| rfolco|rover | k | 18:55 |
| weshay|ruck | maybe put it in both.. lolz.. they are getting close | 18:55 |
| * weshay|ruck is not sure tbh | 18:56 | |
| rfolco|rover | weshay|ruck, or maybe just in fs blacklist ? | 18:57 |
| weshay|ruck | rfolco|rover, naw | 18:57 |
| rfolco|rover | k | 18:57 |
| *** jmasud has quit IRC | 18:57 | |
| weshay|ruck | rfolco|rover, fyi.. scen010-kvm is working in ussuri :) | 19:04 |
| rfolco|rover | ok nice | 19:04 |
| rfolco|rover | weshay|ruck, https://review.opendev.org/735659 Add TestNetworkAdvancedServerOps to skip list | 19:05 |
| weshay|ruck | rfolco|rover, perfect.. thanks | 19:06 |
| weshay|ruck | rfolco|rover, ussuri should also promote today | 19:07 |
| weshay|ruck | you are working on scen010-train.. which should pass next test... and stein just had fs001.. | 19:07 |
| weshay|ruck | probably can get 4 releases promoted soon :) | 19:07 |
| *** dsneddon has quit IRC | 19:11 | |
| weshay|ruck | rlandy, refresh my memory.. | 19:12 |
| weshay|ruck | rlandy, it's problem to have component jobs that do not have the keywork component in the name right? | 19:13 |
| weshay|ruck | it's just a regular periodic job w/o that keyword | 19:13 |
| rlandy | weshay|ruck: upstream? | 19:15 |
| rfolco|rover | weshay|ruck, well, scen010-train may need a kvm version to pass, its failing 100% of the time | 19:15 |
| weshay|ruck | rlandy, ya | 19:15 |
| weshay|ruck | rfolco|rover, k | 19:15 |
| chandankumar | rfolco|rover, does all my depends on does not merged? | 19:15 |
| rfolco|rover | merged | 19:15 |
| chandankumar | ah ok | 19:16 |
| rfolco|rover | but... | 19:16 |
| rfolco|rover | https://review.rdoproject.org/r/#/c/28091/ | 19:16 |
| rfolco|rover | chandankumar, can you please take a quick look ?^ | 19:16 |
| *** jmasud has joined #oooq | 19:17 | |
| rfolco|rover | chandankumar, [Errno 2] No such file or directory: '/home/zuul/tripleo-heat-installer-templates/ci/environments/octavia-kvm.yaml'> | 19:17 |
| rlandy | weshay|ruck: its' not the keyword, it's the var | 19:17 |
| rlandy | what's the issue? | 19:17 |
| weshay|ruck | k | 19:22 |
| weshay|ruck | chandankumar, https://review.rdoproject.org/r/28095 | 19:22 |
| *** dsneddon has joined #oooq | 19:28 | |
| rlandy | chandankumar: https://code.engineering.redhat.com/gerrit/203263 Update rpm to install cert. | 19:44 |
| rlandy | seems like our cert was old | 19:44 |
| rlandy | containers are pushing now | 19:45 |
| rlandy | weshay|ruck: ^^ going better now | 19:45 |
| rlandy | chandankumar: also your patch helped the BM failure - thank you | 19:45 |
| weshay|ruck | nice | 19:50 |
| weshay|ruck | rfolco|rover, hrm.. ussuri is not kicking off at all on the promoter server | 19:58 |
| rfolco|rover | weshay|ruck, looking | 20:00 |
| *** jfrancoa has quit IRC | 20:16 | |
| rfolco|rover | weshay|ruck, https://ci.centos.org/view/rdo/view/promotion-pipeline/ - where is ussuri here ? | 20:30 |
| weshay|ruck | meh | 20:36 |
| weshay|ruck | rfolco|rover, worried about upstream.. not ci.centos atm | 20:36 |
| *** jmasud has quit IRC | 20:36 | |
| weshay|ruck | rfolco|rover, aye.. let's run the python command | 20:42 |
| weshay|ruck | let's see if we get a trace | 20:42 |
| rfolco|rover | k, there is some duplication error | 20:42 |
| rfolco|rover | that I don't understand | 20:42 |
| weshay|ruck | where | 20:42 |
| weshay|ruck | rfolco|rover, backports.configparser.DuplicateOptionError: W | 20:43 |
| weshay|ruck | ah.. maybe in criteria | 20:43 |
| rfolco|rover | aaaaaah | 20:43 |
| weshay|ruck | rfolco|rover++ | 20:46 |
| rfolco|rover | weshay|ruck, you did everything boss | 20:46 |
| rfolco|rover | I just watched | 20:46 |
| weshay|ruck | I thought I found a hash where fs001 passed | 20:47 |
| rfolco|rover | https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-latest-released&job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-ussuri | 20:49 |
| rfolco|rover | previous one passed | 20:49 |
| rfolco|rover | oh stein you mean | 20:50 |
| rfolco|rover | you switch context too fast | 20:51 |
| *** ccamacho has quit IRC | 20:51 | |
| weshay|ruck | rfolco|rover, so 035 passed | 20:51 |
| weshay|ruck | rfolco|rover, no.. I'm still on ussuri :) | 20:52 |
| weshay|ruck | I suspect we can get fs001 on ussuri to pass too if 035 did | 20:52 |
| weshay|ruck | same job.. just ssl is off | 20:52 |
| weshay|ruck | and 039 passed | 20:52 |
| weshay|ruck | so.. f it | 20:52 |
| rfolco|rover | weshay|ruck, ok timed out, and 2 pass before | 20:52 |
| weshay|ruck | on node import | 20:53 |
| weshay|ruck | oh we were waiting on the scenario010 fix | 20:53 |
| * weshay|ruck kickes this pipeline again | 20:53 | |
| weshay|ruck | let's see what happens | 20:53 |
| weshay|ruck | we're close | 20:53 |
| weshay|ruck | rfolco|rover, k.. killed master pipeline, kicked ussuri | 20:55 |
| weshay|ruck | rfolco|rover, anything else I should kick? | 20:55 |
| weshay|ruck | 24hr? | 20:55 |
| rfolco|rover | weshay|ruck, well, train is missing scen10 + fs020, waiting on testproject | 20:56 |
| rfolco|rover | weshay|ruck, don't need to rekick full pipeline I think | 20:56 |
| weshay|ruck | k.. | 20:56 |
| weshay|ruck | cloud is busy.. queued for 2hr | 20:56 |
| weshay|ruck | 3hr | 20:57 |
| rfolco|rover | weshay|ruck, will watch later | 20:57 |
| weshay|ruck | rfolco|rover, k.. will revisit tomorrow | 20:57 |
| weshay|ruck | rfolco|rover, get out of here | 20:57 |
| rfolco|rover | weshay|ruck, yeah, I agree. Will check testproject results later | 20:58 |
| rfolco|rover | ttyl | 20:58 |
| *** jmasud has joined #oooq | 21:01 | |
| rlandy | weshay|ruck: ugh - that buildtest packages patch is no good - but needed downstream | 21:13 |
| rlandy | all jobs are falling over there | 21:13 |
| *** rfolco|rover has quit IRC | 21:24 | |
| *** rfolco|rover has joined #oooq | 21:24 | |
| *** jmasud has quit IRC | 21:46 | |
| *** rlandy is now known as rlandy|bbl | 22:14 | |
| *** jmasud has joined #oooq | 22:30 | |
| *** ysandeep|away is now known as ysandeep | 22:43 | |
| *** tosky has quit IRC | 23:04 | |
| ysandeep | rlandy|bbl, Hey o/, we need https://review.opendev.org/#/c/727200/ as depends-on now on BM? | 23:14 |
| weshay|ruck | rlandy|bbl, ya.. the dep finding is busted.. | 23:23 |
| weshay|ruck | ysandeep, you shouldn't need it... unless you run the job as check | 23:23 |
| weshay|ruck | which is not the primary use case | 23:24 |
| ysandeep | weshay|ruck, ack o/ | 23:24 |
| weshay|ruck | ysandeep, we will need it working, not urgent | 23:24 |
| ysandeep | weshay|ruck, when we run testproject they run in check pipeline.. so for test project i will need that depends-on correct? | 23:26 |
| weshay|ruck | ysandeep, if it's a tq,tqe or tripleo-ci change though.. the change is not built into a rpm | 23:26 |
| weshay|ruck | only if you ran w/ tht, tripleo-common etc | 23:26 |
| weshay|ruck | or some neutron, glance package | 23:27 |
| weshay|ruck | s/package/gerrit-change/ | 23:27 |
| weshay|ruck | ah.. I think you do have some tht changes... though | 23:27 |
| weshay|ruck | those will have to merge | 23:27 |
| weshay|ruck | ysandeep, feel free to ping me w/ reviews so I can assist getting them merged | 23:28 |
| ysandeep | weshay|ruck, everthing is merged.. just one tripleo-quickstart-extras pending - testing and refining that | 23:28 |
| weshay|ruck | ysandeep, k.. so you do not need build-test-packages for that :) | 23:29 |
| ysandeep | weshay|ruck, thanks! :) | 23:29 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!