| *** Goneri has quit IRC | 01:03 | |
| *** dtantsur|afk has quit IRC | 01:05 | |
| *** pmannidi has quit IRC | 03:07 | |
| *** udesale has joined #oooq | 03:34 | |
| *** saneax has joined #oooq | 03:59 | |
| *** ykarel has joined #oooq | 04:44 | |
| *** marios has joined #oooq | 05:01 | |
| bhagyashris | zbr, i am trying to fix mini integration pipeline | 05:22 |
|---|---|---|
| *** jtomasek has joined #oooq | 05:37 | |
| *** ysandeep|away is now known as ysandeep|pto | 05:53 | |
| *** ykarel has quit IRC | 05:59 | |
| *** ykarel has joined #oooq | 06:00 | |
| *** amoralej|off is now known as amoralej | 06:34 | |
| *** apetrich has joined #oooq | 06:44 | |
| zbr | bhagyashris: thanks | 07:19 |
| *** udesale has quit IRC | 07:20 | |
| *** yolanda has quit IRC | 07:20 | |
| *** yolanda has joined #oooq | 07:20 | |
| *** jpena|off is now known as jpena | 07:25 | |
| *** udesale has joined #oooq | 07:28 | |
| bhagyashris | zbr, will need to use build-containers-ubi-8-push method to build the container so i changed that in the updated patch | 07:37 |
| *** udesale_ has joined #oooq | 07:39 | |
| *** udesale has quit IRC | 07:40 | |
| *** tosky has joined #oooq | 07:41 | |
| *** ykarel_ has joined #oooq | 07:43 | |
| *** ykarel has quit IRC | 07:44 | |
| *** ykarel__ has joined #oooq | 07:44 | |
| *** ykarel_ has quit IRC | 07:47 | |
| *** ykarel__ is now known as ykarel | 07:52 | |
| *** tosky_ has joined #oooq | 08:12 | |
| *** tosky is now known as Guest98160 | 08:13 | |
| *** tosky_ is now known as tosky | 08:13 | |
| *** Guest98160 has quit IRC | 08:15 | |
| *** jbadiapa has quit IRC | 08:32 | |
| *** derekh has joined #oooq | 08:43 | |
| *** ykarel_ has joined #oooq | 09:03 | |
| *** ykarel has quit IRC | 09:03 | |
| *** ykarel_ is now known as ykarel | 09:25 | |
| *** dtantsur has joined #oooq | 09:26 | |
| * marios biab | 09:32 | |
| chandankumar | arxcruz: soniya29|ruck akahat kopecmartin added tempest agenda here: https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view#2020-10-07 | 10:05 |
| kopecmartin | chandankumar: i took sick day today so won't attend, i added links for tempest bumps though | 10:11 |
| sshnaidm | chandankumar, commented https://review.opendev.org/#/c/752419/ | 10:11 |
| chandankumar | kopecmartin: ok cool, will take care of that, anything which is blocked there? | 10:15 |
| chandankumar | kopecmartin: and get well soon man :-) | 10:15 |
| chandankumar | sshnaidm: sure will fix that | 10:15 |
| kopecmartin | chandankumar: nothing blocked, just review the spec patch, .. it feels a bit hacky | 10:19 |
| kopecmartin | but i don't see other way | 10:20 |
| soniya29|ruck | chandankar, ack | 10:27 |
| *** ykarel has quit IRC | 10:50 | |
| *** ykarel has joined #oooq | 10:50 | |
| marios | sshnaidm: i know it merged but can you check comment @ https://review.opendev.org/#/c/749243/2/roles/undercloud-deploy/templates/containers-prepare-parameter.yaml.j2 | 10:56 |
| marios | sshnaidm: i am working in that file now for undercloud upgrade | 10:56 |
| sshnaidm | marios, we set docker_registry_host to provider address in release files: https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/CentOS-8/master.yml#L20 | 11:01 |
| marios | sshnaidm: ack... so it should be equivalent then... but maybe better to be explicit there for less confusion | 11:01 |
| sshnaidm | marios, in this case we could use '{{ job.registry_ip_address }}:5001' | 11:01 |
| sshnaidm | marios, but I'd like to keep it in one place | 11:02 |
| marios | sshnaidm: well we need both one sets the insecure registries (this file i pointed to) the other in release file sets the docker_registry passed to job | 11:02 |
| sshnaidm | marios, maybe you're right | 11:02 |
| marios | sshnaidm: so we unfortunately have to keep it... we have same pattern for standalone | 11:03 |
| marios | sshnaidm: but ok fine for now just checking i didn't miss somethign thanks | 11:03 |
| sshnaidm | chandankumar, I'm afraid we need to add commas in the end of every line in repos: https://review.opendev.org/#/c/752419/23/playbooks/multinode-standalone-pre.yml | 11:04 |
| *** jbadiapa has joined #oooq | 11:20 | |
| *** jpena is now known as jpena|lunch | 11:28 | |
| *** holser has quit IRC | 11:36 | |
| *** amoralej is now known as amoralej|lunch | 11:41 | |
| *** holser has joined #oooq | 11:42 | |
| *** Tengu has quit IRC | 11:44 | |
| *** Tengu has joined #oooq | 11:46 | |
| *** rlandy has joined #oooq | 12:01 | |
| *** rlandy is now known as rlandy|rover | 12:01 | |
| rlandy|rover | weshay: soniya29|ruck: hey ... https://review.opendev.org/755555 just merged - could we rekick stein line after this current run is done? | 12:04 |
| soniya29|ruck | rlandy|rover yes | 12:05 |
| weshay | rlandy|rover, sure | 12:05 |
| weshay | rlandy|rover, see my review re: 16.2 | 12:06 |
| rlandy|rover | weshay: yeah - I would have gone to twice but ok | 12:06 |
| marios | sshnaidm: any recommended image to use in upshift for reproducer? | 12:07 |
| sshnaidm | marios, we have there images for reproducer, they're configured by default in reproducer role | 12:07 |
| marios | sshnaidm: i mean for the vm that will run reproducer | 12:08 |
| sshnaidm | marios, if you use one of our CI tenant | 12:08 |
| marios | sshnaidm: anything centos8 fine? | 12:08 |
| sshnaidm | marios, oh, I see, yeah | 12:08 |
| marios | sshnaidm: thanks | 12:08 |
| rlandy|rover | weshay: let a comment in https://code.engineering.redhat.com/gerrit/#/c/214354/ | 12:08 |
| sshnaidm | marios, then choose podman as container engine | 12:08 |
| rlandy|rover | we usually specify the times we want to run | 12:08 |
| rlandy|rover | I am not familiar with /8 | 12:09 |
| rlandy|rover | guess that's very 8 hours? | 12:09 |
| marios | sshnaidm: thanks ... but where do you mean to set that do we have a config for the reproducer to set the container engine? | 12:09 |
| rlandy|rover | weshay: soniya29|ruck: we need to get network component through - it's failing scenario010 | 12:10 |
| sshnaidm | marios, yes | 12:10 |
| rlandy|rover | tempest timeout | 12:10 |
| marios | sshnaidm: ack | 12:11 |
| *** rfolco has joined #oooq | 12:11 | |
| weshay | rlandy|rover, ya.. after I submitted I had the same thought.. I'll update it | 12:11 |
| rlandy|rover | weshay: just gives us more control if we have a busy hour | 12:12 |
| weshay | rlandy|rover, debugging the promoter.. not seeing any candidates after https://trunk.rdoproject.org/api-centos8-train/api/civotes_agg_detail.html?ref_hash=8bda3ae5b99012fd3fd4a121ff82d065 | 12:12 |
| weshay | for current-tripleo-rdo :( | 12:12 |
| rlandy|rover | weshay: yep - only ever check ui component | 12:12 |
| rlandy|rover | we need it to check the agg hash | 12:13 |
| rlandy|rover | will look at it after I deal with network master | 12:13 |
| soniya29|ruck | rlandy|rover yes | 12:15 |
| weshay | rlandy|rover, updated | 12:18 |
| zbr | pojadhav: what is happening with py2-py3 conversion around telegraf? | 12:20 |
| pojadhav | zbr, moving scripts from py2 to py3 | 12:21 |
| pojadhav | zbr : https://review.rdoproject.org/r/#/c/29235/ | 12:21 |
| zbr | one thing that I did not understand was why folders and renames. | 12:21 |
| zbr | i ported a lot of code from py2 to py3 but never by renaming files. | 12:21 |
| pojadhav | not renaming files.. just moving files from py2 dir to py3 dir.. | 12:22 |
| pojadhav | if both outputs are identical | 12:23 |
| zbr | why do we need a telegraph_py3 folder? | 12:24 |
| weshay | chandankumar, is this a tempest conf bug? https://bugzilla.redhat.com/show_bug.cgi?id=1853067 | 12:24 |
| openstack | bugzilla.redhat.com bug 1853067 in openstack-glance "[OSP-16.1] Glance API Error - 409 Conflict Image status transition from saving to saving is not allowed" [High,New] - Assigned to chkumar | 12:24 |
| weshay | kopecmartin, ^ | 12:24 |
| chandankumar | weshay: checking | 12:24 |
| pojadhav | in future.. telegraf(py2) folder will be deprecated once everything switched to py3 thats why.. | 12:27 |
| *** jpena|lunch is now known as jpena | 12:28 | |
| weshay | chandankumar, rlandy|rover neither of those bugs seem like core tempest | 12:30 |
| chandankumar | weshay: it hitted the tempest api, tempest says what is the issue. | 12:30 |
| chandankumar | the issue is caused due to glance side | 12:30 |
| zbr | pojadhav: weshay: i really do not understand the multiple folder approach around telegraf porting to py3 but if you say is better for me to not look at it, i will ignore it. | 12:30 |
| chandankumar | but i need to check the logs | 12:30 |
| pojadhav | weshay, ^ am I giving correct answers of zbr's question..?? | 12:31 |
| chandankumar | weshay: and attached logs is giving 404 | 12:31 |
| akahat | rfolco, ping.. meeting. | 12:31 |
| akahat | chandankumar, ^^ | 12:31 |
| weshay | zbr, two different containers | 12:32 |
| weshay | chandankumar, aye agree | 12:32 |
| weshay | zbr, ya.. we can catch up today and chat | 12:32 |
| weshay | zbr, about the mini-integration and cockpit containers | 12:33 |
| zbr | weshay: i still do not see any reason for duplicating source code. python code can be written to work with both without too much effort. | 12:33 |
| weshay | zbr, ya.. I agree.. this approach does allow for a transition.. but you are correct | 12:34 |
| weshay | zbr, once we have it all migrated to py3.. the telegraf container can be killed | 12:34 |
| weshay | akahat, got anything? | 12:35 |
| akahat | weshay, yes.. I'll talk to you after meeting. okay? | 12:35 |
| weshay | akahat, we have a 1-1 right now :) | 12:35 |
| akahat | weshay, :) | 12:36 |
| * pojadhav brb | 12:36 | |
| weshay | oh.. content provider | 12:36 |
| weshay | akahat, roger that | 12:36 |
| zbr | who can help me read buildah logs? i am looking at https://logserver.rdoproject.org/17/29917/5/check/periodic-tripleo-ci-build-containers-ubi-8-push-victoria/8ac553b/logs/build.log and become dizzy | 12:36 |
| zbr | is see failed as expected number is different than buid but no real error in the entire file. | 12:37 |
| *** jtomasek has quit IRC | 12:37 | |
| zbr | i wonder if we could get rid of the time column, it makes much harder to read the file | 12:38 |
| rlandy|rover | chandankumar: https://bugzilla.redhat.com/show_bug.cgi?id=1853067 updated - you no longer own it | 12:38 |
| openstack | bugzilla.redhat.com bug 1853067 in openstack-glance "[OSP-16.1] Glance API Error - 409 Conflict Image status transition from saving to saving is not allowed" [High,New] - Assigned to cyril | 12:38 |
| weshay | zbr, try to find the container that failed in that build log.. and then the details are in https://logserver.rdoproject.org/17/29917/5/check/periodic-tripleo-ci-build-containers-ubi-8-push-victoria/8ac553b/logs/buildah-builds/container-builds/734d5d01-1696-41e2-9b6f-16faf39ee499/base/ | 12:38 |
| chandankumar | rlandy|rover: thanks :-) | 12:39 |
| weshay | zbr, temepst https://logserver.rdoproject.org/17/29917/5/check/periodic-tripleo-ci-build-containers-ubi-8-push-victoria/8ac553b/logs/containers-build-errors.log | 12:39 |
| zbr | weshay: i searched for "error" and "fail" in the file and found nothing. my ability to read 1.8MB char by char is very low. | 12:39 |
| *** jtomasek has joined #oooq | 12:39 | |
| weshay | zbr, https://logserver.rdoproject.org/17/29917/5/check/periodic-tripleo-ci-build-containers-ubi-8-push-victoria/8ac553b/logs/buildah-builds/container-builds/734d5d01-1696-41e2-9b6f-16faf39ee499/base/os/tempest/tempest-build.log | 12:40 |
| weshay | zbr, look at the error log | 12:40 |
| chandankumar | weshay: zbr on above bhagyashris is on it, tempest-all package is missing which is blocked on octavia-tempest-plugin which need some package from base repo https://bugs.centos.org/view.php?id=17139 & https://review.rdoproject.org/r/#/c/29940/ | 12:40 |
| *** amoralej|lunch is now known as amoralej | 12:41 | |
| zbr | we thrive at hiding errors. zuul console reported failed task, i had to look at task detail in order to identify that output was redirected to a file named build.log, navigated to it and found it also cryptic, only mentioned that not enough containers were build, without any indication about which one failed or where to look for that. | 12:42 |
| zbr | huh, saved by bhagyashris :D | 12:46 |
| zbr | weshay: let me know if you have few minutes to talk around improving readability of these errors. | 12:47 |
| bhagyashris | zbr, i have updated the reviews that fixed the issues while doing the build push container jobs for victoria here : https://hackmd.io/sasvoKjTSkabGlFOTkK_kw#bugs-and-reviews | 12:49 |
| bhagyashris | weshay, zbr, we need to merged this https://review.opendev.org/#/c/756441/1 and https://review.rdoproject.org/r/29940 | 12:50 |
| *** Goneri has joined #oooq | 12:51 | |
| bhagyashris | zbr, fyi i have also changed the victoria-containers-build-push job with build-containers-ubi-8-push-victoria | 12:52 |
| bhagyashris | https://review.rdoproject.org/r/#/c/29915/5/zuul.d/integration-jobs-victoria.yaml@13 | 12:52 |
| weshay | bhagyashris, does this fix the tempest container build error that zbr is hitting? | 12:52 |
| weshay | https://review.rdoproject.org/r/#/c/29940/ | 12:53 |
| bhagyashris | weshay, zuul is -1 let me check | 12:54 |
| zbr | weshay: i am inclined to propose changing exclude_containers logic to be to fallback to "master" for any branch name missing from the list. see https://review.opendev.org/#/c/756441/2/roles/build-containers/vars/main.yaml | 12:59 |
| zbr | is very easy to implement and seems to be more future proof. | 12:59 |
| zbr | as long you do not define a custom list, it will will make use of master one. | 12:59 |
| weshay | zbr, we'll chat today and go through it.. k | 13:00 |
| weshay | ? | 13:00 |
| zbr | sure | 13:00 |
| * zbr was only looking at ways to avoid future work. | 13:01 | |
| rlandy|rover | weshay: soniya29|ruck: look at this time trend ... interesting ... https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario010-standalone-network-master | 13:02 |
| rlandy|rover | the test runs at approx 1hr30 mins until 2020-09-18 12:05:27 | 13:03 |
| *** rf0lc0 has joined #oooq | 13:03 | |
| rlandy|rover | where is runs a whole hour longer | 13:03 |
| rlandy|rover | 2 hrs 44 mins 1 sec from then | 13:03 |
| rlandy|rover | with the sporadic 3hr8min timeout | 13:03 |
| *** rfolco has quit IRC | 13:03 | |
| *** jmasud has quit IRC | 13:04 | |
| arxcruz | chandankumar: am i the only one in the meeting ? | 13:04 |
| chandankumar | arxcruz: in another meeting, joining soon | 13:05 |
| *** jmasud has joined #oooq | 13:07 | |
| rlandy|rover | weshay: merging https://code.engineering.redhat.com/gerrit/#/c/214354 | 13:08 |
| rlandy|rover | jason seems happy with it | 13:08 |
| sshnaidm | marios, if not with reproducer then you can try to run jobs on testproject in rdo as well, with dependencies | 13:13 |
| sshnaidm | marios, will be faster than upstream I think | 13:13 |
| marios | sshnaidm: ack thanks good idea | 13:13 |
| bhagyashris | chandankumar, https://review.rdoproject.org/r/#/c/29940/ +w it | 13:18 |
| *** rlandy|rover is now known as rlandy|rover|mtg | 13:26 | |
| weshay | zbr, sent a mtg | 13:31 |
| *** holser has quit IRC | 13:36 | |
| *** holser has joined #oooq | 13:36 | |
| *** TrevorV has joined #oooq | 13:45 | |
| weshay | zbr, talking to panda for a hot sec.. few min late | 13:58 |
| *** rlandy|rover|mtg is now known as rlandy|rover | 14:02 | |
| zbr | nopb | 14:04 |
| sshnaidm | rf0lc0, what is the reason behind hhttps://review.opendev.org/#/c/756128/3..4/zuul.d/base.yaml ? | 14:09 |
| sshnaidm | marios, btw, we may change this as well:https://github.com/openstack/tripleo-ci/blob/master/playbooks/tripleo-ci/install-built-repo.yml#L12 | 14:11 |
| sshnaidm | marios, when you run yum/dnf update in the upgrade job we should know which repo to use | 14:12 |
| sshnaidm | marios, or maybe not.. | 14:14 |
| weshay | zbr, I'm on now | 14:17 |
| marios | sshnaidm: yes, install-built-repo was another thing i wanted to discuss at some point - yeah for the standalone-upgrade we have build-test-packages called during the standalone-upgrade playbook | 14:21 |
| marios | sshnaidm: we will either have to set two of them or ideally use the same trick with dynamic name like for registry | 14:21 |
| sshnaidm | marios, do we build package for deploy or target release? | 14:28 |
| marios | sshnaidm: target let me fetch pointer | 14:30 |
| marios | sshnaidm: there https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/5728258f832dfd6d4eceffd6473dfd555d857467/playbooks/multinode-standalone-upgrade.yml#L38 | 14:30 |
| sshnaidm | marios, hmm.. need to remove this if job is consumer and take package from provider repo | 14:32 |
| sshnaidm | marios, shouldn't it be with "build" tag? instead of standalone-upgrade | 14:32 |
| marios | sshnaidm: ack yes it is part of my reviews i mean doing this stuff | 14:32 |
| sshnaidm | marios, not sure if naming in install-built-repo will help, because it adds repo in the job start.. | 14:33 |
| sshnaidm | marios, and we need to add this repo before upgrade afaiu | 14:33 |
| marios | sshnaidm: yes not clear to me either but i was focussed more on the containers first and wanted to discuss the gating repo things after sorting containers | 14:34 |
| marios | sshnaidm: so indeed, we don't need the install-built-repo for the upgrade job... at least not at the start like the others | 14:34 |
| marios | sshnaidm: actually... it does it for both ... i mean first it runs the deployment with https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/5728258f832dfd6d4eceffd6473dfd555d857467/playbooks/multinode-standalone.yml#L22 | 14:35 |
| marios | sshnaidm: and then it runs it before upgrade again with 17:30 < marios> sshnaidm: there | 14:36 |
| marios | https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/5728258f832dfd6d4eceffd6473dfd555d857467/playbooks/multinode-standalone-upgrade.yml#L38 | 14:36 |
| sshnaidm | marios, hmm.. but patch has only one branch. Is it just skipped with wrong branch? | 14:37 |
| marios | sshnaidm: i am guessing so... and we are really only gating the target i.e. the 2nd one 'wins' should be | 14:37 |
| sshnaidm | marios, yeah, seems like that | 14:38 |
| marios | sshnaidm: will dig more into that | 14:38 |
| sshnaidm | cool | 14:38 |
| *** ykarel is now known as ykarel|away | 15:00 | |
| *** sshnaidm is now known as sshnaidm|afk | 15:01 | |
| rlandy|rover | arxcruz: chandankumar: hey .. where would I see where the number of tempest tests was increased ... see the diff of these two logs - same test - one day apart - 09/17 09/18 ... | 15:11 |
| rlandy|rover | https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/4a06c77/logs/undercloud/var/log/tempest/stestr_results.html.gz | 15:11 |
| rlandy|rover | https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/cc5b009/logs/undercloud/var/log/tempest/stestr_results.html.gz | 15:11 |
| rlandy|rover | weshay: ^^ fyi - that's the time diff | 15:11 |
| weshay | rlandy|rover, oh.. so increasing the job timeout worked.. we just have a lot more tests? | 15:12 |
| weshay | rlandy|rover, that's a bit of a dynamic thing | 15:12 |
| rlandy|rover | weshay: idk - it's still running bit the test number dramatically increased | 15:13 |
| rlandy|rover | ie: the number of tests running jumped from 1 per section to 24, 60 40 etc.per section | 15:13 |
| arxcruz | rlandy|rover: maybe new tests were added in octavia | 15:14 |
| rlandy|rover | 145 of them? | 15:14 |
| arxcruz | rlandy|rover: https://github.com/openstack/octavia-tempest-plugin/commit/89bdbcd1252dbce8645c24fa6db3e2d32fa779c3 | 15:16 |
| arxcruz | last month, a lot of testes were added, and otheres that was being skipped by default, is now being executed | 15:16 |
| arxcruz | let's check the octavia rpm version | 15:16 |
| rlandy|rover | arxcruz: k - if that is expected, we can increase the test timeout | 15:17 |
| arxcruz | rlandy|rover: python3-octavia-tests-tempest-1.4.1-0.20200826090107.c611b45.el8.noarch and python3-octavia-tests-tempest-1.4.1-0.20200917112332.a80a185.el8.noarch | 15:17 |
| arxcruz | they are different | 15:17 |
| arxcruz | probably the second one got the new tests | 15:17 |
| rlandy|rover | arxcruz: k - increasing test timeout - thanks | 15:18 |
| *** derekh has quit IRC | 15:23 | |
| *** derekh has joined #oooq | 15:23 | |
| weshay | rlandy|rover++ | 15:31 |
| *** ykarel|away has quit IRC | 15:32 | |
| *** TrevorV has quit IRC | 15:33 | |
| *** TrevorV has joined #oooq | 15:33 | |
| weshay | rlandy|rover, think I fixed the promoter.. | 15:39 |
| weshay | https://trunk.rdoproject.org/api-centos8-train/api/civotes_agg_detail.html?ref_hash=2a0ebfadbca9de25dd857f351246c19b | 15:39 |
| weshay | is promoting | 15:39 |
| rlandy|rover | weshay++ | 15:39 |
| rlandy|rover | weshay++ | 15:39 |
| rlandy|rover | what was it?? | 15:39 |
| weshay | promoter Containers promote 'aggregate: 2a0ebfadbca9de25dd857f351246c19b, commit: e02878dcbc8f359dd0bcc9af449b507cb873bb00, distro: 5cbbd0b4457e9ba08ec63a2f091e1bcaa6cd43da, component: network, timestamp: 1601988056' to current-tripleo-rdo: Attempting promotion | 15:40 |
| weshay | rlandy|rover, there was a "break" in a function | 15:40 |
| weshay | should not have been there | 15:40 |
| rlandy|rover | weshay: nice - yeah - now the numbers should match up | 15:40 |
| rlandy|rover | will make my life much happier | 15:40 |
| weshay | rlandy|rover, I'll show you what I added later | 15:40 |
| *** udesale__ has joined #oooq | 15:41 | |
| rlandy|rover | weshay: interesting how it promoted before | 15:41 |
| rlandy|rover | was it just that ui changed? | 15:41 |
| *** udesale_ has quit IRC | 15:44 | |
| weshay | rlandy|rover, tbh.. ya.. something fluky like that | 15:44 |
| rlandy|rover | lol | 15:44 |
| rlandy|rover | nice catch | 15:45 |
| *** amoralej is now known as amoralej|off | 15:51 | |
| weshay | chandankumar, is there a tempest-all rpm RUN dnf -y install iputils openstack-tempest-all && dnf clean all && rm -rf /var/cache/dnf": error while running runtime: exit status 1\n' | 15:56 |
| weshay | https://logserver.rdoproject.org/17/29917/7/check/periodic-tripleo-ci-build-containers-ubi-8-push-victoria/60eed58/logs/containers-build-errors.log | 15:56 |
| weshay | yup https://review.rdoproject.org/r/#/c/29940/4/openstack-tempest.spec | 15:57 |
| weshay | line 143 | 15:57 |
| weshay | soo.. perhaps that's not in promoted-components | 15:57 |
| weshay | zbr, /me looks | 15:57 |
| weshay | https://trunk.rdoproject.org/centos8-victoria/component/tempest/consistent/openstack-tempest-all-25.0.0-0.20201007134423.c8d7a01.el8.noarch.rpm | 15:58 |
| weshay | yup.. not here | 15:58 |
| weshay | https://trunk.rdoproject.org/centos8-victoria/component/tempest/promoted-components/ | 15:58 |
| weshay | zbr, background fyi ^ | 15:58 |
| * weshay repromotes consistent to promoted-components | 15:58 | |
| weshay | and will recheck | 15:58 |
| chandankumar | weshay: we need to promote tempest component from consistent | 15:59 |
| chandankumar | to component-ci-testing | 15:59 |
| chandankumar | then to promoted component | 15:59 |
| weshay | chandankumar, doing it now .. consistent -> promoted-components | 15:59 |
| weshay | that's all we need atm | 15:59 |
| weshay | chandankumar, thanks | 15:59 |
| * zbr wants to ask weshay why consistent had the missing rpm but promoted-components did not. | 15:59 | |
| weshay | zbr, because after dlrn builds rpms.. they are placed into current.. if repo-closure works.. they are placed into consistent | 16:00 |
| weshay | zbr, and we pick up rpms in the component line... at consistent | 16:01 |
| weshay | however in victoria | 16:01 |
| weshay | there is no component line.. | 16:01 |
| weshay | so I'm bypassing it.. and promoting consistent -> promoted-components | 16:01 |
| weshay | where the integration lines start :) | 16:01 |
| weshay | zbr, https://hackmd.io/5uYRmLaOTI2raTbHWsaiSQ#Overview | 16:02 |
| weshay | you've probably seen that before | 16:03 |
| chandankumar | there is no component ci testing now as no component pipeline | 16:03 |
| weshay | right | 16:03 |
| chandankumar | https://trunk.rdoproject.org/centos8-victoria/component/tempest/promoted-components/openstack-tempest-all-25.0.0-0.20201007134423.c8d7a01.el8.noarch.rpm | 16:03 |
| chandankumar | zbr: bhagyashris can you recheck your patches, | 16:03 |
| weshay | zbr, this time.. tempest should build https://review.rdoproject.org/r/#/c/29917/ | 16:03 |
| weshay | chandankumar, I did | 16:03 |
| chandankumar | weshay: ok cool then | 16:03 |
| zbr | so the previous tempest component promotion was done when that rpm was missing from consistent | 16:04 |
| weshay | zbr, ya.. that rpm was not built at all | 16:05 |
| chandankumar | https://trunk.rdoproject.org/centos8-victoria/component/tempest/consistent/openstack-tempest-all-25.0.0-0.20201007134423.c8d7a01.el8.noarch.rpm | 16:05 |
| weshay | chandankumar, although I don't see it here https://trunk.rdoproject.org/centos8-victoria/report.html | 16:06 |
| weshay | or here https://trunk.rdoproject.org/centos8-victoria/status_report.html | 16:06 |
| chandankumar | status report contains tempest-all packages | 16:07 |
| *** Tengu has quit IRC | 16:09 | |
| weshay | really.. | 16:10 |
| weshay | I don't see it | 16:10 |
| weshay | https://trunk.rdoproject.org/centos8-victoria/status_report.html | 16:10 |
| *** tosky has quit IRC | 16:21 | |
| *** dtantsur is now known as dtantsur|afk | 16:24 | |
| chandankumar | weshay: i went to https://trunk.rdoproject.org/centos8-victoria/status_report.html and done ctrl + f <openstack-temepst> and then click on success url https://trunk.rdoproject.org/centos8-victoria/component/tempest/c8/d7/c8d7a01014d4b82dd9960679db5e77dc31f4d87a_7951d240/rpmbuild.log | 16:25 |
| chandankumar | it contains tempest-all subpackage | 16:25 |
| weshay | rlandy|rover, ping w/ a skip list | 16:27 |
| rlandy|rover | ack | 16:27 |
| rlandy|rover | it may pass | 16:27 |
| rlandy|rover | on rerun | 16:27 |
| rlandy|rover | you mean for scenario010?? | 16:28 |
| weshay | chandankumar, ah. thanks | 16:28 |
| weshay | rlandy|rover, no.. for periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2failure (non-voting) | 16:28 |
| rlandy|rover | rerun in progress | 16:29 |
| rlandy|rover | will add skiplist of no pass on rerun | 16:29 |
| chandankumar | weshay: there is alos an url to check status for a particular project need to dig that | 16:30 |
| weshay | rlandy|rover, this is what I found missing https://code.engineering.redhat.com/gerrit/#/c/214449/1/zuul.d/integration-pipeline-rhos-16.2.yaml | 16:45 |
| rlandy|rover | looking | 16:46 |
| rlandy|rover | weshay: k - those jobs need to be defined first | 16:47 |
| rlandy|rover | will do that | 16:47 |
| *** udesale__ has quit IRC | 16:47 | |
| rlandy|rover | that is why zuul is objecting | 16:47 |
| weshay | ya.. I figured.. but might as well let it tell us that | 16:47 |
| rlandy|rover | weshay: yeah - it's right - I'll define those | 16:48 |
| weshay | arxcruz, tqe/playbooks/tempest.yml | 16:51 |
| weshay | for the netstat thing? | 16:51 |
| weshay | before line 196? | 16:51 |
| arxcruz | weshay: yup | 16:51 |
| weshay | arxcruz, thanks | 16:52 |
| rlandy|rover | lunch - brb | 16:57 |
| *** marios is now known as marios|out | 16:58 | |
| weshay | zbr, ur in bizness https://registry.rdoproject.org:8443/oapi/v1/namespaces/tripleovictoria/imagestreamtags/ | 17:03 |
| *** sshnaidm|afk has quit IRC | 17:09 | |
| weshay | s u c c e s s https://review.rdoproject.org/r/#/c/29917/ | 17:09 |
| *** sshnaidm has joined #oooq | 17:10 | |
| *** ykarel has joined #oooq | 17:10 | |
| *** derekh has quit IRC | 17:10 | |
| *** ykarel has quit IRC | 17:18 | |
| weshay | rf0lc0, can you try to move the tripleo-quickstart and tqe repos to content provider | 17:20 |
| weshay | maybe there are patches already .. haven't looked | 17:20 |
| *** Tengu has joined #oooq | 17:20 | |
| weshay | but should be another good way to road test | 17:21 |
| rf0lc0 | weshay, k will check and do it | 17:21 |
| rf0lc0 | weshay, my depends-on had a typo in the testproject patch... now it ran the install-built-repo.yml https://review.rdoproject.org/zuul/stream/b4662e31b5e2443085a61f1049bcbc63?logfile=console.log | 17:22 |
| rf0lc0 | weshay, waiting to check the gate repo file | 17:22 |
| *** Tengu_ has joined #oooq | 17:27 | |
| weshay | thanks | 17:27 |
| *** Tengu has quit IRC | 17:27 | |
| *** Tengu_ has quit IRC | 17:29 | |
| rlandy|rover | weshay: periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-16.2testprojectmastercheck189440,931 hr 45 mins 53 secs2020-10-07 15:17:36SUCCESS | 17:31 |
| rlandy|rover | ^^ passed on rerun | 17:31 |
| weshay | ok.. | 17:31 |
| weshay | so just waiting on fs001 | 17:31 |
| *** ysandeep|pto is now known as ysandeep | 17:37 | |
| rlandy|rover | periodic-tripleo-ci-centos-8-scenario010-standalone-network-mastertestprojectmastercheck16882,163 hrs 32 mins 6 secs2020-10-07 14:09:03SUCCESS | 17:45 |
| rlandy|rover | woohoo - so we just need more time ^^ | 17:45 |
| rlandy|rover | https://review.rdoproject.org/r/29951 Increase timeout for scenario010 test | 17:45 |
| rlandy|rover | https://review.opendev.org/756543 Increase timeout for scenario010 test | 17:45 |
| *** jpena is now known as jpena|off | 17:46 | |
| weshay | rlandy|rover, is that the right review? | 17:47 |
| rlandy|rover | https://review.opendev.org/#/c/756543/1/zuul.d/standalone-jobs.yaml | 17:47 |
| rlandy|rover | that one won't work | 17:47 |
| rlandy|rover | let's merge the rdo one though | 17:47 |
| weshay | k | 17:47 |
| rlandy|rover | https://review.rdoproject.org/r/29951 | 17:47 |
| rlandy|rover | idk about upstream | 17:47 |
| rlandy|rover | we may hit a problem | 17:47 |
| rlandy|rover | maybe the tempest tests are diff on check | 17:47 |
| weshay | thanks | 17:47 |
| *** Tengu has joined #oooq | 17:48 | |
| rlandy|rover | weshay: https://review.opendev.org/#/c/756543 w-1 that one | 17:48 |
| rf0lc0 | weshay, can you teach me how to check gate repo on https://logserver.rdoproject.org/94/29894/6/check/tripleo-ci-centos-8-undercloud-containers/b4662e3/logs/undercloud/home/zuul/ | 17:48 |
| rlandy|rover | we may need to reconsider that if it shows up after promotion | 17:48 |
| weshay | rf0lc0, sure | 17:49 |
| rf0lc0 | weshay, I see install-built-repo.yml ran fine | 17:49 |
| weshay | rf0lc0, https://logserver.rdoproject.org/94/29894/6/check/tripleo-ci-centos-8-undercloud-containers/b4662e3/logs/undercloud/etc/yum.repos.d/gating.repo.txt.gz | 17:49 |
| rf0lc0 | \o/ | 17:49 |
| rf0lc0 | weshay, thanks :) | 17:49 |
| weshay | looks good to me.. double check the provider built the rpm | 17:50 |
| weshay | rf0lc0, was it tht? | 17:50 |
| rf0lc0 | weshay, yep checking rpm from tht patch | 17:50 |
| weshay | rf0lc0, openstack-tripleo-heat-templates noarch 12.7.1-0.20201007161303.ffd106a.el8 gating-repo 546 k | 17:50 |
| weshay | rf0lc0, worked https://logserver.rdoproject.org/94/29894/6/check/tripleo-ci-centos-8-undercloud-containers/b4662e3/logs/undercloud/var/log/dnf.log.txt.gz | 17:51 |
| rf0lc0 | weshay, nice, now the only remaining thing for undercloud is https://review.opendev.org/#/c/756128/4/zuul.d/base.yaml | 17:52 |
| rf0lc0 | weshay, which I think its the right place to do it | 17:52 |
| weshay | rf0lc0, ooh.. not sure if we cant that one the base | 17:53 |
| weshay | maybe | 17:53 |
| weshay | sorry.. not sure if we want that on the base job def | 17:53 |
| rf0lc0 | weshay, an intermediate layer maybe ? a new parent between just for undercloud jobs? | 17:54 |
| weshay | will let sagi make that call | 17:54 |
| weshay | rf0lc0, where do you see that plugged in elsewhere? | 17:54 |
| weshay | e.g. where in the standalone zuul? | 17:54 |
| rf0lc0 | weshay, okay, I'll propose the new layer... yes its plugged to undercloud-upgrade jobs as well | 17:54 |
| weshay | rf0lc0, wait | 17:54 |
| weshay | rf0lc0, before you do that.. | 17:55 |
| weshay | show me where it is for the standalone | 17:55 |
| rf0lc0 | weshay, standalone has its own parent, multinode too | 17:55 |
| weshay | rf0lc0, show me | 17:55 |
| rf0lc0 | searching | 17:55 |
| weshay | rlandy|rover, tempest timed out | 17:55 |
| weshay | fs001 | 17:55 |
| rf0lc0 | weshay, multinode is here https://github.com/openstack/tripleo-ci/blob/master/zuul.d/base.yaml#L184 | 17:56 |
| rf0lc0 | standalone https://github.com/openstack/tripleo-ci/blob/master/zuul.d/base.yaml#L376 | 17:56 |
| rf0lc0 | they were added to the direct parent | 17:56 |
| weshay | ya.. ok | 17:57 |
| *** Tengu has quit IRC | 17:57 | |
| weshay | so you just followed that pattern .. | 17:57 |
| rlandy|rover | weshay: I think we can still promote | 17:57 |
| *** Tengu has joined #oooq | 17:57 | |
| rlandy|rover | I will rerun | 17:57 |
| rlandy|rover | it's timed out before | 17:58 |
| weshay | seems fine to me, rf0lc0 but does the undercloud use the multindoe-centos-8 base | 17:58 |
| weshay | rf0lc0, you may not have to do anything | 17:58 |
| * weshay is not sure.. | 17:58 | |
| rlandy|rover | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-16.2 | 17:58 |
| weshay | but worth checking | 17:58 |
| rf0lc0 | no, undercloud parent is tripleo-ci-base-singlenode-centos-8 | 17:58 |
| weshay | rlandy|rover, can we increase the timeout on fs001? | 17:58 |
| rlandy|rover | weshay:ack - will do that on rerun | 17:58 |
| weshay | rlandy|rover, thanks | 17:58 |
| weshay | rf0lc0, so perhaps the singlenode-centos-8 base is the right place | 17:59 |
| weshay | ya.. | 17:59 |
| weshay | k | 17:59 |
| rf0lc0 | weshay, the only concern is that parent is shared between many jobs, buildimage, many others like upgrade undercloud etc | 17:59 |
| rlandy|rover | periodic-tripleo-centos-7-rocky-containers-build-pushtimed_out (non-voting) | 17:59 |
| rlandy|rover | weshay: ^^ any work do whether we can drop rocky? | 18:00 |
| rlandy|rover | word | 18:00 |
| rlandy|rover | on whether | 18:00 |
| rlandy|rover | queens does fine | 18:00 |
| weshay | rf0lc0, ya.. centos-8 jobs should all be parent/child | 18:00 |
| rlandy|rover | rocky not so muhc | 18:00 |
| rlandy|rover | much | 18:00 |
| weshay | I would NOT say this is safe on centos-7 anything | 18:00 |
| weshay | rlandy|rover, ya.. ignore for now | 18:00 |
| rlandy|rover | weshay: great - thanks | 18:01 |
| rf0lc0 | weshay, the intermediate layer would be a safer move IMHO, we only reparent off new layer the ones we want, undercloud containers in this case will be the first one... the rest will keep the original parent | 18:02 |
| rf0lc0 | weshay, open to hear pros and cons | 18:02 |
| rf0lc0 | I see only pros | 18:02 |
| akahat | rf0lc0, i went through your reviews.. i found c8 jobs there.. do you have any idea about c7 content provider jobs? | 18:07 |
| rf0lc0 | akahat, c7 chandankumar was taking care of | 18:10 |
| rf0lc0 | akahat, as soon as he confirms c7 jobs are moved to content provider, we can replicate in tripleo repos | 18:11 |
| rf0lc0 | akahat, lets do it for c8 master first. Once we confirm all templates and jobs are good, we just replicate | 18:11 |
| akahat | rf0lc0, okay. that sounds good. | 18:12 |
| rf0lc0 | for example.... undercloud is wip, depends on that patch... there is one ipa template that is not enabled yet.... exceptions will be handled as we find them | 18:13 |
| ysandeep | weshay, I am updating description of 16.2 pipeline to reflect now we run thrice- https://code.engineering.redhat.com/gerrit/#/c/214354/2/zuul.d/pipelines.yaml@142 | 18:13 |
| *** marios|out has quit IRC | 18:13 | |
| ysandeep | weshay, Really good idea considering 16.2 will be delivered to our customers and once 16.2 is GA - Incase of bugs we want to deliver fixes as soon as possible. | 18:13 |
| ysandeep | weshay, ut I think this change will not help alone and second bit we need is auto promotions to current-tripleo in downstream. | 18:13 |
| ysandeep | But* | 18:14 |
| ysandeep | Right now we are promoting tripleo-ci-testing to current-tripleo manually using script. So till we auto promote to me it make less sense to trigger the pipeline thrice. IMHO manually we can only consider promoting once a day. But yes running thrice increase our chances to promote on daily basis so +1 to your change from me as well | 18:14 |
| rlandy|rover | ysandeep: just promoted 16.2 | 18:17 |
| rlandy|rover | only advantage of running more is catching component promotions | 18:17 |
| ysandeep | rlandy|rover, thanks! i was just checking today's result - one job failed on tempest was checking that | 18:17 |
| rlandy|rover | ysandeep: I reran tempest job | 18:18 |
| rlandy|rover | fs001 timedout on tempest | 18:18 |
| rlandy|rover | happens often | 18:18 |
| rlandy|rover | looking to increase timeout there | 18:18 |
| ysandeep | rlandy++ | 18:19 |
| *** yolanda has quit IRC | 18:21 | |
| rlandy|rover | timestamp: 1601988056': SUCCESSFUL promotion to current-tripleo-rdo | 18:23 |
| rlandy|rover | 2020-10-07 18:11:30,110 28357 INFO promoter Summary: Promoted 1 hashes this round | 18:23 |
| rlandy|rover | woohoo | 18:23 |
| rlandy|rover | 2 hours agonetwork13 hours ago - network promoted | 18:24 |
| ysandeep | rlandy|rover, weshay minor changes in pipeline description - https://code.engineering.redhat.com/gerrit/#/c/214464/ | 18:31 |
| weshay | rlandy|rover, did we promote 16.2 w/ a failing fs001? | 18:32 |
| weshay | http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/ | 18:34 |
| weshay | ? | 18:35 |
| *** ysandeep is now known as ysandeep|away | 18:38 | |
| rlandy|rover | weshay: yes - it fails a lot | 18:44 |
| rlandy|rover | well at least times out | 18:44 |
| weshay | rlandy|rover, it's not criteria? | 18:44 |
| rlandy|rover | weshay: criteria on downstream?? | 18:45 |
| rlandy|rover | it's whatever we chose it to be at the moment | 18:45 |
| weshay | let's chat | 18:45 |
| rlandy|rover | the promoter script does not ask | 18:45 |
| weshay | https://meet.google.com/arm-zkto-vnu?authuser=1 | 18:46 |
| rf0lc0 | sshnaidm, I'll step on your toe a little | 18:59 |
| sshnaidm | rf0lc0, is it painful? | 18:59 |
| rf0lc0 | sshnaidm, content provider templates and jobs on tq and tqe | 18:59 |
| rf0lc0 | sshnaidm, you tell me | 19:00 |
| rf0lc0 | if it hurts you yell | 19:00 |
| rlandy|rover | weshay: https://code.engineering.redhat.com/gerrit/#/c/214472/ | 19:01 |
| rlandy|rover | weshay: https://code.engineering.redhat.com/gerrit/214477 | 19:17 |
| *** tosky has joined #oooq | 19:26 | |
| weshay | rlandy|rover, just merged https://code.engineering.redhat.com/gerrit/#/c/214199/ | 19:44 |
| weshay | those are running just 1 tempest test https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-train/fd93077/logs/undercloud/var/log/tempest/stestr_results.html.gz | 19:45 |
| rlandy|rover | ok | 19:46 |
| rlandy|rover | oh - and the ovb change as well - thanks | 19:47 |
| rlandy|rover | weshay: I'll take https://code.engineering.redhat.com/gerrit/#/c/214449/ - need to make that depends -on the definitions | 19:48 |
| * rlandy|rover goes to bank ... brb | 19:50 | |
| *** rlandy|rover is now known as rlandy|rover|brb | 19:50 | |
| *** apetrich has quit IRC | 20:06 | |
| *** jbadiapa has quit IRC | 20:33 | |
| *** rlandy|rover|brb is now known as rlandy|rover | 20:40 | |
| *** TrevorV has quit IRC | 21:00 | |
| rlandy|rover | weshay: soniya29|ruck: fyi ... https://bugs.launchpad.net/tripleo/+bug/1898931 - not sure is this is consistent yet | 21:08 |
| openstack | Launchpad bug 1898931 in tripleo "periodic-tripleo-ci-centos-8-scenario003-standalone-master is failing overcloud deploy "Service mistral_executor has not started yet"" [Undecided,Triaged] | 21:08 |
| weshay | rlandy|rover, I think mistral is gone... | 21:08 |
| weshay | rlandy|rover, we may need to look for that patch and make sure it's removed from scenario003 env | 21:09 |
| rlandy|rover | weshay: patch? | 21:09 |
| rlandy|rover | like in THT? | 21:09 |
| rlandy|rover | weshay: https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario003-standalone.yaml#L9 | 21:10 |
| weshay | rlandy|rover, I think kevin was tearing it out.. I saw something last week about it | 21:12 |
| rlandy|rover | weshay: k - will ask him - thanks | 21:12 |
| weshay | rlandy|rover, apparently setting last hashes to anything to over 100 causes... and indexOutOfRange error on the promoter | 21:31 |
| rlandy|rover | nice | 21:31 |
| rlandy|rover | 100 should be good enough | 21:31 |
| weshay | rlandy|rover, ussuri is promoting.. not quite out of the woods w/ minor tweaks.. but getting there | 21:31 |
| weshay | ya | 21:31 |
| rlandy|rover | oif it runs regularly | 21:31 |
| rlandy|rover | cool | 21:31 |
| weshay | it's current-tripleo.. will rerun to get rdo | 21:32 |
| weshay | ok.. ussuri rdo promoting | 21:38 |
| weshay | still funkyness | 21:38 |
| rlandy|rover | https://code.engineering.redhat.com/gerrit/214492 WIP: Add missing OVB jobs: | 21:55 |
| rlandy|rover | weshay: ^^ | 21:55 |
| *** Goneri has quit IRC | 22:01 | |
| *** dsneddon has quit IRC | 22:06 | |
| *** dsneddon has joined #oooq | 22:08 | |
| *** tosky has quit IRC | 22:22 | |
| *** holser has quit IRC | 22:28 | |
| *** Goneri has joined #oooq | 22:34 | |
| *** rlandy|rover has quit IRC | 23:02 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!