| ssbarnea | weshay need workflow on https://review.openstack.org/#/c/613797/ | 00:05 |
|---|---|---|
| ssbarnea | this being another one ready if there is anyone around https://review.openstack.org/#/c/610491/ | 00:09 |
| *** ssbarnea has quit IRC | 00:29 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-multinode-1ctlr-featureset010 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024 @ (3 more messages) | 00:41 |
| *** rlandy|bbl is now known as rlandy | 01:59 | |
| *** huynq has joined #oooq | 02:08 | |
| *** apetrich has quit IRC | 02:16 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (3 more messages) | 02:41 |
| *** udesale has joined #oooq | 03:50 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (3 more messages) | 04:41 |
| *** rlandy has quit IRC | 05:30 | |
| *** ratailor has joined #oooq | 05:35 | |
| *** saneax has joined #oooq | 05:50 | |
| *** chkumar|off is now known as chandankumar | 06:21 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (3 more messages) | 06:41 |
| *** gkadam has joined #oooq | 06:58 | |
| *** apetrich has joined #oooq | 07:03 | |
| *** udesale has quit IRC | 08:02 | |
| *** ykarel has joined #oooq | 08:16 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (3 more messages) | 08:41 |
| *** ssbarnea has joined #oooq | 08:54 | |
| ssbarnea | finally few CR merged, first decent morning of this week | 08:56 |
| ssbarnea | quiquell|off - let me know when you are back, | 08:57 |
| *** udesale has joined #oooq | 09:16 | |
| *** ykarel is now known as ykarel|lunch | 09:38 | |
| ssbarnea | marios : i am reading the code around pip installation on overcloud-prep-containers.yml and tbh it does not make much sense to me, neither original one or proposed one. do you have some time to chat? | 09:41 |
| ssbarnea | i would prefer to avoid using easy_install | 09:42 |
| marios | ssbarnea: o/ guess you are referring to https://review.openstack.org/#/c/610491/26/roles/overcloud-prep-containers/tasks/overcloud-prep-containers.yml@21 | 09:45 |
| ssbarnea | marios exactly, I am working not to rewrite the entire logic. on py3 we should not do anything. | 09:46 |
| ssbarnea | no more messing with system libraries | 09:46 |
| marios | ssbarnea: ok so it looks like you didn't change it on v 27 so perhaps the syntax is ok like that? name: foo then a list | 09:46 |
| ssbarnea | marios that was a rebase bug, anyway. in less than an hour I will have the new version | 09:47 |
| marios | ssbarnea: ack added comment there | 09:48 |
| ssbarnea | marios do you know any reason why we may want to upgrade pip on python3 systems? -- hint: it comes with the system. | 09:49 |
| marios | ssbarnea: i don't follow, i mean i see pip is installed there but that was pre-existing this just makes it f28 compatible no | 09:51 |
| marios | ssbarnea: are you saying if it is python 3 we shouldn't perhaps | 09:53 |
| marios | ssbarnea: but i wonder if that can be followup, since this review is pretty hard to check already (see that nit i pointed at for xample) | 09:54 |
| ssbarnea | marios: yes. which is what I am doing now. | 09:54 |
| marios | ssbarnea: well i guess its "python3" and this review is about py3 and dnf/yum so... | 09:54 |
| marios | k | 09:54 |
| ssbarnea | marios: i know, is grew too much. but I have an idea. i will create two spin-offs which would be merged before it, making its size smaller. | 09:55 |
| marios | ssbarnea: maybe the split should be "dnf/yum" and "python3" | 09:55 |
| panda|off | yum3 | 09:56 |
| *** panda|off is now known as panda | 09:56 | |
| ssbarnea | yeah, this was part of my plan, to put the pkg_mgr in other review. | 10:04 |
| panda | lol it's true, yum is at version 3 and dnf is at version 2 | 10:05 |
| panda | wwwwww | 10:05 |
| panda | pythonN, dnf N-1 | 10:06 |
| panda | pythonN-1, yum N | 10:06 |
| panda | weeee | 10:09 |
| *** chandankumar has quit IRC | 10:09 | |
| panda | ssbarnea: today is holiday almost everywhere | 10:10 |
| marios | panda: why? what's on? | 10:11 |
| panda | marios: where ? | 10:11 |
| marios | 12:10 < panda> ssbarnea: today is holiday almost everywhere | 10:11 |
| panda | like karate kid | 10:11 |
| panda | what's on, what's off | 10:12 |
| panda | marios: it's all saints | 10:12 |
| *** apetrich has quit IRC | 10:12 | |
| marios | panda: ah thanks | 10:12 |
| panda | marios: almost all europe and other places | 10:12 |
| ssbarnea | not in uk, didn't know | 10:14 |
| panda | uk is not europe | 10:14 |
| marios | brexited | 10:15 |
| ssbarnea | not yet, you need to live to see it happening, like any other major changes... | 10:16 |
| ssbarnea | ...not rally in denial, but still. | 10:16 |
| panda | and we've not been bought by IBM. | 10:17 |
| marios | 12:16 < ssbarnea> not yet, you need to live to see it happening, like any other major changes... | 10:17 |
| ssbarnea | marios panda : are you also on telegram? | 10:20 |
| panda | ssbarnea: yes, ...- --.-- .-. -.-.. -... -- . -- . | 10:21 |
| *** jaosorior has quit IRC | 10:23 | |
| ssbarnea | asking because I had a sudden need for some extra privacy and Telegram seemed to be quite popular among RedHatter at least while I was in Tel Aviv, Brno, Munich. -- obviously for non-work stuff, like misc(cellaneous) kind of it. | 10:24 |
| *** apetrich has joined #oooq | 10:25 | |
| ssbarnea | i want some preliminary feedback on this file: https://review.openstack.org/#/c/610491/28/roles/overcloud-prep-containers/tasks/overcloud-prep-containers.yml -- no need redo entire CR. | 10:27 |
| *** chandankumar has joined #oooq | 10:30 | |
| ssbarnea | an ansible question: is possible to import a role without accessing the default tasks/main.yml? i guess not, and this should be fine, just wanted to know if I can assume main.yml is always run. | 10:33 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ (3 more messages) | 10:41 |
| sshnaidm|ruck | ssbarnea, possible, import_tasks_from or kind of | 10:42 |
| ssbarnea | i hope we are not using that feature, it would be a nightmare. | 10:43 |
| *** jtomasek has quit IRC | 10:43 | |
| sshnaidm|ruck | ssbarnea, why? | 10:43 |
| *** jtomasek has joined #oooq | 10:45 | |
| sshnaidm|ruck | ssbarnea, panda need to fix everything that left unfixed in https://review.openstack.org/#/c/613797/ | 10:46 |
| ssbarnea | sshnaidm|ruck: i will, after dealing with the extra one. | 10:48 |
| sshnaidm|ruck | ssbarnea, please add me to reviewers there | 10:49 |
| ssbarnea | sshnaidm|ruck you were part of it from day one, see https://review.openstack.org/#/c/610491/ | 11:00 |
| ssbarnea | sshnaidm|ruck it has being suggested to split it by role to make testing easier | 11:01 |
| ssbarnea | and not only testing, but also merging. | 11:01 |
| panda | ssbarnea: it this only for libvirt ? https://tree.taiga.io/project/tripleo-ci-board/task/68?kanban-status=1447274 | 11:08 |
| panda | ssbarnea: can we sync up when you have time ? | 11:09 |
| ssbarnea | panda sure but give me half an hour, i even skipped my morning coffee. | 11:13 |
| chandankumar | arxcruz: I am working on creating the reproducer env https://trello.com/c/Z8lbGX5E/802-cixlp1800742tripleociproa-tempestlibexceptionsidentityerror-got-identity-error-undercloud-containers let see if the error reproduced I need your help to debug | 11:15 |
| arxcruz | chandankumar: ok, but i really think this is not a real bug, probably the keystone hang out because some tests pass, then you get it, i'll update trello with this info | 11:16 |
| panda | chandankumar: arxcruz aren't you both on holiday today ? | 11:16 |
| chandankumar | arxcruz: holiday? | 11:17 |
| chandankumar | sorry | 11:17 |
| arxcruz | panda: holiday? | 11:17 |
| chandankumar | panda: holiday | 11:17 |
| chandankumar | ? | 11:17 |
| arxcruz | let me check | 11:17 |
| panda | ... | 11:17 |
| chandankumar | panda: arxcruz I will be available next week mon and tue | 11:17 |
| chandankumar | then wed thurs public holiday | 11:17 |
| arxcruz | panda: hmmm all saint's day | 11:18 |
| chandankumar | friday might take leave for extended diwali celebration | 11:18 |
| arxcruz | panda: but my wife had german classes today, i guess is not holiday in berlin | 11:18 |
| arxcruz | apetrich: ^ | 11:18 |
| chandankumar | arxcruz: sure | 11:19 |
| panda | arxcrux: BW, BY, NW, RP, SL | 11:19 |
| panda | chandankumar: I read now, it's only bangalore | 11:20 |
| chandankumar | panda: yes | 11:20 |
| * chandankumar keep floating from pune to Bangalore and vice versa | 11:21 | |
| sshnaidm|ruck | is US on holiday too? | 11:21 |
| panda | no | 11:22 |
| sshnaidm|ruck | and Brazil? | 11:22 |
| panda | no | 11:22 |
| sshnaidm|ruck | ssbarnea, I talked about quickstart.sh changes specifically | 11:23 |
| panda | In Brazil is Holiday tomorrow | 11:23 |
| *** udesale has quit IRC | 11:24 | |
| *** zul has joined #oooq | 11:25 | |
| ssbarnea | panda https://bluejeans.com/2655417928 ? | 11:31 |
| panda | ssbarnea: joining | 11:33 |
| *** jaosorior has joined #oooq | 11:35 | |
| *** ratailor has quit IRC | 11:36 | |
| chandankumar | arxcruz: tempest runned successfully | 11:37 |
| chandankumar | arxcruz: not able to reproduce the error | 11:37 |
| chandankumar | arxcruz: it might be a network hiccup or infra issue? | 11:38 |
| chandankumar | arxcruz: feel free to update the trello | 11:38 |
| arxcruz | chandankumar: already did | 11:38 |
| arxcruz | it's keystone issue, and you open a bug with the similar error message from keystone | 11:38 |
| chandankumar | arxcruz: thanks :-) | 11:38 |
| arxcruz | but i follow german calendar :) | 11:39 |
| arxcruz | so, only rfolco|rover will be on holiday tomorrow | 11:39 |
| *** gkadam has quit IRC | 11:46 | |
| *** gkadam has joined #oooq | 11:50 | |
| sshnaidm|ruck | ssbarnea, I hope you saw it: http://logs.openstack.org/91/610491/30/check/tripleo-ci-fedora-28-standalone/2ac2d2b/job-output.txt.gz#_2018-11-01_11_11_41_648503 | 11:58 |
| *** gkadam_ has joined #oooq | 12:04 | |
| sshnaidm|ruck | is EU and US moving to winter time soon? | 12:07 |
| *** gkadam has quit IRC | 12:07 | |
| panda | sshnaidm|ruck: EU already has | 12:10 |
| weshay | quiquell|off, you hit a bump http://logs.openstack.org/68/614168/1/gate/tripleo-ci-centos-7-undercloud-containers/a7d8e31/job-output.txt.gz#_2018-11-01_11_55_34_728161 | 12:11 |
| weshay | sshnaidm|ruck, panda nov 4 | 12:11 |
| weshay | sshnaidm|ruck, thanks for adding vexhost to sova | 12:16 |
| weshay | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch is interesting | 12:16 |
| weshay | as of today it appears to be working again | 12:16 |
| weshay | rfolco|rover, ^ | 12:16 |
| weshay | arxcruz, chandankumar where are we at w/ fs001 and tempest | 12:17 |
| weshay | brb | 12:18 |
| arxcruz | weshay: did chandankumar fix that with the undercloud_tempest_run ? | 12:18 |
| *** trown|outtypewww is now known as trown | 12:18 | |
| rfolco|rover | sshnaidm|ruck, let me know when you want to switch | 12:20 |
| weshay | arxcruz, does not appear to be fixed | 12:23 |
| arxcruz | weshay: okay, let me check | 12:24 |
| arxcruz | i have a env up | 12:25 |
| arxcruz | this is the nhicher issue right? sorry, still with one glass of coffee on my system | 12:26 |
| chandankumar | weshay: arxcruz https://tree.taiga.io/project/tripleo-ci-board/issue/306?kanban-status=2027734 | 12:26 |
| chandankumar | last night patches got abandoned | 12:27 |
| weshay | chandankumar, ya.. k.. thank you both! | 12:28 |
| weshay | chandankumar, do you want to 1-1? | 12:29 |
| chandankumar | weshay: yes sir | 12:29 |
| weshay | I do not have anything | 12:29 |
| weshay | sure | 12:29 |
| weshay | k | 12:29 |
| * weshay going to blue | 12:29 | |
| arxcruz | weshay: so, do i need to do something ? i saw the patch abandoned... | 12:30 |
| chandankumar | arxcruz: restored the patch | 12:30 |
| * arxcruz kinda lost today... | 12:30 | |
| weshay | panda, did you decide a subject in the user story is not a good idea? | 12:31 |
| *** udesale has joined #oooq | 12:32 | |
| *** ykarel_ has joined #oooq | 12:32 | |
| weshay | panda, imho this looks good https://tree.taiga.io/project/tripleo-ci-board/search?milestone=203564&text=os_tempest | 12:33 |
| *** ykarel|lunch has quit IRC | 12:35 | |
| weshay | apetrich, I'll probably be running late today for our 1-1 | 12:39 |
| *** rlandy has joined #oooq | 12:41 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000 (2 more messages) | 12:41 |
| panda | weshay: Yes but I tried to play with tags first... they're just not visible in the board | 12:44 |
| nhicher | sshnaidm|ruck: hello, we need to merge https://review.rdoproject.org/r/#/c/16917/ for -master and -rocky on vexxhost | 12:48 |
| nhicher | sshnaidm|ruck: I will update the commit message | 12:48 |
| *** agopi|brb is now known as agopi | 12:49 | |
| *** huynq has quit IRC | 12:49 | |
| nhicher | sshnaidm|ruck: also https://review.openstack.org/#/c/614321/ and https://review.openstack.org/#/c/613640/ | 12:50 |
| *** zul has quit IRC | 12:52 | |
| arxcruz | panda: isn't this something we can open a issue on taiga? | 12:53 |
| *** udesale has quit IRC | 12:56 | |
| nhicher | weshay: ^ | 12:57 |
| *** ykarel_ is now known as ykarel | 12:57 | |
| *** gkadam_ has quit IRC | 12:57 | |
| chandankumar | nhicher: tempest related stuff is here https://tree.taiga.io/project/tripleo-ci-board/issue/306?kanban-status=2027734 | 12:58 |
| sshnaidm|ruck | nhicher, ack | 12:58 |
| rlandy | marios: hey the town hall in raleigh moved - so we can chat more after the meeting | 12:58 |
| rlandy | wanted to know how the playbook thing went | 12:59 |
| panda | is the scrum now ? | 13:01 |
| marios | rfolco|rover: better here | 13:01 |
| ykarel | sshnaidm|ruck, hi | 13:01 |
| *** udesale has joined #oooq | 13:01 | |
| panda | or daylight savings? | 13:01 |
| weshay | rfolco|rover, can you hear us? | 13:01 |
| sshnaidm|ruck | ykarel, hey | 13:01 |
| ykarel | sshnaidm|ruck, u remember kolla directory missing in logs? | 13:01 |
| sshnaidm|ruck | ykarel, yeah, I didn't find it last time | 13:02 |
| ykarel | sshnaidm|ruck, afair it used to be in base log | 13:02 |
| sshnaidm|ruck | ykarel, base log? | 13:02 |
| panda | meeting I'd ? | 13:02 |
| ykarel | sshnaidm|ruck, i mean around https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/0852913/logs/ | 13:02 |
| weshay | panda, chandankumar, probably should join | 13:02 |
| panda | meeting Id .. | 13:03 |
| chandankumar | panda: https://bluejeans.com/5878458097/ | 13:03 |
| sshnaidm|ruck | ykarel, yeah, maybe | 13:03 |
| weshay | panda, waiting on you brotha | 13:04 |
| sshnaidm|ruck | ykarel, seems like we lost it in transition somewhere | 13:04 |
| ykarel | sshnaidm|ruck, i think https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-ci-periodic-base/containers-build.yaml#L21 is the issue | 13:05 |
| ykarel | sshnaidm|ruck, earlier it used to be WORKSPACE/logs/ | 13:05 |
| ykarel | $WORKSPACE/logs/ | 13:05 |
| sshnaidm|ruck | ykarel, you're right, will fix it.. | 13:06 |
| ykarel | sshnaidm|ruck, ack | 13:06 |
| ykarel | sshnaidm|ruck, also is the containers-build issue knows and fix is going somewher? | 13:06 |
| ykarel | i mean https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-centos-7-master-containers-build | 13:07 |
| ykarel | No package mod_xsendfile required to build crane container | 13:07 |
| sshnaidm|ruck | ykarel, I saw this, but didn't submit a bug yet | 13:07 |
| sshnaidm|ruck | ykarel, got distracted on other things | 13:07 |
| ykarel | sshnaidm|ruck, no issue, u can log once u got time | 13:08 |
| ykarel | i just wanted to know that it's known and primary blocker | 13:08 |
| sshnaidm|ruck | ykarel, yeah, it's a blocker | 13:10 |
| ykarel | sshnaidm|ruck, also ther is one more | 13:11 |
| ykarel | sshnaidm|ruck, https://trello.com/c/lLin0AIv/925-poiscenario002-3-cinder-tests-failing, this will be seen in scenario002 once containers-build start passing | 13:12 |
| ykarel | i mean featureset018 and wherever cinder is tested | 13:12 |
| sshnaidm|ruck | ykarel, ack, thanks | 13:13 |
| sshnaidm|ruck | ykarel, do we have a LP about it^ ? | 13:14 |
| ykarel | sshnaidm|ruck, nope | 13:14 |
| sshnaidm|ruck | ykarel, ok, will submit too | 13:14 |
| ykarel | sshnaidm|ruck, Thanks | 13:14 |
| ykarel | i asked Alan from cinder to fix that, he has some patches for it | 13:14 |
| ykarel | sshnaidm|ruck, i added links for those in card | 13:15 |
| sshnaidm|ruck | ykarel, cool | 13:15 |
| ykarel | u can add him once the bug is in CIX card, in RDo ci board i can't find him | 13:16 |
| sshnaidm|ruck | ok | 13:16 |
| ssbarnea | https://review.openstack.org/#/c/614762/ | 13:22 |
| ssbarnea | https://review.openstack.org/#/c/614762/ -- extras - working on it. | 13:23 |
| *** ykarel_ has joined #oooq | 13:40 | |
| sshnaidm|ruck | ssbarnea, you uploaded a new patch with a change that is not related to all issues written by me and panda: https://review.openstack.org/#/c/610491/ | 13:41 |
| sshnaidm|ruck | ssbarnea, please avoid to do so, because it erases everything and may lead to merging patch without fixing and responding to issues | 13:42 |
| *** ykarel has quit IRC | 13:42 | |
| sshnaidm|ruck | ssbarnea, try please not to upload a new patch while you didn't fix and respond all issues from previous patches - that's how we work in upstream | 13:42 |
| ykarel_ | sshnaidm|ruck, ocker pull failed: Error while pulling image: Get http://192.168.24.1:8787/v1/repositories/tripleomaster/centos-binary-mariadb/images: dial tcp 192.168.24.1:8787: getsockopt: connection refused", new issue in phase 1 job or already known | 13:43 |
| *** ykarel_ is now known as ykarel | 13:43 | |
| ssbarnea | sshnaidm|ruck i added -W to it, i still have other things to fix in it, working now to address all of them. | 13:44 |
| sshnaidm|ruck | ssbarnea, it doesn't matter, better not to upload a new patch if you didn't fix all issues | 13:44 |
| *** vinaykns has joined #oooq | 13:44 | |
| ssbarnea | sshnaidm|ruck : sorry but because we have drafts disabled there is no way to test without uploading new version. | 13:45 |
| sshnaidm|ruck | ssbarnea, that's not correct, there are various ways to test it | 13:45 |
| ssbarnea | with CI, not really. anyway i will try to minimize the number of updates. | 13:46 |
| sshnaidm|ruck | ssbarnea, there are various ways to test and I hope as part of CI team you're familiar with ways to test things without CI patch to save load on CI | 13:47 |
| sshnaidm|ruck | ssbarnea, if it's rare case and impossible to test anywhere - you can create a dummy patch from your one and test there "aside" | 13:47 |
| ykarel | yup also try to keep minimum number of jobs by adjusting zuul.d/ when testing patches | 13:50 |
| ykarel | atleast the people aware should use to save load on CI | 13:50 |
| marios | rlandy: which story was it this one can't see the review https://tree.taiga.io/project/tripleo-ci-board/task/284?kanban-status=1447275 https://review.openstack.org/#/c/613414/1/roles/nodepool-setup/tasks/main.yml this? | 13:51 |
| marios | rlandy: (we can have a call in a bit better) | 13:52 |
| rlandy | marios:ack - ok | 13:52 |
| * chandankumar relocates to home | 13:54 | |
| sshnaidm|ruck | nhicher, I wonder why rocky job didn't run anything: https://logs.rdoproject.org/32/614632/1/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-vexxhost/36ebcca/job-output.txt.gz | 13:55 |
| sshnaidm|ruck | nhicher, any ideas? | 13:55 |
| ssbarnea | panda sshnaidm|ruck : do we have bash inside our containers? | 13:55 |
| *** saneax has quit IRC | 13:55 | |
| nhicher | sshnaidm|ruck: probably https://review.rdoproject.org/r/#/c/16917 | 13:55 |
| panda | ssbarnea: ... do we have libc ? :) | 13:56 |
| sshnaidm|ruck | ssbarnea, I suppose so, but you can check it very easily :) | 13:56 |
| sshnaidm|ruck | nhicher, ok, merged, let's try again | 13:56 |
| ssbarnea | to be correct: in a container you do not make guarantees, some systems do not have bash. but i assume we have it. | 13:56 |
| ssbarnea | trying to fix the yum detection issue that sshnaidm|ruck pointed earlier today. | 13:57 |
| sshnaidm|ruck | ssbarnea, docker run tripleomaster/centos-binary-base:current-tripleo bash -c "echo success" | 14:02 |
| sshnaidm|ruck | ssbarnea, if you see success, we have bash | 14:02 |
| ssbarnea | thanks! | 14:03 |
| *** gkadam has joined #oooq | 14:13 | |
| weshay | panda, https://review.openstack.org/#/q/topic:reduce-tripleo-usage+(status:open+OR+status:merged) | 14:16 |
| ssbarnea | sshnaidm|ruck weshay rlandy : a quick comment on https://review.openstack.org/#/c/610491/30/roles/collect-logs/tasks/collect.yml line 20 to make a decision. | 14:18 |
| weshay | panda, https://review.openstack.org/#/c/509728/ | 14:20 |
| weshay | panda, sorry dropped, back | 14:22 |
| sshnaidm|ruck | ssbarnea, do you want to call file "packages.txt"? | 14:27 |
| ssbarnea | only if the others agree. i do agree with panda that having different filename does not make much sense (harder to compare between builds,...). | 14:38 |
| ssbarnea | i am ready to make the switch to a single filename, but i want to hear others take on this. | 14:39 |
| sshnaidm|ruck | ssbarnea, pandas commend it very reasonable | 14:39 |
| ssbarnea | sure, i prefer shorter urls, but with two already on package-list-installed.txt - i guess that would be the winnder. | 14:41 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000 (2 more messages) | 14:41 |
| weshay | ssbarnea, this is what I meant https://review.openstack.org/#/c/582735/ | 14:45 |
| weshay | added comment to https://review.openstack.org/#/c/614516 | 14:46 |
| ssbarnea | thanks | 14:47 |
| weshay | sshnaidm|ruck, ykarel thinking we should close the trello board https://trello.com/b/U1ITy0cu/tripleo-and-rdo-ci | 14:49 |
| weshay | wdyt | 14:49 |
| sshnaidm|ruck | weshay, I think need to make it ro, not close | 14:49 |
| sshnaidm|ruck | weshay, there is a lot of data I wouldn't like to lose.. | 14:50 |
| weshay | sshnaidm|ruck, didn't see that as an option, did I miss it? | 14:50 |
| weshay | sshnaidm|ruck, agree | 14:50 |
| ykarel | weshay, fine for me | 14:50 |
| panda | right, rename it to ρ | 14:50 |
| rlandy | panda: ping ... should collect_logs have access to zuul. vars? | 14:51 |
| weshay | rfolco|rover, 1yr | 14:52 |
| weshay | congrats :) | 14:52 |
| rlandy | ie: in collect logs, should I be able to access zuul.executor.*?? | 14:52 |
| panda | rlandy: can't see why not | 14:52 |
| rlandy | marios: we do here ... https://github.com/openstack-infra/tripleo-ci/blob/475da5cdf82ed3db25d0ce2fca2140bdaffcea01/playbooks/tripleo-buildimage/post.yaml#L8 | 14:52 |
| rlandy | panda: error tell me zuul. is undefined | 14:53 |
| panda | rlandy: ok, we are actually not passing that file as extra vars to collect-logs call currently | 14:53 |
| rlandy | hmmm ... | 14:54 |
| rlandy | looking for where we pass that to other playbook calls | 14:54 |
| weshay | sshnaidm|ruck, nhicher afaict .. ovb jobs are working atm | 14:54 |
| weshay | rocky is passing in check | 14:54 |
| weshay | am_i_right? | 14:54 |
| weshay | in rdo | 14:54 |
| rlandy | --extra-vars @/home/zuul/workspace/logs/zuul-variables.yaml | 14:54 |
| rlandy | ha | 14:54 |
| panda | rlandy: yep | 14:54 |
| rlandy | marios: ^^ | 14:55 |
| nhicher | weshay: this one https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch ? | 14:55 |
| weshay | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch | 14:55 |
| weshay | ya | 14:55 |
| rlandy | panda: ok - will put in a review to address that | 14:55 |
| weshay | nhicher, oof | 14:55 |
| weshay | nhicher, nvrmind | 14:56 |
| sshnaidm|ruck | weshay, actually they work, except of fs001 which still fails on ironic | 14:56 |
| weshay | sshnaidm|ruck, ya.. /me checking bmc node for a job where ironic failed | 14:56 |
| sshnaidm|ruck | weshay, 035 has pretty nice rate of 60% | 14:57 |
| weshay | sshnaidm|ruck, heh.. not good enough | 14:57 |
| sshnaidm|ruck | weshay, but load today is less because holidays | 14:57 |
| weshay | sshnaidm|ruck, aye | 14:57 |
| sshnaidm|ruck | weshay, we'll see how it goes in weekend.. | 14:57 |
| weshay | so sshnaidm|ruck nhicher nobody object to turning off check jobs in tripleo afaict | 14:57 |
| weshay | sshnaidm|ruck, aye | 14:57 |
| weshay | rfolco|rover, we rolling logs yet? | 14:58 |
| weshay | sshnaidm|ruck, the data in sova is taking awhile to refresh | 14:59 |
| ssbarnea | sshnaidm|ruck panda : thanks for the comments, i think I addressed all of them on https://review.openstack.org/#/c/610491/ | 15:01 |
| marios | rlandy: so its using the zuul-variables? | 15:01 |
| marios | for the executor | 15:01 |
| sshnaidm|ruck | weshay, yeah, need to work on sova :) | 15:01 |
| marios | rlandy: (or rather, can you use the zuul-variables.yaml for the executor) | 15:02 |
| rlandy | it apears | 15:02 |
| rlandy | appears | 15:02 |
| rlandy | putting in a review to check | 15:02 |
| ssbarnea | weshay: now a newbie question related to testing. Why de do not have a pattern set on "roles/build-test-packages/.*" to run the job(s) needed for validating a change on it? is not possible? | 15:04 |
| sshnaidm|ruck | ssbarnea, thanks, let's wait CI jobs to finish | 15:04 |
| rfolco|rover | weshay, 1yr: thanks | 15:05 |
| ssbarnea | sshnaidm|ruck i didn't remove the rdopkg part from it, do you want me to do so? i did it because i wanted to avoid rebase. | 15:05 |
| sshnaidm|ruck | ssbarnea, which part? | 15:05 |
| rlandy | panda: https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/templates/toci_quickstart.sh.j2#L154? | 15:06 |
| rlandy | looks like we are passing zuul_vars to collect logs | 15:06 |
| ssbarnea | sshnaidm|ruck bottom of https://review.openstack.org/#/c/610491/32/roles/build-test-packages/tasks/main.yml | 15:06 |
| rfolco|rover | weshay, logs... please review https://review.rdoproject.org/r/#/c/17102/ and https://review.rdoproject.org/r/#/c/17192/ sshnaidm|ruck and panda | 15:07 |
| rlandy | oh wait | 15:07 |
| rfolco|rover | manual test works | 15:07 |
| rlandy | that's different | 15:07 |
| *** gkadam has quit IRC | 15:09 | |
| sshnaidm|ruck | ssbarnea, it's unrelated change which is not mentioned in comment | 15:09 |
| sshnaidm|ruck | ssbarnea, better to create a patch on top with it.. | 15:10 |
| rlandy | marios: https://review.openstack.org/#/c/614788/ | 15:10 |
| panda | rlandy: nope we are passing it twice because of the dryrun | 15:10 |
| rlandy | panda: so the review is incorrect? | 15:11 |
| rlandy | panda: ^^? | 15:11 |
| ssbarnea | sshnaidm|ruck sure... bit more pressure on zuul, i will rebase rdopkg bumping on top of this one. | 15:11 |
| marios | rlandy: thanks | 15:12 |
| panda | rlandy: the review is correct | 15:12 |
| rlandy | cool | 15:12 |
| sshnaidm|ruck | ssbarnea, in case we need to revert py3 stuff, we'll also revert rdopkg, and vice versa. That's why to have different commits for different changes | 15:12 |
| rlandy | marios: https://review.openstack.org/614645 | 15:12 |
| rlandy | ^^ will watch the ci on this ... | 15:13 |
| rlandy | see if it does any better | 15:13 |
| ssbarnea | panda sshnaidm|ruck rlandy : qs.sh fix on yum install, please have a brief look, even without zuul reponse https://review.openstack.org/#/c/614762 | 15:13 |
| *** ykarel_ has joined #oooq | 15:13 | |
| panda | marios: you're very brave +2 before the jobs results :) | 15:14 |
| ssbarnea | sshnaidm|ruck : i can do a nasty jinja2 hack: use old version for py2 and use new version for py3 :D | 15:14 |
| marios | panda: well that one just passes in the zuul-variables | 15:14 |
| marios | panda: i mean it isn't doing anything yet so that one is pretty safe i think | 15:14 |
| ssbarnea | sshnaidm|ruck do you find the jinja2 approach acceptable? it would be safe. | 15:15 |
| *** ykarel has quit IRC | 15:16 | |
| *** ykarel_ is now known as ykarel | 15:16 | |
| sshnaidm|ruck | ssbarnea, idk, let's see | 15:16 |
| sshnaidm|ruck | ssbarnea, what's the problem with rdopkg? | 15:16 |
| ssbarnea | py3 runtime bug.... something with basestring :D | 15:16 |
| ssbarnea | old version is known not to work on py3. so better to put conditional here and after we get result from wes job we can remove the version branching. | 15:17 |
| *** apetrich has quit IRC | 15:18 | |
| ykarel | ssbarnea, isn't the rdopkg bug fixed with: https://softwarefactory-project.io/r/#/c/14080/ | 15:19 |
| ykarel | ? | 15:19 |
| ykarel | sorry if i missed the discussion, just asking | 15:20 |
| ssbarnea | ykarel yes is fixed in rdopkg=0.47.3 but we currently have rdopkg=0.46.3 and the question was about which change would go first, changing the pinning requires extra testing. | 15:20 |
| ykarel | ssbarnea, okk got it | 15:21 |
| *** ykarel is now known as ykarel|away | 15:22 | |
| *** saneax has joined #oooq | 15:22 | |
| ykarel|away | may be testing can go in parallel | 15:24 |
| ssbarnea | i branched it, it do I would see what happens with f28 job, https://review.openstack.org/#/c/610491/34/roles/build-test-packages/tasks/main.yml | 15:24 |
| ssbarnea | ykarel|away in fact there is job already created for this, it will go in parallel. | 15:25 |
| ykarel|away | okk cool | 15:26 |
| ssbarnea | reharding r8 please real last comment on https://bugzilla.redhat.com/show_bug.cgi?id=1632162 -- we may have a new surprise. still, probably we should focus on f28 for the moment. | 15:27 |
| openstack | ssbarnea: Error: Error getting bugzilla.redhat.com bug #1632162: NotPermitted | 15:27 |
| panda | ssbarnea: .... | 15:35 |
| panda | is ruby installed by default ? | 15:35 |
| ssbarnea | panda where? i doubt is default anywhere. | 15:36 |
| *** apetrich has joined #oooq | 15:36 | |
| panda | ssbarnea: if we switch to puppet for example, and ruby is installed by default maybe we can work around this | 15:37 |
| panda | anda there are still people that consider that once we complete f28, the others will be easy | 15:38 |
| panda | any distro/version we add will be the same amount of work every time | 15:38 |
| ssbarnea | haha, dream on. anyway py* may not prove such a big problem in the end. | 15:39 |
| ssbarnea | and regarding puppet / ruby.... i hope it was a joke. | 15:39 |
| panda | ssbarnea: no, of course, you have to run a pre run, ssh -c "please please please install python | 15:39 |
| panda | ssbarnea: yes it was, I was really meaning salt | 15:40 |
| ssbarnea | panda there is work on upstream python to deal with this, i think is called pyv or something like this, a python launcher. | 15:40 |
| ssbarnea | so we should be ok | 15:40 |
| ssbarnea | not to mention that there always the option do do a "ln -s" | 15:41 |
| panda | well, it all depends on how the images are built | 15:41 |
| panda | I don't think they will leave out python by defaukt | 15:41 |
| panda | ssbarnea: OOO | 15:42 |
| panda | ssbarnea: OOOOOHHHHH | 15:42 |
| panda | ansible myhost --sudo -m raw -a "yum install -y python2" | 15:42 |
| panda | problem solved | 15:42 |
| panda | thanks ansible | 15:42 |
| panda | well it will be of course "dnf install -y python3" | 15:43 |
| *** agopi is now known as agopi|brb | 15:46 | |
| *** agopi|brb has quit IRC | 15:50 | |
| ssbarnea | panda to be practical, should we involve our PM with the bug above, if we do it now it make make our life much easier when the moment comes. we have some assumptions with f28, lets be sure others don't break them. | 15:55 |
| weshay | sshnaidm|ruck, rfolco|rover ok.. ack to turn off 3rd party tripleo | 15:55 |
| sshnaidm|ruck | weshay, maybe after a weekend? | 15:55 |
| weshay | sshnaidm|ruck, k | 15:56 |
| rfolco|rover | weshay, sshnaidm|ruck fyi holiday tomorrow | 15:58 |
| sshnaidm|ruck | rfolco|rover, are you on holiday? | 15:58 |
| rfolco|rover | tomorrow yes | 15:58 |
| sshnaidm|ruck | rfolco|rover, ack, I don't work in Fri too | 15:59 |
| panda | ssbarnea: there's no time to solve problems before the happen | 15:59 |
| rfolco|rover | sshnaidm|ruck, I think I forgot to rebase log rotation patches, could you please merge it ? | 16:04 |
| sshnaidm|ruck | rfolco|rover, link? | 16:08 |
| rfolco|rover | https://review.rdoproject.org/r/17192 | 16:08 |
| rfolco|rover | https://review.rdoproject.org/r/17102 | 16:08 |
| sshnaidm|ruck | rfolco|rover, done | 16:09 |
| vinaykns | weshay: I have reproduced an environment for the changes that I have pushed | 16:10 |
| vinaykns | weshay: but I couldn't login into the overcloud...as nova list is empty | 16:10 |
| *** agopi has joined #oooq | 16:12 | |
| ssbarnea | panda : put a comment there, warning that this behavior would have this impact and if we are lucky, the release may not have this behavior. | 16:15 |
| *** ykarel|away has quit IRC | 16:21 | |
| ssbarnea | weshay : is in our power to control how http://logs.openstack.org/62/614762/3/check/tripleo-ci-centos-7-containers-multinode/9e6904f/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-11-01_14_03_07 looks? | 16:38 |
| ssbarnea | i find very hard to read the output due to two layers of json-inification of stdout/stderr. | 16:39 |
| ssbarnea | for ansible we can tell it to use yaml output which is clearly easier to read. | 16:39 |
| rlandy | marios: http://logs.openstack.org/45/614645/3/check/tripleo-ci-centos-7-standalone/3b3b516/logs/quickstart_collect_logs.log | 16:40 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000 (2 more messages) | 16:41 |
| marios | rlandy: hm some error | 16:41 |
| rlandy | marios: says the files aren;t there | 16:42 |
| marios | rlandy: damn | 16:42 |
| rlandy | the copy is correct | 16:42 |
| rlandy | not done yet :) | 16:42 |
| marios | rlandy: ack, i will update mine tomorrow in the least to use zuul rather than reading it myself from zuul-variables and if this is successful then use it for playbooks. otherwise i can keep going down the inheritance path route but it is not nice | 16:43 |
| marios | s/nice/sane | 16:43 |
| rlandy | I'll work on it more | 16:43 |
| panda | ssbarnea: before you go, any reviews to review ? | 16:43 |
| panda | ssbarnea: review me up, before you go go. | 16:44 |
| rlandy | ah - maybe logs not readable | 16:45 |
| ssbarnea | panda failed but not sure why https://review.openstack.org/#/c/614762/ | 16:45 |
| panda | marios: rlandy https://tree.taiga.io/project/tripleo-ci-board/task/284?kanban-status=1447275 last comment. | 16:46 |
| rlandy | panda: ack - I know all that info | 16:47 |
| marios | rlandy: thanks | 16:47 |
| marios | er panda | 16:47 |
| rlandy | but the failure is saying the file is not there | 16:47 |
| rlandy | panda: pls see ... | 16:47 |
| panda | :( | 16:47 |
| rlandy | https://review.openstack.org/#/c/614645/3/roles/create-reproducer-script/tasks/main.yml | 16:47 |
| rlandy | panda: you see that I am trying to pick up the json file | 16:47 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 16:48 | |
| rlandy | from zuul.executor.log_root | 16:48 |
| panda | never an original idea from me | 16:48 |
| rlandy | panda: it may be that the logs are not readable | 16:48 |
| rlandy | I am adding a change to make them readable | 16:49 |
| panda | rlandy: maybe we can get a ls to start | 16:49 |
| rlandy | panda: sec - putting in new review for you to see | 16:49 |
| ssbarnea | few weeks back i tried to propose using yaml output instead of default json. weshay should remember. is doable. for main zuul one is not so easy but for ours it is. | 16:49 |
| weshay | ssbarnea, for the ansible output? | 16:49 |
| weshay | ya.. that merged | 16:50 |
| ssbarnea | hmm, probably is not used, based on what i see at http://logs.openstack.org/62/614762/3/check/tripleo-ci-centos-7-containers-multinode/9e6904f/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-11-01_14_03_07 -- i would say that this is JSON, not yaml. | 16:51 |
| ssbarnea | my browser chokes on that file. | 16:52 |
| rlandy | panda: https://review.openstack.org/#/c/614645/4/roles/create-reproducer-script/tasks/main.yml | 16:53 |
| rlandy | marios: ^^ | 16:54 |
| rlandy | panda: it's good to know that your chat with infra confirmed our direction was correct | 16:54 |
| panda | rlandy: direct check https://review.openstack.org/614811 I want to know what's available in that dir | 16:57 |
| *** rfolco|rover is now known as rfolco|ruck | 16:58 | |
| panda | ssbarnea: http://logs.openstack.org/62/614762/3/check/tripleo-ci-fedora-28-standalone/4a45f55/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz | 16:59 |
| rlandy | panda: I think the files are there but not readable | 16:59 |
| panda | ssbarnea: no yum command found for fedora28 | 16:59 |
| rlandy | panda: per https://git.zuul-ci.org/cgit/zuul-jobs/tree/roles/upload-logs/tasks/main.yaml?id=ad683253ed90202d3b15e41232daf90e9f96309e#n17 | 17:00 |
| rlandy | I added that to my commit | 17:00 |
| panda | rlandy: yep I see it here http://logs.openstack.org/45/614645/2/check/tripleo-ci-centos-7-containers-multinode/f1fb82a/job-output.txt.gz#_2018-11-01_04_14_35_691451 | 17:00 |
| panda | not wure why they are not readable by default | 17:00 |
| ssbarnea | panda : thanks, time to make it: $(command -v dnf || command -v yum ) -- bash failes me less than other come complex solutions. | 17:02 |
| rlandy | panda: there is a lot I am not sure about these days ... | 17:03 |
| *** udesale has quit IRC | 17:06 | |
| panda | rlandy: ok, but you are Wure about a lot of things instead. | 17:08 |
| rlandy | panda: no, I haven't reach that level yet | 17:10 |
| rlandy | reached | 17:11 |
| rlandy | panda: marios: the issue with this whole approach is that if there is a failure, the reproducer will only run playbooks to the point of the failure - and no more | 17:11 |
| *** trown is now known as trown|lunch | 17:12 | |
| rlandy | but - if we get it working, it's something | 17:12 |
| panda | 30 minutes to wait for check queue | 17:12 |
| panda | :( | 17:12 |
| rlandy | we can always compare to a working job to get the whole list | 17:12 |
| marios | rlandy: thanks i have to go now willl check that review in the morning for progress thanks for the help! | 17:14 |
| panda | neat, I have a webcam again. | 17:14 |
| rlandy | marios; sure - hope we have something to show you | 17:14 |
| *** sshnaidm|afk is now known as sshnaidm|off | 18:00 | |
| rlandy | "stderr": "chmod: cannot access ‘/var/lib/zuul/builds/0b4da4e9c9fa4d42bdc1a19f3f827267/work/logs/’: No such file or directory", | 18:39 |
| rlandy | panda: ^^ | 18:39 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000 (3 more messages) | 18:41 |
| rlandy | panda: want to ask you infra buddies about that?? | 18:43 |
| rlandy | it should follow their exact suggestion | 18:44 |
| panda | rlandy: where's the log ? | 18:44 |
| rlandy | http://logs.openstack.org/45/614645/4/check/tripleo-ci-centos-7-standalone/0b4da4e/logs/quickstart_collect_logs.log | 18:44 |
| rlandy | panda: ^^ | 18:45 |
| panda | rlandy: I have it | 18:47 |
| panda | rlandy: the log_root is in the zuul executor | 18:52 |
| panda | rlandy: not on undercloud | 18:52 |
| panda | rlandy: to use it in collect logs we need to copy it from the executor to undercloud | 18:53 |
| panda | rlandy: still trying to get https://review.openstack.org/614811 to show me what's in this dir | 18:53 |
| rlandy | panda: delegate localhost | 18:53 |
| *** rfolco has joined #oooq | 18:54 | |
| rlandy | panda: how on earth are we supposed to copy? | 18:54 |
| rlandy | we can't reach anything | 18:54 |
| * rlandy gets | 18:54 | |
| panda | rlandy: delegate_localhost in collect logs, localhost is still the undercloud | 18:55 |
| panda | mmhh | 18:55 |
| rlandy | panda: so get rid of that line? | 18:55 |
| rlandy | pls comment on the review so I can fix | 18:55 |
| *** rfolco|ruck has quit IRC | 18:56 | |
| panda | rlandy: not that easy | 18:56 |
| *** apetrich has quit IRC | 18:56 | |
| panda | rlandy: in post-run we run post.yaml | 18:56 |
| panda | rlandy: which calls collect_logs.sh | 18:56 |
| panda | rlandy: which runs ansible on undercloud | 18:57 |
| panda | rlandy: we need a copy in the post.yaml itself, then we can use the file in collect_logs.sh | 18:57 |
| panda | oh I hate myself still can't get my review right | 19:04 |
| *** vinaykns has quit IRC | 19:04 | |
| rlandy | panda: ok - so ... https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/post.yaml#L18 | 19:05 |
| rlandy | I want to sync the other way | 19:05 |
| rlandy | before collect_logs is called | 19:05 |
| rlandy | more this line really ... | 19:06 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/post.yaml#L22 | 19:06 |
| panda | rlandy: yes | 19:07 |
| panda | rlandy: but from log_root | 19:08 |
| rlandy | ok - review in progress | 19:08 |
| *** apetrich has joined #oooq | 19:08 | |
| panda | rlandy: and then now sure what you want to use in undercloud as dir .. I don't think you can use zuul.executor.log_root value, doesnt make much sense in undercloud | 19:08 |
| rlandy | sec ... let's go with a review step by step | 19:09 |
| panda | rlandy: http://logs.openstack.org/11/614811/6/check/tripleo-ci-fedora-28-standalone/e2fff96/ls-log.txt | 19:13 |
| panda | rlandy: this is ls -l log_root on the executor | 19:13 |
| panda | rlandy: at the start of the run playbook | 19:14 |
| panda | rlandy: so we have it | 19:14 |
| *** trown|lunch is now known as trown | 19:14 | |
| rlandy | panda: https://review.openstack.org/614837 ? | 19:15 |
| rlandy | will do the analysis of the file in reproducer step | 19:16 |
| rlandy | only want the step here to get the file | 19:16 |
| rlandy | ugh - forgot a / | 19:17 |
| panda | rlandy: commented ps 1 | 19:19 |
| panda | rlandy: also, you'll probably just need the .json | 19:20 |
| *** panda is now known as panda|off | 19:20 | |
| rlandy | panda|off: thanks - patch 3 in test | 19:23 |
| rlandy | if you're still around, a quick review would be appreciated | 19:23 |
| rfolco | weshay, I see 4 gate failures today, and 3 post failures (timeouts)... failures are all the same --> pbr pypi fails to install: http://logs.openstack.org/68/614168/1/gate/tripleo-ci-centos-7-undercloud-containers/a7d8e31/job-output.txt.gz#_2018-11-01_11_55_19_711212 | 19:26 |
| weshay | ssbarnea, test passed +2 on https://review.openstack.org/#/c/614516/ | 19:27 |
| rfolco | weshay, wondering if this is not just a network issue... http://mirror.iad.rax.openstack.org/pypi/simple/pbr/ seems to be reachable atm | 19:27 |
| weshay | rfolco, ya.. so this is the type of issue I'm going to be leaning on you and ssbarnea on | 19:28 |
| weshay | rfolco, I'm not quite sure how to debug that | 19:28 |
| panda|off | rlandy looks good hope rsync doesn't complain | 19:28 |
| weshay | rfolco, can you pull other packages? | 19:28 |
| rlandy | we'll see ... complain about what? | 19:29 |
| panda|off | anything | 19:31 |
| panda|off | never seen sync module work at the first try | 19:31 |
| panda|off | on | 19:31 |
| ssbarnea | weshay: i am aware about the mirror flakiness, i already spoke few days ago on infra about that, now they are transparent mirrors in front of CDN. I know for sure that PYPI CDN happened to return 404 historically for some requests... there is no known solution right now, not without changing pip itself. i will try to create a bug with pip or find an existing one. | 19:32 |
| weshay | ssbarnea++ thanks | 19:32 |
| hubbot1 | weshay: ssbarnea's karma is now 3 | 19:32 |
| ssbarnea | mainly "recheck" for now and if you want to do something: find a way to count these failures across time, this would help escalate it. | 19:32 |
| weshay | ssbarnea, elastic search no? | 19:32 |
| weshay | and openstack-health | 19:33 |
| ssbarnea | i hate to give the recheck answer, i really do. | 19:33 |
| weshay | ssbarnea, rfolco I need you guys to be core on elastic-recheck | 19:33 |
| weshay | ssbarnea, rfolco /me creates a task for you two | 19:34 |
| ssbarnea | outch, this is an areas where I have no knowledge yet. but if boss says we need take that hill down, we do :D | 19:34 |
| weshay | ssbarnea, in the meeting the infra folks thought they may just give it to you | 19:35 |
| weshay | but it would be a really good idea for you two to study it a little | 19:35 |
| weshay | after f28 merges ;) | 19:36 |
| weshay | ssbarnea, rfolco http://status.openstack.org/elastic-recheck/ | 19:36 |
| rfolco | I was not sure how to ask "hey can I be core?" | 19:36 |
| weshay | https://github.com/openstack-infra/elastic-recheck | 19:36 |
| ssbarnea | my personal wish was to improve a little bit the log browsing experience, a bit. | 19:37 |
| rfolco | non-classified tripleo bugs may justify that | 19:37 |
| weshay | we talked about it in the meeting we had w/ them | 19:37 |
| weshay | rfolco, ssbarnea the point is.. I need folks that can merge new queries | 19:37 |
| weshay | if that doesn't make sense right now.. | 19:37 |
| weshay | make sure it makes sense by next week and we'll chat | 19:37 |
| weshay | about it | 19:37 |
| ssbarnea | sure, cc me anyway. doing reviews is the first step anyway, | 19:38 |
| weshay | exactly | 19:38 |
| rfolco | weshay, ssbarnea maybe we can start classifying tripleo bugs and add queries there | 19:39 |
| weshay | https://tree.taiga.io/project/tripleo-ci-board/issue/308?kanban-status=2027733 | 19:39 |
| weshay | rfolco, ding! | 19:39 |
| ssbarnea | depends, it makes sense only for things likely to occur. | 19:39 |
| weshay | rfolco, yes.. we just havent been able to do it.. because.... | 19:39 |
| weshay | wait for it | 19:39 |
| weshay | wait for it | 19:39 |
| weshay | wait for it | 19:39 |
| weshay | wait for it | 19:39 |
| weshay | we have no cores | 19:40 |
| rfolco | deadlock | 19:40 |
| rfolco | weshay, I'll take a look at the uncategorized tripleo issues | 19:42 |
| *** leifmadsen has left #oooq | 19:42 | |
| weshay | rfolco, have something for you actually | 19:44 |
| * weshay emailing and tasking atm | 19:44 | |
| rfolco | weshay, you always do | 19:44 |
| rfolco | :) | 19:45 |
| weshay | rfolco, let me know if you have ANY questions https://tree.taiga.io/project/tripleo-ci-board/issue/309?kanban-status=2027733 | 19:50 |
| rfolco | weshay, fs021 has CR repo on. Then you point to results on fs20 | 19:52 |
| weshay | bah.. ur right https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-master%09 | 19:53 |
| weshay | rfolco, that's right.. this is worse | 19:53 |
| weshay | saw this the other day | 19:53 |
| weshay | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-master/02cf30a/ | 19:53 |
| weshay | no logs | 19:53 |
| weshay | rfolco, suggestion.. pull a reproducer for fs20, change the fs to 21 | 19:54 |
| weshay | rfolco, or try a known good master deployment | 19:54 |
| weshay | w/ that repo added to it | 19:54 |
| weshay | rfolco, ideally we though.. we need daily ci results | 19:55 |
| weshay | rfolco, get help from the team if you need it | 19:57 |
| weshay | do not, I repeat do not sit on this | 19:57 |
| weshay | RHEL 7.6 shipped on oct 30 | 19:58 |
| rfolco | weshay, why all these are skipped ? https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-master%09 | 20:20 |
| rfolco | coz containers-build job failed | 20:21 |
| rfolco | ? | 20:21 |
| rfolco | rlandy, you know ^? | 20:22 |
| rlandy | looking | 20:25 |
| rlandy | rfolco: likely | 20:26 |
| *** rlandy is now known as rlandy|brb | 20:26 | |
| rfolco | rlandy, thx | 20:26 |
| rfolco | rlandy|brb, you too, thx | 20:26 |
| rfolco | :) | 20:26 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000 (3 more messages) | 20:41 |
| *** saneax has quit IRC | 20:50 | |
| *** trown is now known as trown|outtypewww | 21:11 | |
| *** rlandy|brb is now known as rlandy | 21:21 | |
| *** agopi is now known as agopi|pto | 21:36 | |
| *** agopi|pto has quit IRC | 21:41 | |
| ssbarnea | got into few new errors with f28: shyaml fails to run due to missing yaml module, yum installs part of python2 even if it has an exclude on it! http://logs.openstack.org/16/614516/2/check/tripleo-ci-fedora-28-standalone/7b10b3a/job-output.txt.gz | 22:03 |
| ssbarnea | found the place but I have question, I want to know if toci_gate_test.sh.j2 would run from a machine with same system, mainly if I can use python.version.major from ansible to decide what to do, i need to use bash. | 22:11 |
| ssbarnea | weshay rlandy rfolco : https://review.openstack.org/#/c/614884/ -- i am making it now a dependency but ready for review. | 22:33 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000 (3 more messages) | 22:41 |
| *** agopi|pto has joined #oooq | 23:10 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!