| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/560445, stable/queens: tripleo-ci- (1 more message) | 00:17 |
|---|---|---|
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 02:17 |
| *** saneax has joined #oooq | 02:26 | |
| *** skramaja has joined #oooq | 03:38 | |
| *** skramaja_ has joined #oooq | 03:43 | |
| *** skramaja has quit IRC | 03:43 | |
| *** skramaja_ is now known as skramaja | 03:59 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 04:17 |
| *** udesale has joined #oooq | 04:33 | |
| *** ykarel has joined #oooq | 05:49 | |
| *** ratailor has joined #oooq | 06:08 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 06:17 |
| *** jschluet has quit IRC | 06:23 | |
| *** jschlueter has joined #oooq | 06:27 | |
| *** jtomasek has joined #oooq | 06:53 | |
| *** apetrich has joined #oooq | 07:06 | |
| *** kopecmartin|off is now known as kopecmartin | 07:21 | |
| *** ykarel is now known as ykarel|lunch | 07:29 | |
| *** agopi_ has joined #oooq | 07:37 | |
| *** agopi_ is now known as agopi | 07:40 | |
| *** tosky has joined #oooq | 07:45 | |
| chandankumar | marios: quiquell|off \o/ | 08:08 |
| chandankumar | marios: I was trying to run os_tempest on standalone but it is not working | 08:08 |
| chandankumar | marios: https://review.openstack.org/628421 -> adds os_tempest as TQE requires in TQ | 08:09 |
| chandankumar | marios: https://review.openstack.org/628415 adds playbook for os_tempest in TQE | 08:09 |
| chandankumar | marios: https://review.openstack.org/627500 in tripleo ci is called with standalone job | 08:10 |
| chandankumar | but it is just showing this http://logs.openstack.org/00/627500/21/check/tripleo-ci-centos-7-standalone-os-tempest/eab1278/job-output.txt.gz#_2019-01-07_06_22_18_305515 | 08:11 |
| chandankumar | it is just showing play but it does not get executed | 08:11 |
| marios | chandankumar:o/ will have a look did you update the name issue/host name from last time (but there are more reviews now too looks like) | 08:11 |
| chandankumar | marios: earlier I was placing playbook in tripleo-ci with primary host there it was working | 08:12 |
| chandankumar | marios: better later on realized that that is not the right place to keep playbook so moved it to TQE | 08:12 |
| chandankumar | there I am using undercloud as host which is not working | 08:13 |
| marios | chandankumar: yeah i mean just looking at the output it looks like the same issue as last time (i.e. it isn't running because of maybe some conditional, or host mismatch/cant find it in the registry) | 08:13 |
| marios | chandankumar: i'll check the reviews see if i can spot something anyway gimme few | 08:13 |
| chandankumar | marios: sure take your time :-) | 08:13 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 08:17 |
| *** agopi has quit IRC | 08:18 | |
| marios | chandankumar: yeah i think it is indeed the 'undercloud' thing | 08:21 |
| marios | chandankumar: i am adding a comment but you are running the job with standalone so there is no undercloud | 08:21 |
| *** ykarel|lunch is now known as ykarel | 08:23 | |
| chandankumar | marios: for standalone, are we using different host? | 08:24 |
| *** saneax has quit IRC | 08:24 | |
| marios | http://logs.openstack.org/00/627500/21/check/tripleo-ci-centos-7-standalone-os-tempest/eab1278/zuul-info/inventory.yaml chandankumar | 08:24 |
| *** rascasoft has joined #oooq | 08:25 | |
| chandankumar | marios: so basically hosts depends on environment type? | 08:27 |
| chandankumar | marios: on undercloud job I am seeing env type to singlenode or for standalone to standalone | 08:28 |
| marios | chandankumar: yeah i mean it comes down to the nodeset i guess right and in this case there is only one | 08:28 |
| chandankumar | marios: sure updating the patch, let see how it goes. :-) | 08:33 |
| *** ratailor has quit IRC | 08:41 | |
| *** ratailor has joined #oooq | 08:44 | |
| *** skramaja_ has joined #oooq | 08:49 | |
| *** skramaja has quit IRC | 08:49 | |
| panda | SPAIN: why u no celebrate liek other countries!!!! | 08:54 |
| panda | also RDO developers: why u all from SPAIN ??? | 08:54 |
| *** ykarel has quit IRC | 09:06 | |
| *** ykarel has joined #oooq | 09:06 | |
| *** saneax has joined #oooq | 09:15 | |
| *** skramaja has joined #oooq | 09:15 | |
| *** skramaja_ has quit IRC | 09:16 | |
| *** bogdando has joined #oooq | 09:17 | |
| *** holser_ has joined #oooq | 09:25 | |
| *** derekh has joined #oooq | 09:36 | |
| *** ssbarnea|bkp2 has joined #oooq | 09:37 | |
| *** chem has joined #oooq | 09:38 | |
| *** ssbarnea has quit IRC | 09:39 | |
| *** saneax has quit IRC | 09:44 | |
| *** saneax has joined #oooq | 09:44 | |
| *** saneax has quit IRC | 09:45 | |
| *** saneax has joined #oooq | 09:45 | |
| ssbarnea|bkp2 | who has +wf on rdo? i need https://review.rdoproject.org/r/#/c/17732/ | 09:56 |
| *** jaosorior has joined #oooq | 10:03 | |
| *** dtantsur|afk is now known as dtantsur | 10:08 | |
| panda | ssbarnea|bkp2: ykarel | 10:11 |
| ykarel | looking | 10:14 |
| ssbarnea|bkp2 | panda: ykarel: also, more important (ovb related) one: https://review.rdoproject.org/r/#/c/18072/ -- branch already exists. | 10:16 |
| *** agopi has joined #oooq | 10:17 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 10:17 |
| ykarel | ssbarnea|bkp2, is this https://review.rdoproject.org/r/#/c/17732/ tested somehow | 10:20 |
| ykarel | in past i have seen some hardlink errors, but can't recall | 10:20 |
| *** agopi is now known as agopi_ | 10:20 | |
| *** agopi_ is now known as agopi | 10:21 | |
| arxcruz|brb | chandankumar: ping | 10:22 |
| ssbarnea|bkp2 | ykarel: i tested via https://review.openstack.org/#/c/620902 | 10:30 |
| ykarel | looking | 10:30 |
| *** sshnaidm|off is now known as sshnaidm | 10:30 | |
| ykarel | ssbarnea|bkp2, i can see following:- | 10:32 |
| ykarel | 2018-12-18 10:41:01.625412 | TASK [fetch-zuul-cloner-fork : Make repositories writable so that people can hardlink] | 10:32 |
| ykarel | 2018-12-18 10:41:14.240501 | primary | changed | 10:32 |
| ykarel | seems that's not tested, can u link the logs where u see it's tested | 10:32 |
| ssbarnea|bkp2 | ykarel: hmm, let me rebase it. | 10:32 |
| ssbarnea|bkp2 | ykarel: are you sure rdo-jobs can be testes like this? | 10:33 |
| ssbarnea|bkp2 | we should be able to find-out in few mintues. | 10:33 |
| chandankumar | arxcruz|brb: pong | 10:35 |
| arxcruz|brb | chandankumar: nevermind | 10:35 |
| *** arxcruz|brb is now known as arxcruz | 10:35 | |
| chandankumar | arxcruz|brb: ok | 10:35 |
| arxcruz | sshnaidm: hey, do you know where tripleo-ci-base-ovb-periodic is declared? | 10:35 |
| arxcruz | in which repo... | 10:35 |
| chandankumar | arxcruz: https://codesearch.rdoproject.org/?q=tripleo-ci-base-ovb-periodic&i=nope&files=&repos= | 10:36 |
| chandankumar | arxcruz: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo-rdo-base.yaml#L72 | 10:36 |
| arxcruz | chandankumar: thanks | 10:36 |
| sshnaidm | arxcruz, https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo-rdo-base.yaml#L72 | 10:36 |
| sshnaidm | chandankumar, ah! 2 seconds :D | 10:37 |
| * arxcruz hate this distributed zuul to search for jobs | 10:37 | |
| sshnaidm | arxcruz, totally.. | 10:37 |
| chandankumar | sshnaidm: :-) | 10:37 |
| ykarel | arxcruz, you can look at zuul jobs | 10:37 |
| ykarel | arxcruz, like https://review.rdoproject.org/zuul/job/tripleo-ci-base-ovb-periodic | 10:37 |
| arxcruz | ykarel: you know, i was looking there for the periodic job that inherit from it, and forgot to check the base job... | 10:38 |
| arxcruz | ykarel: thanks ! | 10:38 |
| ykarel | sshnaidm, don't think so depends-on rdo-jobs work there | 10:38 |
| ykarel | sorry ssbarnea|bkp2 ^^ | 10:39 |
| *** udesale has quit IRC | 10:54 | |
| rfolco | panda, ping | 11:00 |
| ssbarnea|bkp2 | ykarel: if this approach does not work, how we are supposed to test such a change to rdo-jobs? | 11:01 |
| ykarel | ssbarnea|bkp2, u can try running some jobs in rdo-review itself | 11:01 |
| rfolco | panda, trying to figure out what vars need change to build fedora containers before pushing to registry... wes pointed me to https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/ddb06e8/logs/kolla/kolla-build.conf | 11:02 |
| ykarel | ssbarnea|bkp2, for testing in upstream ci, i think u can ask with paul, tristanC, they can guide, /me not aware much on zuul side | 11:03 |
| ssbarnea|bkp2 | ykarel: i will switch to #rdo channel and ask there, i need to know. | 11:04 |
| ykarel | ack | 11:04 |
| ykarel | i too want to know :) | 11:04 |
| panda | rfolco: oink | 11:31 |
| rfolco | panda, boooh! | 11:31 |
| panda | rfolco: oida ? | 11:31 |
| rfolco | panda, which language is this ? piggy lang ? | 11:32 |
| panda | rfolco: german/austrian/bavarian | 11:32 |
| rfolco | panda, forgive my ignorance | 11:33 |
| rfolco | panda, centos-binary... besides your change to push the containers.... | 11:33 |
| rfolco | panda, is this the right place to change to make fedora containers ? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/ddb06e8/logs/kolla/kolla-build.conf | 11:33 |
| panda | rfolco: more or less, but this is generated | 11:35 |
| panda | rfolco: you'll have to modify the sources | 11:35 |
| panda | rfolco: fetching link | 11:35 |
| rfolco | https://github.com/rdo-infra/ansible-role-rdo-kolla-build/blob/master/templates/kolla-build.conf.j2 | 11:35 |
| rfolco | panda, this is the jinja | 11:35 |
| panda | rfolco: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/tripleo-upstream/build-containers-images.sh#L32 | 11:36 |
| panda | rfolco: we don't call kolla directly | 11:36 |
| panda | rfolco: we use an ansible role | 11:36 |
| panda | rfolco: we modify the variables there, then the role generates the conf with the variables | 11:36 |
| rfolco | panda, yes so besides base, trying to find what else to change | 11:38 |
| panda | the trunk registry | 11:46 |
| panda | and probably nothing else | 11:46 |
| rfolco | panda, k, wondering if I can test it locally | 11:47 |
| rfolco | running the role with the modified vars in the yaml file | 11:47 |
| panda | rfolco: take a look at the defaults https://github.com/rdo-infra/ansible-role-rdo-kolla-build/blob/master/defaults/main.yml | 11:48 |
| panda | rfolco: everything that has a "centos" needs to be modified | 11:48 |
| panda | rfolco: I count 4 | 11:48 |
| panda | kolla_virtualenv trunk_repository trunk_deps_repository kolla_base | 11:49 |
| panda | and kolla_virtualenv is probably even temporary | 11:49 |
| panda | so it probably doesn't affect the build if it's not changes | 11:49 |
| panda | d | 11:49 |
| *** ratailor has quit IRC | 11:51 | |
| rfolco | panda, obrigado, mestre. | 11:53 |
| ssbarnea|bkp2 | rfolco: do you think you could be qe for https://tree.taiga.io/project/tripleo-ci-board/task/556 ? | 11:54 |
| *** agopi has quit IRC | 11:56 | |
| rfolco | ssbarnea|bkp2, I am f*ed with f28 and tc stuff for the rest of this sprint. Probably not. | 11:56 |
| ssbarnea|bkp2 | rfolco: let me know if I can help on one of f28 ones | 11:57 |
| rfolco | ssbarnea|bkp2, thanks man, sorry for not being of help this time. | 11:58 |
| ssbarnea|bkp2 | sshnaidm: am I wrong to assume that in regards to ovb, we also want to merge https://review.openstack.org/#/c/628212/ ? | 12:10 |
| ssbarnea|bkp2 | if we do not, new patches to stable will be missed. | 12:11 |
| sshnaidm | ssbarnea|bkp2, no, it's not used in jobs | 12:11 |
| ssbarnea|bkp2 | sshnaidm: someone is using it, right? or is just a randome default value? :D | 12:12 |
| sshnaidm | ssbarnea|bkp2, rarely | 12:12 |
| sshnaidm | ssbarnea|bkp2, it should be merged, but it doesn't block anything | 12:12 |
| ssbarnea|bkp2 | sshnaidm: ok, clearly not a blocker. i will ping bnemec as now we should be able to do the switch. | 12:12 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 12:17 |
| sshnaidm | marios, can you +w please? https://review.openstack.org/#/c/628212/ | 12:18 |
| *** agopi has joined #oooq | 12:24 | |
| marios | sshnaidm: k | 12:25 |
| ssbarnea|bkp2 | panda: sshnaidm : wf https://review.openstack.org/#/c/627521/ about replace-scen4? | 12:26 |
| sshnaidm | +w | 12:28 |
| marios | thanks sshnaidm sshnaidm :) good doing business with you | 12:28 |
| marios | err ssbarnea|bkp2 ^ | 12:28 |
| *** ykarel is now known as ykarel|afk | 12:30 | |
| *** rlandy has joined #oooq | 12:42 | |
| *** holser_ is now known as holser|lunch | 12:43 | |
| *** bogdando_ has joined #oooq | 12:49 | |
| chandankumar | kopecmartin: arxcruz is it ok if I send the os_tempest report tomorrow instead of wednesday? | 12:51 |
| *** bogdando has quit IRC | 12:51 | |
| *** bogdando_ is now known as bogdando | 12:51 | |
| arxcruz | chandankumar: no big deal for me but u need to check with weshay|ruck | 12:53 |
| chandankumar | arxcruz: sure :-) | 12:53 |
| *** ykarel|afk is now known as ykarel | 13:15 | |
| rfolco | panda, do you have a min ? want to make sure its safe to test my script locally with push=false to test container build on fedora | 13:23 |
| rfolco | panda, if you have 5 min to tmate I appreciate | 13:23 |
| panda | rfolco: I have 5 minutmate | 13:24 |
| rfolco | panda, cool, pls jump in my blue and I'll give the tmate hash | 13:25 |
| *** trown|outtypewww is now known as trown | 13:27 | |
| weshay|ruck | chandankumar, arxcruz fine by me | 13:27 |
| chandankumar | weshay|ruck: will send tomorrow then , Thanks :-) | 13:31 |
| *** ykarel is now known as ykarel|away | 13:32 | |
| *** udesale has joined #oooq | 13:36 | |
| *** skramaja has quit IRC | 13:40 | |
| *** ykarel|away has quit IRC | 13:47 | |
| weshay|ruck | panda, rfolco ssbarnea|bkp2 marios update taiga tasks please | 13:58 |
| ssbarnea|bkp2 | BJ gives me "An unexpected error occurred. Please try again later." | 14:00 |
| *** holser|lunch is now known as holser_ | 14:01 | |
| ssbarnea|bkp2 | found it: when OS has proxy enabled, it breaks BJ. | 14:01 |
| rfolco | panda, joining scrum? | 14:04 |
| weshay|ruck | sshnaidm, ^ | 14:04 |
| panda | I have nothing on my calendar | 14:05 |
| chandankumar | kopecmartin: https://tree.taiga.io/project/tripleo-ci-board/us/230 feel free to qe it and it is done from my side | 14:08 |
| kopecmartin | chandankumar, ok | 14:09 |
| chandankumar | kopecmartin: last patch is also workflowed, we just need to QE it | 14:09 |
| chandankumar | by the A big thanks kopecmartin and arxcruz on completing/helping python-tempestconf with os_tempest big chunk is done :-) | 14:10 |
| chandankumar | kopecmartin++ | 14:10 |
| hubbot1 | chandankumar: kopecmartin's karma is now 4 | 14:10 |
| weshay|ruck | https://review.rdoproject.org/r/#/c/17719/ | 14:10 |
| chandankumar | arxcruz++ | 14:10 |
| hubbot1 | chandankumar: arxcruz's karma is now 11 | 14:10 |
| weshay|ruck | panda, , https://review.rdoproject.org/r/#/c/17780/6/ci-scripts/dlrnapi_promoter/config/CentOS-7/master.ini | 14:10 |
| *** holser_ has quit IRC | 14:11 | |
| *** holser_ has joined #oooq | 14:11 | |
| *** ykarel|away has joined #oooq | 14:12 | |
| rlandy | panda: you may want to re-enable fs020 on master https://review.rdoproject.org/r/#/c/17780/7/ci-scripts/dlrnapi_promoter/config/CentOS-7/master.ini | 14:14 |
| *** irclogbot_0 has quit IRC | 14:14 | |
| panda | rlandy: done | 14:16 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 14:17 |
| bogdando | rlandy: hi. So the deployment #2 does not look overlapping to mariadb issues | 14:26 |
| bogdando | and I can see nothing weird in logs around the moment X | 14:26 |
| rlandy | bogdando: thanks for looking into that bug | 14:26 |
| bogdando | I like digging logs | 14:26 |
| rlandy | bogdando: well, this one will be fun - I can try set up a reproducer if that will help | 14:27 |
| bogdando | yes, we can try | 14:27 |
| rlandy | rocky does fine with the same workflow | 14:27 |
| rlandy | so something is up with the re-install of the undercloud with master | 14:27 |
| bogdando | not sure, as I said, reinstall happens after the DB issues start | 14:28 |
| ssbarnea|bkp2 | anyone from our team with wf perm on rdo-jobs? for https://review.rdoproject.org/r/#/c/17732/ | 14:29 |
| rlandy | ssbarnea|bkp2: I do - looking | 14:30 |
| rlandy | done | 14:31 |
| *** irclogbot_0 has joined #oooq | 14:39 | |
| *** ykarel|away is now known as ykarel | 14:45 | |
| weshay|ruck | rlandy, I think workflow on https://review.rdoproject.org/r/#/c/17744 | 14:50 |
| rlandy | panda: ^^ ready for that to merge? | 14:51 |
| rlandy | bogdando: so I see the same aborted connections on the fs001 test job log: https://logs.rdoproject.org/71/18071/2/check/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset001-master/3e94eba/logs/undercloud/var/log/containers/mysql/mariadb.log.txt.gz#_2019-01-06_19_45_53 which looks to be at the time of the reinstall https://logs.rdoproject.org/71/18071/2/check/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset001-master/ | 14:56 |
| rlandy | 3e94eba/logs/undercloud/home/zuul/undercloud_reinstall.log.txt.gz | 14:56 |
| * rlandy compares with passing job | 14:56 | |
| *** rlandy is now known as rlandy|rover | 14:56 | |
| rlandy|rover | same here on passing job ... http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/4a9c3ca/logs/undercloud/var/log/containers/mysql/mariadb.log.txt.gz | 14:58 |
| sshnaidm | rlandy|rover, talk to me if you need to reproduce ovb with new zuul container stuff | 14:58 |
| rlandy|rover | sshnaidm: I'd like to try it when I get off rover shift | 14:58 |
| rlandy|rover | so wednesday | 14:59 |
| sshnaidm | rlandy|rover, ack | 14:59 |
| bogdando | rlandy|rover: oh well then that was a red herring message and real problems could have started later, then overlap happens | 15:04 |
| rlandy|rover | bogdando: there is something else going on - setting up reproducer so we can investigate more | 15:06 |
| *** irclogbot_0 has quit IRC | 15:06 | |
| panda | rlandy|rover: doing final check | 15:08 |
| rlandy|rover | panda: ping me when ready for merge | 15:14 |
| *** irclogbot_0 has joined #oooq | 15:14 | |
| bogdando | rlandy|rover: so we were apparently chasing the wrong rabbit | 15:15 |
| bogdando | the right rabbit is...well, rabbit | 15:15 |
| bogdando | why I'm not surprised | 15:15 |
| rlandy|rover | bogdando: :) ... looking | 15:16 |
| panda | rlandy|rover: I found nothing else, we can merge | 15:21 |
| rlandy|rover | k | 15:22 |
| * panda ducks | 15:22 | |
| rlandy|rover | here goes ... | 15:22 |
| weshay|ruck | rfolco, panda | 15:31 |
| rlandy|rover | weshay|ruck: worth investigating the current failures on master promotion fs030 and fs035? | 15:35 |
| bogdando | rlandy|rover: that recent message about failed futures and rabbits is 20 min later than the 2nd undercloud deployment finishes :) | 15:40 |
| bogdando | not overlapping again :/ | 15:40 |
| bogdando | the 2nd deploy ends at 2019-01-05 12:14:51.197 | 15:41 |
| rlandy|rover | hmmm -ok - working on setting up test env | 15:43 |
| bogdando | rlandy|rover: I gtg, it's late here :) you can try to reach for oslo/rabbit guys maybe | 15:43 |
| rlandy|rover | bogdando: k - will do - thanks for your help here | 15:43 |
| bogdando | or some pdb&guru-meditaion 80lvl gurus | 15:43 |
| rlandy|rover | hmmm ... another reproducer failure ... sudo pip install gear | 15:44 |
| rlandy|rover | weshay|ruck: ^^ didn't you fix this some time back | 15:44 |
| rlandy|rover | echo 'Running with te-broker' | 15:44 |
| rlandy|rover | thought we got rid of that | 15:44 |
| *** saneax has quit IRC | 15:45 | |
| weshay|ruck | sorry chatting w/ panda and rfolco | 15:45 |
| rlandy|rover | ImportError: No module named docutils.core | 15:46 |
| bogdando | rlandy|rover: asked pidone folks to join you, perchance | 15:47 |
| rlandy|rover | bogdando: in what sense? | 15:47 |
| rlandy|rover | bogdando: in ci work in general? this bug in particular? | 15:48 |
| ykarel | rlandy|rover, for fs030 master promotion i filed https://bugs.launchpad.net/tripleo/+bug/1810706 | 15:51 |
| openstack | Launchpad bug 1810706 in tripleo "[promotion job][master][FS030][OVN] Tempest test failing with No IPv4 addresses found in: <port dict> " [Undecided,New] | 15:51 |
| *** bogdando has quit IRC | 15:51 | |
| ykarel | u can add if i missed some bits | 15:52 |
| rlandy|rover | ykarel: thanks | 15:52 |
| rlandy|rover | sshnaidm: is this line 'sudo pip install gear' even needed anymore? | 15:54 |
| rlandy|rover | giving errors on old reproducer | 15:54 |
| sshnaidm | rlandy|rover, no needed | 15:54 |
| rlandy|rover | ImportError: No module named docutils.core | 15:54 |
| rlandy|rover | sshnaidm: ok - will put in a patch to remove | 15:54 |
| sshnaidm | rlandy|rover, then we need to remove all te-broker section | 15:55 |
| rlandy|rover | ack | 15:55 |
| *** kopecmartin is now known as kopecmartin|off | 16:13 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 16:17 |
| * rlandy|rover wants to kill old reproducer | 16:20 | |
| *** zul has joined #oooq | 16:20 | |
| * panda hands a kris to rlandy|rover | 16:31 | |
| rlandy|rover | impossible to work with now :( | 16:39 |
| *** udesale has quit IRC | 16:40 | |
| marios | have a good day folks ttyl bai | 16:54 |
| marios | rfolco: mail me re the scen2/3 i will have at it tomorrow morning | 16:54 |
| marios | with ssbarnea|bkp2 | 16:54 |
| rfolco | marios, don't worry about it, focus on 1/4 :) | 16:57 |
| rfolco | marios, thanks for your help | 16:57 |
| marios | rfolco: ack ok i'll review anyway then & np | 16:57 |
| *** rfolco has quit IRC | 16:58 | |
| *** rfolco has joined #oooq | 16:59 | |
| *** zul has quit IRC | 17:07 | |
| weshay|ruck | rlandy|rover, see anything like this lately? http://logs.openstack.org/97/626597/2/gate/tripleo-ci-centos-7-undercloud-oooq/75c36a0/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2019-01-07_15_31_43 | 17:12 |
| * weshay|ruck trying to track it down | 17:12 | |
| rlandy|rover | weshay|ruck: not in the errors I have investigated | 17:12 |
| weshay|ruck | k.. thanks | 17:13 |
| weshay|ruck | "explanation": "The resource could not be found.", "code": 404, "error": {"message": "The Stack (overcloud) could not be found.", "traceback": | 17:14 |
| rlandy|rover | weshay|ruck: who is our default contact for neutron bugs? https://bugs.launchpad.net/tripleo/+bug/1810706 | 17:24 |
| openstack | Launchpad bug 1810706 in tripleo "[promotion job][master][FS030][OVN] Tempest test failing with No IPv4 addresses found in: <port dict> " [Undecided,New] | 17:24 |
| weshay|ruck | rlandy|rover, ya.. I was looking for that on the CIX board earlier | 17:25 |
| weshay|ruck | sec | 17:25 |
| *** trown is now known as trown|lunch | 17:26 | |
| weshay|ruck | rlandy|rover, I'm going to say Daniel Alvarez | 17:30 |
| *** rf0lc0 has joined #oooq | 17:30 | |
| weshay|ruck | rlandy|rover, oh.. it wasn't escalated because the status importance | 17:31 |
| *** rf0lc0 has quit IRC | 17:31 | |
| weshay|ruck | rlandy|rover, ykarel :) | 17:31 |
| weshay|ruck | fixing | 17:31 |
| chandankumar | rlandy|rover: slaweq is the best guy for neutron bugs :-) | 17:31 |
| weshay|ruck | ok.. now it will be escalated | 17:32 |
| *** rfolco has quit IRC | 17:33 | |
| rlandy|rover | thanks | 17:33 |
| weshay|ruck | rlandy|rover, I added a card | 17:34 |
| rlandy|rover | weshay|ruck: thanks - saw that | 17:34 |
| rlandy|rover | will tracking - adding to ruck/river etherpad | 17:34 |
| *** ykarel has quit IRC | 17:39 | |
| *** rfolco has joined #oooq | 17:42 | |
| *** dtantsur is now known as dtantsur|afk | 17:45 | |
| weshay|ruck | rfolco, panda I've updated https://tree.taiga.io/project/tripleo-ci-board/epic/445 please review my changes to the epic and user stories | 17:46 |
| panda | weshay|ruck: will do | 17:55 |
| weshay|ruck | panda, there are now two ways to build containers | 17:58 |
| weshay|ruck | perhaps we can consolidate | 17:59 |
| *** ykarel has joined #oooq | 18:00 | |
| *** derekh has quit IRC | 18:00 | |
| panda | weshay|ruck: I don't know the other way | 18:01 |
| weshay|ruck | panda, read the epic :) | 18:02 |
| *** ykarel has quit IRC | 18:06 | |
| *** agopi has quit IRC | 18:14 | |
| *** agopi has joined #oooq | 18:15 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades @ (1 more message) | 18:17 |
| *** agopi has quit IRC | 18:18 | |
| *** agopi has joined #oooq | 18:19 | |
| *** trown|lunch is now known as trown | 18:20 | |
| *** agopi_ has joined #oooq | 18:26 | |
| *** agopi has quit IRC | 18:29 | |
| rlandy|rover | weshay|ruck; have you ever used the old reproducer with f28? | 18:54 |
| weshay|ruck | rlandy|rover, long time ago | 18:54 |
| weshay|ruck | with a lot of changes | 18:54 |
| rlandy|rover | weshay|ruck: afaict, you would need to change https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/multinodes/tasks/main.yml#L45 | 18:54 |
| rlandy|rover | to use a fedora image | 18:55 |
| weshay|ruck | rlandy|rover, ya.. I wouldn't even bother at this point tbh | 18:55 |
| weshay|ruck | rlandy|rover, what are you trying to do? | 18:55 |
| weshay|ruck | not that I've had any luck w/ the new repro yet | 18:55 |
| rlandy|rover | weshay|ruck: just chatting with rfolco about setting up to test building containers in his own tenant | 18:56 |
| rlandy|rover | before trying anything that may push | 18:56 |
| rfolco | yeah having a f28 undercloud isn't possible with current repro | 18:57 |
| weshay|ruck | rfolco, STOP | 18:58 |
| weshay|ruck | do NOT use the reproducer | 18:58 |
| rfolco | weshay|ruck, manual tripleo deploy ? | 18:59 |
| weshay|ruck | that's what we said right? | 18:59 |
| weshay|ruck | just install the undercloud | 18:59 |
| weshay|ruck | then run the script | 18:59 |
| rfolco | weshay|ruck, ok, that requires additional steps... like creating a network etc for my tenant... | 19:00 |
| rfolco | I got stuck there | 19:00 |
| weshay|ruck | you got stuck creating a network in your tenant? | 19:01 |
| rfolco | yes | 19:01 |
| rfolco | not that simple | 19:01 |
| weshay|ruck | help me understand | 19:01 |
| rfolco | weshay|ruck, can you bj and I show you? | 19:02 |
| weshay|ruck | k.. going in | 19:03 |
| rfolco | https://bluejeans.com/5878458097 weshay|ruck | 19:04 |
| *** tosky has quit IRC | 19:05 | |
| *** holser_ is now known as holser|eod | 19:08 | |
| *** holser|eod has quit IRC | 19:09 | |
| weshay|ruck | rfolco, https://github.com/openstack-infra/tripleo-ci/tree/master/playbooks/tripleo-buildcontainers | 19:13 |
| rlandy|rover | omg - ovb reproducer == disaster | 19:23 |
| weshay|ruck | rlandy|rover, heh | 19:26 |
| weshay|ruck | rfolco, http://logs.openstack.org/84/628984/1/check/tripleo-build-containers-centos-7/92263a3/ | 19:26 |
| weshay|ruck | rfolco, https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/b64bbb2/job-output.txt.gz | 19:27 |
| weshay|ruck | rfolco, https://review.openstack.org/#/c/624532/ | 19:30 |
| weshay|ruck | rlandy|rover, you want to chat about the repro? | 19:31 |
| weshay|ruck | I could use a second opinion there | 19:31 |
| rlandy|rover | weshay|ruck: sure | 19:33 |
| rlandy|rover | flogging the sam horse here | 19:33 |
| weshay|ruck | oh man | 19:33 |
| weshay|ruck | my blue | 19:33 |
| *** agopi__ has joined #oooq | 19:34 | |
| rlandy|rover | joining | 19:34 |
| *** agopi_ has quit IRC | 19:36 | |
| rlandy|rover | weshay|ruck, https://bugs.launchpad.net/openstack-gate/+bug/1721093 | 19:53 |
| openstack | Launchpad bug 1721093 in OpenStack-Gate "Zuul v3 tasks can end up in an UNREACHABLE state" [Undecided,Confirmed] | 19:53 |
| weshay|ruck | rlandy|rover, https://tree.taiga.io/project/tripleo-ci-board/epic/558 | 19:59 |
| *** agopi__ has quit IRC | 20:09 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000 (1 more message) | 20:17 |
| rlandy|rover | weshay|ruck: https://review.openstack.org/#/c/626758/ worked for me - will w+ it - can I remove the WIP? | 20:24 |
| weshay|ruck | sure | 20:26 |
| weshay|ruck | rlandy|rover, updated | 20:26 |
| rlandy|rover | thanks +2'ed | 20:27 |
| rlandy|rover | sometimes the simplicity of the old reproducer does seems attractive | 20:27 |
| *** trown is now known as trown|outtypewww | 22:01 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000 (1 more message) | 22:17 |
| *** tosky has joined #oooq | 22:52 | |
| *** rascasoft has quit IRC | 23:10 | |
| weshay|ruck | rlandy|rover, ya.. I wouldn't be suprised at all if podman some how broke the idempotency | 23:16 |
| weshay|ruck | rlandy|rover, just proves the point that more robust testing of dependencies needs to happen | 23:16 |
| -openstackstatus- NOTICE: The Etherpad service at https://etherpad.openstack.org/ has been offline since 23:22 UTC due to a hypervisor issue in our service provider, but should hopefully return to service shortly. | 23:48 | |
| *** tosky has quit IRC | 23:48 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!