| rlandy | weshay: why remove export USER_PYTHON_OVERRIDE=python2 | 00:01 |
|---|---|---|
| rlandy | export USER_OVERRIDE_SUDO_CHECK=1 | 00:01 |
| *** tosky has quit IRC | 00:18 | |
| *** rlandy is now known as rlandy|bbl | 00:36 | |
| *** rascasoft has joined #oooq | 00:38 | |
| *** rascasoft has quit IRC | 00:45 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, stable/ocata: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ https://review.openstack.org/564291 | 00:55 |
| *** saneax has joined #oooq | 02:50 | |
| *** saneax has quit IRC | 02:52 | |
| *** saneax has joined #oooq | 02:53 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, stable/rocky: (1 more message) | 02:55 |
| *** rlandy|bbl is now known as rlandy | 03:12 | |
| *** rlandy has quit IRC | 03:19 | |
| *** ykarel has joined #oooq | 03:26 | |
| *** udesale has joined #oooq | 04:02 | |
| *** vinaykns has quit IRC | 04:07 | |
| *** weshay has quit IRC | 04:34 | |
| *** saneax has quit IRC | 04:42 | |
| *** ykarel has quit IRC | 04:47 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ (1 more message) | 04:55 |
| *** ykarel has joined #oooq | 05:05 | |
| *** ratailor has joined #oooq | 05:08 | |
| *** udesale has quit IRC | 05:37 | |
| *** chandankumar is now known as chkumar|ruck | 05:47 | |
| *** skramaja has joined #oooq | 06:05 | |
| *** sshnaidm is now known as sshnaidm|afk | 06:39 | |
| *** saneax has joined #oooq | 06:45 | |
| *** ccamacho has quit IRC | 06:49 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ (1 more message) | 06:55 |
| *** quiquell|off is now known as quiquell|rover | 07:02 | |
| *** kopecmartin|off is now known as kopecmartin | 07:05 | |
| *** udesale has joined #oooq | 07:09 | |
| *** jfrancoa has joined #oooq | 07:21 | |
| *** holser_ has joined #oooq | 07:31 | |
| *** ykarel is now known as ykarel|lunch | 07:33 | |
| chkumar|ruck | quiquell|rover: once of the tempest tests failed https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/3993256/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-02-12_02_33_19 | 07:34 |
| chkumar|ruck | on fs020 rocky | 07:34 |
| chkumar|ruck | quiquell|rover: I am trying to grep in the logs what happened | 07:34 |
| chkumar|ruck | no trace | 07:34 |
| chkumar|ruck | quiquell|rover: 475ade7e-a45a-4986-b605-4f3fe40bd0f3 img id used | 07:35 |
| chkumar|ruck | leading to request timedout | 07:35 |
| ykarel|lunch | chkumar|ruck, there is already bug for it, try to find it | 07:35 |
| ykarel|lunch | it's a race issue | 07:35 |
| ykarel|lunch | iirc i filed it | 07:35 |
| ykarel|lunch | and added the finding | 07:35 |
| chkumar|ruck | ykarel|lunch: finding | 07:35 |
| ykarel|lunch | if can't find, i will find post lunch | 07:36 |
| chkumar|ruck | ykarel|lunch: is this one https://bugs.launchpad.net/tripleo/+bug/1744907 | 07:38 |
| openstack | Launchpad bug 1744907 in tripleo "Tempest test: "test_create_second_image_when_first_image_is_being_saved" failing in featureset020 periodic job" [High,Triaged] | 07:38 |
| chkumar|ruck | ykarel|lunch: ok | 07:39 |
| *** ykarel|lunch has quit IRC | 07:41 | |
| *** ykarel has joined #oooq | 07:43 | |
| *** quiquell|rover is now known as quique|rover|brb | 07:53 | |
| *** ccamacho has joined #oooq | 07:58 | |
| *** ccamacho has quit IRC | 07:59 | |
| *** ccamacho has joined #oooq | 07:59 | |
| *** rascasoft has joined #oooq | 08:02 | |
| *** gkadam has joined #oooq | 08:08 | |
| *** quique|rover|brb is now known as quiquell|rover | 08:12 | |
| *** gkadam_ has joined #oooq | 08:15 | |
| *** gkadam has quit IRC | 08:17 | |
| *** tosky has joined #oooq | 08:40 | |
| ykarel | chkumar|ruck, you find fs020 bug? | 08:41 |
| ykarel | glance one | 08:41 |
| chkumar|ruck | ykarel: nope | 08:41 |
| ykarel | image save deleting one | 08:41 |
| ykarel | chkumar|ruck, ok /me finding | 08:41 |
| chkumar|ruck | ykarel: yes, but it is not related to the above failure | 08:41 |
| chkumar|ruck | ykarel: I have retriggered the fs020 rocky in test project | 08:42 |
| ykarel | which above failure | 08:42 |
| chkumar|ruck | ykarel: current fs020 rocky tempest failure is not related to this bug https://bugs.launchpad.net/tripleo/+bug/1744907 | 08:42 |
| openstack | Launchpad bug 1744907 in tripleo "Tempest test: "test_create_second_image_when_first_image_is_being_saved" failing in featureset020 periodic job" [High,Triaged] | 08:42 |
| ykarel | chkumar|ruck, the one i was refering is differnt https://bugs.launchpad.net/tripleo/+bug/1744907 | 08:42 |
| chkumar|ruck | ok | 08:42 |
| ykarel | chkumar|ruck, but symptoms look saving | 08:43 |
| ykarel | s/saving/same | 08:43 |
| ykarel | looks like image quickly went to saving state | 08:44 |
| ykarel | befor actual checked happened | 08:44 |
| chkumar|ruck | ykarel: yes | 08:45 |
| chkumar|ruck | ykarel: sorry the image was in active state but failed to reach in saving state | 08:45 |
| chkumar|ruck | leading to timedout | 08:46 |
| ykarel | yup it went too quickly to active | 08:46 |
| ykarel | good to check glance logs | 08:46 |
| ykarel | iirc i added comments on the bug, so similar thing needs to be checked in this case as well | 08:46 |
| *** amoralej|off is now known as amoralej | 08:47 | |
| *** jpena|off is now known as jpena | 08:53 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ (1 more message) | 08:55 |
| *** bogdando has joined #oooq | 09:03 | |
| *** panda|off is now known as panda | 09:04 | |
| panda | marios: when's a good time for you to instill your knowledge upon me ? | 09:15 |
| *** ykarel has quit IRC | 09:20 | |
| marios | panda: in 10 mins we can have a call? | 09:22 |
| panda | marios: we'll cut at 14 minutes. | 09:23 |
| marios | panda: ok then lets make it in 40 mins at the start of the hour, 1000 UTC | 09:24 |
| marios | panda: er 36 mins | 09:24 |
| panda | marios: we'll cut at 14 minutes anyway, I don't want you to get sick. | 09:24 |
| marios | panda: oh i thought you meant in 14 mins | 09:24 |
| marios | k lets talk now | 09:24 |
| marios | panda: ? | 09:24 |
| panda | marios: I'm already in your channel | 09:24 |
| marios | https://redhat.bluejeans.com/7661925373/ | 09:24 |
| marios | panda: joining | 09:24 |
| *** ykarel has joined #oooq | 09:27 | |
| *** dtantsur has joined #oooq | 09:28 | |
| *** skramaja has quit IRC | 09:41 | |
| *** skramaja has joined #oooq | 09:41 | |
| chkumar|ruck | quiquell|rover: marios sshnaidm|afk panda arxcruz kopecmartin I will be on PTO on 13th and 14th feb, need to attend one of my friend's marriage | 09:43 |
| arxcruz | chkumar|ruck: you are attending too many weddings, wondering when I will attend yours :P | 09:43 |
| quiquell|rover | arxcruz: implicit stuff there :-) | 09:44 |
| arxcruz | lol | 09:44 |
| chkumar|ruck | arxcruz: time will come soon might be in 2-3 years, WIP currently :-) | 09:44 |
| arxcruz | quiquell|rover: he's training, collecting information | 09:44 |
| quiquell|rover | wip lover | 09:46 |
| chkumar|ruck | quiquell|rover: one more help here http://logs.openstack.org/04/634904/16/check/openstack-tox-linters/48177ee/job-output.txt.gz#_2019-02-12_09_12_08_463656 linters is failing here | 09:46 |
| chkumar|ruck | quiquell|rover: https://review.openstack.org/#/c/634904/ | 09:46 |
| quiquell|rover | chkumar|ruck: 306 Shells that use pipes should set the pipefail option | 09:47 |
| quiquell|rover | use pipefail there | 09:47 |
| chkumar|ruck | ok got it | 09:48 |
| chkumar|ruck | need to use set -o pipefail it hink | 09:48 |
| chkumar|ruck | I think | 09:48 |
| quiquell|rover | yep | 09:48 |
| *** sshnaidm|afk is now known as sshnaidm | 09:52 | |
| marios | panda: https://github.com/openstack/tripleo-heat-templates/blob/23302ca1ce276830aed19c20a9ffa0a8cef9cbb4/common/deploy-steps.j2#L309# | 09:52 |
| *** derekh has joined #oooq | 10:27 | |
| *** jtomasek has quit IRC | 10:46 | |
| *** udesale has quit IRC | 10:49 | |
| *** jtomasek has joined #oooq | 10:55 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, stable/rocky: (1 more message) | 10:55 |
| panda | rfolco: ping | 11:26 |
| panda | marios: is this the right way to pass parameters ? https://review.openstack.org/#/c/634723/9/ci/common/vbmc_setup_standalone.yaml@10 | 11:27 |
| panda | marios: do I need to add also "action" or "name" ? | 11:27 |
| marios | panda: commented see the example later in file | 11:32 |
| marios | panda: needs to go into properties | 11:32 |
| panda | mmhhh | 11:32 |
| panda | marios: but only server is needed ? | 11:35 |
| panda | marios: "servers" | 11:35 |
| panda | well, let's hope it's enough | 11:36 |
| marios | servers should be panda | 11:36 |
| panda | ok | 11:36 |
| panda | properties: servers: "panda" | 11:36 |
| panda | marios: got it. Thanks. | 11:37 |
| marios | :) | 11:37 |
| *** holser_ is now known as holser|lunch | 11:42 | |
| rfolco | panda, o/ | 11:49 |
| *** amoralej is now known as amoralej|lunch | 12:00 | |
| panda | rfolco: commented on your pull request | 12:03 |
| rfolco | panda, where is the gerrithub link? | 12:10 |
| panda | rfolco: in .gitreview | 12:10 |
| rfolco | panda, thx | 12:11 |
| panda | rfolco: git review -s should setup everything | 12:11 |
| panda | rfolco: but you have to sign in to gerrithub and allow it to connect to github accnount | 12:11 |
| rfolco | yep I don't recall using it | 12:12 |
| chkumar|ruck | quiquell|rover: for fedora the output is totally different http://logs.openstack.org/04/634904/16/check/tripleo-ci-fedora-28-standalone/95a7781/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 12:23 |
| chkumar|ruck | quiquell|rover: is it not enabled for f28? | 12:23 |
| chkumar|ruck | i mean tripleo image prepare | 12:23 |
| *** jpena is now known as jpena|lunch | 12:26 | |
| quiquell|rover | chkumar|ruck: this is doing nothing | 12:27 |
| chkumar|ruck | quiquell|rover: we need to enable there in order to test my changes here | 12:27 |
| quiquell|rover | chkumar|ruck: wait... I remember changing the script to work with py3 :-/ | 12:27 |
| quiquell|rover | chkumar|ruck: to fix f28 job | 12:27 |
| quiquell|rover | chkumar|ruck: let me remember what I did to fix it | 12:28 |
| chkumar|ruck | quiquell|rover: ok | 12:28 |
| quiquell|rover | chkumar|ruck: https://review.openstack.org/#/c/624357 | 12:30 |
| quiquell|rover | chkumar|ruck: It was to make it work for standalone sorry | 12:30 |
| quiquell|rover | chkumar|ruck: But I remember changing stuff for python3 in the script | 12:30 |
| quiquell|rover | chkumar|ruck: so look like something is broken and again is not updating the contaners with the chang e | 12:31 |
| quiquell|rover | :-/ | 12:31 |
| chkumar|ruck | quiquell|rover: let me dig that and get back to you | 12:32 |
| quiquell|rover | chkumar|ruck: look at the Depends-On too | 12:32 |
| quiquell|rover | chkumar|ruck: yep I remember !!! | 12:32 |
| *** panda is now known as panda|lunch | 12:32 | |
| quiquell|rover | chkumar|ruck: in the tripleo-container-image-prepare.py | 12:32 |
| quiquell|rover | chkumar|ruck: I make it use ansible-playbook-2 or ansible-playbook-3 depending on the environment | 12:33 |
| chkumar|ruck | quiquell|rover: chekcing | 12:33 |
| *** ratailor has quit IRC | 12:34 | |
| chkumar|ruck | quiquell|rover: ykarel by the way fs020 rocky is working fine http://logs.rdoproject.org/43/13943/80/check/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/3431096/ | 12:36 |
| chkumar|ruck | it might be a race condition or unrelated timeout in the morning | 12:36 |
| chkumar|ruck | it is from recent testproject run | 12:36 |
| ykarel | chkumar|ruck, yup it's intermittent issue | 12:39 |
| ykarel | chkumar|ruck, did you checked logs? | 12:39 |
| ykarel | in failing ones like the similar bug i pointed | 12:40 |
| ykarel | good to know if it's same race | 12:40 |
| chkumar|ruck | ykarel: I tried to grep through image id there is nothing there in glance api the request response are fine | 12:40 |
| ykarel | chkumar|ruck, i think you need to check nova api as well, | 12:41 |
| ykarel | it's image create from server | 12:41 |
| ykarel | you need to check, how much time it took to move to saving --> active | 12:41 |
| chkumar|ruck | ykarel: ok on it | 12:45 |
| *** holser|lunch is now known as holser_ | 12:47 | |
| *** ykarel is now known as ykarel|away | 12:48 | |
| *** ykarel|away has quit IRC | 12:54 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/560445, stable/queens: tripleo-ci- (1 more message) | 12:55 |
| *** weshay has joined #oooq | 12:57 | |
| zbr | quiquell|rover: any idea what could have caused https://gist.github.com/ssbarnea/10935fc1a3c5c8087b037c7a58dcba19 ? | 12:59 |
| weshay | morning.. quiquell|rover sorry I did not get that mariadb stuff yesterday | 13:00 |
| quiquell|rover | weshay: no problem have just merged it I needed for the rr | 13:01 |
| quiquell|rover | weshay: commented at your docker group review | 13:01 |
| weshay | ah k | 13:01 |
| * weshay looks | 13:01 | |
| quiquell|rover | zbr: this one issue was driving me crazy with libvirt, let me remember how did I fix it | 13:02 |
| quiquell|rover | zbr: Yes I remember | 13:02 |
| quiquell|rover | zbr: How are you running stuff with the playbook ? | 13:02 |
| zbr | quiquell|rover: root/libvirt/reproducer.sh | 13:05 |
| quiquell|rover | zbr: the issue is here https://github.com/openstack/tripleo-quickstart/blob/master/playbooks/libvirt-nodepool.yml#L22-L26 | 13:05 |
| quiquell|rover | This is all wrong | 13:05 |
| quiquell|rover | supermin does not like empty env values | 13:05 |
| weshay | quiquell|rover, sshnaidm so.. success w/ libvirt on f28 and centos-7 now w/ a few of those patches | 13:05 |
| quiquell|rover | zbr: at the new reproducer we don't do that and that fixed the issue | 13:06 |
| quiquell|rover | zbr: remove the environment section there and you are all good | 13:06 |
| sshnaidm | quiquell|rover, zbr we need to fix that finally, and set env vars only if it's configured to set them | 13:06 |
| quiquell|rover | sshnaidm: this playbook is only used by old reproducer (I think) | 13:07 |
| sshnaidm | quiquell|rover, still, most of people use it now | 13:08 |
| zbr | sshnaidm: ok, glad to hear we agree on a direction. still, I wanted to use old repo to investigate some build errors. and againt, weare back at code that we don't test in ci and gets borkend. | 13:08 |
| quiquell|rover | zbr: ok so quick fix is that, maybe you can put a review to make env vars optional and not empty | 13:09 |
| quiquell|rover | zbr: so we fix the playbook | 13:09 |
| zbr | sshnaidm: do you remember the recipe for defining env vars only when they are needed? using a dic key, right? | 13:09 |
| quiquell|rover | zbr: it was not easy | 13:09 |
| sshnaidm | zbr, dic key..? | 13:09 |
| quiquell|rover | we have the ugly solution of template source file | 13:09 |
| zbr | quiquell|rover: environmebt: env_dic .... and setting keys with set_fact | 13:10 |
| quiquell|rover | and the nice convolutes solution of defaults and jinja | 13:10 |
| *** udesale has joined #oooq | 13:10 | |
| quiquell|rover | zbr: yep set_fact can work | 13:10 |
| chkumar|ruck | quiquell|rover: weshay need to leave have to catch flight see ya on friday, | 13:10 |
| weshay | k | 13:10 |
| quiquell|rover | chkumar|ruck: enjoy don't get too dr... | 13:10 |
| weshay | safe travels | 13:10 |
| *** chkumar|ruck is now known as chkumar|pto | 13:10 | |
| weshay | oh he will | 13:10 |
| chkumar|pto | weshay: quiquell|rover thanks :-) | 13:11 |
| zbr | i do not particulary like having to loop-set-fact but if nobody has a better idea... | 13:11 |
| *** trown|outtypewww is now known as trown | 13:11 | |
| quiquell|rover | zbr: Let me look in the interwebs | 13:12 |
| sshnaidm | zbr, no need, just "when: myenv is defined" | 13:12 |
| quiquell|rover | zbr: the issue whas that env vars does not work with "omit" I think | 13:12 |
| sshnaidm | and the same task without it | 13:12 |
| sshnaidm | quiquell|rover, yeah, they don't | 13:13 |
| *** panda|lunch is now known as panda | 13:13 | |
| sshnaidm | quiquell|rover, zbr from all issues in ansible I saw, there is no nice or supported way to do it | 13:13 |
| zbr | quiquell|rover: sshnaidm : nevermind, making a fix to it now and adding you for review. | 13:14 |
| quiquell|rover | ack | 13:14 |
| quiquell|rover | zbr: at least you can fix your repro manually man | 13:14 |
| zbr | sshnaidm: regarding ansible-issues... yes, but you want to go back to puppet? ;) | 13:14 |
| sshnaidm | zbr, no, but I'd like "omit" would work for env vars :D | 13:14 |
| zbr | make a patch, i did few in the past, it was much easier to get them merged than in openstack. | 13:15 |
| weshay | sshnaidm, when you have a moment https://review.openstack.org/#/c/636230/ | 13:16 |
| *** quiquell|rover is now known as quique|rover|eat | 13:17 | |
| *** rlandy has joined #oooq | 13:17 | |
| *** agopi has joined #oooq | 13:26 | |
| *** amoralej|lunch is now known as amoralej | 13:28 | |
| *** jpena|lunch is now known as jpena | 13:30 | |
| rlandy | marios: hi - need to borrow your standalone expertise | 13:31 |
| rlandy | marios: pacemaker | 13:32 |
| *** agopi has quit IRC | 13:34 | |
| marios | rlandy: o/ hi. what about it /me opens the env files | 13:37 |
| *** quique|rover|eat is now known as quiquell|rover | 13:39 | |
| panda | mmhh two hours for the scenario12 to finish | 13:42 |
| *** ykarel|away has joined #oooq | 13:43 | |
| marios | rlandy: there? i see we have it in all four scenarios but do you think we should remove it? we are also pointing the services to the clusterized versions like https://github.com/openstack/tripleo-heat-templates/blob/9b073e6b762de4557e02361ccc6bef89e24b4279/ci/environments/scenario001-standalone.yaml#L26-L30 | 13:46 |
| marios | rlandy: looks like the default standalone env doesn't carry pacemaker at all | 13:46 |
| rlandy | marios: idk about other scenarios | 13:46 |
| weshay | anyone have a sec.. for a review.. only a few lines https://review.openstack.org/#/c/636230/ | 13:46 |
| marios | rlandy: i mean at https://github.com/openstack/tripleo-heat-templates/blob/master/environments/standalone/standalone-overcloud.yaml | 13:47 |
| rlandy | marios: I am not sure what to do about the one I am working on | 13:47 |
| marios | rlandy: if pacemaker is a problem for you don't deploy it. we may end up removing from the rest too | 13:47 |
| rlandy | marios: it looked like it needed pacemaker | 13:47 |
| rlandy | but I am not sure why | 13:47 |
| rlandy | now I am sitting with this error: | 13:47 |
| weshay | rlandy, I don't think your error was related to pace | 13:48 |
| weshay | but I would certainly remove services that are not needed | 13:48 |
| rlandy | http://logs.openstack.org/64/635464/5/check/tripleo-ci-centos-7-scenario009-standalone/da47051/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-11_21_41_16 | 13:49 |
| rlandy | which looks like bug ... | 13:49 |
| rlandy | https://bugs.launchpad.net/tripleo/+bug/1814564 | 13:49 |
| openstack | Launchpad bug 1814564 in tripleo "standalone fails if hostname is not properly set and pacemaker is used" [High,In progress] - Assigned to Alex Schultz (alex-schultz) | 13:49 |
| rlandy | but the hostname is defined | 13:49 |
| rlandy | - and again ... not sure I am going down the right path here | 13:49 |
| rlandy | just adding services and seeing what falls over | 13:50 |
| weshay | 2019-02-11 21:41:16 | "stderr: Error: unable to find resource 'galera-bundle'", | 13:50 |
| marios | rlandy: what happens if you don't use pcs (or does your scenario need it explicitly?) | 13:50 |
| rlandy | marios: I didn't have it originally ... | 13:50 |
| weshay | that's diff | 13:50 |
| * rlandy gets that results | 13:51 | |
| marios | rlandy: weshay re the pcs in standalone - i thought it was on by default (i would have sworn we stiched to it, aaaages ago. maybe switch back?0 but apparently not at https://github.com/openstack/tripleo-heat-templates/blob/9b073e6b762de4557e02361ccc6bef89e24b4279/overcloud-resource-registry-puppet.j2.yaml#L172-L173 and the standalone env doesn't have it either in https://github.com/openstack/tripleo-heat-templates/blob/master/environments/standa | 13:51 |
| *** ykarel|away is now known as ykarel | 13:51 | |
| rlandy | marios: it's not clear to me what services are actually needed | 13:53 |
| marios | rlandy: should be 'whatever was in the multinode' or is it not so simple for scen9 /me lokos | 13:53 |
| marios | (looks) | 13:53 |
| rlandy | so - mysql was missing | 13:55 |
| rlandy | I added that ... https://review.openstack.org/#/c/635208/6..9/ci/environments/scenario009-standalone.yaml | 13:55 |
| rlandy | then that result seemed to indicate pacemaker | 13:55 |
| panda | marios: works! thanks. I mean, tempest fails, but at least it runs | 13:55 |
| rlandy | so I got going with those | 13:55 |
| marios | rlandy: ack i added a cmment just now https://review.openstack.org/#/c/635208/9/ci/environments/scenario009-standalone.yaml but you replied here.. soyou mean you had error and added the sql | 13:56 |
| rlandy | marios: yes - then I got this ... | 13:56 |
| marios | panda: cool \o/ | 13:56 |
| marios | rlandy: ah | 13:56 |
| marios | rlandy: try docker/services/database instaed of pacemaker/database | 13:57 |
| marios | if you arent gonna have pcs | 13:57 |
| marios | ? | 13:57 |
| rlandy | marios: an error that should have been fixed with setting the hostname | 13:57 |
| * rlandy reads comment | 13:57 | |
| marios | rlandy: just commented at https://review.openstack.org/#/c/635208/9/ci/environments/scenario009-standalone.yaml | 13:58 |
| rlandy | marios: that is where I started | 13:58 |
| rlandy | with just those lines | 13:58 |
| marios | rlandy: no i added another comment now re the mysql | 13:58 |
| marios | rlandy: like add mysql but not ../../docker/services/pacemaker/database/mysql.yaml since you want to try without pcs | 13:59 |
| marios | use docker/services/database/mysql.yaml instead | 13:59 |
| marios | rlandy: or you already tried that too | 13:59 |
| rlandy | marios: ah | 13:59 |
| rlandy | no - let me try ... | 13:59 |
| marios | k | 13:59 |
| rlandy | thanks | 13:59 |
| marios | np dont thank yet :) | 13:59 |
| marios | sorry about the pcs confusion thanks for pointing at it we should remove it at some point... | 14:00 |
| marios | rlandy: ^ | 14:00 |
| rlandy | yeah - was not quite sure why standalone would use it- but hey, I'm a newbie at this | 14:00 |
| *** agopi has joined #oooq | 14:02 | |
| rlandy | marios: https://review.openstack.org/#/c/635208/10/ci/environments/scenario009-standalone.yaml | 14:05 |
| rlandy | correct now? | 14:05 |
| marios | ack rlandy lets see (yeah that's what i meant) | 14:05 |
| *** EmilienM has joined #oooq | 14:06 | |
| rlandy | ok - here we go | 14:06 |
| EmilienM | quiquell|rover, weshay : https://www.redhat.com/en/blog/working-container-storage-library-and-tools-red-hat-enterprise-linux | 14:06 |
| EmilienM | "In Docker world there are two overlay drivers, overlay and overlay2, today most users use the overlay2 driver, so we just use that one, and called it overlay. If you accidentally use overlay2 in the config containers storage is smart enough to alias it to overlay." | 14:06 |
| rlandy | weshay: wrt reproducer - there is a still a problem with picking up the mirror_fqdn if the env var is not set | 14:07 |
| rlandy | will have to set it either way | 14:07 |
| *** zul has joined #oooq | 14:07 | |
| weshay | rlandy, hrm.. I did not hit that | 14:07 |
| rlandy | weshay: did you run libvirt? | 14:12 |
| weshay | rlandy, aye | 14:12 |
| rlandy | weshay: yep - it's non-libvirt | 14:12 |
| weshay | k.. I will try rdo | 14:12 |
| rlandy | weshay: your job will not start | 14:13 |
| rlandy | since the mirror_fqdn will be empty | 14:13 |
| * rlandy will fix | 14:14 | |
| rlandy | weshay: testing the fix - sec | 14:17 |
| rlandy | https://github.com/ansible/ansible/issues/15575 | 14:18 |
| weshay | arxcruz, can you join my blue | 14:22 |
| arxcruz | weshay: yes | 14:22 |
| weshay | community mtg if anyone has anything as well | 14:22 |
| quiquell|rover | EmilienM: So it has being always overlay2 ? | 14:23 |
| EmilienM | yup | 14:23 |
| quiquell|rover | EmilienM: ack | 14:23 |
| ykarel | panda, sshnaidm can you check https://bugs.launchpad.net/tripleo/+bug/1815048, it needed unified solution for both ovb and non-ovb | 14:23 |
| openstack | Launchpad bug 1815048 in tripleo "[ovb][collect_logs] ovb collect logs job runs twice " [High,Fix released] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 14:23 |
| EmilienM | I'm spawning an instance to confirm | 14:23 |
| EmilienM | but I'm almost sure | 14:23 |
| *** skramaja has quit IRC | 14:23 | |
| EmilienM | I don't think it was vfs, really too slow | 14:23 |
| sshnaidm | ykarel, does it happen in non-ovb too? | 14:24 |
| ykarel | sshnaidm, yup | 14:24 |
| rfolco | CI community meeting (office hours) starts now at https://bluejeans.com/4113567798 - agenda: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 14:24 |
| sshnaidm | ykarel, ook, will look then | 14:24 |
| ykarel | sshnaidm, in post it runs twice | 14:24 |
| quiquell|rover | EmilienM: but maybe is better to be explicit and let it be overlay2 ? or it can give us problems ? | 14:25 |
| ykarel | sshnaidm, for ovb once in run other in post | 14:25 |
| ykarel | sshnaidm, you fixed ^^ | 14:25 |
| EmilienM | quiquell|rover: tbh I would stick on defaults for this one | 14:28 |
| *** sshnaidm has quit IRC | 14:30 | |
| *** sshnaidm has joined #oooq | 14:30 | |
| quiquell|rover | EmilienM: humm tripleo-ci-centos-7-scenario003-standalone is failing in the review with noop overlay2 | 14:30 |
| rlandy | weshay: did you try rdo at all? | 14:33 |
| quiquell|rover | EmilienM: Do you see any failure here ? http://logs.openstack.org/17/636317/4/check/tripleo-ci-centos-7-standalone/f710cfc/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 14:37 |
| sshnaidm | quiquell|rover, do you remember where is json-output.txt is saved? On the node or some zuul folder.. I fount it once, but forgot | 14:38 |
| quiquell|rover | sshnaidm: zuul-executor | 14:39 |
| quiquell|rover | sshnaidm: temporal dir | 14:39 |
| sshnaidm | quiquell|rover, oh! right, in /tmp | 14:39 |
| sshnaidm | quiquell|rover, yeah, thanks | 14:39 |
| quiquell|rover | /tmp/bublefoobar/ansible-foobarfoo | 14:39 |
| weshay | rlandy, ya.. I ran it on sunday | 14:42 |
| weshay | worked | 14:42 |
| weshay | w/ f28 as my client | 14:42 |
| rlandy | weshay; I thought I fixed it | 14:42 |
| rlandy | maybe my current env is just messed up | 14:42 |
| rlandy | I was trying to run ovb | 14:43 |
| *** EmilienM has left #oooq | 14:43 | |
| rlandy | it also worked for me on last week | 14:43 |
| arxcruz | weshay: https://github.com/openstack/tripleo-quickstart-extras/commit/3073296883c0df6cffe6ae02d68153091655b0f3 | 14:45 |
| arxcruz | weshay: https://review.openstack.org/#/c/591982/ | 14:46 |
| weshay | arxcruz, sec | 14:47 |
| weshay | arxcruz, so.. I guess we need a different list that both f20 and f21 would use? | 14:52 |
| *** udesale has quit IRC | 14:52 | |
| arxcruz | weshay: we need these plugins skipped on fs21 for sure | 14:52 |
| weshay | k.. | 14:53 |
| *** gkadam_ has quit IRC | 14:54 | |
| weshay | arxcruz, perhaps the skip list is not the right tool | 14:54 |
| *** gkadam has joined #oooq | 14:54 | |
| arxcruz | weshay: indeed, because we want to test f21 without skip list | 14:54 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/560445, stable/queens: tripleo-ci- (1 more message) | 14:55 |
| *** udesale has joined #oooq | 15:02 | |
| *** holser_ has quit IRC | 15:05 | |
| panda | rlandy: don't know if you were at the meeting, using -ug and -ur on the reproducer script works, thanks for adding those for the crazy multiuser people | 15:07 |
| rlandy | joining | 15:07 |
| rlandy | panda: must have missed it - thanks for testing that | 15:09 |
| rlandy | panda: did you run on rdo? and did you job actually launch and run> | 15:09 |
| panda | rlandy: honestly I have no idea, I can open the zuul page, I had logs scrolling, the job failed. But don't know where the node is, probaly rdo-cloud as it's the default | 15:12 |
| rlandy | panda: k - but it ran far enough to make sure you launched with the correct user | 15:13 |
| panda | rlandy: yes, all previous attempt were failing to launch the zuul container. | 15:15 |
| rlandy | marios: ok - I'm back ... | 15:18 |
| rlandy | http://logs.openstack.org/64/635464/5/check/tripleo-ci-centos-7-scenario009-standalone/411424a/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_15_03_59 | 15:18 |
| marios | rlandy: looks like that address already in use issue again http://logs.openstack.org/64/635464/5/check/tripleo-ci-centos-7-scenario009-standalone/411424a/logs/undercloud/var/log/containers/mysql/mariadb.log.txt.gz#_2019-02-12_15_10_27 :/ | 15:25 |
| rlandy | marios: I have been hitting that error every time | 15:25 |
| rlandy | marios: is that purely an infra issue or something I am missing? | 15:25 |
| marios | rlandy: really don't know trying to find more but not sure what the issue is here yet | 15:26 |
| rlandy | marios: sorry to dump this on you - perhaps we can debug together once so I know what you are looking at | 15:26 |
| marios | rlandy: at the moment this is just debug/i mean i don't know soehting more /specific to the standalone to be checkign | 15:27 |
| marios | rlandy: so no specific thing :) just looking in the logs you pointed at for now | 15:27 |
| rlandy | ok - I'll go through it | 15:27 |
| rlandy | sshnaidm: do you have the instructions somewhere of what needs to be set for ovb - trying that now | 15:28 |
| rlandy | weshay: and wrt reproducer my env is messed out keeps relauncging the old job | 15:28 |
| sshnaidm | rlandy, just create a zuul.yaml file in repo root with such text: http://paste.openstack.org/show/744946/ | 15:30 |
| sshnaidm | rlandy, then "git add zuul.yaml; git commit; git review" | 15:30 |
| sshnaidm | rlandy, if need - include depends-on in commit msg | 15:31 |
| sshnaidm | rlandy, later need to think how to pass cloud settings in more convenient mode, because "vars" overwrite "vars" from job definition | 15:32 |
| rlandy | sshnaidm: yeah - just thinking | 15:32 |
| sshnaidm | rlandy, that's why you need to specify nodes and featureset too, and playbooks if you have them different | 15:32 |
| rlandy | sshnaidm: so - why the rdocloud stuff if you have clouds.yaml? | 15:33 |
| sshnaidm | rlandy, this is for creating clouds.yaml in ovb jobs | 15:33 |
| sshnaidm | rlandy, my clouds.yaml is for nodepool - to create nodes there for jobs, both for multinode | 15:35 |
| weshay | rfolco, source of truth :) https://tree.taiga.io/project/tripleo-ci-board/epic/445 | 15:35 |
| weshay | everything is parked | 15:35 |
| rlandy | sshnaidm: yeah ok - will juts have to see how we can work this out from the job | 15:36 |
| rlandy | so t heusers won;t need to add it off hand | 15:36 |
| rlandy | the users | 15:36 |
| rfolco | weshay, ah need to change us status as well, it is not automatic. | 15:36 |
| zbr | rfolco: i will rework https://review.openstack.org/#/c/636160/1 if you do not mind, to make it more future proof, as i told you yesterday. ok? | 15:38 |
| marios | rlandy: can't spot something yet there, i'll have a look again in my morning reviews run see if you get any progress today | 15:39 |
| rfolco | zbr, ok, add you as co-author please | 15:39 |
| rlandy | marios; ack - thanks | 15:39 |
| *** holser_ has joined #oooq | 15:40 | |
| sshnaidm | rlandy, we can define a new "-test" job inherited from the required one and pass there only cloud variables, it won't override it but will merge | 15:40 |
| sshnaidm | rlandy, but it'll be a different job name, so not sure if it's ok.. | 15:40 |
| rlandy | sshnaidm: we will have the clouds.yaml - so some we can just fill out | 15:40 |
| rfolco | zbr, this is the test with that patch: http://logs.openstack.org/88/636188/4/check/tripleo-build-containers-fedora-28/f7889ea/job-output.txt.gz#_2019-02-12_13_13_15_969999 | 15:41 |
| rfolco | zbr, need to fix this ^ | 15:41 |
| sshnaidm | rlandy, if you mean cloud variables fill - then yeah, we can take them either from clouds.yaml or environment, as we did before | 15:41 |
| *** saneax has quit IRC | 15:41 | |
| zbr | rfolco: sure. | 15:41 |
| rlandy | sshnaidm: also - you are defining a whole job, in reproducer we use a job definition and just override stuff | 15:42 |
| rlandy | nodes: 3ctlr_1comp | 15:42 |
| rlandy | featureset: '001' | 15:42 |
| rlandy | ^^ should be defined | 15:42 |
| sshnaidm | rlandy, where am I defining a whole job? | 15:42 |
| *** ykarel is now known as ykarel|away | 15:43 | |
| *** vinaykns has joined #oooq | 15:43 | |
| rlandy | sshnaidm: what I mean is cloud_settings looks standard for all ovb | 15:44 |
| rlandy | so we can just slot that in | 15:44 |
| rlandy | all your setting are there | 15:44 |
| rlandy | but the ones we have to work out are not so bad | 15:44 |
| * rlandy tries | 15:44 | |
| weshay | zbr++ | 15:46 |
| hubbot1 | weshay: zbr's karma is now 1 | 15:47 |
| weshay | quiquell++ | 15:47 |
| weshay | quiquell|rover++ | 15:47 |
| hubbot1 | weshay: quiquell|rover's karma is now 6 | 15:47 |
| sshnaidm | rlandy, forgot one, also pass in variables: remove_ovb_after_job: false | 15:48 |
| sshnaidm | rlandy, it will leave ovb nodes after a job | 15:48 |
| rlandy | got it | 15:48 |
| rlandy | sshnaidm: working on review in launcher playbook - will take me a bit to get all the spacing right with jinja - will ping you with review | 15:49 |
| sshnaidm | rlandy, cool | 15:49 |
| quiquell|rover | What happend ? | 15:50 |
| weshay | need this for centos-7 repro to work https://review.openstack.org/#/c/636230/ just a couple lines | 15:51 |
| quiquell|rover | weshay: commented | 15:52 |
| weshay | quiquell|rover, line 201 is the yum install | 15:52 |
| weshay | 207 is an update | 15:53 |
| weshay | quiquell|rover, on centos.. using bindep you get ansible 2.4 | 15:53 |
| weshay | once the transaction completes, you can update ansible to 2.7 | 15:53 |
| weshay | and the meta:reset_connection works | 15:54 |
| weshay | in 2.4 it fails | 15:54 |
| weshay | quiquell|rover, bindep always returns 1 if there is a missing package | 15:54 |
| weshay | so you have to use bindep || yum install `bindep` | 15:54 |
| weshay | make sense? | 15:54 |
| quiquell|rover | packages=$(bindep ...) | 15:55 |
| quiquell|rover | -y install $packages | 15:55 |
| quiquell|rover | so you don't have to duplicate | 15:55 |
| weshay | quiquell|rover, ok.. I'll try that on top of this patch | 15:56 |
| weshay | this patch is about running yum update | 15:56 |
| weshay | which would be the same w/ ^ | 15:56 |
| quiquell|rover | Yep | 15:56 |
| quiquell|rover | so we are updating all the packages ' | 15:57 |
| quiquell|rover | ? | 15:57 |
| weshay | no | 15:57 |
| weshay | look at line 207 | 15:57 |
| weshay | thought it would be better to only ensure the packages in bindep are fully updated | 15:58 |
| quiquell|rover | ahh ack | 15:59 |
| quiquell|rover | only the missing ones | 15:59 |
| quiquell|rover | Argg please convert "bindep -b -l newline -f bindep.txt" into a function | 16:00 |
| quiquell|rover | and call the function | 16:00 |
| weshay | it updates all the packages that match | 16:00 |
| quiquell|rover | Hehe | 16:00 |
| weshay | why? | 16:00 |
| quiquell|rover | btw you don't need to specify -f | 16:00 |
| weshay | the function is to install and update | 16:00 |
| quiquell|rover | bindep.txt is the default | 16:00 |
| weshay | quiquell|rover, you do | 16:00 |
| weshay | no | 16:00 |
| weshay | when you have extra_requirements.txt | 16:00 |
| weshay | it prompts | 16:00 |
| quiquell|rover | ack | 16:01 |
| quiquell|rover | ack +2 | 16:01 |
| weshay | quiquell|rover, in tq.. there are two bindep files | 16:01 |
| quiquell|rover | weshay: humm I though you have fusion them | 16:02 |
| weshay | quiquell|rover, https://github.com/openstack/tripleo-quickstart/blob/master/other-requirements.txt | 16:03 |
| weshay | it's not clear to me we want to change that file | 16:04 |
| weshay | I could put up a change and try it | 16:04 |
| weshay | add those to bindep.txt | 16:04 |
| quiquell|rover | What prevent your from doing it ? | 16:05 |
| weshay | http://git.openstack.org/cgit/openstack-infra/bindep/tree/bindep/depends.py#n86 | 16:05 |
| weshay | it's just not 100% clear to me what's it's for. | 16:06 |
| weshay | but will put up a change | 16:06 |
| quiquell|rover | well looks like you have to chose what standard do you want | 16:06 |
| quiquell|rover | or not... | 16:07 |
| quiquell|rover | maybe other-requirements.txt was the old name or something | 16:07 |
| quiquell|rover | and this is only for retrocompatibility not as a feature | 16:07 |
| weshay | https://review.openstack.org/636366 | 16:09 |
| *** gkadam has quit IRC | 16:09 | |
| quiquell|rover | weshay: now you can remove -f bindep.txt | 16:09 |
| weshay | aye | 16:09 |
| weshay | after it merges :) | 16:09 |
| quiquell|rover | ooook | 16:09 |
| weshay | I'll put a review on top | 16:10 |
| quiquell|rover | +2 | 16:10 |
| rlandy | panda: are vars listed in the zuul inventory (not down from zuul) like featureset and nodes, accessible from collect_logs? | 16:12 |
| * rlandy thinks not | 16:12 | |
| *** gkadam has joined #oooq | 16:14 | |
| panda | rlandy: they should, we are passing the fail with the zuul inventory to collect logs playbook too | 16:16 |
| panda | file* | 16:16 |
| rlandy | panda: so featureset would be accessed as zuul.featureset or just featureset? | 16:17 |
| panda | rlandy: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/roles/run-test/templates/toci_quickstart.sh.j2#n58 | 16:17 |
| *** quiquell|rover is now known as quiquell|off | 16:17 | |
| *** jfrancoa has quit IRC | 16:18 | |
| panda | rlandy: zuul.featureset | 16:18 |
| rlandy | panda: zuul_variable just contains zuul. vars | 16:18 |
| rlandy | https://logs.rdoproject.org/64/635464/5/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/7a22e38/logs/zuul-variables.yaml | 16:18 |
| rlandy | so featureset is not there | 16:18 |
| rlandy | compare with zuul_inventory | 16:19 |
| panda | rlandy: yep, I see it | 16:19 |
| rlandy | panda; ok - I'll work with it | 16:19 |
| panda | rlandy: well we are dumping only the zuul dict, not all the inventory | 16:20 |
| rlandy | panda:correct - I though that was the case | 16:20 |
| panda | rlandy: we could create a similar dump, calling it job-variables. | 16:20 |
| rlandy | panda: that would be useful in the reproducer case | 16:21 |
| rlandy | although I may be able to get by w/o it | 16:22 |
| *** ccamacho has quit IRC | 16:23 | |
| *** ccamacho has joined #oooq | 16:24 | |
| *** udesale has quit IRC | 16:37 | |
| rlandy | sshnaidm: do you know if there is a way to query the clouds.yaml | 16:40 |
| rlandy | short of finding the file location and grepping the file | 16:40 |
| rlandy | for username password project | 16:40 |
| sshnaidm | rlandy, to look for it in ~/.config/openstack/clouds.yaml ? | 16:40 |
| sshnaidm | rlandy, or /etc/openstack/ | 16:41 |
| rlandy | openstack --os-cloud rdo-cloud | 16:41 |
| rlandy | ^^ something more like | 16:41 |
| *** ykarel|away has quit IRC | 16:49 | |
| *** ccamacho has quit IRC | 16:53 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/560445, stable/queens: tripleo-ci- (1 more message) | 16:55 |
| weshay | rlandy, /me running centos + rdo now | 16:56 |
| rlandy | weshay: when you get the zuul.yaml file - pls check mirror_fqdn | 16:57 |
| rlandy | thank you | 16:57 |
| *** hamzy has quit IRC | 17:06 | |
| sshnaidm | weshay, rlandy please take a look: https://review.openstack.org/#/q/topic:freeipa+(status:open+OR+status:open) | 17:09 |
| marios | sshnaidm: any idea about this (I saw your name on the rdo side where i took this from) https://review.openstack.org/#/c/635222/14/zuul.d/build-containers.yaml maybe we can talk some more tomorow if you have time in the morning /me going in a minute ... (but zuul doesn't like it decryption failed) | 17:10 |
| marios | sshnaidm: maybe we need to generate a new token for this job (i didn't run this through encryp | 17:11 |
| marios | sshnaidm: (i mean http://git.zuul-ci.org/cgit/zuul/tree/tools/encrypt_secret.py ) | 17:11 |
| marios | sshnaidm: i took the token from https://github.com/rdo-infra/review.rdoproject.org-config/blob/b883feda83d9ae1b0777a9a57027d453675ff0cd/zuul.d/secrets.yaml#L647 | 17:12 |
| sshnaidm | marios, seems like wrong key was used for encryption | 17:12 |
| sshnaidm | marios, rdo zuul id different and its keys are different, it's impossible to use encrypted by one zuul in another | 17:13 |
| marios | sshnaidm: ack well the token is from rdo job | 17:13 |
| marios | sshnaidm: and this is for gerrit | 17:13 |
| sshnaidm | marios, it this was a question.. | 17:13 |
| marios | sshnaidm: so yea | 17:13 |
| *** amoralej is now known as amoralej|off | 17:13 | |
| marios | sshnaidm: confirm we need to generate a new key for it i guess | 17:13 |
| sshnaidm | marios, for using in upstream zuul - yeah | 17:13 |
| sshnaidm | marios, I mean just encrypt the same password, but using key from upstream zuul | 17:14 |
| marios | sshnaidm: k i might ask you more tomorrow please | 17:14 |
| sshnaidm | marios, sure | 17:14 |
| marios | thx sshnaidm | 17:14 |
| *** gkadam has quit IRC | 17:19 | |
| zbr | out of curiosity: can we speedup pip install execution inside containers by sharing the pip cache with the host? would this make sense? a similar question could apply to yum but i am not sure if this could work of if it would be safe. | 17:25 |
| sshnaidm | zbr, only if readonly, writing to same directory from multiple containers is bad idea | 17:41 |
| zbr | sshnaidm: yeah, i know rw is tricky but if is ro, we lose most of the cache benefits. | 17:41 |
| sshnaidm | zbr, like running once in host itself and then mount all dockers with ro | 17:41 |
| sshnaidm | zbr, it's not really tricky, it just doesn't work | 17:42 |
| zbr | sshnaidm: which means that we should spawn own own cache-builder service. | 17:42 |
| sshnaidm | zbr, and if you have selinux... | 17:42 |
| rlandy | panda: so ... how would we dump all the job vars | 17:43 |
| sshnaidm | rlandy, weshay did we plan to figure out how to test patches in zuul-ci without submitting them to upstream infra? | 17:43 |
| rlandy | the rest are just listed under vars | 17:43 |
| rlandy | sshnaidm: we kind of benched that | 17:44 |
| rlandy | the zuul guys had their objections | 17:44 |
| rlandy | people would still be filling up upstream zuul with their patch changes | 17:44 |
| rlandy | which is unfortunate | 17:44 |
| rlandy | even if they are testing in a local test project | 17:45 |
| rlandy | they would have to add that project to local | 17:45 |
| *** dtrainor has quit IRC | 17:46 | |
| panda | rlandy: we may trey to dump host_vars['localhost] to the job-vars dict then job-vars file. But it may end up being a huge dump | 17:47 |
| rlandy | panda: the size is my concern | 17:47 |
| rlandy | because it will be all vars | 17:47 |
| *** dtrainor has joined #oooq | 17:48 | |
| rlandy | panda: otherwise we need some smart way to append the vars we need to zuul | 17:48 |
| rlandy | featureset and nodes at this point | 17:48 |
| panda | rlandy: yep, but we need to add them one by one to a copy: content: task | 17:50 |
| rlandy | hmmm ... we may have these | 17:50 |
| rlandy | {{ featureset_conf }} | 17:52 |
| *** bogdando has quit IRC | 17:52 | |
| rlandy | but not nodes | 17:56 |
| *** derekh has quit IRC | 17:56 | |
| *** dtantsur is now known as dtantsur|afk | 17:58 | |
| *** kopecmartin is now known as kopecmartin|off | 17:58 | |
| panda | rlandy: you need just featureset and nodes ? | 18:01 |
| panda | rlandy: I think you can add them directly to the zuul-variables.yaml then | 18:01 |
| rlandy | panda: for the moment yes | 18:01 |
| rlandy | panda: yep - that was what I was looking at | 18:01 |
| rlandy | not sure quite how yet though | 18:02 |
| rlandy | lineinfile | 18:03 |
| panda | rlandy: yes, or directly here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/roles/prepare-node/tasks/main.yaml#n46 add some other lines to the content with featureset: {{ featureset }} | 18:05 |
| rlandy | hmmm - could do | 18:06 |
| *** agopi is now known as agopi|food | 18:06 | |
| panda | mmmhh scenario 12 fails again tempest ... | 18:06 |
| *** panda is now known as panda|off | 18:06 | |
| panda|off | I'll tempest derek tomorrow | 18:07 |
| rlandy | weshay: how did the job run go? | 18:07 |
| weshay | rlandy, perfect timing.. so the standalone is deploying | 18:08 |
| weshay | you want to see the zuul.yaml | 18:08 |
| rlandy | weshay: pls | 18:08 |
| weshay | rlandy, k.. let's blue in 3 | 18:08 |
| rlandy | weshay: you can just paste it | 18:08 |
| rlandy | or tell me if the mirror_mqdn is there | 18:09 |
| rlandy | fqdn | 18:09 |
| weshay | http://pastebin.test.redhat.com/713271 | 18:09 |
| weshay | mirror_fqdn: mirror.regionone.rdo-cloud.rdoproject.org | 18:10 |
| weshay | rlandy, ^ | 18:12 |
| rlandy | weshay: yay - thank you | 18:13 |
| rlandy | my local zuul kept giving me back the same bad file | 18:13 |
| * rlandy breathes again | 18:13 | |
| *** jpena is now known as jpena|off | 18:23 | |
| rlandy | weshay: fyi - for ovb ... https://review.openstack.org/#/c/636408/1/roles/create-zuul-based-reproducer/templates/launcher-playbook.yaml.j2 | 18:26 |
| rfolco | zbr, I have some fixes for https://review.openstack.org/#/c/636160/... are you close to submit a new patchset? | 18:27 |
| *** agopi|food is now known as agopi | 18:30 | |
| *** panda|off has quit IRC | 18:34 | |
| *** gcerami_ has joined #oooq | 18:37 | |
| *** trown is now known as trown|lunch | 18:41 | |
| zbr | rfolco: yes, very close. as in 1-2min | 18:43 |
| *** hamzy has joined #oooq | 18:43 | |
| weshay | rlandy, deploy passed | 18:44 |
| rlandy | yay - how nice | 18:44 |
| rfolco | zbr, ok thanks, will review it | 18:45 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/560445, stable/queens: tripleo-ci- (1 more message) | 18:55 |
| *** dtantsur|afk has quit IRC | 19:04 | |
| *** jschlueter has quit IRC | 19:04 | |
| *** zul has quit IRC | 19:04 | |
| *** jschlueter has joined #oooq | 19:05 | |
| rlandy | sshnaidm: ever see this error when running ovb: http://10.10.122.58:8000/01/1001/1/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-dlrn-hash-tag/2176a62/job-output.txt.gz | 19:12 |
| rlandy | 2019-02-12 13:54:01.665466 | ERROR! the role 'configure-unbound' was not found in | 19:12 |
| sshnaidm | rlandy, the problem there is not with ovb, seems like it doesn't find a code at all | 19:13 |
| sshnaidm | rlandy, are zuul-mergers working? | 19:14 |
| *** gcerami_ is now known as panda | 19:14 | |
| rlandy | my whole zuul system is not happy today | 19:17 |
| *** trown|lunch is now known as trown | 19:44 | |
| *** jfrancoa has joined #oooq | 19:52 | |
| *** jfrancoa has quit IRC | 19:56 | |
| *** zul has joined #oooq | 19:59 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, stable/ocata: tripleo-ci-centos-7-ovb-3ctlr_1comp- (1 more message) | 20:55 |
| *** matbu has quit IRC | 21:06 | |
| *** matbu has joined #oooq | 21:07 | |
| *** holser_ has quit IRC | 21:15 | |
| *** dsneddon has quit IRC | 21:22 | |
| *** jtomasek has quit IRC | 21:35 | |
| *** trown is now known as trown|outtypewww | 21:36 | |
| *** agopi_ has joined #oooq | 21:37 | |
| *** agopi has quit IRC | 21:39 | |
| *** agopi_ is now known as agopi | 21:41 | |
| *** hamzy has quit IRC | 21:46 | |
| rlandy | weshay: been playing with options to add featureset and nodes to zuul-variables | 21:57 |
| rlandy | none of which are successful | 21:57 |
| rlandy | the colon throws it off | 21:58 |
| rlandy | trying other options new | 21:58 |
| rlandy | now | 21:58 |
| weshay | rlandy, http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&panelId=63&fullscreen | 21:58 |
| weshay | scaring me | 21:59 |
| rlandy | weshay: sorry - I am giving up on it | 21:59 |
| rlandy | ovb requires we store those variables | 21:59 |
| rlandy | but they are not passed to collect logs | 21:59 |
| rlandy | this is my last shot | 22:00 |
| weshay | rlandy, don't we already dump all the vars? | 22:00 |
| rlandy | weshay: only zuul vars | 22:01 |
| weshay | no | 22:01 |
| weshay | we dump more | 22:01 |
| rlandy | weshay: where? | 22:01 |
| rlandy | I need the job vars | 22:01 |
| weshay | getting | 22:02 |
| weshay | rlandy, https://github.com/openstack/tripleo-quickstart-extras/tree/master/roles/dump_vars | 22:03 |
| rlandy | weshay: correct no job vars there | 22:04 |
| weshay | rlandy, so.. why couldn't there be? | 22:05 |
| weshay | just depends when you run it I suppose | 22:05 |
| rlandy | well they are not passed to the nodes | 22:05 |
| weshay | job vars | 22:05 |
| weshay | rlandy, hold on.. hrm.. maybe I don't understand fully | 22:05 |
| weshay | want to chat or no... another time? | 22:05 |
| weshay | "{{ {'zuul': zuul} }}" | 22:07 |
| weshay | what is that? | 22:07 |
| rlandy | weshay: compare these two ... | 22:08 |
| rlandy | http://logs.openstack.org/67/631067/62/check/tripleo-ci-centos-7-undercloud-containers/db7e371/logs/zuul-variables.yaml | 22:08 |
| rlandy | ^^ which is built from "{{ {'zuul': zuul} }}" | 22:08 |
| rlandy | and | 22:08 |
| rlandy | http://logs.openstack.org/67/631067/62/check/tripleo-ci-centos-7-undercloud-containers/db7e371/zuul-info/inventory.yaml | 22:08 |
| rlandy | note that | 22:08 |
| rlandy | job vars are only in the second | 22:08 |
| rlandy | environment_type: singlenode | 22:09 |
| rlandy | featureset: '027' | 22:09 |
| rlandy | ^^ I need those vars | 22:09 |
| rlandy | we dump either the zuul vars or the vars on the nodes we work on with toci | 22:09 |
| rlandy | neither contain job vars | 22:09 |
| * rlandy brb | 22:10 | |
| weshay | rlandy, I think the diff is just that https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/dump_vars/templates/dump_variables_vars.j2 is run on the inner playbooks | 22:12 |
| weshay | if run from infra playbooks I suspect you'd get all those vars | 22:12 |
| * rlandy looks | 22:14 | |
| rlandy | weshay: yeah - trying capturing the hostvars now - of that doesn;t work, I'll use this | 22:14 |
| weshay | rlandy, in theory https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/dump_vars/templates/dump_variables_hostvars.j2 | 22:15 |
| rlandy | weshay: current patch https://review.openstack.org/#/c/636406/8/roles/prepare-node/tasks/main.yaml | 22:19 |
| weshay | rlandy, not sure if I had luck w/ to_nice_yaml but maybe they fixed it now | 22:19 |
| rlandy | weshay: we shall see | 22:20 |
| weshay | aye | 22:20 |
| rlandy | sorry that messed the stats | 22:20 |
| weshay | rlandy, ur fine.. | 22:20 |
| weshay | I just thought upstream was going down there for a sec | 22:20 |
| rlandy | no - I am going down | 22:21 |
| rlandy | oh no | 22:29 |
| *** jjoyce|gone is now known as jjoyce | 22:30 | |
| weshay | http://logs.openstack.org/06/636406/8/check/tripleo-ci-fedora-28-standalone/3e609f6/job-output.txt.gz#_2019-02-12_22_29_30_625850 | 22:31 |
| rlandy | weshay: put too many vars | 22:35 |
| rlandy | playbooks is defined in both | 22:35 |
| rlandy | we can;t do that | 22:36 |
| rlandy | weshay: so we need two vars, nodes and fs | 22:36 |
| rlandy | we have zuul.job | 22:36 |
| rlandy | and we could get those vars from zuul.job | 22:37 |
| rlandy | if and only if the job name is always the same format | 22:37 |
| rlandy | but I don;t think that is for certain | 22:37 |
| *** agopi has quit IRC | 22:43 | |
| rlandy | weshay: still here? | 22:52 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, stable/ocata: tripleo-ci-centos-7-ovb-3ctlr_1comp- (1 more message) | 22:55 |
| *** rascasoft has quit IRC | 23:13 | |
| *** rascasoft has joined #oooq | 23:16 | |
| *** vinaykns has quit IRC | 23:26 | |
| *** rascasoft has quit IRC | 23:27 | |
| *** agopi has joined #oooq | 23:33 | |
| *** rascasoft has joined #oooq | 23:42 | |
| *** rascasoft has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!