| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-containerized-undercloud-upgrades, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7 (1 more message) | 00:42 |
|---|---|---|
| *** marios has quit IRC | 01:03 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-containerized-undercloud-upgrades, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7 (1 more message) | 02:43 |
| *** skramaja has joined #oooq | 03:20 | |
| *** ykarel has joined #oooq | 03:21 | |
| *** saneax has quit IRC | 03:37 | |
| *** ykarel_ has joined #oooq | 03:38 | |
| *** ykarel has quit IRC | 03:40 | |
| *** ykarel__ has joined #oooq | 03:54 | |
| *** ykarel_ has quit IRC | 03:56 | |
| *** ykarel__ is now known as ykarel | 03:59 | |
| *** ratailor has joined #oooq | 04:11 | |
| *** gkadam has joined #oooq | 04:28 | |
| *** apetrich has joined #oooq | 04:42 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ (1 more message) | 04:43 |
| *** ratailor has quit IRC | 04:57 | |
| *** ratailor has joined #oooq | 04:58 | |
| *** udesale has joined #oooq | 05:06 | |
| *** kopecmartin has joined #oooq | 05:31 | |
| *** marios has joined #oooq | 05:37 | |
| *** quiquell has joined #oooq | 06:19 | |
| *** jfrancoa has joined #oooq | 06:26 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ (1 more message) | 06:43 |
| *** holser_ has joined #oooq | 06:45 | |
| *** udesale has quit IRC | 06:54 | |
| *** saneax has joined #oooq | 06:55 | |
| *** ccamacho has joined #oooq | 06:58 | |
| *** udesale has joined #oooq | 06:58 | |
| *** sanjayu_ has joined #oooq | 07:00 | |
| *** ratailor_ has joined #oooq | 07:00 | |
| *** saneax has quit IRC | 07:02 | |
| *** ratailor has quit IRC | 07:03 | |
| quiquell | ssbarnea: Are you around ? | 07:06 |
| ssbarnea | quiquell: yep | 07:06 |
| quiquell | ssbarnea: I am chaking some failures at one job, and looks like ansible_hostname is not as expected | 07:06 |
| quiquell | ssbarnea: Can you help me ? | 07:07 |
| ssbarnea | sure | 07:07 |
| quiquell | quiquell: So, look for "Lookup deployment UUID" here http://logs.openstack.org/72/587572/6/check/tripleo-ci-centos-7-3nodes-multinode/8c81b17/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-08-08_17_59_39 | 07:07 |
| quiquell | ssbarnea: This task has being included in the task before included: /var/lib/mistral/overcloud/ControllerApi/deployments.yaml for centos-7-rax-iad-0001216943 | 07:08 |
| quiquell | so the expected ansible_hostname is centos-7-rax-iad-0001216943 | 07:08 |
| quiquell | The if you go to the included tasks http://logs.openstack.org/72/587572/6/check/tripleo-ci-centos-7-3nodes-multinode/8c81b17/logs/undercloud/var/lib/mistral/overcloud/ControllerApi/deployments.yaml.txt.gz | 07:09 |
| quiquell | You see something like this deployment_uuid: "{{ lookup('file', tripleo_role_name ~ '/' ~ ansible_hostname | lower ~ '/' ~ item) | from_yaml | json_query(item ~ '.id')}}" | 07:09 |
| quiquell | and ansible_hostname is now centos-7-rax-iad-0001216958 | 07:10 |
| quiquell | I suspect is the defference between ansible host and ansible remote | 07:10 |
| quiquell | And we are choosing the wrong one | 07:10 |
| ssbarnea | looking..., let me try to debug this jinja2 filter. i never used json_query. | 07:11 |
| quiquell | ssbarnea: The main problemis ansible_hostname | 07:11 |
| ssbarnea | ahh, right. is not the some | 07:12 |
| ssbarnea | it means someone used a delegate on a block that does that include | 07:13 |
| quiquell | ssbarnea: Humm look t his | 07:13 |
| ssbarnea | is the only thing that could make sense, because task fails on host foo, but jinja code inside the task repost ansible_hostname as bar. | 07:14 |
| quiquell | so it's expected secondary-1 http://logs.openstack.org/72/587572/6/check/tripleo-ci-centos-7-3nodes-multinode/8c81b17/zuul-info/host-info.secondary-1.yaml | 07:14 |
| quiquell | but it got secondary-2 http://logs.openstack.org/72/587572/6/check/tripleo-ci-centos-7-3nodes-multinode/8c81b17/zuul-info/host-info.secondary-2.yaml | 07:14 |
| quiquell | Humm... so maybe they are not generating the playbooks for the third node... | 07:15 |
| quiquell | ssbarnea: Running with hosts: Controller:overcloud | 07:18 |
| ssbarnea | do we pass "-e ansible_host=xxx" somewhere in ansible? | 07:18 |
| quiquell | ssbarnea: Let's check the inventory | 07:18 |
| quiquell | ssbarnea: Maybe it's not generating the inventory correctly | 07:18 |
| *** amoralej|off is now known as amoralej | 07:19 | |
| ssbarnea | the mail loop has correc hostname "..943", only the variable is wrong which makes me believe something tells ansible to override it. Once such case would be if var is passed from CLI. | 07:19 |
| quiquell | ssbarnea: ansible_hostname var ? | 07:20 |
| ssbarnea | yeah, apparently ansible allows you to do this. | 07:20 |
| ssbarnea | see this comment https://github.com/ansible/ansible/issues/14958#issuecomment-314064930 | 07:21 |
| quiquell | ssbarnea: Humm also checking the inventory http://logs.openstack.org/72/587572/6/check/tripleo-ci-centos-7-3nodes-multinode/8c81b17/logs/undercloud/var/lib/mistral/overcloud/tripleo-ansible-inventory.yaml.txt.gz | 07:21 |
| quiquell | Controller points to centos-7-rax-iad-0001216958 | 07:22 |
| *** ratailor_ has quit IRC | 07:34 | |
| *** ratailor__ has joined #oooq | 07:35 | |
| *** ykarel has quit IRC | 07:36 | |
| *** ykarel has joined #oooq | 07:39 | |
| *** ratailor_ has joined #oooq | 07:50 | |
| *** tosky has joined #oooq | 07:52 | |
| *** ratailor__ has quit IRC | 07:52 | |
| *** udesale has quit IRC | 07:53 | |
| ssbarnea | quiquell: where can I find the source code of "Lookup deployment UUID" ?task? | 08:02 |
| *** pliu has joined #oooq | 08:03 | |
| *** bogdando has joined #oooq | 08:03 | |
| quiquell | ssbarnea: The problem is using inventory like Controller:overcloud | 08:05 |
| quiquell | ssbarnea: overcloud has to children ControllerApi and Controller | 08:06 |
| holser_ | tripleo-ci-centos-7-scenario001-multinode-oooq-container Timed out | 08:07 |
| holser_ | Are any known issues? | 08:07 |
| quiquell | holser_: I have see some jobs taking like more than one hore at prepare container | 08:08 |
| holser_ | quiquell - looks like just slow infra http://logs.openstack.org/20/576520/6/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/dd9853e/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 08:11 |
| quiquell | holser_: ack | 08:12 |
| holser_ | job started at 21:32:50, overcloud deployment started at 22.30 | 08:13 |
| * holser_ sighs | 08:13 | |
| *** udesale has joined #oooq | 08:24 | |
| *** dtantsur|afk is now known as dtantsur | 08:33 | |
| *** jfrancoa has quit IRC | 08:40 | |
| *** jfrancoa has joined #oooq | 08:42 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, tripleo-ci-centos-7-containerized-undercloud-upgrades, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: (1 more message) | 08:43 |
| *** ratailor_ has quit IRC | 08:46 | |
| *** ratailor_ has joined #oooq | 08:46 | |
| *** ykarel is now known as ykarel|lunch | 09:11 | |
| *** ykarel|lunch is now known as ykarel|mtg | 09:30 | |
| quiquell | panda|ruck, marios: TAGS review ready for workflow https://review.openstack.org/#/c/584508/ | 09:32 |
| arxcruz | chandankumar: please take a look at https://trello.com/c/UgBONzPU and move to done if you agree :) | 09:40 |
| chandankumar | arxcruz: done | 09:41 |
| *** ykarel_ has joined #oooq | 09:42 | |
| arxcruz | marios: https://review.openstack.org/#/c/577780 can you +2 +w ? | 09:43 |
| *** ykarel|mtg has quit IRC | 09:44 | |
| quiquell | arxcruz: marios is our morning core :-) | 09:45 |
| arxcruz | quiquell: yup | 09:45 |
| chandankumar | marios: https://review.openstack.org/584368 this one needs +2 and +w | 09:47 |
| quiquell | marios: Thanks man, we can +1w too but maybe it's better to wait for panda|ruck | 10:04 |
| *** ykarel_ has quit IRC | 10:06 | |
| *** gkadam has quit IRC | 10:13 | |
| *** gkadam has joined #oooq | 10:13 | |
| *** ykarel has joined #oooq | 10:14 | |
| marios | quiquell: ack | 10:19 |
| marios | arxcruz: i'll check that patch butneeds a bit of review time :) will try check it some more/later | 10:19 |
| *** ratailor_ has quit IRC | 10:33 | |
| *** ratailor has joined #oooq | 10:33 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, tripleo-ci-centos-7-containerized-undercloud-upgrades, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: (1 more message) | 10:43 |
| *** amoralej is now known as amoralej|lunch | 11:01 | |
| *** holser_ has quit IRC | 11:06 | |
| panda|ruck | arxcruz: chandankumar can you take a look at this ? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-queens/a528a93/job-output.txt.gz#_2018-08-09_03_51_04_135774 | 11:17 |
| quiquell | panda|ruck: +1w for the sprint TAGs review https://review.openstack.org/#/c/584508/ | 11:26 |
| quiquell | playbook executions are ok | 11:26 |
| *** agopi has quit IRC | 11:31 | |
| *** agopi has joined #oooq | 11:49 | |
| *** ratailor has quit IRC | 12:01 | |
| weshay | arxcruz, chandankumar why are you guys on this call? | 12:01 |
| weshay | is there a specific tempest issue? | 12:02 |
| chandankumar | weshay: the call just started | 12:02 |
| arxcruz | weshay: i got an invite | 12:02 |
| weshay | arxcruz, chandankumar you guys can drop | 12:02 |
| weshay | arxcruz, chandankumar I'll get w/ you if there is a tempest issue | 12:02 |
| weshay | you don't want to be here | 12:03 |
| arxcruz | ok | 12:03 |
| arxcruz | indeed i don't :D | 12:03 |
| weshay | arxcruz, chandankumar I'll need you both on asaf's call re: the upgrade tool | 12:03 |
| weshay | though | 12:04 |
| weshay | hopefully that one is better | 12:04 |
| arxcruz | weshay: sure, but i need to drop in the middle, my wife leave and i have to take my daughter at school :/ | 12:04 |
| weshay | panda|ruck, fyi https://bugs.launchpad.net/tripleo/+bug/1786109 | 12:05 |
| openstack | Launchpad bug 1786109 in tripleo "did not reach state "available", the state is "clean wait", error: None'" [Medium,Triaged] - Assigned to wes hayutin (weshayutin) | 12:05 |
| weshay | panda|ruck, closed as a dupe https://bugs.launchpad.net/tripleo/+bug/1786223 | 12:05 |
| openstack | Launchpad bug 1786109 in tripleo "duplicate for #1786223 did not reach state "available", the state is "clean wait", error: None'" [Medium,Triaged] - Assigned to wes hayutin (weshayutin) | 12:05 |
| panda|ruck | weshay: ok | 12:06 |
| arxcruz | panda|ruck: this is fs021 this two tests are supposed to fail | 12:08 |
| panda|ruck | arxcruz: so the tests fail but hte job shouldn't ? | 12:10 |
| arxcruz | panda|ruck: this job will fail it's our way to track down the issues that was already fixed upstream/downstream etc | 12:15 |
| panda|ruck | arxcruz: ok, so it's periodic but not in the criteria for promotion, now I understand | 12:19 |
| panda|ruck | arxcruz: thanks | 12:19 |
| arxcruz | panda|ruck: np | 12:19 |
| weshay | panda|ruck, not sure if the introspection issue is a blocker btw | 12:20 |
| *** trown|outtypewww is now known as trown | 12:22 | |
| rfolco|rover | panda|ruck, launchpad bugs page isn't loading CSS, just for me? | 12:27 |
| panda|ruck | rfolco|rover: yes | 12:27 |
| rfolco|rover | :( | 12:27 |
| rfolco|rover | k thx | 12:27 |
| apetrich | weshay, 1:1? | 12:30 |
| weshay | apetrich, ah.. ur here | 12:32 |
| weshay | apetrich, https://bluejeans.com/4113567798 | 12:33 |
| apetrich | weshay, aye | 12:33 |
| weshay | quiquell, #sf-dfg | 12:34 |
| weshay | join please | 12:34 |
| weshay | quiquell, and #rdo | 12:36 |
| weshay | please join | 12:36 |
| weshay | quiquell, ah.. conversation is in fucking #rhos-dev | 12:38 |
| quiquell | weshay: ok, goind there | 12:38 |
| quiquell | weshay: Want to check stuff first, there is something at our cockpit that does not sum up | 12:39 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, (1 more message) | 12:43 |
| *** quiquell has quit IRC | 12:58 | |
| *** amoralej|lunch is now known as amoralej | 13:02 | |
| *** holser_ has joined #oooq | 13:12 | |
| *** skramaja has quit IRC | 13:25 | |
| *** jfrancoa has quit IRC | 13:25 | |
| *** apetrich has quit IRC | 13:34 | |
| *** quiquell|mtg has joined #oooq | 13:36 | |
| *** quiquell|mtg is now known as quiquell | 13:36 | |
| quiquell | weshay: The issue with the cockpit and NODE_FAILURE is zuul not reporting start/end time for those, so it's using the server time at that moment | 13:37 |
| quiquell | weshay: We can totally ignore NODE_FAILURE | 13:37 |
| quiquell | weshay: Or calculate from gerrit | 13:37 |
| *** ccamacho has quit IRC | 13:44 | |
| *** ccamacho has joined #oooq | 13:44 | |
| *** ykarel is now known as ykarel|away | 13:46 | |
| quiquell | ssbarnea: Have generalize the comparing function to include more files to compare https://review.openstack.org/588475 | 13:48 |
| *** ccamacho has quit IRC | 13:48 | |
| *** quiquell is now known as quiquell|off | 13:48 | |
| ssbarnea | quiquell|off: do you have two CR number to test with? | 13:49 |
| ssbarnea | btw, where to get jenkins credentials for ci.centos.org ? | 13:52 |
| *** ykarel|away has quit IRC | 13:53 | |
| *** quiquell|off has quit IRC | 13:53 | |
| *** apetrich has joined #oooq | 13:54 | |
| rfolco|rover | please review https://review.openstack.org/#/c/587572/ when you get a chance --> fix tripleo-ci-centos-7-3nodes-multinode job | 13:56 |
| *** vinaykns has joined #oooq | 13:57 | |
| ssbarnea | asking because I want to check why the jobs on jenkins have no ansible coloring, either misisng the ansi wrapper and/or the environment variable. easy fix. | 13:59 |
| *** apetrich has quit IRC | 14:01 | |
| ssbarnea | also tox.ini in tripleo-quickstart has a jjb target (environmnet) which cannot run because there is no "jenkins" folder. Is this env jus relic? where are the jobs now? | 14:02 |
| *** ccamacho has joined #oooq | 14:25 | |
| *** ccamacho has quit IRC | 14:25 | |
| *** ccamacho has joined #oooq | 14:26 | |
| *** gkadam_ has joined #oooq | 14:26 | |
| *** gkadam has quit IRC | 14:30 | |
| *** vinaykns has quit IRC | 14:35 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, (1 more message) | 14:43 |
| *** dtantsur is now known as dtantsur|brb | 14:43 | |
| weshay | panda|ruck, fyi.. ocata p2 is promoting https://review.rdoproject.org/r/#/c/15364/1 | 15:08 |
| weshay | panda|ruck, rfolco|rover based on what I see we could promote queens.. | 15:12 |
| weshay | panda|ruck, rfolco|rover http://cistatus.tripleo.org/promotion/ | 15:12 |
| weshay | looks like fs035 has been passing regularly.. and hit a blip | 15:13 |
| weshay | this time | 15:13 |
| rfolco|rover | weshay, is the ceph issue resolved ? | 15:14 |
| weshay | rfolco|rover, ah.. good point /me checks | 15:14 |
| rfolco|rover | wasn't last time I checked | 15:15 |
| rfolco|rover | for fs016 I still see ceph problem, and fixes not merged | 15:27 |
| rfolco|rover | http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-queens/8761bba/logs/subnode-2/var/log/extra/docker/containers/gnocchi_db_sync/stdout.log.txt.gz | 15:27 |
| rfolco|rover | for others like fs021 see tempest failures | 15:27 |
| weshay | kopecmartin, http://logs.openstack.org/42/589742/3/check/build-openstack-sphinx-docs/2a676c0/html/ | 15:28 |
| rfolco|rover | http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-queens/a528a93/logs/undercloud/home/zuul/tempest.log.txt.gz | 15:28 |
| weshay | ssbarnea, can you help me review http://logs.openstack.org/42/589742/3/check/build-openstack-sphinx-docs/2a676c0/html/ | 15:28 |
| weshay | ssbarnea, https://review.openstack.org/#/c/589742 | 15:28 |
| weshay | ssbarnea, we need fresh eyes | 15:28 |
| *** rfolco|rover is now known as rfolco|rover|brb | 15:28 | |
| *** dtantsur|brb is now known as dtantsur | 15:29 | |
| *** ccamacho has quit IRC | 15:29 | |
| weshay | kopecmartin, ssbarnea formatting looks off on https://docs.openstack.org/python-tempestconf/latest/cli/cli_options.html#discover-tempest-config | 15:35 |
| weshay | in cli args | 15:35 |
| *** sanjayu_ has quit IRC | 15:35 | |
| kopecmartin | weshay, ssbarnea well, it's auto-generated | 15:37 |
| weshay | ssbarnea, that was quick | 15:44 |
| weshay | lolz | 15:44 |
| weshay | panda|ruck, bah.. finally out of mtgs | 15:46 |
| ssbarnea | weshay: i only reviewed that change by looking at the generated doc, not following the doc steps to check if they are valid. it would take few hours to do QE on the entire docs. | 15:47 |
| panda|ruck | weshay: tha's why you want to join other two now ? :) | 15:48 |
| weshay | panda|ruck, heh.. I guess so | 15:50 |
| weshay | panda|ruck, /me gets coffee, tell me where to join you | 15:50 |
| weshay | panda|ruck, I'm here https://redhat.bluejeans.com/u/whayutin/ | 15:52 |
| weshay | panda|ruck, where are you atm :) | 15:53 |
| *** kopecmartin has quit IRC | 15:53 | |
| panda|ruck | weshay: in my happy place | 15:53 |
| weshay | panda|ruck, I want to go there tooo :( | 15:53 |
| * weshay wonders what panda|ruck's happy place looks like.. | 15:54 | |
| *** ykarel|away has joined #oooq | 16:00 | |
| weshay | rfolco|rover|brb, what was the bug on the ceph issue | 16:03 |
| weshay | rfolco|rover|brb, panda|ruck think it was fs165 | 16:04 |
| weshay | 16 | 16:04 |
| weshay | rfolco|rover|brb, panda|ruck looks fixed https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-queens/112aa82/ | 16:04 |
| weshay | panda|ruck, http://cistatus.tripleo.org/promotion/ | 16:05 |
| weshay | panda|ruck, https://review.openstack.org/#/c/590169/ | 16:09 |
| weshay | panda|ruck, https://review.openstack.org/#/c/590036/ | 16:10 |
| *** holser_ has quit IRC | 16:12 | |
| panda|ruck | arxcruz: can you check this one ? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/649eef0/job-output.txt.gz#_2018-08-09_04_15_46_381840 | 16:17 |
| weshay | panda|ruck, https://review.rdoproject.org/r/15373 | 16:20 |
| weshay | panda|ruck, | 16:21 |
| weshay | dlrnapi --url https://trunk.rdoproject.org/api-centos-queens --username review_rdoproject_org report-result --commit-hash ef6b5c335f1dd9551d040eeddeb665052ae88e71 --distro-hash 3562a5fc3f7d8dedb4969ac97355de40a03e7c46 --job-id periodic-multinode-1ctlr-featureset037 --info-url https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updat | 16:21 |
| weshay | es-queens/b2eaafd --timestamp 1533751819 --success True | 16:21 |
| *** gkadam_ has quit IRC | 16:32 | |
| *** tosky has quit IRC | 16:39 | |
| *** ykarel|away has quit IRC | 16:40 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, tripleo-ci-centos-7-containerized-undercloud-upgrades, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: (1 more message) | 16:43 |
| *** rfolco|rover|brb is now known as rfolco|rover | 16:48 | |
| rfolco|rover | weshay, https://bugs.launchpad.net/tripleo/+bug/1777249 | 16:51 |
| openstack | Launchpad bug 1777249 in tripleo "ceph.client.manila.keyring contains uncessary backslashes in caps mon" [Low,Fix released] - Assigned to John Fulton (jfulton-org) | 16:51 |
| rfolco|rover | weshay, queens branch did not merge, only master fix | 16:51 |
| rfolco|rover | https://review.openstack.org/#/c/589186/ | 16:51 |
| rfolco|rover | queens ^ | 16:52 |
| rfolco|rover | https://review.openstack.org/#/c/575915/ | 16:52 |
| rfolco|rover | master ^ | 16:52 |
| *** agopi has quit IRC | 16:54 | |
| *** amoralej is now known as amoralej|off | 16:58 | |
| *** bogdando has quit IRC | 17:00 | |
| myoung | weshay: panda|ruck and I are in room if still works for ya. | 17:05 |
| weshay | myoung, which room | 17:07 |
| panda|ruck | weshay: myoung room, 705 .. | 17:08 |
| weshay | chandankumar, setup a meeting w/ me tomorrow please | 17:08 |
| chandankumar | weshay: done, does scheduled time work for you? | 17:10 |
| weshay | panda|ruck, https://docs.google.com/document/d/1q_EEUdUBkECKMWC49ZIOv0CURihfU9POdbWFBP_dZLQ/edit | 17:11 |
| weshay | myoung, panda|ruck https://docs.google.com/presentation/d/1TAK312AC0AzOiw9HNSUZIWEkQX6C_jxgnlvDZpdcXAY/edit#slide=id.gb6edfa308_2_15 | 17:11 |
| *** udesale has quit IRC | 17:23 | |
| chandankumar | weshay: today is state bandh, internet is cutted off this time in night | 18:06 |
| *** agopi has joined #oooq | 18:14 | |
| panda|ruck | weshay: https://review.rdoproject.org/r/15376 https://review.rdoproject.org/r/15374 https://review.openstack.org/590428 | 18:20 |
| panda|ruck | weshay: never again | 18:21 |
| panda|ruck | I probably messed up a million configurations variables | 18:21 |
| panda|ruck | I'm going directly to sleep | 18:21 |
| *** panda|ruck is now known as panda|ruck|off | 18:22 | |
| *** dtantsur is now known as dtantsur|afk | 18:24 | |
| weshay | heh | 18:25 |
| weshay | panda|ruck|off, thanks | 18:25 |
| *** trown is now known as trown|lunch | 18:26 | |
| *** apetrich has joined #oooq | 18:33 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, tripleo-ci-centos-7-containerized-undercloud-upgrades, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: (1 more message) | 18:43 |
| *** apetrich has quit IRC | 19:18 | |
| *** trown|lunch is now known as trown | 19:43 | |
| myoung | weshay, panda|ruck|off, what's the timeline for adding rocky jobs? If we can sort out moving all the legacy jobs --> v3 format/syntax first, we might be able to avoid standing up rocky.* as legacy... | 19:44 |
| weshay | now :( | 19:44 |
| myoung | k | 19:44 |
| *** myoung is now known as myoung|biaf | 19:57 | |
| *** chem has quit IRC | 19:58 | |
| *** apetrich has joined #oooq | 20:05 | |
| rfolco|rover | weshay, are you aware of the time outs on gate jobs ? | 20:08 |
| jrist | hey guys | 20:11 |
| jrist | got a few minutes? | 20:11 |
| jrist | a few things that are happening on latest UI | 20:11 |
| jrist | using quickstart | 20:11 |
| jrist | plans are not being loaded (no default plan) | 20:11 |
| jrist | no validations | 20:11 |
| jrist | seems like the validations are not connected to the UI | 20:11 |
| jrist | somehow | 20:11 |
| jrist | cannot even log into the UI because it's trying to go to https://192.168.24.2/keystone/v3/auth/tokens | 20:11 |
| jrist | wrong network. | 20:11 |
| jrist | that was changed before by quickstart-extras | 20:12 |
| jrist | it no longer does | 20:12 |
| jrist | bueller | 20:16 |
| *** apetrich has quit IRC | 20:20 | |
| *** apetrich has joined #oooq | 20:35 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-containerized-undercloud-upgrades, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/560445, (1 more message) | 20:43 |
| *** myoung|biaf is now known as myoung | 20:49 | |
| *** ssbarnea has quit IRC | 20:52 | |
| weshay | rfolco|rover, time outs should be tracked here http://status.openstack.org/elastic-recheck/ | 20:53 |
| weshay | jrist, sorry was 1-1 | 20:54 |
| weshay | jrist, master? | 20:54 |
| jrist | weshay: master yes. | 20:56 |
| jrist | omg vscode is so nice for diffs | 20:57 |
| weshay | jrist, can we add a check / test for that? | 20:57 |
| weshay | https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-6620/console.txt.gz | 20:57 |
| weshay | 2018-08-09 19:14:07.304 | TASK [validate-ui : Execute the simple tripleo-ui validation] ****************** | 20:57 |
| weshay | rfolco|rover, some time outs happen from time to time | 20:57 |
| weshay | rfolco|rover, that is something to track, debug and figure out w/ an infra hat on | 20:58 |
| weshay | :) | 20:58 |
| weshay | rfolco|rover, we need the ara stats from the overcloud deployments | 20:58 |
| weshay | maybe you help push that along | 20:58 |
| weshay | rfolco|rover, https://review.openstack.org/#/c/565077/ | 20:59 |
| weshay | rfolco|rover, as long as we're hitting close to 100% here http://cistatus.tripleo.org/gates/ | 21:02 |
| weshay | rfolco|rover, I'm ok | 21:02 |
| weshay | rfolco|rover, the data needs to be double checked though :) | 21:03 |
| jrist | weshay: just a sec, doing a rebase | 21:03 |
| weshay | jrist, mutha fuka | 21:04 |
| *** trown is now known as trown|outtypewww | 21:04 | |
| * jrist puts on morcheeba | 21:07 | |
| jrist | LET'S DO THIS | 21:07 |
| jrist | ok so | 21:07 |
| jrist | put a check? | 21:07 |
| jrist | weshay: is that check not already tehre? | 21:07 |
| jrist | UI is up but can't login | 21:07 |
| jrist | needs that next step | 21:07 |
| weshay | ya | 21:07 |
| jrist | weshay: sorry for the stupid question but where is that validate-ui check? | 21:23 |
| weshay | jrist, codesearch bro | 21:24 |
| weshay | jrist, http://codesearch.openstack.org/?q=Execute%20the%20simple%20tripleo-ui%20validation&i=nope&files=&repos= | 21:24 |
| weshay | guess I'm brogrammer :( | 21:24 |
| weshay | I'm a | 21:24 |
| weshay | what's ur's say.. | 21:25 |
| weshay | null | 21:25 |
| jrist | thanks | 21:25 |
| weshay | what's ur's say.. | 21:25 |
| weshay | dev | 21:25 |
| jrist | what my what say | 21:25 |
| weshay | jrist, btw.. ptg and summit https://www.openstack.org/summit/denver-2019/ | 21:25 |
| jrist | yeah | 21:26 |
| jrist | ok so where is that validate-ui task run? undercloud or from my quickstart install dir | 21:28 |
| jrist | weshay: ^ | 21:28 |
| weshay | jrist, http://codesearch.openstack.org/?q=tripleoui-validate&i=nope&files=&repos= | 21:31 |
| weshay | jrist, it's called from that playbook | 21:31 |
| jrist | so on my laptop | 21:31 |
| * jrist looks | 21:31 | |
| weshay | jrist, if you want to just run that.. either use a tag or just that call from the cli | 21:31 |
| jrist | yeah | 21:31 |
| jrist | that was my plan | 21:31 |
| jrist | I don't now where it got created though | 21:32 |
| weshay | a playbook w/ just http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/playbooks/quickstart-extras-validate.yml#n28 | 21:32 |
| weshay | jrist, where what was created? | 21:32 |
| jrist | the validate-ui sh | 21:32 |
| *** apetrich has quit IRC | 21:32 | |
| *** agopi has quit IRC | 21:51 | |
| weshay | jrist, oh.. that is dropped in {{ local_working_dir }} | 21:52 |
| weshay | jrist, your python virt env | 21:52 |
| weshay | .quickstart | 21:52 |
| weshay | jrist, https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-6620/ | 21:52 |
| jrist | I have this dir, weshay | 21:52 |
| jrist | /home/jrist/.quickstart-galena/usr/local/share/ansible/roles/validate-ui/README.md | 21:52 |
| weshay | https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-6620/ui_validate_simple.sh | 21:52 |
| weshay | jrist, ya.. not ther | 21:52 |
| jrist | and a jinja for the validation role | 21:52 |
| weshay | e | 21:52 |
| weshay | .quickstart-galena | 21:53 |
| weshay | look there | 21:53 |
| jrist | ah | 21:54 |
| jrist | weshay: so similarly | 21:54 |
| jrist | we could do https://172.16.213.154/keystone/v3/auth/tokens | 21:55 |
| jrist | curl ^ | 21:55 |
| jrist | right now it's curl --insecure "https://172.16.213.154/login" | grep tripleo_ui.js | 21:55 |
| jrist | but that's just for login | 21:55 |
| jrist | if we can't hit that keystone url | 21:55 |
| jrist | we can't hit the UI login | 21:55 |
| weshay | jrist, k.. throw up a review brotha | 21:56 |
| jrist | weshay: totally new role? | 21:56 |
| weshay | jrist, no no | 21:57 |
| weshay | just add to it | 21:57 |
| jrist | add to that file | 21:57 |
| jrist | to the jinja | 21:57 |
| weshay | ya | 21:57 |
| jrist | k thx | 21:57 |
| weshay | rfolco|rover, http://status.openstack.org/elastic-recheck/#1686542 | 21:57 |
| jrist | weshay: does it handle timeouts? | 21:59 |
| weshay | jrist, what do you mean? | 21:59 |
| weshay | elastic-recheck? | 21:59 |
| jrist | weshay: like for now we're grepping for tripleo_ui.js | 21:59 |
| jrist | but what about just trying the URL | 22:00 |
| jrist | which will timeout | 22:00 |
| weshay | always? | 22:00 |
| weshay | jrist, you guys have some real tests around this now don't you? | 22:00 |
| weshay | jrist, you want to chat about this? | 22:01 |
| jrist | lol | 22:01 |
| jrist | it will if the URL doesn't get changed, which it isn't | 22:01 |
| jrist | right now when the UI is set up by quickstart | 22:01 |
| weshay | k | 22:01 |
| jrist | it puts the following in the config file: | 22:01 |
| jrist | weshay: 'keystone': 'https://192.168.24.2:443/keystone/v3', | 22:02 |
| jrist | and so on | 22:02 |
| jrist | wrong IP | 22:02 |
| jrist | used to get configured by tripleo-quickstart-extras | 22:02 |
| jrist | to change to {{the right ip}} | 22:02 |
| weshay | oh k | 22:02 |
| weshay | right | 22:03 |
| jrist | in the case of the validate ui jinja: virthost_addr | 22:03 |
| jrist | {{ virthost_addr }} | 22:03 |
| jrist | does not work now | 22:03 |
| jrist | that is one of the bugs I was mentioning like an hour ago :) | 22:03 |
| jrist | thus this check to catch it. (1) | 22:03 |
| jrist | now we also need to fix that problem (2) | 22:03 |
| jrist | doing the check right now | 22:03 |
| weshay | jrist, first thing would be to doc this in a bug | 22:04 |
| weshay | but ya | 22:04 |
| jrist | it is in a bug. | 22:04 |
| jrist | lp quickstart | 22:04 |
| jrist | https://trello.com/c/24gwdQtW/1002-get-ui-working-properly-with-a-quickstart-install | 22:04 |
| jrist | weshay: specifically https://bugs.launchpad.net/tripleo/+bug/1782438 | 22:05 |
| openstack | Launchpad bug 1782438 in tripleo "tripleo ui endpoints misconfigured in containerized undercloud" [High,Triaged] | 22:05 |
| weshay | jrist, https://github.com/openstack/tripleo-quickstart-extras/blob/b3794ff03ab841bbbc15a339e35a7ec02b193111/roles/undercloud-deploy/tasks/tripleo-ui.yml | 22:06 |
| jrist | weshay: why are you showing me this? | 22:07 |
| weshay | jrist, you want to fix this right? | 22:07 |
| weshay | just figuring out which param to update | 22:07 |
| weshay | I think it's https://github.com/openstack/tripleo-quickstart/blob/master/roles/libvirt/setup/undercloud/templates/ssh-tunnel.service.j2 | 22:07 |
| weshay | to be honest | 22:07 |
| jrist | nope | 22:08 |
| jrist | tunnel works fine | 22:08 |
| jrist | weshay: the UI config doesn't get properly written | 22:08 |
| jrist | weshay: I will upload a copy. one moment | 22:08 |
| weshay | ok.. so ansible_default_ipv4.address | 22:08 |
| jrist | window.tripleOUiConfig = { | 22:09 |
| jrist | // Service URLs (defaults to UI host) | 22:09 |
| jrist | // | 22:09 |
| jrist | // A valid Keystone service URL is required. The other endpoints | 22:09 |
| jrist | // will then be obtained automatically from the Keystone catalog. | 22:09 |
| jrist | // | 22:09 |
| jrist | 'keystone': 'https://192.168.24.2:443/keystone/v3', | 22:09 |
| jrist | 'heat': 'https://192.168.24.2:443/heat/v1/%(project_id)s', | 22:09 |
| jrist | 'ironic': 'https://192.168.24.2:443/ironic', | 22:09 |
| jrist | 'ironic-inspector': 'https://192.168.24.2:443/ironic-inspector', | 22:09 |
| jrist | 'mistral': 'https://192.168.24.2:443/mistral/v2', | 22:09 |
| jrist | 'nova': 'https://192.168.24.2:443/nova/v2.1', | 22:09 |
| jrist | 'swift': 'https://192.168.24.2:443/swift/v1/AUTH_%(project_id)s', | 22:09 |
| jrist | 'zaqar-websocket': 'wss://192.168.24.2:443/zaqar', | 22:09 |
| jrist | // Default websocket queue name | 22:09 |
| jrist | 'zaqar_default_queue': 'tripleo', | 22:09 |
| jrist | // Languages | 22:09 |
| jrist | // | 22:09 |
| jrist | // By default, all available languages are enabled. Use this setting to | 22:09 |
| jrist | // disable certain languages. | 22:09 |
| jrist | // | 22:09 |
| jrist | 'excludedLanguages': [], | 22:09 |
| jrist | // Logging | 22:09 |
| jrist | 'loggers': ["console","zaqar"] | 22:09 |
| jrist | }; | 22:09 |
| jrist | oh christ | 22:09 |
| jrist | meant to paste http://paste.openstack.org/show/727780/ but my crtl-v didn't work | 22:09 |
| jrist | sorry about that. | 22:10 |
| jrist | ah weshay so this check won't work curl --insecure "https://{{ virthost_addr }}/keystone/v3/auth/tokens" | grep "401" | 22:12 |
| jrist | because it will succeed | 22:12 |
| jrist | (because the tunnel works) | 22:12 |
| jrist | the correct IP gets written to that validate ui script | 22:12 |
| weshay | https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-6620/undercloud/var/log/extra/dump_variables_vars.json.gz | 22:14 |
| jrist | weshay: that's helpful | 22:14 |
| jrist | any idea why that didn't get passed through properly? | 22:14 |
| weshay | sec.. I think bogdan changed something | 22:15 |
| weshay | https://review.openstack.org/#/c/585741/1/config/general_config/minimal.yml | 22:16 |
| weshay | jrist, ^ | 22:16 |
| jrist | weshay: what am I looking at? | 22:16 |
| jrist | major imposter syndrome here | 22:16 |
| weshay | jrist, lolz | 22:16 |
| weshay | let's blue | 22:17 |
| jrist | k | 22:17 |
| weshay | https://redhat.bluejeans.com/u/whayutin/ | 22:17 |
| jrist | mine or yorus | 22:17 |
| jrist | k | 22:17 |
| weshay | jrist, https://review.openstack.org/#/c/585741/1/config/general_config/minimal.yml | 22:21 |
| weshay | jrist, https://github.com/openstack/tripleo-heat-templates/blob/master/net-config-undercloud.j2.yaml#L88 | 22:22 |
| *** tosky has joined #oooq | 22:30 | |
| weshay | jrist, https://github.com/openstack/tripleo-quickstart-extras/blob/b3794ff03ab841bbbc15a339e35a7ec02b193111/roles/undercloud-deploy/tasks/tripleo-ui.yml | 22:30 |
| weshay | jrist, https://github.com/openstack/tripleo-quickstart-extras/blob/b3794ff03ab841bbbc15a339e35a7ec02b193111/roles/undercloud-deploy/tasks/tripleo-ui.yml#L104 | 22:31 |
| weshay | jrist, | 22:38 |
| weshay | http://codesearch.openstack.org/?q=tripleo_ui_config.js&i=nope&files=&repos= | 22:38 |
| weshay | jrist, http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/docker/services/tripleo-ui.yaml | 22:41 |
| weshay | jrist, http://git.openstack.org/cgit/openstack/kolla/tree/docker/tripleo-ui/Dockerfile.j2 | 22:42 |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-containerized-undercloud-upgrades, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/560445, (1 more message) | 22:43 |
| jrist | thanks weshay | 22:47 |
| *** jjoyce has quit IRC | 23:44 | |
| *** jjoyce has joined #oooq | 23:45 | |
| *** tosky has quit IRC | 23:48 | |
| *** ssbarnea has joined #oooq | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!