| *** makowals has quit IRC | 00:00 | |
| *** dhill_ has quit IRC | 00:04 | |
| *** wolverineav has quit IRC | 00:04 | |
| *** makowals has joined #openstack-infra | 00:08 | |
| *** jascott1 has joined #openstack-infra | 00:13 | |
| *** jascott1 has quit IRC | 00:17 | |
| *** jkilpatr has quit IRC | 00:22 | |
| *** bobh has joined #openstack-infra | 00:23 | |
| *** bobh has quit IRC | 00:28 | |
| *** masuberu has joined #openstack-infra | 00:28 | |
| *** masber has quit IRC | 00:29 | |
| *** jamesmcarthur has joined #openstack-infra | 00:30 | |
| *** calebb has quit IRC | 00:33 | |
| *** calebb has joined #openstack-infra | 00:34 | |
| *** bobh has joined #openstack-infra | 00:34 | |
| *** bobh has quit IRC | 00:38 | |
| *** dave-mccowan has joined #openstack-infra | 00:40 | |
| *** jamesmcarthur has quit IRC | 00:41 | |
| *** jamesmcarthur has joined #openstack-infra | 00:43 | |
| *** tosky has quit IRC | 00:47 | |
| *** liusheng has quit IRC | 00:52 | |
| *** salv-orlando has joined #openstack-infra | 00:52 | |
| EmilienM | pabelanger: can you get https://review.openstack.org/#/c/522864/ please | 00:52 |
|---|---|---|
| *** jamesmcarthur has quit IRC | 00:56 | |
| *** makowals has quit IRC | 00:57 | |
| *** salv-orlando has quit IRC | 00:58 | |
| *** yuanying has quit IRC | 00:59 | |
| *** makowals has joined #openstack-infra | 01:02 | |
| *** dave-mccowan has quit IRC | 01:09 | |
| *** yuanying has joined #openstack-infra | 01:12 | |
| openstackgerrit | Merged openstack-infra/project-config master: instack-undercloud: remove legacy job https://review.openstack.org/522864 | 01:12 |
| openstackgerrit | Merged openstack-infra/project-config master: Clean up releasenotes jobs https://review.openstack.org/522830 | 01:13 |
| EmilienM | ianw: thx | 01:15 |
| *** ijw has joined #openstack-infra | 01:19 | |
| *** hongbin has joined #openstack-infra | 01:19 | |
| *** cuongnv has joined #openstack-infra | 01:19 | |
| *** bobh has joined #openstack-infra | 01:22 | |
| *** ijw has quit IRC | 01:23 | |
| *** dave-mccowan has joined #openstack-infra | 01:24 | |
| *** yamamoto has joined #openstack-infra | 01:29 | |
| *** Wei_Liu has joined #openstack-infra | 01:30 | |
| *** liujiong has joined #openstack-infra | 01:33 | |
| *** makowals has quit IRC | 01:33 | |
| openstackgerrit | Cao Xuan Hoang proposed openstack-infra/project-config master: Update neutron-vpnaas grafana dashboard https://review.openstack.org/522995 | 01:37 |
| *** makowals has joined #openstack-infra | 01:38 | |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade master: Switch baremetal nics/ports tests over https://review.openstack.org/522989 | 01:41 |
| *** masuberu has quit IRC | 01:43 | |
| *** salv-orlando has joined #openstack-infra | 01:54 | |
| *** kiennt26 has joined #openstack-infra | 01:59 | |
| *** salv-orlando has quit IRC | 02:00 | |
| openstackgerrit | Cao Xuan Hoang proposed openstack-infra/project-config master: Update neutron-vpnaas grafana dashboard https://review.openstack.org/522995 | 02:04 |
| *** ramishra has joined #openstack-infra | 02:09 | |
| openstackgerrit | Cao Xuan Hoang proposed openstack-infra/project-config master: Update neutron-vpnaas grafana dashboard https://review.openstack.org/522995 | 02:10 |
| *** liusheng has joined #openstack-infra | 02:14 | |
| *** kiennt26 has quit IRC | 02:18 | |
| *** gcb has joined #openstack-infra | 02:33 | |
| *** makowals has quit IRC | 02:35 | |
| *** gmann_afk is now known as gmann | 02:37 | |
| *** makowals has joined #openstack-infra | 02:47 | |
| *** threestrands_ has joined #openstack-infra | 02:49 | |
| *** threestrands_ has quit IRC | 02:50 | |
| *** threestrands has quit IRC | 02:50 | |
| *** threestrands_ has joined #openstack-infra | 02:50 | |
| *** kiennt26 has joined #openstack-infra | 02:52 | |
| *** masber has joined #openstack-infra | 02:54 | |
| *** salv-orlando has joined #openstack-infra | 02:56 | |
| *** dave-mccowan has quit IRC | 03:01 | |
| *** salv-orlando has quit IRC | 03:02 | |
| *** dhinesh has joined #openstack-infra | 03:06 | |
| *** dave-mccowan has joined #openstack-infra | 03:07 | |
| *** dhinesh has quit IRC | 03:07 | |
| *** ralonsoh has quit IRC | 03:08 | |
| *** masber has quit IRC | 03:09 | |
| *** dave-mccowan has quit IRC | 03:18 | |
| *** makowals has quit IRC | 03:20 | |
| *** kiennt26 has quit IRC | 03:23 | |
| *** makowals has joined #openstack-infra | 03:25 | |
| *** masber has joined #openstack-infra | 03:29 | |
| *** sree has joined #openstack-infra | 03:38 | |
| *** psachin has joined #openstack-infra | 03:50 | |
| *** e0ne has joined #openstack-infra | 03:51 | |
| *** salv-orlando has joined #openstack-infra | 03:57 | |
| *** raissa has joined #openstack-infra | 03:58 | |
| *** e0ne has quit IRC | 03:59 | |
| *** daidv has joined #openstack-infra | 04:00 | |
| *** raissa has quit IRC | 04:02 | |
| *** raissa has joined #openstack-infra | 04:03 | |
| *** raissa has quit IRC | 04:03 | |
| *** salv-orlando has quit IRC | 04:04 | |
| *** raissa has joined #openstack-infra | 04:04 | |
| *** raissa has quit IRC | 04:04 | |
| Jeffrey4l | hi, why neutron-dynamic-routing-master.tar.gz file in http://tarballs.openstack.org/neutron-dynamic-routing/ is not updated since 06 Oct | 04:04 |
| *** raissa has joined #openstack-infra | 04:04 | |
| Jeffrey4l | checked the zuul jobs, think post jobs should be run. but found nothing the logs logs.o.o site. | 04:05 |
| *** raissa has quit IRC | 04:05 | |
| *** raissa has joined #openstack-infra | 04:05 | |
| *** RuiChen has joined #openstack-infra | 04:05 | |
| *** raissa has quit IRC | 04:06 | |
| Jeffrey4l | infra-root ^^ | 04:06 |
| *** e0ne has joined #openstack-infra | 04:06 | |
| *** raissa has joined #openstack-infra | 04:06 | |
| *** raissa has quit IRC | 04:06 | |
| *** bobh has quit IRC | 04:10 | |
| *** e0ne has quit IRC | 04:10 | |
| *** makowals has quit IRC | 04:11 | |
| *** hongbin has quit IRC | 04:12 | |
| *** jascott1 has joined #openstack-infra | 04:13 | |
| ianw | Jeffrey4l: did you find no logs, or nothing in the logs? | 04:13 |
| Jeffrey4l | ianw, i can not found the jobs is ran. | 04:14 |
| *** bobh has joined #openstack-infra | 04:14 | |
| Jeffrey4l | nothing from here 78ae7cb8f5bcecaf4cb17577e38401b7d8628f20 | 04:14 |
| ianw | Jeffrey4l: here's the periodic logs : http://logs.openstack.org/periodic/git.openstack.org/openstack/neutron-dynamic-routing/master/ | 04:14 |
| Jeffrey4l | http://logs.openstack.org/78/78ae7cb8f5bcecaf4cb17577e38401b7d8628f20 | 04:14 |
| ianw | which job should publish it? | 04:14 |
| Jeffrey4l | publish-openstack-python-branch-tarball i think in the post pipeline. | 04:15 |
| *** jascott1 has quit IRC | 04:18 | |
| *** bobh has quit IRC | 04:19 | |
| ianw | Jeffrey4l: hmm, to start with, that doesn't seem to be in the post: jobs for neutron-dynamic-routing? | 04:20 |
| *** Dinesh_Bhor has joined #openstack-infra | 04:21 | |
| *** makowals has joined #openstack-infra | 04:22 | |
| Jeffrey4l | ianw neutron-dynamic-routing is using "openstack-python-jobs-neutron" tempalte, and this template contains the publish-openstack-python-branch-tarball jobs | 04:22 |
| Jeffrey4l | in the post pipeline. | 04:23 |
| ianw | ok, that makes sense | 04:24 |
| *** ykarel|afk has joined #openstack-infra | 04:24 | |
| Jeffrey4l | but it wont' run. no idea why :( | 04:24 |
| *** links has joined #openstack-infra | 04:26 | |
| ianw | hmm, let me poke a bit | 04:27 |
| Jeffrey4l | thanks | 04:27 |
| ianw | sigh ... status.openstack.org seems to be unresponsive | 04:29 |
| Jeffrey4l | :/ | 04:31 |
| ianw | it's got the xen kernel timeout errors like we've seen before | 04:33 |
| Jeffrey4l | try reboot..which may solve all the issue ;p | 04:34 |
| ianw | #status log rebooted status.o.o as it was hung, likely migration failure as had xen timeout errors on the console | 04:34 |
| openstackstatus | ianw: finished logging | 04:35 |
| *** bhavik1 has joined #openstack-infra | 04:35 | |
| Jeffrey4l | ianw, status.o.o works now. doesn't this cause the jobs not run? | 04:36 |
| ianw | no that's not it, just noticed it when looking at jobs | 04:37 |
| Jeffrey4l | roger. | 04:37 |
| *** bobh has joined #openstack-infra | 04:37 | |
| ianw | Jeffrey4l: ok, there's definitely a non-trivial error here -> http://paste.openstack.org/show/627401/ | 04:41 |
| ianw | this was in response to me triggering : zuul enqueue-ref --tenant openstack --trigger gerrit --pipeline post --project openstack/neutron-dynamic-routing --ref refs/heads/master --oldrev 183c3fa4840d22be1974534eb9e1b28b552f4a42 --newrev 78ae7cb8f5bcecaf4cb17577e38401b7d8628f20 | 04:41 |
| ianw | but when i look back through the logs, the same thing happened with the original event | 04:41 |
| *** bobh has quit IRC | 04:41 | |
| *** yamamoto has quit IRC | 04:43 | |
| ianw | 2017-11-26 06:14:06,148 for reference in the logs | 04:43 |
| *** bobh has joined #openstack-infra | 04:45 | |
| Jeffrey4l | ianw, hrm, what that means? how to fix it? | 04:46 |
| *** ijw has joined #openstack-infra | 04:47 | |
| Jeffrey4l | ianw, seems publish-openstack-sphinx-docs is a final jobs | 04:47 |
| Jeffrey4l | but publish-openstack-sphinx-docs-neutron try to change the require-projects variable for that jobs. | 04:47 |
| ianw | yeah | 04:47 |
| Jeffrey4l | guess it is not very easy to fix :( , | 04:48 |
| *** yamamoto has joined #openstack-infra | 04:49 | |
| *** bobh has quit IRC | 04:50 | |
| Jeffrey4l | is the openstack/neutron really required by this template? | 04:50 |
| *** hongbin has joined #openstack-infra | 04:51 | |
| ianw | I don't 100% know ... maybe not | 04:53 |
| *** udesale has joined #openstack-infra | 04:53 | |
| Jeffrey4l | btw, publish-openstack-sphinx-docs-horizon should have the same issue. | 04:54 |
| ianw | you know, I think I remember that final: wasn't working and maybe we've switched back, which has broken this | 04:54 |
| *** makowals has quit IRC | 04:54 | |
| Jeffrey4l | no idea for this :( | 04:56 |
| *** dhajare has joined #openstack-infra | 04:56 | |
| *** salv-orlando has joined #openstack-infra | 04:59 | |
| *** makowals has joined #openstack-infra | 05:01 | |
| Jeffrey4l | but, i think change the required-projects is useless. | 05:03 |
| Jeffrey4l | how about just remove this in the -neutron project? ianw | 05:04 |
| *** jascott1 has joined #openstack-infra | 05:04 | |
| ianw | Jeffery4l: yeah, just proposing that and we can get reviews, give me a sec :) | 05:04 |
| Jeffrey4l | cool. thanks. | 05:04 |
| *** salv-orlando has quit IRC | 05:05 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Remove neutron and horizion sphinx jobs https://review.openstack.org/523018 | 05:08 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Use only base publish-openstack-sphinx-docs job https://review.openstack.org/523019 | 05:08 |
| *** e0ne has joined #openstack-infra | 05:09 | |
| ianw | AJaeger -- i think you're probably one of the best to review ^ :) other solution being to un-finalise the job, i guess | 05:09 |
| *** e0ne has quit IRC | 05:14 | |
| *** e0ne has joined #openstack-infra | 05:16 | |
| *** jaosorior has joined #openstack-infra | 05:19 | |
| *** e0ne has quit IRC | 05:21 | |
| *** adisky_ has joined #openstack-infra | 05:29 | |
| *** yamamoto has quit IRC | 05:44 | |
| *** bobh has joined #openstack-infra | 05:45 | |
| *** hongbin has quit IRC | 05:46 | |
| *** dklyle has joined #openstack-infra | 05:47 | |
| *** yamamoto has joined #openstack-infra | 05:48 | |
| *** david-lyle has quit IRC | 05:48 | |
| *** makowals has quit IRC | 05:48 | |
| *** bobh has quit IRC | 05:49 | |
| *** udesale__ has joined #openstack-infra | 05:52 | |
| *** e0ne has joined #openstack-infra | 05:53 | |
| *** udesale has quit IRC | 05:53 | |
| *** udesale has joined #openstack-infra | 05:55 | |
| *** makowals has joined #openstack-infra | 05:56 | |
| *** udesale__ has quit IRC | 05:57 | |
| *** e0ne has quit IRC | 06:01 | |
| *** salv-orlando has joined #openstack-infra | 06:01 | |
| *** bobh has joined #openstack-infra | 06:02 | |
| openstackgerrit | Cao Xuan Hoang proposed openstack-infra/project-config master: Update neutron-vpnaas grafana dashboard https://review.openstack.org/522995 | 06:05 |
| *** e0ne has joined #openstack-infra | 06:05 | |
| *** bobh has quit IRC | 06:06 | |
| *** lihi has quit IRC | 06:06 | |
| *** salv-orlando has quit IRC | 06:07 | |
| *** lihi has joined #openstack-infra | 06:08 | |
| AJaeger | ianw: we had some discussion earlier last week about that as well. mordred 's work on topic:updated-pti will solve this as well - but there might be other jobs like these | 06:08 |
| Jeffrey4l | AJaeger, when this issue will be resolved. this block the kolla's jobs. if it takes long. i will push some workaround. | 06:09 |
| AJaeger | as soon as we switch over those docs jobs, they will fail completely right now ;( | 06:09 |
| *** e0ne has quit IRC | 06:10 | |
| AJaeger | Jeffrey4l: why does the *docs* job block your tarball? | 06:10 |
| Jeffrey4l | AJaeger, not the docs. it is the publish-branch-tarball. and the network-dynmaic-router project master tarball isn't update since 6 Oct. and there is a bug in the old tar file. | 06:11 |
| Jeffrey4l | the docs jobs prevent the publish-branch-tarball jobs. | 06:11 |
| AJaeger | But if we merge this change, the docs job will fail - and those are voting jobs in the gate. Then nothing can merge for any of these projects including network-dynamic-router ;( | 06:12 |
| Jeffrey4l | OK. i will push a workaround in kolla, and make the gate green. | 06:13 |
| Jeffrey4l | btw, if there any bug track such issue? AJaeger | 06:13 |
| AJaeger | ianw, Jeffrey4l, I see two options: get mordred's changes in and then merge the two changes by ianw - that's the best way forward. | 06:14 |
| AJaeger | for short-term, we could remove the final from the job. | 06:14 |
| Jeffrey4l | AJaeger, please if it is possible ;D | 06:15 |
| AJaeger | but I don't know the implications of that one, so will not review that without discussion with jeblair | 06:15 |
| AJaeger | infra-root, can we figure out a way forward on this one today, please? ^ | 06:15 |
| *** janki has joined #openstack-infra | 06:15 | |
| AJaeger | Jeffrey4l: https://etherpad.openstack.org/p/zuulv3-issues - please add a few lines to it | 06:16 |
| *** janki has quit IRC | 06:16 | |
| AJaeger | but short-term fix would be to change the post jobs - let me propose this | 06:16 |
| *** janki has joined #openstack-infra | 06:16 | |
| Jeffrey4l | AJaeger, which sections should i put? "Issue with jobs" or Issue with zuul? | 06:17 |
| *** dsariel has joined #openstack-infra | 06:18 | |
| AJaeger | with jobs | 06:18 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove required-jobs from final docs job https://review.openstack.org/523023 | 06:18 |
| AJaeger | ianw, config-core, infra-root, what do you think of this short-term hack? ^ | 06:18 |
| AJaeger | Jeffrey4l: ^ | 06:20 |
| AJaeger | config-core, please review https://review.openstack.org/522927 to get translations again on our health board | 06:21 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Reorg non detailed instance listing columns https://review.openstack.org/522103 | 06:27 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove required-jobs from final docs job https://review.openstack.org/523023 | 06:28 |
| *** makowals has quit IRC | 06:29 | |
| *** jbadiapa has joined #openstack-infra | 06:30 | |
| *** bobh has joined #openstack-infra | 06:32 | |
| *** d0ugal has quit IRC | 06:35 | |
| *** bobh has quit IRC | 06:38 | |
| *** annp has joined #openstack-infra | 06:39 | |
| Jeffrey4l | infra-root seems tarballs.o.o is down | 06:39 |
| *** kiennt26 has joined #openstack-infra | 06:41 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Improve test case node_assignment_at_quota https://review.openstack.org/506134 | 06:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add cloud quota handling https://review.openstack.org/503838 | 06:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Don't fail on quota exceeded https://review.openstack.org/503051 | 06:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Make max-servers optional https://review.openstack.org/504282 | 06:42 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support cores limit per pool https://review.openstack.org/504283 | 06:42 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support ram limit per pool https://review.openstack.org/504284 | 06:42 |
| *** bobh has joined #openstack-infra | 06:42 | |
| *** spectr has joined #openstack-infra | 06:44 | |
| *** bobh has quit IRC | 06:46 | |
| *** namnh has joined #openstack-infra | 06:47 | |
| *** makowals has joined #openstack-infra | 06:47 | |
| *** rcernin has quit IRC | 06:47 | |
| *** d0ugal has joined #openstack-infra | 06:48 | |
| *** rcernin has joined #openstack-infra | 06:49 | |
| *** spectr has quit IRC | 06:55 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Reorg non detailed instance listing columns https://review.openstack.org/522103 | 06:59 |
| ianw | Jeffrey4l: i'm not seeing issues with tarballs.o.o ... transient issue? | 07:03 |
| *** salv-orlando has joined #openstack-infra | 07:03 | |
| *** bobh has joined #openstack-infra | 07:03 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: POC: logs collection from native zuulv3 job https://review.openstack.org/522554 | 07:04 |
| Jeffrey4l | it seems OK right now. tarballs.o.o return very slow just now. | 07:06 |
| *** bobh has quit IRC | 07:07 | |
| *** salv-orlando has quit IRC | 07:09 | |
| *** bobh has joined #openstack-infra | 07:12 | |
| *** Wei_Liu has quit IRC | 07:12 | |
| *** Wei_Liu has joined #openstack-infra | 07:12 | |
| *** gcb has quit IRC | 07:13 | |
| *** liujiong has quit IRC | 07:13 | |
| *** liujiong has joined #openstack-infra | 07:14 | |
| *** armaan has joined #openstack-infra | 07:16 | |
| *** bobh has quit IRC | 07:17 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Collect testrepository.subunit.gz for translations https://review.openstack.org/522927 | 07:18 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: translations: Use installed os-testr https://review.openstack.org/522928 | 07:18 |
| AJaeger | ianw: added README ^ | 07:18 |
| *** makowals has quit IRC | 07:20 | |
| *** florianf has joined #openstack-infra | 07:21 | |
| *** bobh has joined #openstack-infra | 07:22 | |
| openstackgerrit | Niraj Singh proposed openstack-infra/project-config master: Add masakari-dashboard project https://review.openstack.org/516550 | 07:24 |
| *** makowals has joined #openstack-infra | 07:26 | |
| *** bobh has quit IRC | 07:26 | |
| *** ykarel|afk is now known as ykarel | 07:27 | |
| *** rcernin has quit IRC | 07:28 | |
| *** Wei_Liu has quit IRC | 07:28 | |
| *** Wei_Liu has joined #openstack-infra | 07:29 | |
| *** ralonsoh has joined #openstack-infra | 07:30 | |
| *** mguiney has quit IRC | 07:31 | |
| *** andreas_s has joined #openstack-infra | 07:32 | |
| *** ccamacho has quit IRC | 07:32 | |
| *** ccamacho has joined #openstack-infra | 07:32 | |
| *** bobh has joined #openstack-infra | 07:33 | |
| *** salv-orlando has joined #openstack-infra | 07:36 | |
| *** salv-orlando has quit IRC | 07:37 | |
| *** salv-orlando has joined #openstack-infra | 07:37 | |
| *** ykarel is now known as ykarel|lunch | 07:37 | |
| *** bobh has quit IRC | 07:38 | |
| *** bobh has joined #openstack-infra | 07:45 | |
| *** guinm has joined #openstack-infra | 07:46 | |
| *** guinm is now known as valkyrie_ | 07:47 | |
| *** jtomasek has joined #openstack-infra | 07:47 | |
| *** valkyrie_ is now known as valkyrja | 07:47 | |
| *** valkyrja has quit IRC | 07:47 | |
| *** guinm has joined #openstack-infra | 07:48 | |
| *** vtapia has joined #openstack-infra | 07:48 | |
| *** bobh has quit IRC | 07:50 | |
| *** guinm is now known as help | 07:52 | |
| *** help is now known as Guest61083 | 07:53 | |
| *** bobh has joined #openstack-infra | 07:53 | |
| *** Guest61083 is now known as guinm | 07:53 | |
| *** martinkopec has joined #openstack-infra | 07:54 | |
| *** bobh has quit IRC | 07:57 | |
| *** makowals has quit IRC | 08:00 | |
| *** bobh has joined #openstack-infra | 08:02 | |
| *** makowals has joined #openstack-infra | 08:03 | |
| *** guinm has quit IRC | 08:03 | |
| *** hashar has joined #openstack-infra | 08:04 | |
| *** yamamoto has quit IRC | 08:04 | |
| *** hashar has quit IRC | 08:04 | |
| *** hashar has joined #openstack-infra | 08:04 | |
| *** coolsvap has joined #openstack-infra | 08:04 | |
| *** e0ne has joined #openstack-infra | 08:06 | |
| *** bobh has quit IRC | 08:06 | |
| *** guinm has joined #openstack-infra | 08:10 | |
| *** flaper87 has quit IRC | 08:11 | |
| *** pcaruana has joined #openstack-infra | 08:11 | |
| *** gcb has joined #openstack-infra | 08:12 | |
| *** flaper87 has joined #openstack-infra | 08:13 | |
| *** bobh has joined #openstack-infra | 08:13 | |
| *** Hal has joined #openstack-infra | 08:17 | |
| *** AJaeger has quit IRC | 08:18 | |
| *** bobh has quit IRC | 08:18 | |
| *** armaan_ has joined #openstack-infra | 08:20 | |
| *** slaweq has joined #openstack-infra | 08:24 | |
| *** armaan has quit IRC | 08:24 | |
| *** armaan has joined #openstack-infra | 08:24 | |
| *** armaan_ has quit IRC | 08:27 | |
| *** sree_ has joined #openstack-infra | 08:29 | |
| *** amoralej|off is now known as amoralej | 08:29 | |
| *** sree_ is now known as Guest13269 | 08:29 | |
| *** ykarel|lunch is now known as ykarel | 08:31 | |
| *** sree has quit IRC | 08:31 | |
| *** andreas_s has quit IRC | 08:33 | |
| *** yamamoto has joined #openstack-infra | 08:33 | |
| *** andreas_s has joined #openstack-infra | 08:33 | |
| *** jpena|off is now known as jpena | 08:38 | |
| *** andreas_s has quit IRC | 08:42 | |
| *** andreas_s has joined #openstack-infra | 08:43 | |
| *** andreas_s has quit IRC | 08:45 | |
| *** andreas_s has joined #openstack-infra | 08:46 | |
| *** bobh has joined #openstack-infra | 08:50 | |
| *** yamamoto has quit IRC | 08:51 | |
| *** bobh has quit IRC | 08:54 | |
| *** AJaeger has joined #openstack-infra | 08:55 | |
| *** bobh has joined #openstack-infra | 08:55 | |
| *** jpich has joined #openstack-infra | 08:56 | |
| *** rossella_s has joined #openstack-infra | 08:57 | |
| *** bobh has quit IRC | 08:59 | |
| *** sree has joined #openstack-infra | 09:00 | |
| *** Guest13269 has quit IRC | 09:01 | |
| *** bhavik1 has quit IRC | 09:04 | |
| *** bobh has joined #openstack-infra | 09:08 | |
| *** jchhatbar has joined #openstack-infra | 09:12 | |
| *** janki has quit IRC | 09:13 | |
| *** gmann is now known as gmann_afk | 09:13 | |
| *** bobh has quit IRC | 09:13 | |
| *** bobh has joined #openstack-infra | 09:13 | |
| *** bobh has quit IRC | 09:18 | |
| *** Liced has joined #openstack-infra | 09:28 | |
| *** derekh has joined #openstack-infra | 09:28 | |
| *** yamamoto has joined #openstack-infra | 09:31 | |
| *** bobh has joined #openstack-infra | 09:33 | |
| *** dbecker has joined #openstack-infra | 09:34 | |
| *** rpittau has joined #openstack-infra | 09:35 | |
| *** spectr has joined #openstack-infra | 09:37 | |
| *** spectr has quit IRC | 09:37 | |
| *** bobh has quit IRC | 09:37 | |
| *** yamamoto has quit IRC | 09:39 | |
| *** claudiub|2 has joined #openstack-infra | 09:44 | |
| *** bobh has joined #openstack-infra | 09:48 | |
| *** bobh has quit IRC | 09:53 | |
| *** shardy has joined #openstack-infra | 09:53 | |
| *** annp has quit IRC | 10:03 | |
| *** ociuhandu has joined #openstack-infra | 10:03 | |
| *** sree has quit IRC | 10:03 | |
| *** ociuhandu has quit IRC | 10:04 | |
| *** sree has joined #openstack-infra | 10:04 | |
| *** liujiong has quit IRC | 10:06 | |
| *** sdague has joined #openstack-infra | 10:07 | |
| *** owalsh is now known as owalsh_afk | 10:08 | |
| *** dtantsur|afk is now known as dtantsur | 10:08 | |
| *** sree has quit IRC | 10:08 | |
| *** andreas_s has quit IRC | 10:09 | |
| *** andreas_s has joined #openstack-infra | 10:10 | |
| *** stakeda has quit IRC | 10:10 | |
| *** ramishra has quit IRC | 10:10 | |
| *** ramishra has joined #openstack-infra | 10:12 | |
| *** dsariel has quit IRC | 10:13 | |
| sshnaidm | pabelanger, clarkb fixed previous issue, still post failure and no logs: http://logs.openstack.org/54/522554/5/check/tripleo-ci-centos-7-3nodes-multinode/d5cb089/job-output.txt.gz#_2017-11-27_08_59_18_313452 | 10:14 |
| *** hjensas has joined #openstack-infra | 10:16 | |
| *** sambetts_ has joined #openstack-infra | 10:16 | |
| *** jascott1 has quit IRC | 10:16 | |
| *** bobh has joined #openstack-infra | 10:17 | |
| *** andreas_s has quit IRC | 10:18 | |
| *** cuongnv has quit IRC | 10:19 | |
| *** ramishra has quit IRC | 10:19 | |
| *** ramishra has joined #openstack-infra | 10:20 | |
| *** bobh has quit IRC | 10:21 | |
| *** spectr has joined #openstack-infra | 10:22 | |
| *** spectr has quit IRC | 10:23 | |
| *** bobh has joined #openstack-infra | 10:23 | |
| *** wolverineav has joined #openstack-infra | 10:25 | |
| *** namnh has quit IRC | 10:25 | |
| *** yamamoto has joined #openstack-infra | 10:27 | |
| *** bobh has quit IRC | 10:27 | |
| *** andreas_s has joined #openstack-infra | 10:29 | |
| andreykurilin | hi folks! I have an issue with zuul-cloner. It fails at one patch(only in one job). Does anyone can help me with investigation the issue? | 10:30 |
| *** yamamoto has quit IRC | 10:32 | |
| *** bobh has joined #openstack-infra | 10:34 | |
| *** bobh has quit IRC | 10:39 | |
| *** andreas_s has quit IRC | 10:48 | |
| *** andreas_s has joined #openstack-infra | 10:48 | |
| *** yamamoto has joined #openstack-infra | 10:53 | |
| *** bobh has joined #openstack-infra | 10:53 | |
| *** ldnunes has joined #openstack-infra | 10:54 | |
| *** yamamoto has quit IRC | 10:55 | |
| *** pcaruana has quit IRC | 10:55 | |
| *** electrofelix has joined #openstack-infra | 10:55 | |
| *** sambetts_ is now known as sambetts | 10:57 | |
| *** bobh has quit IRC | 10:57 | |
| *** pcaruana has joined #openstack-infra | 10:59 | |
| *** Hal has quit IRC | 11:01 | |
| *** yamamoto has joined #openstack-infra | 11:01 | |
| *** ianw has quit IRC | 11:03 | |
| *** ianw_ has joined #openstack-infra | 11:03 | |
| *** ianw_ is now known as ianw | 11:04 | |
| *** bobh has joined #openstack-infra | 11:05 | |
| *** yamamoto has quit IRC | 11:06 | |
| *** andreas_s has quit IRC | 11:07 | |
| *** finucannot is now known as stephenfin | 11:07 | |
| *** andreas_s has joined #openstack-infra | 11:07 | |
| *** udesale has quit IRC | 11:07 | |
| *** bobh has quit IRC | 11:10 | |
| *** rfolco has joined #openstack-infra | 11:11 | |
| *** bobh has joined #openstack-infra | 11:12 | |
| *** andreas_s has quit IRC | 11:16 | |
| *** bobh has quit IRC | 11:17 | |
| *** alexchadin has joined #openstack-infra | 11:19 | |
| *** ccamacho is now known as ccamacho|doc | 11:22 | |
| *** andreas_s has joined #openstack-infra | 11:27 | |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config master: Ensure checkout overrides are correctly set for networking-cisco https://review.openstack.org/512588 | 11:28 |
| *** sree has joined #openstack-infra | 11:28 | |
| *** andreas_s has quit IRC | 11:28 | |
| *** andreas_s has joined #openstack-infra | 11:29 | |
| *** dsariel has joined #openstack-infra | 11:29 | |
| *** Hal has joined #openstack-infra | 11:30 | |
| *** yamamoto has joined #openstack-infra | 11:31 | |
| *** sree has quit IRC | 11:33 | |
| *** andreas_s has quit IRC | 11:34 | |
| *** spectr has joined #openstack-infra | 11:35 | |
| *** spectr has quit IRC | 11:35 | |
| *** yamamoto has quit IRC | 11:36 | |
| *** alexchadin has quit IRC | 11:40 | |
| *** alexchadin has joined #openstack-infra | 11:41 | |
| *** Kevin_Zheng has quit IRC | 11:41 | |
| *** threestrands_ has quit IRC | 11:42 | |
| *** erlon has joined #openstack-infra | 11:42 | |
| *** sree has joined #openstack-infra | 11:44 | |
| *** gcb has quit IRC | 11:45 | |
| *** bobh has joined #openstack-infra | 11:46 | |
| *** bobh has quit IRC | 11:51 | |
| *** alexchadin has quit IRC | 11:51 | |
| *** alexchadin has joined #openstack-infra | 11:51 | |
| openstackgerrit | Tobias Brox proposed openstack-infra/shade master: bugfix for https://storyboard.openstack.org/#\!/story/2001302 - with unit test modified to pass https://review.openstack.org/523095 | 11:53 |
| *** icey has quit IRC | 11:53 | |
| *** RuiChen has quit IRC | 11:53 | |
| *** ajo has quit IRC | 11:53 | |
| *** patrickeast has quit IRC | 11:53 | |
| *** kong has quit IRC | 11:53 | |
| *** samueldmq has quit IRC | 11:53 | |
| *** ying_zuo has quit IRC | 11:53 | |
| *** fkautz has quit IRC | 11:53 | |
| *** zhenguo has quit IRC | 11:53 | |
| *** electrical has quit IRC | 11:53 | |
| *** larainema has quit IRC | 11:54 | |
| *** chrisyang_0660 has quit IRC | 11:54 | |
| *** kmalloc has quit IRC | 11:54 | |
| *** vdrok has quit IRC | 11:54 | |
| *** hrybacki has quit IRC | 11:54 | |
| *** berendt has quit IRC | 11:54 | |
| *** csatari has quit IRC | 11:54 | |
| *** jbryce has quit IRC | 11:54 | |
| *** ildikov has quit IRC | 11:54 | |
| *** andrewbogott has quit IRC | 11:54 | |
| *** bobh has joined #openstack-infra | 11:54 | |
| *** kklimonda has quit IRC | 11:54 | |
| *** zaro has quit IRC | 11:54 | |
| *** xgerman_ has quit IRC | 11:54 | |
| *** fmccrthy has quit IRC | 11:54 | |
| *** davidlenwell has quit IRC | 11:55 | |
| *** icey has joined #openstack-infra | 11:55 | |
| *** chrisyang_0660 has joined #openstack-infra | 11:55 | |
| *** zhenguo has joined #openstack-infra | 11:55 | |
| *** zaro has joined #openstack-infra | 11:55 | |
| *** berendt has joined #openstack-infra | 11:55 | |
| *** fkautz has joined #openstack-infra | 11:55 | |
| *** samueldmq has joined #openstack-infra | 11:55 | |
| *** kong has joined #openstack-infra | 11:55 | |
| *** electrical has joined #openstack-infra | 11:55 | |
| *** hrybacki has joined #openstack-infra | 11:55 | |
| *** patrickeast has joined #openstack-infra | 11:55 | |
| *** kklimonda has joined #openstack-infra | 11:56 | |
| *** xgerman_ has joined #openstack-infra | 11:56 | |
| *** andrewbogott has joined #openstack-infra | 11:56 | |
| *** fmccrthy has joined #openstack-infra | 11:56 | |
| *** vdrok has joined #openstack-infra | 11:56 | |
| *** davidlenwell has joined #openstack-infra | 11:56 | |
| *** cloudnull has quit IRC | 11:56 | |
| *** kmalloc has joined #openstack-infra | 11:56 | |
| *** ildikov has joined #openstack-infra | 11:56 | |
| *** csatari has joined #openstack-infra | 11:56 | |
| *** bobh has quit IRC | 11:58 | |
| *** cloudnull has joined #openstack-infra | 11:59 | |
| *** tosky has joined #openstack-infra | 11:59 | |
| *** yamamoto has joined #openstack-infra | 12:01 | |
| *** bobh has joined #openstack-infra | 12:02 | |
| *** LindaWang1 has joined #openstack-infra | 12:03 | |
| openstackgerrit | Tobias Brox proposed openstack-infra/shade master: bugfix for https://storyboard.openstack.org/#!/story/2001302 - with unit test modified to pass https://review.openstack.org/523095 | 12:05 |
| openstackgerrit | Alex Kavanagh proposed openstack-infra/project-config master: Modify charm-ceph-mon to move project jobs to repo https://review.openstack.org/523098 | 12:05 |
| *** yamamoto has quit IRC | 12:06 | |
| *** bobh has quit IRC | 12:08 | |
| *** dave-mccowan has joined #openstack-infra | 12:08 | |
| *** alexchadin has quit IRC | 12:09 | |
| *** dave-mcc_ has joined #openstack-infra | 12:09 | |
| *** alexchadin has joined #openstack-infra | 12:10 | |
| *** smatzek has joined #openstack-infra | 12:12 | |
| *** dave-mccowan has quit IRC | 12:12 | |
| *** sree has quit IRC | 12:13 | |
| *** bobh has joined #openstack-infra | 12:13 | |
| *** jkilpatr has joined #openstack-infra | 12:13 | |
| *** adarazs|ruck is now known as adarazs|ruck|lnc | 12:14 | |
| *** jkilpatr has quit IRC | 12:14 | |
| *** jkilpatr has joined #openstack-infra | 12:14 | |
| *** bobh has quit IRC | 12:17 | |
| *** kjackal has joined #openstack-infra | 12:18 | |
| *** fried_turkey is now known as efried | 12:18 | |
| *** andreas_s has joined #openstack-infra | 12:22 | |
| *** szaher has quit IRC | 12:22 | |
| AJaeger | config-core, could you review https://review.openstack.org/522927 to get translations in OpenStack health status board, please? | 12:24 |
| openstackgerrit | Tobias Brox proposed openstack-infra/shade master: Allow grant_role to select users outside default domain https://review.openstack.org/523095 | 12:24 |
| *** giblet is now known as gibi | 12:24 | |
| *** szaher has joined #openstack-infra | 12:25 | |
| *** bobh has joined #openstack-infra | 12:27 | |
| *** udesale has joined #openstack-infra | 12:29 | |
| *** links has quit IRC | 12:29 | |
| *** bobh has quit IRC | 12:33 | |
| *** ccamacho|doc is now known as ccamacho | 12:33 | |
| openstackgerrit | Merged openstack-infra/project-config master: Collect testrepository.subunit.gz for translations https://review.openstack.org/522927 | 12:33 |
| openstackgerrit | Merged openstack-infra/project-config master: translations: Use installed os-testr https://review.openstack.org/522928 | 12:33 |
| AJaeger | thanks, frickler ! | 12:33 |
| *** alexchadin has quit IRC | 12:35 | |
| *** alexchadin has joined #openstack-infra | 12:36 | |
| *** bobh has joined #openstack-infra | 12:37 | |
| *** andreas_s has quit IRC | 12:38 | |
| *** andreas_s has joined #openstack-infra | 12:41 | |
| openstackgerrit | Tobias Brox proposed openstack-infra/shade master: Allow grant_role to select users outside default domain https://review.openstack.org/523095 | 12:41 |
| *** bobh has quit IRC | 12:42 | |
| *** amoralej is now known as amoralej|lunch | 12:42 | |
| *** links has joined #openstack-infra | 12:43 | |
| andreykurilin | `bin/bash: line 21: /usr/zuul-env/bin/zuul-cloner: No such file or directory` did anyone see this before? | 12:44 |
| AJaeger | andreykurilin: which job? | 12:45 |
| *** dhajare has quit IRC | 12:46 | |
| andreykurilin | AJaeger: rally-install-centos-7 . It was moved without modification to our repo and started failing several days ago. http://logs.openstack.org/21/512321/6/check/rally-install-centos-7/69f1208/ara/ | 12:46 |
| *** yamamoto has joined #openstack-infra | 12:47 | |
| andreykurilin | https://github.com/openstack/rally/blob/master/tests/ci/playbooks/rally-install/run.yaml#L33 | 12:47 |
| andreykurilin | I suppose it can somehow relate to https://github.com/openstack-infra/zuul-jobs/commit/6bb1f6046b692ee2038eafd0965bd3095d6fa787 ... | 12:47 |
| AJaeger | andreykurilin: hope somebody else can help you later on this - the links above should be sufficient | 12:49 |
| andreykurilin | AJaeger: thanks | 12:50 |
| *** yamamoto has quit IRC | 12:51 | |
| *** adarazs|ruck|lnc is now known as adarazs|ruck | 12:53 | |
| *** bobh has joined #openstack-infra | 12:54 | |
| *** jpena is now known as jpena|lunch | 12:55 | |
| *** lennyb__ has joined #openstack-infra | 12:56 | |
| *** bobh has quit IRC | 12:59 | |
| frickler | andreykurilin: iiuc zuul-cloner is no longer to be used for non-legacy jobs, see https://docs.openstack.org/infra/manual/zuulv3.html#converting-other-legacy-changes . | 12:59 |
| frickler | andreykurilin: the difference between the original converted jobs and the ones you moved into your repo seems to be that you no longer use legacy-base as parent | 13:00 |
| *** dtantsur is now known as dtantsur|bbl | 13:03 | |
| *** owalsh_afk is now known as owalsh | 13:03 | |
| *** rosmaita has joined #openstack-infra | 13:03 | |
| *** dprince has joined #openstack-infra | 13:05 | |
| *** trown|outtypewww is now known as trown | 13:05 | |
| *** amoralej|lunch is now known as amoralej | 13:06 | |
| *** raissa has joined #openstack-infra | 13:06 | |
| *** jkilpatr has quit IRC | 13:11 | |
| *** rhallisey has joined #openstack-infra | 13:11 | |
| *** ramishra has quit IRC | 13:14 | |
| *** ramishra has joined #openstack-infra | 13:15 | |
| *** markvoelker has quit IRC | 13:16 | |
| *** markvoelker has joined #openstack-infra | 13:16 | |
| *** ykarel has quit IRC | 13:19 | |
| *** armaan has quit IRC | 13:20 | |
| *** ramishra has quit IRC | 13:20 | |
| *** ganso has joined #openstack-infra | 13:22 | |
| *** jkilpatr has joined #openstack-infra | 13:29 | |
| *** gcb has joined #openstack-infra | 13:31 | |
| *** rlandy has joined #openstack-infra | 13:31 | |
| *** ldnunes has quit IRC | 13:33 | |
| *** makowals has quit IRC | 13:34 | |
| *** rlandy is now known as rlandy|rover | 13:34 | |
| *** eharney has joined #openstack-infra | 13:34 | |
| openstackgerrit | boden proposed openstack-infra/project-config master: periodic py jobs for vmware-nsx ocata and pike https://review.openstack.org/521538 | 13:35 |
| *** makowals has joined #openstack-infra | 13:37 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/project-config master: Add tripleo-ipsec project https://review.openstack.org/523124 | 13:37 |
| *** daidv has quit IRC | 13:38 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/project-config master: Add tripleo-ipsec project https://review.openstack.org/523124 | 13:40 |
| *** kgiusti has joined #openstack-infra | 13:41 | |
| Liced | AJeaeger : Hi. Have you found something for bgpvpn translation issue? | 13:41 |
| Liced | AJaeger : Hi. Have you found something for bgpvpn translation issue? | 13:42 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/project-config master: Add tripleo-ipsec to zuul.d/projects.yaml https://review.openstack.org/523125 | 13:44 |
| *** yamamoto has joined #openstack-infra | 13:45 | |
| *** yamamoto has quit IRC | 13:45 | |
| *** ldnunes has joined #openstack-infra | 13:45 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/project-config master: Add tripleo-ipsec project https://review.openstack.org/523124 | 13:48 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/project-config master: Add tripleo-ipsec to zuul.d/projects.yaml https://review.openstack.org/523125 | 13:48 |
| *** yamamoto has joined #openstack-infra | 13:49 | |
| *** jpena|lunch is now known as jpena | 13:49 | |
| *** armaan has joined #openstack-infra | 13:50 | |
| AJaeger | Liced: yes, I'm closer - we have a problem with our post jobs and it looks like if one is misconfigured, then all fail. I have a patch up to unblock this, see https://review.openstack.org/523023 . Once that is in, I hope publishing works again for bgpvpn. | 13:50 |
| AJaeger | Liced: sorry for all this - and thanks for pinging. This is a fallout from the switch to Zuul v3 and we need to get this fixed. | 13:50 |
| *** jbryce has joined #openstack-infra | 13:51 | |
| *** ajo has joined #openstack-infra | 13:51 | |
| *** larainema has joined #openstack-infra | 13:51 | |
| *** ying_zuo has joined #openstack-infra | 13:51 | |
| AJaeger | yeah, the translation jobs show up on the health dashboard again! | 13:52 |
| *** RuiChen has joined #openstack-infra | 13:52 | |
| *** haleyb|away is now known as haleyb | 13:53 | |
| Liced | AJaeger: Okay. Thank you. | 13:53 |
| *** mriedem has joined #openstack-infra | 13:54 | |
| *** coolsvap has quit IRC | 13:54 | |
| *** jaypipes has joined #openstack-infra | 13:56 | |
| *** yamamoto has quit IRC | 13:57 | |
| *** apetrich has quit IRC | 13:58 | |
| *** apetrich has joined #openstack-infra | 13:59 | |
| *** alexchadin has quit IRC | 14:10 | |
| *** hjensas has quit IRC | 14:13 | |
| *** sree has joined #openstack-infra | 14:15 | |
| *** smatzek has quit IRC | 14:18 | |
| *** smatzek has joined #openstack-infra | 14:19 | |
| mordred | morning AJaeger | 14:19 |
| *** dhill_ has joined #openstack-infra | 14:19 | |
| *** sree has quit IRC | 14:19 | |
| AJaeger | mordred: good morning! | 14:20 |
| *** sree has joined #openstack-infra | 14:21 | |
| sshnaidm | pabelanger, clarkb, I fixed previous issue, but still post failure and no logs: http://logs.openstack.org/54/522554/5/check/tripleo-ci-centos-7-3nodes-multinode/d5cb089/job-output.txt.gz#_2017-11-27_08_59_18_313452 | 14:21 |
| AJaeger | mordred, could you review my discussion from earlier today (around 6 UTC) with ianw and Jeffrey4l, please? And then advise whether we should go forward with https://review.openstack.org/523023 as temporary step. We need your updated-pti stack in quickly but I don't know what's still missing ;( | 14:21 |
| openstackgerrit | Trygve Vea proposed openstack-infra/shade master: Implement availability_zone_hints for networks and routers. https://review.openstack.org/522004 | 14:24 |
| *** smatzek has quit IRC | 14:24 | |
| *** dtantsur|bbl is now known as dtantsur | 14:24 | |
| openstackgerrit | Trygve Vea proposed openstack-infra/shade master: Implement availability_zone_hints for networks and routers. https://review.openstack.org/522004 | 14:24 |
| *** ramishra has joined #openstack-infra | 14:27 | |
| *** makowals has quit IRC | 14:28 | |
| *** makowals has joined #openstack-infra | 14:30 | |
| mordred | AJaeger: yes - on it | 14:31 |
| *** camunoz has joined #openstack-infra | 14:31 | |
| *** links has quit IRC | 14:32 | |
| *** esberglu has joined #openstack-infra | 14:33 | |
| *** gcb has quit IRC | 14:33 | |
| mordred | AJaeger: yah - I think we should do that patch as a temporary step - then hopefully we can get both the sphinx job patches AND the reworked base jobs done this week | 14:36 |
| *** shardy is now known as shardy_afk | 14:37 | |
| *** udesale has quit IRC | 14:42 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove required-jobs from final docs job https://review.openstack.org/523023 | 14:42 |
| *** LindaWang1 has quit IRC | 14:43 | |
| *** jascott1 has joined #openstack-infra | 14:45 | |
| Jeffrey4l | ianw, patch is merged ^^ could you trigger the network-dynamic-router post jobs again? | 14:46 |
| *** jascott1 has quit IRC | 14:50 | |
| *** pblaho has joined #openstack-infra | 14:50 | |
| AJaeger | Jeffrey4l: ianw is sleeping now - he's in Australia | 14:50 |
| Jeffrey4l | ah, ok could you help me. AJaeger | 14:50 |
| Jeffrey4l | command like: zuul enqueue-ref --tenant openstack --trigger gerrit --pipeline post --project openstack/neutron-dynamic-routing --ref refs/heads/master --oldrev 183c3fa4840d22be1974534eb9e1b28b552f4a42 --newrev 78ae7cb8f5bcecaf4cb17577e38401b7d8628f20 | 14:50 |
| * AJaeger doesn't have the permissions ;( | 14:50 | |
| Jeffrey4l | ok. | 14:51 |
| Jeffrey4l | thanks.. | 14:51 |
| openstackgerrit | Merged openstack-infra/shade master: Switch baremetal nics/ports tests over https://review.openstack.org/522989 | 14:55 |
| AJaeger | mordred: could you enque the above - and do something similar for networking-bpgpvn as well, please? Let me create one for you... | 14:56 |
| AJaeger | zuul enqueue-ref --tenant openstack --trigger gerrit --pipeline post --project openstack/networking-bgpvpn --ref refs/heads/master --oldrev fea7bba0f11bb31806ef8c112a16903d04681d38 --newrev dd08171669b37caab0ff787dffe74c281551b93e | 14:57 |
| *** salv-orlando has quit IRC | 14:57 | |
| *** yamamoto has joined #openstack-infra | 14:57 | |
| *** salv-orlando has joined #openstack-infra | 14:58 | |
| *** armaan has quit IRC | 14:58 | |
| *** yamahata has quit IRC | 14:59 | |
| *** yamahata has joined #openstack-infra | 15:00 | |
| mordred | AJaeger, Jeffrey4l one sec | 15:00 |
| mordred | AJaeger: both done | 15:01 |
| Jeffrey4l | thanks a lot. | 15:02 |
| *** ihrachys has joined #openstack-infra | 15:02 | |
| *** salv-orlando has quit IRC | 15:03 | |
| AJaeger | thanks, mordred | 15:03 |
| AJaeger | mordred: I only see the dynamic-routing in post. Any error logs? | 15:03 |
| *** yamamoto has quit IRC | 15:04 | |
| AJaeger | I know why - patch coming... | 15:05 |
| Jeffrey4l | some jobs have been the queue more than 5 hours. something is wrong? | 15:05 |
| *** iyamahat has joined #openstack-infra | 15:05 | |
| *** hamzy has quit IRC | 15:06 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove required-projects from post docs jobs https://review.openstack.org/523144 | 15:09 |
| AJaeger | mordred: we need this as well ^ | 15:09 |
| fungi | i'm having trouble ssh'ing into ze05... wonder if we lost it | 15:09 |
| fungi | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=580 suggests it died a little over 5 hours ago | 15:10 |
| fungi | i'll check the oob console | 15:10 |
| openstackgerrit | Trygve Vea proposed openstack-infra/shade master: Implement availability_zone_hints for networks and routers. https://review.openstack.org/522004 | 15:11 |
| fungi | Jeffrey4l: thanks for the heads up. i expect those changes had jobs running from ze05 when it suddenly became unresponsive | 15:11 |
| *** bobh has joined #openstack-infra | 15:11 | |
| *** marst has joined #openstack-infra | 15:12 | |
| *** andreas_s has quit IRC | 15:12 | |
| *** bnemec has quit IRC | 15:13 | |
| openstackgerrit | Merged openstack-infra/shade master: Allow grant_role to select users outside default domain https://review.openstack.org/523095 | 15:13 |
| *** felipemonteiro has joined #openstack-infra | 15:14 | |
| *** felipemonteiro_ has joined #openstack-infra | 15:16 | |
| fungi | oh boy, lots of | 15:16 |
| fungi | memory consumption | 15:17 |
| AJaeger | ;( | 15:17 |
| fungi | oom galore | 15:17 |
| *** andreas_s has joined #openstack-infra | 15:17 | |
| AJaeger | Jeffrey4l: http://tarballs.openstack.org/neutron-dynamic-routing/?C=M;O=D | 15:17 |
| fungi | zuul-executor got sniped by the oom killer | 15:17 |
| *** mat128 has joined #openstack-infra | 15:17 | |
| fungi | oh, that was a long time ago | 15:17 |
| AJaeger | mordred, ianw, so that worked | 15:18 |
| Jeffrey4l | AJaeger, yep. i see. thanks | 15:18 |
| fungi | more recently, just lots of messages about hung tasks | 15:18 |
| *** rbrndt has joined #openstack-infra | 15:18 | |
| fungi | and the console getty isn't responding to carriage return | 15:18 |
| Shrews | fungi: ooh, i wonder if that's been the cause of our finger daemon dying | 15:18 |
| AJaeger | Now, config-core, please help getting https://review.openstack.org/523144 in to allow other post jobs to run again for these three repos | 15:18 |
| *** smatzek has joined #openstack-infra | 15:18 | |
| Shrews | fungi: the finger daemon will appear as a zuul-executor process since it is forked off the main process | 15:19 |
| fungi | unfortunately the kernel messages are all timestamped in seconds since boot, so i don't know whether any of this corresponds to the server suddenly stopping | 15:19 |
| *** felipemonteiro has quit IRC | 15:20 | |
| *** mat128 has quit IRC | 15:20 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Remove tripleo-buildimage-overcloud-full-centos-7 from gate https://review.openstack.org/523146 | 15:20 |
| fungi | there's a 58-day gap between the oom messages and the ones about hung kernel tasks | 15:20 |
| fungi | so i'm guessing the oom situation was unrelated to the present hang | 15:20 |
| fungi | i'll go ahead and nova reboot this | 15:20 |
| openstackgerrit | Alex Kavanagh proposed openstack-infra/project-config master: Modify charm-ceph-mon to make py27 a voting job https://review.openstack.org/523098 | 15:21 |
| *** andreas_s has quit IRC | 15:21 | |
| *** smatzek has quit IRC | 15:21 | |
| *** smatzek has joined #openstack-infra | 15:21 | |
| *** rkukura has quit IRC | 15:25 | |
| fungi | the regular (soft) reboot isn't working, gonna have to try --hard | 15:27 |
| *** mat128 has joined #openstack-infra | 15:28 | |
| dmsimard | fungi: dmesg -T for real dates :) | 15:28 |
| fungi | dmsimard: yep, too bad you can't do that on a hung vm | 15:28 |
| dmsimard | fungi: oh, you were looking at oob console -- I wasn't that far yet in reading the backlog | 15:29 |
| fungi | this was just the kmesg entries on the console from before it ceased responding, yep | 15:29 |
| dmsimard | fungi: pabelanger mentioned some executor issues last friday, he did a #status about it | 15:29 |
| fungi | dmsimard: right, the one on friday (ze03) turns out to have been a hung xen host which rackspace eventually rebooted, according to trouble tickets i found a few minutes ago | 15:30 |
| fungi | the zuul-executor processes have started at boot on ze05 now, so let's see if the hung jobs get retried correctly now | 15:31 |
| fungi | #status log Hard rebooted ze05.openstack.org after it was found hung (unresponsive even at console, determination of cause inconclusive, no smoking gun in kmesg entries) | 15:33 |
| openstackstatus | fungi: finished logging | 15:33 |
| *** fultonj has joined #openstack-infra | 15:33 | |
| *** hongbin has joined #openstack-infra | 15:35 | |
| openstackgerrit | Sam Betts proposed openstack-infra/devstack-gate master: Add fallback to -eol tag if stable branch doesn't exists https://review.openstack.org/512245 | 15:36 |
| AJaeger | fungi, could you +2A https://review.openstack.org/523144, please? | 15:37 |
| *** sree has quit IRC | 15:37 | |
| AJaeger | thanks, fungi | 15:40 |
| *** jamesmcarthur has joined #openstack-infra | 15:43 | |
| pabelanger | fungi: oh, interesting. So, there was an issue with ze03 last week | 15:43 |
| fungi | pabelanger: yep, see my comment about the ticket rax opened to let us know about a hung xen host for that one | 15:43 |
| fungi | this looked different | 15:44 |
| clarkb | fungi: did it have the kernel timeout messages in the console log? | 15:44 |
| clarkb | I think rax has been live migrating our instances and some of them (maybe all of them) have failed to do so properly and required a hard reboot | 15:44 |
| fungi | clarkb: just kernel warning messages about hung tasks not responding for n cycles | 15:44 |
| clarkb | I had to reboot es03 over the weekend and I think logstash-worker03 is in the same boat | 15:45 |
| clarkb | fungi: ya that | 15:45 |
| clarkb | and ianw had to hard reboot status.o.o | 15:45 |
| fungi | clarkb: right, i wouldn't be surprised if that were the cause for the es03 and ze05 hangs, but have no direct evidence nor tickets from the provider to corroborate | 15:45 |
| pabelanger | fungi: cool, I didn't check is server had be recently rebooted, I should have I guess | 15:46 |
| fungi | also because the kmesg entries on the console are timestamped in seconds since boot, it's hard to know whether those warnings were recent | 15:46 |
| *** andreas_s has joined #openstack-infra | 15:46 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove required-projects from post docs jobs https://review.openstack.org/523144 | 15:47 |
| clarkb | fungi: ya the onl time we got evidence from why this was happening was when ianw called when it happened to review.o.o aiui | 15:48 |
| AJaeger | infra-root, can you reenque the following, please: zuul enqueue-ref --tenant openstack --trigger gerrit --pipeline post --project openstack/networking-bgpvpn --ref refs/heads/master --oldrev fea7bba0f11bb31806ef8c112a16903d04681d38 --newrev dd08171669b37caab0ff787dffe74c281551b93e | 15:48 |
| fungi | seems likely to be the cause for the others (excepting ze03 on friday) | 15:48 |
| fungi | AJaeger: sure, doing that now | 15:48 |
| AJaeger | thanks! | 15:49 |
| clarkb | I think from rax's perspective the migrations were either in progress or successful so we never got the notification that something went wrong | 15:49 |
| fungi | AJaeger: done | 15:49 |
| fungi | clarkb: yep, unsurprising. we've seen the same pattern in the past when they've been doing mass forced live migrations | 15:50 |
| *** hamzy has joined #openstack-infra | 15:50 | |
| AJaeger | yeah, running now! | 15:50 |
| *** andreas_s has quit IRC | 15:50 | |
| clarkb | sshnaidm: reading that log I think it didn't find any hosts to run on | 15:53 |
| clarkb | sshnaidm: possible that primary and all:!localhost:!127.0.0.2 didn't match any nodes? | 15:53 |
| jeblair | inc0, Jeffrey4l: with 518797, do you want different secret values on different branches, or do you just want one value? could you just omit the secret definition from that change and use the one defined on master? | 15:53 |
| *** rkukura has joined #openstack-infra | 15:54 | |
| *** bnemec has joined #openstack-infra | 15:54 | |
| Jeffrey4l | jeblair, there is bug in zuul: can not re-define the same secret in different branch. and stable branch can not use the secrets from master branch. | 15:55 |
| Jeffrey4l | so this is workaround. | 15:55 |
| jeblair | Jeffrey4l: why can't stable use secrets from master? | 15:55 |
| *** ramishra has quit IRC | 15:56 | |
| clarkb | sshnaidm: I want to say that the collect logs role is meant to run on the executor (eg localhost) and copy from the other nodes? | 15:56 |
| Jeffrey4l | jeblair, i do not know the details. But check the patch 2. | 15:56 |
| clarkb | sshnaidm: so by running it elsewhere the desired effect isn't achieved | 15:56 |
| jeblair | Jeffrey4l: oh i think you mean PS3? | 15:56 |
| *** martinkopec has quit IRC | 15:56 | |
| Jeffrey4l | ah, right. jeblair | 15:56 |
| pabelanger | clarkb: sshnaidm: the zuul.executor patch isn't correct. It should be checked out into {{ zuul.executor.work_root }}/src/git.openstack.org/openstack-infra/tripleo-ci/ ... | 15:57 |
| pabelanger | path* | 15:57 |
| jeblair | Jeffrey4l: yeah, that looks like a bug. also, since secrets can be defined in-project, we do need to handle the case where they are defined on multiple branches, so that when you branch a project you don't create an invalid config. so 2 things need fixing here i think. | 15:58 |
| *** andrewbogott has quit IRC | 15:58 | |
| *** andrewbogott has joined #openstack-infra | 15:58 | |
| AJaeger | Liced: check https://translate.openstack.org/iteration/view/networking-bgpvpn/master/documents?dswid=9066 and http://tarballs.openstack.org/translation-source/networking-bgpvpn/master/ | 15:58 |
| *** slaweq has quit IRC | 15:58 | |
| *** salv-orlando has joined #openstack-infra | 15:58 | |
| AJaeger | Woot! first time build pot files for networking-bgpvpn! | 15:58 |
| *** slaweq has joined #openstack-infra | 15:59 | |
| *** slaweq has quit IRC | 15:59 | |
| *** slaweq has joined #openstack-infra | 16:00 | |
| *** slaweq has quit IRC | 16:00 | |
| *** makowals has quit IRC | 16:00 | |
| *** slaweq has joined #openstack-infra | 16:00 | |
| Liced | AJaeger: Yes !!! Thank you for your help | 16:02 |
| dmsimard | clarkb: btw re: configuring unbound minimum TTL | 16:03 |
| AJaeger | Liced: note that right now docs publishing is broken for bgpvpn, mordred is working on it. But the rest works now - so, 2 steps ahead but 1 still missing ;) | 16:03 |
| dmsimard | clarkb: I wanted to send a patch for that but I'd like to do it in zuul-jobs/openstack-zuul-jobs so that it's an easier revert than embedding it in the image. What do you think ? | 16:03 |
| *** jchhatbar has quit IRC | 16:03 | |
| clarkb | dmsimard: ya making it a job option is probably best and that way jobs that don't talk to many/any external resources can juts leave it as is | 16:04 |
| *** salv-orlando has quit IRC | 16:04 | |
| dmsimard | I don't anticipate any issues, especially if it's integration tested -- merging it in the nodepool image and hoping for the best scares me | 16:04 |
| *** slaweq has quit IRC | 16:05 | |
| AJaeger | config-core, a couple of cleanups on project-config for review sometime, please: https://review.openstack.org/522894 https://review.openstack.org/522895 https://review.openstack.org/522888 | 16:05 |
| clarkb | dmsimard: you don't have to just hope for the best, it is relatively easy to build images locally and check them | 16:06 |
| clarkb | dmsimard: thta was a big reason for moving away from snapshot images | 16:06 |
| *** shardy_afk is now known as shardy | 16:07 | |
| dmsimard | clarkb: nodepool images still aren't integration tested though, we've been bitten by that before | 16:07 |
| dmsimard | so I'm reluctant to touch them | 16:07 |
| *** makowals has joined #openstack-infra | 16:10 | |
| *** alex_xu has quit IRC | 16:11 | |
| pabelanger | dmsimard: clarkb: historically, it was in DIB because of ready-scripts, but now we might be able to use a job to configure the DNS. It really depends on how many DNS requests happen before the role is run to setup unbound | 16:13 |
| clarkb | pabelanger: we can't completely ues the job to do it imo or shouldn't because we know dns doesn't function in all cloud regions and I think we want working dns regardless of job config | 16:14 |
| dmsimard | pabelanger: I'm designing the role to support the installation and configuration so we can eventually switch 100% to it if we want | 16:14 |
| pabelanger | but, i agree with clarkb, you should be able to rebuild it locally and testing settings. Maybe over holidays, I'll hack on DIB build jobs with our elements | 16:14 |
| *** xinliang has quit IRC | 16:14 | |
| dmsimard | clarkb: surely the cloud provider dhcp provides working DNS.. | 16:14 |
| clarkb | dmsimard: they do not | 16:14 |
| dmsimard | wth ? | 16:14 |
| pabelanger | clarkb: yah, I think that might be better too | 16:14 |
| clarkb | dmsimard: because some clouds blacklist IPs then don't white list them again when the IP is recycled by another tenant | 16:14 |
| *** fultonj has quit IRC | 16:14 | |
| *** sree has joined #openstack-infra | 16:15 | |
| clarkb | dmsimard: so you get completely unworking dhcp assigned dns resolvers | 16:15 |
| clarkb | and this is why we have unbound baked into our images | 16:15 |
| dmsimard | eh | 16:15 |
| dmsimard | btw something I realized just now is that the default bogus-ttl for unbound is 60 seconds | 16:15 |
| dmsimard | so it means we're actually sending a bunch of DNS queries for the junk domain names projects have in their tests | 16:16 |
| dmsimard | which likely further doesn't help potential rate limiting | 16:16 |
| dmsimard | I'll leave it as is for the time being but something to keep in the back of our heads | 16:17 |
| *** andreas_s has joined #openstack-infra | 16:18 | |
| *** apetrich has quit IRC | 16:22 | |
| *** andreas_s has quit IRC | 16:23 | |
| *** apetrich has joined #openstack-infra | 16:24 | |
| mordred | clarkb: do you think it's worth considering running caching dns resolvers in each region like http mirrors? so that we configure unbound on each node to forward to our region-local resolver? | 16:25 |
| mordred | clarkb: we could potentially even just add that to the mirror servers | 16:25 |
| clarkb | mordred: the problem with that is the same as running open squid | 16:25 |
| mordred | clarkb: ah - right | 16:25 |
| clarkb | how do you keep the rest of the cloud/internet from using them inappropriately | 16:25 |
| mordred | nod. excellent point | 16:26 |
| *** sdague has quit IRC | 16:26 | |
| clarkb | it would be a reasonable thing to do if we had better control over source IP addresses and/or could funnel thta information around and updating things quickly enough | 16:26 |
| *** Goneri has joined #openstack-infra | 16:27 | |
| mordred | clarkb: it's too bad that the security group references don't work super well with high volume users like us - this seems like an actual good use case for making a security group which is "allow connections from all the hosts in security group X" | 16:27 |
| *** alex_xu has joined #openstack-infra | 16:27 | |
| clarkb | mordred: ya but also not all cloud ssecuriy group :/ | 16:27 |
| clarkb | (which may actually be due to the other thing too) | 16:28 |
| mordred | clarkb: but, iirc, using that at our scale blows things up - and yes, also the not-all-clouds-security-group thing | 16:28 |
| *** xinliang has joined #openstack-infra | 16:28 | |
| *** xinliang has quit IRC | 16:28 | |
| *** xinliang has joined #openstack-infra | 16:28 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Setting default log url for tempestmail https://review.openstack.org/518723 | 16:30 |
| fungi | we could consider some means of authenticating recursive lookups, i suppose | 16:30 |
| *** sdague has joined #openstack-infra | 16:33 | |
| *** xinliang has quit IRC | 16:34 | |
| sshnaidm | pabelanger, we need to find a way to see errors without calling infra cores every time | 16:37 |
| sshnaidm | pabelanger, it's only beginning of POC and it's already complicated | 16:38 |
| dmsimard | sshnaidm: what issues are you not finding ? | 16:38 |
| *** xinliang has joined #openstack-infra | 16:39 | |
| clarkb | sshnaidm: to be fair the last one totally didn't need any infra core input | 16:39 |
| clarkb | and the logs even said exactly what the fix was (which imo is a vastly better experience than before) | 16:40 |
| pabelanger | sshnaidm: I didn't look at debug logs for that, I just did a code review. But yes, as mentioned, we have an open issue to bubble up errors into code review to make it easier | 16:40 |
| sshnaidm | clarkb, if I saw log this patch doesn't exist, I'd know at least what's wrong | 16:40 |
| sshnaidm | s/patch/path | 16:40 |
| clarkb | sshnaidm: sure there are bound to be bugs I just don't think its fair to characterise this as needing an infra core every time | 16:40 |
| sshnaidm | clarkb, let's say somebody with access to zuul-executor logs | 16:41 |
| clarkb | no you didn't need zuul executor logs is what I'm saying | 16:41 |
| clarkb | it was in the job log and said "to fix this use quotes" | 16:42 |
| clarkb | which is imo a much better job debugging experience | 16:42 |
| *** e0ne has quit IRC | 16:44 | |
| sshnaidm | clarkb, I'm talking about next patch | 16:44 |
| *** jpena is now known as jpena|brb | 16:44 | |
| sshnaidm | clarkb, it was wrong path of cloned repos on host, as pabelanger mentioned | 16:44 |
| sshnaidm | clarkb, I didn't see this path in docs, maybe I missed something | 16:44 |
| jeblair | sshnaidm: can you point me at the patch? | 16:45 |
| sshnaidm | jeblair, https://review.openstack.org/#/c/522554/ | 16:45 |
| jeblair | sshnaidm: docs for that start at https://docs.openstack.org/infra/zuul/feature/zuulv3/user/jobs.html#git-repositories | 16:46 |
| *** jascott1 has joined #openstack-infra | 16:47 | |
| jeblair | sshnaidm: i agree, we should have output for that error there | 16:49 |
| sshnaidm | yeah, it would be very helpful and useful | 16:49 |
| *** sree has quit IRC | 16:50 | |
| jeblair | i'm checking to see if this is one of the known conditions where we are missing logs | 16:50 |
| *** sree has joined #openstack-infra | 16:50 | |
| *** Liced has quit IRC | 16:51 | |
| *** makowals has quit IRC | 16:51 | |
| *** jascott1 has quit IRC | 16:51 | |
| *** armaan has joined #openstack-infra | 16:53 | |
| jeblair | sshnaidm, mordred: i've added that to the zuulv3-issues etherpad right under a similar issue (missing a 'role not found' error message). so hopefully mordred can look at the two of those together -- they seem similar. | 16:54 |
| sshnaidm | jeblair, thanks | 16:54 |
| *** sree has quit IRC | 16:56 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 16:56 |
| *** rhallisey has quit IRC | 16:57 | |
| *** rhallisey has joined #openstack-infra | 16:58 | |
| sshnaidm | clarkb, jeblair pabelanger I wonder if there is RFE for zuulv3 to take vars from files in job definitions. | 16:58 |
| sshnaidm | like vars_files: in ansible playbooks | 16:59 |
| *** salv-orlando has joined #openstack-infra | 17:00 | |
| *** gyee_ has joined #openstack-infra | 17:00 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: POC: logs collection from native zuulv3 job https://review.openstack.org/522554 | 17:00 |
| pabelanger | sshnaidm: I think it is a fine line TBH, we don't wany too much ansible logic in zuul.yaml files. I think the path you are on is right, using vars_file inside your playbooks. Today, vars in zuul.yaml is extra vars on CLI using ansible-playbooks -e@foo.yaml | 17:04 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: tripleo-buildimage: deploy repo in pre-run https://review.openstack.org/523167 | 17:04 |
| *** sree has joined #openstack-infra | 17:04 | |
| *** felipemonteiro_ has quit IRC | 17:05 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Use py3 in docs generation https://review.openstack.org/523169 | 17:05 |
| sshnaidm | pabelanger, sorry, can you explain your last sentence? | 17:06 |
| *** salv-orlando has quit IRC | 17:06 | |
| pabelanger | sshnaidm: the vars section in zuul.yaml, is actually ansible 'extra-vars' that we pass to the ansible-playbook client from the CLI. That is the -e flag on ansible-playbook | 17:07 |
| sshnaidm | pabelanger, and what is zuul.yaml..? | 17:07 |
| pabelanger | sshnaidm: no, sorry, I am wrong. vars in zuul.yaml is the host variables we add to inventory file. We use -e for secrets | 17:08 |
| *** slaweq has joined #openstack-infra | 17:08 | |
| sshnaidm | pabelanger, you mentioned "-e@foo.yaml" - it's file including, so I thought maybe we can use it | 17:08 |
| pabelanger | sshnaidm: zuul.yaml, isn't that where you are asking to use a vars file? | 17:09 |
| *** sree has quit IRC | 17:09 | |
| pabelanger | sshnaidm: yah, I spoke incorrectly. We only use that for secrets and site vars files, which are global (not per job) | 17:09 |
| *** pcaruana has quit IRC | 17:09 | |
| sshnaidm | pabelanger, I meant like this https://github.com/openstack-infra/tripleo-ci/blob/6bfb25eae0d5b2c3f2dab3e74afdcac548938b9b/zuul.d/ovb-jobs.yaml#L143 jobs definition file, it it "zuul.yaml" you mean? | 17:10 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Set ceph_use_fqdn for rdocloud https://review.openstack.org/519744 | 17:10 |
| *** psachin has quit IRC | 17:11 | |
| sshnaidm | pabelanger, and for which hosts do you add them in inventory? | 17:11 |
| pabelanger | sshnaidm: yes, that is zuul.yaml | 17:12 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job https://review.openstack.org/522618 | 17:12 |
| pabelanger | sshnaidm: you can see in the inventory file the location: http://logs.openstack.org/54/522554/5/check-tripleo/tripleo-ci-centos-7-ovb-ha-oooq/7c16193/zuul-info/inventory.yaml | 17:13 |
| *** slaweq has quit IRC | 17:13 | |
| pabelanger | ara_generate_html for example, is from zuul.yaml vars section, IIRC | 17:14 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: tripleo-buildimage: deploy repo in pre-run https://review.openstack.org/523167 | 17:14 |
| sshnaidm | pabelanger, so we can't inject vars like "-e " or "-e @" to ansible, but only to add them to inventory? | 17:15 |
| *** salv-orlando has joined #openstack-infra | 17:15 | |
| *** salv-orlando has quit IRC | 17:16 | |
| *** salv-orlando has joined #openstack-infra | 17:16 | |
| sshnaidm | pabelanger, I think it will be a lot of fun trying to migrate tripleo jobs.. we use ansible in a very different way | 17:16 |
| pabelanger | sshnaidm: mostly because we don't copy -e files to logs servers, and have decided to only use them for secrets today | 17:17 |
| pabelanger | sshnaidm: how so? Are you relying on extra-vars for a lot of things? | 17:18 |
| sshnaidm | pabelanger, for *all* things, and "-e @file.yaml" for all configs | 17:18 |
| sshnaidm | pabelanger, and we don't use 1 playbook in job, usually it's about 10 "atomic" playbooks | 17:19 |
| *** sree has joined #openstack-infra | 17:19 | |
| pabelanger | sshnaidm: right, so we don't support that today. The best you could do, is move those variables into group_vars with some sane defaults and toggle then using zuul.yaml vars today. But, it would be a good use case to bring on with other zuul contributors and see if there is a way to support it. | 17:20 |
| *** jamesmcarthur has quit IRC | 17:21 | |
| EmilienM | jeblair: when you have time this week, I'll need your help on https://review.openstack.org/#/c/521967/ - pabelanger and I couldn't figure out why puppet-openstack-unit-3.* are still running | 17:21 |
| *** sree has quit IRC | 17:24 | |
| *** iyamahat has quit IRC | 17:25 | |
| *** jpena|brb is now known as jpena | 17:25 | |
| *** Hal has quit IRC | 17:25 | |
| sshnaidm | pabelanger, yeah, only vars can help us right now, but it's about a few hundreds of lines in variables, it's a nightmare to add them all to vars. This is like ansible run looks in average tripleo job: http://logs.openstack.org/52/522052/3/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/53d972e/job-output.txt.gz#_2017-11-26_11_15_00_420487http://logs.openstack.org/52/522052/3/check/tripleo-ci-centos-7-scenario002-multinode-oooq-co | 17:26 |
| sshnaidm | ntainer/53d972e/job-output.txt.gz#_2017-11-26_11_15_00_420487 | 17:26 |
| sshnaidm | sorry, pasted twice: http://logs.openstack.org/52/522052/3/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/53d972e/job-output.txt.gz#_2017-11-26_11_15_00_420487 | 17:26 |
| pabelanger | sshnaidm: yah, keep in mind we also don't support tags in zuul.yaml | 17:27 |
| pabelanger | sshnaidm: so, in this case, there is going to be some changes to your playbooks | 17:28 |
| pabelanger | or have some discussions in #zuul to try and support your use case | 17:28 |
| *** jamesmcarthur has joined #openstack-infra | 17:29 | |
| jeblair | if it's a bunch of static vars, can't the playbook just load it with vars_file? | 17:29 |
| jeblair | EmilienM: ack | 17:30 |
| *** rwsu has joined #openstack-infra | 17:32 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/irc-meetings master: Update Designate's meeting time https://review.openstack.org/523177 | 17:32 |
| sshnaidm | jeblair, yeah, we also have about 25 playbooks and tags: https://github.com/openstack-infra/tripleo-ci/blob/e12f1b9514e9cc51d5f09fde365bd65b10d9ec18/toci-quickstart/playbooks/ovb.yml#L2 | 17:32 |
| sshnaidm | jeblair, tags - to run or not to run part of playbooks.. | 17:32 |
| sshnaidm | so we need to convert tags to variables and to include "when"s everywhere.. | 17:33 |
| *** dsariel has quit IRC | 17:33 | |
| jeblair | sshnaidm: or maybe we need tag support in zuul | 17:33 |
| sshnaidm | yeah, if it runs ansible it's just to add to CLI "--tags" "--skip-tags" | 17:34 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: ovb: run ovb-containers-oooq only on pike https://review.openstack.org/522579 | 17:35 |
| jeblair | sshnaidm: we're trying to not reimplement ansible in zuul, but try to make it so that if you can do something with ansible-playbook, zuul can do it for you. at least, in places where that makes sense and is safe from a security point of view. tags *seems* at first glance to be something that could fit, but we haven't discussed it yet. if we added it, it'd probably be after the v3.0 release which we're focused on now. feel free to ... | 17:36 |
| jeblair | ... log a story in storyboard about it | 17:36 |
| jeblair | apparently we have discussed it and i don't remember it. so a story would be very helpful in that regard. :) | 17:37 |
| sshnaidm | jeblair, yeah, will create one | 17:37 |
| *** efried is now known as efried_rollin | 17:38 | |
| sshnaidm | jeblair, the whole idea was to create ansible playbooks runnable everywhere - both locally and CI.. and it's fine to use -e -e@ or --tags with ansible locally | 17:40 |
| sshnaidm | jeblair, but we didn't expect it won't be supported | 17:41 |
| jeblair | sshnaidm: yep, we'll have to work through it. | 17:41 |
| *** Apoorva has joined #openstack-infra | 17:45 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 17:46 |
| *** harlowja has joined #openstack-infra | 17:47 | |
| *** armaan has quit IRC | 17:49 | |
| *** armaan has joined #openstack-infra | 17:49 | |
| *** armax has joined #openstack-infra | 17:53 | |
| *** sree has joined #openstack-infra | 17:53 | |
| *** sree has quit IRC | 17:57 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 18:00 |
| *** ralonsoh has quit IRC | 18:00 | |
| *** derekh has quit IRC | 18:02 | |
| *** armaan has quit IRC | 18:07 | |
| *** armaan has joined #openstack-infra | 18:07 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 18:09 |
| *** markvoelker has quit IRC | 18:10 | |
| *** felipemonteiro has joined #openstack-infra | 18:17 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 18:18 |
| *** felipemonteiro_ has joined #openstack-infra | 18:18 | |
| pabelanger | jeblair: fungi: okay, so we have leaked jobs again on status.o.o (407868,74 is 9+ hours), it does look like we can leak them is our launcher is killed for some reason. In this case ze05.o.o seems to have been rebooted, due to live migration this morning | 18:19 |
| pabelanger | right now, I have no good way to abort those jobs | 18:20 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 18:20 |
| pabelanger | because they are no longer running on the executor | 18:20 |
| pabelanger | and last week, I had to abandon / restore the patches to clean our zuul pipelines | 18:20 |
| pabelanger | which did leave nodes leacked in 'in-use' state | 18:20 |
| *** jpena is now known as jpena|off | 18:20 | |
| pabelanger | which, eventually, cleared because we had to restart the scheduler due to high memory usage | 18:21 |
| *** armaan has quit IRC | 18:22 | |
| *** felipemonteiro has quit IRC | 18:22 | |
| *** armaan has joined #openstack-infra | 18:23 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 18:23 |
| *** kgiusti has quit IRC | 18:24 | |
| *** kgiusti has joined #openstack-infra | 18:26 | |
| *** tosky has quit IRC | 18:26 | |
| *** armaan has quit IRC | 18:27 | |
| *** armaan has joined #openstack-infra | 18:27 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 18:29 |
| *** trown is now known as trown|lunch | 18:29 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 18:31 |
| dmsimard | I love ara | 18:31 |
| dmsimard | makes finding errors so fast | 18:31 |
| dmsimard | <3 | 18:31 |
| *** dhinesh has joined #openstack-infra | 18:32 | |
| *** rbrndt has quit IRC | 18:32 | |
| jeblair | pabelanger: that's a known bug. it's in storyboard somewhere | 18:33 |
| dmsimard | sshnaidm: in the meantime, while it's not ideal, a workaround to consider could be to wrap a nested ansible or something like that. | 18:33 |
| *** jascott1 has joined #openstack-infra | 18:33 | |
| *** armaan has quit IRC | 18:34 | |
| pabelanger | jeblair: okay, good to know | 18:34 |
| dmsimard | sshnaidm: however that means that the zuul ara report will have one long task because from the perspective of the executor you're really just running one "command" task | 18:34 |
| *** vhosakot has joined #openstack-infra | 18:38 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 18:44 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 18:48 |
| sshnaidm | dmsimard, well, it's like we do it now actually | 18:52 |
| *** gouthamr has joined #openstack-infra | 18:52 | |
| sshnaidm | dmsimard, zuul runs shell and we run ansible in this shell | 18:52 |
| *** dhinesh has quit IRC | 18:52 | |
| dmsimard | yeah, it'd be nice to run the right playbooks etc directly. | 18:52 |
| *** dhinesh has joined #openstack-infra | 18:53 | |
| dmsimard | odyssey4me, cloudnull, evrardjp: do you have any plans to fully ansibilize your things and make them runnable directly by zuul v3 ? | 18:53 |
| *** jascott1 has quit IRC | 18:53 | |
| *** jascott1 has joined #openstack-infra | 18:57 | |
| *** eharney has quit IRC | 18:58 | |
| *** ccamacho has quit IRC | 19:02 | |
| *** electrofelix has quit IRC | 19:04 | |
| *** jpich has quit IRC | 19:04 | |
| *** linkmark has joined #openstack-infra | 19:04 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 19:06 |
| *** hughhalf has quit IRC | 19:09 | |
| *** HeOS has quit IRC | 19:09 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: WIP: Add command socket support to zuul-merger https://review.openstack.org/523197 | 19:11 |
| AJaeger | config-core, one cleanup review, please - https://review.openstack.org/#/c/513926 | 19:13 |
| jeblair | pabelanger: https://storyboard.openstack.org/#!/story/2000899 is the story | 19:13 |
| *** hughhalf has joined #openstack-infra | 19:13 | |
| *** dtantsur is now known as dtantsur|afk | 19:14 | |
| *** pblaho has quit IRC | 19:15 | |
| *** pcaruana has joined #openstack-infra | 19:16 | |
| *** HeOS has joined #openstack-infra | 19:21 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Use py3 in docs generation https://review.openstack.org/523169 | 19:21 |
| sshnaidm | pabelanger, is it possible to run ansible at least with "-vv" ? | 19:22 |
| *** tosky has joined #openstack-infra | 19:27 | |
| *** salv-orlando has quit IRC | 19:28 | |
| *** jascott1 has quit IRC | 19:28 | |
| *** salv-orlando has joined #openstack-infra | 19:28 | |
| *** sree has joined #openstack-infra | 19:28 | |
| pabelanger | sshnaidm: yes, but it isn't expsed on a per job level. I also am not sure if that info will get logged to zuul stream | 19:29 |
| pabelanger | jeblair: thanks, that is excatly what happen last week to ze03 and ze06 today | 19:29 |
| pabelanger | good to know | 19:29 |
| *** rbrndt has joined #openstack-infra | 19:30 | |
| Shrews | pabelanger: jeblair: have we seen any finger daemons abnormally terminated on the executors since https://review.openstack.org/517437 merged? | 19:31 |
| pabelanger | Shrews: I think ze04 might have | 19:32 |
| pabelanger | Shrews: I haven't fully looked | 19:32 |
| *** sree has quit IRC | 19:33 | |
| pabelanger | zuul 20080 0.0 0.0 0 0 ? Z Nov20 0:27 [zuul-executor] <defunct> | 19:34 |
| pabelanger | doesn't look healthy | 19:34 |
| *** felipemonteiro_ has quit IRC | 19:38 | |
| Shrews | pabelanger: i see that process was killed by the OOM killer back on the 23rd. perhaps log streaming is eating memory, though i'm not sure why that would be yet | 19:46 |
| Shrews | but at least it's something to look at | 19:46 |
| jeblair | true, i don't think we thought to look there | 19:46 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Create pre-generated fake subunit file https://review.openstack.org/522955 | 19:46 |
| *** trown|lunch is now known as trown | 19:47 | |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Fix keypress handling on board view https://review.openstack.org/438654 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Make board view easier to read https://review.openstack.org/438655 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Add tag editting to story view https://review.openstack.org/438707 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Fix inconsistency name https://review.openstack.org/452648 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Actually allow for overriding the lock file https://review.openstack.org/506953 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Add help text for new story command https://review.openstack.org/523203 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Start using alembic https://review.openstack.org/523204 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Add unique constraint to story_tags table https://review.openstack.org/523205 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Fix tag editing https://review.openstack.org/523206 | 19:47 |
| openstackgerrit | James E. Blair proposed openstack/boartty master: Fix creating new stories https://review.openstack.org/523207 | 19:47 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Improve configure-unbound role to support configuring TTLs https://review.openstack.org/523178 | 19:47 |
| openstackgerrit | Merged openstack/boartty master: Fix keypress handling on board view https://review.openstack.org/438654 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Make board view easier to read https://review.openstack.org/438655 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Add tag editting to story view https://review.openstack.org/438707 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Fix inconsistency name https://review.openstack.org/452648 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Actually allow for overriding the lock file https://review.openstack.org/506953 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Add help text for new story command https://review.openstack.org/523203 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Start using alembic https://review.openstack.org/523204 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Add unique constraint to story_tags table https://review.openstack.org/523205 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Fix tag editing https://review.openstack.org/523206 | 19:49 |
| openstackgerrit | Merged openstack/boartty master: Fix creating new stories https://review.openstack.org/523207 | 19:49 |
| jeblair | SpamapS, clarkb: i *think* we have enough there to create and tag stories for the zuul roadmap now | 19:50 |
| SpamapS | jeblair: woot | 19:50 |
| * SpamapS will take a look after lunch | 19:51 | |
| clarkb | cool, I'm starting my drive home frkm long weekend will be at computer probably 2:30 pacific ish | 19:51 |
| jeblair | At least, i did https://storyboard.openstack.org/#!/story/2001324 entirely through boartty | 19:51 |
| SpamapS | jeblair: reminds me that I need to finish my assignment dropdown fix | 19:51 |
| jeblair | SpamapS: that would be awesome, because assigning to anyone but me in boartty is not fun, and i don't want everything assigned to me | 19:52 |
| *** dsariel has joined #openstack-infra | 19:52 | |
| *** eharney has joined #openstack-infra | 19:53 | |
| *** eharney has quit IRC | 19:53 | |
| *** eharney has joined #openstack-infra | 19:53 | |
| *** amoralej is now known as amoralej|off | 19:55 | |
| *** camunoz has quit IRC | 19:55 | |
| *** ijw has quit IRC | 19:56 | |
| *** sambetts is now known as sambetts|afk | 19:59 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job https://review.openstack.org/522618 | 20:04 |
| *** dave-mcc_ is now known as dave-mccowan | 20:04 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add command socket support to zuul-merger https://review.openstack.org/523197 | 20:07 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Move send_command() into ZuulApp https://review.openstack.org/523211 | 20:07 |
| dmsimard | boartty ? lol | 20:13 |
| dmsimard | nice | 20:13 |
| *** ldnunes has quit IRC | 20:14 | |
| fungi | if you ever wondered what happens when you plug a tty into a wild boar... | 20:15 |
| pabelanger | mmmtty | 20:15 |
| dmsimard | pabelanger, fungi, clarkb: patch for unbound TTL configuration is ready for review: https://review.openstack.org/#/c/523178/ | 20:16 |
| dmsimard | note that configure-unbound is included in the base playbook | 20:16 |
| *** sree has joined #openstack-infra | 20:16 | |
| dmsimard | one thing I wanted to add to the integration tests was "unbound-checkconf -o cache-min-ttl" (and cache-max-ttl) but it doesn't seem like it works that way. | 20:17 |
| dmsimard | Just to make sure the configuration is effective. | 20:17 |
| dmsimard | that review made me realize an issue | 20:18 |
| dmsimard | If you use with_first_found within a role (to seek out vars specific to an OS) and you have a "vars" directory in your playbook directory, it'll import those instead of the role's | 20:19 |
| dmsimard | so we should be careful with those and prefix {{ role_path }}/vars to the list of files | 20:19 |
| *** slaweq has joined #openstack-infra | 20:19 | |
| *** slaweq has quit IRC | 20:20 | |
| *** slaweq has joined #openstack-infra | 20:20 | |
| *** sree has quit IRC | 20:21 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove tripleo-buildimage-overcloud-full-centos-7 from gate https://review.openstack.org/523146 | 20:24 |
| cloudnull | dmsimard: I want to say yes. but evrardjp would have a clearer idea of what the plan is | 20:26 |
| dmsimard | cloudnull: I'm interested in the general use case of making something easy to use by both ansible and zuul v3 | 20:26 |
| *** yamahata has quit IRC | 20:32 | |
| *** threestrands_ has joined #openstack-infra | 20:32 | |
| *** kjackal has quit IRC | 20:32 | |
| cloudnull | sounds like a worth while thing to do | 20:33 |
| *** kjackal has joined #openstack-infra | 20:33 | |
| *** threestrands_ has quit IRC | 20:33 | |
| *** threestrands_ has joined #openstack-infra | 20:34 | |
| *** threestrands has joined #openstack-infra | 20:34 | |
| *** threestrands has quit IRC | 20:34 | |
| *** threestrands has joined #openstack-infra | 20:34 | |
| cloudnull | at the moment I think we're just calling the scripts using a shell task, which is keeping our existing functionality prior to zuulv3 | 20:34 |
| cloudnull | I can see the benifit in just having a single interface | 20:34 |
| cloudnull | but there may be complications in that we're also installing ansible and other bits. | 20:35 |
| cloudnull | but i've not put a lot of thought into it, so i may be over thinking it :) | 20:35 |
| fungi | the real challenge i see comes when your primary deliverable is ansible content, and you're wanting to test that it works under multiple versions of ansible | 20:35 |
| fungi | especially if, as we've been also working to support, you want to test pull requests proposed to upstream ansible | 20:36 |
| fungi | e.g., your jobs are executing ansible content you want to be able to depends-on to proposed changes for ansible itself | 20:37 |
| *** efried_rollin is now known as efried | 20:38 | |
| fungi | because we're almost certainly not going to want to trust unreviewed ansible patches to run on our zuul executors | 20:39 |
| fungi | odds are that will at most have to be relegated to running only on test nodes | 20:39 |
| ianw | dmsimard: i certainly don't have an issue with the var renames etc in that, but should we split it up? i think fiddling the ttl's is probably on the riskier side (these things always have unexpected consequences) and it would be nice to consider in isolation if we revert etc | 20:40 |
| jeblair | our goals -- not all of which have been achieved yet -- are for folks to be able to use ansible in zuul in the way that they would normally use it in most circumstances | 20:40 |
| jeblair | there are some small things cropping up now (such as support for tags and extra var files) that we may need to add to zuul to achieve those goals | 20:41 |
| jeblair | so experimenting with that and continuing to refine zuul to support that would be great | 20:41 |
| jeblair | (changes like that may not arrive until after the v3.0 release, as we focus on that) | 20:41 |
| jeblair | we also know of some larger changes we may need to support the case of folks maintaining ansible roles which should work in multiple versions of ansible | 20:42 |
| jeblair | we may need to alter zuul to support running with different versions. that's a slightly larger task, but not out of the question for future work. | 20:42 |
| jeblair | cloudnull, dmsimard, evrardjp, fungi: ^ | 20:43 |
| jeblair | (at the risk of repeating things people may or may not know) | 20:43 |
| fungi | yup, and that's what i expect will take longer before we can move away from an ansible task running ansible | 20:43 |
| jeblair | fungi: ya. and even if we achieve all our goals, i'm certain there are still going to be some things that just won't be tested correctly unless we do that | 20:44 |
| jeblair | fungi: as in your example of a proposed change to ansible itself | 20:44 |
| jeblair | so building good support for that case as well is desirable | 20:44 |
| fungi | and then there will be cases like zuul integration testing, where we will probably never be able to stop running a nested ansible (since it'll be running as a subprocess of a daemon started as part of the job content) | 20:44 |
| jeblair | ianw: how's the _project rename coming? | 20:44 |
| *** threestrands_ has quit IRC | 20:44 | |
| *** iyamahat has joined #openstack-infra | 20:45 | |
| *** camunoz has joined #openstack-infra | 20:45 | |
| ianw | jeblair: ok ... testing is the issue | 20:46 |
| *** dhill_ has quit IRC | 20:50 | |
| pabelanger | jeblair: speaking of roadmap, should we consider adding ansible 2.4 support into our 3.0 release? | 20:51 |
| jeblair | ianw: it's the copy repos role that's tricky right? | 20:51 |
| jeblair | ianw: maybe we could create a new copy of that role for testing? | 20:51 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Move to dictionary list of projects zuul._projects (take 2) https://review.openstack.org/518815 | 20:53 |
| openstackgerrit | Merged openstack-infra/project-config master: Skip openstack-tox-py35 on requirements for newton https://review.openstack.org/522904 | 20:54 |
| ianw | jeblair: ^ ok, that was my thinking on it, sorry it's just been sitting as [dnm]. i'm open to testing ideas ... | 20:54 |
| ianw | a complete copy of the role and test that might be the easiest thing, hadn't thought of that | 20:54 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Fix sub_nodes and sub_nodes_private with multiple nodes https://review.openstack.org/523226 | 20:55 |
| *** e0ne has joined #openstack-infra | 20:56 | |
| *** jamesmcarthur has quit IRC | 20:59 | |
| *** jamesmcarthur has joined #openstack-infra | 21:00 | |
| dmsimard | ianw: the plan is to do it in the role instead of the base image exactly so it's easy to revert | 21:04 |
| *** jamesmcarthur has quit IRC | 21:04 | |
| dmsimard | ianw: perhaps we can add an "unbound_configure_ttl" which would default to false if you'd like | 21:04 |
| *** yamahata has joined #openstack-infra | 21:05 | |
| *** sree has joined #openstack-infra | 21:05 | |
| ianw | dmsimard: i just mean split the existing change into two different changes, with the renaming stuff etc first, and we can merge that no problems, then the actual interesting bits in a second change | 21:05 |
| dmsimard | ianw: oh, I thought you meant to split it into another role | 21:05 |
| pabelanger | keep in mind, it isn't too hard to revert images in nodepool either | 21:05 |
| *** Apoorva_ has joined #openstack-infra | 21:05 | |
| dmsimard | ianw: sure, let me split it up in two changes and also add a toggle to disable the ttl configuration if need be | 21:06 |
| ianw | no no, just one change, one thing ... :) | 21:06 |
| *** dsariel has quit IRC | 21:07 | |
| *** Apoorva has quit IRC | 21:09 | |
| *** dhill_ has joined #openstack-infra | 21:09 | |
| *** sree has quit IRC | 21:09 | |
| *** rockyg has joined #openstack-infra | 21:13 | |
| *** smatzek has quit IRC | 21:18 | |
| pabelanger | ianw: dmsimard: clarkb: left a -1 on unbound change, with comments | 21:18 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: tripleo-buildimage: deploy repo in pre-run https://review.openstack.org/523167 | 21:19 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Add support for configuring TTLs in configure-unbound https://review.openstack.org/523178 | 21:19 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: General improvements to configure-unbound role https://review.openstack.org/523228 | 21:19 |
| *** jtomasek has quit IRC | 21:19 | |
| dmsimard | ianw: ^ split | 21:19 |
| dmsimard | pabelanger: oops, looking | 21:20 |
| *** vhosakot has quit IRC | 21:20 | |
| *** vhosakot has joined #openstack-infra | 21:20 | |
| dmsimard | pabelanger: sure | 21:20 |
| *** kgiusti has left #openstack-infra | 21:21 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: General improvements to configure-unbound role https://review.openstack.org/523228 | 21:22 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Add support for configuring TTLs in configure-unbound https://review.openstack.org/523178 | 21:22 |
| dmsimard | pabelanger: ^ fixed defaults and quotes | 21:22 |
| *** ijw has joined #openstack-infra | 21:22 | |
| *** rhallisey has quit IRC | 21:23 | |
| *** rockyg has quit IRC | 21:25 | |
| *** hamzy has quit IRC | 21:27 | |
| *** petevg_afk is now known as petevg | 21:29 | |
| *** e0ne has quit IRC | 21:29 | |
| pabelanger | thanks, that's likely our safest route for now | 21:30 |
| *** benj_ has quit IRC | 21:30 | |
| *** ijw has quit IRC | 21:30 | |
| dmsimard | totally fine with that, we can likely track down which jobs are often hit by github or other dns lookup issues and then try it selectively that way | 21:31 |
| *** benj_ has joined #openstack-infra | 21:31 | |
| *** shardy has quit IRC | 21:36 | |
| dmsimard | pabelanger: oh, fyi cache-bogus-ttl is not what we thought it was | 21:38 |
| *** ijw has joined #openstack-infra | 21:39 | |
| dmsimard | pabelanger: it sounds like what we'd like to do doesn't exist (yet), it would be the opposite of "cache-max-negative-ttl" see https://bugzilla.redhat.com/show_bug.cgi?id=1360222 | 21:39 |
| openstack | bugzilla.redhat.com bug 1360222 in unbound "cache-max-negative-ttl don't work as expected" [High,Closed: eol] - Assigned to pwouters | 21:39 |
| dmsimard | Basically "cache-max-negative-ttl" would define the TTL for NXDOMAIN (or such) queries and "cache-max-positive-ttl" would define the TTL for successful ones | 21:40 |
| *** pcaruana has quit IRC | 21:41 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/puppet-zanata master: Preparation for Zanata 4 version upgrade https://review.openstack.org/506795 | 21:41 |
| *** ijw has quit IRC | 21:43 | |
| *** hamzy has joined #openstack-infra | 21:44 | |
| *** vhosakot has quit IRC | 21:48 | |
| *** ganso has quit IRC | 21:55 | |
| jhesketh | o/ | 21:56 |
| pabelanger | o/ | 21:56 |
| *** trown is now known as trown|outtypewww | 21:56 | |
| *** apetrich has quit IRC | 21:57 | |
| *** admcleod_ is now known as admcleod | 21:57 | |
| *** apetrich has joined #openstack-infra | 21:57 | |
| *** esberglu has quit IRC | 21:57 | |
| *** esberglu has joined #openstack-infra | 21:58 | |
| *** jamesmcarthur has joined #openstack-infra | 21:59 | |
| jeblair | it's zuul meeting time in #openstack-meeting-alt | 22:00 |
| *** mat128 has quit IRC | 22:03 | |
| *** rtjure has quit IRC | 22:08 | |
| *** vhosakot has joined #openstack-infra | 22:09 | |
| *** ijw has joined #openstack-infra | 22:09 | |
| *** markvoelker has joined #openstack-infra | 22:09 | |
| *** vhosakot has quit IRC | 22:10 | |
| *** ijw has quit IRC | 22:10 | |
| openstackgerrit | K Jonathan Harker proposed openstack/diskimage-builder master: zypper: fix package removal https://review.openstack.org/517658 | 22:11 |
| *** jascott1 has joined #openstack-infra | 22:13 | |
| *** jascott1 has quit IRC | 22:15 | |
| *** dprince has quit IRC | 22:16 | |
| *** ihrachys has quit IRC | 22:16 | |
| *** ihrachys has joined #openstack-infra | 22:17 | |
| *** jascott1 has joined #openstack-infra | 22:17 | |
| *** d0ugal has quit IRC | 22:21 | |
| *** ijw has joined #openstack-infra | 22:22 | |
| *** dhill_ has quit IRC | 22:28 | |
| *** wolverineav has quit IRC | 22:29 | |
| *** dhill_ has joined #openstack-infra | 22:33 | |
| *** rcernin has joined #openstack-infra | 22:34 | |
| clarkb | infra-root I will be sending the infra is back email shortly (just fyi) | 22:42 |
| *** esberglu has quit IRC | 22:46 | |
| *** esberglu has joined #openstack-infra | 22:46 | |
| fungi | thanks clarkb! happy to be back too! | 22:47 |
| *** florianf has quit IRC | 22:47 | |
| mordred | clarkb: ++ | 22:50 |
| *** esberglu has quit IRC | 22:51 | |
| *** hamzy has quit IRC | 22:55 | |
| *** Apoorva_ has quit IRC | 22:57 | |
| *** wolverineav has joined #openstack-infra | 22:58 | |
| *** Apoorva has joined #openstack-infra | 22:58 | |
| *** ijw has quit IRC | 22:59 | |
| *** esberglu has joined #openstack-infra | 22:59 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job https://review.openstack.org/522618 | 23:01 |
| *** wolverineav has quit IRC | 23:02 | |
| *** esberglu has quit IRC | 23:04 | |
| *** raissa_ has joined #openstack-infra | 23:05 | |
| *** hashar has quit IRC | 23:07 | |
| *** hamzy has joined #openstack-infra | 23:08 | |
| *** raissa has quit IRC | 23:08 | |
| *** gmann_afk is now known as gmann | 23:11 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Ensure ara is updated on executors https://review.openstack.org/516740 | 23:12 |
| clarkb | jeblair: dmsimard fungi ^ fyi | 23:12 |
| dmsimard | clarkb: so.. is it taboo to talk about migrating from puppet 3 to puppet 4 or consider rewriting things to Ansible ? | 23:14 |
| dmsimard | Perhaps a topic for tomorrow's meeting :p | 23:14 |
| *** ijw has joined #openstack-infra | 23:15 | |
| dmsimard | I ask because this pip thing is a non-issue in puppet 4 | 23:15 |
| dmsimard | I guess puppet 5 is a thing too. | 23:15 |
| clarkb | dmsimard: no puppet 4 is a semi in progress item and is on the "we need help" list | 23:16 |
| clarkb | dmsimard: how does puppet 4 fix this? | 23:16 |
| clarkb | you can pass in pip args for package installs? | 23:16 |
| dmsimard | puppet 4 has a native pip provider, and the package type has an install_options to pass args like that | 23:17 |
| dmsimard | Unless that's puppet 5, see my comment in the patch | 23:17 |
| clarkb | well they all have a native pip provider | 23:18 |
| clarkb | the issue is whether or not you can instruct it to only update the deps that no longer fit into the requirements | 23:18 |
| *** jamesmcarthur has quit IRC | 23:18 | |
| clarkb | based on quick git history search of the provider latest 4 and 5 provide it at least | 23:19 |
| jeblair | (i would personally also be happy with a migrateto-ansible | 23:19 |
| jeblair | grr | 23:19 |
| jeblair | (i would personally also be happy with a migrate-to-ansible plan, if it seemed feasible to fully accomplish a migration and not leave us stradling two config management systems) | 23:20 |
| *** bobh has quit IRC | 23:20 | |
| clarkb | my hunch is that ^ would require us to do much of nothing else for a significant chunk of time | 23:21 |
| jeblair | yeah, or have a really detailed plan about how to accomplish it over a longer period | 23:22 |
| dmsimard | it depends | 23:22 |
| dmsimard | for example, if we write a new infrastructure component | 23:22 |
| dmsimard | it should be written in the right language | 23:22 |
| dmsimard | otherwise we keep growing the debt | 23:22 |
| fungi | for the record, the pip option we want is --upgrade-strategy=only-if-needed | 23:22 |
| dmsimard | I think we can also probably migrate progressively from one to the other without it being an all-or-nothing deal | 23:23 |
| dmsimard | I mean, it's not like infracloud has ansible calling puppet calling ansible or anything :) | 23:23 |
| clarkb | dmsimard: I think the all or nothing bit is the long term "we don't want to support both forever" | 23:23 |
| *** pbourke has quit IRC | 23:23 | |
| clarkb | dmsimard: so we either need a detailed plan like jeblair says or we need to migrate and focus on that until its done | 23:23 |
| *** dave-mccowan has quit IRC | 23:24 | |
| *** slaweq has quit IRC | 23:25 | |
| clarkb | because the most debt we can find ourselves in is running multiple systems for long periods of time I think | 23:25 |
| *** pbourke has joined #openstack-infra | 23:25 | |
| dmsimard | It's probably something I would love to have time for, any little free time I have right now I try to spend it on ara to get 1.0 out the door so I don't enter zuul v3 branch territory :) | 23:25 |
| *** linkmark has quit IRC | 23:26 | |
| dmsimard | otherwise I'm still keen on implementing the ara replacement for puppetboard spec and the proactive monitoring spec | 23:26 |
| dmsimard | writing the specs would already be a good start :D | 23:27 |
| clarkb | yes | 23:27 |
| *** slaweq has joined #openstack-infra | 23:27 | |
| *** raissa_ has quit IRC | 23:29 | |
| *** raissa has joined #openstack-infra | 23:29 | |
| mordred | wow. I just spent the entire day down a maze of rabbit holes on what was supposed to be a quick 5 minute diversion this morning | 23:31 |
| * mordred apologizes for being completely useless | 23:31 | |
| dmsimard | mordred: I'm currently down about 2k lines worth of rabbit-holing in ara for what was supposed to be a straightforward change, I know the feeling. | 23:32 |
| dmsimard | mordred: hopefully your rabbit hole was useful :) | 23:33 |
| *** slaweq has quit IRC | 23:33 | |
| mordred | dmsimard: I don't think 'useful' is quite the right word - but I did manage to make it out the other side completing the task rather than rage quitting | 23:33 |
| dmsimard | Rabbit hole or Yak shaving? :p | 23:34 |
| dmsimard | huh, Wikipedia describes a rabbit hole as "a slang expression for a psychedelic experience, from the same usage". Psychadelic. Heh. | 23:35 |
| mordred | dmsimard: maybe I was rabbit shaving | 23:35 |
| *** raissa has quit IRC | 23:36 | |
| * fungi is reminded that yak-shaving day comes again soon. hoping to get some coleslaw in my boots this year | 23:36 | |
| dmsimard | fungi: there's a yak shaving day ?? | 23:36 |
| *** raissa has joined #openstack-infra | 23:37 | |
| *** rkukura_ has joined #openstack-infra | 23:37 | |
| fungi | ren and stimpy show reference | 23:37 |
| dmsimard | fungi: oh that's where it's from | 23:38 |
| fungi | one of my favorite episodes | 23:38 |
| dmsimard | I didn't really speak english when I watched Ren and Stimpy, and if I watched it it was probably in french :) | 23:38 |
| clarkb | I just drove from sunny relatively warm weather to cold foggy and wet. Not sure what I was thinking | 23:38 |
| fungi | dmsimard: it probably made about as much sense translated to swahili | 23:38 |
| dmsimard | clarkb: we got snow up here (finally!) | 23:39 |
| clarkb | (I've just checked for sanity purposes and looks like ansible's pip module also has an extra_args parameter that you can use to set flags like this) | 23:40 |
| *** rkukura has quit IRC | 23:40 | |
| *** rkukura_ is now known as rkukura | 23:40 | |
| pabelanger | clarkb: jeblair: pushed https://review.openstack.org/469983/ up back in June for ansiblify control plane, need to reply to comments / updates but we could keep iterating on that | 23:40 |
| *** dhinesh has quit IRC | 23:41 | |
| *** dhinesh has joined #openstack-infra | 23:42 | |
| mordred | clarkb: yas - I have used the extra_args field with success in some jobs even | 23:48 |
| mordred | dmsimard: ^^ pabelanger's comment | 23:48 |
| *** claudiub|2 has quit IRC | 23:49 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Remove legacy jobs from oaktree repos https://review.openstack.org/523247 | 23:50 |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Remove unused legacy oaktree jobs https://review.openstack.org/523248 | 23:52 |
| *** aeng has joined #openstack-infra | 23:57 | |
| *** marst has quit IRC | 23:57 | |
| *** raissa has quit IRC | 23:57 | |
| *** raissa has joined #openstack-infra | 23:58 | |
| *** mriedem has quit IRC | 23:59 | |
| *** tosky has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!