| *** claudiub has quit IRC | 00:00 | |
| *** r-daneel has quit IRC | 00:05 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add job for testing against devel of ansible https://review.openstack.org/531934 | 00:06 |
|---|---|---|
| *** david-lyle has quit IRC | 00:07 | |
| *** dklyle has joined #openstack-infra | 00:07 | |
| *** dklyle has quit IRC | 00:07 | |
| *** xarses_ has quit IRC | 00:07 | |
| *** jascott1 has quit IRC | 00:16 | |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-openstackci master: Lay groundwork for zuulv2/v3 coexisting https://review.openstack.org/523951 | 00:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Use setup.py if we can't get the name from setup.cfg https://review.openstack.org/531936 | 00:24 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add job for testing against devel of ansible https://review.openstack.org/531934 | 00:25 |
| openstackgerrit | Merged openstack-dev/pbr master: doc: Rework features doc https://review.openstack.org/512248 | 00:25 |
| *** jascott1 has joined #openstack-infra | 00:26 | |
| *** sdague has quit IRC | 00:29 | |
| openstackgerrit | Merged openstack-dev/pbr master: doc: Minor rework of usage doc https://review.openstack.org/512249 | 00:33 |
| *** edmondsw has joined #openstack-infra | 00:36 | |
| *** esberglu has quit IRC | 00:37 | |
| *** felipemonteiro_ has joined #openstack-infra | 00:44 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Adding mapping for SUSE package https://review.openstack.org/529370 | 00:45 |
| clarkb | mordred: does override-branch on zuul do the right thing there? eg it will checkout your change and not tip of override-branch value? | 00:47 |
| clarkb | mordred: left a comment on https://review.openstack.org/#/c/531936/1 but I'm not sure its worth a -1. Curious to know how bad the errors were when it broke on you | 00:50 |
| portdirect | hey - I would like to be able to use the HWE kernel on our ubuntu nodes in the OSH jobs | 00:54 |
| portdirect | is there any guidance on how to do this? | 00:55 |
| clarkb | portdirect: I think the method there is going to be installing the kernel and rebooting the test instance in the job, but I'm not sure anyone has done that yet to sort out the problems involved with doing that | 00:56 |
| clarkb | portdirect: can you provide more detail? OSH? and what the hardware enablement kernel will allow you to do (there might be better options like using a different distro or something | 00:56 |
| *** caphrim007_ has joined #openstack-infra | 00:59 | |
| *** caphrim007 has quit IRC | 01:00 | |
| *** kjackal has quit IRC | 01:02 | |
| portdirect | clarkb: cheers, thats what I thought | 01:02 |
| portdirect | OSH - OpenStack-Helm | 01:02 |
| *** kjackal has joined #openstack-infra | 01:02 | |
| clarkb | I'm guessing you are looking for container features in newer kenrels? | 01:03 |
| portdirect | We really want to start using cephfs as well as RBD for network attached storage, as some of our workloads could benefit from it | 01:03 |
| clarkb | don't both of those work fine with xenials default kernel? | 01:04 |
| portdirect | the ubuntu 4.4 kernel does not support subvolume mounts | 01:04 |
| *** caphrim007_ has quit IRC | 01:04 | |
| portdirect | which the k8s provisioner uses, | 01:04 |
| clarkb | I see | 01:04 |
| portdirect | we (ATT) have filed a bug with canoncial, but I expect it to take some time to trickle down | 01:05 |
| clarkb | then ya either distro with newer kernel (fedora?) or install and reboot | 01:05 |
| portdirect | install and reboot it is | 01:05 |
| portdirect | we gate with centos and fedora as well as ubuntu | 01:06 |
| portdirect | they are fine :D | 01:06 |
| clarkb | centos' kernel is older though, did red hat backport those features? | 01:07 |
| *** felipemonteiro_ has quit IRC | 01:12 | |
| *** markvoelker has joined #openstack-infra | 01:17 | |
| *** kiennt26 has joined #openstack-infra | 01:18 | |
| portdirect | clarkb: yup | 01:20 |
| portdirect | that kernel always confuses me, 3.10 going on 4.somthing.or.other ; | 01:21 |
| portdirect | ;) | 01:21 |
| *** zhurong has joined #openstack-infra | 01:21 | |
| *** liujiong has joined #openstack-infra | 01:24 | |
| *** armax has joined #openstack-infra | 01:25 | |
| corvus | portdirect: i like that naming system! :) | 01:26 |
| *** abelur_ has joined #openstack-infra | 01:27 | |
| *** aeng has quit IRC | 01:34 | |
| portdirect | hmm - I think must be missing something obvious I'm getting a lot of "Waiting for logger" http://zuulv3.openstack.org/stream.html?uuid=addf3af17d6c43688891636268eb8fc7&logfile=console.log | 01:35 |
| portdirect | when trying this patch to get the HWE kernel running: https://review.openstack.org/#/c/531944/2/tools/gate/playbooks/upgrade-host/tasks/main.yaml | 01:35 |
| clarkb | portdirect: ya I think that was something mordred mentioned as a known issue with reboots in jobs | 01:36 |
| clarkb | the console lpg streaming currently relies on a separate daemon | 01:37 |
| clarkb | nkt sure if there is a good way to start it again | 01:37 |
| portdirect | any way to get it back up? | 01:37 |
| clarkb | mordred: ^ | 01:37 |
| corvus | portdirect, clarkb: use this role: https://docs.openstack.org/infra/zuul-jobs/roles.html#role-start-zuul-console | 01:39 |
| portdirect | corvus: so just drop it in on line 36? https://review.openstack.org/#/c/531944/2/tools/gate/playbooks/osh-infra-upgrade-host.yaml | 01:40 |
| *** mriedem1 has joined #openstack-infra | 01:40 | |
| corvus | portdirect: yep that should do it. | 01:40 |
| *** mriedem has quit IRC | 01:42 | |
| portdirect | trying now, thanks man | 01:42 |
| *** aeng has joined #openstack-infra | 01:47 | |
| *** zhongjun has joined #openstack-infra | 01:52 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add /node-list to the webapp https://review.openstack.org/499969 | 01:54 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add /label-list to the webapp https://review.openstack.org/500254 | 01:54 |
| *** mriedem has joined #openstack-infra | 01:56 | |
| *** mriedem1 has quit IRC | 01:58 | |
| *** felipemonteiro_ has joined #openstack-infra | 02:02 | |
| portdirect | corvus, clarkb got the HWE kernel up and working like a chap, thanks for all your help | 02:02 |
| *** jamesmcarthur has joined #openstack-infra | 02:07 | |
| *** annp has joined #openstack-infra | 02:09 | |
| *** namnh has joined #openstack-infra | 02:17 | |
| *** vivsoni has quit IRC | 02:19 | |
| *** vivsoni has joined #openstack-infra | 02:19 | |
| *** caphrim007 has joined #openstack-infra | 02:21 | |
| *** caphrim007 has quit IRC | 02:25 | |
| *** cuongnv has joined #openstack-infra | 02:27 | |
| *** edmondsw has quit IRC | 02:30 | |
| *** Apoorva has quit IRC | 02:32 | |
| *** harlowja has quit IRC | 02:32 | |
| *** armaan has quit IRC | 02:33 | |
| *** simonmcc has quit IRC | 02:36 | |
| *** simonmcc has joined #openstack-infra | 02:37 | |
| *** rlandy_ has quit IRC | 02:37 | |
| *** rlandy|bbl is now known as rlandy | 02:38 | |
| *** rlandy has quit IRC | 02:38 | |
| *** xinliang has quit IRC | 02:43 | |
| ianw | infra-core: just pushed dib 2.10.0 so worth keeping a closer eye than usual on the builds | 02:45 |
| *** udesale has joined #openstack-infra | 02:46 | |
| *** jamesmcarthur has quit IRC | 02:50 | |
| *** jamesmcarthur has joined #openstack-infra | 02:51 | |
| *** xinliang has joined #openstack-infra | 02:54 | |
| *** hongbin has joined #openstack-infra | 02:57 | |
| *** nicolasbock has quit IRC | 02:58 | |
| *** markvoelker has quit IRC | 03:06 | |
| *** adreznec has quit IRC | 03:07 | |
| *** adreznec has joined #openstack-infra | 03:08 | |
| *** jascott1 has quit IRC | 03:10 | |
| *** caphrim007 has joined #openstack-infra | 03:10 | |
| *** iyamahat has quit IRC | 03:13 | |
| *** yamahata has quit IRC | 03:15 | |
| *** adreznec has quit IRC | 03:16 | |
| *** adreznec has joined #openstack-infra | 03:21 | |
| *** ijw has quit IRC | 03:21 | |
| *** hongbin has quit IRC | 03:24 | |
| *** hongbin has joined #openstack-infra | 03:25 | |
| openstackgerrit | lei zhang proposed openstack/diskimage-builder master: Fix the incorrect git.openstack.org source URL https://review.openstack.org/532001 | 03:35 |
| *** jamesmcarthur has quit IRC | 03:40 | |
| *** yamamoto has joined #openstack-infra | 03:40 | |
| *** gyee has quit IRC | 03:48 | |
| *** zhurong has quit IRC | 03:55 | |
| *** ramishra has joined #openstack-infra | 04:13 | |
| *** Apoorva has joined #openstack-infra | 04:15 | |
| *** esberglu has joined #openstack-infra | 04:16 | |
| *** esberglu has quit IRC | 04:17 | |
| *** jamesmcarthur has joined #openstack-infra | 04:23 | |
| *** jamesmcarthur has quit IRC | 04:23 | |
| *** jamesmcarthur has joined #openstack-infra | 04:23 | |
| *** hongbin has quit IRC | 04:26 | |
| *** felipemonteiro_ has quit IRC | 04:34 | |
| *** rosmaita has quit IRC | 04:36 | |
| *** mriedem has quit IRC | 04:47 | |
| *** janki has joined #openstack-infra | 04:47 | |
| *** markvoelker has joined #openstack-infra | 04:56 | |
| *** felipemonteiro_ has joined #openstack-infra | 05:01 | |
| *** coolsvap has joined #openstack-infra | 05:02 | |
| *** psachin has joined #openstack-infra | 05:04 | |
| *** links has joined #openstack-infra | 05:07 | |
| *** ijw has joined #openstack-infra | 05:19 | |
| *** claudiub has joined #openstack-infra | 05:29 | |
| *** sree has joined #openstack-infra | 05:31 | |
| *** sree has quit IRC | 05:32 | |
| *** sree has joined #openstack-infra | 05:32 | |
| *** Apoorva has quit IRC | 05:49 | |
| *** smatzek has joined #openstack-infra | 05:53 | |
| *** iyamahat has joined #openstack-infra | 05:56 | |
| *** xinliang has quit IRC | 05:57 | |
| *** smatzek has quit IRC | 05:57 | |
| AJaeger | ianw, frickler, could you review https://review.openstack.org/#/c/531430/ , please? - we need this to fix manuals publishing | 05:58 |
| *** jamesmcarthur has quit IRC | 06:02 | |
| *** jamesmcarthur has joined #openstack-infra | 06:03 | |
| *** sandanar has joined #openstack-infra | 06:03 | |
| *** dhajare has joined #openstack-infra | 06:05 | |
| *** bhavik1 has joined #openstack-infra | 06:07 | |
| *** jamesmcarthur has quit IRC | 06:07 | |
| *** felipemonteiro_ has quit IRC | 06:08 | |
| *** CrayZee has joined #openstack-infra | 06:08 | |
| *** xinliang has joined #openstack-infra | 06:09 | |
| *** xinliang has quit IRC | 06:09 | |
| *** xinliang has joined #openstack-infra | 06:09 | |
| *** yamahata has joined #openstack-infra | 06:13 | |
| *** aeng has quit IRC | 06:16 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/openstack-zuul-jobs master: Restore branch conditional on tempest-full https://review.openstack.org/531058 | 06:16 |
| *** gouthamr has quit IRC | 06:17 | |
| *** sree has quit IRC | 06:23 | |
| *** sree has joined #openstack-infra | 06:24 | |
| *** sree has quit IRC | 06:28 | |
| *** dbecker has quit IRC | 06:31 | |
| *** links has quit IRC | 06:35 | |
| *** rcernin has quit IRC | 06:38 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix sql reporting with postgres https://review.openstack.org/532023 | 06:44 |
| *** dbecker has joined #openstack-infra | 06:45 | |
| *** jamesmcarthur has joined #openstack-infra | 06:46 | |
| *** zhurong has joined #openstack-infra | 06:50 | |
| *** jamesmcarthur has quit IRC | 06:51 | |
| *** links has joined #openstack-infra | 06:52 | |
| ianw | AJaeger: LGTM, thanks for pasting the debug info in | 06:54 |
| *** pcaruana has joined #openstack-infra | 06:56 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove explicit bindep parameters https://review.openstack.org/531430 | 06:58 |
| *** gcb has joined #openstack-infra | 06:58 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] handle fedora27 python3 changes https://review.openstack.org/532026 | 06:59 |
| *** links has quit IRC | 07:03 | |
| *** pgadiya has joined #openstack-infra | 07:04 | |
| *** florianf has joined #openstack-infra | 07:05 | |
| *** armaan has joined #openstack-infra | 07:10 | |
| *** numans has quit IRC | 07:11 | |
| *** numans has joined #openstack-infra | 07:13 | |
| *** links has joined #openstack-infra | 07:14 | |
| *** jlk has quit IRC | 07:17 | |
| *** jascott1 has joined #openstack-infra | 07:19 | |
| *** jlk has joined #openstack-infra | 07:21 | |
| *** jlk has joined #openstack-infra | 07:21 | |
| *** jtomasek has joined #openstack-infra | 07:26 | |
| *** jtomasek has quit IRC | 07:27 | |
| *** felipemonteiro_ has joined #openstack-infra | 07:27 | |
| *** jtomasek has joined #openstack-infra | 07:27 | |
| *** links has quit IRC | 07:27 | |
| *** makowals has joined #openstack-infra | 07:39 | |
| *** felipemonteiro_ has quit IRC | 07:39 | |
| *** sree has joined #openstack-infra | 07:40 | |
| *** slaweq has joined #openstack-infra | 07:41 | |
| openstackgerrit | megan guiney proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation https://review.openstack.org/524862 | 07:42 |
| *** slaweq has quit IRC | 07:44 | |
| *** e0ne has joined #openstack-infra | 07:44 | |
| *** sree has quit IRC | 07:44 | |
| *** links has joined #openstack-infra | 07:45 | |
| *** udesale has quit IRC | 07:46 | |
| *** slaweq has joined #openstack-infra | 07:47 | |
| *** notmyname has quit IRC | 07:49 | |
| *** rhagarty has quit IRC | 07:49 | |
| *** hashar has joined #openstack-infra | 07:50 | |
| *** d0ugal has joined #openstack-infra | 07:51 | |
| *** slaweq has quit IRC | 07:52 | |
| *** notmyname has joined #openstack-infra | 07:52 | |
| *** rhagarty has joined #openstack-infra | 07:52 | |
| AJaeger | thanks, ianw for reviewing. | 07:52 |
| *** slaweq has joined #openstack-infra | 08:00 | |
| *** eumel8 has joined #openstack-infra | 08:02 | |
| *** armax has quit IRC | 08:08 | |
| *** slaweq has quit IRC | 08:08 | |
| *** dingyichen has quit IRC | 08:09 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/puppet-zanata master: [WiP] Change Zanata install medium to war files https://review.openstack.org/531809 | 08:17 |
| *** tesseract has joined #openstack-infra | 08:20 | |
| *** alexchadin has joined #openstack-infra | 08:22 | |
| *** slaweq has joined #openstack-infra | 08:23 | |
| *** sree has joined #openstack-infra | 08:24 | |
| *** ralonsoh has joined #openstack-infra | 08:25 | |
| *** sree has quit IRC | 08:28 | |
| *** psachin has quit IRC | 08:30 | |
| *** sree has joined #openstack-infra | 08:31 | |
| *** sree has quit IRC | 08:36 | |
| *** yamamoto_ has joined #openstack-infra | 08:36 | |
| *** daidv has quit IRC | 08:38 | |
| *** namnh has quit IRC | 08:38 | |
| *** cuongnv has quit IRC | 08:38 | |
| *** kiennt26 has quit IRC | 08:38 | |
| *** annp has quit IRC | 08:38 | |
| *** cuongnv has joined #openstack-infra | 08:38 | |
| *** namnh has joined #openstack-infra | 08:38 | |
| *** kiennt26 has joined #openstack-infra | 08:39 | |
| *** daidv has joined #openstack-infra | 08:39 | |
| *** annp has joined #openstack-infra | 08:39 | |
| *** yamamoto has quit IRC | 08:39 | |
| *** psachin has joined #openstack-infra | 08:40 | |
| *** sree has joined #openstack-infra | 08:44 | |
| *** jpena|off is now known as jpena | 08:45 | |
| *** sree has quit IRC | 08:49 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-lint job https://review.openstack.org/532083 | 08:50 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-import-to-galaxy job https://review.openstack.org/532084 | 08:50 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-spec job https://review.openstack.org/532085 | 08:50 |
| *** priteau has joined #openstack-infra | 08:56 | |
| *** jpich has joined #openstack-infra | 09:09 | |
| *** iyamahat has quit IRC | 09:14 | |
| *** jlk has quit IRC | 09:18 | |
| *** lucas-afk is now known as lucasagomes | 09:18 | |
| *** gema has joined #openstack-infra | 09:19 | |
| *** dtantsur|afk is now known as dtantsur | 09:20 | |
| dtantsur | AJaeger: morning. how can we move forward https://review.openstack.org/#/c/531148/ please? | 09:21 |
| dtantsur | should I do a fake release? or should we just go ahead, and see if it works with the first release? | 09:21 |
| *** flwang1 has quit IRC | 09:22 | |
| *** flwang has joined #openstack-infra | 09:24 | |
| *** jlk has joined #openstack-infra | 09:26 | |
| *** jlk has joined #openstack-infra | 09:26 | |
| AJaeger | dtantsur: please do a fake release. mordred was looking on automating. So, we might do this different in a couple of days for new projects... See https://review.openstack.org/#/c/531898/ for his work | 09:27 |
| *** jamesmcarthur has joined #openstack-infra | 09:28 | |
| *** efoley has joined #openstack-infra | 09:28 | |
| *** sree has joined #openstack-infra | 09:30 | |
| *** jamesmcarthur has quit IRC | 09:32 | |
| *** derekh has joined #openstack-infra | 09:33 | |
| dtantsur | AJaeger: okay, here it is https://pypi.python.org/pypi/ironic-tempest-plugin | 09:34 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: Refactor status functions, add web endpoints, allow params https://review.openstack.org/531390 | 09:34 |
| *** sree has quit IRC | 09:34 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: Add a separate module for node management commands https://review.openstack.org/531703 | 09:34 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] webapp: add optional admin endpoint https://review.openstack.org/531718 | 09:34 |
| *** jistr|off|mtg is now known as jistr | 09:36 | |
| AJaeger | dtantsur: +2 | 09:39 |
| dtantsur | thnx | 09:39 |
| *** sree has joined #openstack-infra | 09:41 | |
| *** iyamahat has joined #openstack-infra | 09:42 | |
| *** iyamahat has quit IRC | 09:42 | |
| *** sree has quit IRC | 09:45 | |
| *** armaan has quit IRC | 09:46 | |
| *** armaan has joined #openstack-infra | 09:47 | |
| *** iyamahat has joined #openstack-infra | 09:47 | |
| *** bhavik1 has quit IRC | 09:47 | |
| *** sree has joined #openstack-infra | 09:48 | |
| *** sree has quit IRC | 09:52 | |
| *** yamamoto_ has quit IRC | 09:52 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: zk: automatically retry command when connection is lost https://review.openstack.org/523640 | 09:54 |
| *** erlon has joined #openstack-infra | 09:54 | |
| *** iyamahat has quit IRC | 09:56 | |
| *** jaosorior has joined #openstack-infra | 09:59 | |
| *** udesale has joined #openstack-infra | 09:59 | |
| *** stakeda has quit IRC | 09:59 | |
| *** tosky has joined #openstack-infra | 10:01 | |
| *** jascott1 has quit IRC | 10:02 | |
| *** armaan has quit IRC | 10:07 | |
| *** armaan has joined #openstack-infra | 10:07 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Add entry for the Diversity WG https://review.openstack.org/531900 | 10:07 |
| *** sshnaidm is now known as sshnaidm|afk | 10:07 | |
| *** apetrich has quit IRC | 10:09 | |
| *** sree has joined #openstack-infra | 10:09 | |
| *** cuongnv has quit IRC | 10:11 | |
| AJaeger | config-core, I put more entries to review on https://etherpad.openstack.org/p/Nvt3ovbn5x - please help going over the list... | 10:11 |
| *** pbourke has quit IRC | 10:12 | |
| *** pbourke has joined #openstack-infra | 10:13 | |
| openstackgerrit | Merged openstack-infra/project-config master: Publish ironic-tempest-plugin releases to pypi https://review.openstack.org/531148 | 10:15 |
| *** yamamoto has joined #openstack-infra | 10:16 | |
| *** sambetts|afk is now known as sambetts | 10:16 | |
| *** liujiong has quit IRC | 10:16 | |
| *** d0ugal has quit IRC | 10:19 | |
| *** d0ugal has joined #openstack-infra | 10:21 | |
| *** sree has quit IRC | 10:23 | |
| dtantsur | can someone please land https://review.openstack.org/#/c/531382/ ? this is required to start releasing networking-generic-switch correctly | 10:38 |
| *** janki has quit IRC | 10:38 | |
| *** sdague has joined #openstack-infra | 10:40 | |
| *** priteau has quit IRC | 10:43 | |
| *** priteau has joined #openstack-infra | 10:44 | |
| *** hemna_ has quit IRC | 10:48 | |
| *** priteau has quit IRC | 10:48 | |
| *** namnh has quit IRC | 10:48 | |
| *** ldnunes has joined #openstack-infra | 10:48 | |
| *** kopecmartin has joined #openstack-infra | 10:50 | |
| *** janki has joined #openstack-infra | 10:51 | |
| *** yamahata has quit IRC | 10:53 | |
| *** alexchadin has quit IRC | 10:53 | |
| *** alexchadin has joined #openstack-infra | 10:59 | |
| *** jascott1 has joined #openstack-infra | 11:04 | |
| *** priteau has joined #openstack-infra | 11:11 | |
| *** janki has quit IRC | 11:13 | |
| *** janki has joined #openstack-infra | 11:13 | |
| *** priteau has quit IRC | 11:15 | |
| *** wolverineav has joined #openstack-infra | 11:17 | |
| *** jchhatbar has joined #openstack-infra | 11:19 | |
| *** gtmanfred has quit IRC | 11:21 | |
| *** janki has quit IRC | 11:22 | |
| *** udesale has quit IRC | 11:24 | |
| *** jokke_ has joined #openstack-infra | 11:26 | |
| *** therve_ is now known as therve | 11:35 | |
| *** sree has joined #openstack-infra | 11:35 | |
| *** udesale has joined #openstack-infra | 11:36 | |
| *** priteau has joined #openstack-infra | 11:38 | |
| *** udesale has quit IRC | 11:39 | |
| *** udesale has joined #openstack-infra | 11:39 | |
| *** sree has quit IRC | 11:40 | |
| *** apetrich has joined #openstack-infra | 11:41 | |
| *** jamesmcarthur has joined #openstack-infra | 11:49 | |
| *** jamesmcarthur has quit IRC | 11:53 | |
| *** sree has joined #openstack-infra | 11:56 | |
| *** sree has quit IRC | 12:00 | |
| *** apetrich has quit IRC | 12:01 | |
| *** efoley has quit IRC | 12:04 | |
| *** yamamoto has quit IRC | 12:06 | |
| *** yamamoto has joined #openstack-infra | 12:06 | |
| *** annp has quit IRC | 12:08 | |
| *** ijw has quit IRC | 12:08 | |
| *** alexchadin has quit IRC | 12:14 | |
| *** alexchadin has joined #openstack-infra | 12:14 | |
| *** jkilpatr has quit IRC | 12:18 | |
| openstackgerrit | Lenny Verkhovsky proposed openstack/diskimage-builder master: Fixing interface up in slow networks https://review.openstack.org/529976 | 12:20 |
| *** sree has joined #openstack-infra | 12:21 | |
| *** nicolasbock has joined #openstack-infra | 12:21 | |
| *** pcichy has quit IRC | 12:22 | |
| *** hareesh has joined #openstack-infra | 12:23 | |
| *** tlaxkit has joined #openstack-infra | 12:24 | |
| *** sree has quit IRC | 12:25 | |
| *** sree has joined #openstack-infra | 12:30 | |
| *** pgadiya has quit IRC | 12:32 | |
| *** jpena is now known as jpena|lunch | 12:32 | |
| *** gtmanfred has joined #openstack-infra | 12:33 | |
| *** lucasagomes is now known as lucas-hungry | 12:33 | |
| *** hareesh has quit IRC | 12:34 | |
| *** sree has quit IRC | 12:35 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] webapp: add optional admin endpoint https://review.openstack.org/531718 | 12:35 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove neutron and horizion sphinx jobs https://review.openstack.org/523018 | 12:36 |
| openstackgerrit | Amy Marrich (spotz) proposed openstack-infra/irc-meetings master: Updated WoO meeting information https://review.openstack.org/532173 | 12:37 |
| *** pgadiya has joined #openstack-infra | 12:37 | |
| *** pcichy has joined #openstack-infra | 12:38 | |
| *** sree has joined #openstack-infra | 12:38 | |
| *** dsariel has quit IRC | 12:39 | |
| *** ijw has joined #openstack-infra | 12:40 | |
| *** rhallisey has joined #openstack-infra | 12:42 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use only base publish-openstack-sphinx-docs job https://review.openstack.org/523019 | 12:45 |
| openstackgerrit | Merged openstack-infra/project-config master: switch whereto doc job to python3 https://review.openstack.org/531766 | 12:48 |
| *** sree has quit IRC | 12:48 | |
| *** sree has joined #openstack-infra | 12:49 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove remaining sphinx-horizon/neutron jobs https://review.openstack.org/532177 | 12:53 |
| *** jkilpatr has joined #openstack-infra | 12:54 | |
| *** smatzek has joined #openstack-infra | 12:56 | |
| *** zhurong_ has joined #openstack-infra | 12:56 | |
| *** bobh has joined #openstack-infra | 12:58 | |
| openstackgerrit | Merged openstack-infra/project-config master: Update the list of ironic jobs in nova's experimental queue https://review.openstack.org/531868 | 12:59 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy jobs from Keystone https://review.openstack.org/524232 | 12:59 |
| *** dsariel has joined #openstack-infra | 13:00 | |
| *** makowals has quit IRC | 13:00 | |
| *** makowals has joined #openstack-infra | 13:01 | |
| mnaser | AJaeger: i guess we missed a job :< | 13:02 |
| mnaser | (re: publish-openstack-sphinx-docs-horizon) | 13:02 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Use only base publish-openstack-sphinx-docs job https://review.openstack.org/532179 | 13:04 |
| *** trown|outtypewww is now known as trown | 13:04 | |
| mnaser | AJaeger: ^ | 13:04 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/openstack-zuul-jobs master: Remove neutron and horizion sphinx jobs https://review.openstack.org/523018 | 13:04 |
| *** dhajare has quit IRC | 13:05 | |
| *** udesale has quit IRC | 13:06 | |
| *** zhurong_ has quit IRC | 13:07 | |
| *** zhurong has quit IRC | 13:07 | |
| *** udesale has joined #openstack-infra | 13:08 | |
| AJaeger | mnaser: we did, yes... | 13:10 |
| AJaeger | mnaser: those were added after the change was done. | 13:12 |
| mnaser | ah that must have been why | 13:12 |
| *** rosmaita has joined #openstack-infra | 13:13 | |
| AJaeger | mnaser: care to review https://review.openstack.org/#/c/530124/ and https://review.openstack.org/#/c/530100/ as well, please? | 13:14 |
| mnaser | sure let me check them out | 13:14 |
| *** dave-mccowan has joined #openstack-infra | 13:15 | |
| AJaeger | thanks | 13:15 |
| *** bobh has quit IRC | 13:17 | |
| mnaser | AJaeger: done :) | 13:19 |
| *** dave-mccowan has quit IRC | 13:21 | |
| *** edmondsw has joined #openstack-infra | 13:21 | |
| *** sshnaidm|afk is now known as sshnaidm | 13:23 | |
| *** alexchadin has quit IRC | 13:23 | |
| *** alexchadin has joined #openstack-infra | 13:24 | |
| *** dave-mccowan has joined #openstack-infra | 13:26 | |
| *** lucas-hungry is now known as lucasagomes | 13:31 | |
| *** rlandy has joined #openstack-infra | 13:31 | |
| *** jpena|lunch is now known as jpena | 13:32 | |
| *** jamesmcarthur has joined #openstack-infra | 13:38 | |
| *** pcichy has quit IRC | 13:41 | |
| *** pcichy has joined #openstack-infra | 13:43 | |
| Roamer` | hm, I think something's wrong with the legacy-tempest-dsvm-py35 job at http://zuulv3.openstack.org/stream.html?uuid=2f2be286eef5475a955badf6b3d07126&logfile=console.log (change 140733/17 for Nova) - it has been stuck like that for, I think, more than two hours... I guess it will time out eventually, but I'd never seen devstack stuck for two hours before | 13:46 |
| Roamer` | or, of course, the stream is not capturing the actual output, and the actual tests have been running for more than two hours now and will time out soon | 13:47 |
| *** kiennt26_ has joined #openstack-infra | 13:47 | |
| *** esberglu has joined #openstack-infra | 13:48 | |
| *** dprince has joined #openstack-infra | 13:48 | |
| *** katkapilatova1 has joined #openstack-infra | 13:48 | |
| *** katkapilatova1 has left #openstack-infra | 13:49 | |
| pabelanger | Roamer`: looks like it just timed out, so you should have access to logs shortly | 13:49 |
| *** apetrich has joined #openstack-infra | 13:49 | |
| Roamer` | pabelanger, ah, so it did... | 13:50 |
| Roamer` | yeah,, thanks | 13:50 |
| *** coolsvap has quit IRC | 13:52 | |
| fungi | too bad we didn't catch that one sooner... i'd have ben curious to see what behavior we got attempting to ssh into the node for it | 13:52 |
| *** mriedem has joined #openstack-infra | 13:54 | |
| *** jchhatbar has quit IRC | 13:54 | |
| *** mihalis68 has joined #openstack-infra | 13:55 | |
| *** sree has quit IRC | 13:55 | |
| *** sree has joined #openstack-infra | 13:56 | |
| *** sree has quit IRC | 14:00 | |
| Roamer` | fungi, actually I think the node may still be acting up - worlddump is taking, well, some time | 14:02 |
| Roamer` | or something on the node, of course, not the node itself | 14:02 |
| Roamer` | sorry for being vague, "some time" is "close to 15 minutes now" | 14:03 |
| AJaeger | config-core, could you review https://review.openstack.org/#/c/530124/ https://review.openstack.org/#/c/530100/ https://review.openstack.org/532179 for project-config job removals , please? | 14:03 |
| Roamer` | aaaand of course it ended as soon as I said that... I guess my timeouts are pretty close to the jobs' timeouts :) | 14:05 |
| *** alexchadin has quit IRC | 14:06 | |
| Roamer` | sorry for the flood, I will shut up soon, but just to point out that it seems there is *still* something wrong with that node: some rsync failure and now "Remove the build SSH key" is stuck | 14:07 |
| *** kgiusti has joined #openstack-infra | 14:07 | |
| *** jamesmcarthur has quit IRC | 14:07 | |
| *** alexchadin has joined #openstack-infra | 14:10 | |
| fungi | i'm not yet at a computer where i can easily dig up that node's ip address, but if some infra-root who is wants to try an ssh into the root account on it and see whether it gets stuck, that would be a helpful data point | 14:11 |
| fungi | the one i looked at last week hung after the key exchange stage (according to ssh -vvv) and after a couple minutes i got a socket disconnect | 14:11 |
| *** pgadiya has quit IRC | 14:12 | |
| pabelanger | looking now | 14:13 |
| pabelanger | seems to be in inap | 14:13 |
| Roamer` | it ended like a minute ago... if the node was brought up on demand, it may be gone now :/ | 14:13 |
| fungi | so was the one i looked at last week | 14:13 |
| fungi | in inap as well, i mean | 14:13 |
| Roamer` | http://logs.openstack.org/33/140733/17/check/legacy-tempest-dsvm-py35/2f2be28/ just in case | 14:14 |
| *** Goneri has joined #openstack-infra | 14:14 | |
| pabelanger | We've seen this happen a few times when another node comes online to claim the IP (ghost instance) | 14:14 |
| pabelanger | or maybe it is just a routing issue | 14:14 |
| pabelanger | waiting for mgagne to come online to maybe help dig more :) | 14:15 |
| pabelanger | https://review.openstack.org/512130/ might be something we could try too, which enables SSH retries for ansible. To help with network issues | 14:16 |
| fungi | the rogue instance issue usually resulted in key mismatches, i thouht | 14:17 |
| *** shardy_afk has quit IRC | 14:17 | |
| fungi | not hangs | 14:17 |
| mgagne | pabelanger: IP doesn't ping anymore so I don't think there is a ghost instance involved | 14:17 |
| pabelanger | fungi: oh, right. My mistake | 14:17 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] webapp: add optional admin endpoint https://review.openstack.org/531718 | 14:18 |
| *** jroll has quit IRC | 14:19 | |
| *** hjensas has quit IRC | 14:19 | |
| *** jroll has joined #openstack-infra | 14:19 | |
| *** afred312 has quit IRC | 14:20 | |
| *** afred312 has joined #openstack-infra | 14:20 | |
| *** makowals has quit IRC | 14:21 | |
| *** makowals has joined #openstack-infra | 14:22 | |
| openstackgerrit | Honza Pokorny proposed openstack-infra/openstack-zuul-jobs master: Add nodejs 8 jobs https://review.openstack.org/532195 | 14:27 |
| *** psachin has quit IRC | 14:28 | |
| *** smatzek has quit IRC | 14:28 | |
| *** smatzek has joined #openstack-infra | 14:28 | |
| *** smatzek has quit IRC | 14:28 | |
| *** makowals has quit IRC | 14:29 | |
| openstackgerrit | Honza Pokorny proposed openstack-infra/project-config master: Configure tripleo-ui to use nodejs8 jobs https://review.openstack.org/532197 | 14:29 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: deprecations: Deprecate support for '-py{N}' requirements https://review.openstack.org/512306 | 14:32 |
| *** makowals has joined #openstack-infra | 14:34 | |
| *** felipemonteiro_ has joined #openstack-infra | 14:36 | |
| *** yamamoto_ has joined #openstack-infra | 14:37 | |
| *** jamesmcarthur has joined #openstack-infra | 14:39 | |
| *** yamamoto has quit IRC | 14:40 | |
| *** markvoelker has quit IRC | 14:40 | |
| *** eharney has joined #openstack-infra | 14:41 | |
| *** jamesmcarthur has quit IRC | 14:43 | |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-openstackci master: Add partial support for zuulv3 scheduler https://review.openstack.org/526759 | 14:45 |
| *** pcichy has quit IRC | 14:45 | |
| *** kaisers has quit IRC | 14:45 | |
| openstackgerrit | Honza Pokorny proposed openstack-infra/openstack-zuul-jobs master: Add nodejs 8 jobs https://review.openstack.org/532195 | 14:46 |
| *** pcichy has joined #openstack-infra | 14:48 | |
| *** kaisers has joined #openstack-infra | 14:51 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Create stable/queens specific ACL https://review.openstack.org/532205 | 14:53 |
| *** srwilkers_ has quit IRC | 14:54 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add error handling for pidfile https://review.openstack.org/532206 | 14:55 |
| *** alexchadin has quit IRC | 14:56 | |
| *** dizquierdo has joined #openstack-infra | 14:57 | |
| *** srwilkers has quit IRC | 14:58 | |
| *** srwilkers_ has joined #openstack-infra | 14:59 | |
| *** srwilkers_ is now known as srwilkers | 14:59 | |
| *** gouthamr has joined #openstack-infra | 15:00 | |
| *** sshnaidm is now known as sshnaidm|mtg | 15:01 | |
| *** slaweq has quit IRC | 15:02 | |
| *** nicolasbock has quit IRC | 15:02 | |
| *** sree has joined #openstack-infra | 15:03 | |
| *** slaweq has joined #openstack-infra | 15:04 | |
| *** rlandy is now known as rlandy|mtg | 15:05 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add Templates to Kingbird-Dashboard. https://review.openstack.org/530452 | 15:06 |
| *** slaweq has quit IRC | 15:09 | |
| *** ramishra has quit IRC | 15:10 | |
| *** smatzek has joined #openstack-infra | 15:11 | |
| *** smatzek has quit IRC | 15:11 | |
| *** pcichy has quit IRC | 15:11 | |
| *** smatzek has joined #openstack-infra | 15:11 | |
| *** pcichy has joined #openstack-infra | 15:12 | |
| *** bobh has joined #openstack-infra | 15:12 | |
| *** markvoelker has joined #openstack-infra | 15:13 | |
| *** markvoelker has quit IRC | 15:16 | |
| *** sree has quit IRC | 15:17 | |
| *** armax has joined #openstack-infra | 15:21 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Use setup.py if we can't get the name from setup.cfg https://review.openstack.org/531936 | 15:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Capture and report errors in sibling installation https://review.openstack.org/532216 | 15:25 |
| *** sree has joined #openstack-infra | 15:25 | |
| *** markvoelker has joined #openstack-infra | 15:25 | |
| dmsimard | Random question -- we're able to get numbers for the amount of "first party" jobs we run (through graphs or otherwise data from zuul's database).. Could we get something similar for third party ? I'm guessing it would have to be through gerrit... | 15:26 |
| fungi | dmsimard: worse, you'd have to scrape gerrit comments and attempt to parse job results out of them since it's all just freeform text reporting | 15:27 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Capture and report errors in sibling installation https://review.openstack.org/532216 | 15:28 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Use setup.py if we can't get the name from setup.cfg https://review.openstack.org/531936 | 15:28 |
| fungi | anybody happen to know if nb04 is supposed to up accessible? it closes the connection as soon as i attempt to ssh into it | 15:28 |
| fungi | er, supposed to be up and accessible | 15:28 |
| mordred | dmsimard, fungi: once we've upgraded gerrit we'll have the new CI results API | 15:29 |
| dmsimard | fungi: yuck | 15:29 |
| *** sree has quit IRC | 15:29 | |
| fungi | yep | 15:30 |
| fungi | until then... as you say, "yuck" | 15:30 |
| dmsimard | mordred: okay, I'm doing some stats for a talk and was curious about the amount of 3rd party jobs due to the amount of reviews (according to stackalytics) -- there's 146k "code reviews" for 40k commits (pike) and yet Zuul runs on average ~10k jobs per day so it doesn't add up. | 15:31 |
| *** caphrim007 has quit IRC | 15:31 | |
| *** caphrim007 has joined #openstack-infra | 15:32 | |
| *** makowals has quit IRC | 15:32 | |
| fungi | dmsimard: consider that lots of proposed changes never merge but we still run jobs on them, and the average number of patch sets we run jobs on for a commit which does merge is likely somewhere around 5-10 (complete guess on my part but certainly >1) | 15:33 |
| dmsimard | We had 325k jobs in november alone for example | 15:33 |
| fungi | plus, a number of projects run 10 or more jobs against each patch set | 15:33 |
| fungi | and then there's the rechecks | 15:33 |
| fungi | so, so, so many rechecks | 15:33 |
| dmsimard | yeah, nova neutron and cinder have a heavy amount of third party jobs | 15:33 |
| *** Apoorva has joined #openstack-infra | 15:33 | |
| smcginnis | Way too heavy. | 15:33 |
| corvus | stackalytics was pretty good about filtering actual reviews i thought. i would be surprised if a ci report counted as a code review. | 15:33 |
| fungi | and are merge-check jobs being counted in your number? every time a change merged, we ran a merge-check against every open change for the same project+branch | 15:34 |
| dmsimard | fungi: whatever would be reported through sqlreporter | 15:34 |
| dmsimard | (in zuul) | 15:34 |
| frickler | dmsimard: zuul posts "Verified+-1" which is probably different from "Code-Review+-1" and not counted at all in your 146k above | 15:34 |
| smcginnis | dmsimard: Are you looking for something similar to the last two links here: http://cinderstats.ivehearditbothways.com/ | 15:34 |
| mordred | dmsimard: yah - third-party-ci report as verified +- 1 as well | 15:35 |
| dmsimard | smcginnis: that's neat | 15:35 |
| dmsimard | frickler, mordred: oh, you're right -- third party reports a verified vote, not a code review vote | 15:35 |
| dmsimard | so it's stats on the verified votes I want then :D | 15:35 |
| *** sree has joined #openstack-infra | 15:36 | |
| *** links has quit IRC | 15:36 | |
| *** caphrim007 has quit IRC | 15:37 | |
| mordred | dmsimard: yah - of course, keep in mind that each verified vote can represent more than one job | 15:37 |
| *** slaweq has joined #openstack-infra | 15:38 | |
| dmsimard | I'll do with the numbers I have for the time being but I'll keep that at the back of my head | 15:38 |
| mordred | ++ | 15:38 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add support for a secure ZooKeeper configuration https://review.openstack.org/531882 | 15:38 |
| *** sar has joined #openstack-infra | 15:40 | |
| AJaeger | team, could you review https://review.openstack.org/#/c/530124/ https://review.openstack.org/#/c/530100/ https://review.openstack.org/532179 for project-config job removals , please? | 15:40 |
| *** slaweq has quit IRC | 15:42 | |
| *** sree has quit IRC | 15:42 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add error handling for pidfile https://review.openstack.org/532206 | 15:42 |
| *** kiennt26_ has quit IRC | 15:43 | |
| *** david-lyle has joined #openstack-infra | 15:44 | |
| *** makowals has joined #openstack-infra | 15:45 | |
| *** felipemonteiro__ has joined #openstack-infra | 15:45 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: tripleo-upgrades: add publish-openstack-python-tarball https://review.openstack.org/532222 | 15:46 |
| *** rosmaita_ has joined #openstack-infra | 15:46 | |
| *** felipemonteiro_ has quit IRC | 15:49 | |
| *** rosmaita has quit IRC | 15:49 | |
| fungi | infra-root: it looks like nb04 stopped responding to snmp from cacti a couple weeks ago. is it supposed to be in use or is it slated to get decommissioned? | 15:51 |
| fungi | i can't ssh into it | 15:51 |
| dmsimard | fungi: is the VM up ? what's console say ? | 15:51 |
| fungi | looks like it probably broke around 00:00 utc on december 23 | 15:52 |
| fungi | hrm... lots of support tickets open for this account. gonna skim those first | 15:53 |
| *** sree has joined #openstack-infra | 15:55 | |
| fungi | #status log Trove maintenance scheduled for 04:00-12:00 UTC on 2018-01-17 impacting Wiki_MySQL and cacti_MySQL instances | 15:56 |
| openstackstatus | fungi: finished logging | 15:56 |
| *** slaweq has joined #openstack-infra | 15:56 | |
| fungi | #status log Trove maintenance scheduled for 04:00-12:00 UTC on 2018-01-23 impacting zuul_v3 instance | 15:56 |
| openstackstatus | fungi: finished logging | 15:56 |
| mriedem | clarkb: are we maybe getting job timeouts uploading all of the giant console logs now because on the new zuulv3 jobs like tempest-full, the console log is way bigger because it contains all of the ansible stuff plus the full devstack log, in addition to the actual devstacklog we had before | 15:57 |
| *** xarses_ has joined #openstack-infra | 15:57 | |
| fungi | #status log Trove maintenance scheduled for 04:00-12:00 UTC on 2018-01-24 impacting paste_mysql_5.6 instance | 15:57 |
| openstackstatus | fungi: finished logging | 15:57 |
| Shrews | fungi: hrm, i'm not sure which nodes are our current builders | 15:57 |
| fungi | (i skipped logging the ones for -dev servers) | 15:57 |
| fungi | Shrews: nb01 and nb02 are no longer in dns and i believe were decommissioned a while back. nb03 is reachable and seems to be working normally. nb04 seems broken | 15:58 |
| fungi | i suspect we're supposed to be using nb03 and nb04 | 15:59 |
| *** jamesmcarthur has joined #openstack-infra | 15:59 | |
| clarkb | mriedem: maybe? that shouldn't include all of the devstack log that is a regression | 15:59 |
| clarkb | mriedem: file sizes should roughly be the same if we port jobs proprely | 15:59 |
| *** jamesmcarthur has quit IRC | 15:59 | |
| *** jamesmcarthur has joined #openstack-infra | 15:59 | |
| *** sree has quit IRC | 15:59 | |
| mriedem | clarkb: yeah http://logs.openstack.org/36/526436/21/check/tempest-full/5e38a2b/job-output.txt.gz#_2018-01-09_02_26_02_780157 | 16:00 |
| mriedem | i guess it's dumped because of the ansible task that runs it? | 16:00 |
| mriedem | and all of the ansible stuff is dumped into the console logs | 16:00 |
| *** apetrich has quit IRC | 16:00 | |
| clarkb | ansible is just running stack.sh though and the output from that script should be dumped to the log file | 16:01 |
| clarkb | I'm guessing it is currently being tee'd | 16:01 |
| fungi | infra-root: someone should probably take a look at ze02 and make sure it's sane... there's a ticket from rackspace saying they did a reboot migration on it yesterday | 16:02 |
| *** felipemonteiro__ has quit IRC | 16:03 | |
| clarkb | mriedem: looks like the linked job passed and did not timeout though | 16:03 |
| *** felipemonteiro__ has joined #openstack-infra | 16:03 | |
| clarkb | and the compressed output file is only 608k | 16:03 |
| clarkb | so not very large | 16:03 |
| *** janki has joined #openstack-infra | 16:03 | |
| *** sandanar has quit IRC | 16:03 | |
| *** yamahata has joined #openstack-infra | 16:04 | |
| fungi | infra-root: and graphite around 12 hours ago... something about cinder volumes having to be reattached so may need to look closely at those | 16:04 |
| fungi | "It appears that this migration errored because the volume 4b5d9f10-0427-4bfb-b45b-a4d682ac4ba5 attached as xvdb was deleted and could not be found in the API." | 16:05 |
| fungi | "Your cbs devices will be detached and re-attached on completion." | 16:05 |
| fungi | #thanks to Matt Van Winkle for volunteering to act as internal advocate at Rackspace for our control plane account there! | 16:08 |
| openstackstatus | fungi: Added thanks to Thanks page | 16:08 |
| *** markvoelker has quit IRC | 16:09 | |
| pabelanger | confirmed, I cannot ssh into nb04 also | 16:15 |
| pabelanger | ssh_exchange_identification: read: Connection reset by peer | 16:15 |
| clarkb | is nb04 the one in rax? | 16:15 |
| fungi | yeah, looks like a live migration gone wrong based on what i see on the oob console | 16:15 |
| fungi | yes | 16:15 |
| fungi | rax | 16:15 |
| clarkb | ya that was my next question | 16:16 |
| clarkb | it has the kernel hung task timeout messages? | 16:16 |
| pabelanger | yah, rax | 16:16 |
| clarkb | hard reboot tends to get that sorted out | 16:16 |
| fungi | soft reboot through nova api not working either. gonna --hard | 16:16 |
| fungi | #status log rebooted nb04 through the nova api; oob console content looked like a botched live migration | 16:16 |
| openstackstatus | fungi: finished logging | 16:16 |
| pabelanger | we still need to move nb03.o.o from vexxhost, could do that today if needed | 16:16 |
| fungi | part of what got me looking at the builders is gema is looking for approximate sizing for setting up an arm64 region for our nodepool | 16:17 |
| *** hongbin has joined #openstack-infra | 16:17 | |
| fungi | and i expect we'll need a native arm64 builder | 16:17 |
| *** udesale has quit IRC | 16:20 | |
| *** pcichy has quit IRC | 16:22 | |
| *** jkilpatr_ has joined #openstack-infra | 16:24 | |
| dmsimard | Still no kernels published by ubuntu... wth ? | 16:24 |
| dmsimard | Oh I guess today was the coordinated release so they're "in time" | 16:24 |
| *** jkilpatr has quit IRC | 16:26 | |
| clarkb | fungi: ze02 looks ok to me. zuul-executor is running and grepping for tracebacks in the executor debug log doesn't show any problems with corrupted git repos | 16:28 |
| dmsimard | clarkb, fungi: huh, so you know what .. maybe /sys/kernel/debug/x86/pti_enabled is actually a RHEL/CentOS only thing (perhaps due to the weird kernel version?) my fedora laptop doesn't have it but it has the insecure bug in /proc/cpuinfo | 16:32 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Change patchset column to string https://review.openstack.org/532237 | 16:32 |
| dmsimard | CentOS has CONFIG_KAISER in the /boot/config file but Fedora doesn't have it.. heh | 16:33 |
| clarkb | ya because KAISER is now page table isolation aiui | 16:33 |
| openstackgerrit | Alex Krzos proposed openstack-infra/grafyaml master: Add legend for graphs https://review.openstack.org/530799 | 16:34 |
| clarkb | I'm guessing its a side effect of frankenstien kernel backporting mess | 16:34 |
| dmsimard | yeah that's what I was saying :) | 16:34 |
| clarkb | likely to have similar things with ubuntu particularly on the older kernel(s) | 16:34 |
| dmsimard | CONFIG_PAGE_TABLE_ISOLATION is in fedora, yup. | 16:34 |
| *** kopecmartin has quit IRC | 16:34 | |
| AJaeger | clarkb, dmsimard : Yeah, different kernel versions got different patch sets... | 16:34 |
| *** e0ne has quit IRC | 16:35 | |
| *** jamesmcarthur has quit IRC | 16:35 | |
| fungi | gema: i've replied to your initial e-mail, but recommend that we move the discussion to the openstack-infra ml once you're closer to being ready for us to try to set up a native amd64 nodepool builder | 16:36 |
| fungi | er, i mean arm64 | 16:36 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-snmpd master: Enable snmpd so it works on systemd systems https://review.openstack.org/531520 | 16:37 |
| *** makowals has quit IRC | 16:37 | |
| *** markvoelker has joined #openstack-infra | 16:37 | |
| clarkb | infra-root ^ fixing will be nice especially as we systemd more. Also that may be my new least favorite puppet lint check | 16:37 |
| fungi | dmsimard: i believe the cpu_insecure bug flag in /proc/cpuinfo reflects you running a recent microcode update which exposes that in conjunction with the kernel patches... so we may not see that on virtual machines anyway if the hypervisor doesn't also expose it to guest instances | 16:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Use setup.py if we can't get the name from setup.cfg https://review.openstack.org/531936 | 16:38 |
| fungi | i need to disappear to run some lunchtime errands, but will return asap | 16:39 |
| *** sar has quit IRC | 16:40 | |
| *** yamamoto_ has quit IRC | 16:40 | |
| *** rlandy|mtg is now known as rlandy | 16:41 | |
| rcarrillocruz | oh nice | 16:41 |
| rcarrillocruz | good to see the arm nodepool thing is moving along | 16:42 |
| rcarrillocruz | kudos gema :-) | 16:42 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] webapp: add optional admin endpoint https://review.openstack.org/531718 | 16:42 |
| clarkb | fungi: gema for the builder the two things that that intsance needs are disk space and network bandwidth to the cloud(s) using the imagesbuilt | 16:43 |
| clarkb | faster disk helps too | 16:43 |
| clarkb | then for the actual test nodes I'm not sure we know how to size those as arm64 is completely different than amd64 | 16:43 |
| *** Swanson has joined #openstack-infra | 16:43 | |
| clarkb | will likely require us to just try some stuff to find something that is reasonable | 16:43 |
| *** felipemonteiro_ has joined #openstack-infra | 16:45 | |
| openstackgerrit | Merged openstack-infra/project-config master: tripleo-upgrades: add publish-openstack-python-tarball https://review.openstack.org/532222 | 16:46 |
| *** caphrim007 has joined #openstack-infra | 16:48 | |
| *** felipemonteiro__ has quit IRC | 16:49 | |
| *** pbourke has quit IRC | 16:54 | |
| *** smatzek has quit IRC | 16:56 | |
| *** smatzek has joined #openstack-infra | 16:57 | |
| *** smatzek_ has joined #openstack-infra | 16:59 | |
| *** jamesmcarthur has joined #openstack-infra | 16:59 | |
| chandankumar | AJaeger: hello | 16:59 |
| *** smatzek_ has quit IRC | 17:00 | |
| *** smatzek_ has joined #openstack-infra | 17:00 | |
| chandankumar | AJaeger: regarding adding a package entry on pypi, do i register the package from my account? then how infra will push the package there? | 17:00 |
| *** gyee has joined #openstack-infra | 17:01 | |
| *** smatzek has quit IRC | 17:02 | |
| *** panda|rover is now known as panda|rover|afk | 17:02 | |
| *** slaweq has quit IRC | 17:02 | |
| *** armax has quit IRC | 17:05 | |
| clarkb | chandankumar: you have to give our tooling account access as well | 17:05 |
| *** janki has quit IRC | 17:06 | |
| clarkb | chandankumar: this should be documented in the new project docs (though I think some of the details are changing due to changes in pypi) | 17:06 |
| chandankumar | clarkb: ok, i will register few tempest plugin which is needed by other projects | 17:06 |
| clarkb | chandankumar: you might make sure that they don't already have plans to register their own first | 17:06 |
| clarkb | (as that may cause confusion to see existing things) | 17:07 |
| *** jpich has quit IRC | 17:07 | |
| *** sar has joined #openstack-infra | 17:07 | |
| *** alex_xu_ has quit IRC | 17:07 | |
| chandankumar | clarkb: sure, i will check first then create | 17:07 |
| *** tlaxkit has quit IRC | 17:07 | |
| *** hongbin has quit IRC | 17:08 | |
| *** hongbin has joined #openstack-infra | 17:08 | |
| *** alex_xu has joined #openstack-infra | 17:08 | |
| *** jkilpatr has joined #openstack-infra | 17:09 | |
| *** jkilpatr_ has quit IRC | 17:09 | |
| *** mihalis68 has quit IRC | 17:10 | |
| *** sshnaidm|mtg is now known as sshnaidm | 17:10 | |
| *** felipemonteiro_ has quit IRC | 17:13 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add alembic.ini https://review.openstack.org/532260 | 17:14 |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Create stable/queens specific ACL https://review.openstack.org/532205 | 17:14 |
| *** smatzek_ has quit IRC | 17:15 | |
| pabelanger | clarkb: +2 | 17:15 |
| *** smatzek has joined #openstack-infra | 17:15 | |
| *** rkukura_ has joined #openstack-infra | 17:16 | |
| spotz | thanks for the fast turn around on those patches ttx | 17:16 |
| *** lucasagomes is now known as lucas-afk | 17:16 | |
| *** hashar is now known as hasharAway | 17:17 | |
| *** rkukura has quit IRC | 17:18 | |
| *** rkukura_ is now known as rkukura | 17:18 | |
| *** smatzek has quit IRC | 17:21 | |
| *** david-lyle has quit IRC | 17:23 | |
| ttx | spotz: no pb! | 17:23 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Updated WoO meeting information https://review.openstack.org/532173 | 17:25 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Fix error handling for pidfile https://review.openstack.org/532206 | 17:25 |
| *** jaosorior has quit IRC | 17:25 | |
| *** david-lyle has joined #openstack-infra | 17:26 | |
| *** rossella_s has quit IRC | 17:27 | |
| *** sree has joined #openstack-infra | 17:27 | |
| *** smatzek has joined #openstack-infra | 17:29 | |
| *** rossella_s has joined #openstack-infra | 17:29 | |
| *** smatzek has quit IRC | 17:30 | |
| *** smatzek has joined #openstack-infra | 17:30 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Use setup.py if we can't get the name from setup.cfg https://review.openstack.org/531936 | 17:31 |
| *** smatzek has quit IRC | 17:31 | |
| *** Apoorva has quit IRC | 17:31 | |
| *** smatzek has joined #openstack-infra | 17:32 | |
| *** makowals has joined #openstack-infra | 17:33 | |
| *** jamesmcarthur has quit IRC | 17:36 | |
| *** jascott1 has quit IRC | 17:36 | |
| *** jascott1 has joined #openstack-infra | 17:36 | |
| *** smatzek has quit IRC | 17:36 | |
| *** smatzek has joined #openstack-infra | 17:37 | |
| *** yamahata has quit IRC | 17:38 | |
| clarkb | any infra-roots willing to review https://review.openstack.org/#/c/529875/ and its depends on so that I can continue work to deploy logstash.o.o on xenial? (thank you corvus for reviewing) | 17:39 |
| *** smatzek has quit IRC | 17:39 | |
| *** smatzek has joined #openstack-infra | 17:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Fix error handling for pidfile https://review.openstack.org/532206 | 17:40 |
| *** yamamoto has joined #openstack-infra | 17:41 | |
| *** smatzek has quit IRC | 17:44 | |
| *** sree has quit IRC | 17:46 | |
| *** smatzek has joined #openstack-infra | 17:46 | |
| *** jamesmcarthur has joined #openstack-infra | 17:47 | |
| *** markvoelker has quit IRC | 17:47 | |
| *** trown is now known as trown|lunch | 17:48 | |
| *** sree has joined #openstack-infra | 17:49 | |
| *** yamamoto has quit IRC | 17:49 | |
| *** smatzek has quit IRC | 17:50 | |
| *** smatzek has joined #openstack-infra | 17:51 | |
| *** sree has quit IRC | 17:54 | |
| *** smatzek has quit IRC | 17:56 | |
| *** tesseract has quit IRC | 17:57 | |
| *** eumel8 has quit IRC | 17:58 | |
| *** sree has joined #openstack-infra | 17:58 | |
| *** edmondsw has quit IRC | 17:58 | |
| *** smatzek has joined #openstack-infra | 17:58 | |
| *** derekh has quit IRC | 18:00 | |
| *** edmondsw has joined #openstack-infra | 18:01 | |
| *** Apoorva has joined #openstack-infra | 18:02 | |
| *** Apoorva has quit IRC | 18:02 | |
| *** smatzek has quit IRC | 18:03 | |
| *** smatzek has joined #openstack-infra | 18:03 | |
| *** rmcall has joined #openstack-infra | 18:05 | |
| *** edmondsw has quit IRC | 18:06 | |
| *** david-lyle has quit IRC | 18:06 | |
| *** CrayZee has quit IRC | 18:09 | |
| *** jamesmcarthur has quit IRC | 18:14 | |
| *** jamesmcarthur has joined #openstack-infra | 18:14 | |
| *** CrayZee has joined #openstack-infra | 18:15 | |
| clarkb | Hello infra this is your 45 minute warning for the meeting. Any last minute agenda items I need to take into account? | 18:15 |
| mordred | clarkb: I'd like someone to magically bring me a sandwich - if I put that on the agenda will it happen? | 18:16 |
| clarkb | mordred: I don't think it will unfortunately | 18:16 |
| *** yamahata has joined #openstack-infra | 18:16 | |
| *** jkilpatr has quit IRC | 18:17 | |
| *** dtantsur is now known as dtantsur|afk | 18:17 | |
| *** CrayZee has quit IRC | 18:17 | |
| mordred | clarkb: that's a shame. we should fix that | 18:19 |
| *** jamesmcarthur has quit IRC | 18:19 | |
| *** ralonsoh has quit IRC | 18:21 | |
| *** jamesmcarthur has joined #openstack-infra | 18:22 | |
| *** pcichy has joined #openstack-infra | 18:24 | |
| openstackgerrit | Mark Hamzy proposed openstack/diskimage-builder master: fix the grub root label when an XFS disk label is truncated https://review.openstack.org/532279 | 18:24 |
| *** jamesmcarthur has quit IRC | 18:25 | |
| clarkb | mordred: now I want a sandwich | 18:25 |
| *** jamesmcarthur has joined #openstack-infra | 18:25 | |
| mordred | clarkb: see. this is a pervassive issue and deserves attention | 18:28 |
| *** jpena is now known as jpena|off | 18:29 | |
| *** dtruong has joined #openstack-infra | 18:29 | |
| *** jamesmcarthur has quit IRC | 18:30 | |
| *** Hunner_ is now known as Hunnur | 18:30 | |
| *** Hunnur is now known as Hunner | 18:31 | |
| *** esberglu has quit IRC | 18:31 | |
| *** esberglu has joined #openstack-infra | 18:31 | |
| gema | fungi: ack | 18:31 |
| gema | will do | 18:32 |
| *** Apoorva has joined #openstack-infra | 18:32 | |
| *** gema has quit IRC | 18:32 | |
| *** armax has joined #openstack-infra | 18:36 | |
| *** jkilpatr has joined #openstack-infra | 18:38 | |
| *** slaweq has joined #openstack-infra | 18:44 | |
| *** dprince has quit IRC | 18:45 | |
| *** gouthamr has quit IRC | 18:46 | |
| *** slaweq has quit IRC | 18:46 | |
| EmilienM | pabelanger: can you please review https://review.openstack.org/#/c/531030/ ? | 18:47 |
| EmilienM | pabelanger: and https://review.openstack.org/#/c/531229 | 18:47 |
| *** dimak has quit IRC | 18:47 | |
| *** snapiri has quit IRC | 18:48 | |
| *** gouthamr has joined #openstack-infra | 18:48 | |
| *** oanson has quit IRC | 18:48 | |
| *** robled has quit IRC | 18:50 | |
| *** felipemonteiro has joined #openstack-infra | 18:51 | |
| pabelanger | yah, we'll need to create AFS bits too | 18:53 |
| pabelanger | bookmarked for review in a bit | 18:53 |
| pabelanger | or happy to walk new infra-root with creating AFS things too | 18:54 |
| *** pcichy has quit IRC | 18:54 | |
| *** wolverineav has quit IRC | 18:54 | |
| *** robled has joined #openstack-infra | 18:54 | |
| *** robled has joined #openstack-infra | 18:54 | |
| *** wolverineav has joined #openstack-infra | 18:55 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] webapp: add optional admin endpoint https://review.openstack.org/531718 | 18:56 |
| clarkb | Infra meeting now | 19:00 |
| openstackgerrit | Merged openstack-infra/project-config master: Convert training-labs legacy jobs https://review.openstack.org/530124 | 19:00 |
| *** trown|lunch is now known as trown | 19:01 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Use setup.py if we can't get the name from setup.cfg https://review.openstack.org/531936 | 19:01 |
| *** sree has quit IRC | 19:05 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove swift3 legacy tox jobs https://review.openstack.org/530100 | 19:06 |
| openstackgerrit | Merged openstack-infra/project-config master: Use only base publish-openstack-sphinx-docs job https://review.openstack.org/532179 | 19:06 |
| *** jtomasek has quit IRC | 19:06 | |
| *** dizquierdo has quit IRC | 19:10 | |
| *** david-lyle has joined #openstack-infra | 19:10 | |
| *** markvoelker has joined #openstack-infra | 19:13 | |
| *** markvoelker has quit IRC | 19:13 | |
| *** dizquierdo has joined #openstack-infra | 19:14 | |
| *** quite has quit IRC | 19:14 | |
| *** quite has joined #openstack-infra | 19:16 | |
| *** markvoelker has joined #openstack-infra | 19:23 | |
| *** sambetts is now known as sambetts|afk | 19:23 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove swift3 legacy tox jobs https://review.openstack.org/530101 | 19:27 |
| ianw | fatal: loose object 17e6950aa08101b5f3bec0f2f9c32cfd5f51fa36 (stored in /opt/dib_cache/source-repositories/swift_bc937512e22692ed65870f5b957e4f67348e47ef/.git/objects/17/e6950aa08101b5f3bec0f2f9c32cfd5f51fa36) is corrupt | 19:28 |
| ianw | infra-root: ^ do we know about this during image builds? | 19:28 |
| clarkb | ianw: was that on 04? | 19:29 |
| clarkb | ianw: if so that server was part of a failed migration and had to be hard rebooted. Guessing that is fallout and we'll have to clear out the dib cache | 19:29 |
| ianw | ahh, would make sense. ok, i'll poke around at it later | 19:29 |
| *** apetrich has joined #openstack-infra | 19:30 | |
| *** priteau has quit IRC | 19:32 | |
| fungi | entirely possible 04 needs its cache cleaned out | 19:33 |
| fungi | no clue what might have corrupted when it hung | 19:33 |
| *** harlowja has joined #openstack-infra | 19:33 | |
| *** harlowja_ has joined #openstack-infra | 19:41 | |
| *** markvoelker has quit IRC | 19:42 | |
| *** markvoelker has joined #openstack-infra | 19:43 | |
| *** harlowja has quit IRC | 19:43 | |
| *** markvoelker has quit IRC | 19:48 | |
| *** gouthamr has quit IRC | 19:53 | |
| *** rlf1 has joined #openstack-infra | 19:53 | |
| *** dougwig has joined #openstack-infra | 19:54 | |
| *** harlowja_ has quit IRC | 19:55 | |
| *** harlowja has joined #openstack-infra | 19:55 | |
| clarkb | mmedvede: o/ | 20:02 |
| *** gouthamr has joined #openstack-infra | 20:02 | |
| *** dizquierdo has quit IRC | 20:04 | |
| mmedvede | clarkb: before I dove back into puppet-openstackci zuulv3 support again, wanted to make sure I am on the same page with were it needs to go | 20:04 |
| fungi | omelettes will be made, eggs will probably be broken | 20:05 |
| corvus | it's lunch time and everyone is talking about food | 20:05 |
| clarkb | mmedvede: sure, I haven't had a chance to review them but I think fungi has | 20:06 |
| clarkb | fungi: ^ did the changes make sense to you? | 20:07 |
| mmedvede | clarkb: original idea was to avoid breaking zuulv2 deployments when v3 is out, but it looks more like adding full support for v3 to puppet-openstackci now | 20:07 |
| fungi | clarkb: made enough sense for a +2 | 20:07 |
| clarkb | mmedvede: I think thats fine. It default to supporting the v2 stuff out of the box but then if you want v3 you flip the toggle and get v3 right? | 20:08 |
| mmedvede | most of zuulv3 deployment currently is in system-config, which needs to move to puppet-openstackci to avoid having it in two places | 20:08 |
| fungi | mmedvede: i think priority #1 is to make sure that when we merge feature/zuulv3 to master (in both zuul and nodepool repos) anyone with an updated puppet-openstackci deployment won't notice anything. priority #2 is to be able to use puppet-openstackci for deploying zuul v3. only the first one is a blocker for us moving forward with release | 20:09 |
| mmedvede | clarkb: yes, we want to keep v2 while also enabling v3 deployment with it | 20:09 |
| clarkb | fungi: ++ and I think we can do the transition from system-config to puppet-openstackci is a staged manner | 20:09 |
| corvus | then priority #3 would be flipping the default, so you have to enable the flag to use v2, and v3 is the default. | 20:09 |
| fungi | yep | 20:09 |
| clarkb | eg carry the duplication for a quick moment when we transition | 20:09 |
| corvus | after a nice long notification period | 20:09 |
| mmedvede | ok, so then this is for #1 https://review.openstack.org/#/c/523951/ | 20:10 |
| *** yamamoto has joined #openstack-infra | 20:11 | |
| pabelanger | I don't think we can use 0.4.0, unless we also setup zookeeper | 20:12 |
| pabelanger | for nodepool | 20:12 |
| *** rlf1 has quit IRC | 20:12 | |
| mmedvede | I followed the comment here https://review.openstack.org/#/c/523951/1/contrib/single_node_ci_site.pp@63 | 20:13 |
| fungi | pabelanger: great point, so the child change for that one becomes part of priority #1 if we go with the version bump there | 20:13 |
| mtreinish | puppet people, any idea what I did wrong for: http://logs.openstack.org/92/520192/9/check/legacy-system-config-puppet-syntax-3/4216a85/job-output.txt.gz#_2017-12-19_20_52_52_506400 ? | 20:13 |
| pabelanger | yah, the puppet modules for install zookeeper are pretty good too, so might be safe to add that | 20:13 |
| mtreinish | I can't reproduce that failure running puppet lint locally | 20:14 |
| pabelanger | fungi: mmedvede: but we'd need to also update nodepool.yaml for new zookeeper syntax with one of the patches | 20:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add Ansible version to job header https://review.openstack.org/532304 | 20:15 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/devstack-gate master: Switch from lib/neutron-legacy to lib/neutron https://review.openstack.org/436798 | 20:15 |
| *** rlf1 has joined #openstack-infra | 20:15 | |
| fungi | pabelanger: see, this is why we require >1 +2 review ;) | 20:15 |
| *** ldnunes has quit IRC | 20:15 | |
| fungi | mtreinish: that's a several-week-old error. i'd start with a recheck, we had a variety of interesting issues with our puppet jobs before the holidays | 20:16 |
| *** slaweq has joined #openstack-infra | 20:17 | |
| fungi | that one doesn't look familiar, but there were quite a few one-off error conditions my changes were hitting back then which are gone now | 20:17 |
| mtreinish | fungi: ah, ok. I guess that's what I get for taking weeks to look at error logs :) | 20:17 |
| fungi | it seemed like people in here were fixing bugs with jobs faster than they were being reported | 20:18 |
| *** jamesmcarthur has joined #openstack-infra | 20:18 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/devstack-gate master: Switch from lib/neutron-legacy to lib/neutron https://review.openstack.org/436798 | 20:20 |
| *** ijw has quit IRC | 20:20 | |
| clarkb | mmedvede: ok I'm off to find lunch then will dig into the openstackci changes | 20:20 |
| *** armaan has quit IRC | 20:22 | |
| *** armaan has joined #openstack-infra | 20:23 | |
| *** eharney has quit IRC | 20:24 | |
| *** rlf1 has quit IRC | 20:25 | |
| *** eharney has joined #openstack-infra | 20:26 | |
| *** rlf1 has joined #openstack-infra | 20:26 | |
| *** dsariel has quit IRC | 20:32 | |
| openstackgerrit | Merged openstack-infra/system-config master: Install yum-crontab on CentOS servers https://review.openstack.org/531401 | 20:32 |
| openstackgerrit | Merged openstack-infra/system-config master: Remove centos-release-openstack-ocata https://review.openstack.org/531837 | 20:32 |
| *** markvoelker has joined #openstack-infra | 20:33 | |
| *** rlf1 has quit IRC | 20:33 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] webapp: add optional admin endpoint https://review.openstack.org/531718 | 20:34 |
| *** dprince has joined #openstack-infra | 20:37 | |
| *** sar has quit IRC | 20:39 | |
| *** wolverineav has quit IRC | 20:39 | |
| *** wolverineav has joined #openstack-infra | 20:40 | |
| *** armaan has quit IRC | 20:44 | |
| *** dbecker has quit IRC | 20:45 | |
| *** dbecker_ has joined #openstack-infra | 20:45 | |
| ianw | i'm thinking about python3 virtualenv for https://review.openstack.org/#/c/532026/1/diskimage_builder/elements/pip-and-virtualenv/install.d/pip-and-virtualenv-source-install/04-install-pip | 20:47 |
| ianw | can anyone think why we actually need it? | 20:47 |
| ianw | it seems you just call "virtualenv -p pythonX" | 20:48 |
| *** yamamoto has quit IRC | 20:48 | |
| ianw | and what that's provided for doesn't matter | 20:48 |
| ianw | s/for/by/ | 20:48 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add support for a secure ZooKeeper configuration https://review.openstack.org/531882 | 20:49 |
| *** yamamoto has joined #openstack-infra | 20:49 | |
| clarkb | ianw: I think its there for python3 only installs? | 20:51 |
| clarkb | at least on debuntu they are separate I think | 20:51 |
| clarkb | greghaynes: ^ seems like you had a recent change around this | 20:51 |
| fungi | you can, at least, definitely use python2.7-installed virtualenv to build usable python3.x environments | 20:51 |
| fungi | (and vice versa) | 20:52 |
| greghaynes | clarkb: ianw pretty sure I didnt give any thought to using the builtin virtualenv bits when we did that | 20:53 |
| greghaynes | so it may be a good idea | 20:53 |
| *** armax has quit IRC | 20:53 | |
| clarkb | ya problem is if only using python 3 you just want virtualenv installed under python three so it works by default? | 20:53 |
| ianw | hmm, yeah, i'm pretty sure only python2-virtualenv provides the "virtualenv" binary | 20:53 |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-openstackci master: Lay groundwork for zuulv2/v3 coexisting https://review.openstack.org/523951 | 20:53 |
| clarkb | and not require python2 too | 20:54 |
| ianw | then python3 has "python3 -m venv" builtin, iirc? | 20:54 |
| *** yamamoto has quit IRC | 20:54 | |
| clarkb | ianw: 3.4 and greater? something like that | 20:54 |
| *** priteau has joined #openstack-infra | 20:55 | |
| fungi | python3.new.enough.for.us | 20:55 |
| ianw | at any rate, "virtualenv-3" seems to have no point | 20:55 |
| fungi | worth keeping in mind that pyenv != virtualenv (and least not completely, but maybe enough for our purposes?) | 20:56 |
| * fungi rereads https://docs.python.org/3/library/venv.html to confirm | 20:57 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Switch to python3-dev / python3-devel in bindep.txt https://review.openstack.org/532310 | 20:57 |
| greghaynes | ianw: This does make me wonder what that package actually does on py3k... | 20:58 |
| fungi | yeah, i dunno, for example, whether tox can use the built-in venv module or still requires the more full-featured virtualenv package | 20:58 |
| greghaynes | oh what fungi is getting at, huh | 20:58 |
| clarkb | tox still requires virtualenv aiui | 20:58 |
| *** rhallisey has quit IRC | 20:59 | |
| fungi | i'm pretty sure the venv module is not 1:1 equivalent to the virtualenv package | 20:59 |
| ianw | does tox actually like api call virtualenv bits, or just essentially system(virtualenv)? | 20:59 |
| clarkb | ianw: it uses python api I think | 20:59 |
| fungi | compare to the pip integration in py3k where it will basically bootstrap a full copy of pip into place | 21:00 |
| fungi | in the python{3} -m pip case it's using actual pip, but -m venv doesn't use pypa/virtualenv | 21:00 |
| ianw | hmm, ok. the issue is now with these fedora python3 changes, while we used to overwrite /usr/bin/virtualenv it now *creates* /usr/local/bin/virtualenv, so the python3 version effectively overrides the python2 version | 21:01 |
| *** wolverineav has quit IRC | 21:01 | |
| *** wolverineav has joined #openstack-infra | 21:02 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add support for a secure ZooKeeper configuration https://review.openstack.org/531882 | 21:03 |
| clarkb | ianw: I think maybe we want to do it only if no python2 is being installed? but I guess we don't really know that until late in the build | 21:04 |
| clarkb | or does fedora always have python2? | 21:05 |
| *** xarses_ has quit IRC | 21:05 | |
| ianw | well, pip-and-virtualenv the dib package is always bringing in python2 stuff, it doesn't have a notion of python3 only | 21:05 |
| *** xarses_ has joined #openstack-infra | 21:06 | |
| clarkb | ah | 21:06 |
| mtreinish | fungi: hmm, looks like I'm still hitting it after recheck: http://logs.openstack.org/92/520192/9/check/legacy-system-config-puppet-syntax-3/da53e54/job-output.txt.gz#_2018-01-09_20_37_17_029756 | 21:06 |
| fungi | at least now we know it's probably legit? | 21:07 |
| mtreinish | heh, I guess | 21:07 |
| ianw | so yeah, tox depends on python3 virtualenv, so that solves that -- we want it in the python3 environment. if "virtualenv-3" as an entrypoint script is useful is questionable | 21:07 |
| *** jascott1 has quit IRC | 21:07 | |
| fungi | mtreinish: i'll see if i can spot why it's angry about subunit_db_server.pp line 60 | 21:08 |
| ianw | and we want to make sure "virtualenv" called just bare is python2, for backwards compat, so we'll have to make sure to clear out any /usr/local/bin overrides from python3 | 21:08 |
| clarkb | mtreinish: looking at the failure now | 21:08 |
| fungi | huh, that's the mysql_grant block | 21:09 |
| clarkb | mtreinish: you need to name your define | 21:09 |
| clarkb | so mysql_grant { ' | 21:09 |
| clarkb | er | 21:09 |
| clarkb | so mysql_grant { 'some name here': | 21:10 |
| fungi | ahh, yep | 21:10 |
| fungi | it needs a name | 21:10 |
| mtreinish | hmm, ok. I wonder why puppet-lint locally isn't failing on that | 21:10 |
| fungi | looks like it's the puppet 3.x syntax checker, not the linter, raising this | 21:10 |
| mtreinish | ah, that would explain it | 21:10 |
| clarkb | you'd think a linter would catch those problems though | 21:11 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Do not merge: print role path https://review.openstack.org/532314 | 21:11 |
| clarkb | (this is why we have the noop apply tests) | 21:11 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Add subunit2sql check database server and workers https://review.openstack.org/520192 | 21:11 |
| *** makowals has quit IRC | 21:12 | |
| *** markvoelker has quit IRC | 21:13 | |
| *** markvoelker has joined #openstack-infra | 21:13 | |
| *** jascott1 has joined #openstack-infra | 21:14 | |
| clarkb | pabelanger: fungi mmedvede ok want to resurrect the previous discussion about nodepool versions now that I've had lunch and reviewed the first change in the stack | 21:17 |
| clarkb | pabelanger: fungi mmedvede the default zuul and nodepool revisions are for master | 21:17 |
| clarkb | pabelanger: fungi mmedvede I think that means we may want new tags at tip of master then default to those in openstackci | 21:17 |
| clarkb | pabelanger: its fine to use 0.4.0 or later there since the default is master anyways | 21:17 |
| *** markvoelker has quit IRC | 21:17 | |
| clarkb | in fact it would be broken to set the revision to something older than 0.4.0 as we would break anyone using zk with the defaults | 21:18 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: DNM print role_path https://review.openstack.org/532319 | 21:18 |
| *** florianf has quit IRC | 21:19 | |
| clarkb | mmedvede: I also think we may want to honor zuul_revision and nodepool_revision if set so that people can use older installs without updating their puppet | 21:20 |
| *** wolverineav has quit IRC | 21:20 | |
| clarkb | mmedvede: and then don't have two sets of version parameters but let zuulv2 value determine if we install a zuulv2 using the set revision or a zuulv3? | 21:20 |
| clarkb | (this is somewhat tricky) | 21:20 |
| pabelanger | clarkb: yah, I guess that means people running latest master of nodepool, had already setup zookeeper them self | 21:21 |
| *** wolverineav has joined #openstack-infra | 21:21 | |
| clarkb | pabelanger: yes exactly | 21:21 |
| clarkb | and since that is the default I think we want to preserve that | 21:21 |
| pabelanger | but, would be a new service for users coming from 0.3.1 | 21:21 |
| clarkb | (to prevent them from firedrilling again) | 21:21 |
| pabelanger | so, we should add zookeeper into puppet-openstackci and update nodepool.yaml syntax to be safe? | 21:22 |
| clarkb | ya that might be the best option for everyone, I think ship has somewhat sailed on that though and we don't have to make that a blocker | 21:22 |
| clarkb | eg its broken this way now. Sorry. We'll try to fix it | 21:23 |
| clarkb | I'm going to try and brain dump allthis into the change review itself now | 21:24 |
| pabelanger | sure, wfm | 21:24 |
| *** smatzek has quit IRC | 21:24 | |
| *** smatzek has joined #openstack-infra | 21:24 | |
| *** smatzek has quit IRC | 21:24 | |
| *** smatzek has joined #openstack-infra | 21:24 | |
| *** smatzek has quit IRC | 21:24 | |
| mmedvede | what is a minimal change required to keep people from breaking once feature/zuulv3 is merged? | 21:25 |
| *** smatzek has joined #openstack-infra | 21:25 | |
| pabelanger | I don't think there is a minimal change TBH, a lot of syntax changed for both nodepool and zuul | 21:26 |
| mmedvede | I remember going down the path of imagining logic required to "automagically" deduce what version is needed, and it was ugly | 21:26 |
| clarkb | mmedvede: ya I'll have a comment on the change shortly that I think accomplishes this | 21:26 |
| clarkb | or something similar to it | 21:27 |
| *** smatzek has quit IRC | 21:30 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Drop tools/owners.py https://review.openstack.org/532327 | 21:30 |
| clarkb | mmedvede: ok posted, can you let me know if that makes sense? | 21:31 |
| clarkb | I think if we can get consensus on what I've sketched up we can push new tags | 21:31 |
| *** kgiusti has left #openstack-infra | 21:31 | |
| clarkb | mmedvede: also note that my puppet there is not valid its more pseudo code | 21:32 |
| dtruong | Hi. I'm trying to understand how tempest test should be triggered by zuul v3 for a openstack project. It looks like a lot of openstack projects (e.g. ceilometer) set DEVSTACK_GATE_TEMPEST_NOTESTS=1 in run.yaml and then execute tempest from a post_test_hook.sh | 21:32 |
| dtruong | Is that the recommended way? | 21:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Fix typo with yum-cron package / service https://review.openstack.org/532331 | 21:32 |
| pabelanger | clarkb: ianw: ^fixes a typo with yum-cron package. For some reason, I thought it was yum-crontab | 21:33 |
| *** kjackal has quit IRC | 21:33 | |
| *** armax has joined #openstack-infra | 21:34 | |
| clarkb | dtruong: ceilometer appears to be using the legacy tempest jobs which rely on devstack-gate which provides the hook interface | 21:34 |
| mmedvede | clarkb: ok, I think it makes sense. I'll update the patch | 21:35 |
| *** rlandy is now known as rlandy|brb | 21:35 | |
| clarkb | dtruong: the zuulv3 native stuff that is going into tempest right now does not use devstack-gate anymore. Instead there are ansible playbooks and roles https://git.openstack.org/cgit/openstack/tempest/tree/playbooks and https://git.openstack.org/cgit/openstack/tempest/tree/roles | 21:36 |
| clarkb | dtruong: if you are starting from scratch here it may be easier to just run tempest using the roles above | 21:36 |
| clarkb | dtruong: and avoid devstack-gate entirely (though I'm not sure how ready this stuff is for consumption outside of tempest itself) | 21:36 |
| clarkb | mmedvede: did you see my note about using the current tags for now then we can decide if we need new tags before making the switch | 21:37 |
| clarkb | (I think that will make it easier to review | 21:37 |
| *** rlf1 has joined #openstack-infra | 21:38 | |
| *** pcaruana has quit IRC | 21:38 | |
| *** slaweq has quit IRC | 21:39 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Move CRD tests to test_gerrit_legacy_crd https://review.openstack.org/531886 | 21:39 |
| dtruong | @clarkb: are there any openstack projects that are using those ansible playbooks and roles already? | 21:39 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add skipped CRD tests https://review.openstack.org/531887 | 21:39 |
| mmedvede | clarkb: saw that. | 21:39 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Replace change.equals with __eq__ https://review.openstack.org/532333 | 21:39 |
| *** david-lyle has quit IRC | 21:40 | |
| clarkb | dtruong: tempest is, the jobs are defined in tempest's .zuul.yaml | 21:40 |
| clarkb | dtruong: but I'm nto sure of any non tempest projects yet | 21:40 |
| *** dave-mccowan has quit IRC | 21:40 | |
| *** rlf1 has quit IRC | 21:41 | |
| *** eharney has quit IRC | 21:41 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Add subunit2sql check database server and workers https://review.openstack.org/520192 | 21:43 |
| *** threestrands has joined #openstack-infra | 21:44 | |
| *** threestrands has quit IRC | 21:44 | |
| *** threestrands has joined #openstack-infra | 21:44 | |
| *** trown is now known as trown|outtypewww | 21:45 | |
| *** Goneri has quit IRC | 21:45 | |
| clarkb | corvus: what is involved in adding authenticated smtp connections to lists.o.o particularly now that we are splitting mailman installs? | 21:49 |
| clarkb | corvus: this would be a layer above mailman in exim right? as it accepts the port 25 connections | 21:50 |
| clarkb | Zara: hey on the zuulv3-issues ehterpad you have listed that storyboard js builds act differently. Is that still a problem? I know there was owrk on those jobs to zuulv3ify them I think | 21:51 |
| corvus | clarkb: yeah, it'd be some exim config. i don't think the multiple mm install needs to be involved, unless you want it to be (different users for different domains). | 21:52 |
| *** rmcall has quit IRC | 21:52 | |
| *** yamamoto has joined #openstack-infra | 21:52 | |
| clarkb | corvus: ya I expect we can get away without that separation since this is a narrow use case of release jobs not being rejected when sending email | 21:52 |
| * clarkb files a storyboard bug with that info | 21:53 | |
| corvus | clarkb: it's now really easy to add a new exim router. though we may want to switch to the submission port, 587, for this, which i don't think we're listening on now. | 21:54 |
| clarkb | corvus: and tls it? | 21:54 |
| corvus | so that might be a little more puppet-exim work to turn that on | 21:54 |
| corvus | clarkb: yeah, submission generally implies starttls + auth | 21:54 |
| corvus | clarkb: (as opposed to 465, which is the ssl port (no starttls)) | 21:54 |
| fungi | i have a working exim config with starttls available on 25 and 587, where auth is required for relaying and tls is required for auth | 21:55 |
| fungi | if needed | 21:55 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: DNM print role_path https://review.openstack.org/532319 | 21:55 |
| fungi | though mine is for the sa-exim variant (where spamassassin is integrated as an inline filter to allow rejection at receipt), it's likely more or less the same in those bits | 21:56 |
| fungi | might have some extra cruft related to bypassing spam classification for authenticated submission | 21:56 |
| *** jamesmcarthur has quit IRC | 21:58 | |
| clarkb | fungi: corvus https://storyboard.openstack.org/#!/story/2001458 has been filed | 21:58 |
| fungi | gracias | 21:58 |
| *** jamesmcarthur has joined #openstack-infra | 21:59 | |
| *** yamamoto has quit IRC | 21:59 | |
| SotK | clarkb: the storyboard js-build issue was fixed by the zuulv3ification | 21:59 |
| clarkb | SotK: thank you for confirming | 21:59 |
| clarkb | mordred: there is an issues with jobs entry for tox siblings. have we largely fixed those at this point? | 22:00 |
| clarkb | mordred: I know you are making changes to tox siblings now too | 22:00 |
| *** rlandy|brb is now known as rlandy | 22:02 | |
| *** yamahata has quit IRC | 22:02 | |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-openstackci master: Lay groundwork for zuulv2/v3 coexisting https://review.openstack.org/523951 | 22:02 |
| *** jamesmcarthur has quit IRC | 22:03 | |
| *** dprince has quit IRC | 22:04 | |
| *** rcernin has joined #openstack-infra | 22:06 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: webapp: add optional admin endpoint https://review.openstack.org/531718 | 22:10 |
| clarkb | mmedvede: ya I think we want latest tags because the old default is master | 22:10 |
| clarkb | mmedvede: which means anyone using the old defaults is already using the latest tag or newer code | 22:10 |
| clarkb | mmedvede: I think we may want to tag tip of master on both repos if they have moved and then use that in the final patchset though | 22:10 |
| clarkb | but think getting agreement on the structure before we worry about that is worthwhile | 22:10 |
| *** jamesmcarthur has joined #openstack-infra | 22:13 | |
| clarkb | mmedvede: and commented. Otherwise I think that looks great | 22:14 |
| clarkb | infra-root can you please review https://review.openstack.org/523951 I think it is close to being ready for merging. Mostly looking to see if others like he approach and whether or not you think we need newer tags than we already have | 22:15 |
| mmedvede | clarkb: with regards to tags, could we just use sha temporarily? | 22:15 |
| mmedvede | i.e. 'git rev-parse master' | 22:15 |
| clarkb | mmedvede: possibly, though you can't fetch a sha1 so depending on how the puppet stuff is implemented that may not work ( I want to say the puppet git stuff supports raw sha1s though) | 22:16 |
| mordred | clarkb: I believe it does | 22:16 |
| *** aeng has joined #openstack-infra | 22:17 | |
| clarkb | ya the good news is if it doesn't work it will fail safely | 22:17 |
| clarkb | and we can fix it | 22:17 |
| clarkb | mmedvede: it is also easy enough for us to make tags | 22:18 |
| clarkb | I just have to plug in my gpg key device | 22:18 |
| clarkb | and typey typey | 22:18 |
| * clarkb looks to see how old these tags are compared to master | 22:18 | |
| *** jamesmcarthur has quit IRC | 22:18 | |
| mmedvede | clarkb: 2.6.0 of zuul is basically master | 22:19 |
| mmedvede | master is ahead of it by commit/revert | 22:19 |
| *** aeng has quit IRC | 22:19 | |
| clarkb | nodepool 0.4.0 is fairly old though | 22:19 |
| clarkb | and probably deserves a tag | 22:19 |
| clarkb | master includes a fix for jenkins | 22:20 |
| clarkb | so ya we probably want a nodepool tag | 22:20 |
| mmedvede | 0.4.1? | 22:20 |
| mmedvede | or probably 0.5.0 | 22:21 |
| clarkb | I think 0.5.0 | 22:21 |
| clarkb | its largely fixes but the jenkisn thing could be seen as a feature as it changes how it talks to jenkins aiui | 22:21 |
| clarkb | there is also some keypair cleanup that changes a config interface I think | 22:22 |
| clarkb | so ya 0.5.0 | 22:22 |
| clarkb | infra-root ^ should I go ahead and push an 0.5.0 for nodepool. Then mmedvede can update 523951 and we can merge that and then merge feature branches? | 22:23 |
| corvus | clarkb: wfm | 22:23 |
| fungi | sounds fine | 22:23 |
| clarkb | and ya zuul doesn't need a new tag as its just zuulv3 testing pipelines that were updated after the 2.6.0 tag | 22:24 |
| clarkb | new nodepool tag coming right up | 22:24 |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-openstackci master: Lay groundwork for zuulv2/v3 coexisting https://review.openstack.org/523951 | 22:25 |
| *** rfolco has quit IRC | 22:25 | |
| *** yee37932 has joined #openstack-infra | 22:26 | |
| *** andreww has joined #openstack-infra | 22:26 | |
| ianw | i'm just running git fsck over the dib cache (rather than recreating it) and will do some builds when i think things are fixed up | 22:27 |
| ianw | on nb04 i should say | 22:27 |
| fungi | thanks ianw | 22:27 |
| ianw | i think disk i/o is the bottleneck, so it's probably just as fast to re-clone everything :) | 22:28 |
| *** jamesmcarthur has joined #openstack-infra | 22:28 | |
| *** yee379 has quit IRC | 22:28 | |
| fungi | certainly less effort | 22:29 |
| *** markvoelker has joined #openstack-infra | 22:29 | |
| *** markvoelker has quit IRC | 22:30 | |
| clarkb | nodepool 0.5.0 pushed | 22:30 |
| clarkb | mmedvede: ^ fyi | 22:30 |
| *** xarses has quit IRC | 22:30 | |
| mmedvede | clarkb: thanks, I have preemptively updated 523951 to use 0.5.0 above | 22:31 |
| clarkb | mmedvede: +2 | 22:32 |
| *** jamesmcarthur has quit IRC | 22:33 | |
| *** zerick_ has quit IRC | 22:33 | |
| clarkb | I keep not wanting to dive into things like cleaning up our infra-specs changes thinking that I will have to context switch and patch all the things shortly | 22:34 |
| *** zerick has joined #openstack-infra | 22:34 | |
| clarkb | but then no patches arrive | 22:34 |
| fungi | better to just assume ubuntu has no fixes until it has fixes | 22:35 |
| *** gouthamr has quit IRC | 22:35 | |
| *** afred312 has quit IRC | 22:36 | |
| ianw | ok, too many corrupt repos. starting again | 22:38 |
| *** priteau has quit IRC | 22:40 | |
| *** bobh has quit IRC | 22:41 | |
| clarkb | pabelanger: is AJaeger's comment in https://review.openstack.org/#/c/495463/ about it being safe to abandon correct? There is also https://review.openstack.org/#/c/503751/6 | 22:42 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: webapp: add optional admin endpoint https://review.openstack.org/531718 | 22:46 |
| *** yamahata has joined #openstack-infra | 22:48 | |
| *** aeng has joined #openstack-infra | 22:48 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/openstack-zuul-jobs master: Install ovsdbapp master in neutron + ovsdbapp periodic job https://review.openstack.org/532350 | 22:48 |
| *** zerick has quit IRC | 22:49 | |
| clarkb | config-core ^^ is another random item that needed fixing on the zuulv3-issues etherpad that is easier to just fix | 22:49 |
| clarkb | quick review much appreciated (especially if we can get it merged so that periodic jobs test it tonight) | 22:49 |
| *** gouthamr has joined #openstack-infra | 22:53 | |
| *** aeng has quit IRC | 22:56 | |
| *** hongbin has quit IRC | 22:57 | |
| *** e0ne has joined #openstack-infra | 22:58 | |
| ianw | clarkb: i'm not getting what you mean by "neturon doesn't depend on itself"? | 23:02 |
| *** jlk is now known as Guest76 | 23:03 | |
| *** hasharAway has quit IRC | 23:04 | |
| clarkb | ianw: currently that job modifies requirements.txt in neutron to update entries for neutron to point at neutron's git repo rather than neutron on pypi | 23:04 |
| clarkb | which fails because those aren't valid text replacements | 23:04 |
| clarkb | neutron does however have a dep on ovsdbapp which that job wants to update | 23:04 |
| *** sdague has quit IRC | 23:04 | |
| *** aeng has joined #openstack-infra | 23:05 | |
| clarkb | ianw: I can update the commit message and fix the .txt and try ot be more verbose if you think it will help | 23:05 |
| *** Guest76 is now known as jlk | 23:06 | |
| *** afred312 has joined #openstack-infra | 23:06 | |
| clarkb | dmsimard: there is an item on zuulv3-issues more closely related to ara than anything else. Playbook sytnax errors are not visible in ara | 23:09 |
| clarkb | dmsimard: is that something that is even fixable? if so I don't mind filing a bug but based on how ara/ansible works I question if that is feasible | 23:09 |
| *** markvoelker has joined #openstack-infra | 23:11 | |
| *** markvoelker has quit IRC | 23:11 | |
| *** afred312 has quit IRC | 23:11 | |
| ianw | clarkb: oh right, so it's running under a project name of neutron, but actually it's ovsdbapp | 23:11 |
| ianw | actually it's testing | 23:11 |
| clarkb | ya the job is executing from within a neutron context but it intends on testing ovsdbapp master with neutron (rather than whatever the last release on pypi was) | 23:12 |
| *** aeng has quit IRC | 23:16 | |
| ianw | yay, trusty has built. i think i'll have to trigger a few of the others | 23:17 |
| *** afred312 has joined #openstack-infra | 23:20 | |
| mwhahaha | what's the deal with the TIMED_OUT jobs today? | 23:20 |
| clarkb | mwhahaha: I was just starting to look at that over in #zuul. The first one I found I think just has a job timeout that is too short (neutron can't run all its unittests in under 30 minutes reliably I guess) | 23:20 |
| clarkb | but I'm digging more | 23:20 |
| mwhahaha | well we're not hitting that, but it seems that nodes are getting flakey and disapearing or something | 23:21 |
| mwhahaha | resulting in some TIMED_OUT results | 23:21 |
| clarkb | mwhahaha: do you have an example? I Just pciked the top of gate queue when I started looking | 23:22 |
| clarkb | (whcih ended up being the issue I descriebd) | 23:22 |
| mwhahaha | http://logs.openstack.org/43/502943/25/gate/tripleo-ci-centos-7-undercloud-oooq/26ab603/job-output.txt.gz | 23:22 |
| mwhahaha | just hung | 23:22 |
| mwhahaha | http://logs.openstack.org/43/502943/25/gate/tripleo-ci-centos-7-undercloud-oooq/26ab603/job-output.txt.gz#_2018-01-09_22_36_32_085205 | 23:23 |
| mwhahaha | looks like the node went away or something | 23:23 |
| mwhahaha | i'd have to dig through, but i've noticed a few today | 23:23 |
| mwhahaha | that one is currently in the tripleo gate waiting to fail out | 23:24 |
| mwhahaha | I have a feeling http://zuulv3.openstack.org/stream.html?uuid=e110618dcbcf488e88ea67858d7297dd&logfile=console.log is another one like that | 23:24 |
| clarkb | http://logs.openstack.org/43/502943/25/gate/tripleo-ci-centos-7-undercloud-oooq/26ab603/zuul-info/inventory.yaml shows it ran in inap | 23:24 |
| clarkb | I wonder if nap is still having the arp issues and that is breaking ssh | 23:24 |
| mwhahaha | that's the top one from the tripleo gate | 23:24 |
| mwhahaha | been running for a while | 23:24 |
| clarkb | will have to make a logstash query and see if it is cloud specific | 23:25 |
| *** jlk is now known as Guest76 | 23:29 | |
| *** aeng has joined #openstack-infra | 23:31 | |
| clarkb | mwhahaha: logstash says its fairly evenly distributed across a few clouds and regions so not cloud specific | 23:31 |
| clarkb | corvus: I'm noticing we don't put the zuul executor details in the logsatsh stuff | 23:32 |
| mwhahaha | :/ | 23:32 |
| * clarkb checks if zuul status can show me that | 23:32 | |
| *** sdague has joined #openstack-infra | 23:34 | |
| *** gmann has joined #openstack-infra | 23:35 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Index zuul executor https://review.openstack.org/532357 | 23:38 |
| clarkb | corvus: ^ that addresses some debuggability in zuulv3 | 23:38 |
| clarkb | now to figure out what I can figure out with existing data | 23:38 |
| *** pahuang has quit IRC | 23:42 | |
| *** markvoelker has joined #openstack-infra | 23:43 | |
| *** markvoelker has quit IRC | 23:43 | |
| mordred | clarkb: ++ | 23:43 |
| corvus | +2 | 23:44 |
| corvus | +3even | 23:44 |
| mordred | clarkb, corvus, pabelanger, ianw: if you get bored and have a sec, https://review.openstack.org/#/c/531919/ https://review.openstack.org/#/c/531936/ https://review.openstack.org/#/c/532216/ are all good to go | 23:45 |
| *** tosky has quit IRC | 23:46 | |
| openstackgerrit | Merged openstack-infra/project-config master: Index zuul executor https://review.openstack.org/532357 | 23:49 |
| openstackgerrit | Clark Boylan proposed openstack-infra/git-review master: Fix compare_review's use of fetch_review https://review.openstack.org/532359 | 23:50 |
| clarkb | fungi: ^ this came up in #gerrit just now | 23:50 |
| *** ianw has quit IRC | 23:50 | |
| *** ianw has joined #openstack-infra | 23:51 | |
| mordred | clarkb: +2 | 23:53 |
| clarkb | one wonders why pyflakes didn't complain | 23:53 |
| *** ijw has joined #openstack-infra | 23:54 | |
| clarkb | but currently digging through nodepool and zuul logs so that will have to wait for later | 23:54 |
| clarkb | mwhahaha: nothing I can find shows that nodepool deleted the node before the job was done | 23:55 |
| clarkb | mwhahaha: so its not that | 23:55 |
| mwhahaha | yea i don't think it's nodepool | 23:57 |
| mwhahaha | it's likely a locked up system or network issues | 23:57 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Change patchset column to string https://review.openstack.org/532237 | 23:57 |
| ianw | clarkb: is git-review ci in a good state? i had that on my todo list as it seemed all recent changes failed | 23:57 |
| mwhahaha | but hard to tell without logs :D | 23:57 |
| clarkb | mwhahaha: http://logs.openstack.org/43/502943/25/gate/tripleo-ci-centos-7-undercloud-oooq/26ab603/job-output.txt.gz#_2018-01-09_16_48_04_076569 is where it appears to timeout the first time. Are there other logs for the delorean builds to see if anything else will gives us clues? | 23:59 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Slack driver https://review.openstack.org/530521 | 23:59 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Add support for a secure ZooKeeper configuration https://review.openstack.org/531882 | 23:59 |
| clarkb | ianw: it may not be post zuulv3? | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!