| *** jamesmcarthur has joined #openstack-infra | 00:00 | |
| *** dingyichen has joined #openstack-infra | 00:04 | |
| *** agopi has joined #openstack-infra | 00:07 | |
| *** jamesmcarthur has quit IRC | 00:09 | |
| *** eernst has quit IRC | 00:27 | |
| *** yamamoto has joined #openstack-infra | 00:39 | |
| *** jamesmcarthur has joined #openstack-infra | 00:45 | |
| *** yamamoto has quit IRC | 00:45 | |
| *** jamesmcarthur has quit IRC | 00:47 | |
| *** jamesmcarthur has joined #openstack-infra | 00:47 | |
| *** jamesmcarthur has quit IRC | 00:49 | |
| *** jamesmcarthur has joined #openstack-infra | 00:49 | |
| *** wolverineav has joined #openstack-infra | 00:49 | |
| *** bobh has joined #openstack-infra | 00:53 | |
| *** bdodd has quit IRC | 00:54 | |
| *** wolverineav has quit IRC | 00:54 | |
| *** jamesmcarthur has quit IRC | 00:56 | |
| *** jamesmcarthur has joined #openstack-infra | 00:58 | |
| *** bobh has quit IRC | 01:02 | |
| tommylikehu | hey anyone could take a look at this patch? https://review.openstack.org/#/c/535201/ thanks :) | 01:04 |
|---|---|---|
| ianw | tommylikehu: best to firstly rebase that so it applies (which might get it some attention), but pinging in #openstack-qa will probably help more for tempest things after that | 01:09 |
| tommylikehu | ianw: ha, thanks :) | 01:10 |
| *** edmondsw has joined #openstack-infra | 01:11 | |
| *** hongbin has joined #openstack-infra | 01:13 | |
| *** edmondsw has quit IRC | 01:15 | |
| *** jamesmcarthur has quit IRC | 01:22 | |
| *** s-shiono has joined #openstack-infra | 01:40 | |
| *** yamamoto has joined #openstack-infra | 01:41 | |
| *** yamamoto has quit IRC | 01:47 | |
| *** yamamoto has joined #openstack-infra | 01:51 | |
| *** jamesmcarthur has joined #openstack-infra | 01:53 | |
| *** yamamoto has quit IRC | 01:54 | |
| *** jamesmcarthur has quit IRC | 01:59 | |
| *** jamesmcarthur has joined #openstack-infra | 02:07 | |
| *** gongysh has joined #openstack-infra | 02:09 | |
| *** akhilaki_ has joined #openstack-infra | 02:11 | |
| *** akhilaki has quit IRC | 02:11 | |
| *** akhilaki_ has quit IRC | 02:12 | |
| *** akhilaki has joined #openstack-infra | 02:12 | |
| *** yamamoto has joined #openstack-infra | 02:45 | |
| *** psachin has joined #openstack-infra | 02:48 | |
| *** bobh has joined #openstack-infra | 02:50 | |
| *** jamesmcarthur has quit IRC | 02:50 | |
| *** jamesmcarthur has joined #openstack-infra | 02:51 | |
| *** dave-mcc_ has quit IRC | 02:54 | |
| *** bobh has quit IRC | 02:54 | |
| *** dhill_ has quit IRC | 02:56 | |
| *** edmondsw has joined #openstack-infra | 02:59 | |
| *** bdodd has joined #openstack-infra | 03:00 | |
| *** jamesmcarthur has quit IRC | 03:00 | |
| *** jamesmcarthur has joined #openstack-infra | 03:00 | |
| *** gongysh has quit IRC | 03:01 | |
| *** graphene has quit IRC | 03:03 | |
| *** edmondsw has quit IRC | 03:04 | |
| *** graphene has joined #openstack-infra | 03:04 | |
| *** yamamoto has quit IRC | 03:16 | |
| *** yamamoto has joined #openstack-infra | 03:16 | |
| *** yamamoto has quit IRC | 03:20 | |
| *** psachin has quit IRC | 03:22 | |
| *** graphene has quit IRC | 03:32 | |
| *** graphene has joined #openstack-infra | 03:33 | |
| *** psachin has joined #openstack-infra | 03:38 | |
| mtreinish | infra-root: it looks like the openstack-qa list (which is only an archive) is getting spammed right now | 03:42 |
| mtreinish | pleia2 and I configured it to autodiscard messages sent to it, but I still get a ntotification email for all of those | 03:43 |
| mtreinish | the idea was back then that if a someone didn't know the list was dead I could point them in the right direction | 03:45 |
| mtreinish | but I think at this point I probably dont need to get notifications anymore | 03:45 |
| jhesketh | mtreinish: where is the list hosted? I can't see it on lists.o.o | 03:47 |
| mtreinish | infra-root: I'm not really in a place to check the mailman settings (on vacation) but if someone could take a look for me that'd be great | 03:47 |
| mtreinish | jhesketh: its there we just hide it by default because its not active | 03:47 |
| jhesketh | mtreinish: right, I also can't see it in system-config, but maybe I'm blind | 03:47 |
| mtreinish | jhesketh: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa | 03:48 |
| jhesketh | yeah, was this set up before puppet? | 03:48 |
| mtreinish | it might have been, we retired the list like 4 years ago | 03:49 |
| *** akhilaki has quit IRC | 03:51 | |
| *** slaweq has quit IRC | 03:52 | |
| *** udesale has joined #openstack-infra | 03:55 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/system-config master: Create openstack-qa-owner alias to go to spam https://review.openstack.org/584702 | 03:58 |
| jhesketh | mtreinish: once that ^ merges, I can change the owner of the list to the new alias and then you should be free of spam | 03:59 |
| *** slaweq has joined #openstack-infra | 03:59 | |
| *** lpetrut has joined #openstack-infra | 04:00 | |
| *** eernst has joined #openstack-infra | 04:04 | |
| *** jamesmcarthur has quit IRC | 04:04 | |
| *** graphene has quit IRC | 04:05 | |
| *** graphene has joined #openstack-infra | 04:06 | |
| *** ykarel has joined #openstack-infra | 04:09 | |
| *** ramishra has joined #openstack-infra | 04:20 | |
| *** jamesdenton has quit IRC | 04:27 | |
| *** owalsh has quit IRC | 04:43 | |
| *** lpetrut has quit IRC | 04:45 | |
| *** edmondsw has joined #openstack-infra | 04:47 | |
| *** ijw has quit IRC | 04:47 | |
| *** ijw has joined #openstack-infra | 04:48 | |
| *** edmondsw has quit IRC | 04:52 | |
| *** yamamoto has joined #openstack-infra | 04:52 | |
| chkumar|ruck | tristanC: one ssh error issue from today's morning http://logs.openstack.org/24/567224/64/check/tripleo-ci-centos-7-undercloud-oooq/d4fe68f/job-output.txt.gz#_2018-07-23_02_17_01_516558 | 04:54 |
| *** janki has joined #openstack-infra | 04:54 | |
| *** hongbin has quit IRC | 04:55 | |
| *** pgadiya has joined #openstack-infra | 05:02 | |
| *** yamamoto has quit IRC | 05:03 | |
| *** janki has quit IRC | 05:03 | |
| *** janki has joined #openstack-infra | 05:04 | |
| tristanC | chkumar|ruck: seems like an issue between executor and packethost cloud, check the zuul-info/inventory.yaml to get the affected cloud name | 05:05 |
| *** yamamoto has joined #openstack-infra | 05:07 | |
| *** eernst has quit IRC | 05:08 | |
| *** gongysh has joined #openstack-infra | 05:16 | |
| *** pcichy has joined #openstack-infra | 05:20 | |
| *** gongysh has quit IRC | 05:21 | |
| *** armaan has joined #openstack-infra | 05:26 | |
| *** yamamoto has quit IRC | 05:28 | |
| *** armaan has quit IRC | 05:35 | |
| *** cshastri has joined #openstack-infra | 05:37 | |
| *** links has joined #openstack-infra | 05:38 | |
| *** udesale_ has joined #openstack-infra | 05:42 | |
| *** udesale has quit IRC | 05:45 | |
| *** quiquell|off is now known as quiquell | 05:48 | |
| *** pcichy has quit IRC | 05:52 | |
| *** zhurong has joined #openstack-infra | 05:58 | |
| *** pgadiya has quit IRC | 06:00 | |
| *** psachin has quit IRC | 06:01 | |
| *** psachin` has joined #openstack-infra | 06:01 | |
| *** vivsoni has joined #openstack-infra | 06:04 | |
| *** AJaeger has quit IRC | 06:07 | |
| openstackgerrit | zhurong proposed openstack-infra/irc-meetings master: Murano convert weekly meeting to office hours https://review.openstack.org/584734 | 06:10 |
| *** jtomasek has joined #openstack-infra | 06:11 | |
| *** AJaeger has joined #openstack-infra | 06:12 | |
| *** jtomasek has quit IRC | 06:13 | |
| *** mschuppert has joined #openstack-infra | 06:13 | |
| *** dhajare has joined #openstack-infra | 06:17 | |
| *** lpetrut has joined #openstack-infra | 06:29 | |
| *** jtomasek has joined #openstack-infra | 06:30 | |
| *** edmondsw has joined #openstack-infra | 06:35 | |
| *** ccamacho has joined #openstack-infra | 06:39 | |
| *** edmondsw has quit IRC | 06:39 | |
| *** links has quit IRC | 06:41 | |
| *** udesale__ has joined #openstack-infra | 06:43 | |
| *** udesale_ has quit IRC | 06:45 | |
| *** dtantsur|afk is now known as dtantsur | 06:45 | |
| *** jamesmcarthur has joined #openstack-infra | 06:47 | |
| *** jamesmcarthur has quit IRC | 06:51 | |
| *** jaosorior has joined #openstack-infra | 06:55 | |
| *** armaan has joined #openstack-infra | 06:59 | |
| *** links has joined #openstack-infra | 07:00 | |
| *** florianf has joined #openstack-infra | 07:00 | |
| *** zoli is now known as zli|wfh | 07:00 | |
| *** alexchadin has joined #openstack-infra | 07:00 | |
| *** zli|wfh is now known as zoli|wfh | 07:00 | |
| *** pcaruana has joined #openstack-infra | 07:02 | |
| *** pcichy has joined #openstack-infra | 07:04 | |
| *** evrardjp_ is now known as evrardjp | 07:09 | |
| *** zoli|wfh is now known as zoli | 07:11 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Turn on the future parser for groups.o.o https://review.openstack.org/584638 | 07:11 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Turn on the future parser for graphite.o.o https://review.openstack.org/584745 | 07:11 |
| *** jpich has joined #openstack-infra | 07:13 | |
| mpeterson | Hi, I've got a question re the lower-constraints job, the job is currently succeeeding even though it has a neutron version for which it shouldn't succeed. The thing is that tox_install_sibling_packages is replacing neutron with a version that's higher, even lower-constraints.txt directs neutron==12.0.0 | 07:19 |
| *** pguimaraes has joined #openstack-infra | 07:19 | |
| mpeterson | is that an expected behavior? | 07:19 |
| mpeterson | perhaps dhellmann has some input on this | 07:19 |
| *** yamamoto has joined #openstack-infra | 07:20 | |
| *** numans has quit IRC | 07:20 | |
| mpeterson | e.g https://review.openstack.org/#/c/584626/ | 07:21 |
| *** numans has joined #openstack-infra | 07:21 | |
| *** tesseract has joined #openstack-infra | 07:21 | |
| *** quiquell is now known as quiquell|bbl | 07:22 | |
| *** zhurong has quit IRC | 07:24 | |
| *** yamamoto has quit IRC | 07:24 | |
| openstackgerrit | Merged openstack-infra/puppet-askbot master: Use earlier six release https://review.openstack.org/559624 | 07:25 |
| openstackgerrit | Merged openstack-infra/puppet-kibana master: Fix template variables https://review.openstack.org/582169 | 07:34 |
| *** links has quit IRC | 07:34 | |
| *** tosky has joined #openstack-infra | 07:36 | |
| *** rpittau has joined #openstack-infra | 07:38 | |
| *** ykarel is now known as ykarel|lunch | 07:38 | |
| *** graphene has quit IRC | 07:45 | |
| *** graphene has joined #openstack-infra | 07:47 | |
| *** links has joined #openstack-infra | 07:47 | |
| *** zhurong has joined #openstack-infra | 07:48 | |
| *** abelur has joined #openstack-infra | 07:49 | |
| abelur | JJB has tox-cover jobs filing with the below error "Unsupported parameters for (stat) module: register Supported parameters include: checksum_algorithm, follow, get_attributes, get_checksum, get_md5, get_mime, path". Would anyone here know what is causing this? | 07:50 |
| abelur | http://logs.openstack.org/81/584581/3/check/tox-cover/b23a83f/ | 07:50 |
| *** quiquell|bbl is now known as quiquell | 07:52 | |
| AJaeger | abelur: add a comment "recheck" to rerun the tests - this was fixed yesterday. | 07:55 |
| abelur | AJaeger: awesome, trying that now | 07:56 |
| openstackgerrit | Dirk Mueller proposed openstack-infra/project-config master: Drop requirements-check from renderspec https://review.openstack.org/584753 | 08:01 |
| *** armaan has quit IRC | 08:04 | |
| *** hashar has joined #openstack-infra | 08:05 | |
| openstackgerrit | Merged openstack-infra/puppet-redis master: Add Xenial support https://review.openstack.org/581627 | 08:09 |
| *** Adri2000 has quit IRC | 08:15 | |
| *** dtantsur is now known as dtantsur|brb | 08:18 | |
| *** owalsh has joined #openstack-infra | 08:22 | |
| *** edmondsw has joined #openstack-infra | 08:23 | |
| *** armaan has joined #openstack-infra | 08:25 | |
| *** Adri2000 has joined #openstack-infra | 08:25 | |
| *** owalsh_ has joined #openstack-infra | 08:27 | |
| *** owalsh has quit IRC | 08:27 | |
| *** edmondsw has quit IRC | 08:28 | |
| *** ykarel|lunch is now known as ykarel | 08:28 | |
| *** pbourke has quit IRC | 08:30 | |
| *** derekh has joined #openstack-infra | 08:33 | |
| *** gongysh has joined #openstack-infra | 08:35 | |
| *** pbourke has joined #openstack-infra | 08:36 | |
| openstackgerrit | wangxiyuan proposed openstack-infra/project-config master: Update oslo.limit testing jobs https://review.openstack.org/579690 | 08:39 |
| *** graphene has quit IRC | 08:40 | |
| *** graphene has joined #openstack-infra | 08:41 | |
| *** e0ne has joined #openstack-infra | 08:43 | |
| *** zhurong has quit IRC | 08:47 | |
| openstackgerrit | Rico Lin proposed openstack-infra/project-config master: Remove heat-templates python jobs https://review.openstack.org/584774 | 08:49 |
| *** jaosorior has quit IRC | 08:50 | |
| openstackgerrit | Rico Lin proposed openstack-infra/project-config master: Remove heat-templates python jobs https://review.openstack.org/584774 | 09:01 |
| *** wolverineav has joined #openstack-infra | 09:01 | |
| *** r-mibu has joined #openstack-infra | 09:02 | |
| *** dingyichen has quit IRC | 09:03 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Add variables to project https://review.openstack.org/584230 | 09:04 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: doc: Move zuul variable references to a section https://review.openstack.org/584779 | 09:04 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Use definition list for job status https://review.openstack.org/584780 | 09:04 |
| *** wolverineav has quit IRC | 09:06 | |
| *** pguimaraes has quit IRC | 09:14 | |
| *** s-shiono has quit IRC | 09:17 | |
| *** alexchadin has quit IRC | 09:17 | |
| *** alexchadin has joined #openstack-infra | 09:23 | |
| *** CrayZee has joined #openstack-infra | 09:24 | |
| slaweq | andreaf: hi, are You around? | 09:24 |
| abelur | AJaeger: thanks that worked :) | 09:29 |
| *** jistr|off is now known as jistr | 09:29 | |
| andreaf | slaweq: hi - what's up? | 09:33 |
| *** links has quit IRC | 09:34 | |
| slaweq | hi andreaf, can You maybe take a look at https://review.openstack.org/#/c/578796/ | 09:34 |
| slaweq | andreaf: I'm trying to migrate multinode job to zuulv3 but I probably are doing something wrong and on compute-1 node there is no openstack services running | 09:35 |
| slaweq | job logs are in http://logs.openstack.org/96/578796/3/check/neutron-tempest-plugin-dvr-multinode-scenario-zuulv3/464aefa/ | 09:35 |
| slaweq | maybe You will have an idea what is wrong with this patch :) | 09:35 |
| *** yamamoto has joined #openstack-infra | 09:36 | |
| andreaf | slaweq looking.... | 09:36 |
| slaweq | thx andreaf | 09:36 |
| *** dtantsur|brb is now known as dtantsur | 09:38 | |
| AJaeger | abelur: great! | 09:39 |
| *** links has joined #openstack-infra | 09:41 | |
| *** yamamoto has quit IRC | 09:41 | |
| *** owalsh_ is now known as owalsh | 09:41 | |
| *** panda|off is now known as panda | 09:48 | |
| andreaf | slaweq: still looking... the variables setup looks ok, services are enabled on the subnode | 09:50 |
| andreaf | slaweq: however this http://logs.openstack.org/96/578796/3/check/neutron-tempest-plugin-dvr-multinode-scenario-zuulv3/464aefa/ara-report/result/ad7b22ea-d383-41d9-b432-e8a93b5457fb/ stops execution of the orchestrate-devstack on compute1 | 09:51 |
| andreaf | slaweq: which is then preventing devstack from running | 09:51 |
| andreaf | slaweq: I'm not sure why that would fail though | 09:51 |
| andreaf | slaweq: perhaps you have the tls-proxy disabled which stops the CA code from running? | 09:52 |
| *** finucannot has quit IRC | 09:53 | |
| tosky | but the case when tls-proxy is disabled should already be properly handled | 09:54 |
| andreaf | slaweq: I think I know what's going on - you disabled tls proxy on the master but not on the subnode | 09:54 |
| *** stephenfin has joined #openstack-infra | 09:55 | |
| andreaf | tosky: it is handled here http://git.openstack.org/cgit/openstack-dev/devstack/tree/roles/orchestrate-devstack/tasks/main.yaml#n15 | 09:55 |
| andreaf | slaweq: you need to disable tls-proxy on your subnode as well | 09:56 |
| *** sshnaidm|afk is now known as sshnaidm|rover | 09:56 | |
| openstackgerrit | Adam Harwell proposed openstack/diskimage-builder master: cache-url requires curl https://review.openstack.org/542350 | 09:56 |
| andreaf | slaweq: adding something like this should fix the issue http://paste.openstack.org/show/726434/ | 09:57 |
| tosky | yes, I remember that snippet :) | 09:58 |
| *** cshastri has quit IRC | 09:58 | |
| slaweq | andreaf: thx a lot | 09:58 |
| slaweq | I knew that You will help me with that and figure out what's wrong much faster than me :) | 09:58 |
| andreaf | slaweq: np, let me know if that helps | 09:58 |
| slaweq | sure, I will | 09:58 |
| slaweq | thx once again | 09:59 |
| *** rosmaita has joined #openstack-infra | 10:00 | |
| *** sshnaidm|rover is now known as sshnaidm|ruck | 10:00 | |
| *** chkumar|ruck is now known as chkumar|rover | 10:01 | |
| *** kiennt26 has joined #openstack-infra | 10:01 | |
| *** pblaho has joined #openstack-infra | 10:08 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: packaging: Remove support for pyN requirement files https://review.openstack.org/580417 | 10:08 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: builddoc: Remove support for Sphinx < 1.6 https://review.openstack.org/580416 | 10:08 |
| *** edmondsw has joined #openstack-infra | 10:11 | |
| *** kiennt26 has quit IRC | 10:14 | |
| *** edmondsw has quit IRC | 10:16 | |
| *** kiennt26 has joined #openstack-infra | 10:20 | |
| *** jaosorior has joined #openstack-infra | 10:28 | |
| *** alexchadin has quit IRC | 10:30 | |
| *** janki has quit IRC | 10:33 | |
| *** kiennt26 has quit IRC | 10:33 | |
| *** alexchadin has joined #openstack-infra | 10:36 | |
| *** janki has joined #openstack-infra | 10:39 | |
| *** quiquell is now known as quiquell|bbl | 10:39 | |
| *** alexchadin has quit IRC | 10:41 | |
| *** graphene has quit IRC | 10:41 | |
| *** graphene has joined #openstack-infra | 10:42 | |
| *** links has quit IRC | 10:44 | |
| *** alexchadin has joined #openstack-infra | 10:45 | |
| *** links has joined #openstack-infra | 10:48 | |
| *** zoli is now known as zoli|lunch | 10:53 | |
| *** neiloy has joined #openstack-infra | 10:57 | |
| *** bhavik1 has joined #openstack-infra | 10:57 | |
| *** kiennt26 has joined #openstack-infra | 10:58 | |
| *** lucasagomes is now known as lucas-hungry | 11:03 | |
| *** gfidente has joined #openstack-infra | 11:03 | |
| *** gfidente has joined #openstack-infra | 11:03 | |
| *** lucas-hungry is now known as lucasagomes | 11:03 | |
| *** quiquell|bbl is now known as quiquell | 11:03 | |
| *** dave-mccowan has joined #openstack-infra | 11:05 | |
| *** roman_g has joined #openstack-infra | 11:06 | |
| *** pguimaraes has joined #openstack-infra | 11:09 | |
| *** links has quit IRC | 11:13 | |
| *** links has joined #openstack-infra | 11:15 | |
| *** gongysh has quit IRC | 11:15 | |
| *** gongysh has joined #openstack-infra | 11:16 | |
| *** dhill_ has joined #openstack-infra | 11:21 | |
| *** udesale__ has quit IRC | 11:25 | |
| *** numans has quit IRC | 11:30 | |
| *** pguimaraes has quit IRC | 11:34 | |
| *** bhavik1 has quit IRC | 11:37 | |
| *** numans has joined #openstack-infra | 11:38 | |
| *** jamesdenton has joined #openstack-infra | 11:41 | |
| *** tpsilva has joined #openstack-infra | 11:42 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool master: Ignore files produced by tox-cover https://review.openstack.org/584823 | 11:45 |
| *** rosmaita has quit IRC | 11:56 | |
| *** quiquell is now known as quiquell|lunch | 11:59 | |
| *** gongysh has quit IRC | 12:00 | |
| *** boden has joined #openstack-infra | 12:04 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool master: Invalidate az cache on bad request https://review.openstack.org/582746 | 12:04 |
| *** zoli|lunch is now known as zoli | 12:11 | |
| *** zoli is now known as zoli|wfh | 12:12 | |
| *** zoli|wfh is now known as zoli | 12:12 | |
| *** edmondsw has joined #openstack-infra | 12:13 | |
| *** pguimaraes has joined #openstack-infra | 12:13 | |
| *** agopi has quit IRC | 12:14 | |
| *** jcoufal has joined #openstack-infra | 12:14 | |
| *** kiennt26 has quit IRC | 12:15 | |
| *** Bhujay has joined #openstack-infra | 12:21 | |
| *** quiquell|lunch is now known as quiquell | 12:24 | |
| *** trown|outtypewww is now known as trown | 12:24 | |
| *** yamamoto has joined #openstack-infra | 12:26 | |
| *** yamamoto has quit IRC | 12:30 | |
| *** neiloy has quit IRC | 12:37 | |
| *** beagles_afk is now known as beagles | 12:38 | |
| *** rlandy has joined #openstack-infra | 12:39 | |
| *** wolverineav has joined #openstack-infra | 12:40 | |
| *** jistr is now known as jistr|mtg | 12:41 | |
| *** eernst has joined #openstack-infra | 12:42 | |
| *** yamamoto has joined #openstack-infra | 12:43 | |
| *** agopi has joined #openstack-infra | 12:44 | |
| *** yamamoto has quit IRC | 12:46 | |
| *** jamesmcarthur has joined #openstack-infra | 12:48 | |
| *** yamamoto has joined #openstack-infra | 12:49 | |
| *** jamesmcarthur has quit IRC | 12:52 | |
| *** graphene has quit IRC | 12:53 | |
| *** mriedem has joined #openstack-infra | 12:53 | |
| *** arxcruz is now known as arxcruz|off | 12:53 | |
| *** yamamoto has quit IRC | 12:54 | |
| *** graphene has joined #openstack-infra | 12:54 | |
| *** jbadiapa_ is now known as jbadiapa | 12:55 | |
| *** camunoz has joined #openstack-infra | 12:55 | |
| *** udesale has joined #openstack-infra | 12:56 | |
| *** simondodsley_ has quit IRC | 13:00 | |
| *** simondodsley_ has joined #openstack-infra | 13:01 | |
| *** yamamoto has joined #openstack-infra | 13:01 | |
| *** yamamoto has quit IRC | 13:03 | |
| *** jesusaur has quit IRC | 13:05 | |
| *** gongysh has joined #openstack-infra | 13:06 | |
| *** eernst has quit IRC | 13:12 | |
| *** gfidente has quit IRC | 13:17 | |
| *** numans_ has joined #openstack-infra | 13:17 | |
| *** numans has quit IRC | 13:19 | |
| *** ccamacho has quit IRC | 13:20 | |
| *** alexchadin has quit IRC | 13:21 | |
| openstackgerrit | Merged openstack-infra/system-config master: Create openstack-qa-owner alias to go to spam https://review.openstack.org/584702 | 13:25 |
| openstackgerrit | Merged openstack-infra/nodepool master: Invalidate az cache on bad request https://review.openstack.org/582746 | 13:25 |
| openstackgerrit | Merged openstack-infra/nodepool master: Ignore files produced by tox-cover https://review.openstack.org/584823 | 13:25 |
| *** graphene has quit IRC | 13:25 | |
| *** kgiusti has joined #openstack-infra | 13:25 | |
| *** camunoz has quit IRC | 13:26 | |
| *** alexchadin has joined #openstack-infra | 13:26 | |
| *** Bhujay has quit IRC | 13:26 | |
| *** sthussey has joined #openstack-infra | 13:26 | |
| *** graphene has joined #openstack-infra | 13:27 | |
| *** numans has joined #openstack-infra | 13:27 | |
| jhesketh | fungi: shall I edit the openstack-qa list owner (and possibly moderators) manually for 584702? | 13:28 |
| *** numans_ has quit IRC | 13:30 | |
| *** lbragstad has joined #openstack-infra | 13:31 | |
| fungi | jhesketh: should be able to use the mailman cli and set the list to just drop all posts into oblivion | 13:32 |
| fungi | is it still in use for automated notifications of any kind? | 13:32 |
| fungi | i know the periodic job failures used to report there | 13:32 |
| * fungi isn't really here, has to be careful his wife doesn't see him doing openstack on vacation | 13:33 | |
| *** numans has quit IRC | 13:33 | |
| jhesketh | fungi: are you talking about autodiscarding everything? | 13:34 |
| jhesketh | because I believe that's what it's already doing, but the problem is mtreinish gets notified of each one | 13:34 |
| fungi | i think there's a setting to not notify the list owner on discards. i should check the infra ml moderation queue anyway so i'll see if i can spot a likely setting | 13:36 |
| jhesketh | fungi: okay, let me konw if you do, or if you run out of time/get caught I can poke around | 13:37 |
| *** numans has joined #openstack-infra | 13:38 | |
| *** jcoufal has quit IRC | 13:40 | |
| *** camunoz has joined #openstack-infra | 13:40 | |
| *** alexchadin has quit IRC | 13:40 | |
| *** alexchadin has joined #openstack-infra | 13:41 | |
| *** alexchadin has quit IRC | 13:41 | |
| *** alexchadin has joined #openstack-infra | 13:42 | |
| *** bdodd has quit IRC | 13:42 | |
| *** alexchadin has quit IRC | 13:42 | |
| fungi | if generic_nonmember_action is set to discard, then setting forward_auto_discards to no ought to solve it, i think | 13:42 |
| *** bdodd has joined #openstack-infra | 13:42 | |
| *** alexchadin has joined #openstack-infra | 13:43 | |
| *** alexchadin has quit IRC | 13:43 | |
| fungi | "forward_auto_discards (privacy): Should messages from non-members, which are automatically discarded, be forwarded to the list moderator?" | 13:43 |
| fungi | defaults to yes | 13:43 |
| *** alexchadin has joined #openstack-infra | 13:43 | |
| *** alexchadin has quit IRC | 13:44 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Fix delegate_to for ansible synchronize https://review.openstack.org/584656 | 13:47 |
| *** ihar has joined #openstack-infra | 13:48 | |
| *** r-daneel has joined #openstack-infra | 13:49 | |
| *** myoung has joined #openstack-infra | 13:49 | |
| *** gongysh has quit IRC | 13:50 | |
| *** gongysh has joined #openstack-infra | 13:50 | |
| jhesketh | fungi: cool, did you want to set those for openstack-qa while you're there (assuming you still are), otherwise I can | 13:50 |
| jhesketh | and whether or not you want to change the owner to openstack-qa-owner since we've created it now | 13:51 |
| *** ykarel is now known as ykarel|away | 13:51 | |
| *** jcoufal has joined #openstack-infra | 13:51 | |
| *** bdodd has quit IRC | 13:51 | |
| *** ykarel|away has quit IRC | 13:51 | |
| *** eernst has joined #openstack-infra | 13:54 | |
| *** hongbin has joined #openstack-infra | 13:59 | |
| *** jamesmcarthur has joined #openstack-infra | 14:00 | |
| *** quiquell is now known as quiquell|off | 14:02 | |
| *** alexchadin has joined #openstack-infra | 14:02 | |
| *** eharney has joined #openstack-infra | 14:03 | |
| *** camunoz has quit IRC | 14:06 | |
| *** camunoz has joined #openstack-infra | 14:06 | |
| *** camunoz has quit IRC | 14:07 | |
| *** alexchadin has quit IRC | 14:08 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack-infra/bindep master: [WIP] Add real life testing of --profiles https://review.openstack.org/584896 | 14:08 |
| *** psachin` has quit IRC | 14:08 | |
| *** gongysh has quit IRC | 14:08 | |
| *** ccamacho has joined #openstack-infra | 14:09 | |
| *** bobh has joined #openstack-infra | 14:11 | |
| *** camunoz has joined #openstack-infra | 14:13 | |
| *** jistr|mtg is now known as jistr | 14:13 | |
| *** jamesmca_ has joined #openstack-infra | 14:15 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Update README and add TESTING similar to Zuul repo https://review.openstack.org/581348 | 14:16 |
| *** derekh has quit IRC | 14:19 | |
| *** yolanda__ has joined #openstack-infra | 14:19 | |
| pabelanger | frickler: do you mind removing your -2 on https://review.openstack.org/511823/ I'm happy to take over on merging / testing so of these changes related to logging work mordred created | 14:19 |
| *** yolanda_ has quit IRC | 14:21 | |
| *** rpioso|afk is now known as rpioso | 14:21 | |
| *** links has quit IRC | 14:22 | |
| *** derekh has joined #openstack-infra | 14:23 | |
| *** yolanda__ is now known as yolanda | 14:26 | |
| *** alexchadin has joined #openstack-infra | 14:26 | |
| mordred | pabelanger: I think frickler is on vacation - maybe we should remove it for him, since I believe it was a procedural -2 | 14:29 |
| *** armaan has quit IRC | 14:32 | |
| *** armaan has joined #openstack-infra | 14:32 | |
| *** dave-mccowan has quit IRC | 14:33 | |
| pabelanger | mordred: wfm, I'm not sure how to remove somebody else -2 however | 14:34 |
| mordred | pabelanger: I have removed it for you | 14:37 |
| mordred | pabelanger: (if you're an admin there's a little x next to each person's vote which lets you remove it) | 14:38 |
| *** Bhujay has joined #openstack-infra | 14:40 | |
| pabelanger | ah | 14:40 |
| pabelanger | cool | 14:40 |
| fungi | jhesketh: the owner address is already openstack-qa-owner. that's baked into mailman. messages for the owner go there and then get distributed to the configured owners | 14:42 |
| *** dave-mccowan has joined #openstack-infra | 14:42 | |
| fungi | jhesketh: the alias we're adding in the change above merely shadows that since we process local aliases before we process mailman's built-in ones | 14:43 |
| jhesketh | fungi: right, I'm clearly unfamiliar with mailman in detail, but does that mean Matt will no longer see discard messages? | 14:44 |
| fungi | it depends some on mailman's internal routing. if the discard notifications are looping out to exim and then back to mailman, it'll get shunted by that alias we added | 14:46 |
| fungi | which is to say, i'm not sure but i suppose we can check the exim mainlog to see if it's still sending to him | 14:46 |
| * fungi needs to disappear again to work on lunch | 14:48 | |
| *** bdodd has joined #openstack-infra | 14:49 | |
| *** neiloy has joined #openstack-infra | 14:49 | |
| *** jesusaur has joined #openstack-infra | 14:51 | |
| jhesketh | okay, I'll poke at logs tomorrow to see if I can help, otherwise it's past my bed time | 14:51 |
| *** felipemonteiro has joined #openstack-infra | 14:51 | |
| jhesketh | o/ | 14:51 |
| mordred | jhesketh: sleep well! | 14:52 |
| *** felipemonteiro has quit IRC | 14:56 | |
| *** kgiusti has quit IRC | 14:57 | |
| *** beekneemech is now known as bnemec | 15:00 | |
| *** bnemec is now known as beekneemech | 15:01 | |
| *** beekneemech is now known as bnemec | 15:01 | |
| *** eernst has quit IRC | 15:01 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: Add tenant yaml validation option to zuul client https://review.openstack.org/574265 | 15:01 |
| *** eharney has quit IRC | 15:02 | |
| *** bobh has quit IRC | 15:07 | |
| *** bobh has joined #openstack-infra | 15:07 | |
| *** manjeets has quit IRC | 15:12 | |
| *** kgiusti has joined #openstack-infra | 15:14 | |
| *** yamamoto has joined #openstack-infra | 15:15 | |
| *** eharney has joined #openstack-infra | 15:15 | |
| fungi | yeah, looking at the exim mainlog, it may still be sending to him on the order of ~5/hr | 15:16 |
| fungi | i'll see if i can work out how to disable forward_auto_discards | 15:17 |
| *** xgerman_ has quit IRC | 15:18 | |
| *** mrhillsman has quit IRC | 15:18 | |
| *** samueldmq has quit IRC | 15:19 | |
| *** mshalamov has quit IRC | 15:19 | |
| *** patrickeast has quit IRC | 15:19 | |
| *** _Cyclone_ has quit IRC | 15:19 | |
| *** mshalamov has joined #openstack-infra | 15:19 | |
| *** hashar has quit IRC | 15:19 | |
| *** larainema has quit IRC | 15:19 | |
| *** diablo_rojo_phon has quit IRC | 15:19 | |
| *** berendt_ has quit IRC | 15:19 | |
| *** csatari has quit IRC | 15:19 | |
| *** clarkb has quit IRC | 15:19 | |
| *** mancdaz has quit IRC | 15:19 | |
| *** yamamoto has quit IRC | 15:20 | |
| *** ihar has quit IRC | 15:20 | |
| *** sthussey has quit IRC | 15:20 | |
| *** AJaeger has quit IRC | 15:20 | |
| *** apetrich has quit IRC | 15:20 | |
| *** sdoran has quit IRC | 15:20 | |
| *** d34dh0r53 has quit IRC | 15:20 | |
| *** lamt has quit IRC | 15:20 | |
| *** TheJulia has quit IRC | 15:20 | |
| *** andreaf has quit IRC | 15:21 | |
| *** mrhillsman has joined #openstack-infra | 15:21 | |
| *** patrickeast has joined #openstack-infra | 15:21 | |
| *** samueldmq has joined #openstack-infra | 15:21 | |
| *** xgerman_ has joined #openstack-infra | 15:21 | |
| *** ihar has joined #openstack-infra | 15:22 | |
| *** lamt has joined #openstack-infra | 15:22 | |
| *** chason has quit IRC | 15:22 | |
| *** diablo_rojo_phon has joined #openstack-infra | 15:23 | |
| *** larainema has joined #openstack-infra | 15:23 | |
| *** sthussey has joined #openstack-infra | 15:23 | |
| *** sdoran has joined #openstack-infra | 15:23 | |
| *** csatari has joined #openstack-infra | 15:23 | |
| *** berendt_ has joined #openstack-infra | 15:23 | |
| *** chason has joined #openstack-infra | 15:24 | |
| *** felipemonteiro has joined #openstack-infra | 15:24 | |
| *** sdoran has quit IRC | 15:24 | |
| *** sdoran has joined #openstack-infra | 15:24 | |
| *** lamt is now known as Guest23581 | 15:24 | |
| *** andreaf has joined #openstack-infra | 15:25 | |
| *** d34dh0r53 has joined #openstack-infra | 15:25 | |
| panda | pabelanger: I'll put some test in place. two underclouds an two ipxe nodes in the same network. I'll let you know if my dhcp filter works. | 15:25 |
| *** _Cyclone_ has joined #openstack-infra | 15:25 | |
| *** sdoran has quit IRC | 15:25 | |
| pabelanger | panda: great, thanks | 15:26 |
| *** hashar has joined #openstack-infra | 15:26 | |
| *** sdoran has joined #openstack-infra | 15:26 | |
| *** hashar has quit IRC | 15:26 | |
| *** hashar has joined #openstack-infra | 15:26 | |
| *** sdoran has quit IRC | 15:27 | |
| *** sdoran has joined #openstack-infra | 15:27 | |
| *** mancdaz has joined #openstack-infra | 15:27 | |
| *** bdodd has quit IRC | 15:28 | |
| *** pguimaraes has quit IRC | 15:28 | |
| *** bdodd has joined #openstack-infra | 15:29 | |
| *** clarkb has joined #openstack-infra | 15:33 | |
| *** AJaeger has joined #openstack-infra | 15:33 | |
| *** apetrich has joined #openstack-infra | 15:33 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add container spec https://review.openstack.org/560136 | 15:34 |
| *** imacdonn has quit IRC | 15:34 | |
| *** TheJulia has joined #openstack-infra | 15:35 | |
| clarkb | I've noticed we also leaked images | 15:38 |
| clarkb | so cleaning that up | 15:38 |
| clarkb | I think the lesson here is that after a prolonged cloud outage nodepool leaks stuff | 15:39 |
| clarkb | images are particularly problematic because they are a multi state transition process | 15:39 |
| *** r-mibu has quit IRC | 15:40 | |
| *** bdodd_ has joined #openstack-infra | 15:40 | |
| clarkb | there are not new leaked floating ips or ports though | 15:42 |
| corvus | Shrews: has nodepool been restarted so we can verify it before release this week? | 15:43 |
| *** bdodd has quit IRC | 15:43 | |
| *** graphene has quit IRC | 15:43 | |
| clarkb | corvus: aiui no, and there were a couple more changes shrews wanted to get in first (I reivewed a few end of week last week) | 15:44 |
| Shrews | corvus: no. Waiting for some merges first | 15:44 |
| *** gfidente has joined #openstack-infra | 15:44 | |
| corvus | Shrews: links? | 15:44 |
| *** gfidente has quit IRC | 15:44 | |
| *** gfidente has joined #openstack-infra | 15:44 | |
| corvus | i can review them with urgency now :) | 15:44 |
| *** alexchadin has quit IRC | 15:44 | |
| *** sileht has joined #openstack-infra | 15:45 | |
| clarkb | looking through gerrits merged changes list they may have managed to merge | 15:45 |
| *** graphene has joined #openstack-infra | 15:45 | |
| Shrews | corvus: 564687 | 15:45 |
| clarkb | shrews should confirm | 15:45 |
| Shrews | Last one | 15:45 |
| corvus | ack. i think i agree with the general consensus that it's a possible footgun, but perhaps one that's necessary in some environemnts. | 15:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool master: Docs: increase visibility of env secrets warning https://review.openstack.org/584950 | 15:48 |
| *** gyee has joined #openstack-infra | 15:51 | |
| *** jcoufal_ has joined #openstack-infra | 15:52 | |
| *** lpetrut has quit IRC | 15:53 | |
| *** anteaya has joined #openstack-infra | 15:54 | |
| *** jcoufal has quit IRC | 15:55 | |
| *** udesale has quit IRC | 15:56 | |
| *** jcoufal_ has quit IRC | 15:56 | |
| *** graphene has quit IRC | 15:57 | |
| *** graphene has joined #openstack-infra | 15:58 | |
| *** rpittau has quit IRC | 15:59 | |
| *** graphene has quit IRC | 16:00 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Use detail listing in integration testing https://review.openstack.org/584542 | 16:00 |
| *** dave-mccowan has quit IRC | 16:00 | |
| *** graphene has joined #openstack-infra | 16:01 | |
| corvus | clarkb, tobiash: zuul memory use is still basically unchanged since the last restart. | 16:03 |
| *** rpittau has joined #openstack-infra | 16:04 | |
| openstackgerrit | Merged openstack-infra/nodepool master: builder: support setting diskimage env-vars in secure configuration https://review.openstack.org/564687 | 16:04 |
| tobiash | \o/ | 16:04 |
| corvus | i'd probably give it another day or two before we declare victory (or defeat), but so far so good | 16:04 |
| *** ijw has quit IRC | 16:05 | |
| *** ijw has joined #openstack-infra | 16:05 | |
| corvus | Shrews: do you want to take care of restarting nodepool? need any help? | 16:07 |
| *** ijw has quit IRC | 16:07 | |
| *** ijw has joined #openstack-infra | 16:07 | |
| corvus | i guess we'll want to wait a bit longer, until that's actually deployed :) | 16:07 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack-infra/bindep master: Fix user profile display https://review.openstack.org/584955 | 16:07 |
| clarkb | corvus: Shrews we probably want to restart builders too since there was a change to the builder code | 16:08 |
| Shrews | corvus: I'm out for lunch so can't do it myself right now anyway. | 16:08 |
| pabelanger | corvus: clarkb: mind a review on https://review.openstack.org/584548/ think that is what is stopping ze11.o.o from coming online properly | 16:08 |
| corvus | Shrews: perfect, we'll regroup when you're back | 16:08 |
| pabelanger | pip10 thing* | 16:08 |
| evrardjp | I know it's fairly unimportant, but I want to learn: anyone wants to help me write tests for https://review.openstack.org/584955 ? | 16:08 |
| *** e0ne has quit IRC | 16:08 | |
| corvus | pabelanger: +3 | 16:08 |
| pabelanger | thanks you | 16:08 |
| pabelanger | thank* | 16:08 |
| *** kjackal has quit IRC | 16:09 | |
| *** kjackal_ has quit IRC | 16:09 | |
| *** dave-mccowan has joined #openstack-infra | 16:10 | |
| corvus | evrardjp: i'm pretty sure "want to learn" and "write tests" are actually the most important things. :) unfortunately, i don't know much about the test framework in bindep; perhaps clarkb, pabelanger, or ianw could point you in the right direction... | 16:10 |
| evrardjp | haha ofc corvus : ) | 16:12 |
| evrardjp | let me rephrase for clarkb pabelanger or ianw : I'd like to add a real functional test into bindep/tests/test_main,with a file I'd drop somewhere. Not adding a unit test. | 16:12 |
| evrardjp | I could too, but here it's obviously lacking some functional testing. | 16:13 |
| *** pblaho has quit IRC | 16:13 | |
| clarkb | evrardjp: the test suite has /etc/os-release fixtures for the distro lib. I would look at how thos are set up if you want to use other file fixtures | 16:13 |
| *** jcoufal has joined #openstack-infra | 16:13 | |
| clarkb | but basically you add files to the fixtures dir then tell bindep to look at them. | 16:13 |
| evrardjp | And as this is not pyttest, and I am complete python noob, I am lost :p | 16:13 |
| clarkb | and in your case rather than call specific internal functions you'd execute top level bindep cmd since you want a functional test | 16:14 |
| evrardjp | ok | 16:14 |
| clarkb | that said I think a unittest owuld handle this reasonably well | 16:14 |
| clarkb | since your function has a clear input and output you can unittest that well | 16:15 |
| *** armax has joined #openstack-infra | 16:18 | |
| *** jamesmcarthur has quit IRC | 16:19 | |
| evrardjp | clarkb: I think this function was already unit-tested, so I just wanted to extend the testing, but what you said confirmed what I thought, so when I failed earlier, I suppose I pebkac-ed | 16:23 |
| evrardjp | hahah | 16:23 |
| *** dave-mccowan has quit IRC | 16:23 | |
| *** dave-mccowan has joined #openstack-infra | 16:24 | |
| *** eernst has joined #openstack-infra | 16:25 | |
| *** Bhujay has quit IRC | 16:25 | |
| clarkb | evrardjp: also the old behavior was to print distro profiles in --profiles right? | 16:26 |
| evrardjp | it does on top | 16:26 |
| clarkb | evrardjp: I think this is a behavior change that may not be desireable (fixing the bug should be fine though) | 16:26 |
| evrardjp | the bottom was never meant for that: | 16:26 |
| evrardjp | well according to this code: | 16:27 |
| evrardjp | https://github.com/openstack-infra/bindep/blob/master/bindep/tests/test_main.py#L51-L70 | 16:27 |
| *** zoli is now known as zoli|gone | 16:27 | |
| evrardjp | https://images.duckduckgo.com/iu/?u=https%3A%2F%2Fcdn-images-1.medium.com%2Fmax%2F1200%2F1*UtZzMT32fRMnSN-HmgiSVQ.gif&f=1 | 16:28 |
| *** zoli|gone is now known as zoli | 16:28 | |
| clarkb | that tests checks both are listed? | 16:28 |
| clarkb | I would expect your change fails that test? | 16:29 |
| evrardjp | nope | 16:29 |
| evrardjp | because | 16:29 |
| evrardjp | ... | 16:29 |
| evrardjp | https://github.com/openstack-infra/bindep/blob/master/bindep/main.py#L62-L67 | 16:29 |
| clarkb | oh those are two different listing functions | 16:30 |
| evrardjp | clarkb: you see the problem, I have the impression I open a can of worms | 16:30 |
| *** ekhugen has quit IRC | 16:30 | |
| evrardjp | I will then refactor that main | 16:30 |
| *** dave-mccowan has quit IRC | 16:30 | |
| evrardjp | then refactor the testing | 16:30 |
| clarkb | I don't think it is a can of works | 16:30 |
| clarkb | just a bug | 16:30 |
| evrardjp | then refactor moar functions | 16:30 |
| *** panda is now known as panda|off | 16:30 | |
| * fungi is not here, but thought we had a bindep functional test exercising the fallback list on all configured platforms | 16:30 | |
| clarkb | the profiles() method current returns more data than intended | 16:30 |
| *** jpich has quit IRC | 16:30 | |
| clarkb | it also doesn't handle grouped rules in the parsely grammar | 16:31 |
| *** mdrabe has quit IRC | 16:31 | |
| evrardjp | clarkb: I am just joking, I wanted to say I have no limits personally :p | 16:31 |
| evrardjp | clarkb: isn't it better this way? | 16:31 |
| evrardjp | partition clean things | 16:31 |
| evrardjp | I could also use only the profiles function | 16:32 |
| clarkb | I think cleaning that up is fine, I just didn't realize we had multiple listing functions until I read through that test carefully | 16:32 |
| *** ijw has quit IRC | 16:32 | |
| evrardjp | yeah I am just hesitating of adding more into the profiles function, and simplifying the main | 16:32 |
| evrardjp | because you and me both had the same reaction when seeing the code | 16:32 |
| evrardjp | it's less intuitive? | 16:32 |
| evrardjp | it can be done in a separate patch I guess | 16:32 |
| evrardjp | same as adding extra testing | 16:33 |
| evrardjp | what's your opinion? | 16:33 |
| *** ekhugen has joined #openstack-infra | 16:34 | |
| clarkb | if it helps fix the other bug its fine to have it in this change. I think the most confusing aspect of it is the different rule profile types, but unsure how tightly coupled to parsely that is | 16:34 |
| *** gfidente has quit IRC | 16:35 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul master: Don't install python-yaml package https://review.openstack.org/584548 | 16:35 |
| *** eernst has quit IRC | 16:36 | |
| evrardjp | as I generally say, let's iteratively improve the thing :) | 16:38 |
| *** ddurst has quit IRC | 16:39 | |
| *** ddurst has joined #openstack-infra | 16:39 | |
| *** betherly is now known as betherly-afk | 16:40 | |
| *** betherly_ is now known as betherly | 16:40 | |
| clarkb | unrelated to bindep, interesting thing I notice is that all failed ssh in packethost is in the upper half of the FIP /24 | 16:41 |
| clarkb | I wonder if it is a routing issue | 16:41 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Fixed SS base model serialization https://review.openstack.org/583635 | 16:42 |
| *** trown is now known as trown|lunch | 16:42 | |
| *** dave-mccowan has joined #openstack-infra | 16:43 | |
| fungi | that's an astute observation | 16:44 |
| clarkb | fungi: ya .130 is the lowest IP I see with that problem (looking at logstash) | 16:44 |
| clarkb | if .129 is a router then that makes sense | 16:45 |
| clarkb | I wonder if I can see our fip allocations ranges | 16:45 |
| *** armaan has quit IRC | 16:45 | |
| *** armaan has joined #openstack-infra | 16:45 | |
| *** pcaruana has quit IRC | 16:47 | |
| *** dklyle has joined #openstack-infra | 16:49 | |
| AJaeger | config-core, please review https://review.openstack.org/584571 to remove an unused translation job. | 16:49 |
| *** ccamacho has quit IRC | 16:49 | |
| *** akhilaki has joined #openstack-infra | 16:49 | |
| clarkb | heh that explains it our floating ip range is only the /25 upper half of that /24 | 16:54 |
| evrardjp | TIL astute == clever. Thanks fungi for the word, and clarkb for being at the source of that sentence! :p | 16:55 |
| fungi | oh, wow | 17:01 |
| fungi | diagnosis in seconds | 17:01 |
| *** felipemonteiro has quit IRC | 17:01 | |
| *** hashar is now known as hasharDinner | 17:02 | |
| *** felipemonteiro has joined #openstack-infra | 17:02 | |
| *** tesseract has quit IRC | 17:02 | |
| *** derekh has quit IRC | 17:02 | |
| * fungi has strip steak and corn slowly roasting over damp applewood | 17:02 | |
| *** felipemonteiro_ has joined #openstack-infra | 17:02 | |
| * fungi is not paying super close attention | 17:02 | |
| Shrews | clarkb: have you noticed the packethost errors from nodepool-builder? | 17:03 |
| Shrews | clarkb: http://paste.openstack.org/show/726462/ | 17:03 |
| clarkb | Shrews: not from builder, but I also haven't really looked beyond cleaning up leaked images from when the cloud networking was rebuilt | 17:03 |
| clarkb | Shrews: that is new for me, we have up to date images in that cloud | 17:05 |
| *** jamesmca_ has quit IRC | 17:05 | |
| *** gongysh has joined #openstack-infra | 17:05 | |
| clarkb | Shrews: glance source says "this generally means a rate limiting or quota threshold was breached" | 17:06 |
| Shrews | odd error for rate/quota issue. meh | 17:07 |
| *** felipemonteiro has quit IRC | 17:07 | |
| Shrews | clarkb: corvus: mordred: i'm ready to restart nb01 and watch logs for a bit before restarting the other builders. anyone want to volunteer to do the same for one of the launchers? | 17:08 |
| mordred | Shrews: I can do once I'm off the call I'm on now | 17:08 |
| Shrews | mordred: spectacular | 17:08 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul master: Make sure python3-yaml isn't installed https://review.openstack.org/584974 | 17:09 |
| pabelanger | clarkb: corvus: ^another patch to puppet-zuul for python-yaml, starting to think container or virtualenv is longer fix for pip10. But willing to try^ to see if we can get a new server online today | 17:10 |
| Shrews | nb01 restarted... | 17:11 |
| *** jcoufal has quit IRC | 17:13 | |
| *** jcoufal has joined #openstack-infra | 17:15 | |
| *** ramishra has quit IRC | 17:16 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Clean up remote ansible synchronize tests https://review.openstack.org/584978 | 17:17 |
| openstackgerrit | Merged openstack-infra/nodepool master: Docs: increase visibility of env secrets warning https://review.openstack.org/584950 | 17:21 |
| *** e0ne has joined #openstack-infra | 17:22 | |
| *** e0ne has quit IRC | 17:22 | |
| *** gongysh has quit IRC | 17:24 | |
| Shrews | mordred: i'm not seeing much of any activity from nb01 to gauge if anything is broken, so i'm going to go ahead with nl01 | 17:24 |
| *** dtantsur is now known as dtantsur|afk | 17:24 | |
| mordred | Shrews: okie | 17:24 |
| mordred | Shrews: *fingers crossed* | 17:25 |
| clarkb | Shrews: you can trigger an image rebuild manually | 17:25 |
| Shrews | clarkb: oh, good idea. stuck in watching nl01 now though | 17:26 |
| *** felipemonteiro_ has quit IRC | 17:27 | |
| *** felipemonteiro_ has joined #openstack-infra | 17:27 | |
| Shrews | nl01 is restarted, btw | 17:27 |
| mordred | Shrews: it looks like it's mostly working? | 17:29 |
| *** myoung is now known as myoung|bbl | 17:29 | |
| *** felipemonteiro_ has quit IRC | 17:31 | |
| Shrews | mordred: i think so | 17:31 |
| mordred | yah. me too - I opened a tail -f window and haven't seen any disturbing patterns yet | 17:32 |
| *** felipemonteiro has joined #openstack-infra | 17:33 | |
| cmurphy | clarkb: I proposed a few more future parser switches starting with https://review.openstack.org/584637 in case you feel like hammering them out today | 17:34 |
| fungi | clarkb: are you going to follow up with that last e-mail to studarus with that fip /25 route observation? | 17:34 |
| *** felipemonteiro_ has joined #openstack-infra | 17:35 | |
| *** CrayZee has quit IRC | 17:35 | |
| clarkb | fungi: see above our entire fip range is that /25 so not really helpful afterall | 17:36 |
| corvus | pabelanger: where does python3-yaml come from in 584974? | 17:36 |
| *** david-lyle has joined #openstack-infra | 17:36 | |
| *** dklyle has quit IRC | 17:37 | |
| corvus | pabelanger: (are you sure that's installed on a new node, or is that leftover from the previous error?) | 17:37 |
| Shrews | ok, going to see if i can remember how to rebuild an image as clarkb suggested | 17:37 |
| mordred | corvus: it's probably a good safety net anyway, given cloud-init pulling in python-yaml | 17:38 |
| *** felipemonteiro has quit IRC | 17:39 | |
| *** yamahata has quit IRC | 17:39 | |
| corvus | mordred: hrm... it's just the only package we would have set to absent, so i thought it worth a closer look | 17:39 |
| *** jamesmcarthur has joined #openstack-infra | 17:40 | |
| Shrews | fedora-27 being rebuilt | 17:40 |
| mordred | corvus: totes | 17:40 |
| *** isviridov_away has quit IRC | 17:44 | |
| *** vaidy has quit IRC | 17:44 | |
| *** akhilaki has quit IRC | 17:44 | |
| *** mshalamov has quit IRC | 17:45 | |
| *** jamesmcarthur has quit IRC | 17:45 | |
| *** lpetrut has joined #openstack-infra | 17:46 | |
| pabelanger | corvus: Oh, i think it is getting pulled in via python-crypto | 17:47 |
| pabelanger | let me see what is installing that | 17:47 |
| *** trown|lunch is now known as trown | 17:47 | |
| kmalloc | mordred: is the coverage report something we inherit from the global zuul config? 'openstack-tox-cover' ? | 17:49 |
| Shrews | mordred: image build should take another 30m or so before we see any uploads of it. do we wanna go ahead with the other 3 launchers? | 17:49 |
| kmalloc | mordred: i ask because in kyestone im having issues where coverage isn't working -- fails in POST because the coverage_stat file isn't created, but all the tests run. Locally I can at least find the html output | 17:50 |
| pabelanger | corvus: Oh, I think that is something rax is installing for the images | 17:50 |
| pabelanger | 2018-07-07 02:00:52 install python-crypto:amd64 <none> 2.6.1-6ubuntu0.16.04.3 | 17:50 |
| kmalloc | cc clarkb ^ | 17:50 |
| clarkb | kmalloc: I want to say that was a bug over the weekend that TheJulia fixed? | 17:50 |
| pabelanger | 2018-07-23 16:39:25 startup archives unpack | 17:50 |
| pabelanger | is when we started the VM | 17:50 |
| kmalloc | clarkb: ah cool! | 17:50 |
| clarkb | kmalloc: is it still happening today? | 17:50 |
| corvus | pabelanger: ok | 17:50 |
| kmalloc | clarkb: haven't seen a repeat yet, have a recheck in | 17:50 |
| kmalloc | clarkb: or two. | 17:50 |
| *** kjackal has joined #openstack-infra | 17:50 | |
| *** armaan has quit IRC | 17:50 | |
| *** kjackal_ has joined #openstack-infra | 17:51 | |
| clarkb | kmalloc: if it persists today then somethin else is probably broken | 17:51 |
| kmalloc | clarkb: (someone else rechecked). I was having a hard time finding out where the issue was :) pounded my head on the wall all weekend. | 17:51 |
| *** armaan has joined #openstack-infra | 17:51 | |
| kmalloc | but my guess was TheJulia fixed and i wasn | 17:51 |
| kmalloc | t seeing the issue because it was fixed. | 17:51 |
| kmalloc | but i wanted to hold on rechecks perosnally until i chatted here :) | 17:52 |
| kmalloc | thanks! | 17:52 |
| clarkb | kmalloc: but yes it was a bug in the ansible for that job which came out of shared global job config | 17:53 |
| *** larainema has quit IRC | 17:56 | |
| *** ijw has joined #openstack-infra | 17:58 | |
| *** armaan has quit IRC | 18:01 | |
| *** armaan has joined #openstack-infra | 18:01 | |
| *** jamesmcarthur has joined #openstack-infra | 18:02 | |
| *** kjackal_ has quit IRC | 18:03 | |
| *** kjackal has quit IRC | 18:03 | |
| *** david-lyle has quit IRC | 18:04 | |
| *** dklyle has joined #openstack-infra | 18:04 | |
| *** armaan has quit IRC | 18:06 | |
| *** david-lyle has joined #openstack-infra | 18:07 | |
| *** armaan has joined #openstack-infra | 18:07 | |
| *** dklyle has quit IRC | 18:07 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Fix zuul reporting build failure with only non-voting jobs https://review.openstack.org/584990 | 18:07 |
| *** shrircha has joined #openstack-infra | 18:10 | |
| *** r-daneel has quit IRC | 18:10 | |
| shrircha | hi folks, I'm seeing an error in the upstream CI with the message "Failed to start libvirt guest: libvirtError: Requested operation is not valid: cpu affinity is not supported", but I'm not setting any explicit numa configs | 18:11 |
| shrircha | wondering if someone might be able to help me figure out how to get past this error (I'm attempting to run tempest on our project (networking-vpp) upstream) | 18:12 |
| *** luzC has quit IRC | 18:12 | |
| *** yamahata has joined #openstack-infra | 18:12 | |
| clarkb | shrircha: http://status.openstack.org/elastic-recheck/#1781710 I think that is tracking the issue, also there is a bug | 18:12 |
| clarkb | shrircha: but maybe they are different? | 18:13 |
| shrircha | taking a peek | 18:13 |
| shrircha | thanks for the pointer | 18:13 |
| shrircha | clarkb: hm these might be different | 18:14 |
| clarkb | shrircha: do you have a link to logs for your example? | 18:15 |
| clarkb | shrircha: however, it seems like it would be a cloud configuration or nova issue? I'm not sure how much help we wil lbe | 18:15 |
| *** luzC has joined #openstack-infra | 18:16 | |
| shrircha | clarkb: http://logs.openstack.org/94/569494/7/check/networking-vpp-tempest/ebcebee/controller/logs/screen-n-cpu.txt.gz#_Jul_20_23_28_19_082640 | 18:17 |
| shrircha | clarkb: https://review.openstack.org/#/c/569494/ to my review in case I messed something up in my zuul conf | 18:17 |
| clarkb | shrircha: http://logs.openstack.org/94/569494/7/check/networking-vpp-tempest/ebcebee/controller/logs/screen-n-cpu.txt.gz#_Jul_20_23_32_10_707120 that configures numa for the instance | 18:18 |
| *** jamesmca_ has joined #openstack-infra | 18:19 | |
| shrircha | clarkb: I didn't set that though (I've just been trying to run vanilla tempest against our project); is this a nova.conf option I need to explicitly disable? | 18:19 |
| *** eernst has joined #openstack-infra | 18:19 | |
| shrircha | clarkb: or does enabling hugepages (which we require) automatically configure numa? | 18:20 |
| clarkb | shrircha: hugepages imply numa aiui | 18:20 |
| clarkb | at least with nova | 18:20 |
| *** akhilaki has joined #openstack-infra | 18:21 | |
| shrircha | clarkb: darn it. looks like hw virt isn't available on the upstream VMs (at least, I'm not seeing /dev/kvm sooo) | 18:21 |
| clarkb | shrircha: correct, it isn't available in all clouds and when available isn't always reliable so we disable by default everywhere | 18:21 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Scope all_known_hosts in mult-node-known-hosts https://review.openstack.org/584993 | 18:21 |
| shrircha | clarkb: gotcha | 18:22 |
| shrircha | clarkb: thanks for the help! into configland I delve to suss this out :) | 18:22 |
| clarkb | shrircha: what is your test doing that requires huge pages? | 18:22 |
| *** david-lyle is now known as dklyle | 18:22 | |
| shrircha | clarkb: our project configures VPP, which requires hugepages to be enabled | 18:23 |
| clarkb | shrircha: in the nested VMs? | 18:23 |
| *** tosky has quit IRC | 18:23 | |
| shrircha | clarkb: sorry, it should be on the host | 18:23 |
| shrircha | not on the VMs spun up during the tempest test | 18:24 |
| clarkb | shrircha: ok, the thing that is failing due to numa is the nested VMs booted by nova | 18:24 |
| shrircha | clarkb: ah okay | 18:24 |
| shrircha | clarkb: I'd misunderstood | 18:24 |
| shrircha | clarkb: hm then this shouldn't be an issue | 18:25 |
| *** e0ne has joined #openstack-infra | 18:26 | |
| *** vaidy has joined #openstack-infra | 18:28 | |
| shrircha | clarkb: http://logs.openstack.org/94/569494/7/check/networking-vpp-tempest/ebcebee/controller/logs/screen-n-cpu.txt.gz#_Jul_20_23_28_04_520237 | 18:28 |
| shrircha | clarkb: looks like we're setting hugepages in the nested vm anyway | 18:28 |
| shrircha | clarkb: that might be the ticket | 18:28 |
| *** isviridov_away has joined #openstack-infra | 18:29 | |
| *** dhill_ has quit IRC | 18:32 | |
| *** ianychoi has quit IRC | 18:33 | |
| *** felipemonteiro_ has quit IRC | 18:33 | |
| *** felipemonteiro_ has joined #openstack-infra | 18:34 | |
| *** ianychoi has joined #openstack-infra | 18:34 | |
| *** dhill_ has joined #openstack-infra | 18:34 | |
| shrircha | clarkb: in case this is required (I've sent a note to our devs to confirm), is it possible in the zuul config to request a node with hw virt enabled? | 18:35 |
| Shrews | #status log All nodepool launchers restarted with latest code, include switch from shade to openstacksdk | 18:39 |
| openstackstatus | Shrews: finished logging | 18:39 |
| *** jcoufal has quit IRC | 18:39 | |
| mordred | infra-root, project-config-core: there is a change in progress in ansible upstream that is going to affect us and is going to be moderately painful to deal with - but the end result will be a better world: http://paste.ubuntu.com/p/xJFyJJ7rcy/ | 18:39 |
| Shrews | mordred: ^^^. still waiting on builders for a bit | 18:39 |
| mordred | infra-root, project-config-core: tl;dr - all 'facts', like ansible_all_ipv4_addresses are moving from being top-level to being inside of the 'ansible_facts' dict | 18:40 |
| mordred | things that are not 'facts', like, ansible_user - will stay put | 18:40 |
| mordred | the first step, putting them into ansible_facts, has already happened as of 2.5 - so it is possible to migrate existing code using them to the new form today | 18:40 |
| Shrews | pabelanger: how was nb03 built? i can't ssh in, but i can to nb01 and nb02 | 18:44 |
| mordred | Shrews: ++ | 18:45 |
| pabelanger | Shrews: IIRC, ianw provisioned it, but should be from puppet | 18:47 |
| pabelanger | let me look | 18:47 |
| *** lbragstad_ has joined #openstack-infra | 18:47 | |
| Shrews | pabelanger: mordred: need one of you to restart nodepool-builder on nb03. that's the only one left | 18:47 |
| *** dhajare has quit IRC | 18:47 | |
| pabelanger | I see an account for you | 18:47 |
| pabelanger | I can restart it | 18:47 |
| mordred | there were puppet issues with ssh keys at one point | 18:48 |
| Shrews | oh, it might help if i ssh from the right machine | 18:48 |
| *** lbragstad has quit IRC | 18:48 | |
| Shrews | doh | 18:48 |
| Shrews | restarting it... | 18:48 |
| mordred | :) | 18:48 |
| pabelanger | restarted | 18:48 |
| pabelanger | oh, :) | 18:48 |
| Shrews | np | 18:48 |
| Shrews | i noticed :) | 18:49 |
| Shrews | #status log All nodepool builders restarted with latest code, include switch from shade to openstacksdk | 18:49 |
| openstackstatus | Shrews: finished logging | 18:49 |
| mordred | \o/ | 18:49 |
| Shrews | dang, i *still* didn't fix s/include/including/ | 18:49 |
| Shrews | bah | 18:49 |
| Shrews | consistency for the win | 18:49 |
| *** lbragstad_ is now known as lbragstad | 18:51 | |
| mordred | http://grafana.openstack.org/?orgId=1 looks weird - did something happen there? | 18:54 |
| clarkb | cmurphy: ok I think I'm ready to start with the future parser stuff now. Going to pull those changes up now | 18:54 |
| clarkb | mordred: ianw upgraded grafana | 18:54 |
| mordred | or have I just not loooked at the front page in a while | 18:54 |
| mordred | AH. cool. so something happened | 18:55 |
| mordred | apparently there are no "Installed Apps" :) | 18:55 |
| clarkb | cmurphy: https://review.openstack.org/#/c/582168/'s parent was abandoned in favor of https://review.openstack.org/#/c/582011/ I am going to rebase that change | 18:56 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-drupal master: Fix template variables https://review.openstack.org/582168 | 18:59 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-drupal master: Fix template variable https://review.openstack.org/584293 | 18:59 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-drupal master: Monkey-patch drush module for puppet 4 https://review.openstack.org/584341 | 18:59 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-drupal master: Add beaker tests https://review.openstack.org/584294 | 18:59 |
| *** hasharDinner has quit IRC | 19:00 | |
| *** hashar has joined #openstack-infra | 19:00 | |
| *** felipemonteiro_ is now known as felipemonteiro | 19:02 | |
| *** felipemonteiro_ has joined #openstack-infra | 19:04 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Allow launch-retries to indefinitely retry for openstack https://review.openstack.org/584488 | 19:04 |
| *** felipemonteiro has quit IRC | 19:07 | |
| clarkb | mordred: is https://review.openstack.org/#/c/584637/1/modules/openstack_project/files/puppetmaster/groups.txt the right way to express multiple regex members in an ansible group? | 19:07 |
| * clarkb looks for docs | 19:08 | |
| mordred | clarkb: yes - see line 14 | 19:08 |
| clarkb | mordred: 14 is only globs though right? I guess you have to specify that each : separated value is a regex if you are treating it as such | 19:08 |
| mordred | clarkb: it's the same syntax as the hosts line or the first parameter to the ansible command | 19:09 |
| mordred | yes - each : should be a full regex | 19:09 |
| mordred | clarkb: but it's certainly worth double-checking ... ansible --list-hosts 'futureparser ~review-dev\d*\.openstack\.org:~groups-dev\d*\.openstack\.org' should work to test it | 19:10 |
| mordred | or, rather | 19:10 |
| mordred | ansible --list-hosts '~review-dev\d*\.openstack\.org:~groups-dev\d*\.openstack\.org' | 19:10 |
| mordred | clarkb: yes. that works | 19:11 |
| mordred | clarkb: http://paste.openstack.org/show/726469/ | 19:11 |
| clarkb | TIL | 19:11 |
| clarkb | about --list-hosts | 19:11 |
| clarkb | fungi: john says that all the instance uuids I gave him ran on the same hypervisor and that is the hypervisor that acts as the netowrk node too (I realized I didn't cc you and others on that email) | 19:13 |
| clarkb | fungi: in any case that gives john some ideas for debugging and fixing | 19:13 |
| mordred | \o/ | 19:13 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Add logging when pausing due to node abort https://review.openstack.org/585007 | 19:17 |
| cmurphy | clarkb: o/ | 19:18 |
| *** hrubi has quit IRC | 19:18 | |
| clarkb | cmurphy: I'm going to approve the groups-dev future parser change just as soon as its depends on merges | 19:19 |
| cmurphy | clarkb: cool | 19:19 |
| *** felipemonteiro_ has quit IRC | 19:20 | |
| *** graphene has quit IRC | 19:21 | |
| *** felipemonteiro_ has joined #openstack-infra | 19:21 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack-infra/bindep master: Fix user profile display https://review.openstack.org/584955 | 19:21 |
| *** hrubi has joined #openstack-infra | 19:22 | |
| *** graphene has joined #openstack-infra | 19:22 | |
| openstackgerrit | Merged openstack-infra/puppet-drupal master: Fix template variables https://review.openstack.org/582168 | 19:30 |
| *** eharney has quit IRC | 19:35 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul master: Make sure python3-yaml isn't installed https://review.openstack.org/584974 | 19:43 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Turn on the future parser for groups-dev https://review.openstack.org/584637 | 19:46 |
| cmurphy | clarkb: can you +A https://review.openstack.org/584341 and https://review.openstack.org/584293 too? those are needed for https://review.openstack.org/584637 | 19:46 |
| cmurphy | sorry i had the wrong depends-on there | 19:46 |
| *** janki has quit IRC | 19:47 | |
| *** r-daneel has joined #openstack-infra | 19:47 | |
| clarkb | ah the whole stack was needed? | 19:48 |
| *** jcoufal has joined #openstack-infra | 19:49 | |
| cmurphy | yeah | 19:49 |
| *** jcoufal_ has joined #openstack-infra | 19:49 | |
| *** jcoufal is now known as Guest8355 | 19:50 | |
| *** Guest8355 has quit IRC | 19:50 | |
| *** jcoufal_ has quit IRC | 19:51 | |
| clarkb | cmurphy: iirc explicitly passing nil doesn't override a default parameter value right? which is why we have to set 6.x for git_tag instead of just setting it to nil? | 19:53 |
| *** felipemonteiro_ has quit IRC | 19:53 | |
| *** felipemonteiro_ has joined #openstack-infra | 19:54 | |
| cmurphy | clarkb: s/nil/undef correct, just passing undef didn't work (i tried) | 19:55 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added permissions schema per entity fields https://review.openstack.org/585017 | 19:55 |
| clarkb | cmurphy: thanks, I seem to recall running into that behavior too | 19:55 |
| mordred | infra-root, project-config-core: how does https://etherpad.openstack.org/p/53jps0a3sz look as an email to send to folks? | 19:56 |
| mriedem | from my random sampling of job timeout and post failures in the last 5 minutes, is it true that the majority of those types of failures are on packethost nodes? | 19:56 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added permissions schema per entity fields https://review.openstack.org/585017 | 19:57 |
| clarkb | mriedem: there is a known hypervisor issue being addressed on packethost with network connectivity. I don't know that I can attribute them directly to timeouts and post failures though both can be caused by that | 19:57 |
| Shrews | mordred: ++ | 19:57 |
| clarkb | mriedem: I've been watching it and it seems to be about 3 per hour or so? | 19:59 |
| clarkb | based on the existing e-r query | 19:59 |
| mriedem | clarkb: i don't know what the e-r query is for either | 19:59 |
| mriedem | we can't get an e-r query on POST_FAILURE since there are no logs right? | 19:59 |
| mriedem | i just know it's taking about a week to merge code | 20:00 |
| clarkb | mriedem: not necessarily, post failure could happen after copying logs (not sure how common that is though) | 20:00 |
| clarkb | mriedem: the e-r query is the one you updated last week | 20:00 |
| clarkb | mriedem: http://status.openstack.org/elastic-recheck/#1781871 | 20:00 |
| clarkb | mriedem: it manifests as ssh connectivity going away | 20:01 |
| mriedem | ok yeah i guess i just saw this for POST_FAILURE in one job http://status.openstack.org/elastic-recheck/index.html#1781871 | 20:01 |
| mriedem | yup | 20:01 |
| clarkb | looking at the job timeout query in kibana its distributed across all clouds with rax-ord having the most occurences | 20:02 |
| clarkb | and ya was 3 times in previous hour that ssh connectivity had problems | 20:05 |
| *** hongbin has quit IRC | 20:05 | |
| fungi | #status log set forward_auto_discards=0 on openstack-qa@lists.openstack.org ml to combat spam backscatter | 20:06 |
| openstackstatus | fungi: finished logging | 20:06 |
| *** edmondsw has quit IRC | 20:09 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for test_tagged_attachment bug 1775947 https://review.openstack.org/585022 | 20:09 |
| openstack | bug 1775947 in tempest "tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest failing" [Medium,Confirmed] https://launchpad.net/bugs/1775947 - Assigned to Deepak Mourya (mourya007) | 20:09 |
| clarkb | mordred: is next release of nasible going to be 3.0? | 20:13 |
| clarkb | mordred: also for things that remain in top level namespace of vars will they also be in the facts dict? | 20:14 |
| mordred | clarkb: no, and no | 20:14 |
| clarkb | seems like both of those things would be helpful to people, the first to communicate that a major non backward compat change is happening and the second for consistency so you don't have to think about where to lookup a var | 20:14 |
| mordred | clarkb: right now discussions are still ongoing as to when support for the current behavior will be removed | 20:14 |
| clarkb | ah | 20:15 |
| mordred | clarkb: so yeah - the "are we ready to announce the deprecation yet" is the current discussion | 20:15 |
| mordred | since announcing the deprecatoin would imply communicating at what point the old behavior would go away | 20:15 |
| openstackgerrit | Merged openstack-infra/puppet-drupal master: Fix template variable https://review.openstack.org/584293 | 20:17 |
| mordred | clarkb: for the other thing - I *think* the things that stay top-leve are actually going to be a pretty small set | 20:18 |
| *** felipemonteiro__ has joined #openstack-infra | 20:18 | |
| *** felipemonteiro_ has quit IRC | 20:22 | |
| *** jamesmca_ has quit IRC | 20:22 | |
| *** kjackal has joined #openstack-infra | 20:22 | |
| openstackgerrit | Merged openstack-infra/puppet-drupal master: Monkey-patch drush module for puppet 4 https://review.openstack.org/584341 | 20:23 |
| *** kjackal_ has joined #openstack-infra | 20:23 | |
| clarkb | mriedem: for a nova hypervisor what does state up status disabled mean? | 20:24 |
| clarkb | mriedem: I think that john may have disabled the problem hypervisor if status: disabled means don't run any new VMs here | 20:25 |
| mriedem | means nova-compute is running and reporting into the scheduler but it's disabled from scheduling | 20:26 |
| mriedem | if you're using the proper scheduler filter, which everyone should be | 20:26 |
| clarkb | mriedem: ok in that case I think we should start to see the ssh issues fall off, (will double check later today) | 20:27 |
| *** felipemonteiro__ has quit IRC | 20:29 | |
| *** felipemonteiro__ has joined #openstack-infra | 20:29 | |
| *** kgiusti has left #openstack-infra | 20:30 | |
| *** lpetrut has quit IRC | 20:30 | |
| *** slaweq has quit IRC | 20:34 | |
| *** eharney has joined #openstack-infra | 20:34 | |
| *** slaweq has joined #openstack-infra | 20:35 | |
| *** yamamoto has joined #openstack-infra | 20:42 | |
| clarkb | I've also confirmed no VMs running on that hypervisor now according to hypervisor show | 20:44 |
| *** r-daneel_ has joined #openstack-infra | 20:44 | |
| *** r-daneel has quit IRC | 20:44 | |
| *** r-daneel_ is now known as r-daneel | 20:44 | |
| *** jamesmcarthur has quit IRC | 20:46 | |
| *** yamamoto has quit IRC | 20:46 | |
| *** e0ne has quit IRC | 20:50 | |
| clarkb | Shrews: mordred http://paste.openstack.org/show/726480/ did we need to restart all launchers at the same time again? | 20:51 |
| openstackgerrit | Merged openstack-infra/nodepool master: Add logging when pausing due to node abort https://review.openstack.org/585007 | 20:51 |
| Shrews | clarkb: looking, but there were no data model changes requiring that | 20:51 |
| *** jamesmcarthur has joined #openstack-infra | 20:52 | |
| *** camunoz has quit IRC | 20:52 | |
| clarkb | er that paste may not be a complete traceback | 20:52 |
| Shrews | that's.... od | 20:53 |
| Shrews | odd | 20:53 |
| clarkb | Shrews: http://paste.openstack.org/show/726481/ thats a correct traceback | 20:53 |
| Shrews | clarkb: that seems off since it's a traceback including docstrings | 20:54 |
| Shrews | clarkb: which host? | 20:54 |
| clarkb | Shrews: nl02, I'm guessing code updated after the restart and now tracebacks don't line up | 20:55 |
| clarkb | you mgiht be able to find an older exception to see correct lines | 20:55 |
| clarkb | Shrews: oh hrm its the other way around looks like those are from before the restart | 20:56 |
| *** edmondsw has joined #openstack-infra | 20:56 | |
| clarkb | and none after (implying that ya maybe we needed to update all of them together?) | 20:56 |
| *** trown is now known as trown|outtypewww | 20:56 | |
| clarkb | Shrews: I122268dc7723901c04a58fa6f5c1688a3fdab227 is the change that did it, it allowed aborted into the valid states | 20:57 |
| Shrews | clarkb: doh! | 20:57 |
| clarkb | I think it is ok now though | 20:57 |
| clarkb | didn't create catastrophe like last time :) | 20:57 |
| Shrews | clarkb: yeah, this one should recover ok | 20:58 |
| Shrews | clarkb: looks like we should add another release note for this | 20:58 |
| clarkb | Shrews: have all launchers been restarted at this point? | 20:58 |
| Shrews | clarkb: yes | 20:58 |
| clarkb | ya I'm not seeing any ill effects currently | 21:00 |
| clarkb | Shrews: corvus do we want a release note for that? | 21:02 |
| Shrews | clarkb: yes (see above) ;) | 21:02 |
| Shrews | working on it | 21:02 |
| clarkb | oh derp you mentioned that | 21:02 |
| clarkb | (sorry I'm acting as a sink for all the data inputs to confirm it is happy now) | 21:02 |
| *** hashar has quit IRC | 21:05 | |
| *** jamesmcarthur has quit IRC | 21:06 | |
| * mordred pipes more data into clarkb | 21:07 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Add upgrade release note about aborted node status https://review.openstack.org/585035 | 21:07 |
| Shrews | clarkb: ^^^ | 21:08 |
| Shrews | clarkb: np. i had exactly 3 conversations going on at that time, too. i know confusion | 21:08 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for test_tagged_attachment bug 1775947 https://review.openstack.org/585022 | 21:09 |
| openstack | bug 1775947 in tempest "tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest failing" [Medium,Confirmed] https://launchpad.net/bugs/1775947 - Assigned to Deepak Mourya (mourya007) | 21:09 |
| *** hongbin has joined #openstack-infra | 21:10 | |
| *** e0ne has joined #openstack-infra | 21:19 | |
| *** e0ne has quit IRC | 21:21 | |
| *** rcernin has joined #openstack-infra | 21:26 | |
| *** felipemonteiro_ has joined #openstack-infra | 21:31 | |
| *** felipemonteiro__ has quit IRC | 21:34 | |
| openstackgerrit | Merged openstack-infra/system-config master: Turn on the future parser for groups-dev https://review.openstack.org/584637 | 21:38 |
| clarkb | cmurphy: ^ ok I'll watch that | 21:38 |
| *** jamesmcarthur has joined #openstack-infra | 21:39 | |
| clarkb | 2200UTC puppet run will likely be the one that includes it | 21:39 |
| cmurphy | clarkb: o7 | 21:39 |
| *** kjackal_ has quit IRC | 21:41 | |
| *** kjackal has quit IRC | 21:41 | |
| *** agopi is now known as agopi|brb | 21:42 | |
| pabelanger | clarkb: is rdns-create broken for rax? | 21:42 |
| pabelanger | I get | 21:42 |
| pabelanger | ERROR: n/a (HTTP 400) - n/a | 21:42 |
| pabelanger | when running the commands for new server | 21:42 |
| clarkb | pabelanger: I think fungi discovered we had to generate a new token but not sure if he did that? | 21:43 |
| *** jamesmcarthur has quit IRC | 21:43 | |
| pabelanger | http://paste.openstack.org/show/726482/ | 21:44 |
| pabelanger | that is the error I get | 21:44 |
| pabelanger | ah | 21:44 |
| pabelanger | DNS_CLOUD_SERVERS_API_KEY I guess? | 21:45 |
| clarkb | ya I think that one, I'm guessing it wasn't updated if it still isn't working | 21:45 |
| pabelanger | clarkb: k, checking web | 21:45 |
| mordred | I believe we found that wasn't *really* the issue last time | 21:46 |
| mordred | but I do not remember what the issue actually was? | 21:46 |
| *** agopi|brb has quit IRC | 21:47 | |
| pabelanger | is that the rackspace API key in web? | 21:47 |
| mordred | not sure. probably | 21:47 |
| mordred | we activelyignore it for the openstack things | 21:48 |
| mordred | since it's silly | 21:48 |
| pabelanger | k, that is different then what our nova.sh has, let me test it | 21:48 |
| mordred | kk | 21:48 |
| clarkb | ya its only an issue for dns as its not openstack api | 21:48 |
| *** gongysh has joined #openstack-infra | 21:51 | |
| pabelanger | clarkb: mordred: do we know which user we should be using for rackdns? I see we are configured for the old one, but IIRC, we got a new username? | 21:54 |
| mordred | pabelanger: unfortunately I do not know the answer to that | 21:56 |
| clarkb | pabelanger: yes it would be the new user and consequently we think needing a new token | 21:56 |
| clarkb | but it was fungi that was looking into it iirc and I assume he figured it out since he was doing it for a mail server which apparently sends and receives email | 21:56 |
| pabelanger | k, I'm not sure how to proceed. ze11.o.o booted have only used CLI to setup DNS before | 21:57 |
| clarkb | pabelanger: I think either update the credentials to use the new thing or use the web ui | 21:58 |
| mordred | I think fungi use the web ui | 21:59 |
| clarkb | ah | 21:59 |
| *** dhill_ has quit IRC | 22:00 | |
| pabelanger | yah, rackspace api key doesn't seem to be it | 22:00 |
| pabelanger | that is the only token I see | 22:01 |
| *** bobh has quit IRC | 22:01 | |
| clarkb | pabelanger: I think we have to generate a new one. I am looking | 22:01 |
| pabelanger | ack, thanks | 22:02 |
| *** jmorgan1_ has joined #openstack-infra | 22:02 | |
| clarkb | of course the docs for this process are a 404 | 22:03 |
| *** jmorgan1 has quit IRC | 22:03 | |
| *** edmondsw has quit IRC | 22:04 | |
| *** jamesmcarthur has joined #openstack-infra | 22:05 | |
| clarkb | https://github.com/kwminnick/rackspace-dns-cli seems to imply we can just use the password | 22:05 |
| mordred | ++ | 22:05 |
| clarkb | oh thats a not endorsed by rackspace project | 22:05 |
| clarkb | mordred: is that what we use ^ | 22:05 |
| mordred | dunno. just saying - the other services also allow password - so I'd be surprised if we coudln't | 22:06 |
| clarkb | actuall reverse dns is on the openstackci account resources | 22:08 |
| clarkb | which makes me extra confused because reverse dns should've just continued to work | 22:08 |
| clarkb | pabelanger: did you change any values in any files yet? | 22:09 |
| *** jamesmcarthur has quit IRC | 22:10 | |
| pabelanger | clarkb: I've reverted back to originals | 22:10 |
| clarkb | pabelanger: did you change openstack-rs-nova.sh DNS_CLOUD_SERVERS_API_KEY ? | 22:11 |
| *** jmorgan1_ has quit IRC | 22:12 | |
| clarkb | oh forward records didn't work either so both are unworking? | 22:12 |
| *** rh-jelabarre has quit IRC | 22:12 | |
| *** jmorgan1 has joined #openstack-infra | 22:12 | |
| *** agopi has joined #openstack-infra | 22:12 | |
| mordred | clarkb: both forward and reverse need to be done on the account with the domain | 22:13 |
| mordred | reverse can reference servers in a different account | 22:13 |
| mordred | which is why we pass them in by full url | 22:13 |
| pabelanger | clarkb: no, that has always been value | 22:13 |
| clarkb | mordred: ya reverse is weird, when you do it through webui its on the account that created the instance | 22:13 |
| mordred | clarkb: that makes more sense to me ... | 22:14 |
| cmurphy | clarkb: how's groups-dev doing? | 22:14 |
| mordred | clarkb: it'salso possible I'm speaking stupid words | 22:14 |
| clarkb | cmurphy: still waiting for nasible to get around to it | 22:15 |
| clarkb | mordred: anyways I'm now confused because the contents in openstack-rs-nova look correct except for the DNS_* env vars but those shouldn't be needed there | 22:15 |
| *** gfidente has joined #openstack-infra | 22:15 | |
| *** gfidente has joined #openstack-infra | 22:15 | |
| clarkb | mordred: also I odn't see where rackdns uses DNS_* env vars | 22:15 |
| clarkb | I'm guessing I'm looking at the wrong repo | 22:16 |
| clarkb | rackspace-dnsclient is the thing | 22:17 |
| clarkb | which doesn't show up in pypi | 22:17 |
| clarkb | it is https://github.com/kwminnick/rackspace-dns-client according to PKG-INFO | 22:19 |
| clarkb | which doesn't exist anymore | 22:20 |
| clarkb | ugh | 22:20 |
| clarkb | cmurphy: http://paste.openstack.org/show/726484/ seems to have worked | 22:22 |
| clarkb | and site is still up | 22:22 |
| *** slaweq has quit IRC | 22:23 | |
| *** jamesmcarthur has joined #openstack-infra | 22:24 | |
| *** neiloy has quit IRC | 22:25 | |
| *** owalsh_ has joined #openstack-infra | 22:25 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Add upgrade release note about aborted node status https://review.openstack.org/585035 | 22:25 |
| clarkb | mordred: I don't understand how any of this ever worked :/ | 22:26 |
| mordred | clarkb: well, it has literally never worked WELL given the need to use a proprietary API | 22:27 |
| mordred | and the way it's failing right now is the exactly the way I'd expect it to fail- which is that something has bitrotted and nobody cares :( | 22:27 |
| clarkb | I'm not convinced those DNS_* env vars do anything and instead its using the regular cloud creds | 22:27 |
| mordred | clarkb: I think that's entirely likely | 22:27 |
| *** owalsh has quit IRC | 22:28 | |
| cmurphy | clarkb: hmm i wasn't expecting settings.php to change | 22:28 |
| *** owalsh- has joined #openstack-infra | 22:28 | |
| *** rlandy has quit IRC | 22:28 | |
| *** jamesmcarthur has quit IRC | 22:28 | |
| clarkb | cmurphy: possibly related to the drush checkout? | 22:29 |
| clarkb | like maybe it moved ahead on that drush branch? | 22:29 |
| *** owalsh has joined #openstack-infra | 22:31 | |
| clarkb | pabelanger: did the contents of openstackci-rs-nova.sh change? | 22:31 |
| clarkb | oh wait no I get it now | 22:31 |
| *** owalsh_ has quit IRC | 22:31 | |
| pabelanger | clarkb: I added test vars to it, but reverted | 22:32 |
| pabelanger | so date may have changed | 22:32 |
| *** florianf has quit IRC | 22:32 | |
| cmurphy | clarkb: the drush checkout should live in /usr/share/drush not in /srv/vhosts | 22:33 |
| clarkb | pabelanger: gotcha | 22:33 |
| clarkb | cmurphy: ya but maybe it gets the php settings from from thta repo (I've confirmed it is in /usr/share/drush and not /srv/vhosts) | 22:34 |
| *** owalsh- has quit IRC | 22:34 | |
| *** wolverineav has quit IRC | 22:34 | |
| clarkb | the last commit there doesn't appera to be very ne wthough | 22:34 |
| clarkb | pabelanger: mordred adding in debug I've confirmed that it appears to use the regular openstack auth creds not the DNS_* creds | 22:35 |
| *** wolverineav has joined #openstack-infra | 22:35 | |
| clarkb | pabelanger: mordred and it complains about _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed | 22:35 |
| *** owalsh_ has joined #openstack-infra | 22:35 | |
| pabelanger | do we wrong CA installed? | 22:36 |
| pabelanger | do we have* | 22:36 |
| cmurphy | clarkb: i think that drush::git resource isn't idempotent, i'm not so worried about that but about what changed in the settings, that comes from puppet not from the drush checkout | 22:36 |
| *** owalsh has quit IRC | 22:36 | |
| clarkb | cmurphy: got it | 22:38 |
| clarkb | pabelanger: s_client seems fine with it | 22:38 |
| *** mriedem has quit IRC | 22:39 | |
| clarkb | though I don't see that if I run s_cleint from puppetmaster, interesting | 22:39 |
| clarkb | cert was issues june 5, 2018 | 22:40 |
| *** wolverineav has quit IRC | 22:40 | |
| clarkb | Thawte TLS RSA CA G1 is what we are missing on puppetmaster I think | 22:43 |
| *** boden has quit IRC | 22:44 | |
| ianw | is that due to it's distro release? didn't we have problems with this with ... something? | 22:44 |
| clarkb | yup if I wget https://thawte.tbs-certificats.com/Thawte_TLS_RSA_CA_G1.crt and pass -CAfile to that file when running s_client it verifies properly | 22:46 |
| clarkb | ianw: yup | 22:46 |
| pabelanger | would have expected ca-certificates to get an update | 22:46 |
| clarkb | we can manually add that cert to the list of things we trust (have puppet do it out of band of distro), but having the distro update would be ideal | 22:47 |
| *** bdodd_ has quit IRC | 22:48 | |
| pabelanger | Yah, maybe somebody at rackspace can help push a distro update | 22:48 |
| *** bdodd has joined #openstack-infra | 22:48 | |
| *** hongbin has quit IRC | 22:50 | |
| clarkb | cmurphy: I've checked the settings.php file and the three entries in conf dict in groups_dev.pp are in there | 22:51 |
| clarkb | cmurphy: if I had to guess its different ordering of items in a dict with different puppet version | 22:51 |
| clarkb | cmurphy: the order in file now is the same as defined in groups_dev.pp | 22:52 |
| clarkb | not sure what it was before | 22:52 |
| clarkb | pabelanger: I think for today the easiest thing may be to use the web ui | 22:53 |
| clarkb | pabelanger: and we'll sort out how we want to add that cert in the future | 22:54 |
| clarkb | ? | 22:54 |
| *** tpsilva has quit IRC | 22:54 | |
| pabelanger | clarkb: sure, I can use web then add work on adding SSL cert to puppetmaster | 22:55 |
| pabelanger | clarkb: how do we do a rdns via web? | 22:55 |
| pabelanger | don't think I've done that before | 22:55 |
| clarkb | pabelanger: you go the the "server" page for that instance and its one of the options on that page (add reverse dns record iirc) | 22:56 |
| cmurphy | clarkb: is there a $conf['cron_key'] = ... line? | 22:56 |
| pabelanger | ack | 22:57 |
| clarkb | cmurphy: $conf['cron_key'] = 'stuff'; yes | 22:57 |
| *** tosky has joined #openstack-infra | 22:58 | |
| *** myoung|bbl is now known as myoung | 22:59 | |
| pabelanger | clarkb: https://review.openstack.org/584546/ should pass now | 23:03 |
| pabelanger | ze11.o.o dns setup and ssh hostkey added to puppetmaster.o.o | 23:03 |
| cmurphy | clarkb: if you do find /opt/puppetlabs/puppet/cache/clientbucket -name 9fabf951176649a3cead83892966b80c and then cat the file named 'contents' in that directory you should be able to see what the old file was | 23:03 |
| *** mschuppert has quit IRC | 23:04 | |
| *** yamamoto has joined #openstack-infra | 23:04 | |
| clarkb | cmurphy: there is no /opt/puppetlabs as its still puppet3 I guess | 23:04 |
| cmurphy | oh right | 23:04 |
| cmurphy | /var/lib/puppet/cache | 23:05 |
| pabelanger | ianw: if you also like to review: https://review.openstack.org/584546/ bring ze11.o.o online | 23:05 |
| cmurphy | er /var/lib/puppet/clientbucket | 23:05 |
| clarkb | ya found it in /var/lib | 23:05 |
| clarkb | cmurphy: the old file had cron_key defined twice | 23:06 |
| clarkb | same value for both | 23:06 |
| cmurphy | oh nice | 23:06 |
| clarkb | cmurphy: https://git.openstack.org/cgit/openstack-infra/puppet-drupal/tree/templates/settings.php.erb#n10 it was writing it there too I think | 23:07 |
| clarkb | and now we don't | 23:07 |
| *** yamamoto has quit IRC | 23:08 | |
| *** r-daneel has quit IRC | 23:08 | |
| cmurphy | clarkb: yeah i think https://review.openstack.org/#/c/159426/ might have been an error and the template wasn't updated but the variable is still defined in system-config so it still works | 23:08 |
| clarkb | cmurphy: it is a scoping situation similar to how the vhosts had to be updated | 23:08 |
| clarkb | cmurphy: it worked in the past because conf_cron_key was set in site.pp, and @conf_cron_key would find that one I think | 23:08 |
| clarkb | cmurphy: but now the scoping is to puppet-drupal only and we don't set it there | 23:09 |
| cmurphy | yeah i guess that's why it was doubled | 23:09 |
| cmurphy | neat | 23:09 |
| clarkb | similar situation wtih groups.o.o | 23:09 |
| cmurphy | okay i gotta go to bed though, i may or may not be around tomorrow night and then after that i'm leaving for a week and a half so won't be around for this stuff | 23:10 |
| clarkb | ok, enjoy vacation if that is what it is | 23:10 |
| cmurphy | yep | 23:10 |
| clarkb | and thank you for pushing on this. I think groups.o.o is safe too given the above conclusion | 23:10 |
| *** anteaya has quit IRC | 23:10 | |
| cmurphy | yeah i think so too | 23:10 |
| clarkb | but maybe I will save that for morning just in case | 23:10 |
| *** slaweq has joined #openstack-infra | 23:11 | |
| *** felipemonteiro_ has quit IRC | 23:11 | |
| clarkb | pabelanger: change to add ze11 lgtm, just need zuul to confirm and anothe rroot to review | 23:13 |
| *** slaweq has quit IRC | 23:16 | |
| *** graphene has quit IRC | 23:20 | |
| *** mriedem has joined #openstack-infra | 23:21 | |
| *** graphene has joined #openstack-infra | 23:21 | |
| ianw | pabelanger: oh, this is one where we need the dns entry first for the iptables rules? is that in? | 23:23 |
| clarkb | ianw: ya that was the problem earlier with rackdns command not working, I think pabelanger added records in webui to get around the ssl problems | 23:24 |
| *** eernst has quit IRC | 23:26 | |
| *** lpetrut has joined #openstack-infra | 23:26 | |
| *** tosky has quit IRC | 23:27 | |
| pabelanger | yah | 23:27 |
| pabelanger | will push up patch to puppet for SSL cert on puppetmaster tomorrow morning | 23:27 |
| pabelanger | reading up on best practice for it, I think there is an extra folder we create | 23:28 |
| *** eernst has joined #openstack-infra | 23:28 | |
| *** eernst has quit IRC | 23:29 | |
| *** eernst_ has joined #openstack-infra | 23:29 | |
| clarkb | pabelanger: infracloud did some of that too if y ouneed examples | 23:29 |
| clarkb | (it added the cafile for the control plane to all the nodes) | 23:29 |
| *** threestrands has joined #openstack-infra | 23:30 | |
| pabelanger | k | 23:30 |
| *** lpetrut has quit IRC | 23:30 | |
| *** ijw has quit IRC | 23:32 | |
| *** larainema has joined #openstack-infra | 23:33 | |
| clarkb | looks like john completely removed that disabled compute host in packethost | 23:33 |
| *** eernst_ has quit IRC | 23:33 | |
| *** eernst has joined #openstack-infra | 23:33 | |
| clarkb | and the lsat ssh issue according to logstsah was 3.5 hours ago | 23:34 |
| pabelanger | nice | 23:34 |
| clarkb | also 97 instances in use and 2 building and one deleting | 23:35 |
| clarkb | use all that capacity | 23:35 |
| pabelanger | clarkb: so, is this an openstack cloud built atop of the packet.net baremetal servers? | 23:36 |
| *** jamesmcarthur has joined #openstack-infra | 23:36 | |
| clarkb | yup | 23:36 |
| pabelanger | clarkb: community driven? I think they offer resources for OSS projects | 23:36 |
| clarkb | its a collaboration between packet and platform9 (packet provides baremetal, platform9 the openstack) and is managed by john studarus | 23:36 |
| pabelanger | ah, cool | 23:37 |
| *** eernst has quit IRC | 23:38 | |
| pabelanger | clarkb: it was https://www.worksonarm.com/ I was thinking off | 23:38 |
| *** edmondsw has joined #openstack-infra | 23:38 | |
| pabelanger | providing CI/CD infra | 23:38 |
| pabelanger | https://github.com/worksonarm/cluster/ | 23:39 |
| pabelanger | would be a fun project | 23:39 |
| clarkb | pabelanger: I think they intend on adding arm hardware to this cloud as a next step | 23:41 |
| pabelanger | nice | 23:41 |
| pabelanger | guess kata is setup too | 23:42 |
| pabelanger | https://github.com/WorksOnArm/cluster/issues/31 | 23:42 |
| clarkb | they are in the process of adding that aiui | 23:42 |
| clarkb | I've mostly focused on the x86 stuff as that is where we can have virt | 23:42 |
| pabelanger | clarkb: yah, seen jobs running on zuul.o.o today. Nice work | 23:43 |
| *** edmondsw has quit IRC | 23:43 | |
| *** jamesmcarthur has quit IRC | 23:45 | |
| *** jamesmcarthur has joined #openstack-infra | 23:45 | |
| pabelanger | clarkb: ianw: I've +3 https://review.openstack.org/584546/ | 23:49 |
| pabelanger | will monitor and make sure it starts launching jobs properly | 23:49 |
| clarkb | pabelanger: ok let me know if I can help (I'm still around) | 23:49 |
| ianw | "Nothing says software development like stickers and magnets" | 23:50 |
| *** mriedem has quit IRC | 23:50 | |
| pabelanger | ++ | 23:50 |
| pabelanger | (to both) | 23:50 |
| *** rpioso is now known as rpioso|afk | 23:51 | |
| *** akhilaki has quit IRC | 23:52 | |
| clarkb | I'm the odd one out on the stickers front | 23:55 |
| *** sthussey has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
