| *** dingyichen has joined #openstack-infra | 00:00 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add non-voting initial gate for LOCI https://review.openstack.org/450473 | 00:03 |
|---|---|---|
| *** signed8bit is now known as signed8bit_Zzz | 00:04 | |
| *** hongbin_ has quit IRC | 00:07 | |
| *** EricGonczer_ has quit IRC | 00:08 | |
| *** EricGonczer_ has joined #openstack-infra | 00:11 | |
| *** thorst has joined #openstack-infra | 00:12 | |
| *** thorst has quit IRC | 00:13 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Fetch list of AZs from nova if it's not configured https://review.openstack.org/450345 | 00:14 |
| *** signed8bit_Zzz is now known as signed8bit | 00:16 | |
| *** gcb has quit IRC | 00:18 | |
| *** oomichi has quit IRC | 00:19 | |
| *** pvaneck has quit IRC | 00:22 | |
| *** oomichi has joined #openstack-infra | 00:23 | |
| *** mtanino_ has joined #openstack-infra | 00:26 | |
| *** mtanino has quit IRC | 00:27 | |
| *** viks has quit IRC | 00:28 | |
| *** adrian_otto has quit IRC | 00:33 | |
| *** EricGonczer_ has quit IRC | 00:36 | |
| openstackgerrit | Lana Brindley proposed openstack-infra/infra-manual master: Adding instructions for rebasing https://review.openstack.org/447341 | 00:40 |
| *** thorst has joined #openstack-infra | 00:43 | |
| *** gyee has quit IRC | 00:47 | |
| *** EricGonczer_ has joined #openstack-infra | 00:47 | |
| *** Julien-zte has joined #openstack-infra | 00:48 | |
| *** Apoorva_ has joined #openstack-infra | 00:48 | |
| *** rlandy has quit IRC | 00:48 | |
| *** Apoorva has quit IRC | 00:52 | |
| *** Apoorva_ has quit IRC | 00:52 | |
| *** jamielennox is now known as jamielennox|away | 00:54 | |
| openstackgerrit | Dean Troyer proposed openstack-infra/project-config master: Add OSC functional tips job to osc-lib and os-client-config https://review.openstack.org/450419 | 00:54 |
| *** jamielennox|away is now known as jamielennox | 00:59 | |
| *** gongysh has joined #openstack-infra | 01:01 | |
| *** thorst has quit IRC | 01:02 | |
| *** EricGonczer_ has quit IRC | 01:02 | |
| *** namnh has joined #openstack-infra | 01:03 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Don't fail ping check for pypi mirror on nonstandard port https://review.openstack.org/450420 | 01:06 |
| *** liujiong has joined #openstack-infra | 01:08 | |
| *** amotoki has joined #openstack-infra | 01:08 | |
| *** thorst has joined #openstack-infra | 01:11 | |
| *** thorst has quit IRC | 01:11 | |
| *** markvoelker has quit IRC | 01:13 | |
| *** amotoki has quit IRC | 01:14 | |
| *** markvoelker has joined #openstack-infra | 01:14 | |
| *** rfolco has joined #openstack-infra | 01:18 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Ignore errors for nodepool mirror checks https://review.openstack.org/450456 | 01:18 |
| *** tuanluong has joined #openstack-infra | 01:21 | |
| *** knangia has quit IRC | 01:21 | |
| *** john-davidge has joined #openstack-infra | 01:22 | |
| *** rfolco_ has joined #openstack-infra | 01:24 | |
| *** yanyanhu has joined #openstack-infra | 01:24 | |
| *** signed8b_ has joined #openstack-infra | 01:24 | |
| *** xgerman has quit IRC | 01:25 | |
| *** timburke has quit IRC | 01:25 | |
| *** mnencia has quit IRC | 01:25 | |
| *** larainema has quit IRC | 01:26 | |
| *** xgerman has joined #openstack-infra | 01:26 | |
| *** liusheng has quit IRC | 01:27 | |
| *** larainema has joined #openstack-infra | 01:27 | |
| *** rfolco has quit IRC | 01:27 | |
| *** signed8bit has quit IRC | 01:27 | |
| *** liusheng has joined #openstack-infra | 01:27 | |
| *** timburke has joined #openstack-infra | 01:28 | |
| *** knangia has joined #openstack-infra | 01:28 | |
| *** john-davidge has quit IRC | 01:28 | |
| *** bstinson has quit IRC | 01:30 | |
| *** Sihan has joined #openstack-infra | 01:30 | |
| Sihan | Hi Guys, how to config zuul triggered by specific file path? I could not find it in the doc. | 01:31 |
| *** bstinson has joined #openstack-infra | 01:32 | |
| *** mnencia has joined #openstack-infra | 01:32 | |
| *** rhallisey has quit IRC | 01:34 | |
| *** gongysh has quit IRC | 01:34 | |
| *** Sihan has quit IRC | 01:36 | |
| *** signed8b_ is now known as signed8bit_Zzz | 01:38 | |
| *** rbuzatu has quit IRC | 01:42 | |
| *** markvoelker has quit IRC | 01:42 | |
| *** markvoelker has joined #openstack-infra | 01:44 | |
| openstackgerrit | Akira Yoshiyama proposed openstack/os-client-config master: Add allow_identity_admin to CloudConfig.get_session_endpoint() https://review.openstack.org/450500 | 01:47 |
| *** yamahata has quit IRC | 01:48 | |
| *** ym200 has joined #openstack-infra | 01:48 | |
| *** signed8bit_Zzz is now known as signed8b_ | 01:50 | |
| *** zhurong has joined #openstack-infra | 01:52 | |
| *** ramishra has joined #openstack-infra | 01:57 | |
| *** zz_dimtruck is now known as dimtruck | 01:58 | |
| *** nicolasbock has quit IRC | 02:00 | |
| *** gcb has joined #openstack-infra | 02:07 | |
| openstackgerrit | chenhb proposed openstack-infra/project-config master: Fix rally ci jobs for stable/0.9 branch https://review.openstack.org/450509 | 02:08 |
| *** signed8b_ is now known as signed8bit_Zzz | 02:08 | |
| *** mtanino_ has quit IRC | 02:12 | |
| *** thorst has joined #openstack-infra | 02:12 | |
| *** thorst has quit IRC | 02:17 | |
| openstackgerrit | ChangBo Guo(gcb) proposed openstack-infra/project-config master: Make tempest-neutron-dsvm-src-oslo.messaging-zmq-ubuntu-xenial non-voting https://review.openstack.org/450235 | 02:17 |
| *** oomichi has quit IRC | 02:28 | |
| *** hongbin has joined #openstack-infra | 02:30 | |
| *** oomichi has joined #openstack-infra | 02:32 | |
| *** signed8bit_Zzz is now known as signed8b_ | 02:33 | |
| *** esberglu has joined #openstack-infra | 02:36 | |
| *** oomichi has quit IRC | 02:39 | |
| *** jiaohaolin has joined #openstack-infra | 02:40 | |
| *** esberglu has quit IRC | 02:40 | |
| jiaohaolin | hi everybody，which version of tempest should I use to test Liberty? | 02:41 |
| *** oomichi has joined #openstack-infra | 02:42 | |
| *** gongysh has joined #openstack-infra | 02:43 | |
| *** rbuzatu has joined #openstack-infra | 02:43 | |
| *** dharinic has joined #openstack-infra | 02:46 | |
| *** yamahata has joined #openstack-infra | 02:47 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Install python-minimal for Xenial https://review.openstack.org/450526 | 02:49 |
| *** ym200 has quit IRC | 02:50 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: launch-node: ignore failure to restart eth0 https://review.openstack.org/450529 | 02:53 |
| *** markvoelker has quit IRC | 02:54 | |
| *** markvoelker has joined #openstack-infra | 02:54 | |
| jiaohaolin | hi everybody，which version of tempest should I use to test Liberty? | 02:56 |
| clarkb | jiaohaolin: I think 13.0.0 based on the tag info | 02:59 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add planet01.openstack.org https://review.openstack.org/450534 | 03:02 |
| *** jerryz has quit IRC | 03:05 | |
| ianw | jhesketh: rax dns question ... the rdns record is based on the server's id ? | 03:06 |
| ianw | PTR records can only be added for Rackspace Cloud Servers and Load Balancers. | 03:08 |
| *** mjblack has quit IRC | 03:08 | |
| ianw | i'm guessing yes | 03:08 |
| jhesketh | ianw: not sure exactly sorry.. best bet would be to ask rax support... Looking at the api it seems to be attached to the tenant https://developer.rackspace.com/docs/cloud-dns/v1/api-reference/reverse-dns-ptr-records/ | 03:10 |
| jhesketh | ianw: probably just exposed by the server ui though in the panel | 03:10 |
| ianw | jhesketh: yeah, no problems ... i'm just bringing up a control plane server in vexxhost | 03:13 |
| *** ym200 has joined #openstack-infra | 03:13 | |
| *** thorst has joined #openstack-infra | 03:13 | |
| ianw | i guess vexxhost own the ip and have to setup the reverse, i can't find any doc on that though | 03:14 |
| jhesketh | ianw: yeah, you'd have to ask them | 03:14 |
| *** lihi has quit IRC | 03:15 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add planet01.openstack.org https://review.openstack.org/450534 | 03:15 |
| *** lihi has joined #openstack-infra | 03:15 | |
| *** rfolco_ has quit IRC | 03:16 | |
| *** rbuzatu has quit IRC | 03:16 | |
| *** dimak has quit IRC | 03:20 | |
| clarkb | ianw: thats right, and they do have designate at vexxhost so should be possible somehow, mnaser likely knows | 03:20 |
| clarkb | fwiw we havent really donrle reverse ptrs for our mirror hosts | 03:20 |
| *** dimak has joined #openstack-infra | 03:21 | |
| *** rbuzatu has joined #openstack-infra | 03:22 | |
| *** john-davidge has joined #openstack-infra | 03:23 | |
| openstackgerrit | Mark T. Voelker proposed openstack-infra/project-config master: Rename DefCore Committee -> Interop WG https://review.openstack.org/433414 | 03:23 |
| ianw | clarkb: yeah, i guess it's mostly just if it's sending mail | 03:25 |
| *** mjblack has joined #openstack-infra | 03:26 | |
| ianw | just the launch-node script goes a bit bananas :) | 03:26 |
| *** rbuzatu has quit IRC | 03:26 | |
| *** john-davidge has quit IRC | 03:27 | |
| openstackgerrit | Mark T. Voelker proposed openstack-infra/project-config master: Rename DefCore Committee -> Interop WG https://review.openstack.org/433414 | 03:28 |
| *** thorst has quit IRC | 03:33 | |
| *** links has joined #openstack-infra | 03:36 | |
| *** mriedem has joined #openstack-infra | 03:38 | |
| *** vhosakot has joined #openstack-infra | 03:38 | |
| jiaohaolin | clarkb: thanks ,I will try it | 03:39 |
| *** hrubi has quit IRC | 03:42 | |
| *** hrubi has joined #openstack-infra | 03:45 | |
| *** IRCFrEAK has joined #openstack-infra | 03:47 | |
| *** IRCFrEAK has left #openstack-infra | 03:48 | |
| *** jkilpatr has quit IRC | 03:50 | |
| *** ajo has quit IRC | 03:50 | |
| *** adarazs has quit IRC | 03:50 | |
| *** rbrndt has joined #openstack-infra | 03:51 | |
| *** rbrndt has quit IRC | 03:53 | |
| *** adarazs has joined #openstack-infra | 03:53 | |
| *** ajo has joined #openstack-infra | 03:54 | |
| openstackgerrit | Jamie Lennox proposed openstack/os-client-config master: Stop special-casing idenity catalog lookups https://review.openstack.org/450259 | 03:54 |
| *** vipuls has quit IRC | 03:56 | |
| *** spiffxp has quit IRC | 03:56 | |
| *** vipuls has joined #openstack-infra | 03:56 | |
| *** zhurong has quit IRC | 03:57 | |
| *** spiffxp has joined #openstack-infra | 03:58 | |
| mnaser | ianw we'll have to do the rdns for you | 04:01 |
| mnaser | please let me know if you need anything changed :> | 04:01 |
| *** armax has quit IRC | 04:03 | |
| *** mriedem has quit IRC | 04:05 | |
| *** nmathew has joined #openstack-infra | 04:06 | |
| *** signed8b_ is now known as signed8bit_Zzz | 04:06 | |
| *** signed8bit_Zzz is now known as signed8b_ | 04:07 | |
| *** vhosakot has quit IRC | 04:08 | |
| *** jamielennox is now known as jamielennox|away | 04:09 | |
| *** ym200 has quit IRC | 04:12 | |
| *** beslemon has joined #openstack-infra | 04:13 | |
| *** kornicameister has joined #openstack-infra | 04:14 | |
| ianw | mnaser: if you like you can setup rdns for planet01.openstack.org (A and AAAA) | 04:18 |
| mnaser | ianw: sure, one sec | 04:18 |
| mnaser | ianw: 22.45.204.199.in-addr.arpa domain name pointer planet01.openstack.org. | 04:21 |
| *** markvoelker has quit IRC | 04:21 | |
| *** psachin has joined #openstack-infra | 04:22 | |
| ianw | mnaser: cool, thanks. just to confirm, no programmatic way to do this? just i'll do something with the launch-node script to say something sensible | 04:22 |
| mnaser | ianw not at the moment but its a wip with us integrating designate properly again | 04:22 |
| *** rbuzatu has joined #openstack-infra | 04:23 | |
| *** gouthamr has quit IRC | 04:23 | |
| mnaser | i will ping once we (hopefully) have that settled down and done | 04:23 |
| ianw | mnaser: cool, thanks | 04:23 |
| *** markvoelker has joined #openstack-infra | 04:23 | |
| mnaser | ianw feel free to ping me or support@vexxhost.com if im not around, note on the account should have enough info | 04:23 |
| *** beslemon_ has joined #openstack-infra | 04:24 | |
| *** beslemon has quit IRC | 04:26 | |
| *** hongbin has quit IRC | 04:27 | |
| openstackgerrit | Mark T. Voelker proposed openstack-infra/project-config master: Rename DefCore Committee -> Interop WG https://review.openstack.org/433414 | 04:27 |
| *** sree has joined #openstack-infra | 04:28 | |
| *** thorst has joined #openstack-infra | 04:30 | |
| *** jamielennox|away is now known as jamielennox | 04:30 | |
| *** shu-mutou has joined #openstack-infra | 04:31 | |
| *** thorst has quit IRC | 04:34 | |
| *** sree_ has joined #openstack-infra | 04:34 | |
| *** signed8b_ is now known as signed8bit_Zzz | 04:34 | |
| *** sree_ is now known as Guest80488 | 04:34 | |
| *** signed8bit_Zzz is now known as signed8b_ | 04:36 | |
| *** beslemon_ has quit IRC | 04:36 | |
| *** kornicameister has quit IRC | 04:36 | |
| *** beslemon has joined #openstack-infra | 04:37 | |
| *** sree has quit IRC | 04:38 | |
| *** claudiub has joined #openstack-infra | 04:40 | |
| *** ym200 has joined #openstack-infra | 04:41 | |
| *** bhavik1 has joined #openstack-infra | 04:49 | |
| *** signed8b_ has quit IRC | 04:52 | |
| *** signed8bit has joined #openstack-infra | 04:54 | |
| *** zhurong has joined #openstack-infra | 04:56 | |
| *** rbuzatu has quit IRC | 04:57 | |
| *** ym200 has quit IRC | 05:00 | |
| *** ym200 has joined #openstack-infra | 05:01 | |
| *** markvoelker has quit IRC | 05:04 | |
| *** makowals_ has joined #openstack-infra | 05:04 | |
| *** kornicameister has joined #openstack-infra | 05:05 | |
| *** aNuposic has quit IRC | 05:05 | |
| *** makowals has quit IRC | 05:07 | |
| *** makowals has joined #openstack-infra | 05:09 | |
| *** kornicameister has quit IRC | 05:11 | |
| *** makowals_ has quit IRC | 05:11 | |
| *** john-davidge has joined #openstack-infra | 05:24 | |
| *** ym200 has quit IRC | 05:28 | |
| *** john-davidge has quit IRC | 05:29 | |
| *** kornicameister has joined #openstack-infra | 05:36 | |
| *** rcernin has joined #openstack-infra | 05:38 | |
| *** vikrant has joined #openstack-infra | 05:39 | |
| jiaohaolin | why can't I use tempest to test Cinder ? It always throws error like this: 'TypeError: __init__() got an unexpected keyword argument 'chunked' ' | 05:44 |
| *** rbuzatu has joined #openstack-infra | 05:53 | |
| *** nadya has joined #openstack-infra | 06:00 | |
| *** beslemon has quit IRC | 06:05 | |
| *** nadya has quit IRC | 06:07 | |
| *** nadya has joined #openstack-infra | 06:08 | |
| *** andreas_s has joined #openstack-infra | 06:08 | |
| *** nadya has quit IRC | 06:08 | |
| *** jascott1_ has quit IRC | 06:08 | |
| *** tjones has quit IRC | 06:10 | |
| *** jascott1_ has joined #openstack-infra | 06:10 | |
| jiaohaolin | Is anybody here ? | 06:11 |
| jiaohaolin | why can't I use tempest to test Cinder ? It always throws error like this: 'TypeError: __init__() got an unexpected keyword argument 'chunked' ' | 06:11 |
| *** kfox1111 has quit IRC | 06:12 | |
| AJaeger | jiaohaolin: better ask on #openstack-qa | 06:14 |
| *** jascott1_ has quit IRC | 06:15 | |
| *** dharinic has quit IRC | 06:21 | |
| *** isaacb has joined #openstack-infra | 06:26 | |
| *** rbuzatu has quit IRC | 06:27 | |
| *** thorst has joined #openstack-infra | 06:31 | |
| *** thorst has quit IRC | 06:35 | |
| *** jamielennox is now known as jamielennox|away | 06:36 | |
| *** oomichi has quit IRC | 06:48 | |
| *** sambetts|afk has quit IRC | 06:48 | |
| *** john-davidge has joined #openstack-infra | 06:52 | |
| *** oomichi has joined #openstack-infra | 06:52 | |
| *** sambetts_ has joined #openstack-infra | 06:52 | |
| *** jiaohaolin has quit IRC | 06:54 | |
| *** mtreinish has quit IRC | 06:57 | |
| *** mtreinish has joined #openstack-infra | 06:58 | |
| *** john-davidge has quit IRC | 06:58 | |
| *** jiaohaolin has joined #openstack-infra | 06:58 | |
| *** matrohon has joined #openstack-infra | 07:00 | |
| *** marst has quit IRC | 07:06 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Change mutex to counting semaphore https://review.openstack.org/442563 | 07:12 |
| *** rbuzatu has joined #openstack-infra | 07:12 | |
| *** amotoki has joined #openstack-infra | 07:13 | |
| *** pcaruana has joined #openstack-infra | 07:13 | |
| *** tesseract has joined #openstack-infra | 07:18 | |
| *** amotoki has quit IRC | 07:18 | |
| *** aviau has quit IRC | 07:19 | |
| *** aviau has joined #openstack-infra | 07:20 | |
| *** florianf has joined #openstack-infra | 07:20 | |
| andreykurilin | AJaeger: hi hi:) https://review.openstack.org/#/c/450337/ here is a new version of my patch which bases on discussion with clarkb at #openstack-qa channel | 07:21 |
| *** jamielennox|away is now known as jamielennox | 07:21 | |
| *** nadya has joined #openstack-infra | 07:21 | |
| *** Hal has joined #openstack-infra | 07:22 | |
| *** aarefiev_afk is now known as aarefiev | 07:24 | |
| AJaeger | andreykurilin: then let clarkb please review it, this is a very special set up. Syntax is fine but that's all I can say ;( | 07:26 |
| *** jaosorior has joined #openstack-infra | 07:29 | |
| *** rcernin has quit IRC | 07:29 | |
| *** marst has joined #openstack-infra | 07:33 | |
| *** nadya has quit IRC | 07:34 | |
| *** nadya has joined #openstack-infra | 07:34 | |
| *** hashar has joined #openstack-infra | 07:35 | |
| *** makowals has quit IRC | 07:37 | |
| *** martinkopec has joined #openstack-infra | 07:39 | |
| *** ssbarnea has joined #openstack-infra | 07:41 | |
| openstackgerrit | Isaac Beckman proposed openstack-infra/puppet-openstackci master: Pass Java arguments to Jenkins master https://review.openstack.org/263021 | 07:41 |
| *** nadya has quit IRC | 07:42 | |
| *** nadya has joined #openstack-infra | 07:42 | |
| *** nadya has quit IRC | 07:43 | |
| andreykurilin | AJaeger: ok, np, I'll wait. Thanks | 07:44 |
| *** rcernin has joined #openstack-infra | 07:47 | |
| *** john-davidge has joined #openstack-infra | 07:48 | |
| openstackgerrit | Merged openstack-infra/project-config master: [Rally] increase CLI job timeout https://review.openstack.org/450209 | 07:48 |
| *** makowals has joined #openstack-infra | 07:50 | |
| *** liusheng has quit IRC | 07:51 | |
| *** liusheng has joined #openstack-infra | 07:51 | |
| *** amoralej|off is now known as amoralej | 07:53 | |
| openstackgerrit | Merged openstack-infra/project-config master: Enable check-requirements job for valenceclient https://review.openstack.org/450382 | 07:54 |
| openstackgerrit | Merged openstack-infra/project-config master: [magnum] Install dib in virtualenv https://review.openstack.org/450329 | 07:54 |
| *** furlongm has joined #openstack-infra | 07:57 | |
| *** rossella_s has joined #openstack-infra | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add collectd-ceilometer-plugin jobs https://review.openstack.org/448612 | 08:00 |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| openstackgerrit | Merged openstack-infra/project-config master: Increase concurrency for ironic jobs to 4 https://review.openstack.org/447525 | 08:00 |
| *** bhavik1 has quit IRC | 08:00 | |
| *** rcarrillocruz has joined #openstack-infra | 08:01 | |
| *** jpich has joined #openstack-infra | 08:01 | |
| openstackgerrit | Merged openstack-infra/project-config master: Skip non-doc jobs for networking-odl doc changes https://review.openstack.org/447405 | 08:02 |
| *** rcernin has quit IRC | 08:06 | |
| *** rcernin has joined #openstack-infra | 08:07 | |
| *** efoley has joined #openstack-infra | 08:09 | |
| *** nadya has joined #openstack-infra | 08:10 | |
| *** ssbarnea has quit IRC | 08:10 | |
| *** rcernin has quit IRC | 08:10 | |
| *** rcernin has joined #openstack-infra | 08:10 | |
| *** acabot has quit IRC | 08:11 | |
| *** nadya_ has joined #openstack-infra | 08:11 | |
| *** markus_z has joined #openstack-infra | 08:12 | |
| *** nadya has quit IRC | 08:12 | |
| *** amotoki has joined #openstack-infra | 08:14 | |
| *** ccamacho has joined #openstack-infra | 08:14 | |
| *** ralonsoh has joined #openstack-infra | 08:16 | |
| *** panda|zZ is now known as panda | 08:16 | |
| *** amotoki has quit IRC | 08:19 | |
| *** dizquierdo has joined #openstack-infra | 08:19 | |
| *** quite has quit IRC | 08:21 | |
| *** florianf has quit IRC | 08:22 | |
| *** quite has joined #openstack-infra | 08:22 | |
| *** dgonzalez has quit IRC | 08:24 | |
| *** florianf has joined #openstack-infra | 08:27 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/glean master: Revert "Enable network.service with systemd" https://review.openstack.org/450627 | 08:28 |
| *** dingyichen has quit IRC | 08:29 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Have simple-init enable network.service https://review.openstack.org/450436 | 08:29 |
| ianw | pabelanger / clarkb: argh, getting more complicated but I think that's right ^ | 08:29 |
| *** dgonzalez has joined #openstack-infra | 08:30 | |
| *** yamamoto has quit IRC | 08:31 | |
| *** yamahata has quit IRC | 08:31 | |
| *** thorst has joined #openstack-infra | 08:33 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack-infra/bindep master: bindep: depends: Add distro family aliases https://review.openstack.org/450634 | 08:34 |
| *** derekh has joined #openstack-infra | 08:34 | |
| *** e0ne has joined #openstack-infra | 08:35 | |
| *** lucas-afk is now known as lucasagomes | 08:35 | |
| odyssey4me | morning all, I'd appreciate some project-config reviews for https://review.openstack.org/449272 | 08:35 |
| *** alexchadin has joined #openstack-infra | 08:36 | |
| *** yuvalb has quit IRC | 08:36 | |
| *** rossella_s has quit IRC | 08:42 | |
| *** rossella_s has joined #openstack-infra | 08:43 | |
| *** rcernin has quit IRC | 08:46 | |
| *** rcernin has joined #openstack-infra | 08:46 | |
| *** electrofelix has joined #openstack-infra | 08:48 | |
| *** rcernin_ has joined #openstack-infra | 08:48 | |
| *** thorst has quit IRC | 08:52 | |
| *** ssbarnea has joined #openstack-infra | 08:59 | |
| *** tosky has joined #openstack-infra | 09:14 | |
| *** amotoki has joined #openstack-infra | 09:15 | |
| *** alexchadin has quit IRC | 09:16 | |
| *** amotoki has quit IRC | 09:19 | |
| *** knangia has quit IRC | 09:21 | |
| *** wolverineav has joined #openstack-infra | 09:22 | |
| *** tosky has quit IRC | 09:29 | |
| *** yamamoto has joined #openstack-infra | 09:31 | |
| *** ym200 has joined #openstack-infra | 09:32 | |
| *** tosky has joined #openstack-infra | 09:35 | |
| *** ym200 has quit IRC | 09:36 | |
| *** gongysh has quit IRC | 09:39 | |
| *** gongysh has joined #openstack-infra | 09:41 | |
| *** sshnaidm|off is now known as sshnaidm | 09:45 | |
| *** gongysh has quit IRC | 09:47 | |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Add support for custom branching https://review.openstack.org/450337 | 09:47 |
| *** thorst has joined #openstack-infra | 09:47 | |
| *** shardy has joined #openstack-infra | 09:51 | |
| *** yamamoto has quit IRC | 09:56 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config master: Add a new project sushy-tools https://review.openstack.org/450685 | 09:58 |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Add Barbican grenade gate https://review.openstack.org/447689 | 09:59 |
| *** s-shiono_ has quit IRC | 10:01 | |
| *** brianw has joined #openstack-infra | 10:02 | |
| brianw | having an issue using ansible and openstack modules because of shade. someone suggested I ask in this channel. When running ansible using openstack module commands I am getting error that "shade is required for this module". I have shade installed on the target host I am pointing to in ansible hosts file. Anyone have any ideas how to resolve. i have tried installing and re-installing | 10:04 |
| *** Hal has quit IRC | 10:04 | |
| brianw | not using virtenv | 10:04 |
| *** yamamoto has joined #openstack-infra | 10:04 | |
| *** masayukig[m] has quit IRC | 10:04 | |
| *** csmart has quit IRC | 10:04 | |
| *** csmart has joined #openstack-infra | 10:07 | |
| *** nadya_ has quit IRC | 10:07 | |
| *** tuanluong has quit IRC | 10:09 | |
| *** shu-mutou is now known as shu-mutou-AWAY | 10:10 | |
| *** nicolasbock has joined #openstack-infra | 10:11 | |
| openstackgerrit | Sreeram Vancheeswaran proposed openstack-infra/project-config master: Enable functional test job in nova-dpm https://review.openstack.org/450700 | 10:12 |
| *** tuanluong has joined #openstack-infra | 10:13 | |
| strigazi | hi infra. I just noticed the following: I did 3 commits in a row 450696 450697 450698. For 6 and 8 we got a msg from gerrit in out channel but for 450697 we didn't | 10:14 |
| *** namnh has quit IRC | 10:14 | |
| openstackgerrit | Sreeram Vancheeswaran proposed openstack-infra/project-config master: Enable functional test job in nova-dpm https://review.openstack.org/450700 | 10:15 |
| *** amotoki has joined #openstack-infra | 10:16 | |
| *** Julien-zte has quit IRC | 10:17 | |
| *** liujiong has quit IRC | 10:19 | |
| *** thorst has quit IRC | 10:19 | |
| *** amotoki has quit IRC | 10:20 | |
| *** zhurong has quit IRC | 10:20 | |
| *** Julien-zte has joined #openstack-infra | 10:23 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use unicode for change number extraction https://review.openstack.org/450704 | 10:24 |
| *** wolverineav has quit IRC | 10:24 | |
| *** yanyanhu has quit IRC | 10:25 | |
| *** adarazs is now known as adarazs_lunch | 10:27 | |
| *** sdague has joined #openstack-infra | 10:27 | |
| *** zhurong has joined #openstack-infra | 10:28 | |
| dims | AJaeger : https://review.openstack.org/#/c/449641/ is clearly an experiment without governance. right now there is no person tagged with owning the openstack code in the k8s repo. So we will not be worsening the situation either... :) | 10:29 |
| dims | hogepodge : ^ | 10:29 |
| florianf | AJaeger: Hi! I have a question regarding the use of the upper-constrainst file (if you have a moment...) | 10:33 |
| AJaeger | florianf, better ask on #openstack-requirements | 10:37 |
| AJaeger | dims: ok, will look later | 10:38 |
| * AJaeger needs to catch the plane to Kubecon now... | 10:38 | |
| dims | AJaeger : safe travels! | 10:38 |
| AJaeger | thanks, dims | 10:38 |
| *** AJaeger is now known as AJaeger_ | 10:38 | |
| florianf | AJaeger_: OK, thanks! | 10:39 |
| *** masayukig[m] has joined #openstack-infra | 10:41 | |
| *** alexchadin has joined #openstack-infra | 10:43 | |
| *** alexchadin has quit IRC | 10:43 | |
| *** shardy has quit IRC | 10:44 | |
| *** alexchadin has joined #openstack-infra | 10:44 | |
| *** shardy has joined #openstack-infra | 10:44 | |
| *** nadya has joined #openstack-infra | 10:44 | |
| *** alexchadin has quit IRC | 10:48 | |
| *** alexchadin has joined #openstack-infra | 10:52 | |
| *** rbuzatu has quit IRC | 10:59 | |
| *** yamamoto has quit IRC | 11:00 | |
| *** jkilpatr has joined #openstack-infra | 11:00 | |
| *** ym200 has joined #openstack-infra | 11:03 | |
| *** markus_z has quit IRC | 11:04 | |
| *** sambetts_ is now known as sambetts | 11:04 | |
| openstackgerrit | Oksana Voshchana proposed openstack-infra/devstack-gate master: WIP: Test SSL wsgi https://review.openstack.org/414986 | 11:06 |
| *** ym200 has quit IRC | 11:07 | |
| *** oomichi has quit IRC | 11:09 | |
| openstackgerrit | Merged openstack-infra/project-config master: Kubernetes External Cloud Provider for OpenStack https://review.openstack.org/449641 | 11:10 |
| *** askb has quit IRC | 11:10 | |
| *** oomichi has joined #openstack-infra | 11:13 | |
| openstackgerrit | Oksana Voshchana proposed openstack-infra/devstack-gate master: WIP: Test SSL wsgi https://review.openstack.org/414986 | 11:14 |
| *** amotoki has joined #openstack-infra | 11:17 | |
| *** markus_z has joined #openstack-infra | 11:17 | |
| *** furlongm has quit IRC | 11:19 | |
| *** thorst has joined #openstack-infra | 11:20 | |
| *** amotoki has quit IRC | 11:21 | |
| *** zhurong has quit IRC | 11:25 | |
| *** thorst has quit IRC | 11:25 | |
| *** yamamoto has joined #openstack-infra | 11:25 | |
| *** lucasagomes is now known as lucas-hungry | 11:28 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing TLS everywhere for libvirt https://review.openstack.org/450729 | 11:30 |
| *** alexchadin has quit IRC | 11:30 | |
| *** furlongm has joined #openstack-infra | 11:31 | |
| *** openstackgerrit has quit IRC | 11:33 | |
| *** nmathew has quit IRC | 11:33 | |
| *** mat128 has joined #openstack-infra | 11:35 | |
| *** openstackgerrit has joined #openstack-infra | 11:36 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: WIP: Add lvm management to diskimage-builder https://review.openstack.org/444403 | 11:36 |
| openstackgerrit | Sreeram Vancheeswaran proposed openstack-infra/project-config master: Enable functional test job in nova-dpm https://review.openstack.org/450700 | 11:38 |
| *** erlon has joined #openstack-infra | 11:38 | |
| *** flepied has quit IRC | 11:41 | |
| openstackgerrit | Sreeram Vancheeswaran proposed openstack-infra/project-config master: Enable functional test job in nova-dpm https://review.openstack.org/450700 | 11:41 |
| *** EricGonczer_ has joined #openstack-infra | 11:45 | |
| *** thorst has joined #openstack-infra | 11:45 | |
| *** hashar is now known as hasharLunch | 11:47 | |
| *** dizquierdo is now known as dizquierdo_afk | 11:49 | |
| *** rfolco has joined #openstack-infra | 11:51 | |
| *** EricGonczer_ has quit IRC | 11:51 | |
| sshnaidm | does anybody know what are new zuul urls like http://zm01.openstack.org/p/openstack/heat-agents and how to treat them? | 11:51 |
| *** rbuzatu has joined #openstack-infra | 11:51 | |
| *** jcoufal has joined #openstack-infra | 11:55 | |
| *** flepied has joined #openstack-infra | 11:56 | |
| *** adarazs_lunch is now known as adarazs | 11:56 | |
| *** jaypipes has joined #openstack-infra | 11:56 | |
| sshnaidm | pabelanger, ^ | 11:56 |
| fungi | strigazi: the change for the missing report was for the stable/ocata branch, and looking here i can see why it was silent: http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerritbot/channels.yaml#n549 | 11:57 |
| *** rhallisey has joined #openstack-infra | 11:57 | |
| fungi | sshnaidm: that's a url for a zuul merger. zuul-cloner relies on those (usually passed in the $ZUUL_URL envvar) to construct the fetch urls for the merge commits zuul has constructed for that particular job build | 11:58 |
| *** florianf has quit IRC | 11:59 | |
| strigazi | fungi thanks | 12:00 |
| fungi | sshnaidm: also i don't know that i'd call those urls "new" since zuul has been relying on them for at least a couple years (ever since the merger was split out into a separate gearman-connected daemon) | 12:00 |
| sshnaidm | fungi, as I understood when I do "git clone http://zm01.openstack.org/p/openstack/heat-agents" I already have all changes from patch merged, right? | 12:00 |
| *** jordanP has joined #openstack-infra | 12:00 | |
| sshnaidm | fungi, we relied for ZUUL_URL to do "git clone" and then "git fetch ... refspec ...", not it stopped to work | 12:00 |
| fungi | sshnaidm: not really, you should only expect to be able to fetch the $ZUUL_REF from that version of the repository. you should mostly be relying on local copies of the repos provided in /opt and the upstream mirrors at git.openstack.org for other refs | 12:01 |
| fungi | sshnaidm: using zuul-cloner will do all that behind the scenes for you | 12:01 |
| sshnaidm | fungi, I know, but I use ansible git module in this case.. | 12:02 |
| sshnaidm | fungi, http://paste.openstack.org/show/604465/ | 12:03 |
| fungi | sshnaidm: you could replicate what zuul-cloner does with ansible in that case (clone from the local copy in /opt, switch to git.openstack.org as the origin remote and remote update it, then fetch $ZUUL_REF from the $ZUUL_URL provided to that job build) | 12:03 |
| *** florianf has joined #openstack-infra | 12:04 | |
| *** Julien-zte has quit IRC | 12:05 | |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Add ocata to the gerritbot https://review.openstack.org/450745 | 12:07 |
| *** markus_z has quit IRC | 12:09 | |
| fungi | sshnaidm: looking at your paste, there are a few issues with what you have there... first, $ZUUL_URL isn't always zm01 (at least in our ci, there are 8 different mergers and it could be assigned any one of those at random for a given build) | 12:10 |
| sshnaidm | fungi, yeah, it's just for test, actually it's env ZUUL_URL there | 12:11 |
| fungi | sshnaidm: also, as i said, they're not guaranteed to have all refs, just the refs necessary to serve up $ZUUL_REF | 12:11 |
| sshnaidm | fungi, yep, that's the problem with this task.. | 12:11 |
| *** tuanluong has quit IRC | 12:11 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add non-voting initial gate for LOCI https://review.openstack.org/450473 | 12:11 |
| *** eroux has joined #openstack-infra | 12:12 | |
| fungi | sshnaidm: and most importantly, we tune then for serving a small number of git trfs to job workers, not to serve full git clone operations (serving to clients doing, say, `git clone nova` is very resource-intensive) | 12:12 |
| sshnaidm | fungi, but if using git.openstack.org, everything should work I think | 12:12 |
| fungi | sshnaidm: yes, though again if this is running in the upstream ci or on other ci systems building worker images with dib and our git cache element, you should be doing the initial clone from the local /opt filesystem | 12:13 |
| strigazi | Was there a recent change in devstack to use python3 only? This error look like missing python2 http://logs.openstack.org/96/450696/1/check/gate-functional-dsvm-magnum-k8s-ubuntu-xenial/6f2497e/logs/devstacklog.txt.gz#_2017-03-28_10_27_37_417 | 12:13 |
| pabelanger | sshnaidm: agree with fungi, should update your task to use zuul-cloner and just set up environmental vars | 12:14 |
| pabelanger | caching from /opt/git | 12:15 |
| sshnaidm | pabelanger, this task is not for tripleo-ci only, but for rdo-ci and local usage too | 12:15 |
| fungi | sshnaidm: the repos in /opt are usually less than a day old, and no more than a few days old, so if you clone from there, then change the origin to git.openstack.org and update, it only pulls a handful of refs over the network. benefits include reducing load on our git servers (again lots of jobs doing git clone nova or something _will_ overwhelm our servers pretty quickly), lowers network | 12:15 |
| fungi | utilization (reducing chances that a network hiccup will cause your job to fail prematurely), and is also faster | 12:15 |
| openstackgerrit | jstockto2000 proposed openstack-infra/jenkins-job-builder master: Add support for stash-pullrequest-builder plugin https://review.openstack.org/450750 | 12:16 |
| pabelanger | sshnaidm: you can still use zuul-cloner, but might not have the cache. | 12:16 |
| sshnaidm | fungi, we clone repos in very exceptional cases, usually just copying from /opt/stack/new where zuul-cloner cloned them | 12:16 |
| sshnaidm | pabelanger, ^^ | 12:16 |
| sshnaidm | fungi, so it doesn't create any load on infra | 12:17 |
| *** chlong has joined #openstack-infra | 12:17 | |
| fungi | strigazi: i don't know, but you might check with the devstack maintainers (they mostly congregate in #openstack-qa) | 12:17 |
| pabelanger | sshnaidm: right, this is because you are still using devstack-gate to setup repos. If you use zuul-cloner directly, it will do the same thing | 12:17 |
| *** dizquierdo_afk has quit IRC | 12:17 | |
| pabelanger | devstack-gate was the old way projects would get repos from zuul | 12:17 |
| sshnaidm | pabelanger, there is no /opt/git and zuul-cloner on usual developers laptop | 12:18 |
| strigazi | fungi ok, thanks again | 12:18 |
| pabelanger | strigazi: that is fine, it is just a cache. it means you'll download more things | 12:18 |
| fungi | strigazi: i know there's been ongoing work to exercise python 3 in devstack+tempest jobs to find places where services break, but i don't know how far along that effort is. dims may know | 12:18 |
| pabelanger | sshnaidm: keep in mind, we are changing this process in zuulv3. A job will just push the changes to your node for your, and you'll just ask zuul which repos | 12:19 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing TLS everywhere for libvirt https://review.openstack.org/450729 | 12:20 |
| dims | strigazi : there's nothing python3 related in that log | 12:20 |
| sshnaidm | pabelanger, very good | 12:20 |
| pabelanger | sshnaidm: http://git.openstack.org/cgit/openstack-infra/zuul/tree/playbooks/roles/prepare-workspace/tasks/main.yaml?h=feature/zuulv3#n18 | 12:20 |
| pabelanger | that's the basic idea | 12:20 |
| sshnaidm | pabelanger, and when zuulv3 will be in production? | 12:21 |
| dims | strigazi : (0 - "1") on python2.7 gives the identical message | 12:21 |
| strigazi | dims after a closer look yes, there isn't. I'll investigate | 12:21 |
| pabelanger | sshnaidm: still a few months out | 12:23 |
| sshnaidm | pabelanger, ok.. so we need to live with v2 now | 12:24 |
| sshnaidm | need to survive | 12:24 |
| pabelanger | sshnaidm: agree | 12:24 |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config master: Stop special-casing idenity catalog lookups https://review.openstack.org/450259 | 12:24 |
| openstackgerrit | Sreeram Vancheeswaran proposed openstack-infra/project-config master: Enable functional test job in nova-dpm https://review.openstack.org/450700 | 12:24 |
| *** dprince has joined #openstack-infra | 12:24 | |
| *** dave-mccowan has joined #openstack-infra | 12:25 | |
| *** gcb has quit IRC | 12:25 | |
| *** markmcd has joined #openstack-infra | 12:26 | |
| *** gcb has joined #openstack-infra | 12:26 | |
| *** Goneri has joined #openstack-infra | 12:27 | |
| pabelanger | ianw: clarkb: thanks for taking over the patch. Looks like a +2 now | 12:27 |
| pabelanger | clarkb: Oh, new patch from ianw | 12:27 |
| pabelanger | checks | 12:28 |
| *** lucas-hungry is now known as lucasagomes | 12:28 | |
| *** eroux has quit IRC | 12:28 | |
| *** xyang1 has joined #openstack-infra | 12:30 | |
| pabelanger | clarkb: https://review.openstack.org/#/c/450627 from ianw too | 12:30 |
| *** hasharLunch is now known as hashar | 12:32 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config master: Add a new project sushy-tools https://review.openstack.org/450685 | 12:34 |
| *** rlandy has joined #openstack-infra | 12:35 | |
| *** nmathew has joined #openstack-infra | 12:40 | |
| *** nmathew has quit IRC | 12:41 | |
| *** trown|outtypewww is now known as trown | 12:41 | |
| *** aysyd has joined #openstack-infra | 12:42 | |
| *** gordc has joined #openstack-infra | 12:42 | |
| *** rossella_s has quit IRC | 12:42 | |
| *** rossella_s has joined #openstack-infra | 12:43 | |
| *** gongysh has joined #openstack-infra | 12:43 | |
| *** ym200 has joined #openstack-infra | 12:44 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add non-voting initial gate for LOCI https://review.openstack.org/450473 | 12:44 |
| gcb | yolanda: Could you please help review https://review.openstack.org/450235, which blocks oslo.messaging commits to be merged | 12:46 |
| *** gouthamr has joined #openstack-infra | 12:46 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Move futures to requirements https://review.openstack.org/450765 | 12:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Move futures to requirements https://review.openstack.org/450765 | 12:52 |
| *** mdrabe has joined #openstack-infra | 12:54 | |
| *** esberglu has joined #openstack-infra | 12:54 | |
| *** rhallisey has quit IRC | 12:54 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: WIP: Add lvm management to diskimage-builder https://review.openstack.org/444403 | 12:54 |
| *** sbezverk has quit IRC | 12:55 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Fetch list of AZs from nova if it's not configured https://review.openstack.org/450345 | 12:56 |
| *** markus_z has joined #openstack-infra | 12:56 | |
| *** Julien-zte has joined #openstack-infra | 12:57 | |
| *** mat128 has quit IRC | 12:57 | |
| openstackgerrit | Merged openstack-infra/project-config master: Make tempest-neutron-dsvm-src-oslo.messaging-zmq-ubuntu-xenial non-voting https://review.openstack.org/450235 | 12:59 |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add non-voting initial gate for LOCI https://review.openstack.org/450473 | 13:01 |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add non-voting initial gate for LOCI https://review.openstack.org/450473 | 13:02 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Remove unused timing constants https://review.openstack.org/450770 | 13:03 |
| *** signed8bit has quit IRC | 13:03 | |
| *** ym200 has quit IRC | 13:06 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Fetch list of AZs from nova if it's not configured https://review.openstack.org/450345 | 13:06 |
| *** gcb has quit IRC | 13:07 | |
| *** amoralej is now known as amoralej|lunch | 13:08 | |
| *** jiaohaolin has quit IRC | 13:09 | |
| *** rbuzatu has quit IRC | 13:11 | |
| *** Gravious has quit IRC | 13:13 | |
| *** baoli has joined #openstack-infra | 13:13 | |
| *** baoli has quit IRC | 13:14 | |
| *** baoli has joined #openstack-infra | 13:14 | |
| *** hichihara has joined #openstack-infra | 13:14 | |
| *** amotoki has joined #openstack-infra | 13:18 | |
| *** eharney has joined #openstack-infra | 13:18 | |
| *** signed8bit has joined #openstack-infra | 13:19 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean master: [WIP] Fix debian-jessie systemd https://review.openstack.org/450775 | 13:21 |
| *** rhallisey has joined #openstack-infra | 13:21 | |
| *** jkilpatr has quit IRC | 13:22 | |
| *** jkilpatr has joined #openstack-infra | 13:22 | |
| *** links has quit IRC | 13:22 | |
| *** signed8b_ has joined #openstack-infra | 13:23 | |
| *** amotoki has quit IRC | 13:23 | |
| *** ym200 has joined #openstack-infra | 13:23 | |
| *** Guest80488 has quit IRC | 13:25 | |
| *** signed8bit has quit IRC | 13:25 | |
| *** sree has joined #openstack-infra | 13:26 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Log return code on failed dib build https://review.openstack.org/450782 | 13:30 |
| pabelanger | dirk: what https vs http the issue for opensuse DIB failure? | 13:32 |
| *** isaacb has quit IRC | 13:32 | |
| *** sree has quit IRC | 13:32 | |
| *** sree has joined #openstack-infra | 13:32 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Support upgrades in multinode CI https://review.openstack.org/450784 | 13:33 |
| *** rbuzatu has joined #openstack-infra | 13:34 | |
| *** rbuzatu has quit IRC | 13:34 | |
| *** rbuzatu has joined #openstack-infra | 13:34 | |
| *** rbuzatu has quit IRC | 13:34 | |
| *** amotoki has joined #openstack-infra | 13:34 | |
| *** rbuzatu has joined #openstack-infra | 13:35 | |
| *** rbrndt has joined #openstack-infra | 13:35 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Support upgrades in multinode CI https://review.openstack.org/450784 | 13:37 |
| *** sree has quit IRC | 13:37 | |
| *** nadya has quit IRC | 13:39 | |
| *** sbezverk has joined #openstack-infra | 13:40 | |
| *** dimtruck is now known as zz_dimtruck | 13:41 | |
| Shrews | mordred: i assume you still want 449173 even though dsvm jobs are passing now, yeah? | 13:42 |
| *** dizquierdo has joined #openstack-infra | 13:42 | |
| *** pblaho has quit IRC | 13:42 | |
| mordred | Shrews: yah - I think it'll make it easier for us to debug issues later | 13:43 |
| Shrews | k k | 13:43 |
| dirk | pabelanger: thanks for pinging on this again | 13:44 |
| dirk | pabelanger: I looked at it and I'm not able to reproduce it on either xenial nor on a opensuse host | 13:44 |
| dirk | pabelanger: so what I currently think is happening is that there is a $http_proxy set that is not being cleared | 13:45 |
| dirk | pabelanger: because I see that all apt-get calls in the same log have a "http_proxy=" (e.g. unset) before their invocation | 13:45 |
| dirk | but the zypper one hasn't | 13:45 |
| dirk | so my guess is that there is a proxy that blocks outside access. is that possible? | 13:45 |
| dirk | pabelanger: http:// is actualyl the correct url, although https:// might also work | 13:45 |
| *** knangia has joined #openstack-infra | 13:46 | |
| pabelanger | dirk: should be no proxy, workers on public interwebs | 13:46 |
| pabelanger | possible to add more debugging to zypper to see what is going on? | 13:46 |
| dirk | pabelanger: is there a way to make it collect /var/log/zypper.log and put it on logs.openstack.org? | 13:48 |
| *** markvoelker has joined #openstack-infra | 13:49 | |
| *** jordanP has quit IRC | 13:49 | |
| dirk | pabelanger: we could also try to stick in a -vvv somewhere. but I guess that requires a small diskimagebuilder-change | 13:49 |
| *** ihrachys has joined #openstack-infra | 13:49 | |
| dirk | pabelanger: can I add a depends-on for that? we should be able to figure it out then | 13:49 |
| dirk | pabelanger: https://etherpad.openstack.org/p/nodepool-with-opensuse-images captures the current state and action items that we talked about in atlanta btw | 13:50 |
| *** EricGonczer_ has joined #openstack-infra | 13:50 | |
| pabelanger | dirk: make sure, make the change to diskimage-builder, I will depend-on that, and collect the log | 13:50 |
| *** toidiu has joined #openstack-infra | 13:52 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean master: [WIP] Fix debian-jessie systemd https://review.openstack.org/450775 | 13:52 |
| *** toidiu has quit IRC | 13:52 | |
| *** sree has joined #openstack-infra | 13:52 | |
| *** sree has quit IRC | 13:52 | |
| *** sree has joined #openstack-infra | 13:53 | |
| *** EricGonc_ has joined #openstack-infra | 13:53 | |
| *** ldnunes has joined #openstack-infra | 13:54 | |
| *** hamzy has quit IRC | 13:54 | |
| *** EricGonczer_ has quit IRC | 13:55 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Force deploy commands to timeout https://review.openstack.org/450302 | 13:55 |
| *** krtaylor has quit IRC | 13:56 | |
| *** sree has quit IRC | 13:57 | |
| fungi | mordred: are you planning to be around for the infra team meeting today? i wanted to directly bring https://review.openstack.org/446426 up since it's also on the tc meeting agenda and it seemed to come as a surprise last week to at least some people who were responsible for parts of the code in it | 13:57 |
| openstackgerrit | Dirk Mueller proposed openstack/diskimage-builder master: DNM: give me more debug https://review.openstack.org/450800 | 13:59 |
| mordred | fungi: yes I am! | 13:59 |
| fungi | cool, adding you as a party on that topic item. thanks! | 13:59 |
| mordred | fungi: I have also spoken with at least some non-negative percentage of the people who may have been surprised last week | 13:59 |
| *** krtaylor has joined #openstack-infra | 14:00 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Use shorter timeout for update step https://review.openstack.org/448778 | 14:00 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Force deploy commands to timeout https://review.openstack.org/450302 | 14:00 |
| fungi | cool, i figure the discussion in the infra meeting will help inform my governance vote on it | 14:00 |
| *** ldnunes has quit IRC | 14:00 | |
| dirk | pabelanger: done | 14:01 |
| openstackgerrit | Dirk Mueller proposed openstack-infra/nodepool master: [WIP]: Add opensuse 42.2 DIB testing https://review.openstack.org/450045 | 14:01 |
| mordred | fungi: also - quick heads up - there is a bug in the shade 1.18.0 we released yesterday - we were pulling in a requirements via a transitive depend on heatclient - but didn't notice it was missing from our requirements because it WAS in our test-requirements | 14:01 |
| *** hongbin has joined #openstack-infra | 14:01 | |
| mordred | fungi: patch is in flight to fix and will cut a new 1.18.1 as soon as it lands | 14:01 |
| fungi | mordred: very cool, thanks | 14:01 |
| dirk | pabelanger: thinking about it, this might actually be the "zypper-minimal" element issue described under "Fix the opensuse-minimal element" on https://etherpad.openstack.org/p/nodepool-with-opensuse-images | 14:02 |
| *** yamahata has joined #openstack-infra | 14:02 | |
| dirk | pabelanger: I have a local hack for that for my experiments of porting the stuff over on xenial. I can upload that one as well and we'll compare the run | 14:02 |
| *** bh526r has joined #openstack-infra | 14:02 | |
| fungi | mordred: i continue to think that richard's tool to check imports against directly declared install_requires would be a good way to catch stuff like that | 14:02 |
| dirk | pabelanger: I'd gladly appreciate any review or progress on the pending reviews that are pointed out on that etherpad btw | 14:03 |
| *** amoralej|lunch is now known as amoralej | 14:03 | |
| fungi | jeblair: do you mind if i add scheduling the lists.o.o upgrade to the agenda for today's infra meeting? | 14:03 |
| pabelanger | dirk: yes, sounds like it | 14:03 |
| *** yamahata has quit IRC | 14:04 | |
| dirk | pabelanger: from what I remember I only had the issue on -t pattern, not on zypper ref. so that sounds weird. | 14:04 |
| dirk | pabelanger: anyway, we'll get to it.. whats the best way to make it always collect /var/log/zypper.log if it exists ? | 14:04 |
| dirk | there is probably some artifact collection code somewhere on teardown of a job? | 14:05 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Do not install test-requirements for ansible test https://review.openstack.org/450803 | 14:05 |
| mordred | Shrews: ^^ there's a stab at catching similar dep issues | 14:05 |
| *** xarses_ has quit IRC | 14:06 | |
| *** pblaho has joined #openstack-infra | 14:08 | |
| fungi | mordred: here we go... this is the tool i was trying to remember https://pypi.python.org/pypi/pip_check_reqs/2.0.1 | 14:08 |
| *** mriedem has joined #openstack-infra | 14:09 | |
| *** zz_dimtruck is now known as dimtruck | 14:10 | |
| openstackgerrit | Anton Studenov proposed openstack-infra/project-config master: [rally] Move experimental jobs to check section https://review.openstack.org/450807 | 14:11 |
| dmsimard | mordred: I think there's something wrong with the new shade release ? | 14:11 |
| dmsimard | mordred: http://paste.openstack.org/raw/604485/ | 14:11 |
| *** jaosorior has quit IRC | 14:12 | |
| *** yamamoto has quit IRC | 14:12 | |
| weshay | pabelanger, if one wanted to drop a README file at the top of the log directory for jobs obviously only certain jobs, where would one do that? I've failed twice in trying to get that done | 14:13 |
| *** jamesdenton has joined #openstack-infra | 14:14 | |
| pabelanger | weshay: which jobs? | 14:16 |
| *** andreas_s has quit IRC | 14:16 | |
| weshay | tripleo | 14:16 |
| pabelanger | I'd have to see how the publisher is setup | 14:16 |
| *** yamahata has joined #openstack-infra | 14:16 | |
| dmsimard | mordred: okay, just saw https://review.openstack.org/#/c/450765/ -- can you cut a 1.18.1 with it ? | 14:16 |
| fungi | dmsimard: yeah, he just mentioned he was going to asap | 14:17 |
| dmsimard | great | 14:17 |
| *** markvoelker has quit IRC | 14:17 | |
| fungi | dmsimard: and we were just discussing ways to catch that same class of bug going forward | 14:17 |
| pabelanger | weshay: devstack-gate has a help folder, you can see how we do it for tempest | 14:18 |
| weshay | hrm.. /me pokes around | 14:19 |
| mordred | dmsimard: yup. just waiting for it to clear the gate | 14:19 |
| pabelanger | weshay: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n1190 would be the publisher part | 14:19 |
| pabelanger | you'd do that same for tripleo | 14:19 |
| *** tphummel has joined #openstack-infra | 14:20 | |
| weshay | thanks pabelanger :) | 14:20 |
| *** Goneri has quit IRC | 14:24 | |
| *** ldnunes has joined #openstack-infra | 14:25 | |
| openstackgerrit | Merged openstack-infra/shade master: Move futures to requirements https://review.openstack.org/450765 | 14:25 |
| *** kgiusti has joined #openstack-infra | 14:25 | |
| *** eharney has quit IRC | 14:26 | |
| *** tonytan4ever has joined #openstack-infra | 14:26 | |
| *** hamzy has joined #openstack-infra | 14:27 | |
| *** armax has joined #openstack-infra | 14:27 | |
| jeblair | fungi: lists.o.o sounds good | 14:27 |
| mordred | infra-root, dmsimard: 1.18.1 tag pushed | 14:27 |
| *** burgerk has joined #openstack-infra | 14:28 | |
| fungi | jeblair: thanks, added! | 14:28 |
| *** Goneri has joined #openstack-infra | 14:30 | |
| openstackgerrit | Andrey Volkov proposed openstack-infra/project-config master: Add placement api ref jenkins job https://review.openstack.org/449557 | 14:30 |
| *** marst has quit IRC | 14:31 | |
| dmsimard | mordred: \o/ it's on pypi too https://pypi.python.org/pypi/shade/1.18.1 | 14:32 |
| * dmsimard tries his stuff again | 14:32 | |
| *** gcb has joined #openstack-infra | 14:33 | |
| *** eroux has joined #openstack-infra | 14:35 | |
| *** eroux has quit IRC | 14:35 | |
| *** gongysh has quit IRC | 14:38 | |
| *** marst has joined #openstack-infra | 14:38 | |
| *** yatinkarel has joined #openstack-infra | 14:39 | |
| *** flepied has quit IRC | 14:39 | |
| *** eroux has joined #openstack-infra | 14:41 | |
| *** armax has quit IRC | 14:41 | |
| *** eroux has quit IRC | 14:41 | |
| *** eharney has joined #openstack-infra | 14:41 | |
| *** armax has joined #openstack-infra | 14:41 | |
| *** vikrant has quit IRC | 14:48 | |
| openstackgerrit | Anton Studenov proposed openstack-infra/project-config master: [rally] Move experimental jobs to check section https://review.openstack.org/450807 | 14:48 |
| *** eroux has joined #openstack-infra | 14:53 | |
| *** eroux has quit IRC | 14:53 | |
| *** flepied has joined #openstack-infra | 14:54 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Fetch list of AZs from nova if it's not configured https://review.openstack.org/450345 | 14:54 |
| *** eroux has joined #openstack-infra | 14:56 | |
| *** xarses has joined #openstack-infra | 14:56 | |
| *** eroux has quit IRC | 14:57 | |
| *** ggnel_t has quit IRC | 14:58 | |
| *** xarses_ has joined #openstack-infra | 14:59 | |
| *** hichihara has quit IRC | 14:59 | |
| *** xarses has quit IRC | 15:00 | |
| *** andreww has joined #openstack-infra | 15:01 | |
| *** xarses has joined #openstack-infra | 15:02 | |
| *** xarses has quit IRC | 15:02 | |
| *** xarses has joined #openstack-infra | 15:02 | |
| *** xarses_ has quit IRC | 15:04 | |
| *** andreww has quit IRC | 15:05 | |
| *** eroux has joined #openstack-infra | 15:06 | |
| *** adrian_otto has joined #openstack-infra | 15:06 | |
| *** eroux has quit IRC | 15:06 | |
| breton | win 23 | 15:07 |
| breton | :( | 15:07 |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config master: Made ironic-dsvm-standalone voting https://review.openstack.org/450822 | 15:08 |
| *** dasm has quit IRC | 15:08 | |
| *** eroux has joined #openstack-infra | 15:09 | |
| *** martinkopec has quit IRC | 15:10 | |
| *** mriedem has quit IRC | 15:10 | |
| *** eroux has quit IRC | 15:10 | |
| *** dasm has joined #openstack-infra | 15:11 | |
| *** yamamoto has joined #openstack-infra | 15:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean master: [WIP] Fix debian-jessie systemd https://review.openstack.org/450775 | 15:15 |
| *** sbezverk has quit IRC | 15:20 | |
| *** yamamoto has quit IRC | 15:23 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Update ARA's Ansible testing for 2.1.x to 2.1.5.0 https://review.openstack.org/450847 | 15:23 |
| *** aarefiev is now known as aarefiev_afk | 15:24 | |
| *** wolverineav has joined #openstack-infra | 15:24 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Test with bonded public interface https://review.openstack.org/385562 | 15:25 |
| *** eroux has joined #openstack-infra | 15:25 | |
| *** eroux has quit IRC | 15:26 | |
| clarkb | pabelanger: ianw I will grab the latest dib change and rerun my local builds this morning | 15:26 |
| clarkb | pabelanger: ianw also thinking we don't want to release the new glean revert until dib is released with the simple-init change | 15:26 |
| pabelanger | clarkb: I am okay with that | 15:28 |
| *** baoli has quit IRC | 15:29 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Update ARA's Ansible version with releases for 2.1.x, 2.2.x and 2.3.x https://review.openstack.org/450847 | 15:29 |
| clarkb | (I know dib tests this but I am teh paranoid) | 15:29 |
| *** matrohon has quit IRC | 15:31 | |
| *** rcernin has quit IRC | 15:31 | |
| *** rcernin_ has quit IRC | 15:31 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add non-voting initial gate for LOCI https://review.openstack.org/450473 | 15:31 |
| *** eroux has joined #openstack-infra | 15:32 | |
| *** eroux has quit IRC | 15:33 | |
| *** beslemon has joined #openstack-infra | 15:35 | |
| *** eroux has joined #openstack-infra | 15:36 | |
| strigazi | fungi I had the same issue just now with stable/mitaka gerritbot not posting to our channel | 15:36 |
| strigazi | fungi mitaka is there though | 15:36 |
| *** eroux has quit IRC | 15:36 | |
| fungi | strigazi: what was the change it skipped this time? | 15:37 |
| strigazi | fungi https://review.openstack.org/#/c/450857/ | 15:37 |
| *** sbezverk has joined #openstack-infra | 15:37 | |
| *** beslemon has quit IRC | 15:39 | |
| strigazi | fungi, have a look at [1] too. [1]: [magnum] Add ocata to the gerritbot https://review.openstack.org/#/c/450745 | 15:39 |
| *** beslemon has joined #openstack-infra | 15:39 | |
| strigazi | fungi one more query, the releasenote job runs on mitaka while it shouldn't (we did't have reno in mitaka) see: https://review.openstack.org/#/c/441899/ | 15:41 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Docs: Remove cron references https://review.openstack.org/449140 | 15:41 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Docs: Correct availability-zones documentation. https://review.openstack.org/449147 | 15:42 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Docs: Remove refs to removed nodepool commands https://review.openstack.org/449152 | 15:43 |
| *** alexchadin has joined #openstack-infra | 15:45 | |
| *** e0ne has quit IRC | 15:46 | |
| *** eroux has joined #openstack-infra | 15:47 | |
| *** eroux has quit IRC | 15:47 | |
| fungi | strigazi: i don't have a lot of explanation for 450857. the bot's debug log saw the event from gerrit and then indicated it had an empty set of channels for that event | 15:47 |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: Add Barbican grenade gate https://review.openstack.org/447689 | 15:49 |
| strigazi | fungi ok, I'll push again. I think my patch needs one more change | 15:50 |
| strigazi | fungi Can you help about the releasenotes job that runs on mitaka? | 15:51 |
| fungi | strigazi: you'll want to set an exclusion for stable/mitaka for that job name in the jobs section of the openstack-infra/project-config:zuul/layout.yaml file | 15:52 |
| clarkb | there is already a pretty long string of excluded rpojects for release nodes | 15:52 |
| fungi | you should probably see some example negative lookahead regular expressions for other releasenotes jobs doing the same | 15:52 |
| clarkb | s/nodes/notes/ you likely can just add yours to that | 15:52 |
| strigazi | I | 15:52 |
| fungi | right, that | 15:53 |
| strigazi | I'll have a look | 15:53 |
| strigazi | thanks clarkb fungi | 15:53 |
| strigazi | btw clarkb the multinode job for magnum looks ok now | 15:53 |
| clarkb | strigazi: cool so you got the services you want running on each node? | 15:55 |
| strigazi | clarkb yes | 15:55 |
| TheJulia | Guys, is there any way to delete a patchset out of gerrit? | 15:56 |
| fungi | strigazi: grepping through channel logs, the most recent stable/mitaka change echoed by the bot in that channel was almost a week ago at http://eavesdrop.openstack.org/irclogs/%23openstack-containers/%23openstack-containers.2017-03-22.log.html#t2017-03-22T17:50:18 | 15:56 |
| fungi | TheJulia: that's not a feature gerrit has, no | 15:56 |
| thomas_oneill | fungi: that was a question on my behalf I just pushed something I shouldn't have, what do you recommend? shall I abandon the change and re-submit? | 15:57 |
| fungi | TheJulia: and even if we could, the ref has already propagated to all push-based mirrors, cached locally by people running gertty, et cetera | 15:57 |
| thomas_oneill | oh dear lol | 15:57 |
| *** eroux has joined #openstack-infra | 15:57 | |
| TheJulia | thomas_oneill: change all the things? | 15:58 |
| strigazi | fungi it's the change blocked by the reno job :) but this is irrelevant, i think | 15:58 |
| *** eroux has quit IRC | 15:58 | |
| fungi | thomas_oneill: TheJulia: it's like trying to "recall" an e-mail you've sent to a public mailing list | 15:58 |
| TheJulia | fungi: that is what I figured :) | 15:58 |
| TheJulia | fungi: thanks! | 15:58 |
| fungi | thomas_oneill: TheJulia: is the content of teh change sensitive in nature? | 15:58 |
| *** pblaho has quit IRC | 15:59 | |
| jeblair | lwin 24 | 15:59 |
| thomas_oneill | fungi: I'll /q you | 15:59 |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config master: Remove out of date comment https://review.openstack.org/450871 | 15:59 |
| *** alexchadin has quit IRC | 16:00 | |
| *** sbezverk has quit IRC | 16:01 | |
| *** vhosakot has joined #openstack-infra | 16:03 | |
| *** sbezverk has joined #openstack-infra | 16:04 | |
| *** xarses_ has joined #openstack-infra | 16:04 | |
| *** eroux has joined #openstack-infra | 16:05 | |
| *** erlon has quit IRC | 16:05 | |
| *** eroux has quit IRC | 16:05 | |
| *** eroux has joined #openstack-infra | 16:06 | |
| mordred | clarkb: I want to make an rpojects | 16:06 |
| *** eroux has quit IRC | 16:06 | |
| clarkb | mordred: my typing driver is an excellent name generator | 16:07 |
| *** efoley_ has joined #openstack-infra | 16:07 | |
| mordred | clarkb: it is! | 16:07 |
| *** sbezverk has quit IRC | 16:07 | |
| *** eroux has joined #openstack-infra | 16:07 | |
| *** eroux has quit IRC | 16:07 | |
| *** ym200 has quit IRC | 16:08 | |
| clarkb | pabelanger: local boot test of locally built image on latest dib simple-init patchset works great | 16:08 |
| clarkb | so will +1 | 16:08 |
| *** eroux has joined #openstack-infra | 16:08 | |
| *** eroux has quit IRC | 16:09 | |
| pabelanger | clarkb: k, do we want to ask EmilienM / tripleo to +3 and release? | 16:09 |
| clarkb | pabelanger: I'm not sure who is actively reviewing dib today. But would be best to get at least one of those individuals to weigh in | 16:09 |
| EmilienM | o/ | 16:10 |
| EmilienM | clarkb: I'm not DIB expert tbh | 16:10 |
| *** efoley has quit IRC | 16:10 | |
| clarkb | mordred: elsewhere I tyoped "South" as "Sourth" | 16:11 |
| mordred | that's the right spelling | 16:12 |
| * fungi lives in the sourth, can confirm | 16:12 | |
| *** mriedem has joined #openstack-infra | 16:13 | |
| *** ym200 has joined #openstack-infra | 16:14 | |
| *** eroux has joined #openstack-infra | 16:14 | |
| bkero | Is that pronounced "Surth"? | 16:14 |
| fungi | more or less | 16:15 |
| *** eroux has quit IRC | 16:15 | |
| mordred | bkero: it has more dipthongs | 16:15 |
| fungi | though the addition of extra vowels and syllables while dropping random consonants can vary widely across the sourth | 16:16 |
| *** pblaho has joined #openstack-infra | 16:16 | |
| mordred | the biggest mistake non-southerners have in trying to speak southern is not adding NEARLY enough dipthongs | 16:16 |
| mordred | fungi: thisis very true | 16:16 |
| bkero | mordred: I'd remind you we're in public and to not use vulgar terms | 16:16 |
| mordred | :) | 16:16 |
| mordred | my aunt calls my dad "Buddy" - except there are at least 8 vowels where that u sits | 16:16 |
| *** eroux has joined #openstack-infra | 16:17 | |
| * fungi sees mordred's diphthong and raises him an umlaut | 16:17 | |
| *** flwang has quit IRC | 16:17 | |
| *** eroux has quit IRC | 16:17 | |
| clarkb | is it weird that I really find regional accents in this country so fascinating? I think its a side effect of living in the pacific northwest where the most interestign it get is "warshington" | 16:17 |
| * mordred tosses a digraph onto the pile | 16:17 | |
| bkero | An umlaut? In 'murican? | 16:17 |
| fungi | bkero: it was probably actually just a smudge | 16:18 |
| mordred | clarkb: native washingtonians say boat funny too | 16:18 |
| *** eroux has joined #openstack-infra | 16:18 | |
| *** eroux has quit IRC | 16:18 | |
| bkero | clarkb: We have creek/crick too | 16:18 |
| bkero | and some creeping Canadian influence | 16:19 |
| mordred | also - in Seattle the phrase "storm of the century" is used to describe weather that is a best "a mild summer rain shower" anywhere else | 16:19 |
| *** alexchadin has joined #openstack-infra | 16:19 | |
| EmilienM | pabelanger: hey I found useful to report some issue with multinode & networking | 16:19 |
| EmilienM | pabelanger: http://logs.openstack.org/56/450756/2/check/gate-tripleo-ci-centos-7-nonha-multinode-oooq/5ac2866/console.html#_2017-03-28_14_54_14_438722 | 16:19 |
| clarkb | mordred: maybe I say boat funny and so don't notice | 16:20 |
| pabelanger | EmilienM: it is possible the release of DIB will fix that | 16:20 |
| EmilienM | pabelanger: how can I help? | 16:20 |
| clarkb | bkero: and I wouldn't say there is a creeping canadian influence, people that talk with canadian accent is easiest way to find people from canada | 16:21 |
| EmilienM | pabelanger: do you want me to submit a release patch or something? please tell me | 16:21 |
| *** eroux has joined #openstack-infra | 16:21 | |
| *** yamahata has quit IRC | 16:21 | |
| pabelanger | EmilienM: https://review.openstack.org/#/c/450436/ fixes an issue where networking wasn't setup properly on centos hosts. | 16:22 |
| pabelanger | which was causing multi-nodes jobs to fail | 16:22 |
| *** eroux has quit IRC | 16:22 | |
| pabelanger | so, we need a new release and rebuild images with nodepool-builder | 16:22 |
| bkero | clarkb: I'd say that a Vancouver accent is pretty close to a northern Washington accent | 16:23 |
| *** dizquierdo has quit IRC | 16:23 | |
| EmilienM | bnemec: can you review https://review.openstack.org/#/c/450436 please ? | 16:23 |
| EmilienM | pabelanger: I'm preparing the release patch now. | 16:23 |
| jeblair | i'm encountering "soarry" more frequently from non-canadians. (but still not from canadiens) | 16:24 |
| *** eroux has joined #openstack-infra | 16:24 | |
| *** psachin has quit IRC | 16:24 | |
| *** eroux has quit IRC | 16:24 | |
| bkero | clarkb: reminds me of https://www.youtube.com/watch?v=zuQK6t2Esng | 16:24 |
| * bnemec wants a wicked.service in his distro | 16:24 | |
| EmilienM | clarkb, pabelanger: https://review.openstack.org/450876 | 16:25 |
| *** eroux has joined #openstack-infra | 16:25 | |
| bkero | bnemec: are you actually running wicked? | 16:25 |
| *** hashar has quit IRC | 16:25 | |
| clarkb | EmilienM: thanks | 16:25 |
| EmilienM | clarkb: no, thank *you* - we need stable multinode jobs :) | 16:25 |
| *** eroux has quit IRC | 16:26 | |
| *** pcaruana has quit IRC | 16:26 | |
| clarkb | bkero: mordred I find that more than acecnt in the NW though its terminology that makes our english distinguishable | 16:27 |
| clarkb | pop is really common up here instead of soda or coke for example | 16:27 |
| bkero | Really? I use/hear soda here a lot. | 16:27 |
| bkero | I thought this was 'soda' country. | 16:27 |
| clarkb | its about 50/50 iirc | 16:27 |
| bnemec | bkero: No, just reviewing https://review.openstack.org/#/c/450436/7/diskimage_builder/elements/simple-init/post-install.d/80-simple-init | 16:27 |
| clarkb | which is higher percentage of "pop" than just about anywhere else aiui | 16:28 |
| EmilienM | bnemec: thanks for the review :-) | 16:28 |
| bkero | I'm from Montana, pop is the term there. | 16:28 |
| bkero | Definitely | 16:28 |
| *** aNuposic has joined #openstack-infra | 16:28 | |
| bkero | http://images.huffingtonpost.com/2012-11-09-Screenshot20121109at3.05.00PM.png | 16:29 |
| clarkb | and the classic "the 5" vs "I5" | 16:29 |
| *** flwang has joined #openstack-infra | 16:29 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 16:29 |
| bkero | When I lived in the bay area it was either -- it was "Take 680 north, merge onto 580 east, and get off at the Airway blvd exit" | 16:31 |
| bkero | neither* | 16:31 |
| *** rcarrillocruz has quit IRC | 16:33 | |
| *** chlong has quit IRC | 16:34 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Catch and log url pattern formatting errors https://review.openstack.org/450468 | 16:35 |
| *** tphummel has quit IRC | 16:37 | |
| *** nadya has joined #openstack-infra | 16:38 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 16:38 |
| openstackgerrit | Andrey Volkov proposed openstack-infra/project-config master: Add placement api ref jenkins job https://review.openstack.org/449557 | 16:39 |
| *** claudiub has quit IRC | 16:40 | |
| *** rossella_s has quit IRC | 16:42 | |
| *** markus_z has quit IRC | 16:42 | |
| *** tellesnobrega has quit IRC | 16:43 | |
| *** lucasagomes is now known as lucas-afk | 16:43 | |
| strigazi | fungi I pushed again against mitaka and gerritbot echoed in our channel | 16:44 |
| *** jpich has quit IRC | 16:44 | |
| *** Apoorva has joined #openstack-infra | 16:45 | |
| *** chlong has joined #openstack-infra | 16:49 | |
| *** derekh has quit IRC | 16:50 | |
| *** trown is now known as trown|lunch | 16:51 | |
| *** tphummel has joined #openstack-infra | 16:53 | |
| manjeets_ | https://review.openstack.org/#/c/439114/ | 16:54 |
| manjeets_ | review(s) needed | 16:55 |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Skip reno job in mitaka https://review.openstack.org/450888 | 16:58 |
| strigazi | clarkb fungi ^^ | 16:58 |
| *** nadya has quit IRC | 16:59 | |
| *** inc0 has quit IRC | 16:59 | |
| fungi | strigazi: actually, i misread the gerritbot debug log earlier. it did indeed think it echoed to the channel... | 16:59 |
| fungi | 2017-03-28 15:32:59,145 INFO gerritbot: Compiled Message #openstack-containers: Spyros Trigazis proposed openstack/magnum stable/mitaka: Fix tox.ini upper-constraints url https://review.openstack.org/450857 | 16:59 |
| fungi | but it doesn't show up in the channel log for that timestamp, i agree http://eavesdrop.openstack.org/irclogs/%23openstack-containers/latest.log.html | 16:59 |
| *** alexchadin has quit IRC | 16:59 | |
| *** inc0 has joined #openstack-infra | 16:59 | |
| fungi | er, the actual debug log entry i meant to paste was the one after that | 16:59 |
| fungi | 2017-03-28 15:32:59,145 INFO gerritbot: Sending "Spyros Trigazis proposed openstack/magnum stable/mitaka: Fix tox.ini upper-constraints url https://review.openstack.org/450857" to #openstack-containers | 17:00 |
| *** Swami has joined #openstack-infra | 17:01 | |
| *** yamahata has joined #openstack-infra | 17:01 | |
| *** rbuzatu has quit IRC | 17:02 | |
| *** rbuzatu has joined #openstack-infra | 17:03 | |
| *** prometheanfire has joined #openstack-infra | 17:04 | |
| *** rbuzatu_ has joined #openstack-infra | 17:04 | |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Skip reno job in mitaka https://review.openstack.org/450888 | 17:04 |
| pabelanger | speaking of summit, which hotels are people staying at | 17:05 |
| clarkb | pabelanger: btw feel free to approve https://review.openstack.org/#/c/414640/1 you can always change the sudo command later if you want | 17:07 |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Skip reno job in mitaka https://review.openstack.org/450888 | 17:07 |
| clarkb | pabelanger: I think i am in the westin? | 17:07 |
| *** rbuzatu has quit IRC | 17:07 | |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Skip reno job in mitaka https://review.openstack.org/450888 | 17:08 |
| *** rbuzatu_ has quit IRC | 17:08 | |
| *** bh526r has quit IRC | 17:08 | |
| pabelanger | clarkb: thanks, was going to look at this this afternoon | 17:10 |
| pabelanger | the patch | 17:10 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder master: Wrapper pre-scm-buildstep: added support for failOnError parameter https://review.openstack.org/349219 | 17:10 |
| *** rbuzatu has joined #openstack-infra | 17:11 | |
| *** mtanino has joined #openstack-infra | 17:11 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-nodepool master: Rename nodepoold to nodepool-launcher for zuulv3 https://review.openstack.org/450894 | 17:12 |
| openstackgerrit | Merged openstack/diskimage-builder master: Have simple-init enable network.service https://review.openstack.org/450436 | 17:12 |
| *** baoli has joined #openstack-infra | 17:12 | |
| *** mtanino has quit IRC | 17:13 | |
| *** jamesdenton has quit IRC | 17:13 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Don't set worker counts in nonha jobs https://review.openstack.org/425420 | 17:13 |
| *** flepied has quit IRC | 17:13 | |
| *** MasterOfBugs has joined #openstack-infra | 17:14 | |
| *** pramodrj07 has joined #openstack-infra | 17:14 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Use convert_mapping_to_xml for CAV plugin https://review.openstack.org/431798 | 17:15 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Re-enable test_timer_sshkey https://review.openstack.org/450897 | 17:15 |
| *** ldnunes has quit IRC | 17:16 | |
| *** baoli has quit IRC | 17:18 | |
| *** amoralej is now known as amoralej|off | 17:18 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 17:20 |
| openstackgerrit | Merged openstack/os-client-config master: Stop special-casing idenity catalog lookups https://review.openstack.org/450259 | 17:21 |
| *** tellesnobrega has joined #openstack-infra | 17:22 | |
| *** baoli has joined #openstack-infra | 17:23 | |
| *** jamesdenton has joined #openstack-infra | 17:25 | |
| prometheanfire | mordred: you did the shade and os_nova_flavor thing for ansible right? ansible seems to be hanging on https://github.com/ansible/ansible-modules-core/blob/devel/cloud/openstack/os_nova_flavor.py#L205 for me | 17:25 |
| mordred | prometheanfire: looking | 17:25 |
| *** rbrndt has quit IRC | 17:26 | |
| mordred | wow - it's hanging on get_flavor? | 17:26 |
| prometheanfire | yep | 17:26 |
| mordred | prometheanfire: can you try running: http://paste.openstack.org/show/604533/ | 17:27 |
| prometheanfire | ya, will be a tic | 17:29 |
| *** efoley__ has joined #openstack-infra | 17:30 | |
| *** amotoki has quit IRC | 17:31 | |
| *** flepied has joined #openstack-infra | 17:32 | |
| *** nadya has joined #openstack-infra | 17:33 | |
| *** nadya has quit IRC | 17:33 | |
| *** efoley_ has quit IRC | 17:33 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 17:36 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder master: docker-build-publish: add registry + server support https://review.openstack.org/336276 | 17:36 |
| *** adrian_otto has quit IRC | 17:37 | |
| andreykurilin | clarkb: hi! Please, re-review my change to project-config about features.yaml when you have free time | 17:38 |
| clarkb | mordred: for the shade futures thing, why wasn't that caught by the shade + nodepool integration test? that should only install with requirements not test requirements | 17:38 |
| prometheanfire | mordred: works with envars | 17:38 |
| mordred | clarkb: good question | 17:38 |
| clarkb | andreykurilin: https://review.openstack.org/#/c/450337/5 right? | 17:39 |
| mordred | prometheanfire: awesome. so something is potentially not working right in plumbing your auth into os_nova_flavor for you one would imagine | 17:39 |
| clarkb | andreykurilin: if so I have it pulled up will take a look in a bit | 17:39 |
| *** adrian_otto has joined #openstack-infra | 17:39 | |
| mordred | prometheanfire: are you trying to do envvars there - or are you using clouds.yaml ? | 17:39 |
| prometheanfire | neither | 17:39 |
| prometheanfire | using the auth param | 17:39 |
| andreykurilin | clarkb: yes | 17:39 |
| mordred | ah - ok | 17:39 |
| andreykurilin | Thanks | 17:40 |
| prometheanfire | http://docs.ansible.com/ansible/os_nova_flavor_module.html | 17:40 |
| mordred | prometheanfire: I think you should be able to pass all of those values directly in to the shade.openstack_cloud call in that snippet - lemme test real quick | 17:40 |
| mordred | (trying to isolate where it's going poorly) | 17:40 |
| prometheanfire | thanks | 17:41 |
| *** efoley__ has quit IRC | 17:41 | |
| fungi | pabelanger: yeah, i'm in the westin copley place as well | 17:42 |
| fungi | er, copely | 17:42 |
| *** tosky has quit IRC | 17:44 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Test with bonded public interface https://review.openstack.org/385562 | 17:46 |
| mordred | prometheanfire: example of pasting all the things in: http://paste.openstack.org/show/604547/ | 17:46 |
| pabelanger | fungi: ya, looks like still rooms. | 17:47 |
| *** baoli has quit IRC | 17:47 | |
| prometheanfire | k | 17:48 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Use custom network-isolation-absolute.yaml in ipv4 net-iso jobs https://review.openstack.org/422297 | 17:48 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Test with bonded public interface https://review.openstack.org/385562 | 17:48 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Add storage-mgmt network to compute nodes https://review.openstack.org/419049 | 17:48 |
| *** baoli has joined #openstack-infra | 17:49 | |
| *** baoli has quit IRC | 17:50 | |
| *** adrian_otto has quit IRC | 17:50 | |
| *** jerryz has joined #openstack-infra | 17:52 | |
| prometheanfire | mordred: yep, hanging | 17:52 |
| pabelanger | ianw: when you are online, do you mind doing a diskimage-builder release. It looks like there is some confusion about using openstack-releases for it or manually tagging | 17:52 |
| *** rbrndt has joined #openstack-infra | 17:53 | |
| *** electrofelix has quit IRC | 17:53 | |
| mordred | prometheanfire: neat! | 17:55 |
| prometheanfire | hanging on the ssl connection | 17:55 |
| prometheanfire | my auth_url is http though | 17:55 |
| mordred | prometheanfire: so - do you have things in OS_* that yuou don't have in those args? | 17:55 |
| prometheanfire | shouldn't | 17:56 |
| prometheanfire | no OS_ stuff | 17:57 |
| *** pvaneck has joined #openstack-infra | 17:58 | |
| *** baoli has joined #openstack-infra | 17:59 | |
| prometheanfire | it's just the public endpoint failing | 17:59 |
| prometheanfire | so that's neat | 18:00 |
| openstackgerrit | Clark Boylan proposed openstack-dev/pbr master: Lazy import pkg_resources https://review.openstack.org/450913 | 18:00 |
| prometheanfire | might be a problem on my end | 18:00 |
| prometheanfire | or shade should have a timeout :P | 18:00 |
| prometheanfire | or use the timeout | 18:02 |
| *** shardy has quit IRC | 18:03 | |
| *** trown|lunch is now known as trown | 18:03 | |
| *** sslypushenko_ has joined #openstack-infra | 18:04 | |
| *** tesseract has quit IRC | 18:06 | |
| *** adrian_otto has joined #openstack-infra | 18:06 | |
| *** ldnunes has joined #openstack-infra | 18:08 | |
| *** jkilpatr has quit IRC | 18:09 | |
| *** jkilpatr has joined #openstack-infra | 18:09 | |
| *** tosky has joined #openstack-infra | 18:09 | |
| *** nadya has joined #openstack-infra | 18:13 | |
| *** neiljerram has quit IRC | 18:15 | |
| openstackgerrit | Clark Boylan proposed openstack-dev/pbr master: Make version_string() lazy load version info https://review.openstack.org/450918 | 18:16 |
| *** flepied has quit IRC | 18:18 | |
| *** Hal has joined #openstack-infra | 18:20 | |
| *** adrian_otto has quit IRC | 18:21 | |
| *** signed8b_ is now known as signed8bit_Zzz | 18:23 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Refactor: block-device filesystem creation, mount and fstab https://review.openstack.org/444586 | 18:23 |
| *** gyee has joined #openstack-infra | 18:26 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 18:26 |
| openstackgerrit | Clark Boylan proposed openstack-dev/pbr master: Make version_string() lazy load version info https://review.openstack.org/450918 | 18:27 |
| openstackgerrit | Monty Taylor proposed openstack-dev/pbr master: WIP Add method to return deferred version string https://review.openstack.org/450921 | 18:27 |
| clarkb | mordred: heh I'm writing the patch too :P | 18:27 |
| clarkb | mordred: but doing it the way sdague wants it | 18:27 |
| mordred | clarkb: :) | 18:28 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Fix dev doc link in README.rst https://review.openstack.org/450922 | 18:29 |
| *** markvoelker has joined #openstack-infra | 18:32 | |
| pabelanger | mgagne: we are seeing some launch failures in internap-mtl01 today, happen to be away of any issues? | 18:39 |
| mgagne | pabelanger: I can verify. AFAIK, no deployment or maintenance are in progress. | 18:40 |
| pabelanger | also looks like ovh is down | 18:40 |
| pabelanger | mgagne: cool, wanted to make sure you were aware | 18:40 |
| mgagne | pabelanger: so this is not occurring atm but happened earlier? | 18:41 |
| pabelanger | mgagne: http://grafana.openstack.org/dashboard/db/nodepool-internap | 18:41 |
| mgagne | yea, checking at the graph | 18:42 |
| pabelanger | looks like 3 blips today | 18:42 |
| pabelanger | seen another happen not long ago | 18:42 |
| pabelanger | so, just wanted to ask | 18:42 |
| mgagne | sure, I'm asking around in our team and will check the logs | 18:42 |
| mgagne | if you have at least 1 UUID, that would be great | 18:43 |
| pabelanger | Ya, ovh-bhs1 is having issues. But working on it | 18:46 |
| pabelanger | mgagne: sure, 1 sec | 18:46 |
| pabelanger | mgagne: 0fe526bf-1c61-4d21-adf1-c77c61754933 was the most recent failure | 18:47 |
| mgagne | tyvm, will check | 18:47 |
| *** rkukura has quit IRC | 18:48 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 18:49 |
| *** signed8bit_Zzz is now known as signed8b_ | 18:49 | |
| mgagne | pabelanger: so far, no error in the control plane. Could it be the instance was not answering to SSH for some reasons and got deleted? | 18:50 |
| mgagne | I found spike in the "Time to Ready" graph matching the same period. Could it be related? | 18:52 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-iptables master: Ensure IP tables is always running https://review.openstack.org/450927 | 18:52 |
| asselin | fungi, ^^ | 18:52 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 18:55 |
| *** nadya has quit IRC | 18:55 | |
| *** tphummel has quit IRC | 18:55 | |
| *** armax has quit IRC | 18:55 | |
| fungi | thanks asselin! | 18:56 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix dev doc link in README.rst https://review.openstack.org/450922 | 18:56 |
| *** markvoelker has quit IRC | 18:57 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 18:58 |
| *** e0ne has joined #openstack-infra | 18:58 | |
| ianw | pableanger: will do, it's not under releases | 18:58 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-iptables master: Ensure IP tables is always running https://review.openstack.org/450927 | 18:58 |
| *** tphummel has joined #openstack-infra | 18:58 | |
| *** tphummel has quit IRC | 18:59 | |
| pabelanger | ianw: we should then remove it from: http://git.openstack.org/cgit/openstack/releases/tree/deliverables/_independent/diskimage-builder.yaml | 19:00 |
| *** ralonsoh has quit IRC | 19:00 | |
| fungi | it's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour | 19:00 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-iptables master: Ensure IP tables is always running https://review.openstack.org/450927 | 19:01 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: properties: cover lockable-resources plugin https://review.openstack.org/371292 | 19:03 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Update JaCoCo plugin https://review.openstack.org/320758 | 19:03 |
| *** nadya has joined #openstack-infra | 19:03 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: docker-build-publish: add registry + server support https://review.openstack.org/336276 | 19:03 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Update Gradle plugin to use convert_mapping_to_xml function https://review.openstack.org/416377 | 19:03 |
| *** neiljerram has joined #openstack-infra | 19:05 | |
| *** arxcruz has quit IRC | 19:05 | |
| *** ym200 has quit IRC | 19:05 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-iptables master: Ensure IP tables is always running https://review.openstack.org/450927 | 19:09 |
| *** signed8bit has joined #openstack-infra | 19:12 | |
| *** matrohon has joined #openstack-infra | 19:13 | |
| *** signed8b_ has quit IRC | 19:15 | |
| *** baoli has quit IRC | 19:15 | |
| *** nadya has quit IRC | 19:16 | |
| *** signed8b_ has joined #openstack-infra | 19:16 | |
| ianw | pabelanger: 2.2.0 out | 19:17 |
| pabelanger | ianw: danke! | 19:17 |
| *** signed8bit has quit IRC | 19:18 | |
| *** arxcruz has joined #openstack-infra | 19:23 | |
| *** hashar has joined #openstack-infra | 19:23 | |
| *** hashar_ has joined #openstack-infra | 19:24 | |
| *** hashar has quit IRC | 19:24 | |
| *** hashar has joined #openstack-infra | 19:24 | |
| openstackgerrit | Merged openstack-infra/system-config master: Fixes to get gem mirror working https://review.openstack.org/414640 | 19:26 |
| SamYaple | pabelanger: would you mind reviewing this again? https://review.openstack.org/#/c/450473/ got it to pass the gate, and I believe I understand the relationship between builders and job templates now | 19:29 |
| *** sree has joined #openstack-infra | 19:29 | |
| *** sree has quit IRC | 19:33 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Fix dsvm-nodepool-src jobs https://review.openstack.org/450945 | 19:34 |
| pabelanger | ianw: clarkb: ^when you can, 450945 missed an edge case with dsvm-nodepool-src job | 19:34 |
| *** mdrabe has quit IRC | 19:35 | |
| *** florianf has quit IRC | 19:35 | |
| pabelanger | SamYaple: I can look after meeting | 19:36 |
| dirk | pabelanger: it looks like https://review.openstack.org/#/c/450045/ didn't make use of the depends-on. looks like it installs the released pip version of diskimage-builder. so I need another way to capture the log I guess | 19:37 |
| SamYaple | pabelanger: thanks. take your time. no worries | 19:37 |
| pabelanger | dirk: dsvm-nodepool-src will | 19:38 |
| *** nadya has joined #openstack-infra | 19:41 | |
| *** nadya has quit IRC | 19:42 | |
| *** baoli has joined #openstack-infra | 19:43 | |
| *** Sihan has joined #openstack-infra | 19:43 | |
| *** ym200 has joined #openstack-infra | 19:44 | |
| *** rbuzatu has quit IRC | 19:44 | |
| Sihan | Hi Guys, how could I set the log link which send to the review patch in zuul? | 19:44 |
| *** e0ne has quit IRC | 19:46 | |
| *** sree has joined #openstack-infra | 19:48 | |
| clarkb | pabelanger: why not xenial? I think we should efault to xenail now | 19:48 |
| *** ym200 has quit IRC | 19:48 | |
| pabelanger | clarkb: we'd need to update nodepool for that too | 19:49 |
| *** e0ne has joined #openstack-infra | 19:49 | |
| pabelanger | I mean, we could | 19:49 |
| pabelanger | just haven't yet | 19:49 |
| clarkb | pabelanger: so that the test script looks for the right name? | 19:49 |
| pabelanger | clarkb: also, it seems we are not able to boot vms again in nodepool, so I'd like to stick with trusty until we fixed that | 19:49 |
| pabelanger | clarkb: we need to change some pause defaults | 19:50 |
| clarkb | ah | 19:50 |
| pabelanger | I can propose a change for that, on both branches | 19:50 |
| *** armax has joined #openstack-infra | 19:50 | |
| *** sree has quit IRC | 19:52 | |
| *** rockyg has joined #openstack-infra | 19:53 | |
| *** ym200 has joined #openstack-infra | 19:54 | |
| *** flepied has joined #openstack-infra | 19:55 | |
| *** rbuzatu has joined #openstack-infra | 19:55 | |
| *** rbuzatu has quit IRC | 19:55 | |
| *** rbuzatu has joined #openstack-infra | 19:55 | |
| openstackgerrit | Merged openstack-infra/project-config master: Fix dsvm-nodepool-src jobs https://review.openstack.org/450945 | 19:57 |
| *** mdrabe has joined #openstack-infra | 19:59 | |
| *** rbuzatu has quit IRC | 20:00 | |
| openstackgerrit | Merged openstack/os-client-config master: Remove out of date comment https://review.openstack.org/450871 | 20:00 |
| clarkb | cmurphy: ianw: just throwing this out there, haven't thought about if it helps or not, but maybe we attach all the rooter keys to the single user (to avoid sharing keys?) | 20:01 |
| clarkb | that doesn't prevent us from getting access to third party ci setups generally, but might be a good compromise for the concerns ianw listed and goals of switching to dib without puppet? | 20:01 |
| *** tphummel has joined #openstack-infra | 20:01 | |
| brianw | Hi, I am looking for real world feedback on SDn providers for Openstack and what people have experience with and can provide good feedback, experience, etc | 20:01 |
| brianw | Big Switch, Nokia, Midokura, etc | 20:02 |
| ianw | cmurphy / clarkb: i'm not sure if it's a concern so much ... just thought we should probably have some thoughts on access *before* we switch | 20:02 |
| ianw | some common thoughts | 20:02 |
| clarkb | brianw: our infracloud cloud uses provider networks with linuxbridge iirc. So not sure we have a lot of feedback for all the things | 20:02 |
| pabelanger | what if we add our keys to the known_hosts for jenkins / zuul user? | 20:02 |
| ianw | does dib need to get that out of puppet/hiera though? | 20:03 |
| pabelanger | that still means we remove shell accounts | 20:03 |
| clarkb | brianw: we did that because its simple and reliable (I can't recall any networking issues in infracloud?) but isn't really SDN | 20:03 |
| *** marst has quit IRC | 20:03 | |
| clarkb | ianw: ya or we could double account it | 20:03 |
| *** marst has joined #openstack-infra | 20:03 | |
| clarkb | ianw: so its not a perfect solution I don't think | 20:03 |
| ianw | can you grab pub keys from gerrit like you can from github accounts? | 20:04 |
| *** marst_ has joined #openstack-infra | 20:04 | |
| pabelanger | we put the zuul user public key on nodepool-builder disk today, we could do that say | 20:04 |
| pabelanger | same* | 20:04 |
| *** Apoorva has quit IRC | 20:04 | |
| * pabelanger I am not a fan of that approach | 20:04 | |
| brianw | yeah, we currently use linux bridge for that same reason. We are looking to look at SDN providers and whats next | 20:05 |
| *** Apoorva has joined #openstack-infra | 20:05 | |
| ianw | oh sure, i guess we could just bake public keys into the dib element | 20:05 |
| cmurphy | we could move the keys into a hiera yaml file and have dib read out of it | 20:05 |
| cmurphy | so there's only one source of truth | 20:05 |
| clarkb | its also possible it doesn't make sense to do it at all and sharing a key is fine | 20:07 |
| ianw | otherwise i was thinking something like a well-known script on nodepool.o.o "ssh-to-host" or something that uses all the right -i flags, maybe even knows how to translate a nodepool id into an ip | 20:07 |
| clarkb | ianw: I'd be happy with that too | 20:07 |
| cmurphy | clarkb: that would be easiest | 20:07 |
| pabelanger | or a playbook :) | 20:07 |
| dirk | imho you don't want to share a key.. the work of adding a single key or many keys should be similar | 20:08 |
| *** marst has quit IRC | 20:08 | |
| *** adrian_otto has joined #openstack-infra | 20:08 | |
| *** sree has joined #openstack-infra | 20:08 | |
| ianw | it is a nice luxury to be able to just jump onto a host quickly. i'm aware of my privilege in that regard :) | 20:08 |
| pabelanger | IMO: this is the downside of moving this into DIB, we something like hiera / vault | 20:10 |
| clarkb | dirk: I mean its already effectively a shared key | 20:10 |
| clarkb | dirk: its just not typically used by us because we bake our users and keys onto the hosts | 20:10 |
| clarkb | but nothing prevents us from using nodepool as a jump host today and nodepool uses the key everywhere | 20:10 |
| pabelanger | s/like/lack | 20:10 |
| clarkb | pabelanger: except you don't | 20:10 |
| openstackgerrit | Dirk Mueller proposed openstack/diskimage-builder master: DNM: give me more debug https://review.openstack.org/450800 | 20:10 |
| clarkb | pabelanger: just use hiera or vault they both work indepednelty aiui | 20:11 |
| clarkb | and dib doesn't care if you use things like that within it | 20:11 |
| ianw | ok, i put my preferences in a comment ... if nobody else has thoughts don't necessarily hold up forward progress for me | 20:12 |
| mordred | clarkb: fwiw - the jenkins user will not have sudo access if revoke-sudo has already been run -while our individual accounts WILL have it | 20:12 |
| pabelanger | clarkb: right, eventually we want to have this on the outside of dib. So something in the cleanup.d phase would set it | 20:12 |
| mordred | however, now I'm just being chatty | 20:12 |
| clarkb | mordred: ya Ithink thats a minor concern, and probably workaround is to just boot the thing and start over without sudo revocation | 20:13 |
| *** sree has quit IRC | 20:13 | |
| clarkb | and in the case of rootwrap just abuse the dd abilities of stack >_> | 20:13 |
| pabelanger | in zuulv3 we could have a pre-run playbook to populate keys | 20:13 |
| *** sree has joined #openstack-infra | 20:13 | |
| dirk | pabelanger: fungi : I'd really appreciate if you could review/add comments on https://etherpad.openstack.org/p/nodepool-with-opensuse-images . I'd be happy to work on additional items as well as get reviews on the individual open reviews that I posted | 20:14 |
| clarkb | another alternative is to always use glean + config drive | 20:14 |
| pabelanger | dirk: yes, it is on my list | 20:14 |
| clarkb | rather than bake any keys in | 20:14 |
| pabelanger | clarkb: I thought there was something stopping us from doing that | 20:15 |
| pabelanger | but, I would be okay with doing it | 20:15 |
| dirk | pabelanger: thanks. I'm trying to chase down the zypper failure in your nodepool test review meanwhile | 20:15 |
| clarkb | pabelanger: well right now glean only sets the root account, so we'd need to add a feature for not root | 20:15 |
| clarkb | pabelanger: or maybe bake not root in and then have glean add our keys with root | 20:15 |
| pabelanger | ya, that would make sense | 20:16 |
| *** jkilpatr has quit IRC | 20:16 | |
| pabelanger | also means SSHd change, I think we disable root logins by default | 20:16 |
| clarkb | no that got fixed because tripleo was building and booting our images and ran into that | 20:17 |
| pabelanger | ah, cool | 20:17 |
| *** sree has quit IRC | 20:18 | |
| *** jkilpatr has joined #openstack-infra | 20:18 | |
| clarkb | the more I think about it the more I think the best way to deal with this is probably just have zuul/jenkins with the single key. Then add support to nodepool to specify a metadata key that includes all our root keys | 20:19 |
| *** EricGonc_ has quit IRC | 20:19 | |
| clarkb | and have the ansible for cloud configuration set that in all our clouds | 20:19 |
| clarkb | ianw: cmurphy ^ thoughts? its nice bceause it forces us to use our images like anyone else would | 20:19 |
| *** Goneri has quit IRC | 20:20 | |
| cmurphy | clarkb: would the long-lived servers be able to use those keys? my concern is storing the keys in puppet and somewhere else and keeping the two sources in sync | 20:20 |
| pabelanger | I think we want to break the dependency on system-config for DIBs too, so it would mean 2 sources | 20:21 |
| clarkb | cmurphy: we'd have to rebuild them but yes | 20:21 |
| *** annegentle has joined #openstack-infra | 20:21 | |
| *** Goneri has joined #openstack-infra | 20:22 | |
| *** EricGonczer_ has joined #openstack-infra | 20:22 | |
| cmurphy | sounds fine to me then | 20:23 |
| *** ccamacho has quit IRC | 20:25 | |
| _ari_ | zaro: ping | 20:25 |
| cmurphy | in the mean time do we go forward with https://review.openstack.org/#/c/450029/ or wait? | 20:26 |
| pabelanger | fungi: if you don't get a volunteer for Apr. 11, I'm happy to give it a go again | 20:26 |
| ianw | clarkb: so nodepool.conf gets the keys put in by puppet? | 20:26 |
| *** e0ne has quit IRC | 20:26 | |
| *** kgiusti has left #openstack-infra | 20:27 | |
| fungi | pabelanger: no volunteers so far that i've seen, so it's all yours if you're available | 20:27 |
| clarkb | ianw: I think we'd have the ansible cloud config playbook in system-config read our keys from public hiera (which is wherep uppet could read them from) and add a keyname in each of our clouds called infra-root | 20:27 |
| clarkb | ianw: then nodepool would be updated to apply the keyname infra-root to all our hosts booted by nodepool | 20:27 |
| pabelanger | fungi: sure | 20:27 |
| jeblair | did someone say infra-root? a lot? :) | 20:28 |
| fungi | a whole lot, yes | 20:28 |
| clarkb | jeblair: I did, sorry was unintential pings but likely interesting idea to infra-roots :) | 20:28 |
| jeblair | clarkb: ya, catching up now :) | 20:28 |
| jeblair | clarkb: "ansible for cloud configuration" -- what's that? | 20:29 |
| *** e0ne has joined #openstack-infra | 20:30 | |
| clarkb | https://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/clouds_layouts.yml that thing | 20:30 |
| clarkb | https://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/run_cloud_launcher.yaml is playbook that injests that | 20:30 |
| *** mat128 has joined #openstack-infra | 20:31 | |
| *** rkukura has joined #openstack-infra | 20:32 | |
| pabelanger | how do we take nodepool to use that? | 20:32 |
| pabelanger | glean looks for public_keys today in metadata | 20:33 |
| clarkb | pabelanger: its a nova boot option, so likely would be somthing we associate to labels | 20:33 |
| clarkb | or maybe provider | 20:33 |
| pabelanger | k | 20:34 |
| *** sree has joined #openstack-infra | 20:34 | |
| ianw | the point mordred made ... we're keeping the ci user right (jenkins) and logging in as zuul? we don't want to be without sudo | 20:35 |
| *** mat128 has quit IRC | 20:35 | |
| pabelanger | jenkins will go away, so we'll only have zuul | 20:36 |
| clarkb | right that | 20:36 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: WIP: create a README_LOGS to document log files https://review.openstack.org/447653 | 20:36 |
| pabelanger | but ya, that is the edge today, revoke-sudo | 20:36 |
| clarkb | but if we glean + ssh keys fron config drive we always get root | 20:36 |
| *** ldnunes has quit IRC | 20:36 | |
| pabelanger | ya | 20:37 |
| clarkb | and its the way we expect our users to use the images already | 20:37 |
| *** askb has joined #openstack-infra | 20:37 | |
| clarkb | and it avoids the sahred ssh key problem | 20:37 |
| pabelanger | also means we can start sharing our images easier | 20:37 |
| jeblair | clarkb: i don't understand how the cloud launcher relates to nodepool images | 20:37 |
| clarkb | jeblair: the cloud launcher is how we can write our keys down in one place (hiera) and avoid double accounting but still have our keys put in place on long lived servers and in the cloud for nodepool | 20:38 |
| clarkb | jeblair: its just a tool to make sure all the clouds have the right keyname and contents in place | 20:38 |
| *** sree has quit IRC | 20:38 | |
| ianw | i really think it's a regression if we're trying to log in to debug and might not have sudo | 20:38 |
| jeblair | clarkb: oh you're saying use it to create OpenStack(R) Nova(TM) keypairs? | 20:39 |
| clarkb | ianw: ya thats where my current idea came from | 20:39 |
| clarkb | jeblair: yup | 20:39 |
| ianw | i understand not baking the keys in, so yeah, that leaves external injection | 20:39 |
| jeblair | things are clicking | 20:39 |
| jeblair | i would bet $0.10 that not having sudo won't be a problem | 20:40 |
| jeblair | however, keypairs still might be a better idea | 20:40 |
| pabelanger | we just removed keypair support it nodepool, would that need to be reverted? Or only if we want nodepool to create them directly | 20:41 |
| clarkb | baically in nodepool config we say "boot servers with this keypair, putting the keypair in noav is exercise for the user" | 20:41 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config master: Manage NTP on zuul workers in DIB https://review.openstack.org/450971 | 20:41 |
| clarkb | then we use th ansible tool for cloud config as our exercise machine | 20:41 |
| clarkb | pabelanger: only if we want nodepool to create them directly | 20:42 |
| jeblair | i don't think we want nodepool to do that | 20:42 |
| clarkb | pabelanger: I think it makes sense to not do that and instead just specify a name | 20:42 |
| pabelanger | k, that's what I thought | 20:42 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Move NTP management into o_p::server https://review.openstack.org/450972 | 20:42 |
| jeblair | clarkb: what do you want to have happen to the keys? where will they be injected? | 20:43 |
| pabelanger | Ya, ansible / shade seem better managing keypairs now | 20:43 |
| clarkb | jeblair: I think we just use current glean behavior which is apply them to root's authorized_keys | 20:43 |
| clarkb | jeblair: which is what users of our images that are not us have to do | 20:43 |
| clarkb | so its a way of making this consistent across humans | 20:44 |
| *** kzaitsev_mb has joined #openstack-infra | 20:44 | |
| jeblair | clarkb: makes sense | 20:44 |
| jeblair | we'll need to allow logins as root | 20:45 |
| clarkb | which we currently do (because humans not us already ran into that) | 20:45 |
| jeblair | clarkb: we don't have that in our sshd config on workers? | 20:45 |
| clarkb | not anymore | 20:45 |
| jeblair | neat? | 20:45 |
| clarkb | or rather our nodepool booted single use nodes allow root login | 20:46 |
| clarkb | via ssh | 20:46 |
| *** ldnunes has joined #openstack-infra | 20:46 | |
| clarkb | (which I think is acceptable given the nature of the images booted for this) | 20:46 |
| zaro | clarkb: question for you, https://review.openstack.org/#/c/399822 | 20:47 |
| *** neiljerram has quit IRC | 20:47 | |
| *** e0ne has quit IRC | 20:47 | |
| jeblair | i actually think it's acceptable on production servers as long as the fingerprint is logged. but i've never cared enough to press the issue. :) | 20:47 |
| jeblair | so i'm certainly okay with workers. just don't remember that happening. | 20:47 |
| *** esberglu has quit IRC | 20:47 | |
| *** esberglu has joined #openstack-infra | 20:48 | |
| clarkb | zaro: I'm guessing the reason zuul_prod.pp is wrong is it broke and we just fixed it by hand | 20:48 |
| zaro | clarkb: it might be good to try a Gerrit reindex run with current prod data to get an idea of how much downtime will be required for the reindex. | 20:49 |
| clarkb | zaro: so we can update that one. The reason I would like hiera is we have ~10 host keys we need on gerrit servers, itself, 8 git servers, and github | 20:49 |
| zaro | clarkb: opportunity to kill 2 birds with one stone? | 20:49 |
| clarkb | zaro: so we might as well just try to represent that cleanly in hiera | 20:49 |
| *** neiljerram has joined #openstack-infra | 20:50 | |
| dmsimard | hi cores, if you have a sec -- an easy jjb review: https://review.openstack.org/#/c/450847/ | 20:50 |
| *** tonytan4ever has quit IRC | 20:50 | |
| *** trown is now known as trown|outtypewww | 20:50 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Use support report command to collect node logs https://review.openstack.org/418562 | 20:51 |
| *** esberglu has quit IRC | 20:52 | |
| *** dprince has quit IRC | 20:53 | |
| clarkb | zaro: as for naming the hiera key, my example at https://review.openstack.org/#/c/399822/3/manifests/site.pp is gerrit_ssh_known_hosts | 20:53 |
| *** jcoufal has quit IRC | 20:55 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/tripleo-ci master: Use support report command to collect node logs https://review.openstack.org/418562 | 20:55 |
| zaro | clarkb: ok. guess that should be updated in zuul_prod.pp as well? | 20:55 |
| *** sambetts is now known as sambetts|afk | 20:55 | |
| *** rhallisey has quit IRC | 20:55 | |
| clarkb | zaro: since zuul only has to talk to gerrit I think its fine to leave that in place without hiera and just update the value. Separate change is fine for that | 20:55 |
| *** rbuzatu has joined #openstack-infra | 20:56 | |
| *** baoli has quit IRC | 20:57 | |
| *** e0ne has joined #openstack-infra | 20:58 | |
| *** markvoelker has joined #openstack-infra | 20:59 | |
| zaro | clarkb: any interest in trying gerrit reindex with prod data? | 20:59 |
| clarkb | zaro: how would we do that? | 21:00 |
| clarkb | zaro: I think it makes sense to try, but not sure what is involved | 21:00 |
| fungi | clarkb: jeblair: yeah, i personally have no concerns about enabling remote root login via (multiple) rsa keys on single-use nodes | 21:01 |
| *** e0ne has quit IRC | 21:01 | |
| fungi | seems perfectly fine to me | 21:01 |
| *** e0ne has joined #openstack-infra | 21:02 | |
| fungi | lots of people have a knee-jerk reaction to seeing the root account used, without applying logic | 21:02 |
| *** matrohon has quit IRC | 21:02 | |
| openstackgerrit | Merged openstack-infra/project-config master: [magnum] Skip reno job in mitaka https://review.openstack.org/450888 | 21:02 |
| fungi | i don't think we need to be those people | 21:02 |
| *** e0ne has quit IRC | 21:02 | |
| *** e0ne has joined #openstack-infra | 21:02 | |
| fungi | and i'm happy to explain the choice and the associated risk analysis to anyone who asks | 21:03 |
| *** e0ne has quit IRC | 21:03 | |
| Sihan | Does anyone know how could set the log link in zuul ? Currently it directly show the log link on the review, could I set the hyperlink? | 21:03 |
| zaro | clarkb: you can do that by setting up a gerrit similar to prod (Gerrit 2.11 +mysql). copy git repos to git folder, db dump. run upgrade then reindex | 21:03 |
| *** e0ne has joined #openstack-infra | 21:04 | |
| *** markvoelker has quit IRC | 21:04 | |
| *** e0ne has quit IRC | 21:04 | |
| asselin | pabelanger, want to reapprove https://review.openstack.org/#/c/263021/16? was simply rebased. | 21:04 |
| *** rfolco has quit IRC | 21:04 | |
| zaro | *db dump/db import | 21:04 |
| *** e0ne has joined #openstack-infra | 21:04 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Publish shade docs to docs.o.o/developer https://review.openstack.org/450977 | 21:04 |
| *** e0ne has quit IRC | 21:05 | |
| *** tellesnobrega has quit IRC | 21:05 | |
| *** e0ne has joined #openstack-infra | 21:05 | |
| *** rockyg has quit IRC | 21:05 | |
| *** e0ne has quit IRC | 21:05 | |
| *** e0ne has joined #openstack-infra | 21:06 | |
| *** e0ne has quit IRC | 21:06 | |
| clarkb | zaro: ok sounds liek a lot of work, maybe we plan for that after we get the other items out of the way? | 21:07 |
| *** Goneri has quit IRC | 21:07 | |
| *** ianychoi has quit IRC | 21:07 | |
| clarkb | zaro: I mostly don't want to get stuck in a rut on something difficult and instead focus on simpler things to start just to get momentum | 21:07 |
| fungi | Sihan: we configure our zuul layout.yaml to apply a set_node_options parameter function on all jobs, which we define in http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/openstack_functions.py#n18 | 21:07 |
| *** baoli has joined #openstack-infra | 21:08 | |
| *** ianychoi has joined #openstack-infra | 21:08 | |
| *** john-davidge has quit IRC | 21:08 | |
| *** prometheanfire has left #openstack-infra | 21:09 | |
| *** jamesdenton has quit IRC | 21:09 | |
| *** dimtruck is now known as zz_dimtruck | 21:10 | |
| *** gouthamr has quit IRC | 21:10 | |
| *** bulat has joined #openstack-infra | 21:11 | |
| bulat | hi. Folks please help with review https://review.openstack.org/#/c/447318/ | 21:11 |
| bulat | I used utf-8 symbols on my project, and without this patch jjb does not work :( | 21:11 |
| Sihan | @fungi How do you set the hyperlink stuff? | 21:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Change metadata to align with team affiliation https://review.openstack.org/450981 | 21:12 |
| *** john-davidge has joined #openstack-infra | 21:12 | |
| fungi | Sihan: as you can see there, we're assembling ours with a format string where we inject zuul job details | 21:12 |
| asselin | Sihan, is this for 3rd party ci? I believe the log link is documented....will look it up | 21:13 |
| Sihan | Yes, it is 3rd party :) | 21:13 |
| Sihan | that is why I am asking | 21:13 |
| asselin | http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/manifests/single_node_ci.pp#n90 | 21:13 |
| asselin | you should populate it here: http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/contrib/single_node_ci_data.yaml#n32 | 21:14 |
| asselin | Sihan, btw, full docs are here: https://docs.openstack.org/infra/openstackci/ | 21:14 |
| *** ldnunes has quit IRC | 21:15 | |
| *** hashar has quit IRC | 21:15 | |
| zaro | clarkb: haha, yeah it is a lot of work. | 21:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Force os-client-config to use ipv4 https://review.openstack.org/450983 | 21:16 |
| *** john-davidge has quit IRC | 21:16 | |
| *** kzaitsev_mb has quit IRC | 21:17 | |
| zaro | clarkb: so in terms of next steps. i think it’s upgrade review-dev.o.o and test there, i guess i would also suggest, in parallel, you can do the reindex with prod data. you’ll probably want to do that anyways to see if there’s any glaring defects with migrating prod data. | 21:19 |
| *** annegentle has quit IRC | 21:19 | |
| clarkb | zaro: makes sense | 21:19 |
| clarkb | zaro: and to upgrade review-dev.o.o we want to get your chagnes in, are you able to address the review comments? | 21:19 |
| *** vhosakot_ has joined #openstack-infra | 21:20 | |
| clarkb | zaro: also bulat has https://review.openstack.org/#/c/447318/2 to fix a jjb issue | 21:20 |
| *** ym200 has quit IRC | 21:20 | |
| *** vhosakot has quit IRC | 21:22 | |
| *** eharney has quit IRC | 21:22 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config master: Manage NTP on zuul workers in DIB https://review.openstack.org/450971 | 21:24 |
| clarkb | EmilienM: http://status.openstack.org/elastic-recheck/index.html#1669162 is currently top job failure bug and its marked won't fix. Do you know why you wouldn't want to address that somehow? | 21:26 |
| EmilienM | let me click | 21:26 |
| SpamapS | amrith: have you proposed a new MySQL diet with a higher sort buffer size yet? | 21:26 |
| EmilienM | clarkb: it was a mirror issue | 21:27 |
| *** Goneri has joined #openstack-infra | 21:27 | |
| EmilienM | clarkb: mwhahaha closed it because we can't do anything | 21:27 |
| EmilienM | clarkb: we already use mirrors where we can | 21:27 |
| clarkb | EmilienM: and we can't mirror the packages because they are updated many time a day? or? | 21:27 |
| EmilienM | clarkb: but we don't have a mirror for RDO trunk | 21:27 |
| EmilienM | clarkb: yeah, like every 4h | 21:28 |
| clarkb | gotcha | 21:28 |
| EmilienM | clarkb: but pabelanger and I are investigating something | 21:28 |
| *** vhosakot_ has quit IRC | 21:28 | |
| EmilienM | (mostly Paul is driving the work) | 21:28 |
| EmilienM | and he's going to mention it tomorrow during RDO weekly meeting on #rdo (freenode) | 21:28 |
| EmilienM | so more updates soon | 21:28 |
| *** burgerk has quit IRC | 21:28 | |
| pabelanger | ya, I want to get rsync enabled first on rdo | 21:29 |
| *** mriedem has quit IRC | 21:29 | |
| pabelanger | that figure out what we have to mirror | 21:29 |
| EmilienM | pabelanger: ++ | 21:29 |
| pabelanger | clarkb: was going to bring this up in meeting today, but we ran out of time. What is the down side of us having squid proxies in each mirror region for project to use, if the want to opt in | 21:30 |
| zaro | Clark: one change left? https://review.openstack.org/#/c/399822/ | 21:30 |
| zaro | clarkb: that change doesn’t block the next steps | 21:30 |
| *** vhosakot has joined #openstack-infra | 21:30 | |
| *** bulat has quit IRC | 21:31 | |
| *** ihrachys has quit IRC | 21:32 | |
| zaro | clarkb: it might even be better to do the prod data reindex before upgrading review-dev.o.o because you can’t roll back. but if you don’t care that much about keep review-dev on 2.11 then should do those in parallel. | 21:32 |
| mordred | pabelanger: I do not believe that squid proxies can cache https content | 21:32 |
| clarkb | zaro: https://review.openstack.org/#/c/449835/2/manifests/init.pp | 21:32 |
| clarkb | mordred: they can but you have to mitm yourself | 21:33 |
| mordred | yah | 21:33 |
| clarkb | http://wiki.squid-cache.org/Features/SslBump looks like there are several ways to do it over several squid versions | 21:33 |
| pabelanger | Right, but assuming http for the moment. I cannot think of a reason not to provide one | 21:33 |
| clarkb | zaro: I am +2 on that but I think we need it in before we can try review-dev | 21:33 |
| pabelanger | aside from setting it up and managing | 21:33 |
| mordred | yah. I think that's the other reason : ) | 21:34 |
| clarkb | pabelanger: the downside is we decided to go wtith a more cdn like explicit apprach | 21:34 |
| mordred | but I'm not opposed to it or anything | 21:34 |
| mwhahaha | we've got a squid setup for OVB that swaps https/http for this specific reason. so i think there's some other work arounds to be had | 21:34 |
| clarkb | mwhahaha: and aiui it causes problems all the time? | 21:34 |
| mwhahaha | i'm not aware of any | 21:34 |
| clarkb | bkero: ^ havne't you had issues with it in the past? | 21:34 |
| pabelanger | clarkb: what would a cdn look like? | 21:34 |
| zaro | clarkb: ohh you just mean changes need to merge. i thought you meant changes need to be fixed. | 21:35 |
| clarkb | pabelanger: what we do with afs is essentially a cdn, just without aliasing things in front of it | 21:35 |
| clarkb | zaro: ya that | 21:35 |
| pabelanger | mordred: ya, I might be volunteering to do the work :) | 21:35 |
| clarkb | mwhahaha: from past experience things like python requests + ssl + squid == sadness | 21:35 |
| zaro | clarkb: agree | 21:35 |
| mwhahaha | clarkb: these aren't python requests, it's yum requests which we could proxy via config | 21:36 |
| clarkb | mwhahaha: right but you aren't going to selectively turn on or off your proxy are you? | 21:36 |
| clarkb | its just going to be on and cache everything | 21:36 |
| clarkb | including whatever python-requests does | 21:36 |
| mwhahaha | not necessarily | 21:36 |
| pabelanger | clarkb: ya, I'm not sure what is looks like with AFS currently. since each region would download different data | 21:36 |
| clarkb | pabelanger: I'm not saying add squid and afs to gether. I am saying the design we have aimed for is cdn like not caching proxy like | 21:37 |
| clarkb | so I think it makes more sense to see how this would work in the afs cdn system then build a second system that solves the same problem | 21:37 |
| clarkb | s/then/than/ | 21:37 |
| pabelanger | Ah | 21:37 |
| mwhahaha | i would argue that we don't have a cdn like system but that's for another day | 21:38 |
| clarkb | the upside to afs cdn is it has been relatively reliable, but downside is high initial overhead to host content | 21:38 |
| clarkb | squid basically inverts those two things | 21:38 |
| pabelanger | right, I think that is the current complaint for AFS. high initial overhead | 21:39 |
| clarkb | squid has low initial overhead (for consumers) and ime is complicated and leads to unreliability in other eways | 21:39 |
| pabelanger | yes | 21:39 |
| clarkb | mwhahaha: the design of it is actually very similar to how eg youtubes cdn works. We just don't do dns magic in front of it | 21:39 |
| pabelanger | with squid, it would provide some relief before moving into longer AFS mirror | 21:39 |
| pabelanger | that's why I also said, opt-in for projects if they wanted to use the squid. | 21:40 |
| *** mriedem has joined #openstack-infra | 21:40 | |
| mwhahaha | clarkb: eh except you're having to know what to sync upfront rather than being dynamic | 21:41 |
| mtreinish | fungi: so I spent the morning playing with different mqtt brokers. I think vernemq https://github.com/erlio/vernemq will meet our needs | 21:41 |
| clarkb | mwhahaha: thats true of any cdn | 21:41 |
| mwhahaha | clarkb: not necessarily | 21:41 |
| clarkb | mwhahaha: you write the data to the source of truth, then point the network at it | 21:41 |
| mtreinish | I was able to get it spun up pretty easily in a similar config to mosquitto | 21:41 |
| mtreinish | https://github.com/mtreinish/puppet-vernemq | 21:41 |
| clarkb | then network updates its aches as necessary to serve content locally | 21:41 |
| clarkb | (typically on demand which is what afs gives us) | 21:41 |
| mtreinish | I haven't tested the websockets side yet though | 21:42 |
| mtreinish | nor the clustering stuff (but that was a future item anyway) | 21:42 |
| mwhahaha | no what you've done is predownload everything, not what's necessary. which is why we need something more dynamic for the rdo case where we only need the caching for a few hours | 21:42 |
| mwhahaha | so yes, you have a type of cdn. no you do not have something dynamic enough for things that go stale quick | 21:43 |
| clarkb | mwhahaha: right you are basically arguing rdo should be the source of truth not an afs volume | 21:43 |
| clarkb | which is fair | 21:43 |
| clarkb | and also potentially doable | 21:43 |
| mwhahaha | i have done it with squid and yes you can get stale cache issues but that's something i think we wouldn't hit with tripleo because of the rdo hashing. but it's something that would need to be tested | 21:44 |
| mwhahaha | and we can limit the caching to just yum if necessary | 21:44 |
| mwhahaha | anyway | 21:44 |
| clarkb | its not just stale cache issues, its the now you need to manage a cert authority (in order to cache for ssl), you have to convince python tools to work with it, and so forth | 21:45 |
| pabelanger | clarkb: mwhahaha: actually, maybe you can help review: https://github.com/pabelanger/rubygems-mirror/commit/dfdb464ce187bfd1ee76774a465a704239257924 and help me with apache rewrite configuration. If we consider forking ruby-mirror into infra for now, we can at least get a gem mirror into AFS | 21:45 |
| mwhahaha | i don't k now why you're fixated on python tools, the problem isn't those it's just yum | 21:45 |
| pabelanger | I tested it with AFS over the weekend, and it works propelry | 21:45 |
| mwhahaha | we dpm | 21:45 |
| pabelanger | all 300GB of gems | 21:45 |
| clarkb | mwhahaha: because as a service provider I know the instance that goes up someone somewhere is going to use it with pip | 21:46 |
| clarkb | I don't have to just consider your specific user case | 21:46 |
| pabelanger | foobar.gem becomes f/fo/foobar.gem on disk | 21:46 |
| mwhahaha | pabelanger: i think you still get too many clusters based on names | 21:46 |
| clarkb | it may work fine for you, but then I will spend all my time supporting someone else | 21:46 |
| mwhahaha | pabelanger: would be better to md5 the name or something | 21:46 |
| *** dizquierdo has joined #openstack-infra | 21:46 | |
| clarkb | mwhahaha: and since we happen to be predominantly a group of python projects/devs I know and expect that particular issue | 21:47 |
| pabelanger | mwhahaha: right, md5sum means adding something to apache2 to rewrite to md5 | 21:47 |
| pabelanger | do able | 21:47 |
| pabelanger | just wanted to make sure infra was okay with that | 21:47 |
| pabelanger | filename means no external apps to apache2 | 21:48 |
| mwhahaha | clarkb: that's fine and something we can consider. i'm not actually proposing anything at the moment. you guys brought me into this :D Just providing my $0.02 based on my cdn building experiances :) | 21:48 |
| jlvillal | Gerrit UI patch for review: https://review.openstack.org/#/c/448786/ Basically show POST_FAILURE in same color as FAILURE. Currently it appears as black text and easy to miss. | 21:48 |
| * mwhahaha goes back to doing other things | 21:49 | |
| pabelanger | clarkb: so, maybe we can sit down tomorrow or when you are free and hash out some of this. Because, I'd like to get something in place which potentially will help tripelo, and maybe some of our docker mirror issues too | 21:50 |
| *** thorst has quit IRC | 21:51 | |
| clarkb | pabelanger: so I think default to opt in is a reasonable idea (eg don't configure any proxy if any globally on the test VMs) | 21:52 |
| *** adrian_otto has quit IRC | 21:53 | |
| pabelanger | clarkb: right | 21:53 |
| clarkb | and probably what I would do to start if a proxy is what people want to try is just add it to the job. You won't get much benefit since the job life is slow and may only request each artifact once, but it will allow you to modify the job to use it and decide if taht additional setup is worthwhile | 21:54 |
| clarkb | you're going to find that dealing with ssl is a pita and so on | 21:54 |
| clarkb | but maybe its doable and at that point worry about deploying per region | 21:54 |
| *** claudiub has joined #openstack-infra | 21:54 | |
| clarkb | then you just change your proxy url and the job should continue to magically work using the proxy that is shared | 21:55 |
| pabelanger | ya, that seems like a plan for job side | 21:56 |
| *** annegentle has joined #openstack-infra | 21:56 | |
| pabelanger | giving us time to dive into logistics of squid, if we need spec, and where it will live (assuming existing mirrir) | 21:56 |
| clarkb | alternatives include the 404 handler hack in web server to grab contents on the fly (that fits in more with our mirror model but without afs) | 21:57 |
| clarkb | not sure how difficult it would be to get that right for files that are larger | 21:57 |
| clarkb | I wonder if anyone has published versison of that | 21:57 |
| pabelanger | https://github.com/voxpupuli/puppet-squid | 21:58 |
| pabelanger | thanks nibalizer ^ | 21:58 |
| pabelanger | :) | 21:58 |
| nibalizer | heh | 21:58 |
| nibalizer | im not even on the contributors list | 21:59 |
| nibalizer | also ohai infra | 21:59 |
| *** markvoelker has joined #openstack-infra | 21:59 | |
| clarkb | thats going to have a higher initial setup than squid but lower than afs likely | 22:00 |
| clarkb | (the proxy pass 404 caching type deal) | 22:00 |
| clarkb | you'd basically do a /rdo_repo/ proxy to rdo.wherever.org/packages type proxy with caching | 22:01 |
| *** gouthamr has joined #openstack-infra | 22:01 | |
| clarkb | the difference there is rather than have open proxy on internet (another thing to consider with squid) you'll only get to rdo | 22:01 |
| clarkb | pabelanger: ^ thats another thing to keep in mind, proxies like that could potentially be abused so you might need some way of restricting who can use them | 22:02 |
| clarkb | or restrict what they can talk to etc | 22:02 |
| bkero | clarkb: I haven't touched the OVB jobs before. I do know that TripleO CI jobs have classically had squid setups.\ | 22:02 |
| *** EricGonczer_ has quit IRC | 22:03 | |
| *** mriedem has quit IRC | 22:04 | |
| *** sslypushenko_ has quit IRC | 22:05 | |
| *** markvoelker has quit IRC | 22:06 | |
| clarkb | bkero: I seem to recal unreachable proxy at one point? | 22:06 |
| clarkb | and you thought we ran it | 22:06 |
| mtreinish | fungi: ooh, vernemq also has graphite integration builtin to report metrics | 22:06 |
| *** cody-somerville has joined #openstack-infra | 22:08 | |
| pabelanger | clarkb: right, we'd likely need ACLs in place for local network access on squid. will have to look into it more for sure | 22:09 |
| bkero | clarkb: Yes, it was a package manager | 22:09 |
| bkero | tripleo-ci/scripts/proxy-server/proxy-server.pp | 22:10 |
| clarkb | pabelanger: just be careful that doesn't quite work on all of our public clouds :) | 22:10 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Remove os-cloud-config https://review.openstack.org/451092 | 22:10 |
| clarkb | pabelanger: you could us a security group, except not all clouds security group and also group to group rules hit nova hard | 22:10 |
| bkero | export http_proxy=${http_proxy-"http://192.168.1.100:3128/"} in toci_gate_test.sh | 22:10 |
| pabelanger | bkero: yes, that is what I am trying to delete from tripleo | 22:10 |
| pabelanger | bkero: because, private infrastructure | 22:10 |
| bkero | pabelanger: toci_gate_test exports http_proxy="" | 22:10 |
| clarkb | pabelanger: so its actually tricky, you can't really use authentication because its trivail to expose the auth details on the test boxs | 22:11 |
| *** jamesdenton has joined #openstack-infra | 22:11 | |
| bkero | pabelanger: it looks like it gets set and unset quite a lot in tripleo-ci. | 22:11 |
| pabelanger | clarkb: We can likely get an etherpad going for this, hash out ideas, and see what it looks like | 22:11 |
| pabelanger | bkero: yes, more and more they don't use it | 22:12 |
| pabelanger | IIRC | 22:12 |
| clarkb | pabelanger: good idea | 22:12 |
| bkero | pabelanger: TBH I've never touched it. I've expected it to work well, and haven't seen any problems that I can directly attribute to it. Definitely had some centos mirror problems though. | 22:13 |
| bkero | That was/is a tripleo blocker from the past few days. | 22:13 |
| *** john-davidge has joined #openstack-infra | 22:13 | |
| pabelanger | bkero: ya, I've been offered access a few times. I've politely declined :) | 22:13 |
| pabelanger | bkero: ya, the buildlogs issues was an osic-cloud problem, but it still highlights the need to mirror. So, I'm trying to get the ball rolling and import more things into AFS if possible | 22:14 |
| bkero | pabelanger: so: On oooq tests the proxy is not used. http_proxy is set to "" and never overridden. Only in tripleo.sh is it used/set up. | 22:14 |
| pabelanger | gem mirrors is likely first, if we can agree to fork (for now) and what directory hashing looks like | 22:14 |
| bkero | pabelanger: dlrn mirrors in afs? | 22:15 |
| pabelanger | bkero: yes, if they enable rsync | 22:15 |
| clarkb | pabelanger: worth noting that the way we do the mirros and the way squid works you won't actually 100% fix that problem that osic had | 22:15 |
| clarkb | pabelanger: since they both rely on external connectivity to get resources that are not cached locally | 22:15 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Retire os-cloud-config https://review.openstack.org/451093 | 22:15 |
| clarkb | pabelanger: https://etherpad.openstack.org/p/caching-artifacts-in-cloud-regions | 22:15 |
| *** baoli has quit IRC | 22:16 | |
| pabelanger | clarkb: true, we might have been able to reach a little sooner? Since our AFS mirrors would have been failing with TCP connection resets | 22:16 |
| bkero | pabelanger: I would think that could be done, but then again I don't run trunk.rdoproject.org package repos. | 22:16 |
| pabelanger | clarkb: actually, did we even see issues with AFS during that? | 22:16 |
| *** jamesden_ has joined #openstack-infra | 22:16 | |
| *** jamesdenton has quit IRC | 22:17 | |
| *** mriedem has joined #openstack-infra | 22:18 | |
| *** john-davidge has quit IRC | 22:18 | |
| bkero | pabelanger: Almost all of the errors I'm seeing with TripleO jobs come down to package retrieval. I would consider that the top priority to mirror. | 22:19 |
| bkero | Or solve, if not solved by mirroring. | 22:19 |
| clarkb | pabelanger: its udp and ya we say some issues according to e-r | 22:19 |
| clarkb | s/say/saw/ | 22:19 |
| openstackgerrit | Merged openstack/python-jenkins master: Add py35 to tox envlist https://review.openstack.org/444184 | 22:19 |
| pabelanger | bkero: yes, but tripleo in some cases was not using the mirrors we provide. EmilienM is doing a good job to fix that now | 22:20 |
| *** wolverineav has quit IRC | 22:20 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Minimise database queries when resolving cards https://review.openstack.org/444615 | 22:21 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Minimise database queries when populating automatic worklists https://review.openstack.org/444644 | 22:21 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Allow stories and tasks to be filtered by boards/worklists containing them https://review.openstack.org/444614 | 22:21 |
| bkero | Yep, $NODEPOOL_MIRROR_HOST is being set for oooq jobs | 22:21 |
| bkero | for centos and epel | 22:21 |
| bkero | That's been there since 2017-02-08 at least | 22:22 |
| *** sree has joined #openstack-infra | 22:24 | |
| pabelanger | bkero: ceph mirror is the recent one | 22:25 |
| *** marst_ has quit IRC | 22:25 | |
| *** sree has quit IRC | 22:28 | |
| EmilienM | pabelanger: did you manage to get dib release? | 22:29 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test new ovb with images building https://review.openstack.org/445228 | 22:29 |
| pabelanger | EmilienM: yes, ianw did it. | 22:30 |
| *** yamamoto has joined #openstack-infra | 22:30 | |
| ianw | bkero: on a slight tangent, i'm guessing tripleo doesn't really want epel either | 22:31 |
| EmilienM | pabelanger: nice | 22:31 |
| *** xyang1 has quit IRC | 22:31 | |
| bkero | ianw: That's true. It should be able to run without EPEL IIRC. | 22:31 |
| ianw | EmilienM: yep, 2.2.0 ... i bumped despite our outstanding ppc issues | 22:31 |
| EmilienM | cool | 22:32 |
| *** jamesden_ has quit IRC | 22:32 | |
| pabelanger | Just kicked off new centos-7 DIBs too | 22:32 |
| ianw | bkero: yeah, i have changes in flight to turn epel off on images | 22:32 |
| ianw | i need to get back to them. the sticking point was haveged and randomness introduction, it doesn't seem there's actually a good solution for that | 22:33 |
| clarkb | haveged makes a huge difference too (we've noticed with things like pbr's test that use gpg to sign stuff) | 22:34 |
| bkero | ianw: I thought haveged was in base | 22:34 |
| ianw | no, not for centos/rhel. if you're in a hetrogenous environment there's kvm randomness passthrough or whatever it's called | 22:35 |
| bkero | Sure | 22:35 |
| ianw | homogeneous rather | 22:35 |
| bkero | virt-random or whatever | 22:35 |
| ianw | but we launch on xen, kvm, who knows what else :) | 22:36 |
| bkero | Hmm. I wonder if it could be installed statically in the image | 22:36 |
| bkero | That's not ideal though. | 22:36 |
| clarkb | pabelanger: based on that etherpad brain dump I think a good next step would be to mock up both the squid forward proxy and an apache caching reverse proxy in two different changes and sort through the further issues with each | 22:37 |
| *** beslemon has quit IRC | 22:37 | |
| clarkb | pabelanger: then compare against afs things which we already have ap retty good handle on | 22:37 |
| *** yamamoto has quit IRC | 22:37 | |
| ianw | bkero: my plan was to just enable epel for specific things -> https://review.openstack.org/#/c/439917/ | 22:37 |
| fungi | mtreinish: is it terrible that vernemq conjures images of jim varney for me? | 22:37 |
| JayF | fungi: LOL | 22:38 |
| JayF | fungi: Ernest goes to Cloud | 22:38 |
| clarkb | and now for something completely different. Who can we bug to look into why libvirt is crashing? | 22:39 |
| bkero | ianw: I'd be concerned about intermittent failures due to entropy pool problems | 22:39 |
| clarkb | dansmith: ^ any ideas on who might want to debug libvirt crashes? | 22:39 |
| *** tellesnobrega has joined #openstack-infra | 22:39 | |
| *** jaypipes has quit IRC | 22:39 | |
| *** rbrndt has quit IRC | 22:39 | |
| dansmith | clarkb: mdbooth or kashyap | 22:39 |
| dansmith | clarkb: but tomorrow of course | 22:39 |
| clarkb | they are both in a more easterly timezone right? | 22:39 |
| clarkb | I will try to ping them and let them know what we know | 22:40 |
| dansmith | yeah, GMT, GMT+2 I think | 22:40 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add planet01.openstack.org https://review.openstack.org/450534 | 22:40 |
| *** marst has joined #openstack-infra | 22:41 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: scm: handle "bitbucketweb" browser for mercurial https://review.openstack.org/449692 | 22:41 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add planet01.openstack.org https://review.openstack.org/450534 | 22:41 |
| pabelanger | clarkb: ack | 22:41 |
| ianw | clarkb: do we have core dumps? | 22:42 |
| clarkb | ianw: I don't think so, just stacktraces | 22:42 |
| clarkb | ianw: so likely step zero is enable core dumps on the hosts then grab them with devstack-gate | 22:42 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added RSVP delete endpoint https://review.openstack.org/451099 | 22:43 |
| clarkb | pretty sure they are disabled by default | 22:43 |
| ianw | clarkb: yeah, that's what i'm thinking, devstack should have an option to do that | 22:43 |
| ianw | where should mean "does not but should" | 22:43 |
| clarkb | in addition to that it appears taht rax and not rax are affected so don't think its a xen or not xen thing. Also only xenial appears to be affected | 22:43 |
| fungi | we did see plenty of libvirt-related mischief when enabling uca on trusty, so not entirely surprised by this | 22:44 |
| *** signed8b_ is now known as signed8bit_Zzz | 22:44 | |
| *** ihrachys has joined #openstack-infra | 22:45 | |
| *** gordc has quit IRC | 22:46 | |
| clarkb | ianw: apparently core dump not expected to be helpful https://bugs.launchpad.net/nova/+bug/1643911/comments/11 | 22:47 |
| openstack | Launchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] | 22:47 |
| fungi | whee! that's some fun times right there | 22:48 |
| ianw | oh ... yeah. i guess we should just start porting it to rust then | 22:49 |
| *** vhosakot has quit IRC | 22:49 | |
| mtreinish | fungi: haha | 22:50 |
| *** david-lyle has quit IRC | 22:51 | |
| *** thorst has joined #openstack-infra | 22:51 | |
| *** Goneri has quit IRC | 22:52 | |
| mtreinish | fungi: it was actually my second choice, I tried emqttd first. I had a bunch of issues with it though like I couldn't get anonymous read only working, and it didn't like being shutdown (it kept running multiple processes after systemd said it shutdown) | 22:52 |
| *** yamamoto has joined #openstack-infra | 22:53 | |
| *** ihrachys has quit IRC | 22:55 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix constructor arguments to source https://review.openstack.org/451102 | 22:55 |
| *** thorst has quit IRC | 22:55 | |
| *** annegentle has quit IRC | 22:55 | |
| *** yamamoto has quit IRC | 22:56 | |
| *** zz_dimtruck is now known as dimtruck | 22:56 | |
| *** markvoelker has joined #openstack-infra | 23:02 | |
| *** yamamoto has joined #openstack-infra | 23:03 | |
| clarkb | Gerrit CI also suffering from OOMing :) | 23:06 |
| clarkb | we should have a club | 23:06 |
| *** markvoelker has quit IRC | 23:06 | |
| ianw | clarkb: so yeah, i added a comment. let me see about getting cores, and also i think we can use some much lighter weight malloc debug tools like electric fence to start | 23:07 |
| *** Goneri has joined #openstack-infra | 23:07 | |
| ianw | if anyone wants to take a quick look at 450534 that would be good too ... puppet for the new planet server, fairly trivial i hope | 23:08 |
| *** thorst has joined #openstack-infra | 23:08 | |
| *** kfox1111 has joined #openstack-infra | 23:08 | |
| kfox1111 | can I get a few eyeballs please: https://review.openstack.org/#/c/450471/ ? | 23:08 |
| *** dizquierdo has quit IRC | 23:09 | |
| clarkb | ianw: commented | 23:11 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add planet01.openstack.org https://review.openstack.org/450534 | 23:13 |
| clarkb | heh google electric fence and get back lowes urls | 23:13 |
| *** xarses has quit IRC | 23:13 | |
| *** xarses_ has quit IRC | 23:13 | |
| clarkb | ianw: ooh looks like you can just LD_PRELOAD electric fence | 23:14 |
| clarkb | ianw: is that something you can just enable across the baord on ubuntu in devstack temporarily (so that we get enough data?) | 23:14 |
| *** adrian_otto has joined #openstack-infra | 23:14 | |
| clarkb | oh you still need to collect core dumps though | 23:15 |
| *** sdague has quit IRC | 23:15 | |
| ianw | clarkb: yep ... possibly. all it really does is put a no-permissions page after malloc()s to cause a segfault | 23:15 |
| ianw | so it doesn't catch everything, but it's worth a try | 23:15 |
| kfox1111 | clarkb: thanks. :) | 23:15 |
| ianw | yeah, coredumps we will want anyway ... looking at best ways to do that now | 23:15 |
| clarkb | ianw: also there is info in there that it appears to happen in a subset of tempest jobs and always on the vm deletion | 23:16 |
| ianw | daemon_corefile_limit ? | 23:16 |
| clarkb | ianw: so possible that we could rig up a test that is just boot and delete lots of VMs until segfault on xenial | 23:16 |
| clarkb | ianw: is that something that we can just magically set with systemd? | 23:16 |
| ianw | i'm not sure, it's a new one for me. googling :) | 23:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 23:17 |
| ianw | is libvirt setuid? that seems to add another layer of complexity | 23:18 |
| clarkb | ianw: LimitCORE= in the unit file apparently | 23:18 |
| clarkb | devstack could probably just set that into the unit file, reload config, and restart the service | 23:19 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepoold to nodepool-launcher https://review.openstack.org/450877 | 23:19 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Force os-client-config to use ipv4 https://review.openstack.org/450983 | 23:19 |
| clarkb | not sure if LimitCORE=unlimited will work | 23:20 |
| ianw | do we just enable systemd-coredump maybe? | 23:21 |
| *** xarses_ has joined #openstack-infra | 23:22 | |
| clarkb | it looks like you may still have to set LimitCORE with that | 23:23 |
| *** yamamoto has quit IRC | 23:24 | |
| *** thorst has quit IRC | 23:25 | |
| pabelanger | clarkb: ianw: EmilienM: centos-7 DIBs uploading now | 23:25 |
| *** thorst has joined #openstack-infra | 23:25 | |
| EmilienM | pabelanger: ack | 23:25 |
| clarkb | actually coredump.conf may configure the limits itself | 23:26 |
| clarkb | so mayn things | 23:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add canonical hostname to source object https://review.openstack.org/451110 | 23:26 |
| clarkb | ianw: you have the option to store external (eg on disk) or in the journal too. Probably best to do external ? so we don't fill journal up with libvirt cores | 23:27 |
| ianw | clarkb: yeah, we need them copied, we just dump the messages from journal | 23:27 |
| *** mriedem has quit IRC | 23:28 | |
| clarkb | ianw: also the reason our pupept testing didn't catch your regex thing is we comment that out in testing so that puppet will evaluate the manifests on a host with any name | 23:28 |
| clarkb | ianw: so thats a limitation of how we apply puppet site.pp contents | 23:29 |
| *** thorst has quit IRC | 23:29 | |
| clarkb | I remember when it was easy to get core dumps, I just wrote bad code | 23:29 |
| *** dave-mccowan has quit IRC | 23:31 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added RSVP delete endpoint https://review.openstack.org/451099 | 23:32 |
| mnaser | mordred does shade have anything to cache requests | 23:33 |
| mnaser | ooooor force it to be told "this is an id, don't do a list to lookup" | 23:33 |
| ianw | fun | 23:34 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Use libyaml if possible https://review.openstack.org/451113 | 23:36 |
| mordred | mnaser: it can | 23:37 |
| mnaser | mordred trying to speed this up - http://paste.openstack.org/show/604587/ | 23:37 |
| mordred | mnaser: it supports configurable caching ... | 23:37 |
| mnaser | for everyone it seems to run a UserList, ProjectList, RoleAssignmentList .. | 23:38 |
| mnaser | let me see | 23:38 |
| mnaser | i searched the codebase on github quickly without anything | 23:38 |
| mordred | mnaser: these are my cache settings from my clouds.yaml file: http://paste.openstack.org/show/604588/ | 23:39 |
| *** EricGonczer_ has joined #openstack-infra | 23:39 | |
| mnaser | ooo interesting | 23:39 |
| mordred | mnaser: it's possible we don't have the resources you're talking about there annotated yet | 23:39 |
| mnaser | no worries | 23:39 |
| mnaser | i was just wondering if there was a super straightforward quick way.. or we could just wait longer | 23:40 |
| mnaser | or openstack projects could start using consistent role names so we dont have to give everyone $role when we add a new service *shrug* | 23:40 |
| *** Sihan has quit IRC | 23:40 | |
| mordred | right? | 23:40 |
| mordred | mnaser: https://docs.openstack.org/developer/os-client-config/#cache-settings has a little bit of docs | 23:40 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config master: Setup known_hosts file on gerrit servers https://review.openstack.org/399822 | 23:41 |
| mnaser | mordred fyi we shipped out barbican today if you want to have some fun with that | 23:41 |
| mordred | it's easy to add resources into the caching infrastructure if they're not though | 23:41 |
| mordred | woot! | 23:41 |
| mnaser | and magnum should be going up tomorrow-ish | 23:41 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Remove tripleo-test-cloud-rh2 from nodepool https://review.openstack.org/451115 | 23:41 |
| mordred | mnaser: I'll take a look at the caching layer for keystone resource in the morning | 23:41 |
| mnaser | mordred i think what would be nicer if it we can force it to accept the name_or_id as id | 23:42 |
| mnaser | preventing extra lookups | 23:42 |
| *** gyee has quit IRC | 23:43 | |
| *** erlon has joined #openstack-infra | 23:44 | |
| *** Julien-zte has quit IRC | 23:44 | |
| ianw | ffs, all i want is a quick xenial vm for testing this dump stuff ... now my testing cloud has managed to associate the same floating ip to two vm's, handily breaking both | 23:45 |
| *** xarses_ has quit IRC | 23:46 | |
| *** Swami has quit IRC | 23:49 | |
| clarkb | ianw: thats a neat bug is it openstack? | 23:51 |
| ianw | yep. i just restarted everything and we're ok | 23:52 |
| ianw | so libvirt-bin on xenial isn't systemd | 23:52 |
| clarkb | sigh | 23:53 |
| kfox1111 | can I get a second for https://review.openstack.org/#/c/450471/ please? | 23:53 |
| openstackgerrit | Tim Burke proposed openstack-infra/project-config master: Add release notes jobs for swift https://review.openstack.org/451118 | 23:55 |
| *** ym200 has joined #openstack-infra | 23:55 | |
| *** ym200 has quit IRC | 23:55 | |
| *** thorst has joined #openstack-infra | 23:56 | |
| *** ym200 has joined #openstack-infra | 23:57 | |
| *** dingyichen has joined #openstack-infra | 23:58 | |
| ianw | anyone have a xenial ci host open? do we have this "apport" thing installed? | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
