| *** thorst has quit IRC | 00:01 | |
| *** EricGonczer_ has quit IRC | 00:01 | |
| *** EricGonczer_ has joined #openstack-infra | 00:02 | |
| *** amotoki has joined #openstack-infra | 00:05 | |
| *** masahito has joined #openstack-infra | 00:07 | |
| *** Apoorva_ has quit IRC | 00:07 | |
| *** rwsu has quit IRC | 00:08 | |
| *** Apoorva has joined #openstack-infra | 00:08 | |
| *** EricGonczer_ has quit IRC | 00:10 | |
| *** amotoki has quit IRC | 00:10 | |
| *** signed8b_ is now known as signed8bit_Zzz | 00:11 | |
| *** tphummel has quit IRC | 00:12 | |
| openstackgerrit | Merged openstack-infra/shade master: Add bare parameter to get/list/search server https://review.openstack.org/451264 | 00:13 |
|---|---|---|
| *** jmorgan1_ has joined #openstack-infra | 00:15 | |
| *** jmorgan1_ has quit IRC | 00:15 | |
| *** jmorgan1_ has joined #openstack-infra | 00:16 | |
| *** jmorgan1 has quit IRC | 00:16 | |
| *** aNuposic has quit IRC | 00:16 | |
| masahito | hi infra team | 00:17 |
| masahito | I want to add branch creation access rights to blazar-release team in gerrit and added "create" key in project-config/gerrit/acls/blazar.config. | 00:17 |
| masahito | but it doesn't work well. I can't see create branch button in gerrit UI. | 00:18 |
| clarkb | masahito: did a change get merged to project-config to update the ACLs? | 00:18 |
| clarkb | masahito: link to the chang ewould likely be helpful | 00:19 |
| masahito | clarkb: yes. https://github.com/openstack-infra/project-config/blob/master/gerrit/acls/openstack/blazar.config#L3 | 00:19 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 00:19 |
| *** aNuposic has joined #openstack-infra | 00:19 | |
| clarkb | masahito: https://review.openstack.org/#/admin/projects/openstack/blazar,access it hasn't applied yet | 00:20 |
| clarkb | mordred: ^ that is likely fallout of the jeepyb change as well | 00:20 |
| masahito | how do I apply the change to gerrit? | 00:21 |
| clarkb | masahito: there have been changes to jeepyb recently to do more caching of information to speed up updates, but there have been a few problems. My guess is this is related to those jeepyb updates | 00:21 |
| clarkb | masahito: mordred has been debugging most of them as he was author of the jeepyb changes. I have to run soon, but can likely take a look tomorrow if no one else beats me to it | 00:21 |
| clarkb | masahito: maybe send email to the openstack-infra mailing list so that we don't forget | 00:21 |
| *** armax has quit IRC | 00:23 | |
| *** dizquierdo has quit IRC | 00:23 | |
| masahito | clarkb: ok. after the the change will be merged or debbuged, the acl config automatically is applied, right? | 00:23 |
| clarkb | masahito: yes | 00:24 |
| clarkb | masahito: this is a problem in the automation that applies the config | 00:25 |
| clarkb | so once we fix that it should get applied | 00:25 |
| masahito | clarkb: do you have the link to the change? I want to add the link as a remainder. | 00:26 |
| masahito | clarkb: anyway, thanks for the help! | 00:28 |
| *** EricGonczer_ has joined #openstack-infra | 00:28 | |
| *** mtanino_ has joined #openstack-infra | 00:29 | |
| *** xarses has joined #openstack-infra | 00:29 | |
| *** Julien-zte has joined #openstack-infra | 00:29 | |
| *** jcoufal has joined #openstack-infra | 00:29 | |
| *** mtanino has quit IRC | 00:30 | |
| *** armax has joined #openstack-infra | 00:31 | |
| *** mriedem has joined #openstack-infra | 00:32 | |
| *** erikmwilson has quit IRC | 00:33 | |
| *** thorst has joined #openstack-infra | 00:36 | |
| *** erikmwilson has joined #openstack-infra | 00:38 | |
| *** armax has quit IRC | 00:39 | |
| *** yamamoto has joined #openstack-infra | 00:41 | |
| *** rossella_s has quit IRC | 00:42 | |
| *** rossella_s has joined #openstack-infra | 00:43 | |
| *** jkilpatr has quit IRC | 00:44 | |
| *** Julien-zte has quit IRC | 00:45 | |
| *** david-lyle_ has joined #openstack-infra | 00:45 | |
| *** gcb has quit IRC | 00:46 | |
| *** yamamoto has quit IRC | 00:46 | |
| *** aNuposic has quit IRC | 00:47 | |
| *** david-lyle has quit IRC | 00:48 | |
| *** jcoufal has quit IRC | 00:49 | |
| *** aNuposic has joined #openstack-infra | 00:49 | |
| *** mriedem has quit IRC | 00:49 | |
| *** john-davidge has joined #openstack-infra | 00:52 | |
| *** thorst has quit IRC | 00:53 | |
| *** xarses has quit IRC | 00:55 | |
| rajinir | clarkb: you were right, we were in the process of upgrading to a new hardware and didn't turn off the old one , so that nodepool must have been sending those rough delete signals. Turning that off worked | 00:55 |
| *** jkilpatr has joined #openstack-infra | 00:56 | |
| *** john-davidge has quit IRC | 00:57 | |
| *** jcoufal has joined #openstack-infra | 00:57 | |
| *** sihan has quit IRC | 01:00 | |
| *** jcoufal has quit IRC | 01:00 | |
| *** jkilpatr has quit IRC | 01:01 | |
| *** Goneri has quit IRC | 01:03 | |
| *** amotoki has joined #openstack-infra | 01:05 | |
| *** jkilpatr has joined #openstack-infra | 01:06 | |
| *** pedh has quit IRC | 01:10 | |
| *** amotoki has quit IRC | 01:10 | |
| *** gyee has quit IRC | 01:11 | |
| *** jkilpatr has quit IRC | 01:16 | |
| *** Apoorva has quit IRC | 01:16 | |
| *** jkilpatr has joined #openstack-infra | 01:17 | |
| *** liujiong has joined #openstack-infra | 01:18 | |
| openstackgerrit | mark mcclain proposed openstack-infra/project-config master: Enable Python3.5 gating for networking-arista https://review.openstack.org/451572 | 01:19 |
| *** luzC has quit IRC | 01:20 | |
| *** jlwhite has quit IRC | 01:20 | |
| *** castulo has quit IRC | 01:21 | |
| *** lcastell has quit IRC | 01:21 | |
| *** lcastell has joined #openstack-infra | 01:24 | |
| *** luzC has joined #openstack-infra | 01:25 | |
| *** jlwhite has joined #openstack-infra | 01:25 | |
| *** castulo has joined #openstack-infra | 01:26 | |
| *** mtanino_ has quit IRC | 01:28 | |
| pabelanger | clarkb: fungi: did we figure out the caching review proxy change? | 01:30 |
| *** dingyichen has quit IRC | 01:31 | |
| *** david-lyle__ has joined #openstack-infra | 01:34 | |
| *** Julien-zte has joined #openstack-infra | 01:34 | |
| *** yanyanhu has joined #openstack-infra | 01:34 | |
| *** tuanluong has joined #openstack-infra | 01:35 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 01:36 |
| *** david-lyle_ has quit IRC | 01:37 | |
| *** ihrachys has joined #openstack-infra | 01:37 | |
| *** yamamoto has joined #openstack-infra | 01:42 | |
| *** aysyd has quit IRC | 01:43 | |
| *** ihrachys has quit IRC | 01:44 | |
| *** neiljerram has quit IRC | 01:45 | |
| *** yamamoto has quit IRC | 01:46 | |
| *** chlong has joined #openstack-infra | 01:47 | |
| *** mdrabe has joined #openstack-infra | 01:48 | |
| *** oomichi has quit IRC | 01:48 | |
| *** isq has quit IRC | 01:50 | |
| *** thorst has joined #openstack-infra | 01:51 | |
| *** oomichi has joined #openstack-infra | 01:52 | |
| *** isq has joined #openstack-infra | 01:52 | |
| *** hongbin has quit IRC | 01:52 | |
| *** gcb has joined #openstack-infra | 01:53 | |
| *** mdrabe has quit IRC | 01:57 | |
| *** thorst has quit IRC | 02:04 | |
| *** amotoki has joined #openstack-infra | 02:06 | |
| *** amotoki has quit IRC | 02:11 | |
| *** jcoufal has joined #openstack-infra | 02:18 | |
| *** askb has quit IRC | 02:18 | |
| *** jkilpatr has quit IRC | 02:19 | |
| *** jkilpatr has joined #openstack-infra | 02:19 | |
| *** knangia has quit IRC | 02:21 | |
| *** yuanying has quit IRC | 02:22 | |
| *** pedh has joined #openstack-infra | 02:26 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/tripleo-ci master: Replace "ara generate" by "ara generate html", move report to ara_oooq https://review.openstack.org/448108 | 02:27 |
| *** pedh has quit IRC | 02:31 | |
| *** isq has quit IRC | 02:31 | |
| *** thorst has joined #openstack-infra | 02:31 | |
| *** jmorgan1 has joined #openstack-infra | 02:31 | |
| *** adrian_otto has joined #openstack-infra | 02:32 | |
| *** jmorgan1_ has quit IRC | 02:32 | |
| *** masahito has quit IRC | 02:32 | |
| *** isq has joined #openstack-infra | 02:32 | |
| *** knangia has joined #openstack-infra | 02:35 | |
| *** jkilpatr has quit IRC | 02:36 | |
| *** thorst has quit IRC | 02:38 | |
| *** mat128 has joined #openstack-infra | 02:40 | |
| *** jkilpatr has joined #openstack-infra | 02:41 | |
| *** mat128 has quit IRC | 02:41 | |
| *** yamamoto has joined #openstack-infra | 02:43 | |
| *** Rockyg has quit IRC | 02:43 | |
| openstackgerrit | chenhb proposed openstack-infra/project-config master: [rally] Add support for custom branching https://review.openstack.org/450337 | 02:43 |
| *** mat128 has joined #openstack-infra | 02:44 | |
| *** yamamoto has quit IRC | 02:49 | |
| *** mat128 has quit IRC | 02:50 | |
| *** adrian_otto has quit IRC | 02:53 | |
| *** john-davidge has joined #openstack-infra | 02:53 | |
| *** adrian_otto has joined #openstack-infra | 02:53 | |
| *** john-davidge has quit IRC | 02:58 | |
| *** pedh has joined #openstack-infra | 02:58 | |
| *** adrian_otto has quit IRC | 03:03 | |
| *** adrian_otto has joined #openstack-infra | 03:03 | |
| *** bmjen has quit IRC | 03:04 | |
| *** adrian_otto has quit IRC | 03:04 | |
| *** cmurphy has quit IRC | 03:06 | |
| *** erikmwil_ has joined #openstack-infra | 03:06 | |
| *** erikmwilson has quit IRC | 03:06 | |
| *** pedh has quit IRC | 03:07 | |
| *** amotoki has joined #openstack-infra | 03:07 | |
| *** dharinic has joined #openstack-infra | 03:08 | |
| *** oomichi has quit IRC | 03:08 | |
| *** bmjen has joined #openstack-infra | 03:10 | |
| *** amotoki has quit IRC | 03:12 | |
| *** oomichi has joined #openstack-infra | 03:12 | |
| *** ihrachys has joined #openstack-infra | 03:15 | |
| *** yamahata has quit IRC | 03:19 | |
| *** pedh has joined #openstack-infra | 03:22 | |
| *** EricGonczer_ has quit IRC | 03:25 | |
| *** dave-mccowan has quit IRC | 03:27 | |
| *** pedh has quit IRC | 03:28 | |
| *** cmurphy has joined #openstack-infra | 03:29 | |
| *** thorst has joined #openstack-infra | 03:36 | |
| *** ihrachys has quit IRC | 03:38 | |
| *** Julien-zte has quit IRC | 03:43 | |
| *** yamamoto has joined #openstack-infra | 03:45 | |
| *** Julien-zte has joined #openstack-infra | 03:46 | |
| *** adrian_otto has joined #openstack-infra | 03:48 | |
| *** liusheng has quit IRC | 03:49 | |
| *** liusheng has joined #openstack-infra | 03:50 | |
| *** yamamoto has quit IRC | 03:50 | |
| *** jerryz has quit IRC | 03:50 | |
| *** gouthamr has quit IRC | 03:55 | |
| *** dhajare has joined #openstack-infra | 03:55 | |
| *** thorst has quit IRC | 03:55 | |
| *** dimtruck is now known as zz_dimtruck | 03:55 | |
| *** dingyichen has joined #openstack-infra | 03:57 | |
| *** nmathew has joined #openstack-infra | 04:11 | |
| *** psachin has joined #openstack-infra | 04:11 | |
| *** arxcruz has quit IRC | 04:13 | |
| *** arxcruz has joined #openstack-infra | 04:14 | |
| *** yuanying has joined #openstack-infra | 04:19 | |
| *** adrian_otto has quit IRC | 04:20 | |
| *** links has joined #openstack-infra | 04:26 | |
| *** adrian_otto has joined #openstack-infra | 04:27 | |
| *** dhajare has quit IRC | 04:28 | |
| *** gongysh has joined #openstack-infra | 04:33 | |
| *** niska has quit IRC | 04:36 | |
| *** adrian_otto has quit IRC | 04:37 | |
| *** niska has joined #openstack-infra | 04:41 | |
| *** ianw has quit IRC | 04:43 | |
| *** ianw has joined #openstack-infra | 04:43 | |
| *** adam_g has quit IRC | 04:44 | |
| *** adam_g has joined #openstack-infra | 04:45 | |
| *** Apoorva has joined #openstack-infra | 04:45 | |
| *** yamamoto has joined #openstack-infra | 04:47 | |
| *** yamamoto has quit IRC | 04:50 | |
| *** yamamoto has joined #openstack-infra | 04:50 | |
| *** aeng_ has quit IRC | 04:52 | |
| *** john-davidge has joined #openstack-infra | 04:54 | |
| *** Apoorva has quit IRC | 04:56 | |
| *** dhajare has joined #openstack-infra | 04:56 | |
| *** Apoorva has joined #openstack-infra | 04:57 | |
| *** john-davidge has quit IRC | 04:59 | |
| *** gongysh has quit IRC | 05:01 | |
| *** Apoorva has quit IRC | 05:05 | |
| *** oomichi has quit IRC | 05:09 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove os-cloud-config https://review.openstack.org/451092 | 05:11 |
| *** liusheng has quit IRC | 05:11 | |
| *** oomichi has joined #openstack-infra | 05:12 | |
| *** rkukura_ has joined #openstack-infra | 05:12 | |
| *** dhellmann has quit IRC | 05:13 | |
| *** liusheng has joined #openstack-infra | 05:13 | |
| *** stevebaker has quit IRC | 05:14 | |
| *** dalvarez has quit IRC | 05:14 | |
| *** dmellado has quit IRC | 05:14 | |
| *** wfoster has quit IRC | 05:14 | |
| *** mdbooth has quit IRC | 05:14 | |
| *** chlong has quit IRC | 05:14 | |
| *** rkukura has quit IRC | 05:14 | |
| *** radez has quit IRC | 05:14 | |
| *** pabelanger has quit IRC | 05:14 | |
| *** rkukura_ is now known as rkukura | 05:14 | |
| *** dmanchad has quit IRC | 05:15 | |
| *** ansiwen has quit IRC | 05:15 | |
| *** njohnston_ has joined #openstack-infra | 05:15 | |
| *** mrunge- has joined #openstack-infra | 05:15 | |
| *** pots has quit IRC | 05:15 | |
| *** kashyap` has joined #openstack-infra | 05:15 | |
| *** ansiwen has joined #openstack-infra | 05:15 | |
| *** wfoster has joined #openstack-infra | 05:15 | |
| *** mdbooth has joined #openstack-infra | 05:15 | |
| *** mrunge has quit IRC | 05:15 | |
| *** njohnston has quit IRC | 05:15 | |
| *** kashyap has quit IRC | 05:15 | |
| *** dmanchad has joined #openstack-infra | 05:16 | |
| *** dhellmann has joined #openstack-infra | 05:16 | |
| *** dmellado has joined #openstack-infra | 05:16 | |
| *** claudiub has joined #openstack-infra | 05:17 | |
| *** stevebaker has joined #openstack-infra | 05:18 | |
| *** oomichi has quit IRC | 05:18 | |
| *** vikrant has joined #openstack-infra | 05:19 | |
| *** pabelanger has joined #openstack-infra | 05:20 | |
| *** jcoufal has quit IRC | 05:20 | |
| *** dalvarez has joined #openstack-infra | 05:20 | |
| *** nadya has joined #openstack-infra | 05:22 | |
| *** oomichi has joined #openstack-infra | 05:23 | |
| *** nadya has quit IRC | 05:23 | |
| *** ccamacho has quit IRC | 05:27 | |
| *** pgadiya has joined #openstack-infra | 05:28 | |
| *** nadya has joined #openstack-infra | 05:28 | |
| *** nadya has quit IRC | 05:28 | |
| openstackgerrit | Dirk Mueller proposed openstack-dev/pbr master: Add sphinx built manpages if they exist https://review.openstack.org/184899 | 05:29 |
| *** yamamoto has quit IRC | 05:30 | |
| *** jasondotstar has quit IRC | 05:31 | |
| *** yamahata has joined #openstack-infra | 05:32 | |
| *** kornica is now known as kornicameister | 05:34 | |
| *** jasondotstar has joined #openstack-infra | 05:35 | |
| *** florianf has joined #openstack-infra | 05:36 | |
| openstackgerrit | fumihiko kakuma proposed openstack-infra/devstack-gate master: Add commands listed in DEVSTACK_GATE_ADD_TEMPEST_SUDO to tempest sudoers https://review.openstack.org/355682 | 05:37 |
| *** yamamoto has joined #openstack-infra | 05:41 | |
| *** yamamoto has quit IRC | 05:41 | |
| *** rcernin has joined #openstack-infra | 05:45 | |
| *** radez has joined #openstack-infra | 05:48 | |
| *** thorst has joined #openstack-infra | 05:52 | |
| *** sree has joined #openstack-infra | 05:53 | |
| *** thorst has quit IRC | 05:57 | |
| *** hichihara has joined #openstack-infra | 05:58 | |
| *** gongysh has joined #openstack-infra | 06:04 | |
| *** rbuzatu has quit IRC | 06:08 | |
| *** rbuzatu has joined #openstack-infra | 06:08 | |
| *** zhurong has joined #openstack-infra | 06:15 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add requirements gate jobs to shade and requestsexceptions https://review.openstack.org/451301 | 06:18 |
| *** saibarspeis has joined #openstack-infra | 06:20 | |
| *** dharinic has quit IRC | 06:21 | |
| *** flepied has joined #openstack-infra | 06:23 | |
| *** isaacb has joined #openstack-infra | 06:25 | |
| *** rwsu has joined #openstack-infra | 06:27 | |
| *** eventingmonkey has quit IRC | 06:30 | |
| *** saibarspeis has quit IRC | 06:31 | |
| *** eventingmonkey has joined #openstack-infra | 06:32 | |
| *** gongysh has quit IRC | 06:36 | |
| *** martinkopec has joined #openstack-infra | 06:36 | |
| openstackgerrit | Dinesh Bhor proposed openstack-infra/project-config master: Use job-template check-requirements in masakari projects https://review.openstack.org/451668 | 06:37 |
| *** zhurong has quit IRC | 06:38 | |
| *** prateek has joined #openstack-infra | 06:38 | |
| openstackgerrit | Merged openstack-infra/shade master: Change metadata to align with team affiliation https://review.openstack.org/450981 | 06:38 |
| *** yamamoto has joined #openstack-infra | 06:42 | |
| *** yamamoto has quit IRC | 06:47 | |
| *** pcaruana has joined #openstack-infra | 06:47 | |
| *** amotoki has joined #openstack-infra | 06:51 | |
| *** thorst has joined #openstack-infra | 06:53 | |
| *** john-davidge has joined #openstack-infra | 06:54 | |
| *** thorst has quit IRC | 06:57 | |
| *** d0ugal has joined #openstack-infra | 06:58 | |
| *** d0ugal has quit IRC | 06:58 | |
| *** d0ugal has joined #openstack-infra | 06:58 | |
| *** john-davidge has quit IRC | 06:59 | |
| *** matrohon has joined #openstack-infra | 07:02 | |
| *** tuanluong_ has joined #openstack-infra | 07:06 | |
| *** oanson has joined #openstack-infra | 07:06 | |
| *** Dinesh_Bhor has quit IRC | 07:07 | |
| *** Dinesh_Bhor has joined #openstack-infra | 07:07 | |
| *** lihi has joined #openstack-infra | 07:07 | |
| *** tuanluong has quit IRC | 07:07 | |
| *** matrohon has quit IRC | 07:07 | |
| *** dineshbhor has joined #openstack-infra | 07:09 | |
| *** dineshbhor has quit IRC | 07:09 | |
| *** shardy has joined #openstack-infra | 07:14 | |
| *** lindsayk has quit IRC | 07:16 | |
| *** tesseract has joined #openstack-infra | 07:16 | |
| *** mariojv has quit IRC | 07:17 | |
| *** kukacz has quit IRC | 07:17 | |
| *** mariojv has joined #openstack-infra | 07:18 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add initial gate for LOCI https://review.openstack.org/450473 | 07:18 |
| *** oanson has quit IRC | 07:18 | |
| *** kukacz has joined #openstack-infra | 07:18 | |
| *** aviau has quit IRC | 07:19 | |
| *** oanson has joined #openstack-infra | 07:19 | |
| *** aviau has joined #openstack-infra | 07:20 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Add upper constraints limitation of buildimage jobs https://review.openstack.org/451259 | 07:22 |
| *** Dinesh_Bhor has quit IRC | 07:24 | |
| *** Dinesh_Bhor has joined #openstack-infra | 07:25 | |
| *** ssbarnea has joined #openstack-infra | 07:26 | |
| *** Dinesh_Bhor has quit IRC | 07:30 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add Barbican grenade gate https://review.openstack.org/447689 | 07:32 |
| *** Dinesh_Bhor has joined #openstack-infra | 07:32 | |
| *** chem_gone has joined #openstack-infra | 07:35 | |
| *** john-davidge has joined #openstack-infra | 07:35 | |
| *** chem_gone is now known as chem | 07:35 | |
| *** chem has quit IRC | 07:36 | |
| *** chem has joined #openstack-infra | 07:36 | |
| openstackgerrit | Merged openstack-infra/project-config master: Test pyeclib against libec master https://review.openstack.org/449774 | 07:36 |
| openstackgerrit | Merged openstack-infra/project-config master: Add placement api ref jenkins job https://review.openstack.org/449557 | 07:37 |
| openstackgerrit | Merged openstack-infra/project-config master: Add accessbot to #openstack-forum https://review.openstack.org/451442 | 07:38 |
| *** zhurong has joined #openstack-infra | 07:38 | |
| *** ralonsoh has joined #openstack-infra | 07:39 | |
| *** Dinesh_Bhor has quit IRC | 07:40 | |
| *** alexchadin has joined #openstack-infra | 07:40 | |
| openstackgerrit | Merged openstack-infra/project-config master: Fix $PROJECTS for Cinder LIO job https://review.openstack.org/451443 | 07:42 |
| *** hichihara has quit IRC | 07:42 | |
| *** Dinesh_Bhor has joined #openstack-infra | 07:43 | |
| *** lindsayk has joined #openstack-infra | 07:43 | |
| *** yamamoto has joined #openstack-infra | 07:43 | |
| *** aarefiev_afk is now known as aarefiev | 07:46 | |
| openstackgerrit | Merged openstack-infra/project-config master: osc-placement: an OSC plugin for Placement service https://review.openstack.org/448115 | 07:46 |
| *** yamamoto has quit IRC | 07:47 | |
| *** jpich has joined #openstack-infra | 07:48 | |
| *** knangia has quit IRC | 07:51 | |
| *** markus_z has joined #openstack-infra | 07:51 | |
| *** thorst has joined #openstack-infra | 07:54 | |
| *** amoralej|off is now known as amoralej | 07:58 | |
| *** zzzeek has quit IRC | 08:00 | |
| d0ugal | dims: ping | 08:01 |
| d0ugal | What does upper-constraints-xfails.txt mean? | 08:01 |
| *** zzzeek has joined #openstack-infra | 08:02 | |
| *** saibarspeis has joined #openstack-infra | 08:03 | |
| *** openstackgerrit has quit IRC | 08:03 | |
| *** rbuzatu has quit IRC | 08:04 | |
| *** hashar has joined #openstack-infra | 08:04 | |
| *** rbuzatu has joined #openstack-infra | 08:05 | |
| *** rbuzatu_ has joined #openstack-infra | 08:07 | |
| *** rbuzatu__ has joined #openstack-infra | 08:08 | |
| *** rbuzat___ has joined #openstack-infra | 08:09 | |
| *** rbuzatu has quit IRC | 08:10 | |
| *** rbuzatu has joined #openstack-infra | 08:10 | |
| *** rbuzatu_ has quit IRC | 08:12 | |
| *** rbuzatu_ has joined #openstack-infra | 08:12 | |
| *** thorst has quit IRC | 08:12 | |
| *** rbuzatu__ has quit IRC | 08:13 | |
| *** rbuzatu__ has joined #openstack-infra | 08:13 | |
| *** rbuzat___ has quit IRC | 08:13 | |
| *** nadya has joined #openstack-infra | 08:14 | |
| *** rbuzat___ has joined #openstack-infra | 08:14 | |
| *** rbuzatu has quit IRC | 08:15 | |
| *** neiljerram has joined #openstack-infra | 08:15 | |
| *** rbuzatu has joined #openstack-infra | 08:15 | |
| *** rbuzatu_ has quit IRC | 08:16 | |
| *** rbuzatu_ has joined #openstack-infra | 08:17 | |
| *** rbuzatu__ has quit IRC | 08:17 | |
| *** rbuzatu__ has joined #openstack-infra | 08:18 | |
| *** rbuzat___ has quit IRC | 08:19 | |
| *** rbuzat___ has joined #openstack-infra | 08:20 | |
| *** ccamacho has joined #openstack-infra | 08:20 | |
| *** rbuzatu has quit IRC | 08:20 | |
| *** rbuzatu has joined #openstack-infra | 08:21 | |
| *** rbuzatu_ has quit IRC | 08:21 | |
| *** openstackgerrit has joined #openstack-infra | 08:22 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing TLS everywhere for libvirt https://review.openstack.org/450729 | 08:22 |
| *** rbuzatu_ has joined #openstack-infra | 08:22 | |
| *** rbuzatu__ has quit IRC | 08:23 | |
| *** rbuzat___ has quit IRC | 08:24 | |
| *** andreas_s has joined #openstack-infra | 08:25 | |
| *** rbuzatu has quit IRC | 08:25 | |
| *** dizquierdo has joined #openstack-infra | 08:25 | |
| *** rbuzatu has joined #openstack-infra | 08:26 | |
| *** yamamoto has joined #openstack-infra | 08:26 | |
| *** rbuzatu_ has quit IRC | 08:26 | |
| *** lucas-afk is now known as lucasagomes | 08:27 | |
| *** gongysh has joined #openstack-infra | 08:27 | |
| *** rbuzatu has quit IRC | 08:31 | |
| *** yuanying has quit IRC | 08:33 | |
| *** neiljerram has quit IRC | 08:33 | |
| *** derekh has joined #openstack-infra | 08:34 | |
| *** kashyap` is now known as kashyap | 08:35 | |
| *** tuanluong has joined #openstack-infra | 08:37 | |
| yolanda | hi AJaeger_ , can you revisit https://review.openstack.org/#/c/451259/ ? | 08:38 |
| *** tuanluong_ has quit IRC | 08:39 | |
| *** electrofelix has joined #openstack-infra | 08:39 | |
| *** rbuzatu has joined #openstack-infra | 08:44 | |
| *** makowals_ has joined #openstack-infra | 08:47 | |
| *** sputnik13 has quit IRC | 08:48 | |
| *** makowals has quit IRC | 08:48 | |
| *** BobBall_ has quit IRC | 08:51 | |
| *** liusheng has quit IRC | 08:51 | |
| *** liusheng has joined #openstack-infra | 08:52 | |
| thomas-oneill | this is the best place to ask about gerrit stuff right? I'm just trying to split a PS into a couple of separate PS's, so I've checked out master and I'm wanting to 'git apply $MY_OLD_PS_HASH -p' but obviously with master fetched the old PS hash isn't in the local tree. I know I can curl the base64 patch | base64 --decode | git apply -p # is this the best workflow do you think, or is there something | 08:52 |
| thomas-oneill | better? | 08:52 |
| *** saibarspeis has quit IRC | 08:54 | |
| kashyap | ianw: Hi there, maybe you know top off your head -- what is the current status of Fedora or CentOS jobs in upstream CI? Is it maintained? | 08:58 |
| *** markus_z has quit IRC | 08:58 | |
| thomas-oneill | oh not git apply -p, just git apply and then git add -p | 09:00 |
| *** neiljerram has joined #openstack-infra | 09:00 | |
| *** pgadiya has quit IRC | 09:01 | |
| *** markus_z has joined #openstack-infra | 09:01 | |
| *** zhurong has quit IRC | 09:01 | |
| *** ralonsoh has quit IRC | 09:04 | |
| *** neiljerram has quit IRC | 09:07 | |
| *** neiljerram has joined #openstack-infra | 09:08 | |
| *** sshnaidm|off is now known as sshnaidm | 09:09 | |
| *** thorst has joined #openstack-infra | 09:09 | |
| *** ralonsoh has joined #openstack-infra | 09:10 | |
| *** ralonsoh_ has joined #openstack-infra | 09:12 | |
| *** thorst has quit IRC | 09:13 | |
| *** sree has quit IRC | 09:14 | |
| *** ralonsoh has quit IRC | 09:15 | |
| *** pgadiya has joined #openstack-infra | 09:16 | |
| ianw | kashyap: yes they are maintained | 09:17 |
| *** sputnik13 has joined #openstack-infra | 09:17 | |
| kashyap | ianw: Hmm. Clark says: http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-03-29.log.html#t2017-03-29T16:30:50 | 09:18 |
| kashyap | ianw: The relevant quote: "if you can get openstack to actually work on centos/fedora that would be awesome, but the current jobs that exist don't work" | 09:18 |
| Dinesh_Bhor | Hi all, I have two patches to sync requirements of masakari related projects. Can someone please check those? https://review.openstack.org/#/q/status:open++topic:bp/enable-openstack-proposal-bot+owner:%22Dinesh+Bhor+%253Cdinesh.bhor%2540nttdata.com%253E%22 | 09:18 |
| *** sputnik13 has quit IRC | 09:19 | |
| *** yuanying has joined #openstack-infra | 09:19 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/451716 | 09:19 |
| ianw | kashyap: well, the jobs don't get as many eyes as other jobs | 09:20 |
| *** dtardivel has joined #openstack-infra | 09:20 | |
| *** matrohon has joined #openstack-infra | 09:21 | |
| ianw | but if something is wrong, it's "just" a bug. it can be investigated and fixed just like any other bug | 09:21 |
| *** pgadiya has quit IRC | 09:22 | |
| kashyap | ianw: :-) | 09:23 |
| kashyap | ianw: Okay, we need humans to look at the jobs, which is the perennial problem... | 09:24 |
| ianw | what i mean by that is that centos isn't special in any way. it uses all the same build & launch infrastructure as everything else | 09:24 |
| ianw | as you say, it's just a matter of who is paying attention | 09:24 |
| *** masayukig[m] has quit IRC | 09:24 | |
| *** csmart has quit IRC | 09:24 | |
| kashyap | ianw: Where can I read the details about these two jobs? | 09:24 |
| kashyap | Like a YAML file somwehere that tells me: "Here, this is where you can read the Fedora / CentOS job definitions" | 09:25 |
| ianw | they are done the same as all the other jobs in project-config | 09:25 |
| *** tosky has joined #openstack-infra | 09:26 | |
| *** csmart has joined #openstack-infra | 09:26 | |
| *** ralonsoh_ is now known as ralonsoh | 09:26 | |
| * kashyap looks | 09:26 | |
| kashyap | Thank you | 09:26 |
| ianw | kashyap: i also (when the cron hasn't broken) keep http://people.redhat.com/~iwienand/output.html updated | 09:26 |
| kashyap | Ah, thanks! | 09:27 |
| *** sambetts|afk is now known as sambetts | 09:27 | |
| *** rbuzatu has quit IRC | 09:27 | |
| *** alexchadin has quit IRC | 09:27 | |
| kashyap | ianw: The CentOS & Fedora jobs are using disk-image-builder? | 09:28 |
| *** alexchadin has joined #openstack-infra | 09:28 | |
| ianw | unfortunately, because they are non-voting, they are not captured by openstack-health | 09:28 |
| *** alexchadin has quit IRC | 09:28 | |
| ianw | (i have things in progress around that though) | 09:28 |
| ianw | yes, daily image builds for centos & fedora | 09:28 |
| *** alexchadin has joined #openstack-infra | 09:29 | |
| ianw | you can see them at http://nodepool.openstack.org/dib-image-list | 09:29 |
| *** alexchadin has quit IRC | 09:29 | |
| ianw | another thing i have in progress is a nicer html page for that | 09:29 |
| *** alexchadin has joined #openstack-infra | 09:29 | |
| *** alexchadin has quit IRC | 09:30 | |
| ianw | f25 has needed some help for a while. it's just fairly low priority, for me at least | 09:30 |
| *** alexchadin has joined #openstack-infra | 09:30 | |
| *** alexchadin has quit IRC | 09:30 | |
| *** alexchadin has joined #openstack-infra | 09:31 | |
| ianw | actually, i just realised we changed the name | 09:32 |
| kashyap | ianw: /me is just reading the scroll | 09:33 |
| kashyap | ianw: What does it take to turn them to voting? | 09:33 |
| ianw | a lot more than just me looking at them | 09:34 |
| *** pgadiya has joined #openstack-infra | 09:34 | |
| *** masayukig[m] has joined #openstack-infra | 09:34 | |
| kashyap | :-) | 09:35 |
| kashyap | Thanks for the details, and useful URLs | 09:36 |
| *** alexchadin has quit IRC | 09:37 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: New ovb playbook and settings https://review.openstack.org/431567 | 09:37 |
| ianw | kashyap: if you want any help adding a job running on centos to a project, i can help. you can even make it voting if you want. | 09:37 |
| ianw | the thing is with devstack, semi-frequently we have to push through something to unbreak the whole gate, and if it's blocked because centos isn't working then it's a big issue | 09:38 |
| *** Julien-zte has quit IRC | 09:38 | |
| kashyap | ianw: Thank you, will let you know. Just neck-deep buried in competing priorites, this other Gate issue with libvirt made me think of this topic again | 09:38 |
| *** yamahata has quit IRC | 09:38 | |
| kashyap | We also have this thing in progress (for over a year or more) as a DevStack plugin -- https://review.openstack.org/#/c/313568/ ("Plugin to setup libvirt/QEMU from tar releases") | 09:38 |
| ianw | yes, i think i've reviewed that in the past | 09:39 |
| ianw | there is nothing technical stopping such a thing. the only thing is developer bandwidth to build and maintain it | 09:40 |
| kashyap | ianw: How about shortage of nodes in Nodepool? It is not a problem any more? | 09:40 |
| kashyap | ianw: "such a thing" == the plugin? | 09:40 |
| kashyap | Yeah | 09:40 |
| kashyap | Probably not, you meant the Fedora / CentOS voting job maintainance | 09:41 |
| ianw | no, centos is equal to all the other node types | 09:41 |
| *** sdague has joined #openstack-infra | 09:41 | |
| ianw | i mean some sort of job installing later libvirt and testing it | 09:41 |
| ianw | it's great to get it setup, but it's not much good if people don't maintain it | 09:41 |
| kashyap | Precisely | 09:41 |
| kashyap | Shoot and forget is useless | 09:41 |
| openstackgerrit | Lenny Verkhovsky proposed openstack-infra/puppet-openstackci master: added Single Use Slave plugin https://review.openstack.org/450296 | 09:44 |
| *** markus_z has quit IRC | 09:44 | |
| ianw | right, that has happened a little with the fedora jobs, as we lack the bandwidth to prioritise it's stability | 09:45 |
| *** e0ne has joined #openstack-infra | 09:45 | |
| kashyap | ianw: One more, just to be clear: shortage of node-pool nodes is not a problem in this case, right? | 09:45 |
| *** shardy has quit IRC | 09:47 | |
| ianw | kashyap: no, not at all. nodepool satisfies demand equally. ubuntu, centos and fedora nodes are all the same, there is no priority | 09:47 |
| kashyap | Ah-ha, noted. | 09:48 |
| ianw | you can see at http://nodepool.openstack.org/image-list we have images in every region | 09:49 |
| * kashyap clicks | 09:49 | |
| *** Dinesh_Bhor has quit IRC | 09:49 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test new ovb with images building https://review.openstack.org/445228 | 09:50 |
| *** saibarspeis has joined #openstack-infra | 09:52 | |
| *** saibarspeis has quit IRC | 09:52 | |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Fix dib elements path https://review.openstack.org/451728 | 09:56 |
| sshnaidm | hi all, I have a question - do we have ip ranges of all cloud provider we use? | 09:56 |
| sshnaidm | if I want to report something from within a job to my server, but I'd like to prevent spam and fakes, for example by filtering IP | 09:57 |
| sshnaidm | or is there any other way to do it? | 09:57 |
| *** markus_z has joined #openstack-infra | 09:58 | |
| *** esberglu has joined #openstack-infra | 09:58 | |
| frickler | sshnaidm: no, the ip ranges can change without notice. I'm not sure how uploading logs is authenticated, you might want to look into that | 09:58 |
| sshnaidm | frickler, in which repo..? | 09:59 |
| frickler | sshnaidm: good question, probably start at devstack-gate? | 09:59 |
| ianw | sshnaidm: see https://docs.openstack.org/infra/system-config/logstash.html | 10:01 |
| ianw | it's a pull process | 10:02 |
| *** esberglu has quit IRC | 10:02 | |
| *** nadya has quit IRC | 10:03 | |
| ianw | i'm not sure about reporting things back in jobs ... i guess we don't really care if it's passed project review. but we do like to keep things open as much as possible, so sending things to black boxes isn't ideal | 10:04 |
| *** jkraj has joined #openstack-infra | 10:07 | |
| *** thorst has joined #openstack-infra | 10:10 | |
| *** liujiong has quit IRC | 10:10 | |
| *** yanyanhu has quit IRC | 10:10 | |
| *** Dinesh_Bhor has joined #openstack-infra | 10:12 | |
| sshnaidm | yeah, that's the problem to keep open key, how can I be sure I'm not compromised with fake reports | 10:14 |
| *** thorst has quit IRC | 10:14 | |
| *** dingyichen has quit IRC | 10:15 | |
| frickler | sshnaidm: so what data do you want to report that cannot fit into the logs reported by the jobs onto logs.o.o already? | 10:17 |
| frickler | sshnaidm: if the data is there, your server could just watch the gerrit event stream and process the reports accordingly | 10:19 |
| *** dingyichen has joined #openstack-infra | 10:20 | |
| sshnaidm | frickler, interesting.. where can I read about watching gerrit event stream? | 10:20 |
| frickler | sshnaidm: https://gerrit-review.googlesource.com/Documentation/cmd-stream-events.html and maybe https://github.com/j-griffith/sos-ci for a sample application | 10:22 |
| sshnaidm | frickler, thanks! | 10:22 |
| frickler | sshnaidm: there is also a mashed up stream https://specs.openstack.org/openstack-infra/infra-specs/specs/firehose.html | 10:22 |
| *** vsaienk0 has quit IRC | 10:24 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Merge branch 'master' into feature/zuulv3 https://review.openstack.org/445325 | 10:30 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 10:32 |
| *** onovy has quit IRC | 10:35 | |
| *** onovy has joined #openstack-infra | 10:35 | |
| *** mat128 has joined #openstack-infra | 10:36 | |
| Zara | just saw an email about ansiblefest in London; is anyone planning to go? | 10:36 |
| *** ccamacho has quit IRC | 10:38 | |
| *** alexchadin has joined #openstack-infra | 10:38 | |
| *** nadya has joined #openstack-infra | 10:43 | |
| *** ccamacho has joined #openstack-infra | 10:44 | |
| *** rossella_s has quit IRC | 10:45 | |
| *** matrohon has quit IRC | 10:45 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Fix test_leaked_node_not_deleted for v3 https://review.openstack.org/449375 | 10:46 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool feature/zuulv3: Merge branch 'master' into feature/zuulv3 https://review.openstack.org/445325 | 10:46 |
| *** tbarron is now known as tbarron|out | 10:54 | |
| *** arxcruz is now known as arxcruz|pto | 10:58 | |
| *** hashar is now known as hasharAway | 11:00 | |
| *** gongysh has quit IRC | 11:03 | |
| Dinesh_Bhor | AJaeger_: Hi, Could you please check this? https://review.openstack.org/#/c/451668/ | 11:06 |
| *** amoralej is now known as amoralej|lunch | 11:08 | |
| *** yamamoto has quit IRC | 11:10 | |
| *** thorst has joined #openstack-infra | 11:11 | |
| *** yamamoto has joined #openstack-infra | 11:11 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: New ovb playbook and settings https://review.openstack.org/431567 | 11:11 |
| *** yamamoto_ has joined #openstack-infra | 11:12 | |
| *** neiljerram has quit IRC | 11:14 | |
| *** thorst has quit IRC | 11:15 | |
| *** yamamoto has quit IRC | 11:16 | |
| *** neiljerram has joined #openstack-infra | 11:17 | |
| *** john-davidge has quit IRC | 11:20 | |
| *** john-dav_ has joined #openstack-infra | 11:20 | |
| *** gcb has quit IRC | 11:24 | |
| *** jkilpatr has quit IRC | 11:25 | |
| *** nmathew has quit IRC | 11:25 | |
| *** lucasagomes is now known as lucas-hungry | 11:28 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Support upgrades in multinode CI https://review.openstack.org/450784 | 11:32 |
| *** skelso has joined #openstack-infra | 11:33 | |
| *** thorst has joined #openstack-infra | 11:34 | |
| *** ldnunes has joined #openstack-infra | 11:40 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Support container upgrades in multinode OOOQ CI https://review.openstack.org/450784 | 11:42 |
| *** gcb has joined #openstack-infra | 11:44 | |
| *** matrohon has joined #openstack-infra | 11:44 | |
| *** EricGonczer_ has joined #openstack-infra | 11:44 | |
| *** kaisers has quit IRC | 11:45 | |
| *** gordc has joined #openstack-infra | 11:47 | |
| *** pgadiya has quit IRC | 11:47 | |
| *** dizquierdo is now known as dizquierdo_afk | 11:48 | |
| *** dprince has joined #openstack-infra | 11:49 | |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Add ocata to gerritbot https://review.openstack.org/450745 | 11:49 |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config master: jenkins-jobs: enable nova novnc proxy with TLS https://review.openstack.org/396289 | 11:50 |
| *** rhallisey has joined #openstack-infra | 11:54 | |
| *** dizquierdo_afk has quit IRC | 11:55 | |
| *** rossella_s has joined #openstack-infra | 11:56 | |
| *** aysyd has joined #openstack-infra | 11:59 | |
| *** florianf has quit IRC | 11:59 | |
| *** hasharAway is now known as hashar | 12:00 | |
| *** kaisers has joined #openstack-infra | 12:00 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Remove ovb updates job from promotion pipeline https://review.openstack.org/451779 | 12:00 |
| *** njohnston_ is now known as njohnston | 12:02 | |
| *** trown|outtypewww is now known as trown | 12:04 | |
| *** florianf has joined #openstack-infra | 12:07 | |
| *** jaypipes has joined #openstack-infra | 12:10 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Retire os-cloud-config https://review.openstack.org/451093 | 12:14 |
| *** pradk has joined #openstack-infra | 12:15 | |
| *** dave-mccowan has joined #openstack-infra | 12:16 | |
| *** tuanluong has quit IRC | 12:22 | |
| *** rbuzatu has joined #openstack-infra | 12:30 | |
| *** ggnel_t has joined #openstack-infra | 12:33 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Add support for $DO_BOOTSTRAP_SUBNODES https://review.openstack.org/437434 | 12:37 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Add generated nodepool key to authorized_keys https://review.openstack.org/437742 | 12:37 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Fix setting of node and node_private https://review.openstack.org/437433 | 12:37 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Update pip in DLRN venv https://review.openstack.org/438746 | 12:37 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Support running toci_gate_test-oooq.sh locally https://review.openstack.org/437431 | 12:37 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Stop dstat at the end of a job https://review.openstack.org/438745 | 12:37 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Guard adding bridge interface with $STABLE_RELEASE https://review.openstack.org/440838 | 12:37 |
| *** lucas-hungry is now known as lucasagomes | 12:39 | |
| *** EricGonc_ has joined #openstack-infra | 12:39 | |
| strigazi | AJaeger_ Do you have a moment? From a specific host when I pull magnum's repo with https it is very fast. But when I pull with ssh it is very slow. Do you have any suggestion? | 12:40 |
| *** EricGonczer_ has quit IRC | 12:41 | |
| *** kgiusti has joined #openstack-infra | 12:43 | |
| strigazi | AJaeger_ Pulling from github over ssh from that specific host is super fast. I have this only with review.openstack.org | 12:45 |
| *** alexchadin has quit IRC | 12:46 | |
| *** shardy has joined #openstack-infra | 12:49 | |
| *** shardy is now known as shardy_mtg | 12:50 | |
| *** amoralej|lunch is now known as amoralej | 12:52 | |
| *** rhallisey has quit IRC | 12:53 | |
| *** rbuzatu has quit IRC | 12:53 | |
| *** krtaylor has joined #openstack-infra | 12:54 | |
| *** links has quit IRC | 12:54 | |
| *** rhallisey has joined #openstack-infra | 12:54 | |
| *** shardy_mtg is now known as shardy | 12:56 | |
| fungi | strigazi: gerrit itself handles a lot of load from all sorts of things, and isn't optimized for serving git repositories directly. we don't have a recommended ssh-based git mirror but we do offer http/https and git protocol from git.openstack.org | 12:57 |
| fungi | since gerrit is a java-based application the ssh interface to it is provided by a java ssh library called minasshd running within the gerrit jvm. as you can imagine that's not super efficient | 12:57 |
| *** adarazs is now known as adarazs_afk | 12:57 | |
| *** EricGonc_ has quit IRC | 12:59 | |
| fungi | our recommendation is to set your origin remote to git.openstack.org (using http, https or git protocols) | 12:59 |
| *** EricGonczer_ has joined #openstack-infra | 12:59 | |
| *** knangia has joined #openstack-infra | 13:00 | |
| strigazi | fungi thanks, I'll try git.o.o. I was trying to understand what's wrong, is it my setup? is it server side? | 13:00 |
| fungi | strigazi: i didn't catch where anything is wrong. git clone/pull over ssh from review.openstack.org is slow, but that's to be expected | 13:01 |
| *** gouthamr has joined #openstack-infra | 13:01 | |
| *** gongysh has joined #openstack-infra | 13:02 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test new ovb with images building https://review.openstack.org/445228 | 13:02 |
| strigazi | fungi I see a lot of difference in speed from one host to another. Same OS, same git version, same network | 13:02 |
| fungi | also gerrit itself may be slower than usual at the moment since it's gone almost a week since restarting and over time tends to work itself int a java garbage collection death spiral | 13:03 |
| *** markus_z has quit IRC | 13:03 | |
| *** jcoufal has joined #openstack-infra | 13:06 | |
| strigazi | fungi 10s vs 2m20s something is really wrong | 13:09 |
| strigazi | fungi I guess it's me, but I can't find what it is | 13:09 |
| fungi | are the git remotes listed in .git/config identical? | 13:10 |
| fungi | you haven't been very specific about what you're seeing happen in 10s vs 2m20s and from where | 13:11 |
| *** slaweq__ has quit IRC | 13:11 | |
| fungi | without a detailed explanation of what tests you're performing and the contexts around them, i'm afraid i can't be much help | 13:11 |
| *** rbuzatu has joined #openstack-infra | 13:15 | |
| *** mdrabe has joined #openstack-infra | 13:15 | |
| *** zz_dimtruck is now known as dimtruck | 13:15 | |
| *** rbuzatu has quit IRC | 13:15 | |
| *** rbuzatu has joined #openstack-infra | 13:16 | |
| strigazi | fungi I try this [1] on my laptop and on my workstation on which I run devstack. [1] http://paste.openstack.org/show/604844/ in one case it is runs in 10s and in the other it takes 2m20s | 13:16 |
| fungi | strigazi: i don't know why that would be faster on one machine than another (lack of reverse dns entry? broken ipv6 connectivity?) but if it helps, the infra manual recommends https://git.openstack.org/openstack/magnum.git as the clone url (which will then end up as your origin remote): https://docs.openstack.org/infra/manual/developers.html#starting-work-on-a-new-project | 13:17 |
| *** eharney has joined #openstack-infra | 13:18 | |
| fungi | we don't expect pulling a large number of refs directly from gerrit's ssh interface to be particularly performant | 13:19 |
| *** markus_z has joined #openstack-infra | 13:19 | |
| fungi | strigazi: another possibility for the discrepancy might be mtu differences/fragmentation | 13:19 |
| *** signed8bit_Zzz is now known as signed8b_ | 13:19 | |
| strigazi | fungi I'll switch to https. Thanks for your help! | 13:20 |
| fungi | my pleasure | 13:20 |
| *** mat128 has quit IRC | 13:21 | |
| *** dizquierdo has joined #openstack-infra | 13:23 | |
| *** mat128 has joined #openstack-infra | 13:24 | |
| *** nadya has quit IRC | 13:25 | |
| *** annegentle has joined #openstack-infra | 13:27 | |
| *** baoli has joined #openstack-infra | 13:27 | |
| *** Julien-zte has joined #openstack-infra | 13:27 | |
| *** chem has quit IRC | 13:28 | |
| *** chem has joined #openstack-infra | 13:29 | |
| *** trown is now known as trown|brb | 13:30 | |
| *** esberglu has joined #openstack-infra | 13:31 | |
| *** nmathew has joined #openstack-infra | 13:33 | |
| *** chlong has joined #openstack-infra | 13:34 | |
| *** jlvillal is now known as jlvillal_pto | 13:35 | |
| *** jaosorior is now known as jaosorior_away | 13:37 | |
| *** adarazs_afk is now known as adarazs | 13:38 | |
| *** nmathew- has joined #openstack-infra | 13:39 | |
| *** nmathew has quit IRC | 13:42 | |
| *** wolverineav has joined #openstack-infra | 13:42 | |
| *** liusheng has quit IRC | 13:43 | |
| strigazi | fungi it was related to ipv6. But I'll use https from now on to try to be more kind to gerrit | 13:45 |
| fungi | strigazi: cool, glad you found the issue, and thanks for taking our systems performance into account! | 13:46 |
| *** dhajare has quit IRC | 13:47 | |
| sshnaidm | pabelanger, do you know if we always have eth0 interface on nodepool images? (and not em1, or anything else) | 13:48 |
| *** trown|brb is now known as trown | 13:50 | |
| fungi | sshnaidm: i believe on at least some platforms they get so-called "predictable" names now (e.g., enp2s0 or whatever) | 13:50 |
| clarkb | correct eth0 isnt universal | 13:50 |
| fungi | mostly depends on kernel version and udev rules | 13:50 |
| sshnaidm | yeah, but I mean only centos-7 images | 13:51 |
| fungi | good question. probably old-style eth0 or whatever | 13:52 |
| fungi | but i generally don't assume that since we aim to write cross-platform-capable software when possible | 13:52 |
| sshnaidm | is there a better way to predict them than parsing ip route output..? | 13:53 |
| fungi | bake that assumption into your scripts now, and when you want to move to centos-8 or a recent fedora or some other distro it becomes much harde4r | 13:53 |
| sshnaidm | yeah, agree | 13:54 |
| fungi | sshnaidm: i guess you're looking for the interface through which your default route lies? yes that's how i'd do it. keep in mind that we already have at least one provider where that differs for ipv4 vs ipv6 | 13:55 |
| sshnaidm | oh, forgot about ipv6 | 13:55 |
| fungi | in osic i want to say that the ipv4 default route is out eth1 while the ipv6 default route is out eth0 instead | 13:55 |
| *** isaacb_ has joined #openstack-infra | 13:55 | |
| fungi | or maybe it's the other way 'round | 13:55 |
| *** nmathew- has quit IRC | 13:56 | |
| fungi | anyway, just assuming you have only one egress interface is not safe | 13:56 |
| sshnaidm | yeah, good to know | 13:57 |
| *** dimtruck is now known as zz_dimtruck | 13:57 | |
| *** dingyichen has quit IRC | 13:57 | |
| *** Goneri has joined #openstack-infra | 13:57 | |
| *** dharinic has joined #openstack-infra | 13:58 | |
| *** isaacb has quit IRC | 13:58 | |
| * fungi notes his home firewall (openbsd) has two egress interfaces... re0 (a physical interface) for ipv4, gif0 (a tunnel interface) for ipv6 | 13:58 | |
| *** rbrndt has joined #openstack-infra | 13:59 | |
| fungi | so this is actually pretty common | 13:59 |
| clarkb | sshnaidm: why do you need to know the interface? | 13:59 |
| *** matrohon has quit IRC | 14:00 | |
| sshnaidm | clarkb, to get IP, to which host could be connected | 14:00 |
| clarkb | but isnt that already done for you? | 14:01 |
| clarkb | (by zuul) | 14:01 |
| clarkb | also note that you cant actually get the IP that way because some clouds use floating IPs | 14:01 |
| *** john-dav_ is now known as john-davidge | 14:02 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/shade master: Updated from global requirements https://review.openstack.org/451835 | 14:03 |
| fungi | though i wonder if it might be nice for zuul to inject an envvar containing the ip address it used to reach the node | 14:03 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/requestsexceptions master: Updated from global requirements https://review.openstack.org/451836 | 14:03 |
| *** burgerk has joined #openstack-infra | 14:03 | |
| clarkb | fungi: openstack does that for us via config drive | 14:04 |
| fungi | true, if it's in use | 14:04 |
| fungi | and if it's a new enough openstack i guess | 14:04 |
| *** neiljerram has quit IRC | 14:05 | |
| *** mriedem has joined #openstack-infra | 14:06 | |
| *** annegentle has quit IRC | 14:06 | |
| sshnaidm | clarkb, by zuul? | 14:06 |
| sshnaidm | clarkb, how does it do..? | 14:06 |
| clarkb | sshnaidm: zuul has already connected to the node and is executing code there | 14:07 |
| *** dizquierdo has quit IRC | 14:07 | |
| clarkb | so that you dont have to worry about the external IP | 14:07 |
| sshnaidm | clarkb, ok, but I need to know this IP | 14:07 |
| *** neiljerram has joined #openstack-infra | 14:07 | |
| sshnaidm | clarkb, is it in /etc/nodepool/provider ? | 14:07 |
| clarkb | sshnaidm: why? but also keep in mind vhecking interfaces isnt sufficient | 14:07 |
| clarkb | sshnaidm: yes that is one place that should have reliable info, but it will be ipv6 in some clouds and ipv4 in others | 14:08 |
| clarkb | if you want complete info I think you have to rely on config drive | 14:08 |
| *** burgerk has quit IRC | 14:08 | |
| fungi | perhaps to restate clarkb's "why" question, what to you intend to do with that ip address once you know it? | 14:09 |
| sshnaidm | clarkb, it's long story.. I need to inject repository on disk to chrooted environment (where disks are built) and I want to run web server on this repo directory | 14:09 |
| fungi | er, what do you | 14:09 |
| fungi | can't you use a loopback address for that? | 14:09 |
| *** eharney has quit IRC | 14:09 | |
| *** felipemonteiro has joined #openstack-infra | 14:10 | |
| fungi | or a virtual interface? | 14:10 |
| fungi | and just assign a stable address | 14:10 |
| sshnaidm | fungi, I wonder if it's accessible in the same way from chroot.. | 14:10 |
| sshnaidm | I'd like to do it as simple as possible | 14:10 |
| fungi | chroot is only a filesystem limitation. chroot does nothing to control network access | 14:10 |
| fungi | the equivalent in the kernel's tcp/ip stack is "network namespacing" | 14:11 |
| sshnaidm | I see | 14:12 |
| *** markus_z has quit IRC | 14:12 | |
| openstackgerrit | Merged openstack-infra/requestsexceptions master: Update tox for OpenStack https://review.openstack.org/451306 | 14:12 |
| sshnaidm | and if I want other hosts to connect to this repo too? | 14:12 |
| fungi | if you're not using namespacing to isolate the processes you run in your chroot from the network interfaces/routes outside teh chroot, then they can bind on teh same interfaces/addresses no problem | 14:12 |
| sshnaidm | fungi, ok, got it, will try localhost | 14:12 |
| fungi | sshnaidm: what other hosts? as soon as you have a need for multiple hosts to interact in a job, that should be designed as a multi-node job | 14:12 |
| sshnaidm | fungi, yeah, either ovb tripleo job.. | 14:13 |
| fungi | and for multi-node jobs, you want overlay networking and there is subnode connectivity info provided in /etc/nodepool/* files to make that possible | 14:13 |
| openstackgerrit | Merged openstack-infra/requestsexceptions master: Remove runtime depend on pbr https://review.openstack.org/451307 | 14:13 |
| *** szaher has quit IRC | 14:14 | |
| *** szaher has joined #openstack-infra | 14:14 | |
| *** ihrachys has joined #openstack-infra | 14:17 | |
| *** john-davidge has quit IRC | 14:19 | |
| *** jamesdenton has joined #openstack-infra | 14:19 | |
| *** mriedem has quit IRC | 14:20 | |
| *** matrohon has joined #openstack-infra | 14:22 | |
| *** john-davidge has joined #openstack-infra | 14:23 | |
| *** markus_z has joined #openstack-infra | 14:24 | |
| *** skelso has quit IRC | 14:24 | |
| openstackgerrit | Thiago da Silva proposed openstack-infra/project-config master: fix missing backslash https://review.openstack.org/451844 | 14:24 |
| *** eharney has joined #openstack-infra | 14:25 | |
| *** nadya has joined #openstack-infra | 14:26 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add gate-tripleo-ci-centos-7-ovb-containers-oooq-nv to cistatus https://review.openstack.org/451845 | 14:26 |
| *** john-davidge has quit IRC | 14:26 | |
| *** john-dav_ has joined #openstack-infra | 14:26 | |
| *** zz_dimtruck is now known as dimtruck | 14:27 | |
| lucasagomes | hi all, if you have some time today can you please take a look at https://review.openstack.org/#/c/450685/ ? This is adding a new repository for the tools that will be used in testing & developing sushy (a library that ironic uses to talk to redfish compat hardware) | 14:29 |
| lucasagomes | thanks | 14:29 |
| *** markus_z has quit IRC | 14:32 | |
| *** alexchadin has joined #openstack-infra | 14:33 | |
| *** vikrant has quit IRC | 14:33 | |
| *** martinkopec has quit IRC | 14:34 | |
| *** Julien-zte has quit IRC | 14:35 | |
| *** yamamoto_ has quit IRC | 14:36 | |
| *** jamesden_ has joined #openstack-infra | 14:37 | |
| *** armax has joined #openstack-infra | 14:37 | |
| *** jamesdenton has quit IRC | 14:38 | |
| *** isaacb_ has quit IRC | 14:41 | |
| openstackgerrit | Merged openstack-infra/project-config master: fix missing backslash https://review.openstack.org/451844 | 14:41 |
| *** john-dav_ has quit IRC | 14:41 | |
| *** tobiash_ is now known as tobiash | 14:42 | |
| *** john-davidge has joined #openstack-infra | 14:43 | |
| *** cody-somerville has joined #openstack-infra | 14:44 | |
| *** eharney has quit IRC | 14:44 | |
| *** martinkopec has joined #openstack-infra | 14:46 | |
| *** isaacb_ has joined #openstack-infra | 14:47 | |
| *** hongbin has joined #openstack-infra | 14:47 | |
| *** mdrabe has quit IRC | 14:51 | |
| *** armax_ has joined #openstack-infra | 14:51 | |
| *** prateek has quit IRC | 14:51 | |
| *** armax has quit IRC | 14:52 | |
| *** armax_ is now known as armax | 14:52 | |
| *** skelso has joined #openstack-infra | 14:52 | |
| *** annegentle has joined #openstack-infra | 14:52 | |
| *** annegentle has quit IRC | 14:53 | |
| *** marst has quit IRC | 14:53 | |
| *** eharney has joined #openstack-infra | 15:00 | |
| *** rcernin has quit IRC | 15:00 | |
| *** hashar has quit IRC | 15:01 | |
| *** mdrabe has joined #openstack-infra | 15:01 | |
| *** marst has joined #openstack-infra | 15:01 | |
| *** annegentle has joined #openstack-infra | 15:02 | |
| *** skelso has quit IRC | 15:04 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Add ability to select flavor by name or id https://review.openstack.org/449784 | 15:04 |
| *** funzo has joined #openstack-infra | 15:05 | |
| pabelanger | clarkb: fungi: did we want to give https://review.openstack.org/#/c/451554/ at test this morning? I have some bandwidth to do validation | 15:09 |
| clarkb | pabelanger: I'm game | 15:09 |
| fungi | pabelanger: go for it | 15:10 |
| fungi | i'll apprive since you already +2'd | 15:10 |
| *** adrian_otto has joined #openstack-infra | 15:11 | |
| *** isaacb__ has joined #openstack-infra | 15:11 | |
| pabelanger | clarkb: fungi: mordred: speaking of default interfaces, it looks like debian-jessie is still eth0. Here is a systemd update to get glean working: https://review.openstack.org/#/c/450775/ the question is, that will actually start eth0 by default now, I am not sure if we want to do that or not. Since we make an effort in dib to remove all interface.d things | 15:11 |
| pabelanger | DefaultInstance=eth0 is the systemd setting in question | 15:12 |
| pabelanger | fungi: danke | 15:12 |
| fungi | clarkb: pabelanger: biggest thing to keep an eye on with 451554 is making sure it doesn't break apache on restart and take down all our mirrors. that could cause quite a lot of job failures | 15:13 |
| fungi | so i guess be ready to log into all the mirrors and quick fix them if we have to | 15:13 |
| pabelanger | agree | 15:13 |
| pabelanger | fungi: or, we can put all mirror -1 into emergency to be safe | 15:14 |
| fungi | if you want to do that, i'm fine with it | 15:14 |
| clarkb | +1 | 15:14 |
| pabelanger | k, I'll set that up now | 15:15 |
| *** mriedem has joined #openstack-infra | 15:15 | |
| *** adrian_otto has quit IRC | 15:15 | |
| *** isaacb_ has quit IRC | 15:15 | |
| *** armax has quit IRC | 15:15 | |
| vkmc | hey all, can somebody take the final push to this one? https://review.openstack.org/#/c/447125/ | 15:17 |
| *** andreas_s has quit IRC | 15:17 | |
| *** gongysh has quit IRC | 15:17 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add infracloud mirrors to cacti.o.o https://review.openstack.org/451863 | 15:18 |
| pabelanger | clarkb: fungi: ^noticed they were missing | 15:18 |
| openstackgerrit | Merged openstack-infra/system-config master: Use caching reverse proxy for rdo repos https://review.openstack.org/451554 | 15:18 |
| fungi | and assuming it runs okay, we'll want to keep a close eye on utilization for / (or /opt when present) filesystems on the mirrors | 15:19 |
| pabelanger | clarkb: also: http://status.openstack.org/elastic-recheck/#1531187 | 15:20 |
| pabelanger | looks like our glean fix is working | 15:20 |
| mordred | pabelanger: woot! | 15:21 |
| *** skelso has joined #openstack-infra | 15:21 | |
| clarkb | as my kids would say "Hurray" while throwing arms up in the air | 15:21 |
| mordred | clarkb: that sounds very similar to me | 15:22 |
| clarkb | mordred: did you see the jeepyb thing late yesterday? | 15:22 |
| pabelanger | http://status.openstack.org/elastic-recheck/#1674681 | 15:22 |
| pabelanger | looks good now too | 15:22 |
| openstackgerrit | Merged openstack-infra/shade master: Fix a few minor annoyances that snuck in https://review.openstack.org/451276 | 15:22 |
| openstackgerrit | Merged openstack-infra/shade master: Enable H238 - classes should be subclasses of object https://review.openstack.org/451277 | 15:22 |
| clarkb | mordred: blazar doesn't have up to date acls in gerrit, was going to try and take a look today if no one beat me to it | 15:22 |
| pabelanger | I'm going to remove the bug from e-r | 15:22 |
| openstackgerrit | Merged openstack-infra/shade master: Enable H201 - don't throw bare exceptions https://review.openstack.org/451278 | 15:23 |
| clarkb | pabelanger: I think usually they let the bugs fall to 0 over the entire indexing period, but you can propose removal now then merge later | 15:23 |
| *** nadya has quit IRC | 15:23 | |
| mordred | clarkb: oh - right - I started looking and then got distracted | 15:24 |
| fungi | clarkb: there was a very recent fix to jeepyb which should fix a regression in the caching implementation where it only performed acl pushes on initial creation due to a logic short-circuit around git cloning... maybe that wasn't in place when the acl change in question landed? | 15:24 |
| clarkb | mordred: too much throwing arms in the air I imagine | 15:24 |
| pabelanger | clarkb: Ah, I can wait | 15:25 |
| mordred | clarkb: yup | 15:25 |
| mordred | clarkb: so - what I did just now is edited /opt/lib/jeepyb/project.cache and removed the acl_sha line from the openstack/blazar entry, then re-ran manage-projects | 15:25 |
| pabelanger | clarkb: so, this caching thing for apache, could it be applied to hub.docker.com too? | 15:25 |
| mordred | clarkb: this has fixed things before | 15:25 |
| pabelanger | for short term | 15:25 |
| openstackgerrit | Merged openstack-infra/shade master: Reenable hacking tests that already pass https://review.openstack.org/451279 | 15:26 |
| openstackgerrit | Merged openstack-infra/shade master: Take care of multiple imports and update explanation https://review.openstack.org/451280 | 15:26 |
| pabelanger | I'm trying to see how we can setup the registry to use AFS backend | 15:26 |
| mordred | clarkb: which makes me think we have a codepath somewhere which is writing the acl sha to the file without actually pushing the new acls | 15:26 |
| openstackgerrit | Merged openstack-infra/shade master: Updated from global requirements https://review.openstack.org/451835 | 15:26 |
| *** sree has joined #openstack-infra | 15:28 | |
| clarkb | pabelanger: right now the max cached file size is 100MB as in the change I pushed | 15:29 |
| *** scarab_ has joined #openstack-infra | 15:29 | |
| clarkb | pabelanger: our mirrors only have a few gigabytes of disk under /opt but otherwise assuming they don't mangle http too badly yes I think it would work | 15:29 |
| pabelanger | clarkb: ya, that is what I think too. | 15:30 |
| pabelanger | and same concern about local HDD space | 15:30 |
| clarkb | mordred: fun, maybe we should do that for all projects, have it "update" across the baord in order to reset the cache properly? | 15:30 |
| *** saibarspeis has joined #openstack-infra | 15:31 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Fetch list of AZs from nova if it's not configured https://review.openstack.org/450345 | 15:31 |
| clarkb | pabelanger: which mirror(s) are not disabled in ansible/puppet? | 15:32 |
| clarkb | (so I can pay attention to the right one) | 15:32 |
| pabelanger | http://mirror.regionone.infracloud-vanilla.openstack.org/ | 15:32 |
| pabelanger | is where I am testing | 15:32 |
| clarkb | pabelanger: the utility of it is going to largely depend on how cacheable the artifacts our jobs request are | 15:33 |
| clarkb | that and how reliable the connection between proxy and backend is | 15:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Cleanup from config syntax change https://review.openstack.org/451868 | 15:33 |
| *** sree has quit IRC | 15:33 | |
| johnsom | Hello infra folks. I have a gerrit ACL issue I could use help with. The ACL displayed in gerrit does not match the ACL in git and the group was not created. This is for openstack/neutron-lbaas | 15:33 |
| clarkb | johnsom: that is a likely bug in jeepyb's new caching code paths that we were just talking about | 15:34 |
| clarkb | mordred: did you want to do what you did for blazar therre ^ or just invalidate the whole cache andforce it to update all the things? | 15:34 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Do not install test-requirements for ansible test https://review.openstack.org/450803 | 15:35 |
| *** adrian_otto has joined #openstack-infra | 15:35 | |
| mordred | clarkb: I do not want to invalidate the whole cache - last time I tried that I was unable to get an entire run to complete :) | 15:35 |
| johnsom | Ok. Currently the octavia team can't merge stable/ocata patches. Please let me know if there is something I can do to help. | 15:36 |
| *** neiljerram has quit IRC | 15:36 | |
| mordred | johnsom: try again | 15:36 |
| *** saibarspeis has quit IRC | 15:36 | |
| *** yamamoto has joined #openstack-infra | 15:36 | |
| johnsom | That looks better | 15:36 |
| mordred | woot | 15:36 |
| johnsom | Can we seed the new group, octavia-release-branch? | 15:37 |
| mordred | clarkb: so - I think we have enough instances of "delete the acl_sha in the cache file" fixing it that investigating _why_ those cache entries are breaking is the next step | 15:37 |
| *** neiljerram has joined #openstack-infra | 15:37 | |
| mordred | johnsom: sure- who with? | 15:37 |
| pabelanger | clarkb: cool it is working. But there is some issues with our manifest. | 15:37 |
| pabelanger | clarkb: working on a patch | 15:37 |
| johnsom | mordred Either add me or copy the folks from octavia-stable-maint. | 15:37 |
| pabelanger | clarkb: http://mirror.regionone.infracloud-vanilla.openstack.org/rdo/f25/report.html | 15:38 |
| mordred | johnsom: I added you | 15:38 |
| johnsom | Thanks! I can take it from there | 15:38 |
| mordred | woot | 15:38 |
| clarkb | pabelanger: yay my apache configs work (at least to serve that page) | 15:39 |
| clarkb | pabelanger: curious what the manifest problem is | 15:39 |
| pabelanger | clarkb: I switched from https to http, to get it working. Trying to figure out why SSL doesn't work | 15:39 |
| pabelanger | we also need proxy_http | 15:39 |
| pabelanger | enabled | 15:39 |
| openstackgerrit | Ari proposed openstack-infra/jenkins-job-builder master: Add support for jms_messaging plugin as a trigger https://review.openstack.org/451871 | 15:40 |
| *** dtardivel has quit IRC | 15:40 | |
| clarkb | ah | 15:40 |
| *** jkraj has quit IRC | 15:40 | |
| clarkb | pabelanger: but the vhost worked as is? | 15:40 |
| pabelanger | clarkb: ya | 15:40 |
| clarkb | once extra module was enabled that is? | 15:40 |
| clarkb | its possible apache doesn't want to have http fronting https? | 15:41 |
| pabelanger | SSLProxyEngine on | 15:42 |
| pabelanger | we need that | 15:42 |
| clarkb | aha | 15:42 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Validate flavor specification in config https://review.openstack.org/451875 | 15:43 |
| *** yamamoto has quit IRC | 15:43 | |
| *** weshay is now known as weshay_mtg | 15:43 | |
| *** ccamacho has left #openstack-infra | 15:43 | |
| mordred | pabelanger: for dockerhub - I found this: https://blog.docker.com/2015/10/registry-proxy-cache-docker-open-source/ - which while slightly more work might be a decent solution for a similar best-effort passthrough cache | 15:44 |
| *** e0ne has quit IRC | 15:44 | |
| clarkb | mordred: I started with apache for the rdo thing precisely because it was low effort (3 lines of config after one time payment of general setup config fees) | 15:45 |
| clarkb | I think part of the issue with the afs stuff ahs been how one off it feels (lots of manual effort and convincing new and specific tools to play nice with afs etc) so wanted to avoid doing that in the cheap not perfect case | 15:46 |
| *** scarab_ has quit IRC | 15:47 | |
| pabelanger | mordred: I too seen that. I would be cool if we can do it without running docker containers on the mirrors. that's what I am trying to figure out | 15:47 |
| mordred | clarkb: totally | 15:49 |
| mordred | pabelanger: we can TOTALLY run docker registry without running docker | 15:49 |
| mordred | pabelanger: it's even packaged in ubuntu | 15:49 |
| mordred | pabelanger: so 'apt-get install docker-registry' gets you a docker registry that is not running in a docker container | 15:49 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Fixed for rdo apache cache on mirrors https://review.openstack.org/451878 | 15:50 |
| pabelanger | mordred: ++ | 15:50 |
| mordred | pabelanger: which means I _think_ we could just add that and a config file to our mirror hosts and have a docker-enabled thing that's similar to the apache thing for rdo ... | 15:50 |
| mordred | it doesn't solve the other problem which is "let's set up an AFS-backed docker registry into which local images can be published" - which I think is a bit more work | 15:50 |
| clarkb | mordred: we could also probably just apache docker | 15:50 |
| openstackgerrit | Ari proposed openstack-infra/jenkins-job-builder master: Add support for jms_messaging plugin as a trigger https://review.openstack.org/451871 | 15:51 |
| clarkb | unless they have decided they are too cool for http and you need a ton of business logic in a special daemon just to do it | 15:51 |
| mordred | clarkb: we might be able to - sure - but I don't know a ton about the api back and forth there | 15:51 |
| mordred | yah | 15:51 |
| clarkb | which wouldn't surprise me unfortunately | 15:51 |
| *** aarefiev is now known as aarefiev_afk | 15:51 | |
| pabelanger | the part I am curious about is how to setup afs things 'vos release' for this. If possible | 15:51 |
| *** neiljerram has quit IRC | 15:51 | |
| pabelanger | mordred: clarkb: 451878 is you don't mind | 15:52 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Fixes for rdo apache cache on mirrors https://review.openstack.org/451878 | 15:52 |
| *** neiljerram has joined #openstack-infra | 15:53 | |
| mordred | pabelanger, clarkb: so anywho - I support whichever approach to passthrough caching of docker things we think is easier to deal with and maintain | 15:53 |
| mordred | but I'm going to be AFK for the rest of the day, so can't likely do any help until tomorrow at earliest | 15:53 |
| *** Swami has joined #openstack-infra | 15:54 | |
| *** isaacb__ has quit IRC | 15:56 | |
| clarkb | pabelanger: commented | 15:56 |
| clarkb | couple small things then I'm good | 15:56 |
| *** lucasagomes is now known as lucas-afk | 15:56 | |
| *** yamamoto has joined #openstack-infra | 15:56 | |
| *** yamamoto has quit IRC | 15:56 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add canonical hostname to source object https://review.openstack.org/451110 | 15:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add source to projects https://review.openstack.org/451596 | 15:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add a project index to Tenant https://review.openstack.org/451597 | 15:57 |
| *** ggnel_t has quit IRC | 15:58 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Remove classic multinode jobs from cistatus https://review.openstack.org/451884 | 15:58 |
| pabelanger | clarkb: ack | 15:59 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Fixes for rdo apache cache on mirrors https://review.openstack.org/451878 | 15:59 |
| pabelanger | and fixed | 15:59 |
| *** sree has joined #openstack-infra | 16:00 | |
| pabelanger | clarkb: one sec on that review, testing something out | 16:01 |
| *** martinkopec has quit IRC | 16:01 | |
| clarkb | oh I just +2'd it but no approval so we are fine | 16:01 |
| *** saibarspeis has joined #openstack-infra | 16:01 | |
| pabelanger | sure | 16:01 |
| _ari_ | zxiiro: review if you have the cycles - https://review.openstack.org/#/c/323495/ thanks! | 16:01 |
| sdague | clarkb: https://review.openstack.org/#/c/449311/ is the journal support in d-g we should probably add | 16:01 |
| pabelanger | clarkb: question, http://mirror.regionone.infracloud-vanilla.openstack.org/rdo/f25/ | 16:01 |
| pabelanger | URL rewrite looks to be correct | 16:01 |
| _ari_ | zaro: if you have the cycles to review https://review.openstack.org/#/c/323495/ | 16:02 |
| pabelanger | meaning, we properly traverse on our mirror | 16:02 |
| pabelanger | clarkb: any idea, why http://mirror.regionone.infracloud-vanilla.openstack.org/rdo/ doesn't? | 16:02 |
| zxiiro | _ari_: ummm? wrong patch? that one's already merged | 16:02 |
| _ari_ | zxiiro: sorry this one https://review.openstack.org/#/c/451871/ | 16:02 |
| _ari_ | zaro: this one sorry https://review.openstack.org/#/c/451871/ | 16:02 |
| clarkb | pabelanger: http://mirror.regionone.infracloud-vanilla.openstack.org/rdo/ just worked for me | 16:02 |
| pabelanger | clarkb: do the links? | 16:02 |
| _ari_ | zxiiro: sorry about that I was looking who reviewed my last one ;) | 16:03 |
| pabelanger | I think I see the issue | 16:03 |
| pabelanger | it is rdo side | 16:03 |
| zxiiro | _ari_: oh it was pushed today it's in my email queue i'll take a look when I go through it. | 16:03 |
| _ari_ | zxiiro: thanks! | 16:03 |
| clarkb | pabelanger: no the links aren't rewritten by apache, just the ehaders | 16:03 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Merge branch 'master' into feature/zuulv3 https://review.openstack.org/445325 | 16:03 |
| clarkb | pabelanger: it isn't introspecting the bodies of the requests and rewriting them | 16:03 |
| zxiiro | _ari_: unfortunately I got a software release to take care of today and am at ONS next week so might not be able to look at it until I get back from ONS | 16:03 |
| pabelanger | clarkb: Ah I see why | 16:04 |
| pabelanger | clarkb: perfect | 16:04 |
| pabelanger | thanks | 16:04 |
| clarkb | and it does rely on things using relative not absolute links | 16:04 |
| *** scarab__ has joined #openstack-infra | 16:04 | |
| *** scarab_ has joined #openstack-infra | 16:04 | |
| _ari_ | zxiiro: is there anyone else that can take a look | 16:04 |
| *** sree has quit IRC | 16:04 | |
| zxiiro | _ari_: maybe electrofelix | 16:04 |
| *** jordanP_ has joined #openstack-infra | 16:05 | |
| pabelanger | clarkb: does: http://mirror.regionone.infracloud-vanilla.openstack.org/rdo/centos7-master/ work for you? | 16:05 |
| _ari_ | electrofelix: if you have the cycles could you review https://review.openstack.org/#/c/451871/ | 16:05 |
| *** alexchadin has quit IRC | 16:05 | |
| clarkb | pabelanger: no, but looks like backend is https only, did you get the fix for https in place and switch the backend url back? | 16:06 |
| *** saibarspeis has quit IRC | 16:06 | |
| esberglu | I am upgrading my multinode environment from newton to ocata. Currently I have to deploy the control, then the compute, then go back to the control and run | 16:06 |
| esberglu | "nova-manage cell_v2 simple_cell_setup" | 16:06 |
| esberglu | Otherwise I get an error about no host to cell mapping and can't deploy any instances | 16:06 |
| esberglu | Not sure if there is some sort of devstack configuration that I am missing or if this the correct way to go about doing it | 16:06 |
| *** matrohon has quit IRC | 16:07 | |
| clarkb | esberglu: nova would be the best place to ask, but aiui yes you need explicit toggling of things for cells | 16:07 |
| pabelanger | clarkb: ya, I have the changes locally still | 16:07 |
| pabelanger | http://mirror.regionone.infracloud-vanilla.openstack.org/rdo/centos7/ | 16:08 |
| pabelanger | works | 16:08 |
| pabelanger | I wonder if I can just looking at wrong URLS | 16:08 |
| pabelanger | I'll pop into #rdo and confirm | 16:08 |
| *** kvcobb has joined #openstack-infra | 16:08 | |
| *** dharinic has quit IRC | 16:08 | |
| clarkb | huh they both redirect to https so ya there goes my theory | 16:08 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Remove support for nodepool_id https://review.openstack.org/451889 | 16:09 |
| *** scarab__ has quit IRC | 16:09 | |
| *** scarab_ has quit IRC | 16:09 | |
| *** EricGonczer_ has quit IRC | 16:10 | |
| pabelanger | clarkb: I think it might be symlinks? | 16:12 |
| pabelanger | I know they do a lot of it in RDO | 16:12 |
| *** nadya has joined #openstack-infra | 16:12 | |
| clarkb | esberglu: https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/devstack-vm-gate.sh#n240 is what we do | 16:12 |
| *** scarab_ has joined #openstack-infra | 16:12 | |
| *** scarab__ has joined #openstack-infra | 16:12 | |
| *** admcleod has quit IRC | 16:13 | |
| *** admcleod has joined #openstack-infra | 16:13 | |
| clarkb | pabelanger: you shoudl eb able to check the apache logs to see why apache isn't happy | 16:13 |
| *** scarab_ has quit IRC | 16:14 | |
| pabelanger | 23.233.28.9 - - [30/Mar/2017:16:14:25 +0000] "GET /rdo/centos7/current-tripleo/ HTTP/1.1" 404 442 "http://mirror.regionone.infracloud-vanilla.openstack.org/rdo/centos7/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" | 16:15 |
| odyssey4me | project-config cores, I'd appreciate a second review for https://review.openstack.org/449272 | 16:15 |
| odyssey4me | it's been sitting for some time | 16:15 |
| *** scarab__ has quit IRC | 16:15 | |
| *** beslemon has joined #openstack-infra | 16:16 | |
| *** nadya has quit IRC | 16:16 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Source the right environment for rh1 https://review.openstack.org/451893 | 16:17 |
| *** jaosorior_away is now known as jaosorior | 16:19 | |
| *** shardy has quit IRC | 16:20 | |
| odyssey4me | thanks fungi | 16:20 |
| clarkb | hrm apache error logs are silent | 16:21 |
| clarkb | almost like the other end is 404ing and proxy is just forwarding on | 16:21 |
| *** mriedem has quit IRC | 16:22 | |
| *** pcaruana has quit IRC | 16:23 | |
| *** psachin has quit IRC | 16:24 | |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: networking-bagpipe: add an experimental fullstack job https://review.openstack.org/451897 | 16:25 |
| *** tphummel has joined #openstack-infra | 16:26 | |
| pabelanger | clarkb: ya, I think it is because the content in RDO is a symlink | 16:29 |
| *** chem is now known as chem_gone | 16:29 | |
| clarkb | pabelanger: we can probably toggle on more verbose debug logs to see if backend is sending a 404 | 16:29 |
| pabelanger | and our apache might be getting confused | 16:29 |
| *** armax has joined #openstack-infra | 16:29 | |
| *** rbuzatu has quit IRC | 16:29 | |
| clarkb | well all our apache is doing is making an http request to their backend | 16:30 |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Add prep upper constraints macro to bifrost jobs https://review.openstack.org/451898 | 16:30 |
| clarkb | it won't know about any symlinks, just what http says | 16:30 |
| openstackgerrit | Merged openstack-infra/project-config master: Add more periodic tests for OpenStack-Ansible https://review.openstack.org/449272 | 16:31 |
| yolanda | TheJulia, does it make sense to you? https://review.openstack.org/451898 | 16:31 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Fixes for rdo apache cache on mirrors https://review.openstack.org/451878 | 16:31 |
| pabelanger | clarkb: same | 16:31 |
| *** weshay_mtg is now known as weshay | 16:32 | |
| clarkb | LogLevel Debug iirc to get the extra logs | 16:32 |
| clarkb | maybe turn that on manually and do a couple requsets then turn it off and dig in? | 16:32 |
| *** yamahata has joined #openstack-infra | 16:33 | |
| *** beslemon_ has joined #openstack-infra | 16:33 | |
| *** beslemon has quit IRC | 16:34 | |
| *** admcleod has quit IRC | 16:35 | |
| *** admcleod has joined #openstack-infra | 16:35 | |
| *** admcleod has quit IRC | 16:36 | |
| *** admcleod has joined #openstack-infra | 16:36 | |
| *** tosky has quit IRC | 16:37 | |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: networking-bagpipe: add an experimental fullstack job https://review.openstack.org/451897 | 16:39 |
| *** xarses has joined #openstack-infra | 16:39 | |
| *** tesseract has quit IRC | 16:40 | |
| *** sambetts is now known as sambetts|afk | 16:41 | |
| *** adrianofr_ has joined #openstack-infra | 16:45 | |
| *** trown is now known as trown|lunch | 16:46 | |
| *** prateek has joined #openstack-infra | 16:48 | |
| *** nadya has joined #openstack-infra | 16:48 | |
| *** chem_gone has quit IRC | 16:49 | |
| *** nadya has quit IRC | 16:49 | |
| *** jaosorior has quit IRC | 16:50 | |
| *** prateek has quit IRC | 16:51 | |
| *** prateek has joined #openstack-infra | 16:51 | |
| *** derekh has quit IRC | 16:53 | |
| jordanP_ | I have some 502/Proxy error with Gerrit | 16:57 |
| bnemec | Was just about to say ^ :-) | 16:57 |
| *** yamamoto has joined #openstack-infra | 16:57 | |
| *** tosky has joined #openstack-infra | 16:58 | |
| clarkb | I think fungi jinxed it earlier by saying it had been running long without a restart | 16:59 |
| pabelanger | [Thu Mar 30 16:59:00.610579 2017] [core:info] [pid 18270:tid 140206764623616] [client 23.233.28.9:60906] AH00128: File does not exist: /var/www/mirror/rdo/centos7/puppet.passed.ci/, referer: http://mirror.regionone.infracloud-vanilla.openstack.org/rdo/centos7/ | 16:59 |
| clarkb | trying to pull up GC info no | 16:59 |
| pabelanger | clarkb: ^ | 16:59 |
| clarkb | s/no/now/ | 16:59 |
| clarkb | pabelanger: so its looking on disk for some reason, fun! | 17:00 |
| fungi | garbage collection activity does seem to be starting to ramp up in the jvm (though not spiking quite as high we usually see for this yet). looks like our last restart was almost a week ago | 17:00 |
| *** Apoorva has joined #openstack-infra | 17:00 | |
| clarkb | pabelanger: likely the rewrite rules above matching for our centos7 mirror? | 17:00 |
| pabelanger | clarkb: Oh | 17:00 |
| pabelanger | you know what | 17:00 |
| pabelanger | maybe | 17:00 |
| clarkb | fungi: I confirm, shall we restart? | 17:00 |
| fungi | yeah, probably should | 17:00 |
| clarkb | pabelanger: cheap fix would be to host at eg port 8080 and use different vhost | 17:01 |
| fungi | i'll get the statusbot notice going | 17:01 |
| pabelanger | clarkb: ++ | 17:01 |
| pabelanger | lets do that | 17:01 |
| clarkb | fungi: and I can go ahead with the restart | 17:01 |
| clarkb | doing that now | 17:01 |
| jordanP_ | thanks | 17:01 |
| *** scarab_ has joined #openstack-infra | 17:01 | |
| fungi | #status notice The Gerrit service on review.openstack.org is being restarted briefly to relieve performance issues, and should return to service again momentarily. | 17:02 |
| openstackstatus | fungi: sending notice | 17:02 |
| *** pbourke has joined #openstack-infra | 17:02 | |
| clarkb | maybe now is good time for zaro to ask for reviews on the upgrade changes :) | 17:02 |
| fungi | well, maybe not while gerrit is offline, but... ;) | 17:02 |
| clarkb | gerrit is running again | 17:03 |
| fungi | is teh coolnez | 17:03 |
| clarkb | fungi: that would be a neat trick though | 17:03 |
| fungi | gertty | 17:03 |
| clarkb | (I edited the db to reflect my +2) | 17:03 |
| fungi | it _is_ a neat trick, in fact ;) | 17:03 |
| -openstackstatus- NOTICE: The Gerrit service on review.openstack.org is being restarted briefly to relieve performance issues, and should return to service again momentarily. | 17:03 | |
| *** Apoorva has quit IRC | 17:03 | |
| *** ralonsoh has quit IRC | 17:04 | |
| *** Apoorva has joined #openstack-infra | 17:04 | |
| *** yamamoto has quit IRC | 17:04 | |
| *** skelso has quit IRC | 17:04 | |
| pabelanger | fungi: and a quick test of stackalytics.o.o shows the session was closed properly when I terminated the processor script | 17:05 |
| fungi | nice | 17:05 |
| openstackstatus | fungi: finished sending notice | 17:05 |
| *** claudiub has quit IRC | 17:06 | |
| clarkb | pabelanger: you working on vhost move? | 17:07 |
| pabelanger | clarkb: unless you want too | 17:07 |
| clarkb | pabelanger: fungi: I wonder if the stackalytics issue is the paramiko one with not explicitly closing clients | 17:07 |
| pabelanger | clarkb: yes, that is the patch I added from nodepool | 17:07 |
| fungi | clarkb: right, that's what pabelanger has attempted to fix | 17:07 |
| clarkb | ah ok | 17:07 |
| clarkb | missed that | 17:07 |
| clarkb | pabelanger: you can write it | 17:08 |
| pabelanger | https://review.openstack.org/#/c/451550/ | 17:08 |
| pabelanger | clarkb: I was just going to use the same vhost | 17:08 |
| pabelanger | vhost template file | 17:08 |
| clarkb | ya that works | 17:09 |
| clarkb | but then separate vhost sections | 17:09 |
| pabelanger | yes | 17:09 |
| *** MasterOfBugs has joined #openstack-infra | 17:10 | |
| rajinir | jhesketh: josh, could you please enable the gerrit account openstack-ironic-ci, we have fixed the problem | 17:10 |
| *** signed8b_ is now known as signed8bit_Zzz | 17:12 | |
| bswartz | have you guys considered setting up a crontab that reboots gerrit every night? or every week? | 17:12 |
| clarkb | bswartz: no, but we have done it with other jvm tools (jenkins) | 17:13 |
| bswartz | I remember back in the days when I used microsoft software, regular reboots were considered good hygiene | 17:13 |
| clarkb | bswartz: currently trying to put effort on upgrading gerrit which is supposed to fix the memory leak in jgit | 17:13 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Move proxy reverse cache to port 8080 https://review.openstack.org/451878 | 17:13 |
| pabelanger | clarkb: ^might need some help | 17:13 |
| pabelanger | not sure if we need servername settings | 17:14 |
| bswartz | clarkb: oh it's a known issue? | 17:14 |
| bswartz | that's encouraging | 17:14 |
| clarkb | bswartz: yes with a fix upstream, we just have to prepare an upgrade and do all the work associated with that | 17:14 |
| clarkb | bswartz: the problem is gerrit caches git objects (really jgit aiui) | 17:15 |
| clarkb | bswartz: and it has no concept of flushing caches | 17:15 |
| clarkb | and so your cache grows until you have no more memroy left whichforces the garbage collector to use all the cpu time | 17:15 |
| bswartz | yeah if you're google you never flush anything because you have infinite resources... | 17:16 |
| clarkb | well if you're google you don't actually use any of this code | 17:16 |
| clarkb | gerrits dirty secret is the gerrit hosting gerrit isn't really gerrit! | 17:16 |
| clarkb | that was a mouthfull | 17:16 |
| bswartz | o_O | 17:16 |
| clarkb | they use guice to patch in all the googly bits | 17:17 |
| clarkb | (its the only real reason guice is required in gerrit too since the open source version doesn't really have any alternative implementations for anything) | 17:17 |
| *** skelso has joined #openstack-infra | 17:18 | |
| mtreinish | clarkb, fungi, pabelanger, mordred: if you get a sec I've got a small stack of firehose patches: https://review.openstack.org/#/q/status:open+branch:master+topic:more-firehose | 17:19 |
| clarkb | pabelanger: commented | 17:19 |
| clarkb | bswartz: there was a write up on their mailing list of all the stuff that is replaced by google if you want to go dig that up | 17:19 |
| bswartz | I don't care that deeply -- it's just sad that gerrit is opensource-but-not-really ala Chrome (vs Chromium) | 17:21 |
| pabelanger | clarkb: should I just drop the serveralias section for 8080? | 17:21 |
| clarkb | pabelanger: ya I think we don't use that anymore | 17:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Move proxy reverse cache to port 8080 https://review.openstack.org/451878 | 17:22 |
| pabelanger | clarkb: done | 17:22 |
| clarkb | pabelanger: and maybe just ninja apply that to running server and see if it breaks too terribly? | 17:23 |
| pabelanger | clarkb: Ya, was going to do that | 17:23 |
| *** kzaitsev_mb has quit IRC | 17:26 | |
| pabelanger | clarkb: http://mirror.regionone.infracloud-vanilla.openstack.org:8080/rdo/centos7/puppet-passed-ci/ | 17:28 |
| pabelanger | rewrite rules | 17:28 |
| *** armax has quit IRC | 17:30 | |
| *** rbuzatu has joined #openstack-infra | 17:30 | |
| pabelanger | clarkb: just need to clean up: http://mirror.regionone.infracloud-vanilla.openstack.org:8080/ | 17:30 |
| pabelanger | will do that shortly | 17:30 |
| *** david-lyle__ is now known as david-lyle | 17:30 | |
| *** skelso has quit IRC | 17:31 | |
| *** jerryz has joined #openstack-infra | 17:31 | |
| *** armax has joined #openstack-infra | 17:31 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Move proxy reverse cache to port 8080 https://review.openstack.org/451878 | 17:34 |
| pabelanger | clarkb: fungi: ^okay, this should be what we need for caching | 17:35 |
| *** chlong has quit IRC | 17:36 | |
| *** signed8bit_Zzz is now known as signed8b_ | 17:36 | |
| clarkb | rajinir: is that the account associated with this thread http://lists.openstack.org/pipermail/openstack-infra/2017-March/005261.html ? | 17:37 |
| clarkb | pabelanger: that was tested? | 17:37 |
| pabelanger | clarkb: yes, running it local | 17:37 |
| rajinir | clarkb: yes we fixed the problem | 17:38 |
| clarkb | pabelanger: +2 lgtm | 17:38 |
| pabelanger | danke! | 17:39 |
| clarkb | rajinir: ok give me a sec and I can reenable the account. Will you be around to watch it to confirm that it is fixed? | 17:39 |
| rajinir | clarkb: yes | 17:39 |
| *** neiljerram has quit IRC | 17:40 | |
| fungi | clarkb: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2017-03-27.log.html#t2017-03-27T13:34:12 is roughly where i looked it up to close connections if you need the id (19339) | 17:40 |
| *** rbrndt has quit IRC | 17:40 | |
| *** rhallisey has quit IRC | 17:41 | |
| *** rhallisey has joined #openstack-infra | 17:41 | |
| clarkb | rajinir: ok all done | 17:41 |
| clarkb | fungi: oh I just dug it up myself | 17:41 |
| rajinir | clarkb: thank you | 17:41 |
| *** SumitNaiksatam has joined #openstack-infra | 17:42 | |
| *** neiljerram has joined #openstack-infra | 17:43 | |
| pabelanger | clarkb: so, how do we control the size of the CacheRoot? | 17:43 |
| pabelanger | if I understand, we allow 100M files | 17:43 |
| pabelanger | but I don't see a cap | 17:43 |
| fungi | pabelanger: probably with a cinder volume/lvm? | 17:43 |
| clarkb | pabelanger: fungi correct, apache has no functionality for that | 17:43 |
| clarkb | so its fs based | 17:43 |
| pabelanger | okay | 17:43 |
| fungi | i didn't see an option to limit the amount of space the cache is allowed to consume, which is why i suggested keeping an eye on disk utilization | 17:43 |
| clarkb | you could set up a quota for the www-data user | 17:43 |
| pabelanger | what about expiring the content, how does that work? | 17:44 |
| *** beslemon_ has quit IRC | 17:44 | |
| fungi | by age | 17:44 |
| rajinir | clarkb: It works | 17:44 |
| clarkb | this is also why I put it in /opt as on some of our servers at least that is a separate fs | 17:44 |
| clarkb | fungi: pabelanger ya expiration happens based on http ttls and such | 17:44 |
| fungi | yeah, on lots of our mirrors /opt is just part of the rootfs though | 17:44 |
| clarkb | fungi: ya so not complete solution | 17:44 |
| clarkb | but also afs cache is its own fs so fallout should be relatively small | 17:45 |
| clarkb | (I hope as I say that and wonder) | 17:45 |
| pabelanger | okay, so we should figure out directory limits then | 17:45 |
| pabelanger | before having tripleo use it | 17:45 |
| fungi | fs quota enforcement does seem like a viable option, maybe, but given that it will probably apply to everything owned by www-data (logfiles too?) that could be a bit fiddly | 17:45 |
| pabelanger | I have a test job going, and I can see it our cache properly | 17:45 |
| pabelanger | 172.99.106.121 - - [30/Mar/2017:17:45:28 +0000] "GET /rdo/centos7/puppet-passed-ci/python-nova-tests-15.0.0-0.20170330115208.6bd67cd.el7.centos.noarch.rpm HTTP/1.1" 200 3961587 "-" "urlgrabber/3.10 yum/3.4.3" | 17:46 |
| clarkb | pabelanger: I'm not super worried because afs uses far less cache than we expected. Butalso I think only way to find out is to try it | 17:46 |
| pabelanger | clarkb: okay | 17:46 |
| pabelanger | fungi: do you mind adding https://review.openstack.org/#/c/451878 to your review pipeline | 17:47 |
| *** erikmwil_ has quit IRC | 17:48 | |
| clarkb | its using 73M right now | 17:48 |
| *** nadya has joined #openstack-infra | 17:48 | |
| *** nadya has quit IRC | 17:49 | |
| clarkb | pabelanger: and we are assuming there will be a fair bit of overlap in the resources requested ya? | 17:49 |
| clarkb | (otherwise cache liek this or squid doesn't make sense) | 17:49 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove ovb updates job from promotion pipeline https://review.openstack.org/451779 | 17:50 |
| pabelanger | clarkb: I am not sure. I expect some of there data to overlap | 17:50 |
| pabelanger | clarkb: but, rdoproject.org now has rsync open too | 17:50 |
| pabelanger | that guess us AFS things | 17:50 |
| pabelanger | but going to be a few days | 17:50 |
| fungi | pabelanger: approved, sorry. trying to focus on reviewing the zuulv3 secrets stack without losing context too often | 17:51 |
| pabelanger | fungi: no rush, and thank you | 17:51 |
| *** prateek has quit IRC | 17:51 | |
| *** prateek has joined #openstack-infra | 17:52 | |
| openstackgerrit | Sorin Sbârnea proposed openstack-infra/gerritbot master: Removed unreasonable banning of newer pbr versions https://review.openstack.org/451925 | 17:52 |
| *** mattfarina_ has joined #openstack-infra | 17:54 | |
| *** Hal has quit IRC | 17:54 | |
| *** trown has joined #openstack-infra | 17:55 | |
| *** jamesmcarthur_ has joined #openstack-infra | 17:55 | |
| *** dham1_ has joined #openstack-infra | 17:55 | |
| *** nikhil_ has joined #openstack-infra | 17:55 | |
| *** nikhil_ is now known as Guest23722 | 17:55 | |
| *** rguillebert_ has joined #openstack-infra | 17:55 | |
| *** TheJulia_ has joined #openstack-infra | 17:56 | |
| *** ameade_ has joined #openstack-infra | 17:56 | |
| *** mikap has joined #openstack-infra | 17:56 | |
| *** colettecello has joined #openstack-infra | 17:57 | |
| *** SamYaple_ has joined #openstack-infra | 17:57 | |
| *** hogepodge_ has joined #openstack-infra | 17:57 | |
| *** vdrok_ has joined #openstack-infra | 17:57 | |
| *** simonmcc_ has joined #openstack-infra | 17:57 | |
| *** johnsom_ has joined #openstack-infra | 17:57 | |
| *** baoli has quit IRC | 17:57 | |
| openstackgerrit | Merged openstack-infra/system-config master: Move proxy reverse cache to port 8080 https://review.openstack.org/451878 | 17:57 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Fix trailing slash on ProxyPass / ProxyPassReverse https://review.openstack.org/451927 | 17:57 |
| *** baoli has joined #openstack-infra | 17:58 | |
| *** Marx314 has quit IRC | 17:59 | |
| *** Ng has quit IRC | 17:59 | |
| *** arif-ali has quit IRC | 17:59 | |
| *** jd_ has quit IRC | 17:59 | |
| *** rm_work has quit IRC | 17:59 | |
| *** ameade has quit IRC | 17:59 | |
| *** johnsom has quit IRC | 17:59 | |
| *** masayukig[m] has quit IRC | 17:59 | |
| *** nikhil has quit IRC | 17:59 | |
| *** rguillebert has quit IRC | 17:59 | |
| *** TheJulia has quit IRC | 17:59 | |
| *** dham1 has quit IRC | 17:59 | |
| *** mika has quit IRC | 17:59 | |
| *** zeroDivisible has quit IRC | 17:59 | |
| *** phschwartz has quit IRC | 17:59 | |
| *** clarkb has quit IRC | 17:59 | |
| *** gothicmindfood has quit IRC | 17:59 | |
| *** Krenair has quit IRC | 17:59 | |
| *** vdrok has quit IRC | 17:59 | |
| *** jamesmcarthur has quit IRC | 17:59 | |
| *** jlk has quit IRC | 17:59 | |
| *** hogepodge has quit IRC | 17:59 | |
| *** simonmcc has quit IRC | 17:59 | |
| *** trown|lunch has quit IRC | 17:59 | |
| *** mordred has quit IRC | 17:59 | |
| *** mattfarina has quit IRC | 17:59 | |
| *** hughsaunders has quit IRC | 17:59 | |
| *** SamYaple has quit IRC | 17:59 | |
| *** mikap is now known as mika | 17:59 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Pass source to project instantiations https://review.openstack.org/451596 | 17:59 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add a project index to Tenant https://review.openstack.org/451597 | 17:59 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Use new tenant project index for zuul role refs https://review.openstack.org/451928 | 17:59 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove unused Tenant.getRepo method https://review.openstack.org/451929 | 17:59 |
| *** jd_ has joined #openstack-infra | 18:00 | |
| *** neiljerram has quit IRC | 18:00 | |
| *** jamesmcarthur_ is now known as jamesmcarthur | 18:00 | |
| *** masayukig[m] has joined #openstack-infra | 18:00 | |
| *** hughsaunders has joined #openstack-infra | 18:00 | |
| *** rm_work has joined #openstack-infra | 18:00 | |
| *** baoli has quit IRC | 18:00 | |
| *** vdrok_ is now known as vdrok | 18:00 | |
| *** dham1_ is now known as dham1 | 18:00 | |
| *** jlk has joined #openstack-infra | 18:00 | |
| *** rguillebert_ is now known as rguillebert | 18:00 | |
| *** baoli has joined #openstack-infra | 18:00 | |
| *** jlk has quit IRC | 18:00 | |
| *** jlk has joined #openstack-infra | 18:00 | |
| *** phschwartz has joined #openstack-infra | 18:00 | |
| *** arif-ali has joined #openstack-infra | 18:00 | |
| *** hogepodge_ is now known as hogepodge | 18:00 | |
| *** prateek has quit IRC | 18:00 | |
| *** clarkb has joined #openstack-infra | 18:00 | |
| *** Guest23722 is now known as nikhil | 18:01 | |
| *** johnsom_ is now known as johnsom | 18:01 | |
| *** ameade_ is now known as ameade | 18:01 | |
| *** simonmcc_ is now known as simonmcc | 18:01 | |
| *** mordred has joined #openstack-infra | 18:01 | |
| *** zeroDivisible has joined #openstack-infra | 18:01 | |
| *** chlong has joined #openstack-infra | 18:02 | |
| *** neiljerram has joined #openstack-infra | 18:02 | |
| *** SamYaple_ has quit IRC | 18:03 | |
| *** SamYaple has joined #openstack-infra | 18:03 | |
| *** skelso has joined #openstack-infra | 18:03 | |
| *** Krenair has joined #openstack-infra | 18:04 | |
| openstackgerrit | Merged openstack-infra/shade master: Update tox build settings https://review.openstack.org/451312 | 18:05 |
| *** eliqiao has quit IRC | 18:05 | |
| *** eliqiao has joined #openstack-infra | 18:06 | |
| *** colettecello has quit IRC | 18:10 | |
| *** gothicmindfood has joined #openstack-infra | 18:10 | |
| openstackgerrit | Lubosz Kosnik (diltram) proposed openstack-infra/project-config master: Enable LXD test on Pike+ https://review.openstack.org/451935 | 18:10 |
| clarkb | pabelanger: probalby the next step is get your chagne in to tripleoci when we acn watch it then pay attention to disk usage in /opt/apache_cache over the next few days | 18:11 |
| clarkb | pabelanger: and while thats happening we can also poke at adding afs mirroring | 18:11 |
| pabelanger | clarkb: ya, I am talking with EmilienM about that now. We are going to pick 1 job and see how it does | 18:11 |
| *** chlong has quit IRC | 18:11 | |
| pabelanger | monitor HDD, etc | 18:11 |
| pabelanger | clarkb: ya, I'm going to setup rsync stuff also | 18:12 |
| pabelanger | Repo-baseurl : http://mirror.regionone.infracloud-vanilla.openstack.org:8080/rdo/centos7/puppet-passed-ci/ | 18:12 |
| pabelanger | from: http://logs.openstack.org/91/451891/2/check/gate-puppet-openstack-integration-4-scenario002-tempest-centos-7/24cc7ae/logs/repolist.txt.gz | 18:13 |
| pabelanger | and job passed | 18:13 |
| clarkb | so on a small scale it works :) | 18:13 |
| pabelanger | looks like | 18:13 |
| pabelanger | and, rdos RPms are less then 100Mb too | 18:13 |
| clarkb | and if there is a lot of overlap we shouldn't end up using too much disk | 18:13 |
| clarkb | we may have to play with timeouts though (if backend doesn't set a ttl for example) | 18:14 |
| *** john-davidge has quit IRC | 18:14 | |
| clarkb | pabelanger: https://httpd.apache.org/docs/2.4/mod/mod_cache.html#cachemaxexpire default actually looks fine there | 18:15 |
| clarkb | so at worst we will have a days worth of things cached | 18:15 |
| clarkb | er I guess 2 days | 18:15 |
| clarkb | depending on how thinsg overlap | 18:15 |
| fungi | mondays might get slow initially if jobs don't request those files over the weekend | 18:16 |
| fungi | but probably not noticeably if so | 18:16 |
| clarkb | ya I'm more curious to see if this combats the jobs failuer at all | 18:16 |
| *** skelso has quit IRC | 18:18 | |
| pabelanger | does apache cache this is some sort of atmoic way? Meaning, if we backed this to AFS, and infracloud-vanilla cached something, would chocolate service it up properly? | 18:18 |
| *** electrofelix has quit IRC | 18:18 | |
| *** skelso has joined #openstack-infra | 18:18 | |
| pabelanger | or would the hashing be different | 18:18 |
| clarkb | I wouldn't trust sharing a cache across hosts, we set up a lock file on this host just to deal with the processes there | 18:19 |
| *** felipemonteiro has quit IRC | 18:19 | |
| openstackgerrit | Merged openstack-dev/pbr master: Initialize sphinx config using application instead of config https://review.openstack.org/448455 | 18:19 |
| clarkb | I think if you AFS'd it would be the normal way and rely on AFS's caching mechanisms | 18:19 |
| clarkb | you can remove the lock entirely but that has thundering herd issues apparently | 18:20 |
| pabelanger | ah | 18:21 |
| pabelanger | http://mirror.regionone.infracloud-chocolate.openstack.org:8080/rdo/f25/report.html updated correctly | 18:21 |
| pabelanger | going to remove other mirrors from emergency | 18:21 |
| EmilienM | it reminds me Puppet CI uses rdo-qemu-ev/x86_64 from centos.org | 18:21 |
| EmilienM | I need to change that | 18:21 |
| pabelanger | fungi: clarkb: https://review.openstack.org/#/c/451927/ is a bug fix with html things for RDO | 18:22 |
| pabelanger | http://mirror.regionone.infracloud-chocolate.openstack.org:8080/rdo/f25 | 18:22 |
| pabelanger | as an example of failure | 18:22 |
| clarkb | pabelanger: isn't the issue that the html itself has rooted urls? | 18:22 |
| EmilienM | pabelanger: I think "rdo-qemu-ev/x86_64" and "updates/7/x86_64" should be fixed at an image level | 18:22 |
| EmilienM | pabelanger: oh wait, that's impossible, since images are the same for cloud providers | 18:23 |
| *** Ng has joined #openstack-infra | 18:23 | |
| EmilienM | I find it ugly to remove the files and recreate them again | 18:23 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Fix typo in cloud number https://review.openstack.org/450442 | 18:23 |
| pabelanger | clarkb: the urls are: <a href="70/a5/70a5526c00a83546a945333d6308a775303f95f0_1c6d6691">repo</a> | 18:24 |
| *** d0ugal has quit IRC | 18:24 | |
| clarkb | pabelanger: oh those are the urls I am talking about | 18:24 |
| pabelanger | http://mirror.regionone.infracloud-chocolate.openstack.org:8080/rdo/f25/report.html | 18:24 |
| clarkb | the ones at http://mirror.regionone.infracloud-chocolate.openstack.org:8080/rdo/ | 18:24 |
| pabelanger | clarkb: ya, we cannot fix them | 18:25 |
| pabelanger | need to update rdo side | 18:25 |
| pabelanger | report.html, I think we need to fix, because it is in a sub directory | 18:25 |
| pabelanger | which rdo dynamically builds | 18:25 |
| clarkb | ya I +2'd | 18:25 |
| *** adrian_otto has quit IRC | 18:28 | |
| *** rbuzatu has quit IRC | 18:30 | |
| *** rbuzatu has joined #openstack-infra | 18:30 | |
| *** rbuzatu_ has joined #openstack-infra | 18:37 | |
| clarkb | EmilienM: re os-cloud-config removal/deprecation is it used by any old branches still? wondering if I can remove it from devstack-gate's project list right now | 18:37 |
| EmilienM | clarkb: maybe but I will investigate before | 18:37 |
| *** rbuzatu has quit IRC | 18:38 | |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: networking-bagpipe: add an experimental fullstack job https://review.openstack.org/451897 | 18:38 |
| clarkb | gcb: Hello, I'd like to take the next step in removing oslo libraries from the devstack-gate PROJECTS list because we install them from pypi by default not from the source repos. We have machinery in place to automatically add libraries into the PROJECTS list if they are being tested as part of the LIBRARIES_FROM_GIT stuff. Is nowish a good time to do that? anything else I can do to help you and oslo | 18:40 |
| clarkb | udnerstand the change? | 18:40 |
| pabelanger | clarkb: do you know how often we expire the afs cache? do we? | 18:40 |
| *** erikmwil_ has joined #openstack-infra | 18:41 | |
| clarkb | pabelanger: my understanding of it is that afs servers tell clients when their cached content is no longer valid, then if the client needs the file again it pulls from the server and recaches | 18:41 |
| clarkb | so its not time based expiration aiui | 18:41 |
| clarkb | (but maybe it does that too) | 18:42 |
| pabelanger | clarkb: ya, I see afs is up to about 40GB on this mirror | 18:42 |
| pabelanger | was curious if we expire any of the old data or not | 18:42 |
| clarkb | old data that has been replaced yes, but old data that is just old I don't know | 18:43 |
| pabelanger | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=3172&rra_id=all | 18:43 |
| pabelanger | show growth over time | 18:43 |
| clarkb | dims: harlowja you may both have thoughts on my question to gcb above | 18:43 |
| clarkb | pabelanger: for repos where things update fairly often (pypi/wheel with new versions coming in all the time) it may make sense to try and add time based expiration too | 18:44 |
| clarkb | so that we don't cache the 5 month old version of lxml that we used 5 months ago | 18:44 |
| harlowja | who what, lol | 18:44 |
| pabelanger | jeblair: can you help be understand, in afs, would we ever reclaim local HDDs if old content is never used again? ^curious why the cacti graph continues to grow | 18:44 |
| pabelanger | clarkb: agree | 18:44 |
| dims | clarkb : +1 to try yanking and see what fails :) | 18:45 |
| clarkb | dims: harlowja gcb let me push up a change that I can WIP so its a bit more concrete too | 18:45 |
| openstackgerrit | Merged openstack-infra/system-config master: Fix trailing slash on ProxyPass / ProxyPassReverse https://review.openstack.org/451927 | 18:45 |
| *** erikmwilson has joined #openstack-infra | 18:45 | |
| fungi | pabelanger: as i understand, it's handled more the way a filesystem cache is... basically you purge the least recently accessed data if you're running up against the max cache size and need to cache something new | 18:46 |
| clarkb | pabelanger: even with the fixes to the proxy stuff we are still at only 3 lines per backend so if this works it should be simple to add for eg docker | 18:46 |
| *** erikmwi__ has joined #openstack-infra | 18:46 | |
| pabelanger | fungi: Oh, so maybe some general maintenance we should be doing? | 18:47 |
| *** rbrndt has joined #openstack-infra | 18:47 | |
| pabelanger | clarkb: yes, I am hoping too | 18:47 |
| pabelanger | however, a little larger then 100MB files I would guess | 18:48 |
| clarkb | pabelanger: ya maybe, I don't know how well the layering chunks that up or if at all | 18:48 |
| pabelanger | same | 18:48 |
| fungi | pabelanger: what sort of maintenances? we tell afs how much space it's allowed to consume with its cache, so it will grow over time until it reaches that limit and then level off | 18:48 |
| *** erikmwil_ has quit IRC | 18:48 | |
| fungi | you can adjust that with the fs setcachesize command | 18:49 |
| pabelanger | fungi: ya, the graph above in cacti. Why are we up to 40GB of data. I would have expected it to shrink over time, assuming old data (pypi package) is not accessed any more | 18:49 |
| clarkb | pabelanger: I think fuingi is saying the cache claening wouldn't happen until you hit the limit | 18:49 |
| pabelanger | ya see that now | 18:50 |
| clarkb | until then its just going to keep as much as possible | 18:50 |
| pabelanger | so, I guess we don't have a limit | 18:50 |
| *** erikmwilson has quit IRC | 18:50 | |
| fungi | adjust how much you want it to be able to use with http://docs.openafs.org/Reference/1/fs_setcachesize.html | 18:50 |
| fungi | we do have a limit, i thought | 18:50 |
| *** yamahata has quit IRC | 18:50 | |
| pabelanger | checking our configs | 18:51 |
| pabelanger | 500000 is our cache_size currently | 18:52 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate master: Remove oslo projects from PROJECTS list https://review.openstack.org/451948 | 18:52 |
| clarkb | gcb: dims harlowja ^ thats roughly what I had in mind, happy to do it when you have a preference though | 18:53 |
| *** kzaitsev_mb has joined #openstack-infra | 18:53 | |
| clarkb | the motivation here is reducing job runtime and reducing load on the git servers | 18:53 |
| harlowja | kk | 18:53 |
| harlowja | seems fair | 18:53 |
| dims | +1 clarkb | 18:54 |
| fungi | pabelanger: yeah, assuming the unit there is kilobyte blocks like the fs_setcachesize manpage suggests, then we shouldn't be occupying more than half a gigabyte with the afs cache | 18:55 |
| *** TheJulia_ is now known as TheJulia | 18:56 | |
| *** Rockyg has joined #openstack-infra | 18:56 | |
| pabelanger | okay, I'll read more up on it tonight | 18:56 |
| pabelanger | not that it is an issue currently, just curious | 18:56 |
| *** kzaitsev_mb has quit IRC | 18:58 | |
| *** baoli has quit IRC | 18:59 | |
| clarkb | also space x launch later today | 18:59 |
| *** chlong has joined #openstack-infra | 19:00 | |
| pabelanger | clarkb: YES | 19:01 |
| pabelanger | thanks you | 19:01 |
| clarkb | my strategy is tell as many people as I can then hope they remind me just before hand when I inevitably forget | 19:02 |
| clarkb | :) | 19:02 |
| *** kzaitsev_mb has joined #openstack-infra | 19:04 | |
| fungi | pabelanger: according to http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n652 it's 50000000 which makes it roughly 47.7gib (you were missing some zeroes) so move along, nothing to see here? ;) | 19:04 |
| SamYaple | can I get an ack on https://review.openstack.org/#/c/450473/ to get my gates going? | 19:06 |
| clarkb | SamYaple: I can take a look | 19:06 |
| *** SumitNaiksatam has quit IRC | 19:06 | |
| *** chlong has quit IRC | 19:07 | |
| *** beslemon has joined #openstack-infra | 19:07 | |
| pabelanger | fungi: Ha, we default puppet-openafs to 500000 but override | 19:07 |
| pabelanger | okay, cool | 19:07 |
| pabelanger | so lets see what happens when we hit 50G :) | 19:07 |
| SamYaple | thanks clarkb | 19:07 |
| fungi | pabelanger: so when we start to bump up against that, i would expect it to stop growing | 19:08 |
| clarkb | SamYaple: would it be possible to have more descriptive template names than gate-{name}-{node} and gate-{name}-ubuntu-xenial ? | 19:08 |
| SamYaple | clarkb: i can put whatever would be useful there. but i dont have an idea for a more useful name | 19:10 |
| pabelanger | fungi: agree, thanks for helping me understand | 19:10 |
| fungi | any time! | 19:10 |
| fungi | glad i occasionally make sense to someone | 19:10 |
| clarkb | SamYaple: anything to scope it to loci, gate-loci-{name}-{node} may be redundant in the end result but at least then the tmeplate itself is more unique | 19:10 |
| SamYaple | clarkb: oh, the {name} is loci-* | 19:11 |
| SamYaple | it includes loci | 19:11 |
| SamYaple | so that means it will show up as gate-loci-loci-keystone-ubuntu-xenial? | 19:11 |
| clarkb | ya... its redundant in the end result, but the template itself should be more uniquely identified too imo | 19:11 |
| clarkb | you could also maybe do gate-loci-{component}-{node} but that becomes more work to plumb | 19:12 |
| SamYaple | i couldnt figure out how to do that | 19:12 |
| SamYaple | ive spent like 20 hours on this patch | 19:12 |
| SamYaple | mainly because i dont understandp roject-config that well | 19:13 |
| *** xyang1 has joined #openstack-infra | 19:13 | |
| pabelanger | fungi: gah, we again on stackalytcs.o.o | 19:13 |
| clarkb | right, I;m suggesting the simple (but redundant in end result) option | 19:13 |
| SamYaple | i really dont care what it is named at this point, i just want to start commiting things again | 19:13 |
| fungi | pabelanger: i figured | 19:13 |
| clarkb | but also brainstorming other ideas | 19:13 |
| *** nadya has joined #openstack-infra | 19:14 | |
| SamYaple | clarkb: surely i can just rename then when the project gets moving again? | 19:14 |
| SamYaple | clarkb: we know we have other gates needed log term, but right now we arent commiting anything because no gate | 19:14 |
| clarkb | SamYaple: the problem isn't so much your project its when the next project comes along and goes "perfect I want gate-{myproject}-{node} that will just magically work for my project won't it?" | 19:15 |
| *** MasterOfBugs has quit IRC | 19:15 | |
| *** nadya has quit IRC | 19:15 | |
| SamYaple | haha i think i copied this from puppet-openstack for what its worth | 19:15 |
| SamYaple | maybe charms | 19:15 |
| SamYaple | none of this is original, i just copy/pasted until it passed the gate | 19:16 |
| pabelanger | fungi: processor is running on stackalytics.o.o too. Appears things are working back to normal | 19:16 |
| pabelanger | going to take some time to input missing data however | 19:16 |
| fungi | <mr_burns>excellent</mr_burns> | 19:16 |
| SamYaple | clarkb: yea i got this from charms | 19:16 |
| clarkb | I'm not seeing it in the charm jobs (they are all scoped to what they do, pep8, python, etc) | 19:17 |
| clarkb | thats all I'm asking for, is something that identifies the template more uniquely | 19:17 |
| SamYaple | oh im misunderstood that point | 19:18 |
| SamYaple | would gate={name}-builder-{node} work for you? | 19:18 |
| SamYaple | thats the closest i can come up with right now? | 19:18 |
| SamYaple | because its just building a docker image at this point | 19:18 |
| clarkb | maybe image-builder? but ya that would work | 19:19 |
| SamYaple | we will need functional testing, but we can test that in the same gate initially and iterate on it | 19:19 |
| SamYaple | ok | 19:19 |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add initial gate for LOCI https://review.openstack.org/450473 | 19:20 |
| *** neiljerram has quit IRC | 19:24 | |
| *** yamahata has joined #openstack-infra | 19:25 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Rename LOCI template https://review.openstack.org/451956 | 19:26 |
| *** neiljerram has joined #openstack-infra | 19:26 | |
| SamYaple | heh jeblair. its so simple when someone else points out how to do it. tunnel vision is a real thing | 19:28 |
| jeblair | SamYaple: np; though that's not complete yet; i still need to update zuul | 19:28 |
| *** MasterOfBugs has joined #openstack-infra | 19:29 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Rename LOCI template https://review.openstack.org/451956 | 19:30 |
| jeblair | clarkb: i think it's probably okay to use 'gate-{name}-ubuntu-xenial' in zuul? there's no namespace conflict there | 19:30 |
| clarkb | jeblair: but wouldnt it need to match jjb? | 19:30 |
| jeblair | clarkb: it will | 19:30 |
| jeblair | clarkb: (referring to my change -- 451956) | 19:31 |
| SamYaple | is that a regex..? | 19:31 |
| SamYaple | i thought it was some variable passing stuff | 19:31 |
| clarkb | will look as soon as I finish lunch | 19:31 |
| jeblair | SamYaple: no, in zuul it's only the {name} substitution; no other variables like in jjb. | 19:32 |
| jeblair | SamYaple: but in jjb, we can add whatever variables we want (and almost wherever we want) | 19:32 |
| *** florianf has quit IRC | 19:33 | |
| SamYaple | i believe i understand what you are saying, but i lack understanding of the relationship between jjb and zuul. so I will probably stay confused here | 19:33 |
| *** eharney has quit IRC | 19:34 | |
| *** baoli has joined #openstack-infra | 19:37 | |
| *** beslemon has quit IRC | 19:37 | |
| *** askb has joined #openstack-infra | 19:42 | |
| fungi | from a configuration standpoint they're entirely separate, except insofar as the resulting job names referenced in zuul/layout.yaml (post expansion) need to match the expanded names of jobs defined in jenkins/jobs/* | 19:43 |
| fungi | historically, one set of configuration was used to configure zuul (telling it when to run jobs), and the other configuring jenkins (telling it what those jobs are) | 19:44 |
| SamYaple | ok. that explains a good bit. | 19:45 |
| fungi | it's muddied by us replacing jenkins with a launcher service in zuul v2.5, which is why the major refactoring in zuul v3 will allow us to get rid of that separate distinction entirely and define jobs together with the instructions on when they should be run | 19:46 |
| *** baoli has quit IRC | 19:47 | |
| * SamYaple stops trying to learn project-config and waits for zuulv3 | 19:47 | |
| *** yolanda has quit IRC | 19:47 | |
| * jeblair goes back to working on zuulv3 | 19:47 | |
| *** rbuzatu_ has quit IRC | 19:48 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add initial gate for LOCI https://review.openstack.org/450473 | 19:49 |
| SamYaple | w00t | 19:49 |
| SamYaple | thanks all | 19:49 |
| *** baoli has joined #openstack-infra | 19:49 | |
| *** eharney has joined #openstack-infra | 19:49 | |
| *** mriedem has joined #openstack-infra | 19:52 | |
| *** baoli has quit IRC | 19:54 | |
| clarkb | pabelanger: EmilienM have we decided on a job to use the caching proxy? | 19:58 |
| clarkb | all of the puppetry should be getting applied now if not already | 19:59 |
| pabelanger | clarkb: yes, we are iterating on https://review.openstack.org/#/c/451891/ right now | 19:59 |
| EmilienM | and I'm doing tripleo in same time (for experimental) | 19:59 |
| pabelanger | EmilienM: hold off on tripleo for now | 19:59 |
| * clarkb looks | 19:59 | |
| pabelanger | lets see how much data we use on pike master | 19:59 |
| EmilienM | ok | 19:59 |
| fungi | yeah, want to start slowly so we have some idea of the filesystem demands this is going to place on the mirror servers | 20:00 |
| fungi | giving us time to make adjustments there if needed | 20:00 |
| EmilienM | pabelanger: my tripleo patch is just for experiment. We won't merge it. | 20:01 |
| EmilienM | and it's 4 jobs running | 20:01 |
| fungi | well, each new job run against that proxy is likely to request slightly different files, and increase the size of the on-disk cache in different ways | 20:01 |
| clarkb | EmilienM: pabelanger btw the nodepool check there isn't really sufficient to know if you should use a mirror (thinking of third party ci), I solved that in https://review.openstack.org/#/c/451492/8/stack.sh by checking if the mirror is configured for use generally | 20:01 |
| clarkb | pabelanger: thats also one way we could fix the ping test in d-g too | 20:01 |
| pabelanger | clarkb: ya, that could be optimized too | 20:03 |
| *** mriedem has quit IRC | 20:04 | |
| *** mriedem has joined #openstack-infra | 20:04 | |
| *** baoli has joined #openstack-infra | 20:05 | |
| *** sree has joined #openstack-infra | 20:06 | |
| *** e0ne has joined #openstack-infra | 20:07 | |
| *** mriedem has quit IRC | 20:07 | |
| openstackgerrit | Dirk Mueller proposed openstack/diskimage-builder master: DNM: give me more debug https://review.openstack.org/450800 | 20:08 |
| *** mriedem has joined #openstack-infra | 20:09 | |
| *** jordanP_ has quit IRC | 20:10 | |
| *** e0ne has quit IRC | 20:11 | |
| *** sree has quit IRC | 20:11 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test new ovb with images building https://review.openstack.org/445228 | 20:13 |
| *** ramishra has quit IRC | 20:14 | |
| openstackgerrit | Dirk Mueller proposed openstack/diskimage-builder master: DNM: give me more debug https://review.openstack.org/450800 | 20:17 |
| *** adrianofr_ has quit IRC | 20:19 | |
| *** eharney has quit IRC | 20:19 | |
| *** mriedem has quit IRC | 20:22 | |
| *** mriedem has joined #openstack-infra | 20:22 | |
| *** mriedem has quit IRC | 20:23 | |
| *** mriedem has joined #openstack-infra | 20:23 | |
| *** adrian_otto has joined #openstack-infra | 20:24 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Pass source to project instantiations https://review.openstack.org/451596 | 20:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add a project index to Tenant https://review.openstack.org/451597 | 20:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove unused Tenant.getRepo method https://review.openstack.org/451929 | 20:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Use new tenant project index for zuul role refs https://review.openstack.org/451928 | 20:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add source to project and remove unused tenant attrs https://review.openstack.org/451969 | 20:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fully qualify project configuration names https://review.openstack.org/451970 | 20:25 |
| *** john-davidge has joined #openstack-infra | 20:27 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Mirror rdo centos7 repo to AFS https://review.openstack.org/451972 | 20:28 |
| *** arijr has joined #openstack-infra | 20:30 | |
| *** mriedem has quit IRC | 20:30 | |
| *** mriedem has joined #openstack-infra | 20:31 | |
| *** mriedem has quit IRC | 20:31 | |
| *** john-davidge has quit IRC | 20:31 | |
| *** mriedem has joined #openstack-infra | 20:32 | |
| *** sbezverk has joined #openstack-infra | 20:32 | |
| *** Hal has joined #openstack-infra | 20:33 | |
| *** mriedem has quit IRC | 20:34 | |
| EmilienM | pabelanger: see your code, it looks like it works: http://logs.openstack.org/91/451891/8/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/3b2bb53/logs/repolist.txt.gz | 20:34 |
| EmilienM | Repo-baseurl : http://mirror.regionone.osic-cloud1.openstack.org:8080/rdo/centos7/c1/7c/c17c8d67b18850e27aea303584d31b7fb1d9c6b7_02d8fdf6/ | 20:34 |
| *** mriedem has joined #openstack-infra | 20:34 | |
| pabelanger | EmilienM: yes, so that is also going to fix broke package downloads you were seeing too | 20:35 |
| pabelanger | EmilienM: if that job passes, you can merge | 20:35 |
| EmilienM | pabelanger: ack | 20:35 |
| pabelanger | then, we'll monitor and see results and how much storage is used | 20:35 |
| EmilienM | mwhahaha: fyi https://review.openstack.org/#/c/451891 | 20:36 |
| *** kzaitsev_mb has quit IRC | 20:36 | |
| mwhahaha | k | 20:36 |
| *** chlong has joined #openstack-infra | 20:36 | |
| openstackgerrit | Sorin Sbârnea proposed openstack-infra/gerritlib master: Removed pbr<2.0.0 to resolve depencency conflicts https://review.openstack.org/451975 | 20:37 |
| *** asselin has joined #openstack-infra | 20:37 | |
| *** kgiusti has quit IRC | 20:38 | |
| pabelanger | EmilienM: mwhahaha: next will be gem mirrors | 20:38 |
| *** aeng_ has joined #openstack-infra | 20:38 | |
| EmilienM | pabelanger: dude I want to hug you now | 20:38 |
| *** bnemec has quit IRC | 20:38 | |
| clarkb | EmilienM: mwhahaha and do let us know if you see job fail using that proxy, we'll want to look into why | 20:39 |
| *** asselin_ has quit IRC | 20:39 | |
| clarkb | or specifically fail using the proxy, if they fail because openstack is broken then thats ok :) | 20:39 |
| *** felipemonteiro has joined #openstack-infra | 20:39 | |
| EmilienM | for sure | 20:40 |
| EmilienM | I think what you did is a huge step forward | 20:40 |
| *** scarab_ has quit IRC | 20:41 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config master: Add functional job for k8s-cloud-provider with sudo privileges https://review.openstack.org/451977 | 20:41 |
| *** rossella_s has quit IRC | 20:42 | |
| openstackgerrit | Alfredo Moralejo proposed openstack-infra/tripleo-ci master: DNM Other test for 451445 https://review.openstack.org/451978 | 20:43 |
| EmilienM | pabelanger: let's talk about gem. Did your PR got merged? | 20:43 |
| *** rossella_s has joined #openstack-infra | 20:43 | |
| *** mriedem has quit IRC | 20:44 | |
| EmilienM | I don't see https://github.com/pabelanger/rubygems-mirror/commit/fceaa8ef7cf2f75d8850439c232322a41dc1dad1 proposed as PR yet | 20:44 |
| *** esberglu has quit IRC | 20:44 | |
| pabelanger | EmilienM: ya, I want to get apache syntax in place first | 20:45 |
| *** mriedem has joined #openstack-infra | 20:45 | |
| pabelanger | then update doc | 20:45 |
| *** trown is now known as trown|outtypewww | 20:45 | |
| pabelanger | I'll likely do that tomorrow | 20:45 |
| pabelanger | then submit a PR | 20:45 |
| EmilienM | k | 20:45 |
| *** dimtruck is now known as zz_dimtruck | 20:46 | |
| *** jamesden_ has quit IRC | 20:47 | |
| *** rfolco has quit IRC | 20:49 | |
| clarkb | pabelanger: so you're taking a path like /someproject and placing it at /s/so/someproject? do you need to check for projects that are one character long? | 20:50 |
| clarkb | I guess /s would just go to /s/s/s | 20:50 |
| pabelanger | clarkb: min would be: s.gem | 20:51 |
| pabelanger | err | 20:51 |
| pabelanger | s-1.gem | 20:51 |
| pabelanger | because filename include version number | 20:51 |
| *** mriedem has quit IRC | 20:52 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fully qualify project configuration names https://review.openstack.org/451970 | 20:52 |
| *** bnemec has joined #openstack-infra | 20:52 | |
| clarkb | pabelanger: so in that case it would go to /s/s-1/s-1.gem? got it | 20:52 |
| clarkb | er /s/s1/s-1.gem | 20:52 |
| pabelanger | ya, believe so | 20:53 |
| *** mriedem has joined #openstack-infra | 20:53 | |
| pabelanger | actually | 20:53 |
| *** skelso has quit IRC | 20:53 | |
| clarkb | also I failed second time too | 20:53 |
| pabelanger | s/s-/s-1.gem | 20:53 |
| clarkb | /s/s-/s-1.gem | 20:53 |
| clarkb | ya that | 20:53 |
| pabelanger | yup | 20:53 |
| pabelanger | so, we need a rewrite rule for apache to do that | 20:53 |
| pabelanger | because clients will always be /gem/foo.gem | 20:53 |
| EmilienM | would be ok to mirror http://yum.puppetlabs.com and http://apt.puppetlabs.com too? | 20:54 |
| pabelanger | and disk is /gem/f/fo/foo.gem | 20:54 |
| clarkb | EmilienM: we do ceph and mariadb so that should be fine too | 20:54 |
| EmilienM | ok I'll submit a patch, thanks | 20:54 |
| pabelanger | clarkb: we could do md5, but my understanding is we need to run an app in apache, since it doesn't support md5sum in vhosts directly | 20:56 |
| pabelanger | and not sure if we want to do that | 20:56 |
| clarkb | pabelanger: ya I think the setup you've done there is roughly what jeblair did for pypi | 20:58 |
| clarkb | and as long as no two character prefix becomes too popular we should be fine | 20:58 |
| clarkb | thats the benefit of md5, is it should giev us an even distribution | 20:58 |
| pabelanger | ya | 20:59 |
| *** sree has joined #openstack-infra | 21:01 | |
| *** rhallisey has quit IRC | 21:01 | |
| EmilienM | omg it's much faster it seems | 21:02 |
| EmilienM | see this patch from 2 days ago: https://review.openstack.org/#/c/450860/ - scenario001 took 42 min and now it takes 29 minutes: https://review.openstack.org/#/c/451891/ | 21:02 |
| *** esberglu has joined #openstack-infra | 21:02 | |
| clarkb | pabelanger: my hunch is that the language of software development project naming is evenly distributed well enough for our purposes | 21:03 |
| clarkb | pabelanger: we can always check dir entries after the fact and then write a paper on it | 21:03 |
| pabelanger | nice | 21:03 |
| EmilienM | mwhahaha, pabelanger ^ fyi | 21:03 |
| *** asselin has quit IRC | 21:03 | |
| *** asselin has joined #openstack-infra | 21:04 | |
| *** funzo has quit IRC | 21:04 | |
| mwhahaha | yea i would expect that we should see alot of improvement in times | 21:05 |
| *** sree has quit IRC | 21:05 | |
| *** dprince has quit IRC | 21:05 | |
| mwhahaha | especially since the cache is going to get populated real quick | 21:05 |
| pabelanger | Ya, hope it cuts down on recheck | 21:07 |
| *** jpich has quit IRC | 21:08 | |
| *** chlong has quit IRC | 21:09 | |
| EmilienM | we'll still have github repos, buildlogs.centos, rubygems (wip), puppetlabs repo (wip) and that's all I think | 21:09 |
| clarkb | EmilienM: whats buildlogs? | 21:10 |
| clarkb | and for github repos those are puppet modules that you grab from source? | 21:10 |
| EmilienM | clarkb: centos CDN | 21:10 |
| pabelanger | buildlogs should be fine | 21:10 |
| EmilienM | clarkb: yes, using r10k | 21:10 |
| clarkb | EmilienM: so its different than the centos repos we already host? | 21:11 |
| EmilienM | yeah buildlogs is reliable I think most of the time | 21:11 |
| pabelanger | I am told it is an CDN after all | 21:11 |
| EmilienM | yeah ^ | 21:11 |
| pabelanger | clarkb: ya, it is centos testing | 21:11 |
| pabelanger | we mirror stable | 21:11 |
| *** dansmith is now known as danased | 21:11 | |
| clarkb | I see | 21:11 |
| *** danased is now known as dansmith | 21:12 | |
| *** aysyd has quit IRC | 21:12 | |
| clarkb | for go related things we likely need some form of read only git repo hosting (as a downstream) anyways | 21:13 |
| clarkb | should consider the general case too | 21:13 |
| clarkb | r10k etc | 21:13 |
| * mwhahaha mirrors git.openstack.org to github.com to be mirroed back on git.cache.openstack.org | 21:15 | |
| *** ldnunes has quit IRC | 21:15 | |
| mordred | heh | 21:15 |
| mnaser | mirrorception | 21:15 |
| mordred | well - for go related things that are _ours_ we should consume them from git.o.o urls and not github urls ... but yes, we'll need to sort how to deal with go repos that are not ours and are hosted on github | 21:16 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config master: Add functional job for k8s-cloud-provider with sudo privileges https://review.openstack.org/451977 | 21:16 |
| mwhahaha | yea we suffer from that in puppet because we have external modules we pull from github | 21:16 |
| mordred | yup | 21:16 |
| clarkb | we do too, though I don't think we really see any errors from it | 21:16 |
| clarkb | since our job volume is low | 21:16 |
| mwhahaha | we do occasionally | 21:16 |
| mnaser | EmilienM / pabelanger: can we rely on something like this now? https://review.openstack.org/#/c/451891/8/run_tests.sh for kolla | 21:16 |
| clarkb | we've definitely seen it in the general case | 21:17 |
| clarkb | github fails a lot | 21:17 |
| mwhahaha | some days are worse than others | 21:17 |
| EmilienM | mnaser: no please don't, wait | 21:17 |
| mnaser | okay :) | 21:17 |
| EmilienM | mnaser: we're still experimenting | 21:17 |
| mnaser | no worries | 21:17 |
| openstackgerrit | Dirk Mueller proposed openstack-infra/nodepool master: [WIP]: Add opensuse 42.2 DIB testing https://review.openstack.org/450045 | 21:17 |
| clarkb | osic apache cache dir is 113M large | 21:17 |
| EmilienM | mnaser: we're evaluating if it works fine, and if it doesn't break anything. | 21:17 |
| mnaser | once that stuff is set in stone i'll switch things around for us | 21:17 |
| clarkb | which isn't too bad assuming that its relatively stable | 21:17 |
| mnaser | EmilienM ok sweet :) i look forward to faster builds! | 21:18 |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: networking-bagpipe: add an experimental fullstack job https://review.openstack.org/451897 | 21:18 |
| EmilienM | mnaser: once pabelanger gives the go to move tripleo forward using it, i'll ping you | 21:19 |
| mnaser | EmilienM great, thank you so much! | 21:19 |
| *** rbuzatu has joined #openstack-infra | 21:20 | |
| EmilienM | mnaser: I already prepared a patch for tripleo, in case you want to see how we're doing https://review.openstack.org/#/c/451938/ | 21:20 |
| EmilienM | mnaser: highly wip | 21:20 |
| *** zz_dimtruck is now known as dimtruck | 21:21 | |
| mnaser | EmilienM added myself to it, so i'll watch it till it merges and then bring that stuff over to kolla | 21:21 |
| *** neiljerram has quit IRC | 21:21 | |
| *** neiljerram has joined #openstack-infra | 21:23 | |
| zigo | pabelanger: Heya! Any news for your fix to the Stretch image generation? Or an ETA? (no stress, just asking ...) | 21:23 |
| *** rbuzatu has quit IRC | 21:24 | |
| openstackgerrit | Dirk Mueller proposed openstack-infra/nodepool master: [WIP]: Add opensuse 42.2 DIB testing https://review.openstack.org/450045 | 21:25 |
| pabelanger | zigo: no eta yet, aiming for next week currently | 21:26 |
| zigo | pabelanger: Ok. | 21:26 |
| pabelanger | mnaser: ya, hold off until we give the go ahead. we want to profile the usage of HDD space | 21:27 |
| *** tosky has quit IRC | 21:28 | |
| *** neiljerram has quit IRC | 21:29 | |
| *** neiljerram has joined #openstack-infra | 21:30 | |
| *** mriedem has quit IRC | 21:32 | |
| *** dave-mccowan has quit IRC | 21:32 | |
| *** mriedem has joined #openstack-infra | 21:32 | |
| *** mriedem has quit IRC | 21:33 | |
| *** mriedem has joined #openstack-infra | 21:33 | |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: networking-bagpipe: add an experimental fullstack job https://review.openstack.org/451897 | 21:36 |
| *** mriedem has quit IRC | 21:36 | |
| *** mriedem has joined #openstack-infra | 21:37 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config master: Add functional job for k8s-cloud-provider with sudo privileges https://review.openstack.org/451977 | 21:39 |
| *** mriedem has quit IRC | 21:40 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: New ovb playbook and settings https://review.openstack.org/431567 | 21:40 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config master: Add functional job for k8s-cloud-provider with sudo privileges https://review.openstack.org/451977 | 21:40 |
| *** mriedem has joined #openstack-infra | 21:40 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Explicitly set REVIEW_RELEASE on master https://review.openstack.org/451445 | 21:42 |
| *** mriedem has quit IRC | 21:43 | |
| *** mriedem has joined #openstack-infra | 21:43 | |
| clarkb | EmilienM: you use UCA always or is it optional? | 21:44 |
| EmilienM | UCA | 21:44 |
| clarkb | EmilienM: trying to figure out if most of the deployment projects are using UCA | 21:44 |
| EmilienM | but the mirror | 21:44 |
| EmilienM | I don't know any other repo with ubuntu packages | 21:44 |
| clarkb | EmilienM: well there is ubuntu itself | 21:44 |
| clarkb | but I guess that doesn't give you the newer releases of openstack | 21:45 |
| EmilienM | openstack packages are old | 21:45 |
| EmilienM | and Canonical recommand UCA | 21:45 |
| *** adrian_otto has quit IRC | 21:45 | |
| clarkb | ya, for our testing we've avoided it bceause it was broken in various ways on precise and trusty | 21:45 |
| clarkb | but xenial libvirt is unhappy so I have a change up to use UCA and trying to understand if anyone doesn't use UCA | 21:45 |
| clarkb | (sounds like most everyone is using UCA because of the newer openstack pacakges and they just happen t oget newer libvirt too | 21:46 |
| EmilienM | we use it in Puppet Ci and it works fine so far | 21:46 |
| EmilienM | it used to be unstable | 21:46 |
| EmilienM | but we complained :D | 21:46 |
| clarkb | EmilienM: my otherquestion then is UCA is per openstack release, you use pike on master? | 21:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Add support for $DO_BOOTSTRAP_SUBNODES https://review.openstack.org/437434 | 21:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Add generated nodepool key to authorized_keys https://review.openstack.org/437742 | 21:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Fix setting of node and node_private https://review.openstack.org/437433 | 21:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Update pip in DLRN venv https://review.openstack.org/438746 | 21:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Support running toci_gate_test-oooq.sh locally https://review.openstack.org/437431 | 21:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Stop dstat at the end of a job https://review.openstack.org/438745 | 21:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Guard adding bridge interface with $STABLE_RELEASE https://review.openstack.org/440838 | 21:48 |
| clarkb | EmilienM: osa said they use ocata on master (and thats what I have tested so far) | 21:48 |
| openstackgerrit | Sorin Sbârnea proposed openstack-infra/gerritlib master: Updated list of Python versions in tox https://review.openstack.org/451992 | 21:49 |
| EmilienM | clarkb: UCA provides repo per release | 21:49 |
| *** Rockyg has quit IRC | 21:50 | |
| *** jcoufal_ has joined #openstack-infra | 21:50 | |
| *** jcoufal_ has quit IRC | 21:50 | |
| EmilienM | clarkb: http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/xenial-proposed/ | 21:50 |
| clarkb | EmilienM: right, more specifically when do you change the repo you use | 21:50 |
| EmilienM | ah | 21:50 |
| EmilienM | when they provide it :D | 21:50 |
| clarkb | I think OSA lags because they don't want the churn of packages in pike until pike is releases | 21:50 |
| clarkb | so they are still on ocata | 21:50 |
| EmilienM | let me check what we're testing | 21:50 |
| clarkb | but do you just jump to pike as soon as its there? | 21:50 |
| EmilienM | clarkb: yeah | 21:50 |
| EmilienM | we test it first in our ci | 21:51 |
| EmilienM | and if it pass, boom | 21:51 |
| clarkb | gotcha | 21:51 |
| EmilienM | pike repo deploys nova 15.0.1-0ubuntu1~cloud0 | 21:51 |
| EmilienM | which is AFIK ocata | 21:51 |
| EmilienM | so they just copy/pasta things for now | 21:51 |
| EmilienM | and when m1 is out, they'll sync | 21:51 |
| EmilienM | and our CI will prob break :D | 21:52 |
| EmilienM | because things tend to break at first release (missing dependencies, etc) | 21:52 |
| EmilienM | our coverage is far better than canonical testing | 21:52 |
| mnaser | am i correct to assume that devstack-gate by default runs smoke tests for all projects | 21:52 |
| EmilienM | they don't test a lot of bits in OpenStack | 21:52 |
| mnaser | and then $project can tweak and run specific things? | 21:52 |
| *** jcoufal has quit IRC | 21:52 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Use custom network-isolation-absolute.yaml in ipv4 net-iso jobs https://review.openstack.org/422297 | 21:53 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Test with bonded public interface https://review.openstack.org/385562 | 21:53 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Add storage-mgmt network to compute nodes https://review.openstack.org/419049 | 21:53 |
| *** wolverineav has quit IRC | 21:53 | |
| clarkb | mnaser: by default it installs a cloud with devstack then runs the default tempest tox command | 21:53 |
| mordred | mnaser: yah. by default it runs devstack then runs tempest against that devstack | 21:53 |
| mordred | clarkb: jinx | 21:53 |
| mnaser | i'm trying to make the testing in kolla consistent to use tempest (rather than a simple nova boot and friends), trying to make something that fits the "tried and tested" model used in devstack-gate | 21:53 |
| clarkb | mnaser: mordred but you can tell it to run a different tox target or not run tempest at all and followup with your own thing | 21:53 |
| mnaser | https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L868-L871 - but just the smoke tests, unless "otherwise" requested i assume | 21:53 |
| *** marst has quit IRC | 21:53 | |
| mnaser | okay ocol | 21:53 |
| EmilienM | clarkb: I'm out in 2 min, let me know if you want to know anything else or I'm leaving | 21:54 |
| mordred | mnaser: so, for instance, shade's functional tests install devstack then run shade's functional tests against that cloud | 21:54 |
| clarkb | EmilienM: ok, probably best for us to use ocata for now since we just need newer libvirt | 21:54 |
| jhesketh | Morning | 21:54 |
| clarkb | EmilienM: thanks for the info | 21:54 |
| EmilienM | clarkb: yes ocata is stable and good to go imho | 21:54 |
| clarkb | there is also a hook point to not run devstack | 21:54 |
| mordred | mnaser: (also, you should run shade's functional tests against a kolla-deployed cloud in a job somewhere) | 21:54 |
| openstackgerrit | Felipe Monteiro proposed openstack-infra/project-config master: Add experimental murano functional test gate. https://review.openstack.org/451996 | 21:54 |
| clarkb | except I think that also doesn't run tempest, hrm | 21:54 |
| mnaser | unfortunately we cant share bits as much because we run tempest in a docker container and the env isnt installed in the same way | 21:55 |
| mordred | mnaser: I'd be happy to help with that | 21:55 |
| mnaser | im mucking aronud here - https://review.openstack.org/#/c/402122/ | 21:55 |
| mnaser | so far i got tempest to run smoke tests successfully | 21:55 |
| mnaser | except 1 fail due to lack of metering agent (which i disabled in this upcoming set of tests) | 21:55 |
| *** wolverineav has joined #openstack-infra | 21:55 | |
| mnaser | openstack-infra/kolla-gate .. maybe. :-p | 21:55 |
| mordred | mnaser: :) | 21:56 |
| clarkb | I thnk you can use devstack-gate for this, we may just have to smooth out some rough edges, but otherwise its all there today | 21:56 |
| mordred | mnaser: actually, devstack-gate is a bad name | 21:56 |
| clarkb | yes its more like "infra does all the things you shouldn't have to think about and then by default runs devstack and tempest unless told otherwise" | 21:56 |
| mnaser | aha :p | 21:57 |
| mordred | yup. it's basically a framework for getting the VM set up in the CI environment and ready to run things - devstack being the first/main one | 21:57 |
| mordred | so it should just be called "infra-gate" | 21:57 |
| mordred | or something | 21:57 |
| mordred | unfortunately the word "devstack" in the name gives people the wrong impression that they need to do something different when they do not (and in fact should not) | 21:57 |
| mnaser | right now the issue is kolla build gates are voting (so we build images), but deploy is not at the moment | 21:57 |
| *** EricGonczer_ has joined #openstack-infra | 21:58 | |
| clarkb | but also renaming it would be a fairly major pain that I don't want to think about | 21:58 |
| mnaser | so hopefully tempest and stability for sometime will hopefully get us to make them voting then we can try having other projects get tested as part of it (ex: shade) | 21:58 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Use custom network-isolation-absolute.yaml in ipv4 net-iso jobs https://review.openstack.org/422297 | 21:58 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Test with bonded public interface https://review.openstack.org/385562 | 21:58 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Add storage-mgmt network to compute nodes https://review.openstack.org/419049 | 21:58 |
| mnaser | clarkb on the plus side, we're at the point where no one says "hey im trying to deploy openstack in prod with devstack" | 21:58 |
| mordred | mnaser: well - I honestly think in addition to tempest we should make a shade function test for kolla - and we also have tests of the os_ ansible modules in shade that we should make you a job to run | 21:58 |
| *** tosky has joined #openstack-infra | 21:58 | |
| *** mdrabe has quit IRC | 21:58 | |
| clarkb | mnaser: I dunno the user survey is pretty scary on that front | 21:59 |
| *** EricGonc_ has joined #openstack-infra | 21:59 | |
| mnaser | mordred those both sound like pretty exciting scenarios | 21:59 |
| mnaser | unfortunately the failure rate of a kolla deployment is not exciting | 21:59 |
| mnaser | and i think nailing that down before adding more jobs would be best. because testing something that never deploys successfully => not a fun time | 22:00 |
| *** esberglu has quit IRC | 22:00 | |
| lifeless | lastlog lifeless | 22:00 |
| mordred | look it's lifeless | 22:00 |
| clarkb | mnaser: +++++ I find that especially when it comes to this stuff using an incremental appraoch leads to ahppyness quicker | 22:00 |
| mordred | mnaser: ++ | 22:00 |
| mnaser | mordred / clarkb - i made pretty graphs to track the progress http://grafana.openstack.org/dashboard/db/kolla-failure-rate | 22:01 |
| mnaser | 20% failure rate for source deploys and 10% failure rate for binary deploys | 22:01 |
| mordred | lifeless: I pinged you a little while ago about a bindep failure that seems to maybe be a bug somewhere in version string parsing | 22:01 |
| *** wolverineav has quit IRC | 22:01 | |
| mnaser | i dont know if that is a good percentage or not.. but to me that seems kinda high | 22:01 |
| mnaser | maybe too high to make that job voting right now | 22:01 |
| clarkb | infra-root https://review.openstack.org/#/c/449835/2 is the puppet change we need to support deploying new gerrit on review-dev. If you arent' comfortable approving it thats fine, I'm happy to do that with zaro and make sure that all the gerrits remain happy. But would definitely be good to get eyeballs on that so we can work on upgrading gerrit | 22:01 |
| *** harlowja has quit IRC | 22:02 | |
| clarkb | mnaser: I think 10% is high | 22:02 |
| *** EricGonczer_ has quit IRC | 22:02 | |
| clarkb | mnaser: its high enough you'll notice it when voting | 22:02 |
| *** mriedem has quit IRC | 22:02 | |
| *** EricGonc_ has quit IRC | 22:02 | |
| mnaser | clarkb my issue right now is that i struggle to imagine how i can cleanly generate that data | 22:02 |
| clarkb | especially if you have 8 voting jobs the chance a single change fails is super high | 22:02 |
| mordred | lifeless: it's easy to reproduce - put the line "haproxy >=2" into a bindep.txt file and run bindep | 22:02 |
| mnaser | because what if someone keeps uploading wip stuff and the rate sits really high | 22:03 |
| *** sshnaidm is now known as sshnaidm|off | 22:03 | |
| clarkb | mnaser: ya thats definitely a problem. One way you can do it is by running daily jobs and using those as baseline | 22:03 |
| mnaser | clarkb: good idea! | 22:03 |
| mordred | also - checking failure rate compared to failure rate of one of hte other jobs that's more stable | 22:03 |
| *** thorst has quit IRC | 22:03 | |
| pabelanger | mordred: mnaser: I am looking forward to zuulv3 job graph for things like image builds, aside from artifact handling, it will make things much easier to depend on image builds 1 job building, 10 jobs consuming the build | 22:04 |
| *** xyang1 has quit IRC | 22:04 | |
| mnaser | no deploy jobs are "stable" because none are voting in kolla at the moment :-( | 22:04 |
| mnaser | pabelanger yeah that would be super great.. tons of useful things (i.e. secrets0 | 22:05 |
| *** funzo has joined #openstack-infra | 22:05 | |
| *** marst has joined #openstack-infra | 22:05 | |
| mnaser | talk about build failures and one just randomly failed now :< | 22:05 |
| clarkb | pabelanger: speaking of, I was thinking about more use cases for RO cinder volumes and that was one I came up with (not sure if great idea but its a thing) | 22:06 |
| clarkb | pabelanger: basically have the build jobs write to a scratch volume, then mount RO in subsequent jobs, and if pass upload artifact to permanent storgae then delete the volume | 22:07 |
| clarkb | it does require jobs to run in same cloud though :/ | 22:07 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove classic multinode jobs from cistatus https://review.openstack.org/451884 | 22:07 |
| pabelanger | clarkb: how would we expose the cinder volumes to jobs? | 22:08 |
| pabelanger | clarkb: also, spaceX webcast starting | 22:08 |
| clarkb | pabelanger: I'm not sureI was just thinking about how to do things | 22:08 |
| pabelanger | ah | 22:09 |
| clarkb | pabelanger: but I think if build job succeeds then you attach to all the subsequent ones via cinder api (maybe zuul does that?) | 22:09 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config master: Add functional job for k8s-cloud-provider with sudo privileges https://review.openstack.org/451977 | 22:09 |
| clarkb | the restriction for running in same cloud annoys me but also if you have gigabytes of data you probably don't want to copy all over the world so maybe its fine | 22:09 |
| pabelanger | clarkb: Ya, that sounds right | 22:09 |
| *** funzo has quit IRC | 22:09 | |
| clarkb | but we can give the device a name like ZUUL_DATA_RO or something | 22:10 |
| pabelanger | clarkb: right, maybe we group jobs in a single cloud, like we do for nodesets. But, not sure how that would look | 22:10 |
| clarkb | (I do think we can and should replace the data we cache on images with a RO volume in each cloud but that requires the feature to be available everywhere) | 22:14 |
| clarkb | no more 20GB images would be nice :) | 22:14 |
| *** baoli has quit IRC | 22:15 | |
| *** jlk` has joined #openstack-infra | 22:17 | |
| mordred | clarkb: ++ | 22:17 |
| *** EricGonczer_ has joined #openstack-infra | 22:20 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add python35 non-voting job networking-arista https://review.openstack.org/451571 | 22:23 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/shade master: Add server security groups to shade https://review.openstack.org/451183 | 22:24 |
| *** gouthamr has quit IRC | 22:30 | |
| *** gordc has quit IRC | 22:33 | |
| *** EricGonc_ has joined #openstack-infra | 22:37 | |
| *** EricGonczer_ has quit IRC | 22:38 | |
| *** jlk has quit IRC | 22:43 | |
| *** jlk` has quit IRC | 22:43 | |
| *** jlk has joined #openstack-infra | 22:44 | |
| *** jlk has quit IRC | 22:44 | |
| *** jlk has joined #openstack-infra | 22:44 | |
| *** kornicameister has quit IRC | 22:47 | |
| *** kornicameister has joined #openstack-infra | 22:48 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fully qualify project configuration names https://review.openstack.org/451970 | 22:51 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add source to project and remove unused tenant attrs https://review.openstack.org/451969 | 22:51 |
| *** gouthamr has joined #openstack-infra | 22:51 | |
| *** EricGonc_ has quit IRC | 22:53 | |
| lifeless | mordred: it is I | 22:54 |
| lifeless | mordred: I don't have the code checked out right now, but... | 22:57 |
| *** hamzy has quit IRC | 22:57 | |
| lifeless | https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version | 22:59 |
| lifeless | read that and tell me how that haproxy version string is valid | 22:59 |
| lifeless | mordred: in particular https://www.debian.org/doc/debian-policy/footnotes.html#f36 | 23:00 |
| *** Goneri has quit IRC | 23:01 | |
| *** arijr has quit IRC | 23:02 | |
| lifeless | mordred: so option1 is make the bindep parser looser than the spec | 23:03 |
| lifeless | mordred: option 2 is to go WTF at Ubuntu's haproxy maintainer until they fix it | 23:03 |
| lifeless | mordred: option 3 is to go WTF at Debian's policy maintainer until they fix it | 23:03 |
| *** rbrndt has quit IRC | 23:03 | |
| lifeless | mordred: separately, I saw a WIP patch for pbr for lazy versions | 23:04 |
| *** thorst has joined #openstack-infra | 23:04 | |
| lifeless | mordred: I want to note that this won't help anything at all - because pkgresource caches all the Distribution data in memory so the actual choice is 'use Distribution metadata or don't' - and the plugin system OpenStack uses uses Distribution data | 23:04 |
| lifeless | mordred: so if we want to shift the dial on reading this I dunno 1-2K of data, we have a tonne of work to do. Where step 1 is 'invent a new package metadata database thats faster' | 23:05 |
| *** funzo has joined #openstack-infra | 23:06 | |
| *** rbuzatu has joined #openstack-infra | 23:06 | |
| *** rbuzatu has quit IRC | 23:06 | |
| *** rbuzatu has joined #openstack-infra | 23:07 | |
| *** signed8b_ is now known as signed8bit_Zzz | 23:07 | |
| clarkb | lifeless: yes I actually went and debugged setuptools | 23:07 |
| clarkb | lifeless: and found an existing bug and threw data on there for why its slow | 23:07 |
| clarkb | lifeless: turns out every packge it can find in the python path has its version parsed and sorted | 23:07 |
| clarkb | this is slow | 23:07 |
| *** thorst has quit IRC | 23:08 | |
| lifeless | clarkb: yep, it is! | 23:09 |
| mordred | mnaser: remote: https://review.openstack.org/452005 Add clouds.yaml file and use it | 23:09 |
| mordred | mnaser: just made you a kolla-ansible patch :) | 23:09 |
| lifeless | clarkb: just wanted to avoid us fixing not-the-probblem | 23:09 |
| clarkb | lifeless: this means the fix is don't use pkg_resources at all or have fewer deps | 23:09 |
| *** funzo has quit IRC | 23:10 | |
| clarkb | I am not sure this can be excised from pkg_resources without breaking half the world | 23:10 |
| *** hongbin has quit IRC | 23:11 | |
| lifeless | clarkb: sure, my point though is that we have another, much trickier dep on pkg_resources | 23:11 |
| *** tphummel has quit IRC | 23:11 | |
| lifeless | clarkb: which will mean 'not using pkg_resources' is actually non-trivial | 23:11 |
| clarkb | yup | 23:11 |
| lifeless | anyhow, I think we're agreed ;) - ttyl | 23:11 |
| mordred | lifeless: haproxy: >=2.0.0 also doesn't work | 23:12 |
| mordred | lifeless: nor does haproxy >=2.0.0-1 | 23:12 |
| lifeless | mordred: the error I saw was about the version of haproxy returned by dpkg | 23:12 |
| lifeless | mordred: not the version in the spec | 23:12 |
| lifeless | mordred: I may have misinterpreted it | 23:12 |
| mordred | lifeless: I may also have miinterpreted it ... | 23:13 |
| lifeless | mordred: paste the error? | 23:13 |
| *** tphummel has joined #openstack-infra | 23:13 | |
| pabelanger | clarkb: mordred : I don't see any meetings going on, at least on eavesdrop.o.o index page, do you mind +3 https://review.openstack.org/#/c/451434/ logging for openstack-forum | 23:13 |
| mordred | lifeless: http://paste.openstack.org/show/604934/ | 23:13 |
| * clarkb checks calendar and agrees no meetings | 23:14 | |
| mordred | lifeless: I get the same traceback no matter what I use for the version | 23:14 |
| *** thorst has joined #openstack-infra | 23:14 | |
| *** xarses has quit IRC | 23:15 | |
| mordred | lifeless: (the line numbers may be slightly off, I have a print statement added to try to debug) | 23:16 |
| *** thorst has quit IRC | 23:17 | |
| fungi | mordred: i missed it... what's the actual version string at issue? "1.5.14-1ubuntu0.15.10.1~ubuntu14.04.1"? (the package from trusty-backports?) | 23:17 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Create dir on subnode for copying deploy.env https://review.openstack.org/447849 | 23:17 |
| *** ihrachys has quit IRC | 23:17 | |
| *** felipemonteiro has quit IRC | 23:17 | |
| mordred | fungi: well - a couple of days ago someone was trying to add a version assertion of haproxy >= 1.5.0 to nova's bindep | 23:18 |
| mordred | ignoring whether that's a good idea or not - I started trying to poke at the traceback that resulted from him having attempted that | 23:18 |
| mordred | fungi: so far I cannot manage to add _any_ version string to a bindep file and have bindep not crash | 23:19 |
| lifeless | mordred: and whats the entire bindep file you're using? | 23:19 |
| *** tphummel has quit IRC | 23:19 | |
| mordred | lifeless: right now I just have one with haproxy >= 1.5.0 | 23:20 |
| mordred | ok - I just tried something else ... | 23:20 |
| mordred | which is adding a line for a package I do have installed and that shows up in dpkg -l | 23:20 |
| mordred | and that does not crash | 23:20 |
| openstackgerrit | Merged openstack-infra/system-config master: Add #openstack-forum to meetbot channels https://review.openstack.org/451434 | 23:21 |
| mordred | so perhaps hunch is that because dpkg-query -W -f '${Package} ${Status} ${Version}\n' haproxy is returning "dpkg-query: no packages found matching haproxy | 23:21 |
| mordred | " | 23:21 |
| mordred | we're winding up with None ? | 23:21 |
| mordred | get_pkg_version returns None if dpkg does that | 23:22 |
| *** dimtruck is now known as zz_dimtruck | 23:22 | |
| lifeless | so yes | 23:22 |
| lifeless | I think there are two issues | 23:22 |
| lifeless | one is that the haproxy version on that persons machine, if you go back and look at their error report | 23:22 |
| lifeless | is not compliant with policy | 23:23 |
| lifeless | which is freasky | 23:23 |
| mordred | \o/ | 23:23 |
| lifeless | and fungi will want to look at | 23:23 |
| mordred | yah | 23:23 |
| lifeless | the other is that apparently all our tests tested installed packages | 23:23 |
| *** adrian_otto has joined #openstack-infra | 23:23 | |
| mordred | ok. I found the bug in the thing | 23:24 |
| lifeless | None is the correct result from that function | 23:24 |
| *** harlowja has joined #openstack-infra | 23:24 | |
| mordred | it is | 23:24 |
| mordred | and we test it | 23:24 |
| mordred | but then we don't continue the loop past the version equality check | 23:24 |
| openstackgerrit | Monty Taylor proposed openstack-infra/bindep master: Exit the loop when we don't have version info https://review.openstack.org/452007 | 23:25 |
| mordred | lifeless, fungi: ^^ that (clearly needs a test, so I also skimped on the commit message for now) | 23:25 |
| lifeless | https://github.com/openstack-infra/bindep/blob/master/bindep/tests/test_depends.py#L219 | 23:25 |
| lifeless | is meant to test that already | 23:25 |
| fungi | versions for not-installed packages are also tricksy since that's really not something rpm will be able to tell us about (dpkg knows because it has a database of all available versions while rpm only has a database of installed versions) | 23:26 |
| lifeless | possibly https://github.com/openstack-infra/bindep/blob/master/bindep/tests/test_depends.py#L268 | 23:26 |
| mordred | lifeless: I agree | 23:26 |
| lifeless | fungi: dpkg is the same as rpm at this level | 23:26 |
| fungi | oh, right | 23:26 |
| fungi | apt (and yum/dnf) know about available versions | 23:27 |
| *** john-davidge has joined #openstack-infra | 23:27 | |
| lifeless | mordred: I think if you clone the test in https://github.com/openstack-infra/bindep/blob/master/bindep/tests/test_depends.py#L219 and extend it with a version selector you'll trigger the None bug | 23:27 |
| fungi | there was some corner case for versioning where dpkg was more featureful than rpm but now i don't recall what it was | 23:27 |
| lifeless | got to run, ttyl. fungi would appreciate you looking at the version in the original report and seeing if I"m crazy about its compliance... and how it got past the Ubuntu buildfarm | 23:28 |
| *** Swami has quit IRC | 23:28 | |
| *** stakeda has quit IRC | 23:28 | |
| mordred | lifeless: thank you for the help!!!! | 23:28 |
| fungi | i'll put it on my docket, but well into evening grilling and beering time | 23:28 |
| fungi | where was the bug report? | 23:29 |
| *** MasterOfBugs has quit IRC | 23:30 | |
| *** john-davidge has quit IRC | 23:31 | |
| *** mat128 has quit IRC | 23:34 | |
| *** MasterOfBugs has joined #openstack-infra | 23:35 | |
| mordred | fungi: I believe there is a storyboard one | 23:36 |
| fungi | finding | 23:37 |
| fungi | for reference, https://storyboard.openstack.org/#!/story/2000955 | 23:39 |
| *** ihrachys has joined #openstack-infra | 23:41 | |
| fungi | we had sort of punted on version specifiers in bindep lists previously, on the assumption that for any given distro release there would be one and only one preferred version of a package available to the user, so telling them to install a different version wouldn't help much | 23:41 |
| fungi | granted, people do add secondary package repos to their systems to get around that, so there is some use for it | 23:41 |
| *** openstack has joined #openstack-infra | 23:43 | |
| mordred | fungi: I've almost got a fix for the main error | 23:43 |
| *** ramishra has joined #openstack-infra | 23:43 | |
| fungi | though the description in his patch doesn't match | 23:43 |
| ihrachys | fungi: being able to set version in bindep.txt and make the gate consume it (and fail early on missing dep) | 23:44 |
| fungi | disappearing again for a bit, but will be back | 23:44 |
| ihrachys | at the moment, if we set version, I think it breaks gate, or smth. I don't remember exact details | 23:44 |
| ihrachys | I should have followed up on the patch for a long time | 23:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/bindep master: Exit the loop when we don't have a package https://review.openstack.org/452007 | 23:45 |
| fungi | ihrachys: right, see the storyboard link above. the traceback in the log doesn't look like what you're fixing but i can't be sure yet | 23:45 |
| openstackgerrit | Monty Taylor proposed openstack-infra/bindep master: Exit the loop when we don't have a package https://review.openstack.org/452007 | 23:45 |
| mordred | fungi: ok - that fixes the no-version problem and adds a test which fails without the patch | 23:46 |
| mordred | ihrachys: yah - the version bit turns out to be a simple missing continue line ... that said, matching non-compliant versions may be a different issue | 23:46 |
| ihrachys | fungi: that's the context http://lists.openstack.org/pipermail/openstack-dev/2016-October/105054.html | 23:47 |
| mordred | which is the one I think you're solving | 23:47 |
| *** thorst has joined #openstack-infra | 23:47 | |
| mordred | yah - the issue of it not working on ubuntu is that if the package is not installed at all on ubuntu, the version matching fails spectacularly | 23:47 |
| ihrachys | mordred: I would love to say I will revive the patch tomorrow, but I know I won't, so if there is interest, maybe someone would like to freshen it | 23:48 |
| mordred | ihrachys: understood | 23:48 |
| mnaser | mordred woo got smoke tests running here - https://review.openstack.org/#/c/402122/ | 23:50 |
| mnaser | except tempest on ubuntu is blowing up | 23:50 |
| mnaser | its throwing a ton of ArgsAlreadyParsedError: arguments already parsed: cannot register CLI option | 23:51 |
| *** baoli has joined #openstack-infra | 23:52 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 23:53 |
| mordred | mnaser: neat! | 23:53 |
| openstackgerrit | Merged openstack-infra/shade master: Add server security groups to shade https://review.openstack.org/451183 | 23:54 |
| mnaser | i should make a talk with the list of most popular cryptic openstack errors | 23:55 |
| *** ihrachys has quit IRC | 23:55 | |
| *** sdague has quit IRC | 23:55 | |
| SamYaple | mnaser: and the solution to fix those errors? http://i.imgur.com/YsbKHg1.gif | 23:56 |
| mnaser | haha | 23:56 |
| *** jaypipes has quit IRC | 23:58 | |
| *** skelso has joined #openstack-infra | 23:59 | |
| *** thorst has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
