| *** tsg has joined #openstack-infra | 00:02 | |
| *** achanda has joined #openstack-infra | 00:07 | |
| *** woodster_ has quit IRC | 00:10 | |
| *** achanda has quit IRC | 00:12 | |
| *** subscope has quit IRC | 00:18 | |
| *** tsg has quit IRC | 00:23 | |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/project-config: Remove Nova v21 neutron job & make one for v2.0 https://review.openstack.org/163731 | 00:33 |
|---|---|---|
| *** otter768 has quit IRC | 00:37 | |
| *** matrohon has quit IRC | 00:37 | |
| *** mjturek1 has quit IRC | 00:37 | |
| *** tkelsey has joined #openstack-infra | 00:42 | |
| *** zz_dimtruck is now known as dimtruck | 00:47 | |
| *** tkelsey has quit IRC | 00:47 | |
| *** mwagner_lap has quit IRC | 00:48 | |
| *** funzo has joined #openstack-infra | 00:50 | |
| *** tiswanso has joined #openstack-infra | 00:52 | |
| *** garyh has joined #openstack-infra | 00:53 | |
| *** funzo has quit IRC | 00:55 | |
| *** tiswanso_ has joined #openstack-infra | 00:56 | |
| *** tiswanso has quit IRC | 00:58 | |
| *** ghostpl_ has joined #openstack-infra | 00:59 | |
| *** mwagner_lap has joined #openstack-infra | 01:00 | |
| *** ujuc has joined #openstack-infra | 01:01 | |
| *** garyh has quit IRC | 01:03 | |
| *** jamesmcarthur has joined #openstack-infra | 01:04 | |
| *** garyh has joined #openstack-infra | 01:04 | |
| *** ghostpl_ has quit IRC | 01:06 | |
| *** Longgeek has joined #openstack-infra | 01:08 | |
| *** hdd has quit IRC | 01:09 | |
| *** yfried|afk has quit IRC | 01:10 | |
| *** kiseok7_ has joined #openstack-infra | 01:11 | |
| *** liusheng has quit IRC | 01:13 | |
| *** garyh has quit IRC | 01:15 | |
| *** Longgeek has quit IRC | 01:15 | |
| *** alexpilotti has quit IRC | 01:19 | |
| *** amitgandhinz has joined #openstack-infra | 01:21 | |
| *** harlowja_at_home has joined #openstack-infra | 01:21 | |
| *** yamamoto has joined #openstack-infra | 01:22 | |
| *** salv-orlando has quit IRC | 01:22 | |
| *** mtanino has joined #openstack-infra | 01:25 | |
| *** ddieterl_ has joined #openstack-infra | 01:28 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: WIP: Trying to add a DSVM for magnum https://review.openstack.org/164553 | 01:29 |
| *** jamesmcarthur has quit IRC | 01:30 | |
| *** dimtruck is now known as zz_dimtruck | 01:32 | |
| *** jamesmcarthur has joined #openstack-infra | 01:32 | |
| *** ddieterly has quit IRC | 01:33 | |
| *** Qiming has joined #openstack-infra | 01:33 | |
| *** Qiming_ has joined #openstack-infra | 01:34 | |
| *** Qiming has quit IRC | 01:37 | |
| Qiming_ | anteaya, online? | 01:37 |
| *** amotoki has joined #openstack-infra | 01:38 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool: Remove the need for zmq in proper demand calcs. https://review.openstack.org/98306 | 01:42 |
| *** Qiming_ is now known as Qiming | 01:43 | |
| *** ddieterly has joined #openstack-infra | 01:43 | |
| *** ddieterl_ has quit IRC | 01:44 | |
| *** baoli has quit IRC | 01:48 | |
| *** otter768 has joined #openstack-infra | 01:49 | |
| *** jamesmcarthur has quit IRC | 01:50 | |
| *** harlowja_at_home has quit IRC | 01:50 | |
| *** bknudson has quit IRC | 01:53 | |
| *** jamesmcarthur has joined #openstack-infra | 01:56 | |
| *** baoli has joined #openstack-infra | 02:07 | |
| *** hdd has joined #openstack-infra | 02:13 | |
| *** dims_ has quit IRC | 02:13 | |
| *** ivar-lazzaro has joined #openstack-infra | 02:17 | |
| *** ghostpl_ has joined #openstack-infra | 02:17 | |
| *** amitgandhinz has quit IRC | 02:19 | |
| *** ghostpl_ has quit IRC | 02:22 | |
| *** salv-orlando has joined #openstack-infra | 02:22 | |
| *** asselin_ has quit IRC | 02:23 | |
| *** ivar-lazzaro has quit IRC | 02:23 | |
| *** julim has quit IRC | 02:24 | |
| *** salv-orlando has quit IRC | 02:25 | |
| *** salv-orlando has joined #openstack-infra | 02:29 | |
| *** salv-orlando has quit IRC | 02:38 | |
| *** jyuso1 has quit IRC | 02:42 | |
| *** hogepodge has joined #openstack-infra | 02:43 | |
| *** jyuso1 has joined #openstack-infra | 02:43 | |
| *** Qiming has quit IRC | 02:47 | |
| *** Qiming has joined #openstack-infra | 02:47 | |
| *** ivar-lazzaro has joined #openstack-infra | 02:50 | |
| *** garyh has joined #openstack-infra | 02:53 | |
| *** ivar-lazzaro has quit IRC | 02:57 | |
| *** coolsvap|afk is now known as coolsvap | 02:58 | |
| *** hdd has quit IRC | 02:58 | |
| *** nelsnelson has joined #openstack-infra | 02:59 | |
| *** garyh has quit IRC | 02:59 | |
| *** jamesmcarthur has quit IRC | 03:02 | |
| *** sputnik13 has joined #openstack-infra | 03:04 | |
| *** kiseok7_ has quit IRC | 03:05 | |
| *** kiseok7_ has joined #openstack-infra | 03:06 | |
| *** tiswanso_ has quit IRC | 03:07 | |
| *** Longgeek has joined #openstack-infra | 03:11 | |
| *** salv-orlando has joined #openstack-infra | 03:12 | |
| *** Longgeek has quit IRC | 03:13 | |
| *** jamesmcarthur has joined #openstack-infra | 03:13 | |
| *** Longgeek has joined #openstack-infra | 03:14 | |
| *** dimsum__ has joined #openstack-infra | 03:14 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/nodepool: Remove the need for zmq in proper demand calcs. https://review.openstack.org/98306 | 03:15 |
| *** salv-orlando has quit IRC | 03:16 | |
| *** ghostpl_ has joined #openstack-infra | 03:17 | |
| *** stevemar has joined #openstack-infra | 03:19 | |
| *** Longgeek_ has joined #openstack-infra | 03:19 | |
| *** dimsum__ has quit IRC | 03:19 | |
| *** Longgeek has quit IRC | 03:20 | |
| *** ghostpl_ has quit IRC | 03:22 | |
| *** salv-orlando has joined #openstack-infra | 03:26 | |
| *** amotoki has quit IRC | 03:28 | |
| *** ChuckC has joined #openstack-infra | 03:28 | |
| *** salv-orlando has quit IRC | 03:29 | |
| *** jamesmcarthur has quit IRC | 03:32 | |
| *** jamesmcarthur has joined #openstack-infra | 03:32 | |
| *** salv-orlando has joined #openstack-infra | 03:32 | |
| *** salv-orlando has quit IRC | 03:32 | |
| *** salv-orlando has joined #openstack-infra | 03:36 | |
| *** isviridov has quit IRC | 03:38 | |
| *** salv-orlando has quit IRC | 03:38 | |
| *** salv-orlando has joined #openstack-infra | 03:41 | |
| *** Longgeek has joined #openstack-infra | 03:41 | |
| *** ddieterly has quit IRC | 03:43 | |
| *** Longgeek_ has quit IRC | 03:44 | |
| *** salv-orlando has quit IRC | 03:45 | |
| *** salv-orlando has joined #openstack-infra | 03:50 | |
| *** zz_dimtruck is now known as dimtruck | 03:50 | |
| *** jamesmcarthur has quit IRC | 03:52 | |
| *** ajmiller_ has joined #openstack-infra | 03:55 | |
| *** salv-orlando has quit IRC | 03:57 | |
| *** ajmiller has quit IRC | 03:59 | |
| *** garyh has joined #openstack-infra | 04:00 | |
| *** koolhead17 has joined #openstack-infra | 04:02 | |
| *** SumitNaiksatam has quit IRC | 04:04 | |
| *** garyh has quit IRC | 04:10 | |
| *** achanda has joined #openstack-infra | 04:10 | |
| *** Longgeek has quit IRC | 04:11 | |
| *** Longgeek has joined #openstack-infra | 04:11 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Add nanliu/archive module https://review.openstack.org/164569 | 04:15 |
| *** salv-orlando has joined #openstack-infra | 04:19 | |
| *** ujuc has quit IRC | 04:20 | |
| *** sarob has quit IRC | 04:20 | |
| *** salv-orlando has quit IRC | 04:24 | |
| openstackgerrit | Merged openstack-infra/project-config: Add missing projects to GBP job https://review.openstack.org/164484 | 04:25 |
| *** otter768 has quit IRC | 04:27 | |
| *** salv-orlando has joined #openstack-infra | 04:27 | |
| *** salv-orlando has quit IRC | 04:28 | |
| *** yfried|afk has joined #openstack-infra | 04:31 | |
| *** gokrokve has joined #openstack-infra | 04:40 | |
| openstackgerrit | Jamie Lennox proposed openstack/requirements: Bump keystonemiddleware requirement https://review.openstack.org/164573 | 04:40 |
| *** mtanino is now known as mtanino_away | 04:40 | |
| *** tkelsey has joined #openstack-infra | 04:43 | |
| *** ddieterly has joined #openstack-infra | 04:44 | |
| *** ivar-lazzaro has joined #openstack-infra | 04:45 | |
| *** gokrokve has quit IRC | 04:45 | |
| *** ivar-lazzaro has quit IRC | 04:46 | |
| *** gokrokve has joined #openstack-infra | 04:46 | |
| *** ivar-lazzaro has joined #openstack-infra | 04:46 | |
| *** tkelsey has quit IRC | 04:47 | |
| *** changbl has quit IRC | 04:48 | |
| *** ddieterly has quit IRC | 04:48 | |
| *** baoli has quit IRC | 04:50 | |
| *** gokrokve has quit IRC | 04:50 | |
| *** hdd has joined #openstack-infra | 04:52 | |
| *** ivar-lazzaro has quit IRC | 04:57 | |
| *** BharatK has joined #openstack-infra | 05:00 | |
| *** ghostpl_ has joined #openstack-infra | 05:03 | |
| *** armax has joined #openstack-infra | 05:05 | |
| *** garyh has joined #openstack-infra | 05:11 | |
| *** ghostpl_ has quit IRC | 05:11 | |
| *** dimtruck is now known as zz_dimtruck | 05:12 | |
| *** gokrokve has joined #openstack-infra | 05:16 | |
| *** gokrokve has quit IRC | 05:18 | |
| *** gokrokve has joined #openstack-infra | 05:18 | |
| *** garyh has quit IRC | 05:21 | |
| *** gokrokve has quit IRC | 05:23 | |
| *** hdd has quit IRC | 05:29 | |
| *** ashishb has joined #openstack-infra | 05:32 | |
| *** funzo has joined #openstack-infra | 05:36 | |
| *** funzo has quit IRC | 05:40 | |
| *** sarob has joined #openstack-infra | 05:42 | |
| *** ddieterly has joined #openstack-infra | 05:45 | |
| *** HeOS has joined #openstack-infra | 05:46 | |
| *** sarob has quit IRC | 05:47 | |
| *** ddieterly has quit IRC | 05:49 | |
| *** armax has quit IRC | 05:51 | |
| *** stevemar has quit IRC | 05:55 | |
| *** changbl has joined #openstack-infra | 05:58 | |
| *** Ryan_Lane has joined #openstack-infra | 05:59 | |
| *** dboik_ has quit IRC | 06:05 | |
| *** yfried|afk has quit IRC | 06:07 | |
| *** nelsnelson has quit IRC | 06:09 | |
| *** achanda has quit IRC | 06:15 | |
| *** salv-orlando has joined #openstack-infra | 06:17 | |
| *** gokrokve has joined #openstack-infra | 06:18 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual: Adds project-gating description https://review.openstack.org/138595 | 06:21 |
| *** amotoki has joined #openstack-infra | 06:21 | |
| *** garyh has joined #openstack-infra | 06:21 | |
| *** gokrokve has quit IRC | 06:23 | |
| *** salv-orlando has quit IRC | 06:23 | |
| *** afazekas has joined #openstack-infra | 06:23 | |
| *** garyh has quit IRC | 06:31 | |
| *** salv-orlando has joined #openstack-infra | 06:34 | |
| *** ashishb has quit IRC | 06:37 | |
| *** achanda has joined #openstack-infra | 06:42 | |
| *** amotoki has quit IRC | 06:43 | |
| *** amotoki has joined #openstack-infra | 06:44 | |
| *** Ryan_Lane has quit IRC | 06:45 | |
| *** ddieterly has joined #openstack-infra | 06:46 | |
| *** amotoki has quit IRC | 06:48 | |
| *** salv-orlando has quit IRC | 06:50 | |
| *** ddieterly has quit IRC | 06:50 | |
| *** Longgeek_ has joined #openstack-infra | 06:52 | |
| *** Longgeek has quit IRC | 06:52 | |
| *** tnovacik has joined #openstack-infra | 06:53 | |
| *** ghostpl_ has joined #openstack-infra | 06:55 | |
| *** salv-orlando has joined #openstack-infra | 06:57 | |
| openstackgerrit | Eran Gampel proposed openstack-infra/project-config: change project description test https://review.openstack.org/164501 | 07:02 |
| *** ghostpl_ has quit IRC | 07:02 | |
| *** HeOS has quit IRC | 07:05 | |
| *** mrunge has joined #openstack-infra | 07:08 | |
| *** _nadya_ has joined #openstack-infra | 07:11 | |
| *** achanda has quit IRC | 07:12 | |
| *** pcaruana has quit IRC | 07:14 | |
| *** gokrokve has joined #openstack-infra | 07:18 | |
| *** gokrokve has quit IRC | 07:20 | |
| *** gokrokve has joined #openstack-infra | 07:20 | |
| *** yfried|afk has joined #openstack-infra | 07:22 | |
| *** koolhead17 has quit IRC | 07:22 | |
| *** gokrokve has quit IRC | 07:24 | |
| *** yfried has joined #openstack-infra | 07:26 | |
| *** yfried|afk has quit IRC | 07:26 | |
| *** mtanino_away has quit IRC | 07:28 | |
| *** garyh has joined #openstack-infra | 07:32 | |
| *** hichihara has quit IRC | 07:34 | |
| *** e0ne has joined #openstack-infra | 07:34 | |
| *** andreykurilin_ has joined #openstack-infra | 07:37 | |
| *** salv-orlando has quit IRC | 07:37 | |
| *** mpavone has joined #openstack-infra | 07:38 | |
| *** e0ne has quit IRC | 07:39 | |
| *** garyh has quit IRC | 07:42 | |
| *** _nadya_ has quit IRC | 07:42 | |
| *** rwsu has joined #openstack-infra | 07:43 | |
| *** amotoki has joined #openstack-infra | 07:44 | |
| *** liusheng has joined #openstack-infra | 07:44 | |
| *** ddieterly has joined #openstack-infra | 07:46 | |
| *** ddieterly has quit IRC | 07:51 | |
| *** andreykurilin_ has quit IRC | 07:56 | |
| *** scheuran has joined #openstack-infra | 07:57 | |
| *** ihrachyshka has joined #openstack-infra | 07:58 | |
| *** yamamoto has quit IRC | 08:05 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-zuul: Add fetch_url and poll parameters to zuul.conf https://review.openstack.org/162591 | 08:06 |
| *** markus_z has joined #openstack-infra | 08:08 | |
| *** ihrachyshka has quit IRC | 08:14 | |
| *** mrunge has quit IRC | 08:18 | |
| *** gokrokve has joined #openstack-infra | 08:18 | |
| *** asselin_ has joined #openstack-infra | 08:18 | |
| *** pblaho has joined #openstack-infra | 08:19 | |
| *** mrunge has joined #openstack-infra | 08:20 | |
| *** hashar has joined #openstack-infra | 08:21 | |
| *** ashishb has joined #openstack-infra | 08:21 | |
| *** asselin_ has quit IRC | 08:23 | |
| *** gokrokve has quit IRC | 08:23 | |
| *** Hal has joined #openstack-infra | 08:24 | |
| *** Hal is now known as Guest80159 | 08:24 | |
| *** Somay has joined #openstack-infra | 08:24 | |
| *** tkelsey has joined #openstack-infra | 08:26 | |
| *** skolekonov has joined #openstack-infra | 08:26 | |
| *** shardy has joined #openstack-infra | 08:31 | |
| *** jcoufal has joined #openstack-infra | 08:33 | |
| *** yfried is now known as yfried|afk | 08:34 | |
| *** ihrachyshka has joined #openstack-infra | 08:35 | |
| *** fdegir has joined #openstack-infra | 08:35 | |
| *** yfried|afk is now known as yfried | 08:35 | |
| *** yamamoto has joined #openstack-infra | 08:37 | |
| *** salv-orlando has joined #openstack-infra | 08:38 | |
| *** yamamoto has quit IRC | 08:38 | |
| *** camunoz has quit IRC | 08:39 | |
| *** dtantsur has joined #openstack-infra | 08:40 | |
| *** jlanoux has joined #openstack-infra | 08:41 | |
| *** ujuc has joined #openstack-infra | 08:44 | |
| *** HeOS has joined #openstack-infra | 08:45 | |
| *** Ala has joined #openstack-infra | 08:46 | |
| *** ghostpl_ has joined #openstack-infra | 08:46 | |
| *** ildikov has quit IRC | 08:47 | |
| *** ddieterly has joined #openstack-infra | 08:47 | |
| *** Somay has quit IRC | 08:49 | |
| *** Ala has quit IRC | 08:49 | |
| *** Ala has joined #openstack-infra | 08:50 | |
| *** devvesa has joined #openstack-infra | 08:50 | |
| *** salv-orl_ has joined #openstack-infra | 08:50 | |
| *** jd__ has quit IRC | 08:50 | |
| *** jd__` has joined #openstack-infra | 08:50 | |
| *** jd__` is now known as jd__ | 08:51 | |
| *** ghostpl_ has quit IRC | 08:51 | |
| *** ddieterly has quit IRC | 08:51 | |
| *** salv-orlando has quit IRC | 08:51 | |
| *** Ala has quit IRC | 08:52 | |
| *** Ala has joined #openstack-infra | 08:52 | |
| *** jcoufal has quit IRC | 08:57 | |
| *** salv-orl_ has quit IRC | 08:57 | |
| *** Somay has joined #openstack-infra | 08:58 | |
| *** ihrachyshka has quit IRC | 08:59 | |
| *** salv-orlando has joined #openstack-infra | 09:02 | |
| *** salv-orlando has quit IRC | 09:05 | |
| openstackgerrit | Aleksey Ripinen proposed openstack-infra/storyboard: Branches, milestones and projects ids validation in tasks https://review.openstack.org/162126 | 09:07 |
| *** yfried is now known as yfried|afk | 09:08 | |
| *** garyh has joined #openstack-infra | 09:08 | |
| *** jistr has joined #openstack-infra | 09:11 | |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Allow create references in `networking-midonet` https://review.openstack.org/164606 | 09:12 |
| *** yfried|afk is now known as yfried | 09:12 | |
| *** derekh has joined #openstack-infra | 09:14 | |
| *** koolhead17 has joined #openstack-infra | 09:16 | |
| *** gokrokve has joined #openstack-infra | 09:18 | |
| *** garyh has quit IRC | 09:18 | |
| *** ujuc has quit IRC | 09:21 | |
| *** deepakcs has joined #openstack-infra | 09:22 | |
| *** salv-orlando has joined #openstack-infra | 09:22 | |
| *** gokrokve has quit IRC | 09:22 | |
| GheRivero | morning all | 09:23 |
| *** yolanda has quit IRC | 09:25 | |
| *** salv-orlando has quit IRC | 09:25 | |
| *** SumitNaiksatam has joined #openstack-infra | 09:30 | |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Allow create references in `networking-midonet` https://review.openstack.org/164606 | 09:31 |
| *** salv-orlando has joined #openstack-infra | 09:34 | |
| *** Ala has quit IRC | 09:35 | |
| *** salv-orlando has quit IRC | 09:37 | |
| *** yamamoto has joined #openstack-infra | 09:37 | |
| *** Ala has joined #openstack-infra | 09:37 | |
| *** salv-orlando has joined #openstack-infra | 09:38 | |
| *** marcusvrn1 has joined #openstack-infra | 09:42 | |
| *** salv-orlando has quit IRC | 09:43 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 09:43 | |
| *** salv-orlando has joined #openstack-infra | 09:44 | |
| *** ujuc has joined #openstack-infra | 09:46 | |
| *** dimsum__ has joined #openstack-infra | 09:46 | |
| *** cdent has joined #openstack-infra | 09:52 | |
| *** ghostpl_ has joined #openstack-infra | 09:53 | |
| *** yamamoto has quit IRC | 09:55 | |
| *** salv-orlando has quit IRC | 09:56 | |
| *** lucasagomes has joined #openstack-infra | 09:57 | |
| *** ssam2 has joined #openstack-infra | 09:57 | |
| *** lucasagomes has left #openstack-infra | 09:57 | |
| openstackgerrit | Merged openstack-infra/storyboard: Split auth in api and change user_tokens https://review.openstack.org/153591 | 09:57 |
| *** salv-orl_ has joined #openstack-infra | 09:58 | |
| *** Darkwan has quit IRC | 09:59 | |
| *** ghostpl_ has quit IRC | 09:59 | |
| *** yamamoto has joined #openstack-infra | 10:00 | |
| *** sabari has quit IRC | 10:00 | |
| *** sabari has joined #openstack-infra | 10:02 | |
| *** asettle has joined #openstack-infra | 10:04 | |
| *** Qiming has quit IRC | 10:07 | |
| zigo_ | sqlalchemy-migrate versio 0.9.5 has testtools < 0.9.36, that is *impossible* for me to satisfy in Debian. | 10:07 |
| *** salv-orl_ has quit IRC | 10:08 | |
| *** asettle has quit IRC | 10:08 | |
| *** yolanda has joined #openstack-infra | 10:08 | |
| zigo_ | We need this to be fixed. | 10:08 |
| *** arxcruz has joined #openstack-infra | 10:12 | |
| *** salv-orlando has joined #openstack-infra | 10:13 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Added class and view variable for state transitions. https://review.openstack.org/160526 | 10:14 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Preload Story and Creator in state https://review.openstack.org/161063 | 10:14 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Preload tasks in state resolution. https://review.openstack.org/161064 | 10:15 |
| *** salv-orlando has quit IRC | 10:17 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Moved preference resolution into its own resolver. https://review.openstack.org/163574 | 10:18 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Preference service uses private scope. https://review.openstack.org/163575 | 10:18 |
| *** gokrokve has joined #openstack-infra | 10:18 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Safety check for preference save. https://review.openstack.org/163576 | 10:18 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Removed static pageSize injector. https://review.openstack.org/163577 | 10:18 |
| *** coolsvap is now known as coolsvap|afk | 10:19 | |
| *** garyh has joined #openstack-infra | 10:19 | |
| *** salv-orlando has joined #openstack-infra | 10:22 | |
| *** gokrokve has quit IRC | 10:23 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Sorted Package Dependencies. https://review.openstack.org/161497 | 10:26 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Removed "Do No Evil" jshint and replaced with eslint. https://review.openstack.org/161498 | 10:27 |
| *** salv-orlando has quit IRC | 10:29 | |
| *** Somay has quit IRC | 10:29 | |
| *** garyh has quit IRC | 10:30 | |
| *** ildikov has joined #openstack-infra | 10:31 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Enabled eslint:no-undefined https://review.openstack.org/161499 | 10:32 |
| *** pc_m has joined #openstack-infra | 10:32 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Enabled eslint:indent https://review.openstack.org/161500 | 10:32 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Enabled eslint:key-spacing https://review.openstack.org/161501 | 10:33 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Enabled eslint:no-space-before-semi https://review.openstack.org/161502 | 10:33 |
| *** salv-orlando has joined #openstack-infra | 10:33 | |
| *** salv-orlando has quit IRC | 10:36 | |
| *** funzo has joined #openstack-infra | 10:37 | |
| *** funzo has quit IRC | 10:41 | |
| *** yamamoto has quit IRC | 10:43 | |
| *** jp_at_hp has joined #openstack-infra | 10:44 | |
| *** yfried is now known as yfried|afk | 10:45 | |
| *** rlandy has joined #openstack-infra | 10:45 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Added sane result set indicators. https://review.openstack.org/162265 | 10:45 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Moved template into correct directory. https://review.openstack.org/163138 | 10:46 |
| *** tnovacik_ has joined #openstack-infra | 10:47 | |
| *** ghostpl_ has joined #openstack-infra | 10:48 | |
| *** ghostpl_ has quit IRC | 10:48 | |
| *** tnovacik has quit IRC | 10:48 | |
| *** ghostpl_ has joined #openstack-infra | 10:48 | |
| *** ddieterly has joined #openstack-infra | 10:48 | |
| *** dtantsur is now known as dtantsur|bbl | 10:49 | |
| *** salv-orlando has joined #openstack-infra | 10:51 | |
| *** ddieterly has quit IRC | 10:53 | |
| *** _nadya_ has joined #openstack-infra | 10:53 | |
| *** Somay has joined #openstack-infra | 10:53 | |
| *** ZZelle has joined #openstack-infra | 10:55 | |
| *** salv-orlando has quit IRC | 10:56 | |
| *** _nadya__ has joined #openstack-infra | 10:58 | |
| *** _nadya_ has quit IRC | 10:58 | |
| *** dims_ has joined #openstack-infra | 11:02 | |
| *** dimsum__ has quit IRC | 11:04 | |
| *** yamamoto has joined #openstack-infra | 11:05 | |
| *** yfried|afk is now known as yfried | 11:05 | |
| *** Qiming_ has joined #openstack-infra | 11:05 | |
| *** dims_ has quit IRC | 11:07 | |
| *** Somay has quit IRC | 11:11 | |
| *** dimsum__ has joined #openstack-infra | 11:13 | |
| *** Somay has joined #openstack-infra | 11:16 | |
| *** gokrokve has joined #openstack-infra | 11:18 | |
| *** yamamoto has quit IRC | 11:20 | |
| *** Somay has quit IRC | 11:20 | |
| Qiming_ | anteaya, online? | 11:22 |
| *** tnovacik_ has quit IRC | 11:23 | |
| *** gokrokve has quit IRC | 11:23 | |
| *** Somay has joined #openstack-infra | 11:23 | |
| *** Somay has quit IRC | 11:24 | |
| *** ldnunes has joined #openstack-infra | 11:25 | |
| *** garyh has joined #openstack-infra | 11:30 | |
| *** dizquierdo has joined #openstack-infra | 11:36 | |
| *** ghostpl_ has quit IRC | 11:38 | |
| *** yfried_ has joined #openstack-infra | 11:38 | |
| *** yfried has quit IRC | 11:38 | |
| *** garyh has quit IRC | 11:41 | |
| *** tsufiev_ is now known as tsufiev | 11:42 | |
| *** dizquierdo has quit IRC | 11:43 | |
| *** Guest80159 has quit IRC | 11:45 | |
| *** Hal has joined #openstack-infra | 11:45 | |
| *** Hal is now known as Guest64707 | 11:45 | |
| *** mwagner_lap has quit IRC | 11:46 | |
| dstufft | mordred: (or anyone) if you get a chance to test https://github.com/pypa/pip/pull/2535 that'd be great. I'd like to merge it and get it into the next pip release | 11:47 |
| *** ddieterly has joined #openstack-infra | 11:49 | |
| *** jaypipes has joined #openstack-infra | 11:51 | |
| *** devvesa has quit IRC | 11:51 | |
| *** devvesa has joined #openstack-infra | 11:51 | |
| *** ddieterly has quit IRC | 11:54 | |
| *** weshay has joined #openstack-infra | 11:54 | |
| *** dizquierdo has joined #openstack-infra | 11:55 | |
| *** mwagner_lap has joined #openstack-infra | 11:56 | |
| *** dtantsur|bbl is now known as dtantsur | 11:57 | |
| *** ghostpl_ has joined #openstack-infra | 12:00 | |
| *** kgiusti has joined #openstack-infra | 12:02 | |
| *** Qiming__ has joined #openstack-infra | 12:05 | |
| *** deepakcs has quit IRC | 12:07 | |
| openstackgerrit | Merged openstack-infra/os-loganalyze: Update SUPPORTS_SEV regex to support new devstack logs https://review.openstack.org/159286 | 12:07 |
| *** dprince has joined #openstack-infra | 12:07 | |
| *** salv-orlando has joined #openstack-infra | 12:07 | |
| *** Qiming_ has quit IRC | 12:08 | |
| *** yamamoto has joined #openstack-infra | 12:09 | |
| *** aysyd has joined #openstack-infra | 12:10 | |
| *** tnovacik_ has joined #openstack-infra | 12:12 | |
| openstackgerrit | Merged openstack-infra/puppet-storyboard: Corrected metadata.json https://review.openstack.org/158408 | 12:13 |
| *** ildikov has quit IRC | 12:13 | |
| *** rm_work is now known as rm_work|away | 12:14 | |
| *** salv-orlando has quit IRC | 12:14 | |
| *** ildikov has joined #openstack-infra | 12:16 | |
| openstackgerrit | Sergey Lukjanov proposed openstack-infra/project-config: Add py27-scenario-unit tox non-voting job for Sahara https://review.openstack.org/163776 | 12:17 |
| *** Qiming__ is now known as Qiming | 12:17 | |
| *** gokrokve has joined #openstack-infra | 12:18 | |
| *** baoli has joined #openstack-infra | 12:18 | |
| openstackgerrit | Merged openstack-infra/system-config: Also serve zuul status.json via HTTPS https://review.openstack.org/155474 | 12:20 |
| *** funzo has joined #openstack-infra | 12:21 | |
| openstackgerrit | Merged openstack-infra/system-config: Upgrade openstackid to 1.0.6 https://review.openstack.org/164230 | 12:21 |
| *** salv-orlando has joined #openstack-infra | 12:21 | |
| *** ghostpl_ has quit IRC | 12:21 | |
| openstackgerrit | Christian Berendt proposed openstack-infra/project-config: Add new project ofaa to Stackforge https://review.openstack.org/164668 | 12:21 |
| *** salv-orlando has quit IRC | 12:22 | |
| *** salv-orlando has joined #openstack-infra | 12:22 | |
| *** gokrokve has quit IRC | 12:23 | |
| openstackgerrit | Jaroslav Henner proposed openstack-infra/jenkins-job-builder: Add checking of multijob phase condition. https://review.openstack.org/164669 | 12:24 |
| *** dkliban_afk is now known as dkliban | 12:24 | |
| *** bswartz has quit IRC | 12:26 | |
| *** rfolco has joined #openstack-infra | 12:26 | |
| *** SumitNaiksatam has quit IRC | 12:26 | |
| *** claudiub has joined #openstack-infra | 12:27 | |
| *** alexpilotti has joined #openstack-infra | 12:27 | |
| *** ajmiller_ has quit IRC | 12:27 | |
| *** woodster_ has joined #openstack-infra | 12:27 | |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting Sheepdog test job to cinder check pipeline https://review.openstack.org/153868 | 12:28 |
| *** ghostpl_ has joined #openstack-infra | 12:28 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Configure puppetmaster server from manifest https://review.openstack.org/164670 | 12:28 |
| openstackgerrit | Christian Berendt proposed openstack-infra/project-config: Add new project ofaa to Stackforge https://review.openstack.org/164668 | 12:28 |
| openstackgerrit | Merged openstack-infra/project-config: Add specs repository for os-ansible-deployment https://review.openstack.org/163960 | 12:29 |
| *** macjack has joined #openstack-infra | 12:31 | |
| *** dimsum__ has quit IRC | 12:32 | |
| *** dimsum__ has joined #openstack-infra | 12:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Allow create references in `networking-midonet` https://review.openstack.org/164606 | 12:33 |
| *** macjack has quit IRC | 12:35 | |
| *** radez_g0n3 is now known as radez | 12:38 | |
| mordred | dstufft: I have an error, but it doesn't look like it even got to the pbr part | 12:39 |
| openstackgerrit | Eran Gampel proposed openstack-infra/project-config: Change project description text https://review.openstack.org/164501 | 12:39 |
| dstufft | mordred: output? | 12:40 |
| mordred | dstufft: http://paste.openstack.org/show/192626/ | 12:40 |
| mordred | dstufft: I installed the patch into a venv, then did: git clone git://git.openstack.org/openstack-infra/zuul ; venv/bin/pip install ./zuul | 12:41 |
| dstufft | mordred: I think that's an broken cache dir | 12:41 |
| mordred | dstufft: in case you want an easy-ish test case | 12:41 |
| mordred | dstufft: oh - ok. should I blow it away? | 12:41 |
| dstufft | try with --no-cache-dir | 12:41 |
| *** garyh has joined #openstack-infra | 12:41 | |
| mordred | k. re-trying | 12:42 |
| dstufft | probably an error that cachecontrol should handle if that's the case | 12:42 |
| mordred | dstufft: yup. got much further | 12:42 |
| Shrews | mordred: reviewing your obj iface changes for shade now | 12:43 |
| mordred | Shrews: cool | 12:44 |
| dstufft | Ah yea, https://github.com/pypa/pip/issues/2481 I thought we had an open issue for that | 12:44 |
| dstufft | I should probably go and make a PR to cachecontrol | 12:44 |
| mordred | dstufft: ok - the install succeeded... | 12:45 |
| mordred | dstufft: as far as testing goes - should I expect evil to be apparent? or do Ineed to dig and make sure all the files installed as I expected? | 12:45 |
| mordred | fungi: ^^ do you remember what hapened last time this broke? | 12:45 |
| dstufft | mordred: last time pbr broke because we didn't generate an sdist but we didn't copy the .git dir to the temporary directory, so I'd say if pbr has a version number it's probably OK | 12:46 |
| mordred | dstufft: yes - the thing I installed into the virtualenv has a proper version number | 12:46 |
| dstufft | cool | 12:46 |
| dstufft | it probably works then | 12:47 |
| dstufft | if not, that's what git revert is for | 12:47 |
| dstufft | :D | 12:47 |
| *** sdake has joined #openstack-infra | 12:47 | |
| dstufft | that should make ``pip install /to/a/path`` much less of a special case | 12:48 |
| dstufft | it should be functionally equivilant to generating a sdist and isntalling from that now | 12:48 |
| *** openstackgerrit has quit IRC | 12:50 | |
| *** openstackgerrit has joined #openstack-infra | 12:50 | |
| *** ddieterly has joined #openstack-infra | 12:50 | |
| *** adalbas has joined #openstack-infra | 12:51 | |
| *** che-arne has joined #openstack-infra | 12:51 | |
| *** fifieldt has joined #openstack-infra | 12:51 | |
| *** garyh has quit IRC | 12:52 | |
| *** ddieterly has quit IRC | 12:55 | |
| *** jyuso1 has quit IRC | 12:57 | |
| *** wuhg has joined #openstack-infra | 12:57 | |
| *** dboik has joined #openstack-infra | 12:57 | |
| openstackgerrit | Christian Berendt proposed openstack-infra/project-config: Add new project faafo to Stackforge https://review.openstack.org/164668 | 12:57 |
| mordred | dstufft: woohoo. | 12:58 |
| mordred | ttx: ^^ actually, you should be pleased with this ^^ | 12:59 |
| ttx | I guess I should | 13:00 |
| *** bswartz has joined #openstack-infra | 13:01 | |
| *** yfried_ has quit IRC | 13:01 | |
| *** yfried_ has joined #openstack-infra | 13:02 | |
| *** ayoung has joined #openstack-infra | 13:02 | |
| *** bcourt has quit IRC | 13:04 | |
| *** spredzy is now known as spredzy|afk | 13:04 | |
| *** ddieterly has joined #openstack-infra | 13:05 | |
| *** Ala has quit IRC | 13:07 | |
| *** Ala has joined #openstack-infra | 13:08 | |
| *** julim has joined #openstack-infra | 13:09 | |
| *** dboik has quit IRC | 13:09 | |
| *** ddieterly has quit IRC | 13:10 | |
| *** jamesmcarthur has joined #openstack-infra | 13:11 | |
| *** SumitNaiksatam has joined #openstack-infra | 13:12 | |
| mordred | Shrews: thanks | 13:13 |
| *** dustins has joined #openstack-infra | 13:13 | |
| fungi | mordred: dstufft: yeah, i honestly don't remember the details for what we ran into last time | 13:17 |
| fungi | but definitely in favor of pip install dir being like make sdist then pip install | 13:18 |
| mkj_wk | fungi: I had another thought about my tracking mode patch that might further simplify things... Got a minute to ponder this with me? | 13:18 |
| *** gokrokve has joined #openstack-infra | 13:18 | |
| fungi | mkj_wk: i can ponder in parallel at least | 13:18 |
| mkj_wk | :) | 13:19 |
| mkj_wk | so I was thinking that maybe if options.track is set, I should set the tracking branch when downloading a patch for review | 13:19 |
| mkj_wk | in which case perhaps could either remove or separate the "query the remote gerrit for which branch to submit an untracked branch against" | 13:20 |
| mkj_wk | the "query the remote / interactively ask about remote branch" might make sense as a separare feature | 13:20 |
| mordred | mkj_wk: I think that soudns like a non-terrible thing to try | 13:22 |
| *** gokrokve has quit IRC | 13:22 | |
| mordred | mkj_wk: I've been running locally with an unmerged patch that removes the gerrit remote and instead does a split-remote where the push-url is different than the fetch-url ... so I'm personally more in favor of encoding things we know into the git remote config | 13:22 |
| openstackgerrit | Christian Berendt proposed openstack-infra/project-config: Add new project faafo to Stackforge https://review.openstack.org/164668 | 13:25 |
| *** amotoki has quit IRC | 13:26 | |
| *** tiswanso has joined #openstack-infra | 13:26 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Sometimes clouds have strings for image https://review.openstack.org/164691 | 13:26 |
| *** signed8bit has joined #openstack-infra | 13:27 | |
| *** jamesmcarthur has quit IRC | 13:27 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Small fixes found working on ansible modules https://review.openstack.org/162140 | 13:27 |
| *** claudiub_ has joined #openstack-infra | 13:29 | |
| *** amitgandhinz has joined #openstack-infra | 13:29 | |
| mordred | fungi: can you give me a +3 on https://review.openstack.org/#/c/161963/ ? I need to get a shade release out to fix an upstream ansible bug | 13:29 |
| *** bknudson has joined #openstack-infra | 13:30 | |
| *** sandywalsh has joined #openstack-infra | 13:31 | |
| *** claudiub has quit IRC | 13:32 | |
| *** ChuckC has quit IRC | 13:33 | |
| *** dimsum__ is now known as dims | 13:33 | |
| *** ddieterly has joined #openstack-infra | 13:35 | |
| *** spzala has joined #openstack-infra | 13:35 | |
| openstackgerrit | Aleksey Ripinen proposed openstack-infra/storyboard: Added request statistic https://review.openstack.org/164696 | 13:35 |
| fungi | mkj_wk: yeah, that sounds like an entirely reasonable approach | 13:37 |
| *** mriedem has joined #openstack-infra | 13:37 | |
| *** dboik has joined #openstack-infra | 13:39 | |
| *** dboik has quit IRC | 13:39 | |
| *** dboik has joined #openstack-infra | 13:40 | |
| *** sabeen1 has joined #openstack-infra | 13:42 | |
| *** devvesa has quit IRC | 13:44 | |
| *** sambetts_ is now known as sambetts | 13:45 | |
| mkj_wk | been investigating... One thing shows up: fetch_review() was where I first got the idea of just grabbing the first result that came back | 13:45 |
| mkj_wk | for this to work right, I think I'd need to interactively ask for the branch name in fetch_review if more than one is returned | 13:46 |
| *** gmann_ has joined #openstack-infra | 13:46 | |
| *** bcourt has joined #openstack-infra | 13:47 | |
| *** spredzy|afk is now known as spredzy | 13:48 | |
| mordred | fungi: the nodepool graphs look scary at this time on monday morning | 13:49 |
| fungi | mkj_wk: well, if the review is fetched by number rather than change-id it's unambiguous | 13:51 |
| mkj_wk | fungi: Oh, good, that makes it easier ☺ | 13:51 |
| *** mjturek1 has joined #openstack-infra | 13:51 | |
| mkj_wk | I'll look into that later today, probably won't be terrible to do (famous last words...) ☺ | 13:52 |
| fungi | the change number is a unique key in the database, the change-id is only unique for a given project+branch conbination | 13:52 |
| fungi | er, combination | 13:52 |
| *** BobBall_AWOL is now known as BobBall | 13:52 | |
| *** gmann_ has quit IRC | 13:53 | |
| mordred | fungi: actualy - can you load the zuul status page? | 13:53 |
| *** garyh has joined #openstack-infra | 13:53 | |
| fungi | checking | 13:53 |
| mordred | fungi: mine is hanging on loading the things that aren't the nodepool graphs | 13:53 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Expand scope for neutron dbdeadlock bug 1432065 https://review.openstack.org/164703 | 13:53 |
| uvirtbot | Launchpad bug 1432065 in neutron "DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'DELETE FROM ipallocationpools WHERE ipallocationpools.id" [Critical,Confirmed] https://launchpad.net/bugs/1432065 | 13:53 |
| openstack | bug 1432065 in neutron "DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'DELETE FROM ipallocationpools WHERE ipallocationpools.id" [Critical,Confirmed] https://launchpad.net/bugs/1432065 | 13:53 |
| Kiall | mordred: looks dead to me | 13:53 |
| uvirtbot | Launchpad bug 1432065 in neutron "DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'DELETE FROM ipallocationpools WHERE ipallocationpools.id" [Critical,Confirmed] | 13:53 |
| mordred | Kiall: k. at least it's not just me | 13:53 |
| *** tnovacik_ has quit IRC | 13:53 | |
| fungi | hrm, i think it's not returning status.json | 13:54 |
| mordred | server doesn't seem overloaded | 13:54 |
| mordred | fwiw | 13:54 |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Add stackforge/puppet-setproxy https://review.openstack.org/164197 | 13:55 |
| *** gokrokve has joined #openstack-infra | 13:56 | |
| *** fdegir has quit IRC | 13:56 | |
| dansmith | mordred: fungi: I can't | 13:56 |
| dansmith | I'm getting a "no route to host" even | 13:56 |
| *** fdegir has joined #openstack-infra | 13:56 | |
| dansmith | like a default firewall rule is in place to reject (if the host is actually up) | 13:57 |
| mordred | host is up | 13:57 |
| mordred | checking firewall | 13:57 |
| Kiall | NET_CONNECTION_REFUSED showing for the status.json call for me... | 13:58 |
| mordred | REJECT all -- anywhere anywhere reject-with icmp-host-prohibited | 13:58 |
| mordred | fungi: ^^ | 13:58 |
| *** radez is now known as radez_g0n3 | 13:58 | |
| dansmith | mordred: run with -v | 13:58 |
| dansmith | mordred: that could be tied to a specific interface | 13:58 |
| mordred | 990K 50M REJECT all -- any any anywhere anywhere reject-with icmp-host-prohibited | 13:58 |
| dansmith | or not | 13:58 |
| dansmith | :) | 13:58 |
| fungi | except i can still get to the http service itself | 13:59 |
| fungi | also zuul's log indicates it's still running jobs | 13:59 |
| fungi | and processing results/events | 13:59 |
| mordred | yah - it seems to still be cnnected to gerri | 13:59 |
| mordred | fungi: you can get to zuul.openstack.org/status.json? | 14:00 |
| fungi | oho, i wonder if my http+https patch just merged and is at fault for some reason | 14:00 |
| *** gokrokve_ has joined #openstack-infra | 14:00 | |
| mordred | it did just merge | 14:00 |
| mordred | I was just looking at that | 14:00 |
| mordred | fungi: SSLCertificateFile /etc/ssl/certs/*.pem | 14:02 |
| mordred | fungi: that doesn't look quite right :) | 14:02 |
| fungi | SSLCertificateFile: file '/etc/ssl/certs/*.pem' does not exist or is empty | 14:02 |
| fungi | yeah | 14:02 |
| fungi | aha, the vhost name is apparently * there | 14:02 |
| mordred | hahaha | 14:02 |
| fungi | patch on the way | 14:02 |
| *** ildikov has quit IRC | 14:03 | |
| *** garyh has quit IRC | 14:03 | |
| *** peristeri has joined #openstack-infra | 14:03 | |
| mordred | fungi: in fact - apache is just flat not runnign there | 14:04 |
| *** gokrokve has quit IRC | 14:04 | |
| fungi | yep | 14:04 |
| mordred | love it | 14:04 |
| mordred | I really do wish we had our non-ssl/ssl wrapper pattern more re-usable in our puppet | 14:05 |
| dansmith | fungi: is that important? | 14:05 |
| fungi | when i thought i was able to hit the site, it was actually cached in my browser | 14:05 |
| mordred | I mean, none of the options to do that are happy | 14:05 |
| fungi | dansmith: is what important? | 14:05 |
| dansmith | fungi: running apache.. joking.. :P | 14:05 |
| mordred | fungi: I tink the real solution is taht apache is clearly broken and we should replace it with mongodb | 14:05 |
| fungi | dansmith: technically no, as long as you don't care to see the status info ;) | 14:05 |
| *** tnovacik_ has joined #openstack-infra | 14:05 | |
| dansmith | fungi: well, for the record, I do care :) | 14:06 |
| openstackgerrit | David Kranz proposed openstack-infra/project-config: Start using neutron for periodic stress job https://review.openstack.org/164705 | 14:06 |
| fungi | mordred: unlike apache, mongodb runs at web scale | 14:06 |
| * mordred mug of web scale is running out of web scale ... | 14:06 | |
| *** ddieterly has quit IRC | 14:07 | |
| *** _nadya__ has quit IRC | 14:07 | |
| fungi | mordred: it's not just the cert paths affected by this, but also the log filenames | 14:07 |
| *** doug-fish has joined #openstack-infra | 14:07 | |
| fungi | CustomLog ${APACHE_LOG_DIR}/*-access.log combined | 14:07 |
| fungi | et cetera | 14:08 |
| *** mattfarina has joined #openstack-infra | 14:08 | |
| *** nelsnelson has joined #openstack-infra | 14:09 | |
| mordred | fungi: oh -@vhost_name is * | 14:09 |
| fungi | yep | 14:10 |
| fungi | probably need @fqdn here? | 14:10 |
| mordred | $vhost_name = $::fqdn, | 14:10 |
| *** devvesa has joined #openstack-infra | 14:11 | |
| *** nelsnelson has quit IRC | 14:11 | |
| mordred | fungi: I don't see where it's being turned in to * | 14:11 |
| openstackgerrit | Jaroslav Henner proposed openstack-infra/jenkins-job-builder: Add checking of multijob phase condition. https://review.openstack.org/164669 | 14:12 |
| mordred | fungi: it's vhost_name in params | 14:14 |
| mordred | fungi: in apache::params in openstack-infra/puppet-httpd | 14:14 |
| *** Ala has quit IRC | 14:15 | |
| *** ChuckC has joined #openstack-infra | 14:15 | |
| *** markmcclain has left #openstack-infra | 14:15 | |
| *** asselin_ has joined #openstack-infra | 14:15 | |
| mordred | oh for the love of all that is holy | 14:15 |
| fungi | i bet that's inherited from the puppet apache module we forked | 14:15 |
| mordred | Rename apache2 to httpd globally | 14:15 |
| mordred | how is anything working? | 14:16 |
| fungi | maybe this was the first server to get a patch which reloaded apache? | 14:16 |
| mordred | fungi: let me try real quick in an environment ... | 14:17 |
| fungi | mordred: yeah, that variable has been * in the puppet apache module going back to 2011 | 14:18 |
| fungi | according to git blame | 14:18 |
| *** Ala has joined #openstack-infra | 14:20 | |
| *** yfried_ is now known as yfried|afk | 14:20 | |
| *** zz_dimtruck is now known as dimtruck | 14:21 | |
| mordred | fungi: switching to Httpd does not fix it | 14:22 |
| *** nelsnelson has joined #openstack-infra | 14:22 | |
| fungi | mordred: so it looks like i followed bad advice in https://review.openstack.org/#/c/155471/2/templates/zuul.vhost.erb | 14:22 |
| fungi | mordred: i'm going to go back to the thing nibalizer says is "a very old way" because it actually worked | 14:22 |
| mordred | fungi: awesome | 14:23 |
| *** timcline has joined #openstack-infra | 14:23 | |
| fungi | mordred: since nodepool.o.o is using the same "very old way" and is getting the right variable interpolation | 14:23 |
| mordred | fungi: I'm trying to sort out why we're getting the * passed through | 14:23 |
| *** yfried|afk is now known as yfried_ | 14:24 | |
| fungi | mordred: it looks like @vhost_name in the erb is not treated the same as scope.lookupvar("::zuul::vhost_name") | 14:24 |
| mordred | oh! I get what you're saying | 14:24 |
| *** garyh has joined #openstack-infra | 14:24 | |
| fungi | or is there a shorter form of that which will get the value we're setting? | 14:25 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:25 | |
| *** BharatK has quit IRC | 14:26 | |
| mordred | fungi: actually - we may not be passing in vhost_name right ... | 14:26 |
| mordred | got it | 14:26 |
| *** amotoki has joined #openstack-infra | 14:26 | |
| fungi | if so, wow we've been doing it wrong all over the place | 14:27 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-zuul: Pass vhost_name in to apache::vhost https://review.openstack.org/164711 | 14:27 |
| mordred | fungi: ^^ that works | 14:27 |
| mordred | and yes - I think if we start passing in vhost_name, we can move to the @vhost_name syntax in the template files | 14:27 |
| mordred | but as it was, we weren't passing it in, so the scope lookup was all wrong, and we were having to override where the template looked for data | 14:28 |
| fungi | yep, that looks reasonable | 14:28 |
| *** banix has joined #openstack-infra | 14:28 | |
| fungi | so instead of chaining up to the zuul module to decide on vhost_name it was chaining up to the httpd module | 14:28 |
| mordred | fungi: yup | 14:29 |
| fungi | makes total sense | 14:29 |
| fungi | as soon as it merges, i'll make sure apache is starting again | 14:29 |
| *** ddieterly has joined #openstack-infra | 14:31 | |
| *** yamamoto has quit IRC | 14:31 | |
| *** tiswanso has quit IRC | 14:31 | |
| dstufft | http://www.isg.rhul.ac.uk/tls/RC4mustdie.html (for anyone who get sinterested in things like proof of concept attacks against TLS ciphers ;) ) | 14:31 |
| *** otter768 has joined #openstack-infra | 14:32 | |
| *** amotoki has quit IRC | 14:32 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Fix up and document input parameters https://review.openstack.org/160184 | 14:32 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add support for keystone projects https://review.openstack.org/159941 | 14:32 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Small fixes found working on ansible modules https://review.openstack.org/162140 | 14:32 |
| *** tiswanso_ has joined #openstack-infra | 14:34 | |
| *** zz_jgrimm- is now known as jgrimm | 14:35 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Disable dogpile.cache if cache_interval is None https://review.openstack.org/161963 | 14:35 |
| *** otter768 has quit IRC | 14:36 | |
| BobBall | jeblair, asselin: How should cross-project dependencies work? Change 150267 (nova) depends on 160068 (devstack) so the 160068 change is added to my Nova queue. It's this 160068 change which ends up not having a project so triggers the NoneType issue. | 14:36 |
| *** wenlock has joined #openstack-infra | 14:37 | |
| BobBall | jeblair, asselin: It seems that gerrit.updateChange calls self.sched.getProject with the project name returned for the change, but that looks up the project from the layout - which in my case doesn't exist | 14:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Allow for passing cache class in as a parameter https://review.openstack.org/162148 | 14:38 |
| *** ghostpl_ has quit IRC | 14:39 | |
| *** ghostpl_ has joined #openstack-infra | 14:40 | |
| *** stevemar has joined #openstack-infra | 14:40 | |
| *** ChuckC has quit IRC | 14:43 | |
| *** tiswanso_ has quit IRC | 14:43 | |
| *** tiswanso has joined #openstack-infra | 14:44 | |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Change node param for ec2api rally job. https://review.openstack.org/164717 | 14:44 |
| *** tiswanso_ has joined #openstack-infra | 14:45 | |
| openstackgerrit | Merged openstack-infra/project-config: Add the create permission for stackforge/vmtp https://review.openstack.org/164044 | 14:49 |
| jeblair | mordred, fungi, nibalizer: i think we should consider declaring rocketship alignment an anti-pattern. it makes it look like far more lines are changed than actually are when reading a diff. | 14:52 |
| mordred | jeblair: ++ | 14:52 |
| mordred | jeblair: is there an easy way to turn that off in puppet-lint? | 14:52 |
| fungi | agreed. i have yet to figure out why they went to so much trouble to add that to their linter | 14:52 |
| fungi | someone really has a whitespace fetish | 14:53 |
| jeblair | To disable this check you can add --no-arrow_alignment-check to your puppet-lint command line. | 14:53 |
| fungi | being picky about indentation is one thing. being picky about inter-line columnar alignment is something else entirely | 14:53 |
| jeblair | Alternatively, if you’re calling puppet-lint via the Rake task, you should insert the following line to your Rakefile. | 14:53 |
| jeblair | PuppetLint.configuration.send('disable_arrow_alignment') | 14:54 |
| jeblair | i think we do that second thing. | 14:54 |
| jeblair | i'll propose a patch and we can collect opinions there | 14:54 |
| anteaya | having those aligned is really important in rails | 14:56 |
| jeblair | anteaya: why's that? | 14:56 |
| anteaya | I'm guessing that is where the decision is based | 14:56 |
| anteaya | I have no idea | 14:56 |
| anteaya | it just is | 14:56 |
| *** adalbas has quit IRC | 14:56 | |
| anteaya | it is an are you one of us test | 14:57 |
| mordred | ah. one of those | 14:57 |
| anteaya | right | 14:57 |
| anteaya | every language has them | 14:57 |
| fungi | anteaya: does rails make actual execution decisions based on it, or is there just a large culture of people expecting it to be formatted that way? | 14:57 |
| mordred | well, I think we can safely say that we are not one of the rails community :) | 14:57 |
| anteaya | that is one of rails | 14:57 |
| anteaya | fungi: is there a difference? | 14:57 |
| anteaya | mordred: I think that is one of your 'are you one of us' tests | 14:58 |
| mordred | anteaya: rails does some crazy interesting magic based on inferring what you mean from naming choices | 14:58 |
| anteaya | yes | 14:58 |
| anteaya | convention over configuration | 14:58 |
| mordred | anteaya: I could imagine them having rails do somethign with rocketships - although I wouldn't know what | 14:58 |
| fungi | um. yes there is a difference between the interpreter making decisions based on column alignment and developers merely expecting to see the source formatted that way | 14:58 |
| mordred | :) | 14:58 |
| anteaya | and that is one of the conventions | 14:58 |
| mordred | nod. in this case, it seems more like an openstack hacking check | 14:58 |
| *** tiswanso_ has quit IRC | 14:59 | |
| anteaya | fungi: oh okay yes, so rails the code not rails the people | 14:59 |
| anteaya | well they don't tend to review code that isn't in the format they expect | 14:59 |
| *** gordc has joined #openstack-infra | 14:59 | |
| fungi | good thing our puppet modules aren't likely to get turned into rails webapps | 14:59 |
| anteaya | whether or not one can code something that rails will run, I expect it will | 14:59 |
| anteaya | but I don't know | 14:59 |
| anteaya | I'm best at learning culture | 14:59 |
| *** tiswanso_ has joined #openstack-infra | 15:00 | |
| anteaya | so I just aligned the things | 15:00 |
| dstufft | "i think we should consider declaring rocketship alignment an anti-pattern." <- That sentence made me giggle more than it had a right to. | 15:01 |
| mordred | totally. well, and we were doing our best to learn puppet culture | 15:01 |
| mgagne | Anyone knows how I could use JJB variables to dynamically generate a block of publishers (or anything other section) in a job? My first impression is that I would need to have an external templating engine to generate the job yaml instead | 15:01 |
| *** patrickeast has joined #openstack-infra | 15:01 | |
| anteaya | fungi: my understanding is that puppet is rails based | 15:01 |
| sdague | jeblair: I will love you so much if you disable that, alignment anywhere after the left gutter drives me nuts | 15:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-zuul: Disable arrow alignment puppet lint check https://review.openstack.org/164725 | 15:02 |
| fungi | anteaya: as far as i know puppet is just implemented in ruby. rails is a web application framework for ruby, right? | 15:02 |
| anteaya | it is | 15:02 |
| mordred | fungi: I believe puppetmaster is (for now) A rails app | 15:03 |
| sdague | fungi: at some point the puppet master was rails | 15:03 |
| jeblair | okay ^ no rush on that change -- it's a moderately big policy change, so i want to make sure everyone has time to weigh in on it | 15:03 |
| *** nelsnelson has quit IRC | 15:03 | |
| fungi | ahh, yep the puppetmaster process | 15:03 |
| *** _nadya_ has joined #openstack-infra | 15:04 | |
| jeblair | also, i apparently have awoken unable to construct english sentences correctly. but i think it gets the point across. :) | 15:04 |
| mordred | jeblair: policy change +2 - patches to unalign - -1 | 15:04 |
| jeblair | mordred: oh interesting, why not? | 15:04 |
| *** koolhead17 has quit IRC | 15:04 | |
| mordred | jeblair: because they are noise just as much as patches to align are noise | 15:05 |
| jeblair | mordred: a good point. however, at least for this, i thought it was important to see what we were getting ourselves into. :) | 15:05 |
| EmilienM | if someone admin on openstack-dev mailing list could add [puppet] tag, we agreed on that today in our weekly meeting : http://eavesdrop.openstack.org/meetings/puppet_openstack/2015/puppet_openstack.2015-03-16-13.59.html | 15:05 |
| mordred | jeblair: totally - I'm +2 on that patch | 15:05 |
| mordred | I just want us to be clear that we do not think it would be "helpful" for a bunch of people to get ATC credit by "fixing" our modules :) | 15:05 |
| jeblair | mordred: if we come to consensus on that point, i'm also happy to reroll this with just the policy change. | 15:06 |
| *** armax has joined #openstack-infra | 15:06 | |
| *** prad has joined #openstack-infra | 15:07 | |
| *** tonytan4ever has joined #openstack-infra | 15:08 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Pass vhost_name in to apache::vhost https://review.openstack.org/164711 | 15:08 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Sometimes clouds have strings for image https://review.openstack.org/164691 | 15:12 |
| mtreinish | jeblair, pleia2 : so continuing the subunit2sql db migration from Fri. if the ansible timeout on puppet-apply is only 30min I'm pretty sure we'll never be able to run the migrations in puppet | 15:12 |
| mtreinish | should I remove that bit from the policy? | 15:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Handle image name for boot from volume https://review.openstack.org/164691 | 15:12 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Revert "Cap the subunit2sql version in the nodepool scripts" https://review.openstack.org/164730 | 15:13 |
| jeblair | mordred: ^ you should pay attention to the subunit2sql convo too | 15:13 |
| mordred | mtreinish: the ansible timeout is settable | 15:13 |
| jeblair | mordred: but we have "timeout 30 puppet agent ..." | 15:13 |
| mordred | jeblair: yes - but the 30 is a parameter | 15:13 |
| jeblair | right | 15:13 |
| jeblair | mordred: global or per host? | 15:14 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Add Parameterized Remote Trigger Plugin support. https://review.openstack.org/150327 | 15:14 |
| jeblair | mtreinish: any idea how long it might take? | 15:14 |
| mordred | jeblair: well ... it's per-host, but in an awkward way | 15:14 |
| mordred | jeblair: I can make it less-awkward to do per-host - but I think that's a few patchsets down the line | 15:14 |
| mtreinish | jeblair: long, I was assuming at least an hour, but I haven't run the migration on a db the size of the infra one | 15:14 |
| mordred | jeblair: however, I _do_ think most of those patchsets are up for review | 15:14 |
| openstackgerrit | Brant Knudson proposed openstack/requirements: Update cryptography to >= 0.7 https://review.openstack.org/164731 | 15:14 |
| mtreinish | it's doing an alter table on >65M rows I assume it will be really slow | 15:15 |
| mordred | jeblair: I could put up one additional one that shows a POC for setting per-host timeouts | 15:15 |
| openstackgerrit | Brant Knudson proposed openstack/requirements: Update cryptography to >= 0.7 https://review.openstack.org/164731 | 15:15 |
| *** koolhead17 has joined #openstack-infra | 15:15 | |
| *** david-lyle_afk is now known as david-lyle | 15:15 | |
| jeblair | mordred: thing is, most of the time 30 is fine, it's just we have one event where we think 1 hour or more might be necessary | 15:16 |
| mordred | yeah | 15:16 |
| jeblair | mtreinish: would it be safe to run the migration in the background? | 15:16 |
| mordred | jeblair: perhaps a migration like this should be expressed as a one-off playbook rather than as part of teh state expression | 15:16 |
| mordred | jeblair: I mean, we don't really have a model for that currently | 15:16 |
| mtreinish | jeblair: it should be | 15:16 |
| mtreinish | jeblair: but in the worst case we'd probably just lose the data from when the migration was running, not a big deal | 15:17 |
| mordred | jeblair: ah - yeah - background is probably sane for this | 15:17 |
| *** achanda has joined #openstack-infra | 15:17 | |
| jeblair | so maybe we background it for now, and we consider this requirements gathering for future changes? | 15:17 |
| mordred | ++ | 15:18 |
| *** dannywilson has joined #openstack-infra | 15:18 | |
| mtreinish | jeblair: yeah, that's fine | 15:18 |
| mordred | I would love to think of a better way to handle one-time maint tasks - I've never really be crazy about the idea of putting them in puppet, but also not crazy about them being "hey, admin, go run this script" tasks | 15:18 |
| mtreinish | mordred: heh, yeah that was the same tradeoff I had originally when I wrote the puppet for the subunit2sql stuff | 15:19 |
| mtreinish | it ended up being easy enough to trigger the migrations from inside puppet though | 15:19 |
| EmilienM | fungi: you mentionned t | 15:19 |
| EmilienM | it was manual, right? | 15:20 |
| EmilienM | fungi: the openstack-dev ML config | 15:20 |
| *** bhunter71 has joined #openstack-infra | 15:20 | |
| *** xgerman has joined #openstack-infra | 15:20 | |
| mtreinish | jeblair: to manually run the migration it should just be 'subunit2sql-db-manage --config-file /etc/subunit2sql.conf upgrade head' on logstash.o.o, pleia2 checked on fri and everything was in place to do it | 15:21 |
| *** thedodd has joined #openstack-infra | 15:21 | |
| fungi | EmilienM: defining official mailing list subject tags? yes, it's a manual configuration applied via the mailman admin webui by whoever the admins are for a given mailing list | 15:22 |
| jeblair | mtreinish: what user? | 15:22 |
| EmilienM | fungi: ok, I'll contact openstack-dev admins. | 15:22 |
| mtreinish | jeblair: oh, I'm not sure it really matters, just any user that has read perms on the config file | 15:22 |
| jeblair | mtreinish: i can run that now if you want to set up puppet to run the migration in the background next time | 15:22 |
| mtreinish | jeblair: sure will do | 15:23 |
| openstackgerrit | Christian Berendt proposed openstack-infra/project-config: Add new project faafo to Stackforge https://review.openstack.org/164668 | 15:23 |
| *** nelsnelson has joined #openstack-infra | 15:23 | |
| fungi | EmilienM: according to http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev it's ttx and reed so you could try hitting one of them up in irc | 15:24 |
| *** ashleighfarnham has joined #openstack-infra | 15:24 | |
| mordred | dhellmann: what's the "right" way to check if something is a string/unicode that works in both 2 and 3? | 15:24 |
| EmilienM | fungi: I was sending an email to them :) | 15:24 |
| fungi | that works too i'm sure | 15:25 |
| *** achanda has quit IRC | 15:25 | |
| openstackgerrit | Cedric Brandily proposed openstack/requirements: Bump cliff up to 1.10.0 https://review.openstack.org/164733 | 15:26 |
| *** reed has joined #openstack-infra | 15:26 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Upload cover/ to swift not coverage/ https://review.openstack.org/164734 | 15:26 |
| *** mpopow has joined #openstack-infra | 15:27 | |
| dougwig | good morning infra, simple one, needs one more core, hopefully before it's third merge conflict. combining two jobs into one node. :-) https://review.openstack.org/#/c/160915/ | 15:28 |
| *** mjturek1 has quit IRC | 15:28 | |
| *** mjturek1 has joined #openstack-infra | 15:29 | |
| mordred | dougwig: I will always remove pylint references | 15:29 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql: Run subunit2sql migrations in the background https://review.openstack.org/164736 | 15:29 |
| mtreinish | jeblair: ^^^ | 15:29 |
| openstackgerrit | Peter Hamilton proposed openstack/requirements: Update PyKMIP version in requirements https://review.openstack.org/164449 | 15:29 |
| mtreinish | assuming there isn't a magic puppet way to background something | 15:29 |
| *** rm_work|away is now known as rm_work | 15:30 | |
| dougwig | mordred: it's false annoyance rate does seem to be pretty high. | 15:30 |
| *** markmcclain has joined #openstack-infra | 15:33 | |
| *** ajmiller has joined #openstack-infra | 15:33 | |
| GheRivero | exit | 15:33 |
| jeblair | yeah, most projects don't use it because of that | 15:33 |
| *** HeOS has quit IRC | 15:36 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/elastic-recheck: Check for logs uploaded by swift https://review.openstack.org/164742 | 15:36 |
| *** Longgeek_ has quit IRC | 15:36 | |
| clarkb | mtreinish: mriedem ^ should cleanup some of the results not ready messages in #openstack-qa that happened over the weekend | 15:36 |
| *** HeOS has joined #openstack-infra | 15:37 | |
| mriedem | clarkb: thanks, +W | 15:38 |
| *** dhritishikhar has joined #openstack-infra | 15:39 | |
| mtreinish | oh, I'm too slow... | 15:39 |
| ashishb | Hi everyone, I have a quick query about getting a commit rerun through jenkins | 15:42 |
| openstackgerrit | Merged openstack-infra/project-config: new-project: stackforge/senlin https://review.openstack.org/163292 | 15:42 |
| openstackgerrit | Merged openstack-infra/project-config: neutron-*aas pylint jobs moving into pep8 job https://review.openstack.org/160915 | 15:42 |
| ashishb | ok, so here is the scenario I have: I submitted a patch and then there was some error in CI due to which it could not close the pep8 stuff on it | 15:43 |
| ashishb | now, in my patch there is no change, however, I want the jenkins to rerun it through it's tools on it | 15:43 |
| ashishb | is there a way to do that ..? | 15:43 |
| *** ashleighfarnham has quit IRC | 15:43 | |
| *** ildikov has joined #openstack-infra | 15:44 | |
| reed | was there a resolution that projects applying to 'big tent' should also get a topic on the -dev mailing list? (in other words, is there an agreement that new topics should be added to the mlist?) | 15:46 |
| *** ZZelle_ has joined #openstack-infra | 15:46 | |
| fungi | ashishb: leave a review comment starting with the word "recheck" (you can add whatever else you want after it) | 15:47 |
| *** dhritishikhar has quit IRC | 15:47 | |
| *** _nadya_ has quit IRC | 15:47 | |
| fungi | reed: i'm not aware of any resolution regarding that. i'd say it's up to the -dev mailing list admins as to whether they want to support such things | 15:47 |
| jeblair | reed: i think topics for the tc require notification to -dev a week in advance, so i think that's probably included | 15:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Handle image name for boot from volume https://review.openstack.org/164691 | 15:48 |
| reed | fungi, I wouldn't want that responsibility, frankly: traffic is already huge, I think we should have a discussion | 15:48 |
| *** achanda has joined #openstack-infra | 15:48 | |
| jeblair | reed: i misunderstood | 15:49 |
| reed | jeblair, does that mean that first the project needs to apply to the TC and then the topic gets created? | 15:49 |
| *** wuhg has quit IRC | 15:49 | |
| mordred | jeblair, fungi, clarkb: could you please review the patch above? ^^ there is a bug in the wild that ansible upstream uncovered and I'd like to get that fix out the door | 15:49 |
| mordred | jeblair, fungi, clarkb: (the patch has been tested as fixing the issue for the reporter) | 15:49 |
| jeblair | reed: you meant mailman topic, not, like, discussing the application on the list :) | 15:49 |
| reed | jeblair, yes, mailman :) | 15:49 |
| jeblair | reed: i don't recall that being discussed | 15:50 |
| *** nilasae has joined #openstack-infra | 15:50 | |
| *** mpopow has quit IRC | 15:50 | |
| reed | I'm scared to start another potentially long discussion on -dev about this though :) | 15:50 |
| fungi | mordred: btw your fix to puppet-zuul to explicitly pass vhost_name definitely worked and the zuul status is available again | 15:51 |
| mordred | fungi: woot | 15:51 |
| clarkb | mordred I think openstack can return utf-8 on any query | 15:51 |
| dhellmann | sdague, ttx: can we prioritize capping the oslo libs early this week, so we can sync the results into projects before their freeze? https://review.openstack.org/#/c/162656/ | 15:51 |
| *** kaisers has joined #openstack-infra | 15:51 | |
| mordred | clarkb: the issue here is that usually server.image is a dict-like object | 15:51 |
| jeblair | reed: i agree that traffic is huge, however, it's huge at least partly because stackforge projects already use the list. i'm not sure that big-tent will increase the list traffic that much (though it may increase the desire for more official topics) | 15:51 |
| *** tqtran has joined #openstack-infra | 15:51 | |
| mordred | clarkb: but sometimes it's string-like | 15:51 |
| mordred | clarkb: apparently | 15:52 |
| clarkb | oh huh | 15:52 |
| sdague | dhellmann: that forces a bunch of bumps as well? | 15:52 |
| fungi | mordred: any reason not to just always recast it? simplifies the logic there (unless i'm misreading) | 15:52 |
| ctlaugh | kaisers: sorry - I didn't see you in my nic list earlier | 15:52 |
| *** dhritishikhar has joined #openstack-infra | 15:52 | |
| ashishb | fungi: thanks for the "recheck" tip | 15:52 |
| ashishb | :) | 15:52 |
| reed | jeblair, makes sense, I'll talk to ttx about that too (the other list admin) | 15:52 |
| mordred | fungi: yes - I'm looking for "not a dict" | 15:52 |
| kaisers | ctlaugh: np | 15:52 |
| sdague | I thought we were going to do actual caps and not trust the semver thing | 15:52 |
| ttx | jeblair: actually it may reduce traffic, since at some point stackforge projects would move to their own list | 15:53 |
| mordred | fungi: it has nothing to do with unicode | 15:53 |
| dhellmann | sdague: yes, so that the minimum is the stable version being supported and matches what we're actually testing in the gate right now | 15:53 |
| *** Qiming has quit IRC | 15:53 | |
| ttx | (once all the "openstack" ones actually moved) | 15:53 |
| mordred | fungi: but if x = dict(), then unicode(x) != x | 15:53 |
| fungi | mordred: aha, got it | 15:53 |
| *** gokrokve_ has quit IRC | 15:53 | |
| *** ashishb has quit IRC | 15:53 | |
| mordred | fungi: it's, admittedly, a sad hack - however, there does not seem to be a solid way to test for that that it's valid in python2 and python3 | 15:53 |
| fungi | mordred: yeah at first i thought you were saying sometimes it was unicode and sometimes it was an encoded string | 15:53 |
| fungi | so, wow that's some bad api behavior | 15:54 |
| mordred | fungi: nope. should I update the comment to be better? | 15:54 |
| mordred | fungi: and YES it is | 15:54 |
| ctlaugh | kaisers: so, do you have any information you could share on sos-ci, how it's structured, how well it's working, etc? I am currently trying to setup the os-ext stuff, but was curious about what else others are using and how well it's working. | 15:54 |
| ttx | dhellmann: any reason debtcollector is using >= and not ~= ? | 15:54 |
| fungi | mordred: better comment wouldn't hurt, but my brain jumping to encoding-related conclusions isn't entirely your fault. more like a python knee-jerk reaction | 15:55 |
| dhellmann | ttx: oops, I'll fix that | 15:55 |
| ttx | dhellmann: triggered my inconsistency radar | 15:55 |
| kaisers | ctlaugh: I'm not the author but the short overview is: A bunch of python scripts that use ansible scripts. Python stuff analyses gerrit stream and triggers the ansible scripts that set up devstacks, run the tests, fetch the logs and upload them to a log server. Python scripts then feedback to gerrit. | 15:55 |
| *** harlowja_at_home has joined #openstack-infra | 15:56 | |
| openstackgerrit | Doug Hellmann proposed openstack/requirements: Cap Oslo libraries for kilo https://review.openstack.org/162656 | 15:56 |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/project-config: Add Neutron DVR aiopcpu job for multi-node testing https://review.openstack.org/158415 | 15:56 |
| dhellmann | ttx: yeah, that was just me being sloppy when I updated that line after the first patch | 15:56 |
| sdague | honestly, it seems odd to make this more restrictive than it needs to be, but I don't have the bandwidth this week to think through the implications, as I'm trying to address a couple of nova issues pre freeze | 15:57 |
| sdague | so if you think that's what we want, so be it | 15:57 |
| *** ashleighfarnham has joined #openstack-infra | 15:57 | |
| *** gokrokve has joined #openstack-infra | 15:57 | |
| ctlaugh | kaisers: any issues scaling? | 15:58 |
| fungi | dhellmann: sdague: it's worth noting that ~= only works in very recent pip releases. not sure if that matters in this case, but putting it out there | 15:58 |
| kaisers | ctlaugh: it's not completely running so far, and i don't plan to scale too far. Therefore i did not test too much in that direction | 15:59 |
| *** gyee has joined #openstack-infra | 15:59 | |
| dhellmann | fungi: I'm OK with assuming current tools as long as that's what we're using in our gate. | 15:59 |
| *** tnovacik_ has quit IRC | 15:59 | |
| fungi | dhellmann: fine by me then | 15:59 |
| *** BharatK has joined #openstack-infra | 15:59 | |
| dhellmann | sdague: not sure what you mean there by "more restrictive"? | 15:59 |
| sdague | dhellmann: I'm +2, I think it makes sense for jogo to weigh in before an approv | 15:59 |
| sdague | dhellmann: by uplifting the mins as well | 16:00 |
| dhellmann | sdague: sure, he was +1 on an earlier version but I added some updates | 16:00 |
| dhellmann | sdague: ok, we have some apps relying on more recent versions of libs than the mins said otherwise, so I thought raising them all would make sense. we don't do that for every lib release, but maybe we should when we add features. | 16:01 |
| *** marun has quit IRC | 16:01 | |
| sdague | yeh, there are trade offs all around | 16:01 |
| *** sarob has joined #openstack-infra | 16:02 | |
| *** fdegir has quit IRC | 16:02 | |
| *** mpopow has joined #openstack-infra | 16:02 | |
| ctlaugh | kaisers: Is this it? https://github.com/j-griffith/sos-ci Any tricks to configuring it? The documentation there in github is a bit sparse. | 16:02 |
| *** yfried_ has quit IRC | 16:02 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Disable arrow alignment puppet lint check https://review.openstack.org/164725 | 16:02 |
| kaisers | ctlaugh: yeah, that's it. I did some small mods (github/casusbelli/sos-ci), primarily for loggin | 16:02 |
| kaisers | s/loggin/logging/ | 16:03 |
| *** carl_baldwin has joined #openstack-infra | 16:03 | |
| clarkb | sdague: do you know what I need to do to get nova to use https://review.openstack.org/#/c/163514/ ? | 16:03 |
| clarkb | sdague: I am not sure where libvirt/qemu/nova want that cpu definition to go | 16:04 |
| *** achanda has quit IRC | 16:04 | |
| dstufft | There's a new bandersnatch out, you'll want to update to it at somepoint. It's not urgent though. If you don't update to it before I get around to making some archiectural changes to PyPI (not on my immediate TODO list, but short term) you'll stop syncing | 16:04 |
| *** skolekonov has quit IRC | 16:05 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Handle image name for boot from volume https://review.openstack.org/164691 | 16:05 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Clean up race condition in functional tests https://review.openstack.org/164761 | 16:05 |
| asselin | BobBall, interesting find | 16:05 |
| mordred | clarkb: I believe that race condition patch backs up what you were saying about that test being racey :) | 16:06 |
| openstackgerrit | Merged openstack-infra/project-config: Add stackforge/puppet-setproxy https://review.openstack.org/164197 | 16:06 |
| ctlaugh | kaisers: thank you - I'll take a look at your fork | 16:06 |
| *** marun has joined #openstack-infra | 16:06 | |
| zaro | morning | 16:06 |
| clarkb | dstufft: thank you for the heads up | 16:06 |
| sdague | clarkb: you have to modify /usr/share/libvirt/cpu_map.xml | 16:07 |
| sdague | so that's going to be an inline edit of that file, the way it's nested, you can't just append | 16:07 |
| kaisers | ctlaugh: welcome | 16:07 |
| sdague | check out your local copy of it | 16:07 |
| clarkb | sdague: thanks that is what I needed | 16:08 |
| *** rfolco has quit IRC | 16:08 | |
| *** ZZelle_ has quit IRC | 16:08 | |
| *** rfolco has joined #openstack-infra | 16:08 | |
| clarkb | jeblair: just hit https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning after upgrading gertty for right aligned line numbers | 16:08 |
| clarkb | that error message bugs me too. They say things will fail when they should succeed because of missing features. Isn't that failing gracefully when it comes to security? how is that a warning | 16:09 |
| *** xgerman has quit IRC | 16:09 | |
| openstackgerrit | Merged openstack-infra/system-config: Added Valid OAuth client configuration to storyboard manifest. https://review.openstack.org/158434 | 16:09 |
| sdague | clarkb: you might need to restart libvirt after changing that file as well, I don't know | 16:09 |
| clarkb | "I failed safely for you turn on all the alarms!" | 16:09 |
| clarkb | sdague: ok, we may need to bake some of this into devstack since it installs libvirt | 16:09 |
| openstackgerrit | Merged openstack-infra/system-config: Hostname and CORS domains are now configurable. https://review.openstack.org/158474 | 16:09 |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525 | 16:10 |
| fungi | clarkb: it's a passive-aggressive way of prodding application developers to update their use of your library by annoying their users | 16:10 |
| sdague | clarkb: sure | 16:10 |
| sdague | I'd say hack it into d-g for now to see if you can get anything working, then we can pull it back into devstack once we know what it needs to be | 16:11 |
| mordred | fungi: now, if only rax would update their ssl certs so that requests warning about deprecated features in certs wouldn't annoy me | 16:11 |
| *** openstackgerrit has quit IRC | 16:11 | |
| clarkb | sdague: ok | 16:11 |
| *** openstackgerrit has joined #openstack-infra | 16:12 | |
| *** dhritishikhar has quit IRC | 16:12 | |
| clarkb | fungi: did you end up figuring out what was wrong with dfw over the weekend? | 16:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Fix up and document input parameters https://review.openstack.org/160184 | 16:12 |
| clarkb | anyways gertty is really almost unuseable with this warning popping up constantly. I am going to try using their pyopenssl fix | 16:13 |
| clarkb | will push that up if I get it working | 16:13 |
| fungi | clarkb: it cleared up on its own | 16:13 |
| fungi | i didn't look deeper into it | 16:14 |
| *** marun_ has joined #openstack-infra | 16:14 | |
| fungi | clarkb: but really there wasn't much to look into. we were calling nova boot and it was taking an indefinite amount of time for it to return an instance in that region. no sign of stray alien nodes or anything, nothing mentioned on the rax status page for dfw either | 16:15 |
| BobBall | asselin, jeblair: For now I've just assumed the same as the official layout.yaml... every project must be defined. I don't think that's a long-term workable solution though as there is constant change in the projects defined in the official layout.yaml | 16:15 |
| *** marun has quit IRC | 16:15 | |
| *** amotoki has joined #openstack-infra | 16:16 | |
| fungi | clarkb: and it was across all node types, so not a bad image or anything | 16:16 |
| mestery | Infra folks: https://review.openstack.org/#/c/164678/ this is the fix to unblock the grenade job failure which is currently blocking all neutron patches, if there is an oppurtunity to get it promoted, that would be great! | 16:16 |
| mestery | EmilienM: ^^^^ | 16:16 |
| *** claudiub_ has quit IRC | 16:16 | |
| fungi | clarkb: if it persisted i was going to open a trouble ticket with them | 16:16 |
| *** radez_g0n3 is now known as radez | 16:16 | |
| *** mpavone has quit IRC | 16:17 | |
| *** patrickeast has quit IRC | 16:17 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Check for logs uploaded by swift https://review.openstack.org/164742 | 16:17 |
| *** btran has joined #openstack-infra | 16:17 | |
| clarkb | fungi: thanks | 16:17 |
| *** mpopow has quit IRC | 16:18 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525 | 16:20 |
| mordred | BobBall, asselin: I may be missing context on the problem ... although it's possible that jeblair's zuulv3 work might be helpful (as in, it's possible that we're in violent agreement on long-term not being workable) | 16:20 |
| *** marun_ has quit IRC | 16:20 | |
| asselin | BobBall, looking at the code, there's quite a few test cases....shouldn't be too hard to add a new one that reproduces the issue. | 16:20 |
| *** amotoki has quit IRC | 16:21 | |
| *** mpopow has joined #openstack-infra | 16:23 | |
| *** _nadya_ has joined #openstack-infra | 16:23 | |
| anteaya | clarkb dstufft thank you I have added it to tomorrow's meeting agenda: https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting | 16:23 |
| *** scheuran has quit IRC | 16:24 | |
| dstufft | clarkb: fwiw that warning isn't "I failed when it was dangerous to do so" that warning is "I failed when I otherwise could have successfully and safely not failed, but the SSL library isn't good enough" | 16:26 |
| pleia2 | good morning | 16:26 |
| dstufft | like when requests tried to remove the 1024 bit roots (which are not exactly safe to keep around), but couldn't because OpenSSL is bad | 16:27 |
| clarkb | dstufft: sure but it still failed | 16:27 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Handle image name for boot from volume https://review.openstack.org/164691 | 16:27 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Clean up race condition in functional tests https://review.openstack.org/164761 | 16:27 |
| dstufft | that broke a ton of people, that can be fixed on platforms that don't get that error | 16:27 |
| mordred | clarkb: good point - that was actually two different approaches to cleaning it up combined into one :) | 16:27 |
| clarkb | dstufft: seems like if you fail gracefully then who cares | 16:27 |
| anteaya | pleia2: morning | 16:28 |
| clarkb | dstufft: instead not only do I have to update my application but I have to update it in a way that accomodates the embedded version of urllib3 in requests :/ | 16:28 |
| clarkb | its a terrible interface for both the user and the dev | 16:29 |
| *** marun_ has joined #openstack-infra | 16:29 | |
| dstufft | clarkb: so you'd rather it just flat out broke without any warning? | 16:29 |
| *** yamamoto has joined #openstack-infra | 16:29 | |
| clarkb | dstufft: if it can do so gracefully then yes. aiui this behavior has not changed during the lifetime of python2 | 16:30 |
| *** dboik_ has joined #openstack-infra | 16:30 | |
| clarkb | dstufft: eg its always broken this way | 16:30 |
| EmilienM | is it possible to promote 164678 ? | 16:30 |
| dstufft | What behavior hasn't changed? | 16:30 |
| clarkb | dstufft: the failing | 16:30 |
| clarkb | because ssl stdlib isn't good enough | 16:30 |
| dstufft | the stdlib ssl is good enough in newer versions of Python | 16:31 |
| dstufft | 2.7.9, 3.2+ | 16:31 |
| clarkb | as an alternative urllib3 could hard depend on pyOpenSSL and just make this work | 16:31 |
| *** baoli has quit IRC | 16:31 | |
| dstufft | installing pyopenssl is a worka round for older versions of Python | 16:31 |
| *** arxcruz has quit IRC | 16:31 | |
| *** salv-orlando has quit IRC | 16:31 | |
| dstufft | in particular, it looks to see if the Python version has a SSLContext object | 16:32 |
| *** tiswanso_ has quit IRC | 16:32 | |
| *** baoli_ has joined #openstack-infra | 16:32 | |
| *** otter768 has joined #openstack-infra | 16:32 | |
| *** tiswanso has joined #openstack-infra | 16:33 | |
| *** dboik has quit IRC | 16:33 | |
| *** hashar has quit IRC | 16:33 | |
| *** gokrokve has quit IRC | 16:34 | |
| clarkb | dstufft: ok so the most recent version of python2 is fixed which is included in neither ubuntu trusty or rhel7 | 16:34 |
| clarkb | though knowing rhel theymay have backported this stdlib fix to their python2.7.5 | 16:34 |
| sdague | clarkb: which would have been unfortunately, because I believe it was literally the only functional change in 2.7.9 | 16:34 |
| *** marun_ has quit IRC | 16:35 | |
| sdague | it was released to actually put a non broken ssl implementation out there | 16:35 |
| dstufft | clarkb: I think Trusty *might* be backporting 2.7.9 fixes, Debian Jessie has 2.7.9 I know that | 16:35 |
| clarkb | isn't wheezy the current release? | 16:35 |
| sdague | https://www.python.org/downloads/release/python-279/ | 16:35 |
| *** gokrokve has joined #openstack-infra | 16:36 | |
| clarkb | speaking of ubuntu backports, fungi any idea where we are with python3.4 stuff? | 16:36 |
| dstufft | clarkb: Wheezy is current yes, Jessie is testing | 16:36 |
| *** sandywalsh has quit IRC | 16:36 | |
| *** dhritishikhar has joined #openstack-infra | 16:37 | |
| *** otter768 has quit IRC | 16:37 | |
| fungi | clarkb: not a peep. i think ubuntu is abandonware | 16:37 |
| *** tsg has joined #openstack-infra | 16:37 | |
| fungi | which is why i'm focusing on being able to spin u debian jessie servers with nodepool so we can do our py3k testing there | 16:39 |
| fungi | er, spin up | 16:39 |
| *** funzo has quit IRC | 16:39 | |
| *** hdd has joined #openstack-infra | 16:40 | |
| fungi | speaking of which, this is odd... | 16:40 |
| fungi | https://review.openstack.org/164444 | 16:40 |
| openstackgerrit | Clark Boylan proposed stackforge/gertty: Fix urllib3 InsecurePlatformWarning https://review.openstack.org/164770 | 16:41 |
| clarkb | jeblair: ^ | 16:41 |
| fungi | you can clearly see the only comment i've left on that review is setting Workflow-1 | 16:41 |
| fungi | except gerrit isn't showing it as wip at all | 16:41 |
| fungi | oh, d'oh! | 16:41 |
| fungi | nevermind | 16:41 |
| fungi | my check experimental comment unset wip | 16:42 |
| * fungi sighs | 16:42 | |
| *** funzo has joined #openstack-infra | 16:42 | |
| fungi | operator error | 16:42 |
| clarkb | ok :) | 16:42 |
| jeblair | clarkb: Id6f46e85168bba599f8380b6613d8835d65661c3 is also relevant | 16:42 |
| *** carl_baldwin has quit IRC | 16:43 | |
| anteaya | clarkb: for https://review.openstack.org/#/q/topic:enable_swift+status:open,n,z should your change go in before jhesketh's? | 16:44 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-etherpad_lite: Don't host robots.txt from etherpad repo install https://review.openstack.org/164312 | 16:45 |
| clarkb | anteaya: either order should be fine. my change addresses a small problem with the change that merged late last week | 16:46 |
| *** mwagner_lap has quit IRC | 16:47 | |
| *** tnovacik_ has joined #openstack-infra | 16:47 | |
| *** marun_ has joined #openstack-infra | 16:48 | |
| anteaya | yes I see | 16:49 |
| clarkb | jeblair: thanks, I guess its a decision about what pain is worse. Broken ssl or needing to havea bunch of dev header packages installed for gertty on ubuntu trusty, debian wheezy, and rhel7 | 16:49 |
| jeblair | clarkb: wow. your change is making me very sad :( | 16:49 |
| anteaya | just wondered if I should wait on that before approving jhesketh's | 16:49 |
| anteaya | I approved 156522, yours needs another reviewer | 16:50 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Rename experimental nova py27 to run on dsvm nodes https://review.openstack.org/164777 | 16:50 |
| clarkb | jeblair: ya :/ good news is the line number alignment change is working well | 16:50 |
| *** cdent has quit IRC | 16:52 | |
| *** markus_z has quit IRC | 16:52 | |
| anteaya | zaro: I see nothing in https://review.openstack.org/#/q/status:open+topic:Gerrit-2.9-upgrade,n,z do you have nothing for review? | 16:52 |
| jeblair | anteaya: i think we changed that to gerrit-upgrade | 16:53 |
| anteaya | ah was going by the last week's agenda | 16:53 |
| anteaya | thanks | 16:53 |
| anteaya | yes https://review.openstack.org/#/q/status:open+topic:gerrit-upgrade,n,z has results | 16:53 |
| anteaya | thanks jeblair | 16:53 |
| *** tonytan4ever has quit IRC | 16:54 | |
| *** openstackgerrit has quit IRC | 16:54 | |
| *** openstackgerrit has joined #openstack-infra | 16:54 | |
| *** dhritishikhar has quit IRC | 16:54 | |
| *** nilasae has quit IRC | 16:56 | |
| zaro | clarkb: i think recursive file uploads to swift don't work, are you aware? | 16:57 |
| openstackgerrit | Merged openstack-infra/project-config: Upload python test results to swift https://review.openstack.org/156522 | 16:58 |
| clarkb | zaro: they should work, do you have an example? | 16:58 |
| clarkb | zaro: we had tested them with devstack-gate's specific tempest tests and it appears to do the correct thing for that job | 16:58 |
| zaro | http://logs.openstack.org/68/163068/4/check/python-jenkins-coverage/a1cded6/ | 16:59 |
| openstackgerrit | Merged openstack-infra/project-config: Upload cover/ to swift not coverage/ https://review.openstack.org/164734 | 16:59 |
| jeblair | zaro: ^ i think that change will fix it :) | 16:59 |
| zaro | clarkb: the coverage results didn't get uploaded. | 16:59 |
| clarkb | zaro: see ^ | 16:59 |
| *** mtanino has joined #openstack-infra | 16:59 | |
| *** harlowja_at_home has quit IRC | 17:00 | |
| zaro | ohh thanks. | 17:00 |
| jeblair | zuul has good timing today | 17:00 |
| *** yamahata has quit IRC | 17:00 | |
| anteaya | zaro: I can't verify the tarball in 155463: http://tarballs.openstack.org/ci/gerrit/ where will it come from? | 17:00 |
| openstackgerrit | Merged openstack-infra/release-tools: Add script for creating stable releases of libraries https://review.openstack.org/163880 | 17:00 |
| *** Ryan_Lane has joined #openstack-infra | 17:01 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add FindBugs plugin support https://review.openstack.org/153911 | 17:01 |
| *** dtantsur is now known as dtantsur|afk | 17:02 | |
| *** yfried_ has joined #openstack-infra | 17:02 | |
| *** ivar-lazzaro has joined #openstack-infra | 17:04 | |
| *** tiswanso has quit IRC | 17:04 | |
| *** jlanoux has quit IRC | 17:05 | |
| clarkb | mordred: https://jenkins07.openstack.org/job/gate-shade-pep8/56/console failed due to import issues | 17:05 |
| clarkb | jeblair: ^ is interesting because it hits the same warning as gertty | 17:05 |
| *** marun_ has quit IRC | 17:05 | |
| mordred | clarkb: yup. I actually already pushed a change to the problem patch - it's a depends-on patch though, so it doesn't re-trigger the dependent job | 17:05 |
| mordred | clarkb: I'm just going to wait until the depends-on patch lands and then re-kick that stack | 17:06 |
| pleia2 | so this merged on Thursday https://review.openstack.org/#/c/161078/ but our live http://docs.openstack.org/infra/manual/developers.html hasn't updated, I'm not sure how to figure out what went on here | 17:06 |
| mordred | just in case there is somethign else | 17:06 |
| clarkb | mordred: why are you using depends on there? they are in the same repo | 17:06 |
| *** dboik_ has quit IRC | 17:06 | |
| jeblair | pleia2: it's possible that the post job was lost when i restarted zuul | 17:06 |
| mordred | clarkb: yeah - it was originally for expediency - since I could just add one in the web ui while I was looking at it | 17:06 |
| mordred | clarkb: instead of rebasing everythign together | 17:06 |
| zaro | anteaya: you should -1 that. i was expecting a gerrit build to land into tarballs, but i don't see it. | 17:06 |
| mordred | clarkb: I will not do that again | 17:06 |
| pleia2 | jeblair: aha, could be | 17:06 |
| jeblair | pleia2: but we can find out... gimme a sec | 17:07 |
| *** tiswanso_ has joined #openstack-infra | 17:07 | |
| *** salv-orlando has joined #openstack-infra | 17:07 | |
| anteaya | zaro: I will do so | 17:07 |
| *** dmorita has joined #openstack-infra | 17:08 | |
| anteaya | zaro: thanks | 17:08 |
| *** baoli_ has quit IRC | 17:08 | |
| jeblair | pleia2: post jobs run on the ref that gerrit pushes, which is _either_ the commit itself, or the merge commit if there is one | 17:09 |
| *** marun_ has joined #openstack-infra | 17:09 | |
| *** dboik has joined #openstack-infra | 17:09 | |
| pleia2 | ok | 17:09 |
| jeblair | pleia2: so if we look in git log for that repo, the previous commit it would have run on is this: | 17:09 |
| clarkb | and with tags in the release pipeline its the ref of the tag itself not the ref of the commit the tag is applied to | 17:09 |
| jeblair | pleia2: commit c707aefd4ffd0bf4fbc4c3518b5cab3334dbb082 | 17:09 |
| jeblair | pleia2: Merge "Add reference to IRC services document" | 17:09 |
| clarkb | `git show-ref` is handy for tags | 17:09 |
| jeblair | pleia2: you can construct the log url from that like this: http://logs.openstack.org/c7/c707aefd4ffd0bf4fbc4c3518b5cab3334dbb082/ | 17:10 |
| jeblair | pleia2: (note the c7/ directory, which is the first 2 chars of the commit hash) | 17:10 |
| *** baoli has joined #openstack-infra | 17:11 | |
| pleia2 | gotcha | 17:11 |
| *** gokrokve has quit IRC | 17:11 | |
| jeblair | pleia2: so if you follow that down, that's the log from the last successful post job | 17:11 |
| jeblair | pleia2: http://logs.openstack.org/6b/6b83fcfe04beacc1ed17c48e939309a8a531779e would be the one for the change in question | 17:11 |
| jeblair | pleia2: but does not exist, so that confirms it didn't run | 17:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Fix up and document input parameters https://review.openstack.org/160184 | 17:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add support for keystone projects https://review.openstack.org/159941 | 17:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Small fixes found working on ansible modules https://review.openstack.org/162140 | 17:11 |
| pleia2 | jeblair: ah yes, was just plugging that in | 17:11 |
| jeblair | pleia2: we could check timestamps, but it seems quite reasonable it was lost in my zuul restart | 17:11 |
| *** mpopow has quit IRC | 17:11 | |
| jeblair | pleia2: we could just wait for the next change to merge, or we could trigger the job manually | 17:12 |
| jeblair | pleia2: i reckon we ought to do the latter as a training exercise :) | 17:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Allow for passing cache class in as a parameter https://review.openstack.org/162148 | 17:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Disable dogpile.cache if cache_interval is None https://review.openstack.org/161963 | 17:12 |
| mordred | clarkb: fixed it in the git graph | 17:12 |
| jeblair | i will have to remember how to do that first | 17:12 |
| pleia2 | hehe | 17:12 |
| pleia2 | it is a good training exercise, docs usually don't break all of infra | 17:13 |
| jeblair | pleia2: on zuul.openstack.org: /opt/zuul/tools/trigger-job.py --help | 17:13 |
| pleia2 | I only noticed because fifieldt assigned a bug to -infra related to it, and I was going to close the bug | 17:13 |
| *** tiswanso_ has quit IRC | 17:14 | |
| *** tiswanso has joined #openstack-infra | 17:14 | |
| pleia2 | jeblair: ok, so which of these are required fields? | 17:14 |
| *** patrickeast has joined #openstack-infra | 17:15 | |
| *** marun_ has quit IRC | 17:15 | |
| jeblair | pleia2: i think they all are in this case. i believe the only reason they are listed as optional is that the intent was for the script to support jobs other than post as well. but that part hasn't been written yet (doesn't seem particularly necessary -- 'recheck' is easier) | 17:15 |
| * pleia2 nods | 17:16 | |
| *** ildikov has quit IRC | 17:16 | |
| jeblair | pleia2: this is actually a little easier if the job has actually run (ie, if pypi upload transient failed or something -- you can get all the parameters from jenkins) | 17:16 |
| pleia2 | no doubt | 17:17 |
| jeblair | pleia2: you can still look at an old one and get the gist though: https://jenkins07.openstack.org/job/infra-manual-infra-docs/8/parameters/ | 17:17 |
| pleia2 | ah, great | 17:17 |
| *** dhritishikhar has joined #openstack-infra | 17:17 | |
| jeblair | pleia2: let me know what command line you come up with. i'll try to do the same and we can compare | 17:18 |
| jeblair | pleia2: i just realized this isn't going to work :) | 17:19 |
| *** erlon has joined #openstack-infra | 17:19 | |
| *** bdpayne has joined #openstack-infra | 17:19 | |
| pleia2 | jeblair: doh | 17:20 |
| fungi | pleia2: jeblair: usually the only challenge i run into in that case is figuring out which zuul merger will have the commit needed | 17:20 |
| jeblair | pleia2: i think at this point, this script will _only_ work to retrigger a job that has actually run. it won't work for one that hasn't run because the information won't necessarily be in the zuul merger... | 17:20 |
| jeblair | fungi: indeed :) | 17:20 |
| jeblair | i mean, they might have it, and we could check... | 17:20 |
| jeblair | but since this never ran, perhaps we should take a different tack and see if we can enqueue it | 17:20 |
| fungi | which _can_ be done by prodding all the zuul mergers to see which one is serving it | 17:20 |
| pleia2 | I see | 17:21 |
| jeblair | fungi: right, but since it didn't run, it's not actually guaranteed to be on any of them. it _probably_ is, but still. | 17:21 |
| *** dizquierdo has quit IRC | 17:21 | |
| fungi | yep, agreed. sometimes you get lucky, sometimes you get eaten by a grue | 17:21 |
| jeblair | we can't enqueue into post, can we? | 17:21 |
| jeblair | 'zuul enqueue' seems to require a change id, and not a ref. | 17:22 |
| *** jp_at_hp has quit IRC | 17:22 | |
| fungi | i hadn't tried, but yeah might need some extra support for non-change pipelines | 17:22 |
| *** mpopow has joined #openstack-infra | 17:23 | |
| jeblair | pleia2: okay, so back to the other way :) | 17:24 |
| jeblair | pleia2: git fetch http://zm01.openstack.org/p/openstack-infra/infra-manual 6b83fcfe04beacc1ed17c48e939309a8a531779e | 17:24 |
| jeblair | pleia2: that seems to work, so i guess we can use zm01 as the zuul url | 17:24 |
| pleia2 | ok | 17:24 |
| ctlaugh | asselin, (or anyone else) -- I am seeing this from zuul when an event enters the pipeline: http://paste.ubuntu.com/10610548/ I'm not sure what the problem is. | 17:25 |
| jeblair | mtreinish: sqlalchemy.exc.OperationalError: (OperationalError) (1878, 'Temporary file write failure.') 'ALTER TABLE test_runs ADD COLUMN start_time_microsecond INTEGER' () | 17:25 |
| jeblair | real 20m58.268s | 17:25 |
| fungi | wow, faster than i expected | 17:26 |
| *** _nadya_ has quit IRC | 17:26 | |
| *** tonytan4ever has joined #openstack-infra | 17:26 | |
| *** BharatK has quit IRC | 17:26 | |
| jeblair | fungi: i think we were hitting the 30m timeout because there was an 18m mysqldump included in that | 17:26 |
| *** BharatK has joined #openstack-infra | 17:26 | |
| fungi | makes sense | 17:27 |
| mtreinish | jeblair: um, that's not good | 17:27 |
| jeblair | (in the timeout, not the 20m runtime for the (failed) migration i just posted) | 17:27 |
| jeblair | mtreinish, fungi: do we have 2x empty space on that trove instance? | 17:27 |
| fungi | i think we have 0x empty space | 17:27 |
| * nibalizer reads scrollback | 17:27 | |
| * fungi looks | 17:27 | |
| nibalizer | fungi: mordred awesome that you found/fixed a scoping issue | 17:27 |
| *** harlowja has joined #openstack-infra | 17:28 | |
| jeblair | if you can't figure out what that sentence means, i think i meant to say "are we at less than 50% capacity on the trove instance" | 17:28 |
| mordred | nibalizer: heck yes | 17:28 |
| jeblair | mtreinish, fungi, pleia2: because i think mysql will need to write a temp table and so will briefly occupy double the normal space... | 17:28 |
| fungi | nibalizer: one which i expect is endemic throughout our modules | 17:28 |
| jeblair | mordred: ^ correct? | 17:28 |
| nibalizer | fungi: yep | 17:28 |
| mtreinish | jeblair: yeah that's what google just told me too :) | 17:28 |
| nibalizer | and this is why we dont use scope.lookupvar | 17:28 |
| mtreinish | jeblair: well last time we looked at storage we were using 90GB and you bumped it to 300GB | 17:29 |
| mtreinish | but that was a few weeks ago | 17:29 |
| nibalizer | also we've rejected rocketship alignment ... i am greatly amused | 17:29 |
| fungi | jeblair: mtreinish: mordred: actually we're very close to 2x empty space after all | 17:29 |
| fungi | jeblair: mtreinish: mordred: "105 GB of 300 GB" says rackspace | 17:29 |
| jeblair | nibalizer: well, we're discussing it; definitely want your opinion :) | 17:29 |
| *** jp_at_hp has joined #openstack-infra | 17:29 | |
| nibalizer | well this merged so .. https://review.openstack.org/#/c/164725/ | 17:29 |
| anteaya | ctlaugh: is not registered with Gearman | 17:29 |
| jeblair | nibalizer: i just woke up, read a diff while groggy, and was like "which of these lines _actually_ changed?" | 17:30 |
| anteaya | ctlaugh: is your gearman running? | 17:30 |
| nibalizer | my knee jerk reaction is 'nope puppet best practice and we're on the best practice train' but maybe thats not right | 17:30 |
| nibalizer | jeblair: so in my gerrit view it darkgreens stuff that changed and lightgreens stuff that shifted | 17:31 |
| fungi | nibalizer: fight the power | 17:31 |
| mtreinish | fungi: hmm, I would have thought that would be enough space then, maybe it's not using the same location for the temp file? | 17:31 |
| nibalizer | fungi: yea infra don't need anyones arbitrary rules! we're our own project! W wann we be free! | 17:31 |
| fungi | mtreinish: no idea since trove doesn't expose that to us | 17:31 |
| asselin | ctlaugh, assuming the job is correctly spelled, try adding or removing a jenkins slave to retrigger a gearman update | 17:32 |
| fungi | nibalizer: anarchy in the ci | 17:32 |
| ctlaugh | anteaya: perhaps not -- I don't see anything that resembles it running | 17:32 |
| * fungi is going to have a sex pistols song stuck in his head the rest of the day | 17:32 | |
| asselin | ctlaugh, telnet localhost 4730 | 17:32 |
| mtreinish | fungi: sigh, this trove thing just keeps getting better and better... | 17:33 |
| fungi | mtreinish: yeah, it's one of those | 17:33 |
| fungi | "openstack" services | 17:33 |
| jeblair | mordred: any other ideas about the "Temporary file write failure."? | 17:33 |
| jeblair | SpamapS: calling all mysql experts... | 17:33 |
| * fungi looks around for openstack people | 17:33 | |
| mordred | jeblair: this is in resizing our trove instance, right | 17:34 |
| pleia2 | jeblair: here's what I came up with http://paste.openstack.org/show/192681/ | 17:34 |
| *** HeOS has quit IRC | 17:34 | |
| fungi | mordred: doing an alter table | 17:34 |
| mtreinish | mordred: no, doing an alter table on the subunit2sql trove instance | 17:34 |
| jeblair | mordred: we're doing an alter table on a trove instance. it has 105G used out of 300G total. | 17:34 |
| asselin | ctlaugh, if that works, then type 'status' to see all the jobs zuul will see | 17:34 |
| fungi | mordred: not resizing the instance (we can't resize the instance as 300gb is the max rax supports) | 17:34 |
| *** mpopow has quit IRC | 17:34 | |
| *** BharatK has quit IRC | 17:35 | |
| *** dangers_away is now known as dangers | 17:35 | |
| mordred | it may be temp dir settings | 17:35 |
| *** ssam2 has quit IRC | 17:35 | |
| mordred | on the server | 17:35 |
| *** dprince has quit IRC | 17:36 | |
| *** amitgandhinz has quit IRC | 17:36 | |
| mordred | the 2x request is that mysql alters, depending on what the specific alter is, involve a second copy of the table | 17:36 |
| ctlaugh | asselin, anteaya - I deleted the instance in nova, and deleted the slave in Jenkins. Nodepool is not creating another instance -- debug still shows one node ready. | 17:36 |
| anteaya | zaro: how would you feel about rebasing: https://review.openstack.org/#/c/155448/ | 17:36 |
| jeblair | mordred: can we 'show' the relevant values? | 17:36 |
| mordred | yah - but we may not have much option to change them since it's a trove and not on a server we control | 17:36 |
| asselin | ctlaugh, you should deleve via nodepool instead | 17:37 |
| ctlaugh | asselin: ah | 17:37 |
| mordred | jeblair: what version of mysql server is running there? | 17:37 |
| *** mpopow has joined #openstack-infra | 17:37 | |
| asselin | ctlaugh, nodepool list, nodepool delete <id> | 17:37 |
| *** mpopow has quit IRC | 17:37 | |
| *** marun has joined #openstack-infra | 17:37 | |
| fungi | mordred: 5.6 | 17:38 |
| jeblair | fungi: ^ can you answer? i'm working on pleia2's question | 17:38 |
| jeblair | thx | 17:38 |
| jeblair | :) | 17:38 |
| *** mpopow has joined #openstack-infra | 17:38 | |
| mtreinish | mordred: Server version: 5.6.17 MySQL Community Server (GPL) | 17:38 |
| mtreinish | oh, fungi beat me to it | 17:38 |
| *** Sukhdev has joined #openstack-infra | 17:38 | |
| fungi | mordred: jeblair: looking through the custom trove config parameter list i don't see anything obviously related to temporary file creation, but maybe the name is opaque/cryptic? | 17:39 |
| *** yamahata has joined #openstack-infra | 17:39 | |
| *** gokrokve has joined #openstack-infra | 17:39 | |
| clarkb | mordred: see comment on 160184 | 17:39 |
| pleia2 | jeblair: aha, url should just be http://zm01.openstack.org/p | 17:39 |
| jeblair | pleia2: yep | 17:39 |
| mordred | well, actually it should be using the data dir for the second copy | 17:39 |
| mordred | so it shouldn't matter on that ... | 17:39 |
| *** mpopow has quit IRC | 17:39 | |
| fungi | pleia2: you can grep my .bash_history on zuul for lots of relevant example invocations too | 17:40 |
| mordred | you said we were down to 105G - were we larger than that at one point? | 17:40 |
| pleia2 | fungi: ah, lovely | 17:40 |
| ctlaugh | asselin: ok, I deleted -- now nodepool debug log is showing start:0, min:0, ready:0, capacity:2 | 17:40 |
| jeblair | pleia2: and job should be "infra-manual-infra-docs" -- zuul doesn't do any project name interpolation there, so it's the full job name | 17:40 |
| *** marun_ has joined #openstack-infra | 17:40 | |
| mtreinish | mordred: shouldn't have ever been bigger than that, we've never deleted anything from it | 17:40 |
| pleia2 | jeblair: ah, good to know | 17:40 |
| mordred | k | 17:40 |
| SpamapS | is innodb_file_per_table set ? | 17:40 |
| jeblair | pleia2: oldrev i think you can get with this: git show 6b83fcfe04beacc1ed17c48e939309a8a531779e^1 | 17:40 |
| jeblair | pleia2: (6b83 is the new rev -- the ^1 will get the first parent, which i believe is what would be provided normally. i don't believe we actually use this though, so i suppose it doesn't matter that much) | 17:41 |
| jeblair | pleia2: so at any rate, it would be: --oldrev c707aefd4ffd0bf4fbc4c3518b5cab3334dbb082 | 17:41 |
| SpamapS | mordred: http://dev.mysql.com/doc/refman/5.6/en/alter-table-online-operations.html <-- things are somewhat more complex with 5.6 | 17:41 |
| fungi | jeblair: mtreinish: mordred: i want to say we started with 100gb disk, filled it up, resized to the max (300gb) and have seen mild increase since | 17:42 |
| asselin | ctlaugh, not sure...try restarting nodepool....? | 17:42 |
| jeblair | pleia2: and then the logpath needs the extra stuff stuck onto the end | 17:42 |
| jeblair | pleia2: --logpath 6b/6b83fcfe04beacc1ed17c48e939309a8a531779e/post/infra-manual-infra-docs/manually-triggered/ | 17:42 |
| *** nelsnelson has quit IRC | 17:42 | |
| SpamapS | oh wait thats cluster | 17:42 |
| *** marun has quit IRC | 17:42 | |
| *** angela-s has joined #openstack-infra | 17:42 | |
| SpamapS | http://dev.mysql.com/doc/refman/5.6/en/innodb-online-ddl.html | 17:42 |
| SpamapS | that one | 17:42 |
| mtreinish | fungi: yeah that's what we did, jeblair resized it when we went >90gb | 17:42 |
| jeblair | pleia2: that last component is just a directory name i made up. normally, zuul will create a uuid for that part, but since zuul isn't running it, we need to make one up ourselves | 17:43 |
| * pleia2 nods | 17:43 | |
| *** trey has quit IRC | 17:43 | |
| *** mpopow has joined #openstack-infra | 17:43 | |
| fungi | SpamapS: oh! so we can control that per alter call>? | 17:43 |
| *** jp_at_hp has quit IRC | 17:43 | |
| jeblair | pleia2: so i think it should be http://paste.openstack.org/show/192683/ | 17:43 |
| nibalizer | 17:43 | |
| mordred | mtreinish: this is 1679b5bc102c_add_subsecond_columns_to_test_runs_table.py that you're doing, yeah? | 17:44 |
| mtreinish | mordred: yes | 17:44 |
| jeblair | pleia2: and just so you know, i got two things wrong on my attempt and corrected them based on what you wrote. :) | 17:44 |
| nibalizer | jf | 17:44 |
| nibalizer | 17:44 | |
| nibalizer | `\871 | 17:44 |
| SpamapS | fungi: yeah, but what was the alter? | 17:44 |
| SpamapS | fungi: http://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-overview.html#innodb-online-ddl-summary-grid | 17:44 |
| *** afazekas has quit IRC | 17:44 | |
| mordred | SpamapS: it's in an alembic migration | 17:44 |
| *** mpopow has quit IRC | 17:44 | |
| pleia2 | jeblair: great, that's the same as what I've cobbled together | 17:44 |
| mtreinish | SpamapS: http://git.openstack.org/cgit/openstack-infra/subunit2sql/tree/subunit2sql/migrations/versions/1679b5bc102c_add_subsecond_columns_to_test_runs_table.py | 17:45 |
| *** trey has joined #openstack-infra | 17:45 | |
| pleia2 | jeblair: thanks, that makes me feel better :) | 17:45 |
| SpamapS | adding columns can be done online | 17:45 |
| *** Ala has quit IRC | 17:46 | |
| pleia2 | jeblair: ok, I'm going to give this a try | 17:46 |
| jeblair | pleia2: cool. also, i think the only way we're going to see output from it is by checking jenkins | 17:46 |
| pleia2 | ok, I'll keep an eye on it | 17:46 |
| jeblair | pleia2: so open up 7 tabs with "https://jenkins0n.openstack.org/job/infra-manual-infra-docs/" and see which one it appears in :) | 17:47 |
| anteaya | ha ha ha | 17:47 |
| pleia2 | right :) | 17:47 |
| pleia2 | many jenkinses is fun | 17:47 |
| fungi | pleia2: alternatively, just open the url where you expect the logs to land and keep refreshing periodically | 17:47 |
| *** mpopow has joined #openstack-infra | 17:47 | |
| jeblair | that works too | 17:47 |
| jeblair | until we _only_ do swift logs | 17:47 |
| pleia2 | fungi: sounds like you've done this before | 17:47 |
| fungi | and if the trigger-job.py call returns control to the terminal then the log should normally appear within seconds | 17:48 |
| jeblair | the job won't be running with a valid upload token, so shouldn't be able to upload to swift | 17:48 |
| pleia2 | huzzah http://logs.openstack.org/6b/6b83fcfe04beacc1ed17c48e939309a8a531779e/ | 17:48 |
| fungi | yeah, that will require some additional thinking | 17:48 |
| jeblair | i think the next step in making this less terrible is to do zuul enqueue for post jobs | 17:48 |
| pleia2 | http://docs.openstack.org/infra/manual/developers.html#accessing-gerrit-over-https \o/ | 17:48 |
| pleia2 | thanks jeblair | 17:49 |
| jeblair | pleia2: yay! | 17:49 |
| fungi | agreed. trigger-job.py predates zuul having a cli at all. getting the remaining use cases for it implemented in zuul enqueue means we can stop keeping it around i guess | 17:49 |
| SpamapS | so it looks like adding a column will use INPLACE by default. | 17:49 |
| anteaya | pleia2: nice work | 17:50 |
| fungi | or just keep it around as an example tool for testing injection into gearman anyway | 17:50 |
| *** ghostpl_ has quit IRC | 17:50 | |
| *** achanda has joined #openstack-infra | 17:50 | |
| mtreinish | SpamapS: as a mysql n00b is that a good or bad thing? | 17:50 |
| *** dprince has joined #openstack-infra | 17:50 | |
| jeblair | pleia2: oh, yeah, so you know why this is so weird -- the trigger job script just sends a job over gearman directly -- zuul itself is not actually involved. | 17:50 |
| jeblair | pleia2: 'zuul enqueue' tells zuul to enqueue something and launch jobs | 17:51 |
| *** jistr has quit IRC | 17:51 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 17:51 | |
| fungi | mtreinish: SpamapS: i think it means that the tempfile creation error is probably not space exhaustion because of a table copy? | 17:51 |
| SpamapS | mtreinish: it's slower but uses less space | 17:51 |
| pleia2 | jeblair: I see | 17:51 |
| SpamapS | welllll | 17:51 |
| SpamapS | If the table is still in the old column format, it still has to use the copy method. | 17:51 |
| SpamapS | show table status like `tablename`; will answer that | 17:52 |
| SpamapS | actually it might be 'tablename' | 17:52 |
| *** BharatK has joined #openstack-infra | 17:53 | |
| mtreinish | SpamapS: http://paste.openstack.org/show/192687/ | 17:53 |
| anteaya | jeblair: thanks, I keep notes on tutorials in channel, I called this one retrigger a failed post job | 17:54 |
| reed | fungi, re ask.openstack.org, changing hosts file, should I get a CA warning for the different IP address? | 17:54 |
| fungi | reed: you shouldn't, no | 17:54 |
| fungi | reed: do you? | 17:54 |
| SpamapS | mtreinish: ok, version 10.. let me see.. | 17:54 |
| reed | fungi, then the new server works :) | 17:54 |
| fungi | heh | 17:54 |
| jogo | greghaynes: o/ | 17:55 |
| *** achanda_ has joined #openstack-infra | 17:55 | |
| greghaynes | jogo: ohai | 17:55 |
| greghaynes | jogo: I have to do morning routine but I plan on poking at your things today | 17:56 |
| zaro | fungi, jeblair, clarkb : i was thinking about removing jobs that build gerrit 2.4.x would you have any concerns with that? | 17:56 |
| ctlaugh | asselin: to clean things up, I deleted all the images and instances in nodepool and restarted it, and it's building a new image. anteaya mentioned gearman running -- I don't see anything that looks like gearman running -- what would have started it? | 17:57 |
| jeblair | zaro: oh please do! :) | 17:57 |
| jogo | greghaynes: neato! | 17:57 |
| fungi | zaro: no objection from me. we're very unlikely to switch back to 2.4 now | 17:57 |
| asselin | ctlaugh, jenkins | 17:57 |
| *** salv-orl_ has joined #openstack-infra | 17:57 | |
| SpamapS | mtreinish: oh hm, the column formats might be separate from that. | 17:57 |
| asselin | ctlaugh, it's a jenkins plugin | 17:57 |
| *** adalbas has joined #openstack-infra | 17:58 | |
| *** devvesa has quit IRC | 17:58 | |
| SpamapS | mtreinish: relevant info btw: http://mechanics.flite.com/blog/2014/05/01/upgrading-temporal-columns-from-mysql-5-dot-5-to-mysql-5-dot-6-format/ | 17:58 |
| asselin | ctlaugh, you can check to see if it's running using the command "telnet localhost 4730" then "status" | 17:58 |
| *** achanda has quit IRC | 17:58 | |
| SpamapS | mtreinish: It actually looks like 5.6 added the exact thing that the migration is trying to workaround not having. ;) | 17:58 |
| *** marun_ has quit IRC | 17:59 | |
| clarkb | mordred: re 163517 I am reading http://docs.ansible.com/intro_configuration.html#hash-behaviour can you explain why we would need to use the discouraged behavior? | 17:59 |
| *** jp_at_hp has joined #openstack-infra | 17:59 | |
| SpamapS | mtreinish: there's a query there for finding out if your columns are old format. If they are, then the alter will create a copy and you need 2x the space available. | 18:00 |
| mtreinish | SpamapS: yeah that was the thing when I noticed microseconds were missing, the first thing I did was check the version | 18:00 |
| ctlaugh | asselin: I just looked in the jenkins config and it wasn't enabled. Sorry -- seems like that's the problem. | 18:00 |
| *** yamahata has quit IRC | 18:00 | |
| mkj_wk | fungi: would like your preference for making downloaded branches track upstream... Could do it either only if options.track is set, or do it always. Would be simpler to do it always, and I don't see a downside. Do you? | 18:00 |
| SpamapS | mtreinish: if they're new format, then it will alter the table in place. | 18:00 |
| asselin | ctlaugh, np...glad you got it resolved | 18:00 |
| *** yamahata has joined #openstack-infra | 18:00 | |
| *** baoli has quit IRC | 18:00 | |
| mtreinish | SpamapS: when it showed 5.6 then I asked mordred who said do this. Although to be fair I was trying to be sort of backend agnostic in the code | 18:01 |
| mordred | clarkb: it's not discouraged, it's discouraged in general - in our case, I believe it will be much more complex to keep input variables at both the host and the group-of-hosts level without it | 18:01 |
| *** mwagner_lap has joined #openstack-infra | 18:01 | |
| fungi | mkj_wk: if we successfully solve for all corner cases so that the new behavior introduced doesn't break common workflows, i'd be fine with having it on by default | 18:01 |
| *** baoli_ has joined #openstack-infra | 18:01 | |
| mordred | clarkb: specifically, in order to keep a mapping of which hiera values are needed on which hosts, when some of them are needed by groups | 18:01 |
| *** salv-orlando has quit IRC | 18:01 | |
| mordred | clarkb: without getting very squirelly | 18:01 |
| clarkb | mordred: reading http://docs.ansible.com/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable precedence should work just fine right? | 18:01 |
| mordred | no | 18:02 |
| mordred | because we are going to have complex data | 18:02 |
| mkj_wk | fungi: OK, I'll start with it on by default, and if later we come up with a reason it's bad I'll change | 18:02 |
| mkj_wk | thanks! | 18:02 |
| clarkb | mordred: ok, I don't understand where it will break then | 18:02 |
| mordred | clarkb: yeah - I need to put up another patch to show that more clearly | 18:02 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack-infra/project-config: Add "heat-coe-templates" project https://review.openstack.org/164806 | 18:02 |
| fungi | SpamapS: and when you say 2x space available you mean that the unused space needs to be 2x table size, or just 1x (room to make two copies or just make one copy)? | 18:03 |
| clarkb | mordred: also your merge method doesn't work on scalar values which is going to be even more confusing | 18:03 |
| cinerama | pleia2: i added some graffiti to your etherpad re zanata | 18:03 |
| mordred | don't need it to - it's specifically to allow us to define a hiera dict for groups and for hosts | 18:03 |
| clarkb | mordred: so if I have hostvars: {'string': value, {'somekey': 'value'}} only he inner dict will be merged} | 18:03 |
| pleia2 | cinerama: excellent, thanks :) | 18:03 |
| *** peristeri has quit IRC | 18:04 | |
| *** amitgandhinz has joined #openstack-infra | 18:04 | |
| mordred | clarkb: can we come back to this one - I am juggling too many other things at the moment to properly respond to your question | 18:05 |
| cinerama | pleia2: i'm currently a bit stuck on the openid bits | 18:05 |
| clarkb | mordred: sure | 18:05 |
| mordred | clarkb: thanks. it's a good question - and clearly somethign we need to make sure we understand and if so, document why | 18:05 |
| asselin | curious to know if the governance change needs to merge before this? https://review.openstack.org/#/c/163255/ and how to poke that process along.... | 18:05 |
| *** peristeri has joined #openstack-infra | 18:05 | |
| clarkb | fungi: 164312 needed a puppetlint fix | 18:05 |
| clarkb | mordred: ^ you had previously reviewed that change too | 18:06 |
| anteaya | asselin: I was just looking at that | 18:06 |
| clarkb | its +1 by jenkins now so rereview would be asesome | 18:06 |
| anteaya | asselin: the governance change needs to merge first | 18:06 |
| anteaya | asselin: it is on tomorrow's tc agenda so my expectation it should go in during that meeting | 18:06 |
| *** marun_ has joined #openstack-infra | 18:06 | |
| asselin | anteaya, great, thanks for the info | 18:07 |
| anteaya | I checked to make sure it was on the agenda for tomorrow | 18:07 |
| anteaya | sure | 18:07 |
| fungi | clarkb: thanks! | 18:07 |
| mordred | clarkb: +2 | 18:07 |
| *** nelsnelson has joined #openstack-infra | 18:07 | |
| asselin | anteaya, I don't see it on the agenda | 18:08 |
| fungi | clarkb: also there's a new etherpad release as of yesterday covering yet another security vulnerability which we don't need to be concerned about | 18:08 |
| asselin | anteaya, nevermind...it's at the very very end | 18:08 |
| fungi | clarkb: but it HAS A CVE! | 18:08 |
| *** marun has joined #openstack-infra | 18:09 | |
| anteaya | asselin: thanks for looking | 18:09 |
| fungi | clarkb: small victories | 18:09 |
| clarkb | fungi: yay | 18:09 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack-infra/project-config: Add "heat-coe-templates" project https://review.openstack.org/164806 | 18:09 |
| *** BharatK has quit IRC | 18:09 | |
| anteaya | asselin: you probably don't need to be there, but it doesn't hurt if you can make it | 18:09 |
| asselin | anteaya, ok ....will try to pay attention | 18:09 |
| anteaya | asselin: cool | 18:10 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Small fixes found working on ansible modules https://review.openstack.org/162140 | 18:10 |
| fungi | clarkb: i've got a list put together, mostly complete now, to request cve assignments for historical issues in etherpad-lite back to the 1.0 release in 2011. probably get that knocked out next week since the primary dev is on vacation this week | 18:10 |
| clarkb | fungi: fun :) | 18:10 |
| mordred | clarkb: I fixed your 160184 comment in 162140 ^^ just so that we don't have to retest the entire stack there | 18:11 |
| *** mpavone has joined #openstack-infra | 18:11 | |
| *** marun_ has quit IRC | 18:11 | |
| clarkb | mordred: ok I can update vote on 160184 | 18:11 |
| mordred | clarkb: thanks! | 18:12 |
| *** derekh has quit IRC | 18:12 | |
| *** marun_ has joined #openstack-infra | 18:13 | |
| *** marun has quit IRC | 18:13 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Expand scope for neutron dbdeadlock bug 1432065 https://review.openstack.org/164703 | 18:14 |
| uvirtbot | Launchpad bug 1432065 in neutron "DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'DELETE FROM ipallocationpools WHERE ipallocationpools.id" [Critical,Confirmed] https://launchpad.net/bugs/1432065 | 18:14 |
| openstack | bug 1432065 in neutron "DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'DELETE FROM ipallocationpools WHERE ipallocationpools.id" [Critical,Confirmed] https://launchpad.net/bugs/1432065 | 18:14 |
| uvirtbot | Launchpad bug 1432065 in neutron "DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'DELETE FROM ipallocationpools WHERE ipallocationpools.id" [Critical,Confirmed] | 18:14 |
| fungi | btw ^ ...anybody have any idea what channels soren might still be found in? | 18:14 |
| clarkb | we could just ban uvirtbot >_> | 18:14 |
| clarkb | bodepd: ^ any idea how we can bug soren? | 18:15 |
| zaro | clarkb: can you please take a look? https://jenkins07.openstack.org/job/gerrit-package-buck/6/console | 18:15 |
| fungi | clarkb: pain in the butt since we ideally would have him take all channels inhabited by the openstack meetbot out of his config | 18:15 |
| zaro | clarkb: cp: cannot stat ‘gerrit/buck-out/gen/release.war’: No such file or directory | 18:15 |
| zaro | clarkb: just tried it locally and the file is there for me. | 18:16 |
| openstackgerrit | Merged openstack-infra/puppet-etherpad_lite: Don't host robots.txt from etherpad repo install https://review.openstack.org/164312 | 18:16 |
| bodepd | clarkb: wait till he's awake :) | 18:16 |
| *** fdegir has joined #openstack-infra | 18:16 | |
| nibalizer | clarkb: fungi can also +q the bot to quiet without kicking | 18:17 |
| clarkb | zaro: looks like that job cd's into gerrit/ so the relative path is probably wrong | 18:17 |
| bodepd | I checked a few channel he's usually in, and I didn't see him, I can tell your him your looking for him. | 18:18 |
| *** fdegir has left #openstack-infra | 18:18 | |
| clarkb | bodepd: thank you | 18:18 |
| mtreinish | fungi, SpamapS, mordred, jeblair: so I'm wondering what our next step should be for the migration. Maybe we should just switch to a new db that we admin ourselves and switch the worker to use that instead? | 18:18 |
| *** banix has quit IRC | 18:18 | |
| zaro | clarkb: ohh wow, thanks. | 18:19 |
| jogo | having trouble reading the output of the gate-project-config-layout job | 18:19 |
| jogo | what is the error with http://logs.openstack.org/14/164314/1/check/gate-project-config-layout/7d4c1d7/console.html | 18:20 |
| *** paul-- has quit IRC | 18:20 | |
| *** marun_ has quit IRC | 18:20 | |
| jogo | aha 2015-03-13 21:21:19.381 | Job check-dsvm-keystone not defined | 18:20 |
| clarkb | jogo: http://logs.openstack.org/14/164314/1/check/gate-project-config-layout/7d4c1d7/console.html#_2015-03-13_21_21_19_381 yes that | 18:20 |
| *** marun_ has joined #openstack-infra | 18:21 | |
| EmilienM | yolanda: re- https://review.openstack.org/#/c/164812/ | 18:21 |
| *** tmcpeak has joined #openstack-infra | 18:21 | |
| EmilienM | yolanda: are you sure it's a puppet openstack project ? | 18:21 |
| mordred | mtreinish, jeblair: I'm beginning more and more to start to think about that seriously across the board | 18:21 |
| *** packet has joined #openstack-infra | 18:21 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/project-config: Add new keystone tempest job to only run keystone and scenario tests https://review.openstack.org/164314 | 18:21 |
| clarkb | mtreinish: this was always the big problem with this tool right? | 18:21 |
| clarkb | mtreinish: we need a way to expire old data | 18:22 |
| *** melwitt has joined #openstack-infra | 18:22 | |
| clarkb | I don't think make bigger node fixes that | 18:22 |
| clarkb | it just puts the problem off until we run into it again | 18:22 |
| bdpayne | We'd like to add a new op on the openstack-security channel. I'm currently the only op. What is the process for doing this so that it sticks across logins? | 18:22 |
| mtreinish | clarkb: well the goal was to always try and keep a cycles worth in the db, we're not even there yet | 18:22 |
| *** BharatK has joined #openstack-infra | 18:23 | |
| mordred | sdague: I hate to bother you - I know you're busy ... | 18:23 |
| mtreinish | clarkb: fwiw, adding the python api call to delete a date range is my next todo | 18:24 |
| jeblair | mtreinish: did you find a root cause for the current error? | 18:24 |
| mordred | sdague: but I have a patch in shade: https://review.openstack.org/#/c/164691/ which has to deal with server.image sometimes being a string (when boot from volume) and most of the time being a dict-like object | 18:24 |
| mordred | sdague: is that a bug in python-novaclient? or in nova? or is it a bug at all? | 18:24 |
| jkt | hi there, I've created https://storyboard.openstack.org/#!/story/2000205 to describe a bug in gertty, but I don't see that bugreport when I search for gertty's stories | 18:24 |
| jkt | is that a PEBKAC on my side? | 18:24 |
| fungi | bdpayne: /msg chanserv access #openstack-security whoever_you_want +Aotv | 18:25 |
| jeblair | bdpayne: you should be able to add the necessary acl to chanserv | 18:25 |
| jeblair | heh, like that | 18:25 |
| mtreinish | clarkb: but my suggestion to move to a infra admin'ed db was more to let us have more control. Especially over all the tunables that mordred and SpamapS were bringing up | 18:25 |
| *** peristeri has quit IRC | 18:25 | |
| mordred | clarkb: also - re 164691 - you said something about not voting because of tests - I rebased on top of the functional test fix, fwiw | 18:25 |
| fungi | bdpayne: that matches your access levels listed based on /msg chanserv access #openstack-security list | 18:25 |
| bdpayne | ok thanks guys, I'll give it a try | 18:26 |
| fungi | bdpayne: chanserv has a fair amount of context help too, if you get curious | 18:26 |
| mtreinish | jeblair: I don't think so, but I don't really have enough sql-fu to debug further | 18:26 |
| bdpayne | good to know, thanks | 18:26 |
| mordred | clarkb: I don't think it's about a _bigger_ node for subunit2sql - it's that we've got an opaque db that we're having trouble administering - this should not be as complex as it is | 18:26 |
| mordred | jeblair: the error message was added in 5.6.15 with no additional information about what problems it's actually related to | 18:27 |
| openstackgerrit | Khai Do proposed openstack-infra/project-config: Fix upload built artifact in gerrit job https://review.openstack.org/164815 | 18:27 |
| mordred | jeblair: I'm considering diving in to the source code | 18:27 |
| *** peristeri has joined #openstack-infra | 18:27 | |
| fungi | jkt: the url to the story you added seems to actually be https://storyboard.openstack.org/#!/story/2000205 | 18:28 |
| bdpayne | fungi I'm getting "You are not authorized to execute this command." | 18:28 |
| clarkb | you can tune everything with trove aiui | 18:28 |
| clarkb | but sure if control is what you want thats fine | 18:28 |
| jeblair | mtreinish, mordred: ok. i'm open to the idea of switching back. i don't want to do it _too_ lightly though. we have found that we can tweak some server parameters using the trove api (which may become easier when we can manage it with ansible+shade) | 18:28 |
| sdague | mordred: does it vary by cloud? | 18:28 |
| jkt | fungi: yes; what I linked to was a copy-paste from firefox 35's location bar | 18:29 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 18:29 | |
| *** paul-- has joined #openstack-infra | 18:29 | |
| sdague | there is a lot of magically translation layer here in oslo incubator apiclient, so I'm not sure | 18:29 |
| mordred | jeblair: https://github.com/mysql/mysql-server/blob/7e58ce8e69de5a4e941d913999cbbf905fe8f444/storage/innobase/row/row0ftsort.cc#L840-L853 | 18:29 |
| jeblair | mtreinish, mordred: we've hit enough small problems that maybe they're adding up enough to make it worthwhile. otoh, we might switch subunit2sql and then be staring at the same problem. possibly with more debugging tools. :) | 18:29 |
| mordred | this appears to be related to sort buffer | 18:29 |
| mordred | SpamapS: ^^ | 18:29 |
| mordred | I'm not really sure whats causing sort buffer to be used in this | 18:29 |
| fungi | jkt: you probably need to check the page size at https://storyboard.openstack.org/#!/profile/preferences | 18:30 |
| mordred | mtreinish: can you easily paste me "show create table test_runs \G" ? | 18:30 |
| fungi | jkt: pagination in storyboard is still a work in progress | 18:30 |
| openstackgerrit | Jan Kundrát proposed stackforge/gertty: Add mouse wheel scrolling https://review.openstack.org/133550 | 18:31 |
| mordred | sdague: awesome | 18:31 |
| jkt | fungi: OK, noted | 18:31 |
| mtreinish | mordred: monkey see monkey do: http://paste.openstack.org/show/192699/ | 18:31 |
| fungi | bdpayne: double-checking. it might be that you ended up with fewer permissions than usual. i'll fix in a sec if so | 18:31 |
| sdague | based on that, if I had to guess, it might be a nova inconsistency | 18:31 |
| mordred | sdague: I'm not sure - I've never hit it myself ... it was reported by someone who was using Cisco's OpenStack product | 18:31 |
| *** melwitt_ has joined #openstack-infra | 18:31 | |
| bdpayne | fungi kk, thanks | 18:31 |
| nibalizer | SergeyLukjanov: jeblair others what do you think about reverting and re proposing https://review.openstack.org/#/c/164725/ ? | 18:31 |
| mordred | sdague: but they didn't see it on every thing | 18:31 |
| mordred | sdague: just on servers booted from volume | 18:31 |
| nibalizer | that was supposed to be a forum for conversation and now its just merged | 18:32 |
| jeblair | nibalizer: will do, thanks. | 18:32 |
| sdague | is it reproducable in upstream? | 18:32 |
| crinkle | awesome | 18:32 |
| *** otter768 has joined #openstack-infra | 18:33 | |
| *** Guest64707 has quit IRC | 18:33 | |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-zuul: Revert "Disable arrow alignment puppet lint check" https://review.openstack.org/164818 | 18:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-zuul: Disable arrow alignment puppet lint check https://review.openstack.org/164819 | 18:34 |
| nibalizer | jeblair: great thanks | 18:34 |
| *** tsg has quit IRC | 18:35 | |
| mordred | sdague: unsure - I can try | 18:35 |
| *** ashishb has joined #openstack-infra | 18:35 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Update puppet-setproxy to not be part of puppet-openstack https://review.openstack.org/164820 | 18:35 |
| *** melwitt has quit IRC | 18:35 | |
| mordred | sdague: if it is - you would consider that a bug _somewhere_ right? | 18:35 |
| *** melwitt_ is now known as melwitt | 18:35 | |
| *** dprince has quit IRC | 18:35 | |
| sdague | mordred: yeh | 18:36 |
| mordred | sdague: k | 18:36 |
| jeblair | nibalizer, mordred, sdague, SergeyLukjanov: https://review.openstack.org/#/c/164819/ re-proposed and made WIP so that we don't accidentally merge it. | 18:36 |
| mordred | jeblair: I have asked stewart some questions about internals | 18:36 |
| jeblair | mordred: i think that should be a trove api call | 18:37 |
| sdague | jeblair: oh, it merged too quick? | 18:37 |
| jeblair | sdague: yes, before many of our puppet experts were awake :) | 18:37 |
| sdague | heh | 18:37 |
| sdague | well, it's their own damn fault for living in portland ;) | 18:37 |
| *** otter768 has quit IRC | 18:38 | |
| yolanda | here, i'm trying to push a puppet-setproxy module, to stackforge. But i have a question at where to point the acls | 18:38 |
| mordred | jeblair: hang on - not sure that's the actual problem | 18:39 |
| yolanda | initially i pointed to puppet-modules.config but it's not the place, and it's intented to be modules to deploy openstack services | 18:39 |
| greghaynes | clarkb: So issue with using nodepool instead of the waitForImages hackery I did - starting nodepool causes all images to be built, so if im trying to test the a image-update causes one image to be built its not really possible | 18:39 |
| yolanda | my module is to setup a generic proxy service, more related to infra, but not used by you right now | 18:39 |
| greghaynes | and you also cant determine if it was the command or nodepool that caused the image to be built | 18:39 |
| anteaya | yolanda: why is puppet-modules.config not the place? | 18:39 |
| anteaya | whom do you want to have permissions? | 18:40 |
| yolanda | anteaya, no, puppet-setproxy is not deploying an openstack service, but adding a generic proxy feature | 18:40 |
| yolanda | i was asking with emilienm | 18:40 |
| clarkb | greghaynes: you can determine, you let the images build first, note them then build your image and look for the new one | 18:40 |
| yolanda | i was talking | 18:40 |
| greghaynes | clarkb: hrmmm | 18:40 |
| fungi | bdpayne: yep, needed to have f added to your channel access flags to be able to alter the access list. i also added the r flag (kickban control) for parity with other chanops | 18:41 |
| *** emagana has joined #openstack-infra | 18:41 | |
| fungi | bdpayne: also i left out a parameter in the command i gave you | 18:41 |
| *** ghostpl_ has joined #openstack-infra | 18:41 | |
| greghaynes | clarkb: That works, my concern is that actually makes the test code more complicated and adds the caveat *only tested with nodepool currently running | 18:41 |
| fungi | bdpayne: /msg chanserv access #openstack-security add whoever_you_want +Aefortv | 18:41 |
| mordred | jeblair, SpamapS, mtreinish: ok - so this: https://github.com/mysql/mysql-server/blob/7e58ce8e69de5a4e941d913999cbbf905fe8f444/storage/innobase/row/row0merge.cc#L2352 | 18:41 |
| anteaya | yolanda: so for me to help, I need you to be able to tell me what groups you want to have permssions on the acl file, I can't decide that | 18:42 |
| mordred | is the actual place where it's going bad | 18:42 |
| bdpayne | fungi excellent, thanks, I'll give it a go again | 18:42 |
| mordred | which has to do with secondary index calculation | 18:42 |
| clarkb | greghaynes: ya, I am not sure its better | 18:42 |
| *** ghostpl_ has quit IRC | 18:42 | |
| clarkb | greghaynes: I just don't like hae mostly duplicated code, maybe we can reduce the duplication instead? | 18:42 |
| greghaynes | clarkb: I was thinking of maybe passing in an arg to nodepool.start that makes it no do its main loop | 18:42 |
| greghaynes | just db init | 18:42 |
| mordred | which makes me think that we are doing a table copy, and that it is in the calculation of the secondary index that things are going batty | 18:42 |
| yolanda | so anteaya, infra should be fine for me, as the other puppet infra modules, although you are not end users of that at the moment | 18:42 |
| yolanda | we are using that for puppet downstream consumption of infra | 18:42 |
| mordred | solutions may include increasing sort-buffer-size - HOWEVER - it's tough to say to what we need to increase it | 18:43 |
| bdpayne | fungi it works, thanks again! | 18:43 |
| mordred | the other possibility is trying to work out where it's writing temp files | 18:43 |
| anteaya | yolanda: not sure how infra feels about having permissions on a puppet module in stackforge | 18:43 |
| fungi | bdpayne: yw, sorry for the brief misstep there | 18:43 |
| anteaya | yolanda: better ask jeblair | 18:43 |
| mordred | mtreinish: which is "show global variables like 'tmpdir'" | 18:43 |
| mordred | because it's possible tempdir is pointing to a location that cannot handle the temp files needed to do on-disk sorting | 18:44 |
| yolanda | anteaya, jeblair, i didn't want to propose directly to be part of infra, because you are really not going to be using that upstream | 18:44 |
| yolanda | but it's related to it | 18:44 |
| mtreinish | mordred: http://paste.openstack.org/show/192700/ | 18:44 |
| mtreinish | mordred: heh, that was my original guess as to what was going wrong :) | 18:44 |
| *** dprince has joined #openstack-infra | 18:44 | |
| openstackgerrit | Khai Do proposed openstack-infra/project-config: Remove jobs for Gerrit 2.4.x https://review.openstack.org/164821 | 18:45 |
| mtreinish | mordred: although I don't know if /var/tmp is correct, since I don't know what the trove fs is setup like | 18:45 |
| mordred | right - this is the problem with the opaque service thing ... I have no idea how big /var/tmp is | 18:45 |
| mordred | nor where else we should put things | 18:46 |
| anteaya | yolanda: okay then again we come to, who do you want to have permissions on the repo? | 18:46 |
| *** jp_at_hp has quit IRC | 18:46 | |
| yolanda | anteaya, can we have it? i mean, myself, rcarrillocruz, jedimike and our group | 18:46 |
| openstackgerrit | Merged openstack-infra/shade: Clean up race condition in functional tests https://review.openstack.org/164761 | 18:46 |
| yolanda | not sure if that's possible | 18:47 |
| anteaya | yolanda: your repo, you take responsibility | 18:48 |
| anteaya | it is stackforge, you can do anything you want | 18:48 |
| mtreinish | fungi: ^^^ any ideas on where to poke on the trove instance to figure that out? | 18:48 |
| *** mikeit has joined #openstack-infra | 18:48 | |
| *** tmcpeak has left #openstack-infra | 18:48 | |
| yolanda | so if i can create an acl and we are members, i'm really fine for it | 18:48 |
| anteaya | and yes that seems way more reasonable to me | 18:48 |
| anteaya | yolanda: yes | 18:48 |
| yolanda | anteaya, thx, i'll create a new acl then | 18:48 |
| nibalizer | +1 on a gozer acl file | 18:48 |
| nibalizer | kindof +1, its also hilarious and bad | 18:48 |
| nibalizer | but I like it | 18:48 |
| yolanda | nibalizer, shall i call it gozer really? | 18:49 |
| nibalizer | whynot | 18:49 |
| nibalizer | :shrug: | 18:49 |
| nibalizer | any other name would mean the same thing but be a roundabout way of saying it | 18:49 |
| yolanda | well, that will mean closing to our group? but i cannot think of a good name, i'm so bad on that | 18:50 |
| yolanda | ok, gozer | 18:50 |
| yolanda | :) | 18:50 |
| clarkb | mordred: see comment on 164691, don't want to approve if your discussion with sdague changes anthing | 18:50 |
| mordred | clarkb: cool | 18:51 |
| clarkb | SpamapS: see comment on 159674 | 18:51 |
| mordred | clarkb: honest.y I believe it's going to take a bit of time to track down where, if anywhere, the bug is | 18:52 |
| mordred | clarkb: so, I've added filing it to my TDL - but it may take a while - the patch, otoh, needs to be there even if it is a bug, since that behavior has been observed in the wild :( | 18:53 |
| *** mwagner_lap has quit IRC | 18:53 | |
| clarkb | ya | 18:53 |
| mordred | today is basically just a day pointing out that software generally does not work | 18:54 |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Test snapshot image deletion command https://review.openstack.org/164473 | 18:54 |
| clarkb | mordred: I am reading 163540, any thoughts on how you would disable puppet apply from running on a node? | 18:55 |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Update puppet-setproxy to belong to Gozer group https://review.openstack.org/164820 | 18:55 |
| clarkb | mordred: I dont think it needs to be solved in that change since it won't affect us just yet, but something to think about | 18:55 |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Test snapshot image deletion command https://review.openstack.org/164473 | 18:55 |
| greghaynes | clarkb: ^ hacky but seems to work | 18:55 |
| mordred | clarkb: yes - I do have a thought on that ... | 18:56 |
| fungi | mtreinish: i don't think it exposes any of that implementation detail to us | 18:56 |
| mordred | clarkb: and it's to have a disabled group defined on the puppetmaster that we can add and remove hosts from ... then make all of our puppet run host lines have a ;!disabled in them | 18:56 |
| mordred | clarkb: most of teh time the group will be empty/notexist | 18:57 |
| yolanda | anteaya, thx, change filled | 18:57 |
| fungi | mtreinish: we could prod trove devs to find out if this is even within the scope of the service to tell us at all, or prod rackspace admins if not | 18:57 |
| clarkb | mordred: an inventory group? | 18:57 |
| mordred | yup | 18:57 |
| clarkb | gotcha | 18:57 |
| *** afazekas has joined #openstack-infra | 18:57 | |
| *** Rockyg has joined #openstack-infra | 18:58 | |
| mordred | clarkb: that should make us doing puppet apply on hosts that are out of rotation pretty easy too | 18:58 |
| mordred | since we won't have to override lock files there | 18:58 |
| *** baoli_ has quit IRC | 18:58 | |
| clarkb | greghaynes: I don't think thats hacky at all | 18:58 |
| clarkb | mordred: ya | 18:58 |
| mordred | clarkb: so - my brain is less mush now | 18:59 |
| greghaynes | clarkb: well, updateConfig seems like a method that maybe should be private | 18:59 |
| greghaynes | but its not so I wasnt going to ask questios | 18:59 |
| mordred | clarkb: what I was thinking (which incidentally seems to be what nibalizer and greghaynes came up with independently) is to start off with ansible group_vars defining a mapping of hosts to hiera keys | 18:59 |
| clarkb | greghaynes: not necessarily, its basically a public interface to all of the nasty that updating the config does | 18:59 |
| greghaynes | clarkb: welp, works for me then ;) | 19:00 |
| mordred | clarkb: that way, the ansible on the puppetmaster can read the hiera file, and then pass in the appropriate keys for each host to the hieradata parameter so that they get written to each individual hosts's hieradata file | 19:00 |
| *** baoli has joined #openstack-infra | 19:00 | |
| greghaynes | mordred: our config mgmt arguing over drinks is not to be trusted ;) | 19:00 |
| mordred | clarkb: to do that, and to avoid a bunch of namespace pollution otherwise, I was mainly planning on having a single dict called "hiera" that's defined in the ansible variables | 19:00 |
| clarkb | mordred: so I was thinking about that too, I think it would be better to use the hierarchical properties of hiera and have ansible understand them (possibly by calling hiera directly) | 19:01 |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Revert "Disable arrow alignment puppet lint check" https://review.openstack.org/164818 | 19:01 |
| mordred | clarkb: we could do that - but we still need know which keys a given host wants to get | 19:01 |
| clarkb | mordred: so then you say hiera give me the hierarchy for foo.o.o then pass that verbatim into the ansible play | 19:01 |
| *** mikeit has quit IRC | 19:01 | |
| clarkb | mordred: then you don't have to override any hash behavior | 19:01 |
| mordred | heira doesn't know that | 19:01 |
| clarkb | mordred: yes it does | 19:01 |
| clarkb | thats how hiera works | 19:01 |
| mordred | not once we get rid of the puppetmaster it doesn't | 19:01 |
| *** ildikov has joined #openstack-infra | 19:02 | |
| clarkb | yes it still does | 19:02 |
| *** marun_ has quit IRC | 19:02 | |
| mordred | no. we make hiera calls in our site.pp | 19:02 |
| clarkb | hiera is an independent utility that interprets hierarchical data for you | 19:02 |
| clarkb | then you just ask for it | 19:02 |
| greghaynes | There was also some trickery nibalizer did to get hiera into a yaml format | 19:02 |
| greghaynes | which would let ansible read it | 19:02 |
| mordred | clarkb: we dont use it that way | 19:02 |
| *** mpavone has quit IRC | 19:02 | |
| clarkb | mordred: but we can | 19:02 |
| clarkb | mordred: and everything will still work fine | 19:02 |
| mordred | that's adding crazy complexity | 19:02 |
| clarkb | mordred: its not really | 19:02 |
| clarkb | mordred: you would just need a global set of settings, then any overrides as necessary per node | 19:02 |
| clarkb | mordred: thats it | 19:03 |
| clarkb | right now we are all global and use the lookups to scope | 19:03 |
| *** tsg_ has joined #openstack-infra | 19:03 | |
| mordred | clarkb: I understand what you are saying, I think it is needless complexity | 19:03 |
| clarkb | mordred: what is complex about it? | 19:03 |
| clarkb | mordred: I think it basically removes complexity from ansible and allows hiera to do its job | 19:03 |
| clarkb | mordred: you would still write out a hiera file on the remote host, just have hiera interpret what those contents should be for you | 19:03 |
| clarkb | mordred: so no puppet master is required | 19:04 |
| mordred | but why bother with hiera? | 19:04 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 19:04 | |
| mordred | on the master | 19:04 |
| mordred | you don't need it at all | 19:04 |
| mordred | it's an extra moving part that provides no value | 19:04 |
| clarkb | mordred: because it represents the data precisely how you want it | 19:04 |
| mordred | no | 19:04 |
| mordred | ansible can also just directly do that | 19:04 |
| clarkb | you want hierarchical data with overrides | 19:04 |
| mordred | there is no need to have hiera as well | 19:04 |
| *** baoli has quit IRC | 19:04 | |
| mordred | I'm just not proposing that as step one because it's too many steps | 19:04 |
| mordred | you need hiera where puppet runs | 19:05 |
| clarkb | ok | 19:05 |
| mordred | you do not need hiera where puppet does not run | 19:05 |
| clarkb | yes | 19:05 |
| clarkb | however | 19:05 |
| clarkb | hiera is a generic tool that solves this problem | 19:05 |
| clarkb | maybe ansible should learn to speak hiera as a general thing | 19:05 |
| mordred | yes. but so is the built in ansible group var system | 19:05 |
| mordred | it DOES | 19:05 |
| mordred | if you set merge=True | 19:05 |
| mordred | it does literally the same thing | 19:05 |
| clarkb | but they specifically say do not do this | 19:05 |
| mordred | no | 19:05 |
| mordred | they say not to do it unless you want that behavior | 19:05 |
| mordred | they say it's not _generally_ what you should do- because hierarchical overriding data is more complex | 19:06 |
| mordred | to the point - we've also avoided it so far | 19:06 |
| mordred | however, I have faith taht we're smart enough to comprehend the complexities | 19:06 |
| clarkb | sure, but they also say that std module won't be used that way | 19:06 |
| clarkb | so we can't be sure we get proper behavior out of them with this change | 19:06 |
| clarkb | and they strongly advice against it (for these reasons I assume) | 19:06 |
| mordred | std modules don't ship with variables | 19:06 |
| *** BharatK has quit IRC | 19:07 | |
| clarkb | mordred: uh I just used the git module and it accepted variables iirc | 19:07 |
| *** ashishb has quit IRC | 19:07 | |
| mordred | accepted | 19:07 |
| mordred | this has nothing to do with that | 19:07 |
| clarkb | yes it does | 19:07 |
| mordred | this has to do with how group_vars are read from disck | 19:07 |
| mordred | disk | 19:07 |
| clarkb | you could declare the git remote via -e | 19:07 |
| clarkb | or via inventory | 19:07 |
| mordred | ure - but those are input parameters to the module | 19:07 |
| clarkb | and that has different behaviors based on merge vs replace | 19:07 |
| *** AJaeger has joined #openstack-infra | 19:07 | |
| mordred | the module they ship has no bearing on how the input parameters are constructed | 19:08 |
| mordred | no - it really doesn't | 19:08 |
| mordred | like, not even a little bit | 19:08 |
| clarkb | mordred: then why do they call it out in their docs | 19:08 |
| clarkb | I should just give up on ever reading any ansible docs | 19:08 |
| mordred | clarkb: their docs are terrible :) | 19:08 |
| mordred | yes | 19:08 |
| mordred | clarkb: also, to be fair, we do tons of things that our upstream software authors say not to do | 19:08 |
| mordred | clarkb: like isntalling software globally with pip | 19:09 |
| mordred | clarkb: and just about everythign we do with puppet | 19:09 |
| mordred | clarkb: in fact, I believe the gerrit guys told us running gerrit in a VM was insane | 19:09 |
| clarkb | mordred: sure but the difference there is we tend to understand those tools | 19:09 |
| clarkb | mordred: you are the only person right now that groks ansible | 19:09 |
| mordred | indeed - I aim to fix that | 19:09 |
| clarkb | mordred: this means the rest of us will rely on things like documentation | 19:09 |
| clarkb | mordred: if the documentation straight up says "don't do X" I am going to comment that we shouldn't do X | 19:10 |
| mordred | clarkb: fair | 19:10 |
| clarkb | its less that I actually have an opinion here and more the only opinion I have managed to create comes from the upstream docs | 19:11 |
| mordred | clarkb: also, I probably shouldn't braindump 4/5 of a set of patches - the next patch might make the conversation on this clearer - also might make it clearer that I'm wrong :) | 19:11 |
| mordred | jeblair, mtreinish: another workaround is that we could probably do this: http://paste.openstack.org/show/192684/ | 19:11 |
| mtreinish | fungi: hmm, I guess we don't have login access to the node directly. I guess we can just ask SlickNik about it then | 19:12 |
| mordred | mtreinish: unfortunately, SlickNik works for HP and our trove is at Rackspace | 19:12 |
| mordred | so it may not be apples to apples | 19:12 |
| *** dboik has quit IRC | 19:12 | |
| AJaeger | Hi, this infra-manuals change now has 3 +2s - could somebody approve it, please? https://review.openstack.org/#/c/161454/ | 19:12 |
| *** unicell has joined #openstack-infra | 19:12 | |
| fungi | mtreinish: right, it's first ask SlickNik if there's any feature in trove to expose what we're looking for, and then ask rackspace admins to look for us if he has no suggestions | 19:13 |
| *** dboik has joined #openstack-infra | 19:13 | |
| *** afazekas has quit IRC | 19:13 | |
| *** sputnik13 has quit IRC | 19:14 | |
| mtreinish | mordred: ah, so just do the migration manually to avoid using the alter table. Yeah that should work | 19:14 |
| clarkb | greghaynes: reading 164356 you don't actualyl check the output of the command aiui | 19:14 |
| clarkb | greghaynes: merely that the mocked object was called | 19:14 |
| clarkb | greghaynes: and this is the problem with mocks | 19:14 |
| mtreinish | mordred: the only other thing we probably should do is update the alembic version string too | 19:14 |
| greghaynes | clarkb: what do you mean? I am just asserting the number of rows | 19:15 |
| mordred | clarkb: actually, the docs say "playbooks in the official examples repos do not use this setting" - no mention of modules | 19:15 |
| clarkb | greghaynes: but none of the content | 19:15 |
| mtreinish | mordred: but I can't do that because I don't have access to the privileged account on the db | 19:15 |
| greghaynes | clarkb: I was specifically -1'd from doing that in the previous patch | 19:15 |
| openstackgerrit | Merged openstack-infra/infra-manual: Mention Depends-On where the commit is made https://review.openstack.org/161454 | 19:15 |
| AJaeger | thanks, fungi ! | 19:15 |
| clarkb | greghaynes: oh? /me goes to read | 19:15 |
| openstackgerrit | Merged openstack-infra/infra-manual: Use change instead of patch https://review.openstack.org/161632 | 19:15 |
| clarkb | greghaynes: I do not see any -1 for that | 19:16 |
| greghaynes | clarkb: re: asserting 'ready' -> were asserting column order | 19:16 |
| greghaynes | per, patch 1 | 19:16 |
| clarkb | greghaynes: you can ignore order, load everything into a set and compare | 19:16 |
| mtreinish | mordred: oh, you know what I can probably add a .sql file to that within alembic and push that as a bug fix release | 19:16 |
| tsg_ | sdague: thoughts on https://review.openstack.org/#/c/164112/? | 19:16 |
| mordred | clarkb: so - https://review.openstack.org/#/c/163538/1/playbooks/group_vars/all.yaml <-- if you have that, and then also have playbooks/group_vars/zuul.yml that has hierakeys: {'zuul_ssh_private_key_contents', True} - then with merge we'd wind up with a hierakeys dict that had sysadmins and zuul_ssh_private_key_contents in it for zuul | 19:16 |
| mordred | mtreinish: really? that's a thing you can do in alembic? | 19:17 |
| clarkb | greghaynes: but now I understand why you didn't like my idea from before because you are not actually checking the output just that there is output | 19:17 |
| mordred | mtreinish: (also, that script is untested - we should test it for functional completeness somehwere with a small version of the data) | 19:17 |
| greghaynes | clarkb: which comment? | 19:17 |
| greghaynes | oh, about checking for updates | 19:17 |
| *** banix has joined #openstack-infra | 19:18 | |
| mtreinish | mordred: yeah I'm pretty sure you can. You definitely used to be able to do it with sqlalchemy migrate, and I'm pretty sure it's still in alembic | 19:18 |
| clarkb | greghaynes: yes, you can't check for updates because you can only assert the number of rows | 19:18 |
| clarkb | greghaynes: I still don't see the -1 | 19:18 |
| mtreinish | mordred: heh, I figured that (integer is spelt wrong :) ) | 19:18 |
| greghaynes | clarkb: Yea, my issue is just complexity. Its totally doable, I just dont want to make a case where changing features = lots of test churn because then you end up just rewriting your tests to make changes at which point your tests are useless | 19:18 |
| clarkb | greghaynes: I would assert that this modification to the tests is just barely more useful than useless | 19:19 |
| greghaynes | clarkb: https://review.openstack.org/#/c/164473/1/nodepool/tests/test_commands.py | 19:19 |
| clarkb | greghaynes: its just checking that you got a row back | 19:19 |
| clarkb | what if that is an error message instead of node data? | 19:19 |
| clarkb | it will still be counted as successful because we have mocked it out | 19:19 |
| greghaynes | clarkb: sure, so that seems valid to check (that there is a 'ready column' | 19:19 |
| clarkb | greghaynes: https://review.openstack.org/#/c/164356/2 is the change I am looking at fwiw | 19:20 |
| *** ghostpl_ has joined #openstack-infra | 19:21 | |
| greghaynes | psh, well that would explain the lack of seeing a -1 ;) | 19:21 |
| *** SumitNaiksatam has quit IRC | 19:21 | |
| *** afazekas has joined #openstack-infra | 19:21 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Make mysql clients assume utf8 character encoding https://review.openstack.org/164264 | 19:22 |
| *** ParsectiX_ has joined #openstack-infra | 19:22 | |
| greghaynes | so yes, its basically a little more than a smoke test | 19:22 |
| *** ParsectiX_ has quit IRC | 19:22 | |
| greghaynes | but thats >> than no test | 19:22 |
| greghaynes | I was planning on doing the fail cases next, so maybe ill add in asserting image state in that? | 19:23 |
| clarkb | greghaynes: except we already have a test that checks the same number of explosion points as that one | 19:23 |
| nibalizer | mordred: clarkb reading above I get a little lost | 19:23 |
| clarkb | greghaynes: basically does image build yes or no | 19:23 |
| greghaynes | clarkb: I dont think so because I wrote a bug in a parent patch which was only found by my test | 19:23 |
| clarkb | greghaynes: if we want to confirm beyond that we should actually assert some of the state values returned | 19:23 |
| greghaynes | which was that I broke the deleteImage method | 19:23 |
| clarkb | greghaynes: deleteImage isn't tested yet | 19:24 |
| *** dhritishikhar has quit IRC | 19:24 | |
| *** tsg_ has quit IRC | 19:24 | |
| *** HeOS has joined #openstack-infra | 19:24 | |
| mordred | nibalizer: that's fine - we were mostly just having one of our fun chats - I believe I need to write an additional patch so we can point at something and make faces | 19:24 |
| nibalizer | i like the idea of having ansible gather facts, then feed them to the hiera command line utility to peform lookups | 19:24 |
| nibalizer | but as mordred said that poses a problem because a node, before the puppet run, does not know which values it needs | 19:25 |
| greghaynes | clarkb: right, so I can just add in that they are all in 'ready' state? | 19:25 |
| *** AJaeger has quit IRC | 19:26 | |
| nibalizer | clark and i discussed using a two file structure, putting all common data in common.yaml and all node-specific data in <fqdn>.yaml then having ansible merge common.yaml with fqdn.yaml and shove that over | 19:26 |
| *** ghostpl_ has quit IRC | 19:26 | |
| nibalizer | that doesn't work super great because what do you do with shell globbing | 19:26 |
| clarkb | greghaynes: you may want to assert not ready so maybe make ready a default parameter to that utility method | 19:26 |
| mordred | nibalizer: yah - there's a WAY easier way to deal with that I believe | 19:26 |
| nibalizer | you could also just hand the data to ansible in ansibles way and call it good, but thats a handwavy approach | 19:26 |
| greghaynes | jeblair: You ok with ^ if I change https://review.openstack.org/#/c/164473/1/nodepool/tests/test_commands.py to not depend on column ordering? | 19:26 |
| clarkb | greghaynes: for now ready is sufficient so that can be deffered too | 19:26 |
| mordred | nibalizer: no, that's what I eventually think I want to do - but as a first step I want to have an explicit mapping in yaml file mapping node to which keys it can have | 19:27 |
| nibalizer | I'd really like to bring up the ansible-puppet-apply stuff on my test infra but it doesn't seem ready yet, and I havent' put a huge chunk of time into it | 19:27 |
| clarkb | mordred: ya thats kinda the bit I don't like | 19:27 |
| mordred | clarkb: yeah - it's really just a stepping stone | 19:27 |
| nibalizer | mordred: i have a proposal | 19:27 |
| clarkb | mordred: its very similar to the dib parameters spec thing | 19:27 |
| clarkb | mordred: and its :( | 19:27 |
| nibalizer | wait no i dont | 19:27 |
| mordred | clarkb: so that we don't have to change too many things at once | 19:28 |
| nibalizer | what if we encrypt all the element inn hieradata, then put it in a git repo, then clone it to every host just like project-config | 19:28 |
| mordred | clarkb: I could also just propose a more aggresive design if you want | 19:28 |
| clarkb | mordred: so I think the work around for that is using hiera as a described | 19:28 |
| nibalizer | and only paste.o.o can decryt the secrets for paste.o.o ? | 19:28 |
| clarkb | mordred: because then very little changes, its just hiera continuing to feed data in | 19:28 |
| clarkb | nibalizer: please no | 19:29 |
| mordred | clarkb: your thing is as disruptive as my thing | 19:29 |
| nibalizer | then ansible lays down 3 git repos, hieradata, project-config, and system-config then puppet runs | 19:29 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Make mysql clients assume utf8 character encoding https://review.openstack.org/164264 | 19:29 |
| mordred | clarkb: basically I'm just trying to avoid a step where we need to all of a sudden make the current hieradata file a complex directory structure | 19:29 |
| mordred | while changing something else too | 19:29 |
| *** ayoung has quit IRC | 19:29 | |
| clarkb | mordred: I see | 19:29 |
| mordred | if we're willing to do that | 19:29 |
| mordred | then we might as well just do that | 19:29 |
| nibalizer | im not afraid of a directory structure in hiera | 19:29 |
| clarkb | mordred: one thing I will say is I don't think it needs to be complex, we can change a host at a time | 19:30 |
| *** bswartz has quit IRC | 19:30 | |
| clarkb | mordred: while using puppet agent | 19:30 |
| nibalizer | that is the right direction to take hiera anyways | 19:30 |
| *** afazekas has quit IRC | 19:30 | |
| nibalizer | clarkb: is right | 19:30 |
| nibalizer | its also a safe change because if hiera lookup fails you'll get a catalog compilation | 19:30 |
| mordred | except once we dont' have puppetmaster, there is no need to keep hiera on the master | 19:30 |
| nibalizer | well actually you wont because we spammed defaults into our hiera lookups | 19:30 |
| nibalizer | removing hiera from the master has value because keeping a ruby environment around is pain | 19:31 |
| clarkb | mordred: ya that is the one issue with it. If the end goal is no hiera then probably not worth it | 19:31 |
| mordred | it also is one less thign to describe in terms of how the system hangs together | 19:31 |
| nibalizer | so drop that off the side asap :D | 19:31 |
| openstackgerrit | Merged openstack-infra/project-config: Fix upload built artifact in gerrit job https://review.openstack.org/164815 | 19:31 |
| clarkb | mordred: but I don't think you had ever expressed that before because puppet master != hiera | 19:32 |
| mordred | clarkb: I may not have fully realized it until this conversation | 19:32 |
| clarkb | mordred: did you know chef uses hiera? :) | 19:32 |
| mordred | clarkb: I've actually been trying hard to keep to easily grokkable stepwise chunks - that's probably a mistake I'm sensing | 19:32 |
| nibalizer | there is another option, which is that hiera could remain centralized and we could build a simple remote lookup system | 19:32 |
| mordred | clarkb: nice | 19:33 |
| mordred | nibalizer: nonononono | 19:33 |
| mordred | nononononononononononononono | 19:33 |
| mordred | I'm trying very very hard to get rid of central systems that thigns call in to needing auth from remote system inbound to central system | 19:33 |
| nibalizer | we could move the hieradata into something central like etcd or mysql or whatever and all the nodes could reach in and get the data | 19:33 |
| clarkb | mordred: yes. I agree that having steps is nice but without larger context few have made sense individually | 19:33 |
| mordred | clarkb: ++ I will fix that and write up and end to end spec - I think it's past time | 19:34 |
| mordred | nibalizer: yeah - that's the exact opposite direction | 19:34 |
| nibalizer | mordred: yea a spec would be great | 19:34 |
| mordred | nibalizer: the great thing about puppet apply is that it's decentralized and there is no daemon thing that anything out there has to know how to auth to | 19:34 |
| mordred | nibalizer: using ansible to drive it is great because it's a push not a pull system - so that you don't _need_ anything complex on the master side - the master is a trusted party and can pass secrets outboudn all it wants | 19:35 |
| nibalizer | well so the super base case, ignoring security | 19:36 |
| mordred | nibalizer: once we add central services, then we've reinvented puppetmaster, but without the years of production experience and field testing taht puppetmaster has | 19:36 |
| nibalizer | would be to have ansible copy all of the hieradata dir to /etc/puppet/hiera on every host | 19:36 |
| clarkb | greghaynes: left comment on 164472 but approved because its sort of orthogonal to the change being made there. Curious what you think about it though | 19:36 |
| mordred | nibalizer: yes. that is the super simple version | 19:36 |
| nibalizer | and so the complexity you're adding is the filtering out of that per node | 19:36 |
| mordred | yup | 19:36 |
| *** packet has quit IRC | 19:37 | |
| nibalizer | what if instead of interacting with the data, ansible just copies common.yaml,group.yaml, and fqdn.yaml over to the other host | 19:37 |
| nibalizer | isn't that simpler? | 19:37 |
| greghaynes | clarkb: yes, we should. I didnt because its unclear how to do that in the cmd case but not the nodpool loop case | 19:37 |
| clarkb | greghaynes ya | 19:37 |
| greghaynes | Seems like we need to exception map :/ | 19:38 |
| mordred | nibalizer: maybe - let me think through that a little bit more | 19:38 |
| mordred | nibalizer: biggest issue there is group.yaml | 19:38 |
| mordred | nibalizer: because we're starting to mix worlds again as son as we have one of those | 19:38 |
| *** Sukhdev has quit IRC | 19:38 | |
| mordred | since ansible is where groups will be defined and hiera won't really know anything about that | 19:38 |
| nibalizer | hehe | 19:39 |
| *** mriedem has quit IRC | 19:39 | |
| nibalizer | group.yaml is just an optimization | 19:39 |
| mordred | so if we define a group.yaml - it won't really be grokkable by puppet agent in a graduated manner | 19:39 |
| greghaynes | clarkb: or even better were going to need to pass down logic about if in_cmd: raise else: return | 19:39 |
| mordred | nibalizer: it's kinda necessary given how many things we have that have 20 copies of themselves | 19:39 |
| nibalizer | so that you can have an afs.yaml instead of 4 afsXX.yaml files | 19:39 |
| mordred | right. that's exactly right | 19:39 |
| clarkb | greghaynes ya it will be nasty | 19:39 |
| mordred | and you've just described the ansible group_vars system | 19:39 |
| mordred | which is why I don't want to re-invent it | 19:39 |
| nibalizer | the more i think about it, the more i like the idea of a git repo with hieradata, and ansible jsut selectively scps it over to the hsots | 19:40 |
| nibalizer | and doesnt' read it | 19:40 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Hard code the sql for mysql on migration 1679b5bc102 https://review.openstack.org/164838 | 19:40 |
| mtreinish | mordred: ^^^ so actually from googling it doesn't look like alembic does that natively, but I can hack it in | 19:40 |
| nibalizer | because all the ansible read data, then write out yaml/json is janky and a big reason im nervous about this plan | 19:40 |
| mordred | we could totally do that - but it's still not hiera on the master | 19:40 |
| nibalizer | no hiera on the master | 19:40 |
| mordred | nibalizer: it's totally not janky | 19:40 |
| nibalizer | we use a convention in our hieradata git repo | 19:41 |
| mordred | nibalizer: it's 100% of the way ansible works- if you don't like ansible reading yaml data, then kinda the game is over :) | 19:41 |
| mordred | nibalizer: however ... that doesn't mean I don't like your copy files method ... | 19:41 |
| mordred | it's jst that puppet apply won't know anything about group.yaml files | 19:42 |
| *** dmorita has quit IRC | 19:42 | |
| mtreinish | mordred: the unit tests run the migration with data against a real backend so that should hopefully catch any issues | 19:42 |
| mordred | mtreinish: cool! | 19:42 |
| *** ashishb has joined #openstack-infra | 19:43 | |
| *** packet has joined #openstack-infra | 19:43 | |
| mordred | nibalizer: so once we copy those files into /etc/puppet/hieradata.yaml on the remote hosts - we need to make sure that puppet apply knows to not ignore afs.yaml | 19:43 |
| nibalizer | mordred: so if hiera search lookup is 1) fqdn 2) group 3) common then the values set in group.yaml will overrride common values but be overriden by fqdn values | 19:43 |
| nibalizer | right so i think we copy them to /etc/puppet/heireadata/{common,afs,afs01.o.o}.yaml and we set a value group=afs in puppet | 19:44 |
| mordred | nibalizer: how do we set that value? | 19:44 |
| nibalizer | so group=afs could be passed in by ansible facter thing, or just set in the node definition in site.pp | 19:44 |
| mordred | node def in site.pp would be better | 19:45 |
| mordred | so that puppet apply can be run without ansible driving it | 19:45 |
| nibalizer | yep | 19:45 |
| nibalizer | ya with a comment that shellglobbing in node definitons but not hiera lookups is dumb grupm grump throw salmon at puppetlabs | 19:45 |
| *** aysyd has quit IRC | 19:46 | |
| mordred | nibalizer: ok - ansible hostvars['group_names'] is a thing, so it's easy to see for each host which groups it's a member of | 19:46 |
| *** mriedem has joined #openstack-infra | 19:47 | |
| *** hdd has quit IRC | 19:47 | |
| mordred | nibalizer, clarkb: k. I think I can be on board with this as a thign to explore ^^ since I think it meets my desire of being able to do small steps but also eventually not needing to actually run hiera on the master to run ansible | 19:48 |
| *** soren has joined #openstack-infra | 19:48 | |
| clarkb | mordred nibalizer we would only copy data related to the host still right? | 19:49 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Set $group at node scope for hiera lookups https://review.openstack.org/164842 | 19:49 |
| mordred | clarkb: yes | 19:49 |
| soren | fungi, clarkb: Rumour has it that you were looking for me earlier? | 19:49 |
| nibalizer | i dont think hiera.yaml is managed by puppet ? | 19:49 |
| mordred | clarkb: we'd split hieradata in the current structure into a common/group/fqdn structure, which hiera can still serve to puppet agent | 19:49 |
| clarkb | soren yup, uvirtbot and openstack meetbot fight over bugs | 19:49 |
| nibalizer | we'll need to make sure that file is managed, and identical, on all machines | 19:49 |
| clarkb | bug 12345 | 19:49 |
| openstack | bug 12345 in isdnutils (Ubuntu) "isdn does not work, fritz avm (pnp?)" [Medium,Fix released] https://launchpad.net/bugs/12345 - Assigned to Matthias Klose (doko) | 19:49 |
| uvirtbot | Launchpad bug 12345 in isdnutils "isdn does not work, fritz avm (pnp?)" [Medium,Fix released] https://launchpad.net/bugs/12345 | 19:49 |
| uvirtbot | Launchpad bug 12345 in isdnutils "isdn does not work, fritz avm (pnp?)" [Medium,Fix released] | 19:49 |
| soren | Hah. | 19:49 |
| clarkb | soren ^ like so | 19:49 |
| fungi | botfight!!! | 19:50 |
| nibalizer | im just sad they dont loop | 19:50 |
| *** aysyd has joined #openstack-infra | 19:50 | |
| *** alexpilotti has quit IRC | 19:50 | |
| *** sdake has quit IRC | 19:50 | |
| mordred | nibalizer: ./modules/openstack_project/files/puppetmaster/hiera.yaml | 19:50 |
| soren | uvirtbot: leave | 19:50 |
| uvirtbot | soren: Error: "leave" is not a valid command. | 19:50 |
| soren | Really? | 19:50 |
| soren | Hm. | 19:50 |
| soren | uvirtbot: quit | 19:50 |
| *** uvirtbot has quit IRC | 19:50 | |
| soren | That should do it. | 19:51 |
| mordred | nibalizer: can you do a patch to site.pp adding group=blah in such a way that it woudl do something for hiera? | 19:51 |
| clarkb | soren if we make a list of channels to remove uvirtbot from where should we send it | 19:51 |
| clarkb | (this affects other channels too) | 19:51 |
| fungi | soren: anyway, the list at https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/eavesdrop.pp#n39 is all the channels where we currently have the openstack meetbot residing, and it does what uvirtbot did for us now | 19:51 |
| soren | clarkb: /quit probably applies pretty globally. | 19:51 |
| *** dmorita has joined #openstack-infra | 19:51 | |
| *** sdake has joined #openstack-infra | 19:51 | |
| soren | So it should not be an issue anywhere anymore. | 19:52 |
| nibalizer | mordred: https://review.openstack.org/164842 | 19:52 |
| nibalizer | but that just sets the var | 19:52 |
| mordred | nibalizer: neat! | 19:52 |
| clarkb | soren gotcha | 19:52 |
| nibalizer | the interesting part is telling hiera.yaml about your search space | 19:52 |
| nibalizer | then setting up the files properly | 19:52 |
| mordred | nibalizer: yeah - that's the part I don't fully grok | 19:52 |
| SpamapS | mordred: I lost track of the migration talk in the backscroll, but it looks like the answer is "find out how to get more space for temp tables, or run our own." ? | 19:52 |
| nibalizer | i need lunch but will do that after | 19:52 |
| *** xyang1 has joined #openstack-infra | 19:53 | |
| mordred | SpamapS: nope | 19:53 |
| mordred | SpamapS: new answer "run monty's manual sql migration script" | 19:53 |
| mordred | SpamapS: :) | 19:53 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Hard code the sql for mysql on migration 1679b5bc102 https://review.openstack.org/164838 | 19:53 |
| mordred | SpamapS: I dug into the innodb source code with stewart | 19:53 |
| *** amotoki has joined #openstack-infra | 19:53 | |
| mordred | SpamapS: it has to do with sort buffer, because the table copy was rebuilding secondary indexes | 19:53 |
| SpamapS | mordred: ah, and with your method you build the indexes one insert at a time. | 19:54 |
| mordred | SpamapS: doing the insert into select from step should not need sort buffer for bulk index creation, although it will be _slower_ because it'll be doing key at a time inserts into the secondary indexes | 19:54 |
| mordred | yup | 19:54 |
| mordred | so it will not need a blown-out buffer | 19:54 |
| SpamapS | that's unfortunate thouigh | 19:54 |
| SpamapS | the thing should have ridiculous amounts of temp space available | 19:54 |
| mordred | SpamapS: it's the cloud | 19:54 |
| mordred | SpamapS: who needs swap? | 19:55 |
| SpamapS | perhaps we should file a bug with the trove people about this. | 19:55 |
| *** soren has quit IRC | 19:55 | |
| mordred | SpamapS: you know we reformat our ephemeral drives as swap on our long-lived servers, yeah? | 19:55 |
| SpamapS | we can see free space on the datadir, right? | 19:55 |
| SpamapS | mordred: swap for tmpfs? | 19:55 |
| mordred | SpamapS: well, I didn't want to set tmpdir=datadir | 19:55 |
| mordred | SpamapS: no - I'm just saying - clouds tell us we don't want swap | 19:55 |
| mordred | because Cloud | 19:55 |
| *** salv-orl_ has quit IRC | 19:55 | |
| mordred | but we do want swap | 19:55 |
| *** hdd has joined #openstack-infra | 19:56 | |
| mordred | just like we want temp space on our database server | 19:56 |
| *** sputnik13 has joined #openstack-infra | 19:56 | |
| SpamapS | mordred: because swap is kind of a waste of time for servers IMO. It just makes OOM worse by making it harder to kill misbehaving processes. | 19:56 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Setup hiera lookups to do common,group,fqdn https://review.openstack.org/164845 | 19:56 |
| mordred | SpamapS: except when they just mildly misbehave | 19:56 |
| nibalizer | mordred: okay really going to get lunch now but that should get you started %% | 19:56 |
| nibalizer | er ^^ | 19:56 |
| mordred | nibalizer: yah. cool! | 19:57 |
| SpamapS | mordred: I guess I don't recall having mildly misbehaving processes ever, but mmmmk. ;) | 19:57 |
| *** afazekas has joined #openstack-infra | 19:57 | |
| mordred | SpamapS: imagine you run a long lived java process | 19:57 |
| * SpamapS is scared | 19:57 | |
| * SpamapS is cold | 19:57 | |
| *** amotoki has quit IRC | 19:58 | |
| SpamapS | oh sorry got a little wrapped up in that | 19:58 |
| mordred | SpamapS: that occasionally might want to chew up 60G of RAM for no reason | 19:58 |
| SpamapS | Yeah I've had those, SOLR was like that. | 19:58 |
| mordred | but would be just fine causing swap for a minute or two | 19:58 |
| *** r-daneel has joined #openstack-infra | 19:58 | |
| SpamapS | Like oh crap why are page faults 100x usual.. oh.. no virt mem for caching things. | 19:58 |
| mordred | yah | 19:58 |
| SpamapS | But my solution to that was to let OOM killer obliterate that java process, and then it would be resurrected by whatever process monitor was in place. | 20:00 |
| SpamapS | And we'd get a ticket to investigate why we had a known bad thing happen in our logs. | 20:00 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Make test_queries runnable as gating test https://review.openstack.org/150186 | 20:01 |
| SpamapS | ANd we'd say buy us more RAM for the SOLR servers, and then 8 months later... same problem but with 2x the RAM. ;) | 20:01 |
| *** tsg_ has joined #openstack-infra | 20:01 | |
| *** _nadya_ has joined #openstack-infra | 20:01 | |
| mtreinish | SpamapS: heh, so then you bought even more ram :) | 20:01 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add launchpad test to test_queries https://review.openstack.org/155138 | 20:01 |
| SpamapS | anyway, I acknowledge swap _can_ be useful. I find it makes relatively easy problems easy, and relatively complicated problems incredibly hard. | 20:01 |
| SpamapS | mtreinish: no then I quit. ;) | 20:01 |
| SpamapS | and somebody invented elastic search :) | 20:02 |
| *** shashankhegde has joined #openstack-infra | 20:02 | |
| SpamapS | So, back to the lecture at hand: We should probably be able to rebuild indexes on a production box. | 20:03 |
| *** timcline has quit IRC | 20:04 | |
| SpamapS | http://dev.mysql.com/doc/refman/5.6/en/innodb-create-index-implementation.html <-- explains that tmpdir space will cause alter fail | 20:05 |
| *** ghostpl_ has joined #openstack-infra | 20:06 | |
| rcarrillocruz | clarkb: hiya, re: sync/async of nova delete for tests, i'm not aware of flags for nova to do a poll/sync delete, per your comments it seems there's a way to do that? | 20:06 |
| *** hashar has joined #openstack-infra | 20:06 | |
| *** ivar-laz_ has joined #openstack-infra | 20:06 | |
| *** ivar-lazzaro has quit IRC | 20:09 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack-infra/project-config: Add "heat-coe-templates" project https://review.openstack.org/164806 | 20:09 |
| greghaynes | Theres another thing with stuff like solr where you usually *really* dont want it to swap because then rather than just crash (which you can typically failover for) itll just go really really slow | 20:09 |
| greghaynes | oh, SpamapS said that | 20:10 |
| *** mrunge has quit IRC | 20:10 | |
| greghaynes | \o/ for running full text indexer | 20:10 |
| *** tsufiev is now known as tsufiev_ | 20:10 | |
| clarkb | rcarrillocruz shade implements it | 20:13 |
| *** afazekas has quit IRC | 20:15 | |
| mordred | clarkb, nibalizer: stepped away for a sec - the more I think about it - the more I really like the new copy-files plan | 20:15 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Hard code the sql for mysql on migration 1679b5bc102 https://review.openstack.org/164838 | 20:15 |
| *** marun_ has joined #openstack-infra | 20:16 | |
| clarkb | mordred I am still not sure exactly what that plan is (I ate luncj) | 20:16 |
| mordred | rcarrillocruz: might actually jus twant to use cloud.delete_server in the cleanup function | 20:16 |
| jeblair | so much scrollback | 20:16 |
| mordred | clarkb: don't worry - I'm going to make you a nice new patch | 20:16 |
| mordred | clarkb: which I tink will be way LESS code than what's up now | 20:16 |
| *** afazekas has joined #openstack-infra | 20:16 | |
| clarkb | mordred cool | 20:16 |
| mordred | clarkb: and have a gradual cutover possible - AND not need to run hiera process on master | 20:17 |
| mordred | clarkb: in short- I think it's pretty kickass | 20:17 |
| jeblair | okay good, i'll skip all that then :) | 20:18 |
| *** timcline has joined #openstack-infra | 20:18 | |
| mordred | jeblair: yeah - just wait for the updated patch - WAY less reading | 20:18 |
| *** dimtruck is now known as zz_dimtruck | 20:18 | |
| rcarrillocruz | mordred , clarkb: ok, i found odd to use shade code to test shade code, thus wanted to stick to plain openstack client calls | 20:19 |
| rcarrillocruz | thx | 20:19 |
| greghaynes | jeblair: Heyo, clarkb mentioned that hed like to assert the content of image-list. I think its not a bad idea, but this is kind of at odds with your comment that we shouldnt assert column order (in 164473). I could use a set or somesuch to make it column order independent but IMO thats pretty wierd given that I imagine the content changes a ton more than colmn ordering... | 20:19 |
| *** alexpilotti has joined #openstack-infra | 20:19 | |
| greghaynes | Anyhow, I think we should pick one of the two, not both | 20:19 |
| mordred | rcarrillocruz: sure- I totally agree with you in the tests ... but in the cleanup it's probably fine ... otherwise you're going to have a ton of copied code | 20:19 |
| greghaynes | jeblair: er, for added context, this is from the nodepool command testing | 20:20 |
| *** zz_dimtruck is now known as dimtruck | 20:20 | |
| jeblair | greghaynes: yeah, was just trying to catch up on that, thanks | 20:20 |
| *** dprince has quit IRC | 20:22 | |
| *** marun_ is now known as marun | 20:23 | |
| *** afazekas has quit IRC | 20:23 | |
| *** kgiusti has left #openstack-infra | 20:23 | |
| greghaynes | I kind of think as long as we make the code a bit more clear then a change in column ordering would just result in status_colmn = new_index which isnt too much test churn | 20:24 |
| jeblair | greghaynes, clarkb: so my _primary_ thought is that mostly we want to make sure we don't break the commands -- ie, they should be able to cause images to be built, and the display ones should be able to read data, etc. | 20:25 |
| jeblair | greghaynes, clarkb: i think i'm okay with additional checks of the output in that first test case... | 20:25 |
| clarkb | I dont think we need to confirm age or anything like that. just a small sanity check that the row is valid data | 20:25 |
| jeblair | greghaynes, clarkb: i think the use of the output in the second case, where deleting a snapshot was being tested, was more troubling. | 20:25 |
| clarkb | and not "fail" | 20:25 |
| jeblair | clarkb: that sounds reasonable | 20:26 |
| *** tsg_ has quit IRC | 20:26 | |
| jeblair | greghaynes: but it looks like you've already removed that, so i think we're all on the same page. ++ | 20:26 |
| *** gokrokve has quit IRC | 20:27 | |
| *** gokrokve has joined #openstack-infra | 20:27 | |
| greghaynes | clarkb: yep, agreed on that goal | 20:27 |
| greghaynes | ok, cool | 20:27 |
| *** ashishb has quit IRC | 20:30 | |
| openstackgerrit | Boris Pavlovic proposed openstack-infra/project-config: Use template for Rally py34 job https://review.openstack.org/164858 | 20:31 |
| *** julim has quit IRC | 20:31 | |
| *** otter768 has joined #openstack-infra | 20:34 | |
| *** _nadya_ has quit IRC | 20:34 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Rename config-validate to validate-config https://review.openstack.org/164859 | 20:35 |
| *** salv-orlando has joined #openstack-infra | 20:35 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack-infra/project-config: Add "heat-coe-templates" project https://review.openstack.org/164806 | 20:37 |
| *** afazekas has joined #openstack-infra | 20:38 | |
| *** dkliban is now known as dkliban_brb | 20:39 | |
| *** otter768 has quit IRC | 20:39 | |
| *** e0ne has joined #openstack-infra | 20:39 | |
| *** dboik has quit IRC | 20:40 | |
| fungi | is distutils.spawn.find_executable() really the least revolting python 2.x alternative to shutil.which() from >=3.3? | 20:40 |
| *** dboik has joined #openstack-infra | 20:41 | |
| openstackgerrit | Merged openstack/requirements: Bump PyECLib version to 1.0.3 https://review.openstack.org/164112 | 20:41 |
| clarkb | zaro: http://logs.openstack.org/59/164859/1/check/nodepool-coverage/bf38a9f/cover/index.html cover is working now. Though with a minor issue where zuul swift upload generates an index.html even if there is alread an index.html | 20:42 |
| clarkb | I will work on a fix for that now | 20:42 |
| clarkb | jhesketh: ^ for when you wake | 20:42 |
| fungi | there are times when i wish python 2.x would just hurry up and die already | 20:43 |
| *** Rockyg has quit IRC | 20:43 | |
| mtreinish | mordred, SpamapS: I was just thinking if we can get get and put mysql config opts with trove, would it be bad to set tmpdir to be the data dir? | 20:43 |
| *** thedodd has quit IRC | 20:44 | |
| dkranz | sdague, fungi : I am trying to add some neutron stress tests but need this simple change to run the stress job with neutron https://review.openstack.org/#/c/164705/ | 20:44 |
| mordred | mtreinish: I mean - I'm a bit concerned about doing that - just because eek. however - maybe if we did a subdir of datadir | 20:45 |
| mordred | SpamapS: ^^ thoughts? | 20:45 |
| mordred | devananda: ^^ | 20:45 |
| openstackgerrit | Peter Hamilton proposed openstack/requirements: Update PyKMIP version in requirements https://review.openstack.org/164449 | 20:45 |
| clarkb | zaro: also those files don't appear to have a mime type of html | 20:45 |
| mordred | devananda: beacuse, you know, you don't have anything else better to do than mysql consulting this week | 20:45 |
| mtreinish | mordred: well can we create a subdir? :) | 20:45 |
| fungi | dkranz: i guess no longer running them with nova-net is acceptable loss? | 20:46 |
| mordred | mtreinish: we can - I'm not saying it won't be hacky | 20:46 |
| openstackgerrit | Peter Hamilton proposed openstack/requirements: Update PyKMIP version in requirements https://review.openstack.org/164449 | 20:46 |
| dkranz | fungi: actually, I have no objection to adding another one | 20:46 |
| mordred | mtreinish: but we could create a schema called "tmp" which would cause a subdir to be created in the datadir - and then we could set tmpdir=datadir/tmp which would use that dir | 20:46 |
| mordred | and if it trashes somethign in that dir *meh* | 20:46 |
| clarkb | dkranz: fungi that is probably preferable since nova net is still alive | 20:46 |
| dkranz | fungi: I know we have been trying to avoid more jobs but this is just periodic | 20:46 |
| zaro | clarkb: is that something that's doneon server side? | 20:47 |
| dkranz | clarkb: will do. BTW, it would really help if there were a way to run periodic jobs like this without having to run everything in check experimental | 20:47 |
| *** afazekas has quit IRC | 20:47 | |
| dkranz | clarkb: I know this was discussed in Paris but I | 20:47 |
| clarkb | zaro: I think both issues are related to how we upload the files, we should set the type so that swift knows its html and can be served to browsers that way | 20:47 |
| dkranz | clarkb: am not sure if there was any action planned | 20:48 |
| clarkb | zaro: and we shouldn't upload an index.html if one is laready there | 20:48 |
| fungi | dkranz: you mean like a way to manually trigger all periodic jobs? | 20:48 |
| dkranz | fungi: Ideally a way to manually trigger just one | 20:48 |
| dkranz | fungi: the periodic jobs are mostly unrelated | 20:49 |
| *** e0ne has quit IRC | 20:49 | |
| *** afazekas has joined #openstack-infra | 20:49 | |
| *** packet has quit IRC | 20:50 | |
| dkranz | fungi: right now if I want to test a stress test I have to do check experimental in tempest which runs a zillion other jobs too and is quite wasteful | 20:50 |
| *** hichihara has joined #openstack-infra | 20:50 | |
| mtreinish | mordred: heh, ok. :) I'll go back to debugging the mordred migration failure then: http://logs.openstack.org/38/164838/3/check/gate-subunit2sql-python27/7d4c73c/console.html#_2015-03-16_20_26_18_305 | 20:50 |
| *** andreykurilin_ has joined #openstack-infra | 20:50 | |
| mtreinish | fungi: I think dkranz is looking for something like a regex filter on check experimental, so you do check experimental $REGEX and it only runs experimental jobs that match regex | 20:52 |
| dkranz | mtreinish: +1 | 20:53 |
| *** dangers is now known as dangers_away | 20:53 | |
| fungi | mtreinish: dkranz: yeah, trying to think through the implications from zuul's perspective. right now it knows how to run all jobs in a particular pipeline for a given change's project, filtered based on branch and changed files | 20:53 |
| openstackgerrit | Merged openstack-infra/project-config: Rename experimental nova py27 to run on dsvm nodes https://review.openstack.org/164777 | 20:54 |
| fungi | so mostly pondering how one of those could be hooked to minimize complexity | 20:54 |
| fungi | in this case we'd need to have a special kind of comment trigger i think | 20:54 |
| mordred | mtreinish: it looks like it's still doing something with sqla | 20:54 |
| fungi | can't just use the normal comment trigger since the list of jobs to run would be dependent on the contents of the comment from the stream rather than determined explicitly in the config | 20:55 |
| dkranz | fungi: I understand it is nontrivial. My vision is that we should have all kinds of interesting periodic jobs for "non-golden" configurations and tests that take too long for the gate. | 20:55 |
| *** thedodd has joined #openstack-infra | 20:55 | |
| mordred | mtreinish: what's the microsecond_data_migration supposed to be doing? | 20:55 |
| dkranz | fungi: assuming of course they can be watched and not bitrot... | 20:56 |
| *** dustins has quit IRC | 20:56 | |
| anteaya | SpamapS: so on https://review.openstack.org/#/c/158148/3 both jeblair and ajaeger asked for it to be non-voting to start | 20:56 |
| *** jaypipes has quit IRC | 20:56 | |
| anteaya | SpamapS: is there a reason why you would like to test the job with a non-voting job to begin? | 20:56 |
| anteaya | would not | 20:56 |
| *** jaypipes has joined #openstack-infra | 20:57 | |
| mtreinish | mordred: it loops over all the rows extracts microseconds from the time columns and puts it in the new microseconds column | 20:57 |
| dkranz | fungi: thanks for the +2. After that merges I will add another non-neutron stress job. I need to make a new config for it in tempest first. | 20:57 |
| mordred | mtreinish: how is it doing that? | 20:58 |
| mordred | mtreinish: I didn't think the old column had microseconds? | 20:58 |
| *** harlowja is now known as harlowja_away | 20:58 | |
| mtreinish | mordred: it's skipped on the infra db with that config flag | 20:58 |
| clarkb | dkranz: fungi I think the idea was to run most of those things as post jobs so you wouldn't need to manuall trigger anything. The issue is we need some way of presenting the results to uses | 20:59 |
| mtreinish | I put that there in the case someone was running the db against a backend which didn't strip the microseconds | 20:59 |
| fungi | dkranz: i think the main pushback is going to be that the "recheck" and "check experimental" comments are a terrible interface to start with, and we should be doing this via a zuul api/ui rather than accumulating more technical debt on that front | 20:59 |
| mordred | mtreinish: ah - so the data migration ... gotcha | 20:59 |
| mordred | mtreinish: so the problem here is that clearly the migration did not succeed | 20:59 |
| *** carl_baldwin has joined #openstack-infra | 20:59 | |
| dkranz | fungi: It doesn't really matter much to me what the UX is for this | 20:59 |
| mordred | mtreinish: does that sqlalchemy execution context have a schema associated with it? | 20:59 |
| *** bhunter71 has quit IRC | 20:59 | |
| *** soren has joined #openstack-infra | 21:00 | |
| mtreinish | mordred: yeah, I'm manually stepping through the script now | 21:00 |
| dkranz | clarkb: I don't quite understand what you mean by post jobs | 21:00 |
| mtreinish | mordred: oh, dunno | 21:00 |
| *** packet has joined #openstack-infra | 21:00 | |
| clarkb | zaro: looking at response headers from swift the mime is text/html | 21:00 |
| clarkb | dkranz: jobs that run after every merge | 21:00 |
| mtreinish | mordred: fwiw, when I ran the script manually it failed: http://paste.openstack.org/show/192735/ | 21:00 |
| nibalizer | mordred: sweet let me know how i can help | 21:00 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 21:00 | |
| *** Sukhdev has joined #openstack-infra | 21:01 | |
| dkranz | clarkb: but I want to see them pass first before they are merged | 21:01 |
| dkranz | clarkb: and so do reviewers | 21:01 |
| mordred | mtreinish: k. gimme a sec | 21:01 |
| clarkb | dkranz: sure so thats a separate problem. What we discussed at the summit is the above thing | 21:01 |
| jogo | sdague: quick question related to running https://review.openstack.org/#/c/155138/ as a new job | 21:01 |
| dkranz | clarkb: ok, I missed that distinction, | 21:02 |
| jogo | oh neve rmind | 21:02 |
| clarkb | does anyone know why http://logs.openstack.org/59/164859/1/check/nodepool-coverage/bf38a9f/cover/nodepool_allocation.html wouldn't render given the response headers? | 21:02 |
| mordred | mtreinish: try this: LOCK TABLES test_runs write, test_runs_migration write; instead of the lock table command | 21:03 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Do not generate index.html if one exists https://review.openstack.org/164866 | 21:03 |
| clarkb | zaro: ^ thats part of the fix | 21:03 |
| mordred | mtreinish: and UNLOCK TABLES: instead of the unlock table line | 21:03 |
| mtreinish | mordred: now it complains that the column counts don't match: ERROR 1136 (21S01): Column count doesn't match value count at row 1 | 21:04 |
| mordred | mtreinish: neat. one sec | 21:05 |
| mordred | mtreinish: isn't this fun? | 21:05 |
| fungi | dkranz: so now i'm starting to miss what the connection is between requesting individual runs of experimental jobs and whatever you opened with about periodic jobs | 21:05 |
| *** ivar-laz_ has quit IRC | 21:05 | |
| clarkb | OH! we wrap that entire thing in a span | 21:05 |
| clarkb | os_loganalyze must be breaking that | 21:05 |
| clarkb | zaro: ^ | 21:05 |
| clarkb | er in <pre><span> | 21:06 |
| dkranz | fungi: the connection is that the tempest periodic jobs are also on the experimental queue which is the only way to run them on demand | 21:06 |
| *** ivar-lazzaro has joined #openstack-infra | 21:06 | |
| mordred | mtreinish: INSERT INTO test_runs_migration SELECT id, test_id, run_id, status, start_time, stop_time, 0, 0 from test_runs; | 21:06 |
| mordred | mtreinish: try that for the insert | 21:06 |
| dkranz | fungi: and I can do that, but it is a waste, and will get worse when we have more non-gate jobs that want to be triggered on demand | 21:06 |
| mtreinish | mordred: heh actually yes, I've learned more about mysql in the past few hours then I have since I started this project :) | 21:06 |
| fungi | dkranz: okay, but that's separate from the periodic jobs concern then i guess | 21:07 |
| dkranz | fungi: with respect to the issue of who is watching periodic jobs, yes. | 21:07 |
| * fungi needs to head out to dinner, bbl | 21:07 | |
| dkranz | fungi: sorry I mixed them together | 21:07 |
| mordred | mtreinish: you could do this: INSERT INTO test_runs_migration SELECT id, test_id, run_id, status, start_time, stop_time, microsecond(start_time), microsecond(stop_time) from test_runs; | 21:07 |
| *** ivar-lazzaro has quit IRC | 21:07 | |
| *** Slower has joined #openstack-infra | 21:07 | |
| *** jgrimm is now known as zz_jgrimm | 21:07 | |
| mordred | and avoid the need for the follow on sqlalchemy migration | 21:08 |
| Slower | so I made a channel #kolla, would like to get it registered | 21:08 |
| *** ivar-lazzaro has joined #openstack-infra | 21:09 | |
| *** amotoki has joined #openstack-infra | 21:09 | |
| *** andreykurilin_ has quit IRC | 21:09 | |
| Slower | and I'm not the only one on it :) | 21:09 |
| *** ayoung has joined #openstack-infra | 21:09 | |
| mtreinish | mordred: hmm last line fails now: ERROR 1217 (23000): Cannot delete or update a parent row: a foreign key constraint fails | 21:09 |
| *** tsg has joined #openstack-infra | 21:10 | |
| anteaya | Slower: register it with freenode | 21:10 |
| anteaya | Slower: and then read http://ci.openstack.org/irc.html#access | 21:10 |
| openstackgerrit | Joe Gordon proposed openstack-infra/project-config: Add job for network based elastic-recheck queries test https://review.openstack.org/164869 | 21:11 |
| jogo | sdague: ^ I am fairly sure I did that all wrong | 21:11 |
| devananda | mordred: mysql obfuscates the file names when making temp tables so eg. i couldn't do CREATE TEMP TABLE `../mysql/user` .. or something similar | 21:11 |
| clarkb | zaro yup I see the issue second patch soon | 21:11 |
| devananda | mordred: I mean, it seems crazy because I've never even wanted to do that | 21:11 |
| mtreinish | mordred: hmm, I guess I could, would that make the infra case slower though? | 21:11 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Enabled eslint:no-use-before-define https://review.openstack.org/161510 | 21:12 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Enabled eslint:no-shadow https://review.openstack.org/161508 | 21:12 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Enabled eslint:space-infix-ops https://review.openstack.org/161509 | 21:12 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Enabled eslint:space-unary-ops https://review.openstack.org/161506 | 21:12 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Enabled eslint:consistent-return https://review.openstack.org/161507 | 21:12 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Enabled eslint:strict https://review.openstack.org/161504 | 21:12 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Enabled eslint:no-console https://review.openstack.org/161505 | 21:12 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Enabled eslint:no-underscore-dangle https://review.openstack.org/161503 | 21:12 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Manage hiera.yaml on all hosts https://review.openstack.org/164870 | 21:12 |
| mordred | mtreinish: shouldn't do - it's doing a select from / insert into anyway | 21:12 |
| mordred | mtreinish: so it's traversing every row in teh table regardless - doing math on one of the values will not be noticable | 21:13 |
| Slower | anteaya: thanks! | 21:14 |
| anteaya | Slower: welcome | 21:14 |
| nibalizer | mordred: are we okay with some duplication (is there a need even?) between e.g. afsdb and afs groups/roles/machinecollections? i like keeping it explicit and non-inheriting because "yay simple" but we could get fancy if needed | 21:15 |
| *** ldnunes has quit IRC | 21:15 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Add unit test for create_server https://review.openstack.org/160155 | 21:15 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Add basic unit test for shade.openstack_cloud https://review.openstack.org/159674 | 21:15 |
| mordred | nibalizer: I think it's fine to duplicate - ansible can handle a thing being in more than one group, but the code would also handle copying all of teh related groups over | 21:16 |
| *** afazekas has quit IRC | 21:16 | |
| *** radez is now known as radez_g0n3 | 21:16 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Hard code the sql for mysql on migration 1679b5bc102 https://review.openstack.org/164838 | 21:18 |
| mtreinish | mordred: ^^^ that's the latest, it works except it doesn't move the foreign keys to the new table, so it can't drop the old one | 21:18 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Removed grunt:server https://review.openstack.org/163165 | 21:19 |
| mordred | mtreinish: oh - I can add that ... can I add it in the morning though? | 21:19 |
| mordred | mtreinish: are there inbound foreign keys from other tables? | 21:19 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Hard code the sql for mysql on migration 1679b5bc102 https://review.openstack.org/164838 | 21:20 |
| mtreinish | mordred: yes, the test_run metadata table has a foreign key which is the primary key in test_runs. (or did I get that backwards) | 21:21 |
| *** hashar has quit IRC | 21:22 | |
| *** sambetts has quit IRC | 21:22 | |
| *** tiswanso has quit IRC | 21:22 | |
| mordred | mtreinish: will you pastebin "show create table test_run \G" ? | 21:22 |
| mtreinish | mordred: http://paste.openstack.org/show/192699/ | 21:23 |
| *** afazekas has joined #openstack-infra | 21:23 | |
| mordred | mtreinish: no - that way should be fine .. one sec | 21:24 |
| *** asettle has joined #openstack-infra | 21:24 | |
| *** sambetts has joined #openstack-infra | 21:25 | |
| * anteaya goes to gather sap, now that the neutron meeting is over | 21:26 | |
| *** banix has quit IRC | 21:26 | |
| *** packet has quit IRC | 21:27 | |
| openstackgerrit | Doug Wiegley proposed openstack-infra/devstack-gate: Mechanism to allow easier fine-tuning of devstack-gate services https://review.openstack.org/164874 | 21:27 |
| mordred | mtreinish: http://paste.openstack.org/show/192737 | 21:28 |
| *** dkliban_brb is now known as dkliban | 21:28 | |
| mordred | mtreinish: somethign liek that | 21:29 |
| *** afazekas has quit IRC | 21:29 | |
| mordred | mtreinish: you need to drop the test_run_metadata foreign key so that you can drop test_runs_old; | 21:29 |
| mordred | but then add it back again pointing to the new table | 21:29 |
| mtreinish | mordred: ah, ok thanks | 21:30 |
| *** mriedem is now known as mriedem_away | 21:33 | |
| *** mattfarina has quit IRC | 21:33 | |
| *** garyh has quit IRC | 21:34 | |
| krotscheck | Can someone confirm an edge case for me? https://storyboard.openstack.org/#!/story/10 fails to resolve, throws the user back to the dashboard. Other stories are fine, but this one (and a few others) seem borked. | 21:37 |
| jeblair | krotscheck: confirmed (not logged in) | 21:37 |
| krotscheck | I think it's because that story predates story authors. | 21:37 |
| krotscheck | jeblair: Thanks | 21:38 |
| jeblair | krotscheck: also confirmed while logged in | 21:38 |
| *** cdent has joined #openstack-infra | 21:38 | |
| *** aysyd has quit IRC | 21:38 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Hard code the sql for mysql on migration 1679b5bc102 https://review.openstack.org/164838 | 21:38 |
| pleia2 | krotscheck: throws a bunch of errors (not logged in) | 21:39 |
| pleia2 | (I can screenshot if it's helpful) | 21:39 |
| krotscheck | Thanks everyone. | 21:39 |
| krotscheck | Naah, it's throwing those for me too. | 21:39 |
| krotscheck | and I have advanced debugging things. | 21:39 |
| * pleia2 nods | 21:39 | |
| pleia2 | figured :) | 21:39 |
| *** aysyd has joined #openstack-infra | 21:40 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Check image-list output in cmd tests https://review.openstack.org/164356 | 21:40 |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/project-config: Add Neutron DVR aiopcpu job for multi-node testing https://review.openstack.org/158415 | 21:40 |
| greghaynes | clarkb: ^ | 21:40 |
| greghaynes | Note we now have to wait_for_threads for obvious reasons | 21:41 |
| mtreinish | pleia2: do you remember what the syntax error was on the mysqldump from friday? | 21:42 |
| *** bswartz has joined #openstack-infra | 21:42 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Added null check for story creator_id. https://review.openstack.org/164877 | 21:42 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Hard code the sql for mysql on migration 1679b5bc102 https://review.openstack.org/164838 | 21:42 |
| pleia2 | mtreinish: I don't remember, but I can probably browse logs to see if I put it here/jog my memory | 21:42 |
| *** dizquierdo has joined #openstack-infra | 21:43 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/project-config: Add job for network based elastic-recheck queries test https://review.openstack.org/164869 | 21:43 |
| *** ddieterly has quit IRC | 21:43 | |
| mtreinish | pleia2: cool thanks | 21:44 |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Indicate error when deleting an invalid snapshot https://review.openstack.org/164472 | 21:44 |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Test snapshot image deletion command https://review.openstack.org/164473 | 21:45 |
| pleia2 | mtreinish: so we had the ansible logs http://paste.openstack.org/show/192214/ and the /opt/subunit2sql.sql.gz just contained the response you'd get if you just type "mysqldump" or have the wrong parameters | 21:46 |
| pleia2 | so a couple lines of usage | 21:46 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 21:47 | |
| pleia2 | I didn't find why that happened (what it was trying to run and failing) | 21:47 |
| mtreinish | pleia2: :( so it didn't say what the wrong params were? | 21:47 |
| pleia2 | alas, no | 21:47 |
| *** weshay has quit IRC | 21:48 | |
| pleia2 | meeting on St. Patrick's Day means we got a themed meeting announcement | 21:48 |
| *** pc_m has quit IRC | 21:48 | |
| jhesketh | Morning | 21:50 |
| pleia2 | morning jhesketh | 21:50 |
| jeblair | pleia2: ++! | 21:50 |
| *** prad has quit IRC | 21:53 | |
| *** ivar-laz_ has joined #openstack-infra | 21:54 | |
| clarkb | jhesketh: good morning, will have a couple enable_swift changes for review shortly | 21:55 |
| *** Somay has joined #openstack-infra | 21:55 | |
| clarkb | jeblair: the first one is up, working on a second to os_loganalyze (writing tests is taking more time than I hoped) | 21:55 |
| clarkb | er jhesketh ^ | 21:55 |
| *** Somay has quit IRC | 21:56 | |
| SpamapS | hm | 21:56 |
| jhesketh | clarkb: okay cool, will take a look | 21:56 |
| SpamapS | has anybody ever played with the CLI tool that comes with gearmand (just called gearman) and geard? | 21:56 |
| SpamapS | worker mode doesn't seem to work | 21:56 |
| *** ivar-lazzaro has quit IRC | 21:57 | |
| marun | jogo: help to find pass/fail rate for a job? | 21:57 |
| *** Somay has joined #openstack-infra | 21:57 | |
| *** yamamoto has quit IRC | 21:58 | |
| marun | jogo: ah, got it | 21:58 |
| *** cdent has quit IRC | 21:58 | |
| *** rlandy has quit IRC | 21:58 | |
| *** adalbas has quit IRC | 21:59 | |
| *** aysyd has quit IRC | 21:59 | |
| anteaya | SpamapS: I had a question for you in backscroll | 22:00 |
| *** soren has quit IRC | 22:00 | |
| *** timcline has quit IRC | 22:00 | |
| anteaya | SpamapS: did you see it? | 22:00 |
| anteaya | SpamapS: it was about your patch 158148 | 22:00 |
| mtreinish | SpamapS: I've only ever used the server, seemed to work ok for my local testing | 22:01 |
| jeblair | SpamapS: i have not | 22:01 |
| mtreinish | or was that a different package, it's been several months | 22:01 |
| *** yfried_ has quit IRC | 22:02 | |
| *** dimtruck is now known as zz_dimtruck | 22:03 | |
| *** tkelsey has quit IRC | 22:03 | |
| *** aysyd has joined #openstack-infra | 22:03 | |
| clarkb | I think os_loganalyze tests have melted my brain | 22:07 |
| openstackgerrit | Merged openstack-infra/puppet-nodepool: Parameterise logging config template https://review.openstack.org/164031 | 22:10 |
| *** ddieterly has joined #openstack-infra | 22:10 | |
| anteaya | clarkb: no no I need your brain | 22:10 |
| anteaya | clarkb: I'm reviewing https://review.openstack.org/#/c/164866/1/jenkins/scripts/zuul_swift_upload.py | 22:10 |
| openstackgerrit | Maru Newby proposed openstack-infra/project-config: Configure neutron to gate on neutron-dsvm-api job https://review.openstack.org/164886 | 22:11 |
| *** aysyd has quit IRC | 22:11 | |
| anteaya | clarkb: under what conditions would your new if file_path be false? | 22:11 |
| *** harlowja_away is now known as harlowja | 22:11 | |
| clarkb | anteaya: when make_index_file short circuits because an index file already exists | 22:11 |
| * anteaya looks again | 22:11 | |
| SpamapS | jeblair: it doesn't seem to submit a client id, and that perturbs geard | 22:12 |
| anteaya | I looked at it and looked at it and wasn't seeing it, thanks clarkb | 22:12 |
| *** gordc has quit IRC | 22:12 | |
| *** stevemar has quit IRC | 22:13 | |
| SpamapS | anteaya: re https://review.openstack.org/#/c/158148/3 , I believe python3 in a gate is pretty straight forward .. it works, or it doesn't. So I don't really think it needs time to stabilize. | 22:13 |
| anteaya | SpamapS: you are aware that adding a new untested job to a repo's gate blocks the repo until the job is fixed? | 22:14 |
| anteaya | if it is a broken job? | 22:14 |
| jeblair | SpamapS: that sounds like a potential geard bug | 22:15 |
| *** amitgandhinz has quit IRC | 22:16 | |
| *** tsg has quit IRC | 22:17 | |
| *** soren has joined #openstack-infra | 22:17 | |
| *** mattfarina has joined #openstack-infra | 22:18 | |
| *** mpopow has quit IRC | 22:18 | |
| *** mattfarina has quit IRC | 22:18 | |
| *** sputnik13 has quit IRC | 22:21 | |
| *** carl_baldwin has quit IRC | 22:22 | |
| *** bknudson has quit IRC | 22:22 | |
| *** Somay has quit IRC | 22:23 | |
| *** ivar-laz_ has quit IRC | 22:25 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/system-config: Log server can run on Apache 2.4 / Ubuntu Trusty https://review.openstack.org/164889 | 22:25 |
| greghaynes | clarkb: so interesting thing about pypi-mirror and using it for making a wheel only mirror - a recent feature moves the non arch specific wheels into the main mirror dir, so theres not an easy way to separate them out from the rest of the packages | 22:26 |
| *** ivar-lazzaro has joined #openstack-infra | 22:26 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/os-loganalyze: Update regex for better HTML detection. https://review.openstack.org/164891 | 22:27 |
| clarkb | jhesketh: sdague ^ that will make coverage logs render properly when served via swift | 22:27 |
| clarkb | greghaynes: I don't think thats a problem | 22:28 |
| clarkb | greghaynes: we will just have a few duplicate files sitting around but this way everything gets a wheel in a simple way | 22:28 |
| greghaynes | yep, as long as were fine with the duplication then that works | 22:28 |
| *** dboik_ has joined #openstack-infra | 22:30 | |
| *** ddieterly has quit IRC | 22:30 | |
| *** MarkAtwood_ has quit IRC | 22:31 | |
| jhesketh | clarkb: lgtm | 22:32 |
| clarkb | jhesketh: http://logs.openstack.org/59/164859/1/check/nodepool-coverage/bf38a9f/cover/nodepool_allocation.html is an occurence of the issue | 22:32 |
| clarkb | jhesketh: the doctype first line seems to be what broke it | 22:32 |
| *** dboik has quit IRC | 22:33 | |
| clarkb | jhesketh: https://review.openstack.org/#/c/164866/ is the other related change. coverage generates an index.html for us but we override it | 22:33 |
| *** nelsnelson has quit IRC | 22:34 | |
| clarkb | jhesketh: ^ probably needs more scrutiny simply because swift and apache are both involved | 22:34 |
| *** dboik_ has quit IRC | 22:34 | |
| openstackgerrit | Doug Wiegley proposed openstack-infra/devstack-gate: Mechanism to allow easier fine-tuning of devstack-gate services https://review.openstack.org/164874 | 22:34 |
| SpamapS | jeblair: indeed, I'm investigating now | 22:35 |
| *** garyh has joined #openstack-infra | 22:35 | |
| *** otter768 has joined #openstack-infra | 22:35 | |
| openstackgerrit | Merged openstack-infra/project-config: Use job templates for gerritlib builds https://review.openstack.org/156068 | 22:35 |
| *** peristeri has quit IRC | 22:36 | |
| SpamapS | gearman -c 1 -h localhost -w -f something should be equivalent of w = gear.Worker(client_id='foo'); w.addServer('localhost');w.registerFunction('something'); print(w.getJob()) | 22:36 |
| jhesketh | clarkb: I just approved 164866, but if you're concerned about it needing more eyeballs we have until the images rebuild to have others look at it and push up other chnages | 22:36 |
| clarkb | jhesketh: ya mostly just wondering if the relative paths in an index file that doesn't know about swift will work | 22:37 |
| clarkb | jhesketh: I think they will because your browser should do the correct thing | 22:37 |
| SpamapS | but the former does gets a NO_JOB while the latter gets the job | 22:37 |
| clarkb | jhesketh: so it should all just work | 22:37 |
| *** salv-orl_ has joined #openstack-infra | 22:37 | |
| jhesketh | clarkb: hmm, interesting question... I think it should too | 22:38 |
| *** nelsnelson has joined #openstack-infra | 22:38 | |
| *** salv-orlando has quit IRC | 22:39 | |
| openstackgerrit | Merged openstack-infra/project-config: Stop running useless tests on horizon https://review.openstack.org/164344 | 22:39 |
| openstackgerrit | Merged openstack-infra/project-config: Remove jobs for Gerrit 2.4.x https://review.openstack.org/164821 | 22:39 |
| *** otter768 has quit IRC | 22:40 | |
| *** dims has quit IRC | 22:43 | |
| fungi | pleia2: leprechauns are also welcome to attend | 22:43 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:43 | |
| *** nelsnels_ has joined #openstack-infra | 22:44 | |
| pleia2 | leprechauns++ | 22:44 |
| pleia2 | clarkb: can you take a quick look at https://review.openstack.org/#/c/164311/ ? | 22:45 |
| * fungi also has several jokes about nasty green beer | 22:45 | |
| *** nelsnelson has quit IRC | 22:45 | |
| openstackgerrit | Merged openstack-infra/project-config: Do not generate index.html if one exists https://review.openstack.org/164866 | 22:45 |
| pleia2 | beer that can by dyed green typically aren't worth my time :) | 22:46 |
| clarkb | pleia2: ya let me test in a container | 22:46 |
| pleia2 | clarkb: I think we fixed this with crinkle and co that day we were building images, and I tested it in a 14.04 vm just now | 22:46 |
| *** Somay has joined #openstack-infra | 22:46 | |
| *** dims has joined #openstack-infra | 22:46 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Add Fedora 21 dib image https://review.openstack.org/163982 | 22:47 |
| *** hrubi has quit IRC | 22:47 | |
| *** dims has quit IRC | 22:47 | |
| *** dims has joined #openstack-infra | 22:47 | |
| clarkb | pleia2: you did `gem install rspec` or ran the command with a version? | 22:47 |
| pleia2 | clarkb: no version | 22:48 |
| fungi | infra o' team | 22:48 |
| anteaya | mtreinish: I'm looking at https://review.openstack.org/#/c/164730/1 | 22:48 |
| *** garyh has quit IRC | 22:49 | |
| anteaya | mtreinish: we had a chat in the third party meeting this morning, the original patch upset their systems, and the cap fixed them | 22:49 |
| anteaya | mtreinish: any thoughts on how to proceed keeping in mind this might affect them? | 22:49 |
| ianw | jeblair: do you want me to just merge this down to the original change? -> https://review.openstack.org/#/c/164859/ | 22:50 |
| anteaya | mtreinish: for reference: 15:20:29 <rhe00> are there any infra changes planned for this week? http://eavesdrop.openstack.org/meetings/third_party/2015/third_party.2015-03-16-15.03.log.html | 22:50 |
| pleia2 | anteaya: so to move forward with the latest version of subunit2sql they will need to do the schema update at some point, until then they can edit their instances to cap the version themselves (as rhe00 did on Friday) | 22:51 |
| *** shashankhegde has quit IRC | 22:51 | |
| jeblair | ianw: if you want; i didn't want to lose the reviews we already had. you can squash it if you want, or we could just merge it right afterwords. | 22:52 |
| *** Slower has quit IRC | 22:53 | |
| anteaya | pleia2: okay, care to help me draft an email (if nothing else to give us something to point to) | 22:53 |
| anteaya | or should we just tell them when the roll in | 22:53 |
| jeblair | anteaya: well, i mean, i expect we will make hundreds of infra changes this week | 22:53 |
| anteaya | jeblair: I know, that is what I told them | 22:53 |
| anteaya | this was just the start of the conversation | 22:53 |
| jeblair | anteaya: i also would expect the version problem to have broken us too | 22:53 |
| anteaya | okay | 22:54 |
| anteaya | am I an idiot for asking? | 22:54 |
| pleia2 | yeah, it would have if I didn't push through the cap | 22:54 |
| jeblair | anteaya: i don't think so | 22:54 |
| openstackgerrit | Flavio Fernandes proposed openstack-infra/devstack-gate: Translate semi-colon into newline when parsing $DEVSTACK_LOCAL_CONFIG https://review.openstack.org/164895 | 22:54 |
| anteaya | jeblair: okay thanks | 22:54 |
| *** Slower has joined #openstack-infra | 22:54 | |
| anteaya | pleia2: thanks for the cap | 22:54 |
| pleia2 | anteaya: instead of a specific email, perhaps we can get mtreinish to email release notes for the new version of subunit2sql noting the schema change? then point 3rd party there | 22:54 |
| anteaya | oh I like that | 22:55 |
| anteaya | mtreinish: can we sell you on some release notes? | 22:55 |
| *** sdake has quit IRC | 22:55 | |
| ianw | jeblair: well you have the keys to the castle :) i will go with whatever you think is faster | 22:55 |
| *** shashankhegde has joined #openstack-infra | 22:56 | |
| clarkb | pleia2: anteaya confirmed `gem install puppetlabs_spec_helper` works now | 22:56 |
| clarkb | pleia2: so the removal of the workaround should be fine | 22:56 |
| pleia2 | clarkb: care to +A? | 22:56 |
| clarkb | pleia2: ya I can get it | 22:56 |
| pleia2 | clarkb: and thanks, yay! | 22:57 |
| anteaya | clarkb: yay | 22:57 |
| anteaya | that only took what, a couple of months? | 22:57 |
| anteaya | pleia2: let me know what you think of my comment: https://review.openstack.org/#/c/164730/ | 22:57 |
| pleia2 | anteaya: thumbs up | 22:58 |
| anteaya | thanks | 22:58 |
| *** yamamoto has joined #openstack-infra | 22:58 | |
| anteaya | I appreciate the help getting the larger picture | 22:58 |
| pleia2 | sure thing | 22:59 |
| Somay | Can anyone review my patch related to jenkins job builder here? I am a new contributor and would be really happy to get my first patch merged :) https://review.openstack.org/#/c/163137/ | 23:00 |
| jeblair | anteaya: are third-party folks running their own subunit2sql databases or pulling data from ours? | 23:01 |
| *** amotoki has quit IRC | 23:01 | |
| Somay | (sorry to interrupt) | 23:01 |
| *** Sukhdev has quit IRC | 23:01 | |
| clarkb | jeblair: I think pulling from ours | 23:01 |
| anteaya | jeblair: I don't really know as I didn't ask that question | 23:01 |
| jeblair | clarkb: so i think that means that they were broken because _our_ db did not have the schema run, not because they were using their own | 23:01 |
| jeblair | anteaya: ^ | 23:02 |
| clarkb | jeblair: correct | 23:02 |
| anteaya | rhe00 was using asselin's scripts and the change broke his scripts | 23:02 |
| jeblair | schema change | 23:02 |
| anteaya | jeblair: ah yes, I would go with that assumption | 23:02 |
| jeblair | so no action is needed on the part of third-party folks doing that | 23:02 |
| anteaya | oh good | 23:02 |
| anteaya | I felt it my duty to at least ask | 23:02 |
| jeblair | if anyone is running their own subunit2sql database, then they would need to run a schema change manually | 23:02 |
| anteaya | since they were a bit upset today | 23:03 |
| anteaya | okay | 23:03 |
| jeblair | but if they are doing that, they are probably not using a copy of our prep scripts | 23:03 |
| clarkb | anteaya: they are welcome to build images without using our database | 23:03 |
| anteaya | awesome | 23:03 |
| clarkb | anteaya: not sure that this is something to get upset over | 23:03 |
| anteaya | clarkb: what they know how to do and what they cargo cult | 23:03 |
| anteaya | the problem is I don't know myself | 23:03 |
| pleia2 | unfortunately I think many use the scripts without a firm understanding of what's going on | 23:03 |
| *** patrickeast_ has joined #openstack-infra | 23:03 | |
| anteaya | since I've been holding hands and offering tissues for the last 18 months | 23:04 |
| anteaya | I would dearly like ot know myself | 23:04 |
| *** patrickeast has quit IRC | 23:04 | |
| *** patrickeast_ is now known as patrickeast | 23:04 | |
| anteaya | but thanks for telling me, I appreciate it | 23:04 |
| anteaya | and will try to work it into upcoming questions about it | 23:04 |
| openstackgerrit | Merged openstack-infra/project-config: Remove unneeded workaround https://review.openstack.org/164311 | 23:04 |
| anteaya | pleia2: oh any understanding at all | 23:05 |
| anteaya | firm is a pipedream | 23:05 |
| pleia2 | anteaya: I was trying to be diplomatic :) | 23:05 |
| anteaya | ha ha ha | 23:05 |
| anteaya | go you | 23:05 |
| anteaya | what was that line you used? | 23:05 |
| anteaya | my delicate is all used up | 23:05 |
| anteaya | that is the best line ever | 23:06 |
| pleia2 | sounds like me | 23:06 |
| pleia2 | hehe | 23:06 |
| anteaya | that was you | 23:06 |
| anteaya | I loved it | 23:06 |
| anteaya | still do | 23:06 |
| *** claudiub has joined #openstack-infra | 23:06 | |
| anteaya | I need to go for a walk | 23:07 |
| pleia2 | enjoy | 23:07 |
| anteaya | thanks | 23:07 |
| mtreinish | pleia2, anteaya: I have no idea why people running 3rd party ci systems would be using the subunit2sql stuff against infra's db to be honest (or at all really) | 23:07 |
| mtreinish | it'd be great if they were running there own db, I'd just be surprised | 23:08 |
| *** mwagner_lap has joined #openstack-infra | 23:08 | |
| clarkb | because they use our image build scripts | 23:08 |
| *** ChuckC has joined #openstack-infra | 23:09 | |
| *** lnr has joined #openstack-infra | 23:10 | |
| mtreinish | clarkb: yeah I got that, it's just surprising to me that they wouldn't look at what they're running and realize it had no real benefit for them | 23:10 |
| * mtreinish should probably read the backscroll before talking | 23:10 | |
| clarkb | Somay: done | 23:11 |
| *** thedodd has quit IRC | 23:14 | |
| *** BlackMustard has joined #openstack-infra | 23:15 | |
| *** lnr has left #openstack-infra | 23:15 | |
| *** gyee has quit IRC | 23:16 | |
| *** dims has quit IRC | 23:18 | |
| *** dims has joined #openstack-infra | 23:18 | |
| *** r-daneel has quit IRC | 23:19 | |
| *** andreykurilin_ has joined #openstack-infra | 23:20 | |
| *** salv-orl_ has quit IRC | 23:23 | |
| Somay | clarkb: Thanks! Those are the default values that the plugin specifies | 23:24 |
| *** sdake has joined #openstack-infra | 23:24 | |
| *** wenlock has quit IRC | 23:24 | |
| clarkb | Somay: ok, then we just need to raise an error when they occur | 23:24 |
| *** baoli has joined #openstack-infra | 23:24 | |
| Somay | sure, i’ll make that change | 23:25 |
| fungi | ianw is suggesting bindep should have some better error reporting if run on systems lacking an lsb_release executable (see inline comments at https://review.openstack.org/157979 for details). trying to decide if i should detect and raise a special exception for that condition or take another route, and also what the error should recomment to the end user given that the package providing it will have | 23:25 |
| fungi | different names on different platforms | 23:25 |
| *** ZZelle has quit IRC | 23:25 | |
| clarkb | fungi: doesn't bindep already map package names? | 23:25 |
| clarkb | fungi: should be easy enough to use bindep recursively to write the error message? | 23:26 |
| fungi | it does, but... it uses lsb_release to decide what platform it's being run on. so chicken and egg | 23:26 |
| clarkb | oh right | 23:26 |
| clarkb | fungi: I think you just say missing lsb_release executable and let others figure it out? | 23:26 |
| fungi | i could just punt and say "figure it out" yeah | 23:26 |
| *** amitgandhinz has joined #openstack-infra | 23:27 | |
| fungi | as for the bigger question, there's only one method calling lsb_release but it's entered from a couple places in main. and in a module which doesn't currently handle any ui feedback. i'm guessing raising a custom exception is the best way to signal that problem? | 23:27 |
| *** mpopow has joined #openstack-infra | 23:28 | |
| ianw | fungi: yeah, i proposed devstack do a similar thing, except that does have a go at installing for you because devstack takes over everything -> https://review.openstack.org/#/c/117104/ | 23:28 |
| SpamapS | jeblair: I can confirm that something in geard doesn't work with the C based cli gearman command. I wonder if we can add an optional bit to the functional tests if we have 'gearman' in our PATH | 23:28 |
| *** soren has quit IRC | 23:29 | |
| *** BlackMustard has quit IRC | 23:29 | |
| ianw | fungi: that probably matches the behaviour of setuptools etc which is usually "here's a backtrace..." | 23:29 |
| fungi | ianw: well, it does that already | 23:29 |
| fungi | it will try to exec lsb_release, fail to find it, and raise an exception telling you that | 23:30 |
| fungi | because python | 23:30 |
| *** amitgandhinz has quit IRC | 23:31 | |
| *** salv-orlando has joined #openstack-infra | 23:31 | |
| ianw | fungi: yeah, i just mean if the exception has in it something more obvious than a failed exec that's about right, even if it doesn't give a nice formatted output | 23:31 |
| *** ddieterly has joined #openstack-infra | 23:32 | |
| fungi | okay, i'll see what i can do to catch the oserror and reraise it along with a custom exception. i just hate masking exceptions with other exceptions | 23:33 |
| ianw | the exec failure is obvious to python developers who see it and then correlate it with the code, but a bit generically unfriendly ... and having tryed to use lsb_release generically i do know that it is a common issue that it isn't there yet, so people will hit it | 23:33 |
| fungi | may make it easier on less clue-endowed end users but it angers developers who try to actually troubleshoot the exception | 23:34 |
| fungi | i'll find a happy medium | 23:34 |
| fungi | maybe i'll do what i was hesitant to do initially and add a lbyl conditional | 23:35 |
| *** salv-orlando has quit IRC | 23:35 | |
| ianw | the line between user and developer in openstack is pretty thin, so there's that :) | 23:36 |
| *** BlackMustard has joined #openstack-infra | 23:36 | |
| fungi | ianw: worth noting that the primary use case for bindep is "i want to run `tox -e py27` on nova but don't know what distro packages i need to support the python dependencies for it" | 23:37 |
| *** Qiming has joined #openstack-infra | 23:37 | |
| fungi | (for example, libxml/xslt headers to be able to compile c extensions for teh lxml module) | 23:39 |
| *** shashankhegde has quit IRC | 23:40 | |
| *** ajmiller_ has joined #openstack-infra | 23:40 | |
| fungi | so, yeah, currently fairly developer-centric, though i wouldn't ignore the possibility it could find use for less dev-oriented tasks too | 23:40 |
| fungi | which is why i'm entertaining slightly more user-friendly error handling | 23:40 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Added null check for story creator_id. https://review.openstack.org/164877 | 23:41 |
| ianw | right, it's only logical for devstack to use it rather than it's hand-curated list of the same | 23:41 |
| *** jaypipes has quit IRC | 23:41 | |
| *** r-daneel has joined #openstack-infra | 23:42 | |
| *** ajmiller has quit IRC | 23:44 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 23:44 | |
| openstackgerrit | Somay Jain proposed openstack-infra/jenkins-job-builder: Adding more configurable options in Notifications plugin https://review.openstack.org/163137 | 23:45 |
| ianw | it still seems a shame that distro packagers have figured this stuff out, but their info is really locked in their packages. providing a collaboration point is a positive | 23:45 |
| *** sdake__ has joined #openstack-infra | 23:45 | |
| openstackgerrit | Maru Newby proposed openstack-infra/project-config: Configure neutron to gate on neutron-dsvm-api job https://review.openstack.org/164886 | 23:45 |
| *** Somay has quit IRC | 23:46 | |
| *** tsg has joined #openstack-infra | 23:46 | |
| *** r-daneel has quit IRC | 23:47 | |
| SpamapS | ERROR:gear.Server:Received unknown packet: <gear.Packet 0x7fb6decf9f50 type: UNKNOWN> | 23:47 |
| clarkb | jeblair: mtreinish so did we manage to correct the database for subunit2sql? | 23:47 |
| *** Somay has joined #openstack-infra | 23:47 | |
| *** andreykurilin_ has quit IRC | 23:48 | |
| StevenK | SpamapS: Helpful. | 23:48 |
| fungi | well, to some extent i get the feeling some significant percentage of distro contributors resent/avoid lsb | 23:48 |
| *** dboik has joined #openstack-infra | 23:48 | |
| fungi | ianw: ^ | 23:48 |
| clarkb | fungi: unfortunately as a distro user its the only sane tool that does that job | 23:48 |
| *** gokrokve has quit IRC | 23:49 | |
| fungi | hence their lack of priority around having lsb tools installed by default | 23:49 |
| *** sdake has quit IRC | 23:49 | |
| fungi | clarkb: as a _cross-distro_ user yes. as a single-distro user much less so | 23:49 |
| StevenK | /etc/debian_version or /etc/redhat-release, but they aren't an answer | 23:49 |
| *** dboik_ has joined #openstack-infra | 23:49 | |
| StevenK | I did like lsb_release working on Ubuntu | 23:49 |
| SpamapS | IIRC, LSB in its original incarnation included some really bad redhatisms and was resented by all.. but has since been picked apart into useful cross-distro tools. | 23:50 |
| openstackgerrit | Michael Johnson proposed openstack-infra/git-review: Choose correct branch for rebase when submitting https://review.openstack.org/158877 | 23:50 |
| fungi | agreed. but the resentment still lingers i think | 23:50 |
| greghaynes | clarkb: so for this pypi-mirror periodic job, do I put it under the openstack/pypi-mirror job? | 23:51 |
| ianw | there is redhat-lsb-core package now which installs a reasonable amount, but the original redhat-lsb package brings in the kitchen sink | 23:51 |
| *** ashleighfarnham has quit IRC | 23:51 | |
| *** sputnik13 has joined #openstack-infra | 23:51 | |
| greghaynes | Its wierd since it doesnt need to actually be tied to a repo of any kind | 23:51 |
| SpamapS | ianw: yeah, because the original LSB definition included X | 23:51 |
| clarkb | greghaynes: I wouldn't, I would create a meta project instead | 23:51 |
| fungi | redhat-lsb-core is btw the package which provides the lsb_release executable on rh derivatives (as opposed to the lsb-release package on debian derivatives) | 23:51 |
| greghaynes | clarkb: Is there an example of that I can grep for? | 23:51 |
| *** dannywilson has quit IRC | 23:51 | |
| clarkb | greghaynes: let me check | 23:51 |
| *** ajmiller_ has quit IRC | 23:52 | |
| *** tsg has quit IRC | 23:52 | |
| clarkb | greghaynes: there was at one time but I don't see it anymore. Actuall I would put it under requirements | 23:53 |
| clarkb | greghaynes: since we want to job to fire on requirements changes | 23:53 |
| *** dboik has quit IRC | 23:53 | |
| greghaynes | ah. Yep | 23:53 |
| greghaynes | although I think we also want it to fire under pypi-mirror changes too | 23:53 |
| greghaynes | since theres *tons* of those (not really) | 23:54 |
| * fungi senses jobs he deleted getting re-added | 23:54 | |
| greghaynes | fungi: :) | 23:54 |
| clarkb | fungi: yes, it turns out that we need wheels | 23:54 |
| *** Somay has quit IRC | 23:54 | |
| mkj_wk | fungi: that is definitely an improvement | 23:54 |
| *** Qiming has quit IRC | 23:54 | |
| mkj_wk | fungi: querying Gerry for a branch at submit time had become useless :) | 23:54 |
| clarkb | greghaynes: only if you have a similar job that just builds a mirror but doesn't copy any fiels | 23:55 |
| fungi | clarkb: heh. so now it will be used to generate a list of packages which need wheels built and cached? | 23:55 |
| mkj_wk | fungi: this change also removed the need (at least for now) for my sleep hack | 23:55 |
| *** bknudson has joined #openstack-infra | 23:55 | |
| mkj_wk | so it's a much cleaner patch now | 23:55 |
| clarkb | fungi: it will build the wheels too | 23:55 |
| clarkb | fungi: it always did that, we just lost the feature when we went to bandersnatch | 23:55 |
| fungi | mkj_wk: great news | 23:55 |
| fungi | clarkb: build wheels for us to host from our mirror, or pre-caching them on nodepool images? | 23:56 |
| clarkb | fungi: both, we will host them on our mirror then nodepool images will use that to cache the wheels | 23:56 |
| greghaynes | clarkb: Why? I mean as a post job incase we find a bug in pypi-mirror where it, for example, fails on building certain wheels | 23:56 |
| * mkj_wk crosses his fingers that this is not a "free puppy" now :) | 23:56 | |
| clarkb | greghaynes: I think we would just trigger the job to run in that case | 23:56 |
| clarkb | greghaynes: rather than have it run every time | 23:56 |
| clarkb | fungi: the problem with doing wheel builds on images during build time is you hve to install all those packages that ou are trying to install at run time | 23:57 |
| greghaynes | ah, fair enough | 23:57 |
| *** camunoz has joined #openstack-infra | 23:57 | |
| clarkb | fungi: defeats the purpose | 23:57 |
| *** alexpilotti has quit IRC | 23:57 | |
| fungi | clarkb: great pooint | 23:58 |
| greghaynes | Yes, although as I keep liking to point out this doesnt exclude that, we can just pull them into the image at build time and not have the build deps issue | 23:58 |
| fungi | point too | 23:58 |
| greghaynes | them being the wheels | 23:58 |
| clarkb | greghaynes: yup | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!