| *** baoli has quit IRC | 00:02 | |
| *** Hal has joined #openstack-infra | 00:04 | |
| *** mylu has quit IRC | 00:04 | |
| *** Hal is now known as Guest7687 | 00:04 | |
| *** heyongli_ has quit IRC | 00:05 | |
| *** heyongli_ has joined #openstack-infra | 00:05 | |
| *** baoli has joined #openstack-infra | 00:07 | |
| *** mylu has joined #openstack-infra | 00:07 | |
| *** mylu has quit IRC | 00:10 | |
| *** mylu has joined #openstack-infra | 00:11 | |
| *** zeih has joined #openstack-infra | 00:12 | |
| *** salv-orlando has joined #openstack-infra | 00:12 | |
| *** mylu has quit IRC | 00:13 | |
| *** mylu has joined #openstack-infra | 00:13 | |
| *** heyongli_ has quit IRC | 00:15 | |
| *** heyongli_ has joined #openstack-infra | 00:15 | |
| *** sdake has quit IRC | 00:15 | |
| *** baoli has quit IRC | 00:16 | |
| *** zeih has quit IRC | 00:17 | |
| *** baoli has joined #openstack-infra | 00:17 | |
| *** salv-orlando has quit IRC | 00:17 | |
| *** apoorvad has joined #openstack-infra | 00:18 | |
| *** Guest7687 has quit IRC | 00:19 | |
| *** MarkAtwood has quit IRC | 00:20 | |
| *** keedya_ has quit IRC | 00:21 | |
| *** heyongli_ has quit IRC | 00:25 | |
| *** heyongli_ has joined #openstack-infra | 00:25 | |
| *** sdake has joined #openstack-infra | 00:32 | |
| openstackgerrit | Merged openstack-infra/project-config: python-troveclient didn't have release notes till mitaka https://review.openstack.org/311517 | 00:34 |
|---|---|---|
| *** heyongli_ has quit IRC | 00:35 | |
| *** heyongli_ has joined #openstack-infra | 00:36 | |
| *** sdake_ has joined #openstack-infra | 00:42 | |
| *** kengo_sakai has joined #openstack-infra | 00:42 | |
| *** sdake has quit IRC | 00:44 | |
| *** heyongli_ has quit IRC | 00:46 | |
| *** heyongli_ has joined #openstack-infra | 00:46 | |
| *** kengo_sakai has quit IRC | 00:46 | |
| *** markvoelker has joined #openstack-infra | 00:51 | |
| *** markvoelker has quit IRC | 00:56 | |
| *** heyongli_ has quit IRC | 00:56 | |
| *** heyongli_ has joined #openstack-infra | 00:56 | |
| *** Hal has joined #openstack-infra | 00:57 | |
| *** Hal is now known as Guest81755 | 00:58 | |
| *** baoli has quit IRC | 01:02 | |
| openstackgerrit | Steve Martinelli proposed openstack/requirements: Update keystoneclient lower bound https://review.openstack.org/299593 | 01:02 |
| *** asselin__ has quit IRC | 01:06 | |
| *** heyongli_ has quit IRC | 01:06 | |
| *** zz_dimtruck is now known as dimtruck | 01:07 | |
| *** heyongli_ has joined #openstack-infra | 01:09 | |
| *** salv-orlando has joined #openstack-infra | 01:13 | |
| *** heyongli_ has quit IRC | 01:17 | |
| *** heyongli_ has joined #openstack-infra | 01:17 | |
| *** salv-orlando has quit IRC | 01:18 | |
| *** EricGonczer_ has quit IRC | 01:19 | |
| *** kengo_sakai has joined #openstack-infra | 01:24 | |
| *** heyongli_ has quit IRC | 01:27 | |
| *** heyongli_ has joined #openstack-infra | 01:27 | |
| *** jefrite has joined #openstack-infra | 01:30 | |
| *** ekarlso has quit IRC | 01:33 | |
| *** kengo_sakai has quit IRC | 01:37 | |
| *** heyongli_ has quit IRC | 01:37 | |
| *** heyongli_ has joined #openstack-infra | 01:37 | |
| *** kengo_sakai has joined #openstack-infra | 01:37 | |
| timrc | fungi: Ah yard work. I know this thing all too well. What's sad is that you'd never be able to tell how much effort I've put into my yard. | 01:38 |
| *** fedexo has joined #openstack-infra | 01:43 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add install-bin element https://review.openstack.org/311646 | 01:46 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add tinycorelinux element https://review.openstack.org/311647 | 01:46 |
| *** xarses has quit IRC | 01:46 | |
| *** xarses_ has joined #openstack-infra | 01:48 | |
| *** heyongli_ has quit IRC | 01:48 | |
| *** heyongli_ has joined #openstack-infra | 01:48 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add tinycorelinux element https://review.openstack.org/311647 | 01:49 |
| *** zqfan has joined #openstack-infra | 01:51 | |
| *** markvoelker has joined #openstack-infra | 01:52 | |
| *** ekarlso has joined #openstack-infra | 01:53 | |
| *** thorst has quit IRC | 01:54 | |
| *** thorst has joined #openstack-infra | 01:55 | |
| *** markvoelker has quit IRC | 01:57 | |
| *** heyongli_ has quit IRC | 01:57 | |
| *** heyongli_ has joined #openstack-infra | 01:58 | |
| *** mylu has quit IRC | 01:59 | |
| *** mylu has joined #openstack-infra | 02:02 | |
| *** thorst has quit IRC | 02:03 | |
| *** mylu has quit IRC | 02:03 | |
| *** mylu has joined #openstack-infra | 02:04 | |
| *** heyongli_ has quit IRC | 02:08 | |
| *** heyongli_ has joined #openstack-infra | 02:10 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add tinycorelinux element https://review.openstack.org/311647 | 02:11 |
| *** zeih has joined #openstack-infra | 02:13 | |
| morgan | persia: lol | 02:13 |
| *** hdd has joined #openstack-infra | 02:13 | |
| *** salv-orlando has joined #openstack-infra | 02:14 | |
| *** morgan is now known as notmorgan | 02:17 | |
| *** zeih has quit IRC | 02:17 | |
| *** heyongli_ has quit IRC | 02:18 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add tinycorelinux element https://review.openstack.org/311647 | 02:18 |
| *** salv-orlando has quit IRC | 02:19 | |
| *** kengo_sa_ has joined #openstack-infra | 02:22 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Split pip out of pip-and-virtualenv https://review.openstack.org/311651 | 02:23 |
| *** heyongli_ has joined #openstack-infra | 02:23 | |
| *** kengo_sakai has quit IRC | 02:25 | |
| *** mylu has quit IRC | 02:25 | |
| *** denisra has quit IRC | 02:27 | |
| *** denisra has joined #openstack-infra | 02:28 | |
| *** heyongli_ has quit IRC | 02:28 | |
| *** heyongli_ has joined #openstack-infra | 02:28 | |
| *** heyongli_ has quit IRC | 02:38 | |
| *** heyongli_ has joined #openstack-infra | 02:41 | |
| *** mylu has joined #openstack-infra | 02:44 | |
| *** kengo_sa_ has quit IRC | 02:47 | |
| *** heyongli_ has quit IRC | 02:49 | |
| *** heyongli_ has joined #openstack-infra | 02:49 | |
| *** markvoelker has joined #openstack-infra | 02:49 | |
| *** heyongli_ has quit IRC | 02:59 | |
| *** heyongli_ has joined #openstack-infra | 02:59 | |
| *** thorst has joined #openstack-infra | 03:01 | |
| *** dimtruck is now known as zz_dimtruck | 03:07 | |
| *** mriedem1 has quit IRC | 03:08 | |
| *** thorst has quit IRC | 03:08 | |
| *** heyongli_ has quit IRC | 03:09 | |
| *** heyongli_ has joined #openstack-infra | 03:10 | |
| *** apoorvad has quit IRC | 03:11 | |
| *** nadya has joined #openstack-infra | 03:12 | |
| *** baoli has joined #openstack-infra | 03:14 | |
| *** salv-orlando has joined #openstack-infra | 03:15 | |
| *** obondarev has quit IRC | 03:18 | |
| *** nadya has quit IRC | 03:19 | |
| *** baoli has quit IRC | 03:19 | |
| *** obondarev has joined #openstack-infra | 03:19 | |
| *** nadya has joined #openstack-infra | 03:19 | |
| *** heyongli_ has quit IRC | 03:19 | |
| *** salv-orlando has quit IRC | 03:19 | |
| *** heyongli_ has joined #openstack-infra | 03:20 | |
| *** yamamoto has quit IRC | 03:22 | |
| *** nadya has quit IRC | 03:25 | |
| *** mylu has quit IRC | 03:25 | |
| *** mylu has joined #openstack-infra | 03:26 | |
| *** raginbaji is now known as raginbajin | 03:28 | |
| *** Adri2000 has quit IRC | 03:28 | |
| *** heyongli_ has quit IRC | 03:30 | |
| *** heyongli_ has joined #openstack-infra | 03:30 | |
| *** flepied has quit IRC | 03:31 | |
| *** mylu has quit IRC | 03:39 | |
| *** heyongli_ has quit IRC | 03:40 | |
| *** mylu has joined #openstack-infra | 03:40 | |
| *** heyongli_ has joined #openstack-infra | 03:40 | |
| *** mylu has quit IRC | 03:42 | |
| *** Guest81755 has quit IRC | 03:42 | |
| *** Adri2000 has joined #openstack-infra | 03:43 | |
| *** jpeeler has joined #openstack-infra | 03:44 | |
| *** jpeeler has quit IRC | 03:44 | |
| *** jpeeler has joined #openstack-infra | 03:44 | |
| *** kengo_sakai has joined #openstack-infra | 03:48 | |
| *** heyongli_ has quit IRC | 03:50 | |
| *** heyongli_ has joined #openstack-infra | 03:50 | |
| *** mylu has joined #openstack-infra | 03:54 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove translations from operations-guide https://review.openstack.org/311660 | 03:55 |
| *** kengo_sakai has quit IRC | 03:56 | |
| *** mylu has quit IRC | 03:58 | |
| *** zz_dimtruck is now known as dimtruck | 04:00 | |
| *** heyongli_ has quit IRC | 04:00 | |
| *** mylu has joined #openstack-infra | 04:02 | |
| *** heyongli_ has joined #openstack-infra | 04:03 | |
| *** thorst has joined #openstack-infra | 04:06 | |
| *** links has joined #openstack-infra | 04:07 | |
| *** heyongli_ has quit IRC | 04:11 | |
| *** heyongli_ has joined #openstack-infra | 04:11 | |
| *** hdd has quit IRC | 04:11 | |
| *** apoorvad has joined #openstack-infra | 04:12 | |
| *** thorst has quit IRC | 04:13 | |
| *** yamamoto has joined #openstack-infra | 04:14 | |
| *** zeih has joined #openstack-infra | 04:14 | |
| *** salv-orlando has joined #openstack-infra | 04:16 | |
| *** mylu has quit IRC | 04:17 | |
| *** apoorvad has quit IRC | 04:17 | |
| *** zeih has quit IRC | 04:18 | |
| *** mylu has joined #openstack-infra | 04:19 | |
| *** salv-orlando has quit IRC | 04:20 | |
| *** mylu has quit IRC | 04:21 | |
| *** bnemec has joined #openstack-infra | 04:21 | |
| *** heyongli__ has joined #openstack-infra | 04:21 | |
| *** heyongli_ has quit IRC | 04:25 | |
| *** psachin has joined #openstack-infra | 04:27 | |
| *** heyongli__ has quit IRC | 04:31 | |
| *** heyongli__ has joined #openstack-infra | 04:31 | |
| *** mylu has joined #openstack-infra | 04:34 | |
| *** asettle has joined #openstack-infra | 04:35 | |
| *** asettle has quit IRC | 04:41 | |
| *** heyongli__ has quit IRC | 04:41 | |
| *** asettle has joined #openstack-infra | 04:41 | |
| *** heyongli__ has joined #openstack-infra | 04:42 | |
| *** mylu has quit IRC | 04:45 | |
| *** mylu has joined #openstack-infra | 04:45 | |
| *** mylu has quit IRC | 04:47 | |
| *** mylu has joined #openstack-infra | 04:48 | |
| *** markvoelker has quit IRC | 04:49 | |
| *** asettle has quit IRC | 04:51 | |
| *** asettle has joined #openstack-infra | 04:52 | |
| *** heyongli__ has quit IRC | 04:52 | |
| *** heyongli__ has joined #openstack-infra | 04:52 | |
| *** gildub has joined #openstack-infra | 04:54 | |
| *** asettle has quit IRC | 04:55 | |
| *** oanson has joined #openstack-infra | 05:01 | |
| *** heyongli__ has quit IRC | 05:02 | |
| *** heyongli__ has joined #openstack-infra | 05:02 | |
| *** rcernin has joined #openstack-infra | 05:05 | |
| *** thorst has joined #openstack-infra | 05:11 | |
| *** heyongli__ has quit IRC | 05:12 | |
| *** mylu has quit IRC | 05:12 | |
| *** heyongli__ has joined #openstack-infra | 05:12 | |
| *** Sukhdev has joined #openstack-infra | 05:14 | |
| *** salv-orlando has joined #openstack-infra | 05:16 | |
| *** thorst has quit IRC | 05:19 | |
| *** mylu has joined #openstack-infra | 05:19 | |
| *** abregman has joined #openstack-infra | 05:21 | |
| *** salv-orlando has quit IRC | 05:21 | |
| *** mylu has quit IRC | 05:22 | |
| *** heyongli__ has quit IRC | 05:22 | |
| *** heyongli__ has joined #openstack-infra | 05:23 | |
| *** mylu has joined #openstack-infra | 05:27 | |
| *** mylu has quit IRC | 05:32 | |
| *** yolanda has joined #openstack-infra | 05:32 | |
| *** heyongli__ has quit IRC | 05:33 | |
| *** heyongli__ has joined #openstack-infra | 05:33 | |
| *** apoorvad has joined #openstack-infra | 05:33 | |
| *** yolanda has quit IRC | 05:34 | |
| *** yolanda has joined #openstack-infra | 05:34 | |
| *** heyongli__ has quit IRC | 05:43 | |
| *** heyongli__ has joined #openstack-infra | 05:43 | |
| *** sdake_ has quit IRC | 05:47 | |
| *** markvoelker has joined #openstack-infra | 05:50 | |
| *** sdake has joined #openstack-infra | 05:52 | |
| *** kengo_sakai has joined #openstack-infra | 05:53 | |
| *** heyongli__ has quit IRC | 05:53 | |
| *** heyongli__ has joined #openstack-infra | 05:53 | |
| *** mylu has joined #openstack-infra | 05:54 | |
| *** markvoelker has quit IRC | 05:55 | |
| *** Sukhdev has quit IRC | 05:55 | |
| openstackgerrit | fumihiko kakuma proposed openstack-infra/devstack-gate: Add a way to edit local.conf https://review.openstack.org/311668 | 05:55 |
| *** kengo_sakai has quit IRC | 05:57 | |
| *** salv-orlando has joined #openstack-infra | 05:58 | |
| *** heyongli__ has quit IRC | 06:03 | |
| *** heyongli_ has joined #openstack-infra | 06:06 | |
| *** apoorvad has quit IRC | 06:08 | |
| *** zeih has joined #openstack-infra | 06:09 | |
| *** mylu has quit IRC | 06:11 | |
| *** zeih_ has joined #openstack-infra | 06:12 | |
| *** jtomasek has joined #openstack-infra | 06:12 | |
| *** zeih has quit IRC | 06:12 | |
| *** heyongli_ has quit IRC | 06:14 | |
| *** heyongli_ has joined #openstack-infra | 06:14 | |
| *** thorst has joined #openstack-infra | 06:16 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/309273 | 06:22 |
| *** thorst has quit IRC | 06:23 | |
| *** dimtruck is now known as zz_dimtruck | 06:24 | |
| openstackgerrit | Moshe Levi proposed openstack-infra/irc-meetings: Add Moshe Levi as sriov meeting chair https://review.openstack.org/311472 | 06:24 |
| *** heyongli_ has quit IRC | 06:24 | |
| *** heyongli_ has joined #openstack-infra | 06:24 | |
| *** ramishra has joined #openstack-infra | 06:26 | |
| *** roxanaghe has joined #openstack-infra | 06:28 | |
| *** roxanaghe has quit IRC | 06:29 | |
| openstackgerrit | Ben Nemec proposed openstack/diskimage-builder: Add documentation for dib-lint https://review.openstack.org/311671 | 06:29 |
| *** heyongli_ has quit IRC | 06:34 | |
| *** heyongli_ has joined #openstack-infra | 06:35 | |
| yolanda | good morning | 06:36 |
| AJaeger | morning, yolanda! | 06:42 |
| openstackgerrit | Merged openstack-infra/project-config: Retire kolla-mesos project https://review.openstack.org/311572 | 06:43 |
| yolanda | i still have not been able to solve the afs issue, seems it gets stuck when doing a release | 06:43 |
| yolanda | so all jobs consuming new pypi packages since 29 april will fail | 06:44 |
| yolanda | i will need help from other infra-root to debug the issue | 06:44 |
| *** heyongli_ has quit IRC | 06:44 | |
| *** heyongli_ has joined #openstack-infra | 06:45 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: openstack-manuals: Adjust jobs https://review.openstack.org/311675 | 06:50 |
| AJaeger | yolanda: hope everybody is refreshed again after travels today... | 06:50 |
| *** markvoelker has joined #openstack-infra | 06:51 | |
| AJaeger | yolanda: why not make a #status log to mention this, so that others know about it when reading https://wiki.openstack.org/wiki/Infrastructure_Status ? | 06:52 |
| yolanda | ++ | 06:52 |
| yolanda | #status log AFS mirrors not publishing, they get suck on vos release since 29th April | 06:53 |
| openstackstatus | yolanda: finished logging | 06:53 |
| *** heyongli_ has quit IRC | 06:55 | |
| *** markvoelker has quit IRC | 06:55 | |
| *** heyongli_ has joined #openstack-infra | 06:57 | |
| *** heyongli_ has quit IRC | 07:05 | |
| *** fedexo has quit IRC | 07:05 | |
| *** heyongli_ has joined #openstack-infra | 07:05 | |
| *** HeOS has joined #openstack-infra | 07:05 | |
| *** daemontool has joined #openstack-infra | 07:11 | |
| *** mylu has joined #openstack-infra | 07:12 | |
| *** yolanda has quit IRC | 07:12 | |
| *** ifarkas has joined #openstack-infra | 07:12 | |
| *** heyongli_ has quit IRC | 07:15 | |
| *** heyongli_ has joined #openstack-infra | 07:15 | |
| *** arxcruz has joined #openstack-infra | 07:16 | |
| *** mylu has quit IRC | 07:18 | |
| *** thorst has joined #openstack-infra | 07:21 | |
| *** nmagnezi has joined #openstack-infra | 07:21 | |
| *** daemontool has quit IRC | 07:21 | |
| *** daemontool has joined #openstack-infra | 07:21 | |
| *** jed56 has joined #openstack-infra | 07:23 | |
| *** heyongli_ has quit IRC | 07:25 | |
| *** heyongli_ has joined #openstack-infra | 07:26 | |
| *** acabot has joined #openstack-infra | 07:27 | |
| *** thorst has quit IRC | 07:28 | |
| *** salv-orlando has quit IRC | 07:28 | |
| *** roxanaghe has joined #openstack-infra | 07:29 | |
| *** vincentll has joined #openstack-infra | 07:32 | |
| *** daemontool has quit IRC | 07:33 | |
| *** tesseract has joined #openstack-infra | 07:33 | |
| *** tesseract is now known as Guest37590 | 07:34 | |
| *** abregman_ has joined #openstack-infra | 07:34 | |
| *** roxanaghe has quit IRC | 07:35 | |
| *** abregman has quit IRC | 07:35 | |
| *** heyongli_ has quit IRC | 07:36 | |
| *** heyongli_ has joined #openstack-infra | 07:36 | |
| *** yolanda has joined #openstack-infra | 07:36 | |
| *** daemontool has joined #openstack-infra | 07:37 | |
| *** heyongli_ has quit IRC | 07:46 | |
| *** heyongli_ has joined #openstack-infra | 07:46 | |
| *** gildub has quit IRC | 07:47 | |
| *** markvoelker has joined #openstack-infra | 07:51 | |
| *** abregman_ has quit IRC | 07:52 | |
| *** kengo_sakai has joined #openstack-infra | 07:54 | |
| *** markvoelker has quit IRC | 07:56 | |
| *** heyongli_ has quit IRC | 07:56 | |
| *** heyongli_ has joined #openstack-infra | 07:56 | |
| *** derekh has joined #openstack-infra | 07:57 | |
| *** hashar has joined #openstack-infra | 07:58 | |
| *** kengo_sakai has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** pblaho has joined #openstack-infra | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** yamahata has joined #openstack-infra | 08:01 | |
| *** abregman_ has joined #openstack-infra | 08:05 | |
| *** dmk0202 has joined #openstack-infra | 08:06 | |
| *** heyongli_ has quit IRC | 08:06 | |
| *** heyongli_ has joined #openstack-infra | 08:07 | |
| *** lezbar has joined #openstack-infra | 08:09 | |
| *** lezbar has quit IRC | 08:10 | |
| *** lezbar has joined #openstack-infra | 08:10 | |
| *** permalac has joined #openstack-infra | 08:12 | |
| *** heyongli_ has quit IRC | 08:17 | |
| *** heyongli_ has joined #openstack-infra | 08:17 | |
| *** kushal has joined #openstack-infra | 08:17 | |
| *** derekh has quit IRC | 08:19 | |
| AJaeger | project-config cores, could you review https://review.openstack.org/311675 , please? That's a change for openstack-manuals that the docs team would love to see merged soonish. Thanks! | 08:20 |
| *** abregman_ has quit IRC | 08:22 | |
| *** jistr has joined #openstack-infra | 08:26 | |
| *** thorst has joined #openstack-infra | 08:26 | |
| *** heyongli_ has quit IRC | 08:27 | |
| *** heyongli_ has joined #openstack-infra | 08:27 | |
| *** |-paul-| has joined #openstack-infra | 08:30 | |
| *** thorst has quit IRC | 08:33 | |
| *** jordanP has joined #openstack-infra | 08:33 | |
| *** abregman has joined #openstack-infra | 08:33 | |
| *** heyongli_ has quit IRC | 08:37 | |
| *** heyongli_ has joined #openstack-infra | 08:37 | |
| *** jordanP has quit IRC | 08:39 | |
| *** asettle has joined #openstack-infra | 08:41 | |
| *** tojuvone has quit IRC | 08:42 | |
| *** tojuvone has joined #openstack-infra | 08:43 | |
| openstackgerrit | Nisha Agarwal proposed openstack/diskimage-builder: Install proliantutils in IPA's virtualenv https://review.openstack.org/299116 | 08:47 |
| *** heyongli_ has quit IRC | 08:47 | |
| *** heyongli_ has joined #openstack-infra | 08:48 | |
| *** kengo_sakai has joined #openstack-infra | 08:52 | |
| *** markvoelker has joined #openstack-infra | 08:52 | |
| *** daemontool has quit IRC | 08:54 | |
| *** nadya has joined #openstack-infra | 08:54 | |
| *** kengo_sakai has quit IRC | 08:56 | |
| *** markvoelker has quit IRC | 08:57 | |
| *** heyongli_ has quit IRC | 08:58 | |
| *** heyongli_ has joined #openstack-infra | 08:58 | |
| *** asettle has quit IRC | 09:00 | |
| *** vincentll has quit IRC | 09:01 | |
| *** xnox has quit IRC | 09:04 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Do not run release notes for openstack-manuals/liberty https://review.openstack.org/311701 | 09:05 |
| *** yaume has joined #openstack-infra | 09:05 | |
| *** jordanP has joined #openstack-infra | 09:07 | |
| *** heyongli_ has quit IRC | 09:08 | |
| *** heyongli_ has joined #openstack-infra | 09:08 | |
| *** nadya has quit IRC | 09:08 | |
| *** kengo_sakai has joined #openstack-infra | 09:09 | |
| *** vincentll has joined #openstack-infra | 09:16 | |
| *** heyongli_ has quit IRC | 09:18 | |
| *** heyongli_ has joined #openstack-infra | 09:18 | |
| *** ccamacho has joined #openstack-infra | 09:20 | |
| jistr | Hi, can someone please point us to where should we put the change for libffi-devel and openssl-devel installation? I wasn't able to find where the mentioned jenkins_slave.pp file is. https://review.openstack.org/#/c/311217/ | 09:22 |
| *** xnox has joined #openstack-infra | 09:27 | |
| *** tosky has joined #openstack-infra | 09:28 | |
| *** heyongli__ has joined #openstack-infra | 09:29 | |
| *** nadya has joined #openstack-infra | 09:30 | |
| *** sshnaidm has quit IRC | 09:31 | |
| *** heyongli_ has quit IRC | 09:33 | |
| *** thorst has joined #openstack-infra | 09:33 | |
| *** thorst has quit IRC | 09:38 | |
| *** heyongli__ has quit IRC | 09:39 | |
| *** heyongli__ has joined #openstack-infra | 09:39 | |
| *** sshnaidm has joined #openstack-infra | 09:42 | |
| *** nadya has quit IRC | 09:43 | |
| *** chem has joined #openstack-infra | 09:46 | |
| *** heyongli__ has quit IRC | 09:49 | |
| *** heyongli__ has joined #openstack-infra | 09:49 | |
| *** yamamoto has quit IRC | 09:53 | |
| *** markvoelker has joined #openstack-infra | 09:53 | |
| *** pfallenop has quit IRC | 09:55 | |
| *** hashar has quit IRC | 09:55 | |
| *** hashar has joined #openstack-infra | 09:56 | |
| *** markvoelker has quit IRC | 09:58 | |
| *** heyongli__ has quit IRC | 09:59 | |
| *** heyongli__ has joined #openstack-infra | 09:59 | |
| *** pfallenop has joined #openstack-infra | 10:01 | |
| *** HeOS has quit IRC | 10:04 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/project-config: TipleO CI dep missing https://review.openstack.org/311711 | 10:05 |
| *** heyongli__ has quit IRC | 10:09 | |
| *** ramishra has quit IRC | 10:10 | |
| *** heyongli__ has joined #openstack-infra | 10:10 | |
| *** pfallenop has quit IRC | 10:10 | |
| *** kengo_sakai has quit IRC | 10:11 | |
| *** ramishra has joined #openstack-infra | 10:15 | |
| *** vincentll has quit IRC | 10:15 | |
| *** heyongli__ has quit IRC | 10:20 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/project-config: TipleO CI dep missing https://review.openstack.org/311711 | 10:20 |
| *** heyongli__ has joined #openstack-infra | 10:20 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/project-config: TipleO CI dep missing https://review.openstack.org/311711 | 10:22 |
| *** pfallenop has joined #openstack-infra | 10:22 | |
| *** yamahata has quit IRC | 10:22 | |
| *** pfallenop has quit IRC | 10:28 | |
| *** vincentll has joined #openstack-infra | 10:29 | |
| *** heyongli__ has quit IRC | 10:30 | |
| *** heyongli__ has joined #openstack-infra | 10:30 | |
| *** aorourke__ has joined #openstack-infra | 10:30 | |
| *** hashar_ has joined #openstack-infra | 10:32 | |
| *** eliqiao_ has joined #openstack-infra | 10:33 | |
| *** sflanigan_ has joined #openstack-infra | 10:34 | |
| *** sflanigan has quit IRC | 10:34 | |
| *** vincentll has quit IRC | 10:34 | |
| *** pfallenop has joined #openstack-infra | 10:34 | |
| *** vipul- has joined #openstack-infra | 10:35 | |
| *** megm_ has joined #openstack-infra | 10:35 | |
| *** thorst has joined #openstack-infra | 10:36 | |
| *** vincentll has joined #openstack-infra | 10:36 | |
| ccamacho | ajaeger, hello Andreas, we are getting errors in our CI due to a dep missing error, you have recommended to update the other-requirements file | 10:36 |
| ccamacho | but not able to get it | 10:36 |
| *** kushal has quit IRC | 10:36 | |
| ccamacho | can you elaborate more on that? | 10:36 |
| *** mnencia_ has joined #openstack-infra | 10:37 | |
| *** aorourke_ has quit IRC | 10:37 | |
| *** eliqiao has quit IRC | 10:37 | |
| *** oanson has quit IRC | 10:37 | |
| *** megm has quit IRC | 10:37 | |
| *** vipuls has quit IRC | 10:37 | |
| *** hashar has quit IRC | 10:37 | |
| *** Guest68119 has quit IRC | 10:37 | |
| *** mnencia has quit IRC | 10:37 | |
| *** mnencia_ is now known as mnencia | 10:37 | |
| *** jefrite has quit IRC | 10:37 | |
| *** oanson has joined #openstack-infra | 10:38 | |
| *** zeus- has joined #openstack-infra | 10:38 | |
| *** chlong has joined #openstack-infra | 10:38 | |
| AJaeger | ccamacho: other-requirements.txt is documented here http://docs.openstack.org/infra/manual/drivers.html#package-requirements | 10:39 |
| AJaeger | ccamacho: if you fix a bug, please add that to the commit message, so that we have the full background on *why* you do what you do. | 10:39 |
| AJaeger | Is this for paramiko, ccamacho? | 10:40 |
| *** heyongli__ has quit IRC | 10:40 | |
| ccamacho | ajaeger, yes for paramiko | 10:40 |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: WIP: Add baymodel list call for shade+magnum https://review.openstack.org/308009 | 10:40 |
| AJaeger | I think fungi, pabelanger, and others need to discuss what's the best solution is for this problem. | 10:40 |
| *** heyongli__ has joined #openstack-infra | 10:40 | |
| *** nadya has joined #openstack-infra | 10:41 | |
| AJaeger | if this fails in devstack, you need to add the package requirements for devstack, see the files directory in devstack for list of dependencies | 10:42 |
| *** jefrite has joined #openstack-infra | 10:43 | |
| *** thorst has quit IRC | 10:43 | |
| ccamacho | ajaeger, thanks | 10:45 |
| *** mylu has joined #openstack-infra | 10:47 | |
| *** yolanda has quit IRC | 10:47 | |
| *** vincentll has quit IRC | 10:49 | |
| *** vincentll has joined #openstack-infra | 10:50 | |
| *** heyongli__ has quit IRC | 10:50 | |
| *** heyongli__ has joined #openstack-infra | 10:51 | |
| *** yolanda has joined #openstack-infra | 10:51 | |
| *** markvoelker has joined #openstack-infra | 10:54 | |
| *** rhallisey has joined #openstack-infra | 10:54 | |
| *** zeus- is now known as zeus | 10:58 | |
| *** zeus is now known as Guest25915 | 10:59 | |
| *** markvoelker has quit IRC | 10:59 | |
| *** heyongli__ has quit IRC | 11:01 | |
| *** heyongli__ has joined #openstack-infra | 11:01 | |
| * rcarrillocruz waves | 11:02 | |
| * rcarrillocruz starting to rise from dead after 2 stops and 6h layover yesterday at Madrid | 11:02 | |
| *** hashar_ has quit IRC | 11:02 | |
| *** roxanaghe has joined #openstack-infra | 11:06 | |
| * AJaeger waves | 11:10 | |
| rcarrillocruz | hope everyone had safe travels | 11:10 |
| *** roxanaghe has quit IRC | 11:10 | |
| rcarrillocruz | will try to catch up later in the day with HP NOC for the infra cloud status, got access to the vpn but still awaiting for iLO IPs | 11:10 |
| rcarrillocruz | i'll be on and off, today is bank holiday here | 11:11 |
| *** Na3iL has joined #openstack-infra | 11:11 | |
| *** heyongli__ has quit IRC | 11:11 | |
| *** heyongli__ has joined #openstack-infra | 11:11 | |
| AJaeger | rcarrillocruz: relax and recover ;) | 11:12 |
| rcarrillocruz | :P | 11:13 |
| *** mwagner_lap has joined #openstack-infra | 11:14 | |
| *** yolanda has quit IRC | 11:16 | |
| *** yolanda has joined #openstack-infra | 11:16 | |
| *** thorst has joined #openstack-infra | 11:19 | |
| *** thorst has quit IRC | 11:21 | |
| *** thorst has joined #openstack-infra | 11:21 | |
| *** heyongli__ has quit IRC | 11:21 | |
| *** heyongli__ has joined #openstack-infra | 11:21 | |
| *** thorst has quit IRC | 11:26 | |
| *** yolanda has quit IRC | 11:29 | |
| *** yolanda has joined #openstack-infra | 11:30 | |
| pabelanger | yolanda: pretty sure, we are still doing a restore between ord and dfw. we're xfering 12Mb/s between the 2 | 11:31 |
| *** thorst has joined #openstack-infra | 11:31 | |
| *** heyongli__ has quit IRC | 11:31 | |
| *** heyongli__ has joined #openstack-infra | 11:32 | |
| *** ldnunes has joined #openstack-infra | 11:32 | |
| *** anteaya has joined #openstack-infra | 11:37 | |
| *** nadya has quit IRC | 11:37 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add centos-7 to tripleo cloud https://review.openstack.org/311721 | 11:41 |
| *** heyongli__ has quit IRC | 11:42 | |
| *** heyongli__ has joined #openstack-infra | 11:42 | |
| *** trown|outtypewww is now known as trown | 11:43 | |
| *** e0ne has joined #openstack-infra | 11:44 | |
| yolanda | pabelanger, so do you think the long time to complete is normal? | 11:47 |
| yolanda | a vos status shows the forwardmulti transaction has been inactive for hours | 11:47 |
| anteaya | rcarrillocruz: thanks for the update | 11:48 |
| *** flepied has joined #openstack-infra | 11:49 | |
| *** woodster_ has joined #openstack-infra | 11:49 | |
| *** raginbajin has quit IRC | 11:50 | |
| pabelanger | yolanda: Ya, we have almost 300GB of data to xfter between mirrors. Using 12/Mbps that's about 2.2 days to xfer that | 11:50 |
| *** heyongli__ has quit IRC | 11:52 | |
| *** heyongli__ has joined #openstack-infra | 11:53 | |
| *** kzaitsev_mb has joined #openstack-infra | 11:53 | |
| *** markvoelker has joined #openstack-infra | 11:55 | |
| *** mylu has quit IRC | 11:56 | |
| *** markvoelker has quit IRC | 11:59 | |
| *** amrith is now known as _amrith_ | 11:59 | |
| *** EricGonczer_ has joined #openstack-infra | 11:59 | |
| anteaya | for the record I am absolutely exhausted, will be a few days before I am able to get back in a rhythm, last time I felt this exhausted was my sister's wedding | 12:01 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Add usernname configuration option for launcher https://review.openstack.org/311725 | 12:02 |
| *** heyongli__ has quit IRC | 12:02 | |
| *** heyongli__ has joined #openstack-infra | 12:02 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/infra-ansible: TripleO CI ansible deps https://review.openstack.org/311726 | 12:03 |
| *** markvoelker has joined #openstack-infra | 12:06 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/infra-ansible: TripleO CI ansible deps https://review.openstack.org/311726 | 12:08 |
| AJaeger | anteaya: get well soon! | 12:08 |
| anteaya | oh I'm not sick, just very very very very very tired | 12:08 |
| anteaya | but thank you :) | 12:08 |
| anteaya | AJaeger: I see your airplane's windshield wiper issue must have gotten addressed? | 12:09 |
| *** aysyd has joined #openstack-infra | 12:09 | |
| AJaeger | anteaya: yeah, took 1.5 hours to do that - and then 30 mins of paperwork plus 15 mins of refuelling the plan ;( | 12:10 |
| AJaeger | anteaya: So, is there a better wish? Take your time and recover fully? | 12:10 |
| *** baoli has joined #openstack-infra | 12:11 | |
| anteaya | AJaeger: glad you finally got on your way | 12:11 |
| anteaya | AJaeger: yeah, I guess that is a good wish | 12:11 |
| anteaya | both are great, thank you | 12:12 |
| AJaeger | anteaya: yeah, so was I... | 12:12 |
| anteaya | just the get well soon phrase is usually the response after being informed someone is either ill or injured | 12:12 |
| AJaeger | At least not spending the night at the airport ;) | 12:12 |
| anteaya | no kidding | 12:12 |
| anteaya | congratulations for not spending the night at the airport :) | 12:12 |
| *** baoli_ has joined #openstack-infra | 12:12 | |
| *** heyongli__ has quit IRC | 12:12 | |
| anteaya | there are greeting cards with the phrase 'get well soon' printed on them which you usually send to someone when they are in the hospital | 12:13 |
| anteaya | speaking of, yolanda, I hope your family is well | 12:14 |
| AJaeger | yeah, knew about that - but didn't know what the best way was to wish you all the best ... | 12:14 |
| yolanda | anteaya, thanks | 12:14 |
| anteaya | yolanda: I am sorry you couldn't make it to summit but I understand the circumstances | 12:14 |
| * AJaeger was at least not creative | 12:14 | |
| anteaya | AJaeger: ah thank you | 12:14 |
| anteaya | AJaeger: ha ha ha | 12:14 |
| yolanda | we had better moments, but she is recovering | 12:14 |
| yolanda | i have to be at the hospital for 7-10 days more at leat | 12:14 |
| yolanda | least | 12:14 |
| *** e0ne has quit IRC | 12:14 | |
| anteaya | yolanda: understood, glad she is doing well | 12:15 |
| AJaeger | yolanda: ;( | 12:15 |
| anteaya | yolanda: blessed healing for her and I hope she continues to do well | 12:15 |
| anteaya | yolanda: and take your time | 12:15 |
| *** baoli has quit IRC | 12:16 | |
| yolanda | anteaya, many thanks, much appreciated | 12:16 |
| *** kgiusti has joined #openstack-infra | 12:16 | |
| *** weshay has joined #openstack-infra | 12:17 | |
| *** heyongli_ has joined #openstack-infra | 12:17 | |
| *** tlian has joined #openstack-infra | 12:17 | |
| *** dtardivel has joined #openstack-infra | 12:19 | |
| anteaya | yolanda: :) | 12:20 |
| openstackgerrit | Carlos Camacho proposed openstack-infra/infra-ansible: Adding dependencies for paramiko 2.0 when installing ansible https://review.openstack.org/311726 | 12:21 |
| *** heyongli_ has quit IRC | 12:23 | |
| *** heyongli_ has joined #openstack-infra | 12:23 | |
| AJaeger | any project-config core willing to push https://review.openstack.org/311701 on their review queue, please? This does not run releasenotes jobs on liberty for openstack-manuals, we only enabled this for mitaka. | 12:24 |
| AJaeger | ccamacho, Regarding 311726 let's not add more workarounds but work on a good plan first - and wait for fungi to see what he has to say... | 12:25 |
| *** dprince has joined #openstack-infra | 12:25 | |
| ccamacho | sure | 12:25 |
| *** EricGonczer_ has quit IRC | 12:30 | |
| *** edmondsw has joined #openstack-infra | 12:33 | |
| *** heyongli_ has quit IRC | 12:33 | |
| *** heyongli_ has joined #openstack-infra | 12:33 | |
| *** psilvad has joined #openstack-infra | 12:35 | |
| *** psilvad has quit IRC | 12:36 | |
| *** psilvad has joined #openstack-infra | 12:36 | |
| *** Na3iL has quit IRC | 12:38 | |
| *** timl0101 has joined #openstack-infra | 12:39 | |
| *** jheroux has joined #openstack-infra | 12:40 | |
| *** jheroux has quit IRC | 12:42 | |
| *** jheroux has joined #openstack-infra | 12:43 | |
| *** Hal has joined #openstack-infra | 12:43 | |
| *** Hal is now known as Guest39041 | 12:43 | |
| *** heyongli_ has quit IRC | 12:43 | |
| *** heyongli_ has joined #openstack-infra | 12:43 | |
| *** pfallenop has quit IRC | 12:44 | |
| *** gordc has joined #openstack-infra | 12:45 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/infra-ansible: Adding dependencies for paramiko 2.0 when installing ansible https://review.openstack.org/311726 | 12:47 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Pin paramiko < 2.0.0 https://review.openstack.org/311626 | 12:48 |
| pabelanger | AJaeger: ccamacho: fungi: why we need to pin paramiki<2.0.0 ^ | 12:48 |
| pabelanger | regardless of installing missing libs, zuul won't function | 12:49 |
| ccamacho | pabelanger, paramiko 2.0 uses cryptography | 12:49 |
| *** rfolco has joined #openstack-infra | 12:49 | |
| ccamacho | and need other deps | 12:49 |
| cbader | anteaya, question on dib-image build do you know if someone is working on fix for ffi.h error on creation of build? | 12:49 |
| ccamacho | check https://review.openstack.org/#/q/Ia7e0d64214cfd5412ec236ee4bdd6a076821d258,n,z | 12:49 |
| *** pfallenop has joined #openstack-infra | 12:50 | |
| pabelanger | ccamacho: we need to audit paramiko 2.0 and see what is all needed. I'd rather pin ATM until we can see what all needs to be updated | 12:50 |
| *** krtaylor has joined #openstack-infra | 12:51 | |
| *** jaypipes has joined #openstack-infra | 12:51 | |
| ccamacho | pabelanger, question, the dependecies hsould be added in openstack-infra/infra-ansible / setup_env.sh ? | 12:51 |
| pabelanger | ccamacho: they should be added into jenkins_slave.pp | 12:51 |
| pabelanger | in openstack-infra/system-config | 12:52 |
| pabelanger | and / or bindep-fallback.txt | 12:52 |
| *** heyongli_ has quit IRC | 12:53 | |
| ccamacho | pabelander, like https://review.openstack.org/#/c/311711/3/jenkins/data/bindep-fallback.txt but jenkins_slave.pp does not exist | 12:54 |
| *** heyongli_ has joined #openstack-infra | 12:54 | |
| pabelanger | ccamacho: sorry, jenkins_params.pp and jenkins.pp | 12:55 |
| zxiiro | Do you have to have an entry in default_data.json to appear in stackalytics? | 12:55 |
| pabelanger | AJaeger: I think bindep-fallback.txt might be a good place. Especially since we want to stop using jenkins_params.pp and jenkins.pp | 12:55 |
| *** nelsnelson has quit IRC | 12:56 | |
| pabelanger | zxiiro: I believe so | 12:56 |
| ccamacho | pabelanger, mmm i see | 12:56 |
| ccamacho | in bindep-fallback libffi is already included | 12:57 |
| pabelanger | We need to do some optimizations for DIBs. Since bindep-fallback only runs when jenkins launches. | 12:58 |
| pabelanger | I am adding a package-install to the nodepool-base element | 13:00 |
| *** sdake has quit IRC | 13:00 | |
| pabelanger | that should help | 13:00 |
| *** _ari_ has joined #openstack-infra | 13:00 | |
| *** hrybacki has quit IRC | 13:01 | |
| *** hrybacki has joined #openstack-infra | 13:01 | |
| *** sdake has joined #openstack-infra | 13:01 | |
| ccamacho | pabelanger, thanks | 13:03 |
| ccamacho | I will abandone mi tests then If you are getting into install the packages | 13:04 |
| *** heyongli_ has quit IRC | 13:04 | |
| *** sdake_ has joined #openstack-infra | 13:04 | |
| *** heyongli_ has joined #openstack-infra | 13:04 | |
| anteaya | cbader: I have no idea, my suggest is read the channel backscroll for yourself: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ | 13:05 |
| *** sdake has quit IRC | 13:06 | |
| *** aysyd has quit IRC | 13:06 | |
| *** aysyd has joined #openstack-infra | 13:07 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add new zuul dependencies https://review.openstack.org/311734 | 13:07 |
| pabelanger | that should do it, just testing with ubuntu-trusty and centos-7 now | 13:08 |
| *** _amrith_ is now known as amrith | 13:09 | |
| *** deadnull_ has joined #openstack-infra | 13:10 | |
| *** samuelBartel has joined #openstack-infra | 13:11 | |
| *** rbradfor_home is now known as rbradfor | 13:11 | |
| *** Goneri has joined #openstack-infra | 13:12 | |
| *** kengo_sakai has joined #openstack-infra | 13:12 | |
| AJaeger | pabelanger: and added to devstack/files | 13:12 |
| AJaeger | zxiiro: no, you should appear directly there... | 13:13 |
| *** tpsilva has joined #openstack-infra | 13:13 | |
| *** heyongli_ has quit IRC | 13:14 | |
| *** heyongli_ has joined #openstack-infra | 13:14 | |
| *** rkukura has quit IRC | 13:15 | |
| AJaeger | zxiiro: http://stackalytics.com/?user_id=zxiiro | 13:16 |
| *** kengo_sakai has quit IRC | 13:17 | |
| *** julim has joined #openstack-infra | 13:19 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: workaround to deploy libffi-devel and openssl-devel https://review.openstack.org/311737 | 13:20 |
| zxiiro | AJaeger: thanks, I was asking in regards to one of my interns not showing up but I just realized the issue. We had to search Mitaka since his patches were merged a few months ago. | 13:20 |
| *** julim has quit IRC | 13:20 | |
| *** pilgrimstack has joined #openstack-infra | 13:21 | |
| *** heyongli_ has quit IRC | 13:24 | |
| *** heyongli_ has joined #openstack-infra | 13:24 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: workaround to deploy libffi-devel and openssl-devel https://review.openstack.org/311737 | 13:25 |
| *** pradk has joined #openstack-infra | 13:29 | |
| *** baoli_ has quit IRC | 13:30 | |
| cbader | anteaya, thank you | 13:30 |
| fungi | AJaeger: pabelanger: i approved the paramiko<2.0.0 pin in zuul for now until someone has time to dig into making it compatible. as a side effect that will also avoid us needing libffi-dev and openssl-dev for a little while until we decide if we want to have dib installs for /usr/zuul-env use prebuilt wheels of cryptography from our wheel mirror | 13:31 |
| *** jaosorior has joined #openstack-infra | 13:32 | |
| *** yolanda has quit IRC | 13:32 | |
| AJaeger | fungi, I've seen some changes flying by for the issue like 311737 and 311726 - and do not know what's the best way is and neither have time to dig... | 13:32 |
| *** doug-fish has joined #openstack-infra | 13:33 | |
| openstackgerrit | Merged openstack-infra/zuul: Pin paramiko < 2.0.0 https://review.openstack.org/311626 | 13:34 |
| pabelanger | fungi: works for me. Adding wheels to dibs would be great, but likely a longer term goal. Also testing new pkg-map for nodepool-base element ATM | 13:34 |
| *** heyongli_ has quit IRC | 13:34 | |
| *** heyongli_ has joined #openstack-infra | 13:35 | |
| *** links has quit IRC | 13:35 | |
| *** yolanda has joined #openstack-infra | 13:36 | |
| fungi | i need to head to an appointment but should be back in a couple hours | 13:37 |
| *** otherwiseguy has joined #openstack-infra | 13:37 | |
| pabelanger | ack | 13:37 |
| EmilienM | pabelanger: when you have time this week for the tarballs, please ping me | 13:38 |
| pabelanger | EmilienM: Ha ha, I was just about to ping you about it | 13:38 |
| EmilienM | ok, here is the etherpad: https://etherpad.openstack.org/p/fpHDMxrKuQ | 13:38 |
| pabelanger | EmilienM: great, will start at the top again | 13:38 |
| EmilienM | let's try one command, yeah | 13:38 |
| pabelanger | ++ | 13:38 |
| *** kushal has joined #openstack-infra | 13:39 | |
| *** jaosorior has quit IRC | 13:41 | |
| *** tonytan4ever has joined #openstack-infra | 13:44 | |
| *** heyongli_ has quit IRC | 13:45 | |
| *** heyongli_ has joined #openstack-infra | 13:45 | |
| *** nelsnelson has joined #openstack-infra | 13:46 | |
| *** |-paul-| has quit IRC | 13:47 | |
| EmilienM | pabelanger: puppet-aodh tarball name is wrong | 13:47 |
| EmilienM | puppet-aodh.tar.gz should be with the tag | 13:47 |
| EmilienM | see http://tarballs.openstack.org/puppet-aodh/ | 13:47 |
| pabelanger | EmilienM: http://logs.openstack.org/3f/3f0e8910b2703ed7f23ec22a8ec86e575c5f922c/release/puppet-aodh-tarball/manual_000/console.html | 13:47 |
| pabelanger | that is the log file | 13:47 |
| EmilienM | ok, /me looking | 13:47 |
| anteaya | cbader: welcome | 13:47 |
| pabelanger | EmilienM: just testing puppet-ceilometer now, then stopping | 13:48 |
| EmilienM | http://logs.openstack.org/3f/3f0e8910b2703ed7f23ec22a8ec86e575c5f922c/release/puppet-aodh-tarball/manual_000/console.html#_2016-05-02_13_45_30_195 | 13:48 |
| anteaya | cbader: the third party meeting is in the #openstack-meeting channel at 15:00 utc today, you could consider attending | 13:48 |
| pabelanger | EmilienM: also, there is an issue with publisher | 13:48 |
| cbader | anteaya, I have been attending just not saying anything yet just watching. | 13:48 |
| EmilienM | pabelanger: the "dist/*.whl" thing? | 13:49 |
| *** ccamacho has quit IRC | 13:49 | |
| anteaya | cbader: okay, glad you have been attending | 13:49 |
| *** dmsimard|afk is now known as dmsimard | 13:49 | |
| pabelanger | EmilienM: ya | 13:49 |
| pabelanger | EmilienM: which causes jenkins to report a failure | 13:49 |
| cbader | anteaya, I have this thread up on a monitor all day watching and looking at commits. I has been a great help. | 13:50 |
| anteaya | cbader: oh I'm so glad to hear that | 13:50 |
| anteaya | well done | 13:50 |
| cbader | anteaya, I spent all weekend trying to get my ci back up after dib-images crashed thursday night with no luck. so looking for some clue. will ask today. | 13:51 |
| anteaya | cbader: great, I look forward to seeing you in the meeting | 13:51 |
| EmilienM | pabelanger: the thing is {name}-branch-tarball JJB is doing a mv, so the branch tarballs have the right name. But when it's regular tarball job, it does not perform any mv, so I wonder how to put the tag in the name | 13:52 |
| cbader | anteay, found fix for vms and they are up but dib is still bad | 13:52 |
| anteaya | hopefully one of the other operators might have some insight | 13:52 |
| anteaya | cbader: understood, well done on finding a fix for your vms | 13:52 |
| cbader | anteaya, andrew has been a help | 13:52 |
| EmilienM | pabelanger: if you look at jenkins/jobs/python-jobs.yaml | 13:52 |
| anteaya | cbader: I'm glad to hear that | 13:52 |
| *** mriedem has joined #openstack-infra | 13:53 | |
| pabelanger | EmilienM: http://logs.openstack.org/71/71f6212daf6224d162fa45d136a2092e79e28437/release/puppet-aodh-tarball/manual_000/console.html#_2016-05-02_13_41_27_693 | 13:54 |
| *** Guest25915 is now known as zeus` | 13:54 | |
| pabelanger | EmilienM: looks like puppet module build . is naming the file correctly | 13:54 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:54 | |
| pabelanger | EmilienM: do you have a recent branch-tarball log? | 13:55 |
| *** heyongli_ has quit IRC | 13:55 | |
| *** hashar has joined #openstack-infra | 13:55 | |
| *** heyongli_ has joined #openstack-infra | 13:55 | |
| pabelanger | EmilienM: Oh, I see the issue | 13:56 |
| *** e0ne has joined #openstack-infra | 13:56 | |
| pabelanger | EmilienM: http://logs.openstack.org/39/39d3950cf9a4b07153667c48cd53c041e8b2d483/release/puppet-ceilometer-tarball/manual_000/console.html#_2016-05-02_13_48_49_903 show why we need to rename | 13:56 |
| EmilienM | right | 13:56 |
| pabelanger | so 2 questions | 13:56 |
| EmilienM | we need to skip the rename if we run a tag job | 13:57 |
| pabelanger | 1): why don't you change the metadata.json name from openstack-ceilometer to puppet-ceilometer? | 13:57 |
| pabelanger | 2): we need to add version into for the move command | 13:57 |
| *** inc0 has joined #openstack-infra | 13:58 | |
| EmilienM | for 1), I think puppetlabs named this module because we have the https://forge.puppetlabs.com/openstack namespace | 13:58 |
| EmilienM | and yes for 2), but how to find if whether or not we are running a tag job | 13:58 |
| pabelanger | 1) okay | 13:59 |
| dims | Anyone have an idea on when we can get our pypi mirroring back on its feet? | 14:00 |
| EmilienM | because the mv is useful for the branch-tarball jobs | 14:00 |
| *** mwagner_lap has quit IRC | 14:00 | |
| pabelanger | EmilienM: so, puppet module build . is doing the right thing, we just need to sed openstack with puppet | 14:00 |
| dims | fungi : clarkb : yolanda : ^^ | 14:00 |
| EmilienM | pabelanger: that's what I'm doing, see jenkins/scripts/run-tarball.sh | 14:00 |
| *** mtanino has joined #openstack-infra | 14:00 | |
| *** oanson has quit IRC | 14:01 | |
| pabelanger | dims: we are waiting for the mirror to restore. 300GB @ 12Mbps is like 2.5 days | 14:01 |
| dims | pabelanger : ah i see. do we hold releases until then? | 14:01 |
| pabelanger | EmilienM: right, let me test something locally | 14:01 |
| pabelanger | dims: AFS won't let us release, that is the issue ATM. | 14:02 |
| dims | ack pabelanger | 14:02 |
| pabelanger | So, if I understand correctly, afs is reverting to the last know good state | 14:02 |
| mordred | we reverted something in AFS? | 14:02 |
| dims | pabelanger : ack thanks i'll tell ttx and dhellmann that we'll have to wait until things are back to normal to make releases | 14:02 |
| mordred | also, good morning | 14:03 |
| *** kzaitsev_mb has quit IRC | 14:03 | |
| pabelanger | mordred: I believe the vos release was killed due to some confusion between roots | 14:03 |
| mordred | yay! | 14:03 |
| mordred | well, hopefully jeblair will be up soon | 14:03 |
| pabelanger | I _think_ afs is restoring from last state. But need some help to confirm that | 14:04 |
| anteaya | mordred: good morning | 14:04 |
| *** heyongli_ has quit IRC | 14:05 | |
| *** jamesmcarthur has joined #openstack-infra | 14:05 | |
| *** heyongli_ has joined #openstack-infra | 14:05 | |
| bswartz | infra team: I'm trying to figure out how to create a specs repo... | 14:06 |
| bswartz | I looked at the history of project-config and I see some examples of other projects that have done it | 14:07 |
| bswartz | but I don't understand where the initial content for the repo comes from | 14:07 |
| anteaya | bswartz: have you read this? http://docs.openstack.org/infra/manual/creators.html | 14:07 |
| *** baoli has joined #openstack-infra | 14:07 | |
| bswartz | anteaya: yes that page came up in my googling and I didn't see anything specific about creation of the specs repo | 14:08 |
| anteaya | that is the guide to create any repo | 14:08 |
| anteaya | just call yours specs-foo | 14:08 |
| bswartz | okay I'll peruse that page in more detail | 14:08 |
| anteaya | and you probably don't need a launchpad account for bugs against a specs repo | 14:08 |
| anteaya | but up to you | 14:08 |
| bswartz | it would be openstack/manila-specs | 14:08 |
| anteaya | I figured that was what you were heading towards | 14:09 |
| mordred | morning anteaya ! | 14:09 |
| anteaya | :) | 14:09 |
| anteaya | sorry s/specs-foo/foo-specs | 14:09 |
| pabelanger | EmilienM: okay, I need a log for branch-tarball please | 14:09 |
| pabelanger | EmilienM: have a patch, just need to confirm it will work | 14:09 |
| EmilienM | pabelanger: let me find one | 14:10 |
| *** xarses_ has quit IRC | 14:10 | |
| *** jamesmcarthur has quit IRC | 14:10 | |
| EmilienM | pabelanger: http://logs.openstack.org/e3/e3c649f0877e2deb594864b2aacf5fc480cd4ffa/post/puppet-ceilometer-branch-tarball/ | 14:10 |
| *** sdake_ is now known as sdake | 14:11 | |
| *** ddieterly has joined #openstack-infra | 14:12 | |
| *** mtanino has quit IRC | 14:12 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge https://review.openstack.org/311750 | 14:12 |
| pabelanger | EmilienM: anything newer? | 14:12 |
| EmilienM | pabelanger: not afik | 14:13 |
| EmilienM | pabelanger: let me search | 14:13 |
| *** e0ne has quit IRC | 14:13 | |
| EmilienM | pabelanger: http://logs.openstack.org/a0/a0f957f2e85f8bd533b82675dd15aca0d48ba4f4/post/puppet-neutron-branch-tarball/ | 14:13 |
| EmilienM | from yesterday! | 14:13 |
| yolanda | mordred, so about afs... there is one restore task that seems active, and there is one forwardmulti task that seems inactive since this morning. No idea if that's expected or not | 14:14 |
| *** e0ne has joined #openstack-infra | 14:14 | |
| EmilienM | pabelanger: we still have the pip install ansible thing broking our CI, should we wait for a new DIB build & upload? or find a workaround? | 14:15 |
| *** heyongli_ has quit IRC | 14:15 | |
| *** nmagnezi has quit IRC | 14:15 | |
| *** heyongli_ has joined #openstack-infra | 14:16 | |
| bswartz | anteaya: would it be best to create a skeleton of manila-specs on my personal github and then import that using the "upstream" yaml tag? or could we create a completely empty repo and build the skeleton after the repo is created? | 14:16 |
| pabelanger | EmilienM: we pinned paramiko < 2.0.0 new images builds will pick it up shortly | 14:18 |
| *** rbrndt has joined #openstack-infra | 14:18 | |
| EmilienM | pabelanger: cool | 14:19 |
| clarkb | pabelanger: but not for ansible right? | 14:19 |
| yolanda | mordred, can you take a look at https://review.openstack.org/311610 ? magnum+shade jobs again | 14:19 |
| pabelanger | clarkb: EmilienM: oh, wait. ansible is broken too? | 14:19 |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Add baymodel list and search call for shade+magnum https://review.openstack.org/308009 | 14:19 |
| EmilienM | yeah | 14:19 |
| clarkb | pabelanger: yes, because it depends on paramiko if I understand correctly | 14:19 |
| *** mwagner_lap has joined #openstack-infra | 14:19 | |
| pabelanger | where is that broken? | 14:20 |
| pabelanger | only knew of zuul | 14:20 |
| EmilienM | http://logs.openstack.org/50/311750/1/check-tripleo/gate-tripleo-ci-f22-ha/01518a0/console.html | 14:20 |
| clarkb | devstack-gate is my guess bht I am still half asleep | 14:20 |
| EmilienM | clarkb: good guess ;-) it's in there. | 14:20 |
| pabelanger | I am testing another patch to add depends for nodepool-base element, but still testing | 14:20 |
| *** xyang1 has joined #openstack-infra | 14:22 | |
| pabelanger | guess we need to add ansible to wheel mirror | 14:23 |
| pabelanger | or ask #ansible to upload a wheel | 14:23 |
| clarkb | or install the libffi-dev and other deps | 14:23 |
| pabelanger | I don't have bandwidth atm to check on it let me finish up this | 14:23 |
| clarkb | or pin whatever it is in ansible install | 14:24 |
| *** esker has joined #openstack-infra | 14:25 | |
| *** heyongli_ has quit IRC | 14:26 | |
| EmilienM | clarkb: could we just install the dep in devstack-gate in the meantime? | 14:26 |
| *** heyongli_ has joined #openstack-infra | 14:26 | |
| AJaeger | bswartz: starting from empty repo should just work fine... | 14:27 |
| clarkb | EmilienM: that would be a quick workaround and should work | 14:27 |
| AJaeger | bswartz: run the openstack-dev/specs-cookiecutter to set it up | 14:27 |
| *** ajmiller has joined #openstack-infra | 14:27 | |
| *** dprince has quit IRC | 14:29 | |
| sdake | hey quick question | 14:31 |
| sdake | is it possible via acls to restrict +2's to limited directories in a repo | 14:31 |
| clarkb | no | 14:32 |
| sdake | related to kolla repo and the idea of a kolla-k8s-core review team | 14:32 |
| sdake | clarkb thanks | 14:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Update puppet logic for run-tarball.sh https://review.openstack.org/311754 | 14:32 |
| *** ajmiller has quit IRC | 14:32 | |
| clarkb | we can restrict by git refs not by file path | 14:32 |
| pabelanger | EmilienM: okay ^ should do what we need. But need to test it | 14:32 |
| sdake | clarkb ok thanks | 14:32 |
| *** ajmiller has joined #openstack-infra | 14:32 | |
| jeblair | mordred: i'm up | 14:32 |
| sdake | so the onl yway to restrict is raelly trust | 14:33 |
| anteaya | jeblair: good morning | 14:33 |
| jeblair | mordred: i have not caught up on scrollback | 14:33 |
| anteaya | bswartz: you can create a completely empty repo and build everything in the repo after it is created, that is the way I would suggest | 14:34 |
| AJaeger | sdake: yes, trust. And keep in mind that reverting is always an easy option. | 14:34 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/devstack-gate: devstack-vm-gate-wrap: deploy libffi-dev & openssl-dev https://review.openstack.org/311755 | 14:34 |
| *** salv-orlando has joined #openstack-infra | 14:34 | |
| EmilienM | clarkb: ^ it probably needs review & adjustment | 14:34 |
| *** burgerk has joined #openstack-infra | 14:35 | |
| *** heyongli_ has quit IRC | 14:36 | |
| *** heyongli_ has joined #openstack-infra | 14:36 | |
| EmilienM | pabelanger: commented | 14:38 |
| pabelanger | EmilienM: replied | 14:40 |
| pabelanger | I also have to run an errand for a hour, will be back shortly | 14:40 |
| *** jang has joined #openstack-infra | 14:40 | |
| EmilienM | pabelanger: you missed the "fi", line 33 | 14:41 |
| EmilienM | pabelanger: no problem, ttyl | 14:41 |
| jeblair | i get the feeling that some folks want me to help with something about afs mirrors, but i'd really appreciate a summary of the issue. | 14:41 |
| *** deadnull_ is now known as _deadnull | 14:42 | |
| *** flepied has quit IRC | 14:43 | |
| anteaya | jeblair: my read of backscroll has led me to the conclusion that yolanda is the only one who has a handle on the afs issue, she seems to have seen something but isn't quite certain what action to take to fix it | 14:44 |
| *** xarses_ has joined #openstack-infra | 14:44 | |
| anteaya | and I am unable to summarize the 'something' | 14:44 |
| jeblair | i also saw that someone said something may have gone wrong due to 'confusion between roots' | 14:44 |
| jeblair | which makes me especially hesitant to wade in without someone actually explaining the problem | 14:45 |
| anteaya | I saw that as well | 14:45 |
| anteaya | I can understand your perspective | 14:45 |
| anteaya | indeed there is a lack of detail on this issue | 14:45 |
| *** ramishra has quit IRC | 14:46 | |
| *** heyongli_ has quit IRC | 14:46 | |
| *** heyongli_ has joined #openstack-infra | 14:46 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Update puppet logic for run-tarball.sh https://review.openstack.org/311754 | 14:47 |
| *** ramishra has joined #openstack-infra | 14:47 | |
| anteaya | so, for deprecating the sprints page when we get to it: https://wiki.openstack.org/wiki/Sprints | 14:47 |
| anteaya | a page in a repo should be pretty straightforward | 14:47 |
| anteaya | but what repo? | 14:47 |
| anteaya | new repo? existing repo? | 14:48 |
| *** asselin has joined #openstack-infra | 14:48 | |
| anteaya | virtual sprints could probably go into the same repo, on a separate page: https://wiki.openstack.org/wiki/VirtualSprints | 14:48 |
| AJaeger | anteaya: for scheduling? irc-meeting ;) | 14:48 |
| anteaya | good point | 14:48 |
| AJaeger | anteaya: seriously: I don't think we have something that really fits... | 14:49 |
| anteaya | agreed | 14:49 |
| anteaya | we might have to create something | 14:49 |
| anteaya | but creating a new repo per page or two might be a poor use of new repos | 14:49 |
| mordred | jeblair: I do not have any additional clarity and agree with anteaya's read on the backscroll. I have not begun attempting to investigate the situation for largely the same reason | 14:49 |
| AJaeger | the nice thing about the wiki is the self-service | 14:49 |
| anteaya | all the things from the wiki repo? | 14:49 |
| anteaya | AJaeger: so is gerrit | 14:49 |
| AJaeger | anteaya: gerrit needs at least one core to review. Which has its advantages but can be a burden as well... | 14:50 |
| yolanda | hi jeblair, so afs seems to have a problem since 29 of april, failing with vos release | 14:50 |
| jeblair | i did check the log, but only saw the problem statement: https://wiki.openstack.org/wiki/Infrastructure_Status | 14:50 |
| anteaya | but the patch is in gerrit | 14:50 |
| jeblair | yolanda: yes, that much i know :) | 14:50 |
| jeblair | yolanda: thanks for the log entry :) | 14:50 |
| *** ayoung has joined #openstack-infra | 14:50 | |
| anteaya | AJaeger: folks can read the patch prior to it being merged | 14:50 |
| yolanda | when i checked, no vos release process was running, but even with that, the volumes were locked | 14:51 |
| yolanda | i restarted the fileservers and then release ran again, but it's there forever. vos status shows a restore task for one of the mirrors, that seems to be active, and one forwardmulti task in another, that seems to be inactive | 14:51 |
| yolanda | i don't know how to troubleshoot that | 14:51 |
| AJaeger | anteaya: yes, agreed. I don't have a strong opinion right now, to me it looks like we should write this one down and see what other information needs to move from the wiki and then decide what to do with it. | 14:52 |
| AJaeger | anteaya: but if you have a strong opinion, go for it;) | 14:52 |
| anteaya | AJaeger: fair enough | 14:52 |
| anteaya | I am so out of strong opinons today | 14:53 |
| * AJaeger as well... | 14:53 | |
| *** salv-orlando has quit IRC | 14:53 | |
| anteaya | my cat is back in my lap and I'm trying to avoid strong opinions for as long as I can | 14:53 |
| *** reed has quit IRC | 14:53 | |
| *** reed has joined #openstack-infra | 14:53 | |
| anteaya | and just enjoy my cat | 14:53 |
| jeblair | yolanda: somebody in scrollback said something about killing or deleting locks or vos release commands because of confusion between roots. do you know what that was about? | 14:54 |
| yolanda | jeblair, i was working with pabelaner on that yesterday, he may be talking about that | 14:55 |
| jeblair | yolanda: yes; what was the confusion that happened? | 14:55 |
| yolanda | there was one vos_release running when i restarted the service, then we were talking about that release should have timeout in 30 minutes, and we killed the process and restarted again | 14:56 |
| yolanda | so looks as the process started again yesterday | 14:56 |
| yolanda | it doesn't seem to timeout after 30 minutes, but not sure if that's progressing properly now | 14:56 |
| sdake | AJaeger ia gree revert wins on bad actors :) | 14:56 |
| *** heyongli_ has quit IRC | 14:56 | |
| sdake | plus removal from the acls if needed | 14:56 |
| *** heyongli_ has joined #openstack-infra | 14:57 | |
| yolanda | jeblair, one thing that happened is that afs ran out of quota, that may have caused the initial problem on 29 april | 14:57 |
| sdake | AJaeger can we have a different acl for the same repo ? | 14:57 |
| sdake | the reason iask is one of the propoosals is to have kolla-core and kolla-k8s-core with same permissions on the repo but with different rights in the project | 14:57 |
| jeblair | yolanda: the vos release is not in a timeout -- it is permitted to take as long as it takes; however, if it takes too long, its authentication tokens will expire and will not clean up after itself and get stuck in a state like this | 14:57 |
| anteaya | sdake: how are you differenciating between permissions and rights? | 14:58 |
| *** _deadnull is now known as deadnull_ | 14:58 | |
| sdake | rights = voting rights for policy | 14:58 |
| jeblair | yolanda: running out of quota on one volume (like ceph) should never cause a problem with another (like pypi); and it should also not cause a problem with vos release | 14:58 |
| sdake | permissions = +2/-2 on commits to the repo | 14:58 |
| anteaya | sdake: do you mean approval? | 14:58 |
| *** deadnull_ has quit IRC | 14:58 | |
| jeblair | yolanda: from looking at the logs on the 29th, it seems like there was a network issue between dfw and ord, and the vos release failed. | 14:58 |
| *** mtanino has joined #openstack-infra | 14:59 | |
| notmorgan | o/ mornin all | 14:59 |
| sdake | anteaya apologies lost context there, could you add some :) | 14:59 |
| AJaeger | sdake: yes, you could have two groups on the repo; example: both groups can +2 but onl yone can +A. | 14:59 |
| anteaya | sdake: right now most acls have +2 and +1 workflow for core reviewers | 14:59 |
| yolanda | jeblair, what can we do on that cases? tried vos release several times, and it seems to stuck all the time | 14:59 |
| anteaya | sdake: are you wanting a different group to have +1 workflow? | 14:59 |
| sdake | AJaeger oh winning | 14:59 |
| sdake | anteaya thats a good solution i didn't know existed | 15:00 |
| jeblair | yolanda: i suspect that because of that, subsequent vos release actions are requiring more time to complete -- either because it has more work to do to clean up or because the underlying volume has changed enough that it is taking longer to complete the transfer than our authentication tokens allow | 15:00 |
| *** mriedem is now known as mriedem_meeting | 15:00 | |
| AJaeger | sdake: let me give you a reference... | 15:00 |
| yolanda | jeblair, seems it's doing a complete release, not incremental | 15:00 |
| anteaya | sdake: I was asking if that was what you wanted, not suggesting you do that | 15:00 |
| jeblair | yolanda: so i think we need to run a manual vos release on the fileserver in screen, so that we can avoid it being interrupted | 15:00 |
| ansiwen | AJaeger: do I have to change the upper-constraints.txt as well? | 15:00 |
| AJaeger | sdake: http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerrit/acls/openstack/trove-dashboard.config has two groups - but both have the same permissions | 15:00 |
| jeblair | yolanda: it always says "complete release" even if it's an incremental one. | 15:01 |
| sdake | anteaya yes got it but it seems like a sensible workaround | 15:01 |
| AJaeger | ansiwen: for BAbel? Yes. | 15:01 |
| anteaya | sdake: workaround for what, what are you trying to do? | 15:01 |
| yolanda | jeblair, can you help with that? my wifi is so bad to run anything on servers | 15:01 |
| AJaeger | sdake: you can adopt that trove-dashboard setup and do something similar if needed. | 15:01 |
| sdake | btw, can infra handle the case where kolla has 80 gates which are short running (20 minutes) | 15:01 |
| AJaeger | sdake: But my recommendation : KISS. | 15:01 |
| AJaeger | and social contract ;) | 15:01 |
| jeblair | yolanda: well, right now there is one running on mirror-update | 15:01 |
| sdake | AJaeger right - i like social contract model | 15:01 |
| AJaeger | I doubt you have many problems. | 15:02 |
| sdake | AJaeger i agree | 15:02 |
| anteaya | sdake: do you mean jobs? | 15:02 |
| AJaeger | sdake: On the other hand, looking at the k8s thread: You had many many people that want to be cores - that's really a lot. | 15:02 |
| yolanda | jeblair, yes, running since this morning, and vos status is showing that two tasks i mentioned | 15:02 |
| anteaya | infra has one and only one gate pipeline | 15:02 |
| ansiwen | AJaeger: yes, for babel | 15:02 |
| *** EricGonczer_ has joined #openstack-infra | 15:02 | |
| AJaeger | sdake: Add none, see who reviews, and speed add after one month the first three ;) | 15:02 |
| sdake | AJaeger yes - but i suspect half of them will dorp out in the first 30-60 days | 15:02 |
| jeblair | yolanda: i only see the forwardmulti transaction | 15:03 |
| sdake | AJaeger yar - but we want to move super fast | 15:03 |
| ansiwen | AJaeger: i'm confused, because upper-constrains still points to 2.2.0, but 2.3.4 is out | 15:03 |
| yolanda | restore finished? | 15:03 |
| *** mwagner_lap has quit IRC | 15:03 | |
| jeblair | yolanda: oh the restore is on ord | 15:03 |
| yolanda | yes | 15:03 |
| ansiwen | AJaeger: probably I don't understand the meaning of that file | 15:03 |
| jeblair | yolanda: i believe those are two sides of the same transaction | 15:04 |
| AJaeger | ansiwen: 2.3.3 had problems, so we didn't go up... | 15:04 |
| yolanda | and the forwardmulti says not active since some hours | 15:04 |
| AJaeger | ansiwen: we blacklisted 2.3.2, 2.3.3 - and then it was summit and nobody approved the go to 2.3.4 ;) | 15:04 |
| jeblair | yolanda: that is -- dfw is transmitting a copy of a volume (forwardmulti) to ord (restore) | 15:04 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/devstack-gate: devstack-vm-gate-wrap: handle ansible dependencies https://review.openstack.org/311755 | 15:04 |
| *** mriedem_meeting is now known as mriedem | 15:04 | |
| *** samuelBartel has quit IRC | 15:04 | |
| yolanda | jeblair, so you think everything is fine then? | 15:04 |
| *** arxcruz has quit IRC | 15:04 | |
| AJaeger | ansiwen: submit the change and add me as reviewer ;) | 15:04 |
| ansiwen | AJaeger: so do I remove the antry altogether, or set it to the current release (2.3.4) | 15:04 |
| AJaeger | ansiwen: in upper-constraints: Set it to 2.3.4 | 15:05 |
| ansiwen | AJaeger: ok | 15:05 |
| tonytan4ever | Hi folks, I encountered a problem when installing oslo-config: AssertionError: oslo-config .dist-info directory not found | 15:05 |
| tonytan4ever | Is there anything changed with oslo-config ? | 15:05 |
| *** EricGonc_ has joined #openstack-infra | 15:05 | |
| AJaeger | tonytan4ever: I suggest to ask on the oslo channel... | 15:05 |
| tonytan4ever | Okay thank. | 15:06 |
| AJaeger | tonytan4ever: #openstack-oslo | 15:06 |
| jeblair | yolanda: using iftop, i can see an 11mbit stream from dfw to ord on the afs3-volser port. so i believe that the current vos release is still running. | 15:06 |
| *** heyongli_ has quit IRC | 15:07 | |
| yolanda | ok so just slowness caused by a wrong volume state then? | 15:07 |
| jeblair | yolanda: if it finishes before its tokens expire, everything will be fine. if it takes longer than that, we will probably need to manually vos release on the fileserver | 15:07 |
| *** heyongli_ has joined #openstack-infra | 15:07 | |
| yolanda | ok understood | 15:07 |
| jeblair | yolanda: i suspect we may have a *lot* of changes in pypi because of the pathname change... | 15:07 |
| jeblair | so we may have a couple of problems compounding -- a lot of changes to sync on top of a failed release at an unlucky time. | 15:08 |
| yolanda | jeblair, what is length of token expiration? | 15:08 |
| jeblair | yolanda: 10 hours by default, i think? | 15:08 |
| yolanda | release would take more... | 15:09 |
| openstackgerrit | Sven Anderson proposed openstack/requirements: Require Babel>=2.3.4 https://review.openstack.org/311761 | 15:09 |
| *** EricGonczer_ has quit IRC | 15:09 | |
| jeblair | yolanda: it looks like that is about 36 minutes from now, yeah? | 15:09 |
| *** hrybacki_ has joined #openstack-infra | 15:09 | |
| jeblair | yolanda: so i think we should expect it to fail | 15:09 |
| *** simonmcc_ has joined #openstack-infra | 15:09 | |
| *** rackertom_ has joined #openstack-infra | 15:10 | |
| *** |GonZo2000| has joined #openstack-infra | 15:10 | |
| jeblair | yolanda: i think we should disable puppet on mirror-update, then disable the bandersnatch cron entry, then wait for it to fail, then manually grab the bandersnatch lock, then vos release on the fileserver in screen, then re-enable the cronjob and puppet on mirror-update | 15:10 |
| jeblair | that way we have control over the vos release (because of our holding the lockfile) | 15:11 |
| *** burgerk_ has joined #openstack-infra | 15:11 | |
| *** jaranovich_ has joined #openstack-infra | 15:11 | |
| *** petems_ has joined #openstack-infra | 15:11 | |
| *** charz_ has joined #openstack-infra | 15:11 | |
| jeblair | and when it's done, we can run bandersnatch and vos release manually until we are happy | 15:11 |
| *** EricGonc_ has quit IRC | 15:11 | |
| jeblair | and, for the future to make this more robust -- i think we should stop doing vos releases on the mirror worker, and instead have a gearman job running on the fileserver to do them, so they don't timeout | 15:12 |
| jeblair | if that sounds good, i can do the steps i outlined above | 15:12 |
| *** f1ller has joined #openstack-infra | 15:12 | |
| yolanda | jeblair, a gearman job instructed by zuul? | 15:13 |
| *** r-mibu_ has joined #openstack-infra | 15:13 | |
| jeblair | yolanda: that would be ideal, but doesn't currently work for the mirror-update jobs (though it would for the wheel mirror jobs) | 15:13 |
| *** links has joined #openstack-infra | 15:14 | |
| *** links has quit IRC | 15:14 | |
| jeblair | yolanda: an alternative would actually be to make a job queue in afs (files in a directory) | 15:14 |
| *** bryan_att_ has joined #openstack-infra | 15:14 | |
| *** spiffxp_ has joined #openstack-infra | 15:14 | |
| * yolanda needs to learn more about afs, just reading about it these days | 15:14 | |
| yolanda | jeblair, your steps look sane to me | 15:14 |
| *** cargonza_ has joined #openstack-infra | 15:15 | |
| *** rackertom has quit IRC | 15:15 | |
| jeblair | https://etherpad.openstack.org/p/fix-afs | 15:15 |
| *** rackertom_ is now known as rackertom | 15:15 | |
| *** phschwartz_ has joined #openstack-infra | 15:15 | |
| jeblair | okay, i will start on those now | 15:15 |
| *** lindsayk- has joined #openstack-infra | 15:15 | |
| nibalizer | god morning | 15:15 |
| nibalizer | er good morning eep | 15:16 |
| * nibalizer nits jeblair's etherpad | 15:16 | |
| *** mtreinish_ has joined #openstack-infra | 15:16 | |
| * jeblair names nibalizer in etherpad | 15:16 | |
| *** dirk_ has joined #openstack-infra | 15:17 | |
| *** mriedem1 has joined #openstack-infra | 15:17 | |
| nibalizer | hah | 15:17 |
| *** heyongli_ has quit IRC | 15:17 | |
| *** tpsilva_ has joined #openstack-infra | 15:17 | |
| nibalizer | in all seriousness i can run this procedure if you want | 15:17 |
| *** sulo_ has joined #openstack-infra | 15:17 | |
| *** ashtokolov_ has joined #openstack-infra | 15:17 | |
| jeblair | cool | 15:18 |
| *** cfarquhar has joined #openstack-infra | 15:18 | |
| *** cfarquhar has quit IRC | 15:18 | |
| *** cfarquhar has joined #openstack-infra | 15:18 | |
| jeblair | i have done the first 2 steps | 15:18 |
| jeblair | next is wait... | 15:18 |
| *** fdegir_ has joined #openstack-infra | 15:18 | |
| jeblair | so i think it will be good for others to be able to pick up the process | 15:18 |
| jeblair | so yeah, let's coordinate in here and on the etherpad | 15:18 |
| jeblair | #status log steps to fix the pypi mirror problem in progress: https://etherpad.openstack.org/p/fix-afs | 15:19 |
| openstackstatus | jeblair: finished logging | 15:19 |
| jeblair | i'm going to grab breakfast | 15:19 |
| *** maximov_ has joined #openstack-infra | 15:19 | |
| *** psachin has quit IRC | 15:20 | |
| *** dangers has joined #openstack-infra | 15:20 | |
| *** davidlenwell has quit IRC | 15:20 | |
| yolanda | thanks for helping with that, using my mobile connection now so i cannot volunteer to run anything essential | 15:20 |
| *** skolekonov_ has joined #openstack-infra | 15:20 | |
| *** doug-fis_ has joined #openstack-infra | 15:21 | |
| nibalizer | yah thanks jeblair | 15:21 |
| *** mnencia_ has joined #openstack-infra | 15:21 | |
| *** charz has quit IRC | 15:21 | |
| *** dtantsur|afk has quit IRC | 15:21 | |
| *** hrybacki has quit IRC | 15:21 | |
| *** simonmcc has quit IRC | 15:21 | |
| *** GonZo2000 has quit IRC | 15:21 | |
| *** mancdaz has quit IRC | 15:21 | |
| *** jaranovich has quit IRC | 15:21 | |
| *** petems has quit IRC | 15:21 | |
| *** lindsayk has quit IRC | 15:21 | |
| *** mancdaz has joined #openstack-infra | 15:21 | |
| *** filler_ has quit IRC | 15:21 | |
| *** bryan_att has quit IRC | 15:21 | |
| *** briancline has quit IRC | 15:21 | |
| *** dirk has quit IRC | 15:21 | |
| *** briancline has joined #openstack-infra | 15:21 | |
| *** burgerk has quit IRC | 15:21 | |
| *** whoops has quit IRC | 15:21 | |
| *** clarkb has quit IRC | 15:21 | |
| *** r-mibu has quit IRC | 15:21 | |
| *** skolekonov|away has quit IRC | 15:21 | |
| *** tpsilva has quit IRC | 15:21 | |
| *** cargonza has quit IRC | 15:21 | |
| *** mtreinish has quit IRC | 15:21 | |
| *** mnencia has quit IRC | 15:21 | |
| *** rcernin has quit IRC | 15:21 | |
| *** cfarquhar_ has quit IRC | 15:21 | |
| *** msuriar has quit IRC | 15:21 | |
| *** phschwartz has quit IRC | 15:21 | |
| *** spiffxp has quit IRC | 15:21 | |
| *** ashtokolov has quit IRC | 15:21 | |
| *** fdegir has quit IRC | 15:21 | |
| *** maximov has quit IRC | 15:21 | |
| *** sulo has quit IRC | 15:21 | |
| *** briancline has quit IRC | 15:21 | |
| *** weshay has quit IRC | 15:21 | |
| *** dangers_` has quit IRC | 15:21 | |
| *** mancdaz has quit IRC | 15:21 | |
| *** zul has quit IRC | 15:21 | |
| *** lindsayk_ has quit IRC | 15:21 | |
| *** reed has quit IRC | 15:21 | |
| *** timfreund has quit IRC | 15:21 | |
| *** mriedem has quit IRC | 15:21 | |
| *** jlvillal has quit IRC | 15:21 | |
| *** mtreinish_ is now known as mtreinish | 15:21 | |
| *** jaranovich_ is now known as jaranovich | 15:21 | |
| *** dtantsur has joined #openstack-infra | 15:21 | |
| *** jlvillal has joined #openstack-infra | 15:21 | |
| *** reed has joined #openstack-infra | 15:22 | |
| *** doug-fish has quit IRC | 15:22 | |
| *** dtantsur has quit IRC | 15:22 | |
| *** dtantsur has joined #openstack-infra | 15:22 | |
| *** mnencia_ is now known as mnencia | 15:22 | |
| *** rcernin has joined #openstack-infra | 15:22 | |
| *** simonmcc_ is now known as simonmcc | 15:22 | |
| *** briancline has joined #openstack-infra | 15:22 | |
| *** heyongli_ has joined #openstack-infra | 15:22 | |
| *** weshay has joined #openstack-infra | 15:22 | |
| *** tpsilva_ is now known as tpsilva | 15:22 | |
| *** hrybacki_ is now known as hrybacki | 15:23 | |
| *** maximov_ is now known as maximov | 15:23 | |
| *** ashtokolov_ is now known as ashtokolov | 15:23 | |
| *** mancdaz has joined #openstack-infra | 15:23 | |
| *** dirk_ is now known as dirk | 15:23 | |
| *** fdegir_ is now known as fdegir | 15:23 | |
| *** cargonza_ is now known as cargonza | 15:23 | |
| *** zul has joined #openstack-infra | 15:24 | |
| *** petems_ is now known as petems | 15:24 | |
| *** bryan_att_ is now known as bryan_att | 15:24 | |
| *** kmartin has joined #openstack-infra | 15:25 | |
| *** lindsayk_ has joined #openstack-infra | 15:26 | |
| *** heyongli_ has quit IRC | 15:27 | |
| jeblair | infra-root: my availability will be severely limited this week, sorry. | 15:27 |
| *** amitgandhinz has joined #openstack-infra | 15:28 | |
| *** heyongli_ has joined #openstack-infra | 15:28 | |
| *** davidlenwell has joined #openstack-infra | 15:28 | |
| *** armax has joined #openstack-infra | 15:28 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Add baymodel list, get and search call for shade+magnum https://review.openstack.org/308009 | 15:29 |
| *** nadya has joined #openstack-infra | 15:29 | |
| anteaya | jeblair: hope it is because you are doing things you enjoy | 15:29 |
| *** rcernin has quit IRC | 15:30 | |
| jeblair | AJaeger, pleia2, StevenK: i got the BEST email on friday: http://paste.openstack.org/show/495881/ :) | 15:30 |
| cbader | mmedvede, I can't get into the openstack-third-party-ci chat. | 15:30 |
| StevenK | jeblair: \o/ | 15:30 |
| mmedvede | cbader: ok, can discuss here | 15:30 |
| StevenK | I guess I can destroy my Transifex account too | 15:31 |
| cbader | mmedvede, ok | 15:31 |
| *** mriedem1 is now known as mriedem | 15:31 | |
| cbader | mmedvede, I was looking at the patch https://review.openstack.org/#/c/311080/ | 15:32 |
| cbader | mmedvede, sorry have to step away for second boss here | 15:33 |
| *** qb_ has joined #openstack-infra | 15:34 | |
| *** yolanda has quit IRC | 15:35 | |
| *** salv-orlando has joined #openstack-infra | 15:36 | |
| *** nbloom has joined #openstack-infra | 15:36 | |
| mmedvede | cbader: ok, so I am checking the latest nodepool build logs, and it looks to be fixed. Last ffi.h errors were last seen yesterday | 15:36 |
| *** heyongli_ has quit IRC | 15:37 | |
| nbloom | Hi, can someone help me. A custom job I configured seems to fail somewhere on the gate and I can't figure out what is the cause. Log under http://logs.openstack.org/21/295021/4/experimental/gate-vitrage-dsvm-nv/a7fc25c/ | 15:37 |
| *** heyongli_ has joined #openstack-infra | 15:38 | |
| *** chlong has quit IRC | 15:38 | |
| nbloom | sorry, on the experimental pipeline, not on the gate ** | 15:38 |
| *** mriedem has quit IRC | 15:38 | |
| *** kzaitsev_mb has joined #openstack-infra | 15:39 | |
| *** whoops has joined #openstack-infra | 15:39 | |
| jlk | Hi, I need some assistance. I have a project, giftwrap, and currently it is set to run "python-jobs" and "pypi-jobs", and it also uses templates "merge-check, python-jobs, publish-to-pypi, and python3-jobs". HOwever in one of those it's calling a docs jobs, and I'd like it to not call that job. What's the right way to opt out of the docs jobs? | 15:39 |
| *** mriedem has joined #openstack-infra | 15:40 | |
| pabelanger | jeblair: I likely added to the confusion for afs mirrors. Yolanda asked me to vos release which me not fully understanding what was going on, as such I unlocked the volume when a transaction was going on. From everything I was seeing / reading up on, it appeared the AFS mirror was restoring from the last know good transaction | 15:40 |
| anteaya | nbloom: so if you look here: http://logs.openstack.org/21/295021/4/experimental/gate-vitrage-dsvm-nv/a7fc25c/console.html#_2016-05-02_15_21_51_984 | 15:41 |
| cbader | mmedvede, so you have a successful dib-image build | 15:41 |
| anteaya | nbloom: you will see that devstack failed | 15:41 |
| anteaya | and logs are in logs/devstacklog.txt.gz | 15:41 |
| anteaya | nbloom: when you then look at logs/devstacklog.txt.gz | 15:41 |
| *** msuriar has joined #openstack-infra | 15:42 | |
| anteaya | nbloom: you see http://logs.openstack.org/21/295021/4/experimental/gate-vitrage-dsvm-nv/a7fc25c/logs/devstacklog.txt.gz#_2016-05-02_15_21_55_206 | 15:42 |
| pleia2 | jeblair: yeah :) you should have seen the folks in the i18n work session, they were pretty excited about finally getting to delete it | 15:42 |
| mmedvede | cbader: no, but it works for infra team. Might be due to https://review.openstack.org/#/c/311080/ now installing those missing libraries. My configuration is different, so I might have to install the dependencies somewhere else | 15:42 |
| anteaya | nbloom: it also looks like your current build is having trouble cloning: http://logs.openstack.org/21/295021/4/experimental/gate-vitrage-dsvm-nv/a7fc25c/logs/devstacklog.txt.gz#_2016-05-02_15_21_54_202 | 15:42 |
| pleia2 | (also, my time this week is limited as well) | 15:43 |
| anteaya | pleia2: I hope you too are doing things you enjoy | 15:43 |
| *** fedexo has joined #openstack-infra | 15:43 | |
| *** pilgrimstack has quit IRC | 15:44 | |
| cbader | mmedvede, so I will have to fix somewhere else and not use upstream for infra team? | 15:44 |
| mmedvede | cbader: we could still try adding the dependencies in puppet manfiest to make them explicit | 15:44 |
| cbader | mmedvede, sorry which puppet manifest? | 15:45 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Revert "Create stackviz element and script in nodepool" https://review.openstack.org/311768 | 15:45 |
| cbader | mmedvede, the patch https://review.openstack.org/#/c/311080/ is the one I was looking at to see if I could add that somewhere. | 15:46 |
| pabelanger | infra-root: I propose we revert stackviz element as our DIBs are broken as a result^. Logs can be found on http://nodepool.openstack.org | 15:46 |
| nbloom | anteaya thanks :) I'll check this out though I don't really understand how this can be as I use the same configuration when running devstack ./stack.sh on my VM... | 15:46 |
| *** sputnik13 has joined #openstack-infra | 15:46 | |
| *** e0ne has quit IRC | 15:47 | |
| pabelanger | project-config-core as well ^ | 15:47 |
| anteaya | nbloom: well if you have set cloning to pull from the wrong source our system will error our | 15:47 |
| *** kzaitsev_mb has quit IRC | 15:47 | |
| anteaya | nbloom: the same won't happen locally for you | 15:47 |
| *** heyongli_ has quit IRC | 15:48 | |
| *** bpokorny has joined #openstack-infra | 15:48 | |
| *** heyongli_ has joined #openstack-infra | 15:48 | |
| nbloom | anteaya | 15:48 |
| nbloom | anteaya | 15:48 |
| nbloom | anteaya then it's a problem with the cloning source? | 15:48 |
| anteaya | nbloom: yes | 15:48 |
| nbloom | anteaya thanks | 15:49 |
| anteaya | nbloom: https://git.openstack.org/cgit/openstack-dev/devstack/tree/functions-common#n516 the Cloning not allowed in this configuration error comes from two loops | 15:49 |
| anteaya | nbloom: welcome | 15:49 |
| nibalizer | pabelanger: i don't see the explossion | 15:50 |
| nibalizer | on ubuntu-trusty | 15:50 |
| nibalizer | jeblair: as for waiting for bandersnatch to fail... it hasn't written anything to its log file in the 12 hours it's been running | 15:51 |
| nibalizer | so if we're just waiting for that process to leave 'ps' then that's cool | 15:51 |
| *** vincentll has quit IRC | 15:51 | |
| anteaya | jlk: the python-jobs group has a docs job: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/python-jobs.yaml#n479 | 15:52 |
| jlk | anteaya: yeah, I'd like to opt-out of that particular job in the group | 15:52 |
| anteaya | jlk: the best way to not run the docs job is in zuul/layout.yaml specifiy which of the python jobs you wish to run and don't mention the docs job | 15:52 |
| jlk | I'm not sure if I should make a new group, make a new template, or transpose all the jobs into my project job | 15:52 |
| pabelanger | nibalizer: can't link but here is the error in the log | 15:53 |
| pabelanger | nibalizer: 2016-05-02 15:13:33,861 INFO nodepool.image.build.ubuntu-trusty: /usr/bin/env: node: No such file or directory | 15:53 |
| *** pilgrimstack has joined #openstack-infra | 15:53 | |
| pabelanger | each OS is failing in a different way, but all related to stackviz | 15:53 |
| jlk | anteaya: so I would take out "python-jobs" from the template: block of my project, and instead write out the check/gate/whatever jobs from that template that I would want? | 15:53 |
| jeblair | nibalizer: yes; also, i believe a log rotation/compression happened underneath the bandersnatch process, so when it does fail, we may not see anything | 15:53 |
| jlk | anteaya: (this is the other confusing part, in my job config it' slisted as a template, but it's also a job group? | 15:53 |
| pabelanger | 2016-05-02 14:27:33,193 INFO nodepool.image.build.centos-7: sudo: npm: command not found is another | 15:54 |
| anteaya | jlk: yes in the zuul/layout.yaml file | 15:54 |
| *** apoorvad has joined #openstack-infra | 15:54 | |
| anteaya | jlk: can you link your job config please? | 15:54 |
| anteaya | if I am to respond to a definition question | 15:55 |
| cbader | mmedvede, so looks like wheel_mirror_slave.pp already has libffi-dev under ensure_packages but missing openssl-devel | 15:55 |
| jlk | anteaya: https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n6351 | 15:55 |
| fungi | pabelanger: thanks for spotting. i was trying to see if it was succeeding but it was previously masked by the zuul-env build issues with paramiko 2.0.0 | 15:55 |
| anteaya | thank you | 15:56 |
| mmedvede | cbader: that is not the manifest that is used to configure jenkins slaves | 15:56 |
| cbader | mmedvede, oh | 15:56 |
| *** yaume_ has joined #openstack-infra | 15:56 | |
| anteaya | jlk: right now in zuul/layout.yaml you are using templates | 15:56 |
| cbader | mmedvede, which on is | 15:56 |
| mmedvede | cbader: look at single_use_slave.pp | 15:56 |
| jlk | anteaya: correct | 15:56 |
| *** yaume has quit IRC | 15:56 | |
| nibalizer | jeblair: makes sense | 15:57 |
| *** ddieterly is now known as ddieterly[away] | 15:57 | |
| anteaya | jlk: in the jenkins/jobs/python-jobs.yaml file: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/python-jobs.yaml#n479 | 15:57 |
| *** lakshmiS has joined #openstack-infra | 15:57 | |
| mmedvede | cbader: it uses slave_common.pp. The ffi.h error did happen when zuul env was being installed | 15:57 |
| *** abregman has quit IRC | 15:58 | |
| anteaya | jlk: python-jobs is a job group, consisting of a number of jobs, which have been defined above in that file | 15:58 |
| *** heyongli_ has quit IRC | 15:58 | |
| AJaeger | jeblair: it was a pleasure ;) | 15:58 |
| cbader | mmedvede, I see that | 15:58 |
| *** dmk0202 has quit IRC | 15:59 | |
| *** yolanda has joined #openstack-infra | 15:59 | |
| pabelanger | mmedvede: https://review.openstack.org/#/c/311734/ is the path I am testing right now. that should properly address the build dependencies for zuul, since we use diskimage-builder to install it | 15:59 |
| *** Guest37590 has quit IRC | 15:59 | |
| jlk | anteaya: okay, so my use of "python-jobs" doesn't relate to https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n294 at all? | 15:59 |
| AJaeger | nbloom: read http://jaegerandi.blogspot.de/2016/02/templates-in-openstacks-zuul.html | 15:59 |
| AJaeger | nbloom: sorry, that was meant for jlk | 15:59 |
| AJaeger | jlk, read http://jaegerandi.blogspot.de/2016/02/templates-in-openstacks-zuul.html | 15:59 |
| jlk | well now I'm confused | 16:00 |
| jlk | am I using a template like AJaeger is saying, or a job group like anteaya is saying? | 16:00 |
| fungi | jlk: you're conflating jenkins configuration with zuul configuration. two pieces of software, two sets of configuration | 16:01 |
| jlk | le sigh | 16:01 |
| mmedvede | cbader: see pabelanger 's comment. That is better way to manage the dependency if you use dib | 16:01 |
| AJaeger | jlk: Jenkins != Zuul. Jenkins has job-groups, Zuul has templates. Two different kind of languages | 16:01 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add support for auth-type=form https://review.openstack.org/169467 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add matchers for external references on launchpad https://review.openstack.org/295592 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Use short titles in breadcrumbs https://review.openstack.org/310036 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Make 'title' attribute available on dialog widgets https://review.openstack.org/289748 | 16:02 |
| *** lakshmiS has quit IRC | 16:02 | |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add navigation breadcrumb footer https://review.openstack.org/288250 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add extra columns to change list if there is room https://review.openstack.org/310037 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Fix rendering error in change screen https://review.openstack.org/295423 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add configuration information to docs https://review.openstack.org/286240 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add an option to disable the breadcrumb footer https://review.openstack.org/311770 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Show potential completions https://review.openstack.org/311771 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Make change list searchable https://review.openstack.org/311772 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Make project list searchable https://review.openstack.org/311773 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add a short title to diff screen https://review.openstack.org/311774 | 16:02 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add form authentication info to docs https://review.openstack.org/311775 | 16:02 |
| jlk | well, it's pretty unclear to me what I need to do to "make my project work" | 16:02 |
| jlk | my project doesn't work. It runs a job that doesn't work. I'd like to not run that job. | 16:02 |
| AJaeger | jeblair: two more and my screen is scrolled off ;) | 16:02 |
| jeblair | AJaeger: i had a good flight | 16:02 |
| *** kjorgensen has joined #openstack-infra | 16:02 | |
| AJaeger | jlk: read my blog post, it explains it bottom up ;) | 16:02 |
| AJaeger | jeblair: great! | 16:02 |
| fungi | AJaeger: more specifically, zuul has "project-templates" | 16:03 |
| jlk | AJaeger: except it doesn't explain at all "how do I use this thing, but opt out of that thing" | 16:03 |
| jlk | or what I should do if one job from a template is not desired | 16:03 |
| *** heyongli_ has joined #openstack-infra | 16:03 | |
| anteaya | jlk: I already told you above | 16:03 |
| anteaya | jlk: make a cup of tea and read what has been suggested again | 16:03 |
| fungi | jlk: in the zuul configuration you need to manually expand the project-template you're using and omit any undesired entries | 16:04 |
| AJaeger | jlk: it does it the other way round: It starts with no jobs and adds them and then finally uses the template... | 16:04 |
| jlk | alright, here's what's unclear though. | 16:04 |
| *** thorst has quit IRC | 16:04 | |
| jlk | anteaya told me to read about the job groups | 16:04 |
| jlk | but it seems like AJaeger is telling me to expand the template | 16:04 |
| anteaya | jlk: I told you to specify the jobs you want in zuul/layout.yaml | 16:04 |
| anteaya | and omit the job you don't want | 16:05 |
| anteaya | as fungi has also said | 16:05 |
| jlk | [10:52:05] <anteaya>jlk: the python-jobs group has a docs job: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/python-jobs.yaml#n479 | 16:05 |
| * AJaeger needs to step out a bit... | 16:05 | |
| jlk | [10:52:38] <anteaya>jlk: the best way to not run the docs job is in zuul/layout.yaml specifiy which of the python jobs you wish to run and don't mention the docs job | 16:05 |
| anteaya | yeah I'm out too | 16:05 |
| * anteaya makes a cup of tea | 16:05 | |
| jlk | I'm sorry, that read to me that I need to look at the job group and expand that | 16:05 |
| jlk | I'm sorry for not getting it. | 16:05 |
| * AJaeger makes family dinner... | 16:05 | |
| *** EricGonczer_ has joined #openstack-infra | 16:05 | |
| cbader | mmedvede, I am looking at it. | 16:05 |
| *** sridhar_ram has joined #openstack-infra | 16:05 | |
| *** lakshmiS has joined #openstack-infra | 16:06 | |
| *** kjorgensen has quit IRC | 16:06 | |
| *** kjorgensen has joined #openstack-infra | 16:06 | |
| jeblair | jlk: i think you've got it -- this is correct: 15:53 < jlk> anteaya: so I would take out "python-jobs" from the template: block of my project, and instead write out the check/gate/whatever jobs from that template that I would want? | 16:06 |
| jlk | jeblair: thanks, I think I was thrown off-course by the first mention of the python-jobs file, which is a job group. | 16:07 |
| jeblair | jlk: then i think the conversation spiraled into helpfulness :) | 16:07 |
| *** salv-orlando has quit IRC | 16:08 | |
| *** EricGonczer_ has quit IRC | 16:08 | |
| *** heyongli_ has quit IRC | 16:08 | |
| jeblair | jlk: out of curiosity, why no docs? | 16:08 |
| jlk | I just don't have infra right now to build them out, that'll come soonish | 16:09 |
| jlk | but having the docs job is blocking any chnage from coming in | 16:09 |
| *** heyongli_ has joined #openstack-infra | 16:09 | |
| jeblair | gotcha | 16:09 |
| fungi | jlk: an example would be openstack-infra/lodgeit which runs python27 and pep8 but not docs: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n3257 | 16:10 |
| *** hughhalf has quit IRC | 16:10 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/project-config: Opt out of docs jobs for giftwrap https://review.openstack.org/311778 | 16:10 |
| fungi | so instead of using the project-template named "python-jobs" in the zuul layout, it just specifies which jobs from that project-template it actually needs/can use | 16:10 |
| sdake | so, kolla wnats to implemenet ~80 gates for kolla | 16:10 |
| sdake | short running jobs - 20 mins | 16:10 |
| *** EricGonczer_ has joined #openstack-infra | 16:10 | |
| sdake | will that be a problem? | 16:10 |
| fungi | sdake: 80 gates? i really hope you mean 80 jobs | 16:11 |
| jlk | fungi: okay, that looks like what I just submitted | 16:11 |
| sdake | define job for me fungi :) | 16:11 |
| * fungi worries "gate" the new "smurf" and we are going to gate the gates with gateity gating | 16:11 | |
| fungi | sdake: a "job" is a work unit currently defined in our jenkins configuration, which runs a series of commands and reports back a result | 16:11 |
| sdake | yes 80 jobs | 16:12 |
| *** jordanP has quit IRC | 16:12 | |
| sdake | we have 1 gat e;) | 16:12 |
| sdake | well 2 if yo ucount the humans ;) | 16:12 |
| *** links has joined #openstack-infra | 16:12 | |
| fungi | sounds exciting. i hope they succeed at least 999 times out of 1000 or you're probably never going to be able to merge changes easily | 16:12 |
| sdake | fungi all our jobs are non-voting | 16:13 |
| sdake | becuase we lack mirrors in infra repo for our upstream stuff | 16:13 |
| sdake | the work has started here in infra | 16:13 |
| sdake | but incomplete | 16:13 |
| *** kzaitsev_mb has joined #openstack-infra | 16:13 | |
| fungi | ahh, cool. nondeterministic failures in job results become crippling the more voting jobs you run | 16:14 |
| sdake | fugni right - we want to get to the point wher ewe have 1 voting job for functional teting | 16:14 |
| sdake | the rest would be sort of advisory ;) | 16:14 |
| sdake | but that wont implode infra I assume? | 16:14 |
| sdake | (the 80 jobs0 | 16:14 |
| *** gyee has joined #openstack-infra | 16:14 | |
| *** gyee has quit IRC | 16:15 | |
| *** tqtran has joined #openstack-infra | 16:15 | |
| cbader | mmedvede, the issue pabelanger is for dependencies for zuul will this work for dependencies on dib builder? | 16:17 |
| *** sulo_ is now known as sulo | 16:18 | |
| *** ddieterly[away] is now known as ddieterly | 16:18 | |
| *** EricGonczer_ has quit IRC | 16:18 | |
| anteaya | sdake: infra would not implode, your patches will take a very long time to get all the nodes it needs to finish testing | 16:18 |
| *** fawadkhaliq has joined #openstack-infra | 16:18 | |
| *** heyongli_ has quit IRC | 16:18 | |
| jeblair | sdake: that's about 26 machine-hours, or about 2x the amount of time we spend testing a change to nova | 16:18 |
| sdake | anteaya could you put a WAG on that in terms of time | 16:18 |
| *** EricGonczer_ has joined #openstack-infra | 16:19 | |
| anteaya | sdake: well I don't know what a WAG is, I don't acroymn | 16:19 |
| sdake | wild ass guess | 16:19 |
| anteaya | but I'm going to guess jeblair's estimate is useful to your question | 16:19 |
| anteaya | well I don't like that language so wouldn't use it | 16:19 |
| anteaya | but again I think jeblair's statement is helpful here | 16:20 |
| *** hughhalf has joined #openstack-infra | 16:20 | |
| *** heyongli_ has joined #openstack-infra | 16:21 | |
| mmedvede | cbader: your dib build fails because zuul dependencies fail to install, right? If so, that should fix the problem. | 16:21 |
| *** Swami has joined #openstack-infra | 16:22 | |
| cbader | mmedvede, I made the fix on ci and I am trying the build will see if it works thanks for the help. | 16:22 |
| fungi | cbader: not sure which "this" you're referring to, but for the moment zuul is declaring itself incompatible with the new version of paramiko which now drags in cryptography (for other reasons) | 16:22 |
| cbader | fungi, thank you for the clarification, I am still new to this and haven't figured out all the connections. | 16:22 |
| pabelanger | and, if other projects are getting hit with it, updating bindep-fallback.txt is also an option | 16:23 |
| *** jistr has quit IRC | 16:23 | |
| fungi | when zuul eventually gets patched to support paramiko 2.0.0 (assuming it does, rather than switching to a different ssh implementation) then we'll likely need to revisit the indirect libffi-dev and openssl-dev dependencies | 16:23 |
| fungi | pabelanger: i expect any jobs which are running on our workers won't see this unless they're not using our wheel mirrors | 16:24 |
| *** asselin_ has joined #openstack-infra | 16:24 | |
| fungi | those header packages are needed to build the cryptography package, but using a prebuilt wheel of cryptography instead sidesteps building | 16:24 |
| *** gyee has joined #openstack-infra | 16:25 | |
| pabelanger | fungi: exactly | 16:25 |
| mmedvede | yes, our third-party CI was hit with it, we are not using pre-built wheels | 16:25 |
| *** yaume_ has quit IRC | 16:25 | |
| jeblair | sdake: i think the answer to your question depends on how often changes are proposed to your project, and how busy the system is with other projects. an occasional change launching 80 jobs isn't terrible. if kolla has many changes, each of which consumes 26-machine hours, it will have an impact on other projects and itself. | 16:26 |
| fungi | mmedvede: hit with it running jobs, or just hit with it building images? | 16:26 |
| cbader | fungi, with jobs and building images for me. | 16:26 |
| fungi | cbader: which jobs/projects? i'm curious to see how they address it | 16:27 |
| jeblair | sdake: our current budget is 19392 machine-hours-per-day. (this is not an SI unit, but it should be). overall, we are at about half-capacity. some weeks we are at capacity. | 16:27 |
| mmedvede | fungi: both for me too | 16:27 |
| cbader | fungi, noticed on friday all jobs against cinder patches started failing, looked around and notice my dib builde hadn't work since thursday night. go fix for vms friday night. | 16:28 |
| mmedvede | fungi: I patched jobs right away, now fixing image builds. We are not using dibs, and have our own version of system-config, so it is easy local fix | 16:28 |
| *** xarses_ is now known as xarses | 16:28 | |
| *** thorst has joined #openstack-infra | 16:29 | |
| *** heyongli_ has quit IRC | 16:29 | |
| *** xnox has quit IRC | 16:29 | |
| *** heyongli_ has joined #openstack-infra | 16:29 | |
| fungi | mtreinish: timothyb89: (i don't see austin81 in here at the moment) do you have time to dig into the stackviz build issues we're seeing in our image updates? i'm going to approve the temporary revert in https://review.openstack.org/311768 and we can reintroduce it once you iron out those kinks | 16:29 |
| cbader | fungi, I looked in log for image build and saw libffi error, but didn't know how to integrate vm fix to that process. | 16:29 |
| *** ifarkas has quit IRC | 16:29 | |
| *** EricGonczer_ has quit IRC | 16:30 | |
| pabelanger | fungi: do you mind looking at 311768? We need to revert stackviz to merge our zuul dependency issue | 16:32 |
| pabelanger | anteaya: thanks for the reviewing too | 16:32 |
| pabelanger | woah | 16:32 |
| anteaya | pabelanger: sure | 16:33 |
| fungi | pabelanger: see my above pings to mtreinish and timothyb89 | 16:33 |
| *** sputnik13_ has joined #openstack-infra | 16:33 | |
| *** sputnik13 has quit IRC | 16:33 | |
| pabelanger | fungi: ack | 16:33 |
| anteaya | pabelanger: thanks for figuring out the issue and offering the revert | 16:33 |
| pabelanger | anteaya: fungi: I think we're need some more test coverage for stackviz. Once I have some spare bandwidth, I'll see if we can add something into the gate for all our OSes | 16:34 |
| *** nbloom has quit IRC | 16:35 | |
| mtreinish | fungi: I am around now | 16:35 |
| *** ccarmack has joined #openstack-infra | 16:36 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Create stackviz element and script in nodepool" https://review.openstack.org/311768 | 16:38 |
| anteaya | pabelanger: um okay | 16:38 |
| *** amrith is now known as _amrith_ | 16:39 | |
| dims | anteaya : we are trading some proposals on kolla project. one idea that came up was feature branches. can we have a separate ACL group for a specific feature branch? | 16:39 |
| dims | AJaeger : ^^ | 16:39 |
| *** heyongli_ has quit IRC | 16:39 | |
| *** heyongli_ has joined #openstack-infra | 16:39 | |
| anteaya | dims: repos have one acl, for each repo | 16:39 |
| AJaeger | dims, yes. Similar like on stable/ | 16:39 |
| anteaya | dims: now within one acl file you can specify different things, such as AJaeger's point | 16:40 |
| dims | ack. so it's possible :) | 16:40 |
| anteaya | but it is one acl file | 16:40 |
| anteaya | yes it is possible | 16:40 |
| dims | got it | 16:40 |
| *** yamahata has joined #openstack-infra | 16:40 | |
| dims | inc0 : ^^ | 16:40 |
| *** acabot has quit IRC | 16:40 | |
| inc0 | thanks dims, anteaya | 16:41 |
| inc0 | and AJaeger ;) | 16:41 |
| *** dprince has joined #openstack-infra | 16:42 | |
| dims | :) | 16:42 |
| AJaeger | inc0, dims: Just check how it's done for stable/ branches in projects | 16:42 |
| dims | AJaeger : yep | 16:42 |
| *** tphummel has joined #openstack-infra | 16:42 | |
| *** tonytan4ever has quit IRC | 16:44 | |
| dmsimard | Is there any chance we could pass the --prefix parameter to the meetbot irclogs2html implementation for eavesdrops so that it would include the channel name ? i.e, https://github.com/mgedmin/irclog2html/blob/cfefff0d191a17cbdab4bab4c99a638ce9d15105/src/irclog2html/logs2html.py#L198-L200 | 16:44 |
| dmsimard | For example, in http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ it would say "IRC logs for #openstack-infra" | 16:45 |
| *** harlowja has joined #openstack-infra | 16:45 | |
| mtreinish | fungi: looking at the logs it can't find the npm binary on redhat or node on ubuntu | 16:45 |
| mtreinish | which is odd, because I thought they were added to the package lists | 16:46 |
| fungi | mtreinish: unless that's running before package installation (or in the wrong context? chroot vs host?) | 16:46 |
| anteaya | dmsimard: I don't have any argument for not doing so, so I guess it comes down to whoever offers that patch and how they impliment it | 16:46 |
| *** pradk has quit IRC | 16:46 | |
| dmsimard | anteaya: I tried to look at how that specific script was called but couldn't find it .. all I saw was a cron setup in puppet-meetbot but it doesn't particularly speak to me | 16:47 |
| fungi | mtreinish: anyway, i mainly just don't want to iterate on it in production when each new attempt means another day of missing image updates | 16:47 |
| dmsimard | I'm willing to send a patch, just need to know where | 16:47 |
| *** xnox has joined #openstack-infra | 16:48 | |
| anteaya | dmsimard: have you looked at all at codesearch.openstack.org | 16:48 |
| anteaya | it may help you find the call to that script, if indeed there is a call | 16:48 |
| dmsimard | anteaya: yup: http://codesearch.openstack.org/?q=irclog2html&i=nope&files=&repos= | 16:49 |
| dmsimard | that's where I got the cron from | 16:49 |
| pabelanger | mtreinish: fungi: Maybe a path issue? I also wonder if it would be possible to not use sudo npm install, but rather sandbox our npm installs into a local sandbox. I admit, I don't have much experience with npm to know if possible | 16:49 |
| *** oanson has joined #openstack-infra | 16:49 | |
| anteaya | dmsimard: ah great, then at this point you know more than I | 16:49 |
| *** heyongli_ has quit IRC | 16:49 | |
| *** heyongli_ has joined #openstack-infra | 16:49 | |
| anteaya | dmsimard: have you looked at prior patches to the repo? | 16:49 |
| dmsimard | it looks like it could be hidden in supybot or something, there is a mention of ChannelLogger but it doesn't reveal a whole lot either http://codesearch.openstack.org/?q=ChannelLogger&i=nope&files=&repos= | 16:50 |
| anteaya | dmsimard: sometimes they can be helpful | 16:50 |
| EmilienM | pabelanger: is this workaround acceptable for ansible? https://review.openstack.org/#/c/311755/ | 16:50 |
| anteaya | dmsimard: that is a distinct possibility | 16:50 |
| EmilienM | pabelanger: otherwise, can I just submit a patch in OOO JJB? so we can quickly fix it | 16:51 |
| pabelanger | EmilienM: I think so, we can ask fungi what he thinks | 16:51 |
| mtreinish | pabelanger: the other idea I had was to add a post job to stackviz to build the static part and just have nodepool put that on the disk from the log server | 16:51 |
| *** daemontool has joined #openstack-infra | 16:52 | |
| fungi | pabelanger: mtreinish: worth looking at the npm installation macro in our jjb config for hints. i know there are some path differences too based on where the initial distro packaged npm installs the subsequent npm when bootstrapping | 16:52 |
| pabelanger | mtreinish: Personally, I am happy with that approach. Also makes debugging failures a little easier | 16:53 |
| *** fedexo has quit IRC | 16:53 | |
| fungi | pabelanger: EmilienM: i take it ansible also uses paramiko? | 16:54 |
| pabelanger | fungi: Ya | 16:54 |
| *** sputnik13_ is now known as sputnik13 | 16:54 | |
| fungi | i'm curious how they're handling the sudden header deps in their pip-installed ecosystem | 16:54 |
| *** acabot has joined #openstack-infra | 16:54 | |
| pabelanger | I was going to pop over to #ansible and see if they can also do a wheel | 16:55 |
| pabelanger | not sure that will fix all our issues | 16:55 |
| fungi | pabelanger: well, it's cryptography that would need to be wheeled, and the least-common-denominator "anylinux" wheels pypi recently started supporting intentionally punt on problematic libs like openssl | 16:56 |
| pabelanger | fungi: Ah, okay. | 16:56 |
| pabelanger | fungi: at first pass: https://review.openstack.org/#/c/311734/ works on ubuntu-trusty, testing centos-7 now. If we want to go down that path in the future | 16:57 |
| AJaeger | if any project-config core could please later review 311675 and 311701 for openstack-manuals, it would be great. Those changes are needed for us to move forward with getting rid of DocBook XML building | 16:58 |
| *** heyongli_ has quit IRC | 16:59 | |
| ccarmack | fungi: Hi, I'm working on nova cells v2, and we would like to add a multi-node job for cells. Right now there is a single-node job for cells, but multi-node would be more realistic. Does this seem ok? | 17:00 |
| ccarmack | alaski, bauzas ^ | 17:00 |
| *** hdd has joined #openstack-infra | 17:00 | |
| EmilienM | fungi, pabelanger: should I abandon my patch? | 17:01 |
| EmilienM | I'm trying to restablish tripleo CI which is broken for 2 days | 17:01 |
| *** hashar has quit IRC | 17:01 | |
| EmilienM | maybe I can just send a quick fix in ooo jjb, but I'm not sure if anyone else is affected | 17:02 |
| openstackgerrit | Sridhar Ramaswamy proposed openstack-infra/irc-meetings: Move Tacker team meeting to compensate DST https://review.openstack.org/311789 | 17:02 |
| *** heyongli_ has joined #openstack-infra | 17:02 | |
| pabelanger | EmilienM: is the devstack-gate failing on other clouds? I was under the impression everybody is affected | 17:03 |
| openstackgerrit | Jesse Keating proposed openstack-infra/project-config: Opt out of docs jobs for giftwrap https://review.openstack.org/311778 | 17:03 |
| EmilienM | pabelanger: I have no idea | 17:03 |
| *** oanson has quit IRC | 17:04 | |
| fungi | EmilienM: pabelanger: looks like ansible in at least some of our jobs is using paramiko 1.16.0? http://logs.openstack.org/43/308343/1/gate/gate-tempest-dsvm-trove/e89bc95/console.html#_2016-05-02_16_13_07_644 | 17:05 |
| *** sputnik13 has quit IRC | 17:05 | |
| pabelanger | EmilienM: could you link the tripleo failure again? | 17:06 |
| anteaya | AJaeger: I'm already +2 on 311701, going to have to wait for someone else to review | 17:07 |
| *** clarkb has joined #openstack-infra | 17:07 | |
| openstackgerrit | Merged openstack-infra/project-config: openstack-manuals: Adjust jobs https://review.openstack.org/311675 | 17:08 |
| *** pfallenop has quit IRC | 17:08 | |
| EmilienM | pabelanger: it's every jobs, let me find a link | 17:10 |
| *** heyongli_ has quit IRC | 17:10 | |
| EmilienM | pabelanger: http://logs.openstack.org/37/311737/2/check-tripleo/gate-tripleo-ci-f22-ha/88f25da/console.html#_2016-05-02_13_27_21_943 | 17:10 |
| *** sputnik13 has joined #openstack-infra | 17:10 | |
| *** heyongli_ has joined #openstack-infra | 17:10 | |
| *** yamahata has quit IRC | 17:10 | |
| *** yamahata has joined #openstack-infra | 17:11 | |
| *** oanson has joined #openstack-infra | 17:11 | |
| pabelanger | EmilienM: fungi: The issue is, tripleo is not using our wheel mirror. And since mirror.pypi is behind we are still pinned to paramiko 1.16.0. That explains why all our other jobs are not failing | 17:12 |
| openstackgerrit | Christopher Aedo proposed openstack-infra/project-config: Disable App Catalog periodic job https://review.openstack.org/311790 | 17:12 |
| EmilienM | pabelanger: I see | 17:12 |
| EmilienM | pabelanger: so can I just patch JJB in the meantime? so we can fix it very quickly in the meantime DIB is fixed? | 17:12 |
| *** pradk has joined #openstack-infra | 17:12 | |
| EmilienM | or should we go for the devstack gate patch? | 17:12 |
| EmilienM | I prefer option #1 since it only affects OOO CI | 17:13 |
| pabelanger | EmilienM: fungi: We should likely fix devstack-gate for tripleo, once our pypi mirror resync it will expose it. | 17:13 |
| *** trown is now known as trown|lunch | 17:13 | |
| *** pfallenop has joined #openstack-infra | 17:14 | |
| *** sputnik13 has quit IRC | 17:14 | |
| pabelanger | EmilienM: As a long term plan, I want to work towards common CI for tripleo. See https://review.openstack.org/#/c/311721/ for a first step towards that, replacing the fedora-22 jenkins slave with our centos-7 slave | 17:15 |
| pabelanger | EmilienM: will send out a ML post about that | 17:15 |
| *** markvan has quit IRC | 17:16 | |
| *** markvan has joined #openstack-infra | 17:16 | |
| fungi | pabelanger: EmilienM: yeah, so if `pip install ansible` doesn't work in vanilla cases right now i agree that devstack-gate installing ansible that way means we should make sure it also drags in any system package prerequisites | 17:16 |
| fungi | pabelanger: EmilienM: https://github.com/ansible/ansible/issues/15665 | 17:17 |
| *** johnny has joined #openstack-infra | 17:17 | |
| *** jerryz has joined #openstack-infra | 17:18 | |
| *** johnny is now known as Guest66363 | 17:18 | |
| fungi | which points out that we could manually install a lower version of paramiko in the virtualenv before installing ansible into it | 17:18 |
| *** sputnik13 has joined #openstack-infra | 17:19 | |
| pabelanger | fungi: right, we should infact see if ansible works with paramiko too | 17:19 |
| fungi | or just change the pip command to `/tmp/ansible/bin/pip install paramiko==1.16.0 ansible==$ANSIBLE_VERSION` presumably | 17:19 |
| pabelanger | paramiko 2.0.0 | 17:19 |
| mordred | funny thing is - ansible actually uses openssh for things we do | 17:19 |
| mordred | paramiko is the fallback for "is your system so broken that we can't find a usable ssh client" | 17:20 |
| *** piet has joined #openstack-infra | 17:20 | |
| *** heyongli_ has quit IRC | 17:20 | |
| *** heyongli_ has joined #openstack-infra | 17:20 | |
| fungi | right, similarly zuul could call out to /usr/bin/ssh rather than relying on paramiko? | 17:20 |
| anteaya | do we have any need to set any kind of limit on grafana dashboards? | 17:21 |
| *** sputnik13 has quit IRC | 17:21 | |
| mordred | it could, but that would be a decent amount of work | 17:21 |
| mordred | I'd suggest pinning paramiko for zuul | 17:21 |
| anteaya | either as a total quantity or on a per project basis? | 17:21 |
| anteaya | motivation for my question: https://review.openstack.org/#/c/307511/4 | 17:21 |
| fungi | mordred: agreed (and we already did for now as even with paramiko 2.0.0 successfully installed zuul breaks on it) | 17:21 |
| mordred | ++ | 17:22 |
| fungi | their api changed in non-backward-compatible ways in addition to adding install-time build deps on c library headers | 17:22 |
| mordred | yay! another success for semver | 17:22 |
| mordred | I mean, at least they _told_ us they were releasing a completely incompatible thing | 17:23 |
| anteaya | mordred: mr silver lining | 17:23 |
| pabelanger | clarkb: fungi: I've started a manually building new nodepool dibs | 17:23 |
| *** markvan has quit IRC | 17:23 | |
| mordred | anteaya: that's what people call me | 17:24 |
| anteaya | ha ha ha | 17:24 |
| mordred | anteaya: right after they say "that guy who is always upbeat, positive and who gives criticism in a completely constructive manner at all times" | 17:24 |
| *** daemontool_ has joined #openstack-infra | 17:24 | |
| *** fitoduarte has joined #openstack-infra | 17:24 | |
| anteaya | mordred: yeah right after that | 17:25 |
| openstackgerrit | Merged openstack-infra/project-config: Remove translations from operations-guide https://review.openstack.org/311660 | 17:25 |
| *** hughhalf has quit IRC | 17:25 | |
| *** tosky has quit IRC | 17:26 | |
| *** daemontool has quit IRC | 17:27 | |
| *** hughhalf has joined #openstack-infra | 17:27 | |
| *** david-lyle has joined #openstack-infra | 17:28 | |
| *** david-lyle has quit IRC | 17:29 | |
| *** baoli has quit IRC | 17:29 | |
| anteaya | pabelanger: can you think of any reason why would have a limit on how many grafana dashboards we create? | 17:29 |
| *** sputnik13 has joined #openstack-infra | 17:30 | |
| *** david-lyle has joined #openstack-infra | 17:30 | |
| *** heyongli_ has quit IRC | 17:30 | |
| *** heyongli_ has joined #openstack-infra | 17:30 | |
| mordred | anteaya: there is a limit to the number of bytes of information that can be stored across the sum of all of the computers in the world ... that might be a limit ;) | 17:31 |
| pabelanger | anteaya: maybe if the dashboards are impacting graphite.o.o we want to limit grafana dashboards. | 17:31 |
| AJaeger | thanks for reviews, anteaya ! | 17:31 |
| anteaya | mordred: indeed, how very helpful | 17:31 |
| fungi | EmilienM: pabelanger: so i think a saner temporary workaround for d-g is probably to update the pip install command per above. i've tested the syntax from my comment and it successfully avoids installing cryptography as expected | 17:31 |
| *** markvan has joined #openstack-infra | 17:32 | |
| anteaya | pabelanger: so do I have any way of evaluating that? or just allow dashboards until we have a problem and revert some or call a halt? | 17:32 |
| anteaya | AJaeger: welcome :) | 17:32 |
| *** sputnik13 has quit IRC | 17:33 | |
| *** links has quit IRC | 17:33 | |
| *** eil397 has joined #openstack-infra | 17:34 | |
| *** _amrith_ is now known as amrith | 17:35 | |
| pabelanger | anteaya: I think we'd need to monitor catci.o.o to see how much load graphite.o.o is under then decide to limit dashboards or increase resources. But, at this point in time, I'd personally like to see more dashboards in grafana.o.o | 17:35 |
| *** acabot has quit IRC | 17:35 | |
| *** zeih_ has quit IRC | 17:35 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/ansible-puppet: Remove connection: localhost https://review.openstack.org/310878 | 17:36 |
| anteaya | pabelanger: more dashboards it is | 17:36 |
| *** zeih has joined #openstack-infra | 17:36 | |
| anteaya | pabelanger: lets revisit if/when we need to | 17:36 |
| anteaya | and thank you | 17:36 |
| pabelanger | mordred: nibalizer: fungi: ^ here is the patch phschwartz_ and myself came up with in austin. However, we don't have a good way to test it outside of production | 17:36 |
| pabelanger | anteaya: np | 17:36 |
| pabelanger | mordred: nibalizer: fungi: that is for our OOM issues with ansible-playbook on puppetmaster.o.o | 17:37 |
| mtreinish | AJaeger, fungi, pabelanger: https://review.openstack.org/#/c/311147/1 | 17:38 |
| *** inc0 has quit IRC | 17:39 | |
| mordred | pabelanger: looking | 17:39 |
| mtreinish | I'm not sure how many of those jobs actually output subunit, but on fri AJaeger and I were looking at adding it to the translation jobs | 17:39 |
| *** piet has quit IRC | 17:39 | |
| *** inc0 has joined #openstack-infra | 17:39 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Install constrainted version of babel https://review.openstack.org/309809 | 17:39 |
| mordred | pabelanger: we'll need to make sure root can ssh into itself on puppetmaster to do that | 17:39 |
| *** piet has joined #openstack-infra | 17:39 | |
| fungi | mtreinish: seems reasonable | 17:40 |
| AJaeger | mtreinish: yes, this is needed for https://review.openstack.org/311459 - thanks! | 17:40 |
| mordred | pabelanger: which it cannot | 17:40 |
| *** ddieterly is now known as ddieterly[away] | 17:40 | |
| *** bardia has joined #openstack-infra | 17:40 | |
| mordred | pabelanger: other than that - I think that change is completely safe to just apply | 17:40 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/devstack-gate: devstack-vm-gate-wrap: pin paramiko==1.16.0 https://review.openstack.org/311755 | 17:40 |
| pabelanger | mordred: I thought that was the case, but phschwartz_ mention ansible would simply reuse the current connect | 17:40 |
| mordred | pabelanger: as it should not affect functionality | 17:40 |
| EmilienM | fungi, pabelanger: https://review.openstack.org/311755 | 17:40 |
| *** heyongli_ has quit IRC | 17:40 | |
| mordred | pabelanger: there is no current connect | 17:40 |
| fungi | mtreinish: clarkb may want to have a look at it as well, but i assume there's nothing "special" about log collection for the post pipeline | 17:41 |
| *** heyongli_ has joined #openstack-infra | 17:41 | |
| *** zeih has quit IRC | 17:41 | |
| pabelanger | mordred: ack | 17:41 |
| mordred | pabelanger: we just need to put the authorized_keys file in place | 17:41 |
| EmilienM | adding a comment | 17:41 |
| mtreinish | dhellmann: ^^^ you might want to look at AJaeger's patch too, you were asking me about that for release jobs | 17:41 |
| pabelanger | mordred: right | 17:41 |
| fungi | EmilienM: lgtm, and as mordred mentioned, in our case the ansible calls in devstack-gate should never end up using paramiko anyway | 17:42 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/devstack-gate: devstack-vm-gate-wrap: pin paramiko==1.16.0 https://review.openstack.org/311755 | 17:42 |
| *** pvaneck has joined #openstack-infra | 17:42 | |
| pabelanger | fungi: EmilienM: +A | 17:43 |
| mordred | fungi: I'm running the FIP clean script | 17:43 |
| openstackgerrit | Merged openstack-infra/project-config: grafana: added a dashboard with periodic jobs for neutron https://review.openstack.org/307511 | 17:44 |
| mordred | there were 3 leaked osic fips and bluebox seemed totally leaked | 17:44 |
| mordred | scuse me - 2 leaked fips | 17:44 |
| mordred | Cleaned 2 FIPs from osic-cloud1 | 17:44 |
| mordred | Cleaned 44 FIPs from bluebox-sjc1 | 17:44 |
| EmilienM | fungi, pabelanger: thx guys | 17:45 |
| fungi | mordred: so the osic leak is basically solved but bbox is still up in the air? | 17:45 |
| mordred | yah | 17:46 |
| EmilienM | I just hope I did not break anything | 17:46 |
| mordred | fungi: working theory also is that older openstack does this worse than newer | 17:46 |
| fungi | EmilienM: it's about the most innocuous solution we have | 17:47 |
| openstackgerrit | Ben Kero proposed openstack-infra/system-config: add trusty support for paste.openstack.org https://review.openstack.org/311235 | 17:47 |
| fungi | EmilienM: so should be completely safe (but i think our test jobs will tell us if it's not since d-g is self-testing) | 17:47 |
| mordred | fungi: and our clouds from newest to oldest are osic, bluebox, tripleo - and they exhibit the behavior in "not much, medium, constant" | 17:47 |
| *** acabot has joined #openstack-infra | 17:47 | |
| EmilienM | fungi: could I have your feedback on https://review.openstack.org/#/c/301375/ if you have time this week? | 17:47 |
| EmilienM | this is not urgent ^ | 17:47 |
| *** oanson has quit IRC | 17:49 | |
| *** rbrndt has quit IRC | 17:49 | |
| *** fawadkhaliq has quit IRC | 17:50 | |
| *** heyongli_ has quit IRC | 17:51 | |
| *** asselin__ has joined #openstack-infra | 17:51 | |
| *** heyongli_ has joined #openstack-infra | 17:51 | |
| *** permalac has quit IRC | 17:52 | |
| *** inc0_ has joined #openstack-infra | 17:52 | |
| *** inc0 has quit IRC | 17:52 | |
| *** akshai has joined #openstack-infra | 17:52 | |
| *** julim has joined #openstack-infra | 17:52 | |
| *** deadnull_ has joined #openstack-infra | 17:53 | |
| *** asselin has quit IRC | 17:53 | |
| *** deadnull_ has quit IRC | 17:54 | |
| *** piet has quit IRC | 17:55 | |
| fungi | EmilienM: the example we discussed for considering not moving the script for a proposal job into project-config was in the requirements repo, and on reconsideration it turned out that having one branchless copy of the tool in project-config was probably a saner solution anyway due to modifications having to get backported to each branch of the script otherwise. i see that | 17:55 |
| fungi | openstack/puppet-openstack-integration also has stable branches... do you need multiple branch support in that job? | 17:55 |
| EmilienM | fungi: for now, no. We aim to change the Puppetfile only on master. | 17:56 |
| EmilienM | fungi: ie: proposal job will only run on master | 17:56 |
| dprince | pabelanger: hi, re. https://review.openstack.org/#/c/311217/ how about slave_common.pp in system-config? | 17:57 |
| EmilienM | fungi: so I move it? | 17:58 |
| anteaya | mordred: have you any interest in this patch any longer? https://review.openstack.org/#/c/175229/ | 17:59 |
| *** piet has joined #openstack-infra | 17:59 | |
| mordred | anteaya: nope. abandoned | 17:59 |
| anteaya | mordred: thank you and the system-config change it depended on? | 17:59 |
| mordred | uhm | 18:00 |
| openstackgerrit | Steven Dake proposed openstack-infra/project-config: Make oracle gates for Kolla always runs https://review.openstack.org/311803 | 18:00 |
| *** sputnik13 has joined #openstack-infra | 18:00 | |
| mordred | anteaya: I mean, that change is probably useful to someone, but I'm not really giving it mental energy at this point | 18:00 |
| pabelanger | dprince: we have a few reviews open to address that. 311734 for example fixes in our dib images. However, until tripleo migrates to that, we'll need either slave.pp or slave_common.pp. Would need to look deep into those. | 18:00 |
| dprince | pabelanger: yep, understood this is a "quick fix" sort of thing. And totally agree with moving towards the clean DIB build too | 18:01 |
| *** heyongli_ has quit IRC | 18:01 | |
| dprince | pabelanger: I will look and post it now | 18:01 |
| anteaya | mordred: fair enough, this is the system-config patch that was consided a depends on for the patch you abandoned: https://review.openstack.org/#/c/175227/ | 18:01 |
| *** heyongli_ has joined #openstack-infra | 18:01 | |
| pabelanger | dprince: EmilienM did just make a fix to devstack-gate, so what other issues are you seeing? | 18:01 |
| mordred | anteaya: yah. | 18:01 |
| EmilienM | dprince: yeah, my patch should fix tripleo CI | 18:02 |
| EmilienM | dprince: https://review.openstack.org/311755 | 18:02 |
| dprince | pabelanger: we are blocked because our TripleO images haven't had libffi-devel and openssl-devel since Friday | 18:02 |
| dprince | EmilienM: okay, I will reference that here I suppose | 18:03 |
| dprince | https://bugs.launchpad.net/tripleo/+bug/1576807 | 18:03 |
| openstack | Launchpad bug 1576807 in tripleo "ffi.h: No such file or directory" [Critical,Triaged] | 18:03 |
| *** sputnik13 has quit IRC | 18:03 | |
| dprince | EmilienM: sorry I missed your fix. Thanks for posting it | 18:03 |
| EmilienM | dprince: no worries, I should have updated the LP | 18:03 |
| pabelanger | dprince: EmilienM: Ya, that should fix it for now. Until we see what ansible wants to do | 18:03 |
| EmilienM | pabelanger: do we need to wait for a new image to be built? | 18:04 |
| dprince | pabelanger: is there a way to force an image rebuild for TripleO now? | 18:04 |
| *** jed56 has quit IRC | 18:04 | |
| fungi | dprince: yeah, i'm unconvinced that adding more system packages is necessarily the solution until the ansible community confirms paramiko 2.0.0 support is really intentional | 18:04 |
| EmilienM | maybe we clone devstack-gate, let me check | 18:04 |
| pabelanger | I believe once the commit merges, it should unblock. | 18:05 |
| pabelanger | but easy to test | 18:05 |
| EmilienM | dprince: no need, scripts/bighammer.sh clones devstack-gate | 18:05 |
| fungi | EmilienM: dprince: normal devstack-gate jobs see changes take effect as soon as they merge, since d-g is cloned and updated at job start time | 18:05 |
| EmilienM | pabelanger: you're right, TripleO CI git clones devstack-gate | 18:05 |
| EmilienM | fungi: yeah | 18:05 |
| *** trown|lunch is now known as trown | 18:05 | |
| pabelanger | I am doing new image builds too, so I have no problems kicking off tripleo if needed | 18:05 |
| pabelanger | EmilienM: ack | 18:05 |
| dprince | fungi: cool, sounds like we are close then | 18:05 |
| dprince | thanks guys | 18:05 |
| jeblair | (also, d-g uses zuul-cloner so depends-on should work) | 18:07 |
| *** ccamacho has joined #openstack-infra | 18:07 | |
| AJaeger | project-config cores, https://review.openstack.org/310929 contains a first version of review guidelines - comments and reviews are welcome. | 18:08 |
| JayF | Is there a place I can see exactly what size VMs you all use for check/gate jobs? I want to do some benchmarking around Ironic jobs and want to be sure I'm using the right instnace size | 18:09 |
| *** sputnik13 has joined #openstack-infra | 18:10 | |
| nibalizer | JayF: https://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml is the nodepool config file | 18:10 |
| fungi | JayF: the providers section of our nodepool configuration contains hints for flavor selection across different providers we use http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n121 | 18:10 |
| JayF | exactly what Iw as looking for, thanks to both of you | 18:11 |
| nibalizer | it doesn't say exactly what we use but it gives you enough information | 18:11 |
| *** heyongli_ has quit IRC | 18:11 | |
| nibalizer | the tl;dr is every test vm gets 8G of ram | 18:11 |
| JayF | I mean, I'm a racker, so that tells me 8gb instance :) | 18:11 |
| *** heyongli_ has joined #openstack-infra | 18:11 | |
| JayF | which is what I'll use | 18:11 |
| fungi | right, it in fact doesn't know exactly what to use (because that can change across providers and over time) so instead it uses some heuristics to pick something suitable | 18:12 |
| *** sputnik13 has quit IRC | 18:12 | |
| fungi | JayF: specifically 8gb "performance" (and our images are pvhvm) | 18:12 |
| fungi | in rax | 18:12 |
| JayF | awesome; now time to see if I can get pxe booting to be faster :) | 18:13 |
| *** akshai has quit IRC | 18:13 | |
| mordred | JayF: it's also worth noting that we upload externally built images, and it's entirely possible that we are not adding a magic "run faster" flag on them | 18:14 |
| *** akshai has joined #openstack-infra | 18:14 | |
| mordred | since apparently image flags have an effect on how hypervisors execute them | 18:14 |
| *** nadya has quit IRC | 18:14 | |
| JayF | mordred: My benchmarking is going to be around which qemu pxe provider pxe boots things faster | 18:14 |
| mordred | kk | 18:14 |
| JayF | mordred: so I think my results will be valid, and I have enough anecdotal evidence about how long it takes that if I see anything too offbase, I'll raise a red flag | 18:14 |
| mordred | sweet! | 18:15 |
| *** nadya has joined #openstack-infra | 18:15 | |
| bswartz | AJaeger: I ran the cookiecutter thing and it crashed with an error | 18:20 |
| * bswartz sighs | 18:20 | |
| *** heyongli_ has quit IRC | 18:22 | |
| *** xarses has quit IRC | 18:22 | |
| * bswartz roflcopters | 18:22 | |
| bswartz | there is a bug in the cookie cutter repo | 18:22 |
| *** xarses has joined #openstack-infra | 18:22 | |
| *** xarses has quit IRC | 18:23 | |
| *** xarses has joined #openstack-infra | 18:23 | |
| *** Sukhdev has joined #openstack-infra | 18:24 | |
| *** heyongli_ has joined #openstack-infra | 18:24 | |
| openstackgerrit | Ben Swartzlander proposed openstack-dev/specs-cookiecutter: Fix spelling error in template.rst https://review.openstack.org/311806 | 18:25 |
| bswartz | ^ hope this is an easy change to merge | 18:26 |
| *** Sukhdev has quit IRC | 18:26 | |
| notmorgan | mordred: DUDE I want a magic "run faster" flag on all my cloud resources. (sigh... image flags) | 18:27 |
| *** Sukhdev has joined #openstack-infra | 18:27 | |
| *** notmorgan is now known as morgan | 18:28 | |
| *** sputnik13 has joined #openstack-infra | 18:28 | |
| *** kzaitsev_mb has quit IRC | 18:28 | |
| *** fawadkhaliq has joined #openstack-infra | 18:28 | |
| * morgan remembers to put -infra meeting on schedule again. | 18:28 | |
| *** ddieterly[away] is now known as ddieterly | 18:29 | |
| *** inc0_ has quit IRC | 18:29 | |
| dims | bswartz : +2 | 18:31 |
| *** heyongli_ has quit IRC | 18:32 | |
| *** mkoderer__ has quit IRC | 18:32 | |
| *** heyongli_ has joined #openstack-infra | 18:32 | |
| dims | bswartz : thanks! | 18:33 |
| *** tphummel has quit IRC | 18:36 | |
| *** madhu_ak has joined #openstack-infra | 18:37 | |
| *** tphummel has joined #openstack-infra | 18:38 | |
| timothyb89 | mtreinish: fungi: pabelanger: apologies for stackviz issues, a quick look through the logs suggests nodejs-legacy was missing on the ubuntu images | 18:39 |
| timothyb89 | mtreinish: fungi: pabelanger: if there's desire to rethink the build process, I'd like to throw in this as an alternative: https://etherpad.openstack.org/p/stackviz-deployment | 18:41 |
| *** dprince has quit IRC | 18:42 | |
| *** heyongli_ has quit IRC | 18:42 | |
| *** heyongli_ has joined #openstack-infra | 18:42 | |
| *** nmagnezi has joined #openstack-infra | 18:45 | |
| *** baoli has joined #openstack-infra | 18:45 | |
| openstackgerrit | Merged openstack/gertty: Add configuration information to docs https://review.openstack.org/286240 | 18:46 |
| openstackgerrit | Merged openstack/gertty: Add navigation breadcrumb footer https://review.openstack.org/288250 | 18:46 |
| openstackgerrit | Merged openstack/gertty: Make 'title' attribute available on dialog widgets https://review.openstack.org/289748 | 18:46 |
| openstackgerrit | Merged openstack/gertty: Fix rendering error in change screen https://review.openstack.org/295423 | 18:47 |
| openstackgerrit | Merged openstack/gertty: Add matchers for external references on launchpad https://review.openstack.org/295592 | 18:47 |
| openstackgerrit | Merged openstack/gertty: Use short titles in breadcrumbs https://review.openstack.org/310036 | 18:47 |
| openstackgerrit | Merged openstack/gertty: Add extra columns to change list if there is room https://review.openstack.org/310037 | 18:47 |
| openstackgerrit | Merged openstack/gertty: Add an option to disable the breadcrumb footer https://review.openstack.org/311770 | 18:47 |
| fungi | morgan: turbo button as a service | 18:48 |
| morgan | fungi: hehe | 18:48 |
| openstackgerrit | Merged openstack-dev/specs-cookiecutter: Fix spelling error in template.rst https://review.openstack.org/311806 | 18:49 |
| *** nadya has quit IRC | 18:50 | |
| *** banix has joined #openstack-infra | 18:50 | |
| *** roxanagh_ has joined #openstack-infra | 18:50 | |
| *** banix has quit IRC | 18:51 | |
| *** banix has joined #openstack-infra | 18:51 | |
| pabelanger | timothyb89: I'll look at the etherpad and give comments | 18:52 |
| fungi | timothyb89: on the centos images i gather the failure was different enough to possibly be a second problem | 18:52 |
| *** heyongli_ has quit IRC | 18:52 | |
| *** inc0 has joined #openstack-infra | 18:52 | |
| *** heyongli_ has joined #openstack-infra | 18:52 | |
| timothyb89 | fungi: right, that was a race condition in the build script, I just pushed a patch on our end to fix: https://review.openstack.org/311810 | 18:53 |
| *** rbrndt has joined #openstack-infra | 18:54 | |
| *** hashar has joined #openstack-infra | 18:54 | |
| fungi | timothyb89: excellent. can you also propose a revert of the revert in project-config which adds the nodejs-legacy package when needed? (or have you already and i missed it?) | 18:55 |
| *** roxanagh_ has quit IRC | 18:55 | |
| timothyb89 | fungi: I'm a bit confused about that actually ... in this it seems to be installed already: https://review.openstack.org/#/c/279317/12/nodepool/elements/stackviz/pkg-map.json | 18:56 |
| *** esker has quit IRC | 18:56 | |
| timothyb89 | fungi: but in the logs we got: /usr/bin/env: node: No such file or directory | 18:56 |
| fungi | timothyb89: did you see it getting installed earlier in the log? | 18:56 |
| timothyb89 | fungi: it used the name configured, so it said it was installing 'nodejs' | 18:57 |
| timothyb89 | fungi: 2016-05-02 15:39:08,230 INFO nodepool.image.build.debian-jessie: installing nodejs from stackviz | 18:58 |
| fungi | was the apt-get stdout included after that? shoul dmention the actual package name(s) installed | 18:59 |
| timothyb89 | fungi: ah, no, I'm not seeing it there | 18:59 |
| nibalizer | the bandersnatch process is still .... there | 19:00 |
| openstackgerrit | Joshua Harlow proposed openstack/requirements: UC bumps for pslo releases for week of may 2nd https://review.openstack.org/311813 | 19:00 |
| nibalizer | but it hasn't put anything new in its log file and stracing it, it is just waiting | 19:00 |
| openstackgerrit | Joshua Harlow proposed openstack/requirements: UC bumps for oslo releases for week of may 2nd https://review.openstack.org/311813 | 19:01 |
| *** esker has joined #openstack-infra | 19:01 | |
| *** heyongli_ has quit IRC | 19:02 | |
| thingee | pleia2: ping | 19:03 |
| *** heyongli_ has joined #openstack-infra | 19:03 | |
| *** piet has quit IRC | 19:03 | |
| *** xarses_ has joined #openstack-infra | 19:04 | |
| timothyb89 | fungi: hmm, I take it that file should be just `pkg-map` and not `pkg-map.json`? :) | 19:04 |
| *** xarses has quit IRC | 19:05 | |
| thingee | fungi: ping | 19:05 |
| *** ccamacho has quit IRC | 19:05 | |
| fungi | timothyb89: good question. git ls-files says we have two other elements which just call the file "pkg-map" with no extension | 19:05 |
| fungi | thingee: clothe thy ping, heathen | 19:05 |
| fungi | timothyb89: so anyway, that would likely explain it | 19:05 |
| timothyb89 | fungi: sounds good to me :) any objections if I put the patch up again with corrections? | 19:06 |
| *** esker has quit IRC | 19:06 | |
| *** jbryce has quit IRC | 19:06 | |
| fungi | timothyb89: no objections--quite the opposite, that would be most welcome | 19:07 |
| thingee | fungi, pleia2 not sure who all was part of this summit discussion https://etherpad.openstack.org/p/newton-infra-landing-page-for-contributors ... my apologies for not attending as I had a conflict. But I wanted to say the part that says the work I was doing was "geared towards slide deck + room full" ... | 19:07 |
| timothyb89 | fungi: great! thanks for the help! | 19:07 |
| thingee | fungi, pleia2 my work is actually the stuff under how to prepare - http://docs.openstack.org/upstream-training/#how-to-prepare where IRC, account setup and git setup are the essential things | 19:07 |
| fungi | thingee: i think the mention of your guide work was mostly in regards to theming and related resources we might want to link | 19:08 |
| fungi | thingee: but also, thanks for the clarification | 19:08 |
| *** jbryce has joined #openstack-infra | 19:08 | |
| thingee | fungi: ok, just quoting the etherpad here since there seems to be confusion. But the value of this documentation is to give people something they glance at and feel like they can complete things in five minutes. As oppose to existing documentation that is pretty lengthy, can be intimidating, and is attempting to cover every edge case you'd hit along the way. | 19:09 |
| fungi | yikes, opening our image build logs in firefox is a terrible idea :/ | 19:09 |
| fungi | thingee: sounds good to me. the discussion was two-pronged. we want to move the how to contribute page off the wiki. it also covers lots of non-code-contributor getting started situations that don't need gerrit/git | 19:11 |
| *** hdd has quit IRC | 19:11 | |
| thingee | fungi: I'm trying to solve for those people that might not be a developer, but want to contribute to OpenStack in other ways. Or even the person that just has a two line fix and doesn't want to spend half a day getting started with our process. | 19:11 |
| thingee | fungi: +1 for moving off the wiki | 19:11 |
| fungi | long live "project.openstack.org" | 19:12 |
| *** nmagnezi has quit IRC | 19:12 | |
| jeblair | thingee: oh, i did not realize you were doing that | 19:12 |
| thingee | fungi: what's the reasoning behind "project" being used here? | 19:12 |
| *** oanson has joined #openstack-infra | 19:13 | |
| jeblair | thingee: we put a lot of effort into making http://docs.openstack.org/infra/manual/developers.html exactly that -- the shortest and simplest steps for a developer to get started | 19:13 |
| *** heyongli_ has quit IRC | 19:13 | |
| *** heyongli_ has joined #openstack-infra | 19:13 | |
| dirk | dims: could you merge https://review.openstack.org/#/c/291769/ ? | 19:14 |
| *** yolanda has quit IRC | 19:14 | |
| openstackgerrit | Tim Buckley proposed openstack-infra/project-config: Create stackviz element and script in nodepool https://review.openstack.org/311815 | 19:14 |
| thingee | jeblair: yeah, I'd love to collaborate on how we can make things more friendly for people. I've tried these docs with a variety of people in the mentor programs and upstream and felt it was well received. | 19:14 |
| jeblair | thingee: should we just delete the infra manual? | 19:14 |
| dims | dirk : need a second core | 19:14 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Implement periodic job to update Puppet OpenStack constraints https://review.openstack.org/301375 | 19:14 |
| EmilienM | fungi: I moved the script ^ | 19:14 |
| timothyb89 | fungi: pabelanger: mtreinish: revised patch is here: https://review.openstack.org/311815 | 19:15 |
| dirk | dims: apevec is core and he +1ed it | 19:15 |
| jeblair | the main reason i wrote the infra manual was that we had duplicate documentation, much of which was outdated, wrong, or needlessly complex, and i thought it was important to have just one place with the steps for setting things up as a new developer | 19:16 |
| dims | dirk : apologies i meant requirements-core folks https://review.openstack.org/#/admin/groups/131,members | 19:16 |
| thingee | jeblair: not at all. I think we can collaborate together on keeping what we have, but make it so things *feel* more digestable. I think it's just maybe separating things from one big page as it is right now. | 19:16 |
| jeblair | thingee: ok cool, that sounds good | 19:17 |
| jeblair | thingee: maybe each of the developer's guide sections should be different sphinx pages? | 19:18 |
| thingee | jeblair: yes | 19:18 |
| fungi | i can certainly see where adding more of a "choose your own adventure" feel and hiding less commonly needed details could help | 19:18 |
| dirk | dims: ok, I see. I was apparently looking at the wrong core list. | 19:18 |
| *** dtardivel has quit IRC | 19:19 | |
| jeblair | fungi, thingee: yeah, as thingee says that, i can see how one might mistake the entire developers guide (most of which is a reference) with the minimal 'getting started' section, which is a small (but very important) fraction of the whole | 19:19 |
| jeblair | according to my scrollbar, it's about 25% | 19:20 |
| jeblair | and the transition from getting started to development workflow would make a really good page split | 19:20 |
| thingee | jeblair: From the contributor on-board page, I was picturing something static that would allow you to pick what area you want to contribute to, as fungi mentioned "choose your own adventure" kind of feel. No matter what you choose there would be essential stuff like communication (explaining ML and IRC http://docs.openstack.org/upstream-training/irc.html) and then account setup | 19:21 |
| thingee | (http://docs.openstack.org/upstream-training/accounts.html) (http://docs.openstack.org/upstream-training/git.html) ... of course we don't have to use what I worked on here, but that's eventually the bigger picture of things | 19:21 |
| *** jaosorior has joined #openstack-infra | 19:22 | |
| thingee | jeblair: if you pick translator for example, it would give the essential links and then content for the specific stuff the i18n team specifically wants. | 19:22 |
| *** heyongli_ has quit IRC | 19:23 | |
| *** baoli has quit IRC | 19:23 | |
| *** rockyg has joined #openstack-infra | 19:23 | |
| *** heyongli_ has joined #openstack-infra | 19:23 | |
| fungi | i'm on board as long as we can keep _one_ place where those steps are documented/encoded so that we don't wind up back in the bad old days of multiple walkthroughs which drifted/skewed/diverged | 19:23 |
| thingee | fungi: +1 | 19:24 |
| thingee | This on-boarding page, and good communication (especially in the docs of starting a new project and mentioning it there) ... would hopefully encourage that idea of one place. | 19:24 |
| fungi | we've spent lots of time attempting to squash all alternative walkthroughs so that people got a consistent set of steps (particularly around the terrible nuances of gerrit and foundation accounts, cla, et cetera) | 19:25 |
| jeblair | yeah, which is why i'm (seriously) suggesting that maybe we should delete the infra manual :) | 19:25 |
| AJaeger | fungi, I agree - and try constantly to keep things like http://docs.openstack.org/contributor-guide/quickstart/first-timers.html short but it's longer than I loe ;( | 19:25 |
| *** kzaitsev_mb has joined #openstack-infra | 19:25 | |
| *** rockyg has quit IRC | 19:26 | |
| thingee | AJaeger: I love these docs. I spoke to anne before I started these upstream ones and unfortunately just discovered them on the last day of the summit. | 19:26 |
| AJaeger | thingee: I'm trying hard that we don't duplicate everything in the contributor guide... | 19:27 |
| thingee | AJaeger: I might've just didn't explain things well for her to point out to me that something similar already exists. | 19:27 |
| *** rockyg has joined #openstack-infra | 19:27 | |
| *** mylu has joined #openstack-infra | 19:28 | |
| *** yamahata has quit IRC | 19:28 | |
| *** ddieterly is now known as ddieterly[away] | 19:28 | |
| AJaeger | jeblair: will you sent a patch to split up the developer's guide page? | 19:29 |
| thingee | AJaeger: so if the on-board docs had the "essential stuff" like communication, getting account setup, git, workflow, would the docs team be able to use that and in addition to the on-board page having other references to docs team specific tools documentation, etc? | 19:29 |
| *** fawadkhaliq has quit IRC | 19:29 | |
| *** mylu has quit IRC | 19:30 | |
| AJaeger | thingee: I hope so;) Better discuss with annegentle and loquacities. | 19:30 |
| harlowja | qq, for folks, likely something simple i'm missing here | 19:31 |
| harlowja | i'm trying to create a feature branch for oslo.messaging | 19:31 |
| harlowja | running the following | 19:31 |
| harlowja | ssh -p 29418 harlowja@review.openstack.org 'gerrit create-branch' openstack/oslo.messaging feature/dispatch-router 51f3b7b65fff6194bf1c50043bddd649ae6d19ee | 19:31 |
| harlowja | but ending up getting 'fatal: Cannot create "refs/heads/feature/dispatch-router"' | 19:31 |
| thingee | AJaeger: not sure if you saw earlier conversation, but the bigger picture here is it's an on-boarding contributor launch page ... choose your own adventure style. Based on what you want to contribute to it would give the essential resources always, and then in that same docs repo we can have page(s) dedicated to very specific contributor resource stuff for different teams. | 19:31 |
| *** fawadkhaliq has joined #openstack-infra | 19:31 | |
| fungi | AJaeger: thingee: jeblair: i suspect it would be less of a deletion of the infra-manual, and more of a (partial?) move of the information, or possibly even we build the project.openstack.org site out of files in infra-manual. we didn't really bikeshed on where the content is maintained, so much as where and how we would publish it | 19:32 |
| bswartz | harlowja: I've gotten that before -- it was acls related | 19:32 |
| AJaeger | thingee: I scanned the earlier discussion | 19:33 |
| bswartz | harlowja: it's possible to create branches inside gerrit IIRC | 19:33 |
| harlowja | hmmmm | 19:33 |
| *** heyongli_ has quit IRC | 19:33 | |
| *** jaosorior has quit IRC | 19:33 | |
| bswartz | I mean thru the UI | 19:33 |
| AJaeger | fungi, thingee : Thanks for explanation. I'm happy to help with this ;) | 19:33 |
| *** heyongli_ has joined #openstack-infra | 19:33 | |
| *** jaosorior has joined #openstack-infra | 19:33 | |
| jeblair | AJaeger: i won't have time this week | 19:33 |
| thingee | AJaeger: any thoughts? hate it? I think it would help consolidate things, but also finally give us a place that's organized to reference new potential contributors to. | 19:34 |
| fungi | harlowja: i don't see a "create" privilege in openstack-infra/project-config:gerrit/acls/openstack/oslo.messaging.config | 19:34 |
| AJaeger | thingee: I'm all for consolidation ;) So, like the idea. | 19:34 |
| harlowja | hmmmm, i wonder how the other feature branches for oslo.messaging were made | 19:34 |
| harlowja | https://review.openstack.org/#/admin/projects/openstack/oslo.messaging,branches?filter=feature%252F :-/ | 19:34 |
| rcarrillocruz | harlowja: unless you belong to 'project bootstrappers' group or you have explicit acl perms on your group, you won't be able to create the branch, no... | 19:34 |
| rcarrillocruz | probably someone from infra did for oslo team? | 19:34 |
| clarkb | I want to see that the releases team has blankey create perms | 19:34 |
| thingee | AJaeger: excellent. Just waiting on jeblair and fungi's blessing and I guess I'll start an email to explain things. | 19:34 |
| clarkb | s/see/say/ | 19:34 |
| thingee | AJaeger, fungi, jeblair before going to far with things like splitting, etc | 19:35 |
| fungi | harlowja: gerrit admins and the release management team can create branches by default, others can also create branches on a project-by-project basis with the right acls | 19:35 |
| harlowja | k, so i guess it must of been gerrit admins | 19:35 |
| fungi | those pesky gerrit admins | 19:35 |
| harlowja | :) | 19:35 |
| *** nadya has joined #openstack-infra | 19:35 | |
| harlowja | soooo, can i request a feature branch for oslo.messaging, or should i just get the acl? | 19:35 |
| AJaeger | thingee: make a cross-project spec ;) | 19:36 |
| thingee | AJaeger: that's a good idea. I can do that | 19:37 |
| bswartz | "Give me ACLs or give me death!" --Patrick Henry | 19:37 |
| dirk | dims: could you help me understand http://logs.openstack.org/68/311468/1/check/gate-requirements-integration-dsvm/2075ca2/console.html ? | 19:37 |
| dirk | dims: this package definitely exists in that version on pypi.. is there some mirroring cronjob somewhere that needs to be triggered? | 19:37 |
| harlowja | if i can just request the feature branch, that'd be fine, if someone can do it, `'gerrit create-branch' openstack/oslo.messaging feature/dispatch-router 51f3b7b65fff6194bf1c50043bddd649ae6d19ee ` can be the commmand someone runs :-P | 19:38 |
| dims | dirk : the pypi mirroring is broken, infra folks here are working on resurrecting the mirrors | 19:38 |
| clarkb | harlowja: please ask the release team for release managed repos | 19:38 |
| harlowja | k | 19:39 |
| dims | dirk : the pypi mirrors are listed here - http://docs.openstack.org/infra/system-config/bandersnatch.html | 19:40 |
| openstackgerrit | Dirk Mueller proposed openstack/requirements: Add spec_cleaner for rpm-packaging style checks https://review.openstack.org/311468 | 19:40 |
| thingee | AJaeger: maybe here, if the docs team was fine with it becoming a shared thing since they already own it for how to contribute to docs. http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/contributor-guide | 19:40 |
| *** mylu has joined #openstack-infra | 19:42 | |
| thingee | AJaeger, fungi: and this would build to project.openstack.org ... which I'm still not sure why we decided on project here? | 19:42 |
| AJaeger | thingee: best reach out to loquacities on that one. As I said: I'm for consolidation but haven't had enough convincing arguments yet. But if we move forward with project.o.o, I'll happily help to push that. | 19:42 |
| dirk | dims: ouch, ok | 19:42 |
| *** xarses_ is now known as xarses | 19:42 | |
| AJaeger | thingee: I think the idea was to put it on a separate website with an easy to remember name. | 19:43 |
| AJaeger | thingee: and project was the best idea from that session - which I couldn't attend either | 19:43 |
| *** ccamacho has joined #openstack-infra | 19:43 | |
| *** heyongli_ has quit IRC | 19:43 | |
| fungi | thingee: there was far too much bike shedding over coming up with a name that indicated this was a landing portal for the upstream project-teams, without ending up with a name that creates a battlefield over atcs vs. non-atcs drama | 19:43 |
| thingee | I'm trying to make the connection of telling a new contributor go to project.o.o - seems like contributor.o.o would make sense? | 19:44 |
| *** heyongli_ has joined #openstack-infra | 19:44 | |
| thingee | ah | 19:44 |
| *** mylu has quit IRC | 19:44 | |
| fungi | thingee: "contributor" was contentious | 19:44 |
| thingee | fungi: I think whatever the non-atc wg comes up with, we can add that in? | 19:44 |
| thingee | would it still be a problem then? meet up organizing, etc | 19:45 |
| fungi | thingee: concern was that documentation for the non-atcs should be under the control of a neutral body and not under the thumb of the tc | 19:45 |
| *** baoli has joined #openstack-infra | 19:45 | |
| thingee | welcome.o.o ? :P | 19:46 |
| * thingee takes out some paint buckets | 19:46 | |
| fungi | heh | 19:46 |
| * dims shows up with some brushes | 19:46 | |
| *** baoli_ has joined #openstack-infra | 19:47 | |
| fungi | anyway, the conflation is that www.openstack.org is the site for the openstack foundation, so project.openstack.org seemed to be the least overloaded and most descriptive name for a landing site for the openstack project | 19:47 |
| thingee | ok well I'll leave that open in the spec and walk away for a week or so | 19:47 |
| * AJaeger grabs his camera to record the painting session | 19:47 | |
| thingee | fungi: kind of seems project.o.o should be navigator realted | 19:47 |
| thingee | that's my initial confusion with it | 19:48 |
| AJaeger | thingee: or just say, a common page like project.o.o | 19:48 |
| thingee | project navigator* | 19:48 |
| fungi | thingee: ttx and jeblair had a lot to say on the matter of the site name, so will probably need to be brought in on further naming conversations when they're not vacationing | 19:48 |
| thingee | AJaeger: I think it's a valid concern that project.o.o can be confused for http://www.openstack.org/software/project-navigator/ | 19:49 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add support for testfairy plugin https://review.openstack.org/261587 | 19:49 |
| thingee | fungi: sure. so I do have some blessing to start cross-project spec on this? | 19:49 |
| thingee | the idea of moving infra-manual dev doc content over to whatever location? | 19:50 |
| EmilienM | pabelanger: why do we workflow https://review.openstack.org/#/c/311754/ ? are you still testing something? | 19:50 |
| thingee | jeblair: ^ | 19:50 |
| fungi | thingee: what's cross-project about it? | 19:50 |
| fungi | doesn't seem like it needs changes in lots of projects | 19:50 |
| *** regXboi has joined #openstack-infra | 19:50 | |
| pabelanger | EmilienM: Ya, I want to test it tonight. Make sure we don't break the world tomorrow | 19:51 |
| thingee | fungi: well if it's a one place for learning how to contribute to different efforts, it involves projects, i18n, some working groups and how you contribute to them... etc | 19:51 |
| *** baoli has quit IRC | 19:51 | |
| AJaeger | docs will come into it. Have some specification writtenup would be nice but it doesn't need a full blown cross-spec. | 19:51 |
| fungi | got it. i have no opposition to a cross-project openstack/openstack-specs spec describing a possible implementation we can pick apart | 19:52 |
| jeblair | i do not think i will have time to contribute | 19:52 |
| EmilienM | pabelanger: ok | 19:52 |
| *** ddieterly[away] is now known as ddieterly | 19:52 | |
| thingee | jeblair: no problem. just wanted to make sure with the little details and direction we have right now that it sounded ok to you. | 19:53 |
| fungi | though it also still seems like it could wait a week or so for the key people from that session to be back at computers | 19:53 |
| *** _ari_ has quit IRC | 19:53 | |
| thingee | fungi: sure no problem. I can delay putting it out. I'll just privately start the spec so things are ready. | 19:53 |
| *** matrohon has joined #openstack-infra | 19:53 | |
| *** lucasagomes has quit IRC | 19:53 | |
| *** heyongli_ has quit IRC | 19:54 | |
| AJaeger | put it out - let's just give some time for others to come back before you promote it... | 19:54 |
| *** heyongli_ has joined #openstack-infra | 19:54 | |
| *** yamahata has joined #openstack-infra | 19:58 | |
| *** lucasagomes has joined #openstack-infra | 20:00 | |
| *** chem has quit IRC | 20:02 | |
| *** apoorvad has quit IRC | 20:02 | |
| *** matrohon has quit IRC | 20:03 | |
| *** heyongli_ has quit IRC | 20:04 | |
| *** jamesmcarthur has joined #openstack-infra | 20:05 | |
| *** matrohon has joined #openstack-infra | 20:05 | |
| openstackgerrit | Sarafraj Singh proposed openstack/requirements: Bumping version of websockify>=0.8.0 https://review.openstack.org/311829 | 20:06 |
| *** heyongli_ has joined #openstack-infra | 20:06 | |
| jlk | Can I get a workflow +2 on https://review.openstack.org/#/c/311778/ ? | 20:07 |
| *** baoli has joined #openstack-infra | 20:08 | |
| *** baoli_ has quit IRC | 20:08 | |
| *** sdake_ has joined #openstack-infra | 20:08 | |
| *** sdake has quit IRC | 20:08 | |
| *** baoli has quit IRC | 20:08 | |
| *** baoli has joined #openstack-infra | 20:09 | |
| *** jamesmcarthur has quit IRC | 20:10 | |
| pabelanger | EmilienM: with puppet tarballs, we need puppet-foo-master.tar.gz, puppet-foo-stable-mitaka.tar.gz and puppet-foo-stable-liberty.tar.gz for each branch right? | 20:10 |
| *** rbrndt_ has joined #openstack-infra | 20:10 | |
| EmilienM | pabelanger: yes | 20:11 |
| EmilienM | pabelanger: it used to work until now | 20:11 |
| EmilienM | only tags are broken | 20:11 |
| *** rbrndt has quit IRC | 20:11 | |
| pabelanger | EmilienM: right, I am just looking for how puppet-foo.tar.gz is copied into master, stable-mitaka and stable-liberty. Since run-tarballs doesn't do that now | 20:12 |
| EmilienM | pabelanger: it's in the branch-tarball publisher IIRC | 20:12 |
| EmilienM | pabelanger: see in python-jobs.yaml | 20:12 |
| pabelanger | EmilienM: perfect | 20:13 |
| *** heyongli_ has quit IRC | 20:14 | |
| pabelanger | EmilienM: I just need to confirm what a ZUUL_REFNAME looks like for a tag | 20:14 |
| pabelanger | then happy to remove -1 workflow | 20:14 |
| *** heyongli_ has joined #openstack-infra | 20:14 | |
| *** kzaitsev_mb has quit IRC | 20:15 | |
| pabelanger | ZUUL_REFNAME=refs/tags/1.19.22 IIRC is the format | 20:15 |
| pabelanger | EmilienM: okay, looks good | 20:17 |
| *** nadya has quit IRC | 20:17 | |
| pabelanger | EmilienM: the only other issue, is we may need to update the '{name}-tarball' job for puppet, since we don't have any wheels to upload. That is causing the failure on the job returning a success | 20:18 |
| EmilienM | right | 20:18 |
| *** mylu has joined #openstack-infra | 20:19 | |
| *** jamesmcarthur has joined #openstack-infra | 20:21 | |
| openstackgerrit | Pradeep Kilambi proposed openstack-infra/tripleo-ci: Add aodh endpoints to enable-tls https://review.openstack.org/311833 | 20:21 |
| *** mylu has quit IRC | 20:21 | |
| *** marcusvrn__ has joined #openstack-infra | 20:22 | |
| *** markmcd has joined #openstack-infra | 20:22 | |
| *** serverascode_ has joined #openstack-infra | 20:23 | |
| *** vdrok_ has joined #openstack-infra | 20:23 | |
| *** notmorgan has joined #openstack-infra | 20:24 | |
| *** notmorgan has quit IRC | 20:24 | |
| *** notmorgan has joined #openstack-infra | 20:24 | |
| *** pradk has quit IRC | 20:24 | |
| *** Ng_ has joined #openstack-infra | 20:24 | |
| *** morgan has quit IRC | 20:24 | |
| *** notmorgan is now known as morgan | 20:24 | |
| *** xgerman_ has joined #openstack-infra | 20:24 | |
| *** heyongli_ has quit IRC | 20:24 | |
| *** heyongli_ has joined #openstack-infra | 20:25 | |
| *** sridhar_ram has quit IRC | 20:25 | |
| *** mgkwill_ has joined #openstack-infra | 20:25 | |
| *** sheeprine_ has joined #openstack-infra | 20:25 | |
| *** igorbelikov_ has joined #openstack-infra | 20:26 | |
| *** dougwig_ has joined #openstack-infra | 20:26 | |
| *** Sukhdev has quit IRC | 20:27 | |
| *** andreykurilin___ has joined #openstack-infra | 20:27 | |
| *** warpc_ has joined #openstack-infra | 20:28 | |
| *** sylwesterB_ has joined #openstack-infra | 20:28 | |
| *** AJaeger has quit IRC | 20:28 | |
| *** GheRivero_ has joined #openstack-infra | 20:28 | |
| clarkb | mtreinish: re https://review.openstack.org/#/c/311256/ you should be able to add a [ -d .tox/$venvname ] to devstack and switch on which of the paths is in use | 20:28 |
| *** afazekas_ has joined #openstack-infra | 20:28 | |
| *** bapalm_ has joined #openstack-infra | 20:28 | |
| *** LinuxJedi_ has joined #openstack-infra | 20:28 | |
| clarkb | mtreinish: then after you switch (remember you have to do it in each branch too), you can remove that and go back to the common venv | 20:28 |
| *** cbader has quit IRC | 20:29 | |
| *** niska` has quit IRC | 20:29 | |
| *** GheRivero has quit IRC | 20:29 | |
| *** xgerman has quit IRC | 20:29 | |
| *** sylwesterB has quit IRC | 20:29 | |
| *** cgross has quit IRC | 20:29 | |
| *** bapalm has quit IRC | 20:29 | |
| *** Ng has quit IRC | 20:29 | |
| *** peterlisak has quit IRC | 20:29 | |
| *** teran has quit IRC | 20:29 | |
| *** sheeprine has quit IRC | 20:29 | |
| *** flip214 has quit IRC | 20:29 | |
| *** odyssey4me has quit IRC | 20:29 | |
| *** kaisers has quit IRC | 20:29 | |
| *** andreykurilin has quit IRC | 20:29 | |
| *** mgkwill has quit IRC | 20:29 | |
| *** marcusvrn_ has quit IRC | 20:29 | |
| *** markmcd1 has quit IRC | 20:29 | |
| *** afazekas has quit IRC | 20:29 | |
| *** dstanek has quit IRC | 20:29 | |
| *** warpc has quit IRC | 20:29 | |
| *** morgabra has quit IRC | 20:29 | |
| *** mmedvede has quit IRC | 20:29 | |
| *** vdrok has quit IRC | 20:29 | |
| *** bookwar has quit IRC | 20:29 | |
| *** serverascode has quit IRC | 20:29 | |
| *** fungi has quit IRC | 20:29 | |
| *** dougwig has quit IRC | 20:29 | |
| *** dtroyer has quit IRC | 20:29 | |
| *** igorbelikov has quit IRC | 20:29 | |
| *** andreykurilin__ has quit IRC | 20:29 | |
| *** LinuxJedi has quit IRC | 20:29 | |
| *** LinuxJedi_ is now known as LinuxJedi | 20:29 | |
| *** bookwar_ has joined #openstack-infra | 20:29 | |
| clarkb | mtreinish: I really do like the common venv idea though, makes a lot of sense for what tempest is trying to do I think and will make jobs a lot quicker | 20:29 |
| *** cbader has joined #openstack-infra | 20:29 | |
| *** andreykurilin has joined #openstack-infra | 20:29 | |
| *** sylwesterB_ is now known as sylwesterB | 20:29 | |
| *** niska has joined #openstack-infra | 20:29 | |
| *** vdrok_ is now known as vdrok | 20:29 | |
| *** xgerman_ is now known as xgerman | 20:29 | |
| *** warpc_ is now known as warpc | 20:30 | |
| mtreinish | clarkb: yeah, that should work. I'll push up a depends on patch(es) for that | 20:30 |
| *** kaisers has joined #openstack-infra | 20:30 | |
| *** cgross has joined #openstack-infra | 20:30 | |
| *** mgkwill_ is now known as mgkwill | 20:30 | |
| *** marcusvrn__ is now known as marcusvrn_ | 20:30 | |
| *** peterlisak has joined #openstack-infra | 20:30 | |
| *** dtroyer has joined #openstack-infra | 20:30 | |
| *** GheRivero_ is now known as GheRivero | 20:30 | |
| *** andreykurilin___ is now known as andreykurilin__ | 20:30 | |
| *** dstanek has joined #openstack-infra | 20:31 | |
| clarkb | fungi: pabelanger: I am done running errands for the day what is paramiko/ansible situation and are there changes I should be reviewing to help fix that? | 20:31 |
| *** igorbelikov_ is now known as igorbelikov | 20:31 | |
| *** Ng_ is now known as Ng | 20:31 | |
| *** flip214 has joined #openstack-infra | 20:31 | |
| pabelanger | clarkb: fungi: we landed a fix for devstack-gate. Actually EmilienM did | 20:31 |
| *** baoli has quit IRC | 20:31 | |
| *** morgabra has joined #openstack-infra | 20:32 | |
| pabelanger | clarkb: but it appears only tripleo is affected at the moment. They are not using our pypi mirrors, which don't have the new paramiko release yet | 20:32 |
| clarkb | ha, the unhappy mirrors are good for something | 20:32 |
| nibalizer | oh did i miss a painting session | 20:33 |
| *** banix has quit IRC | 20:33 | |
| nibalizer | thingee: call it portal.openstack.org, everyone needs a portal | 20:33 |
| *** fungi has joined #openstack-infra | 20:33 | |
| *** AJaeger has joined #openstack-infra | 20:34 | |
| *** dougwig_ is now known as dougwig | 20:34 | |
| thingee | nibalizer: only if it's running off of php nuke. | 20:34 |
| *** serverascode_ is now known as serverascode | 20:34 | |
| pabelanger | clarkb: ++ | 20:34 |
| *** odyssey4me has joined #openstack-infra | 20:35 | |
| *** heyongli_ has quit IRC | 20:35 | |
| *** heyongli_ has joined #openstack-infra | 20:35 | |
| pabelanger | clarkb: I have finished centos-7 and ubuntu-trusty dibs pinning paramiko in zuul. ubuntu-xenial building now, with fedora-23 and debian-jessie to follow | 20:35 |
| *** hdd has joined #openstack-infra | 20:35 | |
| *** jamesmcarthur has quit IRC | 20:36 | |
| pabelanger | I'm also testing ubuntu-precise locally to see what happens | 20:36 |
| clarkb | mtreinish: can we WIP https://review.openstack.org/#/c/297446/ based on what happens iwth common tempest env and plugin support in devstack? | 20:36 |
| clarkb | pabelanger: are you uploading those images too or just building them for the moment? | 20:36 |
| pabelanger | clarkb: just building now, but looks like nodepool is uploading them as I go. Since the original uploads failed | 20:37 |
| *** teran has joined #openstack-infra | 20:37 | |
| mtreinish | clarkb: done | 20:38 |
| *** sileht has quit IRC | 20:39 | |
| *** jamesmcarthur has joined #openstack-infra | 20:39 | |
| *** jamesmcarthur has quit IRC | 20:41 | |
| *** sileht has joined #openstack-infra | 20:41 | |
| *** pradk has joined #openstack-infra | 20:41 | |
| *** jamesmcarthur has joined #openstack-infra | 20:41 | |
| *** rockyg has quit IRC | 20:41 | |
| *** denisra has quit IRC | 20:43 | |
| *** matrohon has quit IRC | 20:43 | |
| *** jamesmcarthur has quit IRC | 20:43 | |
| *** heyongli_ has quit IRC | 20:45 | |
| *** jaosorior has quit IRC | 20:45 | |
| *** heyongli_ has joined #openstack-infra | 20:45 | |
| clarkb | ianw: comment on https://review.openstack.org/#/c/296171/2 want to turn that around quickly? I can approve if you are around today | 20:47 |
| *** mylu has joined #openstack-infra | 20:48 | |
| *** kzaitsev_mb has joined #openstack-infra | 20:48 | |
| *** mylu has quit IRC | 20:49 | |
| *** apoorvad has joined #openstack-infra | 20:51 | |
| *** mmedvede has joined #openstack-infra | 20:52 | |
| *** pradk has quit IRC | 20:53 | |
| *** piet has joined #openstack-infra | 20:53 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: devstack-vm-gate-wrap: pin paramiko==1.16.0 https://review.openstack.org/311755 | 20:54 |
| *** kzaitsev_mb has quit IRC | 20:54 | |
| *** yamamoto has joined #openstack-infra | 20:55 | |
| *** heyongli_ has quit IRC | 20:55 | |
| *** heyongli_ has joined #openstack-infra | 20:55 | |
| openstackgerrit | Ben Swartzlander proposed openstack-infra/project-config: Add manila-specs repo https://review.openstack.org/311853 | 20:56 |
| clarkb | I am flushing out as many of the d-g changes as I think I can with single reivew based on comments from other cores and things that were approved but needed small corrections | 20:56 |
| clarkb | hopeflly this will let the localrc -> local.conf work happen more smoothly over the next week or two | 20:57 |
| *** Goneri has quit IRC | 20:57 | |
| *** rfolco has quit IRC | 20:57 | |
| *** inc0 has quit IRC | 20:58 | |
| *** dmk0202 has joined #openstack-infra | 21:00 | |
| openstackgerrit | Jeff Grafton proposed openstack-infra/jenkins-job-builder: Update Google Cloud Storage publisher https://review.openstack.org/310080 | 21:00 |
| *** sdake_ has quit IRC | 21:03 | |
| *** sdake has joined #openstack-infra | 21:03 | |
| *** kgiusti has left #openstack-infra | 21:04 | |
| openstackgerrit | Jeff Grafton proposed openstack-infra/jenkins-job-builder: Update workspace-cleanup to support newer options https://review.openstack.org/310084 | 21:05 |
| *** heyongli_ has quit IRC | 21:05 | |
| *** esker has joined #openstack-infra | 21:05 | |
| *** skolekonov__ has joined #openstack-infra | 21:06 | |
| *** heyongli_ has joined #openstack-infra | 21:06 | |
| *** degorenko has quit IRC | 21:07 | |
| *** burgerk__ has joined #openstack-infra | 21:08 | |
| clarkb | jlvillal: re https://review.openstack.org/#/c/290171/1, how hard would it be to have ironic just log in the logs dir then delete that code entirely? | 21:08 |
| *** tojuvone_ has joined #openstack-infra | 21:08 | |
| clarkb | jlvillal: if that is an easy change it might be preferable but if it isn't easy I can push 290171 through | 21:08 |
| *** ccarmack1 has joined #openstack-infra | 21:08 | |
| *** netsin has joined #openstack-infra | 21:09 | |
| *** apoorvad has quit IRC | 21:09 | |
| *** Sukhdev has joined #openstack-infra | 21:09 | |
| *** netsin has quit IRC | 21:09 | |
| *** amrith has quit IRC | 21:09 | |
| openstackgerrit | Merged openstack-infra/project-config: Opt out of docs jobs for giftwrap https://review.openstack.org/311778 | 21:09 |
| *** apoorvad has joined #openstack-infra | 21:09 | |
| *** eliqiao_ has quit IRC | 21:09 | |
| *** netsin has joined #openstack-infra | 21:09 | |
| *** rodrigods has quit IRC | 21:10 | |
| *** Guest39041 has quit IRC | 21:10 | |
| *** rodrigods has joined #openstack-infra | 21:10 | |
| *** amrith has joined #openstack-infra | 21:10 | |
| *** Sukhdev has quit IRC | 21:10 | |
| *** ccarmack has quit IRC | 21:11 | |
| *** burgerk_ has quit IRC | 21:11 | |
| *** trown is now known as trown|outtypewww | 21:11 | |
| *** tojuvone has quit IRC | 21:12 | |
| *** dmk0202 has quit IRC | 21:12 | |
| *** eliqiao has joined #openstack-infra | 21:12 | |
| *** jlvillal has quit IRC | 21:12 | |
| *** jlvillal has joined #openstack-infra | 21:12 | |
| *** skolekonov__ has quit IRC | 21:13 | |
| jlk | clarkb: hi, are project-config things continuously deployed? Like should I expect that ^^ will be reality in the very near future? | 21:14 |
| *** amrith is now known as _amrith_ | 21:14 | |
| *** xyang1 has quit IRC | 21:14 | |
| clarkb | yes | 21:14 |
| clarkb | jlk: but also you could just add a simple set of docs to your project if it is urgent | 21:15 |
| jlk | and to get reviews to retest, its it just comment with "recheck" | 21:15 |
| clarkb | yes | 21:15 |
| jlk | perfect. Not super urgent, just wanted to understand the process | 21:15 |
| openstackgerrit | Vipin Balachandran proposed openstack/requirements: Add defusedxml to requirements https://review.openstack.org/311857 | 21:15 |
| clarkb | mordred: looks like the bluebox fips that leaked last week are gone, did you clean those out? | 21:15 |
| mordred | clarkb: I did | 21:15 |
| *** |-paul-| has joined #openstack-infra | 21:15 | |
| mordred | clarkb: there are some new ones | 21:15 |
| mordred | bluebox-sjc1 [u'dda930bf-3fc6-4223-8532-adeb48a3d76d', u'f0353ce5-6ce4-4f26-a73c-73addee86ddf'] | 21:16 |
| *** heyongli_ has quit IRC | 21:16 | |
| *** heyongli_ has joined #openstack-infra | 21:16 | |
| *** ldnunes has quit IRC | 21:17 | |
| *** BobBall has quit IRC | 21:17 | |
| clarkb | mordred: I want a project called bilge-pump now | 21:19 |
| mordred | clarkb: I can totally rename my clean-fips.py script to bilge-pump :) | 21:19 |
| clarkb | we can have it restart jenkinses for leaked thread cleanup, and delete leaked fips in clouds | 21:19 |
| mordred | clarkb: oooh. yeah. your thing is better | 21:19 |
| *** pradk has joined #openstack-infra | 21:19 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge https://review.openstack.org/311859 | 21:19 |
| jeblair | mordred, pabelanger, nibalizer: i'm thinking that what will happen with the vos release is that it will run to completion before it fails to update the vldb record | 21:20 |
| jeblair | mordred, pabelanger, nibalizer: so i'm starting to wonder if we should kill it, then manually unlock the volume, then vos release again | 21:21 |
| jeblair | since it's clear at this point, it's doing a complete resync | 21:21 |
| jeblair | and it's already passed the 10 hour credential timeout | 21:21 |
| clarkb | manually unlock means run ro off of whatever is currently there? | 21:22 |
| clarkb | zaro: looking at review.o.o memory use things are still looking good after the upgrade | 21:22 |
| clarkb | zaro: 21GB mean memory use. Max is 30GB (30GB is the current limit) | 21:22 |
| clarkb | zaro GC time is almost non existant | 21:23 |
| *** kzaitsev_mb has joined #openstack-infra | 21:23 | |
| pabelanger | jeblair: I'll defer to you, but sounds reasonable | 21:23 |
| *** aysyd has quit IRC | 21:23 | |
| nibalizer | jeblair: like pabelanger im hardly an expert | 21:24 |
| nibalizer | but sure | 21:24 |
| nibalizer | i support killing it | 21:24 |
| pabelanger | clarkb: only fedora-23 and debian-jessie left for dib-image-build | 21:24 |
| * nibalizer has no faith | 21:24 | |
| pabelanger | clarkb: err image-build | 21:24 |
| nibalizer | no faith in things still working normally | 21:24 |
| *** BobBall has joined #openstack-infra | 21:25 | |
| *** asettle has joined #openstack-infra | 21:25 | |
| *** BobBall is now known as Guest78717 | 21:25 | |
| clarkb | jeblair: did you see that the zuul 2.5 stack failed in the gate on its first attempt in? | 21:26 |
| clarkb | jeblair: I continued to review the changes but didn't approve any more of them. | 21:26 |
| *** heyongli_ has quit IRC | 21:26 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Update multinode setup doc to VXLAN https://review.openstack.org/286880 | 21:26 |
| *** julim has quit IRC | 21:26 | |
| *** heyongli_ has joined #openstack-infra | 21:26 | |
| pabelanger | ianw: if you are around tonight, would use some help debugging: http://paste.openstack.org/show/495912/ | 21:26 |
| pabelanger | ianw: trying precise ubuntu-minimal image | 21:27 |
| mordred | jeblair: I agree with nibalizer and pabelanger | 21:27 |
| mordred | jeblair: I am unsure of the full consequences of killing and restarting the release after manually removing the lock | 21:27 |
| mordred | jeblair: but as the current operation cannot succeed | 21:27 |
| mordred | jeblair: I would be hard pressed to describe how it could be a WORSE outcome | 21:27 |
| nibalizer | yah | 21:28 |
| nibalizer | there is also some afs stuff in the mirror.1.gz log file | 21:28 |
| *** lakshmiS has quit IRC | 21:29 | |
| nibalizer | /var/log/bandersnatch/mirror.log.1.gz, even | 21:29 |
| nibalizer | Starting ForwardMulti from 536870932 to 536870932 on afs01.ord.openstack.org (full release). | 21:29 |
| nibalizer | whatever that means | 21:29 |
| nibalizer | me thinks we should put more 'date' in our scripts | 21:29 |
| jeblair | clarkb: no, the unlock doesn't affect which volumes are used -- currently the vldb says "dfw has new, ord has old" so clients will use dfw and ignore ord | 21:30 |
| jeblair | clarkb: when we unlock the volume, those flags will persist | 21:30 |
| jeblair | clarkb: and the next vos release will copy the (existing) clone volume to ord (again) | 21:31 |
| jeblair | clarkb: and when that completes, assuming it still has credentials (which it should if we run it on the fileserver) it will update the vldb and clear the old/new flags | 21:31 |
| jeblair | so both copies will be used | 21:31 |
| clarkb | gotcha so manual run is mostly to run it on the fileserver with local creds | 21:32 |
| *** weshay has quit IRC | 21:33 | |
| jeblair | clarkb: i did not see the failure, but the zuulv2.5 code is not touched at all by tests, so i suspect either nondeterminism or external forces at play. | 21:33 |
| jeblair | if anyone wants to look into that or recheck this week, that's fine. otherwise, i will get back to zuulv2.5 next week | 21:34 |
| jeblair | nibalizer: the last dated entry is when the vos release started; all of the undated stuff after that is the output of the vos release command | 21:34 |
| *** jheroux has quit IRC | 21:35 | |
| clarkb | I will likely continue to try working through the stack doing reviews. | 21:35 |
| jeblair | nibalizer: "Starting ForwardMulti" means "copy this volume from here to there" | 21:35 |
| *** heyongli_ has quit IRC | 21:36 | |
| *** heyongli_ has joined #openstack-infra | 21:36 | |
| *** yamamoto has quit IRC | 21:37 | |
| *** Hal has joined #openstack-infra | 21:37 | |
| nibalizer | jeblair: what do the numbers mean? are they versions? | 21:37 |
| *** yamamoto has joined #openstack-infra | 21:38 | |
| nibalizer | i guess it is the same number | 21:38 |
| jeblair | nibalizer: there are volume ids | 21:38 |
| *** Hal is now known as Guest31859 | 21:38 | |
| jeblair | nibalizer: so yeah, it's the volume id of the ro volume, which is the same on both servers | 21:38 |
| *** oanson has quit IRC | 21:38 | |
| jeblair | (a vos copy would start the same kind of transaction, but with different volume ids) | 21:38 |
| nibalizer | so i sitll need more dns entries... | 21:39 |
| nibalizer | can we jetison all of infracloud? that stuff is all getting renamed ya? | 21:39 |
| *** markvan has quit IRC | 21:39 | |
| nibalizer | crinkle: yo<tab> rcarrillocruz ^? | 21:40 |
| clarkb | nibalizer: it is all getting reIP'd so even if it doesn't get renamed we are still needing new records for everything | 21:40 |
| *** yamamoto_ has joined #openstack-infra | 21:40 | |
| fungi | nibalizer: clarkb: crinkle: rcarrillocruz: my recollection was we already agreed it was fine to delete all the current *.ic.openstack.org a and aaaa resource records since they'll have to be updated with new addresses later anyway | 21:41 |
| pabelanger | clarkb: jeblair: I only had an issue with 307515 in the stack while testing. The rest seemed fine. Even added a few patches a top it for ansible | 21:42 |
| fungi | and creating vs modifying them is an equivalent amount of work | 21:42 |
| fungi | and recreating them means we can do it in a separate zone and not hit the maximum record limit for zones in rackspace's dns manglement implementation | 21:42 |
| *** yamamoto has quit IRC | 21:43 | |
| nibalizer | yah | 21:43 |
| rcarrillocruz | Yup, fine by me | 21:43 |
| clarkb | nibalizer: can you point me at the traceback/story for the launch node fail you ran into? | 21:43 |
| clarkb | nibalizer: I want ot make sure that gets fixed asap | 21:44 |
| *** thorst has quit IRC | 21:44 | |
| * clarkb checks email | 21:44 | |
| fungi | nibalizer: as to your earlier complaints about being unable to figure out how to get the rackdns client to list more than the first 100 records, i'm pretty sure i came to the conclusion some time ago that it's a bug (doesn't understand the pagination in their api maybe?) | 21:45 |
| clarkb | https://storyboard.openstack.org/#!/story/2000569 email so good | 21:45 |
| nibalizer | fungi: yah | 21:45 |
| nibalizer | https://github.com/kwminnick/rackspace-dns-cli/issues/3 | 21:45 |
| nibalizer | reported 2ish years ago | 21:45 |
| nibalizer | so i tried to patch it, found that setup.py install fails on that project these days.. | 21:46 |
| *** thorst has joined #openstack-infra | 21:46 | |
| *** heyongli_ has quit IRC | 21:46 | |
| *** heyongli_ has joined #openstack-infra | 21:47 | |
| rcarrillocruz | sigh, I wish we had some cloud that used Designate v2 | 21:47 |
| jeblair | nibalizer, clarkb, mordred, pabelanger: i killed the vos release; killed davolser on each of the fileservers (to abort the transfer); unlocked the volume (vos unlock mirror.pypi); and ran vos release on the fileserver in screen | 21:48 |
| mordred | jeblair: woot | 21:48 |
| nibalizer | jeblair: wooo | 21:48 |
| nibalizer | afs wizardry: incanted | 21:48 |
| *** Sukhdev has joined #openstack-infra | 21:48 | |
| jeblair | (bos restarte davolserv automatically, so manually restarting it is not necessary) | 21:48 |
| nibalizer | thanks for hopping in on that | 21:48 |
| jeblair | i also have the bandersnatch lock in screen on mirror-update | 21:49 |
| jeblair | so i will re-enable puppet now | 21:49 |
| *** jang has quit IRC | 21:49 | |
| *** ccamacho has quit IRC | 21:49 | |
| *** Sukhdev has quit IRC | 21:49 | |
| jeblair | nibalizer: np -- i'm trying to help as much as i can today, because i'm only less available as the week goes on | 21:49 |
| nibalizer | sure | 21:50 |
| *** thorst has quit IRC | 21:50 | |
| nibalizer | im sure you'll come back on monday to paul and i knowning more about afs haha | 21:50 |
| openstackgerrit | Merged openstack-infra/shade: Add Designate recordsets support https://review.openstack.org/306502 | 21:50 |
| *** amitgandhinz has quit IRC | 21:51 | |
| jeblair | i hope so, but only because it was the most fun thing you could think to do :) | 21:51 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: tripleo.sh: deploy missing deps to run rdopkg https://review.openstack.org/311868 | 21:51 |
| openstackgerrit | Merged openstack-infra/devstack-gate: support py3 https://review.openstack.org/304572 | 21:51 |
| jeblair | #status log manual vos release of pypi mirror started in screen on fileserver; see https://etherpad.openstack.org/p/fix-afs | 21:51 |
| openstackstatus | jeblair: finished logging | 21:51 |
| *** yamamoto_ has quit IRC | 21:53 | |
| *** yamahata has quit IRC | 21:53 | |
| *** hashar has quit IRC | 21:54 | |
| *** yamahata has joined #openstack-infra | 21:54 | |
| *** nelsnelson has quit IRC | 21:54 | |
| *** yamamoto has joined #openstack-infra | 21:55 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Fix '-f: command not found' error https://review.openstack.org/271463 | 21:56 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Consider to log a package list for pip3 https://review.openstack.org/246017 | 21:56 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Remove doubled configuration in local.conf https://review.openstack.org/297475 | 21:56 |
| *** banix has joined #openstack-infra | 21:56 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: tripleo.sh: deploy missing deps to run rdopkg https://review.openstack.org/311868 | 21:57 |
| *** heyongli_ has quit IRC | 21:57 | |
| *** earlephilhwoer has quit IRC | 21:57 | |
| *** heyongli_ has joined #openstack-infra | 21:57 | |
| *** tpsilva has quit IRC | 21:57 | |
| *** earlephilhower has joined #openstack-infra | 21:58 | |
| *** Sukhdev has joined #openstack-infra | 21:58 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Properly handle volumes in launch node https://review.openstack.org/311872 | 21:58 |
| clarkb | nibalizer: ^ I *think* that will fix it but it is untested | 21:58 |
| clarkb | nibalizer: not sure if you are booting new instances right now, but if so maybe give that a go | 21:59 |
| *** regXboi has quit IRC | 21:59 | |
| *** tonytan4ever has joined #openstack-infra | 22:00 | |
| *** pilgrimstack has quit IRC | 22:01 | |
| *** ddieterly is now known as ddieterly[away] | 22:01 | |
| nibalizer | well we are still out of dns | 22:02 |
| clarkb | nibalizer: ya but you can delete records easy enough | 22:02 |
| clarkb | want me to help? | 22:02 |
| nibalizer | so kinda need to https://xkcd.com/1205/ the fixing dns | 22:02 |
| nibalizer | i think morgan is on the rackdns thing | 22:03 |
| nibalizer | we do need to spin up a new puppetdb server once i work the kinks out | 22:03 |
| nibalizer | and a new storyboard-dev server | 22:03 |
| *** ddieterly[away] is now known as ddieterly | 22:03 | |
| *** thorst has joined #openstack-infra | 22:03 | |
| clarkb | nibalizer: its about 2 minutes of button clicking if you just want it done | 22:03 |
| clarkb | derp I forgot to put the story in my change /me updates things | 22:04 |
| nibalizer | clarkb: oh? i deleted one record and it was anoying | 22:04 |
| pabelanger | jeblair: TIL: kill davolser | 22:04 |
| nibalizer | hundreds sounds :( :( | 22:04 |
| *** earlephilhower has quit IRC | 22:04 | |
| clarkb | nibalizer: I think you just select them then do a bulk op | 22:04 |
| *** earlephilhower has joined #openstack-infra | 22:05 | |
| *** ddieterly has quit IRC | 22:05 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Properly handle volumes in launch node https://review.openstack.org/311872 | 22:05 |
| nibalizer | clarkb: ahh | 22:06 |
| clarkb | ok story and commit updated | 22:06 |
| clarkb | if you can use the above change to see if that makes stuff happy that would be excellent | 22:07 |
| *** heyongli_ has quit IRC | 22:07 | |
| *** mriedem has quit IRC | 22:07 | |
| *** heyongli_ has joined #openstack-infra | 22:07 | |
| jeblair | pabelanger: ya, 'vos release' basically just updates the vldb, then asks the sending volume server to copy a volume to the receiving volume server, then they handle it on their own while the vos release command sits there and waits for completion; so killing 'vos release' doesn't actually cause the volume servers to stop their work | 22:07 |
| *** thorst has quit IRC | 22:08 | |
| *** pvaneck has quit IRC | 22:08 | |
| jeblair | pabelanger: (and when vos release receives completion, it updates the vldb again -- but if it has been too long, it tries to do that with expired tokens -- which is the situation we are in that's causing us to have to do this) | 22:08 |
| pabelanger | jeblair: okay, good to know. not that I'll be killing processes between ord and dfw often | 22:08 |
| jeblair | yeah, it's not a good habit to get into | 22:09 |
| pabelanger | jeblair: makes sense | 22:09 |
| clarkb | and all this started due to a network issue between ord and dfw? | 22:09 |
| clarkb | we have good evidence that this is a chronic problem at least outside of afs :/ | 22:09 |
| jeblair | clarkb: i think so -- at least, according to the log, i believe a final vldb update failed | 22:10 |
| pabelanger | I'm curious what the interlink between ord and dfw looks like | 22:10 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate: Stop execution if ansible fails to install https://review.openstack.org/311876 | 22:11 |
| pabelanger | or is it a cross-connect | 22:11 |
| clarkb | jlvillal: you around now? did you see my question about the ironic old logs change? | 22:11 |
| jeblair | i've been mulling over the best way to rejigger the vos releases so we don't have to do this manually; i think i have some good ideas, and can work on that when i'm back | 22:11 |
| mordred | ++ | 22:11 |
| *** fawadkhaliq has quit IRC | 22:11 | |
| * mordred has heard some of the ideas and supports them | 22:11 | |
| jlvillal | clarkb: Oh hey. I did not see your question. | 22:11 |
| * jlvillal looks at scrollback... | 22:11 | |
| *** fawadkhaliq has joined #openstack-infra | 22:11 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Fixed link to l3 agent in tempest-logs https://review.openstack.org/310902 | 22:12 |
| clarkb | jlvillal: and for this new change, maybe we can try making d-g use errexit with an exit handler for log cleanup | 22:12 |
| clarkb | jlvillal: though that is bound to make things unhappy until we clean up all the things that have been slipping through | 22:12 |
| jlvillal | clarkb: Okay. I'll have to learn about that. | 22:13 |
| *** yamamoto has quit IRC | 22:13 | |
| clarkb | pretty sure non zero exit codes are expected from the workspace setup function for example :/ | 22:13 |
| jlvillal | clarkb: I don't see your question on the ironic old log changes in my scrollback. | 22:14 |
| clarkb | jlvillal: basically can we make ironic log to the log dir and avoid moving anything around at all? | 22:14 |
| jlvillal | Well at least not addressed to me, that I can see | 22:14 |
| clarkb | jlvillal: it is a bit odd that ironic needs a special log dir which leads to these issues | 22:14 |
| jlvillal | clarkb: It has been that way for as long as I can remember. They put the baremetal VM logs into a separate directory. | 22:15 |
| clarkb | jlvillal: ya just asking if we can stop | 22:15 |
| *** markvoelker has quit IRC | 22:15 | |
| clarkb | then we can delete code instead of adding it | 22:15 |
| jlvillal | Hmmm. Maybe, would require a few changes in different places I think. | 22:16 |
| jlvillal | Since there is a hook in place that renames the log file when the VM is shut-down. | 22:16 |
| jlvillal | clarkb: https://github.com/openstack/ironic/blob/master/devstack/files/hooks/qemu | 22:17 |
| *** heyongli__ has joined #openstack-infra | 22:17 | |
| clarkb | jlvillal: looks like if we set LOG_DIR properly it would just work? | 22:17 |
| jlvillal | clarkb: Possibly/probably. Not sure. | 22:18 |
| jlk | argh | 22:18 |
| jlk | is there a way to keep nose unittests from testing a particular python import? | 22:19 |
| jlk | or should I disable the py34 job somehow | 22:19 |
| clarkb | jlvillal: https://github.com/openstack/ironic/blob/master/devstack/lib/ironic#L686-L688 appears to be where that ends up getting actually written | 22:19 |
| mordred | jlk: well... we don't usually use nose anywhere anymore - but it is usually possible to control what testr does if you're using that ... | 22:19 |
| *** yamamoto has joined #openstack-infra | 22:19 | |
| mordred | jlk: repo in question and I can take a look? | 22:19 |
| jlk | https://review.openstack.org/#/c/311862 | 22:20 |
| clarkb | jlvillal: and there is an rm -rf associated with that path :( | 22:20 |
| clarkb | jlvillal: it is just annoying that we solve this problem for devstack then other services decide they are more specialer and we need to resolve it again | 22:20 |
| jlk | it's giftwrap, and nose is chewing through imports and finding bad syntax | 22:20 |
| openstackgerrit | Dirk Mueller proposed openstack/requirements: Add spec_cleaner for rpm-packaging style checks https://review.openstack.org/311468 | 22:21 |
| jlvillal | clarkb: Understandable. I'm not sure if I have time right now to work on it. But I can put it on my todo list | 22:21 |
| clarkb | jlvillal: ok, I will go ahead and +2 the d-g change | 22:21 |
| mordred | jlk: yah - I'm guessing it doesn't have .tox in its excludes list | 22:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Move ubuntu-xenial voting for infra jobs https://review.openstack.org/311878 | 22:21 |
| mordred | jlk: I'm looking now | 22:21 |
| jlvillal | clarkb: Thanks | 22:21 |
| jroll | clarkb: keep in mind we can generate a whole bunch of logs there | 22:21 |
| jroll | clarkb: typical gate job should only be... maybe 10? | 22:21 |
| *** heyongli_ has quit IRC | 22:21 | |
| jroll | but from a day of gate debugging... | 22:22 |
| jroll | $ ls ../ironic-bm-logs/ | wc -l | 22:22 |
| devananda | jroll: won't that change when we are able to do tempest full though? | 22:22 |
| jroll | 167 | 22:22 |
| jroll | yep | 22:22 |
| clarkb | jroll: ya but you only ever want those logs if you are ironicing anyways and they are only generated in that case | 22:22 |
| clarkb | jroll: basically its fine ebcause if ironic then ironic logs | 22:22 |
| clarkb | if not ironic then no logs | 22:22 |
| clarkb | but this copy pasta of code that is only different because ironic has a special logs dir is silly | 22:22 |
| jroll | clarkb: yeah, I like spammy things, I don't mind. but some people can't handle extra log messages so how will they deal with a million files? :) | 22:22 |
| devananda | clarkb: having them not pollute the main log dir is appealing to me | 22:23 |
| jroll | see :) | 22:23 |
| clarkb | devananda: you can still do that | 22:23 |
| devananda | oh? | 22:23 |
| clarkb | devananda: and put them in logs/ironic/ | 22:23 |
| jroll | anyway, got a link to this patch? | 22:23 |
| * JayF calls it, 2y from now someone is going to say it's silly for us to have dozens of bm console logs in the parent dir | 22:23 | |
| clarkb | https://review.openstack.org/#/c/290171/1/functions.sh is the patch | 22:23 |
| jroll | thanks | 22:23 |
| jroll | OH | 22:23 |
| clarkb | its literally mv path/ironic-logs-are-specialer path/logs/ironic-logs | 22:23 |
| jroll | logs/ironic-bm-logs/ is appealing to me | 22:23 |
| jroll | yeah | 22:24 |
| clarkb | instead of just using logs/ironic-logs from the get go | 22:24 |
| clarkb | like a sane person | 22:24 |
| jroll | +2 do it, but let's still call it ironic-bm-logs | 22:24 |
| jroll | for muscle memory sake if nothing else | 22:24 |
| jroll | (those are the qemu console logs for the VMs we use) | 22:24 |
| devananda | also lets mv, not cp | 22:25 |
| mordred | jlk: https://review.openstack.org/311880 | 22:25 |
| jroll | clarkb: I don't think that patch is doing the right thing, though | 22:25 |
| *** burgerk__ has quit IRC | 22:25 | |
| mordred | jlk: feel free to squash that into your change if it's helpful | 22:25 |
| mordred | jlk: I do not need the stackalytics stats :) | 22:25 |
| jlk | lol | 22:25 |
| *** jamielennox|away is now known as jamielennox | 22:25 | |
| clarkb | jroll: what is the issue? | 22:26 |
| jroll | clarkb: that's still a special case? | 22:26 |
| mordred | jlk: oh! wait. it's still not excluding | 22:26 |
| devananda | ah, nvm. everything else here is copying | 22:26 |
| mordred | one sec | 22:26 |
| jlk | yeah, this.. yeah. | 22:26 |
| mordred | jlk: there are at least 2 issues here which are legit | 22:26 |
| clarkb | jroll: I think it is doing what the new side does and this just adds in support for grenade things | 22:26 |
| mordred | jlk: in that the underlying library is actually not python3 compatible | 22:26 |
| clarkb | jroll: functionally it should be fine | 22:27 |
| clarkb | jroll: if you agree I will go ahead and approve so that logs function then we can worry about cleaning stuff up and rearranging | 22:27 |
| jlk | that's what I'm seeing now | 22:27 |
| jlk | I think I need to disable the 34 job | 22:27 |
| jlk | or non-vote it | 22:27 |
| mordred | jlk: yah. or update to a newer gitpython | 22:27 |
| mordred | you're pinning to a very old one | 22:27 |
| *** heyongli__ has quit IRC | 22:27 | |
| jroll | clarkb: I mean, that patch is fine, but why not make that path under logs/ to begin with | 22:27 |
| mordred | but you could do that as a follow on when you're not trying to get something else done | 22:27 |
| jlk | was just noticing that | 22:28 |
| jlk | not sure why | 22:28 |
| clarkb | jroll: ya I think it may be partially a devstack thing | 22:28 |
| *** heyongli__ has joined #openstack-infra | 22:28 | |
| jroll | -echo "IRONIC_VM_LOG_DIR=$BASE/$localrc_oldnew/ironic-bm-logs" >>"$localrc_file" | 22:28 |
| jroll | +echo "IRONIC_VM_LOG_DIR=$BASE/$localrc_oldnew/logs/ironic-bm-logs" >>"$localrc_file" | 22:28 |
| jroll | (no clue why that's in d-s-g) | 22:28 |
| clarkb | devstack is splatting into a gross place and we are trying to rectify? | 22:28 |
| *** david-lyle has quit IRC | 22:28 | |
| jroll | nope, it's all dsg | 22:28 |
| pabelanger | clarkb: welp, time to pin debian-jessie mirrors to something other then http://httpredir.debian.org/debian/ | 22:28 |
| mordred | jlk: I get two fixable errors with gitpython 2.0.1 | 22:28 |
| pabelanger | clarkb: 2 failure today, with more yesterday | 22:28 |
| jroll | clarkb: https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L342 | 22:29 |
| jlk | nod | 22:29 |
| clarkb | jroll: ya because of how devstack works | 22:29 |
| jlk | I'm finding other things to fix up too | 22:29 |
| clarkb | jroll: see line 818 of the patch in question | 22:29 |
| jlk | I think this is going to be a chain of commits | 22:29 |
| clarkb | jroll: basically d-g is doing the ugly thing because everything else does the ugly thing | 22:29 |
| jroll | clarkb: I guess, idk why it can't be a devstack config though | 22:29 |
| jroll | clarkb: I thought the original intent was "put these in the right place to begin with so we don't need to do this extra cp" | 22:30 |
| jroll | but I see it now | 22:30 |
| mordred | jlk: I've got a clean 34 run now - checking I didn't break 27 | 22:30 |
| jlk | I'm getting close, we're racing :) | 22:30 |
| * clarkb is sad that we have to do this much path munging to copy logs properly | 22:30 | |
| jroll | ++ | 22:31 |
| mordred | jlk: https://review.openstack.org/311882 BOOM! | 22:31 |
| mordred | jlk: :) | 22:31 |
| clarkb | jroll: one time I wanted /etc/openstack/nova etc so that we could just copy /etc/openstack, that didn't get very far | 22:31 |
| jroll | clarkb: that's why I thought you were trying to *not* do this extra munging, I figured everything else was happy :/ | 22:31 |
| jroll | heh | 22:31 |
| mordred | jlk: now, I have NO IDEA if giftwrap actually works with latest GitPython :) | 22:31 |
| jlk | mordred: I had those, just not in a commit yet | 22:31 |
| jlk | yeah I'm going to test that too | 22:31 |
| jroll | clarkb: anyway, I gotta run but that seems fine | 22:32 |
| craige | o/ | 22:32 |
| clarkb | jroll: we are looking at some fairly largish changes to devstack and d-g this cycile so trying to get on top of current changes and see if we can make things bettererer | 22:32 |
| mordred | jlk: I have excitement for our new giftwrap overlords | 22:32 |
| mordred | look it's a craige | 22:32 |
| clarkb | before we make everything merge conflict | 22:32 |
| jroll | clarkb: awesome | 22:32 |
| mordred | clarkb: speaking of those | 22:32 |
| mordred | clarkb: I've been noodling on the mirror information devstack-gate/devstack thing | 22:32 |
| mordred | clarkb: hoping to have a proposal for specifics by tomorrow (I've hated all of my ideas today) | 22:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Pin debian-jessie deb mirror to ftp.us.debian.org https://review.openstack.org/311883 | 22:33 |
| pabelanger | clarkb: I believe that is the correct syntax ^ | 22:33 |
| *** lucasagomes has quit IRC | 22:36 | |
| clarkb | pabelanger: there is something funny about http://ftp.fqdn | 22:36 |
| pabelanger | clarkb: don't think so, I was reading from https://www.debian.org/mirror/list | 22:37 |
| *** heyongli__ has quit IRC | 22:38 | |
| clarkb | ya it is correct, I just get a kick out of url and hostname not matching | 22:38 |
| clarkb | we are going to talk http to the ftp server | 22:38 |
| *** heyongli__ has joined #openstack-infra | 22:38 | |
| * fungi enjoys the history there | 22:38 | |
| *** fawadkhaliq has quit IRC | 22:38 | |
| mordred | STAB STAB STAB STAB STAB | 22:38 |
| fungi | it's "we stuck an http service on our ftp server, serving the same file tree so you could get it via this new-fangled protocol" | 22:39 |
| mordred | I thought we raised the openid auth timeout on gerrit? | 22:39 |
| pabelanger | we could do ftp vs http | 22:39 |
| pabelanger | dealers choice | 22:39 |
| clarkb | mordred: requires a gerrit restart | 22:39 |
| clarkb | mordred: not sure if anyone has done that second step since change went in | 22:39 |
| mordred | I just lost a non-trivial comment I tried to to write on a review because gerrit has this obsession with logging me out and making me click 3 times to log in again | 22:40 |
| *** fawadkhaliq has joined #openstack-infra | 22:40 | |
| mordred | I would like to set it to never expire | 22:40 |
| clarkb | mordred: there is a "fix for that" | 22:40 |
| mordred | because the expiration is completely useless | 22:40 |
| clarkb | hit go anonymous, copy comments, then log in | 22:40 |
| mordred | it solves absolutely nothing | 22:40 |
| clarkb | and supposedly newer gerrit will save the stuff to a cookie and restore it post log in | 22:40 |
| fungi | we've surprisingly _not_ had to restart gerrit recently, thanks to bigger server | 22:40 |
| pabelanger | nice | 22:40 |
| mordred | clarkb: awesome. it's great that it doesn't do that across the login boundary | 22:41 |
| mordred | since "go anonymous" would not imply to me that I wouldn't lose my comment | 22:41 |
| mordred | RAGE STAB STAB | 22:41 |
| * mordred goes to delete review.openstack.org | 22:41 | |
| *** xarses has quit IRC | 22:41 | |
| *** pradk has quit IRC | 22:41 | |
| *** lucasagomes has joined #openstack-infra | 22:41 | |
| *** banix has quit IRC | 22:42 | |
| *** Guest31859 has quit IRC | 22:42 | |
| StevenK | mordred: SURE, that will fix EVERYTHING | 22:42 |
| dstufft | mordred: hilariously, Logged In -> Logged Out is arguably the time when you _should_ be clearing session data. | 22:43 |
| anteaya | yay for not having to restart gerrit recently? | 22:43 |
| dstufft | (Assuming "Go Anonymous" is some logged out thing) | 22:43 |
| *** pradk has joined #openstack-infra | 22:44 | |
| clarkb | dstufft: your session expires after 24 hours currently and when you try to do a thing that requires a new login eg leaving a comment it asks you to login (and implicitly lose your data) or go anonymous (and implicitly keep the data) | 22:44 |
| dstufft | clarkb: oh, so this isn't a log out button? | 22:44 |
| clarkb | nope | 22:45 |
| dstufft | (can you tell I totally use gerrit a lot) | 22:45 |
| dstufft | ok ignore me then ;P | 22:45 |
| fungi | just google developers thinking it's insane anyone would stay logged into a web service for 24 hours | 22:45 |
| mordred | dstufft: this is the "for some reason we think making you click a button once a day is more better" | 22:45 |
| clarkb | this was one of the things I pointed out when upstream finally went "well what is wrong with the new interface bceause we think it is amazing" | 22:46 |
| StevenK | But it turns into a button, a redirect another button and another redirect | 22:46 |
| clarkb | and they responded with "well we did fix that one issue very recently, please wait for next gerritrelease" | 22:46 |
| jlk | mordred: new fixes, I had to change some code to make this actually work: https://review.openstack.org/#/c/311862 | 22:46 |
| clarkb | supposedly the mobile diff view will stop sucking in a newer release too | 22:46 |
| anteaya | can you gertty on your phone? | 22:47 |
| clarkb | anteaya: you can ssh to a gertty running elsewhere | 22:47 |
| dstufft | mordred: forcing a new login periodically is not entirely wrong, though if you're going to do that you should take care to not lose state when people submit on old pages | 22:47 |
| clarkb | which I have considered doing, but have never really seriously tried | 22:47 |
| *** heyongli__ has quit IRC | 22:48 | |
| nibalizer | ok morgan and i are throwing in the towel | 22:48 |
| * nibalizer presses buttons | 22:48 | |
| anteaya | clarkb: ah yes | 22:49 |
| *** piet has quit IRC | 22:49 | |
| nibalizer | haha now the web ui is erroring | 22:49 |
| nibalizer | yayyyy | 22:49 |
| *** |-paul-| has quit IRC | 22:49 | |
| anteaya | clarkb: I also have never tried | 22:49 |
| anteaya | nibalizer: because that is what web ui's do | 22:49 |
| *** akshai has quit IRC | 22:50 | |
| nibalizer | 16 out of 18 requests succeeded! | 22:51 |
| *** heyongli_ has joined #openstack-infra | 22:53 | |
| jeblair | mordred: you could change the session cache duration, but i guess it's easier to just be mad at everyone | 22:54 |
| openstackgerrit | Ben Swartzlander proposed openstack-infra/project-config: Add manila-specs repo https://review.openstack.org/311853 | 22:54 |
| nibalizer | i'm wondering if we need ppbelectsionsregistration.openstack.org or ppbelectionsregistration.openstack.org | 22:55 |
| morgan | fixed it! | 22:55 |
| nibalizer | testme.openstack.org also seems like maybe we don't need it | 22:55 |
| morgan | python-novaclient<=2.15.0 (cc nibalizer ) | 22:55 |
| nibalizer | woo | 22:55 |
| *** davidlenwell has quit IRC | 22:55 | |
| anteaya | jeblair: he has a reputation to maintain | 22:56 |
| *** rodrigods has quit IRC | 22:56 | |
| *** rodrigods has joined #openstack-infra | 22:56 | |
| morgan | jeblair: i vote mad at everyone | 22:56 |
| morgan | jeblair: :P | 22:56 |
| jeblair | mordred: http://git.openstack.org/cgit/openstack-infra/puppet-gerrit/tree/templates/gerrit.config.erb | 22:57 |
| *** rbrndt_ has quit IRC | 22:57 | |
| jeblair | mordred: it *used* to be 1 day, but then lifeless set it to 2 minutes. | 22:57 |
| jlk | so, if I'm the PTL, and one of hte few engineers working on a project, is it wrong to +2 or +workflow my own changes? | 22:57 |
| *** rbrndt_ has joined #openstack-infra | 22:58 | |
| jeblair | mordred: you approved that change: https://review.openstack.org/295530 | 22:58 |
| *** rbrndt_ has quit IRC | 22:58 | |
| *** heyongli_ has quit IRC | 22:58 | |
| *** heyongli_ has joined #openstack-infra | 22:58 | |
| jeblair | mordred: considering all of that, i think that, in retrospect, that rage was not warranted and would appreciate less rageing and stabbing. | 23:00 |
| morgan | jlk: really depends | 23:00 |
| morgan | jlk: if you have critical mass of cores from disparate companies, we tend to have a soft rule about +2/+A | 23:01 |
| morgan | jlk: but to be perfectly honest, I am of the opinion we trust people in the community until they give us a reason not to | 23:01 |
| mordred | jeblair: fair enough | 23:01 |
| morgan | jlk: sometimes you need to +2/+A your own things at times | 23:01 |
| morgan | jlk esp. with a very small core team | 23:02 |
| jlk | alright, I'll use it sparingly | 23:02 |
| morgan | jlk: ++ | 23:02 |
| nibalizer | oh sweet, blessed pagination | 23:03 |
| jeblair | mordred: maybe the timeout should be increased to something like 2-3, or even 7 days? | 23:03 |
| *** davidlenwell has joined #openstack-infra | 23:03 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add Moshe Levi as sriov meeting chair https://review.openstack.org/311472 | 23:04 |
| jeblair | i'm not sure i dig '2 months' | 23:04 |
| *** xarses has joined #openstack-infra | 23:04 | |
| nibalizer | fungi: i'm thinking im just gonna post our entire openstack.org dns to a paste for everyone to look at | 23:04 |
| jeblair | (which is likely what lifeless was going for) | 23:04 |
| nibalizer | there are definitely some records we can lose | 23:04 |
| nibalizer | unless you can think of an issue with doing that | 23:04 |
| *** doug-fis_ has quit IRC | 23:04 | |
| *** doug-fish has joined #openstack-infra | 23:05 | |
| mordred | jeblair: yah - I believe he was going for 2 months, which seems fine to me - but my opinion on appropriate length should likely be discounted at the moment | 23:06 |
| mordred | jeblair: I'm happy to follow that change with one that sets to something more appropriate | 23:06 |
| *** thorst has joined #openstack-infra | 23:07 | |
| *** heyongli_ has quit IRC | 23:08 | |
| nibalizer | ERROR: OverLimit Retry... (HTTP 413) | 23:10 |
| nibalizer | how silly of me, thinking i could delete 1 record every 3 seconds | 23:11 |
| *** heyongli_ has joined #openstack-infra | 23:11 | |
| fungi | nibalizer: yeah, i sent 'round a list of those idk subdomains a while back (maybe a year ago?) and removed the ones people got back to me saying were unneeded. unfortunately a bunch of those are possibly created by the foundation staff and i never heard any confirmation i could delete them so left them there | 23:11 |
| fungi | nibalizer: and making a paste is fine--there's nothing "secret" in our domains | 23:12 |
| mordred | fungi: what about the secret domains though? | 23:12 |
| fungi | mordred: since i don't know about them, i'm guessing they're extremely secret | 23:12 |
| bswartz | openstack.xxx? | 23:13 |
| fungi | nibalizer: i'm cool with cleanup, just cautious since the infra team aren't the only people creating records in that domain | 23:14 |
| mordred | bswartz: sadly, someone already registered openstack.ninja | 23:15 |
| *** markvoelker has joined #openstack-infra | 23:16 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-gerrit: Change default login cache length to 7 days https://review.openstack.org/311888 | 23:16 |
| fungi | mordred: your repeated mention of "the gathering" makes me think of something other than openstack project teams ;) | 23:16 |
| * mordred taps three forests and summons a wall | 23:17 | |
| nibalizer | http://dpaste.com/2W5F0HC | 23:17 |
| *** ccarmack1 has quit IRC | 23:17 | |
| *** larainema has quit IRC | 23:17 | |
| nibalizer | (our paste server doesn't allow them to be as long as the file is) | 23:17 |
| bkero | The troll has flying. It attacks. | 23:17 |
| nibalizer | i'm getting rid of the infacloud ones (currently suffering from the api rate limiting blues) | 23:18 |
| bswartz | omg ninjas | 23:18 |
| *** larainema has joined #openstack-infra | 23:18 | |
| *** heyongli_ has quit IRC | 23:19 | |
| *** heyongli_ has joined #openstack-infra | 23:19 | |
| *** fawadkhaliq has quit IRC | 23:20 | |
| *** markvoelker has quit IRC | 23:21 | |
| pabelanger | clarkb: all dibs rebuilt for today. However, I have not uploaded any images to clouds but nodepool image-list appears happy (0 building) | 23:21 |
| pabelanger | clarkb: I wonder if that is not correct | 23:21 |
| *** dingyichen has joined #openstack-infra | 23:21 | |
| *** yamamoto has quit IRC | 23:21 | |
| clarkb | pabelanger: you can check the builder log to see if uploads all failed or whatever | 23:21 |
| clarkb | around now is when I expect them to be done though | 23:22 |
| *** Sukhdev has quit IRC | 23:22 | |
| clarkb | I would like to move our cron back a few hours too, /me will propose that change now | 23:22 |
| fungi | nibalizer: yes, that particular dns service regularly makes me want to just set up some bind 10 servers. and then i remember that i'd need to provide some mechanism for random other non-infra people at the foundation to be able to make near-instantaneous changes to the domain | 23:22 |
| *** yamamoto has joined #openstack-infra | 23:23 | |
| *** tonytan4ever has quit IRC | 23:23 | |
| jlk | question on work flow | 23:23 |
| *** mylu has joined #openstack-infra | 23:23 | |
| jlk | somebody sends up a review. It doesn't pass CI due to other issues. I fix those other issues (which moves master along). Does the original review have to be rebased, or will a recheck work? | 23:23 |
| fungi | jlk: recheck | 23:24 |
| nibalizer | fungi: so vexxhost | 23:24 |
| nibalizer | also lol @ 10 requests/second rate limiting | 23:24 |
| pabelanger | clarkb: okay, I see the uploads. neat | 23:24 |
| fungi | jlk: zuul attempts to merge the commit to the tip of the target branch, and tests it that way | 23:24 |
| jlk | perfect | 23:25 |
| *** thorst has quit IRC | 23:25 | |
| *** fawadkhaliq has joined #openstack-infra | 23:25 | |
| pabelanger | clarkb: also, https://review.openstack.org/#/c/311721/ when you have spare cycles. Starting to change out fedora-22 on tripleo CI | 23:25 |
| *** Sukhdev has joined #openstack-infra | 23:25 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Move image update cron back https://review.openstack.org/311890 | 23:26 |
| pabelanger | https://review.openstack.org/#/c/294339/ could also use a review, that is our nodepool-builder worker settings | 23:26 |
| clarkb | pabelanger: ^ something like that to shift everything back | 23:26 |
| pabelanger | clarkb: checking | 23:26 |
| *** pahuang has joined #openstack-infra | 23:27 | |
| pabelanger | clarkb: WFM | 23:27 |
| *** thorst has joined #openstack-infra | 23:28 | |
| *** heyongli_ has quit IRC | 23:29 | |
| *** heyongli_ has joined #openstack-infra | 23:29 | |
| asselin_ | fungi, jlk really? I though that was only for dependent pipeline, gate checks. I though check was exactly what the patch is based off of...similar to git review -d | 23:30 |
| *** mylu has quit IRC | 23:30 | |
| *** mylu has joined #openstack-infra | 23:30 | |
| jlk | crap, i need a gpg key... | 23:30 |
| clarkb | asselin_: yes, it tests forward state even in check | 23:31 |
| *** heyongli_ has quit IRC | 23:31 | |
| fungi | asselin_: really. there's little point in it testing your change with something other than the branch tip | 23:31 |
| *** sputnik13 has quit IRC | 23:31 | |
| clarkb | that forward state is based off of the target branch though, not off of a proposed future | 23:31 |
| *** gyee has quit IRC | 23:32 | |
| *** keedya has joined #openstack-infra | 23:33 | |
| jeblair | clarkb, asselin_, jlk: what clarkb says is true, except you can also use depends-on to pull another patch into the test scenario in check | 23:33 |
| *** dangers is now known as dangers_away | 23:34 | |
| fungi | and also it stacks your parent changes if there are any in between the change and the branch tip | 23:35 |
| *** Guest66363 has quit IRC | 23:35 | |
| *** mylu has quit IRC | 23:36 | |
| *** keedya has quit IRC | 23:37 | |
| *** keedya has joined #openstack-infra | 23:38 | |
| asselin_ | ok, good to know | 23:38 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Remove the ambassadors meeting https://review.openstack.org/310928 | 23:39 |
| *** sdake_ has joined #openstack-infra | 23:41 | |
| *** mriedem has joined #openstack-infra | 23:41 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Move Tacker team meeting to compensate DST https://review.openstack.org/311789 | 23:41 |
| *** yamamoto has quit IRC | 23:42 | |
| *** sdake has quit IRC | 23:43 | |
| *** gordc has quit IRC | 23:43 | |
| lifeless | jeblair: argh. 2minutes? EIFAIL | 23:44 |
| lifeless | jeblair: yes, I was going for 2 months | 23:44 |
| lifeless | jeblair: is there a fix-up patch already up, or should I submit one? | 23:45 |
| jeblair | lifeless: mordred has https://review.openstack.org/311888 | 23:45 |
| lifeless | jeblair: ack, thanks | 23:45 |
| *** flepied has joined #openstack-infra | 23:45 | |
| fungi | oh, yikes | 23:45 |
| jeblair | so, erm, nobody restart gerrit :) | 23:46 |
| *** doug-fish has quit IRC | 23:46 | |
| *** banix has joined #openstack-infra | 23:46 | |
| fungi | roger that | 23:47 |
| jeblair | in fact... | 23:47 |
| fungi | approving as soon as i see check results pass for this | 23:47 |
| *** doug-fish has joined #openstack-infra | 23:47 | |
| clarkb | https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#cache.name.maxAge says 'days' is good | 23:47 |
| jeblair | #status log due to an error in https://review.openstack.org/295530 which will be corrected in https://review.openstack.org/311888 gerrit should not be restarted until the second change lands | 23:47 |
| openstackstatus | jeblair: finished logging | 23:47 |
| jeblair | infra-root: ^ | 23:47 |
| * jeblair uses tools | 23:47 | |
| fungi | tools! | 23:48 |
| clarkb | http://logs.openstack.org/88/311888/1/check/gate-puppet-gerrit-puppet-beaker-rspec-ubuntu-trusty/c684427/console.html#_2016-05-02_23_45_51_910 | 23:48 |
| clarkb | I am thinking that test fail is due to broken tests/manifests | 23:48 |
| jeblair | clarkb: i was afraid of that, but haven't quite gotten into the habit of finding where the errors actually start in the beaker tests | 23:49 |
| *** heyongli_ has joined #openstack-infra | 23:49 | |
| clarkb | jeblair: it is right where I linked, looks like the reindex step timed out? | 23:49 |
| jeblair | (search for "Failures:" seems to approximate that) | 23:49 |
| jeblair | clarkb: yeah, i'm just saying i scanned a few hundred lines and gave up :) | 23:49 |
| clarkb | which is odd considering there should be nothing to index | 23:49 |
| jeblair | i also saw some bouncycastle errors and thought "internet?") | 23:50 |
| *** doug-fis_ has joined #openstack-infra | 23:51 | |
| *** mylu has joined #openstack-infra | 23:51 | |
| *** chlong has joined #openstack-infra | 23:52 | |
| *** doug-fish has quit IRC | 23:52 | |
| clarkb | the reindex fail would fail to create the init script symlink if there is a subscribe and refreshonly | 23:52 |
| clarkb | actually if the init fails then there may not be anything to symlink to, reading the manifest I Think that is actually how it is failing. The init (index step specifically) fails leaving the symlink dangling | 23:54 |
| *** yamamoto has joined #openstack-infra | 23:54 | |
| jeblair | clarkb: yeah, i think it may actually be the bouncycastle thing | 23:55 |
| *** Swami has quit IRC | 23:55 | |
| clarkb | http://logs.openstack.org/88/311888/1/check/gate-puppet-gerrit-puppet-beaker-rspec-ubuntu-trusty/c684427/console.html#_2016-05-02_23_40_42_508 | 23:55 |
| clarkb | jeblair: ^ and bouncy castle causes that I think? | 23:55 |
| jeblair | clarkb: bouncecastle causes initial-init to fail, which causes reindex to fail (because it is not initted) | 23:55 |
| *** doug-fis_ has quit IRC | 23:55 | |
| *** sdake_ has quit IRC | 23:55 | |
| jeblair | ya | 23:55 |
| clarkb | https://downloads.bouncycastle.org/java/bcprov-jdk16-144.jar is a 404 | 23:55 |
| *** yamamoto has quit IRC | 23:55 | |
| jeblair | thatll do it | 23:56 |
| *** mtanino has quit IRC | 23:56 | |
| fungi | whoops | 23:56 |
| *** sdake has joined #openstack-infra | 23:56 | |
| * fungi starts to rant about people deleting releases of things, and then realizes it won't help | 23:56 | |
| *** sdake has quit IRC | 23:56 | |
| *** sdake has joined #openstack-infra | 23:57 | |
| jeblair | there's bouncycastle stuff in the puppet module -- does our gerrit think it needs something newer and so downloads it instead? | 23:58 |
| *** doug-fish has joined #openstack-infra | 23:58 | |
| *** piet has joined #openstack-infra | 23:58 | |
| jeblair | also, different jdk | 23:58 |
| *** sdake has quit IRC | 23:58 | |
| fungi | the release of gerrit we're running wants a newer bc than the debs on trusty, if memory serves | 23:59 |
| clarkb | jeblair: yes iirc precise/trusty have not been able to keep up with a new enough bouncy castle to do gerrit things | 23:59 |
| *** heyongli_ has quit IRC | 23:59 | |
| clarkb | https://www.bouncycastle.org/releasenotes.html if your release number and version number don't match... | 23:59 |
| jeblair | clarkb: right, but the thing we have in puppet is 1.51 on jdk1.5; gerrit is downloading 1.44 on jdk 1.6 | 23:59 |
| *** heyongli_ has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!