| *** claudiub has quit IRC | 00:00 | |
| *** yamamoto has joined #openstack-infra | 00:05 | |
| *** MarkAtwood has joined #openstack-infra | 00:08 | |
| *** yamamoto has quit IRC | 00:08 | |
| morganfainberg | god. that is awful ^^ | 00:11 |
|---|---|---|
| morganfainberg | but yeah it mostly can work | 00:12 |
| dstufft | iterating is sad though | 00:12 |
| *** armax has joined #openstack-infra | 00:14 | |
| dstufft | you either have to iterate over ints, or do some c style for i in range(len(bytes_obj)): char = bytes_obj[i:i+1] | 00:14 |
| *** markvoelker has joined #openstack-infra | 00:16 | |
| clarkb | clearly you need a def char_iter(bytes_obj): for i in range(len(bytes_obj): yield bytes_obj[i:i+1] iterator in stdlib | 00:18 |
| *** markvoelker has quit IRC | 00:20 | |
| *** armax has quit IRC | 00:32 | |
| *** rm_work is now known as rm_work|away | 00:35 | |
| *** camunoz_ has joined #openstack-infra | 00:37 | |
| *** camunoz has quit IRC | 00:39 | |
| *** hdd has joined #openstack-infra | 00:39 | |
| mordred | clarkb: we have the "split a repo" cantrip documented somewhere, right? | 00:47 |
| clarkb | in the puppet split out spec | 00:48 |
| clarkb | mordred ^ | 00:48 |
| mordred | kk. thanks | 00:48 |
| *** garyh has joined #openstack-infra | 00:48 | |
| *** yamamoto has joined #openstack-infra | 00:48 | |
| *** tmcpeak has joined #openstack-infra | 00:49 | |
| *** markvoelker has joined #openstack-infra | 00:50 | |
| *** amotoki has joined #openstack-infra | 00:53 | |
| *** tmcpeak has quit IRC | 00:53 | |
| *** markvoelker has quit IRC | 00:55 | |
| *** rkukura has quit IRC | 00:56 | |
| *** rkukura has joined #openstack-infra | 00:58 | |
| *** rkukura has quit IRC | 01:03 | |
| *** harlowja_at_home has joined #openstack-infra | 01:06 | |
| *** sdake_ has quit IRC | 01:07 | |
| *** boris-42 has joined #openstack-infra | 01:13 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/gear: Fix byte related python3 behaviors https://review.openstack.org/158157 | 01:14 |
| *** ghostpl_ has joined #openstack-infra | 01:19 | |
| *** achanda has quit IRC | 01:21 | |
| *** salv-orlando has quit IRC | 01:22 | |
| *** ghostpl_ has quit IRC | 01:23 | |
| *** sdake_ has joined #openstack-infra | 01:27 | |
| *** baoli has joined #openstack-infra | 01:28 | |
| *** wenlock has joined #openstack-infra | 01:30 | |
| *** sdake__ has joined #openstack-infra | 01:31 | |
| *** tmcpeak has joined #openstack-infra | 01:32 | |
| *** ociuhandu has quit IRC | 01:32 | |
| *** sdake_ has quit IRC | 01:33 | |
| *** dimsum__ has joined #openstack-infra | 01:38 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/gear: Fix byte related python3 behaviors https://review.openstack.org/158157 | 01:39 |
| *** dims_ has joined #openstack-infra | 01:39 | |
| *** dkranz has quit IRC | 01:39 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/gear: Fix byte related python3 behaviors https://review.openstack.org/158157 | 01:40 |
| SpamapS | woot, py34 passing | 01:40 |
| * SpamapS should probably squash that into one instead of 3 patches | 01:41 | |
| *** baoli has quit IRC | 01:41 | |
| *** armax has joined #openstack-infra | 01:41 | |
| *** dimsum__ has quit IRC | 01:43 | |
| *** amotoki has quit IRC | 01:44 | |
| *** hdd has quit IRC | 01:44 | |
| openstackgerrit | Merged openstack-dev/hacking: Updated from global requirements https://review.openstack.org/157758 | 01:44 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Fix bug in template job yaml expansion. https://review.openstack.org/150666 | 01:50 |
| *** markvoelker has joined #openstack-infra | 01:51 | |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Support the use of lists in trigger-build publisher. https://review.openstack.org/150598 | 01:56 |
| *** markvoelker has quit IRC | 01:57 | |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Support the use of lists in trigger-build builder. https://review.openstack.org/150597 | 01:59 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Support the use of lists in trigger-build builder. https://review.openstack.org/150597 | 01:59 |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Add Parameterized Remote Trigger Plugin support. https://review.openstack.org/150327 | 01:59 |
| *** markvoelker has joined #openstack-infra | 02:00 | |
| *** baoli has joined #openstack-infra | 02:02 | |
| *** MarkAtwood has quit IRC | 02:03 | |
| *** ddieterl_ has joined #openstack-infra | 02:06 | |
| *** MarkAtwood has joined #openstack-infra | 02:08 | |
| *** ddieterly has quit IRC | 02:09 | |
| *** signed8bit has joined #openstack-infra | 02:13 | |
| *** tmcpeak has quit IRC | 02:13 | |
| *** MarkAtwood has quit IRC | 02:15 | |
| *** baoli has quit IRC | 02:16 | |
| *** MarkAtwood has joined #openstack-infra | 02:21 | |
| *** stevemar has joined #openstack-infra | 02:21 | |
| *** achanda has joined #openstack-infra | 02:22 | |
| *** wenlock has quit IRC | 02:26 | |
| *** markvoelker has quit IRC | 02:26 | |
| *** markvoelker has joined #openstack-infra | 02:27 | |
| *** achanda has quit IRC | 02:27 | |
| *** achanda has joined #openstack-infra | 02:27 | |
| *** markvoelker has quit IRC | 02:31 | |
| *** yamahata has quit IRC | 02:32 | |
| *** yamahata has joined #openstack-infra | 02:32 | |
| *** vilobhmm has quit IRC | 02:36 | |
| *** MarkAtwood has quit IRC | 02:37 | |
| *** achanda has quit IRC | 02:38 | |
| openstackgerrit | Brian Curtin proposed openstack/requirements: Avoid httpretty 0.8.7 release https://review.openstack.org/158165 | 02:42 |
| *** koolhead17 has joined #openstack-infra | 02:43 | |
| openstackgerrit | Brian Curtin proposed openstack/requirements: Avoid httpretty 0.8.7 release https://review.openstack.org/158165 | 02:44 |
| *** asettle is now known as asettle-luncheon | 02:45 | |
| *** yamahata has quit IRC | 02:46 | |
| openstackgerrit | Merged openstack-infra/system-config: Add Elizabeth K. Joseph to servers and storyboard https://review.openstack.org/154672 | 02:57 |
| *** Ryan_Lane has joined #openstack-infra | 02:58 | |
| *** sdake_ has joined #openstack-infra | 02:58 | |
| clarkb | pleia2 ^ have fun :) | 02:58 |
| mordred | woot! | 02:59 |
| *** sdake__ has quit IRC | 03:02 | |
| *** Ryan_Lane has quit IRC | 03:02 | |
| *** mriedem has quit IRC | 03:06 | |
| anteaya | looks like mordred's woot! also confirms your assumption clarkb | 03:06 |
| anteaya | :) | 03:06 |
| *** ghostpl_ has joined #openstack-infra | 03:07 | |
| *** wenlock has joined #openstack-infra | 03:09 | |
| *** ghostpl_ has quit IRC | 03:12 | |
| *** otter768 has quit IRC | 03:14 | |
| *** MarkAtwood has joined #openstack-infra | 03:19 | |
| *** amotoki has joined #openstack-infra | 03:19 | |
| *** baoli has joined #openstack-infra | 03:27 | |
| *** oomichi_ has joined #openstack-infra | 03:30 | |
| *** baoli has quit IRC | 03:42 | |
| *** bhunter71 has quit IRC | 03:43 | |
| *** ddieterl_ has quit IRC | 03:47 | |
| *** harlowja_at_home has quit IRC | 03:53 | |
| jhesketh | oh clarkb, are you still around? | 03:53 |
| clarkb | not really | 03:54 |
| clarkb | whats up? | 03:54 |
| jhesketh | heh, all good, I'll catch you tomorrow | 03:54 |
| *** asettle-luncheon is now known as asettle | 03:55 | |
| clarkb | didnt get around to the swift thing. or zuul reviews :( | 03:55 |
| jhesketh | no trouble :-) | 03:56 |
| *** weshay has quit IRC | 04:02 | |
| *** _nadya_ has joined #openstack-infra | 04:10 | |
| *** wenlock has quit IRC | 04:10 | |
| *** vilobhmm has joined #openstack-infra | 04:12 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add ansible role for running puppet https://review.openstack.org/158173 | 04:13 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Install standalone ansible roles https://review.openstack.org/158175 | 04:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-ansible: Put the library file into the role https://review.openstack.org/158176 | 04:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-ansible: Remove embedded ansible role https://review.openstack.org/158177 | 04:14 |
| mordred | clarkb: ^^ those are precursors to the other stuff, but I wanted to be able to submit the next set of patches on top of the new repo | 04:16 |
| SpamapS | mordred: only 4? LA to NYC is a long flight.. ;) | 04:17 |
| mordred | SpamapS: :) | 04:18 |
| mordred | I have some more in waiting | 04:18 |
| *** deepakcs has joined #openstack-infra | 04:19 | |
| *** rwsu-afk has quit IRC | 04:19 | |
| *** armax_ has joined #openstack-infra | 04:19 | |
| *** signed8bit has quit IRC | 04:20 | |
| *** armax has quit IRC | 04:20 | |
| *** armax_ is now known as armax | 04:20 | |
| *** armax has quit IRC | 04:22 | |
| *** ivar-lazzaro has quit IRC | 04:22 | |
| *** vilobhmm1 has joined #openstack-infra | 04:28 | |
| *** BharatK has joined #openstack-infra | 04:28 | |
| *** vilobhmm has quit IRC | 04:30 | |
| *** MarkAtwood has quit IRC | 04:31 | |
| *** bswartz has quit IRC | 04:34 | |
| *** _nadya_ has quit IRC | 04:34 | |
| *** bswartz has joined #openstack-infra | 04:35 | |
| *** unicell has quit IRC | 04:46 | |
| *** unicell has joined #openstack-infra | 04:47 | |
| *** koolhead17 has quit IRC | 04:52 | |
| openstackgerrit | David Pursehouse proposed openstack-infra/jenkins-job-builder: Add support to customize trigger-builds block thresholds https://review.openstack.org/154379 | 04:54 |
| *** ghostpl_ has joined #openstack-infra | 04:56 | |
| *** deepakcs has quit IRC | 05:00 | |
| *** ghostpl_ has quit IRC | 05:01 | |
| notmyname | clarkb: jhesketh: I just got in from a long weekend away. I saw "swift" mentioned a few times in here. anything I should be paying attention to? | 05:01 |
| jhesketh | notmyname: nothing worrying ;-). We've just been working on making the uploads a little faster | 05:03 |
| jhesketh | https://review.openstack.org/#/c/156788/ if you're interested | 05:04 |
| notmyname | cool. ok, thanks | 05:04 |
| notmyname | oh, nice (from the commit message) | 05:04 |
| *** garyh has quit IRC | 05:09 | |
| *** vigneshvar has joined #openstack-infra | 05:13 | |
| *** otter768 has joined #openstack-infra | 05:15 | |
| openstackgerrit | Russell Bryant proposed openstack-infra/project-config: Add stackforge/networking-ovn project https://review.openstack.org/158137 | 05:15 |
| *** baoli has joined #openstack-infra | 05:16 | |
| *** sdake_ has quit IRC | 05:18 | |
| *** otter768 has quit IRC | 05:20 | |
| *** HeOS has joined #openstack-infra | 05:23 | |
| *** baoli has quit IRC | 05:28 | |
| *** hdd has joined #openstack-infra | 05:29 | |
| *** baoli has joined #openstack-infra | 05:32 | |
| *** achanda has joined #openstack-infra | 05:35 | |
| *** yfried has quit IRC | 05:44 | |
| *** pcaruana has quit IRC | 05:45 | |
| *** baoli has quit IRC | 05:46 | |
| *** ddieterly has joined #openstack-infra | 05:49 | |
| *** ddieterly has quit IRC | 05:53 | |
| *** Ryan_Lane has joined #openstack-infra | 05:59 | |
| *** camunoz_ is now known as camunoz_gone | 05:59 | |
| *** Ryan_Lane has quit IRC | 06:03 | |
| *** baoli has joined #openstack-infra | 06:04 | |
| *** garyh has joined #openstack-infra | 06:10 | |
| *** baoli has quit IRC | 06:18 | |
| *** achanda has quit IRC | 06:25 | |
| openstackgerrit | Adrian Moisey proposed openstack-infra/jenkins-job-builder: Add support for github commit status pending https://review.openstack.org/152818 | 06:25 |
| *** cnesa has joined #openstack-infra | 06:25 | |
| *** coolsvap_ is now known as coolsvap | 06:27 | |
| *** ildikov has quit IRC | 06:38 | |
| *** MarkAtwood has joined #openstack-infra | 06:39 | |
| *** woodster_ has quit IRC | 06:40 | |
| *** tnovacik has joined #openstack-infra | 06:40 | |
| *** cnesa has quit IRC | 06:43 | |
| *** FallenPegasus has joined #openstack-infra | 06:44 | |
| *** MarkAtwood has quit IRC | 06:45 | |
| *** FallenPegasus is now known as MarkAtwood | 06:45 | |
| *** ghostpl_ has joined #openstack-infra | 06:45 | |
| *** deepakcs has joined #openstack-infra | 06:45 | |
| *** ddieterly has joined #openstack-infra | 06:50 | |
| *** ghostpl_ has quit IRC | 06:50 | |
| *** hdd has quit IRC | 06:54 | |
| *** ddieterly has quit IRC | 06:54 | |
| *** achanda has joined #openstack-infra | 06:56 | |
| *** jcoufal has joined #openstack-infra | 06:57 | |
| *** cnesa has joined #openstack-infra | 07:00 | |
| *** vilobhmm1 has quit IRC | 07:01 | |
| *** mrunge has joined #openstack-infra | 07:04 | |
| *** unicell has quit IRC | 07:06 | |
| *** afazekas_ has joined #openstack-infra | 07:06 | |
| *** scheuran has joined #openstack-infra | 07:08 | |
| *** belmoreira has joined #openstack-infra | 07:10 | |
| *** unicell has joined #openstack-infra | 07:10 | |
| *** Murad has joined #openstack-infra | 07:13 | |
| *** yfried has joined #openstack-infra | 07:13 | |
| Murad | It seems that devstack now requires python-novaclient 2.21 to work | 07:13 |
| Murad | apt-get installs 2.17, how can I get 2.21 | 07:13 |
| Murad | ? | 07:13 |
| *** vilobhmm has joined #openstack-infra | 07:14 | |
| *** otter768 has joined #openstack-infra | 07:16 | |
| *** yfried_ has joined #openstack-infra | 07:17 | |
| *** yfried has quit IRC | 07:17 | |
| *** stevemar has quit IRC | 07:19 | |
| *** mrmartin has joined #openstack-infra | 07:20 | |
| *** otter768 has quit IRC | 07:21 | |
| *** ildikov has joined #openstack-infra | 07:32 | |
| *** tnovacik has quit IRC | 07:40 | |
| *** vilobhmm1 has joined #openstack-infra | 07:42 | |
| *** achanda has quit IRC | 07:42 | |
| *** vilobhmm has quit IRC | 07:44 | |
| *** yfried_ is now known as yfried|afk | 07:44 | |
| *** hrubi has joined #openstack-infra | 07:47 | |
| *** achanda has joined #openstack-infra | 07:48 | |
| *** sergsh has joined #openstack-infra | 07:48 | |
| *** jlibosva has joined #openstack-infra | 07:48 | |
| *** e0ne has joined #openstack-infra | 07:49 | |
| *** e0ne is now known as e0ne_ | 07:49 | |
| *** ddieterly has joined #openstack-infra | 07:50 | |
| *** baoli has joined #openstack-infra | 07:52 | |
| *** doude has joined #openstack-infra | 07:52 | |
| *** luqas has joined #openstack-infra | 07:52 | |
| *** e0ne_ is now known as e0ne | 07:53 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/storyboard: Add tools to run integration tests https://review.openstack.org/155975 | 07:54 |
| *** ddieterly has quit IRC | 07:55 | |
| *** yfried|afk is now known as yfried_ | 07:56 | |
| *** MarkAtwood has quit IRC | 07:58 | |
| *** MarkAtwood has joined #openstack-infra | 08:00 | |
| *** baoli has quit IRC | 08:06 | |
| *** yfried_ is now known as yfried|afk | 08:13 | |
| openstackgerrit | Radomir Dopieralski proposed openstack/requirements: Add ironic-discoverd https://review.openstack.org/158210 | 08:14 |
| *** ttx2 has joined #openstack-infra | 08:15 | |
| *** ttx2 is now known as ttx | 08:15 | |
| *** vilobhmm1 has quit IRC | 08:16 | |
| *** yfried|afk is now known as yfried_ | 08:16 | |
| *** ghostpl_ has joined #openstack-infra | 08:16 | |
| *** ttx has quit IRC | 08:17 | |
| *** ttx has joined #openstack-infra | 08:17 | |
| *** ttx has quit IRC | 08:17 | |
| *** ttx has joined #openstack-infra | 08:18 | |
| *** ttx has quit IRC | 08:19 | |
| *** ttx has joined #openstack-infra | 08:19 | |
| *** ghostpl_ has quit IRC | 08:20 | |
| *** Ala has joined #openstack-infra | 08:21 | |
| *** achanda has quit IRC | 08:21 | |
| *** achanda has joined #openstack-infra | 08:22 | |
| *** jcoufal has quit IRC | 08:22 | |
| *** jcoufal has joined #openstack-infra | 08:23 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Add caching interval to mutable caches https://review.openstack.org/158138 | 08:27 |
| *** luqas has quit IRC | 08:31 | |
| *** achanda has quit IRC | 08:32 | |
| *** dtantsur|afk is now known as dtantsur | 08:35 | |
| *** achanda has joined #openstack-infra | 08:45 | |
| *** MaxV has joined #openstack-infra | 08:46 | |
| *** mpavone has joined #openstack-infra | 08:46 | |
| *** ildikov has quit IRC | 08:50 | |
| *** ddieterly has joined #openstack-infra | 08:51 | |
| *** jistr has joined #openstack-infra | 08:55 | |
| *** ddieterly has quit IRC | 08:56 | |
| *** e0ne has quit IRC | 08:57 | |
| *** viktors|afk is now known as viktors | 08:58 | |
| *** tnovacik has joined #openstack-infra | 09:03 | |
| *** hashar has joined #openstack-infra | 09:04 | |
| *** hashar has quit IRC | 09:05 | |
| *** sergsh has quit IRC | 09:06 | |
| *** ildikov has joined #openstack-infra | 09:06 | |
| *** mpaolino has joined #openstack-infra | 09:08 | |
| *** devvesa has joined #openstack-infra | 09:12 | |
| *** devvesa has quit IRC | 09:14 | |
| *** devvesa has joined #openstack-infra | 09:14 | |
| *** achanda has quit IRC | 09:16 | |
| *** otter768 has joined #openstack-infra | 09:17 | |
| *** bmwiedemann has left #openstack-infra | 09:19 | |
| *** sergsh has joined #openstack-infra | 09:20 | |
| *** otter768 has quit IRC | 09:22 | |
| *** yamamoto has quit IRC | 09:23 | |
| *** hichihara has quit IRC | 09:24 | |
| *** sergsh has quit IRC | 09:27 | |
| *** ghostpl_ has joined #openstack-infra | 09:29 | |
| *** salv-orlando has joined #openstack-infra | 09:33 | |
| *** sergsh has joined #openstack-infra | 09:33 | |
| *** andreykurilin_ has joined #openstack-infra | 09:34 | |
| *** oomichi_ has quit IRC | 09:34 | |
| *** mpavone has quit IRC | 09:36 | |
| *** koolhead17 has joined #openstack-infra | 09:36 | |
| *** koolhead17 has quit IRC | 09:37 | |
| *** koolhead17 has joined #openstack-infra | 09:37 | |
| *** koolhead17 has joined #openstack-infra | 09:37 | |
| *** mpavone has joined #openstack-infra | 09:40 | |
| *** jp_at_hp has joined #openstack-infra | 09:43 | |
| *** bogdando has joined #openstack-infra | 09:45 | |
| *** pblaho has joined #openstack-infra | 09:47 | |
| *** lucasagomes has joined #openstack-infra | 09:48 | |
| lucasagomes | sdague, https://review.openstack.org/#/c/158224/ created the patch using DEVSTACK_LOCAL_CONFIG, apparently it's very new and I couldn't find any example of use of this var in devstack. So lemme know if the usage is correct on my patch. Thanks | 09:49 |
| *** yamamoto has joined #openstack-infra | 09:51 | |
| *** ddieterly has joined #openstack-infra | 09:52 | |
| *** luqas has joined #openstack-infra | 09:53 | |
| *** ddieterly has quit IRC | 09:56 | |
| *** ssam2 has joined #openstack-infra | 09:58 | |
| *** luqas has quit IRC | 09:58 | |
| *** dizquierdo has joined #openstack-infra | 09:58 | |
| *** e0ne has joined #openstack-infra | 09:59 | |
| *** ParsectiX has joined #openstack-infra | 10:02 | |
| *** claudiub has joined #openstack-infra | 10:03 | |
| *** salv-orlando has quit IRC | 10:04 | |
| *** ihrachyshka has joined #openstack-infra | 10:07 | |
| *** sergsh has quit IRC | 10:14 | |
| *** salv-orlando has joined #openstack-infra | 10:14 | |
| *** marcusvrn has joined #openstack-infra | 10:18 | |
| *** e0ne is now known as e0ne_ | 10:21 | |
| openstackgerrit | John Vandenberg proposed openstack-infra/git-review: Update project links https://review.openstack.org/158235 | 10:22 |
| *** marcusvrn1 has joined #openstack-infra | 10:23 | |
| *** andreykurilin_ has quit IRC | 10:23 | |
| *** marcusvrn has quit IRC | 10:25 | |
| *** marcusvrn1 has quit IRC | 10:25 | |
| *** sergsh has joined #openstack-infra | 10:26 | |
| *** e0ne_ is now known as e0ne | 10:28 | |
| *** marcusvrn has joined #openstack-infra | 10:29 | |
| *** koolhead17 has quit IRC | 10:29 | |
| *** yfried_ is now known as yfried|afk | 10:41 | |
| *** vigneshvar has quit IRC | 10:42 | |
| *** rlandy has joined #openstack-infra | 10:45 | |
| *** ddieterly has joined #openstack-infra | 10:53 | |
| *** armax has joined #openstack-infra | 10:53 | |
| *** ddieterly has quit IRC | 10:57 | |
| *** yamamoto has quit IRC | 10:58 | |
| *** teran has joined #openstack-infra | 11:05 | |
| *** jaypipes has joined #openstack-infra | 11:05 | |
| *** yfried|afk is now known as yfried_ | 11:06 | |
| *** pc_m has joined #openstack-infra | 11:07 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack/requirements: Don't use git diff --quiet to detect local change https://review.openstack.org/158247 | 11:08 |
| ihrachyshka | sdague, ^^ we'll see whether it fixes the issues that adam_g and mriedem saw yesterday. | 11:11 |
| *** armax has quit IRC | 11:12 | |
| yolanda | hi, jhesketh, thx for all reviews | 11:12 |
| sdague | lucasagomes: it's not part of devstack, it's part of the tooling around devstack | 11:14 |
| sdague | look at openstack-infra/project-config and openstack-infra/devstack-gate | 11:14 |
| lucasagomes | sdague, ohh damn! gotcha, lemme take a look | 11:14 |
| lucasagomes | sdague, sorry, I'm not familiar with those bits at all | 11:15 |
| *** yfried_ is now known as yfried|afk | 11:16 | |
| *** yfried|afk is now known as yfried_ | 11:17 | |
| *** otter768 has joined #openstack-infra | 11:18 | |
| *** salv-orlando has quit IRC | 11:19 | |
| *** andreaf_ has joined #openstack-infra | 11:20 | |
| sdague | lucasagomes: those are the parts that actually do all the setup and configure for devstack for all the tests | 11:20 |
| lucasagomes | sdague, yeah found some example in the project-config. Putting a patch up in a second | 11:21 |
| lucasagomes | thanks for pointing it out | 11:21 |
| *** andreaf_ has quit IRC | 11:22 | |
| *** otter768 has quit IRC | 11:22 | |
| *** bogdando has left #openstack-infra | 11:24 | |
| *** MarkAtwood has quit IRC | 11:24 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config: Ironic: Bump memory size for non-concurrency job https://review.openstack.org/158251 | 11:24 |
| lucasagomes | sdague, ^ will wait for the tests see if works. Again, thanks for pointing that out and sorry for the previous misunderstanding | 11:25 |
| *** salv-orlando has joined #openstack-infra | 11:25 | |
| *** SumitNaiksatam has quit IRC | 11:26 | |
| *** SumitNaiksatam has joined #openstack-infra | 11:26 | |
| *** cnesa has quit IRC | 11:27 | |
| *** ldnunes has joined #openstack-infra | 11:27 | |
| *** salv-orlando has quit IRC | 11:29 | |
| *** salv-orl_ has joined #openstack-infra | 11:29 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config: Ironic: Bump memory size for non-concurrency job https://review.openstack.org/158251 | 11:39 |
| *** tnovacik has quit IRC | 11:40 | |
| *** baoli has joined #openstack-infra | 11:41 | |
| *** dhritishikhar_ has joined #openstack-infra | 11:44 | |
| openstackgerrit | Sebastian Kalinowski proposed openstack-infra/project-config: Add fuel-tasks-validator project to stackforge https://review.openstack.org/158263 | 11:49 |
| *** mpaolino has quit IRC | 11:50 | |
| *** ddieterly has joined #openstack-infra | 11:53 | |
| *** prmtl has joined #openstack-infra | 11:55 | |
| *** thelexx is now known as lxsli | 11:56 | |
| *** ddieterly has quit IRC | 11:58 | |
| *** mugsie has joined #openstack-infra | 12:03 | |
| *** spredzy is now known as spredzy|afk | 12:07 | |
| *** dkliban_afk is now known as dkliban | 12:09 | |
| *** rfolco has joined #openstack-infra | 12:09 | |
| *** jcoufal has quit IRC | 12:10 | |
| *** jcoufal_ has joined #openstack-infra | 12:10 | |
| *** dmsimard_away is now known as dmsimard | 12:10 | |
| *** yamamoto_ has joined #openstack-infra | 12:14 | |
| *** Kiall has quit IRC | 12:17 | |
| *** Kiall has joined #openstack-infra | 12:19 | |
| *** vhoward has quit IRC | 12:19 | |
| *** lucasagomes is now known as lucas-hungry | 12:24 | |
| *** tnovacik has joined #openstack-infra | 12:25 | |
| *** adalbas has joined #openstack-infra | 12:29 | |
| *** aysyd has joined #openstack-infra | 12:30 | |
| *** AJaeger has joined #openstack-infra | 12:33 | |
| *** AJaeger has joined #openstack-infra | 12:33 | |
| *** mugsie has quit IRC | 12:33 | |
| *** mugsie has joined #openstack-infra | 12:33 | |
| *** mugsie is now known as Guest94423 | 12:34 | |
| prmtl | hi, I kindly ask for review: https://review.openstack.org/#/c/158263/ | 12:34 |
| *** Guest94423 has quit IRC | 12:35 | |
| *** Guest94423 has joined #openstack-infra | 12:35 | |
| *** Guest94423 is now known as muguie | 12:36 | |
| *** muguie is now known as mugsie_ | 12:36 | |
| *** spredzy|afk is now known as spredzy | 12:37 | |
| *** salv-orl_ has quit IRC | 12:38 | |
| *** salv-orlando has joined #openstack-infra | 12:39 | |
| *** weshay has joined #openstack-infra | 12:40 | |
| *** cdent has joined #openstack-infra | 12:41 | |
| *** salv-orlando has quit IRC | 12:42 | |
| *** salv-orlando has joined #openstack-infra | 12:43 | |
| *** dims_ has quit IRC | 12:44 | |
| *** derekh has joined #openstack-infra | 12:45 | |
| *** sdake_ has joined #openstack-infra | 12:48 | |
| Kiall | Hey folks - The latest release of eventlet is causing some issues for Designate due to a broken monkey patch of dns.reversename.from_address() .. What's the deal with capping the requirement? We've never needed to before... | 12:49 |
| *** koolhead17 has joined #openstack-infra | 12:50 | |
| *** dimsum__ has joined #openstack-infra | 12:51 | |
| *** zigo has quit IRC | 12:52 | |
| *** ZZelle has joined #openstack-infra | 12:52 | |
| *** marcusvrn1 has joined #openstack-infra | 12:53 | |
| *** ddieterly has joined #openstack-infra | 12:54 | |
| *** zigo has joined #openstack-infra | 12:56 | |
| *** marcusvrn has quit IRC | 12:57 | |
| *** ddieterly has quit IRC | 12:58 | |
| *** pelix1 has joined #openstack-infra | 12:59 | |
| *** sdake_ has quit IRC | 13:00 | |
| *** dprince has joined #openstack-infra | 13:00 | |
| *** deepakcs has quit IRC | 13:00 | |
| *** BharatK has quit IRC | 13:01 | |
| *** nicodemos has joined #openstack-infra | 13:02 | |
| *** salv-orlando has quit IRC | 13:04 | |
| openstackgerrit | Kiall Mac Innes proposed openstack/requirements: Block eventlet 0.17.0 https://review.openstack.org/158287 | 13:05 |
| *** sdake_ has joined #openstack-infra | 13:08 | |
| *** dkranz has joined #openstack-infra | 13:09 | |
| *** kgiusti has joined #openstack-infra | 13:09 | |
| *** ParsectiX has quit IRC | 13:10 | |
| *** mwagner_lap has joined #openstack-infra | 13:13 | |
| *** ParsectiX has joined #openstack-infra | 13:16 | |
| *** otter768 has joined #openstack-infra | 13:19 | |
| *** bswartz has quit IRC | 13:20 | |
| *** dimsum__ has quit IRC | 13:22 | |
| *** dimsum__ has joined #openstack-infra | 13:22 | |
| *** mugsie_ is now known as mugsie | 13:23 | |
| *** salv-orlando has joined #openstack-infra | 13:23 | |
| *** lucas-hungry is now known as lucasagomes | 13:23 | |
| *** otter768 has quit IRC | 13:24 | |
| openstackgerrit | Merged openstack/requirements: Don't use git diff --quiet to detect local change https://review.openstack.org/158247 | 13:27 |
| *** marcusvrn has joined #openstack-infra | 13:28 | |
| *** salv-orlando has quit IRC | 13:28 | |
| *** mbacchi has joined #openstack-infra | 13:29 | |
| *** gordc has joined #openstack-infra | 13:30 | |
| *** marcusvrn1 has quit IRC | 13:31 | |
| *** salv-orlando has joined #openstack-infra | 13:33 | |
| *** rfolco has quit IRC | 13:36 | |
| *** rfolco has joined #openstack-infra | 13:36 | |
| *** dimsum__ is now known as dims | 13:39 | |
| *** salv-orlando has quit IRC | 13:41 | |
| *** bknudson has quit IRC | 13:41 | |
| fungi | Kiall: i have a vague memory of eventlet and dnspython monkey patching each other out of existence... maybe you need something like keystone did in https://review.openstack.org/29710 ? | 13:44 |
| Kiall | Nah, we really do need the eventlet patching of dnspython .. They've just managed to break a function which doesn't even do I/O with 0.17.. | 13:45 |
| fungi | crazy | 13:45 |
| Kiall | Yep :) | 13:45 |
| *** salv-orlando has joined #openstack-infra | 13:46 | |
| fungi | ahh, okay, followed the maze of links to the upstream bug report now | 13:46 |
| Kiall | LOL - The commit that broke us, is the commit that fixes the issue Keystone mentioned in their commit message ;) | 13:47 |
| *** woodster_ has joined #openstack-infra | 13:47 | |
| *** Guest30952 has joined #openstack-infra | 13:48 | |
| *** lucasagomes has quit IRC | 13:50 | |
| *** bhunter71 has joined #openstack-infra | 13:50 | |
| *** bswartz has joined #openstack-infra | 13:50 | |
| *** tnovacik has quit IRC | 13:53 | |
| *** salv-orlando has quit IRC | 13:53 | |
| *** ddieterly has joined #openstack-infra | 13:55 | |
| fungi | such irony | 13:58 |
| *** ayoung has joined #openstack-infra | 14:00 | |
| Kiall | Yep, I might have a workaround that doesn't require the cap... But there might be more breakage.. What's the deal with accepting a cap like that for a non-core project? :) | 14:00 |
| *** ddieterly has quit IRC | 14:00 | |
| *** bknudson has joined #openstack-infra | 14:05 | |
| *** dtantsur is now known as dtantsur|brb | 14:06 | |
| *** dkranz has quit IRC | 14:06 | |
| fungi | it's up to the requirements-core reviewers to decide, generally | 14:09 |
| *** mfink_ has joined #openstack-infra | 14:10 | |
| *** dhritishikhar_ has quit IRC | 14:10 | |
| afazekas_ | sdague, https://review.openstack.org/#/c/158296/ | 14:10 |
| *** sdake_ has quit IRC | 14:10 | |
| fungi | the infra team usually stays out of the discussion unless it's a review around the tooling for requirements validation/sync or something causing widespread breakage | 14:11 |
| Kiall | fungi: makes sense :) | 14:11 |
| *** mjturek1 has joined #openstack-infra | 14:12 | |
| *** Guest28285 is now known as mgagne | 14:13 | |
| *** mgagne has joined #openstack-infra | 14:13 | |
| sdague | afazekas: that's a pretty basic service start problem | 14:17 |
| mordred | Kiall: is there any version which works for both of you? | 14:18 |
| Kiall | mordred: yep, anything other than the version eventlet released today ;) | 14:18 |
| mordred | that's so awesome | 14:18 |
| *** dustins has joined #openstack-infra | 14:19 | |
| Kiall | I submitted this o/r cap to block 0.17, https://review.openstack.org/158287 ;) | 14:19 |
| *** dkranz has joined #openstack-infra | 14:19 | |
| sdague | afazekas: yeh, that service hasn't started on any job for some long amount of time | 14:19 |
| *** sdake_ has joined #openstack-infra | 14:19 | |
| sdague | http://logs.openstack.org/55/158255/2/check/check-tempest-dsvm-full/53bc17d/logs/screen-ceilometer-acentral.txt.gz | 14:19 |
| mordred | I mean, I'd be inclined to block 0.17 just on the basis of we know it's provably broken in some way - but i'm still coffeeing | 14:19 |
| mordred | Kiall: that said, I'd probably express that as !=0.17.0 so that we don't have to do another round when they release a fix | 14:20 |
| afazekas_ | sdague: it fails because the dependency is not installed, it is specified as test time only dependency. | 14:20 |
| Kiall | mordred: Yea, I've got a workaround that unblocks us while jeeping 0.17.. I just have no idea if anything else is broken and we just haven't noticed.. I'm having trouble tracing the root casue even though I have the eventlet commit that broke things .. | 14:20 |
| Kiall | keeping* | 14:20 |
| mordred | nod | 14:20 |
| *** tiswanso has joined #openstack-infra | 14:20 | |
| *** sdake__ has joined #openstack-infra | 14:20 | |
| sdague | afazekas_: right, which means ceilometer is broken, because if it's an optional driver why is it always installing? | 14:20 |
| afazekas_ | sdague: I am not sure is it optional or not, but AFAIK we have tempest+swift+ceilometer test and that module is for swift support | 14:22 |
| sdague | afazekas: it's not for swift support, it's for ceph support | 14:22 |
| *** sdake_ has quit IRC | 14:24 | |
| Kiall | looks like eventlet will do a bugfix release real-soon :) I love responsive upstreams! | 14:25 |
| afazekas_ | sdague: rgw :) ok. Looks like there is no option for turning it off. | 14:28 |
| sdague | afazekas: yeh, that's a bug | 14:28 |
| sdague | in ceilometer | 14:28 |
| sdague | it should be listed as a critical bug | 14:29 |
| *** e0ne is now known as e0ne_ | 14:30 | |
| *** e0ne_ is now known as e0ne | 14:31 | |
| afazekas_ | sdague: If someone would like to use the rgw module, but the pip forgets to install the dependency it is also a bug, especially if the import is unconditional. | 14:32 |
| *** stevemar has joined #openstack-infra | 14:33 | |
| sdague | it's a driver, we've always put the burden on the user to do that install | 14:33 |
| sdague | afazekas: is there a bug? | 14:34 |
| afazekas_ | sdague: I am do not know do have any generic guideline/strategy for how to handle optional stuff at openstack and what is the ceilometer policy. Usually there is 3 option. A: the python package depends on all possible runtime dependency. B, conditional import. C, splitting the package to multiple packages. For strategy A the requirement change is the right step. | 14:36 |
| gordc | just an fyi: we're looking at ceilo issue now. | 14:36 |
| sdague | https://bugs.launchpad.net/ceilometer/+bug/1424657 | 14:36 |
| openstack | Launchpad bug 1424657 in Ceilometer "ceilometer unconditionally requires rados gateway support" [Critical,New] | 14:36 |
| sdague | gordc: yeh, that needs to be fixed ASAP, currently blocking devstack | 14:36 |
| *** mrmartin has quit IRC | 14:36 | |
| sdague | afazekas: A is not what we do | 14:37 |
| *** ddieterly has joined #openstack-infra | 14:37 | |
| sdague | if it's a driver, it should be something in a config file that's specified | 14:37 |
| *** amotoki has quit IRC | 14:37 | |
| *** claudiub has quit IRC | 14:38 | |
| *** jcoufal_ has quit IRC | 14:38 | |
| *** jcoufal has joined #openstack-infra | 14:38 | |
| afazekas_ | sdague: At the moment the issue breaks devstack changes, the B or C requires more time. | 14:38 |
| gordc | sdague: yeah, we're looking more to to B. | 14:39 |
| Kiall | sdague: digging into eventlet a few mins ago, and I spotted their tox.ini does some interesting stuff for drivers which I've not seen before: https://github.com/eventlet/eventlet/blob/master/tox.ini#L18 | 14:39 |
| Kiall | (semi related, but no use to devstack ;)) | 14:40 |
| *** mriedem has joined #openstack-infra | 14:40 | |
| sdague | gordc: yeh, it seems like this is a driver, so should be specified in a config file and only loaded if specified | 14:41 |
| gordc | sdague: agreed (main reason why we forced it into test-reqs) | 14:41 |
| *** Guest30952 is now known as annegentle | 14:42 | |
| sdague | gordc: I am kind of curious though, 0 tests for ceilometer see if services can start with just requirements? | 14:43 |
| *** dhritishikhar_ has joined #openstack-infra | 14:43 | |
| sdague | because, the fact that this service is straight dead and nothing noticed, seems .... odd | 14:43 |
| *** e0ne is now known as e0ne_ | 14:44 | |
| gordc | sdague: we don't have any tempest test relating to pollster because i don't think we every fleshed out how to capture data without modifying polling interval (iirc) | 14:45 |
| gordc | ever* | 14:46 |
| *** lucasagomes has joined #openstack-infra | 14:46 | |
| sdague | gordc: you should make sure this is something that gets tested for in your functional tests | 14:46 |
| *** sabeen has joined #openstack-infra | 14:46 | |
| gordc | sdague: sure. seems like a reasonable suggestion. | 14:47 |
| *** mrmartin has joined #openstack-infra | 14:47 | |
| lucasagomes | hi, if someone have a time please take a look at https://review.openstack.org/#/c/158251/, in Ironic we this memory bump so that we can start tests for the new ramdisk in gate. thank you | 14:48 |
| *** beekneemech is now known as bnemec | 14:48 | |
| odyssey4me | looking for a +2 +w in this small review which has been sitting for a few days: https://review.openstack.org/156229 | 14:51 |
| HenryG | Hi folks! How do we get the "Rebase Change" button enabled for a stackforge repo? | 14:51 |
| odyssey4me | HenryG the stackforge repo I do a lot of work in (os-ansible-deployment) has it already enabled | 14:52 |
| odyssey4me | perhaps you could use it for comparison | 14:52 |
| HenryG | odyssey4me: remind me where the repo config is? | 14:52 |
| odyssey4me | HenryG I'm not sure if it's in project-config or somewhere else... I've only been involved in patches for tests - we had the rebase button right from the initiation of the project! | 14:53 |
| *** superdan is now known as dansmith | 14:53 | |
| fungi | HenryG: odyssey4me: it should already be enabled for all projects globally | 14:54 |
| *** e0ne_ has quit IRC | 14:54 | |
| fungi | HenryG: odyssey4me: it will only be displayed if gerrit thinks it can trivially rebase the change (and the jgit gerrit uses is not nearly so robust at figuring out how to handle merge conflicts as cgit, so just because a rebase seems trivial doesn't mean gerrit is going to be able to figure out how to make it happen) | 14:55 |
| *** e0ne has joined #openstack-infra | 14:56 | |
| HenryG | fungi: thanks, yes it is enabled, I was looking at a review that gerrit did not think was trivial :) | 14:56 |
| *** signed8bit has joined #openstack-infra | 14:56 | |
| *** dtantsur|brb is now known as dtantsur | 14:57 | |
| *** deepakcs has joined #openstack-infra | 14:57 | |
| *** patrickeast has joined #openstack-infra | 14:58 | |
| *** wenlock has joined #openstack-infra | 14:59 | |
| *** tmcpeak has joined #openstack-infra | 14:59 | |
| *** wenlock has quit IRC | 14:59 | |
| *** wenlock has joined #openstack-infra | 15:00 | |
| *** tnovacik has joined #openstack-infra | 15:02 | |
| deepakcs | fungi, Hi GM. Can u help with one more manul run, we seem to have some insight into the oom issue | 15:03 |
| fungi | deepakcs: sure, i'll set one going in a few minutes | 15:03 |
| deepakcs | fungi, thanks, i am hoping this would be the last i would be troubling u :) | 15:04 |
| clarkb | deepakcs have you tried reproducing this on a node with 8GB of ram? | 15:05 |
| deepakcs | clarkb, we tried with 6GB and it doesn't re-crate on my local setup, even copied the same tempest conf and local rc | 15:05 |
| afazekas_ | gordc: so what to now quickly ? (A, dep change, B, revert the ceilo change, C simple conditional import, D make the devstack job non voting , E wait until some makes it properly configurable ) | 15:05 |
| *** erikmwil_ has joined #openstack-infra | 15:06 | |
| *** erikmwilson has quit IRC | 15:06 | |
| *** erikmwil_ is now known as erikmwilson | 15:06 | |
| fungi | yeah, confirmed last week that their local tests were with 8gb ram | 15:06 |
| *** erikmwil- has joined #openstack-infra | 15:06 | |
| deepakcs | clarkb, but doing some analsysi it seem before the oom, it was the cinder backup tests that were running, and those are the most IO intensive | 15:06 |
| *** BharatK has joined #openstack-infra | 15:06 | |
| deepakcs | clarkb, so disabling c-bak is what we are trying now | 15:06 |
| gordc | afazekas_: easy patch is revert | 15:06 |
| *** prad has joined #openstack-infra | 15:06 | |
| *** patrickeast_ has joined #openstack-infra | 15:06 | |
| *** dimtruck is now known as zz_dimtruck | 15:06 | |
| gordc | afazekas_: or we can wait 15minutes to try this other option which hopefully works? | 15:06 |
| *** patrickeast has quit IRC | 15:07 | |
| *** patrickeast_ is now known as patrickeast | 15:07 | |
| *** sdake__ has quit IRC | 15:08 | |
| afazekas_ | gordc, The 'check-devstack-dsvm-updown' is not voting on ceilo | 15:08 |
| gordc | yes, i was just made aware. | 15:09 |
| *** lascii is now known as alaski | 15:09 | |
| gordc | afazekas_: reverting change as quickfix. | 15:11 |
| *** prmtl has quit IRC | 15:12 | |
| clarkb | deepakcs and you are testibg with 64bit centos7 install? seems like something that OOMs this hard should be reproduceable | 15:12 |
| clarkb | but maybe its related to the number of concurebt tests and test order | 15:13 |
| deepakcs | clarkb, yes, centos7 and i am surprised too as to why it doesn't reocreate | 15:13 |
| *** dangers_away is now known as dangers | 15:13 | |
| deepakcs | clarkb, i tried to use the same pretty_tox cmdlien that the failing logs have (which are set to concurrency=4) | 15:13 |
| deepakcs | clarkb, but yes, i had just 1 cpu, but looks like the infra VM has 8, which i realised just today, not sure if that adds to the mix ? | 15:13 |
| *** nelsnelson has joined #openstack-infra | 15:13 | |
| mriedem | jogo: sdague: mtreinish: clarkb: do you know if we have /proc/cpuinfo dumped anywhere in the logs for the host on a test run? looking at jogo's live migration multi-host change http://logs.openstack.org/30/141530/7/experimental/check-tempest-dsvm-aiopcpu-full/13c9f37/logs/screen-n-cpu.txt.gz?level=TRACE | 15:14 |
| deepakcs | clarkb, fungi any other ideas to re-create locally are welcome :) | 15:14 |
| clarkb | mriedem setup host log file | 15:14 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config: Ironic: Bump memory size for non-concurrency job https://review.openstack.org/158251 | 15:14 |
| mtreinish | mriedem: http://logs.openstack.org/30/141530/7/experimental/check-tempest-dsvm-aiopcpu-full/13c9f37/logs/devstack-gate-setup-host.txt.gz | 15:15 |
| mriedem | yeah | 15:15 |
| mriedem | thanks | 15:15 |
| clarkb | deepakcs yes I think the 8 cores is factoring in, it determines the number of service daemons to start among other things | 15:15 |
| mriedem | looking | 15:15 |
| deepakcs | clarkb, q on test order, i see the PYTHONHASHSEED is dumped in the logs, how do i set the same seed for my local run ? | 15:15 |
| clarkb | deepakcs just set that env var to that value | 15:16 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 15:16 | |
| deepakcs | clarkb, ok, can try with 4 max :( my laptop has 4 phy cpus only. I am working to see if i can get a server class machine for this | 15:16 |
| fungi | (prior to calling tox, so potentially in the tox.ini) | 15:16 |
| fungi | er, prior to calling python i mean | 15:17 |
| deepakcs | clarkb, ah ok, then i should just say pretty_tox.sh.... shouldn't append bash otherwise the env var won't be carried foarward, rigfht ? | 15:17 |
| afazekas_ | gordc, https://review.openstack.org/#/c/158331/ | 15:17 |
| clarkb | PYTHONHASHSEED=12345 pretty_tox.sh | 15:18 |
| deepakcs | clarkb, ah ok, that was simple :) | 15:18 |
| gilliard | mriedem: so is this a question of the actual CPU flags on the machines that run the tests? | 15:18 |
| afazekas_ | sdague: is it planned to add check-devstack-dsvm-updown job to all project ? | 15:19 |
| deepakcs | clarkb, i will try with 4 cpus and hasseed tomorrow, but i hope (99% sure) with the c-bak disabled, we should not hit oom | 15:19 |
| mriedem | gilliard: i think so | 15:19 |
| sdague | afazekas_: no | 15:19 |
| *** imcsk8 has quit IRC | 15:19 | |
| *** imcsk8 has joined #openstack-infra | 15:19 | |
| deepakcs | clarkb, fungi there isn't /proc/cpuinfo in the logs, so had to determine total cpus from syslog indirectly, any reason cpuinfo is excluded ? | 15:20 |
| *** otter768 has joined #openstack-infra | 15:20 | |
| openstackgerrit | Merged openstack-infra/project-config: Change os-ansible-deployment checks to use scripts https://review.openstack.org/156229 | 15:20 |
| fungi | deepakcs: no reason i'm aware of. seems like that would be a reasonable improvement to the openstack-infra/devstack-gate repo | 15:20 |
| clarkb | it is logged... | 15:21 |
| deepakcs | fungi, ok, i will work on patch :) | 15:21 |
| clarkb | the problem is that jobs dies so hard we dont get the logs | 15:21 |
| clarkb | see mriedems question above | 15:21 |
| deepakcs | clarkb, in devstack setup host log file , u mean ? | 15:22 |
| openstackgerrit | Merged openstack-infra/shade: Don't compare images when image is None https://review.openstack.org/157055 | 15:22 |
| *** claudiub has joined #openstack-infra | 15:22 | |
| anteaya | fungi: when do you begin holidaying? | 15:22 |
| mriedem | i don't suppose /usr/share/libvirt/cpu_map.xml is logged anywhere | 15:23 |
| fungi | deepakcs: i'll include the cpuinfo in this log tarball once the manual run completes | 15:23 |
| sdague | afazekas: I consider projects knowing whether or not their services start to be kind of table stakes, and a responsibility for that project. We can try to put a little more sanity checking inside devstack normal runs, but it wasn't on my short list | 15:23 |
| anteaya | fungi: was thinking you would be in the warm today | 15:23 |
| deepakcs | fungi, thanks and gluster logs too (/var/log/glusterfs). | 15:23 |
| clarkb | deepakcs yes in the setup host log file | 15:23 |
| deepakcs | fungi, if this doesn't resolve oom, i would request providing ssh access to the env | 15:24 |
| fungi | anteaya: i'm heading up to norfolk tomorrow afternoon and working from car/hotel, then flying out wednesday morning and probably working from airline wireless until early afternoon est | 15:24 |
| anteaya | ah | 15:24 |
| deepakcs | clarkb, ok, but logging it separately as cpuinfo.txt would be more evident and useful i feel. | 15:24 |
| anteaya | working travel and warm | 15:24 |
| *** otter768 has quit IRC | 15:24 | |
| anteaya | nice | 15:24 |
| deepakcs | fungi, how long will u be gone ? have nice holidays :) | 15:25 |
| fungi | deepakcs: ~10 days | 15:25 |
| fungi | thanks! | 15:25 |
| AJaeger | fungi, 10 days? We'll miss you! Have a great vacation! | 15:26 |
| fungi | thanks AJaeger | 15:27 |
| *** sdake_ has joined #openstack-infra | 15:28 | |
| *** ssam2 has quit IRC | 15:32 | |
| *** jlvillal|afk is now known as jlvillal | 15:32 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Don't try to add an IP if there is one https://review.openstack.org/156088 | 15:33 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Return extra information for debugging on failures https://review.openstack.org/156411 | 15:33 |
| *** ssam2 has joined #openstack-infra | 15:33 | |
| *** achanda has joined #openstack-infra | 15:34 | |
| *** zigo_ has joined #openstack-infra | 15:36 | |
| *** mestery has quit IRC | 15:36 | |
| *** dkranz has quit IRC | 15:36 | |
| *** alexpilotti has joined #openstack-infra | 15:37 | |
| *** zz_dimtruck is now known as dimtruck | 15:38 | |
| *** radez_g0n3 is now known as radez | 15:38 | |
| *** yfried_ has quit IRC | 15:39 | |
| *** mrmartin has quit IRC | 15:39 | |
| *** dkranz has joined #openstack-infra | 15:40 | |
| *** asselin_ has joined #openstack-infra | 15:40 | |
| *** mattfarina has joined #openstack-infra | 15:41 | |
| *** radez is now known as radez_g0n3 | 15:43 | |
| *** achanda has quit IRC | 15:44 | |
| *** sergsh has quit IRC | 15:45 | |
| *** jawed has joined #openstack-infra | 15:45 | |
| dims | fungi: safe travels! | 15:48 |
| fungi | thanks dims | 15:48 |
| *** ChuckC has joined #openstack-infra | 15:49 | |
| *** mrmartin has joined #openstack-infra | 15:52 | |
| *** HeOS has quit IRC | 15:52 | |
| *** ChuckC_ has quit IRC | 15:52 | |
| *** MarkAtwood has joined #openstack-infra | 15:55 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Make image processing work for v2 https://review.openstack.org/156247 | 15:55 |
| *** xyang1 has joined #openstack-infra | 15:57 | |
| krotscheck | Storyboard Meeting in #openstack-meeting-3 | 16:00 |
| *** patrickeast has quit IRC | 16:00 | |
| *** tiswanso has quit IRC | 16:01 | |
| *** tiswanso_ has joined #openstack-infra | 16:02 | |
| *** dhritishikhar_ has quit IRC | 16:02 | |
| *** dkranz has quit IRC | 16:03 | |
| *** amotoki has joined #openstack-infra | 16:05 | |
| *** carl_baldwin has joined #openstack-infra | 16:06 | |
| *** tmcpeak has quit IRC | 16:06 | |
| anteaya | asselin: let me know how you make out with the mirror config issue | 16:07 |
| *** prmtl has joined #openstack-infra | 16:07 | |
| *** dkranz has joined #openstack-infra | 16:08 | |
| *** che-arne has joined #openstack-infra | 16:09 | |
| *** mpaolino has joined #openstack-infra | 16:11 | |
| *** yfried_ has joined #openstack-infra | 16:12 | |
| *** Kiall has quit IRC | 16:13 | |
| asselin | anteaya, sure | 16:13 |
| *** prmtl has quit IRC | 16:14 | |
| *** harlowja_at_home has joined #openstack-infra | 16:15 | |
| *** jcoufal has quit IRC | 16:15 | |
| *** tmcpeak has joined #openstack-infra | 16:16 | |
| *** erikmwilson has quit IRC | 16:16 | |
| *** erikmwil- is now known as erikmwilson | 16:16 | |
| *** belmoreira has quit IRC | 16:17 | |
| *** tmcpeak has quit IRC | 16:19 | |
| *** teran has quit IRC | 16:19 | |
| *** dannywilson has joined #openstack-infra | 16:20 | |
| jogo | mriedem: one of the issues I think we are seeing with which cpu to pick is xen vs kvn | 16:20 |
| jogo | kvm | 16:20 |
| jogo | since rax has xen (not libvirt xen at that) | 16:20 |
| *** devlaps has joined #openstack-infra | 16:21 | |
| *** mtanino has joined #openstack-infra | 16:21 | |
| *** dhritishikhar_ has joined #openstack-infra | 16:21 | |
| mriedem | hmm | 16:22 |
| mriedem | jogo: can we determine that at runtime in d-g when it's setting up? | 16:23 |
| mtreinish | jogo: can't we just restrict the multinode slaves to a single provider? | 16:23 |
| mtreinish | clarkb: ^^^ ? | 16:23 |
| *** dannywil_ has joined #openstack-infra | 16:23 | |
| *** dannywilson has quit IRC | 16:24 | |
| *** xgerman has joined #openstack-infra | 16:24 | |
| jogo | mriedem: so a few things: I tried using pentium | 16:24 |
| jogo | and that worked, but some tempest tests failed and thought that may be related | 16:25 |
| *** andreykurilin_ has joined #openstack-infra | 16:25 | |
| jogo | mriedem: and I think sorting this out at runtime makes the most sense | 16:25 |
| jogo | mriedem: on a side note, aiopcpu is working with tempest full, just without live migration. So that isn't too bad | 16:26 |
| *** spzala has joined #openstack-infra | 16:28 | |
| jeblair | mtreinish: no, we can never restrict slaves to a single provider. the whole system is fault-tolerant across both clouds | 16:29 |
| *** nelsnelson has quit IRC | 16:30 | |
| jeblair | mtreinish: we had a couple of days last week where we could not get any nodes from hpcloud; did you notice? :) | 16:30 |
| mtreinish | jeblair: no, I realize that. I just meant having both slaves for a multinode job allocated from the same place | 16:30 |
| jeblair | mtreinish: oh, they already are | 16:30 |
| jeblair | mtreinish: same AZ even | 16:30 |
| *** mwagner_ has joined #openstack-infra | 16:31 | |
| *** Kiall has joined #openstack-infra | 16:31 | |
| jogo | mtreinish: but single providers have multiple CPU types | 16:31 |
| mtreinish | jogo: yeah, I remember the problem now | 16:32 |
| mtreinish | for some reason I thought it was across some region boundry that was the issue | 16:32 |
| mtreinish | chalk it up to being before noon | 16:32 |
| *** e0ne has quit IRC | 16:32 | |
| *** mestery has joined #openstack-infra | 16:33 | |
| *** e0ne has joined #openstack-infra | 16:33 | |
| *** amitgandhinz has joined #openstack-infra | 16:33 | |
| jogo | mtreinish: hehe | 16:33 |
| *** dkranz has quit IRC | 16:36 | |
| clarkb | jogo is it possible to set explicit cpu flags rather than model? | 16:38 |
| clarkb | if so we can just union the flags of all nodes at runtime and use that | 16:38 |
| clarkb | if nova cant do that today it may make a good L feature | 16:39 |
| *** deepakcs has quit IRC | 16:39 | |
| *** ddieterly has quit IRC | 16:39 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Add tests for yaml anchor behaviour https://review.openstack.org/158350 | 16:40 |
| jogo | clarkb: great question, no idea | 16:40 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: support aliases defined in parent inside included yaml https://review.openstack.org/157735 | 16:40 |
| jogo | but I don't think nova supports that, not sure about libvirt | 16:41 |
| clarkb | pretty sure libvirt and qemu/kvm do | 16:41 |
| *** yjiang5 has joined #openstack-infra | 16:42 | |
| clarkb | model is a shortcut for more verbosely setting flGs | 16:42 |
| *** tonytan4ever has joined #openstack-infra | 16:42 | |
| clarkb | *flags | 16:42 |
| *** devlaps has quit IRC | 16:42 | |
| clarkb | problem here is the models we get and the flags exposed arent quite sane because we are nested | 16:42 |
| clarkb | eg core 2 never had sse4_2 but its in our core 2 processor flags | 16:43 |
| sdague | clarkb: yeh, so this is a really extreme edge case | 16:43 |
| sdague | so it's not something that was considered | 16:44 |
| sdague | also, what happens when a new n-cpu comes online with different flags? | 16:44 |
| clarkb | nested virt isnt an edge case imo | 16:44 |
| sdague | no, not nested virst | 16:44 |
| clarkb | sdague that is a problem regardless | 16:44 |
| clarkb | it wont be for us if we union flags though | 16:44 |
| sdague | expecting live migration to work across a heterogenous hardware set | 16:44 |
| clarkb | if you ask me live migration shouldnt be a cloud feature at all :) but we support it | 16:45 |
| sdague | well, we support it as an admin command | 16:45 |
| clarkb | and heterogenous hardware isnt really a problem if you can describe a subset to use | 16:46 |
| jeblair | (if you ask me, i'm really happy our production servers can get live-migrated when the cloud needs to perform maintenance) | 16:46 |
| clarkb | problem is we may not be able to because nova only accepts cpu models | 16:47 |
| *** yfried_ has quit IRC | 16:47 | |
| *** nelsnelson has joined #openstack-infra | 16:47 | |
| *** devlaps has joined #openstack-infra | 16:48 | |
| clarkb | jeblair ya but that will go away if we config drive | 16:48 |
| *** mpavone has quit IRC | 16:48 | |
| clarkb | config drive breaks live migration | 16:48 |
| *** amotoki has quit IRC | 16:48 | |
| SpamapS | I think live migration is necessary in small-medium scale clouds | 16:48 |
| *** tmcpeak has joined #openstack-infra | 16:49 | |
| SpamapS | If you have only, say, 36 compute hosts supporting a few thousand VMs, you can't afford to leave some running outdated software forever. | 16:49 |
| *** harlowja_at_home has quit IRC | 16:49 | |
| clarkb | I wonder if nova does support this and that is how hpcloud adds the sse4_2 flag | 16:50 |
| clarkb | jogo ^ | 16:50 |
| SpamapS | If you have 3600 compute hosts, then you just stop sending vms to the old ones till they drain a bit, then send the unlucky few running pets there that they'll be rebooted in a few weeks. | 16:50 |
| jeblair | SpamapS: i (personally) think it's more about use-case/audience. a cloud that is providing "virtual servers" would do well to have live-migration, otherwise they compete poorly with "real servers". but one providing "compute resources" perhaps can afford to ignore it | 16:50 |
| SpamapS | s/there/there notices/ | 16:50 |
| clarkb | mordred this may be a reason to nova agent after all :/ | 16:51 |
| *** deepakcs has joined #openstack-infra | 16:51 | |
| *** dkranz has joined #openstack-infra | 16:51 | |
| SpamapS | jeblair: well said. I think we agree, because I think a cloud running virtual servers is bound to be small - medium. :) | 16:51 |
| jeblair | heh :) | 16:52 |
| SpamapS | jeblair: on another note, in case you missed it, presentty is in Debian NEW | 16:52 |
| jeblair | SpamapS: neato! thanks! | 16:53 |
| *** hdd has joined #openstack-infra | 16:53 | |
| jeblair | i should do more work on it :) | 16:53 |
| *** amitgandhinz has quit IRC | 16:53 | |
| SpamapS | jeblair: also, I took a stab at making gear py3 compatible. | 16:54 |
| SpamapS | Oh wait, I stand corrected! | 16:55 |
| *** ildikov has quit IRC | 16:55 | |
| *** gyee has joined #openstack-infra | 16:55 | |
| SpamapS | jeblair: presentty _cleared_ NEW this morning. | 16:56 |
| mordred | clarkb: what? | 16:56 |
| jeblair | SpamapS: what does that mean? | 16:56 |
| * mordred reading scrollback | 16:56 | |
| fungi | jeblair: means its pending landing in sid now | 16:57 |
| *** reed has joined #openstack-infra | 16:57 | |
| fungi | probably show up in sid/unstable sometime today depending on scheduling | 16:57 |
| *** ChuckC_ has joined #openstack-infra | 16:57 | |
| mordred | SpamapS: rax provides "virtual servesr" - it's why they're useful | 16:57 |
| SpamapS | definitely within 24 hours | 16:57 |
| mordred | clarkb: when you say "config drive breaks live migration" - do you have more info so I can understand the issue? | 16:58 |
| SpamapS | mordred: I don't mean to deride virtual server clouds. I mean to say why I think people get confused and say live migration shouldn't exist in a cloud. | 16:58 |
| fungi | though britney won't consider it for testing until after jessie releases | 16:58 |
| mordred | oh | 16:58 |
| openstackgerrit | Boris Pavlovic proposed openstack/requirements: Add flake8-double-quotas to requirements https://review.openstack.org/158357 | 16:58 |
| mordred | SpamapS: gotcha. I'm just happy that rax disagrees with those people | 16:58 |
| *** ChuckC_ has quit IRC | 16:58 | |
| clarkb | mordred nova cannot live migrate with config drives attached | 16:58 |
| SpamapS | hmm | 16:59 |
| clarkb | there is a long standing open bug about it. something about how your device goes away iirc | 16:59 |
| SpamapS | doesn't rax do config drives? | 16:59 |
| mordred | clarkb: so, if I "nova boot --config-drive=1" that vm can never be live-migrated? | 16:59 |
| mordred | clarkb: or only if the config-drive is mounted in some manner? | 16:59 |
| clarkb | mordred aiui correct. this is why our multinode test envs use metadata server and not config drive | 16:59 |
| jeblair | clarkb: can it live-migrate with cinder volumes attached? | 16:59 |
| jogo | clarkb: I think that is considered a bug, and not a feature at least | 17:00 |
| mordred | well, there are two issues here: | 17:00 |
| mordred | a) what we do for nodepool nodes | 17:00 |
| clarkb | jeblair I do not know about cinder specifically but I think iscsi may make that work better | 17:00 |
| mordred | b) what we do for our servers | 17:00 |
| *** ChuckC has quit IRC | 17:00 | |
| mordred | a) we do not care about live migration | 17:00 |
| mordred | b) we do | 17:00 |
| *** ChuckC_ has joined #openstack-infra | 17:00 | |
| jeblair | mordred: agreed | 17:00 |
| clarkb | mordred correct | 17:00 |
| mordred | I'm more opposed to having nova-agent running in nodepool nodes, so I think that's easy | 17:00 |
| mordred | I _do- have a nova-agent element for dib | 17:00 |
| mordred | so if we need to do that route for long-lived servers but still want to make our own base images, we're set | 17:01 |
| * jogo notes that nova-agent isn't a nova thing | 17:01 | |
| mordred | however, I'd suggest that we poke jogo in the face with a stick until he fixes the config-drive live migration :) | 17:01 |
| *** vigneshvar has joined #openstack-infra | 17:01 | |
| clarkb | also mikal knows all about this bug so we should just ask him in a few hours | 17:01 |
| mordred | jogo: yah. it's a rax thing | 17:01 |
| mordred | jogo: that talks to xenstore | 17:01 |
| mordred | it's their way of doing dhcp without running dhcp | 17:02 |
| *** viktors is now known as viktors|afk | 17:02 | |
| mordred | of course, if rax would just dhcp, none of this would really be necessary | 17:02 |
| jeblair | mordred, clarkb: i'd actually like to not run "nova-rootkit" on our servers if we can avoid it :) | 17:03 |
| clarkb | +1 | 17:03 |
| jogo | clarkb: yeah so next step is to follow the instructions danpb has and do automatic cpu type selection in d-g | 17:03 |
| mordred | jeblair: ++ | 17:03 |
| *** scheuran has quit IRC | 17:03 | |
| clarkb | jogo we should also look into directly setting flags | 17:04 |
| clarkb | jogo I think that may be easier long run | 17:04 |
| jogo | clarkb: hmm yeah, that is worth looking into | 17:04 |
| *** rwsu has joined #openstack-infra | 17:05 | |
| mordred | jeblair: totally agree. it's also possible that something at rax is different and live migration would work there - so it might be worth poking mikal to see if he knows any more informatino | 17:06 |
| *** tiswanso_ has quit IRC | 17:06 | |
| mordred | I would honestly be very surprised if rax let something like that out into the wild, since it would drastically confuse the customers who are used to running multi-year servers there | 17:07 |
| mordred | like, it may be one of those things that's easy to fix for the rax xen case but harder to fix broadly or something | 17:07 |
| *** mrmartin has quit IRC | 17:07 | |
| jeblair | yeah, worth digging into | 17:08 |
| *** HeOS has joined #openstack-infra | 17:08 | |
| *** yfried_ has joined #openstack-infra | 17:09 | |
| fungi | SpamapS: rax does support config drive, but i'm not sure whether they do live migration (they stopped supporting online resize a while back at least) | 17:09 |
| clarkb | mordred yup we should definitel have mikal clue bat me/us | 17:10 |
| *** mpaolino has quit IRC | 17:11 | |
| *** tiswanso_ has joined #openstack-infra | 17:11 | |
| bhunter71 | I'm drawing a blank on something- when nodepool adds nodes to my jenkins01, they're currently getting through the whole gearman thing, and to the final jenkins-slave-song-and-dance. I'm getting "hudson.AbortException: Cannot find SSH User credentials with id: null". I wanted global credentials 'jenkins' here. what did I do wrong? | 17:11 |
| fungi | or have him delegate cluebatting to jhesketh/mattoliverau | 17:11 |
| *** xgerman has quit IRC | 17:11 | |
| *** dkranz has quit IRC | 17:11 | |
| *** jawed has quit IRC | 17:12 | |
| pleia2 | good morning :) | 17:12 |
| pleia2 | clarkb: yay | 17:12 |
| *** BobBall is now known as BobBall_AWOL | 17:12 | |
| fungi | bhunter71: the credentials-id is the uuid of the jenkins credentials object. i usually obtain it from config.xml | 17:13 |
| bhunter71 | ah, thanks fungi. (necessitated by a jenkins master getting corrupted/rebuilt) | 17:14 |
| fungi | bhunter71: essentially, configure a ssh credentials set in jenkins for nodepool's use, then go fishing in the jenkins config on disk to discover the corresponding uuid for it | 17:14 |
| clarkb | its one of my big gripes with jenkins move to the credentials manager | 17:15 |
| clarkb | using the key in $HOME/.ssh worked fine before :/ | 17:15 |
| *** carl_baldwin has quit IRC | 17:16 | |
| fungi | yeah, the admin experience there is sort of lacking | 17:16 |
| *** MarkAtwood_ has joined #openstack-infra | 17:16 | |
| *** JayJ has joined #openstack-infra | 17:17 | |
| *** ihrachyshka has quit IRC | 17:19 | |
| *** teran has joined #openstack-infra | 17:20 | |
| *** tkelsey has joined #openstack-infra | 17:20 | |
| *** unicell has quit IRC | 17:20 | |
| *** otter768 has joined #openstack-infra | 17:21 | |
| *** thedodd has joined #openstack-infra | 17:21 | |
| *** baoli has quit IRC | 17:22 | |
| *** baoli has joined #openstack-infra | 17:22 | |
| *** marun has joined #openstack-infra | 17:23 | |
| *** mpaolino has joined #openstack-infra | 17:23 | |
| *** baoli has quit IRC | 17:23 | |
| *** sdake_ has quit IRC | 17:23 | |
| *** baoli has joined #openstack-infra | 17:24 | |
| *** teran has quit IRC | 17:25 | |
| *** MaxV has quit IRC | 17:25 | |
| *** jlk has quit IRC | 17:25 | |
| *** jlk has joined #openstack-infra | 17:25 | |
| *** dkranz has joined #openstack-infra | 17:25 | |
| *** otter768 has quit IRC | 17:25 | |
| *** johnthetubaguy has joined #openstack-infra | 17:25 | |
| *** baoli has quit IRC | 17:26 | |
| *** baoli has joined #openstack-infra | 17:26 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Add a method to create image snapshots from nova. https://review.openstack.org/158119 | 17:27 |
| afazekas_ | ianw, ping | 17:27 |
| jogo | johnthetubaguy: so your a rax person | 17:27 |
| johnthetubaguy | jogo: hey | 17:27 |
| johnthetubaguy | whats up? | 17:27 |
| jogo | we are having issues with the CPU types for live migration | 17:27 |
| johnthetubaguy | hmm, interesting | 17:28 |
| jogo | johnthetubaguy: see https://review.openstack.org/#/c/141530/ | 17:28 |
| jogo | mriedem: ^ | 17:28 |
| *** ddieterly has joined #openstack-infra | 17:28 | |
| jogo | clarkb mriedem: btw got some feedback from danpb https://review.openstack.org/#/c/158334/ | 17:29 |
| *** jistr has quit IRC | 17:29 | |
| mriedem | jogo: shit creek | 17:29 |
| *** tiswanso_ has quit IRC | 17:29 | |
| mriedem | :) | 17:29 |
| johnthetubaguy | OK, so you want to get a list of minimum CPU requirements in rax, or am I missing something? | 17:30 |
| mriedem | is this why we generally assume live migration doesn't work? | 17:30 |
| *** radez_g0n3 is now known as radez | 17:30 | |
| openstackgerrit | Merged openstack-infra/puppet-apparmor: Update metadata.json https://review.openstack.org/153294 | 17:30 |
| *** tiswanso_ has joined #openstack-infra | 17:30 | |
| rcarrillocruz | hey folks | 17:31 |
| jogo | johnthetubaguy: just sanity checking if what we are seeing is right | 17:31 |
| jogo | johnthetubaguy: we can't even use 'Conroe' | 17:31 |
| jogo | which is very old at this point | 17:32 |
| rcarrillocruz | i've seen in our internal infra that some Gerrit projects have a 'config' branch with metadata from Gerrit ( a groups file and a project.config, that contain acls about that Gerrit project) | 17:32 |
| rcarrillocruz | anyone seen this? | 17:32 |
| clarkb | jogo right I think you need newer to match up with the flags that are set | 17:32 |
| jeblair | rcarrillocruz: i think every project should have it | 17:33 |
| clarkb | jogo OR set all capabilities directly | 17:33 |
| jeblair | rcarrillocruz: "refs/meta/config" | 17:33 |
| zaro | morning | 17:33 |
| rcarrillocruz | really? i've noticed that in some | 17:33 |
| rcarrillocruz | not in all projects | 17:33 |
| rcarrillocruz | hmm | 17:33 |
| *** sabeen3 has joined #openstack-infra | 17:33 | |
| *** sabeen has quit IRC | 17:34 | |
| jeblair | rcarrillocruz: https://review.openstack.org/Documentation/config-project-config.html#_the_code_refs_meta_config_code_namespace | 17:34 |
| zaro | rcarrillocruz: i think some may be empty because it inherits from All-Projects | 17:34 |
| rcarrillocruz | AHA! | 17:34 |
| rcarrillocruz | thx zaro, that may be the reason | 17:34 |
| rcarrillocruz | thx jeblair | 17:34 |
| *** mestery has quit IRC | 17:34 | |
| rcarrillocruz | i was worried we had cruft on our Gerrit instance | 17:34 |
| rcarrillocruz | phew | 17:34 |
| jeblair | rcarrillocruz: i think jeepyb should put something there in all cases | 17:35 |
| *** mestery has joined #openstack-infra | 17:35 | |
| *** SumitNaiksatam has quit IRC | 17:35 | |
| jeblair | zaro: ^ | 17:35 |
| *** devvesa has quit IRC | 17:35 | |
| jogo | clarkb: I think setting capabilities directly would be good | 17:35 |
| fungi | jeblair: governance repo change idea https://review.openstack.org/158371 | 17:35 |
| * anteaya goes for a walk, in the lovely sunny day | 17:35 | |
| rcarrillocruz | jeblair: so your take is that regardless of inheritance, jeepyb should put a config branch for all repos it creates? | 17:36 |
| deepakcs | fungi, ping, any update on the manual run ? | 17:36 |
| *** SumitNaiksatam has joined #openstack-infra | 17:36 | |
| *** derekh has quit IRC | 17:36 | |
| *** ChuckC_ has quit IRC | 17:36 | |
| jeblair | fungi: seems reasonable, at least until we finish up docs.o.o/infra | 17:37 |
| zaro | rcarrillocruz: i believe gerrit provides a refs/meta/config for every project, just that some may not contain anything because of inheritance | 17:37 |
| fungi | deepakcs: yes, it completed and i'm writing a follow-up e-mail now, logs are at http://fungi.yuggoth.org/tmp/logs3.tar | 17:37 |
| zaro | rcarrillocruz: but i think wht jeblair is saying is that jeepyb will always add something to project.config for every subproject? | 17:38 |
| *** e0ne has quit IRC | 17:38 | |
| SpamapS | oh my | 17:38 |
| clarkb | the way we use it it will because we always have acls | 17:38 |
| SpamapS | PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND | 17:38 |
| jeblair | zaro: i think so, yeah. | 17:38 |
| SpamapS | 1006 clint 20 0 14.204g 0.010t 2736 S 202.6 68.7 1578:51 python | 17:39 |
| SpamapS | 1006 pts/0 Sl 1579:07 python -m subunit.run discover -t ./ gear/tests/ --load-list /tmp/tmpFdXUhm | 17:39 |
| SpamapS | rogue unit test | 17:39 |
| rcarrillocruz | hmm ok, let me check inheritance on projects showing that branch and projects without it... | 17:39 |
| jeblair | SpamapS: neat! oh, there are two outstanding gear patches, i don't know if they would have an effect. | 17:39 |
| SpamapS | jeblair: it's spinning on this: | 17:40 |
| SpamapS | pid 1030] sendto(15, "\0REQ\0\0\0\36\0\0\0\0", 12, 0, NULL, 0 <unfinished ...> | 17:40 |
| SpamapS | [pid 1025] epoll_ctl(5, EPOLL_CTL_DEL, 16, {EPOLLRDBAND|EPOLLERR|EPOLLHUP|EPOLLRDHUP|EPOLLET|0x39ca9000, {u32=32730, u64=24297708030296026}}) = 0 | 17:40 |
| zaro | pleia2: threw a logo at you https://review.openstack.org/#/c/158134/ | 17:40 |
| SpamapS | I only noticed because my box slowed to a crawl :-P | 17:41 |
| pleia2 | zaro: good idea, thanks | 17:41 |
| zaro | pleia2: can you help create a publication branch for me? | 17:41 |
| pleia2 | zaro: I can! let me know what you need and I'll see about doing it | 17:43 |
| *** sdake_ has joined #openstack-infra | 17:43 | |
| johnthetubaguy | jogo: mriedem: I think I am missing context, this is getting KVM live working when running on rax? | 17:44 |
| *** ChuckC has joined #openstack-infra | 17:45 | |
| *** Ala has quit IRC | 17:45 | |
| zaro | pleia2: would like a "jenkins-job-builder" branch. | 17:45 |
| *** Sukhdev has joined #openstack-infra | 17:46 | |
| clarkb | johnthetubaguy yes we are trying to test it in our multi node test envs | 17:46 |
| pleia2 | zaro: someone can correct me if I'm wrong, but I *think* you create the branch yourself and then when you need a version of it - like you gave the talk at a conference, is when we need to do tagging | 17:46 |
| *** salv-orlando has joined #openstack-infra | 17:46 | |
| johnthetubaguy | clarkb: so they are quite new xeon chips in performance, so I guess xen might be hiding some CPU flags | 17:47 |
| zaro | pleia2: instructions in README says you need to contact infra to create a branch. | 17:47 |
| zaro | pleia2: i mean README from template branch | 17:48 |
| *** dizquierdo has quit IRC | 17:48 | |
| marcusvrn | hi guys! is there any way to change the ci account name? for example: hds-hnas-ci to hds-ci? | 17:48 |
| pleia2 | zaro: ah, branch name in gerrit | 17:48 |
| johnthetubaguy | clarkb: jogo: I don't think we tell xen to hide anything at this point, but I should check really | 17:48 |
| zaro | pleia2: yep, that's what i need | 17:48 |
| pleia2 | zaro: got it, looking into that now | 17:48 |
| *** AJaeger_ has joined #openstack-infra | 17:50 | |
| *** zul has quit IRC | 17:51 | |
| pleia2 | clarkb: re: publication branches in gerrit, what do I use for "Initial Revision" ? | 17:51 |
| *** coolsvap is now known as coolsvap_ | 17:52 | |
| jeblair | pleia2: i'd use 'template' | 17:52 |
| pleia2 | jeblair: ok, thanks | 17:52 |
| clarkb | pleia2 ya, since thats what the changes should be based on | 17:53 |
| *** zul has joined #openstack-infra | 17:53 | |
| pleia2 | zaro: done! | 17:53 |
| zaro | thanks! | 17:53 |
| jeblair | nibalizer: 151098 safe to aprv? | 17:53 |
| clarkb | avoids rebasing to push for review | 17:53 |
| pleia2 | hm, when I added it through the gerrit UI it added a "master" subsection that the others don't have | 17:54 |
| pleia2 | can someone else take a look? | 17:54 |
| nibalizer | jeblair: yea doesn't affect anything yet | 17:54 |
| nibalizer | except of course, my heard *thumpthump* | 17:54 |
| nibalizer | s/heard/heart/ | 17:54 |
| jeblair | aww | 17:54 |
| jeblair | pleia2: not sure what you are referring to | 17:55 |
| pleia2 | jeblair: https://review.openstack.org/#/admin/projects/openstack-infra/publications,branches | 17:55 |
| openstackgerrit | Merged openstack-infra/puppet-gerrit: Make sure Gerrit sites are using the proper libraries. https://review.openstack.org/151368 | 17:55 |
| openstackgerrit | Merged openstack-infra/puppet-gerrit: Add the Gerrit replication plugin paramater 'defaultForceUpdate' https://review.openstack.org/155142 | 17:55 |
| pleia2 | see "jenkins-job-builder" that I just added for zaro's new slidedeck | 17:55 |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Update openstackdocstheme https://review.openstack.org/158379 | 17:55 |
| *** marun has quit IRC | 17:55 | |
| jeblair | pleia2: where's the master subsection you're talking about? | 17:56 |
| *** yamahata has joined #openstack-infra | 17:56 | |
| *** unicell has joined #openstack-infra | 17:56 | |
| *** bhuvan has joined #openstack-infra | 17:56 | |
| pleia2 | jeblair: oh, nevermind, it's just below jjb, but it's the actual master being alphabetical | 17:56 |
| *** sdake__ has joined #openstack-infra | 17:56 | |
| pleia2 | more coffee \o/ | 17:56 |
| jeblair | ah :) | 17:56 |
| *** unicell has quit IRC | 17:57 | |
| pleia2 | I'll get used to this UI soon :) | 17:57 |
| *** unicell has joined #openstack-infra | 17:57 | |
| *** dimtruck is now known as zz_dimtruck | 17:57 | |
| *** ghostpl_ has quit IRC | 17:59 | |
| *** unicell has quit IRC | 17:59 | |
| *** zz_dimtruck is now known as dimtruck | 18:00 | |
| *** unicell has joined #openstack-infra | 18:00 | |
| *** sdake_ has quit IRC | 18:00 | |
| *** nelsnelson has quit IRC | 18:00 | |
| jeblair | nibalizer, clarkb: is there a reason we're calling our modules "openstackci-foo" instead of "openstackinfra-foo"? | 18:00 |
| jeblair | in general, we're trying to s/ci/infra/ | 18:00 |
| *** nelsnelson has joined #openstack-infra | 18:01 | |
| *** Ryan_Lane has joined #openstack-infra | 18:01 | |
| nibalizer | we already had the openstackci name and user on puppetforge, but I don't care | 18:01 |
| openstackgerrit | Merged stackforge/python-jenkins: Add get_job_info_regex method to get info about a set of jobs https://review.openstack.org/155878 | 18:01 |
| openstackgerrit | Merged openstack-infra/puppet-httpd: Rename apache2 to httpd globally https://review.openstack.org/151098 | 18:02 |
| jeblair | nibalizer: yeah, i think a legacy username is fine, but if it affects how the modules are named / namespaced, it's probably worth another look | 18:02 |
| nibalizer | its ~60 one line commits to change it | 18:03 |
| nibalizer | and much easier to change it now than after we start forge publishing | 18:03 |
| jeblair | nibalizer: is module name completely independent from the user who uploads? | 18:05 |
| jeblair | nibalizer: added it to tomorrow's agenda, just to make sure we're all in agreement before we patch bomb | 18:06 |
| *** dhritishikhar_ has quit IRC | 18:06 | |
| *** ildikov has joined #openstack-infra | 18:07 | |
| *** BadCub has quit IRC | 18:07 | |
| *** beecee has joined #openstack-infra | 18:07 | |
| *** afazekas_ has quit IRC | 18:07 | |
| *** dustins_ has joined #openstack-infra | 18:09 | |
| *** david-lyle_afk is now known as david-lyle | 18:09 | |
| *** unicell has quit IRC | 18:10 | |
| *** ChuckC has quit IRC | 18:11 | |
| *** dustins has quit IRC | 18:12 | |
| jeblair | krotscheck: the bottom of your puppet-storyboard stack (156755) has a commit message that says do not merge until an abandoned change (156764) merges | 18:12 |
| *** jlibosva has quit IRC | 18:13 | |
| *** lucasagomes is now known as lucas-dinner | 18:14 | |
| *** tonytan4ever has quit IRC | 18:14 | |
| *** tonytan4ever has joined #openstack-infra | 18:15 | |
| SpamapS | jeblair: Hey, so I've been poking at gear a bit, and I can't seem to find how to make a blocking client submitJob call that waits for the job response, or a non-blocking way to go ahead and wait for the response if I have nothing else to do. | 18:15 |
| SpamapS | jeblair: I looked and zuul just spins with time.sleep(1) | 18:15 |
| *** lucas-dinner has left #openstack-infra | 18:15 | |
| *** dtantsur is now known as dtantsur|afk | 18:16 | |
| SpamapS | Not sure the protocol has something that helps here, one might just need to go ahead and select/poll on the socket. | 18:16 |
| *** deepakcs has quit IRC | 18:16 | |
| fungi | clarkb: i have a feeling http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/scripts/restrict_memory.sh isn't working for devstack-centos7 images. free says ~24gb | 18:16 |
| *** achanda has joined #openstack-infra | 18:16 | |
| *** mrmartin has joined #openstack-infra | 18:17 | |
| *** achanda has quit IRC | 18:18 | |
| krotscheck | jeblair: I need to fix that. | 18:18 |
| krotscheck | sec | 18:18 |
| * krotscheck just got into the office | 18:18 | |
| *** achanda has joined #openstack-infra | 18:18 | |
| jeblair | SpamapS: you can subclass Worker and implement "handleWorkComplete" (and friends) | 18:19 |
| *** mpaolino has quit IRC | 18:19 | |
| *** tonytan4ever has quit IRC | 18:19 | |
| SpamapS | jeblair: yes, however, I still have to spin on time.sleep() until they are called. | 18:20 |
| SpamapS | unless there's a thread I can join? | 18:20 |
| *** ssam2 has quit IRC | 18:20 | |
| openstackgerrit | Merged openstack-infra/puppet-storyboard: Add metadata.json to puppet-storyboard https://review.openstack.org/136591 | 18:20 |
| *** ivar-lazzaro has joined #openstack-infra | 18:20 | |
| jeblair | SpamapS: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/merger/client.py | 18:20 |
| openstackgerrit | Paul Michali proposed openstack-infra/project-config: Setup non-voting VPNaaS func test for StrongSwan https://review.openstack.org/158389 | 18:20 |
| fungi | clarkb: er, says ~30gb i mean | 18:20 |
| fungi | clarkb: on hpcloud obviously | 18:21 |
| *** teran has joined #openstack-infra | 18:21 | |
| *** amrith has joined #openstack-infra | 18:21 | |
| jeblair | SpamapS: yeah, there isn't an out-of-the box way to do that... | 18:21 |
| jeblair | SpamapS: we could probably put an event on the job and wait on that | 18:21 |
| *** ivar-lazzaro has quit IRC | 18:22 | |
| amrith | hi ... could I get someone on the infra team to help get https://review.openstack.org/#/c/157955/ merged please. It is holding up a number of changes in Trove. | 18:22 |
| *** ivar-lazzaro has joined #openstack-infra | 18:22 | |
| AJaeger_ | amrith: +2ed. | 18:24 |
| *** mestery has quit IRC | 18:24 | |
| amrith | Andreas, thanks! | 18:24 |
| SpamapS | jeblair: might just need an equivalent of this http://gearman.info/libgearman/gearman_client_wait.html | 18:24 |
| jeblair | SpamapS: so we could have the default packet handler for those packets call job.wait_event.set(). then you could just do "job.wait()" | 18:24 |
| *** ryanpetrello has quit IRC | 18:25 | |
| *** patrickeast has joined #openstack-infra | 18:25 | |
| arosen | fungi: have you seen this nodepool issue? http://paste.openstack.org/show/180702/ I have the directory: /home/nodepool/nodepool_dib/devstack-trusty-dib-1424651675.d but not /home/nodepool/nodepool_dib/devstack-trusty-dib-1424651675 | 18:25 |
| *** ryanpetrello has joined #openstack-infra | 18:25 | |
| *** zul has quit IRC | 18:25 | |
| *** mestery has joined #openstack-infra | 18:25 | |
| openstackgerrit | Bharat Kumar Kobagana proposed openstack-infra/project-config: Adding glusterfs job to glusterfs plugin's check pipeline https://review.openstack.org/158392 | 18:25 |
| *** zul has joined #openstack-infra | 18:25 | |
| *** teran has quit IRC | 18:26 | |
| *** rkukura has joined #openstack-infra | 18:26 | |
| *** yfried_ has quit IRC | 18:26 | |
| *** vilobhmm has joined #openstack-infra | 18:27 | |
| fungi | arosen: looks like maybe your image-create never finished so image-upload doesn't find it? | 18:27 |
| *** jp_at_hp has quit IRC | 18:27 | |
| arosen | fungi: i was wondering if its actually missing the file extension ? and should be passing in : devstack-trusty-dib-1424651675.qcow2 | 18:28 |
| arosen | because I have that file as well. | 18:28 |
| arosen | fungi: changing the code to this makes the image upload, i'll see if it works: http://paste.openstack.org/show/180703/ | 18:29 |
| *** vilobhmm has quit IRC | 18:29 | |
| fungi | arosen: oh, maybe... it does look like it wants that to be a file it can read from | 18:29 |
| arosen | yea i'm pretty sure it wants to be that disk file. | 18:29 |
| nibalizer | jeblair: no i believe module-author-name and logged in name must be identical | 18:30 |
| fungi | arosen: i think filename is supposed to include the appropriate suffix | 18:30 |
| *** tiswanso_ has quit IRC | 18:30 | |
| jeblair | nibalizer: do you know who set up the user? | 18:30 |
| *** vilobhmm has joined #openstack-infra | 18:31 | |
| nibalizer | jeblair: the credentials for that user are in hiera | 18:31 |
| nibalizer | i think mordred set it up | 18:31 |
| fungi | yeah, mordred added it to hiera according to the git log | 18:31 |
| *** dustins_ has quit IRC | 18:31 | |
| fungi | i recall seeing it in there | 18:32 |
| *** carl_baldwin has joined #openstack-infra | 18:32 | |
| *** dustins has joined #openstack-infra | 18:32 | |
| *** vilobhmm1 has joined #openstack-infra | 18:32 | |
| mordred | what did I do? | 18:32 |
| mordred | yes | 18:32 |
| *** vilobhmm_ has joined #openstack-infra | 18:32 | |
| jeblair | mordred: i was thinking that we probably want our modulenames to be "openstackinfra-foo" | 18:32 |
| mordred | I set it up AGES ago when things were still called openstackci | 18:32 |
| mordred | jeblair: ++ | 18:33 |
| jeblair | mordred: so i reckon we need a new user, based on what nibalizer just said | 18:33 |
| mordred | ++ | 18:33 |
| *** harlowja has joined #openstack-infra | 18:33 | |
| jeblair | mordred: you think you could take care of that? | 18:34 |
| *** tiswanso_ has joined #openstack-infra | 18:34 | |
| *** ashleighfarnham has joined #openstack-infra | 18:34 | |
| *** mestery has quit IRC | 18:34 | |
| mordred | jeblair: sure | 18:35 |
| *** dustins has quit IRC | 18:35 | |
| *** vilobhmm has quit IRC | 18:35 | |
| *** vilobhmm_ is now known as vilobhmm | 18:35 | |
| *** dustins has joined #openstack-infra | 18:36 | |
| *** mestery has joined #openstack-infra | 18:38 | |
| fungi | clarkb: on further investigation, restrict_memory.sh is only working on devstack-precise, not devstack-trusty nor devstack-centos7 | 18:38 |
| mordred | awesome | 18:38 |
| clarkb | wow | 18:38 |
| *** ivar-laz_ has joined #openstack-infra | 18:38 | |
| clarkb | dib related possibly? | 18:39 |
| *** melwitt has joined #openstack-infra | 18:39 | |
| *** ivar-laz_ has quit IRC | 18:39 | |
| clarkb | though dib rebuilds grub rules | 18:39 |
| fungi | clarkb: maybe... | 18:39 |
| SpamapS | jeblair: I'll think about it some as I make this little custom client thing I've been poking at. Thanks for your thoughts. | 18:39 |
| clarkb | but precise is grub1 the others are grub2 | 18:39 |
| SpamapS | morganfainberg: Hi, so, broadcasting.. | 18:39 |
| fungi | clarkb: this is based entirely on the output of free on representatives of those node types in hpcloud | 18:39 |
| *** ivar-laz_ has joined #openstack-infra | 18:40 | |
| fungi | clarkb: also i can't find where our cgroups enforcement is getting applied any longer | 18:40 |
| *** ivar-laz_ has quit IRC | 18:40 | |
| SlickNik | Please can I get some eyeballs on https://review.openstack.org/#/c/157955/ when folks get a chance? Thanks! | 18:41 |
| *** ivar-laz_ has joined #openstack-infra | 18:41 | |
| *** tkelsey has quit IRC | 18:41 | |
| *** koolhead17 has quit IRC | 18:41 | |
| *** ivar-lazzaro has quit IRC | 18:41 | |
| sdague | clarkb: config drive preventing live migration is a bug, and I think tonyb might be working on that one now | 18:42 |
| *** ivar-laz_ has quit IRC | 18:42 | |
| *** zul has quit IRC | 18:42 | |
| *** marun has joined #openstack-infra | 18:42 | |
| clarkb | no cgroups on single use nodes | 18:42 |
| clarkb | fungi ^ | 18:42 |
| mordred | SlickNik, sdague, jogo: ^^ why would we exempt trove from the stable-client rules? | 18:42 |
| mordred | I just want to make sure I understand | 18:42 |
| *** ivar-lazzaro has joined #openstack-infra | 18:42 | |
| fungi | clarkb: we were apparently including jenkins::cgroups from openstack_project::slave but that doesn't get used by openstack_project::single_use_slave | 18:42 |
| fungi | clarkb: yeah | 18:42 |
| sdague | the clients are all capped on stable branches now | 18:42 |
| sdague | mordred: ^^^ | 18:42 |
| sdague | so it doesn't do anything | 18:43 |
| fungi | clarkb: so, intentional? | 18:43 |
| mordred | sdague: ok. so this isn't special casing trove out of something, it's just reflecting the reality we want? | 18:43 |
| *** ddieterly has quit IRC | 18:43 | |
| clarkb | fungi I think idea was we didnt need it for single use nodes | 18:43 |
| clarkb | fungi but the glusterfs job is evidence for otherwise | 18:43 |
| sdague | it's reflecting that there was 0 direct testing of client compatibility, and capping on stable branches removed all our indirect testing | 18:43 |
| *** ddieterly has joined #openstack-infra | 18:44 | |
| mordred | gotcha | 18:44 |
| *** zul has joined #openstack-infra | 18:44 | |
| mordred | sdague: thanks | 18:44 |
| fungi | clarkb: well, the worker i randomly selected to rerun it on today was in hpcloud, and apparently had 30gb ram available to it (which is what sent me down the path of figuring out why restrict_memory.sh and jenkins::cgroups weren't coming into play and whether that was what caused the job to not oom) | 18:44 |
| sdague | I think if you look at those juno / icehouse jobs you'll see the pip freeze shows it's using an old version even with the LIB_FROM_GIT | 18:44 |
| clarkb | fungi the memory restriction to 8gb is done by kernel boot params | 18:45 |
| SlickNik | mordred — no this isn't special casing for trove — what sdague said. | 18:45 |
| clarkb | fungi not cgroups | 18:45 |
| clarkb | check grub config | 18:45 |
| fungi | clarkb: right, so free should be reporting 8gb rather than 30gb there i think | 18:46 |
| fungi | i'll poke around in them and see what they have configured | 18:46 |
| sdague | clarkb: cpu_mode & cpu_model from nova might give you what you need here | 18:46 |
| clarkb | fungi yup | 18:46 |
| *** asselin_ has quit IRC | 18:47 | |
| clarkb | sdagye not quite the trouble is our rax/hp VMs are a wierd combo of flags that doesnt really overlap with libvirt presets | 18:47 |
| sdague | clarkb: and there is no dumbest common denomentator? | 18:47 |
| greghaynes | SpamapS: Trying to find on storyboard that backlog we created, got any pointers? | 18:47 |
| clarkb | sdague not yet no | 18:48 |
| *** dangers is now known as dangers_away | 18:48 | |
| clarkb | even pentium doesnt work | 18:48 |
| SpamapS | greghaynes: it's in the list of stories ;) | 18:48 |
| greghaynes | ya dont say? | 18:48 |
| sdague | clarkb: so that's all just an xml file in libvirt | 18:49 |
| sdague | /usr/share/libvirt/cpu_map.xml | 18:49 |
| sdague | my guess is hpcloud just has a modified one of those | 18:49 |
| fungi | clarkb: looking on a devstack-trusty-hpcloud node i see that /etc/default/grub is missing mem=8G even though the sed pattern as written in restrict_memory.sh does correctly add it. could this be getting reset somehow? | 18:49 |
| sdague | and that if you *really* wanted to here, you could do the same thing there | 18:50 |
| fungi | clarkb: i'll go digging in nodepool image logs to see if it mentions getting run at all | 18:50 |
| SpamapS | greghaynes: I can't seem to find it actually | 18:50 |
| greghaynes | \O/ its not total pebkac | 18:51 |
| SpamapS | greghaynes: https://storyboard.openstack.org/#!/story/2000175 | 18:51 |
| openstackgerrit | Paul Michali proposed openstack-infra/project-config: Setup non-voting VPNaaS func test for StrongSwan https://review.openstack.org/158389 | 18:51 |
| SpamapS | greghaynes: Had to go to the openstack-infra project, then list stories.. last one listed. :-P | 18:51 |
| greghaynes | ah, ty | 18:52 |
| openstackgerrit | Bharat Kumar Kobagana proposed openstack-infra/project-config: Adding glusterfs job to glusterfs plugin's check pipeline https://review.openstack.org/158392 | 18:52 |
| clarkb | sdague we may have to in order to test live migration | 18:52 |
| *** tonytan4ever has joined #openstack-infra | 18:54 | |
| sdague | clarkb: does 486 work as a cpu model? | 18:54 |
| clarkb | sdague I dont think that has been tried | 18:54 |
| sdague | <feature name='fpu'/> | 18:54 |
| sdague | <feature name='vme'/> | 18:54 |
| sdague | <feature name='pse'/> | 18:54 |
| clarkb | host, pentium, and core2 dobt work | 18:54 |
| sdague | where are the 2 cpu flag sets? | 18:54 |
| openstackgerrit | Merged openstack-infra/project-config: Do not run stable-compat jobs against trunk python-troveclient https://review.openstack.org/157955 | 18:54 |
| fungi | clarkb: does nodepool dib not log to the regular image log? | 18:54 |
| clarkb | rax has heterogenous nodes, hpcloud is homogenous but different than the rax options | 18:55 |
| *** yfried_ has joined #openstack-infra | 18:55 | |
| clarkb | fungi it foes | 18:55 |
| fungi | i feel like every time i pull a loose thread to look into a bug i unravel the entire sweater | 18:55 |
| *** dhritishikhar has joined #openstack-infra | 18:55 | |
| *** SumitNaiksatam has quit IRC | 18:56 | |
| sdague | clarkb: well, actually, rax has heterogenous nodes, but has created a cpu model that works for all of them | 18:56 |
| sdague | sorry, hp cloud | 18:56 |
| *** marun has quit IRC | 18:57 | |
| *** tonytan4ever has quit IRC | 18:58 | |
| sdague | clarkb: pentium doesn't work>? | 18:59 |
| sdague | are you sure? | 18:59 |
| sdague | because all those flags look like they are in there | 18:59 |
| clarkb | sdague according to jogo it failed | 19:00 |
| sdague | do we have a test run to look at? | 19:00 |
| morganfainberg | SpamapS, hi so broadcasting | 19:00 |
| clarkb | jogo has bunches on his change let me find it | 19:00 |
| morganfainberg | SpamapS, and i've been looking at oslo.messaging, i think i have a line in on it now. but i need to stand up a strawman server/client to do this. shouldn't be too bad. | 19:01 |
| morganfainberg | SpamapS, but i started putting some basic structure together for this. | 19:01 |
| clarkb | sdague https://review.openstack.org/#/c/141530/ | 19:01 |
| SpamapS | morganfainberg: cool. I am poking at my idea for efficient streaming of broadcasts here: https://github.com/spamaps/gearhorn | 19:01 |
| * morganfainberg looks | 19:02 | |
| sdague | clarkb: thats cpu_model = Conroe | 19:02 |
| SpamapS | morganfainberg: still just tiny bits of library and tests that kind of imagine how it will work | 19:02 |
| morganfainberg | SpamapS, oh cool | 19:02 |
| clarkb | sdague look at patchsets I think several were tried | 19:02 |
| SpamapS | morganfainberg: but the basic idea is to buffer these things up in workers and then fling them out to subscribers somewhat efficiently via the coalesce functionality. | 19:03 |
| morganfainberg | yep | 19:03 |
| morganfainberg | SpamapS, makes sense to me. | 19:03 |
| fungi | clarkb: so, looking in the dib log, i think maybe 50-restrict-memory is happening too early. it doesn't find either /etc/default/grub or /boot/grub/grub.conf | 19:03 |
| SpamapS | I did get a little side tracked with python3.4 because I'm tired of always waving my hands that "we'll get to that later" ;) | 19:04 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Added a Vagrant module and manifest for easier development. https://review.openstack.org/157465 | 19:04 |
| *** zul has quit IRC | 19:04 | |
| clarkb | fungi that sounds plausible | 19:04 |
| morganfainberg | SpamapS, shouldn't be too hard to | 19:04 |
| *** zul has joined #openstack-infra | 19:04 | |
| clarkb | fungi is it postinstall.d? | 19:04 |
| morganfainberg | SpamapS, get all that in for something useful. py3.4 yeah | 19:04 |
| morganfainberg | SpamapS, i tend to wave my hands on that too :P | 19:04 |
| fungi | clarkb: nope, install.d | 19:04 |
| *** dustins has quit IRC | 19:04 | |
| fungi | clarkb: should be post i guess? | 19:05 |
| *** tonytan4ever has joined #openstack-infra | 19:05 | |
| sdague | clarkb: live migration tests passed on the pentium version | 19:05 |
| sdague | http://logs.openstack.org/30/141530/5/experimental/check-tempest-dsvm-aiopcpu-full/1ae4cfb/console.html.gz#_2015-02-19_01_55_27_450 | 19:05 |
| SpamapS | damnit. Sometimes F7 runs flake8, and sometimes it doesn't. | 19:05 |
| SpamapS | >:| | 19:05 |
| sdague | the fails were network ones | 19:05 |
| clarkb | fungi that would be my guess, maybe spamaps or greghaynes can confirm | 19:05 |
| sdague | on different tests | 19:05 |
| *** bhunter71 has quit IRC | 19:06 | |
| clarkb | sdague ok, I am just rrpeating what jogo has said, is it possible cirros wont boot on pentium? | 19:06 |
| clarkb | does that affect arch. | 19:06 |
| SpamapS | fungi: grub gets installed _really_ late. | 19:06 |
| krotscheck | openstackci/httpd isn't a puppetforge thing, is it. | 19:06 |
| clarkb | SpamapS any reason for that? | 19:06 |
| sdague | clarkb: maybe | 19:06 |
| *** tonytan_brb has joined #openstack-infra | 19:06 | |
| fungi | SpamapS: will sticking something in post-install.d be assured to happen after grub is installed? | 19:07 |
| sdague | but... live migration tests pass if the guest doesn't start? | 19:07 |
| *** dustins has joined #openstack-infra | 19:07 | |
| SpamapS | clarkb: I believe because we don't have our final block device until very late. Might be legacy. | 19:07 |
| fungi | SpamapS: or alternate suggestions for altering the grub configuration? | 19:07 |
| clarkb | oh right the wholr cp everything issue :( | 19:07 |
| SpamapS | fungi: no, finalise.d | 19:07 |
| greghaynes | fungi: looks like it happens in finalise.d | 19:07 |
| fungi | okay | 19:07 |
| greghaynes | 51-finalise.d to be specific | 19:07 |
| SpamapS | but | 19:07 |
| SpamapS | it would make sense to install them earlier, and just try to prevent them from poking at the root dev on install | 19:08 |
| SpamapS | but really, what you need is a .d you can drop things in | 19:08 |
| fungi | so if i do a finalise.d/90-whatever that ought to work? | 19:08 |
| *** bhunter71 has joined #openstack-infra | 19:08 | |
| SpamapS | fungi: 52? ;) | 19:09 |
| greghaynes | Well, is there any reason we have to install grub that late? if its just looking at configs that seems fine, we really just need to run the grub-install or w/e its called nowdays that late | 19:09 |
| *** tonytan4ever has quit IRC | 19:09 | |
| *** ddieterly has quit IRC | 19:10 | |
| clarkb | greghaynes thats my question | 19:10 |
| greghaynes | I think we used to do that, actually | 19:10 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Limit manual jobs https://review.openstack.org/158406 | 19:10 |
| fungi | SpamapS: just in case the dib maintainers decide to bump grub to something higher than 52 ;) | 19:10 |
| *** ddieterly has joined #openstack-infra | 19:10 | |
| SpamapS | It's simpler if we do it late when everything is already on the final block device. | 19:10 |
| *** mrunge has quit IRC | 19:10 | |
| SpamapS | fungi: aaaaand we're back in 8th grade with GW-BASIC ;) | 19:10 |
| greghaynes | yes, IIRC we used to have to install, then force an uninstall/reinstall when we tried it | 19:11 |
| fungi | SpamapS: always increment line numbers by 10! | 19:11 |
| greghaynes | because of some handwavy reason I cant remember | 19:11 |
| *** zul has quit IRC | 19:11 | |
| *** zul has joined #openstack-infra | 19:11 | |
| *** tmcpeak has quit IRC | 19:12 | |
| *** cdent has quit IRC | 19:14 | |
| *** tmcpeak has joined #openstack-infra | 19:15 | |
| *** dhritishikhar has quit IRC | 19:15 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Corrected metadata.json https://review.openstack.org/158408 | 19:16 |
| clarkb | SpamapS not sure its simpler, the only diffedence is one command right? | 19:17 |
| *** Rockyg has joined #openstack-infra | 19:17 | |
| sdague | clarkb / fungi - can you guys look at this - https://review.openstack.org/#/c/157057 so we can change a devstack default and not impact gate log collection | 19:17 |
| SpamapS | clarkb: yes, and I'm not sure either. | 19:18 |
| *** SumitNaiksatam has joined #openstack-infra | 19:19 | |
| fungi | sdague: on 157057 i guess we don't see any benefit to making that configurable at the job level right now? | 19:19 |
| *** peterstac has joined #openstack-infra | 19:19 | |
| *** sputnik13 has joined #openstack-infra | 19:19 | |
| openstackgerrit | Bharat Kumar Kobagana proposed openstack-infra/project-config: Adding glusterfs job to glusterfs plugin's check pipeline https://review.openstack.org/158392 | 19:20 |
| sdague | fungi: right now, I don't think so | 19:20 |
| sdague | we're taking the default atm | 19:20 |
| peterstac | Hi, would anyone here be able to look at https://review.openstack.org/#/c/157955/ ? | 19:20 |
| peterstac | python-troveclient is dead in the water until we get that merged ... | 19:21 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Added new OAuth Parameters https://review.openstack.org/157464 | 19:21 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Updated storyboard configuration for pymysql. https://review.openstack.org/156755 | 19:21 |
| peterstac | oops, nevermind - just checked and it's merged already, thx! | 19:21 |
| *** otter768 has joined #openstack-infra | 19:21 | |
| *** teran has joined #openstack-infra | 19:22 | |
| greghaynes | https://review.openstack.org/#/c/136658/2/elements/redhat-common/pre-install.d/package-installs-redhat-common is what im thinking with how we used to do that | 19:22 |
| greghaynes | although that was redhat specific | 19:22 |
| greghaynes | clarkb: oh, looks like we do explicitly remove grub2 in pre-install.d in the ubuntu element. Youall dont use that though right? | 19:23 |
| *** pelix1 has quit IRC | 19:23 | |
| clarkb | we do | 19:24 |
| *** alexpilotti has quit IRC | 19:24 | |
| greghaynes | which is interesting beacause I believe the redhat-common element installs it in that phase | 19:24 |
| mordred | peterstac: I like to service requests before they are made | 19:24 |
| greghaynes | so at a minimum I think we should be consistent between those two | 19:25 |
| mordred | greghaynes, clarkb: but I want us to move to the ubuntu-minimal element once it's all sorted | 19:25 |
| mordred | just fyi | 19:25 |
| greghaynes | mordred: ah, so ATM youre still not using ubunut-minimal? | 19:25 |
| krotscheck | mordred: What was that channel for the ansible things again? | 19:25 |
| mordred | greghaynes: not yet | 19:25 |
| peterstac | mordred: Great philosophy, I like it! ;) | 19:25 |
| AJaeger_ | We haven't used ERROR_ON_CLONE yet - https://review.openstack.org/#/c/158075/ introduces it. Anybody here that can confirm that this is doing the right thing? | 19:26 |
| *** otter768 has quit IRC | 19:26 | |
| mordred | krotscheck: ##ansible-shade | 19:26 |
| *** tmcpeak has quit IRC | 19:26 | |
| *** teran has quit IRC | 19:27 | |
| *** hdd has quit IRC | 19:28 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Run restrict-memory element later in image build https://review.openstack.org/158413 | 19:28 |
| fungi | clarkb: SpamapS: greghaynes: ^ | 19:28 |
| anteaya | AJaeger_: I see ERROR_ON_CLONE in http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/devstack-vm-gate.sh#n130 | 19:28 |
| *** unicell has joined #openstack-infra | 19:28 | |
| AJaeger_ | anteaya: so do I - since 2012. But we're not using it in project-config at all, so wanted some confirmation about setting it. | 19:29 |
| anteaya | right | 19:29 |
| anteaya | I don't see it used anywhere else, so I don't know what happens if the variable it set to FALSE | 19:30 |
| asselin | Hi, we're seeing pypi errors recently causing our 3rd party ci builds to fail. Is anyone aware of any releases of pip etc that could affect it? | 19:30 |
| *** unicell has quit IRC | 19:30 | |
| *** unicell has joined #openstack-infra | 19:30 | |
| asselin | releases of pip that e.g. would require updates to our .pip/pip.conf and/or .pydistutils.cfg | 19:31 |
| anteaya | AJaeger_: I see more in devstack/ | 19:31 |
| fungi | asselin: what errors are you seeing? | 19:31 |
| AJaeger_ | anteaya: looking at devstack, it should be fine to set it.... | 19:31 |
| *** amitgandhinz has joined #openstack-infra | 19:31 | |
| *** amitgandhinz has quit IRC | 19:31 | |
| asselin | fungi, connection timeouts | 19:31 |
| fungi | asselin: intermittent? | 19:31 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add support for volume attach/detach https://review.openstack.org/153623 | 19:32 |
| *** amitgandhinz has joined #openstack-infra | 19:32 | |
| *** tmcpeak has joined #openstack-infra | 19:32 | |
| sdague | AJaeger: that is definitively the wrong thing to do | 19:32 |
| fungi | asselin: to pypi.python.org or some other host? | 19:32 |
| asselin | fungi, I have it setup to use our mirror and fallback to pypi | 19:32 |
| asselin | fungi, intermiitent, but enough to cause everything to fail: http://15.126.198.151/00/155300/2/check/3par-iscsi-driver-master-client-pip-eos10-dsvm/74edc1f/logs/devstacklog.txt.gz#_2015-02-23_14_23_52_617 | 19:33 |
| *** marun has joined #openstack-infra | 19:33 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/project-config: Create Neutron DVR experimental job for aiopcpu https://review.openstack.org/158415 | 19:33 |
| AJaeger_ | sdague: could you comment on the change and tell vponomaryov how to do it properly, please? | 19:33 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add a method to create image snapshots from nova. https://review.openstack.org/158119 | 19:33 |
| asselin | fungi, I noticed lots of warnings, deprecations, etc recently. I'm wondring if we're actually hitting issues now...my best guess | 19:34 |
| AJaeger_ | sdague: ah, you commented - thanks | 19:34 |
| anteaya | AJaeger_: thanks for asking | 19:34 |
| anteaya | sdague: for my education why is this the wrong approach? | 19:34 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 19:35 | |
| sdague | ERROR_ON_CLONE is a safety measure in devstack that was created for the gate | 19:35 |
| sdague | because zuul has to be the one that sets up all the git trees that you intend to test with | 19:35 |
| AJaeger_ | anteaya, sdague: I found some ways to run less manual jobs - review of https://review.openstack.org/158406 would be welcome. | 19:35 |
| sdague | otherwise you won't be testing what you think it is | 19:35 |
| dstufft | asselin: does your proxy support CONNECT | 19:35 |
| asselin | dstufft, ?? not sure. how do you check? | 19:36 |
| dstufft | try to connect to PyPI using just plain old requests via HTTPS | 19:36 |
| anteaya | sdague: thank you, should I add in a comment in devstack-gate perhaps that states that this variable should never be set to FALSE for our needs? | 19:36 |
| AJaeger_ | sdague: so, the clone in line 454 of https://review.openstack.org/#/c/157880/3/devstack/plugin.sh is wrong? | 19:37 |
| sdague | AJaeger: honestly, I don't know what they did wrong | 19:37 |
| sdague | but I know if they think ERROR_ON_CLONE is the fix, they are incorrect :) | 19:37 |
| mordred | clarkb, jeblair: I updated 156088 to fix your comments - when you get a sec, could I get you to look at it, 156247 and 153623 ? They're the last three I need before I can get the PR merged upstream | 19:37 |
| sdague | because it's never the fix | 19:37 |
| *** mestery_ has joined #openstack-infra | 19:38 | |
| fungi | AJaeger_: git_clone is a function which just uses the local cache if there ERROR_ON_CLONE is set | 19:38 |
| sdague | it just means you are bypassing the testing system, and no longer testing anything meaningful | 19:38 |
| *** mestery has quit IRC | 19:38 | |
| openstackgerrit | Merged openstack-infra/project-config: Added Fuel Mellanox plugin https://review.openstack.org/157704 | 19:38 |
| *** andreykurilin_ has quit IRC | 19:38 | |
| dstufft | asselin: also there was a bug awhile back where requests wouldn't forget the proxy after ~50 some requests or so | 19:38 |
| sdague | fungi: actually, it's not even that | 19:38 |
| dstufft | so if you're not on pip 6.0.8 you might want to upgrade, I forget what version fixed it | 19:38 |
| AJaeger_ | vponomaryov: please read discussion above for your manila change | 19:39 |
| fungi | AJaeger_: or, rather, which just uses the local cache, so that you won't trip the ERROR_ON_CLONE safeties | 19:39 |
| sdague | if the directory is there, it never gets far enough to touch ERROR_ON_CLONE | 19:39 |
| sdague | and that's the point, zuul has to be the one that sets up all the git trees, it's the only thing that has the right global view | 19:39 |
| fungi | yeah, i incorrectly oversimplified | 19:39 |
| asselin | dstufft, I'm on 6.0.8 http://15.126.198.151/00/155300/2/check/3par-iscsi-driver-master-client-pip-eos10-dsvm/74edc1f/logs/devstacklog.txt.gz#_2015-02-23_14_23_40_423 | 19:39 |
| fungi | but anyway point being that git_clone is not necessarily running "git clone" | 19:39 |
| fungi | in our case it pretty much never is | 19:40 |
| *** salv-orlando has quit IRC | 19:40 | |
| AJaeger_ | sdague, fungi thanks for the explanations. I don't understand everything yet but this helps! | 19:40 |
| dstufft | asselin: OK, well I can't stay here, I jsut had to check something real quick. I have a migraine so I'm going back to lay down. An important thing to figure out is if you can access PyPI with requests. That'll tell you if it's a pip problem or a requests problem (pip uses requests to do HTTP, pip 6.0.8 uses requests 2.5.1). | 19:41 |
| fungi | dhellmann is releasing all the things | 19:42 |
| openstackgerrit | Paul Michali proposed openstack-infra/project-config: Setup non-voting VPNaaS func test for StrongSwan https://review.openstack.org/158389 | 19:42 |
| * fungi braces for impact | 19:42 | |
| *** dhritishikhar has joined #openstack-infra | 19:42 | |
| asselin | dstufft, ok thanks. hope you feel better. | 19:42 |
| dhellmann | fungi: I'm done for today. | 19:42 |
| fungi | dhellmann: that was a nice large batch | 19:43 |
| fungi | early and often! | 19:43 |
| *** marun has quit IRC | 19:43 | |
| dhellmann | fungi: most of the actual changes are small. The messaging and db libs will come tomorrow or wed, and those have some bigger changes so I'll space them out more. | 19:43 |
| openstackgerrit | Anita Kuno proposed openstack-infra/devstack-gate: Add a comment for ERROR_ON_CLONE https://review.openstack.org/158417 | 19:44 |
| fungi | cool! i'm excited | 19:44 |
| *** marun has joined #openstack-infra | 19:44 | |
| *** Sukhdev has quit IRC | 19:45 | |
| anteaya | sdague fungi can you look at ^^ to see if this is helpful and relevant? | 19:46 |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/project-config: Create Neutron DVR experimental job for aiopcpu https://review.openstack.org/158415 | 19:46 |
| anteaya | fungi: thanks I'll respin | 19:47 |
| *** zul has quit IRC | 19:48 | |
| fungi | sdague: wow, yeah... http://logs.openstack.org/54/158054/4/check/gate-manila-tempest-dsvm-neutron/ce20e71/logs/devstacklog.txt.gz#_2015-02-21_19_51_27_425 | 19:48 |
| fungi | sdague: their devstack plugin is trying to explicitly clone from github | 19:48 |
| mordred | fungi, sdague we may want to figure out how to hard-fail things that try to do that | 19:49 |
| openstackgerrit | Anita Kuno proposed openstack-infra/devstack-gate: Add a comment for ERROR_ON_CLONE https://review.openstack.org/158417 | 19:49 |
| mordred | in a systemic way | 19:49 |
| fungi | mordred: well, we did | 19:49 |
| *** amrith is now known as _amrith_ | 19:49 | |
| mordred | fungi: oh yeah? what did we do? | 19:49 |
| mordred | fungi: what if we put a fake hostname for github.com in /etc/hosts ... | 19:50 |
| sdague | mordred: well, we do | 19:50 |
| fungi | mordred: die 622 'Cloning not allowed in this configuration' | 19:50 |
| mordred | neat | 19:50 |
| sdague | mordred: no, we explode super hard | 19:50 |
| mordred | sweet | 19:50 |
| fungi | point of the story was, that's why they want to work around it by turning off that safety net | 19:50 |
| mordred | oh - they do? | 19:51 |
| mordred | ?HAHAHAHA | 19:51 |
| mordred | no | 19:51 |
| sdague | right, they saw an error, turned off this flag, and thought that was the fix | 19:51 |
| sdague | which I -2ed and said "this is never the fix" | 19:51 |
| *** e0ne has joined #openstack-infra | 19:51 | |
| *** openstackgerrit has quit IRC | 19:51 | |
| *** dhritishikhar has quit IRC | 19:52 | |
| *** openstackgerrit has joined #openstack-infra | 19:52 | |
| sdague | anteaya: I proposed alternate text | 19:52 |
| anteaya | sdague: awesome thanks | 19:52 |
| * anteaya prepares to copy/paste whatever sdague wrote | 19:52 | |
| * anteaya respings | 19:53 | |
| anteaya | that too | 19:53 |
| *** ivar-lazzaro has quit IRC | 19:54 | |
| *** ivar-lazzaro has joined #openstack-infra | 19:54 | |
| sdague | gerritbot no longer posting in here? | 19:55 |
| openstackgerrit | Anita Kuno proposed openstack-infra/devstack-gate: Add a comment for ERROR_ON_CLONE https://review.openstack.org/158417 | 19:55 |
| sdague | oh, there it is | 19:55 |
| sdague | just slow | 19:55 |
| anteaya | I was slow | 19:55 |
| *** bdpayne has joined #openstack-infra | 19:56 | |
| sdague | +2 | 19:56 |
| anteaya | I posted I was respinning and then edited the patch | 19:56 |
| anteaya | thanks | 19:56 |
| anteaya | mostly this is for me and AJaeger_ for when we hit this again in reviews | 19:56 |
| *** yjiang5 is now known as yjiang5_away | 19:56 | |
| AJaeger_ | anteaya: thanks | 19:56 |
| anteaya | AJaeger_: thanks for asking about it | 19:57 |
| sdague | well, it might also be useful because people start reading d-g code when their job fails, and do things like turning it off (like we saw) | 19:57 |
| *** achanda has quit IRC | 19:57 | |
| sdague | hopefully this will slow a little of that down as well | 19:57 |
| anteaya | maybe a little | 19:57 |
| anteaya | hopefully | 19:57 |
| AJaeger_ | worth trying ;) | 19:58 |
| openstackgerrit | Merged openstack/requirements: Add toposort test requirement https://review.openstack.org/157014 | 19:58 |
| *** achanda has joined #openstack-infra | 19:59 | |
| *** tmcpeak has quit IRC | 20:01 | |
| *** zul has joined #openstack-infra | 20:01 | |
| *** tmcpeak has joined #openstack-infra | 20:01 | |
| *** tkelsey has joined #openstack-infra | 20:05 | |
| *** IAX has joined #openstack-infra | 20:06 | |
| *** rm_work|away is now known as rm_work | 20:07 | |
| *** stevemar has quit IRC | 20:07 | |
| *** IAX has left #openstack-infra | 20:07 | |
| *** markvoelker has joined #openstack-infra | 20:07 | |
| *** mestery_ has quit IRC | 20:09 | |
| *** Rockyg has quit IRC | 20:09 | |
| *** BharatK has quit IRC | 20:11 | |
| *** mestery has joined #openstack-infra | 20:11 | |
| openstackgerrit | Merged openstack-infra/project-config: Add stackforge/networking-ovn project https://review.openstack.org/158137 | 20:12 |
| anteaya | russellb: whom do you want as the first person added to the networking-ovn gerrit groups? yourself? | 20:13 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Added a Vagrant module and manifest for easier development. https://review.openstack.org/157465 | 20:14 |
| *** tmcpeak has quit IRC | 20:15 | |
| *** amitgandhinz has quit IRC | 20:15 | |
| openstackgerrit | Merged openstack-infra/project-config: Adding bgpvpn project to stackforge https://review.openstack.org/157847 | 20:15 |
| *** marcusvrn has quit IRC | 20:15 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Corrected metadata.json https://review.openstack.org/158408 | 20:16 |
| *** thedodd has quit IRC | 20:17 | |
| *** kgiusti has quit IRC | 20:17 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 20:17 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Added new OAuth Parameters https://review.openstack.org/157464 | 20:18 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Updated storyboard configuration for pymysql. https://review.openstack.org/156755 | 20:18 |
| *** teran has joined #openstack-infra | 20:19 | |
| *** andreykurilin_ has joined #openstack-infra | 20:19 | |
| *** ivar-lazzaro has quit IRC | 20:20 | |
| *** ivar-lazzaro has joined #openstack-infra | 20:21 | |
| *** baoli has quit IRC | 20:22 | |
| *** kgiusti has joined #openstack-infra | 20:22 | |
| *** ivar-lazzaro has quit IRC | 20:22 | |
| *** ivar-lazzaro has joined #openstack-infra | 20:23 | |
| russellb | anteaya: me, yes. | 20:23 |
| russellb | anteaya: and mestery was the 2nd person i was going to seed it with | 20:23 |
| *** vigneshvar has quit IRC | 20:23 | |
| mestery | russellb anteaya: ++ | 20:24 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Added Valid OAuth client configuration to storyboard manifest. https://review.openstack.org/158434 | 20:27 |
| *** unicell has quit IRC | 20:29 | |
| *** unicell has joined #openstack-infra | 20:29 | |
| *** stevemar has joined #openstack-infra | 20:30 | |
| *** unicell1 has joined #openstack-infra | 20:30 | |
| *** unicell has quit IRC | 20:30 | |
| *** erlon has joined #openstack-infra | 20:30 | |
| *** unicell has joined #openstack-infra | 20:32 | |
| *** unicell1 has quit IRC | 20:32 | |
| *** sdake__ has quit IRC | 20:32 | |
| *** ddieterly has quit IRC | 20:32 | |
| *** ddieterly has joined #openstack-infra | 20:33 | |
| zaro | AJaeger_: take another look? https://review.openstack.org/#/c/155208/ | 20:33 |
| *** harlowja has quit IRC | 20:34 | |
| *** harlowja_ has joined #openstack-infra | 20:34 | |
| *** ZZelle_ has joined #openstack-infra | 20:34 | |
| *** mriedem has quit IRC | 20:35 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Moved working_directory configuration element into correct block. https://review.openstack.org/158437 | 20:36 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Added new OAuth Parameters https://review.openstack.org/157464 | 20:37 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Updated storyboard configuration for pymysql. https://review.openstack.org/156755 | 20:37 |
| *** ddieterly has quit IRC | 20:37 | |
| *** ddieterly has joined #openstack-infra | 20:38 | |
| *** zul has quit IRC | 20:40 | |
| clarkb | on the new project creation front, remember puppet is still broken with that | 20:40 |
| *** zul has joined #openstack-infra | 20:40 | |
| clarkb | sorry I haven been mostly afk today silly doctors | 20:41 |
| *** salv-orlando has joined #openstack-infra | 20:41 | |
| *** baoli has joined #openstack-infra | 20:44 | |
| *** achanda has quit IRC | 20:45 | |
| *** tnovacik has quit IRC | 20:46 | |
| *** ddieterly has quit IRC | 20:46 | |
| *** marun has quit IRC | 20:49 | |
| *** markvoelker has quit IRC | 20:51 | |
| ianw | afazekas: here, if you are | 20:52 |
| *** marun has joined #openstack-infra | 20:52 | |
| *** yjiang5_away has quit IRC | 20:53 | |
| *** yfried_ has quit IRC | 20:57 | |
| *** hichihara has joined #openstack-infra | 20:59 | |
| *** tnovacik has joined #openstack-infra | 20:59 | |
| *** ghostpl_ has joined #openstack-infra | 20:59 | |
| *** tkelsey has quit IRC | 21:00 | |
| *** sandywalsh_ has joined #openstack-infra | 21:02 | |
| mestery | Howdy infra folks! I'm debugging an issue running OpenDaylight in the gate. I'm getting ECONNREFUSED when neutron tries to talk to ODL on port 8088, this works fine locally, wondering if there is an issue with using htis port in the gate? | 21:03 |
| mestery | For reference, see the logs here: http://logs.openstack.org/76/158076/15/check/check-tempest-dsvm-networking-odl/78313f9/logs/screen-q-svc.txt.gz | 21:03 |
| *** nicodemos has quit IRC | 21:03 | |
| *** ghostpl_ has quit IRC | 21:04 | |
| mestery | The ODL tomcat server uses 8080 be default, which swift also uses, so thus the reason for using 8088 instead. | 21:04 |
| mestery | I had to pick a new port | 21:04 |
| *** sandywalsh has quit IRC | 21:05 | |
| *** bknudson has quit IRC | 21:05 | |
| *** garyh has quit IRC | 21:06 | |
| *** garyh has joined #openstack-infra | 21:06 | |
| mordred | jogo, sdague: question on the NUMA thread ... | 21:07 |
| sdague | numa thread? | 21:07 |
| mordred | sdague: on the mailing list | 21:08 |
| mordred | sdague: [openstack-dev] [nova] Ubuntu, qemu, NUMA support | 21:08 |
| ianw | mestery: is tomcat started and running? it's not clear from the devstack log | 21:08 |
| mestery | ianw: Yes, see here: http://logs.openstack.org/76/158076/15/check/check-tempest-dsvm-networking-odl/78313f9/logs/screen-odl-server.txt.gz | 21:08 |
| mestery | ianw: That's the OpenDaylight logs | 21:08 |
| *** aysyd has quit IRC | 21:08 | |
| ianw | mestery: Listening on port 12001 bound on address 0.0.0.0/0.0.0.0 <- is that something else? | 21:09 |
| mestery | Admitedly, there's not much there ... but it's up and running. I even have a (!!!) 5 minute delay built in to let tomcat "settle" before moving on. Shudder .... | 21:09 |
| russellb | clarkb: ah, i'm getting an error pushing commits to a new empty project ... perhaps a known problem? ! [remote rejected] HEAD -> refs/publish/master (no common ancestry) | 21:09 |
| mestery | ianw: Yes | 21:09 |
| mordred | sdague: does that imply that if I wanted to run a kvm openstack on trusty, that because of qemu compile options I would not be able to? | 21:09 |
| mordred | without a backport package | 21:09 |
| mordred | sdague: this is really just me trying to learn how some things hang together | 21:09 |
| clarkb | russellb thats the broken project creation I think | 21:10 |
| russellb | clarkb: OK, thanks | 21:10 |
| *** mestery has quit IRC | 21:10 | |
| sdague | mordred: the answer should be no | 21:10 |
| sdague | let me go refresh on the thread | 21:11 |
| *** mestery has joined #openstack-infra | 21:11 | |
| russellb | clarkb: worth filing a bug for? | 21:11 |
| mestery | ianw: My VPN bounced back now | 21:11 |
| mestery | ianw: You can see I put in some debug to look at the tomcat config file here: http://logs.openstack.org/76/158076/15/check/check-tempest-dsvm-networking-odl/78313f9/logs/devstacklog.txt.gz#_2015-02-23_20_47_39_996 | 21:11 |
| mordred | russellb: I'm working on it - it just opened up a rabbit hole | 21:11 |
| russellb | mordred: ah ok, thanks! i'll sit tight then | 21:11 |
| mordred | russellb: and I think I took the blue pill by accident | 21:11 |
| russellb | :) | 21:11 |
| russellb | it happens | 21:12 |
| mordred | russellb: I mean, I'm not working on your particular project this instant - which project is it? | 21:12 |
| *** kgiusti has quit IRC | 21:12 | |
| *** ayoung has quit IRC | 21:12 | |
| russellb | mordred: stackforge/networking-ovn ... not urgent by any stretch | 21:12 |
| *** salv-orlando has quit IRC | 21:12 | |
| mordred | kk. thanks | 21:12 |
| russellb | from https://review.openstack.org/#/c/158137/ | 21:12 |
| sdague | mordred: so basically, if you want nodes to do numa'y things, you have to give it some extra info | 21:13 |
| mordred | sdague: and the mechanism to do that is via qemu? | 21:13 |
| *** achanda has joined #openstack-infra | 21:13 | |
| sdague | the way this *should* work is that if the nodes can't do that, it's explicit about it, or fails. Right now the numa code paths are largely untested, so it explodes a lot | 21:13 |
| *** marun has quit IRC | 21:13 | |
| sdague | mordred: flavor and image attributes | 21:14 |
| *** dustins has quit IRC | 21:14 | |
| *** rlandy has quit IRC | 21:14 | |
| mordred | sdague: awesome | 21:14 |
| mordred | sdague: I love exploding things | 21:14 |
| *** ddieterly has joined #openstack-infra | 21:14 | |
| ianw | mestery: nothing obvious ... i'd start the job again and ask carkb or jeblair to put the host on hold for you, and log in to see what's going on | 21:15 |
| *** markvoelker has joined #openstack-infra | 21:15 | |
| *** sdake_ has joined #openstack-infra | 21:15 | |
| fungi | russellb: you said to add you as the initial member of networking-ovn-core and networking-ovn-release? i'll take care of that now | 21:15 |
| russellb | fungi: yes, and mestery while you're at it if you don't mind | 21:15 |
| russellb | fungi: thank you! | 21:15 |
| mestery | ianw: Good call, I'll do that and see. | 21:15 |
| mestery | russellb fungi: Thanks! | 21:15 |
| *** markvoelker has quit IRC | 21:15 | |
| fungi | russellb: mestery: done and done | 21:16 |
| *** markvoelker has joined #openstack-infra | 21:16 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: adds support for not condition in conditional_step https://review.openstack.org/155423 | 21:16 |
| mestery | fungi: thank you sir! | 21:16 |
| russellb | yay | 21:16 |
| openstackgerrit | Douglas Mendizábal proposed openstack-infra/project-config: Add a new devstack-gate job to Barbican's tests https://review.openstack.org/157607 | 21:16 |
| *** dustins has joined #openstack-infra | 21:16 | |
| *** salv-orlando has joined #openstack-infra | 21:17 | |
| *** ldnunes has quit IRC | 21:17 | |
| *** sdake__ has joined #openstack-infra | 21:17 | |
| mestery | clarkb fungi: I'm trying to debug an ODL thing in the gate, any chacne you can keep the -dvsm node from this review around for me for a bit: https://review.openstack.org/#/c/158076/ | 21:17 |
| mestery | fungi clarkb: I'm headed to a Meetup in 30 minutes or so, but will be online when I land there and can look at this during the Meetup | 21:18 |
| fungi | russellb: i don't see any indication that you should be getting an error. did you base your commit on the initial state of that repo? | 21:18 |
| sdague | mtreinish was working on a tempest test that called the numa code kind of on a null path to try to reduce the random explody, but at least at the nova midcycle that wasn't landable | 21:18 |
| russellb | fungi: oh, no ... it's a repo i initialized locally. d'oh. | 21:18 |
| fungi | russellb: that would entirely explain the error message | 21:18 |
| mtreinish | sdague: well I was helping w/ reviews and stuff, but I didn't write it | 21:18 |
| russellb | fungi: quite ... | 21:18 |
| sdague | mtreinish: is that passing yet? | 21:19 |
| russellb | fungi: i'm very jet lagged, that's my excuse | 21:19 |
| mtreinish | sdague: last I looked they finally got it to a point where it wouldn't fail randomly | 21:19 |
| fungi | mestery: local firewall rules blocking access on that port on the worker maybe? | 21:19 |
| sdague | ok, that would be good to land then | 21:19 |
| sdague | to prevent regressions | 21:19 |
| mtreinish | sdague: but we can't land it because it's still a 100% tied to some env config | 21:19 |
| sdague | you got the link?' | 21:19 |
| mestery | fungi: That was my thinking too! Where to look for that? Any pointers? | 21:19 |
| sdague | oh, ok | 21:19 |
| mtreinish | like it broke xenapi ci | 21:19 |
| * fungi goes looking for where devstack iptables gets tweaked | 21:19 | |
| mtreinish | sdague: https://review.openstack.org/#/c/143540/ | 21:19 |
| * mestery wanders behind fungi with a flashlight | 21:19 | |
| russellb | fungi: mordred yeah, project works fine, nevermind. thanks | 21:20 |
| sdague | mtreinish: hmmm.... so nova won't reject a numa flavor create if no driver in the cloud supports it? | 21:21 |
| *** sdake_ has quit IRC | 21:21 | |
| sdague | that seems like the safety lever we need there actually | 21:21 |
| fungi | mestery: i do see at least lib/ironic in openstack-dev/devstack adding rules to iptables to permit additional traffic it needs for its features | 21:21 |
| mtreinish | sdague: yeah that's what it looks like | 21:21 |
| mordred | russellb: yay! | 21:21 |
| mestery | fungi: I'll give that a shot, thanks for the pointer! | 21:21 |
| fungi | perhaps other external devstack plugins do somilar things | 21:22 |
| fungi | similar too | 21:22 |
| *** otter768 has joined #openstack-infra | 21:22 | |
| *** radez is now known as radez_g0n3 | 21:23 | |
| *** ChuckC has joined #openstack-infra | 21:25 | |
| *** vilobhmm2 has joined #openstack-infra | 21:25 | |
| *** vilobhmm has quit IRC | 21:25 | |
| *** vilobhmm1 has quit IRC | 21:25 | |
| *** _amrith_ is now known as amrith | 21:26 | |
| *** otter768 has quit IRC | 21:27 | |
| *** baoli has quit IRC | 21:27 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Explicitly turn on database query logging https://review.openstack.org/157057 | 21:29 |
| *** hdd has joined #openstack-infra | 21:30 | |
| jeblair | clarkb, fungi: can you think of a way to indicate what kind of CPU resources we would need from a potential new nodepool provider cloud? | 21:30 |
| jeblair | clarkb, fungi: i tell people "8 vcpus" as a ballpark, but i mean, that's highly variable depending on what kind of cpu we're talking about, and what kind of oversubscription | 21:31 |
| *** yjiang5 has joined #openstack-infra | 21:31 | |
| fungi | oversubscription aside, aggregate bogomips? | 21:33 |
| fungi | (we want the machine to be able to do absolutely nothing this many times a second) | 21:33 |
| mordred | jeblair: I was just talking to sdague - and apparently we're over 50% idle for large swaths of time | 21:35 |
| sdague | yeh, we have dstat numbers on every job | 21:35 |
| SpamapS | get to the idle times faster? ;) | 21:35 |
| sdague | so you can data mine it | 21:35 |
| jeblair | fungi: they are surprisingly consistent. 5186 on hpcloud and 5200 on rax. | 21:35 |
| *** tmcpeak has joined #openstack-infra | 21:35 | |
| sdague | SpamapS: yeh, but if we are going for agregate through put, I'm not convinced | 21:36 |
| openstackgerrit | Khai Do proposed openstack-infra/jenkins-job-builder: The email publisher needs a list of recipients https://review.openstack.org/153432 | 21:36 |
| sdague | if it's node pool nodes, oversubscribed, but not cpu rate limitted actually works nicely | 21:36 |
| sdague | that's what hp cloud old was | 21:36 |
| fungi | jeblair: is that taking the vcpu count into consideration? | 21:36 |
| jeblair | fungi: i don't know, that's the raw value printed in dmesg | 21:36 |
| fungi | ahh | 21:37 |
| fungi | looks like that's per vcpu | 21:37 |
| *** unicell has quit IRC | 21:39 | |
| *** unicell1 has joined #openstack-infra | 21:39 | |
| sdague | jeblair: maybe we should run a linpack run on our nodepool nodes | 21:40 |
| sdague | and say "at least this fast on linpack" | 21:40 |
| *** dizquierdo has joined #openstack-infra | 21:40 | |
| mordred | oh - actually | 21:40 |
| clarkb | bogomips! | 21:40 |
| mordred | you know what? | 21:40 |
| mordred | there's that new google cloud profiler thing | 21:40 |
| sdague | or something equiv | 21:40 |
| mordred | https://github.com/GoogleCloudPlatform/PerfKitBenchmarker | 21:40 |
| nibalizer | ya that | 21:40 |
| nibalizer | i thought of running it on hpcloud | 21:41 |
| nibalizer | but i didn't want to cry | 21:41 |
| nibalizer | also I think we'd have to kick it a bit to work on openstack | 21:41 |
| fungi | oh, neat | 21:41 |
| sdague | yeh, wasn't that a suggested summer of code project by ttx | 21:41 |
| mordred | yah - well, porting it to shade and then figuring out how to regularly run it on our clouds or something might be informative | 21:41 |
| *** AJaeger_ has quit IRC | 21:42 | |
| jeblair | also, i guess, "runs devstack-gate in < 1hr" is a benchmark too, just not super easy to run right now :) | 21:42 |
| clarkb | jeblair the neutron unittests are much easier | 21:43 |
| *** mrmartin has quit IRC | 21:43 | |
| jeblair | clarkb: good point | 21:43 |
| pleia2 | ok, I think I managed finally to dig myself out of my email backlog, running off to dr appt now, should finally be able to get back to reviews and all things infra after that | 21:43 |
| jeblair | clarkb: the're around 1 hour now? | 21:43 |
| clarkb | pleia2 thats what I keep saying | 21:43 |
| pleia2 | clarkb: hah :) | 21:43 |
| *** tiswanso_ has quit IRC | 21:43 | |
| *** dustins has quit IRC | 21:43 | |
| clarkb | jeblair I think 40minutes | 21:44 |
| clarkb | pleia2 in second doctors office of the day right now | 21:44 |
| pleia2 | clarkb: oh, fun | 21:44 |
| jeblair | clarkb: huh, i found one that's 10 minutes. | 21:44 |
| jeblair | 2015-02-23 20:18:26.670 | Ran: 10967 tests in 510.0000 sec. | 21:44 |
| *** sdake_ has joined #openstack-infra | 21:45 | |
| sdague | yeh, with the drivers out of tree they got shorter | 21:45 |
| *** btran has joined #openstack-infra | 21:45 | |
| clarkb | jeblair 4478 seconds according to random log I looked at | 21:45 |
| *** dustins has joined #openstack-infra | 21:46 | |
| clarkb | http://logs.openstack.org/23/156823/4/check/gate-neutron-python27/6b5224f/console.html | 21:46 |
| *** achanda has quit IRC | 21:46 | |
| jhesketh | Morning | 21:46 |
| asselin | hi, anyone know where ubuntu_mysql.qcow2 is supposed to be cached in 3rd party ci? http://paste.openstack.org/show/180797/ | 21:46 |
| clarkb | asselin into the devstack images dir | 21:47 |
| jeblair | clarkb: yeah, that's the linearized sum, but still shorter in parallel. | 21:47 |
| *** achanda has joined #openstack-infra | 21:47 | |
| clarkb | wait gah | 21:47 |
| clarkb | so many different subunit formatters now they make no sense to me | 21:47 |
| clarkb | ya that one was only 612 secinds | 21:48 |
| clarkb | maybe nova is the long unittest project now | 21:48 |
| jeblair | clarkb: i looked at one which was 12min | 21:48 |
| *** dustins has quit IRC | 21:48 | |
| mtreinish | clarkb: arew you complaining about: https://review.openstack.org/#/c/129418/ | 21:48 |
| *** sdake__ has quit IRC | 21:49 | |
| clarkb | mtreinish I just find it all too noisy | 21:49 |
| asselin | clarkb, I see them here: /opt/stack/new/devstack/files, but which nodepool image script is supposed to save it? maybe I need to search some more. | 21:49 |
| *** MaxV has joined #openstack-infra | 21:50 | |
| clarkb | asselin the devstack nodepool scripts do it | 21:50 |
| mtreinish | clarkb: sure, it's a delicate balance we want some debug info but not too much | 21:50 |
| clarkb | mtreinish also hard to read those logs on my phone :) | 21:50 |
| mtreinish | clarkb: if you've got a better suggestion for the output push it up, nothing there is really set in stone | 21:50 |
| mtreinish | clarkb: that's why I got a nexus 6... | 21:51 |
| clarkb | mtreinish I prefer default testr | 21:51 |
| mtreinish | bigger screen :) | 21:51 |
| *** mestery has quit IRC | 21:51 | |
| clarkb | show me failures, total tests run and run time | 21:51 |
| clarkb | simple and only the info I need | 21:51 |
| *** abhirc has joined #openstack-infra | 21:52 | |
| sdague | clarkb: I hate default testr, any program that runs for > 20 seconds with no status progress is problematic. Especially when you are running code that can deadlock. | 21:52 |
| *** bswartz has quit IRC | 21:53 | |
| clarkb | sdague the cases wherr that happens are infrequent enough to not default to always having necessary output to debug deadlocks | 21:53 |
| fungi | carefully going back over the packages we preinstall on thick slaves right now, are we unable to pip install any of python-libvirt, python-lxml and python-zmq? thinking we should see about trying to leave as many of those as possible up to per-project virtualenvs | 21:53 |
| clarkb | can just turn it on when needed | 21:53 |
| sdague | or all the times where it incorrectly skipped things and passed | 21:54 |
| clarkb | sdague that info is written | 21:54 |
| sdague | and because it wasn't verbose, it took us weeks to realize that we missed tests | 21:54 |
| mordred | fungi: I know we can pip lxml | 21:54 |
| mtreinish | clarkb: from what I've seen you're actually the minority, most people like to see that it's doing something as it runs | 21:54 |
| clarkb | mtreinish yes I am | 21:54 |
| mordred | fungi: I _think_ libvirt is pip installin gnow? | 21:54 |
| clarkb | I know this I just dont grok it | 21:54 |
| clarkb | nordred no they stopped | 21:55 |
| clarkb | mordred ^ | 21:55 |
| fungi | srsly? | 21:55 |
| mordred | clarkb: really? they gave up on that? | 21:55 |
| clarkb | so we dont need libvirt-python at all | 21:55 |
| clarkb | yup using fakes now | 21:55 |
| mordred | OH | 21:55 |
| mordred | that's even better | 21:55 |
| fungi | nice | 21:55 |
| clarkb | sure it just wouldve been nice to do years ago :) | 21:55 |
| jeblair | mtreinish, clarkb: am also in that minority. i would be okay with some output to see deadlocks when run interactively, but i don't actually find 10,000 lines of "everything is okay" to be helpful in that department. | 21:56 |
| fungi | well, i'm 99% of the way to having a complete cross-platform bindep manifest now which will turn "devstack" workers into "bare" workers package-wise | 21:56 |
| jeblair | mtreinish, clarkb: (i also don't find no output to be helpful) | 21:56 |
| clarkb | python-zmq is pip installable too but it grabs C zmq from an external source which may not be good | 21:56 |
| fungi | oh, that a'splains why we don't use it then | 21:57 |
| *** rkukura has quit IRC | 21:57 | |
| clarkb | it will link against local C if it finds it | 21:57 |
| clarkb | libzmq-dev or somesuvh | 21:57 |
| mtreinish | jeblair: yeah, it comes back to that delicate balance. If you guys have a better suggestion feel free to push up a patch :) | 21:57 |
| fungi | ahh, we should probably add that to the bindep manifest when we get around to making improvements to the packageset then | 21:58 |
| clarkb | +1 | 21:59 |
| fungi | first i'm trying to replicate what we have as closely as possible, so as to avoid unpleasant surprises | 21:59 |
| *** rkukura_ has joined #openstack-infra | 22:01 | |
| sdague | clarkb: well, as someone that spends a ton of my time running tests locally, the progress part is really important, especially when I'm making cross cutting changes, because lots of things can make these tests go until the 300s timeout. And waiting 5 minutes for first fails to show up because something went terrible is hugely non productive. | 22:01 |
| *** ddieterly has quit IRC | 22:01 | |
| fungi | one thing i've noticed this is going to require a change in though... we'll need to start running the gerrit-git-prep builder before the revoke-sudo builder if we want projects to be able to specify the system packages to add for a particular job | 22:01 |
| jeblair | fungi: yeah, i think that should be safe | 22:02 |
| clarkb | sdague would a simple: no tests reported data in 30 seconds (last file tests were xyz) message fix that problem? | 22:02 |
| *** ddieterly has joined #openstack-infra | 22:03 | |
| sdague | not really, you're still putting an extra 27 second delay in my iteration loop in that case | 22:03 |
| openstackgerrit | Merged openstack-infra/gear: Fix SSL non-blocking IO support https://review.openstack.org/143716 | 22:03 |
| clarkb | 27 seconds? | 22:04 |
| sdague | most of these nova tests should complete in 3 seconds | 22:04 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add initial compute functional tests to Shade https://review.openstack.org/158463 | 22:04 |
| clarkb | then set the nova timeout to 4 seconds | 22:05 |
| openstackgerrit | Merged openstack-infra/gear: Revert "Pin python-daemon to <2.0" https://review.openstack.org/150257 | 22:05 |
| jeblair | sdague: do you run tests serially locally? | 22:05 |
| sdague | that first line of good output means I can get up and get a cup of coffee | 22:05 |
| clarkb | override for longer tests | 22:05 |
| sdague | jeblair: nope | 22:05 |
| *** baoli has joined #openstack-infra | 22:05 | |
| jeblair | sdague: how do you pick out a stuck test then? | 22:05 |
| *** tkelsey has joined #openstack-infra | 22:06 | |
| rcarrillocruz | mordred: https://review.openstack.org/#/c/158463/1 , if you like the approach i can add more tests in bulk... | 22:06 |
| rcarrillocruz | jeblair: ^ | 22:06 |
| sdague | we're displaying worker id, so you can figure out if a worker is stuck | 22:06 |
| jeblair | sdague: (i'm genuinely curious; for zuul, i run the suite in parallel, get "coffee", then if anything failed or timed out, run that one test in the foreground) | 22:06 |
| clarkb | sdague (timeout is overrideable on the cli too) | 22:06 |
| jeblair | sdague: so you watch the stream and when you don't see a worker id for a while, you figure it's stuck? | 22:06 |
| *** Ryan_Lane has quit IRC | 22:06 | |
| *** tonytan_brb is now known as tonytan4ever | 22:07 | |
| sdague | jeblair: yeh, or more importantly, know that all the workers seem to be off and running | 22:07 |
| jeblair | sdague: oh, you're only concerned that they have run one test successfully to completion? what does that tell you? can't any test hang? | 22:07 |
| fungi | i start tests and then do http://xkcd.com/303/ | 22:08 |
| sdague | clarkb: the argument we are having is that you are actually telling me that I develop software incorrectly, and that I should not develop software the way that works for me. | 22:08 |
| jeblair | sdague: i guess it tells you that the system isn't so broken that all tests hang? | 22:08 |
| mordred | rcarrillocruz: LOVE IT | 22:08 |
| sdague | jeblair: yeh, and honestly, it sometimes is | 22:08 |
| clarkb | no I am only suggesting that your way maybe isnt the best default for everyone | 22:08 |
| rcarrillocruz | sweet | 22:08 |
| rcarrillocruz | i'll build on that then | 22:09 |
| mordred | rcarrillocruz: although I'd put the cloud = openstack_cloud() in the setUp and make it self.cloud = - just so that you don't have to do it every time | 22:09 |
| mordred | but that might just be me | 22:09 |
| sdague | clarkb: sure, I accept that | 22:09 |
| rcarrillocruz | yeah | 22:09 |
| rcarrillocruz | good call | 22:09 |
| jeblair | sdague: (to be clear, i'm not participating in an argument, i am genuinely trying to understand your workflow which requires me asking a lot of simple sounding questions) | 22:09 |
| *** rkukura_ is now known as rkukura | 22:11 | |
| sdague | jeblair: personally without the progress displays my mental iteration loop just doesn't work. And if I can't fast iterate on this stuff my throughput goes way down. | 22:11 |
| sdague | also, after we added the subunit-trace function, which always displays the subunit stderr, we found a bunch of tests that were doing terrible things, stack tracing components and still passing | 22:12 |
| jeblair | yowza | 22:12 |
| jeblair | SpamapS: see comments on 159146 | 22:12 |
| jeblair | SpamapS: correction: 158146 | 22:13 |
| sdague | which is why I feel that possitive output is a really important part of creating good tests | 22:13 |
| *** rkukura has left #openstack-infra | 22:13 | |
| sdague | and test runnings that hide everything except final error code make it easy to make bad mistakes in your test suite so it doesn't do what you think it's doing | 22:13 |
| *** radez_g0n3 is now known as radez | 22:16 | |
| SpamapS | jeblair: ty | 22:16 |
| *** JayJ has quit IRC | 22:16 | |
| clarkb | my favorite is sudo fails but test passes (dont think tracing would catch that though but maybe depends on how its logged) | 22:17 |
| SpamapS | jeblair: I may also squash the series since it's really all the same thing: make python3 work. | 22:17 |
| *** JayJ has joined #openstack-infra | 22:17 | |
| sdague | clarkb: yeh, I know we've still got a bunch of things mocked wrong in nova functional tests where we're running of of ips in tests, stack traces, tests still pass. Haven | 22:17 |
| sdague | 't gotten around to pulling all that appart yet | 22:18 |
| *** vilobhmm1 has joined #openstack-infra | 22:18 | |
| *** vilobhmm2 has quit IRC | 22:18 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add initial compute functional tests to Shade https://review.openstack.org/158463 | 22:19 |
| afazekas | ianw: Is anybody working on switching the kernel in the f21 job ? | 22:20 |
| ianw | afazekas: yeah, i'm digging into it now | 22:20 |
| ianw | afazekas: it seems that the extlinux.conf doesn't get rewritten correctly by a new kernel install | 22:20 |
| SpamapS | jeblair: also I may stop caring much about the py3 work.. there's some demons in the threading there... I keep finding dead backgrounded unit tests running py3.4 tests on my machine. | 22:21 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add initial compute functional tests to Shade https://review.openstack.org/158463 | 22:21 |
| ianw | afazekas: at least on rax | 22:21 |
| afazekas | ianw: The original image has different extlinux.conf then the installer expects | 22:21 |
| SpamapS | one of those "I will debug this when py3.4 is the alternative to a large wall of flame." moments | 22:21 |
| ianw | afazekas: there's two it seems ... /boot/extlinux.conf and /boot/extlinux/extlinux.conf, the latter is an anaconda one, the first seems hand-written | 22:22 |
| ianw | afazekas: it doesn't seem possible to regenerate a new one from scratch ... or if it is it's not obvious | 22:22 |
| jeblair | SpamapS: oh wow :( i'd really like it to work with python 3. i can't pick that up right now, but if you give up i will try to look into it at some point. | 22:23 |
| afazekas | ianw: the installer uses the grubby command, but it may have issues if you use spaces in the name | 22:23 |
| *** sarob has quit IRC | 22:23 | |
| *** JayJ has quit IRC | 22:23 | |
| *** JayJ has joined #openstack-infra | 22:24 | |
| SpamapS | jeblair: they've fixed a lot of GIL things in 3.4, so it's entirely possible things are running better and there are unfound races. | 22:24 |
| afazekas | ianw: extlinux support is not in the grubby's man page, it is kind of new thing.. | 22:24 |
| SpamapS | jeblair: it's also possible python-daemon 2.0.x is still awful. | 22:24 |
| clarkb | afazekas: ianw why would we switch the kernel? | 22:25 |
| ianw | clarkb: because the one in the image doesn't work | 22:25 |
| clarkb | what is wrong with it? and should we also be working on getting the base images fixed? | 22:26 |
| afazekas | clarkb: The f21 job is very likely fails because of kernel issue | 22:26 |
| *** tmcpeak has quit IRC | 22:26 | |
| clarkb | gertty syncing over tethered connection always makes me wish I synced prior to leaving home | 22:27 |
| afazekas | clarkb: the kernel itself is updated in the image, but the old one gets booted | 22:27 |
| ianw | clarkb: you'd think "yum update kernel" would update the kernel, but it doesn't on rax ... yes it's a underlying issue with the rax image | 22:27 |
| *** tmcpeak has joined #openstack-infra | 22:27 | |
| clarkb | afazekas: ianw gotcha | 22:27 |
| clarkb | that sounds ... unpleasant | 22:27 |
| ianw | clarkb: it looks like someone hand-wrote a extlinux.conf that confuses the update mechanism | 22:28 |
| jeblair | clarkb: it should be fast! also, if you are running with 155615, you can open a change directly, refresh, and it will prioritize that over the bg sync | 22:28 |
| afazekas | There is two way for setting default image in extlinux | 22:28 |
| afazekas | one is the default option, the other is the timeout | 22:28 |
| clarkb | jeblair: its much better than before I just think that my connection is sad | 22:28 |
| afazekas | f21 expects the default, as remember the rax image usaes the timeout | 22:28 |
| *** JayJ has quit IRC | 22:28 | |
| *** mattfarina has quit IRC | 22:28 | |
| clarkb | jeblair: I will have to grab 155615 | 22:28 |
| *** harlowja_ has quit IRC | 22:29 | |
| *** JayJ has joined #openstack-infra | 22:29 | |
| jeblair | clarkb: 155773 is my current tip and has that as a parent | 22:29 |
| * afazekas sleep | 22:29 | |
| jeblair | clarkb: (also starred changes) | 22:29 |
| *** harlowja has joined #openstack-infra | 22:30 | |
| ianw | afazekas: thanks for you help, i'll email to update. i think solution might be to copy anaconda config and then update kernel | 22:30 |
| *** pc_m has quit IRC | 22:30 | |
| clarkb | jeblair: thanks will grab that | 22:31 |
| clarkb | ianw: should we poke rax too about hteir image? | 22:31 |
| clarkb | ianw: if so jroll and JayF hav ebeen super handy | 22:31 |
| jroll | hi? | 22:32 |
| jroll | what'd we break now | 22:32 |
| ianw | jroll: on the rax f21 image, there's a /boot/extlinux.conf that looks like http://paste.openstack.org/show/180827/ | 22:33 |
| clarkb | jroll: the f21 base image supplied by rax apparently is configured in a away that prevents its kernel form being updated with yum | 22:33 |
| jroll | goody | 22:33 |
| clarkb | jeblair: woah that was a fast sync, I think there must be even newer magic than what I was expecting on the fast sync front | 22:33 |
| *** JayJ has quit IRC | 22:33 | |
| ianw | jroll: when you yum update, it messes things up. the APPEND line gets copied (so you get the wrong initramfs) and the default boot isn't updated | 22:34 |
| *** JayJ has joined #openstack-infra | 22:34 | |
| *** JayJ has quit IRC | 22:34 | |
| jroll | ianw: mind emailing details to jim@jimrollenhagen.com and I'll pass it along to people that deal with that stuff? | 22:34 |
| ianw | jroll: i think you can probably recreate this with grubby --some --series --of --not --well --documented --commands | 22:34 |
| ianw | jroll: sure | 22:35 |
| clarkb | jroll: you sir are awesome | 22:35 |
| jroll | ianw: clarkb: thanks, I do what I can | 22:35 |
| jroll | :) | 22:35 |
| *** tonytan4ever has quit IRC | 22:35 | |
| *** dkranz has quit IRC | 22:36 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Apache Config is now an optional module. https://review.openstack.org/157600 | 22:37 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Parameterized working directories. https://review.openstack.org/157587 | 22:37 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: StoryBoard is now installed into a Virtualenv https://review.openstack.org/158023 | 22:37 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/puppet-storyboard: Moved all module parameters into one manifest. https://review.openstack.org/157589 | 22:37 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Use StoryBoard optional https manifests. https://review.openstack.org/157983 | 22:37 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Centralized storyboard module configuration. https://review.openstack.org/157981 | 22:37 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Hostname and CORS domains are now configurable. https://review.openstack.org/158474 | 22:37 |
| clarkb | mordred: 158173 is jenkins sad | 22:37 |
| clarkb | mordred: looks like its near the first of the changes needed for the ansible work | 22:37 |
| mordred | clarkb: oh - piddle | 22:37 |
| mordred | clarkb: I was hacking on that one mildly blind on the plane - but yes, it is | 22:38 |
| mordred | I'll jump on it | 22:38 |
| mordred | oh - I get what's wrong | 22:38 |
| mordred | clarkb: patch in just a sec | 22:39 |
| clarkb | mordred: thanks, will review as soon as its available | 22:39 |
| *** mtanino has quit IRC | 22:40 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Add an index on accesstokens table for access_token column https://review.openstack.org/156543 | 22:41 |
| clarkb | if others are able to review https://review.openstack.org/#/c/157592/ and https://review.openstack.org/#/c/156788/ it would be a great help for the swift logs | 22:42 |
| clarkb | one is a bug fix for zuul's handlingof layout values and the other is sets up swift uploads in parallel | 22:42 |
| *** achanda has quit IRC | 22:43 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 22:44 | |
| jeblair | clarkb: ack, will do | 22:45 |
| *** achanda has joined #openstack-infra | 22:45 | |
| *** tkelsey has quit IRC | 22:46 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add ansible role for running puppet https://review.openstack.org/158173 | 22:47 |
| mordred | clarkb: ^^ | 22:47 |
| *** dmsimard is now known as dmsimard_away | 22:47 | |
| *** tkelsey has joined #openstack-infra | 22:50 | |
| *** lnr has joined #openstack-infra | 22:51 | |
| clarkb | jhesketh: can you point out the zuul change that allows reporting as not jenkins? | 22:51 |
| clarkb | jhesketh: you have a bunch of changes and want to make sure I orientate myself properly | 22:51 |
| clarkb | mordred: you need to rebase :0 | 22:52 |
| clarkb | er :) | 22:52 |
| *** lnr has left #openstack-infra | 22:52 | |
| mordred | clarkb: :( | 22:53 |
| *** sarob has joined #openstack-infra | 22:55 | |
| *** radez is now known as radez_g0n3 | 22:55 | |
| clarkb | jhesketh: question for you on the zuul merger run in zuul server change | 22:56 |
| clarkb | jeblair: ^ you may find that interesting too | 22:56 |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/project-config: Create Neutron DVR experimental job for aiopcpu https://review.openstack.org/158415 | 22:56 |
| *** hdd has quit IRC | 22:57 | |
| *** tkelsey has quit IRC | 22:58 | |
| *** tmcpeak has quit IRC | 22:59 | |
| *** e0ne has quit IRC | 23:00 | |
| jeblair | clarkb, jhesketh: yeah, i think that's what i was getting at in my original comment. i think it would be good to be consistent. | 23:01 |
| *** tmcpeak has joined #openstack-infra | 23:02 | |
| jogo | clarkb sdague: re using pentium for live migration | 23:02 |
| jhesketh | clarkb: https://review.openstack.org/#/c/121528/ (and all its dependencies) | 23:02 |
| clarkb | jhesketh: thank you | 23:02 |
| jhesketh | will take a look at that merger process | 23:03 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add ansible role for running puppet https://review.openstack.org/158173 | 23:03 |
| mordred | clarkb: ok - that's actually nicer and smaller now | 23:03 |
| *** harlowja has quit IRC | 23:04 | |
| *** rwsu is now known as rwsu-afk | 23:04 | |
| jogo | sdague: ahh I see you found the results, I wonder if those were random failures or not | 23:05 |
| jhesketh | clarkb: on the swift logs, when we have time, we should have a discussion to what level we want to support source/destination type notation that jenkins uses | 23:05 |
| *** hamzy has quit IRC | 23:06 | |
| *** hamzy has joined #openstack-infra | 23:06 | |
| *** Ryan_Lane has joined #openstack-infra | 23:06 | |
| clarkb | jhesketh: for job configuration? | 23:06 |
| jhesketh | yeah | 23:06 |
| clarkb | jhesketh: I think python has stdlib glob support, if we can just do that for source then hvae the bit for preserve hierarchy thats probably sufficient | 23:07 |
| *** stevemar has quit IRC | 23:07 | |
| *** stevemar has joined #openstack-infra | 23:07 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Set custom cpu_model for live_migrate https://review.openstack.org/141530 | 23:08 |
| *** camunoz_gone is now known as camunoz | 23:08 | |
| jogo | retrying pentium to get some another data point to check for consistent failures (I suspect it will fail the same way) | 23:08 |
| jhesketh | clarkb: hmm, that shouldn't be too hard | 23:08 |
| jhesketh | clarkb: the alternative would be to copy all the files you want into a staging dir and uploading that.. it'll make the indexes more straight forward anyway | 23:09 |
| jhesketh | clarkb: the glob stuff doesn't seem too difficult though so I'll tackle that today | 23:10 |
| clarkb | jhesketh: cool | 23:10 |
| *** baoli has quit IRC | 23:11 | |
| *** mfink_ has quit IRC | 23:12 | |
| *** achanda has quit IRC | 23:12 | |
| clarkb | jogo: sdague I have a hunch its doing something funny to arch that breaks us | 23:13 |
| clarkb | jogo: that said, what is in the kvm64 cpu type? I wonder if that is super generic and would work well | 23:13 |
| *** devlaps has quit IRC | 23:14 | |
| *** devlaps has joined #openstack-infra | 23:14 | |
| *** sarob has quit IRC | 23:14 | |
| jogo | clarkb: once the pentium one fails, we can try kvm64 again | 23:15 |
| jogo | but rax is xen ... | 23:16 |
| *** harlowja has joined #openstack-infra | 23:16 | |
| *** sarob has joined #openstack-infra | 23:16 | |
| *** baoli has joined #openstack-infra | 23:17 | |
| asselin | dstufft, my coworker noticed that the pip packages installed aren't the latest. and don't match what you said they should: http://15.126.198.151/00/155300/2/check/3par-iscsi-driver-master-client-pip-eos10-dsvm/74edc1f/logs/pip-freeze.txt.gz | 23:17 |
| *** MaxV has quit IRC | 23:17 | |
| clarkb | jogo: right but we are using qemu on the nested level | 23:17 |
| jogo | clarkb: https://git.cs.umu.se/cklein/qemu/commit/d1cd4bf41906980434bee1c6656e8aeb9a132df3 | 23:18 |
| jogo | looks pretty generic to me :) | 23:18 |
| openstackgerrit | Khai Do proposed openstack-infra/project-config: Refactor Gerrit plugin build jobs https://review.openstack.org/157603 | 23:18 |
| *** sarob has quit IRC | 23:19 | |
| clarkb | jogo: ya I think that may actually be what we need but we shall see | 23:19 |
| *** sarob has joined #openstack-infra | 23:20 | |
| openstackgerrit | Tim Kelsey proposed openstack-infra/project-config: Adding anchor gate tests https://review.openstack.org/157898 | 23:20 |
| dstufft | asselin: pip doesn't depend on stuff like a normal package, we bundle our dependencies inside of pip._vendor.* | 23:20 |
| dstufft | so you can't use pip freeze to see what version of requests we're using | 23:20 |
| dstufft | (unless youy're using python-pip from a Linux vendor that undoes that) | 23:20 |
| asselin | dstufft, I'm using what -infra/devstack is using... | 23:21 |
| mordred | that should be the latest from git-pip | 23:21 |
| mordred | get-pip | 23:21 |
| clarkb | http://lists.openstack.org/pipermail/openstack-dev/2015-February/057585.html hopefully I am not too far off base on that | 23:22 |
| jogo | https://github.com/qemu/qemu/blob/0c28d0d07fbcd7aa44d231241d444d00882256e2/target-i386/cpu.c | 23:22 |
| jeblair | mordred: i'm pretty sure if git-pip were a thing, it would break us up and down | 23:22 |
| clarkb | jogo: I don't think we want i386 | 23:22 |
| dstufft | jogo: does setuptools-git count? :V | 23:23 |
| dstufft | er | 23:23 |
| asselin | I am seeing this recent change that looks related : https://github.com/openstack-dev/devstack/commit/889f6046423de64de9b5c3ca3ac0b63cbf32dfd4 | 23:23 |
| jeblair | clarkb: i am -2 on having our zuul use two accounts | 23:23 |
| dstufft | jeblair: | 23:23 |
| dstufft | too many j names | 23:23 |
| *** otter768 has joined #openstack-infra | 23:23 | |
| clarkb | jeblair: how would you approcah that problem then? | 23:23 |
| jogo | clarkb: that seems to be where kvm64 and pentium are defined | 23:23 |
| clarkb | jeblair: eg how do we run tests that provide useful info that are not gating and actually looked at by humans | 23:23 |
| jeblair | clarkb: check jobs | 23:24 |
| clarkb | jogo: oh, I should read into paths less then | 23:24 |
| clarkb | jeblair: except we clean check | 23:24 |
| clarkb | jeblair: and jgriffith already blew up at us for even suggesting that route | 23:24 |
| jeblair | clarkb: why not gate? | 23:24 |
| jeblair | blew up? | 23:24 |
| clarkb | jeblair: that is a question for jgriffith | 23:24 |
| clarkb | jeblair: yes | 23:24 |
| clarkb | hold on | 23:24 |
| jeblair | clarkb: i must have missed that meeting | 23:24 |
| *** sarob has quit IRC | 23:24 | |
| clarkb | jeblair: http://lists.openstack.org/pipermail/openstack-dev/2015-February/057391.html precipitated much of this | 23:25 |
| * mordred is also confused | 23:25 | |
| *** achanda has joined #openstack-infra | 23:25 | |
| *** oomichi has joined #openstack-infra | 23:25 | |
| mordred | oh. hrm. interesting | 23:25 |
| clarkb | I have no problems with having these tests check/gate | 23:26 |
| clarkb | but others do | 23:26 |
| mordred | so - the rule of what can run in the gate has always been "is it open source" | 23:26 |
| mordred | it's just that most projects do not have multiple open source options | 23:26 |
| clarkb | (I responded to that email too) | 23:26 |
| StevenK | mordred: I am getting awfully close to teaching delorean about building .debs successfully | 23:28 |
| *** otter768 has quit IRC | 23:28 | |
| StevenK | 2015/02/24 10:27:55 Error response from daemon: Conflict, The name builder-ubuntu is already assigned to 31c8b7c0465d. You have to delete (or rename) that container to be able to assign builder-ubuntu to a container again. | 23:28 |
| StevenK | That ^ turning up every single time is getting annoying | 23:28 |
| jeblair | clarkb: (to complete a thought from earlier, a check job that should not gate should be non-voting because of clean check) | 23:28 |
| clarkb | jeblair: ok | 23:29 |
| clarkb | jeblair: I think the concern there is non voting is often ignored, BUT we can probably fix that as a UI thing? | 23:29 |
| jeblair | clarkb: i think we already have | 23:29 |
| *** dprince has quit IRC | 23:29 | |
| *** sabeen3 has quit IRC | 23:30 | |
| clarkb | it does make me happy that that is the route you would like because its less upfront work to get this going so win win :) | 23:30 |
| mordred | StevenK: neat! | 23:30 |
| mordred | StevenK: and blech | 23:30 |
| jeblair | clarkb: yay! i am also happy you are happy :) | 23:30 |
| *** baoli has quit IRC | 23:30 | |
| * mordred is happy that jeblair is happy that clarkb is happy, and also that clarkb is happy | 23:31 | |
| fungi | i'm still wondering how we reconcile the fact that, to take this specific example, cinder tells the sheepdog driver maintainers they must have integration testing reported on cinder changes to stay supported, infra tells the sheepdog driver maintainers that we're able to run their tests for them no need for an external ci rig, but cinder doesn't want job results from "jenkins" for this | 23:34 |
| asselin | anyone know how this recent devstck change in tools/_venv.sh may ignore pip.conf configurations? https://review.openstack.org/#/c/157535/4 | 23:34 |
| jeblair | fungi: i believe the problem is over-application of bureaucracy. let's use less of it. :) | 23:34 |
| fungi | it almost seems like projects are getting stockholm syndrome from third-party testing | 23:35 |
| *** krtaylor has quit IRC | 23:35 | |
| asselin | * ignore pip.conf configurations for virtual environments | 23:36 |
| *** prad has quit IRC | 23:36 | |
| *** mfisch has quit IRC | 23:36 | |
| fungi | where third-party testing was supposed to be an exceptional workaround for when we can't support the tests needed for some component, it's now been turned into some panacea | 23:36 |
| *** alexpilotti has joined #openstack-infra | 23:36 | |
| *** carl_baldwin has quit IRC | 23:37 | |
| *** gordc has quit IRC | 23:37 | |
| fungi | asselin: global /etc/pip.conf? something else? | 23:37 |
| clarkb | fungi: yes, because projects told everyone you must run third party testing. not "hey work with infra to get your test workload tested" | 23:37 |
| clarkb | fungi: it really set up the solution space in one way | 23:38 |
| asselin | fungi, I updated that too. chatting with dtroyer in -qa | 23:38 |
| openstackgerrit | Khai Do proposed openstack-infra/publications: Update logos on System slide https://review.openstack.org/158134 | 23:39 |
| openstackgerrit | Max Rydahl Andersen proposed openstack-infra/jenkins-job-builder: Initial work for supporting downstream-ext https://review.openstack.org/158117 | 23:40 |
| jeblair | clarkb: will respond to your email | 23:40 |
| clarkb | jeblair: thanks | 23:40 |
| *** abhirc has quit IRC | 23:41 | |
| *** armax has joined #openstack-infra | 23:41 | |
| *** abhirc has joined #openstack-infra | 23:41 | |
| *** ivar-laz_ has joined #openstack-infra | 23:42 | |
| *** andreykurilin_ has quit IRC | 23:43 | |
| *** ivar-lazzaro has quit IRC | 23:45 | |
| *** salv-orlando has quit IRC | 23:45 | |
| *** carl_baldwin has joined #openstack-infra | 23:47 | |
| *** krtaylor has joined #openstack-infra | 23:47 | |
| *** che-arne has quit IRC | 23:47 | |
| *** ddieterly has quit IRC | 23:50 | |
| clarkb | mordred: jenkins still doesn't like you :) | 23:54 |
| clarkb | mordred: I think you can tox -elayout locally to run it on your laptop | 23:54 |
| jhesketh | clarkb: is there a meaning to '**' within jenkings? | 23:55 |
| clarkb | jhesketh: ya its a recursive * | 23:55 |
| clarkb | jhesketh: so foo/**/bar matches foo/stuff/here/toarbitrary/depth/bar | 23:55 |
| jhesketh | ah I see | 23:56 |
| jhesketh | thanks | 23:56 |
| *** MarkAtwood_ has quit IRC | 23:56 | |
| *** MarkAtwood has quit IRC | 23:57 | |
| *** marcusvrn has joined #openstack-infra | 23:57 | |
| jhesketh | clarkb: does't look like that is part of the python glob library | 23:57 |
| *** melwitt has quit IRC | 23:57 | |
| jhesketh | will probably have to walk the tree | 23:57 |
| clarkb | jhesketh: its probably ok to not support * and be more specific in jobs | 23:58 |
| clarkb | jhesketh: I doubt there are any places where we really need ** | 23:58 |
| *** ivar-laz_ has quit IRC | 23:58 | |
| jhesketh | clarkb: true, but we'd need to check nearly every job as most of them use test-results which look like "**/*testr_results.html.gz".. not sure if we can assume they are always in the same place | 23:59 |
| *** ivar-lazzaro has joined #openstack-infra | 23:59 | |
| *** dizquierdo has quit IRC | 23:59 | |
| clarkb | jhesketh: ya, I think since we end up templating and cargo culting so much it won't be that much work | 23:59 |
| clarkb | jhesketh: I can try digging into that if you like | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!