| *** mrodden has quit IRC | 00:04 | |
| *** Ryan_Lane has quit IRC | 00:07 | |
| *** uvirtbot has joined #openstack-infra | 00:08 | |
| *** tvb has joined #openstack-infra | 00:11 | |
| *** tvb has joined #openstack-infra | 00:11 | |
| *** krtaylor has joined #openstack-infra | 00:15 | |
| *** tvb has quit IRC | 00:16 | |
| *** sarob has joined #openstack-infra | 00:16 | |
| *** nosnos has joined #openstack-infra | 00:16 | |
| *** mrodden has joined #openstack-infra | 00:17 | |
| *** Ryan_Lane has joined #openstack-infra | 00:18 | |
| *** kgriffs is now known as kgriffs_afk | 00:20 | |
| *** sarob has quit IRC | 00:20 | |
| *** svarnau has quit IRC | 00:24 | |
| *** reed has quit IRC | 00:24 | |
| *** acabrera has joined #openstack-infra | 00:26 | |
| *** acabrera has left #openstack-infra | 00:27 | |
| *** mrda has joined #openstack-infra | 00:34 | |
| *** matsuhashi has joined #openstack-infra | 00:35 | |
| *** alexpilotti has quit IRC | 00:35 | |
| *** changbl has joined #openstack-infra | 00:36 | |
| *** thomasm has quit IRC | 00:37 | |
| *** rwsu is now known as rwsu-away | 00:38 | |
| *** ryanpetrello has joined #openstack-infra | 00:39 | |
| *** marun has quit IRC | 00:47 | |
| *** amotoki has joined #openstack-infra | 00:56 | |
| *** Ryan_Lane has quit IRC | 00:56 | |
| *** Ryan_Lane has joined #openstack-infra | 00:56 | |
| *** anteaya has quit IRC | 01:02 | |
| *** ryanpetrello has quit IRC | 01:03 | |
| *** Ryan_Lane has quit IRC | 01:03 | |
| *** Ryan_Lane has joined #openstack-infra | 01:04 | |
| *** melwitt has quit IRC | 01:08 | |
| *** gyee has quit IRC | 01:12 | |
| *** dcramer_ has quit IRC | 01:15 | |
| *** pabelanger has quit IRC | 01:23 | |
| *** pabelanger has joined #openstack-infra | 01:25 | |
| *** dkranz has joined #openstack-infra | 01:25 | |
| *** yaguang has joined #openstack-infra | 01:27 | |
| *** Ryan_Lane has quit IRC | 01:28 | |
| *** Ryan_Lane has joined #openstack-infra | 01:29 | |
| *** Ryan_Lane has quit IRC | 01:39 | |
| rockyg | Anyone out there? Is Grizzly Jenkins still having problems? | 01:42 |
|---|---|---|
| *** Ryan_Lane has joined #openstack-infra | 01:42 | |
| *** kong has joined #openstack-infra | 01:45 | |
| clarkb | rockyg: last I saw yes. but I am sitting behind homeplate at a baseball game so cant check | 01:51 |
| pabelanger | clarkb, hardcore | 01:51 |
| rockyg | Thanks. And enjoy! Monty's patch has failed Jenkins, so I am assuming Grizzly is still broken | 01:52 |
| *** dcramer_ has joined #openstack-infra | 01:56 | |
| *** krtaylor has quit IRC | 01:56 | |
| mrodden | its bad when i got to try to update a patchset i didn't have time to get to earlier and i can't remember where my git clone of that project is... | 01:57 |
| *** Ryan_Lane has quit IRC | 01:58 | |
| *** Ryan_Lane has joined #openstack-infra | 01:59 | |
| *** marun has joined #openstack-infra | 02:03 | |
| *** SergeyLukjanov has joined #openstack-infra | 02:03 | |
| *** SergeyLukjanov has quit IRC | 02:03 | |
| *** SergeyLukjanov has joined #openstack-infra | 02:04 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Visualize Zuul's NNFI scheduler https://review.openstack.org/47569 | 02:05 |
| *** Ryan_Lane has quit IRC | 02:09 | |
| *** Ryan_Lane has joined #openstack-infra | 02:11 | |
| *** Ryan_Lane1 has joined #openstack-infra | 02:11 | |
| *** tvb has joined #openstack-infra | 02:13 | |
| *** tvb has joined #openstack-infra | 02:13 | |
| *** Ryan_Lane has quit IRC | 02:15 | |
| *** UtahDave has joined #openstack-infra | 02:16 | |
| *** Ryan_Lane1 has quit IRC | 02:17 | |
| *** tvb has quit IRC | 02:18 | |
| jeblair | clarkb, mordred, fungi: ^ just shoot me. but it works: http://i.imgur.com/IdsXESv.png | 02:18 |
| jeblair | Alex_Gaynor: ^ | 02:18 |
| pleia2 | oooh | 02:18 |
| pleia2 | jeblair: that's awesome :) | 02:19 |
| Alex_Gaynor | neat! | 02:19 |
| *** zehicle_at_dell has joined #openstack-infra | 02:19 | |
| dims | very nice :) | 02:20 |
| jeblair | thanks :) it should handle fairly complex graphs too. anyway, dinnertime here | 02:20 |
| morganfainberg | jeblair, wow, thats cool | 02:21 |
| jog0 | I just noticed elastic-recheck is on infra now, nice! But the groups on gerrit are empty | 02:26 |
| *** dcramer_ has quit IRC | 02:27 | |
| openstackgerrit | Mathew Odden proposed a change to openstack-infra/jenkins-job-builder: Add publisher for Git Publisher support https://review.openstack.org/46417 | 02:34 |
| openstackgerrit | Mathew Odden proposed a change to openstack-infra/jenkins-job-builder: Add publisher for Git Publisher support https://review.openstack.org/46417 | 02:39 |
| *** dcramer_ has joined #openstack-infra | 02:40 | |
| mrodden | git graphs for zuul queue? | 02:42 |
| mrodden | fancy | 02:42 |
| *** ryanpetrello has joined #openstack-infra | 02:43 | |
| *** dims has quit IRC | 02:45 | |
| clarkb | jog0 I will add you to them when I get home | 02:47 |
| clarkb | jeblair: nice | 02:47 |
| fungi | jeblair: looks awesome | 02:47 |
| kong | anybody can see https://review.openstack.org/#/c/48093/, and help? Thanks! | 02:47 |
| clarkb | jeblair: did you see my nnfi update to the overview simulation? | 02:48 |
| clarkb | kong grizzly is broken right now | 02:48 |
| kong | :( | 02:48 |
| kong | what could I do now? | 02:49 |
| kong | just wait, right? | 02:49 |
| clarkb | you could work on fixing it | 02:49 |
| kong | I want help, but I'm afraid I'm not good at it | 02:50 |
| clarkb | fungi: is it a matter of updating grizzly pins? | 02:51 |
| mrodden | so today i realized that i haven't been garbage collecting any of my repos on Gerrit | 02:51 |
| mrodden | that leads to some Bad Things after a few months of usage | 02:52 |
| jog0 | clarkb: thanks | 02:53 |
| *** schwicht has quit IRC | 02:56 | |
| *** ryanpetrello has quit IRC | 02:56 | |
| fungi | clarkb: i think part of the grizzly problem is that we test with current clients | 02:59 |
| clarkb | oh | 03:00 |
| clarkb | fun | 03:00 |
| fungi | which will potentially have conflicting requirements now if not pinned there too | 03:00 |
| fungi | so i think maybe the solution was going to be yet another cliff release | 03:01 |
| fungi | and lest i forget, totally unrelated but, tom f. wasn't on today's call (on the way to russia i think?) so we should hopefully be able to catch up with him euro hours tomorrow | 03:04 |
| *** kiall has quit IRC | 03:04 | |
| clarkb | cool thanks | 03:05 |
| *** SergeyLukjanov has quit IRC | 03:05 | |
| *** cody-somerville has quit IRC | 03:07 | |
| *** sandywalsh has quit IRC | 03:14 | |
| *** cody-somerville has joined #openstack-infra | 03:22 | |
| *** dcramer_ has quit IRC | 03:25 | |
| *** sandywalsh has joined #openstack-infra | 03:26 | |
| *** michchap_ has joined #openstack-infra | 03:34 | |
| *** michchap has quit IRC | 03:37 | |
| *** jerryz has quit IRC | 03:38 | |
| *** nosnos has quit IRC | 03:40 | |
| *** nosnos has joined #openstack-infra | 03:41 | |
| *** nosnos has quit IRC | 03:42 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Create new gates for openstack-manuals https://review.openstack.org/48128 | 03:43 |
| *** nosnos has joined #openstack-infra | 03:44 | |
| *** tvb has joined #openstack-infra | 03:45 | |
| *** kgriffs_afk is now known as kgriffs | 03:48 | |
| *** tvb has quit IRC | 03:49 | |
| *** michchap_ has quit IRC | 03:49 | |
| *** michchap has joined #openstack-infra | 03:49 | |
| *** michchap_ has joined #openstack-infra | 03:53 | |
| *** michchap_ has joined #openstack-infra | 03:54 | |
| *** michchap has quit IRC | 03:56 | |
| *** reed has joined #openstack-infra | 03:58 | |
| *** michchap_ has quit IRC | 03:59 | |
| *** michchap has joined #openstack-infra | 04:01 | |
| *** kgriffs is now known as kgriffs_afk | 04:01 | |
| *** michchap has quit IRC | 04:02 | |
| SpamapS | man the gate fails are getting ridiculous | 04:03 |
| SpamapS | shouldn't we be.. stabilizing.. right now? | 04:03 |
| *** michchap has joined #openstack-infra | 04:04 | |
| *** michchap has joined #openstack-infra | 04:05 | |
| *** kiall has joined #openstack-infra | 04:09 | |
| *** michchap has quit IRC | 04:12 | |
| *** vipul is now known as vipul-away | 04:12 | |
| *** ryanpetrello has joined #openstack-infra | 04:14 | |
| *** vipul-away is now known as vipul | 04:16 | |
| *** sarob has joined #openstack-infra | 04:17 | |
| *** michchap has joined #openstack-infra | 04:21 | |
| *** michchap_ has joined #openstack-infra | 04:23 | |
| *** dtroyer has quit IRC | 04:25 | |
| *** michchap has quit IRC | 04:26 | |
| *** marun has quit IRC | 04:26 | |
| *** michchap_ has quit IRC | 04:27 | |
| *** michchap has joined #openstack-infra | 04:28 | |
| *** sarob has quit IRC | 04:29 | |
| *** sarob has joined #openstack-infra | 04:29 | |
| *** dcramer_ has joined #openstack-infra | 04:30 | |
| *** dtroyer has joined #openstack-infra | 04:31 | |
| *** michchap_ has joined #openstack-infra | 04:32 | |
| *** michchap has quit IRC | 04:33 | |
| *** sarob has quit IRC | 04:34 | |
| *** sarob_ has joined #openstack-infra | 04:34 | |
| *** sdake__ has joined #openstack-infra | 04:35 | |
| *** sdake__ has quit IRC | 04:35 | |
| *** sdake__ has joined #openstack-infra | 04:35 | |
| *** sdake_ has quit IRC | 04:35 | |
| *** michchap_ has quit IRC | 04:35 | |
| *** michchap has joined #openstack-infra | 04:37 | |
| *** sarob_ has quit IRC | 04:38 | |
| *** sarob has joined #openstack-infra | 04:39 | |
| *** sarob has quit IRC | 04:43 | |
| *** dkliban has joined #openstack-infra | 04:45 | |
| *** raies_ has quit IRC | 04:49 | |
| *** dkliban has quit IRC | 04:55 | |
| *** basha has joined #openstack-infra | 05:00 | |
| basha | Hi is anyone available? I had some inputs on a jenkins failure as part of https://review.openstack.org/#/c/47161/ | 05:03 |
| *** dcramer_ has quit IRC | 05:05 | |
| *** reed has quit IRC | 05:07 | |
| *** jhesketh_ has quit IRC | 05:09 | |
| *** DinaBelova has joined #openstack-infra | 05:11 | |
| *** tvb has joined #openstack-infra | 05:15 | |
| *** tvb has joined #openstack-infra | 05:15 | |
| *** nicedice has quit IRC | 05:16 | |
| *** SergeyLukjanov has joined #openstack-infra | 05:18 | |
| *** SergeyLukjanov has quit IRC | 05:18 | |
| *** UtahDave has quit IRC | 05:20 | |
| *** tvb has quit IRC | 05:20 | |
| *** ryanpetrello has quit IRC | 05:21 | |
| *** jhesketh_ has joined #openstack-infra | 05:22 | |
| *** ryanpetrello has joined #openstack-infra | 05:22 | |
| clarkb | jog0: I added you to the ptl group, doesn't look like the ACL applied properly though.. I will look into it tomorrow if no one else beats me to it | 05:41 |
| *** afazekas has joined #openstack-infra | 05:41 | |
| clarkb | basha: you need to make that change in a backward compatbile way | 05:41 |
| clarkb | basha: eg support both commands, update things that use old command, remove old command after some deprecation period | 05:42 |
| basha | clarkb: Do you mean mark the old guy as deprecated and switch to the new command? | 05:43 |
| clarkb | basically | 05:44 |
| clarkb | you need to support both while you transition, then when everyone is happy with some deprecation period you can remove the old one | 05:44 |
| basha | clarkb: Oh ok. Is there some wiki which tells me ow to bring in this deprecation? | 05:45 |
| clarkb | I'm not sure | 05:46 |
| basha | Ok. I'll just ask around. Thanks :) | 05:46 |
| *** DinaBelova has quit IRC | 05:48 | |
| *** pblaho has joined #openstack-infra | 05:49 | |
| *** sarob has joined #openstack-infra | 05:49 | |
| *** DinaBelova has joined #openstack-infra | 05:53 | |
| *** sarob has quit IRC | 05:54 | |
| *** rwsu-away has quit IRC | 06:06 | |
| *** yolanda has joined #openstack-infra | 06:13 | |
| *** blamar has quit IRC | 06:13 | |
| *** lynn has joined #openstack-infra | 06:16 | |
| *** rwsu-away has joined #openstack-infra | 06:17 | |
| *** yaguang has quit IRC | 06:18 | |
| *** lynn has quit IRC | 06:18 | |
| *** kong has quit IRC | 06:20 | |
| *** salv-orlando has quit IRC | 06:24 | |
| *** DinaBelova has quit IRC | 06:33 | |
| *** saschpe_ has quit IRC | 06:48 | |
| *** tkammer has joined #openstack-infra | 07:02 | |
| *** enikanorov-w has quit IRC | 07:15 | |
| *** enikanorov-w has joined #openstack-infra | 07:16 | |
| *** jcoufal has joined #openstack-infra | 07:21 | |
| *** thomasbiege has joined #openstack-infra | 07:25 | |
| *** rpodolyaka has joined #openstack-infra | 07:26 | |
| *** flaper87|afk is now known as flaper87 | 07:30 | |
| *** basha has quit IRC | 07:34 | |
| *** thomasbiege has quit IRC | 07:37 | |
| *** tvb has joined #openstack-infra | 07:43 | |
| *** tvb has joined #openstack-infra | 07:43 | |
| *** dizquierdo has joined #openstack-infra | 07:51 | |
| *** hashar has joined #openstack-infra | 07:57 | |
| hashar | hello there | 07:58 |
| hashar | just in case, does anyone how the review scores are generated on http://status.openstack.org/reviews/ ? | 07:58 |
| ttx | clarkb: looks like the new "related bug" message works: https://bugs.launchpad.net/python-cinderclient/+bug/1220590 | 07:59 |
| uvirtbot | Launchpad bug 1220590 in python-cinderclient "Cannot handle volume name argument" [Undecided,Fix committed] | 07:59 |
| ttx | (that bug contain both the update-bug message for "fixes" and "related") | 07:59 |
| ttx | hashar: you can find the scoring in the code | 08:00 |
| hashar | ttx: just found https://github.com/openstack-infra/reviewday :-) | 08:00 |
| hashar | ttx: I like your idea, will probably set up something similar for us | 08:00 |
| ttx | https://github.com/openstack-infra/reviewday/blob/master/reviewday/mergeprop.py | 08:01 |
| hashar | ttx: thanks :) | 08:06 |
| hashar | we really have a lot to learn from you, one day we should organize a meetup of openstack/wikimedia folks :D | 08:06 |
| *** fbo_away is now known as fbo | 08:07 | |
| *** markmc has joined #openstack-infra | 08:13 | |
| *** ndipanov_gone is now known as ndipanov | 08:14 | |
| *** derekh has joined #openstack-infra | 08:16 | |
| *** boris-42 has joined #openstack-infra | 08:20 | |
| *** cody-somerville has quit IRC | 08:21 | |
| *** mrda has quit IRC | 08:22 | |
| *** salv-orlando has joined #openstack-infra | 08:44 | |
| *** basha has joined #openstack-infra | 08:49 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:58 | |
| *** amotoki has quit IRC | 09:00 | |
| *** DinaBelova has joined #openstack-infra | 09:00 | |
| *** johnthetubaguy1 has joined #openstack-infra | 09:09 | |
| *** johnthetubaguy1 is now known as johnthetubaguy | 09:10 | |
| *** DinaBelova has quit IRC | 09:10 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/config: Clarify that statusbot requires public messages https://review.openstack.org/48209 | 09:13 |
| *** fbo has quit IRC | 09:18 | |
| *** fbo has joined #openstack-infra | 09:19 | |
| *** mrda has joined #openstack-infra | 09:21 | |
| *** DinaBelova has joined #openstack-infra | 09:22 | |
| *** hashar has quit IRC | 09:32 | |
| *** hashar has joined #openstack-infra | 09:34 | |
| LinuxJedi | oh wow, according to stackalytics I'm the 6th top independent contributor by lines of code in Havana (all projects). Not bad for a manager who shouldn't be touching code :) | 09:35 |
| *** tvb has quit IRC | 09:54 | |
| *** DinaBelova has quit IRC | 09:58 | |
| *** DinaBelova has joined #openstack-infra | 10:00 | |
| *** afazekas_ has joined #openstack-infra | 10:01 | |
| *** afazekas has quit IRC | 10:02 | |
| *** tvb has joined #openstack-infra | 10:06 | |
| *** tvb has quit IRC | 10:14 | |
| *** afazekas_ has quit IRC | 10:17 | |
| *** DinaBelova has quit IRC | 10:22 | |
| *** schwicht has joined #openstack-infra | 10:32 | |
| *** afazekas_ has joined #openstack-infra | 10:37 | |
| *** markmc has quit IRC | 10:45 | |
| *** tvb|afk has joined #openstack-infra | 11:01 | |
| *** tvb|afk has joined #openstack-infra | 11:01 | |
| *** ruhe has joined #openstack-infra | 11:05 | |
| *** jhesketh_ has quit IRC | 11:08 | |
| *** dims has joined #openstack-infra | 11:13 | |
| *** alexpilotti has joined #openstack-infra | 11:15 | |
| *** hashar has quit IRC | 11:18 | |
| *** jhesketh_ has joined #openstack-infra | 11:20 | |
| *** tkammer has quit IRC | 11:33 | |
| *** mrda has quit IRC | 11:33 | |
| *** basha has quit IRC | 11:36 | |
| *** krtaylor has joined #openstack-infra | 11:39 | |
| *** weshay has joined #openstack-infra | 11:40 | |
| *** hashar has joined #openstack-infra | 11:40 | |
| *** lcestari has quit IRC | 11:44 | |
| flaper87 | Can I get any core's attention here? https://review.openstack.org/#/c/48089/ It's a very quick review | 11:44 |
| *** lcestari has joined #openstack-infra | 11:47 | |
| *** cody-somerville has joined #openstack-infra | 11:50 | |
| *** ArxCruz has joined #openstack-infra | 11:52 | |
| *** johnthetubaguy has quit IRC | 11:53 | |
| *** johnthetubaguy has joined #openstack-infra | 11:55 | |
| *** beagles has quit IRC | 11:55 | |
| *** ruhe has quit IRC | 12:00 | |
| *** beagles has joined #openstack-infra | 12:01 | |
| *** sandywalsh has quit IRC | 12:07 | |
| *** dprince has joined #openstack-infra | 12:10 | |
| openstackgerrit | Julien Danjou proposed a change to openstack-infra/config: Revert "Revert "Use Ubuntu Cloud Archive on precise nodes"" https://review.openstack.org/48226 | 12:14 |
| openstackgerrit | Eoghan Glynn proposed a change to openstack/requirements: Revert "Update Ceilometer alembic requirement to 0.6.0 version" https://review.openstack.org/48227 | 12:16 |
| *** kong has joined #openstack-infra | 12:16 | |
| Shrews | mordred: I *think* maybe your latest patch to libra broke the 'build_sphinx' command. I keep getting 'invalid command'. Perhaps my environment is toast? | 12:17 |
| *** sandywalsh has joined #openstack-infra | 12:20 | |
| Shrews | mordred: nm. i am an idiot | 12:23 |
| *** thomasm has joined #openstack-infra | 12:28 | |
| *** dkliban has joined #openstack-infra | 12:30 | |
| *** michchap has quit IRC | 12:32 | |
| *** michchap has joined #openstack-infra | 12:33 | |
| *** michchap has quit IRC | 12:35 | |
| *** michchap has joined #openstack-infra | 12:36 | |
| *** dims has quit IRC | 12:36 | |
| *** michchap has quit IRC | 12:36 | |
| *** slong has quit IRC | 12:38 | |
| *** michchap has joined #openstack-infra | 12:39 | |
| *** ruhe has joined #openstack-infra | 12:39 | |
| *** acabrera has joined #openstack-infra | 12:40 | |
| *** acabrera is now known as alcabrera | 12:41 | |
| *** annegentle has quit IRC | 12:42 | |
| *** annegentle has joined #openstack-infra | 12:43 | |
| *** dims has joined #openstack-infra | 12:43 | |
| *** ryanpetrello has quit IRC | 12:43 | |
| *** ryanpetrello has joined #openstack-infra | 12:48 | |
| *** krtaylor has quit IRC | 12:49 | |
| *** adam_g has quit IRC | 12:49 | |
| *** adam_g has joined #openstack-infra | 12:50 | |
| *** adam_g has quit IRC | 12:50 | |
| *** adam_g has joined #openstack-infra | 12:50 | |
| *** mriedem has joined #openstack-infra | 12:57 | |
| *** salv-orlando has quit IRC | 12:59 | |
| *** krtaylor has joined #openstack-infra | 13:01 | |
| *** blamar has joined #openstack-infra | 13:02 | |
| *** anteaya has joined #openstack-infra | 13:02 | |
| *** weshay has quit IRC | 13:09 | |
| anteaya | ttx: I'm a little foggy on the problem that 48209 solves | 13:11 |
| ttx | anteaya: It's a problem we encoutered yesterday. reading the original doc, I thought that the bot would require PRIVMSG | 13:12 |
| ttx | and was enable to complete the status post when I needed it | 13:12 |
| *** julim has joined #openstack-infra | 13:12 | |
| ttx | I figured I should fix the doc | 13:12 |
| anteaya | ah okay | 13:12 |
| mordred | LinuxJedi: haha. | 13:12 |
| mordred | SergeyLukjanov: do you happen to know how to communicate company affiliation to the stackalytics folks? | 13:13 |
| *** saschpe has joined #openstack-infra | 13:13 | |
| *** SergeyLukjanov has quit IRC | 13:15 | |
| mordred | LinuxJedi: ah - I see you've fixed yourself! | 13:16 |
| *** basha has joined #openstack-infra | 13:19 | |
| LinuxJedi | mordred: yep, not sure when the system refreshes to show that fix though | 13:20 |
| *** rfolco has joined #openstack-infra | 13:20 | |
| *** basha has quit IRC | 13:20 | |
| *** basha has joined #openstack-infra | 13:21 | |
| Shrews | he also fixed me, though I know I've provided my company affiliation to someone at least twice | 13:21 |
| *** weshay has joined #openstack-infra | 13:22 | |
| anteaya | both Khai and I are now listed as independent | 13:25 |
| *** kgriffs_afk is now known as kgriffs | 13:25 | |
| *** matty_dubs|gone is now known as matty_dubs | 13:26 | |
| mordred | anteaya: I just submitted https://review.openstack.org/48254 | 13:27 |
| mordred | to fix zaro and Ng and jeblair | 13:27 |
| mordred | anteaya: didn't catch you were in the independent list. perhaps submit a patch to fix yourself? | 13:28 |
| anteaya | oh okay, I can do that too | 13:29 |
| LinuxJedi | Shrews: I have found dozens of HP employees listed as independent on this system so don't worry about it :) | 13:30 |
| LinuxJedi | I suspect the same is true of other companies | 13:30 |
| *** changbl has quit IRC | 13:31 | |
| Ng | mordred: aha, I put a change in for myself earlier, I'll abandon mine | 13:32 |
| anteaya | maybe it is a comment on working style, in which case independent fits | 13:33 |
| Ng | it is annoying to have to keep resubmitting affiliation to each one of these different tools :) | 13:33 |
| openstackgerrit | A change was merged to openstack-infra/config: Make PY3K a voting for python-marconiclient https://review.openstack.org/48089 | 13:34 |
| mordred | Ng: yah | 13:39 |
| fungi | at least some of them seem to scrape/sync up with each other | 13:39 |
| Ng | maybe affiliation should be per-commit! then git-review can handle it and make sure all my commits have the right data ;) | 13:39 |
| Ng | (and handle personal-time commits, changes in employment, etc) | 13:40 |
| mordred | Ng: don't tempt fate, somebody could think that's a good idea | 13:40 |
| Ng | it was a silly idea when I typed it, but now I'm starting to wonder :) | 13:40 |
| fungi | the only sane (for limited definitions of the word) way to do that would be for git-review to make a separate connection to register the affiliation under which a particular patchset was submitted | 13:40 |
| *** odyssey4me has joined #openstack-infra | 13:41 | |
| anteaya | just reading the backscroll and found the jeblair zuul subway map, that is wonderful. I'm thinking the NYC maps could use your touch if you are ever in the mood. | 13:41 |
| *** tvb|afk has quit IRC | 13:41 | |
| anteaya | the zuul status screen is going to be more entertaining than some video games | 13:42 |
| fungi | and even then, that would be messy. commits made on behalf of others? cherry-picked backports? matching the submitted patchset to a particular merge commit on a given branch? | 13:42 |
| mordred | hashar: we welcome any such meetups! | 13:43 |
| mordred | jeblair subway map zuul screen | 13:43 |
| mordred | ++ | 13:43 |
| fungi | yeah, subway map. that never even occurred to me when looking at the screencap... i spend too much time in cities with awful public transit | 13:43 |
| mordred | sdague, fungi: so - reading backscrool ... discussion about client libs and stable branch pins | 13:44 |
| mordred | gets us in to an interesting place where I'm not particularly sure we've got a good model | 13:45 |
| fungi | mordred: yes, what's the path forward for grizzly and cliff? | 13:45 |
| *** julim has quit IRC | 13:45 | |
| mordred | I don't know | 13:45 |
| *** davidhadas_ has joined #openstack-infra | 13:46 | |
| mordred | unless we start pinning client libs for stable releases | 13:46 |
| mordred | which just moves the complexity marker to the point in time where we need a bugfix in a pinned old client | 13:46 |
| fungi | make sure current clients support how many server releases back? | 13:46 |
| mordred | it's not that - current clients are supposed to always support old releases | 13:46 |
| fungi | more of a project-wide question there i guess | 13:46 |
| mordred | from an API perspective | 13:46 |
| mordred | until the end of time | 13:46 |
| mordred | but the requirement chain issue is interesting | 13:47 |
| mordred | if a new client lib updates depends - that is not something we've really considered or modeled | 13:47 |
| fungi | ahh, yeah. so if current clients depend on versions of libs which old servers are unable to work with, therein lies a conflict | 13:47 |
| fungi | or what you said | 13:47 |
| mordred | yes. because the servers consume the clients as, you konw, libraries | 13:47 |
| soren | Does Gerrit verify that one has signed the CLA? | 13:48 |
| mordred | fungi: I'm going to have to think about this for a bit - but I think the answer might be stable pinning | 13:48 |
| fungi | soren: yes | 13:48 |
| mordred | soren: yes | 13:48 |
| soren | What about CCLA? | 13:48 |
| mordred | soren: no | 13:48 |
| fungi | soren: no | 13:48 |
| soren | Does it have any clue about affiliation? | 13:48 |
| fungi | [echo [echo [echo]]] | 13:48 |
| fungi | soren: nope | 13:48 |
| fungi | soren: it knows e-mail addresses, ssh keys, openid urls | 13:49 |
| soren | Ok. How does it verify having signed the CLA? Is there a public list somewhere or does it have special access? | 13:49 |
| fungi | soren: oh, and full name, whatever that means these days | 13:49 |
| fungi | soren: it has an internal database table it consults for the icls | 13:49 |
| fungi | er,icla | 13:49 |
| * soren didn't realise the concept of full names was more complicated today than yesterday | 13:49 | |
| fungi | human names are a complicated topic | 13:50 |
| soren | I see. | 13:50 |
| fungi | but an entirely separate ratsnest | 13:50 |
| soren | (, he said, but he didn't actually see) | 13:50 |
| soren | Yeah, sure. | 13:50 |
| soren | Ok. | 13:50 |
| soren | Cool, thanks. I think that answers my questions. | 13:50 |
| sdague | mordred: I think for stable we should run 2 jobs, frozen clients + stable tree & master clients + stable tree | 13:50 |
| mordred | sdague: yes. I've got the jobs for the second one, but can't get anyone to review it :) | 13:51 |
| fungi | so anyway, gerrit's account_agreements table tracks which agreements are signed by a particular account, if any, and then consults that to confirm whether a commit should be allowed based on whether the target project has cla enforcement enabled | 13:51 |
| sdague | mordred: link it to me | 13:51 |
| *** tvb|afk has joined #openstack-infra | 13:52 | |
| *** tvb|afk has joined #openstack-infra | 13:52 | |
| mordred | soren: it's tied to your gerrit account, gerrit will quite simply not let you submit a patch for review at all for a project if you haven't signed the cla | 13:52 |
| fungi | soren: there's also an account_group_agreements table which allows groups of users to be associated to a particular cla as signatories. we currently use that to authorize usa government employees/contractors under the blanked usgcla | 13:52 |
| mordred | sdague: https://review.openstack.org/#/c/41945/ and https://review.openstack.org/#/c/41931/ | 13:52 |
| mordred | fungi: https://review.openstack.org/#/c/41945/ ? pretty please? | 13:53 |
| fungi | er, s/blanked/blanket/ | 13:53 |
| mordred | sdague: frozen clients against stable branch will be hard to devstack | 13:53 |
| sdague | mordred: why, we aren't tagging clients? | 13:54 |
| mordred | sdague: how do we tell devstack to do that combination? | 13:54 |
| sdague | stackrc | 13:54 |
| sdague | we have per branch specifications | 13:54 |
| mordred | sdague: oh. wow. yeah. ok. gross, but ok | 13:54 |
| sdague | per project specification of branches | 13:54 |
| mordred | so I guess this gets better once havana is the stable branch | 13:55 |
| mordred | since we have requirements processing for havana | 13:55 |
| Mithrandir | the pip problem a little while ago is fixed, right? | 13:55 |
| *** DennyZhang has joined #openstack-infra | 13:55 | |
| mordred | we should probably actually make all of our requirments jobs work with branches | 13:56 |
| mordred | Mithrandir: there was a pip problem? | 13:56 |
| Mithrandir | mordred: hmm, maybe sphinx? Something pulled in a prerelease of something that made the world explode | 13:56 |
| Mithrandir | about a week ago? | 13:56 |
| mordred | Mithrandir: ah, yes. that. nope, it's not fixed globally, but I can tell you all abot it | 13:56 |
| mordred | and what to do about it | 13:56 |
| mordred | Mithrandir: how much do you want to know? | 13:57 |
| Mithrandir | I'd just like https://review.openstack.org/#/c/44525/ retried, if that's possible. | 13:57 |
| jeblair | mordred, sdague: er, don't tell devstack to check out code; that's not the way devstack-gate works | 13:57 |
| fungi | Mithrandir: yep, safe. jjb is now no longer testing with pre-release packages from pypi | 13:57 |
| Mithrandir | fungi: ok, coolie. Can you/whom should I ask to get it retried? | 13:58 |
| mordred | Mithrandir: you can do it yourself! | 13:58 |
| fungi | (except that i already did) | 13:58 |
| mordred | Mithrandir: leave a comment "reverify no bug" and it will re-attempt to land it | 13:58 |
| Mithrandir | mordred: ah, ok. Thanks, good to know for next time. | 13:58 |
| Mithrandir | fungi: thx! | 13:59 |
| fungi | so it's currently worming its way through the bowels of zuul | 13:59 |
| sdague | jeblair: right | 13:59 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Add support for purging obsolete, managed jobs https://review.openstack.org/44525 | 13:59 |
| Mithrandir | sure, no hurry. Just didn't want it dropped on the floor | 13:59 |
| mordred | Mithrandir: (if there is a bug that the gate problem is associated, you should say bug XXXXX instead of no bug) | 13:59 |
| Mithrandir | hurrah! | 13:59 |
| fungi | and landed | 13:59 |
| sdague | jeblair: sorry, otp, but I'm with you | 13:59 |
| Mithrandir | I'll probably take a stab at adding a token one can provide sometime later, so you can have multiple yaml sources and yet only purge your own. | 14:00 |
| mordred | jeblair: re: your zuul change. it lgtm, but I'm mostly just trusting that you tested it and like and stuff | 14:00 |
| jeblair | mordred: i like tested it. i didn't like totally test it though. | 14:01 |
| *** yassine has joined #openstack-infra | 14:01 | |
| *** yaguang has joined #openstack-infra | 14:02 | |
| fungi | i like it, but didn't test it | 14:02 |
| jeblair | mordred: i verified that it works fine on Monday, September 23 2013 at around 11am PDT. | 14:02 |
| mordred | jeblair: fine by me | 14:02 |
| mordred | jeblair: something tells me there is only one good way to test this | 14:02 |
| fungi | chuck norris tests HIS code in production, after all | 14:03 |
| mordred | fungi: chuck norris tests production with his code | 14:03 |
| zul | i have to stop highlighting the word chuck | 14:03 |
| mordred | zul: hahaha | 14:03 |
| fungi | sorry zul | 14:04 |
| jeblair | fine, zul norris tests his code in production | 14:04 |
| mordred | jeblair: ++ | 14:04 |
| * fungi CHUCKles | 14:04 | |
| *** pblaho has quit IRC | 14:04 | |
| hashar | mordred: re OS/wikimedia meetup : it would be easy to bring some bay area OF folks to the wmf office :-) Ryan Lane would certainly be happy to help | 14:04 |
| jeblair | hashar: ++ | 14:05 |
| *** tvb|afk has quit IRC | 14:06 | |
| jeblair | hashar: also, you should all come to the openstack summit :) | 14:07 |
| *** matsuhashi has quit IRC | 14:07 | |
| hashar | if you get one in europe, that should be manageable for me :-) | 14:07 |
| hashar | transatlic flights are expensive for small non-profit! | 14:08 |
| *** nosnos has quit IRC | 14:08 | |
| jeblair | hashar: the next one is in hong kong (in november), so it's not transatlantic for you! easy! :) | 14:08 |
| hashar | anyway, I am attending http://www.openworldforum.org/ in Paris Oct 3th-5th were there will be Openstack french people there. | 14:08 |
| *** ruhe has quit IRC | 14:09 | |
| mordred | ew. french people | 14:09 |
| jeblair | hashar: but i believe we would like to have one in europe one year after that. | 14:09 |
| * mordred agrees with jeblair - next fall _should_ in _theory_ be in the Europe | 14:10 | |
| hashar | \O/ | 14:10 |
| *** dkranz has quit IRC | 14:10 | |
| *** tvb|afk has joined #openstack-infra | 14:11 | |
| jeblair | mordred: i'm not certain 41945 is right | 14:13 |
| jeblair | mordred: i'll work on an example | 14:14 |
| soren | mordred: Yeah, I was just curious how Gerrit and echosign were integrated. | 14:14 |
| *** dcramer_ has joined #openstack-infra | 14:14 | |
| mordred | soren: in zero ways :) | 14:15 |
| *** prad has joined #openstack-infra | 14:15 | |
| *** tvb|afk has quit IRC | 14:16 | |
| *** rnirmal has joined #openstack-infra | 14:18 | |
| *** tvb|afk has joined #openstack-infra | 14:19 | |
| *** tvb|afk has joined #openstack-infra | 14:19 | |
| *** julim has joined #openstack-infra | 14:21 | |
| *** prad has quit IRC | 14:21 | |
| jeblair | mordred: regarding 41945 consider this case: http://paste.openstack.org/show/47499/ | 14:24 |
| *** tvb|afk has quit IRC | 14:24 | |
| *** kgriffs is now known as kgriffs_afk | 14:26 | |
| *** tvb|afk has joined #openstack-infra | 14:27 | |
| *** tvb|afk has joined #openstack-infra | 14:27 | |
| guitarzan | hey guys, are there gate jobs for already released versions? (stable/grizzly) | 14:27 |
| mordred | guitarzan: yes | 14:27 |
| fungi | guitarzan: yes, we currently gate stable backport patches on both grizzly and folsom | 14:28 |
| guitarzan | cool, is there a way to run those to see if upstream dependencies broke? | 14:28 |
| *** jcoufal_ has joined #openstack-infra | 14:28 | |
| *** ruhe has joined #openstack-infra | 14:28 | |
| guitarzan | I'm having a problem with cmd2/pyparsing conflicting | 14:28 |
| *** jcoufal has quit IRC | 14:28 | |
| *** tvb|afk has quit IRC | 14:28 | |
| fungi | guitarzan: we have periodic bitrot jobs which run, the trick is convincing people to keep an eye on the results | 14:28 |
| *** changbl has joined #openstack-infra | 14:28 | |
| mordred | jeblair: why do you think that master would be checked out on nova? | 14:28 |
| fungi | guitarzan: and yes, that pyparsing situation is a known problem we're currently trying to work through | 14:28 |
| guitarzan | fungi: ok, awesome | 14:29 |
| guitarzan | fungi: is this something I can see so I don't have to bug you? | 14:29 |
| mordred | jeblair: if BRANCH=stable/grizzly and ZUUL_BRANCH=stable/grizzly, then it should check out stable/grizzly, yeah? | 14:29 |
| jeblair | mordred: you're right; but that's still not (B) | 14:29 |
| jeblair | mordred: iow, i believe the problem is that it is ignoring changes ahead in the queue | 14:29 |
| fungi | guitarzan: its solution is nontrivial because we want to be able to test current client libraries with stable server releases for backward-compatibility, and at the moment there are dependency confilcts | 14:30 |
| mordred | AH. sorry, I misunderstood your paste | 14:30 |
| mordred | jeblair: let me relook | 14:30 |
| guitarzan | fungi: sure, I was just curious if there was a place I could see that you guys knew it was broken too | 14:30 |
| mordred | jeblair: in which way are those stacked? is that in zuul display order, so B depends on A | 14:30 |
| jeblair | mordred: new paste http://paste.openstack.org/show/47500/ | 14:30 |
| mordred | thanks for follwoing this thread and having a look on wireframes. Regarding the term 'resource class', the naming is what we got into during our initial intents. It's not final version, so if there are concerns, there is no problem in finding more accurate one (we just couldn't find better). As for resource class definition, I tried to explain it a bit more in reply to Rob's mail (in this thread), so | 14:30 |
| jeblair | mordred: yes | 14:30 |
| mordred | if you get that one, I hope it will help to answer and explain the concept of classes a little bit more. | 14:30 |
| mordred | gah | 14:30 |
| *** DennyZhang has quit IRC | 14:31 | |
| mordred | jeblair: gotcha. I see the logic hole you're talking about | 14:32 |
| jeblair | mordred: i'm looking at zuul's merger to see exactly what it provides | 14:32 |
| fungi | guitarzan: the openstack-dev mailing list/archives (there's at least one thread on it there) | 14:32 |
| guitarzan | ok, I'll try and watch that more closely, thanks | 14:32 |
| fungi | guitarzan: http://lists.openstack.org/pipermail/openstack-dev/2013-September/015542.html | 14:33 |
| *** datsun180b has joined #openstack-infra | 14:33 | |
| mordred | jeblair: so we additionally want to use zuul changes if those changes are for a stable branch | 14:33 |
| guitarzan | fungi: thank you | 14:33 |
| *** tvb|afk has joined #openstack-infra | 14:33 | |
| jeblair | mordred: ya | 14:33 |
| *** prad has joined #openstack-infra | 14:35 | |
| mordred | jeblair: so, I think what we need to do is not override zuul_branch | 14:35 |
| mordred | because we'll need to test for it | 14:35 |
| *** tvb|afk has quit IRC | 14:36 | |
| *** basha has quit IRC | 14:37 | |
| jeblair | mordred: i expect something like that. i need breakfast to think about this further though. | 14:37 |
| mordred | jeblair: lemme take a stab real quick | 14:38 |
| mordred | and se what you think | 14:38 |
| mordred | after breakfast | 14:38 |
| *** markmcclain has joined #openstack-infra | 14:39 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/config: Clarify that statusbot requires public messages https://review.openstack.org/48209 | 14:40 |
| *** prad has quit IRC | 14:41 | |
| openstackgerrit | Jay Bryant proposed a change to openstack/requirements: Add pywbem to global-requirements.txt https://review.openstack.org/48269 | 14:42 |
| *** salv-orlando has joined #openstack-infra | 14:43 | |
| *** basha has joined #openstack-infra | 14:43 | |
| *** basha has quit IRC | 14:45 | |
| mordred | zul: ^^ I'm going to defer to you on whether a FFE for that is possible | 14:46 |
| zul | mordred: eh? | 14:46 |
| mordred | zul: we're in requirements freeze - I figure that any FFE that's asked for needs to go through the distro people to check how much of a PITA it would be for you | 14:47 |
| *** tvb|afk has joined #openstack-infra | 14:47 | |
| *** tvb|afk has quit IRC | 14:48 | |
| *** tvb|afk has joined #openstack-infra | 14:48 | |
| zul | mordred: not cool for us if we ant to support EMC | 14:49 |
| mordred | zul: looks lke it's already in ubuntu, actually, I just checked | 14:49 |
| zul | mordred: yeah in universe | 14:49 |
| zul | mordred: meaning i have to do paperwork and pay for more beer | 14:50 |
| mordred | I just -1'd it with explanations | 14:50 |
| mordred | main one from me is "no releases directly on pypi, making use of external_links feature" | 14:50 |
| *** ruhe has quit IRC | 14:51 | |
| *** eharney has joined #openstack-infra | 14:51 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Add ability to do smart backwards compat testing https://review.openstack.org/41945 | 14:52 |
| mordred | jeblair: ^^ tell me what you think of that | 14:52 |
| jeblair | mordred: will do | 14:53 |
| *** amotoki has joined #openstack-infra | 14:53 | |
| *** rcleere has joined #openstack-infra | 14:54 | |
| *** changbl has quit IRC | 14:55 | |
| jeblair | mordred: i think altering CHECKOUT_ZUUL is the wrong approach | 14:57 |
| *** dcramer_ has quit IRC | 14:57 | |
| jeblair | mordred: you _want_ to checkout zuul refs, just ones from a different branch | 14:57 |
| *** cthulhup has joined #openstack-infra | 14:57 | |
| jeblair | for some projects | 14:57 |
| mordred | why would I want to check out zuul refs from a different branch? | 14:58 |
| jeblair | mordred: you want to checkout the future stable/grizzly state of nova | 14:58 |
| mordred | I only want to check out zuul refs if those refs are for the target stable branch | 14:58 |
| mordred | right | 14:58 |
| jeblair | mordred: in what case do you _not_ want to checkout a zuul ref? | 14:58 |
| mordred | if the zuul ref is for master and we're requesting stable/grizzly | 14:59 |
| jeblair | zuul refs are for any number of branches | 15:00 |
| *** DennyZhang has joined #openstack-infra | 15:00 | |
| *** cthulhup has quit IRC | 15:00 | |
| *** markmc has joined #openstack-infra | 15:01 | |
| jeblair | mordred: http://paste.openstack.org/show/47502/ | 15:01 |
| *** cthulhup has joined #openstack-infra | 15:01 | |
| jeblair | mordred: to continue the example from earlier -- those are the zuul refs that are created after change C is set up | 15:02 |
| jeblair | mordred: we want to checkout refs/zuul/stable/grizzly/ZC in the nova project | 15:02 |
| mordred | jeblair: how to we know if there will be one of those? | 15:02 |
| jeblair | mordred: (and obviously checkout refs/zuul/master/ZC in novaclient) | 15:02 |
| jeblair | mordred: we try to fetch it, and if it fails we know there isn't one. :) | 15:02 |
| *** changbl has joined #openstack-infra | 15:03 | |
| fungi | which is how devstack-gate already deals with zuul refs elsewhere. try to check them out, and if that fails fall back to the relevant branch tip | 15:03 |
| jeblair | yeah. so i think the character of this change should be about overriding how it constructs the zuul ref (to use the stable branch instead of the triggering branch) | 15:04 |
| mordred | jeblair: ok. let me take a stab at that | 15:05 |
| jeblair | mordred: fwiw, i'm becoming convinced that the grenade tests should be doing this too. | 15:05 |
| jeblair | (but aren't) | 15:06 |
| *** basha has joined #openstack-infra | 15:09 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Add ability to do smart backwards compat testing https://review.openstack.org/41945 | 15:11 |
| mordred | jeblair: ^^ I'm not thrilled with the if block execution - but is that along the logic lines? | 15:11 |
| *** yaguang has quit IRC | 15:12 | |
| mordred | (before I work on making the if section more elegant) | 15:12 |
| jeblair | omg i read email | 15:14 |
| mordred | jeblair: which email? | 15:17 |
| jeblair | openid | 15:17 |
| mordred | yeah. that quickly became way more emails | 15:18 |
| *** dkranz has joined #openstack-infra | 15:19 | |
| jd__ | if some of you have time for a requirement change review on https://review.openstack.org/#/c/47994/, it's blocking Heat guys to fix bugs | 15:19 |
| sdague | lgtm | 15:20 |
| Alex_Gaynor | Approved | 15:20 |
| Alex_Gaynor | (now enjoy waiting in the gate queue :P) | 15:20 |
| jd__ | Alex_Gaynor: thanks (for the approval, not for the wait queue :-) | 15:21 |
| Alex_Gaynor | jd__: wait queue isn't my fault ;) | 15:21 |
| jd__ | Alex_Gaynor: I don't wanna know ;) | 15:22 |
| *** dcramer_ has joined #openstack-infra | 15:22 | |
| therve | So hum, talking about the gate | 15:22 |
| therve | Can we do something about the tests that have tons of rechecks? | 15:23 |
| Alex_Gaynor | therve: Yell at the people to fix the tests is my preferred approach (well, maybe help them). I don't like skips, because they tend to never get un-skipped | 15:23 |
| Alex_Gaynor | IMO projects with known flaky tests should probably stop everything and fix them, but I don't have teh authority to say things like that :) | 15:24 |
| therve | Alex_Gaynor, Well, I kind of agree about skips, but I don't know who to yell at | 15:24 |
| therve | Also, it feels some tests fail because of infra problems, not really a test problem? | 15:25 |
| therve | Maybe the tests should be more resilient | 15:25 |
| Alex_Gaynor | Most of the ones I see are iether a bug in tempest or a bug in what tempest is testing. | 15:25 |
| Alex_Gaynor | Mostly race conditions, sadly. | 15:25 |
| jd__ | therve: I don't know more than you about this, but I'm starting to be pissed off about it indeed | 15:25 |
| therve | bug #1223161 and bug #1226337 are 2 which are weird | 15:25 |
| uvirtbot | Launchpad bug 1223161 in nova "FAIL: tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_stop_terminate_instance" [Undecided,Confirmed] https://launchpad.net/bugs/1223161 | 15:25 |
| uvirtbot | Launchpad bug 1226337 in tempest "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern flake failure" [High,Triaged] https://launchpad.net/bugs/1226337 | 15:25 |
| *** basha has quit IRC | 15:26 | |
| therve | jd__, Yeah this week has been exceptionally bad. | 15:26 |
| therve | I don't know if it's because people are shoving changes into the respective projects or what | 15:26 |
| Alex_Gaynor | Some of that's just due to teh high volume of patches in general. As we head towards the release there's just more patches up in the air. | 15:27 |
| jd__ | at least I wish we had a way to revert what introduced these tests or bugs | 15:27 |
| fungi | i have to disagree, being on the infra side of things and investigating may of the testing-related bug reports which just get blindly assigned to openstack-ci because the reporter doesn't know how to read/interpret logs, most of them are in no way related to the servers these tests are running on and most are either the tests themselves, the projects being tested, or the changes being evaluated | 15:27 |
| Alex_Gaynor | therve: when are you next in town btw? | 15:27 |
| fungi | s/may/many/ | 15:27 |
| jd__ | fungi: yeah I don't think infra is responsible since it always the same tests failing | 15:28 |
| *** hashar has quit IRC | 15:28 | |
| therve | Alex_Gaynor, Week of Oct 7th :) | 15:29 |
| therve | fungi, Sorry I didn't mean to throw the blame, I was just wondering | 15:29 |
| fungi | part of the reason for recheck tracking is to encourage people to figure out where the bug they're tickling actually resides. if the temptation is to assign it to openstack-ci because the developer can't figure out where the bug is, that's pretty much the wrong solution | 15:30 |
| fungi | therve: no blame assumed! | 15:31 |
| therve | I thought rechecks were there to know how much a problem was appearing | 15:31 |
| fungi | therve: just pointing out a common misconception, which i think is in part due to the image most devs have of the testing apparatus as a big black box, no matter how much we try to document and code it all completely i the open | 15:32 |
| jd__ | The problem is that things like https://bugs.launchpad.net/tempest/+bug/1226337 just have nobody assigned and it's here for a week and we don't even know why it started | 15:32 |
| uvirtbot | Launchpad bug 1226337 in tempest "tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern flake failure" [High,Triaged] | 15:32 |
| jd__ | we should have a giant freeze button to stop everything until such regression is fixed | 15:32 |
| fungi | and yes, tracking frequency is also a big part of its reason for existing (which is why i said "part") | 15:32 |
| *** dizquierdo has left #openstack-infra | 15:33 | |
| *** mrodden has quit IRC | 15:33 | |
| therve | fungi, OK gotcha | 15:33 |
| *** svarnau has joined #openstack-infra | 15:34 | |
| fungi | jd__: agreed on 1226337. hopefully as we get more attuned to researching these issues through analysis tools like logstash, we'll be able to narrow down culprit changes in projects and be able to revert things more aggressively in the future | 15:37 |
| fungi | with gating, we have deterministic failures more or less nailed (with the exception of outside influences like fuzzy dependencies), but nondeterminism is much tougher | 15:38 |
| jd__ | fungi: that's cool if we'll have tool to help to narrow this | 15:39 |
| jd__ | from where I stand, I just don't even know where to start to help fixing such a bug | 15:40 |
| *** reed has joined #openstack-infra | 15:42 | |
| mordred | jd__: jog0 and mtreinish have been working on automatically looking at failures using logstash to match failures to known bugs | 15:44 |
| *** mrodden has joined #openstack-infra | 15:44 | |
| fungi | i think at the moment being able to identify symptoms with at least sufficient granularity to pinpoint a particular project or set of projects contributing to that interaction along with the earliest known occurrence attributable to it should narrow the field of suspect commits considerably | 15:47 |
| *** UtahDave has joined #openstack-infra | 15:49 | |
| *** tvb|afk has quit IRC | 15:52 | |
| *** tvb|afk has joined #openstack-infra | 15:56 | |
| *** julim has quit IRC | 15:57 | |
| *** tvb|afk has quit IRC | 15:59 | |
| ttx | mordred: planning to kill the last -core PPAs, please make sure you see my last post to -dev | 16:00 |
| mordred | ttx: saw it. agree. kill them. | 16:01 |
| mordred | ttx: the openstack project has not produced packages in a LONG time | 16:01 |
| mordred | in fact, I believe it's been at least 2 years | 16:01 |
| ttx | right :) and anyone still using them.. shouldn't | 16:02 |
| mordred | yah | 16:02 |
| *** jcoufal_ is now known as jcoufal | 16:02 | |
| dkranz | jd__, fungi : Right now there are a bunch of bugs that started showing up when we started running the gate jobs with parallel tempest | 16:03 |
| dkranz | They may not be traceable to a code change at this point. | 16:03 |
| dkranz | Hopefully this is a transient situation and new bugs that show up will be more easily traced even if flaky | 16:03 |
| *** SergeyLukjanov has joined #openstack-infra | 16:05 | |
| fungi | dkranz: agreed, this was a known risk and expected consequence of parallel tempest. on a positive note, it didn't really add bugs so much as it exposed already lurking ones | 16:06 |
| fungi | and also i suspect the decreased runtime for those jobs has at least completely offset the delays introduced by exposing those bugs which should be fixed by someone anyway. net win | 16:07 |
| *** sarob has joined #openstack-infra | 16:07 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/storyboard: Update setup code past d2to1 https://review.openstack.org/48291 | 16:07 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/storyboard: Update tox config to use 1.6 https://review.openstack.org/48292 | 16:07 |
| mordred | ttx: ^^ | 16:07 |
| *** yassine has quit IRC | 16:07 | |
| dkranz | fungi: Agreed. It would be nice though if these bugs could get more attention. | 16:08 |
| dkranz | fungi: But they are hard to track down and the customer value if fixing them is unclear. | 16:08 |
| SergeyLukjanov | mordred, hi, are there any questions aboutt stackalytics? | 16:08 |
| SpamapS | is there somewhere that measures the overall gate fail rate? | 16:08 |
| mordred | SergeyLukjanov: figured it out | 16:09 |
| mordred | SpamapS: we have the stats for _everytihng_ in graphite | 16:09 |
| ttx | mordred: +2/APRVed | 16:09 |
| mordred | SergeyLukjanov: I even sent in some patches! | 16:09 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Update setup code past d2to1 https://review.openstack.org/48291 | 16:09 |
| SergeyLukjanov | mordred, k | 16:09 |
| *** tvb|afk_ has joined #openstack-infra | 16:09 | |
| mordred | SpamapS: I'm not sure we have a ready-made graph for that data though | 16:09 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Update tox config to use 1.6 https://review.openstack.org/48292 | 16:09 |
| *** rpodolyaka has left #openstack-infra | 16:10 | |
| *** rpodolyaka has quit IRC | 16:10 | |
| SpamapS | mordred: It would be very interesting to see, because right now it feels like we're at 20% fail rate. | 16:12 |
| SpamapS | I haven't had a single patch go through without a tempest fail the last week. | 16:12 |
| mordred | SpamapS: http://graphite.openstack.org/ | 16:12 |
| mordred | may or may not be your friend | 16:13 |
| pleia2 | morning | 16:13 |
| SpamapS | mordred: pretty interface, but not seeing data in the gate-tempest-* builds | 16:14 |
| *** tvb|afk_ has quit IRC | 16:15 | |
| *** ndipanov has quit IRC | 16:15 | |
| jog0 | clarkb: thanks | 16:16 |
| jeblair | SpamapS: http://graphite.openstack.org/compose/?width=586&height=308&_salt=1380125892.94&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.FAILURE%2C%221h%22%29&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.SUCCESS%2C%221h%22%29 | 16:17 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/zuul: Remove usage of d2to1 https://review.openstack.org/48294 | 16:18 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/zuul: Partial sync with OpenStack requirements. https://review.openstack.org/48295 | 16:18 |
| SpamapS | jeblair: "no data" | 16:19 |
| SpamapS | do I have to log in? | 16:19 |
| jeblair | weird | 16:19 |
| *** julim has joined #openstack-infra | 16:19 | |
| mordred | jeblair: same here | 16:19 |
| jeblair | SpamapS: http://graphite.openstack.org/render/?width=586&height=308&_salt=1380125892.94&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.FAILURE%2C%221h%22%29&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.SUCCESS%2C%221h%22%29 | 16:19 |
| jeblair | s/render/compose/ used to work; perhaps that's changed. | 16:19 |
| jeblair | we are running it CD. | 16:19 |
| mordred | render works for me | 16:20 |
| *** primeministerp has quit IRC | 16:20 | |
| SergeyLukjanov | could please anyone help me with the uploading to pypi problem? I've pushed a 0.2.2 tag to savanna repos about eight hours ago, jobs are in success status but there are no 0.2.2 version at pypi | 16:20 |
| SpamapS | "render" ? | 16:21 |
| jeblair | SpamapS: http://graphite.openstack.org/render/?width=586&height=308&_salt=1380125892.94&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.FAILURE%2C%221h%22%29&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.SUCCESS%2C%221h%22%29 | 16:21 |
| *** julim has quit IRC | 16:21 | |
| mordred | SergeyLukjanov: looking | 16:21 |
| mordred | SergeyLukjanov: all of the savana repos? | 16:22 |
| SpamapS | jeblair: ok | 16:22 |
| jeblair | oh, it's composer | 16:22 |
| jeblair | SpamapS: http://graphite.openstack.org/composer/?width=586&height=308&_salt=1380125892.94&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.FAILURE%2C%221h%22%29&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.SUCCESS%2C%221h%22%29 | 16:22 |
| jeblair | mordred: ^ | 16:22 |
| mordred | jeblair: :) | 16:22 |
| SpamapS | so in a few time periods we had "as many fails as successes" :( | 16:22 |
| SergeyLukjanov | mordred, thank you! repos: savanna, savanna-dashboard, savanna-extra and python-savannaclient | 16:22 |
| jeblair | SpamapS: yeah, and if you look back a bit, you can see it's definitely increased: http://graphite.openstack.org/composer/?width=586&_salt=1380126254.968&height=308&from=-14days&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.FAILURE%2C%2224h%22%29&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.SUCCESS%2C%2224h%22%29 | 16:24 |
| mordred | https://jenkins.openstack.org/job/savanna-pypi-upload/6/console | 16:24 |
| mordred | that looks like we successfully uploaded to pypi | 16:24 |
| mordred | dstufft: ^^ ?? | 16:24 |
| mordred | clarkb: ? | 16:24 |
| SpamapS | btw its working w/o render now | 16:24 |
| jeblair | SpamapS: yeah, i typos compose instead of composer | 16:25 |
| dstufft | mordred: hrm? | 16:25 |
| jeblair | mtreinish, sdague: http://graphite.openstack.org/composer/?width=586&_salt=1380126254.968&height=308&from=-14days&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.FAILURE%2C%2224h%22%29&target=summarize%28stats_counts.zuul.pipeline.gate.job.gate-tempest-devstack-vm-full.SUCCESS%2C%2224h%22%29 | 16:25 |
| mordred | dstufft: I'm looking in to a thing that looks like it thinks it may have uploaded to pypi without an error, but nothing is showing up there | 16:25 |
| mordred | dstufft: how hard is it to see from your end if we did, in fact, upload something to you>/ | 16:26 |
| dstufft | logging sucks on PyPI, I can take a look and see if anything shows up but other than exceptions we don't do much logging | 16:27 |
| SpamapS | btw, kudos for having this all in graphite. really nice to have this to poke around at | 16:27 |
| *** hemnafk is now known as hemna | 16:28 | |
| SergeyLukjanov | mordred, dstufft, there are now new version in all (4) projects at pypi, that's very strange | 16:28 |
| *** odyssey4me has quit IRC | 16:28 | |
| dstufft | we set cache headers on pypi | 16:28 |
| dstufft | you might have had it cached in your browser | 16:28 |
| dstufft | what page were you looking at? | 16:28 |
| mordred | ah | 16:28 |
| mordred | https://pypi.python.org/pypi/savanna | 16:29 |
| dstufft | we have varnish caching too but we invalidate the caching on that | 16:29 |
| mordred | SergeyLukjanov: I do not see 0.2.2 | 16:29 |
| dstufft | yea I don't see a 0.2.2 either | 16:29 |
| jeblair | also https://pypi.python.org/pypi/savanna/0.2.2 does not exist | 16:29 |
| SergeyLukjanov | yep, that's why I'm asking you guys about it | 16:29 |
| dstufft | when did you release it | 16:30 |
| *** svarnau_ has joined #openstack-infra | 16:31 | |
| clarkb | morning | 16:31 |
| *** fbo is now known as fbo_away | 16:32 | |
| clarkb | mordred: we kill curl output pretty aggressively, jenkins may think we uploaded a thing but it failed | 16:32 |
| jeblair | we should write a script that does what we need | 16:32 |
| clarkb | we can probably stop doing that if we put more restrictive permissions around who can view the jenkins job output | 16:32 |
| jeblair | --restrictions ++script that doesn't output credentials on error | 16:33 |
| *** primeministerp has joined #openstack-infra | 16:33 | |
| clarkb | ++ | 16:33 |
| dstufft | I really need to take a look at what interesting things openstack has done with logging infra to see what sort of cool things I can do with pypi logs so I can get some motiivation :V | 16:33 |
| *** svarnau has quit IRC | 16:34 | |
| dstufft | btw where is your curl thing that uploads to pypi defined at? is that in git or something where | 16:35 |
| clarkb | dstufft: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/pypi-upload.sh#n28 | 16:36 |
| dstufft | thanks! | 16:37 |
| SergeyLukjanov | we have a bad karma with pushing releases to pypi… some problems each time :) | 16:38 |
| *** tstevenson has joined #openstack-infra | 16:40 | |
| *** sdake__ is now known as sdake_ | 16:40 | |
| *** markmc has quit IRC | 16:41 | |
| jeblair | mordred: examined https://review.openstack.org/#/c/41945/ | 16:42 |
| SergeyLukjanov | maybe there was some problems with pypi when i've pushed tag... | 16:46 |
| LinuxJedi | yay! I'm now listed as HP, in the top 10 of HP and pushed HP to #1 in commits | 16:46 |
| SergeyLukjanov | maybe jobs retriggering will help? | 16:46 |
| LinuxJedi | now to get myself fired... | 16:46 |
| clarkb | dstufft: the reason that thing exists is there doesn't appear to be a tool that takes an sdist and puts it on pypi without running arbitrary code | 16:49 |
| clarkb | and I am caught up on scrollback | 16:49 |
| clarkb | I am going to rerun manage_projects on review.o.o as `manage-projects -v 'openstack-infra/elastic-recheck'` | 16:50 |
| dstufft | clarkb: devpi-client might do it | 16:50 |
| dstufft | but no idea if it will or not | 16:51 |
| dstufft | sometime in the near future I plan on a redesigned upload API and will be releasing a client that works like that for it | 16:51 |
| *** kong has quit IRC | 16:52 | |
| *** kong has joined #openstack-infra | 16:52 | |
| clarkb | ooh devpi-client may do it | 16:53 |
| *** afazekas_ has quit IRC | 16:54 | |
| clarkb | nope nevermind | 16:54 |
| dstufft | devpi-client is designed to work with devpi-server, I don't know if it'll let you shortcircuit the process | 16:54 |
| clarkb | I have no idea why we all think this workflow is ok | 16:54 |
| dstufft | clarkb: stockholm | 16:54 |
| *** sarob has quit IRC | 16:54 | |
| *** sarob has joined #openstack-infra | 16:55 | |
| dstufft | I need to convince someone to give me minions who will obey me and implement things I tell them too so I can make this "fix the world" process go faster | 16:55 |
| *** marun has joined #openstack-infra | 16:55 | |
| clarkb | we could use it if we set up a devpi-server that we pushed to instaed of tarballs | 16:55 |
| clarkb | or in addition to tarballs.o.o then push from there to pypi, but it is still running arbitrary and semi priveleged code somewhere | 16:56 |
| dstufft | well you have to run arbitrary code somewhere to create the package | 16:57 |
| clarkb | right, but with the magic of jenkins I can do that in isolation and use a thing detached from the process to move the file around | 16:57 |
| clarkb | we could make this a three step process, build sdist and copy to tarballs, push to devpi-server, push to pypi | 16:58 |
| dstufft | yea | 16:58 |
| dstufft | fuck this packaging story | 16:58 |
| dstufft | somedy it'll be better | 16:58 |
| dstufft | I hope | 16:58 |
| dstufft | :[ | 16:58 |
| jeblair | clarkb: is that easier than writing a pycurl script? | 16:58 |
| *** Kiall2 has joined #openstack-infra | 16:58 | |
| *** svarnau has joined #openstack-infra | 16:58 | |
| dstufft | probably not | 16:58 |
| *** Kiall2 has quit IRC | 16:59 | |
| *** Kiall2 has joined #openstack-infra | 16:59 | |
| *** kiall has quit IRC | 16:59 | |
| *** Kiall2 is now known as Kiall | 16:59 | |
| dstufft | write a thing that does this and release it on pypi so I can use it and don't have to write my own | 16:59 |
| dstufft | I M O | 16:59 |
| fungi | clarkb: in previous situations with this symptom, most of the time it's been pypi rejecting the post because of a bad trove classifier | 16:59 |
| *** sarob has quit IRC | 17:00 | |
| fungi | i've usually troubleshot it by rerunning a modified copy of the upload script in the leftover workspace on the upload slave | 17:00 |
| clarkb | jeblair: I agree with dstufft, just brainstorming around existing tools | 17:00 |
| clarkb | there is also a bulk upload option that might work, (it seems to deal with files rather than indexes) | 17:00 |
| dstufft | I thought devpi-client might allow you t do it directly :[ I havne't looked super close to it | 17:00 |
| *** svarnau_ has quit IRC | 17:01 | |
| clarkb | dstufft: ya, there may be a hidden flag or something used for another purpose eg bulkthat will work | 17:01 |
| *** jcoufal has quit IRC | 17:01 | |
| *** matty_dubs is now known as matty_dubs|lunch | 17:01 | |
| dstufft | clarkb: I asked in the channel | 17:01 |
| dstufft | #devpi | 17:01 |
| *** sandywalsh has quit IRC | 17:02 | |
| dstufft | holger is away ATM though so dunno if anyone will answer right now | 17:02 |
| clarkb | thanks | 17:02 |
| *** derekh has quit IRC | 17:03 | |
| clarkb | ok actually kicking manage_projects now | 17:04 |
| *** mkirk_ has quit IRC | 17:04 | |
| dstufft | devpi upload --from-dir | 17:04 |
| dstufft | looks like it would do it | 17:05 |
| *** mkirk_ has joined #openstack-infra | 17:05 | |
| dstufft | I'll try it out! | 17:05 |
| clarkb | dstufft: ya, the only thing with that was how to tell it pypi should be the destination as `push` seems to be the verb for talking to external indexes | 17:06 |
| *** Ryan_Lane has joined #openstack-infra | 17:06 | |
| *** Ryan_Lane has joined #openstack-infra | 17:06 | |
| dstufft | yea i'm looking at it now | 17:06 |
| *** eharney has quit IRC | 17:06 | |
| *** davidhadas_ has quit IRC | 17:07 | |
| fungi | clarkb: mind if i retry the savanna upload manually to see what's breaking it this time? | 17:08 |
| dstufft | bleh | 17:08 |
| dstufft | it doesn't work on python3 | 17:08 |
| dstufft | gotta switch my python | 17:08 |
| clarkb | fungi: go for it | 17:08 |
| SergeyLukjanov | fungi, could you please post link to the job?) | 17:08 |
| *** jhesketh_ has quit IRC | 17:09 | |
| SergeyLukjanov | I think that I've pushed tag in that time when pypi was down... | 17:09 |
| SergeyLukjanov | oh | 17:09 |
| SergeyLukjanov | it's working ok | 17:09 |
| SergeyLukjanov | https://pypi.python.org/pypi/savanna/0.2.2 | 17:09 |
| SergeyLukjanov | could you please retrigger jobs for savanna-extra, savanna-dashboard and python-savannaclient too? | 17:10 |
| mordred | fungi: while you're in as that user, savanna-extra needs registering with pypi | 17:10 |
| *** sarob has joined #openstack-infra | 17:10 | |
| *** sarob has quit IRC | 17:10 | |
| fungi | SergeyLukjanov: i ran the script by hand, modified to output any errors to my terminal, so no associated job but basically retrying the pypi-upload.sh run from https://jenkins.openstack.org/job/savanna-pypi-upload/6/console | 17:10 |
| SergeyLukjanov | oops, I'm sorry, no need to release savanna-extra | 17:10 |
| fungi | mordred: not in as any sort of user, just running the upload script on the slave | 17:11 |
| *** sarob has joined #openstack-infra | 17:11 | |
| fungi | SergeyLukjanov: and yes, no errors, so i think pypi.python.org may have been having some sort of intermittent issue, or our access to it was (network trouble, something) | 17:11 |
| SergeyLukjanov | fungi, great, thank you | 17:12 |
| clarkb | it created the second group this time around, but did not push the ACL file best I can tell. Still digging. manage-projects -v is really verbose :) | 17:12 |
| fungi | SergeyLukjanov: so would you like me to do the same for savanna-dashboard and python-savannaclient i guess? | 17:13 |
| SergeyLukjanov | fungi, yep, it'll be awesome | 17:13 |
| fungi | SergeyLukjanov: no errors from any of the three, so definitely something intermittent there | 17:14 |
| SergeyLukjanov | fungi, thank you, looks correct - https://pypi.python.org/pypi?%3Aaction=search&term=savanna&submit=search | 17:15 |
| fungi | i wonder if the upload job should try to find the upload in pypi's index, and retry every 10 seconds for up to 5 minutes or something until it shows up, then exit the script with a nonzero exit code if it fails... | 17:16 |
| *** afazekas has joined #openstack-infra | 17:16 | |
| dstufft | clarkb: fuck it, I want this too so i'm gonna make it. What's you're wishlist for a upload to pypi utility | 17:17 |
| dstufft | your* | 17:17 |
| dstufft | fungi: pypi was having issues last night | 17:17 |
| dstufft | well all of python.org was | 17:17 |
| dstufft | the LB's both decided they should be leader | 17:17 |
| dstufft | and started fighting over the floating ip | 17:17 |
| *** sandywalsh has joined #openstack-infra | 17:18 | |
| clarkb | dstufft: given an sdist (or other package artifact), inflate the file, use the meta data already encoded in it to push to pypi where creds can be in a pypirc | 17:18 |
| fungi | dstufft: upload a tarball and associated metadata and detached signature(s), validate the ssl cert on the endpoint before submitting credentials, don't spew sensitive info in exceptions, optionally confirm the upload actually shows up | 17:18 |
| clarkb | dstufft: very similar the current process less evaluating any code that isn't part of the uploaders codebase | 17:18 |
| dstufft | sounds like my wishlist, ok | 17:19 |
| fungi | heh | 17:19 |
| dstufft | ``twine upload`` is gonna be a thing | 17:19 |
| dstufft | because i'm a sucker for cutesy names | 17:19 |
| fungi | it's scary we all want the same thing. almost as if it needs to exist | 17:19 |
| *** basha has joined #openstack-infra | 17:20 | |
| dstufft | almost as if executing random python just to upload a thing is a terrible idea | 17:20 |
| fungi | yes, that | 17:20 |
| SergeyLukjanov | guys, I hope that https://review.openstack.org/#/c/37620/ is ready | 17:20 |
| SergeyLukjanov | but it's now missing tempest in direct list | 17:20 |
| SergeyLukjanov | maybe it's time to review/merge it and then fix this list? | 17:21 |
| *** jhesketh_ has joined #openstack-infra | 17:21 | |
| fungi | SergeyLukjanov: yes, i agree. i am going to make time today to go over that | 17:21 |
| SergeyLukjanov | cool, thanks, I hope that it'll help to remove duplicated info | 17:22 |
| SergeyLukjanov | and we'll be able to use some new attrs | 17:22 |
| *** MarkAtwood has joined #openstack-infra | 17:25 | |
| *** tvb|afk has joined #openstack-infra | 17:26 | |
| clarkb | create_groups_file appears to be throwing an exception, I will log that better | 17:27 |
| mriedem | hi, i have a stable/grizzly cherry pick for nova that's failing on a pyparsing requirement: "error: Installed distribution pyparsing 1.5.7 conflicts with requirement pyparsing>=2.0.1" | 17:27 |
| mriedem | with the issues yesterday, wondering if anyone knows what that is | 17:27 |
| mriedem | https://review.openstack.org/#/c/48300/ | 17:27 |
| mriedem | looks like it's because a restriction in python-quantumclient for grizzly | 17:29 |
| mriedem | Downloading/unpacking pyparsing>=1.5.6,<2.0 (from python-quantumclient>=2.2.0,<3.0.0->-r nova.egg-info/requires.txt (line 24)) | 17:29 |
| sdague | python-quantumclient isn't installing from git? | 17:29 |
| mriedem | sdague: doesn't look like it: "Storing download in cache at /var/cache/pip/http%3A%2F%2Fpypi.openstack.org%2Fopenstack%2Fpython-quantumclient%2Fpython-quantumclient-2.2.3.tar.gz" | 17:30 |
| sdague | oh, grrr... the rename | 17:30 |
| *** tvb|afk has quit IRC | 17:30 | |
| sdague | markmcclain you about for this one? | 17:30 |
| sdague | so because stable nova requires python-quantumclient (not python-neutron client) | 17:31 |
| dstufft | clarkb: fungi https://github.com/dstufft/twine | 17:31 |
| dstufft | is where it'll be | 17:31 |
| mordred | sdague: headdesk | 17:32 |
| markmcclain | sdague: yes working on it | 17:32 |
| *** SergeyLukjanov has quit IRC | 17:32 | |
| mriedem | markmcclain: is there a bug or should i open one? | 17:33 |
| markmcclain | mriedem: there is a bug | 17:33 |
| * markmcclain looks for it | 17:33 | |
| markmcclain | https://bugs.launchpad.net/python-neutronclient/+bug/1230397 | 17:33 |
| uvirtbot | Launchpad bug 1230397 in python-neutronclient "quantumclient: pyparsing in requirements.txt is causing gate problems for stable/grizzly ceilometer" [Undecided,New] | 17:33 |
| mriedem | markmcclain: great, thanks | 17:33 |
| sdague | markmcclain: can we just change nova stable/grizzly to reference neutron client? | 17:33 |
| *** sandywalsh_ has joined #openstack-infra | 17:34 | |
| *** cthulhup has quit IRC | 17:34 | |
| markmcclain | sdague: we could | 17:35 |
| markmcclain | would that be allowable under stable? | 17:35 |
| *** matty_dubs|lunch is now known as matty_dubs | 17:35 | |
| mtreinish | jeblair: I'm guessing that peak in failures on the graph yesterday was because of the gate breakage with pyparsing? | 17:35 |
| sdague | well... how screwed are we going to be on the symbols? | 17:35 |
| markmcclain | it will be messy | 17:36 |
| *** tvb|afk has joined #openstack-infra | 17:37 | |
| sdague | yeh, so the newclients + old servers exposed this | 17:37 |
| *** sandywalsh has quit IRC | 17:38 | |
| clarkb | jeblair: mordred: fungi: jog0 I think the bug is in creating new groups and getting their UUIDs (race condition maybe) I have a change to manage_projects that should make the logging much more verbose to help debug | 17:41 |
| jeblair | clarkb: cool, thanks | 17:42 |
| clarkb | the first time it ran the -ptl group was created then it died, the second time it ran the -core group was created then it died, the third time it ran it managed to push the ACLs | 17:43 |
| clarkb | so elastic-recheck is set now. | 17:43 |
| mordred | clarkb: I've seen that before - I think that create group returns before it's really done | 17:43 |
| mtreinish | clarkb: ok cool | 17:44 |
| sdague | mordred: ok, neutronclient switch is going to be too hard I think | 17:44 |
| clarkb | mordred: :/ is that a bug in gerrit or gerritlib? any idea? | 17:44 |
| mordred | sdague: update the depends in quantumclient? | 17:44 |
| mordred | clarkb: gerrit I think? | 17:44 |
| mordred | clarkb: maybe add a sleep(1) ... | 17:45 |
| jeblair | or a busy loop around ls-groups? | 17:45 |
| *** julim has joined #openstack-infra | 17:45 | |
| mordred | jeblair: ++ | 17:45 |
| sdague | mordred: and make a new quantumclient release? | 17:45 |
| mordred | sdague: sigh. gross. | 17:46 |
| mordred | going afk for a minute | 17:46 |
| sdague | we don't get quantumclient out of git... because we removed that | 17:46 |
| sdague | so new clients + old servers sounds good, until those clients are used by old servers to call each other | 17:47 |
| jeblair | https://wiki.openstack.org/wiki/Network/neutron-renaming | 17:47 |
| jeblair | sdague, mordred, markmcclain: did steps 6, 7, 8 happen? | 17:47 |
| *** johnthetubaguy has quit IRC | 17:47 | |
| *** kiall_ has joined #openstack-infra | 17:48 | |
| jeblair | (because if they did, i don't see how quantumclient should be pip-requiring anything other than neutronclient) | 17:48 |
| sdague | jeblair: the issue is nova stable/grizzly requires quantumclient | 17:48 |
| jeblair | sdague: yep, which according to the plan should at this point be nothing but a shim for neutronclient | 17:49 |
| jeblair | (see step 6) | 17:49 |
| *** rwsu-away is now known as rwsu | 17:49 | |
| *** vipul is now known as vipul-away | 17:49 | |
| *** vipul-away is now known as vipul | 17:49 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gearman-plugin: Set a node offline even if there is an exception https://review.openstack.org/46124 | 17:54 |
| markmcclain | working to make the late steps happen | 17:54 |
| jeblair | markmcclain: i expect an infra person will need to make the pypi upload manually; just let us know when it's ready | 17:55 |
| markmcclain | jeblair: willl do | 17:56 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gearman-plugin: Set a node offline even if there is an exception https://review.openstack.org/46124 | 18:00 |
| *** melwitt has joined #openstack-infra | 18:00 | |
| *** prad has joined #openstack-infra | 18:00 | |
| *** rnirmal has quit IRC | 18:01 | |
| fungi | reed: did those atc lists pass the sniff test for you at least? seem okay? need anything rerun? | 18:01 |
| fungi | (from last night) | 18:02 |
| *** afazekas is now known as afazekas_away | 18:02 | |
| jeblair | fungi: ooh, what are the numbers? oh, and do i need to send you my most recent all.csv so you can diff? | 18:02 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gearman-plugin: Set a node offline even if there is an exception https://review.openstack.org/46124 | 18:03 |
| *** julim has quit IRC | 18:03 | |
| jeblair | (a very confusing rebase ^) | 18:04 |
| *** gyee has joined #openstack-infra | 18:05 | |
| fungi | jeblair: for the date range starting september 27, 2012 up to but not including september 6, 2013 there were 1018 active technical contributors (and 1064 up through yesterday) | 18:05 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/gerritlib: Add method to list Gerrit Groups. https://review.openstack.org/48312 | 18:05 |
| fungi | jeblair: and yes, i'd love to diff from your previous run | 18:05 |
| *** julim has joined #openstack-infra | 18:05 | |
| fungi | also, thrilled to see our 1k atc estimate was met and even exceeded slightly! | 18:06 |
| clarkb | nice | 18:06 |
| jeblair | fungi: sent | 18:07 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/jeepyb: Use python logging isntead of print. https://review.openstack.org/48313 | 18:07 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/jeepyb: Wait for groups to be created before getting uuids https://review.openstack.org/48314 | 18:08 |
| clarkb | 48314 requires the gerritlib 48312 change | 18:08 |
| clarkb | and of course my pep8 is not good | 18:09 |
| jeblair | mordred, clarkb, fungi: how does https://review.openstack.org/#/c/40068/ relate to branches? | 18:09 |
| *** julim has quit IRC | 18:10 | |
| fungi | jeblair: the thought is that with the list of projects living in the requirements repo, the list can be branched along with that repo when havana is released | 18:10 |
| *** Kiall has quit IRC | 18:11 | |
| jeblair | fungi: right, but i don't see anything about it proposing changes to particular branches.. am i missing something? | 18:11 |
| fungi | jeblair: tooling would still need to exist to take advantage of that, but was mainly concerned that if the list of projects is embedded in that script, it's harder to deal with later | 18:11 |
| jeblair | eg, if someone updates stable/foo in requirements, it should only update the requirements for the stable/foo branches of the projects in projects.txt | 18:11 |
| fungi | that's the thought, though i think for the moment this job is designed to only run for master | 18:12 |
| jeblair | but unless i'm missing something, that will propose stable/foo requirements changes to master branches of projects | 18:12 |
| fungi | oh, though it's not limited to master in layout.yaml, and probably should be | 18:12 |
| jeblair | ok, i'm not missing something. will -1. | 18:13 |
| fungi | yep, great catch | 18:14 |
| *** jaypipes has quit IRC | 18:17 | |
| *** SergeyLukjanov has joined #openstack-infra | 18:17 | |
| *** sarob has quit IRC | 18:20 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/jeepyb: Use python logging isntead of print https://review.openstack.org/48313 | 18:21 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/jeepyb: Wait for groups to be created before getting uuids https://review.openstack.org/48314 | 18:21 |
| jeblair | clarkb: * modules/etherpad_lite/manifests/apache.pp: Fix broken /etc/ssl/certs | 18:26 |
| jeblair | permissions (0700 -> 0755). | 18:26 |
| jeblair | clarkb: in https://review.openstack.org/#/c/45360/5 | 18:26 |
| jeblair | clarkb: how is that broken? | 18:27 |
| clarkb | jeblair: because the system trusted CA files are in that dir, so to do anything via SSL as not root you need to be able to read things there | 18:27 |
| jeblair | clarkb: gotcha | 18:27 |
| fungi | jeblair: i am seeing some text encoding irregularities... the lists i generated contained some non-ascii characters and were encoded iso-8859-1 (i think because of the current mysql server configuration on review.o.o), while what you sent me was in plain ascii. also python's csv writer seems to use dos-style crlf eol markers (wtf?) | 18:28 |
| jeblair | /etc/ssl/private is the one that should be 700 | 18:28 |
| jeblair | fungi: neat! | 18:29 |
| fungi | jeblair: wondering if maybe your mua munged the list you sent... could you try attaching it multipart-mime? | 18:30 |
| fungi | the diff.py script reports 140 lines between them, but wc -l says it should only be 40 lines difference | 18:31 |
| jeblair | ha! | 18:31 |
| jeblair | fungi: i had an empty message so it elected not to mime encode it. | 18:31 |
| fungi | ahh, yep. that makes sense | 18:31 |
| *** sarob has joined #openstack-infra | 18:31 | |
| *** jtzl has quit IRC | 18:33 | |
| jeblair | fungi: sent++ | 18:33 |
| *** jtzl has joined #openstack-infra | 18:34 | |
| fungi | thanks! | 18:35 |
| fungi | yep, as i suspected... mutt says all.csv [text/csv, quoted, iso-8859-1, 47K] | 18:35 |
| fungi | so that seems much more in line with what i had | 18:35 |
| *** basha has quit IRC | 18:36 | |
| *** yolanda has quit IRC | 18:38 | |
| *** julim has joined #openstack-infra | 18:42 | |
| *** gyee has quit IRC | 18:42 | |
| *** nicedice has joined #openstack-infra | 18:48 | |
| *** atiwari has joined #openstack-infra | 18:50 | |
| Shrews | jeblair: re: your comment on 46741, are you saying that you have cases where you send(N>1) bytes, but read them one at a time with recv? | 18:50 |
| jeblair | clarkb: can you explain your +2 on 46871 in light of the fact that there is no such worker connected to our zuul? | 18:51 |
| jeblair | Shrews: i have no idea what other connections might send | 18:51 |
| clarkb | jeblair: there is a comment there indicating that zuul won't break on that | 18:52 |
| jeblair | clarkb: where's that comment? | 18:53 |
| clarkb | I thought it was in the gerrit comments but may have been in irc (it was from jhesketh) | 18:55 |
| clarkb | looking for it now | 18:55 |
| *** mrmartin has joined #openstack-infra | 18:58 | |
| clarkb | jeblair: found it, it is in jhesketh's first gerrit comment on that change | 18:59 |
| mriedem | sdague: i figured out my flake8 issue in nova | 18:59 |
| mriedem | was running hacking 0.7.0 but needed 0.7.2 - global requirements seems a bit old on hacking, but i've never updated one of those | 19:00 |
| jeblair | clarkb: well, it doesn't ignore them; it SKIPS them. | 19:00 |
| jeblair | i think that change is getting ahead of itself since it starts to run a job that doesn't exist and has no clear plan for existing... | 19:01 |
| clarkb | jeblair: thats fair, I think having the change gives us something concrete to talk about though (useful in that regard) | 19:02 |
| jeblair | clarkb: i suspect that jhesketh is under the impression he may be able to attach his worker to openstack-infra's zuul... | 19:02 |
| jeblair | clarkb: at least that's how i interpret the 2nd paragraph of the commit msg | 19:03 |
| clarkb | hmm ya | 19:03 |
| *** julim has quit IRC | 19:05 | |
| *** julim has joined #openstack-infra | 19:06 | |
| mriedem | in openstack/requirements, should test-requirements match what's in global-requirements? | 19:11 |
| openstackgerrit | A change was merged to openstack-infra/config: Clarify that statusbot requires public messages https://review.openstack.org/48209 | 19:11 |
| clarkb | mriedem: no, test-requirements are the requirements for testing openstack/requirements and I believe are allowed to be divergent | 19:11 |
| mriedem | clarkb: ok, wasn't sure, thanks | 19:11 |
| clarkb | mriedem: I think | 19:11 |
| mriedem | i saw the comment in test-requirements, seemed odd that they wouldn't match what's in global though | 19:12 |
| clarkb | the idea being that maybe you need something different in order to get a change to global-requirements to pass | 19:12 |
| mriedem | clarkb: ah, ok | 19:12 |
| reed | fungi, all seems alright, thanks | 19:12 |
| *** nati_ueno has joined #openstack-infra | 19:12 | |
| fungi | reed: you're welcome | 19:13 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gear: Remove --pre from tox install command https://review.openstack.org/48327 | 19:13 |
| jeblair | we're _still_ writing that patch. yay. | 19:13 |
| jeblair | lifeless: i added you as a reviewer on https://review.openstack.org/#/c/47164/ but you didn't respond. do you not see changes where people add you as a reviewer? | 19:15 |
| clarkb | jeblair: https://review.openstack.org/#/c/48091/1/tools/atc/email-stats.py line 107, what does the rowCount comparison do? should it instead simply check if rowCount is in the json line? | 19:17 |
| openstackgerrit | Matt Riedemann proposed a change to openstack/requirements: Pin hacking>=0.7.2,<0.8 in global-requirements https://review.openstack.org/48328 | 19:17 |
| clarkb | jeblair: doesn't need fixing before merging as we are moving the tools, it just popped up as I was reading through the change | 19:17 |
| *** qba73 has joined #openstack-infra | 19:17 | |
| clarkb | jeblair: or does gerrit give us data in chunks of 500? | 19:18 |
| jeblair | clarkb: it gives us data in chunks of 500 | 19:21 |
| *** reed has quit IRC | 19:22 | |
| openstackgerrit | A change was merged to openstack-infra/config: Initial checkin of ATC scripts https://review.openstack.org/48091 | 19:24 |
| *** rnirmal has joined #openstack-infra | 19:25 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Update tox.ini to current standards https://review.openstack.org/47391 | 19:26 |
| *** dizquierdo has joined #openstack-infra | 19:27 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Add script for mapping reviews to bugs and their milestones https://review.openstack.org/48330 | 19:28 |
| lifeless | jeblair: I'm sure I got such an email | 19:28 |
| lifeless | jeblair: but I get hundreds of gerrit mails/day | 19:28 |
| lifeless | jeblair: and the summary shown by gmail is often not enough to ascertain direct interest or bulk traffic | 19:29 |
| lifeless | jeblair: this one in particular I've already mentioned to mordred as not making sense. | 19:29 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Add script for mapping reviews to bugs and their milestones https://review.openstack.org/48330 | 19:30 |
| *** vipul is now known as vipul-away | 19:30 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add requirements and publish jobs to marconi https://review.openstack.org/47183 | 19:31 |
| jeblair | flaper87: ^ that just happened | 19:32 |
| fungi | lifeless: they'll generally show bolded on the main review dashboard ("my changes") | 19:32 |
| *** samalba has quit IRC | 19:32 | |
| *** dizquierdo has quit IRC | 19:32 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Update savanna project config https://review.openstack.org/48332 | 19:32 |
| *** samalba has joined #openstack-infra | 19:34 | |
| *** tvb|afk has quit IRC | 19:36 | |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Update savanna project config https://review.openstack.org/48332 | 19:36 |
| *** vipul-away is now known as vipul | 19:37 | |
| lifeless | fungi: yeah, I haven't had time this week to poke at gerrit really. | 19:37 |
| lifeless | fungi: EBUSY | 19:37 |
| *** dcramer_ has quit IRC | 19:37 | |
| fungi | lifeless: yep, totally understand that | 19:38 |
| *** sarob_ has joined #openstack-infra | 19:38 | |
| *** sarob has quit IRC | 19:40 | |
| *** sarob_ has quit IRC | 19:42 | |
| *** CaptTofu has quit IRC | 19:43 | |
| *** CaptTofu has joined #openstack-infra | 19:43 | |
| *** CaptTofu_ has joined #openstack-infra | 19:44 | |
| *** CaptTofu_ has quit IRC | 19:44 | |
| clarkb | mordred: https://review.openstack.org/#/c/48295/ did you git review -R that change? or does git review need to do like zuul and better approximate the gerrit jgit merge strategy | 19:45 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gear: Fix geard logging config https://review.openstack.org/48334 | 19:46 |
| flaper87 | jeblair: AWESOME!!!!!! | 19:46 |
| flaper87 | mordred: jeblair thanks | 19:46 |
| *** alexpilotti has quit IRC | 19:46 | |
| *** SergeyLukjanov has quit IRC | 19:47 | |
| *** CaptTofu_ has joined #openstack-infra | 19:48 | |
| *** CaptTofu has quit IRC | 19:48 | |
| flaper87 | jeblair: mordred btw https://review.openstack.org/#/c/47999/ :) | 19:48 |
| *** julim has quit IRC | 19:49 | |
| *** alexpilotti has joined #openstack-infra | 19:49 | |
| clarkb | zaro: if you rebase 48334 on https://review.openstack.org/#/c/48327/ your change should pass check tests | 19:51 |
| *** vipul is now known as vipul-away | 19:52 | |
| *** julim has joined #openstack-infra | 19:52 | |
| *** adalbas has quit IRC | 19:53 | |
| clarkb | flaper87: fyi we had to backout the change that enabled ubuntu's cloud archive on our slaves, which means that we can't currently install a new mongodb. Hopefully libvirt will get sorted out and we can reenable that | 19:53 |
| *** gyee has joined #openstack-infra | 19:53 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gear: Fix geard logging config https://review.openstack.org/48334 | 19:54 |
| flaper87 | clarkb: oh, OK! thanks for the heads up! | 19:54 |
| openstackgerrit | A change was merged to openstack/requirements: Bump ceilometerclient dependency https://review.openstack.org/47994 | 19:54 |
| *** hashar has joined #openstack-infra | 19:55 | |
| fungi | clarkb: flaper87: i believe jd__ has a wip change open to revert that revert, and it links to the bug with details on the issue | 19:55 |
| mriedem | clarkb: let me know if i didn't address your comment here: https://review.openstack.org/#/c/48328/ | 19:56 |
| mriedem | or if you have questions about *why* we'd have an internal pip mirror | 19:56 |
| clarkb | fungi: flaper87: https://review.openstack.org/#/c/48226/ found it thanks | 19:56 |
| fungi | so in theory, you could rebase any changes needing cloud archive on top of that wip change, i suppose | 19:57 |
| *** dcramer_ has joined #openstack-infra | 19:57 | |
| flaper87 | clarkb: fungi thanks, I'll follow that review | 19:58 |
| clarkb | mriedem: I will respond inline | 19:59 |
| clarkb | er on the change | 19:59 |
| *** vipul-away is now known as vipul | 20:00 | |
| *** sarob has joined #openstack-infra | 20:01 | |
| *** sarob has quit IRC | 20:02 | |
| *** sarob has joined #openstack-infra | 20:03 | |
| *** adalbas has joined #openstack-infra | 20:05 | |
| *** tvb|afk has joined #openstack-infra | 20:06 | |
| clarkb | mordred: fungi: https://review.openstack.org/#/c/48058/1 is that test really redundant? | 20:08 |
| clarkb | mordred: fungi: I am not sure that there is any place outside of the mirror build jobs that we actively check we can install every single requirement in one place | 20:08 |
| * clarkb grabs lunch | 20:09 | |
| jeblair | clarkb: it also seems to conflict with https://review.openstack.org/#/c/45329/7 | 20:09 |
| jeblair | clarkb: (which mordred +2d) | 20:09 |
| fungi | clarkb: the pbr integration test was the assertion there i think, though i wonder if we should be running that as part of the check/gate pipeline for openstack/requirements in that case | 20:10 |
| clarkb | jeblair: yup they conflict. Also I want to say there is a reason for the source | 20:10 |
| clarkb | fungi: the pbr integration test does not check that everyting installs cleanly | 20:10 |
| clarkb | it checks that whatever subset nova et al use installs | 20:10 |
| clarkb | jeblair: but it may not be necessary for those particular shell scripts | 20:11 |
| dhellmann | mordred: I'm seeing a UnicodeDecodeError packaging something with pbr, have you ever seen that? | 20:11 |
| clarkb | fungi: maybe the thing that needs fixing is we need to make it easier to prune the list? that way the mirror jobs don't potentially break due to something that isn't in use | 20:12 |
| clarkb | dhellmann: like this http://logs.openstack.org/34/48334/1/check/gate-gear-docs/176d79e/console.html ? | 20:12 |
| fungi | dhellmann: sphinx? | 20:12 |
| clarkb | oh nevermind that is a typeerror not an encoding error | 20:12 |
| clarkb | ok really lunch now | 20:12 |
| dhellmann | clarkb: no, it looks more like http://bugs.python.org/issue11638 | 20:13 |
| *** dprince has quit IRC | 20:13 | |
| dhellmann | fungi: no, simple package with one module and no docs yet | 20:13 |
| fungi | clarkb: yeah, i guess the overlap of devstack installing proposed requirements and pbr integration tests may still not be as complete as the install-them-all test | 20:13 |
| dhellmann | if this package needs more documentation than the readme, I'll be surprised | 20:13 |
| *** rockygrober has joined #openstack-infra | 20:14 | |
| *** tvb|afk has quit IRC | 20:15 | |
| dhellmann | fungi, clarkb : https://github.com/dreamhost/httplib2-ca_certs_locater | 20:15 |
| *** rockyg has quit IRC | 20:16 | |
| *** sarob has quit IRC | 20:17 | |
| *** sarob has joined #openstack-infra | 20:18 | |
| fungi | dhellmann: i wonder if the lack of tags is confusing pbr's version generation? | 20:18 |
| dhellmann | I put a 0.0.0 tag on the first commit and I get a version number of 0.0.0.1.hash | 20:18 |
| fungi | oh, hrm | 20:18 |
| dhellmann | then I tried adding a 0.1.0 tag on HEAD and got that as the version | 20:19 |
| dhellmann | aha, although the version is a unicode string | 20:19 |
| dhellmann | _run_shell_command is returning a unicode object | 20:19 |
| fungi | which would almost definitely be that python bug you linked | 20:20 |
| dhellmann | yep, encoding the version string as utf-8 makes the error go away | 20:20 |
| dhellmann | ok, I'll work on a patch | 20:20 |
| fungi | i wonder what circumstance is causing you to be the first to hit it | 20:21 |
| dhellmann | retribution for the cliff fiasco? :-) | 20:21 |
| dhellmann | seriously, I'm not sure what's special about this rep | 20:22 |
| dhellmann | repo | 20:22 |
| dhellmann | are you able to reproduce it? | 20:22 |
| *** sarob has quit IRC | 20:22 | |
| clarkb | are you on os x? maybe something platform specific? | 20:22 |
| dhellmann | oh, I'm on OS X | 20:22 |
| dhellmann | yeah | 20:22 |
| fungi | ha! that's why it's not happening to me | 20:22 |
| dhellmann | something to do with the filesystem encoding? | 20:22 |
| dhellmann | does not happen to me on linux | 20:23 |
| dhellmann | ok, well, that's something | 20:23 |
| dhellmann | I wonder if I just have a version of 2.7 on OS X that doesn't have the fix from that bug | 20:23 |
| fungi | dhellmann: possibly. os x does some interesting things with regard to utf-8 normalization, as i learned when rsync'ing the music collection from my debian fileserver to my girlfriend's macbook | 20:23 |
| *** nati_ueno has quit IRC | 20:24 | |
| *** davidhadas_ has joined #openstack-infra | 20:24 | |
| fungi | the rsync options necessary to make that work sanely are... interesting | 20:25 |
| dhellmann | haha, I can't even install pbr because of this :-) | 20:25 |
| dhellmann | I can only imagine on the rsync options... | 20:26 |
| clarkb | clearly the solution is to run linux everywhere >_> | 20:27 |
| zaro | anteaya: hi | 20:33 |
| anteaya | hey zaro | 20:33 |
| *** nati_ueno has joined #openstack-infra | 20:33 | |
| zaro | anteaya: ohh was going to ask you about patch for stackanalytics but i noticed that mordred did it instead | 20:35 |
| anteaya | he did | 20:35 |
| anteaya | did he get your info correct? | 20:35 |
| zaro | anteaya: yes, looks corret. didn't realized that this is a good way to find out who's who. | 20:36 |
| *** fbo_away is now known as fbo | 20:37 | |
| anteaya | zaro: neither did I until I did my own patch | 20:40 |
| anteaya | some good info in there | 20:40 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/requirements: add: enum34 https://review.openstack.org/48339 | 20:43 |
| *** afazekas_away is now known as afazekas | 20:46 | |
| *** dizquierdo has joined #openstack-infra | 20:48 | |
| mordred | dhellmann: there is a patch in flight to pbr which pokes at run_shell_command - be sure you pull that in and work on top ofit | 20:50 |
| *** dcramer_ has quit IRC | 20:53 | |
| dhellmann | mordred: ack, thanks | 20:54 |
| *** ArxCruz has quit IRC | 20:55 | |
| *** zehicle_at_dell has quit IRC | 20:57 | |
| *** rnirmal has quit IRC | 21:00 | |
| *** rnirmal has joined #openstack-infra | 21:00 | |
| *** ArxCruz has joined #openstack-infra | 21:00 | |
| *** flaper87 is now known as flaper87|afk | 21:03 | |
| *** DennyZhang has quit IRC | 21:03 | |
| dhellmann | mordred: if I "git review -d" your change, and then rebase mine on top of the branch that creates, is that going to do the right thing for gerrit? | 21:04 |
| *** alcabrera has quit IRC | 21:04 | |
| *** ArxCruz has quit IRC | 21:05 | |
| jeblair | dhellmann: yes | 21:06 |
| *** thomasm has quit IRC | 21:08 | |
| clarkb | jeblair: in your status.js update how are _graph_index and _graph_position different? | 21:08 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Diff between installed packages and requirements https://review.openstack.org/45329 | 21:09 |
| *** julim has quit IRC | 21:09 | |
| jeblair | clarkb: _graph_index is the column index for where the red or green dot should be placed; _graph_position is the distance of the change from the top of the list (so that the changes behind can be sorted) | 21:10 |
| jeblair | clarkb: i admit in retrospect those are confusing; i did not anticipate the need for both of them. | 21:11 |
| *** tvb|afk has joined #openstack-infra | 21:11 | |
| *** thomasm has joined #openstack-infra | 21:12 | |
| *** thomasm has quit IRC | 21:13 | |
| *** flaper87|afk is now known as flaper87 | 21:13 | |
| clarkb | jeblair: thanks, I left a comment on the change | 21:14 |
| dstufft | clarkb: talking to holger, he's gonna make devpi do it | 21:14 |
| clarkb | dstufft: cool | 21:15 |
| *** tvb|afk has quit IRC | 21:17 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-dev/hacking: Revert "Adds ability to ignore hacking validations with noqa" https://review.openstack.org/48345 | 21:18 |
| openstackgerrit | A change was merged to openstack-infra/gearman-plugin: Set a node offline even if there is an exception https://review.openstack.org/46124 | 21:18 |
| *** dkliban is now known as dkliban_afk | 21:19 | |
| *** hashar has quit IRC | 21:20 | |
| dhellmann | jeblair: cool, thanks | 21:21 |
| *** portante is now known as portante|afk | 21:22 | |
| *** mrmartin has quit IRC | 21:27 | |
| jog0 | Alex_Gaynor: ping | 21:27 |
| jog0 | we have a problem https://review.openstack.org/#/c/48345/ | 21:27 |
| *** sarob has joined #openstack-infra | 21:28 | |
| *** rfolco has quit IRC | 21:32 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-dev/hacking: Add noqa support for H231 and H233 https://review.openstack.org/48349 | 21:32 |
| clarkb | jog0: do you know why the decorator isn't working? | 21:33 |
| *** sarob has quit IRC | 21:33 | |
| jog0 | clarkb: no :( | 21:34 |
| jog0 | I have a hunch though | 21:34 |
| jog0 | actually no hunch | 21:35 |
| jog0 | but it doesn't seem worth keeping around or debugging | 21:36 |
| jog0 | clarkb: And I don't know why pypy is pypy is broken | 21:36 |
| jog0 | so unless Alex_Gaynor can fix it I vote we make pypy non-voting today | 21:36 |
| clarkb | I wonder if it is because you yield failures | 21:37 |
| dhellmann | is there some secret trick to get testr to create its database using a format known to both python 2 and 3? I keep having to run the pbr tests one version at a time, removing .testrepository each time. | 21:37 |
| dhellmann | lifeless: ^^ | 21:37 |
| clarkb | so return f(...) derps the yild | 21:38 |
| clarkb | dhellmann: no, there is no trick | 21:38 |
| lifeless | dhellmann: not that I've found | 21:38 |
| dhellmann | lifeless: are you using anydbm, then? | 21:38 |
| jog0 | clarkb: that makes sense | 21:38 |
| jog0 | clarkb: either way still easier to revert then to fix and don't think its worth fixing | 21:39 |
| jog0 | it doesn't even save much code | 21:40 |
| jog0 | just an extra line | 21:40 |
| lifeless | dhellmann: see testrepository/repository/file.py | 21:40 |
| lifeless | db = dbm.open(self._path('times.dbm'), 'c') | 21:40 |
| lifeless | and | 21:40 |
| *** changbl has quit IRC | 21:41 | |
| lifeless | try: | 21:41 |
| lifeless | import anydbm as dbm | 21:41 |
| lifeless | except ImportError: | 21:41 |
| lifeless | import dbm | 21:41 |
| lifeless | at the top | 21:41 |
| lifeless | dhellmann: I'm pondering structured data and sqlite3 tbh | 21:41 |
| dhellmann | lifeless: how about a version specifier in the filename or something? it sounds like I probably can't use testr with detox the way it works now? | 21:42 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Make gate-hacking-pypy non-gating https://review.openstack.org/48352 | 21:42 |
| lifeless | dhellmann: whats detox ? | 21:42 |
| jog0 | clarkb Alex_Gaynor: my emergency disable hacking-pypy patch if we can't fix it in soon | 21:42 |
| dhellmann | lifeless: detox runs your tox envs in parallel on the current host | 21:43 |
| lifeless | dhellmann: ah; so I want to lift that up into testr anyhow | 21:43 |
| dhellmann | yes, please! :-) | 21:43 |
| lifeless | dhellmann: invert a bunch of stuff | 21:43 |
| dhellmann | well, blah, not put it in testr, just make the tools that already exist work | 21:43 |
| lifeless | so you want testr to know that the py2.6 and py3.3 test runs are different | 21:44 |
| lifeless | otherwise whichever test run finishes last will replace your success/failure data | 21:44 |
| lifeless | dhellmann: I don't have a strong opinion about where the boundaries should be | 21:46 |
| lifeless | dhellmann: but - testr already parallelises, so detox doesn' | 21:46 |
| lifeless | t really make sense performance wise. | 21:46 |
| lifeless | dhellmann: clearly you should be able to do it either way around though. I don't want to reimplement tox. | 21:47 |
| *** vipul is now known as vipul-away | 21:47 | |
| *** vipul-away is now known as vipul | 21:47 | |
| *** dmakogon_ has joined #openstack-infra | 21:49 | |
| *** fbo is now known as fbo_away | 21:50 | |
| *** mrda has joined #openstack-infra | 21:50 | |
| clarkb | pleia2: my catching up on reviews is going more slowly than I had hoped, I still haven't gotten to stracing iscsid | 21:52 |
| clarkb | sdague: are you still around? | 21:54 |
| dhellmann | lifeless: testr will parallelize multiple versions of python at the same time? | 21:54 |
| dhellmann | lifeless: because that's what detox *is for* | 21:55 |
| *** boris-42 has quit IRC | 21:55 | |
| clarkb | sdague: thinking about the rechecks based on age script | 21:55 |
| clarkb | sdague: is the initial goal a way of detecting merge conflicts? | 21:56 |
| clarkb | dhellmann: no, testr runs your test suite in parallel under the same python. So you can do for env in "26 27 33 py" ; do tox -epy$env ; done to get the same approximate speed | 21:58 |
| *** dkranz has quit IRC | 21:58 | |
| jeblair | clarkb: i had a chat with sdague about that -- i don't want to leave recheck comment spam just to check merge conflicts for old changes; that should be something that gets built into zuul. | 21:58 |
| dhellmann | clarkb: right. detox replaces the for loop | 21:58 |
| clarkb | dhellmann: but it appares detox isn't running one env at a time. instead it fires them up together | 21:59 |
| jeblair | clarkb: he suggested there were other uses for the script (such as batch rechecks after a "crisis"); i haven't looked at it to see though. | 21:59 |
| dhellmann | clarkb: yes | 21:59 |
| clarkb | jeblair: ya, it seems a bit brute force | 21:59 |
| clarkb | dhellmann: which isn't needed | 21:59 |
| dhellmann | ? | 21:59 |
| clarkb | dhellmann: testr is going to use all of the available cpus if given the --parallel flag | 21:59 |
| *** zehicle_at_dell has joined #openstack-infra | 22:00 | |
| clarkb | so running testr 4 times is equivalent to running 4 test runs under different pythons at the same time assuming the second set of tests use a single cpu each and you are on a 4 cpu machine | 22:00 |
| dhellmann | clarkb: I don't understand how that helps me run tests against multiple versions of python at the same time | 22:01 |
| clarkb | dhellmann: it doesn't | 22:01 |
| clarkb | lifeless is saying that running tests against multiple version of python at the same time should be approximately as fast as running your tests against one python at a time | 22:01 |
| dhellmann | ok, the pbr test suite runs fast enough that it is actually faster to run them in parallel | 22:01 |
| dhellmann | except that it doesn't work | 22:01 |
| dhellmann | clarkb: perhaps | 22:02 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Visualize Zuul's NNFI scheduler https://review.openstack.org/47569 | 22:03 |
| *** dmakogon_ has left #openstack-infra | 22:05 | |
| clarkb | dhellmann: that said you should be able to use the "remote" environment support to do what you want | 22:05 |
| clarkb | where each python has its own workspace essentially | 22:05 |
| dhellmann | clarkb: is that a feature of testr? | 22:06 |
| clarkb | but really I would be curious to see how much quicker it is doing them in parallel (nova and quantum unittests are very cpu bound and may even be slower) | 22:06 |
| clarkb | dhellmann: ya http://testrepository.readthedocs.org/en/latest/MANUAL.html#remote-or-isolated-test-environments | 22:06 |
| dhellmann | clarkb: these are just the pbr tests, so nothing on the scale of nova | 22:06 |
| dhellmann | clarkb: of course, it doesn't really matter until the dbm issue is sorted out, but maybe that remote feature is a way to work around that, too | 22:07 |
| clarkb | oh except, testr is going to write to the local .testrepository... hmm this is a fun one | 22:07 |
| *** datsun180b has quit IRC | 22:08 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack-dev/pbr: Do not pass unicode where byte strings are wanted https://review.openstack.org/48355 | 22:10 |
| sdague | clarkb: I'm back now | 22:11 |
| sdague | jeblair: so I want a script for tempest to do recheck on condition, if you don't want it in config, I can put it somewhere else | 22:12 |
| clarkb | dhellmann: re ^ could we encode to utf8? and ignore the version check? | 22:12 |
| clarkb | dhellmann: I am still digging my way through the python bug | 22:13 |
| *** jerryz has joined #openstack-infra | 22:13 | |
| jeblair | sdague: i don't mean to rehash our conversation from the other day; i was trying to catch clarkb up since we had it in irc and not in the review. | 22:13 |
| dhellmann | clarkb: you know, I started with utf-8 and saw some other asciis in the file and just went into copycat mode | 22:13 |
| sdague | ok, fair | 22:13 |
| dhellmann | clarkb: it does seem to work with utf-8, but I expect all the values in the version to be in the ascii range anyway | 22:14 |
| dhellmann | I guess utf-8 is still safer | 22:14 |
| clarkb | dhellmann: yeah either way *should* work. I only ask because places where I expect ascii to fail where utf8 passes is when we are dealing with utf8 | 22:14 |
| clarkb | dhellmann: if they both fail we are probably running on windows or something funny | 22:14 |
| dhellmann | clarkb: fair point, I'll go ahead and change it | 22:14 |
| dhellmann | haha | 22:15 |
| dhellmann | clarkb: as far as the version check, I think we want that because the bug is only present in python 2, and we don't want to mix byte strings and unicode strings in python 3 | 22:16 |
| clarkb | sdague: jeblair: ya I was just wondering what specific use case was in mind there as there appears to be one | 22:16 |
| clarkb | dhellmann: ok | 22:16 |
| dhellmann | clarkb: the unicode handling is done properly in the tarfile module for py3 | 22:16 |
| sdague | clarkb: so right now I need it on devstack and tempest to flush old changes for recheck | 22:16 |
| jog0 | jeblair: just noticed we are running NFFI scheduler in zuul now | 22:17 |
| jog0 | nice | 22:17 |
| jeblair | jog0: hopefully soon that will be much more apparent | 22:17 |
| sdague | because nothing quiet demotivates people in reviews so much as reviewing something, merge conflict | 22:17 |
| jog0 | jeblair: oh? | 22:17 |
| jeblair | sdague: well, that's what we want in zuul... i thought you had other conditions in mind | 22:17 |
| jeblair | jog0: https://review.openstack.org/#/c/47569/ | 22:18 |
| sdague | jeblair: I do, I haven't had time to write them yet :) | 22:18 |
| sdague | but I needed to start somewhere | 22:18 |
| clarkb | sdague: jeblair: what about check for conflicts locally so that we don't have to retest all the things if there is no conflict? | 22:18 |
| clarkb | (before zuul is able to do this) | 22:18 |
| jog0 | jeblair: cool | 22:18 |
| jog0 | any screen shots of that | 22:19 |
| *** qba73 has quit IRC | 22:19 | |
| jeblair | sdague: ok; well, i mean, you're free to leave any comment you want on any change you want, but i'd rather not start leaving recheck spam everywhere even as an interim measure. | 22:19 |
| jeblair | clarkb: there's even a long-open ci bug about "check all open changes in a project for merge conflicts after a change merges" | 22:20 |
| jeblair | jog0: i posted one last night, sorry i don't have it handy | 22:20 |
| jog0 | jeblair: i can wait till its up | 22:20 |
| jog0 | thanks | 22:20 |
| sdague | jeblair: I was doing this manually a lot, which is actually why I brought up the idea the other day of having something besides "recheck no bug" to indicate this was a freshness check | 22:20 |
| *** mriedem has quit IRC | 22:22 | |
| jeblair | sdague: yeah, it's just it's not that hard to add to zuul, so i hate the idea of spending time on not doing it the right way | 22:22 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-dev/pbr: Do not pass unicode where byte strings are wanted https://review.openstack.org/48355 | 22:24 |
| *** thedodd has quit IRC | 22:25 | |
| clarkb | jeblair: your thoughts on https://review.openstack.org/#/c/48153/ would be greatly appreciated | 22:26 |
| jeblair | clarkb: is it urgent? | 22:26 |
| clarkb | jeblair: not really. I am giving the talk next tuesday so there is time to clean if up if I have gotten nnfi wrong | 22:27 |
| *** mrodden has quit IRC | 22:27 | |
| jeblair | clarkb: oh is this the ales thing? | 22:28 |
| clarkb | yup | 22:28 |
| *** slong has joined #openstack-infra | 22:30 | |
| *** dizquierdo has left #openstack-infra | 22:30 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/zuul: Remove usage of d2to1 https://review.openstack.org/48294 | 22:30 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/zuul: Partial sync with OpenStack requirements. https://review.openstack.org/48295 | 22:30 |
| clarkb | mordred: ^ I rebased those for you as I got curious to why there were conflicts. I think it was because they depended on an old version of a change that was merged | 22:31 |
| *** rwsu has quit IRC | 22:31 | |
| *** nati_ueno has quit IRC | 22:32 | |
| lifeless | dhellmann: thats strange, since testr is also parallel | 22:32 |
| jeblair | clarkb: i think i'd have change #3 start at the same level and then only move out of the way when it's failed | 22:32 |
| jeblair | clarkb: (because having it start out of the way suggests there's something special about it beforehand) | 22:33 |
| jeblair | clarkb: other than that, seems accurate! :) | 22:33 |
| *** afazekas is now known as afazekas_zz | 22:33 | |
| clarkb | jeblair: good point. that was admittedly a shortcut because I couldn't figure out how to make the js stuff move without redoing animation stuff, I will hack on that more | 22:33 |
| clarkb | jeblair: thank you for checking it | 22:33 |
| fungi | well, it's special because it's keystone and the others are nova... | 22:34 |
| jeblair | clarkb: guaranteed to add another 2 minutes to any talk. :) | 22:34 |
| jeblair | fungi: there's that. :) | 22:34 |
| fungi | (at least that was the rationale i assumed) | 22:34 |
| clarkb | fungi: it was a cheap way of making the reattached change 4 head graph not look horrible | 22:34 |
| jeblair | fungi: but seriously, they were all lined up initially to actually highlight that zuul treats them the same. | 22:34 |
| clarkb | ya jeblair's point is a good one. There isn't anything special until after the failure | 22:35 |
| fungi | yep, agreed. it just didn't dawn on me that clarkb might have realigned them to make the line generation take less animation (hanna-barbera would be pleased) | 22:36 |
| sdague | jeblair: so if I wanted to do the merge conflict checking on any patch merge, where would that hook in in zuul? | 22:36 |
| clarkb | lifeless: dhellmann: I wonder if parallelizing the tox sdist/develop step and the discover that testr runs is helpful as those steps are IOy | 22:36 |
| clarkb | fungi: I learned much from my saturday morning cartoons | 22:36 |
| sdague | because this is getting to be a pretty big issue in the large review queues, especially with certain folks that don't seem to understand patch series :) | 22:36 |
| *** rfolco has joined #openstack-infra | 22:36 | |
| jhesketh | jeblair, clarkb: I've just got to work sorry. Read bits of scrollback but not all.. You guys were wondering about my zuul config/worker? | 22:36 |
| fungi | clarkb: i learned everything i know from 1960s superhero cartoon reruns | 22:36 |
| dhellmann | clarkb: could be -- if I have some time tomorrow I'll put some numbers together | 22:37 |
| dhellmann | clarkb, lifeless : you know, it could be faster because in parallel the py33 tests bailed early | 22:37 |
| dhellmann | because of the dbm thing | 22:37 |
| dhellmann | so maybe it's my imagination | 22:37 |
| jeblair | clarkb: check out the Change.merge function; you can copy that to a Change.move and it should be like one line of code to animate it. | 22:38 |
| clarkb | jeblair: thanks | 22:38 |
| jeblair | sdague: i haven't brainstormed how to do that; only the simpler case of 'recheck old changes' which we discussed the other day. let me think about that. | 22:39 |
| *** alexpilotti has quit IRC | 22:39 | |
| clarkb | jhesketh: yes, in particular I think jeblair wants to discuss the running of the tests before we worry about the zuul config | 22:40 |
| jeblair | jhesketh: yeah i left a comment in https://review.openstack.org/#/c/46871/ | 22:40 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gear: Fix geard logging config https://review.openstack.org/48334 | 22:41 |
| jhesketh | jeblair: okay, thanks for that. I'll take a look | 22:41 |
| *** dhellmann is now known as dhellmann_ | 22:42 | |
| fungi | jeblair: clarkb: any objections to me temporarily repointing the puppet master's reporturl to anteaya's beta dashboard (since reports to our dead dashboard are falling on deaf ears anyway)? | 22:42 |
| anteaya | would save me a lot of time | 22:42 |
| jeblair | fungi: nope | 22:42 |
| anteaya | yay, thanks | 22:42 |
| *** sarob has joined #openstack-infra | 22:43 | |
| anteaya | do you have all you need from me, fungi? | 22:43 |
| fungi | then, assuming it works, we can puppet us up a new dashboard all slick and fancy like | 22:43 |
| anteaya | oh I hope this works | 22:43 |
| *** prad is now known as prad-afk | 22:44 | |
| fungi | anteaya: right now it's http://puppet-dashboard.openstack.org:3000/reports/upload so if you could confirm specifically what you want it changed to, that would help | 22:44 |
| anteaya | http://15.185.153.222:3000/ | 22:44 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gear: Fix geard logging config https://review.openstack.org/48334 | 22:44 |
| fungi | anteaya: .../reports/upload still, or just the root url? | 22:44 |
| anteaya | let me check that | 22:45 |
| *** tvb|afk has joined #openstack-infra | 22:45 | |
| clarkb | fine with me | 22:46 |
| clarkb | fungi: anteaya ^ | 22:46 |
| anteaya | thanks clarkb | 22:47 |
| *** rwsu has joined #openstack-infra | 22:47 | |
| jeblair | sdague: i think it could be done very similar to the check-old changes; as a new zuul trigger, except instead of walking all changes on a timer, it responds to merge events and walks all changes for that project. however... | 22:47 |
| jeblair | sdague: because it would run so often, i think it would need two additional things -- support for not reporting to gerrit if the vote is the same, and a way to specify that only a merge check (with no jobs) should be performed. | 22:48 |
| *** tvb|afk has quit IRC | 22:50 | |
| *** prad-afk has quit IRC | 22:51 | |
| anteaya | fungi: the documentation I am reading is consistent with using reports/upload as part of the url to send reports to the dashboard | 22:51 |
| jeblair | sdague: i'm happy to work on those myself, but realistically, i probably won't be able to get to it for a bit. | 22:51 |
| sdague | yeh, I probably won't either. Hence my stop gapping. | 22:51 |
| anteaya | but so far I am not finding the corresponding documentation that tells me how to ensure that path is available for the reports to be received | 22:51 |
| *** atiwari has quit IRC | 22:52 | |
| anteaya | fungi: can we try it and see if dashboard picks it up by default? | 22:53 |
| fungi | anteaya: 20 pending tasks | 22:53 |
| jeblair | sdague: though keep in mind that what your script does is rechecking old changes on a timer, which is the easier of the two in zuul. | 22:54 |
| anteaya | there they are | 22:54 |
| anteaya | fungi a good start | 22:54 |
| fungi | is your queue runner running? | 22:54 |
| jeblair | sdague: though it sounds like the more complex option is what you actually want | 22:54 |
| anteaya | fungi: I don't know, how do I find out? | 22:55 |
| sdague | so I want the auto merge conflict check in zuul on merges, for sure | 22:55 |
| fungi | anteaya: i think it | 22:55 |
| fungi | anteaya: ugh, badly-placed enter key | 22:55 |
| sdague | but I also want bulk recheck on criteria for things like when we slam into a wall like we did recently | 22:55 |
| *** sarob has quit IRC | 22:55 | |
| fungi | anteaya: i think it's the puppetqd process? | 22:55 |
| jeblair | sdague: and that's why i'm glad you're working on that script. :) | 22:56 |
| sdague | and to get that, I needed to start with rechecking on some criteria | 22:56 |
| *** weshay has quit IRC | 22:56 | |
| anteaya | this is the output in my terminal: http://paste.openstack.org/show/47523/ | 22:56 |
| sdague | and date was easy, and immediately useful to me :) | 22:56 |
| *** jhesketh_ has quit IRC | 22:56 | |
| fungi | anteaya: i'll check the old dashboard | 22:56 |
| sdague | anyway time for me to go work on dinner, catch folks in the morning | 22:57 |
| jeblair | sdague: good night! | 22:57 |
| anteaya | fungi: I don't have a process called puppetqd that is currently running | 22:57 |
| *** MarkAtwood has left #openstack-infra | 22:57 | |
| *** UtahDave has quit IRC | 22:58 | |
| *** MarkAtwood has joined #openstack-infra | 23:00 | |
| anteaya | btw, I have an email conversation going with sodabrew, he is very helpful | 23:00 |
| anteaya | and quick to respond | 23:00 |
| fungi | anteaya: the puppet-dashboard-workers initscript (sorry, puppetqd is related to the master not the dashboard) | 23:01 |
| anteaya | no worries | 23:01 |
| anteaya | what command would tell me if that is running? | 23:02 |
| anteaya | ps aux | grep puppet returns the command to run the rails server | 23:02 |
| *** rcleere has quit IRC | 23:03 | |
| *** kong has quit IRC | 23:04 | |
| *** kong has joined #openstack-infra | 23:04 | |
| fungi | that's a good question. our dashboard is not in a known-good state but dashboard/delayed_job.0 processes i think (reviewing the initscript) | 23:05 |
| anteaya | running ps aux | grep dashboard just gives me the running rails server again | 23:06 |
| fungi | also no idea how similar the sodabrew fork is to what we were running before | 23:06 |
| anteaya | in this regard, I don't know either | 23:06 |
| anteaya | I have an email into sodabrew about the queue runner | 23:06 |
| anteaya | so far, no answer yet | 23:06 |
| anteaya | here is my /etc/init.d: http://paste.openstack.org/show/47524/ | 23:08 |
| anteaya | nothing puppety | 23:09 |
| *** dims has quit IRC | 23:14 | |
| anteaya | it looks like running rake jobs:work would be a good place to start: http://paste.openstack.org/show/47525/ | 23:14 |
| anteaya | I'll try that | 23:14 |
| *** reed has joined #openstack-infra | 23:16 | |
| anteaya | ohhh look at that | 23:16 |
| anteaya | and sodabrew just gave me confirmation that that is the correct command | 23:19 |
| anteaya | he also says that Dashboard 1.2 workers cannot process 2.0 jobs. | 23:19 |
| anteaya | is he talking about the jobs coming from the master? | 23:20 |
| anteaya | I'll let this run for a while | 23:22 |
| anteaya | and check back in a bit | 23:22 |
| *** jhesketh__ has joined #openstack-infra | 23:23 | |
| *** boris-42 has joined #openstack-infra | 23:26 | |
| *** julim has joined #openstack-infra | 23:27 | |
| jeblair | reed: http://translate-dev.openstack.org/ | 23:28 |
| jeblair | reed: very basic pootle setup so people can poke at it | 23:28 |
| *** salv-orlando has quit IRC | 23:28 | |
| jeblair | clarkb, fungi, mordred: ^ | 23:28 |
| lifeless | clarkb: thats plausible; OTOH thats part of the whole 'its a scheduler' problem | 23:29 |
| *** dims has joined #openstack-infra | 23:31 | |
| clarkb | jeblair: cool | 23:33 |
| clarkb | lifeless: ya | 23:33 |
| *** flaper87 is now known as flaper87|afk | 23:33 | |
| fungi | anteaya: looks like it | 23:34 |
| fungi | 's doing what we want | 23:34 |
| *** mrda has quit IRC | 23:34 | |
| * fungi now needs to find out what a pootle is | 23:35 | |
| *** mrda has joined #openstack-infra | 23:35 | |
| fungi | aha! a transifex-killer ;) | 23:35 |
| anteaya | fungi: yay, so how long do you want to evaluate it | 23:37 |
| fungi | anteaya: lgtm | 23:37 |
| anteaya | great | 23:37 |
| fungi | anteaya: maybe make sure it runs for a few days without falling over? | 23:38 |
| anteaya | clarkb jeblair does http://15.185.153.222:3000/ meet all your puppet dashboard needs? | 23:38 |
| anteaya | fungi: let's do that | 23:38 |
| anteaya | now in the meantime, this can run and I can work on the puppeting of what I have | 23:38 |
| anteaya | pleia2 zaro mordred ^^ | 23:38 |
| clarkb | anteaya: while we are at it if we can get it on port 80 that would be awesome | 23:39 |
| *** rfolco has quit IRC | 23:40 | |
| jeblair | anteaya: looks great! and hey, we have hosts with errors! | 23:40 |
| jeblair | anteaya: what's the deal with 'pending tasks'? | 23:40 |
| *** julim has quit IRC | 23:40 | |
| anteaya | clarkb: I will see what can be done about port 80 | 23:41 |
| anteaya | yeah there are 8 pending tasks | 23:42 |
| fungi | perpetually | 23:42 |
| fungi | suggests that of the tasks coming in and out, there are 8 which it's failing to handle and leaving in the queue | 23:43 |
| anteaya | hmmm | 23:43 |
| fungi | because when the queue runner wasn't hacking away at them, that number was climbing rapidly | 23:43 |
| fungi | now it's whittled them all away again except for 8 | 23:44 |
| anteaya | here is the output from the shell running rake jobs:work http://paste.openstack.org/show/47527/ | 23:44 |
| * fungi restarts the puppet agent on the gitxx servers--i think that's the pip upgraded itself error we're seeing on them | 23:45 | |
| anteaya | okay | 23:45 |
| fungi | yeah, puppet agent restarts on those seems to be clearing them out of the failed list | 23:51 |
| *** hemna is now known as hemnafk | 23:52 | |
| fungi | oh, actually they're just shifting to the unresponsive list :/ | 23:52 |
| fungi | anteaya: is the time accurate on that dashboard server? | 23:54 |
| anteaya | never checked, I can check now | 23:54 |
| pleia2 | anteaya: dashboard looks good! | 23:54 |
| fungi | the unresponsive list is piling up quickly, which makes me think either the clock is ahead or the timeout is too short | 23:55 |
| anteaya | pleia2: thanks | 23:55 |
| anteaya | and just got another email from sodabrew - I invited him into channel if he wants | 23:56 |
| anteaya | he was saying that dashboard 1.2 workers and dashboard 2.0 workers (his dashboard is 2.0) are incompatible | 23:56 |
| anteaya | which shouldn't be an issue for us | 23:56 |
| anteaya | ubuntu@puppet-dashboard:~$ date | 23:58 |
| anteaya | Wed Sep 25 23:58:03 UTC 2013 | 23:58 |
| jog0 | clarkb: how behind is ElasticSearch right now | 23:58 |
| clarkb | jog0: ~8 hours | 23:58 |
| *** rockygrober has quit IRC | 23:59 | |
| clarkb | jeblair: moving the change in the simulation is easy. Moving the TestSeries is my nemesis. I think I have that sorted now though | 23:59 |
| clarkb | or know what to do | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
