| *** bknudson has joined #openstack-infra | 00:02 | |
| *** atiwari has quit IRC | 00:03 | |
| mordred | ok. restarted | 00:03 |
|---|---|---|
| *** mattoliverau has quit IRC | 00:04 | |
| *** yamahata has quit IRC | 00:04 | |
| *** dizquierdo has quit IRC | 00:10 | |
| *** mattoliverau has joined #openstack-infra | 00:12 | |
| openstackgerrit | A change was merged to openstack-infra/config: Move o.o user creation to it's own manifest. https://review.openstack.org/68779 | 00:19 |
| mordred | fungi: I can't log in to graphite.o.o | 00:21 |
| anteaya | fungi: shall I remove WIP on the reversion patch? 72178 | 00:24 |
| *** sarob has joined #openstack-infra | 00:24 | |
| anteaya | and morning from a few hours ago mattoliverau | 00:24 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix path to graphite db https://review.openstack.org/72193 | 00:24 |
| anteaya | how are you this fine Monday morning? | 00:24 |
| anteaya | mordred fungi so what is the direction for manage-projects? smaller pieces? still under the ministration of puppet if we can manage it? | 00:25 |
| *** CaptTofu has joined #openstack-infra | 00:26 | |
| *** AaronGr_Zzz is now known as AaronGr | 00:26 | |
| *** sarob has quit IRC | 00:28 | |
| mattoliverau | Good morning anteaya :) | 00:32 |
| *** AaronGr is now known as AaronGr_Zzz | 00:32 | |
| mattoliverau | anteaya: I'm doing well, been having another heat wave in Melbourne, today is a bit of a cool change, a great start to the week :) | 00:36 |
| mattoliverau | anteaya: how's you weekend going? | 00:36 |
| *** markmcclain has joined #openstack-infra | 00:38 | |
| anteaya | well not bad | 00:39 |
| anteaya | spent most of it reading and listening to classical music | 00:39 |
| anteaya | how hot for you? | 00:39 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 00:40 |
| anteaya | somebody from nova is awake | 00:40 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 00:41 |
| *** markmcclain has quit IRC | 00:43 | |
| fungi | anteaya: i didn't want to un-wip the revert until we had some consensus on where we want to go next. but i can confirm that the logoutput option at least seems to work for manual invocations of puppet agent --test there | 00:44 |
| fungi | mordred: i suspect puppet may be broken/off on graphite.o.o... checking | 00:45 |
| fungi | yeah, not running | 00:45 |
| fungi | starting it now | 00:46 |
| fungi | mordred: your keys are updated now | 00:48 |
| anteaya | fungi: okay, great let's let it steep for a bit and see what others have to offer | 00:48 |
| mordred | fungi: also, https://review.openstack.org/72193 | 00:49 |
| anteaya | btw I won't be around for the meeting on Tuesday, I have a conflict | 00:49 |
| anteaya | so the only items I have are the manage-projects item and the proposal to change the meeting time to make it easier to include mattoliverau | 00:49 |
| mordred | fungi: I can now confirm that the sync_db runs on every puppet invocation | 00:50 |
| *** gokrokve has joined #openstack-infra | 00:50 | |
| anteaya | mattoliverau: if you can make it to next week's meeting and speak up about the proposal to change the meeting time that would be great | 00:50 |
| mattoliverau | By next week you mean this weeks I assume ;) I'll make sure I am available and there. | 00:51 |
| *** yamahata has joined #openstack-infra | 00:54 | |
| *** yongli has joined #openstack-infra | 00:57 | |
| *** mwagner_lap has joined #openstack-infra | 01:00 | |
| anteaya | this week's, yes | 01:00 |
| anteaya | and thank you | 01:00 |
| anteaya | fungi: I am brewing up an email reply to: http://lists.openstack.org/pipermail/openstack-dev/2014-February/026552.html | 01:00 |
| anteaya | and was wondering if I could have your input: https://etherpad.openstack.org/p/41DH5FjJJ1 | 01:01 |
| fungi | anteaya: do you have a link to any discussion which took place when we blocked their access? i honestly don't recall which account it was or the circumstances surrounding it, so would like to refresh my memory | 01:04 |
| anteaya | fungi: let me dig up some logs | 01:04 |
| fungi | anteaya: thanks! | 01:04 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Correct create-cgitrepos projects.yaml versioning https://review.openstack.org/71991 | 01:09 |
| anteaya | well Dec 31 I sent this: http://lists.openstack.org/pipermail/openstack-dev/2013-December/023228.html and then Jan 9 I sent this: http://lists.openstack.org/pipermail/openstack-dev/2014-January/023818.html | 01:09 |
| anteaya | both in reference to tail-F but I don't think we had mentioned them by name yet | 01:09 |
| anteaya | still digging | 01:09 |
| anteaya | here is the service account request: http://lists.openstack.org/pipermail/openstack-infra/2013-December/000525.html | 01:11 |
| *** nosnos has joined #openstack-infra | 01:12 | |
| *** idella4 has joined #openstack-infra | 01:13 | |
| *** gothicmindfood has quit IRC | 01:13 | |
| *** gothicmindfood has joined #openstack-infra | 01:14 | |
| anteaya | got something, http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-01-07.log timestamp 2014-01-07T12:55:57 | 01:16 |
| anteaya | username kruskakli | 01:16 |
| idella4 | hello there == g'day | 01:17 |
| *** e0ne has quit IRC | 01:17 | |
| idella4 | guessing anteaya is busy; how was the conf.? | 01:21 |
| *** dims has joined #openstack-infra | 01:21 | |
| *** nosnos_ has joined #openstack-infra | 01:21 | |
| *** nosnos_ has quit IRC | 01:21 | |
| *** nosnos has quit IRC | 01:22 | |
| *** nosnos_ has joined #openstack-infra | 01:22 | |
| *** CaptTofu has quit IRC | 01:24 | |
| anteaya | fungi: part two: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-01-10.log 2014-01-10T14:28:36 | 01:24 |
| anteaya | hello idella4 | 01:24 |
| *** dkehn__ has quit IRC | 01:24 | |
| *** locke105 has quit IRC | 01:24 | |
| idella4 | hi | 01:25 |
| *** sarob has joined #openstack-infra | 01:25 | |
| *** dkehn__ has joined #openstack-infra | 01:25 | |
| anteaya | if you are interested in contributing to openstack, please do respond to the comments left on at least one of the bug reports you filed | 01:25 |
| anteaya | if you have attached yourself to me, don't | 01:25 |
| *** locke105 has joined #openstack-infra | 01:25 | |
| anteaya | I am not in the mood and am more than likely to publicly embarass you | 01:25 |
| anteaya | simply as a mechanism to ensure I have the space I need to work | 01:26 |
| *** jasondotstar has joined #openstack-infra | 01:26 | |
| anteaya | fungi: so let me know if you have any thoughts about the email response so far | 01:27 |
| idella4 | thought as much | 01:27 |
| anteaya | okay, I think that was in an email thread that was jammed in with some other stuff | 01:28 |
| anteaya | I will continue to dig | 01:28 |
| *** CaptTofu has joined #openstack-infra | 01:30 | |
| *** sarob has quit IRC | 01:30 | |
| *** yamahata has quit IRC | 01:33 | |
| mattoliverau | Hi idella4, but your g'day reference, I take it your an Aussie? | 01:33 |
| idella4 | \yes matt I am indeed | 01:33 |
| idella4 | I tend to throw the odd au term in mostly as a fun stir | 01:35 |
| mattoliverau | Me too, you'll find a few of us on here. Monday's aren't as busy as it's stil Sunday in the US. Welcome to the channel :) | 01:35 |
| idella4 | ah thanks | 01:35 |
| anteaya | fungi: part of the issue is that there seem to be two players: here is an email from luke gorrie: http://lists.openstack.org/pipermail/openstack-dev/2014-January/023480.html | 01:35 |
| mattoliverau | some American's and Canadians on the channel, just don't like taking weekends :P | 01:36 |
| anteaya | http://lists.openstack.org/pipermail/openstack-dev/2014-January/023510.html | 01:36 |
| anteaya | Jan 15 from tobbe: http://lists.openstack.org/pipermail/openstack-dev/2014-January/024357.html | 01:36 |
| anteaya | http://lists.openstack.org/pipermail/openstack-dev/2014-January/023576.html | 01:37 |
| fungi | anteaya: http://lists.openstack.org/pipermail/openstack-dev/2014-January/023480.html | 01:37 |
| fungi | i think i just found it | 01:37 |
| idella4 | well au is the land of the long weekend and you're here anyway? | 01:37 |
| idella4 | except it's in fact now Monday | 01:37 |
| fungi | anteaya: thread seems to start at http://lists.openstack.org/pipermail/openstack-dev/2014-January/023469.html | 01:38 |
| mattoliverau | idella4: lol, yeah, well although it is actually a wekeend in some parts of Oz today, it isn't here in Melbourne :( | 01:38 |
| mattoliverau | *long weekend | 01:38 |
| anteaya | fungi: yes | 01:38 |
| mattoliverau | idella4: where are you based? | 01:39 |
| idella4 | Perth | 01:39 |
| anteaya | so while Luke acknowledged the email, there was no sense of ownership, for me | 01:39 |
| fungi | anteaya: though the quoted content goes back further... finding the original thread now | 01:39 |
| mattoliverau | Ahh nice, I was there for LCA, lovely city :) | 01:39 |
| idella4 | the only gentoo dev this side of Melbourne I believe | 01:39 |
| anteaya | fungi: k | 01:39 |
| idella4 | oh you were at the conf. | 01:40 |
| mattoliverau | Cool, I once co-created a distro based of gentoo, many years back. it was called kororaa :) But haven't used gentoo in a while now days. | 01:41 |
| *** derekh has quit IRC | 01:41 | |
| idella4 | It's a water oriented city with the Swan river as it is but the state is spooked by a recent history of sharks grabbing the odd swimmer | 01:42 |
| idella4 | mostly down the south-west area of the state | 01:42 |
| idella4 | kororaa vaguely rings a bell | 01:42 |
| mattoliverau | lol, well Vicoria is currently on fire, so I think combined we can scare off all non aussies in the channel :) | 01:42 |
| idella4 | hmm yep | 01:43 |
| idella4 | still like livin' where I live | 01:43 |
| mattoliverau | We released the first XGL live CD when wiggly windows and the 3d desktop cube was brand spakin new. | 01:43 |
| mattoliverau | My best mate, has created kororaa, called it the korora project and it's now a fedora remix.. some for one Linux distra extreme to the next :P | 01:44 |
| mattoliverau | s/created/recreated | 01:44 |
| idella4 | did you do this is a private venture or was it work related in some wau? | 01:45 |
| idella4 | way? | 01:45 |
| *** zehicle_at_dell has quit IRC | 01:46 | |
| mattoliverau | This as in openstack or this as in kororaa. The first is fun and cause I get paid to, the latter was just for fun. | 01:46 |
| *** zehicle_at_dell has joined #openstack-infra | 01:46 | |
| idella4 | hmm that makes for very keen wither way | 01:47 |
| openstackgerrit | Rohan Kanade proposed a change to openstack/requirements: Adds python-seamicroclient dependency for Ironic https://review.openstack.org/70751 | 01:47 |
| fungi | anteaya: so that thread started out on december 22 with http://lists.openstack.org/pipermail/openstack-dev/2013-December/022945.html which was focused on the mellanox account | 01:47 |
| idella4 | either way | 01:47 |
| *** yamahata has joined #openstack-infra | 01:48 | |
| anteaya | fungi: yes and then expanded | 01:50 |
| anteaya | I am trying to find if I emailed directly | 01:50 |
| anteaya | so many emails to select from | 01:51 |
| fungi | i know how you feel :/ | 01:51 |
| idella4 | anyway nice to make yr acquaintance. think I'll check with the innovator of openstack @ gentoo whether he's perused those bugs I files months ago | 01:51 |
| *** yamahata has quit IRC | 01:53 | |
| mattoliverau | idella4: cool, well nice to meet you too :) | 01:55 |
| *** lttrl has quit IRC | 01:56 | |
| *** morganfainberg_Z is now known as morganfainberg | 01:56 | |
| mattoliverau | On that note I think I'll go find some lunch :) | 01:56 |
| idella4 | good move that | 01:56 |
| idella4 | meat pie with tomato sauce plus a vegemite sandwich | 01:57 |
| anteaya | fungi: that's what it was sean collins had emailed mid december: http://lists.openstack.org/pipermail/openstack-dev/2014-January/023475.html | 01:57 |
| *** lttrl has joined #openstack-infra | 01:57 | |
| anteaya | fungi: he did so privately and forwarded me the email | 01:57 |
| fungi | anteaya: oh wow... anyway, i think your text accurately portrays the situation | 01:57 |
| anteaya | fungi: stand by I will dig up the email to confirm and will forward it to you | 01:58 |
| *** yaguang has joined #openstack-infra | 02:02 | |
| anteaya | okay so back to the etherpad | 02:09 |
| anteaya | I'm open as to a next steps or thoughts for redirection as well as a closing | 02:09 |
| anteaya | line 10 is basically you and I comparing notes, I'll let you edit out any part that you think should not be in the final message, fungi | 02:10 |
| *** e0ne has joined #openstack-infra | 02:10 | |
| fungi | anteaya: yeah, that was inline commentary. deleting | 02:12 |
| anteaya | k | 02:12 |
| fungi | anteaya: while perhaps not relevant to the account at hand, note that we're now adding new accounts as members of the Third-Party CI group initially, which can comment on any project's changes but can only cast verify votes on the openstack-dev/sandbox changes | 02:13 |
| anteaya | I love that | 02:13 |
| anteaya | and yes, caught that patch merge earlier today | 02:14 |
| anteaya | the shiny penny that distracted me from remove manage-projects automation | 02:14 |
| fungi | it's still a little unclear to me how we should deal with figuring out when they're ready to be moved to the Voting Third-Party CI group (a meetbot vote during the weekly irc meeting for the project(s) on which they intend to start voting maybe? | 02:14 |
| anteaya | that sounds fair to me | 02:14 |
| anteaya | having meetbot record the votes ensures we have a record with a url | 02:15 |
| fungi | but as far as accounts getting voting access disabled now, it can be as simple as moving them from the full voting group to the one which only votes on sandbox | 02:15 |
| anteaya | yay | 02:15 |
| fungi | which gives them the ability to continue retesting their voting problems on an actual repository at least | 02:16 |
| anteaya | speaking of the sandbox, are their any repos they can track on the sandbox to test the triggers for their systems? | 02:16 |
| anteaya | ah | 02:16 |
| anteaya | the sandbox is a repo? | 02:16 |
| fungi | they can always add a change to the openstack-dev/sandbox repository on gerrit, yes | 02:16 |
| anteaya | so it is: http://git.openstack.org/cgit/openstack-dev/sandbox/ | 02:16 |
| fungi | it's just a normal project repository like any other, except with some special acl rules | 02:17 |
| *** ryanpetrello has joined #openstack-infra | 02:17 | |
| fungi | yup | 02:17 |
| anteaya | well perhaps that might be something we can mention in the wrap up? | 02:17 |
| anteaya | or should that be a separate email that is reference by this one? | 02:17 |
| fungi | i'm not sure it directly addresses the concerns raised by the initial e-mail, unfortunately | 02:17 |
| anteaya | don't want to clog the discussion with too many issues | 02:17 |
| anteaya | yes okay | 02:17 |
| anteaya | so I won't mention it in my wrap up | 02:18 |
| anteaya | trying to find something to offer | 02:18 |
| anteaya | unfortunately the tone of the last irc conversation didn't fill me with hope | 02:18 |
| fungi | and we still need to figure out which, if any, of the 40 existing voting accounts get grandfathered in or demoted, and when/how that will happen | 02:18 |
| anteaya | ah the grandfathering | 02:19 |
| *** ryanpetrello has quit IRC | 02:22 | |
| anteaya | fungi: my ending feels weak but that is all I have right now | 02:25 |
| anteaya | can you think of anything better? | 02:25 |
| *** ryanpetrello has joined #openstack-infra | 02:27 | |
| *** sarob has joined #openstack-infra | 02:28 | |
| *** yongli has quit IRC | 02:31 | |
| *** zhiwei has joined #openstack-infra | 02:31 | |
| *** sarob has quit IRC | 02:32 | |
| *** CaptTofu has quit IRC | 02:32 | |
| zhiwei | fungi: hi, I'am back. we now can pick a date to rename the stackforge project. | 02:33 |
| anteaya | fungi: I like that, can we confirm that tail-f can still provide comments? | 02:33 |
| fungi | anteaya: checking | 02:33 |
| anteaya | thanks | 02:33 |
| fungi | zhiwei: great! it's still on our meeting agenda, so we'll discuss it again on tuesday. you just missed a great opportunity for us to do it last friday since we needed to rename a bunch of oslo projects and could have done yours in the same window | 02:34 |
| zhiwei | oh, a pity. | 02:34 |
| fungi | zhiwei: there is a rename slated for savanna's projects... depending on how soon they figure out what they're changing their name to we could potentially batch your rename up with all theirs | 02:35 |
| zhiwei | ok, thank you very much. I will pay attention on this. | 02:36 |
| fungi | zhiwei: but if you can't show up for the 19:00 utc infrastructure team meeting on tuesday in #openstack-meeting, just keep your eyes on the meeting log at http://eavesdrop.openstack.org/meetings/infra/2014/ for details afterward | 02:36 |
| zhiwei | ok | 02:36 |
| *** yamahata has joined #openstack-infra | 02:36 | |
| anteaya | zhiwei: here is the savanna rename email thread: http://lists.openstack.org/pipermail/openstack-dev/2014-January/025905.html | 02:37 |
| *** ryanpetrello has quit IRC | 02:38 | |
| anteaya | fungi: yeah, I like what you have, after you have confirmed permissions on that account | 02:39 |
| *** ryanpetrello has joined #openstack-infra | 02:40 | |
| fungi | anteaya: their account appears to be intact in the db (we didn't clear their ssh keys or set their account inactive). they're not presently in the Voting Third-Party CI group but i can add them to the probationary Third-Party CI group if there's no consensus from the neutron community that they're ready to reinstate voting access | 02:41 |
| anteaya | so Third-Party CI allows them to vote on the sandbox repo as well as comment on any other repo in gerrit, correct? | 02:42 |
| fungi | yep, just as described. i've gone ahead and stuck them into that group so we don't lose track of their account (so i won't need to look it up again) | 02:43 |
| fungi | in part because i'm lazy | 02:43 |
| anteaya | yay for laziness, and we needed some account to be in that group | 02:43 |
| anteaya | okay let me add some words for that to the etherpad | 02:44 |
| fungi | well, jaypipes has a testing account in there as of a few hours ago too | 02:44 |
| anteaya | coo | 02:44 |
| anteaya | l | 02:44 |
| jaypipes | ty fungi | 02:44 |
| fungi | yw as always, jaypipes! | 02:44 |
| *** ryanpetrello has quit IRC | 02:45 | |
| *** dkehn__ is now known as dkehn_ | 02:45 | |
| anteaya | hey jaypipes | 02:47 |
| jaypipes | hey all :) | 02:48 |
| jaypipes | how are ya, Anita? | 02:48 |
| anteaya | fungi: okay any final comments on the etherpad? | 02:48 |
| anteaya | doing okay, had a concussion after code sprint, didn't know until last Wednesday | 02:48 |
| anteaya | turns out classical music, silk, amber and a good novel is good for a concussion | 02:49 |
| anteaya | finally able to collect a thought and get some work done today | 02:49 |
| *** weshay has quit IRC | 02:49 | |
| anteaya | and I was on stove duty this weekend, thought about letting you know, then figured, naw, he'll just laugh at me | 02:49 |
| jaypipes | :) | 02:49 |
| anteaya | :D | 02:49 |
| anteaya | and thou? | 02:49 |
| fungi | anteaya: i made one more minor clarification in the first paragraph, and also added a note in the second which you may or may not be able to do anything to address | 02:52 |
| * anteaya reads | 02:52 | |
| anteaya | commented | 02:57 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-dev/hacking: Remove tox locale overrides https://review.openstack.org/72206 | 02:59 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-dev/pbr: Remove tox locale overrides https://review.openstack.org/72207 | 02:59 |
| *** dstanek has joined #openstack-infra | 02:59 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/elastic-recheck: Remove tox locale overrides https://review.openstack.org/72215 | 03:01 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/gear: Remove tox locale overrides https://review.openstack.org/72216 | 03:02 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Remove tox locale overrides https://review.openstack.org/72217 | 03:02 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/os-loganalyze: Remove tox locale overrides https://review.openstack.org/72218 | 03:02 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/release-tools: Remove tox locale overrides https://review.openstack.org/72219 | 03:02 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/reviewstats: Remove tox locale overrides https://review.openstack.org/72220 | 03:03 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/storyboard-webclient: Remove tox locale overrides https://review.openstack.org/72221 | 03:03 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-dev/cookiecutter: Remove tox locale overrides https://review.openstack.org/72233 | 03:09 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack/requirements: Remove tox locale overrides https://review.openstack.org/72242 | 03:12 |
| anteaya | fungi: thanks for the help, I posted it: http://lists.openstack.org/pipermail/openstack-dev/2014-February/026728.html | 03:15 |
| fungi | anteaya: you're welcome | 03:16 |
| StevenK | fungi: Are you done with the tox spam? :-) | 03:17 |
| StevenK | fungi: And let me say *damn* for the amount of reviews you've pushed up in a short time | 03:17 |
| anteaya | good night | 03:17 |
| *** morganfainberg is now known as morganfainberg_Z | 03:19 | |
| *** morganfainberg_Z is now known as morganfainberg | 03:20 | |
| fungi | anteaya: good night! | 03:20 |
| fungi | StevenK: almost ;) | 03:22 |
| StevenK | fungi: Heh | 03:22 |
| fungi | StevenK: *now* i am | 03:24 |
| StevenK | Heh | 03:24 |
| fungi | there's nothing like a patch blast to 50 projects to round out a weekend | 03:25 |
| *** dkranz has quit IRC | 03:25 | |
| fungi | and now, i'm off to enjoy the rest of my sunday | 03:25 |
| mordred | fungi: you COULD have put a topic on those so that I could easily search and script votes | 03:27 |
| fungi | mordred: yeah, i noticed that *after* i submitted them :/ | 03:27 |
| fungi | totally meant to add a -t footopic and didn't | 03:28 |
| *** sarob has joined #openstack-infra | 03:28 | |
| mordred | fungi: it's ok- I think I've got a search | 03:29 |
| fungi | i figure people will merge them in their own time... some faster than others if they're blocked on getting py33 unit tests to pass and mine's the only change getting a +1 from jenkins ;) | 03:29 |
| *** harlowja is now known as harlowja_away | 03:30 | |
| *** markmcclain has joined #openstack-infra | 03:31 | |
| mordred | fungi: I'm +1-ing all of them | 03:32 |
| fungi | looks like gear and release-tools have some other bitrot issues to work out first | 03:32 |
| mordred | mainly because I'm low on review stats right now | 03:32 |
| fungi | heh | 03:32 |
| fungi | fair enough. i was low of commit stats ;) | 03:33 |
| fungi | er, low on | 03:33 |
| mordred | even at this point with mostly being awol I don't think I'm low on commit stats | 03:33 |
| fungi | i think you've stocked up enough commits to survive through a nuclear winter | 03:34 |
| mordred | haha | 03:34 |
| *** zhiyan has joined #openstack-infra | 03:34 | |
| *** sarob has quit IRC | 03:34 | |
| fungi | okay, i swear. this time i'm really stepping away from the computer. catch up with everybody in the morning | 03:34 |
| mordred | bah | 03:35 |
| mordred | you'll be back | 03:35 |
| fungi | i will, but only after a short nap ;) | 03:35 |
| *** idella4 has left #openstack-infra | 03:35 | |
| StevenK | fungi: Do you just watch for a few minutes after saying you're stepping away? :-P | 03:35 |
| fungi | oh, and i guess i have expense reports to file too | 03:35 |
| fungi | StevenK: no, i just keep failing at the task at hand | 03:36 |
| * fungi -> up, up and /away! | 03:36 | |
| StevenK | fungi: Funnily enough, I'm just looking at that too | 03:36 |
| *** sarob has joined #openstack-infra | 03:38 | |
| *** banix has quit IRC | 03:40 | |
| mordred | StevenK: you'll love it | 03:42 |
| mordred | StevenK: and by that - I do mean that you WILL love it | 03:42 |
| *** e0ne has quit IRC | 03:42 | |
| * StevenK is looking at the training for it | 03:43 | |
| *** sarob has quit IRC | 03:43 | |
| markmcclain | mordred: thanks for abandoning that review for me | 03:44 |
| mordred | markmcclain: I do what I can to be useful | 03:44 |
| mordred | markmcclain: which is less and less... | 03:44 |
| markmcclain | mordred: hardly… less is more right? | 03:48 |
| openstackgerrit | YAMAMOTO Takashi proposed a change to openstack-dev/hacking: HACKING.rst: Fix odd indentation in an example code https://review.openstack.org/72255 | 03:49 |
| *** mriedem has joined #openstack-infra | 04:10 | |
| *** e0ne has joined #openstack-infra | 04:10 | |
| *** jasondotstar has quit IRC | 04:21 | |
| *** sarob has joined #openstack-infra | 04:32 | |
| *** CaptTofu has joined #openstack-infra | 04:33 | |
| *** sarob has quit IRC | 04:36 | |
| *** CaptTofu has quit IRC | 04:38 | |
| *** jasondotstar has joined #openstack-infra | 04:53 | |
| *** talluri has joined #openstack-infra | 04:56 | |
| *** talluri has quit IRC | 05:08 | |
| *** talluri has joined #openstack-infra | 05:08 | |
| *** talluri has quit IRC | 05:12 | |
| *** chandankumar_ has joined #openstack-infra | 05:17 | |
| *** david_lyle_ has joined #openstack-infra | 05:20 | |
| *** nicedice has quit IRC | 05:20 | |
| *** david_lyle_ is now known as david_lyle | 05:22 | |
| *** david-lyle has quit IRC | 05:23 | |
| *** VijayT has joined #openstack-infra | 05:23 | |
| *** e0ne has quit IRC | 05:24 | |
| *** sarob has joined #openstack-infra | 05:32 | |
| *** coolsvap has joined #openstack-infra | 05:38 | |
| *** sarob has quit IRC | 05:39 | |
| *** mriedem has quit IRC | 05:42 | |
| *** sarob has joined #openstack-infra | 05:42 | |
| *** markwash has joined #openstack-infra | 05:44 | |
| *** mriedem has joined #openstack-infra | 05:45 | |
| *** AaronGr_Zzz is now known as AaronGr | 05:46 | |
| *** mriedem has quit IRC | 05:47 | |
| *** sarob has quit IRC | 05:47 | |
| *** markwash has quit IRC | 05:48 | |
| *** AaronGr is now known as AaronGr_Zzz | 05:48 | |
| *** markwash has joined #openstack-infra | 05:49 | |
| *** nosnos has joined #openstack-infra | 05:56 | |
| *** e0ne has joined #openstack-infra | 05:56 | |
| *** nosnos has quit IRC | 05:57 | |
| *** nosnos has joined #openstack-infra | 05:58 | |
| *** nosnos has quit IRC | 05:58 | |
| *** boris-42_ has quit IRC | 05:58 | |
| *** nosnos has joined #openstack-infra | 05:59 | |
| *** Ryan_Lane has quit IRC | 05:59 | |
| *** nosnos_ has quit IRC | 05:59 | |
| *** e0ne has quit IRC | 06:00 | |
| *** jasondotstar has quit IRC | 06:02 | |
| *** david_lyle_ has joined #openstack-infra | 06:04 | |
| *** amotoki has joined #openstack-infra | 06:05 | |
| *** david_lyle has quit IRC | 06:05 | |
| *** Ryan_Lane has joined #openstack-infra | 06:09 | |
| *** markmcclain has quit IRC | 06:13 | |
| *** markmcclain has joined #openstack-infra | 06:16 | |
| *** markmcclain1 has joined #openstack-infra | 06:18 | |
| *** markmcclain has quit IRC | 06:21 | |
| *** oubiwann_ has quit IRC | 06:27 | |
| jgriffith | Anybody else looked at this: http://logs.openstack.org/91/69191/2/check/check-tempest-dsvm-full/c63b067/logs/screen-n-cpu.txt.gz#_2014-02-10_04_27_30_327 | 06:31 |
| jgriffith | This just started showing up today... | 06:32 |
| jgriffith | funny thing is it seems "bogus" | 06:32 |
| jgriffith | as in cinder doesn't return that response for that call.... | 06:32 |
| jgriffith | infact AFAICT it doesn't return that response for anything | 06:32 |
| jgriffith | and I don't see that error in the Cinder logs anywhere either | 06:33 |
| *** CaptTofu has joined #openstack-infra | 06:34 | |
| *** sarob has joined #openstack-infra | 06:36 | |
| *** CaptTofu has quit IRC | 06:39 | |
| *** sarob has quit IRC | 06:40 | |
| *** markmcclain1 has quit IRC | 06:42 | |
| *** nosnos has quit IRC | 06:47 | |
| *** nosnos_ has joined #openstack-infra | 06:48 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 06:52 | |
| *** saju_m has joined #openstack-infra | 06:53 | |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 06:55 | |
| *** saju_m has quit IRC | 07:00 | |
| *** david_lyle_ has quit IRC | 07:02 | |
| *** chandankumar_ has quit IRC | 07:07 | |
| *** chandan_kumar has joined #openstack-infra | 07:07 | |
| *** david-lyle has joined #openstack-infra | 07:12 | |
| *** saju_m has joined #openstack-infra | 07:16 | |
| *** igor has joined #openstack-infra | 07:16 | |
| *** dims has quit IRC | 07:18 | |
| *** dims has joined #openstack-infra | 07:19 | |
| *** igor has quit IRC | 07:20 | |
| *** igor has joined #openstack-infra | 07:21 | |
| *** pblaho has joined #openstack-infra | 07:23 | |
| *** yolanda has joined #openstack-infra | 07:30 | |
| *** rlandy has joined #openstack-infra | 07:34 | |
| *** sarob has joined #openstack-infra | 07:36 | |
| *** DeeJay1 has joined #openstack-infra | 07:37 | |
| *** plomakin_ has quit IRC | 07:39 | |
| *** plomakin has joined #openstack-infra | 07:39 | |
| *** sarob has quit IRC | 07:43 | |
| *** sarob has joined #openstack-infra | 07:46 | |
| *** HenryG has quit IRC | 07:47 | |
| *** HenryG has joined #openstack-infra | 07:48 | |
| *** rockstar has quit IRC | 07:48 | |
| *** gokrokve has quit IRC | 07:48 | |
| *** rockstar has joined #openstack-infra | 07:48 | |
| *** rockstar has quit IRC | 07:48 | |
| *** rockstar has joined #openstack-infra | 07:48 | |
| *** sarob has quit IRC | 07:51 | |
| *** boris-42_ has joined #openstack-infra | 07:57 | |
| *** salv-orlando has joined #openstack-infra | 07:59 | |
| *** markwash has quit IRC | 07:59 | |
| *** VijayT has quit IRC | 08:00 | |
| *** nosnos_ has quit IRC | 08:02 | |
| *** nosnos has joined #openstack-infra | 08:02 | |
| *** ekarlso has quit IRC | 08:04 | |
| *** ekarlso has joined #openstack-infra | 08:05 | |
| *** ekarlso has quit IRC | 08:12 | |
| *** coolsvap1 has joined #openstack-infra | 08:13 | |
| *** ekarlso has joined #openstack-infra | 08:13 | |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 08:16 | |
| *** coolsvap has quit IRC | 08:17 | |
| SergeyLukjanov | fungi, great job with proposing locale overrides removal, +1s from me to all of them except https://review.openstack.org/#/c/72208/ | 08:17 |
| *** jcoufal has joined #openstack-infra | 08:18 | |
| *** gokrokve has joined #openstack-infra | 08:18 | |
| *** e0ne has joined #openstack-infra | 08:19 | |
| *** gokrokve_ has joined #openstack-infra | 08:20 | |
| *** pleia2 has quit IRC | 08:21 | |
| *** morganfainberg has quit IRC | 08:22 | |
| *** pleia2 has joined #openstack-infra | 08:23 | |
| *** morganfainberg has joined #openstack-infra | 08:23 | |
| *** gokrokve has quit IRC | 08:23 | |
| *** gokrokve_ has quit IRC | 08:26 | |
| *** hashar has joined #openstack-infra | 08:27 | |
| *** e0ne has quit IRC | 08:28 | |
| openstackgerrit | Lukasz Jernas proposed a change to openstack-infra/config: Add fonts-takao package on the Debian build slave https://review.openstack.org/72287 | 08:28 |
| *** Ryan_Lane has quit IRC | 08:30 | |
| *** coolsvap1 has quit IRC | 08:33 | |
| *** CaptTofu has joined #openstack-infra | 08:35 | |
| *** CaptTofu has quit IRC | 08:39 | |
| openstackgerrit | Ruslan Kamaldinov proposed a change to openstack-infra/storyboard: Added base for functional tests https://review.openstack.org/69384 | 08:49 |
| *** flaper87|afk is now known as flaper87 | 08:49 | |
| *** mlipchuk has joined #openstack-infra | 08:51 | |
| DeeJay1 | shouldn't https://review.openstack.org/#/c/72287/ post changes to the associated bug ? | 08:54 |
| *** jooools has joined #openstack-infra | 08:56 | |
| *** matrohon has joined #openstack-infra | 08:58 | |
| *** abhishek has joined #openstack-infra | 08:59 | |
| *** zhiwei has quit IRC | 08:59 | |
| *** yolanda has quit IRC | 09:04 | |
| *** jpich has joined #openstack-infra | 09:04 | |
| *** yolanda has joined #openstack-infra | 09:05 | |
| openstackgerrit | Flavio Percoco proposed a change to openstack-infra/devstack-gate: Archive config files along with logs https://review.openstack.org/69344 | 09:06 |
| *** yolanda has quit IRC | 09:06 | |
| *** e0ne has joined #openstack-infra | 09:07 | |
| *** yolanda has joined #openstack-infra | 09:07 | |
| *** yolanda has quit IRC | 09:08 | |
| *** coolsvap has joined #openstack-infra | 09:08 | |
| *** derekh has joined #openstack-infra | 09:08 | |
| *** yolanda has joined #openstack-infra | 09:12 | |
| *** yassine has joined #openstack-infra | 09:12 | |
| *** yolanda has quit IRC | 09:14 | |
| *** yolanda has joined #openstack-infra | 09:16 | |
| *** yolanda has quit IRC | 09:16 | |
| *** yolanda has joined #openstack-infra | 09:17 | |
| *** yolanda has quit IRC | 09:17 | |
| *** yolanda has joined #openstack-infra | 09:18 | |
| *** yolanda has quit IRC | 09:19 | |
| *** yolanda has joined #openstack-infra | 09:19 | |
| *** gokrokve has joined #openstack-infra | 09:21 | |
| *** mkerrin has joined #openstack-infra | 09:23 | |
| *** yolanda has quit IRC | 09:23 | |
| *** yolanda has joined #openstack-infra | 09:23 | |
| *** yolanda has quit IRC | 09:25 | |
| *** gokrokve has quit IRC | 09:26 | |
| *** yolanda has joined #openstack-infra | 09:26 | |
| *** zhiwei has joined #openstack-infra | 09:27 | |
| *** morganfainberg is now known as morganfainberg_Z | 09:28 | |
| *** afazekas has joined #openstack-infra | 09:29 | |
| *** coolsvap1 has joined #openstack-infra | 09:32 | |
| *** krtaylor has quit IRC | 09:33 | |
| *** krtaylor has joined #openstack-infra | 09:34 | |
| *** yolanda has quit IRC | 09:34 | |
| *** yolanda has joined #openstack-infra | 09:35 | |
| *** coolsvap has quit IRC | 09:35 | |
| *** jcoufal has quit IRC | 09:36 | |
| *** jcoufal has joined #openstack-infra | 09:36 | |
| *** e0ne_ has joined #openstack-infra | 09:38 | |
| *** e0ne has quit IRC | 09:38 | |
| *** sarob has joined #openstack-infra | 09:40 | |
| *** fbo_away is now known as fbo | 09:42 | |
| *** sarob has quit IRC | 09:47 | |
| *** sarob has joined #openstack-infra | 09:50 | |
| abhishek | Hi, Can any one help me regarding this error | 09:55 |
| *** sarob has quit IRC | 09:55 | |
| abhishek | + git clone git://git.openstack.org/openstack-infra/devstack-gate 2014-02-10 08:37:46.680 | fatal: unable to connect to git.openstack.org: 2014-02-10 08:37:46.680 | git.openstack.org: Temporary failure in name resolution | 09:55 |
| *** jasondotstar has joined #openstack-infra | 10:00 | |
| *** jp_at_hp has joined #openstack-infra | 10:01 | |
| *** markmc has joined #openstack-infra | 10:10 | |
| openstackgerrit | Akihiro Motoki proposed a change to openstack-infra/zuul: Correct the example of zuul enqueue command https://review.openstack.org/72308 | 10:10 |
| *** dizquierdo has joined #openstack-infra | 10:17 | |
| *** mrmartin has joined #openstack-infra | 10:18 | |
| *** gokrokve has joined #openstack-infra | 10:21 | |
| *** DinaBelova_ is now known as DinaBelova | 10:21 | |
| *** andre__ has joined #openstack-infra | 10:23 | |
| abhishek | hi can any one help me regarding the error fatal: unable to connect to git.openstack.org | 10:25 |
| abhishek | I am getting this error on patch set https://review.openstack.org/#/c/71913/5 | 10:25 |
| abhishek | how can I resolve this error> | 10:25 |
| *** gokrokve has quit IRC | 10:26 | |
| *** zhiyan has quit IRC | 10:27 | |
| *** zhiyan has joined #openstack-infra | 10:30 | |
| *** jcoufal has quit IRC | 10:32 | |
| *** zhiyan has quit IRC | 10:34 | |
| *** CaptTofu has joined #openstack-infra | 10:36 | |
| *** amotoki has quit IRC | 10:38 | |
| openstackgerrit | Sergey Kolekonov proposed a change to openstack-infra/jenkins-job-builder: Heavy Job Plugin support added https://review.openstack.org/72313 | 10:39 |
| *** jcoufal has joined #openstack-infra | 10:39 | |
| *** CaptTofu has quit IRC | 10:41 | |
| *** yamahata has quit IRC | 10:41 | |
| openstackgerrit | Sergey Kolekonov proposed a change to openstack-infra/jenkins-job-builder: Heavy Job Plugin support added https://review.openstack.org/72313 | 10:41 |
| *** coolsvap_ has joined #openstack-infra | 10:43 | |
| SergeyLukjanov | DeeJay1, issue is under the i18n project and CR is to infra | 10:44 |
| *** sarob has joined #openstack-infra | 10:44 | |
| openstackgerrit | Sergey Kolekonov proposed a change to openstack-infra/jenkins-job-builder: Heavy Job Plugin support added https://review.openstack.org/72313 | 10:44 |
| SergeyLukjanov | abhishek, hi, is it the local error or in jenkins jobs? | 10:44 |
| *** yaguang has quit IRC | 10:45 | |
| *** coolsvap1 has quit IRC | 10:46 | |
| *** coolsvap1 has joined #openstack-infra | 10:48 | |
| *** sarob has quit IRC | 10:48 | |
| *** zhiwei has left #openstack-infra | 10:51 | |
| *** coolsvap_ has quit IRC | 10:51 | |
| *** CaptTofu has joined #openstack-infra | 10:52 | |
| *** coolsvap has joined #openstack-infra | 10:52 | |
| *** yassine has quit IRC | 10:53 | |
| *** coolsvap1 has quit IRC | 10:53 | |
| DeeJay1 | SergeyLukjanov: yeah, I couldn't find the infra project somehow, and issue affects both i18n and infra | 10:57 |
| SergeyLukjanov | DeeJay1, it's named openstack-ci at launchpad | 10:57 |
| abhishek | @SergeyLukjanov: its in jenkins job | 10:58 |
| * DeeJay1 facepalms | 10:58 | |
| SergeyLukjanov | abhishek, https://bugs.launchpad.net/openstack-ci/+bug/1277751, https://bugs.launchpad.net/openstack-ci/+bug/1270382 | 10:59 |
| uvirtbot | Launchpad bug 1277751 in openstack-ci "fatal: unable to connect to git.openstack.org" [Undecided,New] | 10:59 |
| SergeyLukjanov | abhishek, as you can see, 1277751 is on top of http://status.openstack.org/rechecks/ | 11:00 |
| *** jasondotstar has quit IRC | 11:00 | |
| *** ArxCruz has joined #openstack-infra | 11:01 | |
| abhishek | @SergeyLukjanov:so recheck bug 1277751 will work for me right? | 11:01 |
| uvirtbot | Launchpad bug 1277751 in openstack-ci "fatal: unable to connect to git.openstack.org" [Undecided,New] https://launchpad.net/bugs/1277751 | 11:01 |
| SergeyLukjanov | abhishek, yup, but 1270382 is more relevant IMO | 11:02 |
| SergeyLukjanov | abhishek, due to the "git.openstack.org: Temporary failure in name resolution" | 11:02 |
| abhishek | yes, thanks a lot.... | 11:03 |
| *** jasondotstar has joined #openstack-infra | 11:06 | |
| *** mrmartin has quit IRC | 11:16 | |
| *** gokrokve has joined #openstack-infra | 11:21 | |
| *** gokrokve has quit IRC | 11:25 | |
| jpich | Is the linking to ChangeId currently broken (ChangeId urls show as https://review.openstack.org/#q,$1,n,z ) a known issue? I can't quite tell if https://bugs.launchpad.net/openstack-ci/+bug/1265646 is meant to fix it after a restart or if it's something else | 11:28 |
| uvirtbot | Launchpad bug 1265646 in openstack-ci "Links to ChangeIds from Gerrit comments are broken" [Undecided,Fix released] | 11:28 |
| *** CaptTofu has quit IRC | 11:29 | |
| jpich | e.g. https://review.openstack.org/#/c/69526/ in the commit message or any review really | 11:29 |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard-webclient: [WIP] MVP Storyboard Client https://review.openstack.org/70897 | 11:34 |
| *** jasondotstar has quit IRC | 11:35 | |
| markmc | jpich, yeah, I noticed that earlier too ... file a bug? | 11:36 |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard-webclient: [WIP] MVP Storyboard Client https://review.openstack.org/70897 | 11:37 |
| *** DinaBelova is now known as DinaBelova_ | 11:39 | |
| jpich | markmc: I supposed it can simply be marked as duplicate if it is one. Opened https://bugs.launchpad.net/openstack-ci/+bug/1278412 | 11:40 |
| uvirtbot | Launchpad bug 1278412 in openstack-ci "ChangeId linking broken in commit message" [Undecided,New] | 11:40 |
| *** sarob has joined #openstack-infra | 11:44 | |
| *** sarob has quit IRC | 11:51 | |
| *** rfolco has joined #openstack-infra | 11:54 | |
| *** HenryG has quit IRC | 11:58 | |
| *** CaptTofu has joined #openstack-infra | 11:59 | |
| *** pleia2 has quit IRC | 12:01 | |
| *** pleia2 has joined #openstack-infra | 12:02 | |
| *** CaptTofu has quit IRC | 12:04 | |
| *** DinaBelova_ is now known as DinaBelova | 12:05 | |
| *** pleia2 has quit IRC | 12:07 | |
| *** pafuent has joined #openstack-infra | 12:07 | |
| *** pleia2 has joined #openstack-infra | 12:07 | |
| *** pafuent has left #openstack-infra | 12:08 | |
| *** pafuent has joined #openstack-infra | 12:09 | |
| openstackgerrit | Zang MingJie proposed a change to openstack-infra/zuul: Use ssh to fetch packs instead of HTTP https://review.openstack.org/67858 | 12:14 |
| *** HenryG has joined #openstack-infra | 12:14 | |
| openstackgerrit | Zang MingJie proposed a change to openstack-infra/zuul: Use ssh to fetch packs instead of HTTP https://review.openstack.org/67858 | 12:16 |
| openstackgerrit | Sergey Kolekonov proposed a change to openstack-infra/jenkins-job-builder: Heavy Job Plugin support added https://review.openstack.org/72313 | 12:17 |
| *** CaptTofu has joined #openstack-infra | 12:21 | |
| *** gokrokve has joined #openstack-infra | 12:21 | |
| chmouel | is jenkins verifying everytime we add whatever comments now? | 12:21 |
| *** yamahata has joined #openstack-infra | 12:22 | |
| kiall | chmouel: I hope not, because if it is, things are probably pretty close to melting right now ;) Seriously though - I haven't noticed any extra test runs on the last few reviews I've done | 12:22 |
| chmouel | kiall: ii get all those emails saying "Patch Set 8: No score' everytime i add a new comment now | 12:23 |
| chmouel | (or whatever number) | 12:23 |
| chmouel | so i don't know if it's just testing or just add a new comment | 12:23 |
| kiall | From Jenkins, or yourself? | 12:24 |
| chmouel | jenkins | 12:24 |
| *** masayukig has joined #openstack-infra | 12:24 | |
| kiall | On https://review.openstack.org/#/settings/preferences - is "CC Me On Comments I Write" ticked? | 12:24 |
| chmouel | i.e: https://review.openstack.org/#/c/65660/ | 12:24 |
| chmouel | i just add a +2 without approve and he started the check jobs | 12:25 |
| *** gokrokve has quit IRC | 12:25 | |
| *** CaptTofu has quit IRC | 12:26 | |
| kiall | Ah - That does look a little off .. | 12:26 |
| kiall | Nothing I can do to fix it though ;) | 12:27 |
| tristanC | oh, it even restart check jobs (when comment are added) | 12:28 |
| kiall | https://review.openstack.org/#/c/65359/ <-- Just +2'd this without approval, Jenkins doesn't seem to be re-running | 12:28 |
| openstackgerrit | Andreas Rehn proposed a change to openstack-infra/jenkins-job-builder: Added support for Delivery Pipeline Plugin https://review.openstack.org/71658 | 12:28 |
| tristanC | kiall: jenkins job are running in zuul for that one https://review.openstack.org/69187 | 12:29 |
| chmouel | maybe that depend by proj who knows :) | 12:29 |
| kiall | tristanC: wrong link ;) | 12:30 |
| *** CaptTofu has joined #openstack-infra | 12:31 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack-infra/elastic-recheck: Add query for cinder bug #1278416 https://review.openstack.org/72329 | 12:31 |
| uvirtbot | openstackgerrit: Error: Could not parse data returned by Launchpad: The read operation timed out | 12:31 |
| *** CaptTofu has quit IRC | 12:32 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack-infra/elastic-recheck: Add query for cinder bug #1278416 https://review.openstack.org/72329 | 12:32 |
| uvirtbot | Launchpad bug 1278416 in cinder "remove_iscsi_target() fails with ENOENT trying to unlink file" [High,New] https://launchpad.net/bugs/1278416 | 12:32 |
| tristanC | kiall: well chmouel posted a comments on that review, and jenkins is now restarting the tests | 12:32 |
| *** CaptTofu has joined #openstack-infra | 12:32 | |
| *** pafuent has quit IRC | 12:32 | |
| mordred | fungi: morning | 12:32 |
| kiall | Oh, yea... I saw that and tried the same thing (leaving a +2, with no approval) on another review.. it didn't seem to kick Jenkins off again | 12:33 |
| *** CaptTofu_ has joined #openstack-infra | 12:35 | |
| *** CaptTofu has quit IRC | 12:35 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack/requirements: Oslo program has adopted cliff, pycadf, stevedore, and taskflow https://review.openstack.org/72330 | 12:36 |
| *** bookwar has joined #openstack-infra | 12:36 | |
| openstackgerrit | Andreas Rehn proposed a change to openstack-infra/jenkins-job-builder: Added support for Delivery Pipeline Plugin https://review.openstack.org/71658 | 12:38 |
| *** david_lyle_ has joined #openstack-infra | 12:38 | |
| *** bhuvan has joined #openstack-infra | 12:38 | |
| *** smarcet has joined #openstack-infra | 12:39 | |
| *** abhishek has quit IRC | 12:40 | |
| *** david-lyle has quit IRC | 12:42 | |
| openstackgerrit | Andreas Rehn proposed a change to openstack-infra/jenkins-job-builder: Added support for Delivery Pipeline Plugin https://review.openstack.org/71658 | 12:43 |
| *** banix has joined #openstack-infra | 12:45 | |
| *** sarob has joined #openstack-infra | 12:47 | |
| openstackgerrit | Andreas Rehn proposed a change to openstack-infra/jenkins-job-builder: Added support for Delivery Pipeline Plugin https://review.openstack.org/71658 | 12:48 |
| *** coolsvap has quit IRC | 12:50 | |
| *** banix has quit IRC | 12:50 | |
| *** pafuent has joined #openstack-infra | 12:51 | |
| *** sarob has quit IRC | 12:52 | |
| *** masayukig has quit IRC | 12:53 | |
| *** masayukig has joined #openstack-infra | 12:53 | |
| *** nosnos_ has joined #openstack-infra | 12:58 | |
| openstackgerrit | Sascha Peilicke proposed a change to openstack/requirements: Bump lesscpy to >=0.10.1 https://review.openstack.org/70619 | 12:59 |
| *** nosnos has quit IRC | 13:01 | |
| jd__ | chmouel: yep it seems there's something like that, I just saw that pattern too – someone comment and Jenkins start checking against :) | 13:02 |
| *** max_lobur_afk is now known as max_lobur | 13:03 | |
| *** nosnos_ has quit IRC | 13:03 | |
| *** pafuent has quit IRC | 13:04 | |
| *** CaptTofu_ has quit IRC | 13:05 | |
| *** nosnos has joined #openstack-infra | 13:07 | |
| mordred | jd__, chmouel: in zuul, zuul reads all comments, but it has a filter to tell if it should care about the comment or not | 13:10 |
| *** freyes has joined #openstack-infra | 13:10 | |
| mordred | wait - are you saying OpenStack's jenkins is running on every comment? | 13:11 |
| *** saju_m has quit IRC | 13:12 | |
| *** ilyashakhat has quit IRC | 13:14 | |
| SergeyLukjanov | mordred, jd__, chmouel, I think that it's because the latest jenkins run was >48h ago | 13:14 |
| SergeyLukjanov | it was about https://review.openstack.org/#/c/69187/ | 13:15 |
| SergeyLukjanov | as you can see the prev. jenkins run was Feb 6 | 13:15 |
| *** yassine has joined #openstack-infra | 13:16 | |
| *** pdmars has joined #openstack-infra | 13:17 | |
| *** gokrokve has joined #openstack-infra | 13:21 | |
| *** bhuvan has quit IRC | 13:24 | |
| mordred | SergeyLukjanov: grumble. that's not good | 13:24 |
| SergeyLukjanov | mordred, why? we're going with it for about a week ;) | 13:24 |
| mordred | oh | 13:25 |
| mordred | gotcha. I totally understand now | 13:25 |
| mordred | this is the re-check-stale-patches code | 13:25 |
| SergeyLukjanov | mordred, exactly | 13:26 |
| SergeyLukjanov | mordred, the only cons is that now jenkins post "+0, I'm running jobs" each time ;) | 13:26 |
| SergeyLukjanov | btw https://github.com/openstack-infra/config/commit/4a40990c9d1c3982aa06c7efb0cb1cd085ed7616 | 13:27 |
| mordred | yeah - I remember this now | 13:27 |
| *** gokrokve has quit IRC | 13:27 | |
| chmouel | hopeflly the mail server scales because that's a lot of emails being sent :) | 13:28 |
| jd__ | :) | 13:28 |
| mordred | of all of the things I'm worried about scaling - I have 100% confidence in jeblair's ability to scale a mail server | 13:28 |
| mordred | (you should have seen the mail cluster he ran before he worked on openstack) | 13:29 |
| jd__ | hm I just had 3 python26 jobs lost | 13:29 |
| jd__ | and dvsm | 13:30 |
| *** bhuvan has joined #openstack-infra | 13:31 | |
| jd__ | ah I guess it's because new patchset has been uploaded in the meantime | 13:31 |
| *** mrmartin has joined #openstack-infra | 13:31 | |
| jd__ | I was not used to that :) | 13:31 |
| *** ilyashakhat has joined #openstack-infra | 13:32 | |
| *** masayukig has quit IRC | 13:32 | |
| *** masayukig has joined #openstack-infra | 13:33 | |
| SergeyLukjanov | mordred, are you sure that zuul could work ok with two identical jobs in one pipeline? (https://review.openstack.org/#/c/71947/1) | 13:33 |
| *** mrodden1 has quit IRC | 13:35 | |
| mordred | SergeyLukjanov: I believe we've been doing it for a while in other places, yeah | 13:35 |
| *** masayukig has quit IRC | 13:37 | |
| anteaya | can someone else confirm that zuul has almost every patch in check and gate queued? | 13:37 |
| SergeyLukjanov | mordred, yup, I've checked this on zuul that handles savanna-ci and it looks like it works correctly. | 13:39 |
| SergeyLukjanov | anteaya, mm, I'm not sure that I understand what does you mean :( | 13:40 |
| anteaya | there are very few tests running | 13:41 |
| anteaya | tests are queued, not running | 13:41 |
| anteaya | then timing out | 13:41 |
| SergeyLukjanov | anteaya, it looks like there are no free build nodes, so, it shoud be ok | 13:42 |
| sdague | fyi - https://review.openstack.org/#/c/71947/ - looks like gerrit is no longer linking ids correctly | 13:44 |
| anteaya | morning sdague | 13:44 |
| sdague | morning | 13:44 |
| anteaya | also looks like there is a git.o.o issue: https://bugs.launchpad.net/openstack-ci/+bug/1277751 | 13:45 |
| uvirtbot | Launchpad bug 1277751 in openstack-ci "fatal: unable to connect to git.openstack.org" [Undecided,New] | 13:45 |
| sdague | anteaya: yeh, that seems to come and go | 13:45 |
| anteaya | though I can get there via the gui and clone a repo | 13:45 |
| SergeyLukjanov | sdague, ack, link is https://review.openstack.org/#q,$1,n,z atm | 13:45 |
| anteaya | great, a random failure Monday | 13:45 |
| anteaya | we organized it just for you, sdague, as a welcome home | 13:46 |
| chmouel | mordred: i think it would be nice to have an email describing the latest update on the new things hapenning between LOST and the recheck for the 48 hours change | 13:46 |
| *** max_lobur is now known as max_lobur_afk | 13:47 | |
| anteaya | sdague: sorry I am not seeing the failure to link ids correctly in the 71947 patch | 13:47 |
| *** dims has quit IRC | 13:47 | |
| *** sarob has joined #openstack-infra | 13:48 | |
| *** mrodden has joined #openstack-infra | 13:49 | |
| *** dims has joined #openstack-infra | 13:50 | |
| *** ryanpetrello has joined #openstack-infra | 13:54 | |
| hughsaunders | hey all, just noticed a "LOST" status for jenkins jobs: https://review.openstack.org/#/c/71471/2 "gate-rally-pep8 LOST in 2m 36s" | 13:54 |
| *** sarob has quit IRC | 13:54 | |
| *** yamahata has quit IRC | 13:56 | |
| *** eharney has joined #openstack-infra | 13:56 | |
| *** oanufriev has joined #openstack-infra | 13:57 | |
| anteaya | hughsaunders: yes, zuul seems to be having some issues this morning though we haven't figured out why yet | 13:57 |
| anteaya | some neutron patches are also reporting lost test logs | 13:57 |
| hughsaunders | anteaya: thanks for looking into it :) | 13:58 |
| *** dklyle has joined #openstack-infra | 13:58 | |
| anteaya | thanks for reporting your findings | 13:58 |
| *** sarob has joined #openstack-infra | 13:58 | |
| *** tru_tru has joined #openstack-infra | 13:59 | |
| *** thomasem has joined #openstack-infra | 13:59 | |
| *** yamahata has joined #openstack-infra | 13:59 | |
| anteaya | the /srv/static/log storage still has what appears to be about 2T worth of space: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=13 | 14:00 |
| *** mfer has joined #openstack-infra | 14:01 | |
| *** david_lyle_ has quit IRC | 14:01 | |
| sdague | anteaya: try clicking on any of those links in the commit message | 14:01 |
| anteaya | sdague: ah ha | 14:02 |
| anteaya | no worky | 14:02 |
| *** sarob has quit IRC | 14:03 | |
| anteaya | the cpu usage graph for static.o.o spent an aweful lot of time in IOwait for many hours a while ago: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=13&page=2 | 14:03 |
| jpich | sdague, anteaya: I reported the ChangeId issue this morning, fwiw -> https://bugs.launchpad.net/openstack-ci/+bug/1278412 | 14:03 |
| uvirtbot | Launchpad bug 1278412 in openstack-ci "ChangeId linking broken in commit message" [Undecided,New] | 14:03 |
| *** oubiwann_ has joined #openstack-infra | 14:03 | |
| anteaya | almost like some sort of scheduled job was running, or something was unavailable, for a period of time and then it went away | 14:04 |
| anteaya | thanks jpich | 14:04 |
| anteaya | jpich: so the first time you noticed it was about 2 hours ago? | 14:05 |
| *** weshay has joined #openstack-infra | 14:05 | |
| jpich | anteaya: Give or take, yup | 14:06 |
| anteaya | device reads really spiked for a while there, in the IO operations | 14:06 |
| anteaya | jpich: k, trying to track down a time line, something spiked a while ago, might be related | 14:06 |
| anteaya | spiked or disappeared | 14:07 |
| *** esker has joined #openstack-infra | 14:07 | |
| anteaya | anyone know what time zone the cacti graphs use? | 14:07 |
| anteaya | if it is utc something kicked off about 8 hours ago | 14:08 |
| *** dprince has joined #openstack-infra | 14:09 | |
| *** oubiwan__ has joined #openstack-infra | 14:09 | |
| *** e0ne has joined #openstack-infra | 14:10 | |
| anteaya | well I looked at all the cacti graphs for static.o.o all I can confirm is that we have space for logs and something happened a few hours ago | 14:11 |
| anteaya | not sure where to look next | 14:11 |
| *** bhuvan has quit IRC | 14:12 | |
| *** e0ne_ has quit IRC | 14:12 | |
| fungi | mordred: good morning... how broken are we on this fine monday? | 14:13 |
| anteaya | fairly | 14:13 |
| anteaya | start by looking at zuul I think | 14:14 |
| *** oubiwann_ has quit IRC | 14:14 | |
| anteaya | I see most patches in both check and gate with queued tests | 14:14 |
| anteaya | from gerrit has broken links to change ids: https://bugs.launchpad.net/openstack-ci/+bug/1278412 | 14:14 |
| uvirtbot | Launchpad bug 1278412 in openstack-ci "ChangeId linking broken in commit message" [Undecided,New] | 14:14 |
| anteaya | jobs have lost logs | 14:15 |
| anteaya | and some failures connecting to git.o.o: https://bugs.launchpad.net/openstack-ci/+bug/1277751 | 14:15 |
| uvirtbot | Launchpad bug 1277751 in openstack-ci "fatal: unable to connect to git.openstack.org" [Undecided,New] | 14:15 |
| anteaya | take your pick where you want to begin | 14:15 |
| anteaya | I checked cacti for static.o.o, we have about 2T left for writing logs | 14:16 |
| mordred | AH | 14:16 |
| anteaya | and something happened about 8 hours ago that included much IOwait time | 14:16 |
| mordred | I understand what "both check and gate with queued tests" means now - I was reading it wrong | 14:16 |
| *** AJaeger has joined #openstack-infra | 14:16 | |
| anteaya | how could I have been clearer the first time? | 14:17 |
| mordred | nope. you were fine. my brain processed it weirdly | 14:17 |
| anteaya | morning brain | 14:17 |
| mordred | the sentence seems quite obvious now that I grok | 14:17 |
| anteaya | k | 14:17 |
| mordred | I thought you were teling me that the problem was the changes were putting things into both check and gate simultaneously and it was happening for most of the changes in gerrit | 14:18 |
| mordred | which I did not see happening :) | 14:18 |
| anteaya | ah okay, yes, I can understand the non-grokness then if that was your takeaway from my posting | 14:18 |
| openstackgerrit | Ruslan Kamaldinov proposed a change to openstack-infra/storyboard: Extracted DB API layer https://review.openstack.org/72356 | 14:18 |
| mordred | 71393 is running | 14:19 |
| anteaya | also chmouel was mentioning something about jenkins triggering check jobs on comments | 14:19 |
| anteaya | chmouel: did you have a url or a bug for that? | 14:19 |
| fungi | DeeJay1: we've been wrestling with some recent problems in the bug update hook. i'll see if i can spot what happened with that one | 14:19 |
| mordred | or, is running things for three things | 14:19 |
| anteaya | some tests are running on patches, yes | 14:19 |
| chmouel | anteaya: this is what SergeyLukjanov and mordred were talkking about the re-check-stale-patches code | 14:20 |
| *** e0ne_ has joined #openstack-infra | 14:20 | |
| anteaya | but I am seeing much queueness | 14:20 |
| mordred | we seem to be awful low on nodes | 14:20 |
| anteaya | did a host fall over? | 14:20 |
| anteaya | or a zone? | 14:20 |
| mordred | ssh-ing to nodepool.o.o is currently taking a non-trivial amount of time | 14:20 |
| DeeJay1 | fungi: AFAIK this was my issue, as the bug is in another project in launchpad than openstack-ci | 14:21 |
| *** gokrokve has joined #openstack-infra | 14:21 | |
| mordred | fungi: it seems we may be having some sort of fail with nodepool | 14:21 |
| mordred | System load: 3.72 | 14:21 |
| mordred | Swap usage: 85% | 14:22 |
| mordred | fwiw | 14:22 |
| *** oanufriev has quit IRC | 14:22 | |
| mordred | puppet has been running since the 6th | 14:22 |
| mordred | stopping | 14:22 |
| *** oanufriev has joined #openstack-infra | 14:22 | |
| openstackgerrit | Ruslan Kamaldinov proposed a change to openstack-infra/storyboard: Refactor REST API to work on top of DB API https://review.openstack.org/72357 | 14:23 |
| *** e0ne has quit IRC | 14:23 | |
| fungi | DeeJay1: oh, i see SergeyLukjanov already answered your question | 14:24 |
| mordred | fungi: nodepool image-list was very slow in returning - doesn't it just talk to mysql/ | 14:24 |
| *** rossella_s has joined #openstack-infra | 14:25 | |
| *** gokrokve has quit IRC | 14:26 | |
| *** max_lobur_afk is now known as max_lobur | 14:27 | |
| fungi | 1277751 probably shouldn't be on top of the recheck list. i strongly suspect people are hitting bug 1270382 but the title of the other it too vague so they're selecting it because other people are selecting it (since it's remaining so visible on the rechecks list) | 14:27 |
| uvirtbot | Launchpad bug 1270382 in openstack-ci "Name resolution failure on gate job" [Medium,Incomplete] https://launchpad.net/bugs/1270382 | 14:27 |
| *** nicedice has joined #openstack-infra | 14:27 | |
| *** blitzrage is now known as leifmadsen | 14:28 | |
| *** smarcet has quit IRC | 14:29 | |
| *** pblaho has quit IRC | 14:31 | |
| fungi | retitled 1277751 so as to be less vague and marked it as a duplicate of 1266616 | 14:31 |
| *** smarcet has joined #openstack-infra | 14:34 | |
| fungi | man it's taking me a while to get through all the scrollback. problems with zuul? | 14:34 |
| mordred | fungi: looks like nodepool | 14:34 |
| fungi | looking okay to me at the moment, but i've only just glanced at it | 14:34 |
| *** pblaho has joined #openstack-infra | 14:35 | |
| fungi | anteaya: the iowait on static.o.o is the weekly compression job which looks through the filesystem for uncompressed job logs and fixes those | 14:35 |
| *** sirushti has left #openstack-infra | 14:35 | |
| anteaya | ah thank you | 14:36 |
| *** boris-42_ has quit IRC | 14:36 | |
| openstackgerrit | Philippe Godin proposed a change to openstack-infra/jenkins-job-builder: Add best and worst status details for conditional-step plugin. https://review.openstack.org/69608 | 14:37 |
| *** ewindisch has joined #openstack-infra | 14:37 | |
| *** prad has joined #openstack-infra | 14:38 | |
| fungi | okay, caught up. i assume from the lack of mention in scrollback that chmouel and others completely forgot what we discussed last week about zuul rerunning check jobs for any change which receives activity (comments, et cetera) if it hasn't seen a valid check result in the past 48 hours | 14:38 |
| *** chris_johnson has joined #openstack-infra | 14:39 | |
| fungi | so that core reviewers don't go blindly approving changes into the gate pipeline without fresh check results indicating it has some chance of success | 14:39 |
| chmouel | fungi: yep missed that, probably an email on the list may be useful ;) | 14:39 |
| anteaya | I forgot or never knew that discussion took place | 14:39 |
| fungi | chmouel: i could have sworn you asked about it in here last week | 14:39 |
| fungi | anteaya: you asked about it too | 14:39 |
| anteaya | I believe you | 14:40 |
| chmouel | fungi: ah wasn't me :) or maybe there is an another chmouel which i highly doubt :) | 14:40 |
| fungi | anyway, i'll take a poke at nodepool | 14:40 |
| *** mriedem has joined #openstack-infra | 14:43 | |
| fungi | chmouel: ahh, yep. i don't see in my channel log where you were asking about it. i was thinking of jd__ (he did notice the new zuul recheck behavior last week) | 14:43 |
| anteaya | fungi: can you remember about when it merged? I just checked the log for config for that last 10 days and don't see a commit title that rings a bell for me | 14:44 |
| jd__ | yeah, the most troubling thing is the LOST jobs I see a lot today :) | 14:44 |
| *** pdmars has quit IRC | 14:45 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/devstack-gate: Add support for running tempest serially without tenant isolation https://review.openstack.org/72365 | 14:45 |
| *** burt1 has joined #openstack-infra | 14:45 | |
| fungi | anteaya: https://review.openstack.org/69508 | 14:47 |
| *** malini_afk is now known as malini | 14:49 | |
| fungi | mordred: did you say you were rebooting the nodepool server? when i try to log into it i get "The system is going down for reboot NOW! Control-Alt-Delete pressed. Connection to nodepool closed by remote host." | 14:49 |
| fungi | looks like it just came back up (1 minute uptime) | 14:49 |
| fungi | i'll get to work cleaning up the mess | 14:49 |
| *** matsuhashi has joined #openstack-infra | 14:49 | |
| *** shortstop has joined #openstack-infra | 14:49 | |
| anteaya | fungi: thanks, okay, that title did strike me but yes I did pass over it | 14:50 |
| mordred | fungi: woah. I did NOT do that | 14:50 |
| mordred | but fascinating | 14:50 |
| fungi | mordred: okay, then rackspace probably left us a ticket saying "we noticed your vm was very busy, so we figured it was hung and rebooted it for you. kthx!" | 14:51 |
| mordred | sigh | 14:51 |
| mordred | it would be great if we could positively mark things as "we expect this to run hot" | 14:51 |
| *** sarob has joined #openstack-infra | 14:51 | |
| fungi | mordred: now you know that actually using the resources you're allotted is not at all conducive to a heavy oversubscription model ;) | 14:52 |
| chmouel | there is a "support instruction" text box for the RAX support staff that they should check if this get added before rebooting | 14:52 |
| jaypipes | hi all, I see some Jenkins jobs reported by Zuul as LOST... just want to check to see if this is a known thing and if I should just hold off on running a recheck. | 14:52 |
| fungi | "This message is to inform you that your Cloud Server 'nodepool.openstack.org' was found to be swapping as of 8:45 CST Today. The extent of disk IO usage had begun to have a negative impact on other customers on your host machine. As such, we had to reboot your cloud server machine to improve overall performance." | 14:53 |
| anteaya | jaypipes: hi, yes, please hold off | 14:53 |
| jaypipes | anteaya: no problemo. that's why I checked here :) thx! | 14:53 |
| mtreinish | fungi: heh nice | 14:53 |
| anteaya | fungi: would that be due to the file compression job? | 14:53 |
| fungi | anteaya: not at all. nodepool != static | 14:53 |
| anteaya | k | 14:53 |
| fungi | those are two completely separate servers | 14:53 |
| anteaya | hmmmm | 14:54 |
| anteaya | can we swap nodepool hosts? | 14:54 |
| anteaya | after the fire fighting | 14:55 |
| fungi | anteaya: we could take an outage to resize the nodepool server, but i really think we're looking at a memory leak judging from the overall ramp of the memory and swap graphs | 14:55 |
| anteaya | dang | 14:55 |
| *** sarob has quit IRC | 14:56 | |
| anteaya | some newly upgraded tool perhaps | 14:56 |
| *** oubiwan__ has quit IRC | 14:56 | |
| fungi | so i have doubts that a bigger server would do anything other than allow us to go longer between restarts | 14:56 |
| mrmartin | re | 14:56 |
| anteaya | k | 14:56 |
| fungi | anteaya: i don't know that it's anything new. looking at the graph it looks like it ramps pretty continually and then drops sharply (presumably corresponding to restarts of the server) | 14:57 |
| mrmartin | fungi: I want to take-over a patch that was sent to review, but was not accepted. Do you know a guide that describe the steps? | 14:57 |
| fungi | er, to restarts of the service | 14:57 |
| anteaya | fungi: hmmm | 14:57 |
| fungi | mrmartin: not accepted/take over how? | 14:57 |
| fungi | mrmartin: is it just abandoned? | 14:57 |
| mrmartin | fungi: this one: https://review.openstack.org/#/c/69636/4 | 14:58 |
| fungi | (as in the gerrit status for the cahnge says "abandoned"?) | 14:58 |
| mrmartin | it was posted by smarcet, but I want to continue his work | 14:58 |
| mrmartin | without loosing the previous comments. | 14:58 |
| fungi | mrmartin: oh, by not accepted you just mean it hasn't been approved yet | 14:58 |
| fungi | mrmartin: git review -d 69636 | 14:59 |
| mrmartin | yep, not approved is the exact phrase :) | 14:59 |
| anteaya | mrmartin: you can just download the latest patcheset and edit it, submitting a new patchset | 14:59 |
| anteaya | using the command fungi gave you | 14:59 |
| fungi | mrmartin: that will download the latest patchset for change 69636 and create a local topic branch from it | 14:59 |
| *** yamahata has quit IRC | 14:59 | |
| mrmartin | oh great, thank you. | 14:59 |
| anteaya | if you want patchset 4 do, git review -d 69636,4 | 14:59 |
| fungi | mrmartin: the change-id header in the commit message (along with everything else) will be preserved, so when you fir commit -a --amend after making changes and then git review it will continue with subsequent patchsets on the same change | 15:00 |
| mrmartin | cool. | 15:00 |
| *** zhiyan has joined #openstack-infra | 15:00 | |
| *** zehicle_at_dell has quit IRC | 15:00 | |
| fungi | er, s/fir/git/ (fingers were in the wrong place) | 15:00 |
| *** banix has joined #openstack-infra | 15:00 | |
| *** saju_m has joined #openstack-infra | 15:02 | |
| ArxCruz | fungi: anteaya hey, do you guys have an sample about how to inherit from openstack_project recipes in puppet? I'm trying to create the recipes for my CI inheriting from openstack_projects:: but mostly classes has parameters, and these parameters doesn't work well with heritage | 15:02 |
| ArxCruz | I try to use the include, but inside the classes also have other classes that I can't change the parameters | 15:02 |
| *** bhuvan has joined #openstack-infra | 15:02 | |
| hughsaunders | re nodepool instance: performance instances don't have swap by default, so you could move to a performance instance and not get rebooted for "excessive swapping" | 15:02 |
| fungi | ArxCruz: we try to extract anything general people might want to reuse into separate modules outside of the openstack_project module | 15:02 |
| *** yamahata has joined #openstack-infra | 15:03 | |
| fungi | hughsaunders: we set up swap partitions on performance nodes. running the linus kernel with no swap device or file at all is a very bad idea | 15:03 |
| fungi | s/linus/linux/ | 15:03 |
| *** damnsmith is now known as dansmith | 15:03 | |
| *** dkliban has joined #openstack-infra | 15:03 | |
| *** saju_m has quit IRC | 15:03 | |
| *** saju_m has joined #openstack-infra | 15:04 | |
| ArxCruz | fungi: so, I should have my own arx_openstack_project modules inherits from ::service_name ? | 15:05 |
| ArxCruz | that's the way it is right now :) okay ,thanks, just wondering if inheriting was a better idea :) | 15:05 |
| *** CaptTofu has joined #openstack-infra | 15:05 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Add non-isolated serial periodic tempest job https://review.openstack.org/72385 | 15:06 |
| *** pdmars has joined #openstack-infra | 15:06 | |
| fungi | ArxCruz: well, my point was that the openstack_project module is supposed to be an example of how to make use of our other modules in a deployment. that said, there are probably still lots of reusable classes in there which could stand to get mostly extracted out to separate top-level modules | 15:06 |
| ArxCruz | I see | 15:07 |
| *** lcostantino has joined #openstack-infra | 15:07 | |
| *** max_lobur is now known as max_lobur_afk | 15:07 | |
| *** markwash has joined #openstack-infra | 15:07 | |
| fungi | ArxCruz: so i don't really have an example of inheriting bits and pieces from openstack_project::.* | 15:08 |
| mordred | yes. we're working on that at hp too | 15:08 |
| mordred | I wouldn't inherit - I'd copy and edit where needed, tbh | 15:08 |
| mordred | it's a bit of a mess | 15:08 |
| ArxCruz | mordred: that's what I was trying to avoid | 15:08 |
| ArxCruz | inheriting would be more easy to maintain | 15:08 |
| mordred | AaronGr_Zzz and I are working throuh getting more of opensatck_project re-usable | 15:08 |
| ArxCruz | but puppet itself have some limitations about inherit | 15:08 |
| ArxCruz | mordred: count with me, do you have a action plan ? | 15:09 |
| fungi | yes, best effort would be to help untangling the interdependencies between openstack_project classes so that it's more modular | 15:09 |
| mordred | ArxCruz: AaronGr_Zzz does | 15:09 |
| fungi | ultimately openstack_project should be a very, very light shim layer to provide us with specific behaviors nobody else should really want | 15:10 |
| *** rcleere has quit IRC | 15:10 | |
| *** CaptTofu has quit IRC | 15:10 | |
| mordred | yup | 15:10 |
| cody-somerville | +1 | 15:11 |
| cody-somerville | Maybe we should start using juju charms? ;) | 15:11 |
| mordred | it also turns out there is more in openstack_project that people want to reuse than I thinkw e originally thought | 15:11 |
| * mordred punches cody-somerville in the face | 15:11 | |
| mordred | no | 15:11 |
| ArxCruz | mordred: indeed, our CI is all based on openstack_project puppet recipes, with some modifications | 15:12 |
| * cody-somerville runs to HR with a bloody nose and is told by HR he deserved it. :P | 15:12 | |
| *** lcostantino has quit IRC | 15:12 | |
| ArxCruz | but as you said, it's a mess maintain that | 15:12 |
| *** sdague has quit IRC | 15:12 | |
| mordred | well, some of it only comes up when we try to re-consume | 15:13 |
| mordred | for instance - the devstack gate prep scripts in nodepool all assume that you want to clone from review.openstack.org | 15:13 |
| *** lcostantino has joined #openstack-infra | 15:13 | |
| mordred | which is fine -they were written for openstack | 15:13 |
| ArxCruz | mordred: for this case, what I'm doing right now is overwriting the gate yaml file | 15:14 |
| mordred | but trying to track them is hard, because you have to replace that string in a bunch of places | 15:14 |
| mordred | yup | 15:14 |
| ArxCruz | so, I need to get all the files from openstack, run jjb script | 15:14 |
| *** bhuvan has quit IRC | 15:14 | |
| ArxCruz | then I copy my own yaml file, and reran the jjb | 15:14 |
| fungi | deleting all the stale ready and building nodes from before the nodepool server got rebooted | 15:15 |
| dripton | https://review.openstack.org/#/c/65029/ (Nova) seems to have been Approved and passed gate jobs last night, but is not in Git. Is this a gate problem or am I missing something? | 15:15 |
| *** dklyle is now known as david-lyle | 15:15 | |
| *** smarcet has quit IRC | 15:15 | |
| *** jgrimm has joined #openstack-infra | 15:16 | |
| fungi | #status the gate is experiencing delays due to nodepool resource issues (fix in progress, eta 16:00 utc) | 15:16 |
| fungi | oops | 15:16 |
| fungi | #status alert the gate is experiencing delays due to nodepool resource issues (fix in progress, eta 16:00 utc) | 15:16 |
| openstackstatus | NOTICE: the gate is experiencing delays due to nodepool resource issues (fix in progress, eta 16:00 utc) | 15:16 |
| *** ChanServ changes topic to "the gate is experiencing delays due to nodepool resource issues (fix in progress, eta 16:00 utc)" | 15:16 | |
| fungi | dripton: we're working on it | 15:16 |
| dripton | thanks fungi | 15:16 |
| dripton | I failed to read the status line | 15:17 |
| *** rwsu has joined #openstack-infra | 15:17 | |
| fungi | dripton: i only just set it | 15:17 |
| fungi | (you reminded me!) | 15:17 |
| dripton | aha, then I feel less bad. Thanks for the update. | 15:17 |
| fungi | np | 15:17 |
| *** max_lobur_afk is now known as max_lobur | 15:18 | |
| *** lcostantino has quit IRC | 15:19 | |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Implements: blueprint openid-oauth2-infra-implementation-puppet-script [smarcet] - #5478 - Clean up puppet Scripts https://review.openstack.org/69636 | 15:19 |
| *** mrodden has quit IRC | 15:20 | |
| *** AaronGr_Zzz is now known as AaronGr | 15:20 | |
| *** chandan_kumar has quit IRC | 15:20 | |
| ArxCruz | fungi: clarkb mordred I'm having problems to publish my logs outside my network, is it possible I start report success only for gerrit, with a message saying logs will be available in approximately 1 hour at http://my.outside.server ? | 15:21 |
| *** smarcet has joined #openstack-infra | 15:21 | |
| *** gokrokve has joined #openstack-infra | 15:21 | |
| ArxCruz | or I need to have the logs ready when I post the result ? | 15:21 |
| *** oubiwann_ has joined #openstack-infra | 15:22 | |
| krtaylor | ArxCruz, we could make it a bit faster than that, maybe 30 mins for starters | 15:23 |
| ArxCruz | if https://review.openstack.org/#/c/71189/2 and https://review.openstack.org/#/c/71189/ goes upstream... :) | 15:24 |
| *** gokrokve has quit IRC | 15:25 | |
| *** mrodden has joined #openstack-infra | 15:26 | |
| *** gokrokve has joined #openstack-infra | 15:28 | |
| *** sdague has joined #openstack-infra | 15:29 | |
| *** mriedem has quit IRC | 15:30 | |
| *** oubiwann_ has quit IRC | 15:31 | |
| fungi | so, nodepool's running and i'm clearing out the stale ready/building/delete nodes from earlier, but even though we're now well below quota it doesn't seem to be building new nodes | 15:31 |
| fungi | aha! "OperationalError: (OperationalError) (2006, 'MySQL server has gone away') 'UPDATE node SET external_id=%s WHERE node.id = %s' (4519309, 1332924L)" | 15:32 |
| fungi | weird... it's definitely running | 15:32 |
| *** markwash has quit IRC | 15:32 | |
| *** jnoller has joined #openstack-infra | 15:33 | |
| zhiyan | fungi: Hello, can you pls give a hand on https://review.openstack.org/#/c/58482/ ? it pending on merger a while, not sure why.. | 15:33 |
| fungi | zhiyan: see the channel topic | 15:33 |
| zhiyan | fungi: wo, got it. sorry | 15:33 |
| *** lcostantino has joined #openstack-infra | 15:34 | |
| fungi | zhiyan: though that doesn't look necessarily related. i think gerrit may have done what we occasionally see where it doesn't notice that the change is merged in the repository | 15:34 |
| fungi | zhiyan: check the tip of glance master... it's probably merged | 15:35 |
| zhiyan | fungi: seems it's better than i think...will pull | 15:35 |
| fungi | the mysql error in the nodepool log might be a red herring... it's from almost an hour ago (and nodepool hasn't written anything to its log since even though it's running) | 15:36 |
| *** bhuvan has joined #openstack-infra | 15:36 | |
| *** lttrl has quit IRC | 15:36 | |
| fungi | yeah, that's from the time of the reboot... i think it logged that because mysqld got stopped before nodepoold | 15:37 |
| *** matsuhashi has quit IRC | 15:37 | |
| *** nosnos has quit IRC | 15:37 | |
| *** yamahata has quit IRC | 15:37 | |
| *** nosnos has joined #openstack-infra | 15:37 | |
| *** dkranz has joined #openstack-infra | 15:37 | |
| *** matsuhashi has joined #openstack-infra | 15:37 | |
| *** saju_m has quit IRC | 15:37 | |
| *** lttrl has joined #openstack-infra | 15:38 | |
| fungi | restarted nodepoold and it's launching a buuuunch of nodes now | 15:38 |
| *** freyes has quit IRC | 15:39 | |
| *** oubiwann_ has joined #openstack-infra | 15:39 | |
| *** CaptTofu has joined #openstack-infra | 15:39 | |
| *** saju_m has joined #openstack-infra | 15:39 | |
| DeeJay1 | fungi: didn't reconnect to the DB after the restart? | 15:39 |
| *** boris-42_ has joined #openstack-infra | 15:40 | |
| fungi | DeeJay1: it's quite possible we lack proper lsb dependency ordering headers in its initscript or maybe it doesn't wait correctly for the db to be ready when starting. hard to say since it was causing it not to log anything | 15:40 |
| zhiyan | fungi: thanks. it's there, already been merged to trank. | 15:40 |
| zhiyan | *trunk | 15:40 |
| zhiyan | fungi: good luck. brb | 15:41 |
| fungi | DeeJay1: i should have attached an strace to its pid before i stopped and restarted it | 15:41 |
| *** dims has quit IRC | 15:41 | |
| *** zhiyan has left #openstack-infra | 15:41 | |
| DeeJay1 | fungi: oh well, next time ;) | 15:41 |
| *** rcleere has joined #openstack-infra | 15:41 | |
| *** pcrews has joined #openstack-infra | 15:42 | |
| *** markwash has joined #openstack-infra | 15:42 | |
| fungi | yep. my hindsight is always 20/20 | 15:43 |
| *** dims has joined #openstack-infra | 15:45 | |
| *** markwash has quit IRC | 15:46 | |
| *** afazekas has quit IRC | 15:47 | |
| DeeJay1 | fungi: that error is straight from libmysql so I think there's a 90% chance it was it ;) | 15:48 |
| fungi | DeeJay1: right, but that was from prior to shutdown as i surmised. roughly 3 minutes before last says the system booted up | 15:49 |
| *** wenlock has joined #openstack-infra | 15:49 | |
| *** bhuvan has quit IRC | 15:50 | |
| *** sarob has joined #openstack-infra | 15:52 | |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 15:53 | |
| *** dkliban has quit IRC | 15:55 | |
| openstackgerrit | Aaron Greengrass proposed a change to openstack-infra/config: Extend user creation with more granularity https://review.openstack.org/68776 | 15:57 |
| *** DinaBelova is now known as DinaBelova_ | 15:57 | |
| *** sarob has quit IRC | 15:57 | |
| *** AlexF has joined #openstack-infra | 15:59 | |
| *** pblaho has quit IRC | 16:00 | |
| *** chris_johnson is now known as wchrisj|away | 16:01 | |
| *** AlexF has quit IRC | 16:01 | |
| *** wchrisj|away is now known as chris_johnson | 16:02 | |
| *** w_ has joined #openstack-infra | 16:02 | |
| jeblair | fungi: good morning. anything i can help with? | 16:02 |
| *** atiwari has joined #openstack-infra | 16:02 | |
| fungi | jeblair: unravelling the latest nodepool mystery | 16:03 |
| *** olaph has quit IRC | 16:04 | |
| jeblair | i see it's been rebooted. that's nice. | 16:04 |
| fungi | it starts out that we have a fairly severe memory leak, to the point where we finally left it running so long without a nodepoold restart that this morning rackspace "rebooted" it for us. the swap activity was apparently impacting other customers on the same compute node | 16:04 |
| jeblair | (i find that practice annoying, btw) | 16:05 |
| *** zul has joined #openstack-infra | 16:05 | |
| *** mriedem has joined #openstack-infra | 16:06 | |
| fungi | it came up with, as best i can tell, nodepoold not finding the mysqld socket and waiting indefinitely. i'm still digging in the debug log to confirm | 16:06 |
| *** mlipchuk has quit IRC | 16:06 | |
| *** nosnos has quit IRC | 16:06 | |
| fungi | after i manually restarted nodepool and deleted a bunch of stale nodes from before the server reboot, it's been building and running a bunch of jobs | 16:06 |
| fungi | however, zuul had already marked many of them lost, even though jenkins seems to be running the jobs for those lost resutls now | 16:07 |
| fungi | which i think is why we don't yet see any movement in the zuul pipelines | 16:07 |
| *** matsuhashi has quit IRC | 16:07 | |
| *** AlexF has joined #openstack-infra | 16:08 | |
| *** dkliban has joined #openstack-infra | 16:08 | |
| fungi | oh, it's finally started getting some nodes to the oldest pending changes. good | 16:09 |
| jnoller | @fungi @jeblair We rebooted the server(s)? | 16:10 |
| fungi | jnoller: yes, ticket 140210-1822033-2 | 16:11 |
| *** mgagne has joined #openstack-infra | 16:11 | |
| jnoller | ... | 16:11 |
| jeblair | jnoller: yeah; we have a habit of not caring if our servers swap occasionally, and rax has a habit of caring enough about that to reboot them. | 16:11 |
| jeblair | jnoller: which is usually our cue to move to a larger server | 16:11 |
| jnoller | *sigh* | 16:11 |
| jnoller | Yeah, a bigger server helps | 16:12 |
| jnoller | obviously | 16:12 |
| fungi | jnoller: jeblair: happened back on feb 1st as well (ticket 140201-1811175-2) | 16:12 |
| jnoller | Did we notify you *before hand*? | 16:12 |
| fungi | part of the problem is, i think, that e-mail for the openstackci account goes to monty@inaugust.com | 16:12 |
| jnoller | westmaas ^^^ | 16:13 |
| jnoller | ok | 16:13 |
| fungi | is there somewhere better we should have ticket updates going? | 16:13 |
| jeblair | fungi: i could create an @openstack.org alias we could put all of infra-root on... | 16:14 |
| fungi | but yeah, those two tickets i mentioned are "hi there, we've opened this ticket to let you know we rebooted your server" so not really any advance notice no | 16:14 |
| fungi | "This server was experiencing out of memory errors and using the swap space partition for a significant period of time." | 16:15 |
| fungi | i'm not sure using swap is necessarily an "out of memory error" | 16:15 |
| westmaas | poor wording | 16:15 |
| westmaas | :/ | 16:15 |
| westmaas | this isn't a perf flavor though, right? | 16:16 |
| *** mrmartin has quit IRC | 16:16 | |
| westmaas | (catching up) | 16:16 |
| * fungi checks to see if nodepool.o.o was really logging logging an actual oom condition | 16:16 | |
| sc68cal | I'm seeing jenkins doing rechecks when someone posts a comment on a review that doesn't have "recheck" in it | 16:16 |
| fungi | westmaas: no, i think it's not a performance node. i believe we built it before you had those generally available | 16:16 |
| westmaas | fungi: makes sense | 16:16 |
| *** david_lyle_ has joined #openstack-infra | 16:16 | |
| sc68cal | https://review.openstack.org/#/c/56381/ | 16:17 |
| fungi | westmaas: yeah, seems to have been built in august | 16:17 |
| *** KurtMartin is now known as kmartin | 16:17 | |
| sc68cal | also this review at 8:36AM EST https://review.openstack.org/#/c/52983/ | 16:17 |
| fungi | sc68cal: that's because there was no recent check result on the change (within the last 48 hours) so it's being automatically rechecked to make sure it doesn't merge conflict ot have any new issues with the current branch tip code | 16:18 |
| *** reed has joined #openstack-infra | 16:18 | |
| sc68cal | oooh - that's nice, especially for alembic migrations | 16:18 |
| sc68cal | since those can get stale real quick | 16:18 |
| *** thedodd has joined #openstack-infra | 16:19 | |
| fungi | sc68cal: this prevents core reviewers from blindly approving stale changes without some recent assurance that it will have a chance of passing gating jobs | 16:19 |
| sc68cal | That's awesome! | 16:19 |
| fungi | sc68cal: new behavior as of 5 days ago. jeblair implemented it. i agree, very neat stuff | 16:19 |
| *** david-lyle has quit IRC | 16:20 | |
| *** sdake has quit IRC | 16:20 | |
| *** sdake has joined #openstack-infra | 16:21 | |
| *** sdake has quit IRC | 16:21 | |
| *** sdake has joined #openstack-infra | 16:21 | |
| fungi | #status ok jobs are running for changes again, but there's a bit of a backlog so it will still probably take a few hours for everything to catch up | 16:21 |
| openstackstatus | NOTICE: jobs are running for changes again, but there's a bit of a backlog so it will still probably take a few hours for everything to catch up | 16:21 |
| *** ChanServ changes topic to "Discussion of OpenStack Project Infrastructure | Docs http://ci.openstack.org/ | Bugs https://launchpad.net/openstack-ci | Code https://git.openstack.org/cgit/openstack-infra/" | 16:21 | |
| *** DeeJay1 has quit IRC | 16:21 | |
| *** mrda_away is now known as mrda | 16:22 | |
| *** virmitio has joined #openstack-infra | 16:22 | |
| *** david_lyle_ has quit IRC | 16:24 | |
| *** johnthetubaguy has joined #openstack-infra | 16:25 | |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Remove tox locale overrides https://review.openstack.org/72220 | 16:26 |
| *** dprince_ has joined #openstack-infra | 16:28 | |
| *** w_ is now known as olaph | 16:29 | |
| *** dprince has quit IRC | 16:29 | |
| *** beekneemech is now known as bnemec | 16:30 | |
| fungi | all active changes in the gate pipeline finally have nodes assigned for all their jobs, and zuul is working through the check backlog | 16:31 |
| *** reed has quit IRC | 16:31 | |
| *** reed_ has joined #openstack-infra | 16:31 | |
| *** CaptTofu has quit IRC | 16:35 | |
| *** reed_ has quit IRC | 16:36 | |
| *** bhuvan has joined #openstack-infra | 16:37 | |
| *** thuc has joined #openstack-infra | 16:38 | |
| *** thuc has quit IRC | 16:39 | |
| *** dizquierdo has quit IRC | 16:39 | |
| *** thuc has joined #openstack-infra | 16:39 | |
| *** oanufriev has quit IRC | 16:42 | |
| *** lcostantino has quit IRC | 16:43 | |
| *** markmcclain has joined #openstack-infra | 16:43 | |
| *** lcostantino has joined #openstack-infra | 16:44 | |
| *** markmcclain1 has joined #openstack-infra | 16:46 | |
| *** igor has quit IRC | 16:47 | |
| *** AlexF has quit IRC | 16:47 | |
| *** markmcclain has quit IRC | 16:47 | |
| *** igor has joined #openstack-infra | 16:47 | |
| *** lcostantino has quit IRC | 16:48 | |
| *** talluri has joined #openstack-infra | 16:48 | |
| *** lcostantino has joined #openstack-infra | 16:48 | |
| *** jooools has quit IRC | 16:50 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Remove tox locale overrides https://review.openstack.org/72221 | 16:50 |
| *** jooools has joined #openstack-infra | 16:50 | |
| *** igor has quit IRC | 16:52 | |
| *** gyee has joined #openstack-infra | 16:53 | |
| zaro | morning | 16:53 |
| zaro | fungi: any concerns with my last comment here: https://bugs.launchpad.net/openstack-ci/+bug/979227 | 16:54 |
| uvirtbot | Launchpad bug 979227 in openstack-ci "convert gerrit's mysql tables to utf8" [Medium,Triaged] | 16:54 |
| *** sarob has joined #openstack-infra | 16:55 | |
| *** lcostantino has quit IRC | 16:55 | |
| *** lcostantino has joined #openstack-infra | 16:56 | |
| lcostantino | Hi there, qq: i am getting -1 on some reviews because it has a depdency. Does gerrit take care of that and won't let jenkins attempt to merge that until the dependency lands? Or will try to do it and just fail. | 16:56 |
| *** dprince_ has quit IRC | 16:57 | |
| jeblair | lcostantino: it won't merge until the dependency merges. zuul won't even start running merge tests until they are both approved | 16:57 |
| *** gokrokve has quit IRC | 16:57 | |
| lcostantino | jeblair, thanks that i wanted to know. | 16:57 |
| *** smarcet has quit IRC | 16:58 | |
| *** saju_m has quit IRC | 16:59 | |
| *** sarob has quit IRC | 17:00 | |
| *** sarob has joined #openstack-infra | 17:00 | |
| zaro | jeblair: would like your feedback on bug 979227 | 17:01 |
| uvirtbot | Launchpad bug 979227 in openstack-ci "convert gerrit's mysql tables to utf8" [Medium,Triaged] https://launchpad.net/bugs/979227 | 17:01 |
| *** lcostantino has quit IRC | 17:02 | |
| *** jcoufal has quit IRC | 17:02 | |
| fungi | zaro: it may be a good question for #gerrit... i doubt gerrit relies on string collation order for anything of consequence, but it would be good to find out from them | 17:02 |
| *** cadenzajon has joined #openstack-infra | 17:02 | |
| *** gokrokve has joined #openstack-infra | 17:03 | |
| fungi | case (in)sensitivity collisions are a bigger concern for me | 17:03 |
| *** saju_m has joined #openstack-infra | 17:04 | |
| *** CaptTofu has joined #openstack-infra | 17:04 | |
| *** yamahata has joined #openstack-infra | 17:04 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Cancel obsolete builds on reconfiguration https://review.openstack.org/72042 | 17:04 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Remove double accounting of builds https://review.openstack.org/72043 | 17:04 |
| *** markwash has joined #openstack-infra | 17:05 | |
| fungi | jeblair: i assume 72042 solves what we saw with pending tripleo jobs after they were removed from the layout? | 17:05 |
| *** DinaBelova_ is now known as DinaBelova | 17:06 | |
| jeblair | fungi: yes, then 72043 solves it better | 17:06 |
| fungi | ahh | 17:06 |
| jeblair | well, more cleanup and simplification really | 17:06 |
| jeblair | zaro: looking | 17:07 |
| *** david-lyle has joined #openstack-infra | 17:08 | |
| jeblair | zaro: what kind of duplication do you see? | 17:08 |
| *** yamahata has quit IRC | 17:08 | |
| jeblair | zaro: (when going from cs to ci) | 17:08 |
| jeblair | zaro: (which tables/what kind of data in the rows) | 17:08 |
| openstackgerrit | Philippe Godin proposed a change to openstack-infra/jenkins-job-builder: Add best and worst status details for conditional-step plugin. https://review.openstack.org/69608 | 17:08 |
| zaro | fungi: good idea. i'll do that. | 17:09 |
| *** yamahata has joined #openstack-infra | 17:09 | |
| *** andreaf has joined #openstack-infra | 17:09 | |
| *** markmcclain1 has quit IRC | 17:09 | |
| *** markmc has quit IRC | 17:09 | |
| *** smarcet has joined #openstack-infra | 17:10 | |
| *** markmcclain has joined #openstack-infra | 17:10 | |
| *** e0ne_ has quit IRC | 17:10 | |
| *** dprince has joined #openstack-infra | 17:10 | |
| zaro | jeblair: there's quite a few of them, sorry i didn't record every dup error. but i do remember that there were duplicate entries for emails and commit messages. | 17:11 |
| jeblair | why would commit messages have a unique constraint? | 17:12 |
| zaro | sorry, i could be wrong. let me try to get a history and be more accurate. | 17:13 |
| *** rossella_s has quit IRC | 17:13 | |
| jeblair | emails, however, are case-sensitive | 17:13 |
| *** coolsvap has joined #openstack-infra | 17:14 | |
| fungi | at least the local part (the domain part is case insensitive, but the string overall needs case sensitivity because of the local part) | 17:14 |
| jeblair | yup | 17:14 |
| *** dangers_away is now known as dangers | 17:14 | |
| ArxCruz | AaronGr: around? | 17:15 |
| *** hashar has quit IRC | 17:15 | |
| AaronGr | yes, good morning | 17:15 |
| fungi | wow, i love receiving out-of-office autoreplies for code review changes i've submitted | 17:15 |
| fungi | makes my day | 17:16 |
| dhellmann | good morning/afternoon | 17:16 |
| *** reed has joined #openstack-infra | 17:16 | |
| dhellmann | is the process for automatically importing git repos still borked? | 17:17 |
| dhellmann | I'd like to see about moving ahead with oslo-cookiecutter if possible: https://review.openstack.org/#/c/70762/ | 17:17 |
| fungi | dhellmann: semi-borked. it works when triggered by hand, and i identified some of the ways in which it's breaking under puppet yesterday | 17:18 |
| *** gokrokve has quit IRC | 17:18 | |
| fungi | and anteaya has a patch up to get it to log its error messages from stdout/stderr when exec'd under puppet | 17:18 |
| jeblair | dhellmann: we'll run it by hand for openstack projects; i don't intend to let it impact the project | 17:19 |
| fungi | which i have tested and seems to at least work for manual puppet agent --test invocations | 17:19 |
| jeblair | dhellmann: (stackforge projects may be slower to be created though) | 17:19 |
| dhellmann | ok, thanks | 17:19 |
| zaro | jeblair, jeblair : so the other dups i saw were for userid and change messages | 17:19 |
| fungi | dhellmann: reviewing now | 17:19 |
| *** saju_m has quit IRC | 17:19 | |
| malini | dtroyer: ping | 17:19 |
| dhellmann | I don't have any stackforge projects to add for now, but that one is a semi-blocker for automating some of the graduation steps for oslo libs | 17:19 |
| dhellmann | I could just use the github repo, but you know | 17:20 |
| jeblair | dhellmann: please don't, just let us know if something is blocking (as you are doing) :) | 17:20 |
| jeblair | zaro: why would a change message have a unique constraint? | 17:20 |
| *** dkliban has quit IRC | 17:20 | |
| dhellmann | jeblair: yep, not blocked yet, but that's why I was asking :-) | 17:21 |
| jeblair | dhellmann: so i guess we'll just want to try to keep these in sync for the parts that are common? | 17:21 |
| dhellmann | jeblair: yeah, I don't know a good way to do that | 17:22 |
| dhellmann | the other repo is already somewhat out of date, but I don't remember the details about that | 17:22 |
| dhellmann | it seemed like I had to make a few changes to make it work | 17:22 |
| fungi | dhellmann: mordred: jeblair: can openstack-dev/cookiecutter have multiple templates maybe? | 17:22 |
| dhellmann | fungi: that would be ideal, but cookiecutter takes a git url as an arg | 17:22 |
| fungi | ahh, poop | 17:23 |
| dhellmann | it *can* take a directory, but I'm afraid it won't be used that way | 17:23 |
| dhellmann | because of the extra step of checking out the repo | 17:23 |
| fungi | right | 17:23 |
| dhellmann | I'm sure Audrey would be open to suggestions if we can come up with a way to say "this directory inside this git repo" | 17:23 |
| dhellmann | maybe we can make the root of the repo not valid, and put each template in a subdir, for example | 17:24 |
| dims | jeblair, could use your feedback on oslo.vmware (https://review.openstack.org/#/c/70761/) - there are a few reviews against oslo-incubator that are stuck and will need to be rebased to oslo.vmware repo when we get it ready | 17:25 |
| *** whoops has joined #openstack-infra | 17:25 | |
| dtroyer | malini: yo | 17:26 |
| fungi | dhellmann: looking at your github repo, you'll probably want to also add something like https://review.openstack.org/72233 | 17:26 |
| malini | dtroyer: thanks for the review | 17:26 |
| *** jpich has quit IRC | 17:26 | |
| malini | do you want me to update https://review.openstack.org/#/c/69497/ with a new patch set or add a dependent patch ? | 17:27 |
| *** apevec has joined #openstack-infra | 17:27 | |
| jeblair | dims: lgtm | 17:27 |
| *** salv-orlando has quit IRC | 17:27 | |
| dhellmann | fungi: ok, I can do that now if you haven't already approved the import | 17:27 |
| jeblair | fungi, anteaya: how close are we to having m-p logging? | 17:27 |
| malini | dtroyer: not sure how this is usually done | 17:28 |
| anteaya | jeblair: well I think fungi said that logging works | 17:28 |
| dims | jeblair, thanks! | 17:28 |
| dtroyer | malini: You can actually just change https://review.openstack.org/#/c/72412/ to add the is_marconi_enabled() function, it need not be dependent as nothing changes until https://review.openstack.org/#/c/69497/ is merged. No need to make it dependent. | 17:28 |
| mordred | fungi: /me just reading - why do yuo want more than one template? | 17:28 |
| anteaya | with the patch I offered which he tried by hand | 17:28 |
| *** sarob has quit IRC | 17:28 | |
| anteaya | jeblair: seems the m-p script is hitting a github request limit, which is set at 5000/hour | 17:29 |
| jeblair | anteaya: awesome, thx | 17:29 |
| anteaya | np | 17:29 |
| *** yassine has quit IRC | 17:29 | |
| fungi | mordred: oslo projects are apparently special and need to differ from other cookiecutter projects, per https://review.openstack.org/70762 | 17:29 |
| dhellmann | fungi: pushed | 17:29 |
| mordred | ah. well, yeah - I think we need different repos | 17:29 |
| *** sandywalsh has joined #openstack-infra | 17:30 | |
| malini | dtroyer: thanks! Do you expect 69497 to be merged soon? We are trying to get through the marconi graduation requirements & would like to know sooner if we need something else | 17:30 |
| *** salv-orlando has joined #openstack-infra | 17:30 | |
| *** lcostantino has joined #openstack-infra | 17:30 | |
| fungi | jeblair: so https://review.openstack.org/72158 logged in the puppet output when applied from a dev environment and run under puppet agent --test manually yesterday. not sure whether it will also log to syslog with other agent messages when run under puppet agent daemon | 17:31 |
| jeblair | fungi: oh i see, neat. | 17:31 |
| fungi | jeblair: but the easiest way to tell is to reenable the exec for it in puppet and let it run, i suppose | 17:31 |
| dtroyer | malini: I want it to be…I'll prod chmouel and sdague to look at it again... | 17:32 |
| malini | thanks dtroyer!! | 17:32 |
| jeblair | fungi, anteaya: it depends on a WIP patch | 17:32 |
| fungi | jeblair: also, i'm quite positive that 10 minutes is no longer nearly enough for a full manage-projects pass | 17:32 |
| fungi | jeblair: the wip patch is the one to turn the exec back on | 17:32 |
| anteaya | jeblair: fungi suggested I leave the reversion as WIP until further notice | 17:32 |
| *** saju_m has joined #openstack-infra | 17:32 | |
| jeblair | fungi: it looks like the timeout is 900 secs | 17:32 |
| jeblair | fungi: is that enough | 17:32 |
| anteaya | I can un-WIP if you wish | 17:32 |
| jeblair | anteaya: where did you see that we are hitting a rate limit? | 17:33 |
| fungi | jeblair: ahh, yes 900 will just make it (we were hovering around 700-800 when i was testing) | 17:33 |
| anteaya | fungi saw it | 17:33 |
| fungi | jeblair: i was getting it in the logs with a link to github's faq about authenticated queries being limited to 5000 per hour | 17:33 |
| fungi | when rerunning manage-projects multiple times within the span of an hour | 17:34 |
| anteaya | fungi: I had thought you had put that in a bug report comment, had you not? | 17:34 |
| jeblair | that's a lot of queries | 17:34 |
| fungi | anteaya: i mentioned it in my last review comment on 72158 | 17:34 |
| zaro | jeblair: i don't know the answer to your question, but can take a look at the tables to see if i can find the unique contraint | 17:35 |
| anteaya | there it is | 17:35 |
| fungi | but the github rate limit was running it 4 or 5 times in the span of an hour | 17:35 |
| jeblair | fungi: ok, so not an immediate problem | 17:35 |
| *** elasticio__ has joined #openstack-infra | 17:36 | |
| fungi | so i'm guessing we're running somehwere in excess of 1000 github queries for a full run | 17:36 |
| *** elasticio__ is now known as Guest44514 | 17:36 | |
| fungi | jeblair: no, not an immediate concern, but certainly something we need to keep in mind as we continue to grow our project base. we're within sight of the 15-minute timeout and the 5000 github request limits | 17:36 |
| *** lcostantino has quit IRC | 17:37 | |
| fungi | but yeah, next step i think is to merge 72158 and the revert in 72178 and then see if it logs when it breaks | 17:37 |
| *** fbo is now known as fbo_away | 17:37 | |
| *** lcostantino has joined #openstack-infra | 17:38 | |
| fungi | but wanted to discuss it before recommending to anteaya to take that revert out of wip | 17:38 |
| jeblair | fungi, anteaya: both lgtm | 17:38 |
| anteaya | so take revert out of wip? | 17:38 |
| fungi | anteaya: already did | 17:38 |
| fungi | and approved | 17:39 |
| fungi | let's see what transpires | 17:39 |
| anteaya | okay thanks | 17:39 |
| anteaya | let's | 17:39 |
| anteaya | I have to have something to eat and then a walk | 17:39 |
| anteaya | back in a bit | 17:39 |
| *** AlexF has joined #openstack-infra | 17:40 | |
| *** mkoderer has quit IRC | 17:41 | |
| *** johnthetubaguy has quit IRC | 17:41 | |
| *** markmcclain has quit IRC | 17:42 | |
| *** andre__ has quit IRC | 17:42 | |
| *** mkoderer has joined #openstack-infra | 17:43 | |
| *** dkliban has joined #openstack-infra | 17:44 | |
| *** moted has joined #openstack-infra | 17:45 | |
| apevec | is launchpad having issues or is just me? | 17:47 |
| apevec | even openid login via launchpad hangs | 17:47 |
| jeblair | fungi, clarkb, mordred: http://lists.openstack.org/pipermail/openstack-dev/2014-February/026778.html | 17:48 |
| openstackgerrit | A change was merged to openstack-dev/hacking: Remove tox locale overrides https://review.openstack.org/72206 | 17:50 |
| openstackgerrit | A change was merged to openstack-dev/hacking: Updated from global requirements https://review.openstack.org/71412 | 17:50 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Publish draft docs on draft-docs https://review.openstack.org/70594 | 17:51 |
| fungi | jeblair: thanks! | 17:52 |
| openstackgerrit | A change was merged to openstack-dev/hacking: Clarify H403 message https://review.openstack.org/70453 | 17:52 |
| mordred | jeblair: looking | 17:54 |
| *** smurugesan has joined #openstack-infra | 17:55 | |
| clarkb | morning | 17:55 |
| mordred | jeblair: +2 (I'm away from ability to respond to that via email, but I am strongly +2) | 17:55 |
| jeblair | mordred: what happened to your irc-to-email gateway? | 17:56 |
| clarkb | jeblair: did you catch the logstash crm114 problem in scroll back? I am about to catch up on my sb too | 17:56 |
| jeblair | clarkb: no | 17:56 |
| mordred | jeblair: god I need one | 17:57 |
| clarkb | jeblair: I submitted a bug which I will dig up, but tl;dr is that logstash claims the select timeout is being hit without data to read so we don't index the remaining data for that file | 17:57 |
| clarkb | jeblair: I did a quick debug of the problem, but didn't really sort anything out (probably need to run crm by hand against the files that timeout) | 17:57 |
| *** Steap_ has joined #openstack-infra | 17:57 | |
| clarkb | jeblair: https://bugs.launchpad.net/openstack-ci/+bug/1278210 | 17:58 |
| uvirtbot | Launchpad bug 1278210 in openstack-ci "CRM114 timeouts preventing logs from being indexed" [High,Triaged] | 17:58 |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "Stop running manage-projects" https://review.openstack.org/72178 | 17:58 |
| *** igor_ has joined #openstack-infra | 17:58 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add logging to manage-projects https://review.openstack.org/72158 | 17:58 |
| jeblair | clarkb: we don't _index_ or we don't add error_pr values? | 17:58 |
| *** harlowja_away is now known as harlowja | 17:58 | |
| clarkb | jeblair: we don't index because of where the exception is caght | 17:59 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/jeepyb: Allow welcome-message to configure logging https://review.openstack.org/72435 | 18:00 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Enable verbose output for welcome message https://review.openstack.org/72436 | 18:00 |
| jeblair | clarkb: whoops. yeah, we should catch that, log, and proceed | 18:00 |
| *** amcrn has joined #openstack-infra | 18:01 | |
| clarkb | jeblair: ok, I can work on that patch | 18:01 |
| *** derekh has quit IRC | 18:02 | |
| jeblair | (then secondarily, check into why it timed out) | 18:02 |
| *** igor_ has quit IRC | 18:02 | |
| *** markmcclain has joined #openstack-infra | 18:04 | |
| *** johnthetubaguy has joined #openstack-infra | 18:05 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Added base for functional tests https://review.openstack.org/69384 | 18:05 |
| jeblair | clarkb: with the active window, i wonder if we should consider switching gate back to high gearman precedence | 18:08 |
| clarkb | jeblair: oh, good point | 18:09 |
| clarkb | in theory that prevents unnecesary starvation | 18:09 |
| fungi | jeblair: clarkb: i think it's worth revisiting. we no longer consume enough nodes in a gate reset to starve the check pipeline | 18:09 |
| jeblair | clarkb, fungi: also, this is a bad test since we had the nodepool fubar, but i have a suspicion that monday mornings will have a high check load because of the freshness requirement. | 18:10 |
| fungi | likely so | 18:10 |
| *** e0ne has joined #openstack-infra | 18:11 | |
| *** saju_m has quit IRC | 18:12 | |
| clarkb | just like freshly squeezed OJ | 18:15 |
| *** jp_at_hp has quit IRC | 18:16 | |
| fungi | our check queue is high in vitamin c too | 18:16 |
| *** oanufriev has joined #openstack-infra | 18:20 | |
| *** e0ne_ has joined #openstack-infra | 18:21 | |
| *** e0ne has quit IRC | 18:21 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Handle log filter exceptions more gracefully. https://review.openstack.org/72444 | 18:21 |
| clarkb | jeblair: fungi ^ going to apply that on logstash-worker01 | 18:21 |
| clarkb | :( forgot to set related-bug in the commit message, will update | 18:22 |
| *** CaptTofu has quit IRC | 18:22 | |
| jeblair | clarkb: one worry: will that cause a 20 second timeout on every subsequent line? | 18:22 |
| jeblair | clarkb: oh heh | 18:23 |
| jeblair | clarkb: i see you handle it with new_filters | 18:23 |
| *** max_lobur is now known as max_lobur_afk | 18:23 | |
| clarkb | yeah :) | 18:23 |
| jeblair | lgtm | 18:23 |
| openstackgerrit | Bhuvan Arumugam proposed a change to openstack-infra/config: log analyzer for openstack IRC logs https://review.openstack.org/72445 | 18:23 |
| jeblair | ttx: are you ready for https://review.openstack.org/#/c/65103/ to merge? | 18:24 |
| *** gokrokve has joined #openstack-infra | 18:27 | |
| openstackgerrit | A change was merged to openstack-infra/config: Import oslo-cookiecutter repository https://review.openstack.org/70762 | 18:27 |
| *** apevec has quit IRC | 18:27 | |
| fungi | watching the puppet logs on review.o.o now as that ^ gets applied | 18:28 |
| jeblair | SergeyLukjanov: ping | 18:28 |
| *** saju_m has joined #openstack-infra | 18:29 | |
| *** e0ne_ has quit IRC | 18:31 | |
| *** e0ne has joined #openstack-infra | 18:31 | |
| *** alexpilotti has joined #openstack-infra | 18:31 | |
| *** CaptTofu has joined #openstack-infra | 18:34 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/config: Support filtering by review id(s) https://review.openstack.org/72446 | 18:34 |
| *** sarob has joined #openstack-infra | 18:34 | |
| *** rwsu has quit IRC | 18:36 | |
| *** freyes has joined #openstack-infra | 18:38 | |
| sdague | jeblair: question on the recheck bits, where does clean check for gate stand? | 18:39 |
| openstackgerrit | Bhuvan Arumugam proposed a change to openstack-infra/config: log analyzer for openstack IRC logs https://review.openstack.org/72445 | 18:39 |
| jeblair | sdague: just un-wip'd the change | 18:39 |
| sdague | cool | 18:39 |
| jeblair | sdague: https://review.openstack.org/#/c/68817/ | 18:40 |
| *** gokrokve has quit IRC | 18:40 | |
| *** Sukhdev has joined #openstack-infra | 18:43 | |
| sdague | jeblair: what does the middle comment-added block mean? | 18:43 |
| jeblair | fungi: what do you think of climate using "openstack-climate" as the rtfd project name? https://review.openstack.org/#/c/68062/2 that seems premature to me. | 18:43 |
| *** VijayT has joined #openstack-infra | 18:43 | |
| sdague | require approval: verified: 1 on approval: verified: 1 ? | 18:43 |
| fungi | jeblair: somewhat agree--i hadn't see that change yet | 18:44 |
| fungi | jeblair: if "climate" were a descriptive term, then i'd be more okay with it | 18:44 |
| *** Guest44514 has quit IRC | 18:45 | |
| fungi | jeblair: but this isn't "the climate you want when using openstack" because climate is itself already a fairly nondescriptive codename | 18:45 |
| jeblair | sdague: so that when jenkins votes +1 at the completion of a check run it will be enqueued into the gate. | 18:45 |
| *** zigo has quit IRC | 18:45 | |
| jeblair | sdague: (strictly speaking, any +1 vote (including from 3rd party ci) would trigger this too; we could probably narrow that down) | 18:47 |
| jeblair | sdague: in all cases, the normal requirements to be enqueued (can merge, has approved vote, has +2 code review and no -2 code review) still apply | 18:47 |
| sdague | jeblair: ok, I guess it's not obvious to me from the dsl | 18:47 |
| *** gokrokve has joined #openstack-infra | 18:49 | |
| *** zigo has joined #openstack-infra | 18:49 | |
| jeblair | sdague: "a comment-added event with an approval vote of verified=1 will trigger the pipeline iff there is also an approval of verified=1 from jenkins within the last 48h" | 18:50 |
| jeblair | s/also// | 18:50 |
| *** pafuent has joined #openstack-infra | 18:50 | |
| *** rwsu has joined #openstack-infra | 18:51 | |
| *** saju_m has quit IRC | 18:51 | |
| clarkb | jeblair: my logstash-worker change seems to work, I am going to self approve the change and get that running everywhere | 18:51 |
| jeblair | clarkb: cool. i did +2 it i think. :) | 18:52 |
| *** saju_m has joined #openstack-infra | 18:52 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Require a recent (48h) positive check result https://review.openstack.org/68817 | 18:52 |
| jeblair | sdague: there, that's more specific: "a comment-added event with an approval vote of verified=1 from jenkins will trigger the pipeline" | 18:53 |
| *** morganfainberg_Z is now known as morganfainberg | 18:53 | |
| *** burt1 has quit IRC | 18:53 | |
| sdague | ok, I get it now. | 18:55 |
| sdague | it still has to handle sufficiency separately | 18:55 |
| jeblair | yep | 18:55 |
| *** exz has quit IRC | 18:55 | |
| jeblair | that's magic that gets invoked if 'submit' is set on a pipeline | 18:55 |
| sdague | yep | 18:57 |
| *** saju_m has quit IRC | 18:58 | |
| *** talluri has quit IRC | 18:59 | |
| *** talluri has joined #openstack-infra | 18:59 | |
| *** johnthetubaguy has quit IRC | 19:00 | |
| clarkb | btw https://bitbucket.org/hpk42/tox/pull-request/85/fix-command-expansion-and-parsing/diff is a thing | 19:01 |
| clarkb | I figured hg out enough to get that far | 19:01 |
| *** sarob has quit IRC | 19:01 | |
| clarkb | dhellmann: mordred jeblair ^ | 19:01 |
| jeblair | clarkb: from what i hear from mordred, hopefully you won't have to update that. :) | 19:01 |
| *** markmcclain has quit IRC | 19:01 | |
| clarkb | jeblair: I had to update it once already, its basically the github PR model where you do a new commit and add it to the PR, the difference is that hg won't let you squash them best I can tell | 19:02 |
| *** markmcclain has joined #openstack-infra | 19:02 | |
| reed | jeblair, fungi: can you please give your opinion to https://review.openstack.org/#/c/69636/ ? | 19:02 |
| jeblair | oh, ew. | 19:02 |
| clarkb | and if that PR gets rejected on silly grounds I will probably be forking with that as the tip of master :) | 19:03 |
| jeblair | reed: yep, it's about 10 changes down my list; though one thing i can see before i've reviewed it is we need to point them at | 19:03 |
| jeblair | https://wiki.openstack.org/wiki/GitCommitMessages | 19:03 |
| *** talluri has quit IRC | 19:03 | |
| *** AlexF has quit IRC | 19:04 | |
| *** johnthetubaguy has joined #openstack-infra | 19:04 | |
| *** dcramer_ has joined #openstack-infra | 19:04 | |
| *** pafuent1 has joined #openstack-infra | 19:06 | |
| *** pafuent has quit IRC | 19:06 | |
| *** pafuent1 has quit IRC | 19:07 | |
| reed | jeblair, agreed, it's not a good commit message... keep in mind we need that patch when you review it, if you can, avoid -1/2ing just because of the commit message | 19:07 |
| *** lcostantino has quit IRC | 19:09 | |
| *** lcostantino has joined #openstack-infra | 19:10 | |
| jeblair | reed: i'm the last person to needlessly -1 something on style grounds, but that really does need a better commit message and i will probably -1 it for that. it's more than mere style, it's functionally deficient. but it's easy to fix so it shouldn't be a big deal. | 19:10 |
| fungi | does <VirtualHost *:443> work these days? | 19:11 |
| reed | jeblair, the big deal is that any extra step requires 2+ days to be evaluated and that is awful | 19:11 |
| jeblair | reed: they work every other day? | 19:12 |
| clarkb | fungi: yes I believe it does | 19:12 |
| fungi | awesome | 19:12 |
| reed | jeblair, marton took over the change, he's in EU and infra review queue is just long | 19:12 |
| fungi | i need to fix up my apache https vhost configs for that eventually | 19:13 |
| *** sarob has joined #openstack-infra | 19:13 | |
| jeblair | reed: yeah, i know it's long. that's why i'm reviewing changes now. or at least i was. but we're not going to just merge stuff that's broken and not maintainable. we do review for a reason. | 19:14 |
| lcostantino | another qq for this gerrit newbie. Is the normal behavior to get all dependencies review updated after pushing change? ( i cannot find anything explicit on the wiki ). For instance A->B->C , i pushed C and A/B cleared all the +1. Is that right? | 19:14 |
| reed | jeblair, one request to edit anything and the change goes back to the bottom of the queue | 19:14 |
| jeblair | reed: that's not how i run my queue. | 19:14 |
| reed | jeblair, is the commit message the only thing? | 19:14 |
| jeblair | reed: i don't know, i haven't reviewed it yet. it's still 10 changes away. | 19:15 |
| reed | ok | 19:15 |
| clarkb | lcostantino: short answer is it depends :) any time the git commit sha1 updates, the gerrit changes will get new patchsets. So if you had to rebase the stack A,B,C to resolve a merge conflict all will update | 19:15 |
| jeblair | russellb: but i will review the whole thing and leave comments on everything i see in order to minimize the number of round trips. | 19:15 |
| reed | jeblair, thanks, see what you can do | 19:15 |
| reed | i | 19:15 |
| jeblair | gah | 19:15 |
| jeblair | reed: ^ | 19:15 |
| clarkb | lcostantino: if there was no rebase and merge conflict then it shouldn't update (ensure you are using the latest release of git review to get that behavior, old releases always rebased) | 19:15 |
| openstackgerrit | A change was merged to openstack-infra/config: Handle log filter exceptions more gracefully. https://review.openstack.org/72444 | 19:15 |
| reed | jeblair, I'll ping marto, let him know that he may need to edit the commit and hope he can do it before going to bed | 19:16 |
| jeblair | reed: i don't like re-reviewing things any more than anyone likes revising them. | 19:16 |
| lcostantino | clarkb, thanks. This happened when i just created the new C review. So i think i am getting hit but the gerrit version then | 19:16 |
| lcostantino | clarkb, which version should be the right one? | 19:16 |
| clarkb | lcostantino: 1.23 is the latest | 19:16 |
| lcostantino | clarkb, thanks | 19:16 |
| fungi | grr... so openstack-dev/oslo-cookiecutter got created and its acl applied in gerrit, but http://git.openstack.org/cgit/openstack-dev/oslo-cookiecutter says "Repository seems to be empty" and the puppet agent log didn't mention anything between "(/Stage[main]/Openstack_project::Gerrit/Exec[manage_projects]) Triggered 'refresh' from 2 events" and "Finished catalog run in 684.92 secon | 19:17 |
| fungi | ds" | 19:17 |
| fungi | so either manage-projects didn't actually output an error, or the exec logoutput option in puppet only works when you puppet apply --test but doesn't get copied to the syslog stream | 19:18 |
| fungi | AaronGr: nibalizer: ^ do you know much about exec logoutput behavior? | 19:18 |
| *** johnthetubaguy has quit IRC | 19:19 | |
| fungi | if it's the latter, then i think manage-projects needs python logging configuration so it can log on its own | 19:19 |
| *** mburned has joined #openstack-infra | 19:19 | |
| * fungi gets to work cleaning up the oslo-cookiecutter git repo mess | 19:20 | |
| nibalizer | hrm, never used that parameter, gimmie a sec | 19:20 |
| nibalizer | fungi: where is this code? | 19:21 |
| fungi | nibalizer: thanks! http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/manifests/gerrit.pp#n386 | 19:22 |
| *** smarcet has quit IRC | 19:22 | |
| *** smarcet has joined #openstack-infra | 19:23 | |
| *** bhuvan has quit IRC | 19:23 | |
| fungi | though in retrospect, i may have been too hasty to suspect breakage from manage-projects itself... the git repositories on review and also on zuul look correct | 19:24 |
| fungi | maybe create-cgitrepos is failing on the git farm | 19:25 |
| nibalizer | from the puppet docs, its unclear if true means on_failure or if false means on_failure | 19:25 |
| *** melwitt has joined #openstack-infra | 19:25 | |
| nibalizer | but i imagine that true means log the output | 19:25 |
| jeblair | fungi: or is it just that replication happened out of sequence? | 19:25 |
| *** beagles is now known as beagles_brb | 19:26 | |
| nibalizer | ya, logoutput dumps the entire output of the exec | 19:27 |
| fungi | jeblair: seems possible. create-cgitrepos runs clean on all of the git farm. i manually did a gerrit replicate openstack-dev/oslo-cookiecutter and http://git.openstack.org/cgit/openstack-dev/oslo-cookiecutter is okay now | 19:28 |
| *** NikitaKonovalov_ is now known as NikitaKonovalov | 19:28 | |
| *** NikitaKonovalov is now known as NikitaKonovalov_ | 19:28 | |
| fungi | nibalizer: thanks for double-checking! i'll continue to keep an eye on it. maybe we want manage-projects to spit something on stdout rather than silence on success, to exercise that | 19:28 |
| fungi | dhellmann: oslo-cookiecutter is hopefully open for business | 19:29 |
| * fungi needs to step away for a bit and run a couple errands. back soon | 19:29 | |
| nibalizer | so are ya'll syslogging your puppet output then? | 19:29 |
| nibalizer | I generally reccomend against that for security | 19:30 |
| jeblair | nibalizer: can you elaborate? | 19:30 |
| *** thedodd has quit IRC | 19:30 | |
| nibalizer | often you have an exec somewhere, mysql module used to do this (not sure if it still does), that shells out to create the user/password | 19:31 |
| nibalizer | if that fails for whatever reason, the shell command with password expanded will be dumped to puppet output, then to syslog | 19:31 |
| nibalizer | now if you have encrypted logs and all that, no worries | 19:31 |
| jeblair | nibalizer: ah. well we would need to avoid things like that then because moreso than syslog, we actually want all the puppet logs available on the web. | 19:32 |
| nibalizer | yea, thats why i brought it up | 19:32 |
| nibalizer | its possible you dont have this problem | 19:32 |
| jeblair | nibalizer: they used to be until puppet-dashboard broke; there are efforts to replace that though. | 19:32 |
| jeblair | nibalizer: or it's possible we do and didn't know about it. :) | 19:32 |
| nibalizer | don't they go to clarkb's logstash? | 19:32 |
| *** ruhe_ has joined #openstack-infra | 19:32 | |
| jeblair | nibalizer: no, that's only test runs | 19:33 |
| nibalizer | ok | 19:33 |
| *** jamesmcarthur has joined #openstack-infra | 19:33 | |
| nibalizer | i think AaronGr is scheming to bring up puppetboard, which i totally +1 | 19:33 |
| jeblair | yes, Hunner was working on that as well... | 19:34 |
| jeblair | Hunner and AaronGr should probably connect on that. anteaya too. | 19:34 |
| *** chris_johnson is now known as wchrisj|away | 19:35 | |
| SergeyLukjanov | jeblair, pong, thank you :) | 19:36 |
| jeblair | SergeyLukjanov: no, thank you. :) | 19:36 |
| nibalizer | puppetboard can help you identify failing nodes and failing resources on those nodes, which is really all we need from a puppet dashborad | 19:36 |
| nibalizer | plus i have a half-dozen commits in it, so i guess im biased | 19:36 |
| jeblair | nibalizer: it can display logs too, right? | 19:37 |
| *** nicedice_ has joined #openstack-infra | 19:37 | |
| SergeyLukjanov | jeblair, AFAIK yes | 19:37 |
| nibalizer | i dont think it has a page with just the raw text of the puppet output | 19:37 |
| jeblair | nibalizer: basically i want AaronGr to be able to do whatever he wants to the system without an infra-root needing to go and look at logs | 19:38 |
| SergeyLukjanov | ruhe, ping, I remember that you was talking about puppetboard | 19:38 |
| jeblair | nibalizer: oh, er, how do you use it to diagnose failures then? | 19:38 |
| *** markmcclain has quit IRC | 19:38 | |
| nibalizer | puppetboard will get him most of the way there | 19:38 |
| openstackgerrit | Matt Ray proposed a change to openstack-infra/config: Create new Chef cookbook-openstack-integration-test for Tempest support. https://review.openstack.org/68791 | 19:38 |
| *** saju_m has joined #openstack-infra | 19:38 | |
| ruhe_ | SergeyLukjanov: i've deployed puppetboard manually, and i haven't seen any logs on the dashboard | 19:39 |
| nibalizer | well because you see the resource thats failing and the text of its failure, you have all the same information, just tabled and clickable | 19:39 |
| *** mrmartin has joined #openstack-infra | 19:39 | |
| jeblair | nibalizer: ok, good, the text of the failure; that should be sufficient. | 19:39 |
| ruhe_ | SergeyLukjanov: but it can tell you that some deployment went wrong. that's what deployer/admin actually needs | 19:40 |
| *** pafuent has joined #openstack-infra | 19:40 | |
| *** nicedice has quit IRC | 19:40 | |
| SergeyLukjanov | ruhe_, could it share info about props, env, etc/ | 19:41 |
| nibalizer | also if you all want to review https://review.openstack.org/#/c/71739/ that would be awesome | 19:41 |
| *** wchrisj|away is now known as chris_johnson | 19:42 | |
| nibalizer | and here is the basics of the plan AaronGr cooked up to improve the openstack puppet | 19:42 |
| ruhe_ | SergeyLukjanov: probably yes, because it has a query interface for puppetdb. unfortunatelly VM with puppetboard went down and i can't check it now | 19:42 |
| nibalizer | https://etherpad.openstack.org/p/FhzEfZrc49 | 19:42 |
| pafuent | Hi. Can someone take a look at this review https://review.openstack.org/#/c/71614/? | 19:42 |
| clarkb | logstash workers 01-06 are happy now. logstash itself is about half an hour behind. Going to spin up workers on 07 and 08 to help pick up the backlog | 19:43 |
| openstackgerrit | Valeriy Ponomaryov proposed a change to openstack-infra/config: Upgrade manila's devstack-job to tempest-job https://review.openstack.org/66846 | 19:45 |
| pafuent | https://review.openstack.org/#/c/71614/, will enable gate jobs for climate project | 19:45 |
| *** thuc has quit IRC | 19:46 | |
| jeblair | SergeyLukjanov: i had a question about https://review.openstack.org/#/c/66521/ | 19:46 |
| *** thuc has joined #openstack-infra | 19:46 | |
| jeblair | SergeyLukjanov: i'm pretty sure i've asked this before, but i forgot; sorry... | 19:46 |
| jeblair | SergeyLukjanov: why do you need to run that as a 3rd-party test instead of running it in openstack-infra? | 19:47 |
| *** DinaBelova is now known as DinaBelova_ | 19:47 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add bugdaystats to openstack-infra https://review.openstack.org/69489 | 19:48 |
| SergeyLukjanov | jeblair, tl;dr - we need nested virt and up to 7 2vcpu 2GB RAM vms for running one of 5 scenarious | 19:48 |
| *** zigo has quit IRC | 19:48 | |
| SergeyLukjanov | jeblair, I've checked hp and rackspace vms that are used for gating, Hadoop could not start on nested vms on both of them | 19:49 |
| *** thuc has quit IRC | 19:50 | |
| jeblair | SergeyLukjanov: ok. if we can run a job on multiple vms in the future, we could probably do this upstream then, right? | 19:50 |
| jeblair | (that's something I'd like to look into after we stop using jenkins) | 19:51 |
| *** smarcet has quit IRC | 19:52 | |
| *** exz has joined #openstack-infra | 19:52 | |
| SergeyLukjanov | jeblair, if such vms will support nested virt acceleration | 19:52 |
| SergeyLukjanov | jeblair, the another approach is to use TripleO | 19:52 |
| *** johnthetubaguy has joined #openstack-infra | 19:53 | |
| *** smarcet has joined #openstack-infra | 19:53 | |
| jeblair | SergeyLukjanov: oh, you don't need nested virt just because you need lots of vms, you need it because you need to deploy openstack and then within openstack delpoy hadoop vms; so the devstack-deployed version needs to be something can can nest a vm | 19:54 |
| jeblair | SergeyLukjanov: in that case, yes, i agree tripleo might be the best bet in the future | 19:54 |
| *** DinaBelova_ is now known as DinaBelova | 19:54 | |
| SergeyLukjanov | jeblair, yeah | 19:54 |
| jeblair | SergeyLukjanov: okay cool. thanks, and sorry i forgot. :( | 19:55 |
| SergeyLukjanov | jeblair, in fact currently we're running all of it on one 24 CPU, 128 GB RAM server with devstack :) | 19:55 |
| SergeyLukjanov | jeblair, np, I need to recheck and document all issues some day ;) | 19:55 |
| *** markmcclain has joined #openstack-infra | 19:58 | |
| *** hashar has joined #openstack-infra | 19:58 | |
| *** saju_m has quit IRC | 20:01 | |
| openstackgerrit | Matt Ray proposed a change to openstack-infra/config: Create new Chef cookbook-openstack-integration-test for Tempest support. https://review.openstack.org/68791 | 20:01 |
| *** openstack has joined #openstack-infra | 21:33 | |
| jeblair | oh look it's retrying | 21:33 |
| jeblair | openstack: welcome back | 21:33 |
| fungi | good to know it re-resolves names | 21:33 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Add non-isolated serial periodic tempest job https://review.openstack.org/72385 | 21:34 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/devstack-gate: Add cliff, pycadf, stevefore & taskflow https://review.openstack.org/72487 | 21:35 |
| *** mattray has joined #openstack-infra | 21:36 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Set periodic tempest jobs to run with master https://review.openstack.org/71940 | 21:37 |
| openstackgerrit | A change was merged to openstack/requirements: Adds python-seamicroclient dependency for Ironic https://review.openstack.org/70751 | 21:38 |
| anteaya | fungi: I am drawing together my thoughts on what I need to mock a run of manage-projects: https://etherpad.openstack.org/p/mock-manage-projects | 21:41 |
| anteaya | do drop by if you feel so inclined | 21:42 |
| *** amcrn_ has joined #openstack-infra | 21:44 | |
| *** amcrn has quit IRC | 21:45 | |
| *** gokrokve has quit IRC | 21:46 | |
| *** gokrokve has joined #openstack-infra | 21:46 | |
| anteaya | fungi: thanks | 21:46 |
| anteaya | do I need the vm to be a minumum size? | 21:47 |
| anteaya | or will a small or medium flavour suffice | 21:47 |
| *** amcrn_ is now known as amcrn | 21:48 | |
| fungi | anteaya: a 2gb vm will almost certainly work for basic gerrit/jeepyb testing | 21:48 |
| *** eharney has quit IRC | 21:49 | |
| anteaya | great thank you | 21:49 |
| fungi | 1gb or smaller would probably even be fine (i ran it at home for a while in a vm with 256mb ram and 512mb swap and a very slow nfs root filesystem with reasonable performance) | 21:49 |
| *** chris_johnson is now known as wchrisj|away | 21:50 | |
| openstackgerrit | A change was merged to openstack-infra/config: Make files in reviewday docroot accessible https://review.openstack.org/69734 | 21:50 |
| fungi | presumably the goal is to profile the various github api calls made when running manage-projectys for one single project, and then multiply that by the couple hundred projects we have to get a rough count | 21:50 |
| *** gokrokve has quit IRC | 21:50 | |
| fungi | mordred suggested possibly splitting manage-projects into a couple of separate moving parts to allow the github and similar interactions to occur asynchronously and at a different frequency and speed to avoid running into rate limiting issues | 21:51 |
| openstackgerrit | A change was merged to openstack/requirements: Update README with rationale and internal mirror https://review.openstack.org/71934 | 21:51 |
| *** hashar has joined #openstack-infra | 21:52 | |
| anteaya | makes sense | 21:52 |
| anteaya | and yes, I would like to see what happens where, so that when the splitting happens the calls are distributed | 21:52 |
| anteaya | not still left in one component | 21:53 |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/config: tempest-dsvm-neutron-heat-slow graduates from experimental https://review.openstack.org/70037 | 21:55 |
| stevebaker | jeblair, fungi, I fixed the commit message ^ | 21:55 |
| fungi | stevebaker: awesome. lgtm | 21:56 |
| openstackgerrit | A change was merged to openstack/requirements: Remove tox locale overrides https://review.openstack.org/72242 | 21:56 |
| jeblair | jog0: i think https://review.openstack.org/#/c/69735/7 could be done with only one web page fairly easily, and just a js function call when selecting a different pipeline... did you consider that? | 21:57 |
| jog0 | jeblair: I did but my JS is bad | 21:58 |
| jog0 | I tried | 21:58 |
| jog0 | and this was easier | 21:58 |
| jog0 | not bette per se | 21:58 |
| jog0 | it also opens us up for other links | 21:58 |
| jog0 | like status.openstack.org/elastic-recheck/data/uncategorized.html | 21:58 |
| jog0 | sdague: ^ | 21:58 |
| sdague | jeblair: in general we've been going down the path of multi pages in e-r, and trying to move logic out of javascript | 21:58 |
| *** mfer has quit IRC | 21:59 | |
| sdague | especially because our contributor set is more python familiar than js | 21:59 |
| jeblair | sdague: okay, but in this case you're literally only changing the url from which the json blob is fetched; a duplication of the wrapping html seems overkill | 21:59 |
| *** hashar has quit IRC | 22:00 | |
| sdague | jeblair: so honestly we probably want to get to a template model here | 22:00 |
| sdague | honestly, I'd be fine on this as incremental change as it will get useful UI out there right now, and we can refactor the back end later | 22:02 |
| jeblair | jog0: thanks, i just wanted to make sure the approach wasn't discarded for some reason; i would prefer to avoid duplication, but i don't immediately have time to work it up myself and i won't ask you to either | 22:02 |
| jeblair | sdague: wfm | 22:02 |
| *** pdmars has quit IRC | 22:03 | |
| clarkb | jeblair: https://gerrit-review.googlesource.com/#/c/54490 | 22:04 |
| *** smurugesan has joined #openstack-infra | 22:04 | |
| jeblair | clarkb: yay! | 22:05 |
| *** apevec has joined #openstack-infra | 22:06 | |
| *** jhesketh has joined #openstack-infra | 22:06 | |
| jhesketh | Howdy | 22:07 |
| *** gokrokve has joined #openstack-infra | 22:07 | |
| jeblair | jhesketh: hello there | 22:07 |
| *** johnthetubaguy has joined #openstack-infra | 22:07 | |
| *** ywu has joined #openstack-infra | 22:08 | |
| stevemar | question for the infra folks: how could we go about gating against warnings/errors in docs for keystone ? | 22:08 |
| jeblair | stevebaker: errors should already be blocked by the gate-keystone-docs job... | 22:09 |
| jeblair | grr | 22:09 |
| jeblair | stevemar: ^ | 22:09 |
| wenlock | mordred, clarkb, was updating the review https://review.openstack.org/#/c/51425/28 and I came across storyboard::init section where story board is installed with pip... any reason why we wouldn't make that a package install with provider pip with reference to the clone directory? | 22:09 |
| * stevebaker goes back to sleep | 22:09 | |
| dolphm | jeblair: for openstack/keystone/docs ? | 22:09 |
| jeblair | stevemar: about warnings; i'm not sure if the needed bits were added to pbr.... | 22:10 |
| *** salv-orlando_ has joined #openstack-infra | 22:10 | |
| jeblair | dolphm: like this job https://jenkins06.openstack.org/job/gate-keystone-docs/174/consoleFull | 22:10 |
| stevemar | jeblair, specifically warnings for improper RST syntax for docstrings | 22:11 |
| dolphm | jeblair: the log is full of non-fatal ERRORs and WARNINGs that we'd like to gate against | 22:11 |
| stevemar | jeblair, yes, what dolphm said | 22:11 |
| dolphm | "build succeeded, 6 warnings." <-- 3 of those are actually logged as "ERRORs" | 22:11 |
| jeblair | lovely | 22:11 |
| stevemar | jeblair, and hopefully make oslo play nice too :) | 22:12 |
| jeblair | i wonder if sphinx counts them all as warnings, even the errors | 22:12 |
| dolphm | stevemar: ++ might just be able to exclude keystone/openstack/common from the doc job | 22:12 |
| clarkb | jeblair: dolphm stevemar I thought pbr already did that for us and it not doing so is a bug | 22:13 |
| stevemar | dolphm, oh good point | 22:13 |
| jeblair | clarkb: yeah, i see "warningiserror=True" in pbr | 22:14 |
| jeblair | so i'm trying to find out what's up with that | 22:14 |
| *** salv-orlando has quit IRC | 22:14 | |
| *** salv-orlando_ is now known as salv-orlando | 22:14 | |
| jeblair | http://git.openstack.org/cgit/openstack-dev/pbr/tree/pbr/packaging.py#n714 | 22:14 |
| anteaya | hey jhesketh | 22:14 |
| *** rlandy has quit IRC | 22:15 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Give test mysql user create and drop perms. https://review.openstack.org/69519 | 22:16 |
| clarkb | now with resolved merge conflicts ^ | 22:17 |
| jeblair | oh i think 'warnerrors' is needed to enable that function | 22:17 |
| jeblair | what in the world is this? | 22:20 |
| jeblair | http://git.openstack.org/cgit/openstack/keystone/commit/?id=3f27b309 | 22:20 |
| jeblair | dolphm, stevemar: ^ | 22:20 |
| *** dangers is now known as dangers_away | 22:20 | |
| jeblair | mordred: ^ | 22:20 |
| jeblair | dolphm: if i'm reading that right, rather than submitting a two line patch to pbr, the author added a new private sphinx extension to keystone? | 22:21 |
| dolphm | dstanek: ^ | 22:22 |
| jeblair | dolphm: please tell me i'm missing something | 22:22 |
| dolphm | jeblair: i don't think so | 22:22 |
| *** dangers_away has quit IRC | 22:23 | |
| mordred | jeblair: looking | 22:23 |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 22:23 |
| dstanek | jeblair: i did submit a patch, but it wasn't that well received and i haven't gotten back to it | 22:24 |
| jeblair | https://bugs.launchpad.net/pbr/+bug/1260495 is the bug | 22:24 |
| uvirtbot | Launchpad bug 1260495 in keystone "Setting autodoc_tree_index_modules makes documentation builds fail" [High,Confirmed] | 22:24 |
| dstanek | jeblair: my attempt at a pbr patch https://review.openstack.org/#/c/61875/ | 22:25 |
| jeblair | stevemar, dolphm: before we go too far down the rabbit hole, this should address your original question: https://review.openstack.org/72492 | 22:25 |
| * fungi needs to step away and cook dinner, but will be back in a while | 22:26 | |
| dolphm | jeblair: ah, thanks! | 22:26 |
| jeblair | stevemar, dolphm: i would say that it should fail tests because of the current errors, but i don't know the impact of the current pbr/apidoc ext issue on that | 22:26 |
| dolphm | jeblair: dstanek: so, i'm comparing the extension in keystone to the pbr patch; jeblair- what's the ~2 line approach? | 22:26 |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 22:27 |
| jeblair | stevemar, dolphm: eg, it might not be run at all right now | 22:27 |
| jeblair | dolphm: oh, it was the last comment on the bug | 22:27 |
| *** dangers_away has joined #openstack-infra | 22:27 | |
| *** thomasem has quit IRC | 22:27 | |
| dolphm | jeblair: ah... any suggestion on where they should live? | 22:28 |
| *** jergerber has joined #openstack-infra | 22:28 | |
| dolphm | jeblair: assuming you're in favor | 22:28 |
| apevec | anteaya, markmcclain - can we declare stable/havana gate as stable as master? Last few rechecks I saw only bug 1254890 and bug 1253896 which aren't solved on master either | 22:28 |
| uvirtbot | Launchpad bug 1254890 in nova ""Timed out waiting for thing ... to become ACTIVE" causes tempest-dsvm-* failures" [Critical,Confirmed] https://launchpad.net/bugs/1254890 | 22:28 |
| uvirtbot | Launchpad bug 1253896 in tempest "Attempts to verify guests are running via SSH fails. SSH connection to guest does not work." [Critical,In progress] https://launchpad.net/bugs/1253896 | 22:28 |
| jeblair | dolphm: since mordred is around, i'd like him to take the lead on that decision | 22:29 |
| *** masayukig has joined #openstack-infra | 22:29 | |
| markmcclain | apevec: maybe.. I had run rechecks and needed to dig through the results | 22:29 |
| apevec | markmcclain, ok, I'd like we push what was pulled out so I can send 2013.2.2 call for testing | 22:30 |
| *** dkranz has quit IRC | 22:32 | |
| *** gokrokve has quit IRC | 22:32 | |
| *** gokrokve has joined #openstack-infra | 22:32 | |
| *** smurugesan1 has joined #openstack-infra | 22:36 | |
| *** smurugesan has quit IRC | 22:36 | |
| *** gokrokve has quit IRC | 22:36 | |
| clarkb | jeblair: any chance you can poke at the crm 114 thing this week? | 22:37 |
| jeblair | clarkb: to try to identify why it times out? | 22:37 |
| clarkb | jeblair: ya | 22:38 |
| *** salv-orlando_ has joined #openstack-infra | 22:39 | |
| jeblair | clarkb: not for a while, i don't think (i don't think it's very high priority now that it's not blocking the rest of the log stuff) | 22:40 |
| jeblair | clarkb: and the review queue is quite big :( | 22:40 |
| *** mattray has quit IRC | 22:41 | |
| clarkb | jeblair: ya, probably not a big priority. the index backlog is pretty big right now though, but that is more to do with the number of tests that have been running all day | 22:43 |
| *** salv-orlando has quit IRC | 22:43 | |
| *** salv-orlando_ is now known as salv-orlando | 22:43 | |
| * clarkb context switches to reviews | 22:44 | |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: fix some small syntax issues in puppet manifest. https://review.openstack.org/72496 | 22:46 |
| *** wchrisj|away is now known as chris_johnson | 22:47 | |
| *** dcramer_ has quit IRC | 22:49 | |
| *** Sukhdev has quit IRC | 22:54 | |
| anteaya | apevec: the only move I will make is if a patch is stuck in the gate I will remove it to keep the gate moving, but if markmcclain agrees, I won't pull them out if they don't cause issues | 22:54 |
| apevec | anteaya, all patches themselves were innocent :) | 22:55 |
| anteaya | doesn't matter, to me if something is failing in the gate, it needs to be removed | 22:56 |
| anteaya | I don't know the code in question | 22:56 |
| anteaya | otherwise the gate gets blocked | 22:56 |
| anteaya | but if you have markmcclain's confidence, that is fine for me to let them in | 22:57 |
| *** eharney has joined #openstack-infra | 22:58 | |
| *** nati_uen_ has quit IRC | 23:01 | |
| jeblair | i think we need to get bare centos nodes up soon. | 23:01 |
| *** shardy is now known as shardy_afk | 23:01 | |
| clarkb | jeblair: ++, pleia2 is working on fedora which should be very similar | 23:01 |
| clarkb | though fedora has new puppet problems so maybe we tackle them more independently | 23:01 |
| *** packet has quit IRC | 23:02 | |
| pleia2 | I think puppet is ok now, working through some nodepool networking stuff (we still hard code /etc/network/interfaces, which doesn't exist on fedora) | 23:03 |
| *** gokrokve has joined #openstack-infra | 23:04 | |
| pleia2 | it does pull in puppet 3, but the corrections I made in template variables should make it work for simple nodes | 23:04 |
| *** banix has quit IRC | 23:04 | |
| *** jamesmcarthur has quit IRC | 23:05 | |
| pleia2 | that said, this shouldn't be an issue in centos because it can actually pull in a 2.7 version of puppet (fedora20's version of ruby is too new) | 23:05 |
| *** malini_afk has quit IRC | 23:05 | |
| *** stevemar has quit IRC | 23:06 | |
| *** malini_afk has joined #openstack-infra | 23:07 | |
| *** malini_afk is now known as malini | 23:07 | |
| *** VijayT has joined #openstack-infra | 23:07 | |
| jeblair | SergeyLukjanov is now a member of openstack-ci-core and openstack-ci-admins in gerrit. congrats! | 23:08 |
| jeblair | (and we really should finish consolidating those) | 23:09 |
| pleia2 | hooray! | 23:09 |
| anteaya | congratulations SergeyLukjanov | 23:10 |
| anteaya | get ready for the deluge while the rest of us are sleeping | 23:10 |
| *** prad has quit IRC | 23:11 | |
| anteaya | I am making myself shut down now that it is dark | 23:11 |
| anteaya | night | 23:11 |
| clarkb | jog0: are you still around? I am going to approve those changes for you, but hoping you cna help untangle them if anything goes sideways | 23:12 |
| jog0 | clarkb: Iam | 23:12 |
| clarkb | jog0: great, approved | 23:13 |
| *** thedodd has quit IRC | 23:13 | |
| *** alexpilotti has quit IRC | 23:13 | |
| jog0 | sweet | 23:14 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add tab to only plot gate failures to status.o.o/elastic-recheck/ https://review.openstack.org/69735 | 23:14 |
| *** dstanek is now known as dstanek_afk | 23:16 | |
| *** nati_ueno has joined #openstack-infra | 23:17 | |
| mordred | congrats SergeyLukjanov ! | 23:18 |
| clarkb | jog0: I did the config change too, but it is taking longer to merge :( | 23:19 |
| *** jgrimm has quit IRC | 23:19 | |
| *** VijayT has quit IRC | 23:19 | |
| openstackgerrit | A change was merged to openstack-infra/config: Generate two elastic-recheck-graph json files all, and gate only https://review.openstack.org/70223 | 23:20 |
| jog0 | that was pretty quick | 23:21 |
| jog0 | clarkb: how long until the web page updates | 23:22 |
| *** nati_ueno has quit IRC | 23:23 | |
| clarkb | jog0: after the :30 puppet updates up to 10 minutes beyond that | 23:23 |
| *** VijayT has joined #openstack-infra | 23:25 | |
| *** wenlock has quit IRC | 23:26 | |
| jog0 | clarkb: thanks | 23:26 |
| *** esker has quit IRC | 23:29 | |
| *** esker has joined #openstack-infra | 23:29 | |
| jog0 | it just updated | 23:29 |
| jog0 | the data isn't there yet (I think) | 23:30 |
| jog0 | sdague: http://status.openstack.org/elastic-recheck/gate.html is quasi live | 23:30 |
| *** sarob has quit IRC | 23:30 | |
| sdague | jog0: cool | 23:30 |
| sdague | jog0: as in neither work now? | 23:30 |
| *** sarob has joined #openstack-infra | 23:31 | |
| *** jergerber has quit IRC | 23:32 | |
| jog0 | sdague: the data has to be generated | 23:32 |
| *** pdmars has joined #openstack-infra | 23:33 | |
| sdague | but the base page is not working either | 23:33 |
| sdague | and that data should have been there | 23:33 |
| *** sarob has quit IRC | 23:33 | |
| *** esker has quit IRC | 23:34 | |
| clarkb | sdague: didn't the base data file change its name too? | 23:34 |
| *** mtreinish has quit IRC | 23:34 | |
| sdague | oh, I guess so | 23:34 |
| sdague | ok, so the real issue is this was supposed to wait until that all happened | 23:35 |
| zaro | jeblair, fungi : I have attached a file to bug 979227 containing the case insensitive duplicates that i found when attempting to convert the gerrit db | 23:35 |
| uvirtbot | Launchpad bug 979227 in openstack-ci "convert gerrit's mysql tables to utf8" [Medium,Triaged] https://launchpad.net/bugs/979227 | 23:35 |
| *** denis_makogon has quit IRC | 23:36 | |
| *** homeless has quit IRC | 23:36 | |
| *** VijayT has quit IRC | 23:36 | |
| *** dkranz has joined #openstack-infra | 23:38 | |
| *** chris_johnson is now known as wchrisj|away | 23:38 | |
| *** mtreinish has joined #openstack-infra | 23:39 | |
| *** cadenzajon has quit IRC | 23:40 | |
| *** virmitio has quit IRC | 23:40 | |
| *** masayukig has quit IRC | 23:41 | |
| *** CaptTofu has quit IRC | 23:41 | |
| *** jnoller has quit IRC | 23:42 | |
| *** e0ne has quit IRC | 23:42 | |
| jog0 | clarkb: so I don't see any data on http://status.openstack.org/elastic-recheck/gate.html yet | 23:43 |
| *** e0ne has joined #openstack-infra | 23:44 | |
| clarkb | jog0: its running now | 23:46 |
| jog0 | clarkb: thanks | 23:46 |
| *** cadenzajon has joined #openstack-infra | 23:47 | |
| *** mgagne has quit IRC | 23:47 | |
| *** dstanek_afk is now known as dstanek | 23:49 | |
| *** johnthetubaguy has quit IRC | 23:49 | |
| *** smurugesan1 has quit IRC | 23:50 | |
| *** johnthetubaguy has joined #openstack-infra | 23:50 | |
| *** mrda is now known as mrda_away | 23:50 | |
| *** johnthetubaguy has quit IRC | 23:50 | |
| *** markwash has quit IRC | 23:51 | |
| jog0 | sdague: check it out http://status.openstack.org/elastic-recheck/gate.html | 23:52 |
| *** sarob has joined #openstack-infra | 23:52 | |
| jog0 | dims: ^ | 23:52 |
| *** mriedem has quit IRC | 23:52 | |
| *** markmcclain has quit IRC | 23:53 | |
| *** sandywalsh has quit IRC | 23:53 | |
| *** zul has quit IRC | 23:54 | |
| *** wenlock has joined #openstack-infra | 23:56 | |
| *** wenlock has quit IRC | 23:56 | |
| *** wenlock has joined #openstack-infra | 23:57 | |
| clarkb | jog0: sdague: looking at es graphs, we may want to only run those jobs once an hour instead of 4x an hour | 23:57 |
| clarkb | es goes a bit crazy and I think it may contribute to the backlog being so large | 23:58 |
| clarkb | but we can watch it this week and see | 23:58 |
| sdague | clarkb: sure, hourly should be completely sufficient | 23:58 |
| *** whoops has quit IRC | 23:59 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/elastic-recheck: Fix alignment of tabs https://review.openstack.org/72503 | 23:59 |
| jeblair | jog0: ^ prettification | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!