| *** openstack has joined #openstack-infra | 00:09 | |
| *** asettle has quit IRC | 00:09 | |
| clarkb | I Think it cleans up after breaking so logs for fail will be before cleanup | 00:10 |
|---|---|---|
| cody-somerville | jeblair: figured it out. Good catch. Fixed uploaded ^^ | 00:11 |
| fungi | yeah, i end up scrolling up a few dozen lines to look for errors about missing files or whatnot | 00:11 |
| craige | OMG, we have comments mordred! | 00:11 |
| cody-somerville | jeblair: FYI, looks like --open argument in Python3 is broken | 00:11 |
| *** hdd_ has joined #openstack-infra | 00:11 | |
| *** hdd has quit IRC | 00:12 | |
| *** asettle has joined #openstack-infra | 00:12 | |
| pleia2 | bot rescued successfully | 00:12 |
| craige | I'll need to updated the etherpad but as far the actual data migration to Phabricator is concerned, all blocking issues are resolved fungi. | 00:13 |
| anteaya | pleia2: nice work | 00:13 |
| craige | so I'd love more people to hit it and find things wrong (preferrably right) as I stand up phabricator.openstack.org. | 00:14 |
| anteaya | pleia2: I told meeting-4 the bot is back | 00:14 |
| craige | Current dev instance is http://os01.mcwhirter.io/ | 00:14 |
| *** davideag_ has quit IRC | 00:14 | |
| anteaya | pleia2: but the meeting in there was already over | 00:14 |
| SamYaple | this is the closest ERROR i have http://paste.openstack.org/show/489936/ | 00:14 |
| *** davideagnello has joined #openstack-infra | 00:14 | |
| SamYaple | have you seen it before and can you save me troubleshooting time? | 00:14 |
| SamYaple | (ubuntu1 is this hose) | 00:14 |
| pleia2 | anteaya: ah yeah, I didn't bother with -3 or -4 since there wasn't anything going on right now | 00:14 |
| SamYaple | host* | 00:14 |
| anteaya | pleia2: yup | 00:15 |
| *** placko_ has quit IRC | 00:16 | |
| *** Qiming has quit IRC | 00:16 | |
| *** placko_ has joined #openstack-infra | 00:17 | |
| *** openstackgerrit has quit IRC | 00:17 | |
| *** dimtruck is now known as zz_dimtruck | 00:17 | |
| fungi | craige: awesome! now we need people to poke the demo more aggressively and report back on whether there are missing features we need before we can consider a go-live cutover | 00:17 |
| fungi | ttx: ^ (when you're not busy travelling) | 00:18 |
| mordred | craige: WOOT!!!! | 00:18 |
| *** openstackgerrit has joined #openstack-infra | 00:18 | |
| *** mescanef has joined #openstack-infra | 00:18 | |
| *** tonytan4ever has joined #openstack-infra | 00:18 | |
| SamYaple | mordred: you see this dib failure? http://paste.openstack.org/show/489936 | 00:22 |
| *** flepied has quit IRC | 00:22 | |
| craige | Much w00t mordred much w00tness. | 00:23 |
| craige | Yes, that's what i'm hoping fungi. | 00:23 |
| craige | I'm also trying to now stand up phabricator.openstack.org to have a more accurate instance for poking. However people should absolutely hammer my dev instance in the interim. | 00:24 |
| *** nwkarsten has quit IRC | 00:24 | |
| craige | mtreinish, yolanda ^^ | 00:24 |
| *** nwkarsten has joined #openstack-infra | 00:25 | |
| dims | fungi : clarkb : anyone seen weirdness in just simple pep8 jobs? apt-get times out after 30 mins - http://logs.openstack.org/70/290670/1/check/gate-oslo.privsep-pep8/c9336d9/console.html#_2016-03-09_21_56_24_048 | 00:26 |
| *** bpokorny has quit IRC | 00:28 | |
| fungi | dims: pabelanger also noted some very poor i/o performance in vexxhost on another job | 00:28 |
| ianw | fungi: i *really* learnt C thanks to an internship at SGI where i had not too much to do, and plenty of time to play with irix's often hilariously lax security with the benefit of source code. IndyCam's were freaking awesome | 00:29 |
| *** bpokorny has joined #openstack-infra | 00:29 | |
| fungi | ianw: far better than my experience learning korn shell by trying to fix sco openserver! | 00:29 |
| craige | oh, ew. | 00:29 |
| *** yamahata has joined #openstack-infra | 00:30 | |
| dims | fungi : ack thanks | 00:30 |
| *** nwkarsten has quit IRC | 00:30 | |
| * ianw now knows fungi is good for a korn shell port of devstack | 00:30 | |
| dims | haha | 00:30 |
| fungi | dims: apt/dpkg are pretty hard on disk i/o, so while i think this is likely a deeper performance issue there are probably also some simple things we can do to make that a lot bit faster | 00:30 |
| *** xarses has quit IRC | 00:31 | |
| *** asettle has quit IRC | 00:31 | |
| fungi | ianw: i'd sooner port it to almquist | 00:32 |
| *** asettle has joined #openstack-infra | 00:32 | |
| fungi | (which granted would not be hard at all) | 00:32 |
| fungi | ooh, i shoulda said "plan 9 rc" | 00:34 |
| *** flepied has joined #openstack-infra | 00:34 | |
| fungi | that would be sorta fun | 00:34 |
| pleia2 | lol | 00:35 |
| *** dims has quit IRC | 00:35 | |
| fungi | also, on the topic of operating systems which went missing in action, whatever happened to beos? | 00:36 |
| openstackgerrit | Craige McWhirter proposed openstack-infra/system-config: Added RemoteUser Provider & OpenID configs https://review.openstack.org/254066 | 00:36 |
| fungi | oh, right, haiku still exists as a descendant | 00:37 |
| pleia2 | yeah, haiku is still active | 00:37 |
| craige | beos and OS/2 are having a little tea party together. | 00:38 |
| * fungi warps to the sofa | 00:38 | |
| craige | heh | 00:38 |
| craige | Warps 4 the sofa? | 00:38 |
| fungi | *groan* | 00:38 |
| pleia2 | about 10 years ago I would attend a user group that started with os/2 and morphed into a general systems meetup | 00:38 |
| craige | \o/ | 00:38 |
| pleia2 | but their membership didn't, I was always the youngest, by far (and the only woman) | 00:39 |
| craige | sounds like many LUGs. | 00:39 |
| pleia2 | it was fun though :) | 00:39 |
| fungi | in the places i lived, the bugs were always more fun than the lugs | 00:39 |
| craige | I loved OS/2, until Linux came out. | 00:39 |
| fungi | craige: which granted was 1992 if you were spending any time downloading source from usenet | 00:40 |
| SamYaple | mmmm i still have a PS/2 sitting i a closet somewhere | 00:40 |
| craige | No, I linux didn't come out for me until 1994, when I bought an infomagic double CD in Chinatwon, the only place in Sydney to get it. | 00:41 |
| fungi | ooh, infomagic. that brings back memories | 00:41 |
| craige | I'd never even used UNIX before, went home, replaced Os/2 with Linux and start there at the blinking slackware darkstar # prompt with no idea how to do ANYTHING. | 00:42 |
| * craige then went and bought a book... | 00:42 | |
| SamYaple | i few years back i had this customer how was using a ps/2 running os/2 for all of thier vulcanization equipment. been running there for like 30 years then it broke | 00:42 |
| pleia2 | SamYaple: sounds like a very bad day | 00:42 |
| SamYaple | you would think so but no | 00:42 |
| SamYaple | battery had died. went down to radio shack and bought a replacement. cause you know, they had it on the shelf | 00:43 |
| SamYaple | thats what was most shocking | 00:43 |
| SamYaple | the battery, not a ps/2 | 00:43 |
| pleia2 | gah, so they didn't stop using it? | 00:43 |
| *** denisra_ has joined #openstack-infra | 00:43 | |
| SamYaple | no | 00:43 |
| pleia2 | bad day postponed :) | 00:43 |
| * fungi remembers when you could buy basic circuit components at radio shack. also parts for radios | 00:43 | |
| SamYaple | they would have had to spend millions to replace that | 00:43 |
| *** BobBall_ has joined #openstack-infra | 00:43 | |
| *** asselin_ has joined #openstack-infra | 00:44 | |
| *** bpokorny has quit IRC | 00:44 | |
| *** BobBall_ is now known as Guest3102 | 00:44 | |
| SamYaple | i did convince them it was cheaper to buy all the ps/2 stuff on ebay they could find | 00:44 |
| Clint | and forrest mims books | 00:44 |
| SamYaple | that included a 51/2" floppy drive | 00:44 |
| SamYaple | they spent a few thouhsand and have all the spare parts you could want | 00:44 |
| *** bpokorny has joined #openstack-infra | 00:44 | |
| SamYaple | i think it had 4MB of ram | 00:44 |
| *** Volundr has joined #openstack-infra | 00:45 | |
| jokke_ | fungi: yeah, now ye get them from ebay ;) | 00:46 |
| fungi | i went into a radio shack recently, it was like an outpost for best buy. cell phones and flat screens. want a 20-pack of 10kohm .25w 5% resistors? sorry, what decade are you from? | 00:46 |
| *** isq_ has joined #openstack-infra | 00:46 | |
| *** mwhahaha_ has joined #openstack-infra | 00:46 | |
| *** cbader_ has joined #openstack-infra | 00:46 | |
| fungi | i'm a digikey junkie these days | 00:46 |
| *** kuromagi^ has joined #openstack-infra | 00:46 | |
| SamYaple | ive got a store near me with all that stuff if i dont want to wait for china | 00:46 |
| *** evgenyl_ has joined #openstack-infra | 00:47 | |
| bkero | fungi: Oh, you're one of *rolls eyes* THOSE customers. The parts bin is in the back. Don't ask us about it. | 00:47 |
| *** aysyd has quit IRC | 00:47 | |
| *** prometheanfire has quit IRC | 00:47 | |
| *** kukacz_ has joined #openstack-infra | 00:47 | |
| *** Nakato_ has joined #openstack-infra | 00:48 | |
| bkero | I still have/use the original IBM Model M from my first computer, an IBM 386. | 00:48 |
| fungi | i just had such fond memories of the "old" radio shack. it's sad to be reminded of what they've become | 00:48 |
| bkero | I think it was a 386DX. Fancy fancy. | 00:48 |
| *** sheeprine_ has joined #openstack-infra | 00:48 | |
| bkero | fungi: there were some efforts to revitalize it with 'hardware for a newer generation'. Arduinos and shields. | 00:49 |
| jokke_ | fungi: icbm controller accurate enough for hobbyist use is so much cheaper to build nowadays :P | 00:49 |
| *** ruhe has quit IRC | 00:49 | |
| *** mwhahaha has quit IRC | 00:49 | |
| *** ptoohill has quit IRC | 00:49 | |
| *** eglute has quit IRC | 00:49 | |
| *** evgenyl has quit IRC | 00:49 | |
| *** mkoderer__ has quit IRC | 00:49 | |
| *** denisra has quit IRC | 00:49 | |
| *** aeng has quit IRC | 00:49 | |
| *** harlowja_at_home has quit IRC | 00:49 | |
| *** zzzeek has quit IRC | 00:49 | |
| *** kushal has quit IRC | 00:49 | |
| *** cbader has quit IRC | 00:49 | |
| *** gmann has quit IRC | 00:49 | |
| *** Kiall has quit IRC | 00:49 | |
| *** BobBall has quit IRC | 00:49 | |
| *** whoops has quit IRC | 00:49 | |
| *** jbernard_ has quit IRC | 00:49 | |
| *** sheeprine has quit IRC | 00:49 | |
| *** sbadia has quit IRC | 00:49 | |
| *** asselin__ has quit IRC | 00:49 | |
| *** pcrews has quit IRC | 00:49 | |
| *** kuromagi has quit IRC | 00:49 | |
| *** isq has quit IRC | 00:49 | |
| *** kukacz has quit IRC | 00:49 | |
| *** madorn has quit IRC | 00:49 | |
| *** verdurin has quit IRC | 00:49 | |
| *** Nakato has quit IRC | 00:49 | |
| *** ruhe2 has joined #openstack-infra | 00:49 | |
| *** tqtran is now known as tqtran-afk | 00:49 | |
| *** Kiall has joined #openstack-infra | 00:49 | |
| *** ruhe2 is now known as ruhe | 00:49 | |
| *** madorn_ has joined #openstack-infra | 00:49 | |
| fungi | heh | 00:49 |
| *** eglute_s has joined #openstack-infra | 00:49 | |
| *** evgenyl_ is now known as evgenyl | 00:49 | |
| *** mwhahaha_ is now known as mwhahaha | 00:49 | |
| *** gmann has joined #openstack-infra | 00:49 | |
| *** ptoohill has joined #openstack-infra | 00:49 | |
| *** mkoderer___ has joined #openstack-infra | 00:49 | |
| *** aeng has joined #openstack-infra | 00:49 | |
| *** sbadia has joined #openstack-infra | 00:50 | |
| *** harlowja_at_home has joined #openstack-infra | 00:50 | |
| jokke_ | bkero: those are great, but try to find store selling them (or any components) without ridiculous prices | 00:50 |
| *** kushal has joined #openstack-infra | 00:50 | |
| bkero | jokke_: Radio Shcak | 00:50 |
| bkero | Err, Radio Shack* | 00:50 |
| *** jbernard_ has joined #openstack-infra | 00:50 | |
| *** zzzeek has joined #openstack-infra | 00:50 | |
| *** verdurin has joined #openstack-infra | 00:50 | |
| *** prometheanfire has joined #openstack-infra | 00:50 | |
| *** pcrews has joined #openstack-infra | 00:50 | |
| fungi | patronize adafruit.com as much as you can. they're awesome | 00:51 |
| *** cody-somerville has quit IRC | 00:51 | |
| jokke_ | bkero: so they actually stock components still as well, got bit different impression from fungi above ;) | 00:51 |
| bkero | https://engineerzero.wordpress.com/2012/01/04/buying-an-arduino-at-radio-shack/ | 00:51 |
| jokke_ | fungi: them or sparkfun.com | 00:51 |
| jokke_ | sparkfun stocks bit weirder stuff as well | 00:52 |
| bkero | I usually sparkfun, adafruit, or seeed | 00:52 |
| fungi | jokke_: they do. there's usually a corner in the back of the store littered with empty soda cups and old napkins with some inaccurately-marked bins of parts they rarely get around to restocking | 00:52 |
| fungi | yeah, the sparkfun crowd are great too. frencil is a hoot | 00:52 |
| fungi | i've not checked out seeed but doing now | 00:53 |
| Clint | isn't that a german reggae band | 00:54 |
| bkero | There's a really cool PCB marketplace startup in Portland. I'm trying to remember the name of it. | 00:54 |
| fungi | Clint: you're thinking of ß"eed | 00:54 |
| * Clint twitches. | 00:55 | |
| SamYaple | bkero: dvdshack? | 00:55 |
| fungi | er,ßëed | 00:55 |
| bkero | heh no | 00:55 |
| jokke_ | we have store chain called maplin here. Like I tried to get some basic components from them few months back. Can't remember exactly what, but it was something you would expect any hobbyist needing. Their individual store saldo was 2 ... an not like "We've sold everything out, get more next week" But their restock was actually 2 ... and component out sale price was like .30EUR | 00:55 |
| bkero | Tindie! | 00:55 |
| bkero | (not Tinder) | 00:55 |
| bkero | https://www.tindie.com/ just scroll down | 00:55 |
| fungi | their motto totally should have been "just scroll down" | 00:56 |
| SamYaple | fungi: i think "swipe up" would work better for "tindie" | 00:58 |
| fungi | hah | 00:58 |
| clarkb | dmsimard: http://logs.openstack.org/76/290776/8/check/gate-grenade-dsvm/ed652db/logs/devstack-gate-setup-workspace-old.txt.gz that job failed and I wonder if it is because we ran the setup twice for grenade | 00:59 |
| *** Jeffrey4l has joined #openstack-infra | 00:59 | |
| SamYaple | alright i can do this no more. i have no idea why dib is breaking | 00:59 |
| clarkb | SamYaple: sorry just getting back to computer and can look at some pastes | 00:59 |
| clarkb | buying a house is literally the worst thing ever | 01:00 |
| clarkb | highly recommend doing it before you have twins | 01:00 |
| *** asettle has quit IRC | 01:00 | |
| *** rhallisey has quit IRC | 01:00 | |
| jokke_ | I grew up about 15miles out of small city (<90k people) and we had 3 electronics stores in that city ... you got project idea at Fri, made BOM and picked up Saturday morning (normally had to go max to the second store to get all you wanted) you were done Sunday. Now you get project idea at fri evening, you order the parts, sit on your thumb for the weekend and maybe get the parts by Tue or Wed | 01:00 |
| SamYaple | clarkb: that one just finished http://paste.openstack.org/show/489938/ no idea what to make of it | 01:00 |
| SamYaple | clarkb: i just bought mine (and sold one out of state) | 01:00 |
| SamYaple | some places its fine, others it is not | 01:01 |
| SamYaple | in TX it was all online, here in SC it was all on paper :/ | 01:01 |
| clarkb | in portland its just the crazyness of the market making everything crazy | 01:02 |
| SamYaple | jokke_: get your ideas on mon or tue. problem solved! | 01:02 |
| clarkb | Found house we liked then they drop the bomb on you that it burned down 3 years ago | 01:02 |
| clarkb | ya nope | 01:03 |
| pleia2 | clarkb: wow :( | 01:03 |
| SamYaple | yea and there are some new 2016 laws on the books making actually buying it even harder | 01:03 |
| *** tonytan4ever has quit IRC | 01:03 | |
| SamYaple | the lawyers have more to do now which slows things down | 01:03 |
| clarkb | yes apparently closing times have increased ~33% | 01:03 |
| *** markus_z has quit IRC | 01:04 | |
| *** Jeffrey4l has quit IRC | 01:04 | |
| fungi | SamYaple: where in sc? i'm practically next door (nc) | 01:04 |
| SamYaple | well the federal delay only adds 3 days, but the new laws mean people havent streamlined the process | 01:04 |
| clarkb | SamYaple: that log looks like puppet is breaking for some reason | 01:04 |
| SamYaple | fungi: greenville | 01:04 |
| SamYaple | clarkb: youve gotten as far as me! | 01:04 |
| SamYaple | fungi: i go to ashville and charlotte a bit too | 01:05 |
| clarkb | but good news is we found something else that we ust put an offer on so we shall see how that goes | 01:05 |
| jokke_ | fungi: where in nc are you? | 01:05 |
| fungi | oof. you're practically next door to my parents in hendersonville nc, but i'm at the opposite end of the state any more | 01:05 |
| fungi | jokke_: kill devil hills, out on the barrier islands | 01:05 |
| greghaynes | can confirm, PDX house buying is the worst | 01:06 |
| jokke_ | fungi: uuuh ... looks nice | 01:06 |
| jokke_ | fungi: although not exactly coffee meet distance for next week :P I'm landing RDU Sun night and leaving Fri morning | 01:07 |
| *** tonytan4ever has joined #openstack-infra | 01:07 | |
| SamYaple | clarkb: any ideas on the puppet breakage? | 01:07 |
| fungi | jokke_: it is nice. also slow to get to/from. closest "major" airport (and i use the term loosely) is 1.5 hours by car. rdu is 3 hours | 01:07 |
| *** rguillebert has quit IRC | 01:07 | |
| *** markusz has joined #openstack-infra | 01:08 | |
| fungi | i lived in the raleigh area for 15 years. got tired of it | 01:08 |
| jokke_ | fungi: bit like Galway, Ireland (where I'm at) closest international is about an hour away and Dublin Airport just over 2hrs ... Over here needs effort though to manage to spend 3 hours driving from point to point :P | 01:09 |
| *** angdraug has quit IRC | 01:09 | |
| fungi | we desperately need to have a summit in the dublin area | 01:10 |
| jokke_ | fungi: so anything special I definitely should visit there? (You know any good breweries or coffee roasteries around Raleigh?) | 01:10 |
| jokke_ | fungi: that would be handy | 01:10 |
| openstackgerrit | Merged openstack-infra/puppet-askbot: Add captcha to the askbot venv https://review.openstack.org/288916 | 01:11 |
| *** sridhar_ram has quit IRC | 01:11 | |
| *** Daisy has joined #openstack-infra | 01:11 | |
| fungi | jokke_: oh, sure, there are plenty. "cup-a-joe" is the local coffee chain, they do their own roasting. also lots of brewery-pub establishments now. are you staying downtown or nearer the airport? | 01:11 |
| fungi | if you're in downtown raleigh there are a few easy walking distance (i'll omit the "irish" pubs) | 01:12 |
| jokke_ | downtown ... sheraton | 01:13 |
| fungi | yeah, i should be able to give you some recommendations... just a sec | 01:13 |
| jokke_ | Brilliant! | 01:14 |
| fungi | my favorite is aviator, but they're not downtown. you're still likely to find a lot of their stuff on tap at various restaurants there | 01:15 |
| *** yonglihe has joined #openstack-infra | 01:15 | |
| *** thorst_ has quit IRC | 01:15 | |
| *** Daisy has quit IRC | 01:16 | |
| jokke_ | yeah, time is kind of a problem. I'm expecting to sit majority in the office (not exactly a holiday trip) | 01:17 |
| fungi | i see there's a "boylan bridge" brewpub, which is likely an easy walk, though that's a new one on me | 01:17 |
| fungi | clarkb should go check that out if he ever gets to raleigh ;) | 01:17 |
| *** ddieterly has joined #openstack-infra | 01:17 | |
| *** thorst_ has joined #openstack-infra | 01:17 | |
| fungi | crank arm brewery isn't too bad | 01:17 |
| clarkb | wait what | 01:18 |
| *** dalgaaf has quit IRC | 01:18 | |
| jokke_ | fungi: cheers, appreciated. | 01:19 |
| fungi | raleigh brewing on neil st is right off hillsborough (the main drag through downtown next to the main ncsu campus) | 01:20 |
| *** Sukhdev has joined #openstack-infra | 01:20 | |
| fungi | trophy brewing is also right around there | 01:20 |
| fungi | sub noir and big boss breweries aren't right downtown but probably still walkable in a reasonable timeframe | 01:21 |
| *** Sukhdev has quit IRC | 01:22 | |
| fungi | there are a bunch more up on the north side of town where i used to live, but there's not much point in me recommending them | 01:22 |
| *** cody-somerville has joined #openstack-infra | 01:22 | |
| *** nikhil_k has quit IRC | 01:22 | |
| jokke_ | that sounds already like a list that I will need to use taxis from half way of the beer night and hope that they have wheelbarrow at the hotel to deliver me into my room :P | 01:23 |
| *** Odd_Bloke has quit IRC | 01:23 | |
| *** Odd_Bloke has joined #openstack-infra | 01:23 | |
| fungi | heh, they might | 01:23 |
| *** lezbar__ has quit IRC | 01:24 | |
| *** thorst_ has quit IRC | 01:24 | |
| *** Sukhdev has joined #openstack-infra | 01:24 | |
| fungi | there's also lots of great restaurants and bars right around where the downtown sheraton is. you shouldn't have any trouble finding something to do with your evenings | 01:24 |
| *** mriedem has joined #openstack-infra | 01:25 | |
| *** pvaneck has quit IRC | 01:25 | |
| *** gyee has quit IRC | 01:25 | |
| jokke_ | sounds good. As long as I don't need to touch Miller and such but get something local for my taste buds I'll be happy | 01:25 |
| fungi | yeah, raleigh has an excellent local brewing scene. i always drank local beers when i lived there | 01:27 |
| *** Sukhdev has quit IRC | 01:27 | |
| fungi | pretty much any place with more that 2 taps had at least one local keg on | 01:28 |
| *** Sukhdev has joined #openstack-infra | 01:28 | |
| *** aorourke is now known as aorourke-afk | 01:28 | |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Exclude python-troveclient 2.1.0 https://review.openstack.org/290935 | 01:28 |
| *** jpr has joined #openstack-infra | 01:29 | |
| jeblair | drat, southend closed | 01:29 |
| jeblair | i used to walk there from my house | 01:30 |
| *** dims has joined #openstack-infra | 01:31 | |
| jokke_ | fungi: I'm bit spoiled. My local is http://www.galwaybaybrewery.com/salthouse/ | 01:31 |
| fungi | jeblair: 10 years ago, apparently. there's still a southend brewery in charlotte, fwiw | 01:32 |
| fungi | jokke_: that looks amazing. now i have another reason to visit galway | 01:33 |
| bkero | jokke_: Busy Bee Cafe was pretty nice when I was there | 01:34 |
| clarkb | jokke_: is it ok for me to complain that the sign says "america craft beer" then has a blue moon logo? | 01:35 |
| fungi | jeblair: apparently the locations in charlotte nc and charleston sc were affiliated with the one in raleigh | 01:35 |
| *** persia has quit IRC | 01:35 | |
| jeblair | fungi: yeah. good bbq _and_ good beer. | 01:36 |
| jokke_ | clarkb: absolutely | 01:36 |
| jeblair | good ribs even. | 01:36 |
| clarkb | ok good, blue moon isn't exactly craft beer | 01:36 |
| *** Qiming has joined #openstack-infra | 01:37 | |
| jeblair | they probably make less of it than guiness so it counts as craft? i dunno :) | 01:37 |
| jeblair | clarkb: http://www.marketwatch.com/story/how-blue-moon-made-craft-beer-meaningless-2015-05-01 | 01:38 |
| *** persia has joined #openstack-infra | 01:38 | |
| jokke_ | clarkb: I think you should read it as American, craft beer & Blue Moon :P | 01:38 |
| fungi | hah | 01:38 |
| *** jpr has quit IRC | 01:38 | |
| clarkb | I am just proud to live in the state that drinks the most beer brewed locally per capita | 01:38 |
| jokke_ | clarkb: sound like a place I wanna visit :D | 01:39 |
| *** jpr has joined #openstack-infra | 01:39 | |
| *** Sukhdev has quit IRC | 01:39 | |
| *** Sukhdev has joined #openstack-infra | 01:39 | |
| bkero | also most microbreweries per capita | 01:41 |
| bkero | in the world AIUI | 01:41 |
| *** placko_ has quit IRC | 01:41 | |
| jokke_ | Vancouver was nice place beer wise | 01:41 |
| *** Sukhdev has quit IRC | 01:41 | |
| *** Sukhdev has joined #openstack-infra | 01:42 | |
| jeblair | fungi: last time i was in raleigh, my friends took me to all the nice new hip bars and restaurants... but when they weren't looking i went to places they take for granted like the seafood market, clyde coopers, big ed's, etc. | 01:42 |
| SamYaple | asheville, nc pumps out alot of beer | 01:43 |
| SamYaple | the water is just right there so they run it through some charcoal filters and use it | 01:43 |
| fungi | jeblair: yeah, there are still some great old staples there | 01:43 |
| SamYaple | most other places need alot more | 01:43 |
| *** kzaitsev_mb has quit IRC | 01:43 | |
| *** mtanino has quit IRC | 01:43 | |
| clarkb | SamYaple: thats a big reason for it here too. Our water has almost nothing in it | 01:43 |
| *** aeng has quit IRC | 01:44 | |
| clarkb | the water department apparently calls all the breweries when they switch to the well feed | 01:44 |
| fungi | jeblair: seafood market as in the one at the entrance to the farmers market? | 01:44 |
| clarkb | "hey you might need to treat your water now" | 01:44 |
| SamYaple | lol | 01:44 |
| jeblair | fungi: no, the one on the right in the middle of the stands that basically just serves fried-anything-from-the-ocean | 01:44 |
| *** edmondsw has quit IRC | 01:44 | |
| SamYaple | i just had a bunch of west coast beer | 01:44 |
| fungi | jeblair: aha, yep. good stuff | 01:44 |
| bkero | clarkb: AIUI our water is pretty soft | 01:45 |
| SamYaple | firestone walker stuff | 01:45 |
| SamYaple | it was good | 01:45 |
| clarkb | bkero: yup, its like 3 ppm total additives | 01:45 |
| clarkb | bkero: many breweries add stuff to our water | 01:45 |
| bkero | "Portland’s water is very soft. The hardness of Bull Run water is typically 3-8 parts per million (ppm) – approximately ¼ to ½ a grain of hardness per gallon. Portland’s groundwater hardness is approximately 80 ppm (about 5 grains per gallon), which is considered moderately hard." | 01:45 |
| bkero | clarkb: Eugene breweries just take it right out of the Willamette | 01:45 |
| *** armax has quit IRC | 01:46 | |
| *** Sukhdev has quit IRC | 01:46 | |
| *** Sukhdev_ has joined #openstack-infra | 01:46 | |
| fungi | the asheville area is starting to get overrun with larger chain breweries opening though. in the past few years sierra nevada, fat tire and oskar blues have muscled in | 01:46 |
| clarkb | bkero: I don't recommend that up here | 01:47 |
| fungi | mostly because of water supply, again | 01:47 |
| jokke_ | I've lived my whole life max 15miles from sea ... I'd have a trust issue towards sea food market that is located xxx miles inland :P | 01:47 |
| SamYaple | jeez. in TX we had 3ppm of clhorine.... | 01:47 |
| bkero | clarkb: the Willamette is different down there. Not nearly as dirty. | 01:47 |
| clarkb | SamYaple: and when somone pees in the resevoir they drain it and refill | 01:47 |
| clarkb | nevermind all the animals in it | 01:48 |
| *** cody-somerville has quit IRC | 01:48 | |
| fungi | jokke_: raleigh is actually not very far up river. most of the fresh seafood there was caught and hauled ashore just a few miles from my house around sunrise, then spent a few hours in a refrigerated truck | 01:48 |
| SamYaple | i always like to point that out to drinkers of "spring bottled water" | 01:48 |
| *** Sukhdev has joined #openstack-infra | 01:49 | |
| fungi | jokke_: but i agree, much better here right on the ocean ;) | 01:49 |
| *** Sukhdev_ has quit IRC | 01:49 | |
| *** sarob has quit IRC | 01:49 | |
| SamYaple | im from myrtle beach originally. they have some good seafood too | 01:49 |
| *** fawadk has quit IRC | 01:49 | |
| * Clint salivates. | 01:50 | |
| clarkb | in portland the seafood swims upriver to make it easier to eat it | 01:50 |
| jeblair | i think it was easier to get fresh seafood in raleigh than sf | 01:50 |
| jeblair | there are some places here, but it's taken me a while to find them, and they're not where you'd expect. | 01:50 |
| *** takashi has joined #openstack-infra | 01:50 | |
| Clint | baltimore had that problem for a couple years | 01:50 |
| clarkb | jeblair: is it mostly shellfish and salmon? | 01:50 |
| clarkb | jeblair: thats our problem | 01:50 |
| SamYaple | clarkb: i couldnt figure out the puppet thing so i just had it not report error codes | 01:51 |
| *** jpr has quit IRC | 01:51 | |
| clarkb | SamYaple: probably good enough for what you are doing | 01:51 |
| *** sputnik13 has quit IRC | 01:51 | |
| *** Sukhdev has quit IRC | 01:51 | |
| SamYaple | i "fixed" it! | 01:51 |
| clarkb | SamYaple: I will have to dig in toorrow and see if our builds are failing too | 01:51 |
| jokke_ | I have to admit that while living in Dublin, I made sure that the seafood I got was either from West or North, not from the Irish sea ;) | 01:51 |
| fungi | yeah, nice thing about here is we can go right down to the warf and buy right from catches as they're coming off the boat. great if you're, say, having a party and want 15 lbs of fresh shrimp or something | 01:51 |
| *** Sukhdev has joined #openstack-infra | 01:52 | |
| SamYaple | when i was a kid i used to go down and take lobsters out of the lobster traps. i was told they were there for everyone | 01:52 |
| *** thorst_ has joined #openstack-infra | 01:52 | |
| SamYaple | i did not have the best parent in that situation | 01:52 |
| SamYaple | but it was delicious lobster | 01:52 |
| jeblair | clarkb: most of our salmon is _your_ salmon is the big problem :| | 01:52 |
| fungi | SamYaple: the error code reporting in puppet is insane, so this does not surprise me. they have several non-zero "success" exit codes | 01:53 |
| jeblair | ca runs are in very bad shape | 01:53 |
| *** Sukhdev has quit IRC | 01:53 | |
| SamYaple | fungi: in this case its _probably_ validly reporting non-zero, but i dont know anything about puppet | 01:53 |
| SamYaple | and frankly id like to keep it that way | 01:53 |
| clarkb | ya its a valid error, it can't write to /etc/dhcp because the dir doesn't exist | 01:54 |
| *** Sukhdev has joined #openstack-infra | 01:54 | |
| SamYaple | that should all be in the chroot environment, right? | 01:54 |
| clarkb | SamYaple: yup, its not your system's etc/dhcp | 01:54 |
| jokke_ | fungi: that's the lovely part living here by Atlantic. 6 days a week day's catch available from local wholesale at the harbour | 01:54 |
| SamYaple | yea then i dont know | 01:54 |
| SamYaple | either way ill have the fstrim results soo | 01:54 |
| SamYaple | soon* | 01:54 |
| clarkb | SamYaple: I am going to guess its something like simple-init element missing | 01:55 |
| clarkb | SamYaple: maybe add the simple-init next time around if you have patience for it | 01:55 |
| SamYaple | it takes about 10m to run the script so ill do it again | 01:55 |
| *** gongysh has joined #openstack-infra | 01:55 | |
| *** Sukhdev has quit IRC | 01:56 | |
| *** Sukhdev has joined #openstack-infra | 01:56 | |
| *** Jeffrey4l has joined #openstack-infra | 01:57 | |
| SamYaple | clarkb: cool. the results are in and are what i expected. 4.6GB | 01:58 |
| *** gildub has quit IRC | 01:58 | |
| clarkb | SamYaple: nice! | 01:58 |
| SamYaple | patch incoming | 01:58 |
| *** gildub has joined #openstack-infra | 01:58 | |
| *** Sukhdev has quit IRC | 01:58 | |
| clarkb | and this will work with xfs if we choose to make centos7 images that way | 01:58 |
| *** Sukhdev has joined #openstack-infra | 01:59 | |
| *** ddieterly has quit IRC | 01:59 | |
| prometheanfire | SamYaple: mind letting me know what you think? https://review.openstack.org/#/c/290894/ | 01:59 |
| SamYaple | clarkb: correct | 02:00 |
| SamYaple | or btrfs (one can hope) | 02:00 |
| clarkb | SamYaple: I was reading an lkml article from 4 years ago where they said btrfs will be default everywhere soon | 02:00 |
| *** Sukhdev has quit IRC | 02:01 | |
| *** Sukhdev has joined #openstack-infra | 02:01 | |
| fungi | "soon" | 02:01 |
| fungi | on oracle linux | 02:01 |
| prometheanfire | lol | 02:01 |
| prometheanfire | I like my zfs :P | 02:02 |
| openstackgerrit | Sam Yaple proposed openstack/diskimage-builder: Use fstrim to prep the block device https://review.openstack.org/290944 | 02:02 |
| SamYaple | well considering the devs didnt announce it "stable for the core features" until about 18 months ago that article sounds hopeful | 02:02 |
| SamYaple | prometheanfire: yea i jump back and forth | 02:02 |
| SamYaple | clarkb: with the above patch we really dont need zerofree, but i left it in there for now. it just doesnt run if it detects fstrim | 02:03 |
| fungi | was 18 months ago about the point where it got a usable fsck? | 02:03 |
| SamYaple | who needs to fsck a CoW filesystem? | 02:03 |
| prometheanfire | zpool scrub | 02:04 |
| fungi | yeah, repairing errors? it's not like those ever happen | 02:04 |
| prometheanfire | and a friend had to use the btrfs recovery stuff once | 02:04 |
| SamYaple | i mean it has checksum errors | 02:04 |
| SamYaple | if thats what you mean by fsck its had that for a long time | 02:04 |
| SamYaple | you can read all the data and check for errors | 02:04 |
| fungi | oh yeah, i basically lost a very large btrfs some years back. it was nigh unrecoverable and there was no filesystem checker for it | 02:04 |
| prometheanfire | there's one now, but it's discouraged | 02:05 |
| SamYaple | i mean some years back it wasnt annouced as stable :P | 02:05 |
| *** fawadkhaliq has joined #openstack-infra | 02:05 | |
| SamYaple | prometheanfire: its not your standard fsck | 02:05 |
| SamYaple | prometheanfire: there is a btrfs scrub | 02:05 |
| clarkb | what version of the kernel is it considered stable in? | 02:05 |
| SamYaple | its not teh same thing | 02:05 |
| SamYaple | clarkb: 3.14 i believe | 02:05 |
| SamYaple | maybe a bit newer, but in the 3.x branch | 02:06 |
| prometheanfire | ya, scrubs aren't the same | 02:06 |
| fungi | right, there was no basic file recovery for btrfs the last time i tried to use it. luckily i was making frequent backups | 02:06 |
| prometheanfire | 'stable' | 02:06 |
| SamYaple | you just said zfs had that and that was the same thing | 02:06 |
| SamYaple | it has the same checks zfs has to keep it 'stable' | 02:06 |
| prometheanfire | didn't say it was the same thing | 02:06 |
| SamYaple | fungi: not likely to happen either i think. cow is nigh impossible to recover | 02:07 |
| SamYaple | prometheanfire: i guess you made it saound that way | 02:07 |
| SamYaple | 02:04:28 < SamYaple> who needs to fsck a CoW filesystem? | 02:07 |
| SamYaple | 02:04:55 < prometheanfire> zpool scrub | 02:07 |
| fungi | good point | 02:07 |
| prometheanfire | I know of some recoveries of zfs that have been done (mainly due to zil stuff dieing) | 02:08 |
| SamYaple | btrfs has scrubs and checksums, it can be used to validate integrity | 02:08 |
| fungi | essentially the block device started having some sporadic errors. btrfs more or less gave up being mountable. if it were ext3 or reiser or something, some of the files would have just become unreachable but i could still have at least copied most of the content off with it mounted read-only | 02:08 |
| prometheanfire | also, it helps that I've never lost data on zfs | 02:09 |
| SamYaple | prometheanfire: were you using it before it was announced as stable :P | 02:09 |
| prometheanfire | no | 02:09 |
| SamYaple | not really a direct comparision then | 02:09 |
| prometheanfire | were you using it before it was announced as stable | 02:09 |
| prometheanfire | no | 02:09 |
| fungi | i was definitely using btrfs before it was officially stable. this was probably 2009? hard to remember now | 02:09 |
| prometheanfire | I was not using it before it was stable | 02:09 |
| SamYaple | prometheanfire: btrfs? yes i was | 02:09 |
| prometheanfire | only after | 02:09 |
| SamYaple | lol yea that was a while back fungi | 02:10 |
| prometheanfire | so, comparison applicable :P | 02:10 |
| fungi | seems recent to me, but yeah. i guess it's been a while | 02:10 |
| *** yamamoto_ has joined #openstack-infra | 02:10 | |
| SamYaple | i dont think there has been a "all your data are gone now" since its been announced stable | 02:10 |
| SamYaple | but you know if you lost the wrong block on zfs you have to rekick the whole thing, btrfs not so much | 02:11 |
| prometheanfire | there's been a few bad ones though | 02:11 |
| fungi | i do keep meaning to try it again | 02:11 |
| SamYaple | before it was called stable, yes. same with zfs | 02:11 |
| prometheanfire | and after | 02:11 |
| prometheanfire | not the same with zfs | 02:11 |
| SamYaple | show me one? | 02:11 |
| * fungi hopes he | 02:12 | |
| prometheanfire | have to reclone | 02:12 |
| fungi | 's not going to need to make a plea for civility | 02:12 |
| SamYaple | lol fungi me and prometheanfire are good | 02:12 |
| fungi | it's almost like someone called a vi vs emacs throwdown in here | 02:12 |
| prometheanfire | so might be a few :P | 02:13 |
| SamYaple | we both like ZFS, i lean toward btrfs though | 02:13 |
| fungi | heh | 02:13 |
| SamYaple | fungi: so thats what were going to do today? fight? | 02:13 |
| prometheanfire | and I lean toward btrfs | 02:13 |
| SamYaple | i have it in writing! | 02:13 |
| prometheanfire | there are some things I wish zfs had that btrfs does | 02:13 |
| prometheanfire | oh, meh | 02:13 |
| prometheanfire | spelling | 02:13 |
| SamYaple | lol | 02:13 |
| SamYaple | same with zfs | 02:13 |
| SamYaple | i love creaintg zvols | 02:13 |
| prometheanfire | Receiving objects: 1% (79164/4583176), 35.07 MiB | 408.00 KiB/s | 02:13 |
| SamYaple | hate that btrfs will never have it | 02:14 |
| prometheanfire | If oracle doesn't finish their zfs cinder driver I'm going to do it myself | 02:14 |
| prometheanfire | they've been 'working on it' | 02:15 |
| SamYaple | i wrote a zfs driver for opennebula | 02:16 |
| SamYaple | tell me prometheanfire , what is "${VAR##*:}" doing? | 02:16 |
| SamYaple | ##*: is in question | 02:16 |
| prometheanfire | ## | 02:17 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/project-config: Add experimental job to test xstatic packaging https://review.openstack.org/289125 | 02:17 |
| prometheanfire | ## remove longest match from the front of the string | 02:17 |
| prometheanfire | *: remove anything up til the last : from the front of the string | 02:17 |
| Clint | and including the : | 02:17 |
| prometheanfire | bash substring manipulation is awesome | 02:17 |
| prometheanfire | Clint: yes | 02:17 |
| SamYaple | got it | 02:18 |
| SamYaple | grep some of my scripts turns out ive done this before | 02:18 |
| SamYaple | i cannot remember these things | 02:18 |
| prometheanfire | elements_path was /foo/bar/baz:/x/y/z:/What/I/want | 02:18 |
| *** thorst_ has quit IRC | 02:18 | |
| prometheanfire | ya, I have to google every time | 02:19 |
| prometheanfire | Receiving objects: 9% (448897/4583176), 181.92 MiB | 446.00 KiB/s | 02:19 |
| SamYaple | yea i would have gone |awk -F: '{print $(NF)}' | 02:19 |
| prometheanfire | still going lol | 02:19 |
| *** thorst_ has joined #openstack-infra | 02:19 | |
| SamYaple | but yours is bashier | 02:19 |
| prometheanfire | ya, didn't want to call out | 02:19 |
| SamYaple | yup | 02:19 |
| Clint | i remember # vs. % because the 3 is left of the 5 | 02:20 |
| fungi | except on a 10-key pad | 02:20 |
| prometheanfire | that removing from the end? | 02:20 |
| Clint | i haven't seen one of those in a while | 02:21 |
| fungi | so true | 02:21 |
| Clint | prometheanfire: # from the left, % from the right | 02:21 |
| Clint | (# is on the left of $, % is on the right of $) | 02:21 |
| Clint | on us layout anyway | 02:22 |
| *** Roamer` has quit IRC | 02:22 | |
| SamYaple | man bash | 02:23 |
| fungi | ever wanted to waste several hours of your life you'll never get back? man bash! | 02:24 |
| *** Sukhdev_ has joined #openstack-infra | 02:24 | |
| *** Sukhdev has quit IRC | 02:24 | |
| SamYaple | im very happy the new lts for ubuntu is about to land | 02:24 |
| SamYaple | umount -R is your friend | 02:25 |
| fungi | i've slowly grown a finger memory for forward-searching to specific sections of the bash manpage | 02:25 |
| prometheanfire | SamYaple: with zfs support :P | 02:25 |
| SamYaple | (its like a requirement with systemd!) | 02:25 |
| fungi | SamYaple: what's -R get you? | 02:25 |
| SamYaple | recursive unmount! | 02:25 |
| fungi | oh! | 02:25 |
| SamYaple | yes! | 02:25 |
| fungi | yeah, that would be handy | 02:25 |
| prometheanfire | oh, that is nice | 02:25 |
| fungi | especially with chroots | 02:25 |
| *** hdd_ has quit IRC | 02:26 | |
| SamYaple | i actually pull in the new umount binaries on my 14.04 system because i need that thing | 02:26 |
| SamYaple | i neeeeeed it | 02:26 |
| fungi | i always forget to umount sys or proc or something when cleaning up a chroot | 02:26 |
| SamYaple | /dev/pts | 02:26 |
| SamYaple | arg | 02:26 |
| *** roxanagh_ has quit IRC | 02:26 | |
| fungi | i still don't get why /dev/pts has to be a separate virtual fs from /dev | 02:27 |
| SamYaple | umount -R would get right of this crap https://github.com/SamYaple/encrypted-zfs-ubuntu-bootstrap/blob/master/cleanup.sh#L7 | 02:27 |
| *** IlyaG has joined #openstack-infra | 02:27 | |
| prometheanfire | /dev/shm | 02:27 |
| prometheanfire | that's my 'fun' | 02:27 |
| *** baoli has joined #openstack-infra | 02:27 | |
| fungi | that one makes a little more sense as it's just a ramdisk | 02:27 |
| prometheanfire | SamYaple: oh god | 02:27 |
| *** kevinsho_ has quit IRC | 02:28 | |
| SamYaple | prometheanfire: right! | 02:28 |
| prometheanfire | d34dh0r53: wtf you doing here? | 02:28 |
| *** thorst_ has quit IRC | 02:28 | |
| SamYaple | clearly he wanted to join the cool kids | 02:28 |
| *** Sukhdev_ has quit IRC | 02:30 | |
| *** Sukhdev has joined #openstack-infra | 02:31 | |
| *** IlyaG has quit IRC | 02:33 | |
| *** Sukhdev has quit IRC | 02:35 | |
| *** Sukhdev has joined #openstack-infra | 02:35 | |
| *** Roamer` has joined #openstack-infra | 02:36 | |
| *** EricGonczer_ has joined #openstack-infra | 02:37 | |
| *** _ody has quit IRC | 02:38 | |
| *** _ody has joined #openstack-infra | 02:38 | |
| *** ociuhandu has quit IRC | 02:38 | |
| *** Sukhdev has quit IRC | 02:38 | |
| *** ociuhandu has joined #openstack-infra | 02:39 | |
| *** Sukhdev has joined #openstack-infra | 02:39 | |
| *** baoli has quit IRC | 02:40 | |
| *** eharney has quit IRC | 02:40 | |
| *** Jeffrey4l has quit IRC | 02:41 | |
| *** Sukhdev has quit IRC | 02:42 | |
| *** Sukhdev has joined #openstack-infra | 02:42 | |
| *** EricGonc_ has joined #openstack-infra | 02:43 | |
| *** Jeffrey4l has joined #openstack-infra | 02:44 | |
| *** kzaitsev_mb has joined #openstack-infra | 02:45 | |
| *** Sukhdev has quit IRC | 02:45 | |
| *** Sukhdev has joined #openstack-infra | 02:46 | |
| *** EricGonczer_ has quit IRC | 02:46 | |
| *** Sukhdev has quit IRC | 02:48 | |
| *** Sukhdev has joined #openstack-infra | 02:49 | |
| *** yamamoto_ has quit IRC | 02:49 | |
| *** Sukhdev has quit IRC | 02:52 | |
| *** binbincong has quit IRC | 02:52 | |
| *** Sukhdev has joined #openstack-infra | 02:52 | |
| *** fawadkhaliq has quit IRC | 02:53 | |
| *** EricGonc_ has quit IRC | 02:53 | |
| *** tonytan4ever has quit IRC | 02:55 | |
| *** Sukhdev has quit IRC | 02:55 | |
| *** Sukhdev_ has joined #openstack-infra | 02:55 | |
| *** tqtran-afk has quit IRC | 02:56 | |
| *** SergeyLukjanov has quit IRC | 02:56 | |
| *** NikitaKonovalov has quit IRC | 02:56 | |
| *** kzaitsev_mb has quit IRC | 02:57 | |
| *** Sukhdev_ has quit IRC | 02:58 | |
| *** Sukhdev has joined #openstack-infra | 02:59 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: centos-minimal does not provide base https://review.openstack.org/290956 | 03:00 |
| *** HeOS has quit IRC | 03:00 | |
| *** jlk has quit IRC | 03:01 | |
| *** dansmith has quit IRC | 03:01 | |
| *** darvon has quit IRC | 03:01 | |
| *** dansmith has joined #openstack-infra | 03:01 | |
| *** jlk has joined #openstack-infra | 03:01 | |
| *** jlk has quit IRC | 03:01 | |
| *** jlk has joined #openstack-infra | 03:01 | |
| *** Sukhdev has quit IRC | 03:01 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/shade: Add get_stack_outputs method https://review.openstack.org/276487 | 03:02 |
| *** Sukhdev has joined #openstack-infra | 03:02 | |
| *** darvon has joined #openstack-infra | 03:02 | |
| *** HeOS has joined #openstack-infra | 03:02 | |
| *** SergeyLukjanov has joined #openstack-infra | 03:02 | |
| *** gildub has quit IRC | 03:03 | |
| *** NikitaKonovalov has joined #openstack-infra | 03:03 | |
| *** Sukhdev has quit IRC | 03:05 | |
| *** Sukhdev_ has joined #openstack-infra | 03:05 | |
| *** Sukhdev_ has quit IRC | 03:08 | |
| *** Sukhdev has joined #openstack-infra | 03:08 | |
| *** ajmiller has quit IRC | 03:10 | |
| *** thorst_ has joined #openstack-infra | 03:10 | |
| *** hdd has joined #openstack-infra | 03:11 | |
| *** Sukhdev_ has joined #openstack-infra | 03:11 | |
| *** Sukhdev has quit IRC | 03:11 | |
| *** tphummel has quit IRC | 03:14 | |
| *** Sukhdev_ has quit IRC | 03:14 | |
| *** thorst_ has quit IRC | 03:14 | |
| *** dims has quit IRC | 03:14 | |
| *** Sukhdev has joined #openstack-infra | 03:14 | |
| *** baoli has joined #openstack-infra | 03:14 | |
| *** thorst_ has joined #openstack-infra | 03:15 | |
| *** grassass has joined #openstack-infra | 03:15 | |
| *** Sukhdev has quit IRC | 03:15 | |
| *** zz_dimtruck is now known as dimtruck | 03:17 | |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Exclude python-troveclient 2.1.0 https://review.openstack.org/290935 | 03:17 |
| *** mriedem has quit IRC | 03:17 | |
| *** apoorvad has quit IRC | 03:18 | |
| openstackgerrit | Merged openstack/requirements: kilo: pin pysaml2<4.0.4 https://review.openstack.org/290613 | 03:19 |
| *** SergeyLukjanov has quit IRC | 03:19 | |
| *** thorst_ has quit IRC | 03:23 | |
| *** bapalm has quit IRC | 03:24 | |
| *** baoli has quit IRC | 03:24 | |
| *** fawadkhaliq has joined #openstack-infra | 03:24 | |
| *** SergeyLukjanov has joined #openstack-infra | 03:24 | |
| *** asettle has joined #openstack-infra | 03:26 | |
| *** bpokorny has quit IRC | 03:27 | |
| *** bpokorny has joined #openstack-infra | 03:27 | |
| *** markvoelker has joined #openstack-infra | 03:29 | |
| *** hdd has quit IRC | 03:32 | |
| *** bpokorny has quit IRC | 03:32 | |
| *** asettle has quit IRC | 03:33 | |
| *** fawadkhaliq has quit IRC | 03:34 | |
| *** yamamoto_ has joined #openstack-infra | 03:36 | |
| *** gildub has joined #openstack-infra | 03:37 | |
| *** bapalm has joined #openstack-infra | 03:43 | |
| *** markvoelker has quit IRC | 03:43 | |
| *** rlandy has quit IRC | 03:45 | |
| *** yamahata has quit IRC | 03:49 | |
| *** links has joined #openstack-infra | 03:50 | |
| *** kzaitsev_mb has joined #openstack-infra | 03:53 | |
| *** kencjohnston has joined #openstack-infra | 03:55 | |
| *** woodster_ has quit IRC | 03:57 | |
| *** kencjohnston has quit IRC | 03:57 | |
| *** kzaitsev_mb has quit IRC | 03:58 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Clear up "already provided" message https://review.openstack.org/290968 | 03:59 |
| *** dims has joined #openstack-infra | 04:01 | |
| *** jaosorior has quit IRC | 04:02 | |
| *** jaosorior has joined #openstack-infra | 04:03 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/shade: Add get_stack_outputs method https://review.openstack.org/276487 | 04:05 |
| *** gongysh has quit IRC | 04:11 | |
| *** zeih has joined #openstack-infra | 04:12 | |
| *** dims has quit IRC | 04:13 | |
| *** nikhil has joined #openstack-infra | 04:15 | |
| *** nikhil_k has joined #openstack-infra | 04:15 | |
| *** zeih has quit IRC | 04:17 | |
| *** tphummel has joined #openstack-infra | 04:20 | |
| *** thorst_ has joined #openstack-infra | 04:21 | |
| *** takashi has quit IRC | 04:27 | |
| *** takashi has joined #openstack-infra | 04:27 | |
| *** thorst_ has quit IRC | 04:28 | |
| *** hichihara has joined #openstack-infra | 04:28 | |
| *** kevinsho_ has joined #openstack-infra | 04:29 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Add centos-minimal build, experimental job https://review.openstack.org/290974 | 04:31 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Remove Fedora23 install-glean-from-git flag https://review.openstack.org/290975 | 04:33 |
| *** ajmiller has joined #openstack-infra | 04:33 | |
| *** kevinsho_ has quit IRC | 04:33 | |
| *** IlyaG has joined #openstack-infra | 04:37 | |
| *** treaki__ has joined #openstack-infra | 04:37 | |
| *** amotoki has joined #openstack-infra | 04:38 | |
| *** harlowja_at_home has quit IRC | 04:39 | |
| *** harlowja_at_home has joined #openstack-infra | 04:39 | |
| *** zeih has joined #openstack-infra | 04:39 | |
| *** treaki_ has quit IRC | 04:40 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Add centos-minimal build, experimental job https://review.openstack.org/290974 | 04:41 |
| *** jamielennox is now known as jamielennox|away | 04:41 | |
| *** zeih has quit IRC | 04:44 | |
| openstackgerrit | Reinhard Tartler proposed openstack-infra/jenkins-job-builder: Add support for disable-strict-forbidden-file-verification https://review.openstack.org/240360 | 04:44 |
| *** HeOS has quit IRC | 04:45 | |
| *** HeOS has joined #openstack-infra | 04:45 | |
| ianw | while c++ still wins for "one character wrong ridiculous length of backtrace", jjb is getting up there -> http://logs.openstack.org/74/290974/1/check/gate-project-config-layout/28939af/console.html#_2016-03-10_04_33_56_319 | 04:50 |
| *** DinaBelova has quit IRC | 04:51 | |
| *** kragniz has quit IRC | 04:52 | |
| *** DinaBelova has joined #openstack-infra | 04:53 | |
| *** kragniz has joined #openstack-infra | 04:54 | |
| *** heyongli has joined #openstack-infra | 04:57 | |
| *** ajmiller has quit IRC | 05:00 | |
| *** heyongli has quit IRC | 05:00 | |
| *** heyongli has joined #openstack-infra | 05:00 | |
| *** HeOS has quit IRC | 05:01 | |
| *** IlyaG has quit IRC | 05:02 | |
| *** HeOS has joined #openstack-infra | 05:03 | |
| *** kragniz has quit IRC | 05:05 | |
| *** kragniz has joined #openstack-infra | 05:05 | |
| *** heyongli has quit IRC | 05:10 | |
| *** heyongli has joined #openstack-infra | 05:10 | |
| *** nwkarsten has joined #openstack-infra | 05:13 | |
| *** Jeffrey4l has quit IRC | 05:15 | |
| *** nwkarsten has quit IRC | 05:17 | |
| *** fedexo has joined #openstack-infra | 05:19 | |
| *** beisner has quit IRC | 05:19 | |
| *** sheeprine_ has quit IRC | 05:19 | |
| *** sheeprine has joined #openstack-infra | 05:20 | |
| *** heyongli has quit IRC | 05:20 | |
| *** heyongli has joined #openstack-infra | 05:20 | |
| *** beisner has joined #openstack-infra | 05:21 | |
| *** dimtruck is now known as zz_dimtruck | 05:25 | |
| *** zz_dimtruck is now known as dimtruck | 05:25 | |
| *** thorst_ has joined #openstack-infra | 05:26 | |
| *** hdd has joined #openstack-infra | 05:29 | |
| *** heyongli has quit IRC | 05:30 | |
| *** hieulq has joined #openstack-infra | 05:31 | |
| *** flepied has quit IRC | 05:31 | |
| *** hieulq has quit IRC | 05:31 | |
| *** thorst_ has quit IRC | 05:33 | |
| *** baoli has joined #openstack-infra | 05:36 | |
| *** harlowja_at_home has quit IRC | 05:36 | |
| *** infra-red has joined #openstack-infra | 05:38 | |
| *** sheeprine has quit IRC | 05:40 | |
| *** jaosorior has quit IRC | 05:40 | |
| *** dansmith has quit IRC | 05:40 | |
| *** yonglihe has quit IRC | 05:40 | |
| *** evgenyl has quit IRC | 05:40 | |
| *** mwhahaha has quit IRC | 05:40 | |
| *** Volundr has quit IRC | 05:40 | |
| *** mescanef has quit IRC | 05:40 | |
| *** dingyichen has quit IRC | 05:40 | |
| *** cfarquhar has quit IRC | 05:40 | |
| *** baoli has quit IRC | 05:40 | |
| *** flepied has joined #openstack-infra | 05:43 | |
| *** asettle has joined #openstack-infra | 05:45 | |
| *** binbincong has joined #openstack-infra | 05:45 | |
| *** sheeprine has joined #openstack-infra | 05:46 | |
| *** dansmith has joined #openstack-infra | 05:46 | |
| *** yonglihe has joined #openstack-infra | 05:46 | |
| *** evgenyl has joined #openstack-infra | 05:46 | |
| *** mwhahaha has joined #openstack-infra | 05:46 | |
| *** Volundr has joined #openstack-infra | 05:46 | |
| *** mescanef has joined #openstack-infra | 05:46 | |
| *** dingyichen has joined #openstack-infra | 05:46 | |
| *** cfarquhar has joined #openstack-infra | 05:46 | |
| *** jaosorior has joined #openstack-infra | 05:46 | |
| *** jaosorior has quit IRC | 05:46 | |
| *** kushal has quit IRC | 05:47 | |
| *** nwkarsten has joined #openstack-infra | 05:49 | |
| *** asettle has quit IRC | 05:49 | |
| *** armax has joined #openstack-infra | 05:50 | |
| *** jtomasek has joined #openstack-infra | 05:53 | |
| *** ayoung has quit IRC | 05:53 | |
| *** nwkarsten has quit IRC | 05:54 | |
| *** melwitt has quit IRC | 05:54 | |
| *** kukacz_ has quit IRC | 05:55 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:55 | |
| *** med_ has quit IRC | 05:56 | |
| *** mfisch has quit IRC | 05:56 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: WIP: Remove bare-trusty from projects.yaml https://review.openstack.org/290809 | 05:56 |
| *** kukacz has joined #openstack-infra | 05:56 | |
| *** med_ has joined #openstack-infra | 05:56 | |
| *** med_ is now known as Guest93664 | 05:56 | |
| *** melwitt has joined #openstack-infra | 05:59 | |
| *** kzaitsev_mb has quit IRC | 06:00 | |
| *** sheeprine has quit IRC | 06:02 | |
| *** dansmith has quit IRC | 06:02 | |
| *** yonglihe has quit IRC | 06:02 | |
| *** evgenyl has quit IRC | 06:02 | |
| *** mwhahaha has quit IRC | 06:02 | |
| *** Volundr has quit IRC | 06:02 | |
| *** mescanef has quit IRC | 06:02 | |
| *** dingyichen has quit IRC | 06:02 | |
| *** cfarquhar has quit IRC | 06:02 | |
| *** dimtruck is now known as zz_dimtruck | 06:03 | |
| *** ayoung has joined #openstack-infra | 06:04 | |
| *** infra-red has quit IRC | 06:06 | |
| *** fawadkhaliq has joined #openstack-infra | 06:07 | |
| *** sheeprine has joined #openstack-infra | 06:08 | |
| *** dansmith has joined #openstack-infra | 06:08 | |
| *** yonglihe has joined #openstack-infra | 06:08 | |
| *** evgenyl has joined #openstack-infra | 06:08 | |
| *** mwhahaha has joined #openstack-infra | 06:08 | |
| *** Volundr has joined #openstack-infra | 06:08 | |
| *** mescanef has joined #openstack-infra | 06:08 | |
| *** dingyichen has joined #openstack-infra | 06:08 | |
| *** cfarquhar has joined #openstack-infra | 06:08 | |
| *** binbincong has quit IRC | 06:10 | |
| *** abregman has joined #openstack-infra | 06:14 | |
| *** hdd has quit IRC | 06:15 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/291008 | 06:17 |
| *** bapalm has quit IRC | 06:23 | |
| *** Guest38484 has joined #openstack-infra | 06:27 | |
| *** binbincong has joined #openstack-infra | 06:29 | |
| *** fawadkhaliq has quit IRC | 06:30 | |
| *** thorst_ has joined #openstack-infra | 06:31 | |
| *** kushal has joined #openstack-infra | 06:31 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: gerrit-plugin-jobs: Use ubuntu-trusty https://review.openstack.org/290826 | 06:33 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/288849 | 06:33 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Change jobs to use ubuntu-trusty https://review.openstack.org/291022 | 06:35 |
| *** thorst_ has quit IRC | 06:38 | |
| openstackgerrit | Purandhar Sairam Mannidi proposed openstack/diskimage-builder: Add support for building images capable of UEFI https://review.openstack.org/287784 | 06:39 |
| *** bapalm has joined #openstack-infra | 06:42 | |
| *** melwitt has quit IRC | 06:45 | |
| AJaeger | fungi, pabelanger, clarkb: Thanks for reviews! We're down to 6 bare-trusty jobs | 06:45 |
| AJaeger | 5 of them are changed with https://review.openstack.org/290826 and https://review.openstack.org/291022 | 06:46 |
| AJaeger | fungi, clarkb, pabelanger: The final one needs discussion: gate-infra-puppet-apply-bare-trusty | 06:46 |
| AJaeger | Do we have servers running bare-trusty that we need this for? Do we need to setup a job for ubuntu-trusty? | 06:47 |
| AJaeger | more later today - just wanted to dump it here while I investigated | 06:47 |
| *** sheeprine has quit IRC | 06:49 | |
| *** sheeprine has joined #openstack-infra | 06:49 | |
| *** melwitt has joined #openstack-infra | 06:50 | |
| *** melwitt is now known as Guest98534 | 06:50 | |
| *** gongysh has joined #openstack-infra | 06:51 | |
| *** jtomasek has quit IRC | 06:53 | |
| openstackgerrit | Purandhar Sairam Mannidi proposed openstack/diskimage-builder: Add support for building images capable of UEFI https://review.openstack.org/287784 | 06:54 |
| *** Guest98534 is now known as melwitt | 06:55 | |
| *** fedexo has quit IRC | 06:58 | |
| *** melwitt has quit IRC | 06:58 | |
| fungi | AJaeger: no, the server puppet manifests just need to be tested on a representative of the distro/release, so the testing we do on ubuntu-trusty now validates the puppetry for our servers which run on trusty | 06:59 |
| *** melwitt has joined #openstack-infra | 07:00 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/291008 | 07:00 |
| *** kushal has quit IRC | 07:03 | |
| *** melwitt has quit IRC | 07:03 | |
| *** melwitt has joined #openstack-infra | 07:04 | |
| *** melwitt is now known as Guest37020 | 07:05 | |
| *** skraynev has quit IRC | 07:05 | |
| *** skraynev has joined #openstack-infra | 07:06 | |
| *** yamahata has joined #openstack-infra | 07:07 | |
| lennyb | anteaya, mriedem: Hi, I will check the issue with the email. I've changed it to my personal email until the issue is solved. | 07:07 |
| *** sarob has joined #openstack-infra | 07:11 | |
| *** esikachev has joined #openstack-infra | 07:14 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder: Set default locale to image in ubuntu-minimal https://review.openstack.org/290789 | 07:15 |
| *** sarob has quit IRC | 07:16 | |
| *** Kennan has quit IRC | 07:18 | |
| *** zeih has joined #openstack-infra | 07:22 | |
| *** sc68cal has quit IRC | 07:22 | |
| *** ildikov has quit IRC | 07:22 | |
| *** Kennan has joined #openstack-infra | 07:23 | |
| *** Jeffrey4l has joined #openstack-infra | 07:26 | |
| *** zeih has quit IRC | 07:27 | |
| *** roxanagh_ has joined #openstack-infra | 07:27 | |
| *** roxanagh_ has quit IRC | 07:28 | |
| *** andreykurilin__ has joined #openstack-infra | 07:29 | |
| *** melwitt_ has joined #openstack-infra | 07:30 | |
| *** melwitt_ has quit IRC | 07:32 | |
| *** scheuran has joined #openstack-infra | 07:33 | |
| *** rcernin has joined #openstack-infra | 07:33 | |
| *** ildikov has joined #openstack-infra | 07:34 | |
| *** thorst_ has joined #openstack-infra | 07:36 | |
| *** esikachev has quit IRC | 07:38 | |
| *** melwitt_ has joined #openstack-infra | 07:39 | |
| *** melwitt_ is now known as melwitt | 07:40 | |
| *** melwitt has quit IRC | 07:40 | |
| *** melwitt has joined #openstack-infra | 07:41 | |
| *** fhubik has joined #openstack-infra | 07:42 | |
| *** thorst_ has quit IRC | 07:43 | |
| *** esikachev has joined #openstack-infra | 07:43 | |
| openstackgerrit | Merged openstack-infra/puppet-storyboard: Update SSLProtocol and SSLCipherSuite https://review.openstack.org/286612 | 07:44 |
| *** nikhil has quit IRC | 07:45 | |
| *** fhubik has quit IRC | 07:46 | |
| *** nikhil has joined #openstack-infra | 07:47 | |
| yolanda | uh AJaeger, i'm having that error on a diskimage-builder test | 07:47 |
| yolanda | 2016-03-10 07:45:27.466 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/services/client.py", line 18, in <module> | 07:47 |
| yolanda | 2016-03-10 07:45:27.466 | from tempest.services.network.json import base | 07:47 |
| yolanda | 2016-03-10 07:45:27.467 | ImportError: cannot import name base | 07:47 |
| yolanda | do you know about it? | 07:47 |
| yolanda | i'll as in qa as well | 07:48 |
| *** baoli has joined #openstack-infra | 07:48 | |
| dmellado | mmm yolanda, could this be that tox it's not using the system packages? | 07:50 |
| *** baoli has quit IRC | 07:53 | |
| *** dizquierdo has joined #openstack-infra | 07:53 | |
| *** e0ne has joined #openstack-infra | 07:53 | |
| *** mrmartin has joined #openstack-infra | 07:54 | |
| *** jlanoux has joined #openstack-infra | 07:55 | |
| AJaeger | yolanda: no idea on that one. Did we switch jobs on that test to ubuntu-trusty? IT shouldn't change tempest python package installation... | 07:56 |
| *** nmagnezi has joined #openstack-infra | 07:57 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:57 | |
| *** _nadya_ has joined #openstack-infra | 08:00 | |
| *** cyraxjoe has joined #openstack-infra | 08:00 | |
| *** ifarkas has joined #openstack-infra | 08:02 | |
| *** kzaitsev_mb has quit IRC | 08:02 | |
| esikachev | guys, who can help me with ci? | 08:05 |
| esikachev | trigger by time not triggered | 08:05 |
| yolanda | AJaeger dmellado, so question answered in qa, there are patches in review for this | 08:06 |
| yolanda | https://review.openstack.org/#/c/291040/1 and https://review.openstack.org/290983 | 08:07 |
| *** ihrachys has joined #openstack-infra | 08:07 | |
| *** ihrachys has quit IRC | 08:08 | |
| openstackgerrit | Alexey Ovchinnikov proposed openstack-infra/project-config: [WIP] Add publishing capability to buildimage-lxd job https://review.openstack.org/290418 | 08:08 |
| yolanda | dmellado , you seem to be more aware on these tempest tests than me | 08:09 |
| *** daemontool_ has joined #openstack-infra | 08:11 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:11 | |
| *** ihrachys has joined #openstack-infra | 08:13 | |
| *** daemontool has quit IRC | 08:14 | |
| *** fhubik has joined #openstack-infra | 08:16 | |
| *** armax has quit IRC | 08:17 | |
| *** flepied has quit IRC | 08:17 | |
| *** zeih_ has joined #openstack-infra | 08:20 | |
| *** [HeOS] has joined #openstack-infra | 08:20 | |
| *** kushal has joined #openstack-infra | 08:21 | |
| *** HeOS has quit IRC | 08:21 | |
| *** e0ne has quit IRC | 08:21 | |
| *** kushal has joined #openstack-infra | 08:25 | |
| *** e0ne has joined #openstack-infra | 08:25 | |
| *** matrohon has joined #openstack-infra | 08:26 | |
| *** zeih_ has quit IRC | 08:27 | |
| *** lezbar has joined #openstack-infra | 08:27 | |
| *** arxcruz has joined #openstack-infra | 08:28 | |
| *** [1]Thelo has joined #openstack-infra | 08:30 | |
| *** kevinsho_ has joined #openstack-infra | 08:31 | |
| dmellado | yolanda: ? | 08:31 |
| *** zeih_ has joined #openstack-infra | 08:31 | |
| yolanda | dmellado, i'm talking about the tempest failures, i was seeing your comments in openstack_qa | 08:32 |
| *** mikelk has joined #openstack-infra | 08:32 | |
| *** dguitarbite has quit IRC | 08:32 | |
| *** davideagnello has quit IRC | 08:32 | |
| *** Thelo has quit IRC | 08:33 | |
| *** [1]Thelo is now known as Thelo | 08:33 | |
| dmellado | yolanda: what usually works for the tempest plugin is that they add an entry point for testr | 08:34 |
| *** pcaruana has joined #openstack-infra | 08:34 | |
| *** davideagnello has joined #openstack-infra | 08:35 | |
| dmellado | so depending on the repo and implementation there's usually a tox env that exposes the global packages so it could get to tempest too | 08:35 |
| dmellado | anyway this seemed related to the patch ;) | 08:35 |
| *** kevinsho_ has quit IRC | 08:36 | |
| *** andreykurilin__ has quit IRC | 08:36 | |
| *** betherly has quit IRC | 08:36 | |
| *** fkautz has quit IRC | 08:36 | |
| *** fdegir has quit IRC | 08:36 | |
| *** simonmcc has quit IRC | 08:36 | |
| *** etoews has quit IRC | 08:36 | |
| *** sylwesterB has quit IRC | 08:36 | |
| *** hrybacki has quit IRC | 08:36 | |
| *** ashtokolov has quit IRC | 08:36 | |
| *** zhiyan has quit IRC | 08:36 | |
| *** edwarnicke has quit IRC | 08:36 | |
| *** Ng has quit IRC | 08:36 | |
| *** GheRivero has quit IRC | 08:36 | |
| *** zhenguo_ has quit IRC | 08:36 | |
| *** fmccrthy has quit IRC | 08:36 | |
| *** teran has quit IRC | 08:36 | |
| *** skath_ has quit IRC | 08:36 | |
| *** kgalanov has quit IRC | 08:36 | |
| *** ams_ has quit IRC | 08:36 | |
| *** igorbelikov has quit IRC | 08:36 | |
| *** zzzeek has quit IRC | 08:36 | |
| *** bryan_att has quit IRC | 08:36 | |
| *** lindsayk_ has quit IRC | 08:36 | |
| *** dham1 has quit IRC | 08:36 | |
| *** samyewell has quit IRC | 08:36 | |
| *** ameade has quit IRC | 08:36 | |
| *** scottda has quit IRC | 08:36 | |
| *** jpmaxman has quit IRC | 08:36 | |
| *** zxiiro has quit IRC | 08:36 | |
| *** esikachev has quit IRC | 08:37 | |
| *** zzzeek has joined #openstack-infra | 08:38 | |
| yolanda | dmellado ah, i understand. For my particular failure i think that this patch https://review.openstack.org/#/c/290983/3/neutron_fwaas/tests/tempest_plugin/services/client.py will solve my issue | 08:39 |
| *** ashtokolov has joined #openstack-infra | 08:39 | |
| *** skath_ has joined #openstack-infra | 08:39 | |
| *** zhiyan has joined #openstack-infra | 08:39 | |
| *** scottda has joined #openstack-infra | 08:39 | |
| *** ameade has joined #openstack-infra | 08:39 | |
| *** simonmcc has joined #openstack-infra | 08:40 | |
| *** etoews has joined #openstack-infra | 08:40 | |
| *** betherly has joined #openstack-infra | 08:40 | |
| *** fdegir has joined #openstack-infra | 08:40 | |
| *** lindsayk_ has joined #openstack-infra | 08:40 | |
| *** thorst_ has joined #openstack-infra | 08:41 | |
| dmellado | yolanda: if it doesn't work for you let me know and we'll have a look | 08:41 |
| yolanda | thanks | 08:42 |
| *** kgalanov has joined #openstack-infra | 08:42 | |
| *** teran has joined #openstack-infra | 08:42 | |
| dmellado | np ;) | 08:42 |
| *** GheRivero has joined #openstack-infra | 08:42 | |
| *** fmccrthy has joined #openstack-infra | 08:42 | |
| *** igorbelikov has joined #openstack-infra | 08:42 | |
| *** dham1 has joined #openstack-infra | 08:42 | |
| *** zxiiro has joined #openstack-infra | 08:42 | |
| *** e0ne has quit IRC | 08:42 | |
| *** jpmaxman has joined #openstack-infra | 08:43 | |
| openstackgerrit | Merged openstack-infra/system-config: Add a playbook and script for doing adhoc puppet https://review.openstack.org/284352 | 08:43 |
| *** zhenguo_ has joined #openstack-infra | 08:44 | |
| *** Ng has joined #openstack-infra | 08:44 | |
| *** sylwesterB has joined #openstack-infra | 08:44 | |
| *** hrybacki has joined #openstack-infra | 08:44 | |
| *** andreykurilin__ has joined #openstack-infra | 08:44 | |
| *** bryan_att has joined #openstack-infra | 08:44 | |
| *** fkautz has joined #openstack-infra | 08:44 | |
| *** samyewell has joined #openstack-infra | 08:44 | |
| *** edwarnicke has joined #openstack-infra | 08:44 | |
| *** kzaitsev_mb has quit IRC | 08:45 | |
| *** ams_ has joined #openstack-infra | 08:45 | |
| *** dingyichen has quit IRC | 08:45 | |
| *** vincentll has joined #openstack-infra | 08:46 | |
| *** thorst_ has quit IRC | 08:48 | |
| *** zeih_ has quit IRC | 08:48 | |
| *** gildub has quit IRC | 08:48 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Use set-hostnames playbook in launch-node https://review.openstack.org/290777 | 08:49 |
| openstackgerrit | Merged openstack-infra/system-config: Make the shade cache and groups writable by admin https://review.openstack.org/290728 | 08:50 |
| *** Jeffrey4l has quit IRC | 08:50 | |
| openstackgerrit | Merged openstack-infra/system-config: Update for ansible and puppet apply https://review.openstack.org/247099 | 08:51 |
| *** Jeffrey4l has joined #openstack-infra | 08:51 | |
| *** jaosorior has joined #openstack-infra | 08:52 | |
| *** salv-orlando has joined #openstack-infra | 08:55 | |
| *** fhubik is now known as fhubik_brb | 08:55 | |
| *** flepied has joined #openstack-infra | 08:56 | |
| openstackgerrit | Merged openstack-infra/project-config: Added IoTronic project. https://review.openstack.org/286113 | 08:56 |
| *** jordanP has joined #openstack-infra | 08:56 | |
| *** esikachev has joined #openstack-infra | 08:56 | |
| openstackgerrit | Merged openstack-infra/project-config: use the correct branch when proposing a new config file https://review.openstack.org/290316 | 08:56 |
| stevemar | yolanda: *poke* | 08:57 |
| stevemar | yolanda: can you also please review https://review.openstack.org/#/c/290351/ :) | 08:57 |
| *** fhubik_brb is now known as fhubik | 08:58 | |
| yolanda | hi stevemar taking a look, i'm going through the review queue now | 08:58 |
| stevemar | yolanda: thank you! :] | 08:58 |
| rcarrillocruz | nice | 08:59 |
| rcarrillocruz | yolanda: http://logs.openstack.org/77/290777/4/check/gate-system-config-puppet-syntax/454f511/console.html | 08:59 |
| rcarrillocruz | looks like the set-hostnames change ran in a jenkins without the job fixed | 09:00 |
| rcarrillocruz | the locale thing | 09:00 |
| yolanda | older node? ok let's recheck | 09:00 |
| rcarrillocruz | i'm rechecking, but the fact the change merged yesterday and some jenkins not having it is worrying | 09:00 |
| yolanda | mm, and that fix was done on the job directly right? | 09:00 |
| yolanda | maybe jjb problems?= | 09:00 |
| rcarrillocruz | y | 09:01 |
| *** hashar has joined #openstack-infra | 09:01 | |
| yolanda | going to check jjb runs | 09:01 |
| *** zz_dimtruck is now known as dimtruck | 09:03 | |
| *** shardy has joined #openstack-infra | 09:05 | |
| *** Jeffrey4l has quit IRC | 09:05 | |
| openstackgerrit | Merged openstack-infra/project-config: Add a non-voting iPXE job to ironic-inspector https://review.openstack.org/290016 | 09:05 |
| yolanda | rcarrillocruz, i don't see failures in jjb in logs. But going to check job config history in them | 09:06 |
| *** _nadya_ has quit IRC | 09:06 | |
| openstackgerrit | Merged openstack-infra/project-config: don't fail if commands overlap https://review.openstack.org/290351 | 09:08 |
| *** jaosorior has quit IRC | 09:09 | |
| *** jaosorior has joined #openstack-infra | 09:09 | |
| *** jistr has joined #openstack-infra | 09:09 | |
| yolanda | rcarrillocruz, just started looking and jenkins02 has not picked the job updates | 09:10 |
| rcarrillocruz | :/, recheck didn't seem to go thru, i'll retry | 09:10 |
| rcarrillocruz | thx | 09:10 |
| AJaeger | So, manual JJB run on jenkins02 again? ;( Any other jenkins needed? | 09:11 |
| *** dtantsur|afk is now known as dtantsur | 09:11 | |
| yolanda | AJaeger, going to check all | 09:12 |
| yolanda | manual updates yes | 09:12 |
| *** dimtruck is now known as zz_dimtruck | 09:12 | |
| *** jcoufal has joined #openstack-infra | 09:12 | |
| yolanda | jenkins02 is failing with jjb quite often... | 09:13 |
| *** rossella_s has joined #openstack-infra | 09:13 | |
| AJaeger | ;( | 09:14 |
| AJaeger | thanks, yolanda ! | 09:14 |
| *** lucas-dinner is now known as lucasagomes | 09:14 | |
| rcarrillocruz | \o/ now green... | 09:14 |
| *** _skolekonov is now known as skolekonov | 09:14 | |
| yolanda | cool | 09:14 |
| *** amotoki has quit IRC | 09:15 | |
| yolanda | all the other jenkins are fine | 09:15 |
| AJaeger | \o/ | 09:17 |
| *** salv-orlando has quit IRC | 09:20 | |
| yolanda | jjb doing its job on jenkins02, looks as it hadn't run for a long time, because it's reconfiguring a bunch of gate jobs | 09:26 |
| *** dtardivel has joined #openstack-infra | 09:29 | |
| *** openstackgerrit has quit IRC | 09:30 | |
| *** openstackgerrit_ has joined #openstack-infra | 09:30 | |
| *** openstackgerrit_ is now known as openstackgerrit | 09:31 | |
| *** openstackgerrit has quit IRC | 09:31 | |
| *** openstackgerrit_ has joined #openstack-infra | 09:31 | |
| *** pblaho has joined #openstack-infra | 09:31 | |
| *** openstackgerrit_ is now known as openstackgerrit | 09:32 | |
| *** openstackgerrit has quit IRC | 09:32 | |
| AJaeger | ;( | 09:32 |
| *** openstackgerrit_ has joined #openstack-infra | 09:32 | |
| *** flepied has quit IRC | 09:33 | |
| *** openstackgerrit_ is now known as openstackgerrit | 09:33 | |
| *** openstackgerrit has quit IRC | 09:33 | |
| *** e0ne has joined #openstack-infra | 09:33 | |
| *** openstackgerrit_ has joined #openstack-infra | 09:34 | |
| *** sfinucan has joined #openstack-infra | 09:34 | |
| *** openstackgerrit_ is now known as openstackgerrit | 09:34 | |
| *** abregman has quit IRC | 09:35 | |
| *** sarob has joined #openstack-infra | 09:38 | |
| openstackgerrit | Merged openstack-infra/system-config: Add pabelanger to infra root https://review.openstack.org/290104 | 09:39 |
| yolanda | finally! this test has failed a lot of times ^ | 09:40 |
| openstackgerrit | Merged openstack-infra/system-config: Remove pabelanger from asterisk and stackalytics https://review.openstack.org/290143 | 09:40 |
| *** christian_ has joined #openstack-infra | 09:41 | |
| *** electrofelix has joined #openstack-infra | 09:41 | |
| *** _degorenko|afk is now known as degorenko | 09:42 | |
| *** sarob has quit IRC | 09:42 | |
| *** dguitarbite has joined #openstack-infra | 09:44 | |
| *** dguitarbite has quit IRC | 09:44 | |
| *** pkarikh has left #openstack-infra | 09:44 | |
| *** thorst_ has joined #openstack-infra | 09:45 | |
| *** daemontool__ has joined #openstack-infra | 09:46 | |
| *** abregman has joined #openstack-infra | 09:47 | |
| *** daemontool_ has quit IRC | 09:49 | |
| *** amotoki has joined #openstack-infra | 09:50 | |
| *** acabot has joined #openstack-infra | 09:51 | |
| *** flepied has joined #openstack-infra | 09:51 | |
| *** kushal has quit IRC | 09:52 | |
| *** thorst_ has quit IRC | 09:53 | |
| *** placko_ has joined #openstack-infra | 09:56 | |
| *** esikachev has quit IRC | 10:00 | |
| *** markusz is now known as markus_z | 10:00 | |
| *** tpsilva has joined #openstack-infra | 10:00 | |
| *** [HeOS] has quit IRC | 10:02 | |
| *** HeOS has joined #openstack-infra | 10:03 | |
| *** Guest3102 is now known as BobBall | 10:04 | |
| *** esikachev has joined #openstack-infra | 10:04 | |
| *** hichihara has quit IRC | 10:10 | |
| hashar | maar roots ! | 10:10 |
| *** sambetts|afk is now known as sambetts | 10:11 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:11 | |
| *** derekh has joined #openstack-infra | 10:13 | |
| gmann | yolanda: AJaeger: regarding tempest plugin failure | 10:13 |
| gmann | i have opened a bug - https://bugs.launchpad.net/tempest/+bug/1555524 | 10:13 |
| openstack | Launchpad bug 1555524 in tempest "Any tempest plugin import failure leads other plugin tests fails" [Undecided,New] - Assigned to Ghanshyam Mann (ghanshyammann) | 10:13 |
| gmann | let's see we can find goos way to avoid such failure in future | 10:13 |
| gmann | good | 10:14 |
| yolanda | gmann that makes sense yes | 10:15 |
| yolanda | we shall only care about the plugins we are testing | 10:15 |
| gmann | yolanda: basically, other project error should not leads to any project tests failure | 10:15 |
| gmann | yolanda: exactly. | 10:15 |
| gmann | yolanda: we take care while loading, running or registering config etc | 10:16 |
| *** jtomasek has joined #openstack-infra | 10:16 | |
| gmann | but this is special case where testr needs to list all tests and run requested one | 10:16 |
| gmann | and it is import error so comes while listing the tests | 10:17 |
| yolanda | i understand, do you see an easy way to avoid it? enclosing imports in some try block? | 10:17 |
| gmann | yolanda: no, i think we should make it on tempest side, so while discovery if there is any issue in any plugin just log the error and move on | 10:20 |
| *** esikachev has quit IRC | 10:21 | |
| gmann | but catching exception from loader.discovery is not straight forward | 10:21 |
| gmann | anyways this will fix issue as of now - https://review.openstack.org/#/c/290983/3 | 10:22 |
| gmann | i will ping fwaas cores to merge it soon | 10:22 |
| yolanda | thanks | 10:23 |
| Kennan | hi gmann: | 10:23 |
| gmann | Kennan: hi | 10:23 |
| Kennan | when will tempest failure be fixed ? I notice you are working on that | 10:23 |
| Kennan | gmann: seems this ImportError: cannot import name base | 10:24 |
| Kennan | https://bugs.launchpad.net/tempest/+bug/1555524 | 10:24 |
| openstack | Launchpad bug 1555524 in tempest "Any tempest plugin import failure leads other plugin tests fails" [Undecided,New] - Assigned to Ghanshyam Mann (ghanshyammann) | 10:24 |
| gmann | Kennan: this patch fix this issue - https://review.openstack.org/#/c/290983/3 | 10:25 |
| gmann | Kennan: m checking if somehow we can avoid such issues in future | 10:25 |
| Kennan | gmann: so tempest have changes which cause such failure ? | 10:25 |
| openstackgerrit | Purandhar Sairam Mannidi proposed openstack/diskimage-builder: Add support for building images capable of UEFI https://review.openstack.org/287784 | 10:26 |
| gmann | Kennan: theoretically yes but that is not exactly the way other plugin should import | 10:26 |
| gmann | basically tempest interfaces are not stable interfaces except tempest/lib | 10:27 |
| gmann | so if someone importing and using those, we cannot guarantee that those will not be changed | 10:27 |
| gmann | all interfaces under tempest/lib are stable one and there we make sure about any backward incompatibility | 10:28 |
| Kennan | ok gmann: i notice our tempest also failed for that, hope that your change can be merged soon | 10:28 |
| gmann | Kennan: i am trying to reach fwaas core to merge that soon :) | 10:29 |
| Kennan | thanks | 10:29 |
| *** flepied has quit IRC | 10:29 | |
| *** dizquierdo_ has joined #openstack-infra | 10:35 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Follow name_or_id pattern on domain operations https://review.openstack.org/288715 | 10:36 |
| *** dizquierdo has quit IRC | 10:37 | |
| *** flepied has joined #openstack-infra | 10:42 | |
| *** dizquierdo_ has quit IRC | 10:43 | |
| *** dizquierdo has joined #openstack-infra | 10:45 | |
| *** rguillebert has joined #openstack-infra | 10:46 | |
| *** dims has joined #openstack-infra | 10:47 | |
| *** tzn has joined #openstack-infra | 10:48 | |
| *** kushal has joined #openstack-infra | 10:48 | |
| *** thorst_ has joined #openstack-infra | 10:50 | |
| *** bapalm has quit IRC | 10:51 | |
| *** dims has quit IRC | 10:52 | |
| *** yamamoto_ has quit IRC | 10:52 | |
| *** placko_ has quit IRC | 10:53 | |
| *** dizquierdo is now known as dizquierdo_afk | 10:55 | |
| *** dims has joined #openstack-infra | 10:56 | |
| *** ociuhandu has quit IRC | 10:56 | |
| *** fhubik is now known as fhubik_brb | 10:57 | |
| *** salv-orlando has joined #openstack-infra | 10:57 | |
| *** thorst_ has quit IRC | 10:58 | |
| *** sdague has joined #openstack-infra | 11:00 | |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 on the ceph gate job https://review.openstack.org/289445 | 11:01 |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config: net-bgpvpn: control gate settings from rc scripts https://review.openstack.org/291103 | 11:07 |
| *** asettle has joined #openstack-infra | 11:07 | |
| *** larainema has quit IRC | 11:08 | |
| *** bapalm has joined #openstack-infra | 11:08 | |
| *** gongysh has quit IRC | 11:09 | |
| *** pblaho has quit IRC | 11:11 | |
| *** _nadya_ has joined #openstack-infra | 11:11 | |
| *** fhubik_brb is now known as fhubik | 11:11 | |
| dims | sdague : https://review.openstack.org/#/c/289901/ is ready, i print the count of lines the warning message is repeated (example - http://logs.openstack.org/01/289901/4/check/gate-tempest-dsvm-full/5f4c1a4/logs/deprecations.txt.gz) | 11:13 |
| *** dguitarbite has joined #openstack-infra | 11:14 | |
| sdague | cool | 11:15 |
| sdague | dims: awesome, thanks | 11:16 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Follow name_or_id pattern on domain operations https://review.openstack.org/288715 | 11:18 |
| *** dims has quit IRC | 11:19 | |
| *** dims has joined #openstack-infra | 11:21 | |
| *** fhubik is now known as fhubik_brb | 11:22 | |
| *** fhubik_brb is now known as fhubik | 11:24 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-specs: Add extension to openstackci for next phase of work https://review.openstack.org/239810 | 11:24 |
| *** bapalm has quit IRC | 11:24 | |
| *** asettle has quit IRC | 11:27 | |
| *** kmartin has quit IRC | 11:29 | |
| *** Douhet has joined #openstack-infra | 11:29 | |
| *** salv-orlando has quit IRC | 11:30 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard: Fix removing due dates from boards https://review.openstack.org/291121 | 11:39 |
| *** ldnunes has joined #openstack-infra | 11:43 | |
| *** bapalm has joined #openstack-infra | 11:44 | |
| *** yamamoto has joined #openstack-infra | 11:44 | |
| *** yamamoto has quit IRC | 11:44 | |
| *** yamamoto has joined #openstack-infra | 11:44 | |
| *** Guest88500 has quit IRC | 11:45 | |
| *** yamamoto_ has joined #openstack-infra | 11:45 | |
| *** yamamoto has quit IRC | 11:50 | |
| *** Qiming has quit IRC | 11:50 | |
| *** rfolco has joined #openstack-infra | 11:50 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Add playbook to manage power state of ironic servers https://review.openstack.org/284981 | 11:52 |
| *** thorst_ has joined #openstack-infra | 11:56 | |
| *** dims has quit IRC | 11:56 | |
| *** sfinucan has quit IRC | 11:56 | |
| *** yamamoto_ has quit IRC | 11:57 | |
| *** aysyd has joined #openstack-infra | 11:59 | |
| *** kzaitsev_mb has quit IRC | 12:00 | |
| *** jaosorior has quit IRC | 12:02 | |
| *** thorst_ has quit IRC | 12:03 | |
| *** jaosorior has joined #openstack-infra | 12:03 | |
| *** dims has joined #openstack-infra | 12:04 | |
| *** erlon has joined #openstack-infra | 12:05 | |
| *** fhubik is now known as fhubik_brb | 12:07 | |
| *** fhubik_brb is now known as fhubik | 12:08 | |
| *** _amrith_ is now known as amrith | 12:16 | |
| *** xiangxinyong has quit IRC | 12:19 | |
| lucasagomes | hi all, ironic seems to be broken due some tempest import error: http://logs.openstack.org/12/291112/1/check/gate-tempest-dsvm-ironic-pxe_ssh/c7ad0ed/console.html#_2016-03-10_11_45_27_997 | 12:20 |
| openstackgerrit | Merged openstack-infra/storyboard: Fix removing due dates from boards https://review.openstack.org/291121 | 12:20 |
| *** placko_ has joined #openstack-infra | 12:21 | |
| *** kzaitsev_mb has joined #openstack-infra | 12:22 | |
| *** |-paul-| has joined #openstack-infra | 12:23 | |
| *** x00350071 has joined #openstack-infra | 12:25 | |
| lucasagomes | this patch in tempest broken ironic https://review.openstack.org/#/c/290108/ | 12:26 |
| AJaeger | lucasagomes: Read backsroll, this sounds like https://bugs.launchpad.net/tempest/+bug/1555524 | 12:26 |
| openstack | Launchpad bug 1555524 in tempest "Any tempest plugin import failure leads other plugin tests fails" [Undecided,New] - Assigned to Ghanshyam Mann (ghanshyammann) | 12:26 |
| yolanda | lucasagomes, gmann was working on it | 12:26 |
| gmann | lucasagomes: this will fix it - https://review.openstack.org/#/c/290983/ | 12:28 |
| *** Daisy has joined #openstack-infra | 12:28 | |
| lucasagomes | yolanda, gmann AJaeger thanks | 12:28 |
| *** yamamoto has joined #openstack-infra | 12:28 | |
| gmann | and in parallel m checking if we can have permanent solution to such case | 12:28 |
| lucasagomes | yeah it breaks us pretty badly | 12:28 |
| lucasagomes | I will add a patch in ironic depending on that to see if that works | 12:28 |
| lucasagomes | gmann, thanks a lot | 12:28 |
| *** placko_ has quit IRC | 12:29 | |
| *** placko_ has joined #openstack-infra | 12:29 | |
| *** gordc has joined #openstack-infra | 12:31 | |
| *** weshay has joined #openstack-infra | 12:31 | |
| *** kevinsho_ has joined #openstack-infra | 12:32 | |
| *** Daisy has quit IRC | 12:33 | |
| *** Daisy has joined #openstack-infra | 12:34 | |
| *** lucasagomes is now known as lucas-hungry | 12:34 | |
| *** hamzy has quit IRC | 12:35 | |
| *** thorst_ has joined #openstack-infra | 12:35 | |
| *** kevinsho_ has quit IRC | 12:37 | |
| *** Daisy has quit IRC | 12:38 | |
| *** doug-fish has quit IRC | 12:38 | |
| *** yamamoto has quit IRC | 12:39 | |
| *** doug-fish has joined #openstack-infra | 12:39 | |
| *** Qiming has joined #openstack-infra | 12:40 | |
| *** christian_ has quit IRC | 12:40 | |
| *** fesp has joined #openstack-infra | 12:43 | |
| *** yamamoto has joined #openstack-infra | 12:43 | |
| *** doug-fish has quit IRC | 12:44 | |
| *** pcaruana has quit IRC | 12:45 | |
| *** gongysh has joined #openstack-infra | 12:46 | |
| *** ildikov has quit IRC | 12:48 | |
| openstackgerrit | Merged openstack-infra/project-config: Move Manila experimental job to check pipeline https://review.openstack.org/290475 | 12:48 |
| *** fesp has quit IRC | 12:51 | |
| *** |-paul-| has quit IRC | 12:52 | |
| *** e0ne_ has joined #openstack-infra | 12:53 | |
| *** asettle has joined #openstack-infra | 12:53 | |
| *** e0ne has quit IRC | 12:54 | |
| *** pblaho has joined #openstack-infra | 12:55 | |
| *** baoli has joined #openstack-infra | 12:55 | |
| *** fhubik_real has joined #openstack-infra | 12:56 | |
| *** Guest8646 is now known as zeus | 12:56 | |
| *** zeus has quit IRC | 12:56 | |
| *** zeus has joined #openstack-infra | 12:56 | |
| *** fhubik has quit IRC | 12:57 | |
| *** hdd has joined #openstack-infra | 12:57 | |
| *** rhallisey has joined #openstack-infra | 12:57 | |
| *** dizquierdo_afk has quit IRC | 12:58 | |
| *** asettle has quit IRC | 12:58 | |
| *** jpr has joined #openstack-infra | 12:59 | |
| *** yamahata has quit IRC | 13:00 | |
| *** pcaruana has joined #openstack-infra | 13:00 | |
| *** dprince has joined #openstack-infra | 13:01 | |
| *** rossella_s has quit IRC | 13:02 | |
| *** cj has quit IRC | 13:03 | |
| *** sarob has joined #openstack-infra | 13:03 | |
| *** rossella_s has joined #openstack-infra | 13:03 | |
| *** cj has joined #openstack-infra | 13:05 | |
| *** EricGonczer_ has joined #openstack-infra | 13:05 | |
| *** esikachev has joined #openstack-infra | 13:07 | |
| *** sarob has quit IRC | 13:08 | |
| *** Daisy has joined #openstack-infra | 13:09 | |
| *** Daisy has quit IRC | 13:11 | |
| *** Daisy has joined #openstack-infra | 13:11 | |
| *** Hal has joined #openstack-infra | 13:12 | |
| *** kgiusti has joined #openstack-infra | 13:12 | |
| *** Hal is now known as Guest99065 | 13:12 | |
| *** pradk has joined #openstack-infra | 13:13 | |
| *** Guest99065 has quit IRC | 13:14 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Grab upper-constaints file for oslo master jobs https://review.openstack.org/291165 | 13:14 |
| *** Daisy has quit IRC | 13:15 | |
| *** lucas-hungry is now known as lucasagomes | 13:17 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Kill CI job if it doesn't get a testenv quickly https://review.openstack.org/290731 | 13:18 |
| *** larainema has joined #openstack-infra | 13:18 | |
| openstackgerrit | gordon chung proposed openstack/requirements: bump upper constraint for ceilometerclient https://review.openstack.org/291168 | 13:19 |
| *** edmondsw has joined #openstack-infra | 13:21 | |
| *** hamzy has joined #openstack-infra | 13:21 | |
| *** zeih has joined #openstack-infra | 13:22 | |
| *** cj has quit IRC | 13:24 | |
| *** zeih has quit IRC | 13:25 | |
| *** zeih has joined #openstack-infra | 13:25 | |
| *** doug-fish has joined #openstack-infra | 13:26 | |
| *** mtanino has joined #openstack-infra | 13:27 | |
| *** doug-fish has quit IRC | 13:27 | |
| *** doug-fish has joined #openstack-infra | 13:27 | |
| openstackgerrit | Alexis Lee proposed openstack/requirements: WIP oslo.config>=3.9.0 https://review.openstack.org/291175 | 13:29 |
| *** moravec has quit IRC | 13:31 | |
| *** cj has joined #openstack-infra | 13:33 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/project-config: Reduce max-servers for tripleo rh1 https://review.openstack.org/291179 | 13:35 |
| derekh | Small nodepool config tweak if anybody has a minute to take a look ^^ | 13:36 |
| *** acabot has quit IRC | 13:38 | |
| *** dims has quit IRC | 13:38 | |
| *** ogelbukh_ has joined #openstack-infra | 13:39 | |
| *** dims has joined #openstack-infra | 13:39 | |
| *** fhubik_real has quit IRC | 13:39 | |
| *** vgridnev has joined #openstack-infra | 13:40 | |
| anteaya | lennyb: thank you | 13:40 |
| *** cj has quit IRC | 13:41 | |
| *** EricGonczer_ has quit IRC | 13:43 | |
| *** gongysh has quit IRC | 13:43 | |
| *** burgerk has joined #openstack-infra | 13:43 | |
| *** fhubik_real has joined #openstack-infra | 13:44 | |
| *** Daisy has joined #openstack-infra | 13:45 | |
| *** sarob has joined #openstack-infra | 13:46 | |
| lennyb | anteaya: np, I just wasn't aware of the issue and was unreachable yesterday. | 13:48 |
| *** links has quit IRC | 13:48 | |
| AJaeger | dims: do you have some mintues to discuss your upper-constraints change? | 13:49 |
| *** degorenko is now known as _degorenko|afk | 13:49 | |
| dims | AJaeger : sure, i was looking for why the file does not get downloaded in my job and was trying to see if this was an option | 13:50 |
| *** acabot has joined #openstack-infra | 13:50 | |
| *** markvoelker has joined #openstack-infra | 13:50 | |
| *** vgridnev has quit IRC | 13:50 | |
| AJaeger | In general we download the file as part of setup, see http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/python-jobs.yaml#n75 | 13:50 |
| AJaeger | zuul-git-prep-upper-constraints sets up the constraints file in the location for tox to use | 13:51 |
| *** sarob has quit IRC | 13:51 | |
| *** vgridnev has joined #openstack-infra | 13:51 | |
| dims | AJaeger : ah so i should stick in zuul-git-prep-upper-constraints somewhere? | 13:51 |
| AJaeger | can we do something similar for your job? We might need to do this in a generic way - since other integration tests need it as well | 13:51 |
| dims | AJaeger : yes of course | 13:51 |
| AJaeger | dims: I'm currently looking at options, not yet on the drawing board ;) | 13:52 |
| *** rfolco_ has joined #openstack-infra | 13:52 | |
| dims | thanks AJaeger : i've W-1'ed my review | 13:52 |
| AJaeger | dims: you can change gerrit-git-prep to the zuul variant | 13:53 |
| *** rfolco has quit IRC | 13:53 | |
| AJaeger | I mean replace branch-git-prep with a zuul variant | 13:53 |
| AJaeger | dims: might need some more discussion with sdague, clarkb or fungi. | 13:54 |
| dims | AJaeger : here right? http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/python-jobs.yaml#n106 | 13:55 |
| dims | AJaeger : i'll still need to be able to specify branch? | 13:55 |
| AJaeger | and then use zuul-git-branch-prep-upper-constraints with master | 13:55 |
| anteaya | lennyb: I've forwarded you the email mriedem tried to send you yesterday, did you recieve it? | 13:55 |
| *** placko_ has quit IRC | 13:56 | |
| AJaeger | dims: you still might need to change the shell script to link run-tox together with this... | 13:56 |
| dims | AJaeger : ok. will wait for a bit for clarkb and fungi | 13:57 |
| AJaeger | perhaps copy the file around? But that's my suggestion... | 13:57 |
| AJaeger | thanks, dims ! | 13:57 |
| *** fhubik_real is now known as fhubik|rdotest | 13:57 | |
| pabelanger | yolanda: So the failures are a result of IO slowness in vexxhost | 13:57 |
| AJaeger | we could also pass UPPER_CONSTRAINTS_FILE in and only set to $(pwd)/upper-constraints.txt if its not set | 13:57 |
| pabelanger | I'm about to check logstash and see how many are passing and failing in vexxhost | 13:57 |
| *** ddieterly has joined #openstack-infra | 13:58 | |
| AJaeger | yolanda, pabelanger, anteaya: https://review.openstack.org/291022 and https://review.openstack.org/290826 are 5 out of the last 6 bare-trusty node changes (the final one I'll send once those two are in). If you have time to review this sometime this week, it would be great | 13:59 |
| *** mriedem has joined #openstack-infra | 13:59 | |
| pabelanger | AJaeger: checking | 13:59 |
| *** salv-orlando has joined #openstack-infra | 13:59 | |
| yolanda | AJaeger ok | 13:59 |
| *** jaosorior has quit IRC | 13:59 | |
| *** rlandy has joined #openstack-infra | 14:01 | |
| *** EricGonczer_ has joined #openstack-infra | 14:02 | |
| anteaya | AJaeger: looks like you have what you need | 14:02 |
| openstackgerrit | Sergey Belous proposed openstack-infra/project-config: Initial job for Nova with os-vif https://review.openstack.org/286085 | 14:02 |
| AJaeger | thanks, yolanda, anteaya and pabelanger! | 14:03 |
| *** keedya has joined #openstack-infra | 14:03 | |
| *** burgerk has quit IRC | 14:03 | |
| *** openstackgerrit has quit IRC | 14:03 | |
| *** openstackgerrit_ has joined #openstack-infra | 14:03 | |
| *** akshai has joined #openstack-infra | 14:04 | |
| *** amrith is now known as _amrith_ | 14:04 | |
| *** denisra_ has quit IRC | 14:04 | |
| *** ifarkas has quit IRC | 14:04 | |
| *** openstackgerrit_ is now known as openstackgerrit | 14:04 | |
| *** openstackgerrit has quit IRC | 14:04 | |
| *** openstackgerrit_ has joined #openstack-infra | 14:05 | |
| *** openstackgerrit_ is now known as openstackgerrit | 14:05 | |
| *** openstackgerrit has quit IRC | 14:05 | |
| *** openstackgerrit_ has joined #openstack-infra | 14:06 | |
| *** hdd has quit IRC | 14:06 | |
| *** Hal has joined #openstack-infra | 14:06 | |
| *** openstackgerrit_ is now known as openstackgerrit | 14:06 | |
| *** openstackgerrit has quit IRC | 14:07 | |
| *** Hal is now known as Guest40672 | 14:07 | |
| *** openstackgerrit_ has joined #openstack-infra | 14:07 | |
| AJaeger | poor openstackgerrit ;( | 14:07 |
| *** gongysh has joined #openstack-infra | 14:07 | |
| *** openstackgerrit_ is now known as openstackgerrit | 14:08 | |
| *** openstackgerrit has quit IRC | 14:08 | |
| *** openstackgerrit_ has joined #openstack-infra | 14:08 | |
| *** openstackgerrit_ is now known as openstackgerrit | 14:09 | |
| *** openstackgerrit has quit IRC | 14:09 | |
| *** openstackgerrit_ has joined #openstack-infra | 14:09 | |
| *** amotoki has quit IRC | 14:10 | |
| *** openstackgerrit_ is now known as openstackgerrit | 14:10 | |
| *** Guest41345 has joined #openstack-infra | 14:10 | |
| *** hashar has quit IRC | 14:10 | |
| Zara | ;_; | 14:11 |
| *** hashar has joined #openstack-infra | 14:11 | |
| *** burgerk has joined #openstack-infra | 14:12 | |
| sileht | AJaeger, hi, I got a issue with the new ubuntu-trusty image, the locale seems not the same as base-trusty, making some telemetry tests that loads yaml files to fail | 14:12 |
| *** amitgandhinz has joined #openstack-infra | 14:13 | |
| sileht | AJaeger, it works on python2.7 but we have trouble with python3.4 jobs | 14:13 |
| AJaeger | sileht: we discussed this on the channel here yesterday but I don't remember the final conclusion. | 14:14 |
| * sileht will dig into channel logs | 14:14 | |
| AJaeger | My personal statement: IF your test is dependent on a specific sort/locale, your test should set it. | 14:14 |
| AJaeger | sileht: otherwise, it might fail as well for any developer... | 14:14 |
| sileht | AJaeger, agreed | 14:15 |
| AJaeger | sileht: So, sorry for the breakage - but can you make your test robust on this, please? | 14:15 |
| *** nwkarsten has joined #openstack-infra | 14:15 | |
| *** nwkarsten has quit IRC | 14:15 | |
| sileht | AJaeger, the main issue is under underlying dep, I will continue to investigate | 14:16 |
| *** kushal has quit IRC | 14:16 | |
| *** nwkarsten has joined #openstack-infra | 14:16 | |
| *** claudiub has joined #openstack-infra | 14:16 | |
| AJaeger | fungi, anteaya: Was there any solution that we want to do on the infra side for this? | 14:16 |
| pabelanger | sileht: AJaeger do you have a URL to the failing test? | 14:16 |
| lennyb | anteaya: yes. I will catch him later | 14:16 |
| sileht | pabelanger, http://logs.openstack.org/44/269844/31/check/gate-gnocchi-tox-db-py34-mysql/15760a7/console.html#_2016-03-09_20_35_26_710 | 14:16 |
| anteaya | lennyb: did you recieve the email? | 14:17 |
| anteaya | AJaeger: there was a long discussion on the topic yesterday, as you point out | 14:17 |
| anteaya | and as sileht mentioned there seems to be a locale default difference between the two images | 14:17 |
| lennyb | anteaya: yes, I received the email. thank you. | 14:18 |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 on the ceph gate job https://review.openstack.org/289445 | 14:18 |
| pabelanger | sileht: you likely need to set en_US.utf8 for your locale: https://github.com/openstack/python-swiftclient/blob/master/tox.ini#L10 | 14:18 |
| anteaya | if there was a final conclusion I lost it in the noise, so yes sileht's decision to look at the logs is the one I would support | 14:18 |
| anteaya | lennyb: great thanks | 14:18 |
| sileht | pabelanger, as AJaeger said, tests should work in any locale | 14:18 |
| *** salv-orlando has quit IRC | 14:19 | |
| AJaeger | sileht: I said: Work in any developers/image locale. So, if your test is locale dependent, set it to well defined value as pabelanger suggests | 14:19 |
| anteaya | I was reading backscroll at the time of the conversation and still couldn't follow the conclusion if any | 14:19 |
| *** jpr has quit IRC | 14:19 | |
| pabelanger | sileht: right, but we don't set any default now in ubuntu-trusty | 14:19 |
| *** ajmiller has joined #openstack-infra | 14:19 | |
| sileht | AJaeger, yeah if the system doesn't have the defined locale it will fallback to C and I will got the same issue | 14:20 |
| AJaeger | sileht: are you setting a locale? | 14:20 |
| anteaya | that was a fast nova meeting | 14:20 |
| anteaya | <-- breakfast | 14:20 |
| *** sheeprine has quit IRC | 14:20 | |
| AJaeger | anteaya: enjoy! | 14:20 |
| sileht | AJaeger, no and I don't want to | 14:20 |
| anteaya | thanks | 14:20 |
| clarkb | sileht: then your tests need to work against a C locale | 14:21 |
| sileht | Agreed, I will fix the lib that cause issue | 14:22 |
| clarkb | the rough conclusion yesterday was we will set a utf8 locale by default, but as you say tests that actually care about utf8 or sort order shpuld explicitly say so | 14:22 |
| clarkb | AJaeger: ^ | 14:22 |
| *** mtanino has quit IRC | 14:22 | |
| yolanda | pabelanger, clarkb, i fix some fixes to ubuntu-minimal to set a locale https://review.openstack.org/290789 | 14:23 |
| *** nmagnezi has quit IRC | 14:23 | |
| AJaeger | clarkb: thanks | 14:23 |
| *** EricGonc_ has joined #openstack-infra | 14:23 | |
| AJaeger | clarkb: good morning, you're up early -everything ok? | 14:23 |
| yolanda | however the change is not passing tests, due to a bug in tempest | 14:23 |
| clarkb | AJaeger: ya and mostly not here yet :) | 14:24 |
| mordred | clarkb: aw - I was going to start poking you for reviews :) | 14:25 |
| *** Daisy has quit IRC | 14:25 | |
| mordred | pabelanger: how about you - you wanna +A this? https://review.openstack.org/#/c/290734 ? :) | 14:25 |
| *** ildikov has joined #openstack-infra | 14:25 | |
| AJaeger | mordred: that has a +A already, hasn't it... | 14:26 |
| *** Daisy has joined #openstack-infra | 14:26 | |
| AJaeger | mordred: you first need to merge https://review.openstack.org/#/c/251063/7 as dependent change | 14:26 |
| *** salv-orlando has joined #openstack-infra | 14:26 | |
| pabelanger | mordred: sure, checking | 14:26 |
| *** nmagnezi has joined #openstack-infra | 14:26 | |
| AJaeger | Wow, Zuul: "Queue lengths: 15292 events, 295 results. " | 14:27 |
| mordred | pabelanger: it's actually https://review.openstack.org/#/c/290777/ that needs help | 14:27 |
| *** EricGonczer_ has quit IRC | 14:27 | |
| mordred | AJaeger: good point | 14:27 |
| *** nwkarste_ has joined #openstack-infra | 14:28 | |
| clarkb | mordred: shade-launch-node is a thing now too | 14:28 |
| *** dalgaaf has joined #openstack-infra | 14:28 | |
| clarkb | we might want to delete old launch node or rename the shade one | 14:28 |
| *** dims has quit IRC | 14:29 | |
| * yolanda is currently using shade-launch-node successfully | 14:29 | |
| pabelanger | Yay for playbooks | 14:29 |
| *** moravec has joined #openstack-infra | 14:29 | |
| *** salv-orlando has quit IRC | 14:29 | |
| mordred | clarkb: yup. that's 251063 | 14:29 |
| mordred | clarkb: I kinda went on a tear yesterday because there was still parts of the old code trying to import novaclient v1_1 which doesn't exist anymore | 14:30 |
| mordred | clarkb: seemed like time to just finish the job | 14:30 |
| fungi | AJaeger: sileht: i believe yolanda has a patch up to address it in diskimage-builder. the biggest issue is that the default locale provided by ubuntu's glibc in trusty was not a utf-8 locale, but it does have an available utf-8 locale so we're going to see about setting the environment default to that on new images | 14:30 |
| *** Daisy has quit IRC | 14:30 | |
| *** ociuhandu has joined #openstack-infra | 14:30 | |
| yolanda | i set that to C.UTF-8 ... is waiting for a fix in tempest to pass tests, and review from cores | 14:30 |
| *** nwkarsten has quit IRC | 14:31 | |
| *** gongysh has quit IRC | 14:32 | |
| *** akrivoka has joined #openstack-infra | 14:32 | |
| anteaya | clarkb: thanks that helps me | 14:32 |
| AJaeger | fungi, yolanda: Thanks! | 14:32 |
| odyssey4me | It seems like the zuul queue lengths are just climbing and climbing | 14:33 |
| AJaeger | odyssey4me: impressive, isn't it? | 14:33 |
| *** xyang1 has joined #openstack-infra | 14:34 | |
| openstackgerrit | Merged openstack-infra/project-config: Change jobs to use ubuntu-trusty https://review.openstack.org/291022 | 14:34 |
| *** nwkarste_ has quit IRC | 14:34 | |
| anteaya | fungi: inbound zuul traffic is spiking, a runaway ci commenting? http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=397&rra_id=all | 14:34 |
| *** nwkarsten has joined #openstack-infra | 14:34 | |
| AJaeger | yolanda: is manual JJB run on jenkins02 finished? | 14:34 |
| anteaya | AJaeger: thanks for the alert on the zuul queue lengths | 14:34 |
| yolanda | yes, it finished | 14:34 |
| openstackgerrit | Merged openstack-infra/project-config: gerrit-plugin-jobs: Use ubuntu-trusty https://review.openstack.org/290826 | 14:35 |
| AJaeger | thanks, yolanda | 14:35 |
| AJaeger | anteaya: thanks for digging into it | 14:35 |
| *** dims has joined #openstack-infra | 14:35 | |
| *** christian_ has joined #openstack-infra | 14:36 | |
| *** tonytan4ever has joined #openstack-infra | 14:36 | |
| anteaya | well hopefully just finding what fungi can dig into | 14:36 |
| *** zz_dimtruck is now known as dimtruck | 14:36 | |
| anteaya | that was the only anomoly I see on the zuul cacti graphs | 14:36 |
| yolanda | AJaeger, glad to help. job config history in jenkins02 looks much better now | 14:36 |
| *** asettle has joined #openstack-infra | 14:37 | |
| *** _ari_ has joined #openstack-infra | 14:37 | |
| fungi | esikachev: you said you have some problem with "trigger by time" earlier that i don't see addressed by anyone yet. care to elaborate? | 14:37 |
| anteaya | fungi: do we have a runaway ci commenting causing long zuul queue lengths? | 14:38 |
| fungi | yolanda: rcarrillocruz: jjb had been manually run very recently on jenkins02 (all of the jenkins masters in fact). it's been less than 48 hours | 14:38 |
| yolanda | fungi so jenkins02 this morning was buggy again, it didn't pick the utf8 changes fro yesterday | 14:39 |
| yolanda | so i had to do again today | 14:39 |
| anteaya | queue lengths have decreased AJaeger | 14:39 |
| AJaeger | indeed, back to normal, anteaya | 14:39 |
| AJaeger | drastic - from 20k to 10 in seconds... | 14:39 |
| anteaya | 20K is very large though | 14:40 |
| anteaya | I'm used to 3k or so | 14:40 |
| fungi | dims: AJaeger: you mention wanting to discuss some upper-constraints change with me, clarkb or sdague but don't mention what the change is... details? | 14:40 |
| * AJaeger never saw it over 10k before. 2k is normal... | 14:40 | |
| anteaya | AJaeger: yeah me neither | 14:41 |
| AJaeger | fungi: https://review.openstack.org/291193 is the change | 14:41 |
| *** pcaruana has quit IRC | 14:41 | |
| dims | fungi : this one https://review.openstack.org/#/c/291193/ | 14:41 |
| AJaeger | background: nova (and glance, neutron*) use constraints in their tox.ini file | 14:41 |
| AJaeger | But integration tests do not setup the constraints file at all ;( | 14:41 |
| dims | fungi : i am seeing failures in oslo master periodic jobs | 14:41 |
| *** sheeprine has joined #openstack-infra | 14:41 | |
| AJaeger | dims run into this with the olso-master periodic job on nova | 14:41 |
| AJaeger | So, question is how to setup constraints for integration tests in the best way? | 14:42 |
| AJaeger | oslo master is the first one to run into it, others will follow for sure... | 14:42 |
| dims | AJaeger : these are py27/py34 jobs (periodic-nova-py27-with-oslo-master) | 14:42 |
| *** asettle has quit IRC | 14:43 | |
| AJaeger | fungi, it's periodic-{name}-{python}-with-oslo-master job -defined in jenkins/jobs/python-jobs.yaml | 14:44 |
| *** masayukig has quit IRC | 14:45 | |
| *** masayukig has joined #openstack-infra | 14:47 | |
| fungi | dims: AJaeger: yep, looks like we're already doing that in other periodic jobs | 14:47 |
| fungi | AJaeger: dims: oh, you're worried that the constraints file will cause pip not to install libs from source? | 14:48 |
| *** tonytan4ever has quit IRC | 14:48 | |
| fungi | i believe constraints are ignored if you include a requirement from local file path, url, et cetera | 14:49 |
| *** kevinsho_ has joined #openstack-infra | 14:49 | |
| AJaeger | fungi: I haven't thought about that one yet ;) | 14:49 |
| *** tonytan4ever has joined #openstack-infra | 14:49 | |
| openstackgerrit | Merged openstack-infra/system-config: Retire old launch-node.py https://review.openstack.org/251063 | 14:49 |
| dims | fungi : i think if that change looks good, we should merge and see what happens :) | 14:49 |
| yolanda | mordred, are you ok in approving https://review.openstack.org/#/c/285433/ ? the glean hostname fix | 14:49 |
| *** ajmiller has quit IRC | 14:49 | |
| AJaeger | fungi: I was just looking at dims change and didn't consider it correct and wanted to discuss the bigger picture here - but have no energey right now to help with the solution ;/ | 14:49 |
| *** salv-orlando has joined #openstack-infra | 14:50 | |
| openstackgerrit | Merged openstack-infra/system-config: Zero out the inventory cache instead of removing it https://review.openstack.org/290734 | 14:50 |
| dims | AJaeger : i'll work through it if that review does not help with the situation, i'll dig deeper | 14:50 |
| *** _amrith_ is now known as amrith | 14:50 | |
| fungi | AJaeger: what do you think is incorrect about it? | 14:50 |
| AJaeger | I was thinking of using a zuul setup macro instead of running a wget... | 14:51 |
| openstackgerrit | Merged openstack-infra/system-config: Use set-hostnames playbook in launch-node https://review.openstack.org/290777 | 14:51 |
| mordred | yolanda: done | 14:51 |
| fungi | AJaeger: oh, in the run-tox-with-oslo-master.sh script? | 14:51 |
| yolanda | thx | 14:51 |
| *** eharney has joined #openstack-infra | 14:52 | |
| AJaeger | I meant use zuul-git-branch-prep-upper-constraints and then copy the upper-constraints file around | 14:52 |
| fungi | AJaeger: copy it around to where? | 14:52 |
| AJaeger | to the place that run-tox expects it | 14:52 |
| *** esikachev has quit IRC | 14:52 | |
| *** esikachev has joined #openstack-infra | 14:52 | |
| fungi | AJaeger: doesn't zuul-git-branch-prep-upper-constraints do that already? | 14:52 |
| anteaya | mordred yolanda is today the day we get ip addresses on the new 60GB vm? we do need to announce as soon as possible so we can give folks at least 4 weeks to adjust | 14:52 |
| *** Zhongjun_ has joined #openstack-infra | 14:52 | |
| anteaya | and we have set the date for migration as April 11th | 14:53 |
| yolanda | anteaya, shade-launch-node finally worked so i have a vm now with an ip | 14:53 |
| AJaeger | fungi, I suggest to double check that it copies it to the right place | 14:53 |
| yolanda | i was just testing it | 14:53 |
| anteaya | yolanda: oh how lovely | 14:53 |
| fungi | AJaeger: or are you saying projects that use this particular job have tox.ini files not set up the way other projects do? | 14:53 |
| anteaya | yolanda: congratulations! | 14:53 |
| *** pcaruana has joined #openstack-infra | 14:53 | |
| anteaya | yolanda: yup, awesome, thank you | 14:53 |
| yolanda | kudos to mordred | 14:53 |
| * anteaya offers mordred kudos | 14:54 | |
| yolanda | i'm testing the puppet runs, it seems to be giving some problems, but the vm is ready | 14:54 |
| *** _nadya_ has quit IRC | 14:54 | |
| eharney | ianw: did you see my reply on https://review.openstack.org/#/c/289589/ ? curious what you think | 14:54 |
| yolanda | anteaya, shall it be enough to send an announce to openstack-infra mailing list? | 14:54 |
| fungi | AJaeger: that macro does this: cp $REQS_DIR/upper-constraints.txt ./ | 14:54 |
| *** pradk_ has joined #openstack-infra | 14:54 | |
| anteaya | yolanda: yes, the ips are all you need to announce | 14:55 |
| yolanda | no need to enter something in the wiki, or fix any other documentation ? | 14:55 |
| fungi | AJaeger: so it's copying the file into the top level directory of the project's clone workdir | 14:55 |
| anteaya | yolanda: then you/we have 4 weeks to prepare it | 14:55 |
| AJaeger | fungi, I just double checked - I was udner the impression the job changed directories but does not. So, just using the zuul macro should indeed fix this. | 14:55 |
| anteaya | yolanda: I haven't seen that we have done that before, what were you thinking? | 14:55 |
| anteaya | yolanda: whatever you think we need to tell folks the ips | 14:55 |
| yolanda | first time that i will do that, so i wanted to be sure | 14:55 |
| yolanda | mostly to be sure that the announce in mailing list is not ignored | 14:56 |
| fungi | yolanda: ip address and our tentative maintenance date/time | 14:56 |
| *** _nadya_ has joined #openstack-infra | 14:56 | |
| *** hashar has quit IRC | 14:56 | |
| *** jaosorior has joined #openstack-infra | 14:56 | |
| anteaya | yolanda: here is the announcement from last time: http://lists.openstack.org/pipermail/openstack-dev/2015-February/056508.html | 14:56 |
| *** _degorenko|afk is now known as degorenko | 14:56 | |
| yolanda | fungi anteaya, it will be a good idea to announce again every week until we approach to the maintenance window | 14:56 |
| fungi | yolanda: make sure to include both ipv4 and ipv6 addresses of the new server and, for reference, the ones on the current production server | 14:56 |
| AJaeger | fungi: I've been looking to much at integration jobs like jenkins/jobs/bandit.yaml - where there is a cd involved. I expect those jobs to fail the same way like the olso one soon - and those do change directories | 14:56 |
| anteaya | yolanda: I don't know as we did that before | 14:56 |
| *** thiagop has joined #openstack-infra | 14:57 | |
| anteaya | the thread before was one post long: http://lists.openstack.org/pipermail/openstack-dev/2015-February/thread.html#56508 | 14:57 |
| anteaya | I'll be sharing the post at the third party meetings | 14:57 |
| fungi | AJaeger: yes, i think some more complex jobs need more complex solutions if they're going to use constraints. for example, devstack jobs take a completely different (and more complicated) approach to using it | 14:57 |
| yolanda | ok i'll send something like that | 14:57 |
| anteaya | yolanda: awesome, if you want to create a draft in an etherpad we can take a peek before you post, if you like | 14:58 |
| yolanda | sounds good | 14:58 |
| anteaya | thanks for taking care of this | 14:58 |
| yolanda | i did a similar work for Gozer, to happy to do it in upstream this time | 14:58 |
| openstackgerrit | Merged openstack-infra/project-config: Reduce max-servers for tripleo rh1 https://review.openstack.org/291179 | 14:59 |
| mordred | fungi, yolanda: actually - before we send out the email ... | 14:59 |
| mordred | openstack supports fixed ips that are not tied to a specific server but that are not floating ips | 15:00 |
| mordred | rackspace does not allow us to create them | 15:00 |
| mordred | however | 15:00 |
| mordred | maybe we should ask if they'd let us make one | 15:00 |
| mordred | so that we could put it on the new gerrit, but have this be the last time we have to tell people about a new IP? | 15:00 |
| *** _nadya__ has joined #openstack-infra | 15:00 | |
| *** _nadya_ has quit IRC | 15:01 | |
| anteaya | mordred: whom could we ask? | 15:01 |
| mordred | I have no idea how hard that would be permissions-wise. maybe I should shut up and we should just do that next time | 15:01 |
| anteaya | well I don't think it would hurt to ask | 15:01 |
| yolanda | mordred, and how does it work on terms of a server replacement? you update the ip, moving to the new server you create? | 15:01 |
| *** mtanino has joined #openstack-infra | 15:01 | |
| anteaya | as long as we don't delay announcing waiting on the reply | 15:01 |
| mordred | yolanda: same as with floating ips | 15:01 |
| yolanda | ah you associate/disassociate on demand | 15:02 |
| mordred | yolanda: you make the new server, then you detach the port from the old server and attach it to the new server | 15:02 |
| mordred | yah | 15:02 |
| yolanda | who can we talk in rax for that? | 15:02 |
| mordred | jroll: should I ping pvo about that ? ^^ | 15:02 |
| jroll | mordred: sup | 15:02 |
| * AJaeger waits for ages for a "git review" to send changes to our gerrit ;( | 15:03 | |
| jroll | mordred: so you want a fixed ip you can move around? | 15:03 |
| jroll | mordred: or just a regular port? | 15:03 |
| *** dimtruck is now known as zz_dimtruck | 15:04 | |
| *** Jeffrey4l has joined #openstack-infra | 15:04 | |
| openstackgerrit | Merged openstack-infra/puppet-lodgeit: fix puppet-lodgeit to support latest mysql https://review.openstack.org/275244 | 15:05 |
| *** mrmartin has quit IRC | 15:06 | |
| *** burgerk has quit IRC | 15:06 | |
| mordred | jroll: uh. either? I mean, those two things sound the same to my dumb lizard brain | 15:06 |
| mordred | jroll: nova fixed-ip == neutron port best I can tell? | 15:06 |
| mordred | but yes - would love to allocate an IP that we can add to a server, and then remove from it, and then add to a diffrent server | 15:07 |
| openstackgerrit | Merged openstack-infra/project-config: Remove cloudv-ostf-adapter from Infrastructure Systems https://review.openstack.org/285399 | 15:07 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 15:08 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove redundant Manila Tempest job https://review.openstack.org/290623 | 15:08 |
| dmsimard | clarkb: so you think the failures in https://review.openstack.org/#/c/290776/ are legit ? | 15:08 |
| * dmsimard looks | 15:08 | |
| mtreinish | infra-root: when someone gets a chance can they check the logs on health.o.o it looks like something is up with the api server | 15:09 |
| mordred | mtreinish: I blame jaypipes | 15:09 |
| openstackgerrit | Merged openstack-infra/project-config: Remove Fedora23 install-glean-from-git flag https://review.openstack.org/290975 | 15:09 |
| mtreinish | mordred: heh, works for me :) | 15:09 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove bare-trusty from projects.yaml https://review.openstack.org/290809 | 15:11 |
| *** zeih has quit IRC | 15:11 | |
| AJaeger | the change above just passed as recheck without any JJB changes in the diff output, so ready to merge! | 15:11 |
| jroll | mordred: I have learned things | 15:11 |
| *** rbrndt__ has joined #openstack-infra | 15:12 | |
| AJaeger | fungi, do you want to review the removal of bare-trusty from projects.yaml? ^ | 15:12 |
| mtreinish | mordred: oh, yay it's returning 500 on that status page, that's no good | 15:12 |
| *** vgridnev has quit IRC | 15:12 | |
| jroll | mordred: tl;dr: it's possible to do so, but you have to land in the same cell, among other gotchas, I wouldn't do it | 15:12 |
| *** tonytan_brb has joined #openstack-infra | 15:13 | |
| *** vgridnev has joined #openstack-infra | 15:13 | |
| openstackgerrit | Ana Krivokapic proposed openstack-infra/project-config: New project for TripleO UI https://review.openstack.org/291227 | 15:15 |
| * jaypipes blames mordred | 15:15 | |
| *** salv-orlando has quit IRC | 15:16 | |
| *** vgridnev has quit IRC | 15:16 | |
| *** mrmartin has joined #openstack-infra | 15:16 | |
| yolanda | jroll, mordred, so fixed ip is not a good idea? | 15:17 |
| *** Guest40672 has quit IRC | 15:17 | |
| mordred | jroll: gotcha | 15:17 |
| mordred | cool | 15:17 |
| *** tonytan4ever has quit IRC | 15:17 | |
| jroll | mordred: yolanda: correct, I'd recommend waiting for floating ips | 15:17 |
| mordred | jroll: when rax gets floating ips - it's not going to take away our current lovely and wonderful real public ips is it? | 15:18 |
| anteaya | jroll: thanks for the fast turn around on the query | 15:18 |
| jroll | mordred: I don't believe so :) | 15:18 |
| jroll | anteaya: of course :) | 15:18 |
| mordred | jroll: \o/ | 15:18 |
| *** ddieterly is now known as ddieterly[away] | 15:18 | |
| *** Hal has joined #openstack-infra | 15:18 | |
| mordred | the fact that rax does not require floating ips for things is one of my favorite things about it | 15:18 |
| jroll | heh, I know | 15:19 |
| *** Hal is now known as Guest20150 | 15:19 | |
| * mordred just wants to make sure everyone is aware his mind has not magically changed recently | 15:19 | |
| *** zeih has joined #openstack-infra | 15:19 | |
| yolanda | mordred, jroll ++ for not floating ips :) | 15:19 |
| anteaya | mordred: so noted | 15:19 |
| mordred | but ... you're right, the use case I have with wanting a sticky ip for review.o.o is the one actual useful usecase for floating ips | 15:19 |
| *** vgridnev has joined #openstack-infra | 15:19 | |
| *** zz_dimtruck is now known as dimtruck | 15:19 | |
| mordred | it's rare - less than 1% of our servers need it - but it _is_ a useful thing for that 1% | 15:19 |
| jroll | mordred: I even believe that floating IPs will not be implemented via NAT, but am not 100% sure on that | 15:19 |
| mordred | jroll: WOOT | 15:20 |
| mordred | jroll: that makes your floating ips useful | 15:20 |
| jroll | :) | 15:20 |
| mordred | whoever thought NAT for servers was a feature should be very ashamed of themselves | 15:20 |
| fungi | AJaeger: i do want to review that, yes. i'll add it to my priority list for this morning | 15:20 |
| *** esikachev has quit IRC | 15:20 | |
| fungi | still trying to catch up on e-mail | 15:20 |
| AJaeger | fungi: no urgency on that - just happy to see that progress ;) | 15:21 |
| AJaeger | once that's in, we need to review whether infra-manual needs an update as well... | 15:21 |
| *** ddieterly[away] is now known as ddieterly | 15:21 | |
| *** vgridnev has quit IRC | 15:22 | |
| AJaeger | I can ssh to review.openstack.org but I cannot push a change using "git review -d". Any ideas? | 15:22 |
| Clint | AJaeger: push with -d? | 15:22 |
| fungi | jroll: yes! for ages i've wanted nova to support binding additional arbitrary addresses to instance network interfaces directly (aliases in the traditional term) rather than only supporting solving that use case through network address translation | 15:22 |
| *** jaosorior has quit IRC | 15:23 | |
| jroll | fungi: :) | 15:23 |
| AJaeger | Clint: "git review -y" | 15:23 |
| *** vgridnev has joined #openstack-infra | 15:23 | |
| *** ajmiller has joined #openstack-infra | 15:23 | |
| AJaeger | (those patches on top of each other) | 15:23 |
| fungi | i used to attach multiple ip addresses to network interfaces in virtual machines all the time, but that was in the dark ages where i managed my virtual machines by hand | 15:23 |
| anteaya | I don't know the -y flag | 15:23 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove final bare-trusty job https://review.openstack.org/291234 | 15:24 |
| anteaya | --yes | 15:24 |
| AJaeger | anteaya: don't ask for "yes" if you have stacked changes | 15:24 |
| AJaeger | Ah, that one works on its own | 15:24 |
| anteaya | understood | 15:24 |
| anteaya | thanks | 15:24 |
| *** tsg has joined #openstack-infra | 15:24 | |
| jeblair | 30532 opensta+ 20 0 3790004 3.184g 7964 S 100.4 82.6 613:11.88 apache2 | 15:25 |
| *** dizquierdo has joined #openstack-infra | 15:25 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove bare-trusty from nodepool https://review.openstack.org/291238 | 15:25 |
| AJaeger | Bye, bye bare-trusty! ^ | 15:25 |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 on the ceph gate job https://review.openstack.org/289445 | 15:25 |
| AJaeger | so, I manged to push the two patches through as single ones... | 15:25 |
| yolanda | woot! | 15:25 |
| EmilienM | hello infra, would it be possible to land this patch about login.defs on redhat systems? thanks https://review.openstack.org/#/c/288244/ | 15:25 |
| fungi | AJaeger: wow, that change deletes 758 lines! i have a feeling we need to update the examples in the infra manual too | 15:26 |
| jeblair | [Thu Mar 10 15:22:07.549262 2016] [:error] [pid 30532:tid 139826121930496] [remote 88.179.44.181:50645] OperationalError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT runs.run_at AS runs_run_at, runs.passes AS runs_passes, runs.fails AS runs_fails, runs.skips AS runs_skips, run_metadata.value AS run_metadata_value \\nFROM runs INNER JOIN run_metadata ON runs.id = ... | 15:26 |
| jeblair | ... run_metadata.run_id \\nWHERE run_metadata.`key` = %(key_1)s AND runs.run_at >= %(run_at_1)s AND runs.run_at <= %(run_at_2)s'] [parameters: {u'key_1': u'project', u'run_at_1': datetime.datetime(2016, 2, 10, 14, 52, 3, 165000, tzinfo=tzlocal()), u'run_at_2': datetime.datetime(2016, 3, 10, 14, 52, 3, 165000, tzinfo=tzlocal())}] | 15:26 |
| mtreinish | jeblair: hmm, I would have expected it to recover from a loss in connectivity to the db | 15:27 |
| jeblair | mtreinish: there are several of those, that's the most recent | 15:27 |
| jeblair | mtreinish: and did you see the line from top above ^ | 15:27 |
| jeblair | [Thu Mar 10 14:58:33.988192 2016] [cache:info] [pid 31777:tid 139825881478912] [client 103.56.55.33:56659] AH00780: Cache lock file for '/runs/key/build_queue/gate/recent' too old, removing: /tmp/mod_cache-lock/b/Q/bQW2ktpSMvVsBFA0rNFeJw, referer: http://status.openstack.org//openstack-health/ | 15:28 |
| pabelanger | AJaeger: \o/ | 15:28 |
| mtreinish | jeblair: about the memory consumption? | 15:28 |
| jeblair | mtreinish: and cpu consumption | 15:28 |
| openstackgerrit | Merged openstack-infra/system-config: Decouple openstack_project::server from puppetdb class https://review.openstack.org/172871 | 15:29 |
| *** Daisy has joined #openstack-infra | 15:29 | |
| openstackgerrit | Merged openstack-infra/system-config: Add periodic pipeline test result to infra docs https://review.openstack.org/254725 | 15:29 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual: Remove bare-trusty lines https://review.openstack.org/291241 | 15:30 |
| AJaeger | fungi, just two lines for infra-manual | 15:30 |
| mtreinish | jeblair: hmm, I kinda wish there was a big obvious stack trace in the logs. That's easier to debug :) | 15:30 |
| *** aysyd has quit IRC | 15:30 | |
| mtreinish | I'm not quite sure why it's going off the rails | 15:30 |
| *** kingia_ has joined #openstack-infra | 15:30 | |
| *** aysyd has joined #openstack-infra | 15:30 | |
| jeblair | mtreinish: there are several of those, but i haven't yet correlated them with the timeouts, which are more recent. | 15:31 |
| fungi | AJaeger: yep, figured it would be simple, but it reduces the number which will get reintroduced through copying (not by much though, since people mostly seem to copy project creation changes they or others submitted in the past rather than cribbing from the instructions each time) | 15:31 |
| mtreinish | jeblair: well, at the very least I guess that's evidence that mod_cache is doing something. I wasn't 100% that was actually configued correctly :) | 15:32 |
| yolanda | mordred then we leave the external ip for another time? | 15:32 |
| jeblair | [Thu Mar 10 07:55:41.802144 2016] [:error] [pid 30532:tid 139826138715904] [remote 76.103.105.228:50389] mod_wsgi (pid=30532): Exception occurred processing WSGI script '/opt/openstack-health/.venv/bin/openstack-health'. | 15:32 |
| *** eharney has quit IRC | 15:32 | |
| jeblair | mtreinish: first timeout ^ | 15:32 |
| *** kingia has quit IRC | 15:33 | |
| mtreinish | jeblair: ok, yeah that makes more sense | 15:33 |
| mtreinish | does it log the exception anywhere? | 15:33 |
| *** aorourke-afk is now known as aorourke | 15:34 | |
| mordred | yolanda: yes. let's continue with the current plan | 15:34 |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder: Generate fedora-atomic images using dib https://review.openstack.org/287167 | 15:34 |
| *** burgerk has joined #openstack-infra | 15:35 | |
| *** eharney has joined #openstack-infra | 15:35 | |
| openstackgerrit | Ana Krivokapic proposed openstack-infra/project-config: New project for TripleO UI https://review.openstack.org/291227 | 15:35 |
| jeblair | mtreinish: https://etherpad.openstack.org/p/HeDtMwfZCJ | 15:35 |
| openstackgerrit | Merged openstack-infra/project-config: Fix oslo master jobs failures - missing upper constraints https://review.openstack.org/291193 | 15:35 |
| mtreinish | jeblair: hmm, so it lost the connection to the db and that query failed | 15:36 |
| mtreinish | I'd expect that 1 request to get a 500 and then for everything to move on | 15:37 |
| mtreinish | I'm kinda surprised that's enough to bring everything down | 15:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Remove interpret_puppet_exitcodes https://review.openstack.org/291249 | 15:38 |
| *** IlyaG has joined #openstack-infra | 15:38 | |
| mtreinish | jeblair: like it's returning 500 on http://health.openstack.org/ and http://health.openstack.org/status those don't even make db queries | 15:39 |
| jeblair | mtreinish: i expect all of its threads are busy | 15:39 |
| *** Qiming has quit IRC | 15:39 | |
| mtreinish | ah, that could do it | 15:40 |
| mtreinish | jeblair: is there still a connectivity issue to the db from health.o.o? | 15:40 |
| jeblair | there's basically no network traffic right now... | 15:40 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 15:40 | |
| *** cody-somerville has joined #openstack-infra | 15:41 | |
| jeblair | mtreinish: i can connect to the port via telnet. there is no mysql client installed on that host. | 15:41 |
| *** Daisy has quit IRC | 15:42 | |
| *** darvon has quit IRC | 15:42 | |
| mtreinish | hah, I guess that's a disadvantage of using pymysql, mysql never gets installed on the host because you don't need the c bindings anymore | 15:42 |
| *** Daisy has joined #openstack-infra | 15:42 | |
| *** pfallenop has quit IRC | 15:43 | |
| jeblair | i connected from a different host, mysql is idle | 15:43 |
| openstackgerrit | Ana Krivokapic proposed openstack-infra/project-config: New project for TripleO UI https://review.openstack.org/291227 | 15:43 |
| *** vgridnev has quit IRC | 15:44 | |
| mtreinish | jeblair: right, I've been able to connect locally since I woke up ~1hr ago. I was thinking it might have been a connectivity issue more local to the api server | 15:44 |
| jeblair | mtreinish: did you install a thread-dump handler by any chance? | 15:44 |
| mtreinish | jeblair: I'm tempted to just say lets just restart apache and cross our fingers this doesn't happen again | 15:44 |
| mtreinish | jeblair: I don't think so | 15:44 |
| *** sdake has joined #openstack-infra | 15:44 | |
| jeblair | that would tell us what the threads are doing... right now strace just says they are all waiting on futexes | 15:45 |
| mtreinish | jeblair: if you have a package name in mind I can add it to the puppet policy for the api server | 15:45 |
| *** darvon has joined #openstack-infra | 15:45 | |
| *** pfallenop has joined #openstack-infra | 15:45 | |
| jeblair | there are several connections to the mysql server that are in close_wait | 15:45 |
| jeblair | mtreinish: for thread dumping? i mean in the program. we have them in nodepool and zuul. | 15:46 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 15:46 | |
| *** sdake_ has joined #openstack-infra | 15:46 | |
| jeblair | could probably copy it pretty easily, it's fairly standalone | 15:47 |
| mtreinish | jeblair: oh, flask might have something builtin, but I didn't put anything explicitly in the api server | 15:47 |
| *** esikachev has joined #openstack-infra | 15:47 | |
| yolanda | infra-root (or any interested) that is a draft of the email to send for gerrit ip: https://etherpad.openstack.org/p/update-gerrit-host-email . Any feedback? | 15:48 |
| dmsimard | clarkb: found the issue for the swap patch .. fixing | 15:49 |
| jeblair | mtreinish: i'm out of ideas without further access to the internal state. i can restart now if you want. | 15:49 |
| *** dimtruck is now known as zz_dimtruck | 15:49 | |
| *** sdake has quit IRC | 15:49 | |
| *** baoli has quit IRC | 15:50 | |
| mtreinish | jeblair: yeah go for it. I'll work on adding the thread dump to the api server should this happen again | 15:50 |
| jeblair | cool | 15:50 |
| *** ildikov has quit IRC | 15:50 | |
| *** baoli has joined #openstack-infra | 15:51 | |
| sdague | oh... hey, I just discovered this in the gerrit docs - https://review.openstack.org/Documentation/config-gerrit.html#trackingid | 15:51 |
| jeblair | env: htcacheclean: No such file or directory | 15:51 |
| jeblair | mtreinish: ^ that is emitted when it starts | 15:51 |
| *** angdraug has joined #openstack-infra | 15:52 | |
| sdague | seems like if we specify that correctly bug:1523668 would be a valid search bit | 15:52 |
| *** yamahata has joined #openstack-infra | 15:52 | |
| *** zz_dimtruck is now known as dimtruck | 15:53 | |
| * yolanda needs to step out for a while | 15:53 | |
| mtreinish | jeblair: hmm, I haven't seen that before. Shouldn't the apache package install that? | 15:53 |
| jeblair | mtreinish: apparently it's in the apache2-utils package | 15:54 |
| jeblair | which is not installed | 15:54 |
| mtreinish | :/ | 15:54 |
| *** xarses has joined #openstack-infra | 15:54 | |
| pabelanger | mordred: thanks for the 1.5.1 release of shade, it fixed some packaging issues I was having for fedora :) | 15:56 |
| pabelanger | I should finally be able to push this up to rawhide today or tomorrow | 15:56 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-openstack_health: Add apache2-utils package to api server https://review.openstack.org/291264 | 15:58 |
| mtreinish | jeblair: ^^^ | 15:58 |
| *** mrmartin has quit IRC | 15:59 | |
| openstackgerrit | Ben Swartzlander proposed openstack-infra/project-config: Give manila-image-elements its own acls https://review.openstack.org/290865 | 15:59 |
| *** jistr has quit IRC | 15:59 | |
| *** Jeffrey4l has quit IRC | 16:00 | |
| *** armax has joined #openstack-infra | 16:00 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: WIP task notes https://review.openstack.org/291267 | 16:00 |
| *** jtomasek has quit IRC | 16:00 | |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Exclude python-troveclient 2.1.0 https://review.openstack.org/290935 | 16:00 |
| *** mrmartin has joined #openstack-infra | 16:02 | |
| mordred | pabelanger: woot! | 16:03 |
| *** esikachev has quit IRC | 16:03 | |
| anteaya | yolanda: I have added my thoughts to the etherpad, thank you | 16:03 |
| *** pradk has quit IRC | 16:06 | |
| *** pradk_ is now known as pradk | 16:06 | |
| *** oomichi_ has joined #openstack-infra | 16:06 | |
| *** esikachev has joined #openstack-infra | 16:07 | |
| *** oanson has quit IRC | 16:09 | |
| *** sdake_ is now known as sdake | 16:10 | |
| openstackgerrit | Merged openstack-infra/system-config: Upgrade grafana to 2.6.0 https://review.openstack.org/264979 | 16:14 |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Update Fontawesome to 4.5.0 https://review.openstack.org/291274 | 16:16 |
| *** kmartin has joined #openstack-infra | 16:16 | |
| *** zeih has quit IRC | 16:17 | |
| *** hashar has joined #openstack-infra | 16:17 | |
| mtreinish | jeblair: did you just manually install apache2-utils to get it going? | 16:17 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add storyboard exim router/transports https://review.openstack.org/270331 | 16:17 |
| jeblair | mtreinish: no, i ignored it | 16:17 |
| jeblair | mtreinish: it did not seem to be a fatal error | 16:17 |
| *** oomichi_ has quit IRC | 16:18 | |
| *** oomichi_ has joined #openstack-infra | 16:19 | |
| mtreinish | ah, ok | 16:19 |
| fungi | yolanda: lgtm other than including the current ip addresses, which i just added (this makes it easier for people who have to submit requests to their corporate firewall overlords to tell them what ip addresses are replacing what other ip addresses) | 16:19 |
| *** arxcruz has quit IRC | 16:20 | |
| *** esikachev has quit IRC | 16:21 | |
| jeblair | anyone know why we still have two entries for the health dashboard on the status site? | 16:22 |
| *** baoli has quit IRC | 16:23 | |
| *** tonytan_brb has quit IRC | 16:23 | |
| jeblair | ah here it is https://review.openstack.org/286350 | 16:23 |
| *** pblaho has quit IRC | 16:24 | |
| *** tonytan4ever has joined #openstack-infra | 16:24 | |
| *** bpokorny has joined #openstack-infra | 16:24 | |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 on the ceph gate job https://review.openstack.org/289445 | 16:25 |
| *** christian_ has quit IRC | 16:26 | |
| *** ZZelle has quit IRC | 16:27 | |
| *** EricGonc_ has quit IRC | 16:28 | |
| *** tonytan4ever has quit IRC | 16:28 | |
| *** tonytan4ever has joined #openstack-infra | 16:28 | |
| *** ildikov has joined #openstack-infra | 16:29 | |
| anteaya | clarkb: when next you are about, logstash 2.0 | 16:29 |
| *** baoli has joined #openstack-infra | 16:29 | |
| anteaya | clarkb: I'm wondering if there is any way we can get to kibana 4 by summit | 16:29 |
| *** sc68cal has joined #openstack-infra | 16:30 | |
| anteaya | I don't dare to dream yet, just wondering what you think is possible | 16:30 |
| *** nwkarsten has quit IRC | 16:30 | |
| *** nwkarsten has joined #openstack-infra | 16:30 | |
| *** tzn has quit IRC | 16:31 | |
| *** sc68cal_ has joined #openstack-infra | 16:31 | |
| openstackgerrit | Merged openstack-infra/system-config: Remove QA health link from status page https://review.openstack.org/286350 | 16:31 |
| *** _nadya__ has quit IRC | 16:32 | |
| *** amitgandhinz has quit IRC | 16:32 | |
| *** nwkarste_ has joined #openstack-infra | 16:32 | |
| *** amitgandhinz has joined #openstack-infra | 16:32 | |
| *** harlowja_at_home has joined #openstack-infra | 16:33 | |
| *** mrmartin has quit IRC | 16:34 | |
| *** annegentle has joined #openstack-infra | 16:34 | |
| *** nwkarsten has quit IRC | 16:35 | |
| openstackgerrit | Merged openstack-infra/project-config: Build AFS RPMs for Centos7 periodically https://review.openstack.org/290214 | 16:36 |
| *** jlanoux has quit IRC | 16:36 | |
| SamYaple | clarkb: for this fstrim patch, based on what the comments were in teh review it seems like _only_ presenting fstrim and not zerofree would be prefered, in which case i think the cleanest thing would be a revert of the zerofree patch | 16:37 |
| SamYaple | your thoughts? | 16:37 |
| *** mikelk has quit IRC | 16:37 | |
| openstackgerrit | Merged openstack/gertty: Use diff long options and uncap GitPython https://review.openstack.org/290080 | 16:37 |
| openstackgerrit | Merged openstack/gertty: Add config option for git clone URL https://review.openstack.org/169469 | 16:37 |
| clarkb | SamYaple: I dont think there is anything in the zerofree patch that needs to survive without zerofree so that should be fine | 16:38 |
| SamYaple | yea i checked it over, i agree with you there | 16:38 |
| SamYaple | ok will do | 16:38 |
| *** vincentll has quit IRC | 16:39 | |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add configuration information to docs https://review.openstack.org/286240 | 16:39 |
| openstackgerrit | Merged openstack/gertty: Make topic selection a select list https://review.openstack.org/286238 | 16:39 |
| openstackgerrit | Merged openstack/gertty: Indent projects under topics https://review.openstack.org/286239 | 16:39 |
| *** jpr has joined #openstack-infra | 16:39 | |
| clarkb | anteaya: kibana4 work can happen independent of logstash 2.0 I just dont have the time to do both at the same time. I do however intend on getting logstash 2.0 in real soon now. so we shall see | 16:39 |
| openstackgerrit | Merged openstack/gertty: Support batch abandon/restore https://review.openstack.org/290852 | 16:39 |
| *** Daisy has quit IRC | 16:39 | |
| openstackgerrit | Merged openstack/gertty: Handle more than one change result when searching https://review.openstack.org/290694 | 16:39 |
| nibalizer | yolanda: that blurb looks good to me | 16:39 |
| *** Daisy has joined #openstack-infra | 16:40 | |
| *** flepied has quit IRC | 16:40 | |
| *** rfolco has joined #openstack-infra | 16:40 | |
| jeblair | pabelanger: would you please reconsider https://review.openstack.org/284806 ? | 16:40 |
| anteaya | clarkb: is there something like an etherpad or storyboard in place such that I can understand the current status and figure out how to help, or can I compose such a thing? | 16:41 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 16:42 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 16:42 | |
| pabelanger | jeblair: WFM | 16:43 |
| *** rfolco_ has quit IRC | 16:43 | |
| clarkb | anteaya: just the changes in review | 16:43 |
| anteaya | clarkb: do they have topics? | 16:43 |
| anteaya | if yes, what are they? | 16:43 |
| pabelanger | jeblair: left comment about git.o.o, that what my only concern | 16:44 |
| jeblair | pabelanger: i agree, it would be nice. | 16:44 |
| *** Daisy has quit IRC | 16:44 | |
| anteaya | clarkb: I only see two patches with logstash-2.0 as the topic: https://review.openstack.org/#/q/status:open+topic:logstash-2.0 | 16:45 |
| clarkb | anteaya: looks like 'logstash' and 'logstash-2.0' | 16:45 |
| anteaya | thank you | 16:45 |
| anteaya | clarkb: any objection to me creating a storyboard for these? | 16:45 |
| anteaya | if only for my own benefit? | 16:45 |
| clarkb | there are only 3 changes total, just need to get them applied | 16:45 |
| *** sridhar_ram has joined #openstack-infra | 16:45 | |
| anteaya | for kibana 4 as well? | 16:45 |
| pabelanger | won't mind a review of https://review.openstack.org/#/c/287232/ Specifically the manually installation of wiki extensions, I left a comment explaining what needs to be done moving forward | 16:45 |
| clarkb | anteaya: sure, though there isnt much to say other than apply changes | 16:46 |
| clarkb | kibana 4 is a completely unknown entity at this point | 16:46 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Generate a log with deprecation warnings https://review.openstack.org/289901 | 16:46 |
| anteaya | oh okay, thanks | 16:46 |
| *** e0ne_ has quit IRC | 16:46 | |
| *** pcaruana has quit IRC | 16:47 | |
| anteaya | https://review.openstack.org/#/q/status:open+topic:%255Elogstash.* | 16:48 |
| markvan | AJaeger: ping | 16:48 |
| *** ihrachys has quit IRC | 16:48 | |
| *** scheuran has quit IRC | 16:49 | |
| anteaya | clarkb: looks like yolanda is leaving https://review.openstack.org/#/c/247857/ for you to approve and babysit | 16:49 |
| anteaya | markvan: AJaeger prefers if you just ask your question | 16:49 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add storyboard exim router/transports https://review.openstack.org/270331 | 16:50 |
| markvan | anteaya: k, thx. Looking for help on a job-template question. Is it possible to define different job timeouts for templated jobs? Example anywhere? | 16:50 |
| fungi | AJaeger: did you have a change proposed to replace gate-infra-puppet-apply-bare-trusty with gate-infra-puppet-apply-ubuntu-trusty? i'm also noticing we're singling out the centos7 version of that job with a different job-template which i'm looking into now to see if it can be simplified | 16:52 |
| anteaya | can you share which job template you are looking to change? | 16:52 |
| markvan | anteaya: sure, it's this one in Heat: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/heat.yaml#L57 Looks like LBaaSv2 increases devstaack deploy by 10-15min and then testing are almost 5min | 16:53 |
| *** regXboi has joined #openstack-infra | 16:53 | |
| markvan | so would like to have only lbaasv2 side of that template have a longer timeout 80 -> 120 | 16:54 |
| *** asettle has joined #openstack-infra | 16:54 | |
| clarkb | anteaya oh good, though ideally all the changes go in together I can at least get the refactor in first | 16:54 |
| jeblair | why does lbaasv2 take so long to deploy? | 16:55 |
| *** cody-somerville has quit IRC | 16:55 | |
| *** fhubik|rdotest has quit IRC | 16:55 | |
| anteaya | clarkb: ah okay | 16:55 |
| anteaya | clarkb: reviewing them now | 16:55 |
| markvan | jeblair: looks like it the octavia_amphora packages and images, but have not dug into that yet | 16:55 |
| *** flepied has joined #openstack-infra | 16:56 | |
| yolanda | thanks for the feedback, i'll send now | 16:56 |
| rm_work | yes | 16:56 |
| rm_work | we use DIB to build our amp | 16:56 |
| *** ashleighfarnham has joined #openstack-infra | 16:56 | |
| rm_work | which takes... a bit | 16:56 |
| rm_work | ^^ octavia devstack build times | 16:56 |
| rm_work | also we pull in barbican as a dep too | 16:56 |
| rm_work | and neutron and nova and glance etc | 16:57 |
| anteaya | yolanda: did you remove the last sentence? | 16:57 |
| rm_work | so large dep list | 16:57 |
| anteaya | yolanda: or do you disagree? | 16:57 |
| rm_work | jeblair: ^^ | 16:57 |
| yolanda | i'll remove | 16:57 |
| anteaya | yolanda: thanks | 16:57 |
| yolanda | i was used to send emails with that bit at the end :) | 16:57 |
| jeblair | rm_work: do the amphora images need to be rebuilt for all of these jobs, or just certain ones? | 16:57 |
| markvan | anteaya: I was thinking I could add a timeout in the project job descriptions https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/projects.yaml#L2249 and then ref that in the template as {timeout}, would that work? | 16:58 |
| anteaya | markvan: as for the orginal question, I do not believe we have a mechinism to change timeouts for various variables in a job | 16:58 |
| jeblair | rm_work: the trove folks (and some other projects) have a system where they build images and put them on tarballs.o.o for most jobs to use, then you only need to rebuild them in live tests when you're changing something on them. | 16:58 |
| *** fhubik|rdotest has joined #openstack-infra | 16:58 | |
| *** asettle has quit IRC | 16:58 | |
| anteaya | markvan: as jeblair is noting, asking to do so might be indicative of needing to solve a problem within on part of them | 16:58 |
| anteaya | yolanda: I understand | 16:59 |
| amrith | jeblair, we are moving away from that system | 16:59 |
| *** fhubik|rdotest has quit IRC | 16:59 | |
| fungi | amrith: what did the challenges end up being? | 16:59 |
| jeblair | markvan: when we're talking about increasing timeouts because expected run times of jobs are > 1 hour, we should really pause and reflect. | 16:59 |
| amrith | it sounds like a good idea but we've found a lot of challenges with it. | 16:59 |
| fungi | amrith: also do you know if ironic is experiencing similar issues as they also do that? | 16:59 |
| anteaya | markvan: not that I'm aware and as jeblair is pointing out, this is cause for reflection | 16:59 |
| amrith | fungi: let me write up a longer response and then paste-bomb it. | 16:59 |
| fungi | amrith: ml thread might make more sense in that case | 16:59 |
| *** moravec has quit IRC | 17:00 | |
| amrith | fungi, no. it would be ironic if they were facing the same problem ;) | 17:00 |
| fungi | heh | 17:00 |
| amrith | kidding, could not resist. | 17:00 |
| amrith | I don't know. | 17:00 |
| yolanda | ok sent | 17:00 |
| rm_work | jeblair: well | 17:00 |
| anteaya | yolanda: thank you | 17:00 |
| yolanda | thx for the feedback | 17:00 |
| rm_work | jeblair: so it is something we have tried | 17:00 |
| amrith | the issues have mostly to do with frequency of update | 17:00 |
| *** ihrachys has joined #openstack-infra | 17:00 | |
| amrith | we used tarballs in two cases; guest images and trove client. | 17:00 |
| rm_work | jeblair: so the images we generate bake in code from our current codebase which any patch could be changing | 17:01 |
| amrith | I'll summarize in a short email | 17:01 |
| rm_work | so we kinda need to build it? | 17:01 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/devstack-gate: Log deprecations in horizon and keystone https://review.openstack.org/291301 | 17:01 |
| amrith | rm_work, we work around that in a different way. | 17:01 |
| rm_work | hmm ok | 17:01 |
| *** HeOS has quit IRC | 17:01 | |
| openstackgerrit | Merged openstack/requirements: Cap cliff-tablib <2.0 in stable/kilo https://review.openstack.org/272457 | 17:01 |
| amrith | trove guest images that we use for testing/development that is | 17:01 |
| *** tzn has joined #openstack-infra | 17:01 | |
| amrith | the way we do that is that the guest images have a stub of upstart code that rsync's the guest agent code from the host. | 17:01 |
| rm_work | like i said, we have tried it, we did download an image from a CDN for a while instead of building | 17:01 |
| rm_work | but it caused us problems | 17:01 |
| markvan | anteaya: jeblair was looking for short term solution to get the basic lbaas v2 heat test up and going since the resources were added. | 17:02 |
| amrith | that way, if I'm making changes to the code base, server and guest, then launching a new instance automagically gets you the latest guest agent. | 17:02 |
| rm_work | amrith: i am sure if you have experience you are willing to share, we would be interested in hearing it | 17:02 |
| amrith | would love to | 17:02 |
| rm_work | amrith: if you want to talk to us in #openstack-lbaas that would be great | 17:02 |
| rm_work | amrith: specifically johnsom was working on it | 17:02 |
| amrith | sure thing, let me add that group | 17:02 |
| *** rossella_s has quit IRC | 17:03 | |
| mgagne | pabelanger: is the grafana upgrade still in progress? | 17:03 |
| *** rossella_s has joined #openstack-infra | 17:03 | |
| amrith | rm_work, hailing you in #openstack-lbaas | 17:03 |
| *** lucasagomes is now known as lucas-afk | 17:04 | |
| *** flepied has quit IRC | 17:04 | |
| openstackgerrit | Merged openstack-infra/system-config: Customize cgit max-repo-count and set it to 1500 https://review.openstack.org/288745 | 17:04 |
| *** dims has quit IRC | 17:04 | |
| *** derekh has quit IRC | 17:05 | |
| anteaya | markvan: yes, we are aware, and I believe the point jeblair is making is that in cases such as this the most effcient way of dealing with the issue is to understand the problem rather than attempting to cover it over and creating technical debt | 17:05 |
| anteaya | and if indeed that is his point, I agree with him | 17:05 |
| openstackgerrit | Merged openstack-infra/system-config: Cacti: add hosts to alternate graph trees https://review.openstack.org/284448 | 17:05 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Update Fontawesome to 4.5.0 https://review.openstack.org/291274 | 17:06 |
| *** dims has joined #openstack-infra | 17:06 | |
| jeblair | amrith, rm_work: okay, so we have a pretty flexible system with a lot of options for working through this, and this is the first time i'm hearing about either of the issues. there are a lot of folks around here who may have creative ideas for how to deal with this, so please do start a mailing list thread and invite infra folks to join. | 17:06 |
| anteaya | markvan: also given there is no mechinism to do what you want, I don't see there is a short term solution available | 17:06 |
| *** moravec has joined #openstack-infra | 17:06 | |
| rm_work | jeblair: we actually dealt with the timeout and long build issues about 3-4 months ago | 17:06 |
| markvan | anteaya: sure I can see that point. Thinking down the road when devstack starts to enable lbaasv2 as default, could impact many jobs | 17:06 |
| rm_work | dealt with, as in, it was a huge issue for us and neutron and we ended up just cutting out testing down | 17:07 |
| rm_work | and splitting our testing into like 5 different jobs | 17:07 |
| rm_work | if you remember that whole mess | 17:07 |
| anteaya | markvan: why would devstack enable lbaasv2 by default? | 17:07 |
| anteaya | I don't know as we even have neutron enabled by default in devstack yet | 17:07 |
| markvan | anteaya: ok, so my only choice is to create a separate job just for this (it's experiemental right now) with a longer timeout | 17:07 |
| anteaya | someone correct me if I'm wrong | 17:07 |
| jeblair | markvan: what's the expected runtime of the job? | 17:08 |
| anteaya | markvan: well I have been trying to say, work with the folks who have created the longer runtime to solve that issue | 17:08 |
| markvan | anteaya: was thinking as folks move from v1 which is there by default to v2, they will need that enabled. Not sure on roadmap | 17:08 |
| anteaya | markvan: that would be something you would have to discuss with the devstack folks | 17:08 |
| markvan | jeblair: right now it's at 80, and looks to have about 10-20 minutes left | 17:08 |
| anteaya | and if lbaasv1 is enabled by default in devstack, that is news to me | 17:09 |
| jeblair | markvan: we try to keep jobs under 1 hour | 17:09 |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: WIP task notes https://review.openstack.org/291267 | 17:09 |
| markvan | jeblair: sure, that's reasonable goal | 17:09 |
| *** vincentfrancoise has joined #openstack-infra | 17:10 | |
| anteaya | clarkb: rechecked a few of the patches to get fresh logs | 17:10 |
| *** gyee has joined #openstack-infra | 17:10 | |
| *** HeOS has joined #openstack-infra | 17:11 | |
| anteaya | asselin mmedvede lennyb can you help ensure folks see this email post? http://lists.openstack.org/pipermail/openstack-dev/2016-March/088985.html | 17:11 |
| markvan | anteaya: the current lbaasv2 job is at 130, that's why I was going down this road in the first place for this new heat job. | 17:11 |
| jeblair | oy | 17:12 |
| anteaya | what jeblair said | 17:12 |
| mmedvede | anteaya: ok, I'll bring it up during meetings | 17:13 |
| markvan | yeah, puts this heat job in a bad place. I think this heat job is the first one trying to use lbaasv2 outside of the base lbass jobs | 17:13 |
| anteaya | mmedvede: thank you | 17:13 |
| jeblair | markvan: so the heat job should not be changing any amphora code, so it's ripe for being able to use a pre-built image | 17:13 |
| * sc68cal drops in | 17:13 | |
| anteaya | sc68cal: welcome | 17:14 |
| *** abregman is now known as abregman|mtg | 17:14 | |
| markvan | jeblair: yup, that sounds like what we need here | 17:14 |
| *** sc68cal_ has quit IRC | 17:14 | |
| jeblair | (even if that's more difficult for lbaasv2 itself, though i suspect there are untried solutions there as well) | 17:14 |
| *** keedya has quit IRC | 17:14 | |
| markvan | jeblair: anteaya thx for the chat, I'll take it over to the lbaas channel to discuss | 17:15 |
| jeblair | markvan: let us know if you need any help | 17:15 |
| lennyb | anteaya: ok | 17:16 |
| sc68cal | rm_work: still figuring out gating on an amphora image eh? | 17:16 |
| anteaya | lennyb: thank you | 17:16 |
| jeblair | rm_work: do all of the jobs each build an amphora image? | 17:16 |
| anteaya | markvan: thank you | 17:16 |
| *** kushal has joined #openstack-infra | 17:17 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Let Users Add Notes to Tasks https://review.openstack.org/291267 | 17:18 |
| *** vgridnev has joined #openstack-infra | 17:19 | |
| anteaya | Zara: yay notes | 17:19 |
| openstackgerrit | Merged openstack-infra/system-config: Add sysstat and iotop to all hosts https://review.openstack.org/284806 | 17:19 |
| anteaya | Zara: wooooo | 17:19 |
| *** hashar has quit IRC | 17:20 | |
| Zara | anteaya: :) it should replace the old 'task links' patch | 17:20 |
| Zara | since you can put an url in a note (even if you have to hilight and paste it) | 17:20 |
| Zara | I'm hoping we can refine it in the future to make it a bit nicer, though | 17:20 |
| anteaya | I didn't even know about the task links patch | 17:20 |
| anteaya | you invoking a writhing mass of snakes | 17:20 |
| anteaya | that makes me less confident than I would like to be | 17:20 |
| anteaya | is taht still how you would characterize the state of the code? | 17:21 |
| Zara | ah, no it has been improved since I wrote that, though there still might be odd things | 17:21 |
| anteaya | can you update the comment? | 17:21 |
| Zara | before I was switching between 3 variable names for the same sort of thing | 17:21 |
| Zara | it was ugly | 17:21 |
| Zara | sure | 17:21 |
| anteaya | thanks | 17:21 |
| *** Sukhdev has joined #openstack-infra | 17:21 | |
| *** fedexo has joined #openstack-infra | 17:22 | |
| *** vgridnev has quit IRC | 17:22 | |
| *** dizquierdo has quit IRC | 17:23 | |
| jeblair | infra-root: did someone try to stop gerrit? | 17:25 |
| fungi | jeblair: i did not | 17:25 |
| mordred | jeblair: I did not - but I did see the email | 17:25 |
| fungi | why do you ask? | 17:25 |
| clarkb | not me | 17:25 |
| yolanda | jeblair, the sudoers? it may come from the replacement gerrit | 17:25 |
| jeblair | ah | 17:25 |
| yolanda | i'm working on that and tried to sudo with gerrit2 | 17:25 |
| fungi | oh! yes, that makes sense | 17:25 |
| * clarkb is working on paging logstash 2.0 back in | 17:25 | |
| fungi | they are both named review.openstack.org as far as they know | 17:25 |
| yolanda | i may stop mailings on that for the moment | 17:25 |
| jeblair | it's not configured to talk to the database is it? | 17:25 |
| yolanda | yes, they are both named the same | 17:25 |
| *** ddieterly is now known as ddieterly[away] | 17:26 | |
| fungi | yolanda: don't worry about it, we're aware and know to look at received headers | 17:26 |
| pabelanger | jeblair: not me | 17:26 |
| *** matrohon has quit IRC | 17:26 | |
| fungi | but yeah, let's make sure we don't start gerrit on that server | 17:26 |
| jeblair | because if it accesses the production database, it could corrupt it | 17:27 |
| fungi | having it connect to the production database and alter anything there would be a "bad thing"[tm] | 17:27 |
| fungi | right, that | 17:27 |
| fungi | don't cross the streams | 17:27 |
| yolanda | it has its own | 17:27 |
| fungi | yolanda: its own trove instance? | 17:27 |
| yolanda | no, local database | 17:28 |
| fungi | oh, interesting | 17:28 |
| yolanda | but anyway, ned to be careful with replication and other thigns | 17:29 |
| yolanda | but anyway, ned to be careful with replication and other things | 17:29 |
| *** oomichi_ has quit IRC | 17:29 | |
| jeblair | how does it have its own trove database? | 17:29 |
| jeblair | er local database? | 17:29 |
| *** ddieterly[away] is now known as ddieterly | 17:30 | |
| yolanda | yep local at the moment because i was doing some tests | 17:30 |
| jeblair | yolanda: what are you doing? | 17:30 |
| yolanda | i was checking manage_projects, wasn't running | 17:31 |
| * anteaya checked the backup server on cacti and device read/write appear to be operating on devices xvda,b,c,e | 17:31 | |
| *** esker has joined #openstack-infra | 17:32 | |
| yolanda | jeblair, how do you normally test these replacements ? what i did for gozer is to have its own local db, disable replication | 17:32 |
| jeblair | yolanda: what are we testing? | 17:33 |
| *** moravec has quit IRC | 17:33 | |
| openstackgerrit | Merged openstack-infra/git-review: Correct metadata URLs https://review.openstack.org/275445 | 17:33 |
| yolanda | jeblair, if the server comes up cleanly, and everything is up | 17:33 |
| jlk | fungi: jeblair: mordred: have you tried turning bluebox back on in nodepool to see if it's working now? | 17:33 |
| anteaya | jlk: it looks good to me | 17:34 |
| *** apoorvad has joined #openstack-infra | 17:34 | |
| anteaya | jlk: I looked this morning | 17:34 |
| fungi | though i think our grafana is busted at the moment | 17:34 |
| fungi | pabelanger: is that an issue known to you? | 17:34 |
| jeblair | yolanda: we don't. and we definitely don't make alternate manual configuration changes to servers. puppet runs and it either works or we fix it. | 17:34 |
| jlk | anteaya: ah okay. | 17:34 |
| fungi | pabelanger: at the moment i just get "{{alert.title}}" when i browse to it | 17:34 |
| anteaya | um, I get an odd output when I try: http://grafana.openstack.org/ | 17:35 |
| anteaya | fungi: me too | 17:35 |
| yolanda | jeblair so you spin up the instance, and don't do any operation until the deployment day? | 17:35 |
| mordred | jlk: I agree with anteaya | 17:35 |
| jeblair | yolanda: yep. | 17:35 |
| jeblair | pabelanger and i are upgrading grafana to 2.6.0. it looks like we have succeeded. ;) | 17:35 |
| *** esker has quit IRC | 17:35 | |
| anteaya | jlk: I did look this morning, no error node launch attempts | 17:35 |
| *** esker has joined #openstack-infra | 17:35 | |
| *** nwkarste_ has quit IRC | 17:35 | |
| jlk | thanks | 17:35 |
| anteaya | jlk: thanks for asking | 17:36 |
| anteaya | jeblair pabelanger success | 17:36 |
| fungi | jeblair: heh! | 17:36 |
| jeblair | yolanda: mostly we make sure puppet and the gerrit init script are disabled so we don't accidentally start it running | 17:36 |
| *** nwkarsten has joined #openstack-infra | 17:36 | |
| yolanda | mm, gerrit is stopped there, but how can i disable that on puppet? if i put on emergency hosts, the naming is common to both gerrits | 17:37 |
| jeblair | yolanda: there's a thing you can do with uuids i think | 17:37 |
| fungi | yeah, it's documented in the system-config manual | 17:37 |
| *** vincentfrancoise has quit IRC | 17:37 | |
| jeblair | yolanda: (and make sure the gerrit init script is not set to start on boot; that's a manual change that's okay to make :) | 17:37 |
| yolanda | ok | 17:38 |
| jeblair | yolanda: (that way if rackspace reboots the instance for some reason it doesn't start up) | 17:38 |
| yolanda | ah that makes sense | 17:38 |
| fungi | yolanda: http://docs.openstack.org/infra/system-config/sysadmin.html#examples (second example) | 17:38 |
| nibalizer | and 20 minutes later I get the gerrit sudo email | 17:38 |
| *** Zhongjun_ has quit IRC | 17:38 | |
| * nibalizer hi5s fastmail | 17:38 | |
| *** tphummel has quit IRC | 17:39 | |
| fungi | nibalizer: greylisting ftw? | 17:39 |
| Clint | greylolsting | 17:39 |
| nibalizer | ya thats probably what happened | 17:39 |
| fungi | it's probably the first e-mail they've ever seen from that ip address, so not terribly surprising | 17:39 |
| sc68cal | greylist4lyfe | 17:39 |
| nibalizer | also pabelanger now that you're root can you make stackalytics not email me so much | 17:39 |
| anteaya | ha ha ha | 17:40 |
| *** fedexo has quit IRC | 17:40 | |
| *** nwkarsten has quit IRC | 17:41 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove bare-trusty from projects.yaml https://review.openstack.org/290809 | 17:42 |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Let Users Add Notes to Tasks https://review.openstack.org/291267 | 17:44 |
| *** esikachev has joined #openstack-infra | 17:45 | |
| pabelanger | AJaeger: nice work again with bare-trusty! We can likely start looking into system-config to remove some puppet code too | 17:45 |
| *** fawadkhaliq has joined #openstack-infra | 17:47 | |
| pleia2 | ooh, was that the last bare-trusty? | 17:47 |
| pleia2 | \o/ | 17:48 |
| openstackgerrit | Ales Komarek proposed openstack-infra/project-config: Added required package python-yaml for salt-formula-test job https://review.openstack.org/291321 | 17:48 |
| pabelanger | pleia2: Ya, I think so | 17:48 |
| AJaeger | fungi, https://review.openstack.org/291234 is the apply job | 17:48 |
| *** SumitNaiksatam has joined #openstack-infra | 17:49 | |
| AJaeger | pabelanger: thanks. Cleaning up system-config is a task I will *not* sign up for | 17:49 |
| anteaya | yay AJaeger has limits! | 17:49 |
| fungi | AJaeger: oh, awesome--thanks. i figured you probably had since it was the only glaring omission from git grep | 17:49 |
| *** yamamoto has quit IRC | 17:49 | |
| pabelanger | AJaeger: :) I can volunteer for that | 17:50 |
| *** amitgandhinz has quit IRC | 17:51 | |
| *** jtomasek has joined #openstack-infra | 17:51 | |
| pabelanger | AJaeger: fungi: see my comments on ^ 291321. I guess that will be the next step, removing custom apt-get install logic into JJB templates | 17:51 |
| *** nikhil has quit IRC | 17:51 | |
| *** jpr has quit IRC | 17:51 | |
| pc_m | Hi folks. I had a question... https://review.openstack.org/#/c/290513/ was +2/+A at 8:30am, it passed check queue at 9:30am, but I don't see it in gate queue and it now almost 1pm. Any thoughts? | 17:52 |
| *** Sukhdev has quit IRC | 17:52 | |
| *** e0ne has joined #openstack-infra | 17:53 | |
| *** dims has quit IRC | 17:54 | |
| pabelanger | pc_m: is it possible it requires 289558 first? | 17:54 |
| pabelanger | gerrit web is terrible now for me to tell | 17:54 |
| anteaya | pabelanger: the recheck | 17:54 |
| pabelanger | fires up gertty | 17:54 |
| anteaya | sorry pabelanger | 17:54 |
| anteaya | pc_m: the recheck | 17:54 |
| anteaya | pc_m: that sent it into check and told it to return | 17:55 |
| pc_m | pabelanger: Ah, that could be. I think it was based on top of that one. | 17:55 |
| anteaya | oh pabelanger is probably correct here | 17:55 |
| anteaya | I can't read the new gerrit ui dependency tree anymore | 17:56 |
| pc_m | pabelanger: Where is that shown in gerrit. | 17:56 |
| pc_m | ? | 17:56 |
| pabelanger | pc_m: anteaya: Ya, that is the issue. confirmed by gertty | 17:56 |
| pabelanger | pc_m: related changes | 17:56 |
| pabelanger | which is a terrible name for it | 17:56 |
| pc_m | pabelanger: Ah. I didn't know where to look. Used to be right in the center and easy to see. Thanks! | 17:56 |
| anteaya | pc_m: top right corner | 17:56 |
| anteaya | pc_m: yeah | 17:56 |
| pc_m | anteaya: pabelanger: Thanks! | 17:57 |
| anteaya | pc_m: gertty shows the dependenices easier I'm told | 17:57 |
| pabelanger | anteaya: indeed! | 17:57 |
| * pc_m googling for gertty | 17:57 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Let Users Add Notes to Tasks https://review.openstack.org/291267 | 17:57 |
| anteaya | pc_m: https://pypi.python.org/pypi/gertty/1.3.1 | 17:58 |
| pc_m | anteaya: Thanks. Just found it. Will have to look at this | 17:58 |
| *** moravec has joined #openstack-infra | 17:59 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Logstash 2.0 compat ES output rule https://review.openstack.org/247714 | 18:00 |
| *** baoli has quit IRC | 18:00 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-logstash: Install logstash 2.0 https://review.openstack.org/247756 | 18:00 |
| *** tphummel has joined #openstack-infra | 18:00 | |
| clarkb | anteaya: ^ a few updates that apparently got lost when logstash got deprioritized | 18:00 |
| anteaya | clarkb: thanks, will re-review | 18:00 |
| *** sputnik13 has joined #openstack-infra | 18:01 | |
| *** baoli has joined #openstack-infra | 18:01 | |
| asselin | anteaya, mmedvede I added the link to the wiki. https://wiki.openstack.org/wiki/Meetings/ThirdParty | 18:01 |
| *** mriedem has quit IRC | 18:01 | |
| anteaya | I need to call the hardware store about my dryer, it seems to not be heating up reliably | 18:01 |
| anteaya | asselin: good idea, thank you | 18:01 |
| *** dims has joined #openstack-infra | 18:03 | |
| AJaeger | fungi, once once the apply job change is in (291234), we can remove bare-trusty also from nodepool (https://review.openstack.org/291238) | 18:04 |
| *** notnownikki has quit IRC | 18:04 | |
| *** sambetts is now known as sambetts|afk | 18:04 | |
| fungi | AJaeger: yep, i was about to review it, but my workstation (which still has a broken fan i can't seem to get a replacement for) shut itself off again so i'm letting it cool back down | 18:04 |
| fungi | will look at that one shortly once i let it recover | 18:05 |
| Zara | anteaya: thanks for reviewing! :) | 18:05 |
| AJaeger | fungi, no worries - just wanted to point it out. I put the nodepool one on WIP, wasn't sure whether that one can merge directly or needs to wait a bit... | 18:06 |
| *** abregman|mtg is now known as abregman | 18:06 | |
| *** Swami has joined #openstack-infra | 18:07 | |
| fungi | AJaeger: i'm inclined to hold off merging removal for nodepool and system-config yet, since it may be harder to temporarily pivot some jobs back to bare-trusty if we discover a particularly nasty issue which will take more time to work around | 18:07 |
| fungi | at least for a few more days | 18:07 |
| AJaeger | fungi, feel free to -2 my patch then ;) | 18:07 |
| fungi | once my workstation is back up, i'll gladly do that ;) | 18:08 |
| fungi | in the meantime, catching up on more e-mail | 18:08 |
| *** ddieterly is now known as ddieterly[away] | 18:08 | |
| *** ybathia has joined #openstack-infra | 18:09 | |
| * AJaeger renames now tox-nodb to tox again... | 18:09 | |
| *** annegentle has quit IRC | 18:09 | |
| pabelanger | AJaeger: so, what sort of timelines do you think about removing apt-get syntax from JJB and migrating projects to other-requirements.txt? | 18:11 |
| *** nmagnezi has quit IRC | 18:12 | |
| *** jordanP has quit IRC | 18:12 | |
| fungi | pabelanger: we should hold off doing that until we have the database requirements worked out | 18:13 |
| *** mriedem has joined #openstack-infra | 18:13 | |
| fungi | pabelanger: otherwise we're going to wind up with lots of projects adding other-requirements.txt files and unnecessarily including mysql/postgres because of implementation details of some current job-templates | 18:13 |
| *** kzaitsev_mb has quit IRC | 18:13 | |
| AJaeger | pabelanger: I need a few days break first ;) - and wait for fungi to look at the database side first | 18:14 |
| pabelanger | AJaeger: fungi: Yup, fair enough | 18:14 |
| *** kevinsho_ has quit IRC | 18:15 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Rename tox-nodb-ENV to tox-ENV https://review.openstack.org/291349 | 18:15 |
| openstackgerrit | Sam Yaple proposed openstack/diskimage-builder: Revert "Zerofree the image if possible" https://review.openstack.org/291350 | 18:15 |
| openstackgerrit | Sam Yaple proposed openstack/diskimage-builder: Use fstrim to prep the block device https://review.openstack.org/291351 | 18:15 |
| AJaeger | pabelanger, fungi: that's the rename back to tox-ENV ^ | 18:15 |
| *** sputnik13 has quit IRC | 18:16 | |
| fungi | AJaeger: awesome! you are a change-writing machine | 18:16 |
| *** ybathia has quit IRC | 18:16 | |
| *** electrofelix has quit IRC | 18:17 | |
| *** abregman has quit IRC | 18:17 | |
| AJaeger | fungi: I know how to use emacs ;) | 18:17 |
| *** moravec has quit IRC | 18:17 | |
| AJaeger | pabelanger: do you want to approve your first infra-manual change, please? Then the examples are fine as well: https://review.openstack.org/#/c/291241/ | 18:17 |
| *** moravec has joined #openstack-infra | 18:18 | |
| *** esker has quit IRC | 18:18 | |
| pabelanger | AJaeger: done | 18:18 |
| *** esker has joined #openstack-infra | 18:18 | |
| SamYaple | hello, can I get some reviews on this shade patch? https://review.openstack.org/#/c/287504/ more keystone v3 things for shade | 18:19 |
| *** tmorin has joined #openstack-infra | 18:19 | |
| AJaeger | thanks, pabelanger ! | 18:19 |
| tmorin | hi AJaeger yolanda | 18:19 |
| tmorin | it would help a lot if one of you (or perhaps someone else) could have a look at https://review.openstack.org/#/c/291103/ | 18:20 |
| AJaeger | tmorin: please explain what it is and what's important with it | 18:20 |
| tmorin | we need this to go through to enrich DSVM jobs without having to push a change to project-config each and every time | 18:21 |
| tmorin | this is a pretty straightforward job change where we move from builtin devstack parameters to parameters defined in an rcfile in the repo | 18:21 |
| *** dims has quit IRC | 18:21 | |
| *** ybathia has joined #openstack-infra | 18:21 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Rename tox-nodb-ENV to tox-ENV https://review.openstack.org/291349 | 18:21 |
| * AJaeger takes te last emacs reference - I do not know how to use emacs ;( | 18:21 | |
| SamYaple | AJaeger: would you mind reviewing https://review.openstack.org/#/c/289042/ again, i responded to your comment | 18:22 |
| * AJaeger just noticed that I forgot to safe a file for my change | 18:22 | |
| *** annegentle has joined #openstack-infra | 18:22 | |
| *** dims has joined #openstack-infra | 18:22 | |
| *** ihrachys has quit IRC | 18:23 | |
| AJaeger | tmorin: needs as usual a neutron infra liason to +1 | 18:23 |
| *** esker has quit IRC | 18:23 | |
| openstackgerrit | Merged openstack-infra/infra-manual: Remove bare-trusty lines https://review.openstack.org/291241 | 18:23 |
| tmorin | AJaeger: yes, sure, need to find out who can do this, I believe mestery, but maybe more than one person has this role | 18:24 |
| mestery | tmorin: Yes, me or armax or dougwig | 18:25 |
| tmorin | hi kyle | 18:25 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Let Users Add Notes to Tasks https://review.openstack.org/291267 | 18:25 |
| tmorin | I've added the three of you to the change | 18:25 |
| tmorin | kyle, maybe you can have a look right now ? | 18:25 |
| tmorin | that would be awesome :) | 18:25 |
| tmorin | the change is pretty simple | 18:25 |
| AJaeger | tmorin: I commented, the change looks wrong | 18:25 |
| tmorin | as I say above: we just change from builtin devstack parameters to parameters defined in an rcfile in the repo | 18:26 |
| AJaeger | tmorin: at least you do something that is not part of commit message | 18:26 |
| *** asettle has joined #openstack-infra | 18:26 | |
| tmorin | I've replied why I think this is correct | 18:27 |
| tmorin | We really are just moving things in a different place | 18:27 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Rename tox-nodb-ENV to tox-ENV https://review.openstack.org/291349 | 18:28 |
| tmorin | we have the same lines that we remove in the rcfiles recently commited to the repo | 18:28 |
| openstackgerrit | Merged openstack-infra/puppet-openstack_health: Add apache2-utils package to api server https://review.openstack.org/291264 | 18:28 |
| *** jaosorior has joined #openstack-infra | 18:28 | |
| tmorin | AJaeger: see https://github.com/openstack/networking-bgpvpn/blob/master/devstack/devstack-gate-bagpipe-rc | 18:29 |
| AJaeger | tmorin: let's ask sdague to review https://review.openstack.org/#/c/291103, please | 18:29 |
| *** openstackgerrit_ has joined #openstack-infra | 18:30 | |
| tmorin | AJaeger: sure | 18:30 |
| AJaeger | tmorin: it would have helped if you have said in the commit message: REmove bagpipe settings, it's now part of rcfile. | 18:30 |
| *** asettle has quit IRC | 18:30 | |
| *** [1]Thelo has joined #openstack-infra | 18:31 | |
| tmorin | AJaeger: ok, I will update | 18:31 |
| AJaeger | tmorin: right now you only say "enable rcfile so that parameters can be controlled" | 18:31 |
| tmorin | yes, true | 18:31 |
| robcresswell | AJaeger: I finally got to bumping the cisco-ui translation patch. Thankyou for following that up. | 18:31 |
| *** tqtran-afk has joined #openstack-infra | 18:31 | |
| *** degorenko is now known as _degorenko|afk | 18:31 | |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config: net-bgpvpn: control gate settings from rc scripts https://review.openstack.org/291103 | 18:31 |
| tmorin | AJaeger: I can't find sdague email address to add him as a reviewer :-( | 18:32 |
| tmorin | just found it :) | 18:32 |
| *** Thelo has quit IRC | 18:33 | |
| *** [1]Thelo is now known as Thelo | 18:33 | |
| *** tonytan4ever has quit IRC | 18:33 | |
| *** kdas__ has joined #openstack-infra | 18:33 | |
| *** tonytan4ever has joined #openstack-infra | 18:34 | |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config: net-bgpvpn: control gate settings from rc scripts https://review.openstack.org/291103 | 18:34 |
| *** kushal has quit IRC | 18:34 | |
| *** takashi has quit IRC | 18:35 | |
| *** tonytan_brb has joined #openstack-infra | 18:35 | |
| dougwig | tmorin: just looked at it. you got the full range of +1, 0, -1. :) | 18:35 |
| *** nwkarsten has joined #openstack-infra | 18:36 | |
| tmorin | dougwig: I will remove the if lines that are not useful anymore | 18:36 |
| dougwig | tmorin: ok. and is your team too small for the standard two +2's ? | 18:37 |
| tmorin | dougwig: on the single +2 on the dependent change -> the reason is that this dependent change is in a project were we are only two core and the change is proposed by one of them | 18:37 |
| tmorin | dougwig: yes, exactly | 18:37 |
| dougwig | tmorin: figured. ok. | 18:37 |
| tmorin | dougwig: we always apply the "two +2" rule on changes by non-core contributors | 18:38 |
| tmorin | dougwig: :) | 18:38 |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config: net-bgpvpn: control gate settings from rc scripts https://review.openstack.org/291103 | 18:39 |
| AJaeger | EmilienM, pabelanger: I don't understand why https://review.openstack.org/#/c/290906 is needed. What am I missing? | 18:39 |
| tmorin | dougwig, AJaeger: I think I've now addressed your comments on https://review.openstack.org/#/c/291103/ | 18:39 |
| *** tonytan4ever has quit IRC | 18:39 | |
| anteaya | fungi: I wish dyson would get around to creating fans for computers: http://www.dyson.com/air-treatment/cooling-fans.aspx | 18:40 |
| pabelanger | AJaeger: I believe the scripts EmilienM has for the job are expecting the project to be in openstack/puppet-openstack-cookiecutter and not . | 18:40 |
| fungi | anteaya: i know, they have some awesome airfoil-based designs | 18:41 |
| pabelanger | AJaeger: I also think there is the added depends-on support | 18:41 |
| tmorin | I have to go | 18:41 |
| tmorin | bye folks!... | 18:41 |
| *** tmorin has left #openstack-infra | 18:41 | |
| anteaya | fungi: have you considered mounting your workstaation on a block of ice? | 18:41 |
| *** nwkarsten has quit IRC | 18:42 | |
| SamYaple | thanks AJaeger | 18:42 |
| mordred | anteaya: he could also move the workstation to canada | 18:42 |
| mordred | anteaya: I hear you have ice up there | 18:42 |
| *** angdraug has quit IRC | 18:43 | |
| pleia2 | hah | 18:43 |
| SamYaple | ice is a natural sound absorber. thats why it seems so quiet when it snows. it is so quiet! | 18:43 |
| anteaya | mordred: it is melting | 18:43 |
| anteaya | mordred: we have snowbanks still | 18:43 |
| AJaeger | pabelanger: ok, will check again... | 18:44 |
| anteaya | mordred: was considering tapping this season | 18:44 |
| anteaya | but the weather got warm and stayed warm | 18:44 |
| fungi | anteaya: i should submerge it in a mineral oil path circulated through a chiller | 18:44 |
| anteaya | it has to ossilate around zero for a few weeks for good tasting syrup | 18:44 |
| fungi | s/path/bath/ | 18:44 |
| anteaya | fungi: that sounds like it would work | 18:44 |
| anteaya | you may have to wash your hands after every time you type though | 18:45 |
| anteaya | been foggy out my window since monday | 18:45 |
| anteaya | not good syrup weather at all | 18:45 |
| *** esikachev has quit IRC | 18:45 | |
| fungi | anteaya: instead, i've found another barebones system of the same model on surplus somewhere online, so i'm ordering that and then can swap it with the current hardware and send that back to the manufacturer for fan replacement | 18:45 |
| anteaya | fungi: ha ha ha | 18:46 |
| openstackgerrit | Sam Yaple proposed openstack/diskimage-builder: Use fstrim to prep the block device https://review.openstack.org/290944 | 18:46 |
| fungi | since they were disinclined to just tell me the make and model of the fan replacement | 18:46 |
| anteaya | sounds simple and straighforward | 18:46 |
| anteaya | fungi: :( | 18:46 |
| anteaya | and I bet you asked nice | 18:46 |
| *** SumitNaiksatam has left #openstack-infra | 18:46 | |
| fungi | i'm quite capable of replacing the fan cartridge myself, but it has no distinguishing marks on it to indicate who made it, nor even a cryptic sequence of digits i might take to be a model identifier | 18:47 |
| fungi | and because of the tight tolerances in there, it's either get the exact same fan to replace this one, or take power tools to the case and saw a very large hole so i can bolt on something generic | 18:48 |
| *** eharney has quit IRC | 18:48 | |
| fungi | a solution which i'm saving as a last resort | 18:49 |
| SamYaple | anteaya: you ever get maple syrup from the tree and poor it onto ice and then roll that onto a stick? | 18:49 |
| *** amitgandhinz has joined #openstack-infra | 18:49 | |
| SamYaple | its delicious | 18:49 |
| anteaya | maple tafee | 18:49 |
| fungi | SamYaple is now known as SayMaple | 18:49 |
| anteaya | lost a tooth to that when I was a kid | 18:50 |
| *** keedya has joined #openstack-infra | 18:50 | |
| SamYaple | thats the one! | 18:50 |
| rm_work | jeblair: yes, and yes they do, but we're going to look at options again -- just needs to be revisited, and sounds like amrith can give us some pointers here | 18:50 |
| anteaya | the snow has to be the consistentcy of small ice pellets for that to be really good | 18:50 |
| *** yamamoto has joined #openstack-infra | 18:50 | |
| rm_work | (yes we're still figuring out amp images, and yes we build them on every job) | 18:50 |
| amrith | happy to | 18:50 |
| pleia2 | I got some of that when I was in montreal :d | 18:50 |
| SamYaple | thats why i said ice, not snow. there is a sublte difference | 18:50 |
| rm_work | amrith: johnsom is here now | 18:50 |
| anteaya | used to make it every year from last year's syrup while boiling down | 18:50 |
| anteaya | just disappears if you pour it on snow | 18:51 |
| anteaya | fungi: I'm sure you have a spectrum of solutions, hope the manufacturer will insall a fan when you send them the workstation | 18:52 |
| jeblair | pabelanger: looking at the grafana server, i see the puppet run from when it upgraded, but i don't see any errors related to it | 18:52 |
| anteaya | pleia2: yeah that serving was very large in my opinion | 18:52 |
| pabelanger | jeblair: I think we need to restart apache, grafana.o.o is funky | 18:52 |
| anteaya | pleia2: but glad you got a taste | 18:52 |
| jeblair | pabelanger: restarted, no joy | 18:53 |
| pabelanger | Hmm | 18:53 |
| jeblair | pabelanger: i see several errors in my dev console | 18:53 |
| jeblair | a missing css and notably, a missing js file | 18:53 |
| anteaya | rm_work: is there any way you can get amphorae to not build an image on every code change? | 18:53 |
| jeblair | pabelanger: oh there's a grafana server process | 18:54 |
| pleia2 | anteaya: that's right, I was with you then! | 18:54 |
| pabelanger | jeblair: that's right. Forgot about that | 18:54 |
| jeblair | pabelanger: ah. that was easy. :) | 18:54 |
| anteaya | pleia2: :) it was fun | 18:54 |
| jeblair | pabelanger: probably just need to subscribe a restart event to the install in puppet and this would be automatic | 18:54 |
| anteaya | rm_work: from my understanding, building an image on every code change seems to be the root of the issue | 18:55 |
| pabelanger | jeblair: Ya, think so | 18:55 |
| jeblair | jlk: http://grafana.openstack.org/dashboard/db/nodepool-blue-box is up again | 18:55 |
| jlk | thanks | 18:55 |
| pabelanger | need to setup grafana again, and tests elasticsearch backend | 18:56 |
| *** rhallisey has quit IRC | 18:58 | |
| openstackgerrit | Merged openstack-infra/project-config: Give manila-image-elements its own acls https://review.openstack.org/290865 | 18:58 |
| * AJaeger better calls it a day and waves good-bye.... | 18:58 | |
| *** ybathia has quit IRC | 18:58 | |
| *** nwkarsten has joined #openstack-infra | 18:59 | |
| pabelanger | fungi: AJaeger: clarkb: Do we want to do 282878 now? I think we needed shade 1.5.1 for that? | 18:59 |
| *** yamamoto has quit IRC | 18:59 | |
| AJaeger | pabelanger: I want it - but let fungi confirm ;) | 18:59 |
| fungi | okay, i've probably let the processor cool down long enough. i'll fire stuff back up and have a look at that | 19:00 |
| *** cyraxjoe has left #openstack-infra | 19:01 | |
| *** vgridnev has joined #openstack-infra | 19:01 | |
| anteaya | AJaeger: thank you, enjoy your evening | 19:03 |
| *** eharney has joined #openstack-infra | 19:03 | |
| *** nwkarsten has quit IRC | 19:03 | |
| *** pvaneck has joined #openstack-infra | 19:04 | |
| anteaya | clarkb: I've reviewed https://review.openstack.org/#/q/status:open+topic:%255Elogstash.* and they all have fresh jenkins results | 19:04 |
| anteaya | any other infra-root care to review some logstash patches? yolanda I believe has reviewed many | 19:04 |
| *** tonytan_brb is now known as tonytan4ever | 19:07 | |
| *** sarob has joined #openstack-infra | 19:08 | |
| *** ddieterly[away] has quit IRC | 19:09 | |
| pabelanger | anteaya: I can look, but my logstash foo is weak | 19:10 |
| anteaya | pabelanger: thanks it is a lot of puppet | 19:11 |
| anteaya | pabelanger: so your puppet reviewing is greatly appreciated | 19:11 |
| anteaya | and you don't have to review or vote anything you don't feel comfortable voting on | 19:12 |
| anteaya | you can always use +1 as well if that is what you are comfortable with | 19:12 |
| anteaya | glad to have more eyes, thank you | 19:12 |
| anteaya | sometimes the best review is a good clarifying question | 19:13 |
| *** dimtruck is now known as zz_dimtruck | 19:14 | |
| pabelanger | mordred: so, https://github.com/openstack/puppet-ovn is a thing now :) Plan on testing it out with the help of russellb | 19:14 |
| pabelanger | 1 step close to infracloud :p | 19:14 |
| russellb | ++ | 19:14 |
| russellb | pabelanger: let me know how i can help | 19:15 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul: Let zuul-cloner work in post and release pipelines https://review.openstack.org/271585 | 19:15 |
| *** jistr has joined #openstack-infra | 19:15 | |
| pabelanger | russellb: Ya, we'll need ubuntu support for sure. Any issues doing fedora 23? | 19:16 |
| russellb | pabelanger: f23 should be fine | 19:16 |
| pabelanger | cool | 19:16 |
| russellb | pabelanger: puppet may need some tweaks for ubuntu since it hasn't been thested there yet | 19:16 |
| *** dmsimard has quit IRC | 19:17 | |
| russellb | pabelanger: we're also not building the networking-ovn (neutron plugin) as an ubuntu package anywhere yet | 19:17 |
| pabelanger | russellb: Ya, I'm hoping EmilienM will have some CI in place to help gate on the testing | 19:17 |
| russellb | cool | 19:17 |
| EmilienM | russellb: I'll disable beaker tests for ubuntu, since no packaging. No need to waste resources. | 19:18 |
| EmilienM | we are in the middle of a release, I'll put some efforts probably in 2 weeks. | 19:18 |
| EmilienM | to have integration testing with OVN, asap | 19:18 |
| *** akrivoka has quit IRC | 19:19 | |
| russellb | ok | 19:19 |
| russellb | the packaging for OVN itself is done for debian/ubuntu | 19:19 |
| *** nwkarsten has joined #openstack-infra | 19:19 | |
| russellb | just not the neutron plugin | 19:19 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/project-config: Enable heat service in shade jobs https://review.openstack.org/291371 | 19:20 |
| Shrews | mordred: before you get engrossed in Duke v. ND, ^^^^ | 19:21 |
| *** dmsimard has joined #openstack-infra | 19:21 | |
| *** nwkarste_ has joined #openstack-infra | 19:21 | |
| openstackgerrit | Merged openstack/requirements: Exclude python-troveclient 2.1.0 https://review.openstack.org/290935 | 19:22 |
| EmilienM | russellb: I'll start by using packaging in RDO and iterate with Ubuntu if needed. | 19:22 |
| *** ddieterly has joined #openstack-infra | 19:22 | |
| EmilienM | but I'm working on Mitaka release this week and next week so it won't happen before 2 weeks I think | 19:22 |
| russellb | EmilienM: yep no worries | 19:22 |
| russellb | EmilienM: pabelanger fwiw, i have notes about where Fedora/CentOS packages can be found in the packaging section of http://docs.openstack.org/developer/networking-ovn/install.html | 19:23 |
| EmilienM | it's already bookmarked :P | 19:24 |
| *** nwkarsten has quit IRC | 19:24 | |
| *** pcrews_ has joined #openstack-infra | 19:24 | |
| *** zul has quit IRC | 19:25 | |
| *** pcrews has quit IRC | 19:25 | |
| *** zul has joined #openstack-infra | 19:26 | |
| anteaya | rbradfor: nice post on feedback on the bug smash event | 19:26 |
| *** nwkarste_ has quit IRC | 19:26 | |
| anteaya | rbradfor: thanks for attending | 19:26 |
| *** esikachev has joined #openstack-infra | 19:26 | |
| rbradfor | anteaya, thanks for the comments. | 19:27 |
| rbradfor | I actually got a lot from the event as I got to meet 3 openstack regulars for the first time. | 19:27 |
| *** gnuoy_ has joined #openstack-infra | 19:27 | |
| anteaya | well done | 19:28 |
| *** vgridnev has quit IRC | 19:28 | |
| anteaya | rbradfor: how many people would you say attended? | 19:28 |
| *** dims_ has joined #openstack-infra | 19:28 | |
| sc68cal | rbradfor: 15? | 19:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add storyboard exim router/transports https://review.openstack.org/270331 | 19:28 |
| *** ddieterly is now known as ddieterly[away] | 19:28 | |
| * sc68cal crashed the party on monday | 19:28 | |
| anteaya | sc68cal: ah you were there too, very nice | 19:28 |
| *** sobersabre has joined #openstack-infra | 19:28 | |
| rbradfor | and it reminded me of when I started I did all this myself, so there is merit in having beginner guides to help individuals get over the intro hurdle. | 19:28 |
| anteaya | well I don't see how one can crash a bug smash event | 19:28 |
| anteaya | rbradfor: yes, I agree | 19:29 |
| *** jcoufal has quit IRC | 19:29 | |
| anteaya | you made good points there in your post | 19:29 |
| *** dims has quit IRC | 19:29 | |
| sc68cal | ooh, I haven't read it yet. link? | 19:29 |
| rbradfor | sc68cal, anteaya on monday I'd say 15 in the room, and I think 5 of us were full-time contributors. I'd say 8 of the 10 probably had not any experience with OpenStack. It would be good to get these numbers at other sites. | 19:29 |
| anteaya | sc68cal: dev ml | 19:29 |
| *** Sukhdev has joined #openstack-infra | 19:29 | |
| rbradfor | sc68cal, See ML. | 19:29 |
| sc68cal | ack - thx | 19:30 |
| *** rhallisey has joined #openstack-infra | 19:30 | |
| anteaya | sc68cal: http://lists.openstack.org/pipermail/openstack-dev/2016-March/088999.html | 19:30 |
| robcresswell | AJaeger: Does the new translation extraction setup require additional config? Seems to be generating empty .pot files on horizon-cisco-ui. | 19:30 |
| anteaya | interesting | 19:30 |
| anteaya | I wonder how the event was communicated such that folks with no experience were in attendance | 19:31 |
| fungi | pabelanger: AJaeger: on 282878 it looks like nodepool-builder was last restarted at 04:06 utc on tuesday, and shade 1.5.1 was not tagged until 12:57:43 utc on tuesday, so we need a nodepool-builder (and also probably nodepoold) restart before we can rely on that fix | 19:31 |
| anteaya | rbradfor: what other sites were doing an in person event? | 19:31 |
| *** jaosorior has quit IRC | 19:31 | |
| ianw | SamYaple: hopefully that fstrim just needs a sudo, and the ioctl failure isn't actually saying "not supported"? | 19:31 |
| *** jpr has joined #openstack-infra | 19:32 | |
| SamYaple | ianw: dunno havent looked at the gate. will look into it | 19:32 |
| rbradfor | anteaya, well the free tee shirt had a list of locations :) | 19:33 |
| *** andreykurilin__ has quit IRC | 19:33 | |
| anteaya | rbradfor: nice | 19:33 |
| anteaya | found them, they are listed on an etherpad: https://etherpad.openstack.org/p/OpenStack-Bug-Smash-Mitaka | 19:33 |
| rbradfor | anteaya, australia, china, india, russia, united kingdom, united statse (at least) | 19:33 |
| anteaya | cool | 19:34 |
| sobersabre | waynr: can you take a quick look at https://review.openstack.org/#/c/287497/ ? | 19:34 |
| SotK | anteaya: the task notes patch has made it through to storyboard.openstack.org now, if you're interested :) | 19:34 |
| sobersabre | fdegir: have you had a chance to take a look at: https://review.openstack.org/#/c/287497/ ? | 19:34 |
| anteaya | SotK: woooo, I"m interested | 19:34 |
| * anteaya heads to storyboard | 19:34 | |
| sobersabre | fdegir: you said you would, but I guess it's rarely things happen too close to the plan :-| | 19:35 |
| openstackgerrit | Joshua Harlow proposed openstack/requirements: Bump voluptuous to 0.8.9 https://review.openstack.org/291383 | 19:35 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Fix race in tests https://review.openstack.org/290903 | 19:35 |
| pabelanger | fungi: thanks | 19:35 |
| *** dmsimard has quit IRC | 19:36 | |
| pabelanger | ianw: testing centos-minimal again locally, so far, so good | 19:36 |
| *** annegentle has quit IRC | 19:36 | |
| anteaya | SotK: so it is that black square that creates a drop down box? | 19:36 |
| fungi | pabelanger: note that nodepool-builder restarts are disruptive to daily image updates, so probably should happen sometime later on once they're done for the day | 19:36 |
| fungi | i'll try to plan to do one tonight | 19:36 |
| anteaya | SotK: with the prompt, enter task details here? | 19:36 |
| *** annegentle has joined #openstack-infra | 19:36 | |
| pabelanger | fungi: Ack. Ya, I haven't even started looking into the production systems yet. Won't be for a while until I start restarting services :) | 19:37 |
| SotK | anteaya: yeah, I think the square is supposed to be a sticky-note type thing | 19:37 |
| *** dmsimard has joined #openstack-infra | 19:37 | |
| ianw | pabelanger: yeah, you might see some occasional failures, but i believe https://review.openstack.org/#/c/281596/ covers it | 19:37 |
| anteaya | SotK: okay my mind thinks, black square, not sticky note | 19:37 |
| anteaya | SotK: do we have the ability to add a tooltip on hover? | 19:37 |
| pabelanger | ianw: good to know | 19:37 |
| SamYaple | ianw: i dont have alot of time right now so ill just push a patch with sudo for the gate to test it for me | 19:37 |
| *** nwkarsten has joined #openstack-infra | 19:38 | |
| ianw | pabelanger: i spent a while trying to diff rpm-db's to get a true root cause on that, but basically, gave up. too many layers there | 19:38 |
| SamYaple | ianw: worst case is the loop stuff doesnt support discard and we have to use qemu-nbd, but well cross that bridge if we get there | 19:38 |
| ianw | and when it's probably due to version skew, it's too obscure to waste too much time on | 19:38 |
| openstackgerrit | Sam Yaple proposed openstack/diskimage-builder: Use fstrim to prep the block device https://review.openstack.org/290944 | 19:39 |
| SotK | anteaya: sure | 19:39 |
| * SotK prepares a patch | 19:39 | |
| anteaya | SotK: thank you | 19:39 |
| ianw | SamYaple: yeah, that was my concern; it's a good point that we should probably actually capture at least kernel logs for that job, we might have got something useful there | 19:39 |
| ianw | SamYaple: did it work locally? | 19:40 |
| openstackgerrit | Sridhar Ramaswamy proposed openstack-infra/project-config: Remove groups for python-tackerclient https://review.openstack.org/290185 | 19:40 |
| SamYaple | ianw: of course | 19:40 |
| SamYaple | i posted results for you | 19:40 |
| openstackgerrit | Merged openstack-infra/puppet-infracloud: Add examples directory https://review.openstack.org/283794 | 19:40 |
| SamYaple | ianw: i dont push untested patches. thats crazy talk | 19:40 |
| ianw | SamYaple: hmm, what platform | 19:40 |
| *** rcernin has quit IRC | 19:40 | |
| SamYaple | ubuntu (though it was 16.04 preview, i was testing dib xenial stuff) | 19:41 |
| ianw | hmm, that's going to be a much later kernel than trusty... | 19:41 |
| SamYaple | yea 4.4 vs 3.13 | 19:41 |
| SamYaple | the kernel certainly supports it for qemu-nbd | 19:41 |
| SamYaple | maybe no losetup | 19:41 |
| SamYaple | if yo urecall i commented i was suprised it worked in the comments | 19:42 |
| ianw | that doesn't fill me with hope that it's not a kernel issue :) | 19:42 |
| SamYaple | my point is even if it is kernel related, there is still a path forward | 19:42 |
| *** e0ne has quit IRC | 19:43 | |
| *** Sukhdev has quit IRC | 19:43 | |
| ianw | i'm just happy the tests found something for once, rather than just being a PITA :) | 19:43 |
| jeblair | mordred, nibalizer: https://review.openstack.org/270331 has just the right amount of whitespace, quotation marks, and backslashes in just the right places now. | 19:44 |
| openstackgerrit | Aleksandra Fedorova proposed openstack-infra/project-config: Add missing delay-release option in Fuel projects https://review.openstack.org/291390 | 19:45 |
| ianw | what's going to be the best way to find logs for periodic jobs? | 19:45 |
| jeblair | ianw: http://logs.openstack.org/periodic/ | 19:45 |
| *** armax has quit IRC | 19:46 | |
| fungi | ianw: i think there's an intention to get the health dashboard to start providing a more convenient index to those at some point | 19:46 |
| anteaya | is there? | 19:47 |
| *** _nadya_ has joined #openstack-infra | 19:47 | |
| *** IlyaG has quit IRC | 19:47 | |
| *** nelsnelson has quit IRC | 19:47 | |
| jeblair | true. i actually answered the question "what is currently the best way to find logs for the afs build job?" | 19:47 |
| jeblair | there's a whole multiverse of possible answers to the question as written. :) | 19:48 |
| fungi | indeed. it's all about relative spacetime, after all | 19:49 |
| fungi | i wasn't clear on ianw's current relativistic frame of reference | 19:49 |
| *** baoli has quit IRC | 19:50 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove final bare-trusty job https://review.openstack.org/291234 | 19:50 |
| fungi | AJaeger: ^ boom! | 19:50 |
| SamYaple | its finished | 19:50 |
| SamYaple | ring has been dropped into the lava | 19:50 |
| *** SumitNaiksatam has joined #openstack-infra | 19:50 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Add a tooltip when hovering on the "Task notes" button https://review.openstack.org/291391 | 19:50 |
| fungi | SamYaple: if only. we still have jobs running on bare-precise :/ | 19:50 |
| jeblair | https://jenkins.openstack.org/job/perodic-package-afs-centos7 | 19:50 |
| jeblair | this does not exist yet | 19:51 |
| pabelanger | fungi: nice! | 19:51 |
| jeblair | are we stuck on jjb updates? | 19:51 |
| SamYaple | fungi: give it a year that ubuntu will stop releaseing seucirty patcheso n that one | 19:51 |
| fungi | jeblair: i hope not. jenkins.o.o usually manages to update even if nothing else does | 19:51 |
| * mordred is excited about the death of bare-trusty | 19:52 | |
| pabelanger | looking forward to centos-7 next | 19:52 |
| pabelanger | well, not the death of | 19:52 |
| *** ayoung has quit IRC | 19:52 | |
| fungi | mordred: we still need a nodepool-builder restart for shade 1.5.1 | 19:53 |
| fungi | so we're not building these in rax at the moment | 19:53 |
| anteaya | SotK: thank you | 19:53 |
| pabelanger | ianw: centos7-minimal dib successful, nice work | 19:53 |
| fungi | ianw: jeblair: i don't see that job in project-config | 19:53 |
| mordred | fungi: I restarted it the other day after having applied the patch by hand - did it get restarted again with stock 1.5.0? | 19:53 |
| jeblair | fungi: https://review.openstack.org/290214 hrm | 19:54 |
| fungi | mordred: oh! i forgot about that. i was comparing the creation time on the pidfile to your tag time | 19:54 |
| jeblair | fungi: ah typo | 19:54 |
| anteaya | SotK: next question, after I have added a note, where should I be able to view the contents of the note? | 19:54 |
| jeblair | fungi: there it is https://jenkins.openstack.org/job/periodic-package-afs-centos7/ | 19:54 |
| mordred | fungi: I mean, we SHOULD restart to pick up 1.5.1 so that we're not running on a hand-patched file | 19:54 |
| anteaya | SotK: thus far I have not figured out where I should be looking | 19:54 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Add common bash functions to help track metrics. https://review.openstack.org/291392 | 19:55 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Metrics tracking for TripleO deployment tasks https://review.openstack.org/291393 | 19:55 |
| *** esker has joined #openstack-infra | 19:55 | |
| mordred | fungi: builder is ready for a restart any time | 19:55 |
| fungi | mordred: so anyway, it's your assessment that we should be safe to merge 282878? want to +3 if i un-wip? | 19:55 |
| *** gyee has quit IRC | 19:56 | |
| mordred | fungi: yes. shall I restart the builders on actual 1.5.1 ? | 19:56 |
| *** moravec has quit IRC | 19:56 | |
| fungi | mordred: it's safe to do that even if image updates are underway? | 19:57 |
| mordred | fungi: oh - are we updating images right now? no, I do not think it is safe to restart | 19:57 |
| SotK | anteaya: so far, its just in the same place as you added it | 19:57 |
| * fungi doesn't know how long our image updates go these days | 19:57 | |
| *** rguillebert has quit IRC | 19:57 | |
| SotK | (at the moment its just a single big text field for putting notes in) | 19:57 |
| *** ildikov has quit IRC | 19:58 | |
| mordred | fungi: there are images in building state | 19:58 |
| anteaya | SotK: so I have to click on the button to see what I added? | 19:58 |
| * anteaya just heard a very loud noise of unknown origin | 19:58 | |
| *** sshnaidm_ has joined #openstack-infra | 19:58 | |
| *** sshnaidm has quit IRC | 19:59 | |
| *** hashar has joined #openstack-infra | 19:59 | |
| anteaya | SotK: so I do, okay | 19:59 |
| fungi | mordred: anyway, that change is ready for approval | 19:59 |
| anteaya | hmmmmm, is there a larger vision for this feature? | 19:59 |
| fungi | i added relevant comments | 19:59 |
| ianw | jeblair: yeah, i was wondering about the latest run ... i'd like to add it to my nodepool build monitoring script. i feel like scraping dates is wrong | 20:00 |
| mordred | fungi: +A | 20:00 |
| SotK | I'd like an arbitrary number of notes to be able to be added eventually | 20:00 |
| ianw | fungi: it's 7am, so relativistic time is best described as "still a little blurry" | 20:00 |
| SotK | idk what other people's views on the feature are though | 20:00 |
| *** ildikov has joined #openstack-infra | 20:00 | |
| anteaya | well I'd like to be able to see from storyview if any task have notes | 20:01 |
| fungi | ianw: i like to refer to that as "time dilation" | 20:01 |
| anteaya | so I know if clicking the button is worth my time | 20:01 |
| *** rhallisey has quit IRC | 20:01 | |
| anteaya | so if the button changed shape or colour if it had content in the form of a note | 20:02 |
| *** rhallisey has joined #openstack-infra | 20:02 | |
| anteaya | so some other way of saying clicking this button is worth it | 20:02 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade: Add update_service() https://review.openstack.org/287504 | 20:03 |
| anteaya | SotK: and adding multiple notes makes sense to me | 20:03 |
| anteaya | or the ability to do so | 20:03 |
| * SotK makes a patch to change colour if there are notes | 20:04 | |
| anteaya | SotK: thanks :) | 20:04 |
| anteaya | I appreciate you listening to me | 20:04 |
| anteaya | :) | 20:04 |
| SotK | you're welcome :) | 20:05 |
| *** sdake has quit IRC | 20:05 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/glean: Place glean.lock in working dir of script https://review.openstack.org/291399 | 20:06 |
| yolanda | mordred, what do you think about that? ^ | 20:06 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/reviewstats: Support passing just a single project name https://review.openstack.org/291401 | 20:06 |
| mriedem | russellb: ^ do i get ATC cred for that? | 20:06 |
| russellb | sure! | 20:06 |
| mriedem | this is all new to me | 20:07 |
| openstackgerrit | yolanda.robla proposed openstack-infra/glean: Place glean.lock in working dir of script https://review.openstack.org/291399 | 20:07 |
| mriedem | was trying to find a good way to get reviewstats on stable branches | 20:07 |
| openstackgerrit | yolanda.robla proposed openstack-infra/glean: Place glean.lock in working dir of script https://review.openstack.org/291399 | 20:08 |
| russellb | mriedem: i feel like it did that at one point ... | 20:08 |
| mriedem | russellb: it does, but only for stable-maint-core | 20:08 |
| russellb | i see | 20:08 |
| mriedem | russellb: so if i want to see just stable reviews on nova, it needs some work | 20:09 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Add CentOS AFS build job to project-config experimental https://review.openstack.org/291402 | 20:09 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: [WIP] Remove CentOS AFS build experimental job https://review.openstack.org/291403 | 20:09 |
| russellb | mriedem: got it | 20:09 |
| mriedem | plus it doesn't handle gerrit group hierarchy | 20:09 |
| russellb | mriedem: sounds good to me, left a comment though | 20:09 |
| russellb | mriedem: it doesn't? lame on my part | 20:09 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Highlight the notes button if a task has notes https://review.openstack.org/291404 | 20:09 |
| * anteaya reviews | 20:09 | |
| mriedem | russellb: it probably never needed to | 20:09 |
| mriedem | and arguably doesn't need to... | 20:10 |
| mriedem | i.e. stable-maint-core is included in nova-stable-maint, but i probably care more about nova-stable-maint for just nova reviews | 20:10 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade: Test v3 params on v2.0 endpoint; Add v3 unit https://review.openstack.org/289083 | 20:10 |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade: Update create_endpoint() https://review.openstack.org/289051 | 20:10 |
| russellb | mriedem: ok, i could go either way ... i'd probably expect it to show everything included | 20:11 |
| *** e0ne has joined #openstack-infra | 20:12 | |
| SotK | anteaya: thanks :) | 20:12 |
| SamYaple | ianw: sudo took care of it. gate is happy | 20:12 |
| *** mrmartin has joined #openstack-infra | 20:13 | |
| fungi | so with the last bare-trusty using job out of our config, i should plan to make sure our jenkins masters are completely up to date later this evening and then restart zuul when it's quiet, so we no longer have any remaining bare-trusty registrations in gearman | 20:14 |
| clarkb | and update nodepool config | 20:14 |
| fungi | yeah, it's already dialed down to min-ready: 0 | 20:15 |
| fungi | as of that last change | 20:15 |
| *** Swami has quit IRC | 20:15 | |
| ianw | SamYaple: nice. i also have https://review.openstack.org/#/c/289678 up so that we can pull some longer-term graphs of image size over time | 20:15 |
| SamYaple | hey clarkb if you want to run with this patchseries https://review.openstack.org/#/c/290944/ and see if it generates the images you want, i would be interested in seeing how it compares | 20:15 |
| SamYaple | ianw: yay i like data | 20:15 |
| fungi | clarkb: there's a wip change to remove the images, but i wanted to wait at least a couple more days in case we encounter a "zomg must switch back" issue we haven't spotted yet | 20:15 |
| *** |-paul-| has joined #openstack-infra | 20:16 | |
| fungi | so that we don't have to wait for new images to build in that case | 20:16 |
| clarkb | fungi: ya if we set max-servers yo 0 we will still build images | 20:16 |
| *** aeng has joined #openstack-infra | 20:16 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Temporarily stop booting ubuntu-trusty in rax-.*" https://review.openstack.org/282878 | 20:17 |
| *** rguillebert has joined #openstack-infra | 20:18 | |
| anteaya | clarkb: oh the -1 keeps the images from being built | 20:18 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul: Let zuul-cloner work in post and release pipelines https://review.openstack.org/271585 | 20:19 |
| ianw | SamYaple: yeah, the gate failures were due to the image switch and mysql users. should be fixed now | 20:19 |
| *** austin81 has joined #openstack-infra | 20:19 | |
| *** matrohon has joined #openstack-infra | 20:19 | |
| *** armax has joined #openstack-infra | 20:20 | |
| SamYaple | i am always teetering on the edge of staying away from infra and falling into the abyss and keeping up with the goings-ons around here | 20:21 |
| fungi | SamYaple: there is no shallow end in our pool | 20:21 |
| SamYaple | if i knew more about this I would be able to tell if that was a pun or not | 20:22 |
| mriedem | russellb: is it just me or does the default reviewstats run for nova take a looon time | 20:22 |
| mriedem | *loooong | 20:22 |
| russellb | yes it takes a very long time | 20:22 |
| russellb | especially the first time you run it | 20:22 |
| mriedem | heh, i'll have to remember not to run that | 20:22 |
| russellb | it keeps a cache that should speed it up | 20:22 |
| russellb | after it finishes the first time | 20:22 |
| mriedem | ok, i've been running these in venvs | 20:22 |
| fungi | SamYaple: i just like making cryptic comments. i can be safely ignored ;) | 20:22 |
| russellb | ah... | 20:22 |
| russellb | mriedem: the whole thing is a hack | 20:22 |
| russellb | mriedem: it downloads the entire review history of a project... | 20:22 |
| mriedem | i have aspirations to contribute | 20:22 |
| mriedem | wow | 20:23 |
| SamYaple | fungi: you are a cipher wrapped in an enigma | 20:23 |
| russellb | which you can imagine hasn't scaled amazingly | 20:23 |
| mriedem | i thought it was just going back 14 days | 20:23 |
| *** kzaitsev_mb has joined #openstack-infra | 20:23 | |
| *** zz_dimtruck is now known as dimtruck | 20:23 | |
| *** e0ne has quit IRC | 20:23 | |
| russellb | mriedem: i think it downloads everything anyway, to populate its cache to work on | 20:23 |
| *** _nadya_ has quit IRC | 20:24 | |
| russellb | which is just a pickled dict written to disk heh | 20:24 |
| mriedem | yeah, when i did just --stable=liberty it was faster | 20:24 |
| mriedem | mmm pickled dict | 20:24 |
| russellb | ikr? | 20:24 |
| *** ociuhandu has quit IRC | 20:24 | |
| russellb | pickles are gross though | 20:24 |
| fungi | russellb: mriedem: if the struggle to get stackalytics.openstack.org into production ever ends, the hope is that we can try to get whatever statistical features and queries we're missing reimplemented there | 20:24 |
| russellb | fungi: yep | 20:24 |
| mriedem | russellb: i have a jar in my fridge that begs to differe | 20:24 |
| russellb | i haven't worked on reviewstats other than approving people's info updates for the past very long time | 20:24 |
| russellb | pretty low cost to keep it running in the meantime | 20:25 |
| clarkb | I have a tasty pickle with kunch | 20:25 |
| clarkb | *lunch | 20:25 |
| russellb | i'm sorry | 20:25 |
| fungi | i have a pickled plum for breakfast every morning. they're awesome | 20:25 |
| *** annegentle has quit IRC | 20:26 | |
| fungi | had pickled red onions on my pork belly tacos at dinner last night too | 20:26 |
| fungi | don't think i could get by without pickles | 20:27 |
| russellb | it's like i don't even know you people | 20:27 |
| *** annegentle has joined #openstack-infra | 20:27 | |
| russellb | (i don't actually feel this strongly about pickles) | 20:27 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove one bare-trusty line from projects.yaml https://review.openstack.org/291412 | 20:27 |
| dmsimard | clarkb: so I know what the issue is for the swap review | 20:28 |
| anteaya | my grandmother made mustard pickles all the time | 20:28 |
| dmsimard | But I'm having a hard time finding a non-hacky way to fix it | 20:28 |
| anteaya | they were my favourite | 20:28 |
| *** esker has quit IRC | 20:28 | |
| dmsimard | Basically the swap amount, even if it's 8192MB can be reported as low as ~7700MB but it's not a set number | 20:28 |
| *** ddieterly[away] is now known as ddieterly | 20:29 | |
| fungi | AJaeger: i have a feeling we'll need to do a few cleanup patches like 291412 for the next... while | 20:29 |
| clarkb | dmsimard: is the issue we check amiunt of swap? | 20:29 |
| clarkb | ya | 20:29 |
| dmsimard | clarkb: yeah | 20:29 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/reviewstats: Support passing just a single project name https://review.openstack.org/291401 | 20:29 |
| clarkb | dmsimard: I would write a file out that acts as a guard | 20:29 |
| dmsimard | so, I mean, I can add a "fuzz" factor (say 500MB) and then compare "$swapcurrent -lt $(( 8192 - fuzz))" but that feels hacky | 20:29 |
| clarkb | dmsimard: maybe use the swapfile for that | 20:30 |
| dmsimard | oh, hey, that's not a bad idea. Sort of a lock file ? | 20:30 |
| *** Sukhdev has joined #openstack-infra | 20:30 | |
| clarkb | if not exists swapfile then setup swap | 20:30 |
| clarkb | dmsimard: ya | 20:30 |
| *** esikachev has quit IRC | 20:30 | |
| dmsimard | but sometimes we use the ephemeral disk too. | 20:30 |
| *** esker has joined #openstack-infra | 20:30 | |
| clarkb | ya write a 0 byte file in that case | 20:30 |
| dmsimard | yeah, I can create a dummy file. | 20:30 |
| dmsimard | good idea ! | 20:31 |
| clarkb | and dony swapon | 20:31 |
| austin81 | mtreinish, pabelanger: Did y'all manage to come to a consensus on puppet vs dib for stackviz deployment? | 20:31 |
| anteaya | clarkb: can this go in yet? https://review.openstack.org/#/c/247857/ | 20:32 |
| anteaya | clarkb: I think everyone is waiting for you to say what goes in in what order | 20:32 |
| anteaya | the decouple log processing from logstash patch | 20:33 |
| jesusaur | when did we start enforcing subunit sizes <50MB? and what's the reasoning behind that? | 20:33 |
| clarkb | anteaya: that one goes in first, then when it is applied everywhere we merge the two other changes (those have to go in together) | 20:33 |
| clarkb | anteaya: my preference would be to not merge anything until it all has the necessary +2s | 20:33 |
| anteaya | clarkb: the other two changes you offered? | 20:33 |
| clarkb | anteaya: yes | 20:34 |
| anteaya | install logstash and compat ES output rule | 20:34 |
| clarkb | yup | 20:34 |
| mtreinish | austin81: consensus, I don't think so, or at least I don't rememeber ever having that. :) | 20:34 |
| anteaya | okay will see if I can round up some +2's | 20:34 |
| clarkb | anteaya: that way we can blaze through and get it all done at once | 20:34 |
| clarkb | anteaya: and avoid getting it half done today and finished in a couple weeks | 20:34 |
| anteaya | clarkb: yup, I'm all for helping you blaze through | 20:34 |
| anteaya | clarkb: yup | 20:35 |
| anteaya | pabelanger: you rechecked and the jenkins jobs passed now, care to review | 20:35 |
| anteaya | Logstash 2.0 compat ES output rule | 20:35 |
| *** esker has quit IRC | 20:35 | |
| anteaya | https://review.openstack.org/#/c/247714/ | 20:35 |
| anteaya | newlines | 20:35 |
| *** esker has joined #openstack-infra | 20:35 | |
| *** esker has quit IRC | 20:36 | |
| anteaya | pabelanger: and thanks for reviewing the puppet code on install logstash 2.0: https://review.openstack.org/#/c/247756/ | 20:36 |
| austin81 | mtreinish: Gotcha, just checking :) I won't be around next week for infra meeting so I was trying to get it pushed along now if possible | 20:36 |
| *** erlon has quit IRC | 20:36 | |
| austin81 | mtreinish: or at least understand the debate better | 20:37 |
| *** IlyaG has joined #openstack-infra | 20:37 | |
| anteaya | any infra-root in addition to pabelanger able to review 247756 and 247714 ? | 20:37 |
| anteaya | pretty please? | 20:37 |
| nibalizer | sure | 20:37 |
| anteaya | thanks | 20:37 |
| clarkb | jesusaur: I think the reasoning is to keep subunit2sql sane? jesusaur large subunit files take a lot of time to process and use up more room in the database? | 20:37 |
| mtreinish | austin81: personally I would move forward with dib, I still don't understand how writing a puppet module to run 1 build/install command is going to solve getting the output on the nodepool images | 20:38 |
| *** angdraug has joined #openstack-infra | 20:38 | |
| mtreinish | austin81: you're going to have to use dib to get the content on there. Adding a puppet module doesn't make any sense to me | 20:38 |
| fungi | well, we apply some puppet to our images from a dib element, but we'd like to continue to scale that back | 20:39 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/devstack-gate: Use dd to create swapfile, ensure swap amount == 8GB https://review.openstack.org/290776 | 20:40 |
| fungi | the challenge with doing it only in dib is that we're not yet to the point where devstack-based jobs are running on dib-built images in every provider (rackspace is a holdout because of our need to switch to simple-init/glean for network configuration) | 20:40 |
| mtreinish | fungi: well, it was dib and the equiv build script | 20:40 |
| waynr | sobresabre: okay, I have reviewed and given workflow +1, should be merged soonish | 20:40 |
| mtreinish | fungi: I believe austin81 has a patch up already | 20:40 |
| SamYaple | ianw: alot is slowly becoming an accepted word. im making sure of it just to prove my 2nd grade teacher wrong | 20:41 |
| austin81 | fungi: https://review.openstack.org/#/c/279317/ | 20:41 |
| fungi | but i have hopes that we're at most weeks away from moving all devstack-trusty jobs to ubuntu-trusty and then we will be using dib in rackspace for the platform running devstack jobs | 20:41 |
| dmsimard | SamYaple: http://i.imgur.com/hUthyS6.jpg | 20:41 |
| SamYaple | dmsimard: if thats a picutre of a fuzzy animal thing, thats my spirit animal | 20:42 |
| austin81 | fungi: spec at https://review.openstack.org/#/c/287373/ | 20:42 |
| *** esikachev has joined #openstack-infra | 20:42 | |
| SamYaple | it is. thats the one. thanks dmsimard | 20:42 |
| dmsimard | SamYaple: :p it comes from http://hyperboleandahalf.blogspot.ca/2010/04/alot-is-better-than-you-at-everything.html I think | 20:42 |
| jesusaur | clarkb: hm, my zuul-cloner change just barely went over that limit | 20:43 |
| clarkb | jesusaur: oh it shouldn't affect zuul | 20:43 |
| SamYaple | yes dmsimard im ok with this situation | 20:43 |
| fungi | SamYaple: the alot is your spirit animal? i love this alot | 20:43 |
| clarkb | jesusaur: or is the check fialing the job? | 20:43 |
| AJaeger | #success All jobs moved from bare-trusty to ubuntu-trusty | 20:43 |
| openstackstatus | AJaeger: Added success to Success page | 20:43 |
| SamYaple | fungi: indeed | 20:43 |
| anteaya | AJaeger: congratulations | 20:44 |
| nibalizer | clarkb: its safe to land the logstash changes? | 20:44 |
| fungi | a success among successes to be sure | 20:44 |
| *** nwkarsten has quit IRC | 20:44 | |
| clarkb | nibalizer: it is safe to land the first one (would prefer we land that only when the other two have the +2s) | 20:44 |
| clarkb | nibalizer: the other two aren't super safe because there is no forward/backward compatibile path so we have to coordinate it tightly | 20:45 |
| *** IlyaG has quit IRC | 20:45 | |
| clarkb | let me put a paste up | 20:45 |
| anteaya | clarkb: which is the first one? Decouple log processing from logstash? https://review.openstack.org/#/c/247857/ | 20:45 |
| *** sbalukoff has quit IRC | 20:45 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove one bare-trusty line from projects.yaml https://review.openstack.org/291412 | 20:45 |
| anteaya | I think that is the first one | 20:46 |
| clarkb | anteaya: yes decouple is first | 20:46 |
| jesusaur | clarkb: yeah, the subunit filesize check failed the job, and on recheck the filesize was under but an alarm clock failed the job :( | 20:46 |
| anteaya | is it safe to land yet? | 20:46 |
| clarkb | nibalizer: anteaya http://paste.openstack.org/show/490068/ | 20:46 |
| clarkb | yes I am fine if the decoupling changes lands now and the other two have +2s | 20:47 |
| anteaya | woooo | 20:47 |
| jeblair | clarkb, mordred: are we ready to think about nodepool shade? | 20:47 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: upd google-cloud-storage plugin support https://review.openstack.org/287497 | 20:47 |
| openstackgerrit | Billy Olsen proposed openstack-infra/project-config: Delay release for charm projects https://review.openstack.org/291421 | 20:47 |
| clarkb | jeblair: does it pass tests? I havne't checked in on it recently | 20:47 |
| mordred | jeblair: YES | 20:48 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Added npm-install macro to swagger build job https://review.openstack.org/291422 | 20:48 |
| mordred | jeblair: I need to rebase the patch after having pushed the recent logging changes | 20:48 |
| anteaya | nibalizer: care to land 247857? | 20:48 |
| mordred | jeblair: lemme do that real quick | 20:48 |
| jeblair | mordred: okay. cool, i think this might be a good time to try it. | 20:49 |
| jeblair | we can handle some nodepool restarts at our load. | 20:49 |
| AJaeger | thanks everybody for reviewing these ubuntu-trusty changes in quick time! | 20:50 |
| * AJaeger disappears again... | 20:50 | |
| nibalizer | anteaya: sure, if clarkb is ready | 20:50 |
| anteaya | AJaeger: thanks for doing the hard work of shepherding them | 20:50 |
| anteaya | nibalizer: well as long as the other two patches are reviewed and ready to be merged soon after | 20:50 |
| clarkb | nibalizer: anteaya https://review.openstack.org/#/c/247756/4 has two +2's so I think we are ready ot merge its parent | 20:51 |
| clarkb | nibalizer: so go ahead :0 | 20:51 |
| clarkb | er :) | 20:51 |
| nibalizer | i think i've reviewd the things? | 20:51 |
| *** denisra has joined #openstack-infra | 20:51 | |
| clarkb | nibalizer: you did, thats what I was saying, I am ready because you and pabelanger have reviewed | 20:51 |
| anteaya | nibalizer: https://review.openstack.org/#/c/247714/ | 20:51 |
| clarkb | nibalizer: and you saw the paste above right? outlines the rough process | 20:51 |
| nibalizer | clarkb: ya i saw | 20:51 |
| nibalizer | you're on those steps? | 20:51 |
| anteaya | we still need two +2's on https://review.openstack.org/#/c/247714/ | 20:51 |
| clarkb | nibalizer: yup I will work through it | 20:51 |
| nibalizer | ok | 20:52 |
| nibalizer | im around and can hop on if things go sidewase | 20:52 |
| sobersabre | waynr: thanks for the merge!! | 20:52 |
| clarkb | plan is after first change merges and applies to disable puppet in ~5 at a time and update them in chunks | 20:52 |
| clarkb | that way we should never stop processing events | 20:52 |
| sobersabre | finally I can remove all these cherry-picks I'm doing and use master :) | 20:52 |
| *** yamahata has quit IRC | 20:52 | |
| anteaya | and another system-config core in addition to nibalizer, if you could review https://review.openstack.org/#/c/247714/ please | 20:52 |
| clarkb | anteaya: that one I am almost comfortable self approving since its just a small config change and not a large refactor | 20:53 |
| anteaya | clarkb: oh okay fine, I'll stop now | 20:53 |
| anteaya | you have what you need | 20:53 |
| anteaya | thank you all | 20:53 |
| jeblair | it also has some previous +2s | 20:53 |
| anteaya | even better | 20:54 |
| *** moravec has joined #openstack-infra | 20:55 | |
| *** matrohon has quit IRC | 20:55 | |
| pabelanger | anteaya: clarkb reviewed | 20:55 |
| pabelanger | But will yield approval to somebody else :) | 20:55 |
| openstackgerrit | Merged openstack-infra/reviewstats: Support passing just a single project name https://review.openstack.org/291401 | 20:56 |
| clarkb | pabelanger: I can take care of approval, thank you | 20:56 |
| *** hdd has joined #openstack-infra | 20:56 | |
| *** esker has joined #openstack-infra | 20:56 | |
| clarkb | nibalizer: do we have a working "run puppet when emergency file says don't run puppet" playbook thing yet | 20:57 |
| clarkb | that was being worked on during the sprint and I don't recall if we finished that | 20:57 |
| clarkb | (I will likely need that for this) | 20:57 |
| anteaya | pabelanger: thanks | 20:57 |
| nibalizer | clarkb: monty has some code up | 20:57 |
| clarkb | actually | 20:58 |
| mordred | what did I do? | 20:58 |
| mordred | yah. it' smerged | 20:58 |
| clarkb | I don't need it, I can put all the hosts in the emergency file, wait for the module to update on puppetmaster, then remove a handful of them at a time from the emergency file and they will auto update | 20:58 |
| fungi | weather's too nice to spend all afternoon in front of a computer. i'm going out for a quick walk but should be back in roughly an hour | 20:58 |
| openstackgerrit | Merged openstack-infra/puppet-subunit2sql: Switch use of cStringIO.StringIO to io.BytesIO https://review.openstack.org/286926 | 20:59 |
| anteaya | fungi: enjoy | 20:59 |
| openstackgerrit | Merged openstack-infra/puppet-subunit2sql: Remove inaccurate comment https://review.openstack.org/286932 | 20:59 |
| mordred | well, if you nede the other thing - remote_puppet_adhoc.yaml with a --limit argument is your friend | 20:59 |
| mordred | clarkb: ^^ | 20:59 |
| clarkb | mordred: remote_puppet_adhoc.yaml ? | 20:59 |
| openstackgerrit | Merged openstack-infra/system-config: Move statusbot_auth_nicks to hiera https://review.openstack.org/274257 | 20:59 |
| clarkb | mordred: thanks | 20:59 |
| anteaya | weather is overcast and foggy here but was thinking of doing the same | 20:59 |
| pabelanger | any core like to review 291402? It will help ianw with testing afs builds on centos | 20:59 |
| mordred | clarkb: yup! | 20:59 |
| openstackgerrit | Merged openstack-infra/system-config: Add storyboard exim router/transports https://review.openstack.org/270331 | 20:59 |
| openstackgerrit | Merged openstack-infra/puppet-log_processor: Decouple log processing from logstash https://review.openstack.org/247857 | 21:00 |
| *** denisra has quit IRC | 21:00 | |
| *** rfolco has quit IRC | 21:00 | |
| lifeless | dhellmann: we fixed the pbr jobs; you may need to recheck | 21:00 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 21:00 | |
| clarkb | pabelanger: done | 21:01 |
| *** weshay has quit IRC | 21:01 | |
| pabelanger | clarkb: danke | 21:01 |
| *** esikachev has quit IRC | 21:01 | |
| *** amrith is now known as _amrith_ | 21:01 | |
| *** esikachev has joined #openstack-infra | 21:01 | |
| *** rossella_s has quit IRC | 21:03 | |
| *** rossella_s has joined #openstack-infra | 21:03 | |
| *** sputnik13 has joined #openstack-infra | 21:04 | |
| clarkb | nibalizer: hrm any idea why a good chunk of the logstash machines are reporting unreprted? | 21:04 |
| clarkb | nibalizer: in puppetbaord | 21:05 |
| tristanC | ttx: is it time to tag governance repository with march-2016-elections ? | 21:05 |
| nibalizer | that likely means they fail | 21:05 |
| nibalizer | iirc they are timing out? | 21:05 |
| clarkb | nibalizer: huh | 21:05 |
| * clarkb pulls up ansible logs | 21:05 | |
| anteaya | tristanC: I think he would be offline by now, would he not? | 21:06 |
| nibalizer | 2016-03-10 06:53:45,822 p=17767 u=root | fatal: [logstash-worker02.openstack.org]: FAILED! => {"changed": false, "failed": true, "msg": "/usr/bin/timeout -s 9 30m /usr/bin/puppet apply --detailed-exitcodes --logdest syslog --no-noop /opt/system-config/production/manifests/site.pp failed with return code: 4", "rc": 4, "stderr": "Faraday: you may want to install system_timer for reliable timeouts\n", | 21:06 |
| nibalizer | "stdout": "", "stdout_lines": []} | 21:06 |
| clarkb | failed with return code: 4" | 21:06 |
| tristanC | anteaya: you're right, he is probably afk, do you know who can tag the repository ? | 21:06 |
| clarkb | ya will check syslog | 21:06 |
| nibalizer | ya | 21:06 |
| anteaya | tristanC: I think he is the one who does it | 21:06 |
| nibalizer | maybe we need the system_timer gem | 21:06 |
| clarkb | it is entirely possible that this set of changes fixes it if logstash stopped publishing the old stuff or similar | 21:06 |
| clarkb | nibalizer: I think the timer thing is mostly noise | 21:06 |
| anteaya | tristanC: even if someone else did have permissions, I can't see them wanting to tag governance without ttx's say so | 21:07 |
| *** sdake has joined #openstack-infra | 21:07 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 21:07 | |
| clarkb | dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem | 21:07 |
| clarkb | dpkg is unhappy, I will work on fixing | 21:07 |
| *** dtardivel has quit IRC | 21:08 | |
| *** kencjohnston has joined #openstack-infra | 21:08 | |
| openstackgerrit | Merged openstack-infra/project-config: Add CentOS AFS build job to project-config experimental https://review.openstack.org/291402 | 21:09 |
| openstackgerrit | Merged openstack-infra/project-config: Remove groups for python-tackerclient https://review.openstack.org/290185 | 21:09 |
| *** abregman has joined #openstack-infra | 21:09 | |
| openstackgerrit | Ryan Brandt proposed openstack-infra/project-config: Use TEMPEST_SERVICES variable to trigger monasca tempest tests https://review.openstack.org/285577 | 21:10 |
| *** tzn has quit IRC | 21:10 | |
| *** tzn has joined #openstack-infra | 21:11 | |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting job for Cinder Functional tests https://review.openstack.org/287115 | 21:11 |
| tristanC | anteaya: oh well it's mostly for the mail announcement link, i'll send ttx a reminder mail to be sure it's tagged soon | 21:12 |
| *** krtaylor has quit IRC | 21:13 | |
| *** jpr has quit IRC | 21:13 | |
| HeOS | Hello! Could you please review the following request about updates of jeepyb? https://review.openstack.org/#/c/284843/ | 21:14 |
| anteaya | well that and to ensure the state of the repo is static at a certain point in time | 21:14 |
| openstackgerrit | Deklan Dieterly proposed openstack-infra/project-config: Add pre_test_hook to freezer api gate job https://review.openstack.org/291430 | 21:14 |
| anteaya | but I'm sure he will tag as soon as he knows to do so | 21:14 |
| ttx | tristanC: I'm tagging as we speak, processing the las tchanges first | 21:14 |
| anteaya | ttx: ah there you are | 21:14 |
| anteaya | how fortuitous | 21:15 |
| ttx | crazy heh | 21:15 |
| tristanC | ttx: neat! thank :) | 21:15 |
| anteaya | so glad you are staring at a screen at this time of night | 21:15 |
| anteaya | lucky us | 21:15 |
| ttx | tristanC: should I tag march-2016-elections, april-2016-elections ? | 21:15 |
| openstackgerrit | Merged openstack-infra/jeepyb: Remove argparse from requirements https://review.openstack.org/270409 | 21:15 |
| *** ldnunes has quit IRC | 21:16 | |
| ianw | fungi: nodepool tests still failing? http://logs.openstack.org/78/289678/2/check/gate-nodepool-python27/757e0d2/console.html do you already know about this? | 21:16 |
| tristanC | ttx: as you want, both are fine | 21:16 |
| ttx | let's say march, start of period | 21:17 |
| tristanC | ttx: that's what we did for sept-2014-elections, it was the start date | 21:17 |
| openstackgerrit | Stanislaw Pitucha proposed openstack-infra/project-config: Publish Anchor to pypi https://review.openstack.org/291433 | 21:17 |
| ttx | tristanC, fungi: alright, tag is up | 21:18 |
| *** esikachev has quit IRC | 21:19 | |
| openstackgerrit | Merged openstack-infra/infra-manual: Fix path to jenkins/jobs/projects.yaml https://review.openstack.org/289415 | 21:19 |
| rm_work | anteaya: sorry, i'm in and out today -- but yeah, that's part of it (a big part). We don't have a good solution yet that is actually still workable to validate everything works (because any change could include changes to our image agent code), so even on other projects, we could break them if they don't have the absolute latest version of our agent and they are gating against our master branch, since the agent and the controller | 21:20 |
| rm_work | code are tightly coupled | 21:20 |
| rm_work | anteaya: but i agree we should do something | 21:20 |
| rm_work | anteaya: we are going to see what we can learn from _amrith_ | 21:20 |
| *** weshay has joined #openstack-infra | 21:20 | |
| rm_work | maybe he has good ideas that can help us :) | 21:20 |
| openstackgerrit | Stanislaw Pitucha proposed openstack-infra/project-config: Publish Anchor to pypi https://review.openstack.org/291433 | 21:21 |
| openstackgerrit | Merged openstack-infra/jeepyb: Use oauth token to create GitHub project if available https://review.openstack.org/277527 | 21:21 |
| anteaya | rm_work: well infra uses disk image builder to build images daily | 21:21 |
| anteaya | does that use case parallel your use case at all? | 21:22 |
| anteaya | is _amrith_ building images? | 21:22 |
| *** kencjohnston has quit IRC | 21:22 | |
| clarkb | ok I think the dpkg/apt surgery is mostly done | 21:23 |
| clarkb | should make puppet happier | 21:24 |
| *** gnuoy_ has quit IRC | 21:24 | |
| clarkb | I had to run dpkg --configure -a and in one case apt-get -f install | 21:24 |
| *** Sukhdev has quit IRC | 21:25 | |
| *** hashar has quit IRC | 21:26 | |
| *** kgiusti has quit IRC | 21:29 | |
| *** boris-42 has joined #openstack-infra | 21:30 | |
| Shrews | clarkb: did you see my question for you on https://review.openstack.org/278803? | 21:31 |
| *** hdd has quit IRC | 21:32 | |
| clarkb | Shrews: no | 21:32 |
| clarkb | Shrews: but, it needs a new patchset now that they have working dns and ssl | 21:32 |
| *** dprince has quit IRC | 21:32 | |
| clarkb | Shrews: and yes we don't specify any versions in our clouds.yaml currently | 21:32 |
| Shrews | clarkb: ack. i'll wait for the new patch then | 21:33 |
| *** mrmartin has quit IRC | 21:33 | |
| clarkb | nibalizer: puppet has been running for almost 20 minutes on review.openstack.org, any idea if that is normal? | 21:34 |
| *** flepied has joined #openstack-infra | 21:36 | |
| clarkb | Shrews: new patchset pushed with up to date auth_url | 21:37 |
| clarkb | Shrews: and my local testing has shown that no version overrides are required | 21:37 |
| anteaya | going to go for a walk myself | 21:38 |
| nibalizer | clarkb: that doesn't seem normal to me | 21:38 |
| nibalizer | what is it doing? | 21:38 |
| clarkb | nibalizer: not sure, it ends up finishing around 20 minutes iwth no errors | 21:38 |
| clarkb | nibalizer: would probably have to check the syslog on the server | 21:38 |
| clarkb | maybe it is puppeting both servers serially? | 21:39 |
| nibalizer | oh huh we have a new review.o.o don't we | 21:39 |
| *** Sukhdev has joined #openstack-infra | 21:40 | |
| *** Sukhdev has quit IRC | 21:40 | |
| *** jistr has quit IRC | 21:40 | |
| *** hamzy has quit IRC | 21:40 | |
| *** Sukhdev has joined #openstack-infra | 21:41 | |
| *** openstackstatus has quit IRC | 21:42 | |
| *** nikhil_k has quit IRC | 21:42 | |
| *** openstackstatus has joined #openstack-infra | 21:44 | |
| *** ChanServ sets mode: +v openstackstatus | 21:44 | |
| *** annegentle has quit IRC | 21:44 | |
| *** annegentle has joined #openstack-infra | 21:44 | |
| *** nwkarsten has joined #openstack-infra | 21:45 | |
| *** keedya has quit IRC | 21:46 | |
| *** tpsilva has quit IRC | 21:47 | |
| *** amitgandhinz has quit IRC | 21:47 | |
| *** annegentle has quit IRC | 21:47 | |
| *** annegentle has joined #openstack-infra | 21:48 | |
| *** ihrachys has joined #openstack-infra | 21:49 | |
| *** nwkarsten has quit IRC | 21:49 | |
| *** |-paul-| has quit IRC | 21:49 | |
| clarkb | nibalizer: looks like the logprocessor user switch over has happened without any major fallout | 21:50 |
| *** sobersabre has quit IRC | 21:51 | |
| clarkb | waiting for puppet to finish up and figure out why logstash-worker03 and 06 claim to have had ssh errors | 21:51 |
| clarkb | I can manually ssh | 21:51 |
| nibalizer | from laptop or root@puppetmaster? | 21:51 |
| clarkb | root@puppetmaster | 21:51 |
| *** annegentle has quit IRC | 21:51 | |
| clarkb | well both but ya | 21:51 |
| *** sdague has quit IRC | 21:52 | |
| *** dims has joined #openstack-infra | 21:52 | |
| *** annegentle has joined #openstack-infra | 21:52 | |
| *** jamielennox|away is now known as jamielennox | 21:52 | |
| *** dims_ has quit IRC | 21:52 | |
| nibalizer | hrm | 21:53 |
| * nibalizer tries a -m ping | 21:54 | |
| nibalizer | so yea the entire logstash-worker pool replies to pings | 21:54 |
| *** ociuhandu has joined #openstack-infra | 21:55 | |
| *** egonzalez has joined #openstack-infra | 21:55 | |
| *** egonzalez has quit IRC | 21:56 | |
| *** thorst_ has quit IRC | 21:56 | |
| clarkb | only 03 and 06 seem to have been affected, everything else is happy | 21:56 |
| *** ociuhandu has quit IRC | 21:56 | |
| clarkb | I may wait for it to back around again and maybe it was a fluke network blip? | 21:57 |
| jhesketh | Morning | 21:57 |
| *** dgonzalez has quit IRC | 21:57 | |
| *** ociuhandu has joined #openstack-infra | 21:57 | |
| dmsimard | clarkb: is /root/ persisted between grenade runs ? | 21:57 |
| clarkb | dmsimard: yes it should be | 21:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/jenkins-job-builder: Add PLAIN default for branch|project-compare-type https://review.openstack.org/291456 | 21:58 |
| clarkb | dmsimard: we don't really delete anything between grenade runs, we just start services out of the other location | 21:58 |
| dmsimard | My logic could be bad but it looks like it gets created twice | 21:58 |
| dmsimard | http://logs.openstack.org/76/290776/9/check/gate-grenade-dsvm/dfd2eb4/logs/devstack-gate-setup-workspace-new.txt.gz#_2016-03-10_20_42_40_578 | 21:58 |
| dmsimard | http://logs.openstack.org/76/290776/9/check/gate-grenade-dsvm/dfd2eb4/logs/devstack-gate-setup-workspace-old.txt.gz | 21:58 |
| *** doug-fish has quit IRC | 21:59 | |
| *** dgonzalez has joined #openstack-infra | 21:59 | |
| clarkb | hrm | 21:59 |
| clarkb | ya definitely looks like it is touching it both times | 22:00 |
| * clarkb rereads the logic | 22:00 | |
| dmsimard | [[ -e /root/fixed_disk_layout ]] && return 0 || sudo touch /root/fixed_disk_layout | 22:00 |
| dmsimard | for me that reads if the file exists, return 0 or else touch the file | 22:00 |
| *** annegentle has quit IRC | 22:00 | |
| *** melwitt has quit IRC | 22:00 | |
| dmsimard | I tested it locally | 22:00 |
| clarkb | dmsimard: it could be a permissions issue | 22:01 |
| clarkb | I actually think that is the problem | 22:01 |
| *** annegentle has joined #openstack-infra | 22:01 | |
| dmsimard | hm, yeah | 22:01 |
| dmsimard | I can create the file elsewhere.. | 22:01 |
| clarkb | dmsimard: you could write it to /etc/ | 22:01 |
| *** HeOS has quit IRC | 22:01 | |
| dmsimard | yeah | 22:01 |
| dmsimard | either way, looks like the job failed but for other reasons this time (I haven't yet looked into) | 22:02 |
| *** thorst_ has joined #openstack-infra | 22:02 | |
| clarkb | ya it seems to have been happy with the swap that time around | 22:02 |
| *** HeOS has joined #openstack-infra | 22:02 | |
| dmsimard | and it did exactly what I wanted it to, bumped the swap on a vexxhost vm | 22:03 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/devstack-gate: Use dd to create swapfile, ensure swap amount == 8GB https://review.openstack.org/290776 | 22:03 |
| nibalizer | clarkb: im going to split for dinner in a bit | 22:03 |
| nibalizer | need anything else? | 22:03 |
| clarkb | nibalizer: not right now, its mostly working. Once I get 03 and 06 puppeted I will disable puppet on them and approve the other changes then start working through the logstash 2.0 transition | 22:04 |
| *** FallenPegasus has joined #openstack-infra | 22:04 | |
| *** thorst__ has joined #openstack-infra | 22:04 | |
| nibalizer | cool | 22:04 |
| *** thorst_ has quit IRC | 22:06 | |
| *** shardy has quit IRC | 22:07 | |
| *** thorst__ has quit IRC | 22:08 | |
| clarkb | nibalizer: looks like the run that just happened worked | 22:08 |
| clarkb | so transient network blip is what I will attribute that to | 22:08 |
| *** jpr has joined #openstack-infra | 22:09 | |
| clarkb | once ansible is done with this pass I will be putting all 20 logstash workers in the emergency file then approve the logstash 2.0 upgrade changes | 22:10 |
| *** rcernin has joined #openstack-infra | 22:11 | |
| *** esker has quit IRC | 22:13 | |
| *** aysyd has quit IRC | 22:13 | |
| *** _amrith_ is now known as amrith | 22:14 | |
| *** dgonzalez has quit IRC | 22:14 | |
| *** dgonzalez has joined #openstack-infra | 22:15 | |
| *** dgonzalez has quit IRC | 22:16 | |
| *** melwitt has joined #openstack-infra | 22:16 | |
| *** melwitt has quit IRC | 22:16 | |
| *** dgonzalez has joined #openstack-infra | 22:16 | |
| *** dgonzalez has quit IRC | 22:17 | |
| *** abregman has quit IRC | 22:17 | |
| *** dgonzalez has joined #openstack-infra | 22:18 | |
| *** jtomasek has quit IRC | 22:18 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add PLAIN default for branch|project-compare-type https://review.openstack.org/291456 | 22:18 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: [wip] Revert "Add CentOS AFS build job to project-config experimental" https://review.openstack.org/291403 | 22:22 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: CentOS AFS build : don't revoke sudo https://review.openstack.org/291467 | 22:22 |
| openstackgerrit | Merged openstack-infra/system-config: Logstash 2.0 compat ES output rule https://review.openstack.org/247714 | 22:22 |
| *** thorst_ has joined #openstack-infra | 22:22 | |
| *** rcernin has quit IRC | 22:22 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool: Use shade for all OpenStack interactions https://review.openstack.org/226751 | 22:23 |
| openstackgerrit | Merged openstack-infra/puppet-logstash: Install logstash 2.0 https://review.openstack.org/247756 | 22:23 |
| clarkb | ok worker19 is staged to be the first updated | 22:24 |
| clarkb | (well 20 was updated way back when so I guess 19 is technically second) | 22:25 |
| *** sdake_ has joined #openstack-infra | 22:25 | |
| *** thorst__ has joined #openstack-infra | 22:26 | |
| *** woodster_ has joined #openstack-infra | 22:27 | |
| *** sdake has quit IRC | 22:28 | |
| *** thorst_ has quit IRC | 22:29 | |
| fungi | thanks ttx! | 22:30 |
| jesusaur | fungi: i have the zuul-cloner change passing tests locally, but the check queue seems to either hit an alarm or create too large of a subunit file | 22:31 |
| clarkb | jesusaur: the alarm is the test timeout | 22:31 |
| clarkb | jesusaur: and for too alrge of a subunit file that is because zuul is logging all the things | 22:32 |
| fungi | ianw: i thought that failure mode was theorized to be due to the anonymous account still existing, but i guess that wasn't the case because you can see the "DELETE FROM mysql.user WHERE User='\'''\''; FLUSH PRIVILEGES;" there in the job log | 22:32 |
| jesusaur | clarkb: so what is the correct thing to do to mitigate the failures? increase the limits or reduce the logging? | 22:33 |
| *** esker has joined #openstack-infra | 22:33 | |
| clarkb | jesusaur: I think reduce the logging | 22:34 |
| fungi | clarkb: nibalizer: puppet shouldn't be trying to apply on both review.o.o servers, yolanda disabled the new one by uuid in the emergency file today right after it was built | 22:34 |
| ianw | fungi: yeah ... so i presume no other jobs are seeing this | 22:34 |
| clarkb | jesusaur: this is a simple sanity check saying "you are logging too much | 22:34 |
| clarkb | fungi: ok, it has been quicker since the 20 minute run | 22:34 |
| *** mriedem has quit IRC | 22:34 | |
| fungi | clarkb: it might have had a change to stuff manage-projects cares about and triggered the exec for that? it's crazy slow | 22:35 |
| clarkb | that is possible | 22:36 |
| fungi | ianw: nodepool is the only project with a job exhibiting this regression as far as i'm aware | 22:36 |
| clarkb | fungi: nodepool is also one of the only projcets that does db per test | 22:36 |
| ianw | fungi: i haven't had luck seeing it with anything in logstash, but i also see chatter about that | 22:36 |
| fungi | clarkb: maybe the grant we do doesn't provide sufficient permission to create databases? unfortunately it doesn't echo the failing query in the log | 22:37 |
| *** nwkarsten has joined #openstack-infra | 22:37 | |
| ianw | fungi: might it be worth holding a node so we can manually inspect mysql? | 22:37 |
| fungi | pymysql.err.OperationalError: (1044, u"Access denied for user 'openstack_citest'@'%' to database 'WzwysOab_6923'") | 22:38 |
| clarkb | well thats disappointing. erb array.to_s is not the same as irb array.to_s | 22:38 |
| jeblair | clarkb: what's an irb? | 22:38 |
| *** esker has quit IRC | 22:38 | |
| fungi | ianw: i can do that quite easily, yes. want to recheck and let me know the node id? i'm happy to put your ssh key on it | 22:38 |
| clarkb | jeblair: interactive ruby | 22:38 |
| ianw | fungi: just a tick. i think that's going to be quicker than trying to replicate it | 22:38 |
| *** chlong has quit IRC | 22:39 | |
| *** gyee has joined #openstack-infra | 22:40 | |
| fungi | no argument from me. i could probably also hold us an unused bare-trusty node before we delete the remaining one if you want a side-by-side comparison | 22:40 |
| clarkb | http://paste.openstack.org/show/490077/ | 22:40 |
| openstackgerrit | Mathieu Gagné proposed openstack-infra/jeepyb: Add support for GitHub Enterprise https://review.openstack.org/276528 | 22:40 |
| fungi | er, before we delete the remaining oneS | 22:40 |
| clarkb | it appears to be making an array just fine but instaed of to_s resulting in [] and ""s its appending all the entries together | 22:40 |
| ianw | fungi: ubuntu-trusty-ovh-gra1-8687499 | 22:41 |
| ianw | get it while it's hot :) | 22:41 |
| ianw | https://jenkins02.openstack.org/job/gate-nodepool-python27/149/console | 22:42 |
| clarkb | crinkle: have a chance to look at https://review.openstack.org/#/c/247714/6/modules/openstack_project/templates/logstash/indexer.conf.erb vs http://paste.openstack.org/show/490077/ and help me figure that out? | 22:42 |
| *** nwkarsten has quit IRC | 22:42 | |
| *** _ari_ has quit IRC | 22:42 | |
| fungi | ianw: held. i'll aldo grab a bare-trusty out of the fresh pile. what ssh key do you want added? | 22:43 |
| ianw | fungi: https://github.com/ianw.keys | 22:43 |
| *** burgerk has quit IRC | 22:44 | |
| *** EricGonczer_ has joined #openstack-infra | 22:44 | |
| anteaya | morning jhesketh | 22:44 |
| *** hdd has joined #openstack-infra | 22:45 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: puppet: fix cookiecutter job template https://review.openstack.org/290906 | 22:46 |
| EmilienM | hey infra, can someone approve this patch please ? https://review.openstack.org/#/c/288244/ | 22:47 |
| fungi | ianw: jenkins@149.202.179.199 for the held ubuntu-trusty node and jenkins@104.239.192.201 for an unused bare-trusty i held | 22:48 |
| *** EricGonczer_ has quit IRC | 22:49 | |
| openstackgerrit | Merged openstack-infra/project-config: CentOS AFS build : don't revoke sudo https://review.openstack.org/291467 | 22:49 |
| *** hdd has quit IRC | 22:50 | |
| fungi | i too would like to see EmilienM's 288244 change merged since it updates that puppeted file for consistency with centos 7's (and recent fedora's) new defaults, and corrects a definite bug on our workers for those platforms | 22:50 |
| crinkle | clarkb: that is weird | 22:51 |
| clarkb | the to_s is apparently redundant but doesn't change the output | 22:53 |
| *** arxcruz has joined #openstack-infra | 22:54 | |
| *** EricGonczer_ has joined #openstack-infra | 22:55 | |
| *** ddieterly is now known as ddieterly[away] | 22:56 | |
| *** doug-fish has joined #openstack-infra | 22:59 | |
| clarkb | I think I have an alternative I just need to figure out how to rmeove the trailing , | 23:00 |
| *** dingyichen has joined #openstack-infra | 23:00 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 23:01 | |
| crinkle | clarkb: i'm trying to reproduce, not seeing it | 23:02 |
| clarkb | this is puppet 3.8.6 | 23:03 |
| *** doug-fish has quit IRC | 23:03 | |
| crinkle | yep | 23:03 |
| crinkle | ruby 1.9? | 23:03 |
| *** doug-fish has joined #openstack-infra | 23:03 | |
| clarkb | looks like 1.8.7? | 23:03 |
| clarkb | ruby1.9.1 is not installed | 23:04 |
| crinkle | aha with 1.8.7 i can reproduce | 23:04 |
| craige | o/ | 23:05 |
| *** krtaylor has joined #openstack-infra | 23:05 | |
| * clarkb does a bad hack | 23:06 | |
| *** arxcruz has quit IRC | 23:07 | |
| *** doug-fish has quit IRC | 23:08 | |
| *** EricGonczer_ has quit IRC | 23:08 | |
| ianw | fungi: can you please sudoer jenkins so i can query mysql as root | 23:08 |
| fungi | ianw: sure thing, just a sec | 23:09 |
| *** derekh has joined #openstack-infra | 23:09 | |
| fungi | ianw: done | 23:10 |
| *** dmsimard is now known as dmsimard|pto | 23:10 | |
| ianw | ERROR 1141 (42000): There is no such grant defined for user 'openstack_ci' on host '%' | 23:11 |
| ianw | well that seems odd | 23:11 |
| *** krtaylor has quit IRC | 23:11 | |
| *** moravec has quit IRC | 23:12 | |
| *** annegentle has quit IRC | 23:12 | |
| *** annegentle has joined #openstack-infra | 23:12 | |
| crinkle | clarkb: I think .inspect gives you what you want in ruby 1.8.7 | 23:13 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Use ruby 1.8 compat erb for logstash config https://review.openstack.org/291481 | 23:13 |
| clarkb | crinkle: instead of to_s? | 23:13 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Update IRC channel to use #openstack-qa https://review.openstack.org/271474 | 23:13 |
| clarkb | I can try that | 23:13 |
| crinkle | clarkb: yes | 23:13 |
| *** annegentle has quit IRC | 23:14 | |
| clarkb | crinkle: is inspect something in ruby 1.9 and 2.0? | 23:14 |
| *** doug-fish has joined #openstack-infra | 23:14 | |
| crinkle | clarkb: yes and it seems to have the same behavior | 23:15 |
| clarkb | looks like inspect worked, I will amke that update instead | 23:15 |
| *** _tzn has joined #openstack-infra | 23:15 | |
| *** annegentle has joined #openstack-infra | 23:15 | |
| *** tzn has quit IRC | 23:16 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Use ruby 1.8 compat erb for logstash config https://review.openstack.org/291481 | 23:16 |
| *** tzn_ has joined #openstack-infra | 23:16 | |
| *** ihrachys has quit IRC | 23:16 | |
| *** _tzn has quit IRC | 23:16 | |
| clarkb | I self approved that | 23:17 |
| clarkb | mordred: nibalizer the next puppet-ansible thing I want to tackle is fixing how we copy our modules onto the hosts | 23:17 |
| nibalizer | fix how | 23:18 |
| clarkb | mordred: nibalizer we aren't clearing out old files if they have been deleted, I think we just need a slightly better rsync command | 23:18 |
| nibalizer | git clean? | 23:18 |
| clarkb | or that | 23:18 |
| nibalizer | i think yea ibstall_modules should clean | 23:18 |
| clarkb | nibalizer: I don't think the issue is puppetmaster side | 23:18 |
| clarkb | nibalizer: but it might be both | 23:18 |
| *** ddieterly[away] is now known as ddieterly | 23:18 | |
| clarkb | in any case I will try to look into that soonish | 23:18 |
| nibalizer | yea lets do both | 23:19 |
| *** doug-fish has quit IRC | 23:19 | |
| *** asettle has joined #openstack-infra | 23:19 | |
| *** tonytan4ever has quit IRC | 23:20 | |
| *** EricGonczer_ has joined #openstack-infra | 23:20 | |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-exim: Always specify driver first https://review.openstack.org/291482 | 23:20 |
| jeblair | nibalizer: can you check my erbing? | 23:21 |
| nibalizer | jeblair: sorry | 23:22 |
| nibalizer | gerrit on phobe is super bad | 23:22 |
| *** Qiming has joined #openstack-infra | 23:22 | |
| docaedo | I'm trying to debug an issue with the app catalog, wondering if there are more verbose logs from puppet (i.e. something more useful than what I find at http://puppetboard.openstack.org/node/apps.openstack.org) | 23:24 |
| jeblair | is anyone here not on a phone? | 23:25 |
| Clint | i'm not on a phone | 23:25 |
| jeblair | Clint: do you understand ruby erb? | 23:25 |
| Clint | usually | 23:25 |
| *** krtaylor has joined #openstack-infra | 23:25 | |
| * Clint peers. | 23:25 | |
| clarkb | docaedo: unfortunately the ansible stuff has us stashing those only in syslog on the host in question | 23:25 |
| mordred | crinkle: ^^ ? | 23:25 |
| jeblair | Clint: can you let me know if https://review.openstack.org/291482 looks good? | 23:25 |
| mordred | jeblair: it looks good to me - but I'm not sure my erb is so stellar | 23:26 |
| clarkb | I know my erb is bad based on recent events | 23:26 |
| *** dmsimard|pto has quit IRC | 23:26 | |
| clarkb | ruby! | 23:26 |
| *** xyang1 has quit IRC | 23:26 | |
| jeblair | clarkb: i would have asked you but i did not want to prolong your pain | 23:26 |
| docaedo | clarkb: ok thanks .. I think one of the scripts that builds the JSON of assets is misbehaving, just trying to figure out how best to tell | 23:26 |
| mgagne | we have a couple of example in puppet-horizon with ERB: https://github.com/openstack/puppet-horizon/blob/master/templates/local_settings.py.erb | 23:27 |
| mgagne | good luck | 23:27 |
| clarkb | docaedo: I only see recent successes in syslog | 23:27 |
| *** esker has joined #openstack-infra | 23:27 | |
| clarkb | docaedo: nothing indicating one of the execs broke | 23:27 |
| *** esker has quit IRC | 23:28 | |
| fungi | ianw: are you seeing signs that the grant executed by the job didn't "stick"? | 23:28 |
| *** esker has joined #openstack-infra | 23:28 | |
| fungi | or maybe never happened at all? | 23:28 |
| *** nikhil_k has joined #openstack-infra | 23:28 | |
| jeblair | <%- @neutron_options_real.sort.each do |opt_name,opt_val| -%> | 23:28 |
| jeblair | mgagne: wait | 23:28 |
| jeblair | what does that do? | 23:28 |
| mgagne | takes a "dict", sort it and loop | 23:28 |
| docaedo | clarkb: yeah don't think it's anything fully broken - it's just supposed to add a last_modified field to each asset (based on what git blame tells us), and locally works perfectly, but on the server that field is blank | 23:28 |
| jeblair | mgagne: is that a list of hash key, value pairs sorted by key? | 23:28 |
| mgagne | we sort it due to ruby not preserving order in 1.8 | 23:29 |
| mgagne | yes | 23:29 |
| docaedo | clarkb: I can recreate that behavior locally but there are a few different things that could cause it, so I don't know what to try to fix :) | 23:29 |
| jeblair | mgagne: yeah, that's one of the things i'm fixing in https://review.openstack.org/291482 but i didn't know you could do that, i thought you had to just sort the keys and access by key | 23:29 |
| *** melwitt has joined #openstack-infra | 23:29 | |
| clarkb | docaedo: what should I be checking to narrow it down for you? | 23:29 |
| mgagne | both could work I guess | 23:29 |
| jeblair | yeah. at this point, i'm inclined to leave what i now know is not perfectly elegant ruby in place because i have tested it :) | 23:30 |
| mgagne | =) | 23:30 |
| *** bpokorny_ has joined #openstack-infra | 23:30 | |
| jeblair | but i wuld have written it that way had i known | 23:30 |
| docaedo | clarkb: I think I need to merge a change to a script in the app catalog first - right now tools/asset_history.sh creates a file that gets merged with assets.yaml, and then it deletes that file, | 23:30 |
| docaedo | clarkb: so that file not being there when you look would be normal behavior (and it's only there for a moment, if assets.yaml is changed) | 23:31 |
| jeblair | i'm self-approving https://review.openstack.org/291482 because it's a bugfix and it looks okay to Clint | 23:31 |
| docaedo | clarkb: I can have that file live in /tmp/ though, and not remove it until the start of the *next* run .. so until I do that, I don't think there's anything you can do to help | 23:32 |
| clarkb | docaedo: ok | 23:32 |
| docaedo | clarkb: appreciate the offer though - thank you! | 23:32 |
| crinkle | mordred: o/ | 23:32 |
| *** ddieterly is now known as ddieterly[away] | 23:33 | |
| *** bpokorny has quit IRC | 23:34 | |
| *** regXboi has quit IRC | 23:34 | |
| *** ddieterly[away] has quit IRC | 23:36 | |
| *** derekh has quit IRC | 23:39 | |
| *** Qiming has quit IRC | 23:40 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Add job tags https://review.openstack.org/277987 | 23:40 |
| openstackgerrit | Merged openstack-infra/system-config: Use ruby 1.8 compat erb for logstash config https://review.openstack.org/291481 | 23:40 |
| *** Qiming has joined #openstack-infra | 23:41 | |
| *** dmsimard|pto has joined #openstack-infra | 23:41 | |
| *** annegentle has quit IRC | 23:42 | |
| *** sdake_ has quit IRC | 23:42 | |
| clarkb | next puppet run should puppet logstash-worker18 with the above fix | 23:43 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Make all configuration in-repo configuration https://review.openstack.org/268386 | 23:43 |
| clarkb | if that works I will do half and half of the remainder | 23:43 |
| openstackgerrit | Frank Kloeker proposed openstack-infra/system-config: Manifest for configuration translation checksite https://review.openstack.org/276466 | 23:43 |
| *** Sukhdev has quit IRC | 23:44 | |
| openstackgerrit | Merged openstack-infra/puppet-exim: Always specify driver first https://review.openstack.org/291482 | 23:44 |
| *** Sukhdev has joined #openstack-infra | 23:45 | |
| *** yamahata has joined #openstack-infra | 23:46 | |
| *** gongysh has joined #openstack-infra | 23:47 | |
| ianw | fungi: not sure ... think it's something in nodepool that's creating the "random string" db | 23:47 |
| ianw | ahh, i think i see it ... | 23:48 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-exim: Add some whitespace between routers/transports https://review.openstack.org/291490 | 23:50 |
| * anteaya watches the prime minister arrive for the state dinner | 23:52 | |
| mordred | anteaya: arrive at your house? | 23:53 |
| jeblair | SotK, Zara: the storyboard exim patches have landed | 23:53 |
| *** gordc has quit IRC | 23:54 | |
| anteaya | I don't have state dinners | 23:54 |
| anteaya | jeblair: yay the exim patches have landed | 23:55 |
| anteaya | mordred: now mind you if the states wanted to have the prime minister to dinner at my house, I would do my best | 23:55 |
| anteaya | I'd even bring out gramma's china | 23:55 |
| anteaya | the white house | 23:55 |
| anteaya | apparently this is to make the us aware canada is a thing? | 23:56 |
| *** Swami has joined #openstack-infra | 23:56 | |
| anteaya | or to help | 23:56 |
| ianw | fungi: ok, so it's this bit -> https://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/tests/__init__.py#n269 | 23:56 |
| anteaya | you too can watch: http://www.cbc.ca/news/politics/state-dinner-for-trudeau-caps-off-official-visit-at-white-house-1.3483695 | 23:56 |
| anteaya | so far guests are being announced | 23:57 |
| ianw | fungi: the nodepool testing creates a new user and tries to grant them their database | 23:57 |
| *** Sukhdev_ has joined #openstack-infra | 23:57 | |
| *** Sukhdev has quit IRC | 23:57 | |
| fungi | ianw: aha, rather than using the openstack_citest user? | 23:57 |
| ianw | fungi: we have to setup "openstack_citest" WITH GRANT OPTION; if we want them to be able to do that | 23:57 |
| ianw | (grant to other users) | 23:57 |
| mordred | anteaya: ah. I undersatnd now | 23:58 |
| fungi | ianw: sounds like a reasonable addition. if you want to write up that patch we'll get it merged before i go do manual jjb runs this evening | 23:58 |
| fungi | clarkb: ^ sound fine? | 23:58 |
| ianw | fungi: the other approach is to, i guess, refactor nodepool. i'd be willing to be with some "blame" digging it was copied from somewhere else and nobody's ever really thought that much about it | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
