Thursday, 2016-01-07

« Wednesday, 2016-01-06 Index Friday, 2016-01-08 »
sc68calis meetbot misbehaving today?00:00
anteayasc68cal: what are you seeing and in what channel?00:00
sc68caltrying to start my meeting in -alt00:00
sc68calsigh00:01
anteayait is working in -meeting00:01
sc68calit'd help if I spelled it right.00:01
anteayabut I agree it doesn't seem to be working in -alt00:01
fungispelling ftw00:01
sc68caljust one of those days00:01
anteayasc68cal: and you're off, good meeting to you00:01
* sc68cal needs more coffee00:02
*** Swami_ has joined #openstack-infra00:03
*** Swami has quit IRC00:03
*** baoli has quit IRC00:04
openstackgerritSachi King proposed openstack-dev/pbr: Add dependencies for building wheels  https://review.openstack.org/26401000:04
openstackgerritSachi King proposed openstack-dev/pbr: Make setup.py --help-commands work without testrepository  https://review.openstack.org/26392800:04
openstackgerritMerged openstack-infra/project-config: Grafana: Add Rackspace nodepool graph  https://review.openstack.org/26438400:05
openstackgerritMerged openstack-infra/project-config: Grafana: adjust internap graphs to match rax  https://review.openstack.org/26438500:06
jeblairclarkb: seems to work for me...00:06
jeblairclarkb: though, things get weird again before 2015010800:06
jeblairso i'm using that as my from00:06
clarkblet me try that00:07
*** dims_ has quit IRC00:07
clarkbjeblair: what interval are you using in the summarize? I am using 1month maybe thats the problem00:07
clarkbthere it goes, seems order matters00:08
jeblairoh, heh, still 1h00:08
*** tzn has joined #openstack-infra00:10
*** cloudtrainme has quit IRC00:14
openstackgerritJames E. Blair proposed openstack-infra/project-config: Grafana: Add Blue Box dashboard  https://review.openstack.org/26447900:14
clarkbI get 551569300:15
*** tzn has quit IRC00:15
*** otter768 has joined #openstack-infra00:16
*** Sukhdev has joined #openstack-infra00:17
clarkbthats how many jobs we launched last year roughly00:18
clarkbnot bad00:18
fungi~630jph average over the year00:18
*** Qiming has quit IRC00:19
fungithat seems like it could be reasonably accurate00:19
openstackgerritJames E. Blair proposed openstack-infra/project-config: Grafana: Add OVH dashboard  https://review.openstack.org/26448000:19
jeblairyeah, it does not surprise me00:19
*** otter768 has quit IRC00:21
clarkbjeblair: pabelanger in grafana is there a way to tell a dashboard I want to look at more than a few hours of recent activity?00:24
jeblairclarkb: yeah, there's stuff in the top right00:25
jeblairclarkb: 'zoom out' and the little clock symbol00:25
clarkboh sweet00:25
jeblairclarkb: note, i'm unable to get custom times to work, only the predefined 'last X timeperiod' stuff00:25
*** baoli has joined #openstack-infra00:25
clarkb(looking at the git mirror graphs and want to see how it changes through the week and during our image builds00:25
jeblairclarkb: i suspect a grafana->graphite bug00:26
openstackgerritJames E. Blair proposed openstack-infra/project-config: Grafana: Fix node launch error graphs  https://review.openstack.org/26448200:26
*** doug-fish has quit IRC00:26
jeblairmgagne, clarkb: ^ i was wrong about the error graph, that shuld fix it00:26
mgagnejeblair oh, nice catch. should have looked at graphite data ^^'00:27
clarkbhttp is the least used and has the least errors00:27
mgagnethat's why I wasn't able to use aggregate functions and regex to bypass the problem00:27
clarkband errors don't seem to map to when we do our image builds00:27
clarkbI wonder00:27
jeblairclarkb: errors can be client hangups00:28
clarkbjeblair: ya I seem to recall at one point someone had an RST but we werent running at bandwidth capacity00:28
clarkbgreghaynes: ^ was that you?00:28
greghaynesYep00:28
clarkbjroll: JayF the rax network bandwidth limits don't apply different to individual source/destiniations do they?00:28
clarkbjroll: JayF eg if we have 800mbps aggregate allowed, would a client be limited to some smaller number and start getting resets if they went over?00:29
jrollclarkb: no clue, honestly00:29
jrollsec00:29
jeblairclarkb: btw, i don't really know which of those graphs will be interesting over time -- it's basically just as much as i could get, so if you want to adjust that page feel free.00:30
jeblairclarkb: i mean, i took a guess at which i thought would be the most interesting and made those larger, but...00:30
*** Swami_ has quit IRC00:31
*** gampel1 has quit IRC00:31
*** sputnik13 has quit IRC00:31
clarkbjeblair: those looks good, might want to add an aggregate too for connections/bw etc00:31
mgagnejeblair there is a graph on status.o.o/zuul with buiilding/available/in-use/deleting nodes. would there be a way to get this data per provider?00:31
anteayagreghaynes: what is the status of this diskimage-builder patch? there is a project-config patch that depends on it, trying to figure out how much credence to give it00:31
jeblairmgagne: yes!00:31
anteayahttps://review.openstack.org/#/c/239562/00:31
jeblairmgagne: i'll whip that up00:32
mgagnejeblair I only see 2 available nodes on our end so I'm wondering if it's a problem on our side or not.00:32
*** sputnik13 has joined #openstack-infra00:32
mgagnejeblair there are public nodepool logs but only for dib build so not much useful for me00:32
jeblairmgagne: yeah, i think we're going to be able to publish the complete logs soon00:33
*** ajmiller has joined #openstack-infra00:33
mgagnewell, I'm mostly looking for nodepook.log for its periodic tasks output00:33
jeblairmgagne: nodepool tries to utilize providers at the ratio they are configured, so when we're not at full demand, providers with fewer max-servers will be used less, so the current situation may be normal00:34
clarkbmtreinish: the node_provider on dashboard sorts funny, it seems to go left to right top to bottom except right now ovh-gra1 has a higher fail rate than ovh-bhs100:34
*** sabeen3 has quit IRC00:35
mgagneoh, I get it. will wait until tomorrow to get a better idea00:35
jeblairmgagne: yeah, i think that's what's happening right now00:35
anteayaif there is a devstack-gate reviewer that wants a change of scenery, here is a patch that needs one more +2 that upgrades bashate, there is a project-config patch that is ready to go that depends on it: https://review.openstack.org/#/c/236815/00:35
mtreinishclarkb: yeah, I've seen that weirdness before too00:35
mtreinishI'm not sure what causes it00:35
*** samuelBartel has quit IRC00:36
mgagnejeblair should https://review.openstack.org/#/c/264370/ wait until next week to get more data before bumping quota?00:36
clarkb261070 failed grenade again :(00:36
anteaya:(00:36
clarkbseems like grenade has a really high fail rate right now00:36
clarkbanteaya: what project-config change depends on that dib change?00:37
greghaynesanteaya: there is a patch it depends on which needs to be updated00:37
anteayaclarkb greghaynes https://review.openstack.org/#/c/245297/00:37
anteayagreghaynes: awesome00:37
jrollclarkb: sounds like generally no00:38
openstackgerritMerged openstack-infra/project-config: Grafana: Add Blue Box dashboard  https://review.openstack.org/26447900:38
jeblairmgagne, clarkb: it looks like maybe we've done 10 devstack-tempest gate runs on internap?  i'll leave it to the two of you to decide on how big of a baseline data set you want :)00:38
mtreinishclarkb: https://github.com/openstack/openstack-health/blob/master/app/js/controllers/home.js#L10-L1300:38
clarkbjroll: ok, so the limit is the limit whether a single connection is made or many?00:38
mtreinishit seems right to me at least00:38
jrollclarkb: yes00:39
clarkbmtreinish: I bet its comparing fails over time not percentages?00:39
clarkbfailRate to me is semi ambiguous00:39
jrollclarkb: what's the context btw?00:40
jrolljust out of curiousity00:40
mtreinishclarkb: it's supposed to be the percentage00:40
mgagneclarkb jeblair : lets wait until Monday, I'm off on Fridays for the next couple of months so won't be able to react fast if anything goes wrong with the next ratio00:40
clarkbjroll: greghaynes saw a rst when talking to our git mirror a little while back, but our frontend never saw the connection00:40
clarkbjroll: we have seen similar behavior in the psat when hitting rax bw limits. the way rax shapes that is to nuke connections from orbit with gratuitous rsts00:40
clarkbmgagne: ok00:41
*** otter768 has joined #openstack-infra00:41
openstackgerritMerged openstack-infra/project-config: Grafana: Add OVH dashboard  https://review.openstack.org/26448000:41
anteayamgagne: do you want to make a note in the patch that you want it merged monday, or would you like me to set workflow -1 on it?00:41
jrollclarkb: huh, ok00:41
anteayamgagne: or both?00:41
mgagneanteaya will add a note, could be merged before if people can revert if they feel it's going wrong00:42
jeblairyeah, that's always an option00:42
anteayamgagne: sure, either way, as long as you are happy00:42
openstackgerritMerged openstack-infra/project-config: Grafana: Fix node launch error graphs  https://review.openstack.org/26448200:42
*** tsg has joined #openstack-infra00:42
*** sputnik13 has quit IRC00:43
*** sputnik13 has joined #openstack-infra00:48
*** yolanda has quit IRC00:50
anteayapabelanger: do I have any ability as a user to alter the colour of the graphs in grafana at all?00:50
*** sputnik13 has quit IRC00:51
anteayaI'm trying to see data and right now I see the holi festival: http://poraschaudhary.photoshelter.com/image/I0000_13RWqbI1r000:52
anteayaits pretty00:52
jrollclarkb: if you keep having problems with it let me know but I don't think that was it00:53
clarkbjroll: can do and thank you for checking00:53
jrollnp00:53
*** gyee has quit IRC00:54
*** sabeen1 has joined #openstack-infra00:56
*** sputnik13 has joined #openstack-infra00:56
*** dims has joined #openstack-infra00:56
*** baoli has quit IRC00:57
*** thorst has joined #openstack-infra00:58
*** sputnik13 has quit IRC00:58
openstackgerritJames E. Blair proposed openstack-infra/project-config: Grafana: Add per-region node graphs  https://review.openstack.org/26449700:59
jeblairmgagne: ^00:59
*** zz_dimtruck is now known as dimtruck00:59
*** jaypipes has quit IRC00:59
*** eil397 has quit IRC01:03
*** ybathia has quit IRC01:04
*** thorst_ has joined #openstack-infra01:04
*** ybathia has joined #openstack-infra01:04
zarofungi: will this work? https://gerrit-review.googlesource.com/#/c/73650/01:06
*** thorst has quit IRC01:06
jeblairzaro: can you get further information about the submission from the submission id?01:06
zarothe number between the two dashes is the millisecond date01:06
jeblairzaro: that seems like an implementation detail that could change easily in the future01:07
jeblairzaro: if a developer changes the way submission ids are formatted (say they switch to uuids to support multi-master better), they would not think that this needs to change and our usage would break01:08
fungizaro: it might work if that id format is guaranteed to stay consistent (and if the submission ids stored by older versions of gerrit have been using the same pattern)01:08
fungibut i share jeblair's concerns01:08
fungisomething closer to what you get in the labels associative array would make more sense01:08
jeblairzaro: it would be better if the actual api returned a timestampt for the time of submission so that was a promise that the api made to users that way if something changes, developers know what users are using the field for01:08
fungiclearly the information is floating around in gerrit since the ssh api displays it01:09
zarook.  gotta go right now but will thing about that.01:09
*** ybathia has quit IRC01:09
fungizaro: cool--thanks for working on this!01:09
*** tzn has joined #openstack-infra01:11
*** pvaneck has quit IRC01:12
*** Qiming has joined #openstack-infra01:12
*** boris-42 has quit IRC01:13
*** bharathm has quit IRC01:15
*** tzn has quit IRC01:16
openstackgerritMerged openstack/os-testr: Add subunit_trace --color option's doc  https://review.openstack.org/25381401:20
openstackgerritSachi King proposed openstack-dev/pbr: Add dependencies for building wheels  https://review.openstack.org/26401001:20
openstackgerritSachi King proposed openstack-dev/pbr: Make setup.py --help-commands work without testrepository  https://review.openstack.org/26392801:20
*** Sukhdev has quit IRC01:25
*** bhunter71 has quit IRC01:28
*** angdraug has quit IRC01:32
*** angdraug has joined #openstack-infra01:33
*** ybathia has joined #openstack-infra01:35
*** asselin has quit IRC01:36
*** ybathia_ has joined #openstack-infra01:36
*** angdraug has quit IRC01:36
*** Sukhdev has joined #openstack-infra01:38
*** ybathia has quit IRC01:39
*** Thelo has joined #openstack-infra01:40
TheloHi all, is anyone familiar with this nodepool error message "fatal: index file smaller than expected" (http://paste.openstack.org/show/483196/)01:41
*** ybathia_ has quit IRC01:41
*** ddieterly has joined #openstack-infra01:41
clarkbThelo: that looks like a broken git repo01:41
clarkbThelo: but its in the cache so while dib is not running you should be safe to delete that directoy01:42
Thelois it safe to delete everything under dib_cache?01:42
*** Sukhdev has quit IRC01:43
clarkbyes, though it will have to repopulate which will make the next build be slow nad use lots of network traffic01:44
clarkbI think you can just delete that one git repo01:44
Thelook I try again then01:44
Thelothanks01:44
*** tsg has quit IRC01:45
Theloclackb: Just to be curious how long it takes you to build an image?01:46
clarkbon my local build machine with slow spinning disks usually about 45 minutes01:47
clarkbon our nodepool server that we use in production 10-15 minutes seems normal01:47
Thelook thanks01:47
*** mtanino has quit IRC01:47
clarkbthat assumes a warm cache01:49
*** Thelo has quit IRC01:49
*** Thelo has joined #openstack-infra01:49
Theloclarkb: I hit the same issue again, is there anything else I can do?01:52
clarkbI don't know, not familiar with that git error01:52
*** dimtruck is now known as zz_dimtruck01:54
clarkbif we resurrect https://review.openstack.org/#/c/179840/ and update it to handle secure.conf and occ clouds.yaml is there enough revie winterest to get that in?01:57
clarkbI think we shouldn't try to make it perfect this time around if so01:57
clarkblets get something so that we aren't perpetually chasing changes to nodepool01:57
*** julim has joined #openstack-infra02:00
fungiclarkb: Thelo: it implies that the local .git/index in that clone of the repo is probably corrupt. chances are you didn't manage to delete the cached version after all02:01
fungiif glance were actually broken on reclone, we'd be seeing much worse issues i expect02:02
*** bhunter71 has joined #openstack-infra02:03
*** davideagnello has quit IRC02:07
*** bpokorny has quit IRC02:13
*** mriedem has quit IRC02:13
*** cloudtrainme has joined #openstack-infra02:16
*** zhurong has joined #openstack-infra02:17
*** piet has joined #openstack-infra02:20
*** sdake has quit IRC02:22
*** salv-orl_ has quit IRC02:23
*** salv-orlando has joined #openstack-infra02:23
clarkbI have rechecked the sparce swap file change again02:27
clarkbgrenade failed again02:27
*** claudiub has quit IRC02:27
*** fawadkhaliq has joined #openstack-infra02:28
jrollI like these nodepool graphs02:29
*** ddieterly has quit IRC02:30
*** piet has quit IRC02:35
openstackgerritShu Muto proposed openstack/os-testr: remove python 3.3 trove classifier  https://review.openstack.org/26452102:40
*** esker has joined #openstack-infra02:40
openstackgerritShu Muto proposed openstack/os-testr: remove python 3.3 trove classifier  https://review.openstack.org/26452102:40
*** baoli has joined #openstack-infra02:42
*** baoli has quit IRC02:46
*** baoli has joined #openstack-infra02:47
*** baoli has quit IRC02:48
*** baoli has joined #openstack-infra02:54
*** baoli has quit IRC02:54
camunozping StevenK02:55
StevenKcamunoz: Hi02:55
camunozh Steven, happy new year!02:55
camunozquick question...02:55
*** baoli has joined #openstack-infra02:55
camunozwho can I talk to about openstack id... I'm playing with a couple of things on Zanata and Open Id to tighten the integration and need some expertise02:56
*** ddieterly has joined #openstack-infra02:56
*** ddieterly has joined #openstack-infra02:58
*** dims has quit IRC02:58
StevenKcamunoz: According to http://git.openstack.org/cgit/openstack-infra/openstackid , Sebastian Marcet. I'm not sure if he's on IRC.02:58
clarkbsmarcet is the irc nick, not currently here02:59
clarkbI think it is late in the evening for smarcet02:59
*** tlian2 has joined #openstack-infra03:01
*** tlian2 has quit IRC03:01
* craige wonders if clint is about.03:01
camunozthanks StevenK, clarkb I will try to get in touch with him later03:02
*** ddieterly has quit IRC03:02
* Clint squints at craige03:02
*** andreykurilin__ has quit IRC03:02
*** boris-42 has joined #openstack-infra03:03
craigeAs a someone who has multiple storyboard accounts, would it mkae any sense to merge the tickets under one name?03:03
*** tlian has quit IRC03:04
craigeFor authentication purposes, when emails share the same launchpad ID, I'm linking them. I've not thought aboutwhat needs to happen to any tickets held by the other account.03:05
Clintcraige: i would certainly prefer to have everything merged03:07
Clintcraige: though my problem started with having two launchpad ids03:07
StevenKStarted?03:07
StevenKClint: Launchpad accounts can be merged03:08
Clinter, sorry, no03:08
Clinti misspoke03:08
craigeFor the next iteration, I'll merge and see what the consensus is.03:08
Clinti have one launchpad id, but two ubuntu one whatsits03:08
clarkbthe problem iirc is that launchpad makes a new openid for every email03:08
clarkbso when you login with the other email the other openid is used which tells most openid consumers to make a new account03:08
craigeFYI for tchaypo and others, I just ended up wrapping that SQL logic in python try - except - else.03:08
*** thorst_ has quit IRC03:09
clarkbI have worked around this by using a single email in launchpad then add additional emails to whatever service I am using03:09
craigeWhen I query launchpad, with Clint's two email addresses, they return the same OpenID.03:09
*** thorst has joined #openstack-infra03:09
*** links has joined #openstack-infra03:10
*** links has quit IRC03:10
Clintthe original ubuntu one openid was not associated with my launchpad account03:10
craigeSo I need to handle that migration case (there were others too)03:10
clarkbcraige: if you look at the html source of the account you can see if there is >1 openid03:10
clarkbthey are listed in the source and hidden away03:10
craigeoh, niiice.03:10
clarkbthat can at last confirm/deny if that may be part of the problem03:11
craigeI' mextracting the openID via "('(\',)')03:11
craigehrm03:12
*** baoli has quit IRC03:12
Clintneeds more punctuation03:12
*** edmondsw has quit IRC03:12
*** links has joined #openstack-infra03:12
craigeI'm extracting the openID via launchpad.people.find(text=email@nospam)03:13
*** tzn has joined #openstack-infra03:13
craigeI'm then using that openID as the authoritive one.03:13
clarkbClint: what is your lp account page?03:13
craigeI hadn't considered multiple IDs.03:13
Clintclarkb: https://launchpad.net/~clint03:14
clarkbhrm I only see the one in the pag esource03:14
Clintright, the other one isn't associated with a launchpad account03:14
craigeI'm using Launchpad rather than Ubuntu One for a pre-xmas reason that escapes me right now.03:16
*** unicell1 has quit IRC03:16
*** cloudtrainme has quit IRC03:17
*** hparekh has joined #openstack-infra03:17
*** thorst has quit IRC03:18
*** tzn has quit IRC03:18
hparekhHi Can anybody review  https://review.openstack.org/#/c/261249/ ?03:20
*** sridhar_ram1 has quit IRC03:22
*** aopincar has joined #openstack-infra03:26
*** salv-orlando has quit IRC03:26
*** salv-orlando has joined #openstack-infra03:27
*** esker has quit IRC03:35
*** yamahata has quit IRC03:36
*** esker has joined #openstack-infra03:37
*** esker has quit IRC03:41
*** rockyg has quit IRC03:44
*** fawadkhaliq has quit IRC03:49
*** rhallisey has quit IRC03:51
*** [1]Thelo has joined #openstack-infra03:52
*** Thelo has quit IRC03:55
*** [1]Thelo is now known as Thelo03:55
*** mudassirlatif has quit IRC03:56
*** aopincar has quit IRC03:57
*** zz_dimtruck is now known as dimtruck03:58
*** ddieterly has joined #openstack-infra03:58
*** thorst has joined #openstack-infra04:02
*** jerryz has quit IRC04:02
*** ddieterly has quit IRC04:03
*** unicell has joined #openstack-infra04:05
*** esker has joined #openstack-infra04:06
*** thorst has quit IRC04:08
*** thorst has joined #openstack-infra04:08
*** salv-orl_ has joined #openstack-infra04:09
*** esker has quit IRC04:11
*** salv-orlando has quit IRC04:11
*** tzn has joined #openstack-infra04:14
*** bharathm has joined #openstack-infra04:15
*** thorst has quit IRC04:17
*** esker has joined #openstack-infra04:17
*** tzn has quit IRC04:19
*** bharathm has quit IRC04:20
*** coolsvap|away is now known as coolsvap04:21
*** esker has quit IRC04:22
*** mrmartin has joined #openstack-infra04:24
*** lykinsbd has joined #openstack-infra04:24
*** lykinsbd has quit IRC04:26
*** otter768 has quit IRC04:28
*** piet has joined #openstack-infra04:29
*** treaki__ has joined #openstack-infra04:44
*** jaosorior has joined #openstack-infra04:45
*** harlowja_at_home has joined #openstack-infra04:45
*** piet has quit IRC04:47
*** treaki_ has quit IRC04:48
*** annegentle has joined #openstack-infra04:49
*** annegentle has quit IRC04:49
openstackgerritMerged openstack/requirements: Bump taskflow minimum version to 1.25.0  https://review.openstack.org/26107404:51
*** sabeen1 has quit IRC04:58
*** ddieterly has joined #openstack-infra04:59
*** zhurong_ has joined #openstack-infra05:02
*** esker has joined #openstack-infra05:03
*** Ramanjaneya has joined #openstack-infra05:03
*** ddieterly has quit IRC05:06
*** zhurong has quit IRC05:06
*** esker has quit IRC05:08
*** mrmartin has quit IRC05:12
*** dimtruck is now known as zz_dimtruck05:14
*** kevinbenton has quit IRC05:14
*** kevinbenton has joined #openstack-infra05:14
*** tzn has joined #openstack-infra05:15
*** thorst has joined #openstack-infra05:15
*** tzn has quit IRC05:19
*** fawadkhaliq has joined #openstack-infra05:20
*** DinaBelova has quit IRC05:21
*** thorst has quit IRC05:21
*** DinaBelova has joined #openstack-infra05:22
*** harlowja_at_home has quit IRC05:23
*** baoli has joined #openstack-infra05:23
*** baoli has quit IRC05:28
*** kevinbenton has quit IRC05:29
*** piet has joined #openstack-infra05:31
*** zhurong_ has quit IRC05:36
*** sharikapongubala has joined #openstack-infra05:37
*** zhurong has joined #openstack-infra05:37
reedspam on the wiki, folks: https://wiki.openstack.org/wiki/Talk:How_To_Use_The_Wiki05:38
reedI blocked the user05:39
reedand deleted its contributions05:40
docaedoWow, googled the number they listed, they've spammed that EVERYWHERE05:41
*** piet has quit IRC05:43
reeddocaedo, everywhere on *.openstack.org?05:45
reedi didn't check... i simply received a notification in my inbox and decided to block that immediately but I am actually about to shutdown my machine and go to bed (early flight tomorrow)05:46
reedi hand this over to you folks :) good night and good luck05:47
docaedoreed: no like Internet - Ubuntu, github, other random wikis (but seems like it's indexed at least a page or three at openstack)05:47
*** ozamiatin has joined #openstack-infra05:48
docaedoThanks for the catch!05:48
*** rajinir has quit IRC05:55
*** esker has joined #openstack-infra05:58
*** ddieterly has joined #openstack-infra06:02
jaosoriorHi, does anyone know if the Is the Fedora 23 node type still not working?06:06
jaosorior*if the Fedora 23...06:06
*** ddieterly has quit IRC06:06
*** salv-orl_ has quit IRC06:08
*** piet has joined #openstack-infra06:08
*** salv-orlando has joined #openstack-infra06:08
*** markvoelker has quit IRC06:11
*** tzn has joined #openstack-infra06:16
*** esker has quit IRC06:18
*** zhurong has quit IRC06:18
*** gampel has joined #openstack-infra06:19
*** zhurong has joined #openstack-infra06:19
*** thorst has joined #openstack-infra06:20
*** tzn has quit IRC06:20
*** aopincar has joined #openstack-infra06:22
*** piet has quit IRC06:24
*** MeisterEckhart has joined #openstack-infra06:26
*** thorst has quit IRC06:26
*** otter768 has joined #openstack-infra06:29
openstackgerritMerged openstack/requirements: Ensure python 3.3 is also listed for enum34  https://review.openstack.org/25919606:30
stevemarhmm, why would a jenkins run not have the script it should have under the slave_scripts directory? http://logs.openstack.org/34/264534/2/experimental/check-osc-plugins/c2d7770/console.html#_2016-01-07_06_13_25_25806:31
stevemarwhen it has it in this run: http://logs.openstack.org/34/264534/1/experimental/check-osc-plugins/b2fd0e5/console.html#_2016-01-07_03_44_52_75606:31
clarkbwhen was it added?06:32
*** MeisterEckhart has quit IRC06:32
clarkbneed images to be built after merging and old nodes to cycle out06:33
*** zhurong_ has joined #openstack-infra06:33
*** otter768 has quit IRC06:33
*** zhurong has quit IRC06:34
*** MeisterEckhart has joined #openstack-infra06:37
*** PsionTheory has quit IRC06:38
*** MeisterEckhart has quit IRC06:39
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/26359206:44
*** harlowja_at_home has joined #openstack-infra06:44
*** vgridnev has joined #openstack-infra06:48
*** tongli has quit IRC06:51
*** unicell1 has joined #openstack-infra06:51
*** jswarren has quit IRC06:52
*** abregman has joined #openstack-infra06:52
*** jswarren has joined #openstack-infra06:52
*** unicell has quit IRC06:52
*** aeng has quit IRC06:52
*** itsuugo has quit IRC06:54
*** irenab has quit IRC06:54
*** itsuugo has joined #openstack-infra06:55
*** coolsvap is now known as coolsvap|away06:56
*** tongli has joined #openstack-infra06:58
*** esker has joined #openstack-infra07:00
*** ddieterly has joined #openstack-infra07:03
*** esker has quit IRC07:05
*** ddieterly has quit IRC07:08
*** markvoelker has joined #openstack-infra07:11
*** yolanda has joined #openstack-infra07:12
*** markvoelker has quit IRC07:16
*** tzn has joined #openstack-infra07:16
*** rraja_ has quit IRC07:17
*** rraja has quit IRC07:17
*** nadya has joined #openstack-infra07:17
*** nadya is now known as Guest8646207:17
TheloHi all in nodepool > buildimage, can Im specify the release version I want? ( example ubuntu Trusty 14.04.2, instead of taking the latest by defaut)07:18
*** kevinbenton has joined #openstack-infra07:18
*** kevinbenton has quit IRC07:20
*** kevinbenton has joined #openstack-infra07:20
*** irenab has joined #openstack-infra07:20
*** tzn has quit IRC07:21
*** camunoz has quit IRC07:23
*** ParsectiX has joined #openstack-infra07:24
*** thorst has joined #openstack-infra07:25
*** kevinbenton has quit IRC07:26
*** pahuang has quit IRC07:27
*** kevinbenton has joined #openstack-infra07:29
*** ParsectiX has quit IRC07:30
*** ParsectiX has joined #openstack-infra07:31
*** harlowja_at_home has quit IRC07:31
*** thorst has quit IRC07:32
*** gampel has left #openstack-infra07:33
*** isaacb has joined #openstack-infra07:39
*** coolsvap|away is now known as coolsvap07:41
*** hdd_ has quit IRC07:41
*** ParsectiX has quit IRC07:42
*** ParsectiX has joined #openstack-infra07:43
*** mrmartin has joined #openstack-infra07:50
flip214the diff view in gerrit says "Press '?' to view keyboard shortcuts" .... but if I do so, a small input with "javascript regex" pops up, I guess that's for searching.07:51
flip214ah, I need to click _below_ the diff part (bottom pane) to make "?" work.07:51
*** irenab has left #openstack-infra07:51
*** jtomasek has joined #openstack-infra07:53
*** fawadkhaliq has quit IRC08:00
*** otter768 has joined #openstack-infra08:00
*** mpavone has joined #openstack-infra08:01
*** otter768_ has joined #openstack-infra08:02
*** ddieterly has joined #openstack-infra08:04
*** salv-orlando has quit IRC08:04
*** otter76__ has joined #openstack-infra08:04
*** otter768 has quit IRC08:04
*** salv-orlando has joined #openstack-infra08:04
*** mpavone has quit IRC08:05
*** otter76__ has quit IRC08:06
*** otter768 has joined #openstack-infra08:06
*** Guest86462 has quit IRC08:06
*** otter768_ has quit IRC08:06
*** matrohon has joined #openstack-infra08:07
*** otter768_ has joined #openstack-infra08:08
*** otter768 has quit IRC08:08
*** ddieterly has quit IRC08:09
*** otter768 has joined #openstack-infra08:10
*** otter768_ has quit IRC08:10
*** gsagie has quit IRC08:11
*** jyuso1 has joined #openstack-infra08:11
*** otter768_ has joined #openstack-infra08:12
*** rraja_ has joined #openstack-infra08:12
*** yamamoto has joined #openstack-infra08:12
*** otter768_ has quit IRC08:13
*** otter768_ has joined #openstack-infra08:14
*** isaacb has quit IRC08:14
*** otter768 has quit IRC08:14
*** otter768_ has quit IRC08:15
*** otter768 has joined #openstack-infra08:15
*** tzn has joined #openstack-infra08:17
*** otter768_ has joined #openstack-infra08:17
*** otter768 has quit IRC08:17
*** otter768_ has quit IRC08:19
*** otter768 has joined #openstack-infra08:19
*** yamamoto has quit IRC08:20
*** otter768_ has joined #openstack-infra08:21
*** otter768 has quit IRC08:21
*** tzn has quit IRC08:21
*** otter768_ has quit IRC08:23
*** otter768 has joined #openstack-infra08:23
*** otter768_ has joined #openstack-infra08:25
openstackgerritIsaac Beckman proposed openstack-infra/puppet-jenkins: Allow setting java heap size in jenkins default  https://review.openstack.org/26278708:25
*** ihrachys has joined #openstack-infra08:27
*** otter76__ has joined #openstack-infra08:27
*** otter768 has quit IRC08:28
*** craige has quit IRC08:29
*** otter768 has joined #openstack-infra08:29
*** jtomasek has quit IRC08:29
*** jtomasek has joined #openstack-infra08:30
*** salv-orlando has quit IRC08:30
*** salv-orlando has joined #openstack-infra08:30
*** otter768_ has quit IRC08:30
*** otter768_ has joined #openstack-infra08:31
*** otter76__ has quit IRC08:31
*** thorst has joined #openstack-infra08:32
*** otter76__ has joined #openstack-infra08:33
*** yaume has joined #openstack-infra08:33
*** otter768 has quit IRC08:33
*** otter768 has joined #openstack-infra08:34
*** otter76__ has quit IRC08:35
*** otter768_ has quit IRC08:36
*** otter768_ has joined #openstack-infra08:36
*** otter768 has quit IRC08:37
*** ParsectiX has quit IRC08:37
*** thorst has quit IRC08:37
*** ParsectiX has joined #openstack-infra08:37
*** isaacb has joined #openstack-infra08:37
openstackgerritMerged openstack-infra/devstack-gate: Use a sparse swapfile on rootfs for providers without ephemeral disks  https://review.openstack.org/26107008:38
*** otter768_ has quit IRC08:38
*** otter768 has joined #openstack-infra08:38
*** zeih has joined #openstack-infra08:38
*** sshnaidm_ has joined #openstack-infra08:39
*** samuelBartel has joined #openstack-infra08:39
*** markus_z has joined #openstack-infra08:40
*** otter768 has quit IRC08:40
*** ifarkas has joined #openstack-infra08:40
*** otter768 has joined #openstack-infra08:40
*** ruagair_ has joined #openstack-infra08:41
*** ruagair_ has quit IRC08:42
*** otter768_ has joined #openstack-infra08:42
*** ruagair_ has joined #openstack-infra08:44
*** otter76__ has joined #openstack-infra08:44
*** otter768 has quit IRC08:44
*** samuelBartel has quit IRC08:45
*** jlanoux has joined #openstack-infra08:46
*** otter76__ has quit IRC08:46
*** otter768 has joined #openstack-infra08:46
*** otter768_ has quit IRC08:47
*** mkoderer_cloud has joined #openstack-infra08:48
*** arxcruz has joined #openstack-infra08:48
*** otter768_ has joined #openstack-infra08:48
*** otter768 has quit IRC08:48
*** dingyichen has quit IRC08:48
*** sharikapongubala has quit IRC08:49
openstackgerritIsaac Beckman proposed openstack-infra/puppet-jenkins: Allow setting java heap size in jenkins default  https://review.openstack.org/26278708:49
*** otter768 has joined #openstack-infra08:50
*** ruagair_ has quit IRC08:50
*** ruagair has joined #openstack-infra08:51
*** otter76__ has joined #openstack-infra08:52
*** otter768 has quit IRC08:52
*** otter768_ has quit IRC08:52
*** esker has joined #openstack-infra08:53
*** otter76__ has quit IRC08:53
*** otter768 has joined #openstack-infra08:53
*** lezbar__ has joined #openstack-infra08:54
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/os-testr: Use assertTrue/False instead of assertEqual(T/F)  https://review.openstack.org/26467208:55
*** otter768_ has joined #openstack-infra08:56
*** jcoufal has joined #openstack-infra08:57
*** esker has quit IRC08:57
*** otter768 has quit IRC08:57
*** otter76__ has joined #openstack-infra08:58
*** samuelBartel has joined #openstack-infra08:58
*** markus_z has quit IRC08:58
*** fhubik has joined #openstack-infra08:59
*** otter768 has joined #openstack-infra08:59
*** otter76__ has quit IRC08:59
*** jordanP has joined #openstack-infra08:59
*** otter768_ has quit IRC09:01
*** otter768_ has joined #openstack-infra09:01
*** markus_z has joined #openstack-infra09:02
*** sfinucan has joined #openstack-infra09:03
*** otter76__ has joined #openstack-infra09:03
*** otter768_ has quit IRC09:03
*** otter768 has quit IRC09:04
*** zeih has quit IRC09:04
*** ddieterly has joined #openstack-infra09:04
*** otter768 has joined #openstack-infra09:05
*** otter76__ has quit IRC09:05
*** vgridnev has quit IRC09:06
*** nadya has joined #openstack-infra09:07
*** otter768_ has joined #openstack-infra09:07
*** nadya is now known as Guest4637509:07
*** otter76__ has joined #openstack-infra09:09
*** otter768_ has quit IRC09:09
*** ddieterly has quit IRC09:09
*** otter768 has quit IRC09:10
*** jogo has quit IRC09:10
*** otter768 has joined #openstack-infra09:11
*** otter76__ has quit IRC09:11
*** Guest46375 has quit IRC09:11
*** markvoelker has joined #openstack-infra09:12
*** otter768 has quit IRC09:13
*** otter768_ has joined #openstack-infra09:13
*** otter768 has joined #openstack-infra09:14
*** otter768_ has quit IRC09:15
*** jyuso1 has quit IRC09:16
*** otter768 has quit IRC09:16
*** otter768 has joined #openstack-infra09:17
*** markvoelker has quit IRC09:17
*** jaosorior has quit IRC09:17
*** otter768 has quit IRC09:18
*** tzn has joined #openstack-infra09:18
*** otter768 has joined #openstack-infra09:18
*** fhubik is now known as fhubik_brb09:19
*** vgridnev has joined #openstack-infra09:20
*** otter768_ has joined #openstack-infra09:20
*** otter768 has quit IRC09:20
*** rossella_s has joined #openstack-infra09:21
*** mrmartin has quit IRC09:22
*** tzn has quit IRC09:22
*** otter768_ has quit IRC09:22
*** otter768 has joined #openstack-infra09:22
*** sridhar_ram has joined #openstack-infra09:23
*** ruagair is now known as craige09:24
*** otter768 has quit IRC09:24
*** otter768_ has joined #openstack-infra09:24
*** vgridnev has quit IRC09:25
*** zeih has joined #openstack-infra09:25
*** otter768_ has quit IRC09:25
*** craige is now known as ruagair09:26
*** otter768 has joined #openstack-infra09:26
*** fhubik_brb is now known as fhubik09:26
*** sridhar_ram has quit IRC09:27
*** zhurong_ has quit IRC09:27
*** otter768_ has joined #openstack-infra09:28
*** otter768 has quit IRC09:28
*** otter768 has joined #openstack-infra09:30
*** otter768_ has quit IRC09:30
*** oomichi has quit IRC09:32
*** otter768_ has joined #openstack-infra09:32
*** jaosorior has joined #openstack-infra09:32
*** daemontool has joined #openstack-infra09:32
*** otter76__ has joined #openstack-infra09:34
*** jkt has quit IRC09:34
*** otter768 has quit IRC09:34
*** derekh has joined #openstack-infra09:35
*** shardy has joined #openstack-infra09:35
*** otter768 has joined #openstack-infra09:35
*** otter76__ has quit IRC09:35
*** thorst has joined #openstack-infra09:36
*** otter768_ has quit IRC09:36
*** jogo has joined #openstack-infra09:37
*** jogo has joined #openstack-infra09:37
*** otter768_ has joined #openstack-infra09:37
*** otter768_ has quit IRC09:39
*** otter76__ has joined #openstack-infra09:39
*** otter768 has quit IRC09:40
*** jistr has joined #openstack-infra09:41
*** otter76__ has quit IRC09:41
*** otter768 has joined #openstack-infra09:41
*** yamahata has joined #openstack-infra09:41
*** thorst has quit IRC09:42
*** otter768_ has joined #openstack-infra09:43
*** otter768 has quit IRC09:43
*** otter768 has joined #openstack-infra09:45
*** otter76__ has joined #openstack-infra09:47
*** otter768_ has quit IRC09:48
*** tongli has quit IRC09:48
*** otter768_ has joined #openstack-infra09:49
*** otter76__ has quit IRC09:49
*** otter768 has quit IRC09:50
*** otter768_ has quit IRC09:51
*** otter768 has joined #openstack-infra09:51
*** tzn has joined #openstack-infra09:51
*** ozamiatin has quit IRC09:52
*** yamahata has quit IRC09:53
openstackgerritMerged openstack-infra/subunit2sql: Add new graph type to show run_time variance by metadata  https://review.openstack.org/21056909:53
*** otter768_ has joined #openstack-infra09:53
*** kashyap has joined #openstack-infra09:54
kashyapHi, is there a way I can apply a topic-based patch series from Gerrit in command to a local branch via testing?  (I know how to do this from mailing lists, but not from Gerrit.)09:55
*** otter76__ has joined #openstack-infra09:55
*** otter768 has quit IRC09:56
*** otter768 has joined #openstack-infra09:56
*** otter76__ has quit IRC09:57
*** otter768_ has quit IRC09:57
*** otter768_ has joined #openstack-infra09:58
*** rguillebert has joined #openstack-infra10:00
*** otter768_ has quit IRC10:00
*** otter768_ has joined #openstack-infra10:00
*** otter768 has quit IRC10:01
*** otter768 has joined #openstack-infra10:02
*** otter768_ has quit IRC10:02
*** dtantsur|afk is now known as dtantsur10:03
*** otter768_ has joined #openstack-infra10:04
*** otter768 has quit IRC10:04
*** ddieterly has joined #openstack-infra10:05
*** otter768_ has quit IRC10:05
*** ruagair has quit IRC10:06
*** otter768 has joined #openstack-infra10:06
*** craige_ has joined #openstack-infra10:07
*** otter768 has quit IRC10:08
*** otter768 has joined #openstack-infra10:08
*** salv-orl_ has joined #openstack-infra10:08
*** craige_ is now known as craige10:09
*** dizquierdo has joined #openstack-infra10:10
*** otter768_ has joined #openstack-infra10:10
*** ddieterly has quit IRC10:10
*** sputnik13 has joined #openstack-infra10:10
*** salv-orlando has quit IRC10:11
*** otter768_ has quit IRC10:11
*** otter768_ has joined #openstack-infra10:12
*** otter768 has quit IRC10:13
*** otter768 has joined #openstack-infra10:14
*** otter768_ has quit IRC10:14
*** electrofelix has joined #openstack-infra10:15
*** otter768_ has joined #openstack-infra10:16
*** otter768 has quit IRC10:16
*** fawadkhaliq has joined #openstack-infra10:16
*** otter768_ has quit IRC10:17
*** otter768 has joined #openstack-infra10:17
*** otter768 has quit IRC10:19
*** abregman is now known as abregman|afk10:19
*** otter768 has joined #openstack-infra10:19
*** otter768_ has joined #openstack-infra10:21
*** otter768 has quit IRC10:21
*** rossella_s has quit IRC10:22
*** rossella_s has joined #openstack-infra10:22
*** Qiming has quit IRC10:22
*** otter768_ has quit IRC10:23
*** otter768 has joined #openstack-infra10:23
*** otter768_ has joined #openstack-infra10:25
*** otter768 has quit IRC10:26
*** otter768_ has quit IRC10:27
*** otter768 has joined #openstack-infra10:27
*** abregman|afk is now known as abregman10:27
*** ramineni1 has joined #openstack-infra10:27
ramineni1sdague: hi , around?10:28
*** sputnik13 has quit IRC10:28
*** jcoufal has quit IRC10:28
*** sputnik13 has joined #openstack-infra10:28
*** sputnik13 has quit IRC10:28
*** otter768 has quit IRC10:28
*** otter768 has joined #openstack-infra10:29
*** zeih has quit IRC10:30
*** otter768_ has joined #openstack-infra10:31
*** otter768 has quit IRC10:31
*** abregman is now known as abregman|afk10:33
*** jaosorior has quit IRC10:34
*** jaosorior has joined #openstack-infra10:34
*** otter768_ has quit IRC10:35
*** jkt has joined #openstack-infra10:37
*** apoorvad has joined #openstack-infra10:40
*** thorst has joined #openstack-infra10:41
*** esker has joined #openstack-infra10:41
TheloHi all, in my case, nodepool image-build seems to be stuck at "No handlers could be found for logger "oslo_db.sqlalchemy.session"" (http://paste.openstack.org/show/483214/)10:42
Thelois this supposed to retry10:42
Theloits been more than 15 minutes10:43
Thelo?10:43
*** esker has quit IRC10:45
*** sdake has joined #openstack-infra10:46
*** thorst has quit IRC10:47
*** ociuhandu has quit IRC10:51
*** hashar has joined #openstack-infra10:59
*** zeih has joined #openstack-infra11:00
*** tzn has quit IRC11:04
openstackgerritMerged openstack-infra/jenkins-job-builder: Copy config files tree  https://review.openstack.org/24368911:05
openstackgerritMerged openstack-infra/jenkins-job-builder: Add support of 'somewhereElse' value to Copy to slave plugin  https://review.openstack.org/23925011:05
openstackgerritMerged openstack-infra/jenkins-job-builder: Remove deprecated getChildren() usage  https://review.openstack.org/24376811:06
*** ddieterly has joined #openstack-infra11:06
*** salv-orl_ has quit IRC11:07
*** salv-orlando has joined #openstack-infra11:08
*** jordanP has quit IRC11:09
*** dims has joined #openstack-infra11:10
*** ddieterly has quit IRC11:11
*** coolsvap is now known as coolsvap|away11:11
*** zeih has quit IRC11:11
*** markvoelker has joined #openstack-infra11:13
*** Qiming has joined #openstack-infra11:14
*** markvoelker has quit IRC11:18
*** tlian has joined #openstack-infra11:18
*** ociuhandu has joined #openstack-infra11:19
*** lucas-dinner is now known as lucasagomes11:20
*** abregman|afk is now known as abregman11:21
openstackgerritThomas Goirand proposed openstack-infra/project-config: Debian image  https://review.openstack.org/26472611:23
*** ramineni1 has left #openstack-infra11:26
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: Add mimic to global requirements to use in tests  https://review.openstack.org/22026811:29
*** zeih has joined #openstack-infra11:29
*** sdake has quit IRC11:36
*** thorst has joined #openstack-infra11:45
*** BobBall_AWOL is now known as BobBall11:45
*** fhubik is now known as fhubik_brb11:46
BobBallHas anyone seen failures relating to RYU's requirements?  https://github.com/openstack/requirements/blob/master/global-requirements.txt requires ryu>=3.23.2 and oslo.config>3.2.0, but pip-requires for ryu 3.29 now has oslo.config<=3.0.011:48
*** rfolco has joined #openstack-infra11:52
*** thorst has quit IRC11:52
*** devananda has quit IRC11:54
*** devananda has joined #openstack-infra11:55
*** esker has joined #openstack-infra11:56
dimsBobBall : no i haven't in fact we just +2A'ed this one https://review.openstack.org/#/c/264026/11:57
dimsBobBall : looks like it's coming from here https://github.com/osrg/ryu/commit/1ed92b25b5997f804dd70a4f072fd20f6af618e011:59
BobBallHow does that work? :) haha11:59
BobBallYes; but it means that neutronclient can't work I think?11:59
dimsBobBall : i just blocked it now11:59
BobBallhttp://paste.openstack.org/show/483223/ is what our internal neutron CI gives11:59
*** apoorvad has quit IRC12:00
*** esker has quit IRC12:00
BobBallI really don't understand why ryu 3.29 is working in the gate at all...12:00
dimsBobBall : which was the last version that worked?12:01
BobBallbut certainly neutron jobs seem to be using 3.29 (e.g. http://logs.openstack.org/22/254022/11/check/gate-tempest-dsvm-neutron-full/e033211/logs/devstacklog.txt.gz) - which passed12:01
dimswe should probably do this on neutron channel12:02
BobBallSure12:02
*** hichihara has joined #openstack-infra12:04
dimsBobBall : do the commands that use ryu actually run in upstream CI? (/usr/local/bin/neutron-rootwrap-xen-dom0)12:06
*** ddieterly has joined #openstack-infra12:07
BobBallneutron-rootwrap-xen-dom0 doesn't use ryu - it just seems to be checking the installed requirements of Neutron when loading12:07
dimsBobBall : next step, please log a bug in neutron with the details - https://bugs.launchpad.net/neutron/12:07
dimsBobBall : and then we can black list 3.29 in global requirements12:07
openstackgerritLucas Alvares Gomes proposed openstack/diskimage-builder: deploy-ironic: Fix syntax error when checking for root device hints  https://review.openstack.org/26473612:07
BobBallWill do.12:08
*** nadya has joined #openstack-infra12:09
*** nadya is now known as Guest9029312:09
BobBalldims: Against neutron or global-requirements?12:11
*** ddieterly has quit IRC12:11
BobBallwell I can't find a global requirements project atm... I'd already raised it against Neutron - see https://bugs.launchpad.net/neutron/+bug/153183712:11
openstackLaunchpad bug 1531837 in neutron "ryu 3.29 incompatible with oslo.config" [Undecided,New]12:11
dimsBobBall : let's do neutron, not sure if we have a bug tracker for global12:11
*** zhurong has joined #openstack-infra12:12
*** ajmiller_ has joined #openstack-infra12:12
*** ajmiller has quit IRC12:12
*** Guest90293 has quit IRC12:13
*** markvoelker has joined #openstack-infra12:14
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: ryu 3.29 incompatible with oslo.config  https://review.openstack.org/26474112:14
*** nadya has joined #openstack-infra12:17
*** nadya is now known as Guest3344212:17
*** apoorvad has joined #openstack-infra12:17
*** esker has joined #openstack-infra12:17
*** hichihara has quit IRC12:17
*** kaisers has quit IRC12:17
*** markvoelker has quit IRC12:18
*** Guest33442 is now known as _nadya_12:21
*** apoorvad has quit IRC12:22
*** esker has quit IRC12:22
*** mrmartin has joined #openstack-infra12:25
*** jcoufal has joined #openstack-infra12:27
*** fhubik_brb is now known as fhubik12:30
*** gordc has joined #openstack-infra12:31
*** otter768 has joined #openstack-infra12:32
*** tsg has joined #openstack-infra12:32
*** kaisers has joined #openstack-infra12:33
openstackgerritAndrea Frittoli proposed openstack/requirements: Add inflection to requirements  https://review.openstack.org/25958812:34
*** otter768 has quit IRC12:37
*** tzn has joined #openstack-infra12:38
*** doug-fish has joined #openstack-infra12:38
*** amrith is now known as _amrith_12:39
*** apoorvad has joined #openstack-infra12:39
*** nmagnezi_ has joined #openstack-infra12:39
*** yaume_ has joined #openstack-infra12:39
*** baoli has joined #openstack-infra12:41
*** yaume has quit IRC12:42
*** apoorvad has quit IRC12:43
openstackgerritMerged openstack-infra/project-config: Add Job for mistral-docker-imagebuild  https://review.openstack.org/26124912:45
*** thorst has joined #openstack-infra12:47
*** resker has joined #openstack-infra12:50
*** markvan has quit IRC12:51
*** resker has quit IRC12:52
*** markvan has joined #openstack-infra12:53
*** ParsectiX has quit IRC12:53
openstackgerritThierry Carrez proposed openstack/requirements: Add missing license information  https://review.openstack.org/26475412:53
*** woodster_ has joined #openstack-infra12:54
*** claudiub has joined #openstack-infra13:01
*** ParsectiX has joined #openstack-infra13:02
*** salv-orlando has quit IRC13:03
*** salv-orlando has joined #openstack-infra13:03
*** fawadkhaliq has quit IRC13:05
*** zhurong_ has joined #openstack-infra13:05
*** amuller has joined #openstack-infra13:05
*** doug-fish has quit IRC13:05
*** fawadkhaliq has joined #openstack-infra13:05
*** zhurong has quit IRC13:05
*** gongysh has joined #openstack-infra13:06
*** _nadya_ has quit IRC13:06
*** dprince has joined #openstack-infra13:07
*** fhubik is now known as fhubik_brb13:07
*** ddieterly has joined #openstack-infra13:07
*** links has quit IRC13:07
*** fhubik_brb is now known as fhubik13:09
*** weshay_xchat has joined #openstack-infra13:10
dtantsuryolanda, hi! could you reapprove https://review.openstack.org/#/c/258736/ ? the dependencies has merged, but it needs to be kicked to the gate13:10
*** nmagnezi_ has quit IRC13:11
dtantsur(actually we may even need to do something like rebase + reapprove)13:11
*** nmagnezi_ has joined #openstack-infra13:11
*** ddieterly has quit IRC13:12
yolandadtantsur , can you try a recheck comment there? it may do the trick13:12
dtantsuryolanda, last time it didn't work for me, but I can try13:13
*** dizquierdo has quit IRC13:15
*** fawadkhaliq has quit IRC13:15
*** fawadkhaliq has joined #openstack-infra13:16
*** Thelo has quit IRC13:16
*** Thelo has joined #openstack-infra13:16
*** claudiub has quit IRC13:17
*** unicell1 has quit IRC13:17
*** unicell has joined #openstack-infra13:17
*** markvoelker has joined #openstack-infra13:18
*** fawadk has joined #openstack-infra13:19
openstackgerritMerged openstack-infra/puppet-infra-cookiecutter: Update Infra URL  https://review.openstack.org/26422413:19
*** Ramanjaneya has quit IRC13:21
openstackgerritgordon chung proposed openstack-infra/project-config: fix stable/liberty telemetry integration job  https://review.openstack.org/26476913:21
*** fawadkhaliq has quit IRC13:21
*** erlon has joined #openstack-infra13:22
openstackgerritZara proposed openstack-infra/storyboard-webclient: Add unlimited pagination option for preferences  https://review.openstack.org/26477013:22
*** edmondsw has joined #openstack-infra13:22
openstackgerritMerged openstack-infra/puppet-grafyaml: Update Infra URL  https://review.openstack.org/26422513:24
*** doug-fish has joined #openstack-infra13:24
*** rhallisey has joined #openstack-infra13:25
openstackgerritThierry Carrez proposed openstack/requirements: Add missing license information  https://review.openstack.org/26475413:28
openstackgerritMerged openstack-infra/puppet-odsreg: Update Infra URL  https://review.openstack.org/26423113:29
openstackgerritMerged openstack-infra/puppet-infracloud: Update Infra URL  https://review.openstack.org/26422713:29
dtantsuryolanda, like I suspected: https://review.openstack.org/#/c/258736/ recheck worked, but it didn't enter the gate13:31
dtantsuryolanda, maybe you try clicking "rebase" button then just reapprove?13:32
*** aopincar has quit IRC13:33
*** apoorvad has joined #openstack-infra13:34
*** sdake has joined #openstack-infra13:34
*** trown|outttypeww is now known as trown13:35
openstackgerritLucas Alvares Gomes proposed openstack/diskimage-builder: Install IPA in a virtual enviroment  https://review.openstack.org/23584313:36
*** zz_dimtruck is now known as dimtruck13:37
*** apoorvad has quit IRC13:38
*** sharikapongubala has joined #openstack-infra13:39
*** moravec has quit IRC13:40
*** Daisy has joined #openstack-infra13:40
*** rhallisey has quit IRC13:40
*** rhallisey has joined #openstack-infra13:43
*** rhallisey has quit IRC13:45
*** rhallisey has joined #openstack-infra13:46
*** ddieterly has joined #openstack-infra13:47
*** jordanP has joined #openstack-infra13:47
*** ddieterl_ has joined #openstack-infra13:51
*** kgiusti has joined #openstack-infra13:51
*** kgiusti has left #openstack-infra13:51
*** dimtruck is now known as zz_dimtruck13:51
*** kgiusti has joined #openstack-infra13:51
*** _nadya_ has joined #openstack-infra13:52
yolandadone13:53
*** doug-fis_ has joined #openstack-infra13:53
*** [1]Thelo has joined #openstack-infra13:53
pc_myolanda: hi13:54
*** ddieterly has quit IRC13:54
*** doug-fish has quit IRC13:55
*** fhubik is now known as fhubik_brb13:56
*** Thelo has quit IRC13:56
*** [1]Thelo is now known as Thelo13:56
*** rhallisey has quit IRC13:57
openstackgerritMerged openstack/requirements: Bump the lower boundary of pysnmp to 4.2.3  https://review.openstack.org/24656413:57
*** fhubik_brb is now known as fhubik13:57
yolandapc_m, hi13:58
pc_myolanda: Thanks for the reviews on constraint based jobs for VPN and skip-if condition reviews. Are they ready for +A?13:58
*** fhubik is now known as fhubik_brb13:59
yolandalet me double check the reviews for neutron liaisons, i think they are ok yes14:01
pc_myolanda: Yes, Doug +1ed each. Thanks!14:01
yolandacool let me approve14:01
openstackgerritDirk Mueller proposed openstack-infra/project-config: Enable docs/pypi publish for openstack/pymod2pkg  https://review.openstack.org/26478114:01
*** bcourt has quit IRC14:01
openstackgerritMerged openstack-infra/project-config: Add reno jobs for ironic-python-agent  https://review.openstack.org/25873614:02
*** xyang1 has joined #openstack-infra14:03
*** rhallisey has joined #openstack-infra14:04
*** claudiub has joined #openstack-infra14:04
*** zeih has quit IRC14:05
*** zeih_ has joined #openstack-infra14:06
*** zeih has joined #openstack-infra14:07
*** fhubik_brb is now known as fhubik14:07
pc_myolanda: Thanks a lot for reviewing!14:08
yolandayou are welcome14:08
*** zeih_ has quit IRC14:09
*** zeih_ has joined #openstack-infra14:09
*** ddieterl_ has quit IRC14:09
*** fawadk has quit IRC14:10
*** zeih_ has quit IRC14:10
*** zeih has quit IRC14:10
*** esker has joined #openstack-infra14:11
*** mkoderer_cloud has quit IRC14:11
openstackgerritMerged openstack-infra/project-config: Skip py27/34 jobs when VPN doc (only) change  https://review.openstack.org/26340914:14
*** zeih has joined #openstack-infra14:14
*** zeih_ has joined #openstack-infra14:15
openstackgerritMerged openstack-infra/project-config: Constraint based experimental jobs for neutron-vpnaas  https://review.openstack.org/26339814:15
*** bcourt has joined #openstack-infra14:16
*** dkranz has joined #openstack-infra14:17
*** vgridnev has joined #openstack-infra14:17
*** amitgandhinz has joined #openstack-infra14:17
*** mriedem has joined #openstack-infra14:17
*** bhunter71 has quit IRC14:18
*** zeih has quit IRC14:19
*** esker has quit IRC14:20
*** esker has joined #openstack-infra14:21
*** scheuran_ has joined #openstack-infra14:21
*** rossella_s has quit IRC14:21
*** rossella_s has joined #openstack-infra14:22
*** zeih has joined #openstack-infra14:22
*** tsg has quit IRC14:22
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Summit Application API  https://review.openstack.org/22196414:22
*** regXboi has joined #openstack-infra14:24
*** mpavone has joined #openstack-infra14:24
*** mpavone has quit IRC14:25
*** zeih_ has quit IRC14:25
scheuran_Hi. I submitted a small doc fix to infra 3 weeks ago. https://review.openstack.org/#/c/257501/ Can anybody please have a look? Or do I need a bug or something like this?14:26
*** peristeri has joined #openstack-infra14:27
hasharscheuran_: I usually look at the git log history for the file and add recent authors as reviewers14:27
scheuran_hashar, good idea, thanks!14:28
*** amitgandhinz has quit IRC14:29
*** amitgandhinz has joined #openstack-infra14:29
*** _amrith_ is now known as amrith14:30
*** ari has joined #openstack-infra14:31
openstackgerritZara proposed openstack-infra/storyboard-webclient: Add unlimited pagination option for preferences  https://review.openstack.org/26477014:31
*** jsavak has joined #openstack-infra14:32
*** lucasagomes is now known as lucas-hungry14:32
*** otter768 has joined #openstack-infra14:32
*** sharikapongubala has quit IRC14:33
*** daemontool_ has joined #openstack-infra14:33
*** zeih has quit IRC14:33
*** daemontool has quit IRC14:34
*** apoorvad has joined #openstack-infra14:35
*** melania has joined #openstack-infra14:35
*** zeih has joined #openstack-infra14:35
*** bdemers_ has joined #openstack-infra14:36
*** abregman has quit IRC14:36
anteayakashyap: if you run `git review -d <gerrit patch id,patchset number>` in the root of the repo the patch applied to you will get download the patch and all its children locally14:37
*** otter768 has quit IRC14:38
anteayakashyap: git review doesn't pull down patches based on gerrit topic that I know of14:38
*** zz_dimtruck is now known as dimtruck14:38
*** zhurong_ has quit IRC14:39
kashyapanteaya: What do you mean "patch id"?14:40
kashyapanteaya: I only see patch number and individual Gerrit change IDs for respective patches.14:40
*** apoorvad has quit IRC14:40
*** moravec has joined #openstack-infra14:41
*** ddieterly has joined #openstack-infra14:42
*** ddieterly has quit IRC14:42
anteayahttps://review.openstack.org/#/c/247714/ let's say I want to test patchset 3 of this patch locally14:42
*** ddieterly has joined #openstack-infra14:42
anteayaI execute git review -d 247714,314:42
anteayaif i want the latest I execute git review -d 24771414:42
anteayaif I additionally want to test https://review.openstack.org/#/c/264497/ with it14:43
anteayaI can execute git review -x 264497 after git review -d 24771414:43
anteaya-x is cherry pick14:43
anteayaoh I'm wrong14:44
anteayadifferent repos14:44
anteayaso I would have to switch repos for the cherry pick14:44
openstackgerritsebastian marcet proposed openstack-infra/openstackid: Fix on Resource server banning:  https://review.openstack.org/26480214:45
*** fawadkhaliq has joined #openstack-infra14:47
*** esker has quit IRC14:48
*** esker has joined #openstack-infra14:48
openstackgerritMerged openstack-infra/jenkins-job-builder: Warn only once for masked definitions  https://review.openstack.org/26424114:49
*** dslevin_ has quit IRC14:50
*** jcoufal_ has joined #openstack-infra14:50
pabelangeranteaya: Ya, but it is system wide. So, you'd have to update the yaml file to include your new color settings.  I think that logic is exposed... if not, I can add the code into grafyaml.14:50
*** jsavak has quit IRC14:51
*** jcoufal has quit IRC14:53
*** eharney has quit IRC14:54
*** hashar has quit IRC14:55
*** piet has joined #openstack-infra14:57
*** baoli has quit IRC14:58
*** dprince has quit IRC14:58
*** dprince has joined #openstack-infra14:59
*** dizquierdo has joined #openstack-infra14:59
anteayaoh so we would have to agree as a group what the colours should be should we change them?15:00
openstackgerritMerged openstack-infra/openstackid: Fix on Resource server banning:  https://review.openstack.org/26480215:00
*** ParsectiX has quit IRC15:00
*** baoli has joined #openstack-infra15:01
*** pradk has joined #openstack-infra15:01
*** isaacb has quit IRC15:01
*** annegentle has joined #openstack-infra15:03
*** dslev has joined #openstack-infra15:03
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Mention Grafana in README.rst  https://review.openstack.org/26482215:03
pabelangeranteaya: Yup, it would be the same as we do graphite today.  FWIW, I just defaulted it to what ever grafana picks for no reason.  we could simply copy over the existing color schema15:03
pabelangeryolanda: do you mind reviewing? https://review.openstack.org/#/c/229232/15:04
yolandasorry, on a call for a while15:04
yolandai'll take a look15:04
pabelangermordred: FYI, we finally landed the python-shade RPM in fedora rawhide. Working on getting it into epel7 next15:05
*** woodster_ has quit IRC15:06
*** mrmartin has quit IRC15:07
anteayapabelanger: you lost me on "we could simply copy over the existing colour schema" do you mean just use the colours that graphite is using?15:08
anteayaif so, that's fine15:08
*** jsavak has joined #openstack-infra15:09
*** bcourt has quit IRC15:09
*** mriedem is now known as mriedem_meeting15:10
*** eharney has joined #openstack-infra15:10
pabelangeranteaya: I was referring to the colors we use here:15:10
pabelangerhttp://graphite.openstack.org/render/?from=-24hours&height=180&until=now&width=334&bgcolor=ffffff&fgcolor=000000&areaMode=stacked&target=color(alias(sumSeries(stats.gauges.nodepool.nodes.building),%20%27Building%27),%20%27ffbf52%27)&target=color(alias(sumSeries(stats.gauges.nodepool.nodes.ready),%20%27Available%27),%20%2700c868%27)&target=color(alias(sumSeries(stats.gauges.nodepool.nodes.used),%20%2715:11
pabelangerIn%20Use%27),%20%276464ff%27)&target=color(alias(sumSeries(stats.gauges.nodepool.nodes.delete),%20%27Deleting%27),%20%27c864ff%27)&title=Test%20Nodes&_t=0.167118133744224915:11
pabelangerugh15:11
pabelangershould have shorten that15:11
*** doug-fis_ is now known as doug-fish15:11
anteayaI think clarkb wants to move to logstash 2.0 real soon now, if some of infra-root care to review this patch please: https://review.openstack.org/#/c/247714/15:11
anteayapabelanger: I'll get it from the logs, but yeah graphite urls tend to be fairly lengthy15:12
anteayayep, those colours would be great15:13
EmilienMhello infra :-) I have a quick request for Puppet jobs, please look https://review.openstack.org/#/c/264196/ -- thanks!15:14
*** bcourt has joined #openstack-infra15:14
*** jaypipes has joined #openstack-infra15:15
EmilienMpabelanger: hey, I saw https://review.openstack.org/#/c/262050 -- should I do the same for https://github.com/openstack/puppet-openstack_extras/blob/master/manifests/repo/redhat/redhat.pp#L115 ?15:17
*** flepied1 has joined #openstack-infra15:17
*** hdd has joined #openstack-infra15:18
*** yamahata has joined #openstack-infra15:18
pabelangerEmilienM: might be worth it. I think dl.f.o is the new default15:19
openstackgerritMerged openstack-infra/project-config: Grafana: Add per-region node graphs  https://review.openstack.org/26449715:19
*** mtanino has joined #openstack-infra15:19
EmilienMpabelanger: ack thx15:19
*** flepied has quit IRC15:19
*** med_` has joined #openstack-infra15:20
*** med_ has quit IRC15:20
*** jtomasek has quit IRC15:20
*** Daisy_ has joined #openstack-infra15:21
fungidtantsur: yolanda: rechecking a change which already has a verify +1 won't enqueue it into the gate, but adding another workflow +1 will (removing your previous workflow +1 first if necessary)15:22
*** Daisy has quit IRC15:22
yolandafungi, just learned it, that's what i had to do15:22
EmilienMpabelanger: in fact, most of deployments use metalink property https://github.com/openstack/puppet-openstack_extras/blob/master/manifests/repo/redhat/redhat.pp#L107 -- so I think we don't need to change URL15:23
fungiyolanda: yeah, it's a known limitation of gerrit currently that applying a vote (for example zuul applying a verify +1) when the same value for that vote already exists for that account won't cause the vote to appear in the comment event on gerrit's event stream, so zuul doesn't see it and therefore doesn't enqueue15:23
yolandai thought a recheck caused it to read even existing votes, but i was wrong15:24
*** jtomasek has joined #openstack-infra15:25
pabelangerEmilienM: Ya, IIRC, we had mirror issues with DIB, so we just pinned to the main download site15:25
openstackgerritMerged openstack-infra/project-config: run integration jobs for puppet-ironic  https://review.openstack.org/26419615:26
EmilienMpabelanger: why not using metalink?15:26
*** Daisy_ has quit IRC15:27
*** Daisy has joined #openstack-infra15:27
*** nelsnelson has quit IRC15:28
*** nelsnelson has joined #openstack-infra15:29
*** Daisy has quit IRC15:29
*** Daisy has joined #openstack-infra15:29
*** tonytan4ever has joined #openstack-infra15:29
*** sshnaidm_ has quit IRC15:30
pabelangerEmilienM: would need to confirm, but I think when a mirror was picked there was a chance it was down.15:30
pabelangerbut would need to check git log to be sure of reason15:30
*** annegentle has quit IRC15:30
fungiyolanda: recheck comments don't cause zuul to reread anything (votes included), they just reenqueue the change directly into the check pipeline15:30
*** annegentle has joined #openstack-infra15:31
*** dprince has quit IRC15:32
*** isaacb has joined #openstack-infra15:32
fungier, i should have said don't cause zuul to reevaluate the change's state15:32
*** dprince_ has joined #openstack-infra15:32
EmilienMpabelanger: cool thx15:33
fungiso if the change has a verify -1 and a workflow +1 and you recheck it will enqueue into the gate after passing check jobs the next time, but if it already has a verify +1 then you actually need to toggle the workflow vote15:33
*** zeih has quit IRC15:33
*** lucas-hungry is now known as lucasagomes15:34
openstackgerritChristoph Albers proposed openstack-infra/project-config: Removed testjobs and corresponding chef-jobs  https://review.openstack.org/26413015:34
openstackgerritDirk Mueller proposed openstack-infra/project-config: Enable docs publish for openstack/pymod2pkg  https://review.openstack.org/26478115:34
*** dims_ has joined #openstack-infra15:35
*** ozamiatin has joined #openstack-infra15:35
*** bdemers_ has quit IRC15:35
*** dims has quit IRC15:36
isaacbfungi: Hi, can you please look in: https://review.openstack.org/#/c/239363/15:39
*** zeih has joined #openstack-infra15:39
*** sigmavirus24_awa is now known as sigmavirus2415:42
*** mriedem_meeting is now known as mriedem15:42
isaacbfungi: thanks.15:42
*** bhunter71 has joined #openstack-infra15:43
*** [patrol] has joined #openstack-infra15:48
openstackgerritChristoph Albers proposed openstack-infra/project-config: Removed testjobs and corresponding chef-jobs  https://review.openstack.org/26413015:49
[patrol]hey, I have few issues with jenkins-job-builder15:50
[patrol]1. It does not generate "plugin=..." part  as in <hudson.plugins.parameterizedtrigger.BuildTrigger plugin="parameterized-trigger@2.30">15:50
*** electrofelix has quit IRC15:50
*** dprince_ has quit IRC15:51
*** dprince has joined #openstack-infra15:51
*** tongli has joined #openstack-infra15:51
[patrol]2. even if generated xml file is the same as configured by gui some parts may not appear in "configure" tab15:51
*** electrofelix has joined #openstack-infra15:51
*** ggillies has quit IRC15:51
*** ggillies has joined #openstack-infra15:52
[patrol]manual configuration will result in exactly the same xml as submitted in update15:52
pabelanger[patrol]: what is the expect XML format look like?  Is there sufficient unit test coverage in JJB?15:53
pabelanger[patrol]: possible something is not escaped properly?15:53
[patrol]1. expected xml format is as quoted earlier. i.e. ws-cleanup has hardcoded plugin version 0.1415:54
[patrol]current is 0.28 -.-15:54
yolandainfra-root, are you aware of dependency problems when installing zuul?15:54
yolandai get an error on pbr, with " ImportError: No module named testrepository"15:55
pabelangeryolanda: which dependency?15:55
pabelangerthat is PBR issue I think15:55
pabelangerwe should bump it15:55
yolandayep, got it on a clean install of zuul15:55
pabelangerI thought it was fixed... recent15:55
clarkbwe did bump it15:55
yolandamm, i may be outdated with my zuul code15:55
pabelangerYa, back on dec 3015:55
pabelanger[patrol]: easier to see logs of the problem15:56
openstackgerritChristoph Albers proposed openstack-infra/project-config: Removed testjobs and corresponding chef-jobs  https://review.openstack.org/26413015:56
yolandapabelanger, clarkb, thanks. I am out of sync15:56
jeblairmordred: good morning!15:57
*** apoorvad has joined #openstack-infra15:57
[patrol]pabelanger: how retreive logs? what log levels are supported? how do I find tox?15:59
*** apoorvad has quit IRC16:02
*** lezbar__ has quit IRC16:02
*** Daisy has quit IRC16:02
openstackgerritJames E. Blair proposed openstack-infra/project-config: Grafana: Add node sparklines to rax graph  https://review.openstack.org/26484516:02
mordredmorning jeblair !16:02
*** moravec has quit IRC16:03
*** Daisy has joined #openstack-infra16:03
jeblairpabelanger: did you see https://review.openstack.org/264331 ?  and do you think there's anyone else that might want to review those, or should we go ahead and merge them?16:03
jeblairmordred: hey!  want to work on that puppetmaster stuff?16:03
*** ihrachys has quit IRC16:04
mordredjeblair: I do. I llove working on puppetmaster stuff16:05
*** moravec has joined #openstack-infra16:06
jeblairmordred: for starters, are there any outstanding patches i should review?16:06
mordrednope. I believe it's all landed16:07
*** Daisy has quit IRC16:07
jeblairoh good16:08
jeblairwell' i'm landing some old patches of yours anyways :)16:08
*** lezbar has joined #openstack-infra16:08
mordredoh - cool - actually ...16:08
*** salv-orl_ has joined #openstack-infra16:08
pabelangerjeblair: approved16:08
jeblairpabelanger: cool, i figure they've aged long enough :)16:08
mordredjeblair: checking about the inventory patch16:10
*** hdd has quit IRC16:10
openstackgerritMerged openstack-infra/grafyaml: Add support for Y axis labels  https://review.openstack.org/26431416:10
anteayammmmm well aged patches16:11
*** salv-orlando has quit IRC16:11
anteayalike fine wine16:11
*** sean-k-mooney has left #openstack-infra16:11
*** Qiming has quit IRC16:12
*** bcourt has quit IRC16:13
openstackgerritMerged openstack-infra/grafyaml: Add support for Y axis formats  https://review.openstack.org/26433116:13
mordredjeblair: steps needed to get hostname-based logs - we need to add ansible: use_hostnames: True to clouds.yaml16:13
*** woodster_ has joined #openstack-infra16:13
jeblairhttps://etherpad.openstack.org/p/ansible-system-config16:14
jeblairmordred: i also pasted in a common error in the logs --16:15
jeblairmordred: it looks like we're trying to ssh to host "2001", which is the first part of an ipv6 addr16:16
jeblairmordred: this might be related, but doesn't quite seem the same thing as the inventory/hostname stuff16:16
*** dtardivel has joined #openstack-infra16:16
mordredjeblair: oh fun16:17
jeblairmordred: i assume that our rsync is working in general, so maybe we also rsync to the ipv4 addr later?  (i'll check)16:17
openstackgerritMerged openstack-infra/system-config: Add script to downgrade gerrit from 2.10  https://review.openstack.org/24008916:17
mordredjeblair: seems like a bug in the ansible rsync module and quoting16:17
*** zeih has quit IRC16:19
jeblairmordred: hrm, actually, it kind of looks like almost all of that is failing16:19
*** fhubik has quit IRC16:19
pabelangermordred: jeblair: centos7 by chance?16:19
pabelangerI've seen that error if rsync was lacking functionality16:20
pabelangerlike --chmod16:20
mordredjeblair: I';m thrilled that that's stopped working16:20
jeblairpabelanger: i'm pretty sure the actual error is "msg: ssh: connect to host 2001 port 22: Invalid argument"16:20
pabelangerOh, ya. IPv6 maybe?16:21
jeblairpabelanger: and i think we're seeing it on all the remote hosts we're rsyncing to16:21
mordredyah16:21
jeblairpabelanger: yes that's it16:21
mordredit's pulling in an ip address rather than a hostname and not quoting it properly _I_think_16:21
pabelangerYa, I would say that16:21
*** nmagnezi_ has quit IRC16:22
pabelangerhttps://github.com/ansible/ansible/issues/8211 quick google16:22
jeblairhttp://serverfault.com/questions/420158/how-do-i-use-rsync-with-an-ipv6-address-via-ssh16:22
*** _nadya_ has quit IRC16:22
jeblairneeds []16:22
*** tonytan4ever has quit IRC16:24
*** bcourt has joined #openstack-infra16:25
*** melania has quit IRC16:25
*** tonytan_brb has joined #openstack-infra16:26
openstackgerritMatt Riedemann proposed openstack/requirements: Bump upper constraint on python-novaclient to 3.2.0  https://review.openstack.org/26485516:26
jeblairmordred: we're still using puppetmaster with puppet agent, right?  this step is for syncing puppet modules, system-config, and hieradata; but since we're still using puppetmaster and puppet agent, puppet will sync that data even if ansible doesn't?16:26
*** egonzalez has joined #openstack-infra16:26
mordredjeblair: yes, that is correct16:28
*** jbernard has left #openstack-infra16:28
mordredjeblair: we started doing the sync so that we could track working or not - which turns out to be a good tihng16:28
*** isaacb has quit IRC16:29
jeblairok, so that splains why everything's not broken, but we do need to fix before proceeding to apply16:29
[patrol]pabelanger: all tests passed. For ws-cleanup it was sufficient to change plugin version in generated xml to 0.28, but for parameterized-trigger and copyartifact it is not so. xml file generated by jenkins-job-builder is the same as the one genereted by jenkins' gui16:31
*** Eva-i has quit IRC16:32
*** ddieterly has quit IRC16:32
*** weshay_xchat has quit IRC16:32
anteayajesusaurus: any feelings on the future of https://review.openstack.org/#/c/218493/ at all?16:33
*** Eva-i has joined #openstack-infra16:33
*** ifarkas has quit IRC16:33
*** otter768 has joined #openstack-infra16:33
*** weshay_xchat has joined #openstack-infra16:34
jeblairmordred: so we should be able to proceed with the use_hostnames thing even though the rsync still won't work (it will tell us which host it's failing on though :)16:37
*** arxcruz has quit IRC16:37
jeblairmordred: what's the expand_hostvars conversation?16:37
*** piet has quit IRC16:38
*** otter768 has quit IRC16:38
*** med_` is now known as med_16:39
mordredjeblair: yes. that's right16:39
*** med_ has quit IRC16:39
*** med_ has joined #openstack-infra16:39
*** med_ is now known as med16:40
mordredjeblair: expand_hostvars controls whether or not the inventory goes and fills in  more info about the nodes16:40
mordredlike volumes attached and stuff16:40
mordredit's additional api calls16:40
*** med is now known as med_16:40
*** abregman has joined #openstack-infra16:40
mordredjeblair: but we cache the inventory for 24 hours anyway16:40
mordredjeblair: so I don't think it's necessary for us to optimize that16:40
jeblairhrm, i didn't see anything from gerritbot for remote:   https://review.openstack.org/264858 Add use_hostnames to ansible clouds.yaml16:41
*** Sukhdev has joined #openstack-infra16:41
jeblairmordred: are we using any of that info yet?16:41
jeblairmordred: like, it sounds like it might be more useful for ansible launch node, yeah?16:42
mordredyah16:43
mordredwe're not using it now16:43
mordredbut we certainly could do in the future16:43
mordredjeblair: +216:43
*** piet has joined #openstack-infra16:43
jeblairmordred: maybe let's not set it right now, since we have other problems, but do so soon (before we need it) when we're ready to find more problems? :)16:44
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add use_hostnames to ansible clouds.yaml  https://review.openstack.org/26485816:44
openstackgerritMerged openstack/requirements: Fix a little spelling typo in README and error message  https://review.openstack.org/22672916:44
openstackgerritMerged openstack/requirements: Add tap-as-a-service to projects.txt  https://review.openstack.org/25621416:44
openstackgerritMerged openstack/requirements: Add networking-midonet to projects.txt  https://review.openstack.org/25508216:44
*** thiagop has quit IRC16:44
jeblairthere it is ^ :)16:44
*** thiagop has joined #openstack-infra16:45
mordredjeblair: ++16:45
*** scheuran_ has quit IRC16:45
mordredjeblair: so - the other thing we're missing is the "generate static group file from a yaml file we write"16:46
*** zaro has quit IRC16:46
*** sabeen has joined #openstack-infra16:46
*** zaro has joined #openstack-infra16:46
*** baoli has quit IRC16:46
*** dslev has quit IRC16:47
pabelangerjeblair: any chance for a new release of zuul today?16:48
*** sabeen3 has joined #openstack-infra16:48
mordredjeblair: I just braindumped into the etherpad on that16:50
*** sabeen has quit IRC16:51
*** unicell1 has joined #openstack-infra16:51
*** devkulkarni has joined #openstack-infra16:51
*** unicell has quit IRC16:52
pleia2good morning16:52
anteayamorning pleia216:53
openstackgerritFausto Marzi proposed openstack-infra/project-config: Add requirement check to freezer* jobs  https://review.openstack.org/26486316:54
jeblairpabelanger: i think we need https://review.openstack.org/259281 landed first16:54
pabelangerjeblair: roger16:54
*** bpokorny has joined #openstack-infra16:54
*** gyee has joined #openstack-infra16:55
openstackgerritFausto Marzi proposed openstack/requirements: Add freezer* projects and 2 modules as requirements  https://review.openstack.org/26486516:56
jeblairmordred: what depends on the yaml -> static group thing?16:56
*** ihrachys has joined #openstack-infra16:57
*** jlanoux has quit IRC16:58
*** dslevin_ has joined #openstack-infra16:59
*** jordanP has quit IRC16:59
*** dims has joined #openstack-infra16:59
mordredjeblair: properly copying group var hiera files17:00
*** yolanda has quit IRC17:00
*** ihrachys has quit IRC17:00
mordredjeblair: but that's only for apply17:00
mordredjeblair: which is blocked on that ipv6 issue for the moment17:00
mordredjeblair: so, not essential to hack on today17:01
*** asselin has joined #openstack-infra17:01
*** ihrachys has joined #openstack-infra17:02
jeblairmordred: ok; are you going to work on the ipv6 thing while you're bored during your meeting, or is another ansibleish person interested in doing that, or should i figure out how to ansible?17:02
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add range search functionality  https://review.openstack.org/26486917:02
mordredjeblair: I'm looking at it right now17:02
*** dims_ has quit IRC17:02
jeblaircool17:03
*** sridhar_ram has joined #openstack-infra17:05
*** moravec has quit IRC17:05
*** hdd has joined #openstack-infra17:05
*** esker has quit IRC17:05
*** hdd has quit IRC17:06
*** daemontool_ has quit IRC17:06
*** moravec has joined #openstack-infra17:07
*** baoli has joined #openstack-infra17:07
*** jsavak has quit IRC17:08
*** jsavak has joined #openstack-infra17:10
*** lezbar has quit IRC17:10
*** samuelBartel has quit IRC17:11
openstackgerritMerged openstack/requirements: Bump futurist g-r to 0.6.0  https://review.openstack.org/26422017:11
*** moravec has quit IRC17:11
*** bpokorny has quit IRC17:12
*** Sukhdev has quit IRC17:12
*** lezbar has joined #openstack-infra17:12
*** mrmartin has joined #openstack-infra17:13
*** sfinucan has quit IRC17:13
*** Sukhdev has joined #openstack-infra17:13
*** trown is now known as trown|lunch17:13
*** davideagnello has joined #openstack-infra17:15
*** dtantsur is now known as dtantsur|afk17:15
mgagnejeblair +1 for Test Nodes graph thanks!17:16
*** markus_z has quit IRC17:17
*** slagle has joined #openstack-infra17:17
*** moravec has joined #openstack-infra17:17
*** apoorvad has joined #openstack-infra17:18
zaromorning17:19
*** piet has quit IRC17:19
*** piet has joined #openstack-infra17:19
pleia2g'day zaro17:19
jeblairmgagne: i think this will be cool too: https://review.openstack.org/26484517:20
clarkbaccording to melody gerrit GCing has picked up this morning17:20
*** hdd has joined #openstack-infra17:20
clarkbhowever memory use doesn't look too bad, most spikes are under 12GB17:21
*** yolanda has joined #openstack-infra17:21
*** bhunter71 has quit IRC17:21
mgagnejeblair not that useful for us until we spread across multiple regions =)17:21
jeblairmgagne: you'll still have a big number like "In Use: 12" :)17:22
mgagnejeblair could "Nodes" be grouped with the other "Test Node" graphs?17:22
jeblairwith grafana, i'm never sure when i'm supposed to hit the back button.17:22
jeblairmgagne: you mean the sparklines?17:22
anteayaclarkb: I tried to gather some enthusiasm for https://review.openstack.org/#/c/247714/ so you can be in a position to merge it when you are ready (logstash 2.0)17:22
fungizaro: for the record, the other reviewers' comments on https://gerrit-review.googlesource.com/73650 basically echo what jeblair and i were suggesting yesterday17:22
*** _nadya_ has joined #openstack-infra17:23
anteayaclarkb: but thus far, no buyers17:23
mgagneyea but I'm not sure what the graph looks like so I don't know if my suggestion makes sense17:23
fungiso their recommendation looks like a great path forward17:23
clarkbmtreinish: ovh-bhs1 pass % is > than hpcloud-b2 now, I think the swap and qemu changes have helped17:23
*** mrmartin has quit IRC17:23
clarkbanteaya: thanks, its ok today is swamped with adult responsibiltiies anyways17:23
*** apoorvad has quit IRC17:23
mtreinishclarkb: ok cool17:23
anteayaclarkb: okay, have fun today17:23
*** Swami has joined #openstack-infra17:24
*** Swami_ has joined #openstack-infra17:24
mgagnejeblair just so we get related graphs grouped together17:24
openstackgerritDoug Hellmann proposed openstack-infra/project-config: stop proposing constraints updates to stable branches  https://review.openstack.org/26488317:24
*** hdd has quit IRC17:25
jeblairmgagne: they look like the ones at the top of this: http://grafana.openstack.org/dashboard/db/zuul-status17:25
mgagnejeblair or create a new panel: Node States instead of Node Launches which counts operations and timing, not nodes count17:25
mgagnejeblair oh, right, better put them at the top then17:25
*** aopincar has joined #openstack-infra17:26
pleia2wow, such beautiful new dashboards, thanks jeblair17:26
waynrelectrofelix: hey, haven't seen you on IRC in a while17:26
electrofelixwaynr: xmas holidays followed by being ill :(17:26
electrofelixfirst day back at work17:26
mtreinishclarkb: do we know which jobs were relying on swap?17:27
waynri've been trying to review and get some of the master branch commits on jenkins-job-builder merged while you were out17:27
clarkbmtreinish: magnum for sure, and the normal tempest runs swaps a small amount according to dstat but not like the magnum job does17:27
waynrah, sorry to hear that I hope you're feeling better!17:27
mtreinishhmm, ok17:27
*** sigmavirus24 is now known as sigmavirus24_awa17:27
clarkbmtreinish: on the order of a few kb to mb17:27
clarkbwhich is probably just noise based on swapiness17:28
electrofelixwaynr: getting there, need to catch up the JJB reviews, probably be tomorrow17:28
*** egonzalez has quit IRC17:28
waynrcool17:28
waynri made quite a bit of progress on the JJB 2.x work17:29
*** bhunter71 has joined #openstack-infra17:29
mtreinishclarkb: ok, thats about what'd I'd expect for the normal jobs17:29
*** sdake has quit IRC17:29
*** jsavak has quit IRC17:30
*** mudassirlatif has joined #openstack-infra17:30
*** jsavak has joined #openstack-infra17:30
*** ujuc has joined #openstack-infra17:32
*** mrmartin has joined #openstack-infra17:33
clarkbmanilla tests appear to fail a lot there17:33
clarkbwonder if htats a disk availability issue17:33
*** sdake has joined #openstack-infra17:34
*** Sukhdev has quit IRC17:34
clarkbmtreinish: health dashboard idea, reverse sort failure % so highest fail rates are at top of list17:35
clarkb(I can toggle it with a mouse click but doing it by default would be nice)17:36
openstackgerritMerged openstack-infra/tripleo-ci: Don't pass duplicate projects to tripleo.sh --delorean-build  https://review.openstack.org/24981917:36
mtreinishclarkb: https://review.openstack.org/25492717:37
clarkbsweet17:37
openstackgerritMerged openstack-infra/tripleo-ci: Add CI coverage of deleting the deployed overcloud  https://review.openstack.org/25049817:38
mtreinishclarkb: you just need to bug sc68cal to respin it17:39
clarkbya needs a rebase17:40
*** armax has joined #openstack-infra17:40
zarofungi: yep, noticed that, thanks. just found that there is a special SUBM label that is just not exposed from change endpoint. exposing it should do the trick.17:41
fungizaro: yep, i mentioned that a while back. that's exactly how it's treated in the ssh api if you look (label name is SUBM and the grantedOn date is what we're looking for)17:44
*** jcoufal_ has quit IRC17:44
zarosorry, my memory is horrible17:45
*** jistr has quit IRC17:45
*** matrohon has quit IRC17:45
*** ihrachys has quit IRC17:47
fungino problem17:47
fungizaro: also, no luck on the viewAllAccounts global capability17:47
fungizaro: i added it for the registered users group (also tried anonymous users) and still get "not allowed to list email addresses" when authenticating as a non-admistrative user17:48
fungiit's set up on review-dev right now if you want to test17:48
*** bhunter71 has quit IRC17:48
fungiand maybe confirm whether i'm doing something stupid17:49
*** sabeen3 has quit IRC17:49
electrofelixwaynr: I saw, lots of improvements there, though I spotted a slightly worrying size of increase in the time for the comparison job. Any ideas whether than is related to your changes, or just slower VMs?17:49
*** hdd has joined #openstack-infra17:50
*** cschwede has quit IRC17:50
*** bhunter71 has joined #openstack-infra17:51
*** aopincar has quit IRC17:51
fungizaro: i can get /a/accounts/self/emails but not arbitrary /a/accounts/{account-id}/emails17:51
fungithe latter gives that permission error i quoted earlier17:52
fungiso something else seems to be controlling access to it17:52
*** gchamoul has quit IRC17:52
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Enable docs publish for openstack/pymod2pkg  https://review.openstack.org/26478117:52
fungizaro: similarly, the /a/accounts/{account-id}/username method gives "not allowed to get username"17:53
fungiwhich is silly since plain /a/accounts/{account-id} actually displays the username in the response17:53
fungiso their risk model here is sort of inconsistent17:54
electrofelixis there an easy way to report for a project on whether requirements are in line with openstack's global requirements? a standard CI job?17:54
openstackgerritBrant Knudson proposed openstack-infra/project-config: Config file updater use current code  https://review.openstack.org/26489417:54
*** rebase has joined #openstack-infra17:55
fungizaro: ideally we'd have control to make methods like /a/accounts/self/emails and /a/accounts/{account-id}/username available but still restrict access to more sensitive methods like /a/accounts/{account-id}/password.http17:55
*** bcourt has quit IRC17:55
fungier, i mean make methods like /a/accounts/{account-id}/emails and /a/accounts/{account-id}/username            available but still restrict access to more sensitive methods like17:56
*** SumitNaiksatam has joined #openstack-infra17:56
fungi /a/accounts/{account-id}/password.http17:56
anteayaelectrofelix: yes gate-{name}-requirements is the job I believe17:56
electrofelixanteaya: thanks17:57
fungizaro: probably no need for us to restrict access to /a/accounts/{account-id}/sshkeys either17:57
waynrelectrofelix: i saw that also, and I'm hoping it's just slower VMs....here's a run on master branch that appears to have taken longer than normal: https://review.openstack.org/#/c/193660/17:57
anteayaelectrofelix: welcome17:57
waynrthere's nothing I can think of in the jjb-2.0.0-api topic that would lead to increased run times for the comparison job17:57
*** jerryz has joined #openstack-infra17:58
electrofelixwaynr: that was my thought too, seeing runs of 18 minutes though is a bit concerning. I plan to look a little closer anyway by checking them on my workstation17:58
clarkbyou can run it under yappi17:59
clarkbor similar and compare17:59
waynralso electrofelix I made this into a separate change: https://review.openstack.org/#/c/261127 so we could get https://review.openstack.org/#/c/133465/ merged17:59
clarkband run ut locally to have consistent cpu/io/etc17:59
waynrelectrofelix: I recommend starting with this https://review.openstack.org/#/c/262668/ since that is showing increased comparison time also18:00
waynrthe fact that that commit is showing increased comparison time really makes me think it's just slow VM's because that is almost entirely rearranging entrypoint code, no functional changes to the Builder or YamlParser classes18:01
waynrer, i mean you should start with that one because it's the first in that series of commits18:02
jeblairelectrofelix: like actual python 'entrypoints' entrypoints?  those can be super slow in some uses.18:04
*** mrmartin has quit IRC18:04
jeblairwaynr: ^ oops18:04
electrofelixwaynr: I suspect that we need an email to discuss about the hacking module, as it's something that for what we're looking for is just a codification of existing pep8 guidelines https://www.python.org/dev/peps/pep-0008/#imports, but is contentious18:04
jeblairyeah, i don't think anyone objects to the ordering of imports.  i do object to using the hacking tool.18:06
*** esker has joined #openstack-infra18:06
jeblairjust order the imports and move on.18:06
electrofelixjeblair: ah, that's interesting, so we can fix up the styles to make it more readable where it's become a mess, as long as we don't enforce it?18:07
stevemarthis change added a new script to the jenkins/scripts folder: https://review.openstack.org/#/c/261828/ but it looks like it may not have been installed on all images? cause i'm seeing an experimental job fail with the message that the script cannot be found: https://review.openstack.org/#/c/264534/ any advice?18:07
electrofelixI can live with that18:07
*** bcourt has joined #openstack-infra18:07
jlvillalOn the Jenkins build host. Do they do any pre-caching of the pip wheel files?18:07
electrofelix jeblair waynr: the entrypoints only be slow for the initial start of a command in this case, as far as I can tell18:07
waynrjeblair: on the rang of 10-15 minutes kind of slow? https://review.openstack.org/262668 only adds 4 additional entrypoints loaded through the stevedore library I can't imagine that would lead to comparison tests ranging from 14-19 minutes18:07
nibalizerlast night on omfra cloud I set the ram quota to 50 million18:08
nibalizerbecause ram quota is expressed in k18:08
waynrjeblair: i haven't noticed any real difference when actually running jenkins-jobs from the command line after that commit18:08
*** zeih has joined #openstack-infra18:08
clarkbwaynr electrofelix the easiest way to find out would be to use a profiler like yappi18:08
jeblairwaynr, electrofelix: i don't remember details, but i know it was a huge problem with rootwrap a while back18:08
electrofelixclarkb: thanks, I'll look into it18:08
*** trown|lunch is now known as trown18:08
electrofelixI figure I'll just check locally with a consistent workstation performance if there is anything odd going on first as a ballpark test18:09
*** eil397 has joined #openstack-infra18:09
waynrelectrofelix jeblair could we compromise on the hacking guidelines by adding a new tox target that does include the additional flake8 rules? we could even add it as a non-voting test on JJB18:09
jeblairso no actual theory here about why, just more of a heads up that maybe there could be a subtle thing there18:09
anteayanibalizer: how'd that work out?18:10
nibalizeranteaya: reasonably well18:10
*** jordanP has joined #openstack-infra18:10
anteayacan we keep that setting?18:10
anteayasounds great18:10
*** sabeen1 has joined #openstack-infra18:11
nibalizercrinkle: i have two serers I can't delete18:11
*** derekh has quit IRC18:11
*** jswarren_ has joined #openstack-infra18:11
nibalizer29f98122-2da4-4516-84b6-69a1d1dd49ff and 0ec412fd-8e2f-4ebb-b92d-792c07fa6404 have been in state deleting for a long time18:11
arizaro# Can you take another look at this?18:11
*** tongli_ has joined #openstack-infra18:11
*** esker has quit IRC18:11
crinklenibalizer: hmm18:12
crinklenibalizer: can you reach their hypervisors?18:12
jeblairnibalizer: this is something we see in real production clouds :)18:12
jeblairnibalizer: i think they probably periodically go in and do something to fix it.18:12
crinkleoh good so we're doing it right :)18:13
jeblairyep!18:13
*** amuller has quit IRC18:13
*** amuller has joined #openstack-infra18:13
*** esker has joined #openstack-infra18:13
*** jpeeler has quit IRC18:13
*** tongli has quit IRC18:14
*** dkranz has quit IRC18:14
*** eharney has quit IRC18:14
*** markvoelker has quit IRC18:14
fungi"right"18:14
*** jswarren has quit IRC18:14
*** ajo has quit IRC18:14
*** sabeen3 has joined #openstack-infra18:14
fungia.k.a. "coping with openstack's bugs"18:15
*** AJaeger has joined #openstack-infra18:15
nibalizercrinkle: yes these nodes are fairly old18:15
*** sabeen1 has quit IRC18:15
nibalizerso its possible we had some instability18:15
*** trown has quit IRC18:15
*** trown|brb has joined #openstack-infra18:16
*** ari has quit IRC18:16
*** weshay_xchat has quit IRC18:16
clarkbI think for fails like that libvirt/qemu logs may be helpful18:16
*** rhallisey has quit IRC18:16
nibalizerjeblair: i hobbled this together from the d-g readme, http://paste.openstack.org/show/483246/ and I am trying to use it to run a devstack+tempest run to get a timing estimate18:16
nibalizerbut it never completes18:16
nibalizerput another way, what testing do you usually do on a cloud to get timing information before puttign infra nodepool on it18:17
*** amuller is now known as amuller_afk18:17
jeblairnibalizer: ah, i have a thing, gimme a sec18:17
*** yaume_ has quit IRC18:18
openstackgerritDoug Hellmann proposed openstack/requirements: add simpler instructions for adding new dependencies  https://review.openstack.org/26490718:18
openstackgerritSpencer Krum proposed openstack-infra/system-config: Rename puppet-community to voxpupuli  https://review.openstack.org/26391018:19
nibalizerwe renamed 'puppet-community' to 'voxpupuli' so you'll likely start seeing the later around18:20
*** tsg has joined #openstack-infra18:20
*** trown|brb is now known as trown18:21
*** ajo has joined #openstack-infra18:21
openstackgerritJohn L. Villalovos proposed openstack-infra/devstack-gate: TRIVIAL: Fix some misspellings  https://review.openstack.org/26393818:21
*** tonytan_brb has quit IRC18:21
funginibalizer: hah. that's an excellent pun18:22
*** aopincar has joined #openstack-infra18:22
*** e0ne has joined #openstack-infra18:23
openstackgerritMerged openstack/requirements: Add searchlight to projects.txt  https://review.openstack.org/25503118:24
*** _nadya_ has quit IRC18:24
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add exceedingly rough nodepool/devstack benchmark script  https://review.openstack.org/26491018:26
jeblairnibalizer: ^ there's how i've been doing it -- it's a script never intended to see the light of day18:27
*** ihrachys has joined #openstack-infra18:27
jeblairwhich is why it's sort of squirming around and smoking a bit18:27
jeblairnibalizer: but, if i had to hazard a guess, you maybe only need one modification to your current procedure:18:27
*** abregman is now known as abregman|afk18:27
*** eharney has joined #openstack-infra18:27
jeblairnibalizer: add "exec 0</dev/null" to the top of your script18:27
*** dkranz has joined #openstack-infra18:28
*** rhallisey has joined #openstack-infra18:28
*** esker has quit IRC18:28
*** mrmartin has joined #openstack-infra18:28
jeblairnibalizer: turns out, there are some processes in devstack now that are finicky about stdin; and we don't notice this with jenkins because it handles that for us18:28
jeblairnibalizer: so, maybe try that if you're comfortable with what you've got...18:28
*** abregman|afk has quit IRC18:28
*** abregman has joined #openstack-infra18:29
jeblairnibalizer: and then, if you want to have a thing that grabs all the available nodes from your local nodepool and spawns a devstack run on them in parallel and reports the timings when complete, look at that script18:29
*** ihrachys has quit IRC18:29
*** weshay_xchat has joined #openstack-infra18:29
*** ari has joined #openstack-infra18:31
*** gyee has quit IRC18:31
*** zeih has quit IRC18:31
jesusaurusanteaya: not really. i thought it would be useful to publish a manpage draft similar to a docs draft, but it was in response to a single man page change i was reviewing and it didnt seem like anyone else was interested18:32
jeblairnibalizer: i also based my tests on devstack runs on changes (rather than just running master) because then i was fixing at least some of the trees at a point in time and could compare runtimes to previously completed jobs.  i'm not sure how important that is.18:32
*** mriedem has quit IRC18:32
*** esker has joined #openstack-infra18:33
fungiAJaeger: we're coming up on the approval deadline for your 246550 spec on docs job consolidation. do you feel consensus has been reached in light of some of the concerns raised in this week's meeting or should we hold off for more revisions?18:34
*** otter768 has joined #openstack-infra18:34
*** jaosorior has quit IRC18:34
*** jpeeler has joined #openstack-infra18:34
*** jaosorior has joined #openstack-infra18:35
AJaegerfungi: thanks for asking. There were a few comments in the meeting by jeblair and others on the approach that have not been added to the review. I think we should wait with approval so that they can chime in, the time might have been too short.18:35
*** kushal has quit IRC18:35
*** harlowja has quit IRC18:35
AJaegerI also had some concerns with implementation for the post-job - the releasenotes publish always from master while other docs publish from branches, we cannot combine them at all.18:36
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: [WIP] Add support for templating dashboard  https://review.openstack.org/25089318:36
AJaegerI only thought about that this evening and I'm still chewing on that ;(18:36
*** harlowja has joined #openstack-infra18:36
fungiAJaeger: no problem, just didn't want to leave you hanging. thanks for following up18:37
AJaegerfungi, do you mind if I WIP it?18:37
fungiAJaeger: feel free!18:37
AJaegerfungi: highly appreciated, thanks for asking!18:37
*** mrmartin has quit IRC18:37
*** mriedem has joined #openstack-infra18:38
pabelangerjeblair: Nice script, I was actually thinking of something like that over the christmas break.  Something more generic outside devstack / openstack testing, as a way to generate load for nodepool18:39
*** edmondsw has quit IRC18:39
jeblairpabelanger: yeah, we could clean it up a bit fairly easily :)18:39
*** otter768 has quit IRC18:39
*** jsavak has quit IRC18:40
*** jsavak has joined #openstack-infra18:40
pabelangerjeblair: I mean, today I just attand my zuul to a random project on review.o.o and use that as the trigger, disabling comments on success and failures.  But, would be nice to remove that dependency18:40
pabelangerattach*18:40
openstackgerritDarragh Bailey proposed openstack-infra/project-config: Add requirements checking jobs for git-upstream  https://review.openstack.org/26491518:41
*** hdd has quit IRC18:42
AJaegerproject-config cores, could I get a second +2 on https://review.openstack.org/245352, please? - it's a new project creation and I did some edits myself, so don't want to +2A. thanks to anteaya for the first +2.18:42
jeblairpabelanger, nibalizer: i went through and commented on the gotchas in https://review.openstack.org/26491018:43
*** edmondsw has joined #openstack-infra18:44
*** amuller_afk has quit IRC18:45
pabelangerjeblair: nice, will check it out and give it a spin18:45
*** amuller_afk has joined #openstack-infra18:46
*** ayoung has quit IRC18:46
*** ayoung has joined #openstack-infra18:46
*** ari has quit IRC18:47
*** pradk has quit IRC18:47
*** gchamoul has joined #openstack-infra18:47
*** ujuc has quit IRC18:47
*** ari has joined #openstack-infra18:48
*** tonytan4ever has joined #openstack-infra18:48
*** pvaneck has joined #openstack-infra18:50
*** tongli has joined #openstack-infra18:51
*** bcourt has quit IRC18:53
*** tongli has quit IRC18:53
*** tongli_ has quit IRC18:53
*** sigmavirus24_awa is now known as sigmavirus2418:54
*** Daisy has joined #openstack-infra18:55
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Mention Grafana in README.rst  https://review.openstack.org/26482218:55
*** bharathm has joined #openstack-infra18:55
*** jordanP has quit IRC18:55
*** sputnik13 has joined #openstack-infra18:56
AJaegerHi pleia2, do you have time for the translators discussion now?18:58
pleia2AJaeger: yeah sure18:58
*** electrofelix has quit IRC18:58
pleia2I can check on the gettext version, sec18:59
*** Daisy has quit IRC18:59
AJaegerpleia2: I tried another gettext locally and couldn't reproduce it.18:59
AJaegerpleia2: So, I'm puzzled what's happening ;(19:00
pleia2AJaeger: looks like default for 14.04, 0.18.3.119:00
pleia2http://packages.ubuntu.com/trusty/gettext19:00
AJaegerI used 0.19.6 and 0.19.219:01
pleia2how did you go about testing this?19:01
AJaegerpleia2: can you try locally to download the files and see whether msgattrib really cleans this up?19:01
pleia2I can try it on a trusty system I have here19:01
*** squid1 is now known as calebb19:02
pleia2trusty system :D19:02
AJaegerpleia2: download the zip file as Alex explained, the unzip it and run:19:02
*** sridhar_ram has quit IRC19:02
openstackgerritMerged openstack-infra/project-config: stop proposing constraints updates to stable branches  https://review.openstack.org/26488319:02
AJaegerpleia2: msgattrib --translated --no-location --sort-output neutron.po --output=neutron-new.po19:03
pleia2will have to get dependencies too19:03
AJaegerand then checking neutron-new.po whether the translators are removed19:03
*** sabeen3 has quit IRC19:03
pleia2ok19:03
*** markvoelker has joined #openstack-infra19:03
AJaegerwhat confuses me is that this does not happen with every file, just with some ;(19:04
*** jsavak has quit IRC19:04
*** aopincar has quit IRC19:04
*** bcourt has joined #openstack-infra19:05
*** markvoelker_ has joined #openstack-infra19:05
openstackgerritMerged openstack/requirements: Add mimic to global requirements to use in tests  https://review.openstack.org/22026819:06
*** jsavak has joined #openstack-infra19:06
*** rebase has quit IRC19:06
*** esker has quit IRC19:07
*** yamahata has quit IRC19:07
*** ozamiatin has quit IRC19:07
openstackgerritMerged openstack-infra/system-config: Rename puppet-community to voxpupuli  https://review.openstack.org/26391019:08
*** e0ne has quit IRC19:08
*** bpokorny has joined #openstack-infra19:08
*** markvoelker has quit IRC19:09
pleia2AJaeger: doesn't pull them out19:09
pleia2neutron-new.po still has the list of translators at the top19:09
AJaegerpleia2: so, what does? That's the only place in our infra scripts that touches the files, besides the download...19:10
AJaeger(if I read the scripts properly)19:10
*** dslevin_ has quit IRC19:10
*** esker has joined #openstack-infra19:11
pleia2maybe we should be looking for the FIRST AUTHOR <EMAIL@ADDRESS> string somewhere (whether in our scripts or zanata source)19:11
AJaegerwait, we do a msgmerge, don't we? Let me double check19:11
pleia2since something is resetting things19:11
AJaegerwe run update_catalog - let's see what that one does...19:12
*** Sukhdev_ has joined #openstack-infra19:13
anteayajesusaurus: what would you like to do with that patch?19:13
*** Sukhdev_ has quit IRC19:14
*** markvoelker has joined #openstack-infra19:14
*** jerryz has quit IRC19:14
*** pradk has joined #openstack-infra19:14
*** Sukhdev_ has joined #openstack-infra19:14
*** baoli has quit IRC19:14
*** mrmartin has joined #openstack-infra19:15
*** dizquierdo has quit IRC19:15
*** fawadkhaliq has quit IRC19:16
*** markvoelker_ has quit IRC19:17
openstackgerritMerged openstack-infra/project-config: Grafana: Add node sparklines to rax graph  https://review.openstack.org/26484519:17
*** dizquierdo has joined #openstack-infra19:20
*** tzn has quit IRC19:20
*** ajmiller_ is now known as ajmiller19:22
*** dizquierdo has quit IRC19:22
*** mrmartin has quit IRC19:22
*** unicell1 has quit IRC19:24
*** hdd has joined #openstack-infra19:27
AJaegerpleia2: I'm running all commands locally and cannot reproduce it ;(19:28
*** dslevin_ has joined #openstack-infra19:28
pleia2:\19:28
pleia2AJaeger: want me to test a few more?19:28
*** esker has quit IRC19:28
*** yamahata has joined #openstack-infra19:28
*** esheffield has quit IRC19:29
*** vgridnev_ has joined #openstack-infra19:29
*** sputnik13_ has joined #openstack-infra19:29
*** rguillebert_ has joined #openstack-infra19:30
*** nikhil_k has joined #openstack-infra19:31
*** ganso_ has joined #openstack-infra19:31
*** tlian2 has joined #openstack-infra19:31
*** tsg_ has joined #openstack-infra19:31
*** unicell has joined #openstack-infra19:31
*** jraim_ has joined #openstack-infra19:32
*** mriedem1 has joined #openstack-infra19:33
*** dims_ has joined #openstack-infra19:33
*** aviau_ has joined #openstack-infra19:33
*** aviau_ has quit IRC19:33
*** dims has quit IRC19:34
*** aviau_ has joined #openstack-infra19:34
*** yonglihe_ has joined #openstack-infra19:35
*** kgalanov_ has joined #openstack-infra19:35
*** etoews_ has joined #openstack-infra19:35
*** ameade_ has joined #openstack-infra19:35
AJaegerpleia2: nailed it down - that's a change in babel 2.2.019:35
*** ociuhandu has quit IRC19:36
AJaegerit works with 2.1.1 just fine19:36
pleia2in babel?! gah19:36
AJaegerlet's check whether that's an announced change...19:36
pleia2nice sleuthing19:36
*** bexelbie- has joined #openstack-infra19:37
openstackgerritSean M. Collins proposed openstack-infra/project-config: Neutron multinode grenade: Add grenade to PROJECTS  https://review.openstack.org/26493819:37
*** bradjones_ has joined #openstack-infra19:37
*** bradjones_ has quit IRC19:37
*** bradjones_ has joined #openstack-infra19:37
*** reed_ has joined #openstack-infra19:37
*** stevemar_znc has joined #openstack-infra19:37
*** igorbelikov_ has joined #openstack-infra19:37
*** pradk has quit IRC19:38
*** sputnik13 has quit IRC19:38
*** mriedem has quit IRC19:38
*** tsg has quit IRC19:38
*** eil397 has quit IRC19:38
*** vgridnev has quit IRC19:38
*** tlian has quit IRC19:38
*** rguillebert has quit IRC19:38
*** bradjones has quit IRC19:38
*** ganso has quit IRC19:38
*** bexelbie has quit IRC19:38
*** jamespage has quit IRC19:38
*** gnuoy has quit IRC19:38
*** nikhil has quit IRC19:38
*** ameade has quit IRC19:38
*** igorbelikov has quit IRC19:38
*** briancurtin has quit IRC19:38
*** kgalanov has quit IRC19:38
*** reed has quit IRC19:38
*** sileht has quit IRC19:38
*** waynr has quit IRC19:38
*** samyewell has quit IRC19:38
*** lxsli has quit IRC19:38
*** etoews has quit IRC19:38
*** yonglihe has quit IRC19:38
*** vdrok- has quit IRC19:38
*** aviau has quit IRC19:38
*** stevemar has quit IRC19:38
*** jraim has quit IRC19:38
*** bradjones_ is now known as bradjones19:38
*** bexelbie- is now known as bexelbie19:38
*** mriedem1 is now known as mriedem19:38
*** rguillebert_ is now known as rguillebert19:38
*** reed_ is now known as reed19:39
*** yonglihe_ is now known as yonglihe19:39
*** igorbelikov_ is now known as igorbelikov19:39
*** ganso_ is now known as ganso19:39
*** kgalanov_ is now known as kgalanov19:39
*** lxsli has joined #openstack-infra19:39
*** samyewell has joined #openstack-infra19:39
*** _nadya_ has joined #openstack-infra19:40
*** stevemar_znc is now known as stevemar19:40
*** pradk has joined #openstack-infra19:40
openstackgerritMerged openstack-infra/project-config: Mention Grafana in README.rst  https://review.openstack.org/26482219:40
*** ameade_ is now known as ameade19:40
*** jraim_ is now known as jraim19:40
*** gnuoy has joined #openstack-infra19:41
pleia2and yeah, 2.2.0 is the version of babel on the proposal slave19:41
*** jamespage has joined #openstack-infra19:41
*** jamespage has quit IRC19:41
*** jamespage has joined #openstack-infra19:41
*** shardy is now known as shardy_afk19:41
*** sileht has joined #openstack-infra19:41
*** waynr has joined #openstack-infra19:41
AJaegerpleia2: smells like https://github.com/python-babel/babel/commit/e0e7ef168856bb431076cfe45f6c456d1209123319:41
*** etoews_ is now known as etoews19:42
pleia2AJaeger: aha, that does seem likely19:42
AJaegerjust not sure whether this is desired or not19:43
pleia2it does seem like the right behavior :(19:43
openstackgerritJames E. Blair proposed openstack-infra/project-config: Grafana: add sparklines to remaining providers  https://review.openstack.org/26494119:43
pleia2even if it's unpleasant for us19:43
jesusaurusanteaya: i guess i'll abandon it19:43
AJaegerpleia2: yes, it looks correct - the pot header does not include the Translators, so it's not here either...19:44
*** waynr has quit IRC19:44
*** waynr has joined #openstack-infra19:44
pleia2AJaeger: did ones that it updated correctly have it?19:44
AJaegercan we update the master pot to include the translators?19:44
pleia2could mean we just need to update some pot files19:44
pleia2wfm19:44
AJaegerpleia2: this happens when we run update_catalog - which we do not run on the log level ones, so only the normal files are updated19:45
*** tqtran has joined #openstack-infra19:45
pleia2right, gotcha19:45
*** sigmavirus24 is now known as sigmavirus24_awa19:45
*** briancurtin has joined #openstack-infra19:46
*** bgmccollum has joined #openstack-infra19:46
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add range search functionality  https://review.openstack.org/26486919:46
*** vdrok- has joined #openstack-infra19:46
AJaegerpleia2: I'm looking on how to change the default header19:48
*** sigmavirus24_awa is now known as sigmavirus2419:48
*** Sukhdev_ has quit IRC19:48
nibalizerjeblair: thanks so much for 26485819:48
pleia2AJaeger: thanks19:49
*** lucasagomes is now known as lucas-dinner19:49
jeblairnibalizer: i just typed what mordred said to type :)  but yeah, let's see if it works19:49
nibalizeri tested it on my personal setup and it did precisely what I wnated19:49
jeblaircool19:50
nibalizercrinkle: wanna rebase 209617 (again) sorry19:50
jeblairpabelanger: it looks like grafyaml is installed from pip; shall we make a release?19:51
openstackgerritSean McGinnis proposed openstack/requirements: Update to os-brick 0.7.0  https://review.openstack.org/25949419:51
*** abregman is now known as abregman|afk19:51
jeblairpabelanger: oh wow i totally read that wrong, we're deploying from source19:51
jeblairpabelanger: nevermind :)19:52
openstackgerritMerged openstack-infra/project-config: Add Manila with CephFSNative driver CI job  https://review.openstack.org/25932319:52
* timrc perks up at grafyaml -- pabelanger what's left to do to get it to the point where I can push dashboards to gerrit :)?19:53
timrcMy new job is sadly stealing much time away from my time here, with you all.  Much sadness.19:53
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud modules to modules.env  https://review.openstack.org/20961719:54
crinklenibalizer: ^19:54
*** dhellmann_ has joined #openstack-infra19:55
pleia2timrc: sad indeed! but I hope you're enjoying the new job :)19:55
openstackgerritSpencer Krum proposed openstack-infra/system-config: Hound Maintenance  https://review.openstack.org/26432619:55
*** amuller_afk is now known as amuller19:56
*** esker has joined #openstack-infra19:57
*** sridhar_ram has joined #openstack-infra19:57
*** dims_ has quit IRC20:00
openstackgerritSpencer Krum proposed openstack-infra/system-config: puppet-staging is in voxpupuli  https://review.openstack.org/26494520:00
openstackgerritMerged openstack-infra/system-config: Add use_hostnames to ansible clouds.yaml  https://review.openstack.org/26485820:00
*** esker has quit IRC20:01
*** esker has joined #openstack-infra20:02
*** esker has quit IRC20:03
*** sabeen1 has joined #openstack-infra20:04
*** esker has joined #openstack-infra20:04
openstackgerritMerged openstack-infra/system-config: Add accessDatabase and emailReviewers capabilities  https://review.openstack.org/26435720:04
*** MarkAtwood1 has joined #openstack-infra20:06
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add baremetal hpuswest node definition  https://review.openstack.org/20875120:06
AJaegerpleia2: https://github.com/python-babel/babel/issues/318 - I filed a bug20:07
*** esker has quit IRC20:08
pleia2github auto formatting gets no cookies today20:08
AJaegerpleia2: I found no way to modify it and give up. If you want to give it a try...20:08
*** armax has quit IRC20:08
pleia2AJaeger: I can try, but what steps were you using to modify+test?20:08
*** armax has joined #openstack-infra20:09
AJaegerpleia2: just run:  python setup.py  update_catalog  --no-fuzzy-matching --ignore-obsolete=true20:09
*** esker has joined #openstack-infra20:09
AJaegerthat should show the problem...20:09
pabelangerjeblair: IIRC, I think we gate on grafyaml from pip20:09
*** rfolco has quit IRC20:09
pleia2AJaeger: against which file?20:09
pabelangerjeblair: which is fine for me, just means we release more20:09
pabelangerjeblair: so, ya. I'm good with it20:09
AJaegerin neutron tree on - and then run git diff20:09
pleia2the .pot?20:09
AJaegergit diff will show the problems ;)20:10
pleia2right, got it20:10
AJaegerpleia2: it should be on all neutron.pot files20:10
* pleia2 nods20:10
pleia2thanks20:10
AJaegerneutron.po I mean20:10
jeblairpabelanger: oh, hrm.  we should match production to what we gate on, i think; so maybe we should update the gate to install from source20:10
AJaegerpleia2: I tried updating the  header comment in neutron.pot (really pot!) and then running update_catalog again - the update was propagaged but still Translators removed ;(20:11
zaroari: a little busy right now.  but will take a look when i have a chance.20:11
arizaro# understand thanks!20:11
*** esker has quit IRC20:11
pleia2AJaeger: will do20:11
pabelangerjeblair: My thoughts, if we gate on release, we ensure some backwards compat.  While, ensuring we release often, if that makesense20:11
*** dhellmann has quit IRC20:11
*** dhellmann_ is now known as dhellmann20:11
pabelangertimrc: trying to get templating going now20:12
*** dhellmann has quit IRC20:12
AJaegerpleia2: I'm leaving now, if you find anything, please send me an email or comment in that github issue20:12
AJaegerpleia2: thanks!20:12
pleia2AJaeger: yep, will do, have a good night20:12
pabelangertimrc: for the most part it should be working. I think we are up to 7 dashboards now20:12
*** dims has joined #openstack-infra20:12
openstackgerritMerged openstack-infra/project-config: Add requirements checking jobs for git-upstream  https://review.openstack.org/26491520:12
dulekHad anyone seen something like this: http://logs.openstack.org/67/257667/7/check/gate-grenade-dsvm/b6143b9/logs/devstack-gate-setup-workspace-new.txt.gz#_2016-01-07_18_08_03_65120:12
dulekSeems to strike Cinder's grenade check.20:13
*** AJaeger has quit IRC20:13
dulekI don't even understand why Cinder's grenade is pulling zaqar.20:13
*** dhellmann has joined #openstack-infra20:14
smcginnisdulek: That doesn't seem right.20:14
jeblairpabelanger: that's fair, but since we can't land coordinating breaking changes, we get the backwards compat guarantee anyway.  i think needing to release to use a new feature for a fast-moving self-hosted project slows us unecessarily.  however, i'm not opposed to releasing often.  i don't see them as being in conflict -- i think we can perform the gate tests on source (which makes the most sense to me because that's what we want to verify20:15
*** changbl has quit IRC20:15
jeblairpabelanger: (additional benefit: being able to actually use code we just landed before releasing it will reduce broken releases)20:16
clarkbdulek because it is in the PROJECTS list20:16
pabelangerjeblair: Ya, that is valid. +1 for dogfooding20:16
jeblaircool20:16
jeblairi'll propose a change in a bit20:17
openstackgerritDirk Mueller proposed openstack/requirements: Use OpenStack Foundation for setup.py copyright  https://review.openstack.org/26494920:17
clarkbit is going to be in every run of that job, cinder or not. Doesnt mean the service is configured or running though (but may be)20:17
*** rguillebert has quit IRC20:17
clarkbdulek that isnt why the job failed though20:18
clarkb(if it did fail)20:18
dulekclarkb: Okay, sorry for my ignorance, I'm far from understanding of how all this stuff works. Anyway I guess there's a problem with zaqar repo? Or something?20:18
mtreinishare other people have issues getting a response form http://health.openstack.org ?20:19
*** ayoung has quit IRC20:19
*** dims has quit IRC20:19
dulekclarkb: It did.20:19
sdaguedulek: zuul says fatal error when there is not a fatal error20:20
sdagueignore that20:20
sdaguethis is why things failed - http://logs.openstack.org/67/257667/7/check/gate-grenade-dsvm/b6143b9/logs/grenade.sh.txt.gz#_2016-01-07_18_53_38_69020:20
jeblairpabelanger: heh, our grafana doesn't have commit 3c54d14460f7356188718714a872337773f212ba in it which is needed to support time values greater than seconds; i guess we're a bit ahead of the curve :)20:20
*** esker has joined #openstack-infra20:20
jeblairmtreinish: negative20:20
mtreinishjeblair: negative, meaning it's working for you?20:21
jeblairmtreinish: oh you did phrase it that way.  sorry.  it is not working for me.20:21
*** esker has quit IRC20:21
*** aeng has joined #openstack-infra20:22
*** dims has joined #openstack-infra20:22
duleksdague: Cool. Is it Nova's fault or just a random hiccup? I've seen that 2 times in this hour.20:22
mtreinishjeblair: heh, well I barely worded that question in a coherent manner so I would have clarified either way :)20:22
sdaguedulek: I don't know, could be an issue with oslo.service20:23
sdagueare we tracking this bug in ER20:23
sdague?20:23
mtreinishjeblair: is there anything in the logs on it?20:23
jeblairmtreinish: i need to get lunch; can you ping another infra-root?20:23
*** openstackgerrit has quit IRC20:23
*** dims has quit IRC20:23
*** bharathm has quit IRC20:24
mtreinishjeblair: sure, you were just the first to bite :)20:24
jeblairif no one else shows up, i'll stick around20:24
duleksdague: I don't think so, I've just noticed that recently.20:24
mtreinishjeblair: well it seems to be working now20:24
*** openstackgerrit has joined #openstack-infra20:24
mtreinishso you fixed it :)20:24
jeblairmtreinish: glad i could help!20:25
jeblairmtreinish: if no one else has, i'll help you check the logs, etc, after lunch20:25
mtreinishok, thanks20:25
*** _nadya_ has quit IRC20:27
*** jsavak has quit IRC20:27
openstackgerritColleen Murphy proposed openstack-infra/puppet-infracloud: Add bifrost manifest  https://review.openstack.org/23175720:29
*** bharathm has joined #openstack-infra20:29
*** gchamoul has quit IRC20:30
*** jsavak has joined #openstack-infra20:30
openstackgerritColleen Murphy proposed openstack-infra/puppet-infracloud: Add bifrost manifest  https://review.openstack.org/23175720:30
*** migi has quit IRC20:31
*** amrith is now known as _amrith_20:31
openstackgerritDaneyon Hansen proposed openstack-infra/irc-meetings: Removes Magnum Networking Subteam Meeting  https://review.openstack.org/26495420:32
*** gchamoul has joined #openstack-infra20:32
*** migi has joined #openstack-infra20:32
mtreinishfungi, clarkb, pleia2, nibalizer: if you get a sec can you check the logs for the api server on health.o.o? It's struggling a bit right now20:33
*** dprince has quit IRC20:33
*** otter768 has joined #openstack-infra20:35
*** tsg__ has joined #openstack-infra20:35
pleia2mtreinish: gah, cpu 100.2 11.3   9:51.74 apache220:36
pleia2poking around logs now20:36
mtreinishheh, that's probably not a good sign20:36
fungiwell, it dropped off again20:36
fungithe cacti graphs do show load average ticking up to 1 for a bit20:37
*** tsg_ has quit IRC20:37
mtreinishI'm guessing it's probably something to do with the resampling stuff actually being used by the frontend now20:38
fungiIOError: failed to write data20:38
pleia2there are a fair number of errors, and I see a flask restart20:38
mtreinish:(20:38
mtreinishthat's probably when it came back20:39
fungiwasn't a package update... dpkg.log is quiet since ~06:30 utc20:39
*** ayoung has joined #openstack-infra20:39
*** matrohon has joined #openstack-infra20:39
*** dims has joined #openstack-infra20:39
fungithere are a lot of errors about mod wsgi forcibly restarting flask20:39
* pleia2 nods20:39
fungianything sensitive in the api error log? can we just paste some of it?20:40
fungiit's only 432 lines since jabuary 320:40
mtreinishfungi: I don't think there's anything sensitive in there20:40
*** bharathm has quit IRC20:40
mtreinishit might have the private network ip for the mysql server20:40
pleia2probably data from 20:09 onward is useful20:40
fungiyeah, i don't see anything worrying privacy wise20:40
*** otter768 has quit IRC20:40
fungii'll put this up on paste.o.o so you can follow along20:40
mtreinishbut that would be about it. it uses the 'public' creds20:41
fungithat's what i thought20:41
fungimtreinish: http://paste.openstack.org/show/48326520:42
fungiplenty of tracebacks in there to have fun running down20:42
fungiyou'll obviously want to group by session id because there's a lot of interleaving going on there20:43
openstackgerritMatt Riedemann proposed openstack-infra/project-config: Test gate-grenade-dsvm-partial-ncpu on requirements repo  https://review.openstack.org/26495720:43
mriedemdhellmann: dansmith: sdague: ^20:43
pleia2fungi: it cuts off toward the end20:43
fungihuh, paste truncated it. i'll push up one with just stuff from today and see if that's any better20:44
fungiyep, just saw that20:44
fungihere's just today's, not truncated http://paste.openstack.org/show/483266/20:45
fungithe "cache lock too old" might be related to the actual availability issue20:46
mtreinishyeah there are a bunch of errors in there a lot I've not seen before20:46
pleia2what fascinating errors you have, h.o.o20:46
sdaguemriedem: I was about to -2 with fury about deleting that job. But, your comments are clear, +220:48
*** piet has quit IRC20:48
fungibut yeah, mtreinish said something about it in here at 20:10 and the earliest errors in the log are from 20:09 so i assume they're all related to whatever was going on20:48
sdaguemriedem: actually, wait, why do you want that job?20:48
fungier, said something in here at 20:1920:48
*** esker has joined #openstack-infra20:48
*** unicell has quit IRC20:48
openstackgerritMerged openstack-infra/project-config: Neutron multinode grenade: Add grenade to PROJECTS  https://review.openstack.org/26493820:49
sdaguemriedem: what do you believe it's testing that isn't tested now20:49
mriedemsdague: o.vo changes on stable/liberty i think is the goal20:49
mtreinishfungi: heh, I'm not watching closely enough it took me 10mins to notice :)20:49
mriedemdansmith and dhellmann are trying to resolve which version of o.vo should be in u-c on stable/liberty20:49
mriedemover in -releases20:49
mriedemsdague: https://etherpad.openstack.org/p/stable-liberty-constraints-sanity20:50
fungimtreinish: and given the various source ip addresses mentioned in the error log, i have to assume at least most of these are symptoms (something happened, performance tanked, these errors are the fallout from it)20:50
openstackgerritMerged openstack-infra/puppet-infracloud: Specify the dns nameserver for the subnet  https://review.openstack.org/26283020:50
mtreinishfungi: yeah that's what I'm thinking too20:50
*** rfolco has joined #openstack-infra20:50
mtreinishI'm gonna try to write some patches to be defensive against the tracebacks and go from there20:51
mtreinishfungi: the 1 thing I'm not sure about is where that IOError is coming from20:53
fungimtreinish: yeah, nothing in dmesg log about filesystem write issues. i have to assume this is a socket write?20:54
*** rfolco has quit IRC20:54
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for nova py34 fake image service bug 1531988  https://review.openstack.org/26496020:55
openstackbug 1531988 in OpenStack Compute (nova) "py34 tests that use the stubbed out fake image service race fail a lot" [High,In progress] https://launchpad.net/bugs/1531988 - Assigned to Matt Riedemann (mriedem)20:55
crinkleI am attempting to clean out https://storyboard.openstack.org/#!/story/2000175 and there are some tasks that either need clarification, a status update, or to be marked invalid20:56
*** esker has quit IRC20:56
crinkleStevenK: jeblair mordred Clint could you possibly take a minute visit ^ and tell me about the tasks that are assigned to you?20:56
mtreinishfungi, pleia2: patch #1: https://review.openstack.org/#/c/264962/20:57
*** dtardivel has quit IRC20:57
*** matrohon has quit IRC20:58
Clintcrinkle: mine corresponds to https://review.openstack.org/#/c/198136/21:00
*** ociuhandu has joined #openstack-infra21:00
crinkleClint: cool, i'll change the status to "progress"21:01
*** changbl has joined #openstack-infra21:01
Clintcrinkle: ok21:01
*** bgmccollum has left #openstack-infra21:03
*** matrohon has joined #openstack-infra21:03
*** tzn has joined #openstack-infra21:05
*** bharathm has joined #openstack-infra21:06
*** changbl has quit IRC21:06
openstackgerritMerged openstack-infra/elastic-recheck: Add query for nova py34 fake image service bug 1531988  https://review.openstack.org/26496021:07
openstackbug 1531988 in OpenStack Compute (nova) "py34 tests that use the stubbed out fake image service race fail a lot" [High,In progress] https://launchpad.net/bugs/1531988 - Assigned to Matt Riedemann (mriedem)21:07
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add range search functionality  https://review.openstack.org/26486921:08
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add range search functionality  https://review.openstack.org/26486921:09
mtreinishfungi, pleia2: patch #2: https://review.openstack.org/#/c/264967/21:09
*** mudassirlatif_ has joined #openstack-infra21:10
mtreinishthe 3rd unique traceback type is the value error from pandas, I've got to lookup how to check for that condition before we interpolate21:10
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add range search functionality  https://review.openstack.org/26486921:11
*** mudassirlatif has quit IRC21:12
*** mudassirlatif_ is now known as mudassirlatif21:12
*** moravec has quit IRC21:13
*** unicell has joined #openstack-infra21:17
anteayajesusaurus: okay thanks21:18
*** rajinir has joined #openstack-infra21:18
*** piet has joined #openstack-infra21:18
*** moravec has joined #openstack-infra21:19
rajinirasselin: Zuul is now triggering the jobs and I can see them in the status. Apparently gearman plugin port was set to 8080 instead of 4370( but returned sucess on test connection)21:19
jeblaircrinkle: i have not started on mine; but basically boils down to "if we want to destructively deploy often, we need to be able to tell nodepool to gracefully stop using a cloud via some ansible-triggered thing or other"21:20
anteayacrinkle: yay clean-up21:20
asselinrajinir, oh? is that the default? the test probably just checks for a telnet connection works and jenkins is on 8080 by default21:21
*** unicell has quit IRC21:21
adam_gdo the single-node devstack slaves' $STACK_USER have an ssh key created anywhere on them pre-job?21:21
crinklejeblair: okay thank you21:21
jeblaircrinkle: (it is conceivable that we might not need it as our thoughts on how to redeploy evolve; regardless, i don't think it will be too hard)21:21
*** bhunter71 has quit IRC21:21
*** unicell has joined #openstack-infra21:21
jeblairadam_g: stack user is created by devstack-gate i believe21:21
jeblairadam_g: (many words for no)21:22
rajinirasselin: Yes, its the default21:22
adam_gjeblair, yeah, i see some ssh setup for the multinode stuff in there21:22
rajinirasselin: New issue. nodepool->jenkins slaves->credentials. It is not sticking. Causes the slave agent to fail on ssh id: null. What could be wrong?21:22
jeblairadam_g: yeah, i think that is set up by the nodepool ready scripts21:22
rajinirasselin: The credentials are specified in /etc/nodepool/secure.conf21:22
jeblairmtreinish: fungi got you what you need, right?21:23
asselinrajinir, I did run into that once, but I don't remember the root cause...21:24
mtreinishjeblair: yep I think so21:24
*** kgiusti has left #openstack-infra21:24
*** bhunter71 has joined #openstack-infra21:24
asselinrajinir, I think there were some stack traces or clues int he jenkins log file21:24
jeblairpabelanger: tagged grafyaml 0.0.521:24
pabelangerjeblair: nice21:24
rajinirasselin: Looks like I'm stepping into issues every step :(21:25
asselinrajinir, would you please submit some doc updates with your fixes?21:25
*** baoli has joined #openstack-infra21:25
asselinrajinir, sorry to hear that :(21:25
rajinirasselin: Will that be here? https://github.com/openstack-infra/puppet-openstackci/tree/master/contrib21:25
*** baoli_ has joined #openstack-infra21:26
asselingit.openstack.org21:26
asselingithub is just a mirror, you can' submit anything there21:26
asselincan't21:26
rajinirasselin: Learning a lot though with these issues ;)21:26
fungiin fairness you can't submit anything to git.openstack.org either, has to go through gerrit21:26
rajinirok21:26
asselinrajinir, http://docs.openstack.org/infra/manual/developers.html21:27
rajinirwill do, a few more issues to debug21:28
rajiniron the same note, any follow on docs to switch the CI system to the live project from sandbox?21:28
fungilooks like that (puppet-openstackci) repo could stand to have a CONTRIBUTING.rst like cookiecutter provides. something along the lines of http://git.openstack.org/cgit/openstack-infra/bindep/tree/CONTRIBUTING.rst21:28
*** nikhil_k is now known as nikhil21:29
asselinfungi, ack21:29
*** shardy_afk is now known as shardy21:30
*** baoli has quit IRC21:30
rajinirasselin: what's the process to switch the CI system to a live project21:30
rajinirfungi: ok21:30
*** sabeen1 has quit IRC21:30
asselinrajinir, in general, you should avoid commeting to gerrit on live projects until you're confident you system is working.21:31
jeblairthe problem with searching for the word "ops" in source code is that one is overwhelmed with matches for the word "oops"21:32
rajinirasselin: Agree. Just want to plan ahead once we are confident. And the question of what determined confidence21:32
openstackgerritMerged openstack-infra/puppet-gerrit: Allow puppet to set Gerrit sshd options to control connections  https://review.openstack.org/26282021:32
openstackgerritMerged openstack-infra/puppet-hound: Logrotate the hound log  https://review.openstack.org/26438021:32
pleia2jeblair: haha21:32
*** ari has quit IRC21:33
asselinrajinir, when you think all the results your system produces add value to other contributors. You can also ask for permission with links to the log files, and people will tell you.21:33
*** mrmartin has joined #openstack-infra21:34
rajinirasselin: Thank you21:34
*** esker has joined #openstack-infra21:36
openstackgerritMerged openstack-infra/puppet-gerrit: Allow puppet to configure Gerrit diff timeout options  https://review.openstack.org/26348521:36
*** baoli_ has quit IRC21:36
*** baoli has joined #openstack-infra21:37
*** matrohon has quit IRC21:37
*** Sukhdev has joined #openstack-infra21:37
*** tsg__ has quit IRC21:39
*** matrohon has joined #openstack-infra21:39
*** mudassirlatif has quit IRC21:39
pleia2pabelanger: when I saw the note about stackalytics.org not updating, I remembered your recent email about moving it into prod and subsequently noticed that stackalytics.o.o has also stopped (last update 31 Dec), does something need to be looked at there?21:39
*** Swami_ has quit IRC21:39
*** Swami has quit IRC21:39
openstackgerritChristopher Aedo proposed openstack-infra/project-config: Update app-catalog link check job  https://review.openstack.org/26497821:39
pleia2I worded that poorly, moving stackalytics.o.o into prod21:39
*** mudassirlatif has joined #openstack-infra21:39
*** mrmartin has quit IRC21:39
openstackgerritJames E. Blair proposed openstack-infra/system-config: Upgrade grafana to 2.6.0  https://review.openstack.org/26497921:40
jeblairpabelanger: ^ think that'll work?21:40
*** tsg__ has joined #openstack-infra21:40
pabelangerpleia2: we haven't moved anything in the backend, so the 2 system should be independent of each other21:40
*** baoli has quit IRC21:40
pleia2pabelanger: right, their last update times are very different21:40
fungiwho was working on trying to get dib's debian-minimal element working for nodepool purposes?21:40
pabelangerjeblair: not sure. need to double check puppet-grafana module21:40
pleia2pabelanger: but neither of them are updating, is ours not updating intentionally, or broken?21:40
*** _amrith_ is now known as amrith21:40
anteayafungi: I do not know, but it was not I21:40
fungii recall someone was poking at it recently, and was curious to see if they'd be interested in collaborating on 26472621:40
*** baoli has joined #openstack-infra21:40
pabelangerpleia2: no, it should be working. The fact that both aren't means to me something in the code is not working as expected.21:41
openstackgerritMerged openstack-infra/puppet-diskimage_builder: Add lvm2 package to diskimage builder  https://review.openstack.org/25897421:43
pabelangerpleia2: stackalytics-processor is running, so must be something in the code that is stopping the update21:43
pleia2pabelanger: yeah, that was what I was wondering, anything in logs of use>21:43
pleia2?21:43
pabelangerpleia2: actually, I don't think we log anything.  We should fix that21:44
pleia2indeed!21:44
*** baoli has quit IRC21:45
pabelangerjeblair: Ya, I think we should be okay with version bump21:45
pleia2the only changes to stackalytics at the end of the year (before it broke) were config changes, so either something there or some other library updated in stackalytics that broke things21:46
pleia2the changes look ok though21:46
*** baoli has joined #openstack-infra21:46
pabelangerpleia2: FYI: rebooting stackalytics.o.o will kick-start everything again.21:48
pabelangerbest I can offer right now :)21:48
openstackgerritChristopher Aedo proposed openstack-infra/project-config: Update app-catalog link check job  https://review.openstack.org/26497821:48
pleia2pabelanger: that's ok, I can carve out some time to take a look too21:49
*** bharathm has quit IRC21:50
*** samyewell has quit IRC21:50
*** kgalanov has quit IRC21:50
*** jaosorior has quit IRC21:50
*** slagle has quit IRC21:50
*** devkulkarni has quit IRC21:50
*** jaypipes has quit IRC21:50
*** regXboi has quit IRC21:50
*** gongysh has quit IRC21:50
*** markvan has quit IRC21:50
*** craige has quit IRC21:50
*** DinaBelova has quit IRC21:50
*** boris-42 has quit IRC21:50
*** mescanef has quit IRC21:50
*** dtantsur|afk has quit IRC21:50
*** bharathm has joined #openstack-infra21:51
*** samyewell has joined #openstack-infra21:51
*** kgalanov has joined #openstack-infra21:51
*** jaosorior has joined #openstack-infra21:51
*** slagle has joined #openstack-infra21:51
*** devkulkarni has joined #openstack-infra21:51
*** jaypipes has joined #openstack-infra21:51
*** regXboi has joined #openstack-infra21:51
*** gongysh has joined #openstack-infra21:51
*** markvan has joined #openstack-infra21:51
*** craige has joined #openstack-infra21:51
*** DinaBelova has joined #openstack-infra21:51
*** boris-42 has joined #openstack-infra21:51
*** mescanef has joined #openstack-infra21:51
*** dtantsur|afk has joined #openstack-infra21:51
openstackgerritMerged openstack-infra/puppet-stackalytics: Share crontab lock with both processor and dump  https://review.openstack.org/24426821:51
openstackgerritMerged openstack-infra/puppet-jeepyb: Install libxml2-dev{,el} and libxslt-dev{,el}  https://review.openstack.org/25926821:51
*** ihrachys has joined #openstack-infra21:52
*** Kennan has quit IRC21:52
*** sabeen1 has joined #openstack-infra21:53
*** Kennan has joined #openstack-infra21:53
*** amitgandhinz has quit IRC21:53
*** mudassirlatif has quit IRC21:55
*** sabeen3 has joined #openstack-infra21:55
mtreinishfungi, pleia2: patch #3: https://review.openstack.org/#/c/264984/21:56
mtreinishI think that covers all the stack traces we were seeing in the log21:56
*** mudassirlatif has joined #openstack-infra21:56
mtreinishwell except for the IOErrors (which there wasn't a trace associated with)21:56
*** sabeen1 has quit IRC21:58
*** devkulkarni has quit IRC21:58
*** mudassirlatif has quit IRC21:58
*** ayoung has quit IRC21:58
*** mudassirlatif has joined #openstack-infra22:00
*** moravec has quit IRC22:01
*** thorst has quit IRC22:02
*** bhunter71 has quit IRC22:02
*** xyang1 has quit IRC22:02
*** trown is now known as trown|outttypeww22:04
*** mudassirlatif has quit IRC22:05
*** mudassirlatif has joined #openstack-infra22:05
*** Keedya has joined #openstack-infra22:05
*** baoli has quit IRC22:05
*** baoli has joined #openstack-infra22:06
*** rhallisey has quit IRC22:06
TheloHi all, when running nodepool image-build I get the following exception "(pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'logstash.openstack.org' ([Errno 101] Network is unreachable)""(http://paste.openstack.org/show/483270/) Does anybode know how to solve this?22:07
*** vgridnev_ has quit IRC22:07
openstackgerritJames E. Blair proposed openstack-infra/grafyaml: Add support for nullPointMode  https://review.openstack.org/26498522:07
*** _diana_ has quit IRC22:07
clarkbThelo are you behind a firewall/proxy?22:08
fungiThelo: looks like your image build is trying to pre-cache subunit profile information for optimizing jobs, but your firewall may be set up to disallow outbound mysql socket connections through the internet22:08
*** salv-orlando has joined #openstack-infra22:09
Thelobut why the mysql socket connection goes through the internet?22:09
Theloisn't the database local?22:09
fungino, that's the subunit2sql data we22:09
fungi're hosting22:09
fungihowever, there's a good chance your environment won't benefit from it anyway since it's timing details about our job runs, not yours22:10
clarkbya one option would be to remove it from the builds22:10
mtreinishfungi: it's of very questionable benefit for our jobs too :)22:10
fungimtreinish: indeed22:11
Thelois there anyway to skip this part then?22:11
jlvillalStrange issue. When doing a grenade ironic tempest run, apt-get seems to hang. When I did a non-grenade ironic tempest run, things worked. Not sure if anyone has ever seen this issue before. pstree pastebin here: http://paste.openstack.org/show/483271/22:11
mtreinishit's been on my list to make that better, it's just pretty far down22:11
*** Swami has joined #openstack-infra22:11
*** salv-orl_ has quit IRC22:11
*** jsavak has quit IRC22:12
*** matrohon has quit IRC22:12
openstackgerritJames E. Blair proposed openstack-infra/grafyaml: Add tooltip customization support  https://review.openstack.org/26498822:15
jeblairpabelanger: i have 2 more changes, https://review.openstack.org/264988 and https://review.openstack.org/264985 that would help with these nodepool dashboards22:16
openstackgerritMerged openstack-infra/elastic-recheck: Remove query for bug 1501435  https://review.openstack.org/26348822:16
openstackbug 1501435 in python-novaclient "osc 1.7 no longer can boot a server from volume" [Critical,Fix released] https://launchpad.net/bugs/1501435 - Assigned to melanie witt (melwitt)22:16
*** claudiub has quit IRC22:16
*** ihrachys has quit IRC22:16
*** SumitNaiksatam has quit IRC22:17
Thelofungi: on which port should the server connect to logstash.openstack.org?22:17
*** baoli has quit IRC22:17
fungiThelo: 3306, the mysql default query socket port number22:18
*** baoli has joined #openstack-infra22:18
zarojeblair, fungi : here you, hope you like this better: https://gerrit-review.googlesource.com/#/c/73651/22:18
pabelangerjeblair: cool, will review shortly. Just testing templating stuff ATM. Looks to be working22:18
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for neutron ipv6 floating ip bug 1531706  https://review.openstack.org/26498922:18
openstackbug 1531706 in neutron "API job fails with ExternalIpAddressExhausted" [Critical,Fix released] https://launchpad.net/bugs/1531706 - Assigned to Kevin Benton (kevinbenton)22:18
Thelofungi: ok you are right22:19
*** dkranz has quit IRC22:20
fungizaro: yeah, lgtm22:20
jeblairzaro: looks lovely!22:20
Thelotwo more questions: (1) is there any way to specify the version release in nodepool.yaml ( example ubuntu 14.04.02 )?22:20
jeblairzaro: aside from omg i cant copy and paste to show you the code i was talking about.22:20
jeblairnevermind.22:21
Thelo(2) In the CI can I choose not to build image daily and always pick the same one? if yes how?22:21
*** aopincar has joined #openstack-infra22:21
openstackgerritBrant Knudson proposed openstack-infra/project-config: Keystone-only uwsgi job  https://review.openstack.org/26499122:21
jeblairzaro: (i wanted to copy line 116 of ChangeInfo.java, but i guess i just have to say "line 116 of ChangeInfo.java")22:21
fungijlvillal: was there anything useful in the stdout of the controlling terminal? did you look in the grenade log?22:21
*** rossella_s has quit IRC22:21
jlvillalfungi: It just says: Running grenade in the terminal22:22
*** rossella_s has joined #openstack-infra22:22
jlvillalLet me look some more...22:22
*** tsg_ has joined #openstack-infra22:22
fungijlvillal: you might check the grenade.sh.log or whatever it's called (where stdout of grenade.sh is redirected)22:22
jlvillalfungi: I see it finish downloading 193 packages22:23
notmynameit it possible to expand the commit message display box/iframe thing in the new gerrit? https://review.openstack.org/#/c/264922/ seems to be a reasonably formatted message, but gerrit's UI is making it scroll horizontally22:23
fungijlvillal: oh, actually that's a child of stack.sh so have a look at the devstack.log22:23
anteayanotmyname: I don't have any horizontal scrolling in that commit message when I view it with gerrit's gui22:24
anteayanotmyname: vertical yes, not horizontal22:24
jlvillalfungi: http://paste.openstack.org/show/483273/  from grenade log22:24
notmynameanteaya: what browser? I see it in safari and chrome22:24
jlvillalAt end nothing more22:24
anteayafirefox22:24
*** pradk has quit IRC22:24
anteayaI see a horizontal scrollbar in chrome22:24
jlvillalfungi: And devstacklog has same content at end.22:25
fungijlvillal: hrm, yeah that's even stranger22:25
*** camunoz has joined #openstack-infra22:25
jlvillalVery strange that it was working when non-grenade. I am using Vagrant and libvirt. Recreating from scratch each test.22:25
*** tsg__ has quit IRC22:25
fungijlvillal: you might hit up the grenade/devstack maintainers in #openstack-qa for ideas22:26
jlvillalUse ansible to setup the box. So pretty automated22:26
jlvillalfungi: Thanks for your help22:26
fungiyw22:26
*** thorst_ has joined #openstack-infra22:27
*** esker has quit IRC22:27
clarkbthe gerrit GC time did spike briefly to 10%, and its hanging out around 2% currently22:28
zarojeblair: hmm, copy works for me. tried on both chrome and firefox.22:28
*** esker has joined #openstack-infra22:28
clarkbso we may need to restart gerrit in the nearish future and when we do that apply the next set of memory tuning22:28
clarkbzaro: which piece of tuning do you think will have the biggest impact and should go in first?22:28
* zaro looks22:29
*** isaacb has joined #openstack-infra22:29
*** baoli has quit IRC22:29
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config: Add a grenade job for testing oslo.versionedobjects  https://review.openstack.org/26499222:29
*** baoli has joined #openstack-infra22:30
openstackgerritBrant Knudson proposed openstack-infra/project-config: Keystone-only uwsgi job  https://review.openstack.org/26499122:30
*** thorst_ has quit IRC22:32
*** mudassirlatif has quit IRC22:32
openstackgerritPaul Belanger proposed openstack-infra/project-config: [WIP] Switch to templating for nodepool rackspace  https://review.openstack.org/26499422:33
pabelangerjeblair: ^ first test22:34
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config: Add a grenade job for testing oslo.versionedobjects  https://review.openstack.org/26499222:34
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: [WIP] Add support for templating dashboard  https://review.openstack.org/25089322:34
jeblairpabelanger: that's a forward looking statement, right?  the templating stuff hasn't landed?22:34
zaroclarkb: i think this one, https://review.openstack.org/#/c/262075/22:34
*** regXboi has quit IRC22:34
pabelangerjeblair: Ya, just cleaning that up now. I have it working locally22:34
clarkbzaro: thanks22:35
openstackgerritPaul Belanger proposed openstack-infra/project-config: [WIP] Switch to templating for nodepool rackspace  https://review.openstack.org/26499422:35
zaroclarkb: you think GC has stabalized?22:35
*** otter768 has joined #openstack-infra22:36
clarkbzaro: no, but it is starting to GC more and more22:36
openstackgerritMerged openstack-infra/irc-meetings: Removes Magnum Networking Subteam Meeting  https://review.openstack.org/26495422:36
clarkbso I want to be ready with another tuning option when we restart again22:36
*** gyee has joined #openstack-infra22:36
clarkbif we have to restart22:36
*** mudassirlatif has joined #openstack-infra22:37
zaroclarkb: cool.  this one isn't a perf tuning thing but i'm guessing should go in sooner than later. https://review.openstack.org/#/c/262611/22:37
jeblairpabelanger: somehow later on, i take it we get to use the same file for multiple providers?22:38
pabelangerjeblair: Yup, I think so too22:38
clarkbzaro: ok would be good to make sure jhesketh's concern is addressed (I think it is fungi did a check iirc)22:38
pabelangerjeblair: 1 dashboard to rule them all22:39
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: [WIP] Add support for templating dashboard  https://review.openstack.org/25089322:39
zarojhesketh: take another look at https://review.openstack.org/#/c/262611/ ? i think i've addressed your concern.22:39
*** aopincar has quit IRC22:39
*** eharney has quit IRC22:39
*** esker has quit IRC22:40
*** thiagop has quit IRC22:40
*** otter768 has quit IRC22:40
openstackgerritMerged openstack-infra/elastic-recheck: Add query for neutron ipv6 floating ip bug 1531706  https://review.openstack.org/26498922:40
openstackbug 1531706 in neutron "API job fails with ExternalIpAddressExhausted" [Critical,Fix released] https://launchpad.net/bugs/1531706 - Assigned to Kevin Benton (kevinbenton)22:40
anteayacan an irc op stop by #openstack please?22:41
*** baoli has quit IRC22:41
anteayanot sure if this is a mistake or intentional22:41
*** baoli has joined #openstack-infra22:41
*** tonytan4ever has quit IRC22:42
anteayacan someone kick ev1lkry?22:42
jeblairpabelanger: cool, left some comments which i hope are helpful :)22:42
anteayajeblair fungi clarkb help?22:43
pleia2pabelanger: it's official, I have no idea how stackalytics works (at least, not enough to fix this)22:43
anteayasomeone is flooding #openstack22:43
jeblairpabelanger: you tricked me :) my comments are on PS1 and my cover message are on PS2.22:43
pabelangerpleia2: :)22:43
Sam-I-Amanteaya: looks like an accidental paste22:43
Sam-I-Amanteaya: who has ops in there anyway?22:43
anteayaSam-I-Am: when will it stop22:43
pabelangerjeblair: great, will check shortly22:43
Sam-I-Amanteaya: i dunno, do you have ops?22:44
anteayaI don't, notmyname does and kicked them22:44
anteayanotmyname: thank you22:44
Sam-I-Amthere we go22:44
Sam-I-Amhow does one go about getting ops there?22:44
fungibeat me to it22:44
fungiSam-I-Am: a change to add a nick to http://git.openstack.org/cgit/openstack-infra/project-config/tree/accessbot/channels.yaml#n3722:45
*** jtomasek has quit IRC22:45
fungior, well, nickserv account if not the same as your primary nick22:45
*** isaacb has quit IRC22:46
Sam-I-Amfungi: ahh. you guys have code everywhere :)22:46
pleia2it's what we do :)22:46
jeblairSam-I-Am: make sure you're up to date with http://docs.openstack.org/infra/system-config/irc.html22:46
openstackgerritHongbin Lu proposed openstack-infra/project-config: Temporarily remove swarm test from Magnum gate  https://review.openstack.org/26499822:47
jeblairSam-I-Am: and propose any changes to that such as things we should expect from openstack-global irc ops :)22:47
*** jsavak has joined #openstack-infra22:50
pabelangerokay, time for food. BBIAB22:51
Sam-I-Amjeblair: what are the qualifications for this special list?22:51
anteayaSam-I-Am: you can create a patch with your nick in it22:52
jeblairSam-I-Am: i don't think there are any, other than the last few volunteers have said "i know how this works and i'm a reasonable person and i'm in a good timezone without a lot of coverage" and we said "okay". :)22:52
anteayaSam-I-Am: and use good judgement, as jeblair says22:53
jeblairperhaps some, possibly zero-length, subset of those constitute our actual requirements :)22:53
Sam-I-Amwell, my timezone is boring, but i spend a lot of time in #openstack22:53
Sam-I-Amoccasionally there is riffraff in there22:53
*** baoli has quit IRC22:53
*** mriedem has quit IRC22:54
anteayaSam-I-Am: you would be a good candidate for ops, in my opinion22:54
Sam-I-Ami think i have an irc client in my head anymore22:54
*** amuller has quit IRC22:54
*** baoli has joined #openstack-infra22:54
anteayaI believe that22:54
pleia2Sam-I-Am: this would add you to the access list of all the openstack channels in that file22:55
*** mudassirlatif has quit IRC22:55
*** jsavak has quit IRC22:55
pleia2(not saying it's a problem, but fyi)22:55
Sam-I-Amyeah22:56
nibalizerpabelanger: do you want to talk about 264378  ?22:56
*** rossella_s has quit IRC22:56
*** jsavak has joined #openstack-infra22:56
anteayanibalizer: I think he went for dinner22:56
nibalizerI think clark is right that f21 is eol and f22/23 not working isn't the criteria for turning 21 off22:56
clarkbwe have always said we would turn it off when $fedora EOLs22:56
clarkbwe let it go ~5 weeks over22:56
anteayaI think I'm going to go for dinner22:57
*** mudassirlatif has joined #openstack-infra22:57
clarkbzaro: for the git gc scheduling, I think we may already be doing that under gerrit with our repack cron?22:57
*** mudassirlatif has quit IRC22:58
openstackgerritMerged openstack-infra/project-config: Grafana: add sparklines to remaining providers  https://review.openstack.org/26494122:59
pleia2keep reading that commit as sparklies22:59
pleia2which would be better22:59
* nibalizer throws glitter in the air23:00
pleia2:)23:00
jeblairwheee!23:00
*** dims_ has joined #openstack-infra23:00
*** dims has quit IRC23:01
*** mudassirlatif has joined #openstack-infra23:01
craigeo/23:02
*** dingyichen has joined #openstack-infra23:02
fungiwho /invite'd rip_taylor?23:03
fungithough i guess that would have been confetti not glitter23:03
*** moravec has joined #openstack-infra23:04
clarkbI just wrote an email where I said Jenkins +1's after check tests pass and realized we still haven't s/Jenkins/Zuul/ in Gerrit23:05
*** baoli has quit IRC23:05
*** sridhar_ram has quit IRC23:05
*** baoli has joined #openstack-infra23:06
*** erlon has quit IRC23:06
*** rguillebert has joined #openstack-infra23:07
jeblairclarkb: i think zuulv3 would be a nice symbolic time to do that :)23:07
clarkb+123:07
clarkbconsidering jenkins won't exist at that point23:08
*** Keedya has quit IRC23:08
fungifull agreement23:09
TheloHi, does anyone know the answers to these?23:10
Thelo(1) is there any way to specify the os version release in nodepool.yaml ( example ubuntu 14.04.02 )?23:10
Thelo(2) In the CI can I choose not to build image daily and always pick the same one? if yes how?23:10
asselinThelo, not sure, but did you check the nodepool docs?23:10
Theloyes but I couldn't find the answer ( for example you can pick the distribution but I couldn't see anything about the release version)23:11
*** sigmavirus24 is now known as sigmavirus24_awa23:12
*** ujuc has joined #openstack-infra23:14
asselinThelo, for your previous question, yest you can skip the mysql to logstash.openstack.org by deleting the code that does that form the elements in your project-config23:14
*** tonytan4ever has joined #openstack-infra23:14
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud controller node  https://review.openstack.org/20969823:17
*** baoli has quit IRC23:17
*** tonytan4ever has quit IRC23:18
*** baoli has joined #openstack-infra23:18
Theloasselin: Thanks23:19
jesusaurusThelo: are you running your own nodepool or do you mean the one run by openstack-infra?23:19
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud compute node definition  https://review.openstack.org/21398023:19
Thelomy own23:19
Thelojesusaurus:my own23:19
*** sridhar_ram has joined #openstack-infra23:20
jesusaurusThelo: then the answer to (1) is to create images for the different os versions you want23:20
*** ujuc has quit IRC23:20
*** Swami_ has joined #openstack-infra23:21
asselinThelo, if I remember right, nodepool specifies a release and dib elements to use, and you can add additional dib elements in the project-config repo23:22
asselinusually it uses the latest via apt-get upgrade23:22
Thelojesusaurus: So If I create my own image (without nodepool build-image) then how do I tell nodepool to take it23:23
Theloasselin: what is a dib element?23:23
fungiyeah, i have a feeling that the big gotcha there is that apt-get upgrade will turn 14.04.2 into 14.04.later23:24
*** Swami has quit IRC23:24
asselinThelo, dib stands for disk image builder. it's a projects that nodepool uses to build images efficiently23:24
asselinThelo, http://docs.openstack.org/developer/diskimage-builder/23:24
fungiat least if using our bootstrapping scripts, they always want to upgrade the installed package set to whatever their latest equivalents are23:25
openstackgerritJames E. Blair proposed openstack-infra/project-config: Validate grafyaml based on master  https://review.openstack.org/26500923:26
openstackgerritJames E. Blair proposed openstack-infra/project-config: Grafana: Update OVH graph with axis labels  https://review.openstack.org/26501023:26
Theloasselin: Thanks I got this, but my idea was more to take an image which I know has a kernel version compatible with my storage client23:26
openstackgerritJames E. Blair proposed openstack-infra/project-config: Validate grafyaml based on master  https://review.openstack.org/26500923:27
openstackgerritJames E. Blair proposed openstack-infra/project-config: Grafana: Update OVH graph with axis labels  https://review.openstack.org/26501023:27
jeblairno one look at that first patchset23:27
Theloasselin: Taking Trusty for example, does not garantuee a specific kernel version.23:27
*** ujuc has joined #openstack-infra23:27
jeblairor at least if you do, ++ the second one ;)23:28
*** piet has quit IRC23:28
anteayaha ha ha23:28
*** piet has joined #openstack-infra23:28
*** gordc has quit IRC23:28
asselinThelo, I wonder if you can change the kernel version later on as its own dib element. Then when it boots, it will book into that.23:29
*** baoli has quit IRC23:29
*** baoli has joined #openstack-infra23:30
anteayafungi: rip taylor would have used glitter if it had been available23:30
fungitouché23:30
Theloasselin: Can I tell nodepool to use a local image?23:30
fungizaro: did you see me comments earlier about viewAllAccounts not granting access to /accounts/{id}/emails?23:31
fungier, my23:31
*** Qiming has joined #openstack-infra23:32
asselinThelo, i'd check the docs for that. My suggesting was to hack one of the element scripts to just apt-get install <desired kernel> before the image build finishes. Or see if you can skip the part that does the upgrade.23:34
asselinsorry...I haven't done this and not that familiar, just hoping those hints can help you figure it out23:34
Theloasselin: ok, many thanks for your help23:35
boris-42fungi: hi23:36
boris-42fungi: I have some troubles with working with stable branches in rally =(23:37
boris-42fungi: I tried to cherry-pick this one https://review.openstack.org/#/c/264976/23:37
boris-42fungi: and all dsvm jobs failed to start.. should I add some magic parameters in infra project config to make this work?23:37
jlvillalOn my local system, I want to run a modified version of devstack when doing a grenade run in my devstack-gate environment. Is that possible? If so, how can I do that?23:38
anteayapabelanger jeblair so trying to run tox -egrafyaml on 265010 (latest patchset) has me installing zuul-cloner (which is new behaviour) and then fails: http://paste.openstack.org/show/483285/23:38
anteayawhat did I do wrong?23:38
fungiboris-42: you proposed it to a branch called 0.1, but devstack-gate assumes all unregognized branches (e.g., not stable/kilo or stable/liberty) should be tested against the master branch of any other repos23:38
jlvillalI assume I somehow need to replicate how it works when someone submits a patch to 'devstack' and then it runs the test build.23:38
fungis/unregognized/unrecognized/23:39
boris-42fungi: so there is issues with requirements?23:39
boris-42are*23:39
*** dims_ has quit IRC23:39
boris-42fungi: because they are not sinked if we install rally in venv would it help?23:39
fungiboris-42: i believe it will run with the master branch of the openstack/requirements repo too, yes23:39
jeblairanteaya: tox --version23:40
anteayaanita@anita-hp:~/work/project-config$ tox --version23:40
anteaya1.6.1 imported from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc23:40
jeblairanteaya: i'm running with 1.9.0; i wonder if either your tox or your virtualenv is too old to supply the $VIRTUAL_ENV variable i was using23:41
fungiboris-42: does your 0.1 branch not work with master of everything else?23:41
anteayapossibly, I can get me a new tox23:41
*** doug-fish has quit IRC23:41
fungiboris-42: for devstack-gate integration tests, we're really not set up to support any non-master-based branches besides stable branches tied explicitly to the integrated release cycles23:42
fungi(stable/kilo and stable/liberty currently)23:42
boris-42fungi: so the problem is that 0.1 branch has all requirements however everything else must work master23:42
boris-42against master*23:42
boris-42the only thing is with python requirements they are not synced with current master23:43
*** baoli has quit IRC23:44
jeblairzuul.o.o is not quite as responsive as i would like it to be23:44
*** mriedem has joined #openstack-infra23:44
openstackgerritJames E. Blair proposed openstack-infra/project-config: Validate grafyaml based on master  https://review.openstack.org/26500923:45
jeblairanteaya: ^ hopefully that will at least prevent it from messing things up.23:45
jeblairanteaya: you may now want to remove the zuul directory from your home dir23:45
fungiboris-42: well, the other challenge is that syncing in upper-constraints.txt is the only way we even have now to pin requirements to earlier values. you'd somehow need to supply your own constraints list compatible with your branch (which likely requires extending the current job definition somehow)23:45
*** dims_ has joined #openstack-infra23:45
anteayaah thanks23:46
anteayait downloaded quite a few things: http://paste.openstack.org/show/483287/23:46
fungiboris-42: basically if you want to do integration testing with devstack-gate on non-master-based branches, you really need to stick to the integrated release cycle and standard stable branch names23:46
*** pahuang has joined #openstack-infra23:46
boris-42fungi: so just creating virtual env and running setup.py inside it in gates won't work in most cases?23:46
anteayaI guess that will teach me to validate yaml when not in a virtualenv23:47
fungiboris-42: it might--i have no idea whether you install rally in a virtualenv for your jobs now or not. if you don't care that everything devstack installs globally is using master requirements then it might be fine. this could be similar in some ways to how tempest is installed i guess23:47
jeblairzuul's cpu usage is all system time, and i'm unable to ssh into it.23:47
jeblairinfra-root: ^ possible situation developing23:48
fungijeblair: looking23:48
*** MIDENN_ has quit IRC23:48
*** ayoung has joined #openstack-infra23:48
*** shardy has quit IRC23:48
*** jsavak has quit IRC23:48
boris-42fungi: so yep I don't care about what is using OpenStack, because I am working with OpenStack API and it's python clients23:48
jeblairi'm going to log into rax and see if there's a console thing.23:48
boris-42fungi: so we will try to install everything in venv23:49
fungijeblair: i just did23:49
fungiapache tasks blocked23:49
jeblairfungi: then i'll stop :)23:49
*** weshay_xchat has quit IRC23:49
fungijeblair: though looks like a symptom, and since this isn't a teletype i can't look at more than the last page of the console23:49
*** ujuc has quit IRC23:50
*** dims_ has quit IRC23:50
fungijust seeing lots of info kmessages about apache2 tasks blocked for more than 120 seconds23:50
jeblairoh it's out of ram and is swapping23:50
jeblairi just got a shell23:51
fungiit finally let my ssh session through23:51
fungiyeah23:51
fungisystem load was up to ~40 and is falling now23:51
jeblairzuul seems to have a memory leak for the past few weeks23:51
fungiswapping heavily, yes23:52
fungijeblair: possibly ramping up around the time of the gerrit upgrade?23:52
fungimaybe all the unrecognized event types in the new stream are leaving trash in memory23:53
anteayaqueue length events are falling23:53
anteaya870 and dropping23:53
nibalizerjeblair: fungi I'm about23:53
fungior, you know, any of the numerous other things that could now be subtly different about interactions between gerrit and zuul23:54
*** [1]Thelo has joined #openstack-infra23:54
zxiiroanyone ever run into this error with JJB "RuntimeError: Maximum recursion depth exceeded"?  It's strange because I can only reproduce it using a Jenkins job. Running on CLI via the same system doesn't produce this problem.23:54
anteayajeblair: this paste is with your tox change after the zuul issue is addressed: http://paste.openstack.org/show/483288/23:55
jeblairfungi: it's also around the time of the connections work23:55
zxiiroit happens only when I run "jenkins-jobs test --recursive jobs/" too23:55
fungijeblair: on the cacti graphs, that first higher-ish ramp-up at week 51 looks likely23:55
fungiwe have what seem to be two restarts there, week 51 and week 5223:56
*** Thelo has quit IRC23:56
*** [1]Thelo is now known as Thelo23:56
fungithe first of those is the restart during the gerrit maintenance23:56
*** piet has quit IRC23:56
fungimemory was a lot lower prior to that restart23:57
jeblairfungi: i think that was the first restart since connections merged too23:57
anteayait just started using swap 2.5 days ago23:58
zarofungi: yes, i did notice your comments from earlier.23:58
fungientirely likely. looks like it was running a long while (month-ish) without a restart before that23:58
zarofungi: will look into it shortly.23:58
fungianteaya: yeah, but in your mind stack that graph on top of the memory usage graph and you'll see it sort of takes off once memory maxes out23:58
zarofungi: it only prevents non-admins from running the script, correct?23:58
anteayafungi: yup, I see that23:59
fungizaro: right, was mostly just hoping to find a way to configure the emails method to be accessible to non-admins23:59
anteayait hadn't ever been in swap prior to this time, sorry I seem to have communicated that poorly23:59
openstackgerritMerged openstack-infra/tripleo-ci: Include the branch name in row descriptions  https://review.openstack.org/25950523:59
zarocool.  let me run some tests to see what's going on there.23:59
openstackgerritMerged openstack-infra/tripleo-ci: Tweak the report colors a little  https://review.openstack.org/25950623:59
*** bpokorny has quit IRC23:59
openstackgerritMerged openstack-infra/tripleo-ci: Remove parse logs  https://review.openstack.org/25950723:59
« Wednesday, 2016-01-06 Index Friday, 2016-01-08 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!