Thursday, 2014-09-25

*** mmedvede has joined #openstack-infra00:00
nibalizeri don tthing thats right. I think $config_dir = '/etc/project-config' ; require => Vcsrepo[$config_dir] is right though00:01
fungiit's freeze-time00:01
clarkbnibalizer: that requires yo uto use vcsrepo...00:01
clarkbnibalizer: which isn't good parameterization imo00:01
fungilooks like we have three patches in the gate and one more which could use a last-minute +300:01
nibalizerif jeblair confirms that his way works then all good00:02
jeblairseems to! :)00:02
nibalizerim pretty wiped out from puppetconf at this point00:02
nibalizerokay lets do that then!00:02
openstackgerritA change was merged to openstack-infra/config: Add flame project on StackForge  https://review.openstack.org/12311300:02
*** marcoemorais has quit IRC00:02
jogoclarkb: is the logstash config stuff frozen in the big config repo re-org right now?00:03
clarkbjogo: it should be00:03
clarkbjogo: but all of it may not be because some of it is templatized00:03
clarkbjogo: we should treat it as frozen00:03
fungijogo: clarkb: logstash changes? i think we didn't put any of the logstash configuration up on the block for project-config00:03
fungiat least i skipped the logstash configuration changes because it looked like they weren't involved00:04
jogoclarkb fungi: ask because we are not parsing %{logmessage}00:04
jogoovercloud-controller0_logs/os-collect-config.txt *00:04
*** dannywilson has quit IRC00:04
jogocorrectly00:04
clarkbjogo: %{logmessage} comes from blank lines and lines we don't parse properly00:05
openstackgerritA change was merged to openstack-infra/config: Add puppet-manila to Stackforge  https://review.openstack.org/12380200:05
clarkbjogo: so that is the end result of a thing not parsed00:05
fungii was limiting my reviews to changes which touched the files in anteaya's git filter-branch sieve00:05
*** dannywilson has joined #openstack-infra00:05
jogoclarkb: right. so i was thinking filing a tripleo bug and removing that from logstash for now00:05
jogoafter consulting with tripleo folks first00:05
clarkbjogo: can we do it when we are less frozen? I mostly don't want to inadverdently break things (this is why we are frozen)00:06
stevemari guess i can't issue a tox command in a jenkins job :(00:06
clarkbstevemar: the majority of our jobs issue tox commands. do you have a log to where doing so is breaking?00:06
jogoclarkb: yes we can do this when things are less frozen, that is why I asked00:06
clarkbjogo: ok00:06
anteayafungi: tell me when I can clone00:06
stevemarclarkb, https://jenkins01.openstack.org/job/check-osc-dsvm-functional/1/console ctrl+f "Permission denied"00:06
stevemarclarkb, this is what I was kicking off: https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/osc-functional.yaml#L3000:07
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch gerrit to project-config  https://review.openstack.org/12289400:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Doc updates for project-config  https://review.openstack.org/12290800:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch jenkins to project-config  https://review.openstack.org/12289300:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch specs to project-config  https://review.openstack.org/12290400:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch gerritbot to project-config  https://review.openstack.org/12290500:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch nodepool to project-config  https://review.openstack.org/12290200:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch accessbot to project-config  https://review.openstack.org/12290300:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch zuul and zuul-dev to project-config  https://review.openstack.org/12253300:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Move zuul layout into /etc/zuul/layout  https://review.openstack.org/12253200:08
jeblairnow with ordering! ^00:08
anteaya:D00:08
fungiclarkb: jeblair: did one of you want to hit 122922 with a +3? it's the only remaining one i could find we'd want to squeeze in00:08
fungiother than the one still churning in the gate pipeline00:09
clarkbstevemar: that looks like a simple permissions issue. The user runing tox doesn't have perms to update that dir00:09
openstackgerritBoris Pavlovic proposed a change to openstack-infra/config: Add experimental zaqar-rally job  https://review.openstack.org/12339600:09
jeblairfungi: done00:09
clarkbstevemar: I think you want to run as the stack user00:09
fungigah. boris-42 just uploaded another patchset for that00:09
*** dannywilson has quit IRC00:10
boris-42fungi oh=)00:10
boris-42jeblair sorry for delay00:10
mwhudson2014-09-25 00:09:44.934 | Testing ICMP connectivit to mirror.rackspace.com00:10
mwhudson2014-09-25 00:09:44.935 | + ping -c 20 mirror.rackspace.com00:10
*** hdd has joined #openstack-infra00:10
stevemarclarkb, just add an su - stack before that i suppose?00:10
mwhudsonyeah, so about that00:10
*** marcoemorais has joined #openstack-infra00:10
mwhudsonoh well, it's some kind of progress00:10
* mwhudson goes for lunch00:10
fungiboris-42: i had already rebased and approved it for you00:10
stevemarlunch? where the heck is mwhudson working out of?00:11
mwhudsonstevemar: nz00:11
anteayaso we wait or move on and boris-42 can offer to project-config?00:11
boris-42fungi could you re approve it00:11
stevemarmwhudson, that explains that :)00:11
boris-42anteaya what kind of project-config?00:11
anteayastevemar: middle of the pacific00:11
anteayaboris-42: config is frozen00:11
stevemaranteaya, i was gonna say00:11
boris-42anteaya ah00:11
anteayaas of 10 minutes ago00:11
boris-42anteaya oh00:11
anteayaboris-42: http://specs.openstack.org/openstack-infra/infra-specs/specs/config-repo-split.html00:11
stevemartime to head on home00:12
boris-42anteaya okay no worries00:12
boris-42anteaya it won't be blocked forever?)00:12
jeblairanteaya: it was a rebase due to merge conflict of a previously approved patch00:12
anteayaboris-42: http://lists.openstack.org/pipermail/openstack-dev/2014-September/046904.html00:12
*** jergerber has quit IRC00:12
*** boden has joined #openstack-infra00:12
anteayajeblair: so the rebase can go through?00:12
*** dmsimard is now known as dmsimard_away00:12
jeblairwe can probably wait for it to land :)00:12
*** dimtruck is now known as zz_dimtruck00:13
anteayaokay00:13
* anteaya goes to make tea00:13
jeblairi don't think we will approve any changes we have not alrady approved at this point00:13
jeblairnow we will wait for the ones in the queue to land00:13
fungii assume we don't want to try and cram krotscheck's update to switch from group to groups in review.projects.yaml in, and will let it get reproposed to the new project00:14
*** Sincler has quit IRC00:14
mwhudsonoh, that error wasn't fatal, for whatever reason00:14
jeblairfungi: oh i thought we did want to do that00:14
clarkbI thought I +2'd it too00:14
jeblairi thought that was penultimate, and then the remove jobs change is ultimate00:14
fungijeblair: okay, lets00:15
krotscheckoookie dokie.00:15
jeblairoh, it has a conflict00:15
fungikrotscheck: did you need to update that patch one last time once the config gate goes silent?00:15
krotscheckVery likely, yes.00:15
jeblairthat's probably the best approach00:16
krotscheckLemme go see what happens.00:16
openstackgerritA change was merged to openstack-infra/config: UI, automation repos for group based policy project  https://review.openstack.org/12292200:16
*** patrickeast has quit IRC00:16
*** openstackgerrit has quit IRC00:16
fungiyeah, the in-flight change is just a job addition so shouldn't conflict with that file00:16
*** stevemar has quit IRC00:17
fungikrotscheck: but you want to make sure you rebase on top of that one that *just* merged00:17
fungi(since it did hit that file)00:17
*** openstackgerrit has joined #openstack-infra00:17
*** boden has quit IRC00:17
*** dims has quit IRC00:18
openstackgerritMichael Krotscheck proposed a change to openstack-infra/config: Project may now be assigned to multiple project groups.  https://review.openstack.org/11181500:18
*** alexpilotti has quit IRC00:18
krotscheckfungi: That should do it.00:18
*** todd_dsm has joined #openstack-infra00:18
krotscheckTurns out the conflict was due to a different patch of mine, too.00:18
fungihah00:18
openstackgerritIan Wienand proposed a change to openstack-infra/nodepool: Add meta option for passing meta-data  https://review.openstack.org/12363900:19
clarkbthat nova bug fix is going to merge assuming gerrit doesn't refuse to merge it00:19
ianwclarkb: ^ not sure that is clearer than setting on boot, suggestions welcome :)00:19
jogoclarkb: woot!00:20
ianw(the inconsistent use of meta / metadata is a bit annoying : https://review.openstack.org/#/c/123893/)00:20
jogoclarkb: looking at the log breakdown in the last 48 hours00:20
jogoand only half in the check queue have the screen tag on them00:20
krotscheckEvery time I see the project name ‘mistral’ I’m reminded of a really tasty restaurant here in seattle.00:20
clarkbianw: :/00:20
clarkbianw: but ya I think that is more like it00:21
*** _nadya_ has joined #openstack-infra00:21
clarkbianw: will try to properly review in the morning00:21
anteayamistral does ski equipment or did00:22
fungiokay, i'm +2 on the groups change and +2 on the change to remove redundant jobs00:22
ianwclarkb: np, thanks00:22
anteayacan't remember if it was downhill skis or water skis00:22
* clarkb loks at grops thing00:22
clarkbfungi: which chaneg is remove reudndant jobs change?00:22
fungiclarkb: 12219900:23
* anteaya hands clarkb some more vowels00:23
clarkbaeiou00:23
*** hdd has quit IRC00:24
anteayabless you00:24
*** alexpilotti has joined #openstack-infra00:25
jeblairaoeui00:25
clarkbok I am +2 on both00:26
*** alexpilotti has quit IRC00:26
clarkbdidn't approve to give jeblair a chance to look too00:26
jeblairclarkb: +200:26
clarkbare we ready to approve both of them?00:27
jeblairclarkb: we're waiting on https://review.openstack.org/#/c/123396/00:27
clarkbrgr00:27
fungilooks like it's about to get workers in check00:27
jeblairbut that's the last00:27
jeblairclarkb: i think we can aprv krotscheck's change tho00:28
jeblairno conflict with that one00:28
clarkbok I will do that00:28
clarkbdone00:28
*** zns has joined #openstack-infra00:31
*** groknix has quit IRC00:31
*** yaguang has joined #openstack-infra00:33
stevebakerdoes anybody know what this mysql_config not found error might be caused by? http://logs.openstack.org/96/110496/17/check/check-heat-dsvm-functional/2447467/console.html#_2014-09-24_22_47_16_32400:36
*** yamahata_ has joined #openstack-infra00:36
clarkbstevebaker: it means you are missing libmysqlclient-dev as a system package00:36
clarkbstevebaker: you probably need to run devstack with test packages install flage00:37
fungidevstack needs more flage00:37
stevebakerok00:38
jogoclarkb: after a spot check of logstash entries I didn't see anything that really stood out size wise00:38
jogothe breakdown is what I expected00:38
fungistevebaker: by default devstack only sets itself up with the packages necessary to run openstack. packages needed for things like unit tests are notably missing00:38
stevebakerhuh, am I going to hit the config projects freeze to fix check-heat-dsvm-functional?00:39
*** todd_dsm has quit IRC00:39
openstackgerritMichael Krotscheck proposed a change to openstack-infra/puppet-storyboard: StoryBoard Deferred Processors  https://review.openstack.org/11393700:40
*** mmaglana has quit IRC00:40
clarkbjogo: that doesn't really surprise me. I think the overall volume is just too high00:41
clarkbjogo: but I need to dig into it again00:41
clarkbstevebaker: possibly00:41
jogoclarkb: yeah, it doesn't look like we can realistically drop the volume in a meaningful way00:41
jogofor example a large chunk of the dsvm console.html output is just tempst job names00:42
jogoand we need that00:42
clarkbwe do?00:42
jogoand the other large chunk has the screen tag00:42
jogoclarkb: well arguably we can maybe not log the names of things that work00:42
jogoand only things that fail00:42
clarkb(I agree that ew probably don't want to drop tings just bceause but individual test names are not super useful)00:42
jogohttp://logs.openstack.org/81/123581/3/check/check-tempest-dsvm-full/b872ff9/console.html00:42
jogothat file is 417k00:43
anteayafeels like the first four hours of config freeze with be debating if config is frozen00:43
jogoand we parse and load *all* of it00:43
* anteaya considers going for a walk00:43
clarkbjogo: yup00:43
clarkbjogo: and we do all of the devstack log for it too00:43
jogoclarkb: those are half the size00:44
jogo211k00:44
jogodevstacklog.txt.gz that is00:44
jeblairanteaya: what do you mean?00:44
anteayajeblair: just trying to figure out where stevebaker's question fits in00:45
jeblairoh. yep.  frozen.00:45
anteayathanks00:45
jeblairstevebaker: is that a broken, voting job?00:45
clarkbI said possibly because there was no change yet00:46
clarkband it depends on when the fix comes in00:46
stevebakerjeblair: no, a new job which is yet to work00:46
jeblairstevebaker: then yeah, it'll have to wait, sorry00:46
stevebakerI'll propose a WIP anyway. Is there a new projects repo yet?00:47
jeblairstevebaker: no, we can't actually create it for several more hours00:47
stevebakerok, np00:47
fungibasically the reason for the freeze00:47
anteayawe have to wait for final patches to merge00:47
jeblairya00:47
*** melwitt has quit IRC00:47
*** dims has joined #openstack-infra00:48
jeblairso, unfortunately, we're probably behind a 710-deep merge-check queue00:48
anteayawooooo00:48
* krotscheck breaks out the movies and the popcorn.00:48
fungiand we approved them just as we hit the largest spike of merge-check jobs of the day launched, yeah00:48
jeblairanteaya: i would recommend that walk :)00:49
*** dkehn has quit IRC00:49
clarkbI may have opened a beer00:49
fungii am personally taking this as an opportunity to wash dishes00:49
anteayaI can do a one hour, hour and a half or a two hour walk00:49
fungithough i like clarkb's idea far better00:49
*** ChuckC has quit IRC00:49
anteayahow much do I have time for, do you think?00:49
jeblairanteaya: best guess: 1hr00:50
anteayaI'd like to do the most I can, I didn't get any in today but don't want to keep folks waiting on me00:50
anteayavery good00:50
* anteaya gets shoes and coat00:50
clarkbfungi: if you can get them an IPAs made by breakside and gigantic are amazing00:50
jeblairanteaya: also, i don't think you need to stay up for this00:50
jogoclarkb: for fun https://review.openstack.org/#/c/121914/ makes nova unit tests log pretty big too00:50
jeblairanteaya: if you want to do the repo split/import tomorrow morning i think that will be fine00:50
clarkbalso omsi after dark is tonight but I am missing it (have to go to seattle tomorrow and pack)00:50
jeblairanteaya: whatever works for you00:50
fungiclarkb: yeah, had them when visiting the west coast. haven't seen either here much00:50
anteayajeblair: I will check in once I get back from my walk00:50
jeblairanteaya: ok00:50
anteayathanks00:50
jogoconsole for that is now 291k whee00:50
clarkbso I don't know why pretty tox is a thing00:51
clarkbI personally hate it00:51
*** dkehn has joined #openstack-infra00:51
clarkbI only care when things break00:51
clarkbI don't need to see 10000 lines of passed passed passed. But apaprently I am odd00:51
fungiit does feel like people optimizing for a very bad workflow00:51
*** harlowja_ has quit IRC00:51
*** hogepodge has quit IRC00:51
fungibasically http://xkcd.com/303/00:51
fungiexcept staring at the screen rather than chair-jousting in the hallway00:53
clarkbfungi: https://www.omsi.edu/afterdark is tonight I went once a few years back. Its quite crowded but yo uget to drink beer while going through the exhibits00:53
* jeblair stops watching the merge-queue counter00:53
clarkbsuper awesome00:53
fungiclarkb: looks awesome00:54
fungioh, that reminds me, i did end up hitting brewfest on my way back from oscon00:54
clarkbfungi: nice00:54
clarkbfungi: I really wish they did it a bit more beer garden style00:54
clarkbbut it has grown to giant size and they do their best I guess00:55
clarkbI didn't bother with any samplers when I went. the lines were too long so I just pciekd beers that looked good and got full glasses00:55
*** lcheng has joined #openstack-infra00:55
fungii had a saturday evening flight, so slept in, was heading toward the airport on the max, decided to stop off and check out the park, noticed there was beer, went in and ate and drank for several hours, got back on the max and headed the rest of the rest of the way to pdx00:56
fungiit was one of those happy accidents00:56
clarkbwe enforced a time limit so as not to over do it :)00:56
fungii did do samplers because i had a ton of time to kill00:56
fungiso standing in line wasn't a problem00:56
*** homeless has quit IRC00:57
clarkbthe best part was when we had to a sign all kinds of paperwork to get a minor in00:57
clarkbthey advertise being minor friendly then make it such a pita00:57
fungialso, the starter pack of tokens all spent on samplers was plenty enough beer to prep me for an afternoon in the airport followed by an overnight cross-country flight00:57
fungisince they were usually overshooting the sample line on the glass, i'm sure i had ~6 pints, a lot of it high-gravity brews00:59
clarkbya I think most of the pourers are volunteers with OLCC licenses so they are generous01:00
*** Ryan_Lane has quit IRC01:00
clarkbits both good and bad. Gets locals involved but would be nice if the breweries themselves had a larger part in it01:00
*** dane_leblanc has joined #openstack-infra01:01
*** marcoemorais has quit IRC01:02
*** arnaud__ has quit IRC01:02
*** arnaud has quit IRC01:02
*** hdd has joined #openstack-infra01:03
*** emagana has quit IRC01:06
*** emagana has joined #openstack-infra01:06
*** mriedem has joined #openstack-infra01:08
*** zz_dimtruck is now known as dimtruck01:10
*** emagana has quit IRC01:11
*** thedodd_ has joined #openstack-infra01:12
*** boden has joined #openstack-infra01:13
*** thedodd_ has quit IRC01:13
*** thedodd has joined #openstack-infra01:14
*** stevemar has joined #openstack-infra01:14
*** shashankhegde has quit IRC01:14
*** daya_k has joined #openstack-infra01:15
*** marcoemorais has joined #openstack-infra01:15
fungii'm having trouble making heads or tails of the failure in http://logs.openstack.org/46/123846/1/check/gate-zuul-python27/fec879a/console.html01:16
*** hogepodge has joined #openstack-infra01:16
fungioh, nevermind. have to look at the testr_resylts.html01:17
*** boden has quit IRC01:17
*** harlowja has joined #openstack-infra01:18
*** nelsnelson has quit IRC01:18
*** marcoemorais has quit IRC01:19
*** hdd has quit IRC01:21
*** bswartz has joined #openstack-infra01:21
clarkbjeblair: I assume you are still around. Do you have any opposition to me moving all hpcloud devstack-trusty image builds to the dib image once we aer done with this reorg of repos?01:21
*** Sukhdev has quit IRC01:21
clarkbjeblair: https://review.openstack.org/#/c/123629/ is my test change. and the neutron and not neutron tests are both happy with that image01:22
*** Sukhdev has joined #openstack-infra01:22
openstackgerritIan Wienand proposed a change to openstack-infra/nodepool: Add meta option for passing meta-data  https://review.openstack.org/12363901:22
clarkbthen once those are moved to the regular node label I would set up rax with the -dib label and test those too01:22
clarkbbasically make sure ^ is working like we expect it01:22
*** stevemar has quit IRC01:22
clarkband that rax is generally happy with the image(s)01:22
fungii guess it's a question of whether we want to push forward building images different ways in two providers, or wait until we can reasonably switch in both01:23
*** stevemar has joined #openstack-infra01:23
clarkbfungi: I am not too worried about that because status quo is different images today01:23
*** bdpayne has quit IRC01:23
fungitrue, but at least relatively similar methods of assembling them01:23
jeblairclarkb: if i were going to be around next week, i'd say i'd be okay with doing hp in dib while we work on rax01:24
jeblairbut i'm not likely to have to deal with the consequences of that, so i'll abstain :)01:25
clarkbjeblair: thats fair :)01:25
*** otherwiseguy has joined #openstack-infra01:25
jeblair(and yeah, semi-around; eating while waiting for changes to go thru)01:25
*** todd_dsm has joined #openstack-infra01:25
clarkbI should eat dinner too /me does that01:25
fungii'm generally in favor of giving it a try as well, so let's go for it01:26
*** mmedvede has quit IRC01:26
*** daya_k has quit IRC01:27
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for keystone bug 1373680  https://review.openstack.org/12391001:28
uvirtbotLaunchpad bug 1373680 in keystone "keystone fails to start on icehouse side of grenade, "error: [Errno 98] Address already in use"" [Undecided,New] https://launchpad.net/bugs/137368001:28
clarkbI think the big win here is reducing the large number of image builds (save cpu time) and moving to a more consistent base. Even if we start doing that in small chunks that is worthwhile01:28
fungii see making the images we test on downloadable as a huge win. i know that's the step-3-profit part but still01:29
*** zns has quit IRC01:29
*** bhuvan has quit IRC01:29
jogowhat just happened? looks like the top of the gate is in queued and nodepool has free nodes01:31
*** Ryan_Lane has joined #openstack-infra01:32
jogoeven noops are queued heh01:32
*** Ryan_Lane is now known as Guest9493401:32
jogoodd, ignore me. there it goes01:32
clarkbjogo: merge check01:32
jogoclarkb:  oh merge-check can slow other things down?01:33
*** lcheng has quit IRC01:33
*** dmsimard_away is now known as dmsimard01:33
fungijogo: to some degree, yes. 123846 aims to soften the impact from it01:33
clarkbyes, though jeblair has a zuul change up to make it less problematic01:33
*** zns has joined #openstack-infra01:34
*** dmsimard is now known as dmsimard_away01:34
fungibut the way its unit tests are failing is still making my eye twitch01:34
jogoclarkb: oh interesting01:34
fungilooks like we're probably just a couple minutes out from merge-check emptying01:35
*** tchaypo has quit IRC01:35
*** tchaypo has joined #openstack-infra01:36
funginow we just need to wait for those two changes to get bare-precise nodes assigned in the check pipeline to run their apply jobs01:39
*** todd_dsm has quit IRC01:39
*** todd_dsm has joined #openstack-infra01:40
*** thedodd has quit IRC01:40
morganfainbergmriedem, +1 on that e-r query for keeping it on the radar01:41
*** daya_k has joined #openstack-infra01:41
morganfainbergmriedem, though 2x in 10 days is thankfully pretty low.01:41
morganfainbergmriedem, especially considering the volume of checks/gate runs around RC01:41
*** bdpayne has joined #openstack-infra01:42
*** JayJ_ has joined #openstack-infra01:42
*** signed8bit has joined #openstack-infra01:42
*** dmsimard_away is now known as dmsimard01:42
*** emagana has joined #openstack-infra01:43
*** marun has joined #openstack-infra01:44
*** nosnos has joined #openstack-infra01:45
*** todd_dsm has quit IRC01:45
*** arosen1 has joined #openstack-infra01:46
*** arosen1 is now known as arosen-home201:46
*** daya_k has quit IRC01:47
*** armax has quit IRC01:48
anteayaback01:49
*** todd_dsm has joined #openstack-infra01:50
*** zns has quit IRC01:50
*** dmsimard is now known as dmsimard_away01:51
*** zns has joined #openstack-infra01:51
*** dannywilson has joined #openstack-infra01:55
*** rmcall has quit IRC01:56
*** todd_dsm has quit IRC01:56
*** jpich has quit IRC01:57
anteayaany estimates on how long 'til precise nodes?01:57
clarkblooks like one node just went ready01:58
clarkbso things should start moving?01:58
*** flaviof_zzz is now known as flaviof02:00
openstackgerritA change was merged to openstack-infra/config: Add experimental zaqar-rally job  https://review.openstack.org/12339602:00
fungii just deleted a few bare-precise nodes nodepool thought had been ready for more than an hour... i think they missed getting added to a jenkins master at some point and were dead weight02:00
fungiso that ought to speed things up02:00
anteayaone down one to go02:00
*** Guest94934 has quit IRC02:03
*** bknudson has quit IRC02:04
anteayaah okay last one in the gate02:04
anteayathe config patch in check is this one: https://review.openstack.org/#/c/122903/ which isn't approved yet02:04
anteayaplus has unmerged dependencies02:05
jeblairanteaya: where's your remove-jobs change?02:05
anteayahttps://review.openstack.org/#/c/122199/02:05
anteayanot approved yet02:05
jeblairya, standing by to aprv it when next change lands02:06
anteayakk02:06
fungiit was waiting for the others to merge i think, so that we didn't remove jobs they would have otherwise run02:06
jeblair++02:06
anteayado you want 122199 landed before the filter branch?02:06
jeblairyep02:06
anteayakk02:06
fungiand assuming the filtered repo looks okay and someone else reviews 121650 before i start my day tomorrow, i can approve that and get it merged and confirmed created in gerrit before the west coast peeps are up and about for the second installment of fun02:09
openstackgerritA change was merged to openstack-infra/config: Project may now be assigned to multiple project groups.  https://review.openstack.org/11181502:09
jeblair122199 aprvd02:10
clarkbwait I am expected to be up and about?02:10
fungiout of curiosity, was there a plan to address the chicken-and-egg situation 121650 presents?02:10
anteayado expand on the chicken and egg situation you are thinking of02:10
jeblairreviewing that now02:10
fungipresumably we'll need to propose a copy of that change to the new repo once it's created02:10
*** cnesa13 has quit IRC02:10
jeblairfungi: yeah, we should02:11
clarkbfungi: I had not considered that, but yes02:11
fungisince it won't be included in the filtered repo it's importing02:11
anteayafungi: I'm slow, try me again?02:12
anteayaso a backport to project-config02:12
fungianteaya: 121650 uses the filtered repo to seed openstack-infra/project-config02:12
anteayafor the create the project-config repo?02:12
anteayayes02:12
clarkbthe first chane tp project-config should be a copy of 12165002:12
anteayainteresting02:12
fungibut the filtered repo has already been created from a state of openstack-infra/config from before 121650 was merged02:13
clarkbor maybe gitreview things first if necessary02:13
clarkbthen 12165002:13
* anteaya does inception brain for a minute02:13
fungiso it needs a copy of that change02:13
*** dannywilson has quit IRC02:13
fungiyeah, gitreview edits can either be squashed with that or the change prior to it02:13
clarkboh jeblair just -1'd as I +2'd02:13
jeblairanteaya, fungi: left comments on 121650, i think it will need some work02:13
anteayaoh okay02:13
clarkbjeblair: don't we want the puppet integration test tests?02:14
*** boden has joined #openstack-infra02:14
jeblairalso, those jobs that remain will likely fail, but we can probably work an a monolithic commit to fix them02:14
*** emagana has quit IRC02:14
anteaya8 comments02:14
jeblairclarkb: there's no puppet in that repo02:14
*** jpich has joined #openstack-infra02:14
clarkbjeblair: right but the /config repo depends on those files02:14
*** emagana has joined #openstack-infra02:14
fungii assumed they were going to be treated as integration jobs02:14
clarkbso this would be more of a make sure that deleting a thing doesn't break us? though the tests are probably not able to test that effectively02:15
fungialso there is one python script in the new repo i think, though i'm ambivalent about style checking it02:15
jeblairclarkb: but the apply test is --noop02:15
anteayathere is more than one python script now02:15
anteayasince we added 5 files from tools/02:15
clarkbjeblair: oh right, ya so I think the test isn't able to test that effectively02:15
jeblairanteaya: ok, we can keep pep8 then02:16
*** dane_leblanc has quit IRC02:16
* anteaya prepares to rebase02:16
*** dane_leblanc has joined #openstack-infra02:16
*** dimtruck is now known as zz_dimtruck02:16
jeblairclarkb: it's possible this could break the apply tests.  what we probably should do is actually execute the project_config puppet class at the beginning of the integration test02:16
*** thedodd has joined #openstack-infra02:16
jeblairclarkb: then do include the project_config repo in the integration test02:17
*** marun has quit IRC02:17
clarkbjeblair: execute it without --noop?02:17
jeblairthen i think we will effectively test that02:17
jeblairclarkb: yep.  like "puppet apply class {...}"02:17
clarkbthat seems like a reasonable approach02:17
fungiahh, yep02:17
openstackgerritA change was merged to openstack-infra/config: Removes redundant jobs from config gate and check  https://review.openstack.org/12219902:18
jeblairbut we can do that all tomorrow; i think we'll actually have quite a bit of work buliding these tests up again02:18
jeblair(that's what i'm planning on spending most of the day doing)02:18
clarkbwfm02:18
fungiquite probably so, yes02:18
fungii'm going to take this opportunity to knock off and pick up in the am02:18
*** boden has quit IRC02:18
jeblairanteaya: the remove-jobs change has merged, so you are clear to do the filter branch now02:19
jeblairanteaya: (at your leisure)02:19
*** emagana has quit IRC02:19
anteayathanks going to finish fixing 12165002:19
jeblairfungi: and with the comments i've left on 121650, i'm cool with you approving an updated version of that whenever everything is ready to your satisfaction in the morning02:20
fungijeblair: i figured. hopefully it'll all be working and i won't instead discover a dozen new bugs we've merged this afternoon02:20
*** bdpayne has quit IRC02:20
jeblairhehe02:20
jeblairwe merged so many bugs today!02:20
fungiwhich i am fully bracing myself for being a possibility02:20
clarkbfungi: we made sure to fedex next day them for you02:21
fungiindeed02:21
*** bdpayne has joined #openstack-infra02:22
*** arnaud has joined #openstack-infra02:22
*** dims has quit IRC02:23
openstackgerritAnita Kuno proposed a change to openstack-infra/config: Creates a new repo, project-config  https://review.openstack.org/12165002:24
clarkbjenkins should be in http://stackalytics.com/?metric=ci02:24
clarkbis that what "openstack community" is I have a hard time believing jenkisn isn't doing the most testing02:25
*** annegentle has quit IRC02:25
jeblaironce again, "community" is used in a divisive way.02:26
openstackgerritAnita Kuno proposed a change to openstack-infra/config: Creates a new repo, project-config  https://review.openstack.org/12165002:26
*** terryw has joined #openstack-infra02:27
*** annegentle has joined #openstack-infra02:27
*** otherwiseguy has quit IRC02:27
clarkbthis is what happens when it is late and I am reviewing code02:27
clarkbI was only good for 6k loc this cycle02:27
clarkbclearly I should pick up on that :P02:27
*** Sukhdev has quit IRC02:28
*** thedodd has quit IRC02:28
anteayamy ability to work tomorrow morning is entirely dependent on whether my monster neighbours want to drive heavy equipment around backwards tomorrow or not02:28
*** baoli has joined #openstack-infra02:28
anteayaso don't wait for me if any of my patches need fixing02:28
clarkbI am looking at 121650 now02:29
jeblairanteaya: i noticed one more thing on there02:29
anteayathanks02:29
clarkbthe groups thing was addressed02:29
anteayajeblair: what did I miss?02:29
jeblairanteaya: we need to change the job names for three jobs; but those will require jjb changes02:29
anteayacan I do that now?02:29
anteayaI would much rather work tonight when it is quiet02:30
fungiwow, "openstack foundation" made stackalytics' top-ten list of reviews by company02:30
anteayaha ha ha02:30
anteayago fungi02:30
jeblairanteaya: we could continue to do that in 121650, or we could punt and do those in another change (though we would also need to re-propose that to the new repo)02:30
anteayayour call02:30
anteayadoesn't matter to me02:30
clarkbare they not templated?02:30
jeblairclarkb: these are the special one-offs02:31
jeblairit might just be a matter of changing their names02:31
fungipoop. i was ranked #1202:31
* fungi is slacking02:31
anteayafungi: who beat you?02:31
anteaya12 from the foundation?02:31
anteayawho are the other 11?02:31
anteayaand where are they hiding?02:32
fungianteaya: no, i'm #12 on the reviews by engineer list02:32
anteayaoh well02:32
anteayaSergeyLukjanov never sleeps02:32
anteayaand AJaeger is cloned at least 3 times02:32
fungijogo and i barely missed the top 1002:32
anteayaso that is those two02:32
*** bdpayne has quit IRC02:32
anteayadang02:32
jeblairi'm 4th on the foundation list.  i'm not likely to catch up at this point.02:32
anteayayeah what a week left?02:32
anteayaI made it to 3002:33
anteayasecond female for reviews02:33
jeblairanteaya: and i'm not working for the foundation at all during that week02:33
anteayaxing beat me02:33
clarkbAJaeger is crazy. apparently wrote >300k loc02:33
anteayaha ha ha, good point02:33
clarkbthough all the docs folk appera to be high in the loc list. I call hacks02:33
openstackgerritKen'ichi Ohmichi proposed a change to openstack-infra/devstack-gate: Remove Nova v3 API test configuration  https://review.openstack.org/12302702:33
anteayait is almost like they have to write a lot of lines of code02:33
jeblairanteaya, clarkb, fungi: for 121650 i think any of these options would be okay:02:33
jeblaira) ignore the names of the jobs for now because it's probably not important anyway; b) drop the jobs and add them back later; c) changes the names in 12165002:34
clarkbI think I prefer b to a and c would be best but is more work02:34
clarkbso I will say b02:35
*** JadnG has quit IRC02:35
anteayaI have no opinion02:35
*** otter768 has joined #openstack-infra02:35
anteayabut yes due to chicken and egg b might be easier02:35
*** signed8bit has quit IRC02:36
openstackgerritJeffrey Zhang proposed a change to openstack-infra/config: Add nova-formula project  https://review.openstack.org/12391802:36
fungii kinda like a, but maybe that's my it's-late-why-am-i-still-on-the-computer-ishness talking02:36
anteayaoh yeah, a is the least work02:37
* anteaya changes her vote to a02:37
*** praneshp has quit IRC02:37
anteayaclarkb: can I sell you on a?02:37
jeblairwfm. 121650 +202:37
clarkba is fine too, its just has higher potential for wedging02:38
anteayajeblair: thanks02:38
jeblairg'night! :)02:38
anteayanight02:38
*** terryw has quit IRC02:38
* anteaya filter branches02:38
fungii expect we'll be working through the wedging tomorrow regardless02:38
clarkbfungi: godo point02:38
clarkbI do not oppose a02:38
clarkbbut I also think my ability to computer is quickly fleeing02:38
anteayayeah02:39
funginight all. more excitement awaits us on the morrow02:39
anteayaI'm just going to do the filter branch submit to github and call it a night02:39
anteayanight fungi02:39
clarkbsounds good02:39
anteayafungi: redo any of my work as you need to02:39
fungianteaya: i doubt there will be redoing, just more doind02:39
fungidoing02:39
anteayakk02:40
clarkbI will be on a train for much of tomorrow afternoon but in theory I will have 110v and wifi so intend on working02:41
anteayacool02:41
anteayawhere are you going from and to?02:41
*** otherwiseguy has joined #openstack-infra02:42
clarkbportland to seattle. going to finish packing then drive box truck back to portland on saturday and unload all my stuff02:42
anteayacool02:42
anteayayay portland02:42
anteayaI hope everything goes well02:43
clarkbty02:43
clarkbit should be straighforward, I actually have the seattle place for another month (woo leases) so its nota huge rush but camping in the new place is getting old02:43
anteayayeah02:43
anteayatoothbrush in a bag is fun for only so long02:44
anteayaso I'll have to make it to oscon next summer then02:44
anteayato watch everyone else eat your brisket and drink your beer02:45
*** jyuso has joined #openstack-infra02:46
*** zz_dimtruck is now known as dimtruck02:46
anteaya2500 commits to go02:47
*** todd_dsm has joined #openstack-infra02:47
*** marcoemorais has joined #openstack-infra02:48
*** marcoemorais1 has joined #openstack-infra02:50
openstackgerritJeffrey Zhang proposed a change to openstack-infra/config: Add cinder-formula project  https://review.openstack.org/12392002:52
*** marcoemorais has quit IRC02:53
anteayaonce folks are awake again: https://github.com/anteaya/project-config02:57
*** JadnG has joined #openstack-infra02:58
SpamapSmordred: you know how we were wondering how standalone Heat auths without a keystone catalog?02:58
SpamapSmordred: I wonder if we could just have KaaS (Keystone as a service) ...02:58
morganfainbergSpamapS, interesting.02:59
openstackgerritJeffrey Zhang proposed a change to openstack-infra/config: Add cinder-formula project  https://review.openstack.org/12392002:59
*** todd_dsm has quit IRC02:59
openstackgerritJeffrey Zhang proposed a change to openstack-infra/config: Add nova-formula project  https://review.openstack.org/12391803:00
*** rmcall has joined #openstack-infra03:00
openstackgerritAnita Kuno proposed a change to openstack-infra/config: Removes content split into project-config  https://review.openstack.org/12392103:02
lifelesssdague: fixtures is on github now03:02
*** thedodd has joined #openstack-infra03:03
openstackgerritJeffrey Zhang proposed a change to openstack-infra/config: Add neutron-formula project  https://review.openstack.org/12392203:05
*** otter768 has quit IRC03:05
anteayaI'm waiting for jzhang to show up03:06
anteayaI'll give him another few minutes before I head off to bed03:07
*** jeffrey4l has joined #openstack-infra03:08
*** thedodd has quit IRC03:09
*** thedodd has joined #openstack-infra03:09
jeffrey4lhey guys, could anybody explain this "The config repo is frozen. Please stop offering patches. These files won't be in config once we are done the work." ? What's means by *frozen*.03:10
anteayathere you are03:10
*** thedodd has quit IRC03:10
anteayawas wondering how long it would take you to read that03:10
anteayawas about to head off to bed03:10
anteayayeah the config repo is frozen03:10
anteayaplease stop offering patches03:10
jeffrey4lwhat's the reason?03:11
anteayawe are implimenting this spec: http://specs.openstack.org/openstack-infra/infra-specs/specs/config-repo-split.html03:11
anteayahere is the annoucement email: http://lists.openstack.org/pipermail/openstack-dev/2014-September/046904.html03:11
anteayathe changes you want will need to be offered to project-config once it is in gerrit03:12
anteayawhich it isn't yet03:12
jeffrey4lok. Got it.03:12
anteayaand if you clone if from github and work on it from there you are just wasting your time03:12
*** dane_leblanc has quit IRC03:12
anteayawe should be done tomorrow03:12
jeffrey4lUnderstand. Thanks for you information.03:13
anteayanp03:13
anteayaso save your energy03:13
anteayaand offer the patches to project-config tomorrow03:13
anteayagood night03:13
* anteaya heads off to bed03:13
jeffrey4lgood night03:13
*** boden has joined #openstack-infra03:15
*** emagana has joined #openstack-infra03:15
*** hdd has joined #openstack-infra03:16
*** JayJ_ has quit IRC03:16
*** rmcall has quit IRC03:16
*** bradjones has quit IRC03:18
*** boden has quit IRC03:19
*** emagana has quit IRC03:19
*** mbacchi has quit IRC03:19
*** pcrews has quit IRC03:22
*** bradjones has joined #openstack-infra03:23
*** bradjones has joined #openstack-infra03:23
*** dims has joined #openstack-infra03:23
*** jamesmcarthur has joined #openstack-infra03:25
*** reed has quit IRC03:28
*** dims has quit IRC03:28
*** mriedem has quit IRC03:30
lifelessmordred: dhellmann: ping03:30
lifelessre: namespace packages03:30
*** bdpayne has joined #openstack-infra03:32
*** todd_dsm has joined #openstack-infra03:35
*** Ryan_Lane has joined #openstack-infra03:38
*** Ryan_Lane is now known as Guest4397703:38
*** todd_dsm has quit IRC03:41
*** emagana has joined #openstack-infra03:47
*** todd_dsm has joined #openstack-infra03:47
*** koolhead17 has joined #openstack-infra03:48
stevemari'm super excited for this new repo03:48
stevemarthe old one was kinda... full03:49
*** emagana has quit IRC03:52
*** praneshp has joined #openstack-infra03:53
openstackgerritA change was merged to openstack/requirements: Upgrade django-pyscss to 1.0.3 for Horizon  https://review.openstack.org/12150903:56
*** praneshp_ has joined #openstack-infra03:56
*** jamesmcarthur has quit IRC03:57
*** praneshp has quit IRC03:58
*** praneshp_ is now known as praneshp03:58
*** coolsvap is now known as coolsvap|gone03:58
stevemarwhen is the new project-config repo going to be created?03:59
*** baoli has quit IRC03:59
*** jamesmcarthur has joined #openstack-infra03:59
*** jamesmcarthur has quit IRC04:00
*** gnuoy has quit IRC04:00
*** jamespage has quit IRC04:00
*** gnuoy has joined #openstack-infra04:00
*** jamespage has joined #openstack-infra04:00
*** Guest43977 has quit IRC04:00
*** dannywilson has joined #openstack-infra04:04
*** Sukhdev has joined #openstack-infra04:06
*** ChuckC_ has joined #openstack-infra04:11
*** boden has joined #openstack-infra04:15
*** Azher has joined #openstack-infra04:16
*** boden has quit IRC04:20
*** todd_dsm has quit IRC04:20
*** shashankhegde has joined #openstack-infra04:20
*** todd_dsm has joined #openstack-infra04:22
*** todd_dsm has quit IRC04:24
*** otherwiseguy has quit IRC04:24
*** otherwiseguy has joined #openstack-infra04:25
*** zns has quit IRC04:25
*** esker has joined #openstack-infra04:26
*** zns has joined #openstack-infra04:28
*** lcheng has joined #openstack-infra04:29
*** esker has quit IRC04:32
*** todd_dsm has joined #openstack-infra04:33
*** gokrokve has joined #openstack-infra04:35
openstackgerritA change was merged to openstack/requirements: Remove lesscpy  https://review.openstack.org/12230404:38
*** morgan_remote_ has joined #openstack-infra04:39
*** amcrn has quit IRC04:46
*** marun has joined #openstack-infra04:46
*** _nadya_ has quit IRC04:46
*** emagana has joined #openstack-infra04:47
*** emagana has quit IRC04:52
*** todd_dsm has quit IRC04:54
*** JadnG has quit IRC05:03
*** kashyap has joined #openstack-infra05:03
*** gokrokve has quit IRC05:04
openstackgerritKhai Do proposed a change to stackforge/python-jenkins: Add a get_plugin_info method  https://review.openstack.org/12393205:07
*** dannywilson has quit IRC05:09
*** otherwiseguy has quit IRC05:09
*** dannywilson has joined #openstack-infra05:09
*** dannywilson has quit IRC05:14
*** todd_dsm has joined #openstack-infra05:14
*** jpich has quit IRC05:15
*** boden has joined #openstack-infra05:16
openstackgerritKhai Do proposed a change to stackforge/python-jenkins: Add a get_plugin_info method  https://review.openstack.org/12393205:18
*** oomichi has joined #openstack-infra05:19
*** vigneshvar has joined #openstack-infra05:20
*** boden has quit IRC05:20
*** JadnG has joined #openstack-infra05:24
openstackgerritKhai Do proposed a change to stackforge/python-jenkins: Add a get_plugin_info method  https://review.openstack.org/12393205:26
*** coolsvap|gone is now known as coolsvap05:26
*** mrmartin has joined #openstack-infra05:27
*** esker has joined #openstack-infra05:28
*** mrmartin has quit IRC05:28
*** jpich has joined #openstack-infra05:29
jogofungi: time to do a bunch of reviews I guess :)05:29
*** hdd has quit IRC05:29
*** todd_dsm has quit IRC05:31
*** esker has quit IRC05:33
*** arnaud has quit IRC05:34
*** SumitNaiksatam has quit IRC05:34
*** SumitNaiksatam has joined #openstack-infra05:35
*** salv-orlando has quit IRC05:38
*** yaguang has quit IRC05:41
*** Ng has quit IRC05:47
*** Ng has joined #openstack-infra05:48
*** willroberts_ has quit IRC05:48
*** emagana has joined #openstack-infra05:48
*** willroberts_ has joined #openstack-infra05:49
*** jeffrey4l has quit IRC05:49
*** dannywilson has joined #openstack-infra05:49
*** ociuhandu has quit IRC05:50
*** koolhead17 has quit IRC05:50
*** ppai has joined #openstack-infra05:51
*** emagana has quit IRC05:53
*** harlowja is now known as harlowja_away05:53
*** amcrn has joined #openstack-infra05:55
*** shashankhegde has quit IRC05:56
*** Sukhdev has quit IRC06:03
*** flaper87|afk is now known as flaper8706:08
*** vdreamarkitex has quit IRC06:08
*** vdreamarkitex has joined #openstack-infra06:09
*** jeffrey4l has joined #openstack-infra06:09
openstackgerritA change was merged to stackforge/python-jenkins: Support node configuration  https://review.openstack.org/8960406:11
*** jlibosva has joined #openstack-infra06:14
*** yaguang has joined #openstack-infra06:17
*** boden has joined #openstack-infra06:17
*** changbl_ has quit IRC06:18
*** boden has quit IRC06:21
*** andreaf has quit IRC06:23
*** berendt has joined #openstack-infra06:26
*** HeOS has quit IRC06:27
*** esker has joined #openstack-infra06:29
SergeyLukjanovanteaya, nope! I'm sleeping sometimes - I like to sleep in the plane (and push client releases as well)06:31
*** esker has quit IRC06:33
openstackgerritDerek Higgins proposed a change to openstack-infra/config: Run tripleo tests on horizon  https://review.openstack.org/12394706:34
*** liam__ has joined #openstack-infra06:35
*** jeffrey4l has left #openstack-infra06:35
*** k4n0 has joined #openstack-infra06:37
*** MaxV has joined #openstack-infra06:40
*** liam__ has quit IRC06:41
*** JadnG has quit IRC06:42
*** _nadya_ has joined #openstack-infra06:43
*** JadnG has joined #openstack-infra06:45
*** JadnG has left #openstack-infra06:46
*** _nadya_ has quit IRC06:47
*** emagana has joined #openstack-infra06:49
*** camunoz has quit IRC06:52
*** emagana has quit IRC06:54
*** MaxV has quit IRC06:54
*** marun has quit IRC06:57
*** todd_dsm has joined #openstack-infra06:58
*** bradjones has quit IRC07:03
openstackgerritThomas Bechtold proposed a change to openstack-infra/config: Update ssh module to support SUSE  https://review.openstack.org/12395507:08
*** lcheng has quit IRC07:12
*** jcoufal has joined #openstack-infra07:12
*** hashar has joined #openstack-infra07:14
openstackgerritThomas Bechtold proposed a change to openstack-infra/config: Update snmpd module to support SUSE  https://review.openstack.org/12395607:14
*** arosen-home2 has quit IRC07:15
*** arxcruz has joined #openstack-infra07:15
*** zns has quit IRC07:16
*** rkukura has quit IRC07:17
*** boden has joined #openstack-infra07:18
*** trinaths has joined #openstack-infra07:18
*** ociuhandu has joined #openstack-infra07:20
*** jgallard has joined #openstack-infra07:20
*** boden has quit IRC07:22
*** hashar has quit IRC07:25
*** hashar has joined #openstack-infra07:28
*** jpich_ has joined #openstack-infra07:28
*** Guest68510 has joined #openstack-infra07:28
*** esker has joined #openstack-infra07:30
*** MaxV has joined #openstack-infra07:30
*** MaxV has quit IRC07:30
*** bradjones has joined #openstack-infra07:30
*** bradjones has quit IRC07:31
*** bradjones has joined #openstack-infra07:31
*** jpich has quit IRC07:32
*** jpich_ is now known as jpich07:33
*** ashaeron has joined #openstack-infra07:34
*** _nadya_ has joined #openstack-infra07:34
*** esker has quit IRC07:35
*** MaxV has joined #openstack-infra07:35
*** arnaud has joined #openstack-infra07:42
*** dannywilson has quit IRC07:43
*** dannywilson has joined #openstack-infra07:43
*** HeOS has joined #openstack-infra07:44
*** skraynev has quit IRC07:45
*** jcoufal has quit IRC07:46
*** jcoufal has joined #openstack-infra07:46
*** e0ne has joined #openstack-infra07:46
*** dannywilson has quit IRC07:48
*** e0ne has quit IRC07:49
*** pblaho has joined #openstack-infra07:50
*** emagana has joined #openstack-infra07:50
*** skolekonov has joined #openstack-infra07:50
*** e0ne has joined #openstack-infra07:50
*** emagana has quit IRC07:51
*** emagana has joined #openstack-infra07:52
*** arnaud has quit IRC07:54
*** e0ne has quit IRC07:55
*** emagana has quit IRC07:56
*** YorikSar has joined #openstack-infra07:56
*** bdpayne has quit IRC07:56
*** ociuhandu_ has joined #openstack-infra07:57
*** ociuhandu has quit IRC07:57
*** ociuhandu_ is now known as ociuhandu07:57
*** garyh has quit IRC07:58
*** emagana has joined #openstack-infra07:59
*** e0ne has joined #openstack-infra07:59
*** rushiagr_away is now known as rushiagr08:00
*** e0ne has quit IRC08:01
*** jp_at_hp has joined #openstack-infra08:01
*** rkukura has joined #openstack-infra08:02
*** dizquierdo has joined #openstack-infra08:03
*** dkehn has quit IRC08:05
*** dkehn has joined #openstack-infra08:07
*** jistr has joined #openstack-infra08:09
*** emagana has quit IRC08:10
*** morgan_remote_ has quit IRC08:10
*** e0ne has joined #openstack-infra08:10
*** jpich has quit IRC08:11
*** che-arne has quit IRC08:12
*** boden has joined #openstack-infra08:12
*** derekh has joined #openstack-infra08:13
*** IvanBerezovskiy has joined #openstack-infra08:14
*** arnaud has joined #openstack-infra08:14
*** zz_johnthetubagu is now known as johnthetubaguy08:16
*** e0ne has quit IRC08:17
*** stevemar has quit IRC08:23
*** skraynev has joined #openstack-infra08:24
*** andreykurilin_ has joined #openstack-infra08:24
openstackgerritChristian Berendt proposed a change to openstack-infra/subunit2sql: Stop using intersphinx  https://review.openstack.org/12397308:26
openstackgerritThomas Bechtold proposed a change to openstack-infra/config: Update pip module to support SUSE  https://review.openstack.org/12397408:29
AJaegerberendt, seems you have more repos checked out than I have ;)08:29
*** esker has joined #openstack-infra08:31
berendtAJaeger: https://raw.githubusercontent.com/openstack-infra/config/master/modules/openstack_project/files/review.projects.yaml08:31
berendtI use this list08:32
*** esker has quit IRC08:35
*** arnaud has quit IRC08:36
*** marcoemorais1 has quit IRC08:38
*** afazekas has joined #openstack-infra08:39
*** andreaf_ is now known as andreaf08:39
*** andreykurilin_ has quit IRC08:41
*** emagana has joined #openstack-infra08:41
*** ihrachyshka has joined #openstack-infra08:43
*** emagana has quit IRC08:45
*** openstackgerrit has quit IRC08:48
*** trinaths has left #openstack-infra08:50
*** laxity has quit IRC08:52
*** Longgeek has joined #openstack-infra08:52
*** rushiagr is now known as rushiagr_away08:52
*** laxity has joined #openstack-infra08:52
*** lcheng has joined #openstack-infra08:55
*** garyh has joined #openstack-infra08:55
*** koolhead17 has joined #openstack-infra08:55
*** lcheng has quit IRC08:59
*** _nadya_ has quit IRC09:09
*** _nadya_ has joined #openstack-infra09:09
*** che-arne has joined #openstack-infra09:12
*** _nadya_ has quit IRC09:14
AJaegerberendt, I haven't checked out all repos ;)09:23
*** e0ne has joined #openstack-infra09:23
AJaegerSo my grep for intersphinx did not catch all - thanks for getting the rest...09:24
*** alexpilotti has joined #openstack-infra09:24
*** esker has joined #openstack-infra09:31
*** e0ne has quit IRC09:34
*** Longgeek has quit IRC09:35
*** e0ne has joined #openstack-infra09:35
*** esker has quit IRC09:36
*** Guest68510 has quit IRC09:38
*** emagana has joined #openstack-infra09:41
*** Longgeek has joined #openstack-infra09:44
*** emagana has quit IRC09:46
*** maxbit has quit IRC09:49
*** IvanBerezovskiy has left #openstack-infra09:52
*** lttrl has quit IRC09:53
*** unicell1 has joined #openstack-infra09:53
*** unicell has quit IRC09:54
*** cnesa13 has joined #openstack-infra09:55
*** rushiagr_away is now known as rushiagr09:56
sdague lifeless thank you!09:59
*** mwagner_lap has joined #openstack-infra10:01
*** bdpayne has joined #openstack-infra10:06
*** lttrl has joined #openstack-infra10:07
*** loki184 has joined #openstack-infra10:07
*** dims has joined #openstack-infra10:09
*** bdpayne has quit IRC10:10
*** dims has quit IRC10:14
*** vdo has joined #openstack-infra10:17
*** jgallard has quit IRC10:18
*** yamahata_ has quit IRC10:20
*** viktors|afk is now known as viktors10:26
*** dizquierdo has quit IRC10:27
*** esker has joined #openstack-infra10:32
*** maxbit has joined #openstack-infra10:35
*** esker has quit IRC10:37
*** Azher has quit IRC10:38
*** dizquierdo has joined #openstack-infra10:38
*** _nadya_ has joined #openstack-infra10:38
*** IvanBerezovskiy has joined #openstack-infra10:39
*** emagana has joined #openstack-infra10:42
*** emagana has quit IRC10:47
*** ppai has quit IRC10:48
*** jyuso has quit IRC10:50
*** mpaolino has joined #openstack-infra10:51
*** mpaolino has quit IRC10:57
grue_pm.11:02
*** berendt has quit IRC11:02
*** ppai has joined #openstack-infra11:02
grue_pmany of you guys seen mordred around - I kind of need to catch him in a hurry11:07
*** dmsimard_away is now known as dmsimard11:08
*** hashar has quit IRC11:11
*** MaxV has quit IRC11:15
*** MaxV has joined #openstack-infra11:16
*** MaxV has quit IRC11:20
sdaguegrue_pm: this is pretty early for him11:23
*** esker has joined #openstack-infra11:24
grue_pmmy timezone conversion must have been wrong11:25
grue_pmah bother it - doesnt matter anyway.11:26
grue_pmI hate timezones - how dare he sleep ;-p11:26
*** MaxV has joined #openstack-infra11:30
*** pcm_ has joined #openstack-infra11:33
*** loki184 has quit IRC11:35
*** otter768 has joined #openstack-infra11:36
*** coolsvap is now known as coolsvap|afk11:37
*** dims has joined #openstack-infra11:41
*** emagana has joined #openstack-infra11:43
*** denis_makogon has joined #openstack-infra11:45
*** yaguang has quit IRC11:45
*** dims has quit IRC11:46
*** emagana has quit IRC11:47
*** otter768 has quit IRC11:48
*** otter768 has joined #openstack-infra11:48
*** baoli has joined #openstack-infra11:52
*** otter768 has quit IRC11:53
*** julim has joined #openstack-infra11:55
*** dimtruck is now known as zz_dimtruck12:02
*** mmedvede has joined #openstack-infra12:04
*** zz_naotok is now known as naotok12:07
*** dims has joined #openstack-infra12:07
*** bdpayne has joined #openstack-infra12:07
*** doug-fish has joined #openstack-infra12:08
*** amcrn has quit IRC12:10
*** kgiusti has joined #openstack-infra12:11
*** bdpayne has quit IRC12:11
*** signed8bit has joined #openstack-infra12:15
*** signed8bit has quit IRC12:15
*** signed8bit has joined #openstack-infra12:16
*** signed8bit has quit IRC12:16
*** esker has quit IRC12:18
*** kgiusti has quit IRC12:19
*** aysyanne has joined #openstack-infra12:20
*** bswartz has quit IRC12:22
*** kgiusti has joined #openstack-infra12:23
*** rushiagr is now known as rushiagr_away12:26
*** dprince has joined #openstack-infra12:26
*** rushiagr_away is now known as rushiagr12:28
sdaguettx: https://review.openstack.org/#/c/116708/ - how do you feel about approving that in so we can have the README in requirements before we branch it12:28
*** isviridov_meetin is now known as isviridov12:29
*** vigneshvar has quit IRC12:29
AJaegersdague, do you want to fix the two nits I noticed? Or shall I send a followup?12:29
sdagueAJaeger: honestly, I'd rather stop fixing minor typos at this point. That's how things like this die with the thousand paper cuts :)12:30
*** zz_dimtruck is now known as dimtruck12:30
AJaegersdague, that's why I didn't -1 it ;)12:30
*** sdake has quit IRC12:30
*** sdake has joined #openstack-infra12:31
*** adalbas has joined #openstack-infra12:32
AJaegersdague, created https://review.openstack.org/12403612:36
sdagueAJaeger: thanks12:36
*** mjturek has joined #openstack-infra12:38
ttxsdague: sure. I just haven't had time to read it yet12:40
ttxbut then we have time (requirements branch is cut when ALL RC1s are done)12:40
*** mpaolino has joined #openstack-infra12:41
*** JayJ_ has joined #openstack-infra12:41
*** vigneshvar has joined #openstack-infra12:41
*** hdd has joined #openstack-infra12:42
sdaguesure12:42
*** emagana has joined #openstack-infra12:44
*** naotok is now known as zz_naotok12:48
*** emagana has quit IRC12:48
*** e0ne has quit IRC12:48
*** e0ne has joined #openstack-infra12:49
*** signed8bit has joined #openstack-infra12:56
*** signed8bit has quit IRC12:57
*** signed8bit has joined #openstack-infra12:57
*** bswartz has joined #openstack-infra12:59
*** andreaf is now known as andreaf_12:59
*** ppai has quit IRC12:59
*** fifieldt has joined #openstack-infra13:01
*** yaguang has joined #openstack-infra13:03
*** juzuluag has joined #openstack-infra13:08
*** Guest68510 has joined #openstack-infra13:09
*** marcoemorais has joined #openstack-infra13:09
*** dhellmann has quit IRC13:10
*** achuprin_ has joined #openstack-infra13:10
*** dhellmann has joined #openstack-infra13:11
*** marcoemorais1 has joined #openstack-infra13:11
*** mriedem has joined #openstack-infra13:13
*** marcoemorais has quit IRC13:14
*** dane_leblanc has joined #openstack-infra13:16
*** kgiusti has quit IRC13:16
*** dustins has joined #openstack-infra13:16
fungiAJaeger: for checking out repos, i just iterate over the gerrit ls-projects api method output. that will get you *every* project in gerrit13:16
*** pblaho has quit IRC13:18
*** zns has joined #openstack-infra13:21
*** zns has quit IRC13:21
*** kgiusti has joined #openstack-infra13:21
*** _nadya_ has quit IRC13:22
fungiokay, new project-config project change 121650 and its specified upstream repo lgtm, so approving and keeping tabs on it now13:22
*** oomichi has quit IRC13:26
*** zns has joined #openstack-infra13:27
KiallHeya - I'm sure I'm just being blind, but where can I find the git repo containing http://specs.openstack.org/ ? Tis missing the Designate specs link!13:27
*** baoli has quit IRC13:29
*** vhoward has left #openstack-infra13:29
*** juzuluag has quit IRC13:29
*** fifieldt has quit IRC13:30
*** pcm_ has quit IRC13:30
*** baoli has joined #openstack-infra13:30
*** pcm_ has joined #openstack-infra13:31
*** dane_leblanc has quit IRC13:31
*** daneleblanc has joined #openstack-infra13:31
*** zz_jgrimm is now known as jgrimm13:32
*** xyang0 has joined #openstack-infra13:32
*** jaypipes has joined #openstack-infra13:34
fungiKiall: the index file is just plan html and currently resides in the openstack-infra/config repo, but is presently in the process of moving to a new openstack-infra/project-config repo later today13:37
fungiKiall: you might want to hold off and ask again tomorrow, since the project and file path you want to patch will be completely different than it is now13:37
Kiallooh - I never even thought to look there! And, if it's moving.. yea, holding off is better.13:37
fungiwe hope to have it knocked out later today13:38
*** _nadya_ has joined #openstack-infra13:38
KiallSure - No rush13:38
fungibut that's probably going to be pretty late in europe13:38
*** radez_g0n3 is now known as radez13:38
fungialso, we seem to have lost openstackgerrit again13:38
fungii'll go find it13:38
AJaegerKiall, will be in project-config/modules/openstack_project/files/specs/index.html13:38
Kiallfungi: Yea.. He said something about not being appreciated and ragequit ;)13:39
fungiAJaeger: it will only be there momentarily... the file paths are being reorganized in that repo as a next step after import13:39
AJaegerfungi, oops ;( I'm so used to these long paths ;)13:39
fungiAJaeger: the imported repo is just the result of a git filter-branch run so the original file path will match the source copy in the config repo, but then there will be a huge change full of git mv calls13:40
AJaegerfungi, fine with me ;) Thanks for the heads-up. I missed that somehow.13:40
AJaegerI like this ;)13:41
fungithe end goal is to have a much saner organized/structured set of data files which doesn't really have anything to do with puppet13:41
*** zz_gondoi is now known as gondoi13:42
fungifiles will be grouped by purpose and (loosely) by affected system13:42
*** dizquierdo has quit IRC13:44
*** emagana has joined #openstack-infra13:44
AJaegernice ;)13:45
AJaegerI'm looking forward to this13:45
*** dustins_ has joined #openstack-infra13:45
*** changbl_ has joined #openstack-infra13:45
*** mbacchi has joined #openstack-infra13:45
*** dustins has quit IRC13:45
*** hdd has quit IRC13:46
*** otherwiseguy has joined #openstack-infra13:47
*** esker has joined #openstack-infra13:48
*** homeless has joined #openstack-infra13:48
*** eharney has quit IRC13:49
*** emagana has quit IRC13:49
*** mrmartin has joined #openstack-infra13:50
derekhtchaypo: you had some kind of an opinion on this didn't you https://review.openstack.org/#/c/123453/ , I couldn't add you in gerrit for some reason13:52
*** daya_k has joined #openstack-infra13:53
derekhtchaypo: and I guess your asleep, so let me know if you don't get this message ;-)13:53
*** yaguang has quit IRC13:53
*** yaguang has joined #openstack-infra13:53
*** sballe_ has joined #openstack-infra13:54
*** AJaeger has quit IRC13:54
*** liusheng has quit IRC13:55
*** liusheng has joined #openstack-infra13:55
*** dangers_away is now known as dangers13:56
*** pcrews has joined #openstack-infra13:56
*** berendt has joined #openstack-infra13:58
*** morgan_remote_ has joined #openstack-infra13:59
*** radez is now known as radez_g0n313:59
*** nosnos has quit IRC14:00
*** nosnos has joined #openstack-infra14:00
*** gokrokve has joined #openstack-infra14:02
*** dizquierdo has joined #openstack-infra14:02
*** nelsnelson has joined #openstack-infra14:02
*** nosnos has quit IRC14:04
*** dims has quit IRC14:08
*** mfer has joined #openstack-infra14:08
*** dims has joined #openstack-infra14:09
sdakemorning14:11
anteayamorning sdake14:11
anteayafungi: thanks14:11
anteayafungi: anything I can do now?14:11
*** dimtruck is now known as zz_dimtruck14:12
*** dkehnm has quit IRC14:12
*** gokrokve has quit IRC14:14
*** mestery has quit IRC14:14
fungianteaya: probably not yet. poking around now to see if it worked14:14
*** mestery has joined #openstack-infra14:14
*** gokrokve has joined #openstack-infra14:14
*** tsg has joined #openstack-infra14:15
anteayadon't see it on git.o.o yet14:15
*** zns has quit IRC14:15
*** openstackgerrit has joined #openstack-infra14:15
fungiyeah, it's possible we approved one or more broken projects yesterday, so i'm digging through m-p output to figure out what's going on with it14:16
*** stevemar has joined #openstack-infra14:16
fungione thing to note, the -attic projects, because they're set to read-only, are not updatable from m-p so it throws errors about failing to push the config for them14:17
anteayaawesome14:18
fungithough that's been going on for a while and doesn't appear to impact creation of other projects14:18
*** ChuckC has joined #openstack-infra14:19
sandywalshfor you kids today https://pbs.twimg.com/media/ByW9JRfCUAAUiKj.jpg:large14:19
*** bknudson has joined #openstack-infra14:19
*** ChuckC_ has quit IRC14:20
fungisandywalsh: yep, i love that one14:20
*** hdd has joined #openstack-infra14:20
anteayasandywalsh: :D14:20
*** JayJ_ has quit IRC14:20
*** zns has joined #openstack-infra14:21
fungithis is fun... http://paste.openstack.org/show/115331/14:22
fungiOperationalError: (2006, 'MySQL server has gone away')14:22
*** gargola has quit IRC14:22
*** emagana has joined #openstack-infra14:22
*** daya_k has quit IRC14:23
*** david-lyle has joined #openstack-infra14:24
fungiaha, one cause is if the sql socket connection idles for too long and then the program tries to reuse it once it's timed out14:25
*** atiwari has joined #openstack-infra14:25
*** zz_dimtruck is now known as dimtruck14:25
*** ChuckC has quit IRC14:25
fungithis seems like an entirely likely possibility14:25
fungihrm, this one shouldn't be failing... 2014-09-25 14:25:39,785: manage_projects - ERROR - Failed to push config for project: openstack/sahara14:26
anteayaserver has gone away is always a fun error14:26
*** jgrimm is now known as zz_jgrimm14:27
*** david-ly_ has joined #openstack-infra14:27
*** Sincler has joined #openstack-infra14:28
*** torandu has quit IRC14:29
*** david-lyle has quit IRC14:29
*** mpaolino has quit IRC14:30
*** sballe_ has quit IRC14:30
*** koolhead17 has quit IRC14:31
*** JayJ_ has joined #openstack-infra14:32
fungioh, it looks like maybe we've broken puppet or the puppetmaster ran out of memory again14:33
fungiahh, yep, no puppet processes running at all on the master14:34
*** viktors has quit IRC14:34
*** tonytan4ever has joined #openstack-infra14:34
fungi[Wed Sep 24 20:35:58 2014] Out of memory: Kill process 22172 (ruby) score 313 or sacrifice child14:34
fungithat a'splains it14:34
funginothing we approved after 20:35 probably got applied anywhere14:35
*** emagana has quit IRC14:35
anteayayay14:35
*** Sukhdev has joined #openstack-infra14:35
fungii'm rebooting the puppet master server to make sure everything comes back up correctly14:35
*** tonytan4ever has quit IRC14:36
*** emagana has joined #openstack-infra14:36
*** markmcclain has joined #openstack-infra14:36
fungihuh. didn't start on its own either14:37
fungioh, there it goes14:37
*** mpaolino has joined #openstack-infra14:37
fungineeded an incoming connection through apache14:37
*** eharney has joined #openstack-infra14:38
anteayaah14:39
*** datsun180b has joined #openstack-infra14:39
fungii'm manually executing /opt/config/production/run_all.sh now just so i can watch its progress14:40
*** emagana has quit IRC14:40
anteayafungi: enjoy14:40
anteayalike watching progress bars maybe14:40
fungiwell, in this case i want to see what errors, if any, it throws14:40
*** openstackgerrit has quit IRC14:40
*** JayJ_ has quit IRC14:41
*** openstackgerrit has joined #openstack-infra14:41
anteayayes14:42
*** JayJ_ has joined #openstack-infra14:42
anteayatristanC is going to take care of today's election duties14:43
anteayaand tomorrow morning we get to create electral rolls14:43
anteayabut we can figure that out tomorrow morning14:43
*** maxbit has quit IRC14:43
fungii've cleared my morning tomorrow to make sure i can get the rolls assembled and double-checked before i hand them over14:43
anteayathank you14:43
tristanCfungi: alright! many thanks!14:44
clarkbhow many programs are without ptl candidates so far?14:44
anteaya514:44
anteayaand morning clarkb14:45
clarkbgood morning14:45
anteayahey you have commits in every program14:45
anteayaany look appealing to you?14:45
*** JayJ_ has quit IRC14:45
clarkbI do? I mean I might wouldnt surprise me14:46
tristanCclarkb: Trove, Swift, Glance, Sahara and Barbican14:46
*** marcoemorais1 has quit IRC14:46
anteayaclarkb: well all your tox change over patches14:47
*** radez_g0n3 is now known as radez14:47
*** ildikov has joined #openstack-infra14:48
*** tsg has quit IRC14:51
clarkbWoot I was just subscribed to project-config14:52
*** andreykurilin_ has joined #openstack-infra14:52
fungigood, it worked14:52
*** andreaf_ is now known as andreaf14:52
*** david-ly_ has quit IRC14:52
*** david-lyle has joined #openstack-infra14:53
fungii'll get our groups set up in it momentarily14:53
mordredmorning all14:53
anteayamorning mordred14:53
anteayayay14:53
*** juzuluag has joined #openstack-infra14:53
openstackgerritSteven Dake proposed a change to openstack-infra/config: Add Kolla project - Dockerizing OpenStack  https://review.openstack.org/12297214:54
anteayawhat did we decide is the first patch that project-config needs14:54
sdagueclarkb: can we drop lines that don't time parse in logstash instead of making up their time as parse time?14:54
anteayasdake: config is frozen, we have split it out14:54
*** tonytan4ever has joined #openstack-infra14:54
anteayasdake: that will need to be reproposed to project-config once project-config is taking patches, don't do it yet since you will just have to do massive rebases14:55
*** vhoward has joined #openstack-infra14:55
anteayasdake: best to wait a few hours14:55
clarkbsdague we can drop them sure. I dont understand why we would do that14:55
sdaguebecause they mostly confuse the issue14:55
clarkbsdague its better to handle parse failures gracefully imo14:56
clarkbwhat issue?14:56
anteayaI think the .gitreview patch is first14:56
sdagueso lets take - http://logstash.openstack.org/#eyJzZWFyY2giOiIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6OTAwLCJncmFwaG1vZGUiOiJjb3VudCJ914:56
clarkbdropping on the floor is going to make debugging hard impossible hard14:56
sdaguewhich is just the front page14:56
sdagueclarkb: what is the value of - "container-server: Started child 2197"14:57
sdaguethat has a time that's completely unrelated to when that message was created14:57
clarkbsdague the value is its there if I have to debug why it failed14:57
*** JayJ_ has joined #openstack-infra14:57
clarkbin this case its known but when oslo.log rewrites everything without telling anyone we need that info14:57
mriedemwhy don't i see pymongo in a tempest run? http://logs.openstack.org/95/123995/2/check/check-tempest-dsvm-full/d060d8e/logs/pip-freeze.txt.gz14:58
*** JayJ_ has quit IRC14:58
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Adds .gitreview file  https://review.openstack.org/12407414:59
sdagueclarkb: ok, so in this case should we change os-loganalyze to make those things not considered info lines?14:59
*** JayJ_ has joined #openstack-infra15:00
sdagueI guess we could do it on that side as well15:00
* anteaya stands by for reviews on that patch15:00
*** lttrl has quit IRC15:00
sdaguemostly we keep getting confusion that logstash is really 23 hrs behind15:00
*** groknix has joined #openstack-infra15:00
clarkbsdague I think that is slightly better but potentially has the same problem if we overmatch15:00
sdaguebecause there are hits for "now"15:00
sdagueand if oslo scrambled the world15:00
anteayamordred 12407415:00
*** xyang0 has quit IRC15:00
sdagueit would break badly enough that i think having everything jumbled in logstash wouldn't be any better than it not being there at all15:01
*** andreaf is now known as andreaf_15:01
clarkbthats not the point. the point is if its jumbled there you can debug it15:02
clarkbif its not there you have no info at all15:02
clarkbjust that its not there15:02
clarkbthe bug here is some of our logs are bad15:03
clarkbwe have the power to fix this in theory15:03
*** otherwiseguy has quit IRC15:04
*** bcrochet has quit IRC15:04
*** lcheng has joined #openstack-infra15:05
clarkb(granted I have tried and was met with massive resistance)15:05
*** armax has joined #openstack-infra15:05
*** beekneemech is now known as bnemec15:05
clarkbsdague: another option is to not try to parse files we know that are bad15:05
clarkbI am more open to that because we can specifically identifu issues in those cases15:06
sdagueit's not actually that our logs are bad15:06
sdagueit's actually that stuff is printing to screen15:06
sdagueremember, the logs we capture, aren't actually logs, they are screen sessions15:06
clarkbno in this case its swift being bad15:07
sdaguereally? I though that was a pre log handler setup print15:07
clarkbscreen session logging goes through python logging and can/should have timestamps like all loggong15:07
clarkbsdague nope I already match and drop that15:07
sdaguehmmmm15:08
clarkb(with positive matches not fail to parse drops)15:08
*** ZZelle has quit IRC15:08
sdaguecd /opt/stack/old/swift && /opt/stack/old/swift/bin/swift-object-server /etc/swift/object-server/1.conf -v15:08
* anteaya notes she is working on this etherpad to co-ordinate project-config stuff: https://etherpad.openstack.org/p/HT3TDOSChH15:08
sdaguethat's in ER15:08
*** berendt has quit IRC15:08
*** ZZelle has joined #openstack-infra15:08
*** bcrochet has joined #openstack-infra15:09
clarkbits all swift though. we dont have that problem with other screen files15:10
clarkbits possible the positive match is not sifficient for screen-swift*15:10
*** marun has joined #openstack-infra15:10
openstackgerrityolanda.robla proposed a change to openstack-infra/config: Add debootstrap to the needed packages for dib  https://review.openstack.org/12407715:10
sdagueright, because os-loganalyze dumps those lines15:10
sdagueso... that is a possible fix here as well15:11
*** baoli has quit IRC15:11
*** ChuckC has joined #openstack-infra15:12
openstackgerritA change was merged to stackforge/dox: Simplify Popen.communicate() result parsing  https://review.openstack.org/12404215:12
anteayayolanda: config is frozen and that patch is just going to have to be abandoned and reproposed to project-config after it is accepting patches15:13
anteayayolanda: don't propose to project-config yet since ti has to undergo a massive reorganization15:13
yolandaoh, ok, thx15:13
anteayanp15:13
yolandaanteaya, any idea about when?15:13
anteayasometime by end of day today we hope15:14
anteayaprobably your tomorrow15:14
yolandaok, not any urgent issue15:14
*** r-daneel has joined #openstack-infra15:14
*** baoli has joined #openstack-infra15:15
clarkbsdague: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/logstash/indexer.conf.erb#n1215:16
clarkbso a junk screen line starts with +15:16
sdagueyeh, though you don't actually see those except in the swift case15:17
sdaguebecause os-loganalyze considers those level=None15:17
sdagueso level=INFO drops them15:17
clarkbdrops the +?15:18
sdaguedrops the lines entirely15:18
fungiokay... i'm in the infra-core group in gerrit, i included it in the project-config-core gerrit group, i even flushed all gerrit caches, but i can't +2 124074 which is proposed against the openstack-infra/project-config project even though its acl says project-config-core should be able to...15:18
clarkbthen how do we get them in logstash?15:18
* jeblair is waking up15:18
*** otherwiseguy has joined #openstack-infra15:18
*** praneshp has quit IRC15:18
sdaguehttp://logs.openstack.org/01/123801/1/check/check-tempest-dsvm-full/b5f8b37/logs/screen-n-api.txt.gz vs http://logs.openstack.org/01/123801/1/check/check-tempest-dsvm-full/b5f8b37/logs/screen-n-api.txt.gz?level=INFO15:19
clarkbsdague logstash should drop any of those + prefixed lines if they come through15:19
sdaguebecause the swift log files are completely punted15:19
*** skolekonov has quit IRC15:19
clarkbI think the issue is swift logs dont have the prefix15:20
sdaguemaybe something else is going on15:20
*** zz_jgrimm is now known as jgrimm15:20
sdagueI'll look in a few, I'm breaking up some bugs right now in nova15:20
anteayafungi: interesting15:20
*** Longgeek_ has joined #openstack-infra15:20
anteayafungi: and thanks for at least trying to +2 the new gitreview file15:20
*** pradk has joined #openstack-infra15:21
clarkbfungi gerrit has basically told us to take the day off15:21
*** jlibosva has quit IRC15:21
* clarkb tries15:21
anteayagerrit and puppetmaster so far15:21
fungiif it's the same for you, i'll see if we ended up with divergent group uuids in gerrit for it or something like that15:21
anteayaI wonder what other tools are in on the work-to-rule15:22
sdagueclarkb: hmmmm http://logs.openstack.org/21/123021/4/gate/gate-grenade-dsvm/e5fd11e/logs/old/screen-s-object.txt.gz15:22
sdaguedefinitely no +15:22
*** hdd has quit IRC15:22
clarkbI have +2. try logging out and back in?15:22
*** hdd has joined #openstack-infra15:22
sdagueclarkb: oh... grenade runs don't have the +15:22
sdaguehttp://logs.openstack.org/21/123021/4/gate/gate-grenade-dsvm/e5fd11e/logs/old/screen-n-api.txt.gz15:22
*** jlibosva has joined #openstack-infra15:23
sdagueprobably because it's actually screen15:23
*** maxbit has joined #openstack-infra15:23
sdaguevs. the exec15:23
*** Longgeek has quit IRC15:23
*** yamahata_ has joined #openstack-infra15:24
*** annegent_ has joined #openstack-infra15:24
annegent_Anyone seen a reed?15:25
anteayaannegent_: I haven't seen him so far today15:26
annegent_ok thanks anteaya15:26
anteayanp15:26
sdagueclarkb: semi related - message:"%{logmessage}"15:27
sdaguelooks like the parsing for logs/overcloud-controller0_logs/os-collect-config.txt15:29
sdagueis completely bonkered15:29
fungiclarkb: strangeness... i did not log out and back in, but now i suddenly have expected access15:30
anteayaas a note, if anyone is inclined to drive around to look at the autumn colours in ontario/quebec/or north eastern states, they are shaping up to be _spectacular_ this year15:30
jeblairi+2d 12407415:30
*** andreaf has joined #openstack-infra15:30
anteayafungi: you said the magic words apparently15:30
anteayaor something updated15:30
anteayajeblair: thanks15:30
fungii approved it just to see how broken it is15:31
fungistill waiting on workers in check though15:31
anteayayeah, sitting here staring at it15:31
anteayago go15:31
*** lttrl has joined #openstack-infra15:31
*** xyang0 has joined #openstack-infra15:32
JayJ_Anybody have any pointers on how to slipstream a driver into Ubuntu PXE server? My server need a RAID driver to see the volumes15:32
*** rwsu has quit IRC15:33
openstackgerritJames E. Blair proposed a change to openstack-infra/zuul: Add precedence to merge jobs  https://review.openstack.org/12384615:33
fungiJayJ_: wrong channel?15:34
JayJ_fungi: I guess. Thanks :)15:35
fungiJayJ_: i was just trying to figure out how your question applies to the channel topic15:35
*** maxbit_ has joined #openstack-infra15:36
openstackgerritTomas Bezdek proposed a change to openstack-infra/jenkins-job-builder: Flatten attribute for publish plugins  https://review.openstack.org/12408115:36
*** _nadya_ has quit IRC15:37
*** andreaf has quit IRC15:37
*** andreaf has joined #openstack-infra15:38
jeblairanteaya: have you proposed your re-org change to project-config?15:38
anteayajeblair: not yet, I was waiting on the gitreview file15:39
jeblairanteaya: would you go ahead and do that please?15:39
anteayabut I can do so as a dependency on the gitreview file15:39
jeblairi can then start working on changes to make the jobs work15:39
anteayaI will15:39
anteayakk15:39
*** maxbit has quit IRC15:39
anteayadependency on the gitreveiw file okay?15:40
BobBallDo 3rd party CIs really need an email address? If the contact details are in https://wiki.openstack.org/wiki/ThirdPartySystems can we give a blackhole for gerrit emails?15:40
jeblairanteaya: sure; we're going to have to do some creative squashing anyway15:40
*** andreykurilin_ has quit IRC15:40
*** annegent_ has quit IRC15:40
*** asettle has joined #openstack-infra15:40
jeblairi think we will end up with 2 changes: (.gitreview + fix jobs) and (reorg + change paths in jobs)15:41
*** tsg has joined #openstack-infra15:41
sdakeanteaya sure do you have a top level git repo I canwork with?15:41
sdakejust same location with project-config instead of config?15:41
jeblairanteaya: but if you write gitreview and reorg, i can do the others and we can squash them15:41
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Reorganizes project-config  https://review.openstack.org/12408715:42
*** k4n0 has quit IRC15:43
anteayadone15:43
openstackgerritA change was merged to stackforge/dox: Close files after use  https://review.openstack.org/12403715:43
*** markmcclain has quit IRC15:43
anteayasdake: well project-config but it will be undergoing a massive directory shuffle as 124087 outlines15:43
jeblairthanks.  i've got some vms spinning up so i can run the tests; i'm going to go work on that for a little while.  biab.15:43
anteayasdake: honestly I would suggest you work on something else today (or help us?) and come back to it tomorrow15:44
sdakeanteaya cool I'll rebase tonight15:44
morganfainberganteaya, i hate to be the one pointing this out but looks like dreamhost build is posting "Starting" messages on (at least one) reviews https://review.openstack.org/#/c/117035/ at 04:54 this morning15:44
sdakeI'll rebase tomororw then  :)15:44
morganfainberganteaya, not sure if this is the right place to point it out/not15:44
morganfainberganteaya, or if it's a concern.15:44
anteayajeblair: kk15:44
anteayamorganfainberg: it is and it is15:45
anteayamorganfainberg: I will get back to it, thank you15:45
*** rwsu has joined #openstack-infra15:45
morganfainberganteaya, happy to help. let me know if you need anything else.15:45
anteayamorganfainberg: thanks15:45
clarkbsdague: ya jogo was looking at that but we hit config freeze15:46
anteayamorganfainberg: okay yeah, third party ci commenting on keystone is new for me15:46
clarkbsdague I thinkbhe wanted to just stop indexing that file15:46
sdagueclarkb: ok15:46
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for keystone bug 1373680  https://review.openstack.org/12391015:46
uvirtbotLaunchpad bug 1373680 in keystone "keystone fails to start on icehouse side of grenade, "error: [Errno 98] Address already in use"" [Undecided,New] https://launchpad.net/bugs/137368015:46
jeblairanteaya, clarkb, fungi: or maybe we just force-submit the first few changes instead of squashing them.  i kind of like that.15:46
anteayamorganfainberg: do you know who the operator of that ci is?15:46
anteayajeblair: your call15:47
morganfainberganteaya, no idea. the link doesn't go anywhere either.15:47
anteayajeblair: as long as my code is correct15:47
morganfainberganteaya, i've never seen that CI before tbh.15:47
anteayamorganfainberg: kk, we can just disable them then15:47
fungijeblair: wfm15:47
jogosdague: I filed a bug with tripleo over that. The good news is it looks like those bad entries aren't big percentage of the data we have15:47
clarkbjeblair wfm15:47
krtaylorBobBall, are you referring to the email listed in the request?15:48
morganfainberganteaya, it doesn't bother me, it might be an isolated incident, but it just wanted to raise the flag it's an oddity and shocked me15:48
sdaguejogo: yeh, I was just trying to get back to what we need to filter out to get logstash to report the last valid parsed thing15:48
BobBallkrtaylor: the email associated with the gerrit account that gets spammed each time someone comments on a change the CI has voted on15:48
*** praneshp has joined #openstack-infra15:48
anteayamorganfainberg: me too and multiple problems there and I need to focus on project-config so I will just disable15:48
morganfainberganteaya, ++ yeah will let you get back to prject-config stuff! /me lurks in the corner and watches you all do magic.15:49
derekhsdague: jogo https://review.openstack.org/#/c/122466/15:50
openstackgerritDavid Shrewsbury proposed a change to stackforge/dox: Use base class for config file parsers  https://review.openstack.org/12386415:50
anteayafungi: I don't have cloud@hq.newdream.net in my list of third party cis15:50
anteayamorganfainberg: thanks15:51
*** dannywilson has joined #openstack-infra15:51
*** gokrokve_ has joined #openstack-infra15:52
*** slagle_ has joined #openstack-infra15:52
*** mpaolino has quit IRC15:52
*** slagle_ has quit IRC15:52
*** slagle has quit IRC15:53
*** ihrachyshka has quit IRC15:53
*** annegent_ has joined #openstack-infra15:53
*** asettle has quit IRC15:54
*** slagle has joined #openstack-infra15:54
*** bdpayne has joined #openstack-infra15:54
*** jistr has quit IRC15:54
fungianteaya: looks like they're probably using a normal gerrit account15:54
*** gokrokve has quit IRC15:55
*** annegent_ has quit IRC15:55
*** marcoemorais has joined #openstack-infra15:55
anteayafungi: thanks15:56
*** annegent_ has joined #openstack-infra15:56
*** andreykurilin_ has joined #openstack-infra15:56
*** gokrokve_ has quit IRC15:56
*** kgiusti is now known as kgiusti_lunch15:57
fungianteaya: they seem to have created it just over a week ago15:57
anteayayes, dhellmann will email them and I have asked them to talk to me in -dev15:57
*** daneleblanc has quit IRC15:57
*** IvanBerezovskiy has left #openstack-infra15:58
fungiusername:dreamhost15:58
anteayafungi: can you disable that account while you are there?15:58
fungiaccount_id:1330815:58
fungidisabling now15:58
anteayajust so we quell the tide of patches15:58
anteayathanks15:58
dhellmannanteaya: they're on the west coast, so they may not be online yet15:58
anteayadhellmann: fair enough, well they are disabled now so at least we will have some quiet from them15:59
jeblairespecially in socal, they're probably surfing15:59
anteayatide of comments I mean15:59
*** dustins_ has quit IRC15:59
anteayasurfing would be good15:59
fungifwiw their lp profile seems to be set to private because i can't look them up through the api reversing their openid url15:59
anteayaI have never surfed15:59
anteayathat is odd16:00
anteayajeblair: what else can I do now to keep us moving along?16:00
anteayashould I submit the back port of https://review.openstack.org/#/c/121650/ as a dependency to the reorg patch?16:01
jeblairanteaya: good one; ++16:01
anteayakk16:01
fungianteaya: you'd probably need to insert it between 124087 and its parent, or else move the files around to match in your backport16:02
fungi(though git might be smart enough to figure it out from the mv info)16:02
*** vigneshvar has quit IRC16:02
*** mdbooth has joined #openstack-infra16:03
*** tsg has quit IRC16:03
anteayaI'll try as a dependency of the reorg and see if I fail16:03
mdboothAt some point in the past, when you told jenkins to recheck it would post a comment about having queued a job, and remove its -116:04
mdboothIt doesn't seem to be doing that now16:04
mdboothIs that intentional?16:04
*** esker has quit IRC16:04
*** andreykurilin_ has quit IRC16:04
*** dustins has joined #openstack-infra16:05
jeblairclarkb, fungi: what do i do to tox.ini to say "this isn't a python package, don't try to install it?"16:05
clarkbjeblair something like skipsdist = true16:05
fungimdbooth: yes, that was unfortunate noise we had to bear due to shortcomings in older versions of gerrit. we've since managed to upgrade and have turned that behavior back off16:05
jeblairclarkb: thx16:05
*** yaguang has quit IRC16:05
*** ChuckC has quit IRC16:06
mdboothfungi: The -1/-2 removal was very useful. Would it be possible to add that back without the comments?16:06
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for heat security group race bug 1374021  https://review.openstack.org/12409416:06
uvirtbotLaunchpad bug 1374021 in heat "test_server_cfn_init in check-tempest-dsvm-neutron-heat-slow fails with "BadRequest: Security group default already exists for project"" [Undecided,New] https://launchpad.net/bugs/137402116:06
mdboothThe reason is that when I've submitted a lot of changes, I will get a lot of failures16:06
fungianteaya: i've gone over the resulting tree in a checkout of 124087 and it looks perfect16:07
mdboothWhen Jenkins removed its -1/-2 after kicking off a job, it was really easy for me to see which ones I'd already done16:07
mdboothi.e. I could do a quick scan and expect to see either nothing or +1 in that column. -1 always required attention.16:07
mordredjeblair: k. I'm off the phone- where can I help?16:08
fungimdbooth: unfortunately changing a vote in the current versions of gerrit causes a new comment to be created16:08
mdboothfungi: Hmm. Is that so bad? If that's the price, I'd personally have it back, tbh.16:08
mdboothWas it causing problems?16:09
fungimdbooth: we have other systems which provide information on what's currently got jobs running, much more closely coupled to the systems running those jobs16:10
*** shashankhegde has joined #openstack-infra16:10
mdboothfungi: Right, it's just that it was a *really* useful view16:10
fungimdbooth: and yes, every comment on a change is yet one more e-mail sent to subscribers16:10
*** morgan_remote_ has quit IRC16:10
anteayafungi: thank you16:11
mdboothOtherwise I'd have to write a separate tool which hooks into both gerrit and zuul to tell me which changes require recheck attention16:11
*** rushiagr is now known as rushiagr_away16:11
*** koolhead17 has joined #openstack-infra16:11
*** koolhead17 has joined #openstack-infra16:11
fungimdbooth: probably better to spend time helping fix the bugs which are resulting in you needing to recheck over and over. that sounds like a really futile existence16:11
mdboothfungi: Heh, a trivial matter of programming :)16:12
mdboothSeriously, that's not a quick fix.16:12
mdboothAnd yes, it does very much feel like a really futile existence. No need to rub it in ;)16:13
*** acruz_ has joined #openstack-infra16:13
jeblairmordred: are you interested in hacking up the puppet apply test in infra/congfig to add the project-config repo to the zuul cloner (installing it in /etc/project-config) ?16:13
fungimdbooth: it's like that adage "dress for the job you want, not the job you have." we try to optimize for a working system rather than optimize to make it easier to live with a broken system16:14
jeblairmordred: (we figured then we can actually include project-config in our puppet integration tests)16:14
*** vdo has quit IRC16:14
mordredjeblair: ooh, neat16:14
mordredjeblair: yeah - lemme go look at that16:14
mdboothfungi: That's all very well, but in the mean time it leaves us dressed for the wrong job, with no currently outlook of a career change.16:15
fungidoesn't stop me dressing like a fireman ;)16:15
mdboothlol16:16
mdboothFireman is appropriate :)16:16
jeblairany objection to making the irc access check job voting?  (not sure why it isn't, actually)16:16
mordredjeblair: cloner is first-come-first-served, right? so if I want to put in a more specific rule, I put it first and it'll apply more sticky than the general .*/.* ?16:16
clarkbjeblair none here16:16
mdboothOk, my 2p anyway. Vote removal made me happy. At least I know it's intentionally gone.16:16
jeblairmordred: excellent question! :)16:16
*** ramashri has joined #openstack-infra16:16
fungijeblair: oh, no not at all. seems to pass normally16:16
*** arxcruz has quit IRC16:17
*** jlibosva has quit IRC16:17
*** MaxV has quit IRC16:17
fungimdbooth: there is work toward embedding views of testing activity into the gerrit webui, which i expect to be the eventual actual solution to that problem16:17
mdboothfungi: Sounds ideal.16:18
*** mmaglana has joined #openstack-infra16:19
*** arnaud has joined #openstack-infra16:19
*** jp_at_hp has quit IRC16:19
*** acruz_ has quit IRC16:20
jeblairfungi, mdbooth: it's _possible_ we could use the gerrit rest api to remove jenkins as a reviewer from a change (thereby silently removing its vote).  we need zuul to start using the rest api, and then add some features to zuul to enable that.16:20
jeblairthough, we should actually test my hypothesis first :)16:20
jeblairand it's a few days of hacking to actually implement it.  so it's not a short-term thing.16:21
zaroclarkb: you in seattle today?16:22
zaromorning16:22
*** Sukhdev has quit IRC16:23
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add project-config to zuul cloner  https://review.openstack.org/12410216:24
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Tell project-config about itself  https://review.openstack.org/12410316:24
*** bdpayne has quit IRC16:25
anteayamorning zaro16:26
*** packet has joined #openstack-infra16:26
*** mrmartin has quit IRC16:27
*** bswartz has quit IRC16:27
*** Azher has joined #openstack-infra16:28
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Tell project-config about itself  https://review.openstack.org/12410316:28
*** asettle has joined #openstack-infra16:30
*** e0ne has quit IRC16:31
anteayalooks like I need a tox.ini with the .gitreview file in order to pass jobs16:32
*** bradjones has quit IRC16:32
* anteaya works on a tox.ini file16:32
anteayahttps://review.openstack.org/#/c/124074/16:32
jeblairanteaya: please don't :)16:32
jeblairanteaya: that's what i'm working on16:32
anteayaI won't do that16:32
* anteaya gets out of jeblair's way16:32
*** bswartz has joined #openstack-infra16:33
*** dane_leblanc has joined #openstack-infra16:33
*** openstackgerrit has quit IRC16:33
*** cody-somerville has quit IRC16:33
*** gyee has joined #openstack-infra16:34
*** packet has quit IRC16:35
*** shashankhegde has quit IRC16:35
*** cody-somerville has joined #openstack-infra16:36
*** cody-somerville has quit IRC16:36
*** cody-somerville has joined #openstack-infra16:36
*** packet has joined #openstack-infra16:36
clarkbzaro on my way16:37
*** bradjones has joined #openstack-infra16:37
*** bradjones has joined #openstack-infra16:37
*** ashaeron has quit IRC16:38
*** salv-orlando has joined #openstack-infra16:38
*** mmaglana has quit IRC16:41
*** Guest68510 has quit IRC16:41
jeblairclarkb, fungi, mordred, anteaya: https://etherpad.openstack.org/p/HTlrATUqXy16:41
*** mmaglana has joined #openstack-infra16:41
*** tonytan4ever has quit IRC16:42
jeblairhow's that look for a plan?16:44
clarkbseems reasonable16:45
mordred++16:45
fungiyeah, covers everything we know about so far, anyway16:45
* mordred needs to move locations - back online in about 3016:45
jeblairfungi: advantage of using etherpad instead of paste!  we can insert the unknown unknowns later!16:45
*** gokrokve has joined #openstack-infra16:45
*** mmaglana has quit IRC16:46
anteayalooks good16:46
jeblairso i think 124109 is ready for review; i've locally tested: irc access check, compare xml, layout, pep8, bashate, projects-yaml16:47
*** gargola has joined #openstack-infra16:47
*** derekh has quit IRC16:48
*** rushiagr_away is now known as rushiagr16:48
jeblairanteaya: want to write a change to add project-config to gerritbot in this channel? :)16:49
jeblairi've merged the 1st 2 changes, waiting on review of 12410916:49
anteayais that not on the backport?16:49
anteayahttps://review.openstack.org/#/c/124103/2/gerritbot/channels.yaml16:50
*** lcheng has quit IRC16:50
jeblairthen why isn't it announcing in here now?16:50
jeblairor did you add that to the backport?16:50
anteayaI did16:51
jeblairoh ok.  then i guess we'll get it soon16:51
anteayaand I have announcements in the backsrcoll16:51
anteayaunless the bot left16:51
jeblairoh, yeah, i see them16:51
jeblairi guess we must have just lost the bot or something since the merges didn't get announced16:51
* jeblair looks16:51
clarkbso I stayed up much too late last night rereading up on mesos. I dont think it directly solves many or our problems but compute clusters are fun16:51
clarkbjeblair 124109 lgtm16:51
jeblairrestarted gerritbot16:52
fungidid we lose it again? i just restarted it a couple hours ago16:53
fungilooks like so16:53
jeblairyep16:53
*** openstackgerrit has joined #openstack-infra16:53
jeblairfungi: are you reviewing 109?16:53
fungiyeah, almost done16:54
fungilgtm. who was going to force submit those?16:55
jeblairme16:55
jeblairalready did the 1st 216:55
fungioic the first two already went in16:55
fungiyeah16:55
fungihence missing gerritbot noticed16:55
openstackgerritA change was merged to openstack-infra/project-config: Fix test jobs  https://review.openstack.org/12410916:55
jeblairthere we go!16:55
*** markmcclain has joined #openstack-infra16:56
*** patrickeast has joined #openstack-infra16:56
fungiso once zuul picks up the layout change we should be safe to recheck16:56
jeblairoh, it's still in the queue from the first time, so we need to wait for those results before rechecking16:56
jeblairfungi: no, we haven't merged the changes that will cause zuul to use that layout.yaml yet16:57
jeblairwe're still drifting on the old config16:57
fungihah! right16:57
jeblairbut the layout.yaml change there was unimportant16:57
*** asettle has quit IRC16:57
jeblair(it won't trigger the irc check, but other than that, we should be good)16:57
*** che-arne has quit IRC16:58
jeblairwe could probably go ahead and approve the changes to do the switch16:58
fungialso 123846 is passing jobs now if we wanted to work in a zuul restart sometime today16:59
*** baoli has quit IRC16:59
jeblairfungi: how convenient would that be? ;)16:59
*** dane_leblanc has quit IRC16:59
jeblairfungi, clarkb: you want to start at 122532 and work up the stack?17:00
jeblairfungi, clarkb: maybe don't approve 122532 itself until last so they all go in together?17:00
*** baoli has joined #openstack-infra17:00
anteayaI'm not much help on getting the tests working but do tell me if there is anything else I can do17:01
jeblairanteaya: you want to write the change against project-config that renames those jobs from 'gate-config' to 'gate-project-config'?17:01
*** SumitNaiksatam has quit IRC17:01
anteayaI can do that17:01
jeblairanteaya: and review the stack at 122532 if you haven't already (istr you reviewed earlier versions?)17:01
jeblairother than that, we're waiting17:02
anteayaI am reviewing the stack yes17:02
*** dizquierdo has quit IRC17:02
jeblairmordred: reviews of stack at 122532 are helpful17:02
anteayaam I also doing the jjb changes to make sure the gate-project-config jobs work?17:02
fungiyep, reviewing that stack while i lunch17:02
anteayais that the decision?17:02
jeblairanteaya: i don't understand?17:02
mordredjeblair: back on laptop in 1017:03
* clarkb reviews17:03
anteayawell to get those jobs working it was required to make changes in jjb was it not?17:03
anteayathat was my understanding17:03
jeblairanteaya: nope, just to rename them17:03
anteayakk17:03
jeblairanteaya:  the changes to get them working all merged in 12410917:03
*** packet has quit IRC17:03
anteayaah17:03
* jeblair afks for 15m17:03
*** packet has joined #openstack-infra17:03
*** dane_leblanc has joined #openstack-infra17:04
*** MarkAtwood has joined #openstack-infra17:06
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Tell project-config about itself  https://review.openstack.org/12410317:06
anteayathat fixed a whitespace error17:06
*** johnthetubaguy is now known as zz_johnthetubagu17:08
*** michchap has quit IRC17:09
*** shashankhegde has joined #openstack-infra17:09
*** marun has quit IRC17:10
*** lcheng has joined #openstack-infra17:10
*** bdpayne has joined #openstack-infra17:11
*** harlowja_away is now known as harlowja17:12
*** annegent_ has quit IRC17:14
anteayadoes config-compare.xml get renamed to project-config-compare.xml?17:14
fungianteaya: i tend to think yes17:15
anteayaI will17:15
fungianteaya: since project-config is where the jjb configs live now17:15
anteayaand will await reviews17:15
anteayakk17:15
*** mmaglana has joined #openstack-infra17:16
harlowjahave we been seeing any weird eventlet errors recently?17:17
harlowjahttps://github.com/eventlet/eventlet/issues/142#issuecomment-56844972 :-/17:17
harlowjathe requirements repo pretty much lists 0.15.2 (the one that is unstable)17:17
clarkbfungi nice catch on the gerrit switch change17:17
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Update jenkins job names that test project-config  https://review.openstack.org/12411617:18
*** HeOS has quit IRC17:18
clarkbharlowja it doesnt say >= something?17:18
*** bhuvan has joined #openstack-infra17:18
harlowjawell it says >= 0.15.1 and the only other version is 0.15.2 currently :-P17:18
fungii don't think that one's likely to merge-conflict when updated, so i'm going agead reviewing the rest on the assumption those votes will carry over as a trivial rebase17:18
*** lcheng has quit IRC17:18
*** drwahl has joined #openstack-infra17:19
*** Sukhdev has joined #openstack-infra17:20
fungiclarkb: but yeah, that could have been mildly disastrous if we failed to catch it before it got applied17:20
anteayafungi: is that patch kinda-sorta what jeblair has in mind do you think? 12411617:20
*** lcheng has joined #openstack-infra17:21
fungianteaya: well, the job names in jenkins job builder configuration will need to be altered to match17:21
anteayayes I had thought that was a part of it as well17:22
anteayaperhaps I am confused17:22
anteayaI will have something to eat and await jeblair's further guidance17:22
*** sressot has quit IRC17:22
*** john-davidge has joined #openstack-infra17:23
openstackgerritA change was merged to stackforge/dox: Use base class for config file parsers  https://review.openstack.org/12386417:23
fungianteaya: for example http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/infra.yaml17:24
anteayaI can just go ahead and make the changes in that file and add that to that patch17:25
anteayaI will do that17:25
fungigood deal17:25
*** dimtruck is now known as zz_dimtruck17:26
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Update jenkins job names that test project-config  https://review.openstack.org/12411617:27
*** SumitNaiksatam has joined #openstack-infra17:27
anteayayeah so SergeyLukjanov and notmyname and SlickNik either email the list or find someone who will17:28
*** zz_dimtruck is now known as dimtruck17:28
*** AJaeger has joined #openstack-infra17:28
*** arnaud has quit IRC17:29
anteayathis is shaping up to be a sleepless night for me, don't do that17:29
*** ChuckC has joined #openstack-infra17:30
SergeyLukjanovanteaya, I'll do it in an hour, do I have an hour? :(17:30
*** BobBall is now known as BobBallAway17:30
anteayaSergeyLukjanov: you have an hour, but don't delay17:30
notmynameanteaya: for the PTL nominations?17:30
anteayanotmyname: yes17:30
notmynameanteaya: isnt' the deadline tomorrow (/me is planning on doing it today)17:30
SergeyLukjanovanteaya, yeah, I'm trying to do 10 things at a time, but 10 more is waiting :( will try to send an email asap17:30
anteayanotmyname: your tomorrow will be too late17:31
notmyname?17:31
anteayanotmyname: you have to do it by the end of your day today17:31
*** dimtruck is now known as zz_dimtruck17:31
*** koolhead17 has quit IRC17:31
*** ivar-lazzaro has joined #openstack-infra17:31
anteayaAny member of an election electorate can propose their candidacy for the same election until September 26, 05:59 UTC.17:31
anteayathat works out to midnight tonight pst time17:31
anteayanotmyname: you are pst?17:31
notmynameanteaya: ya17:32
fungithat's actually 11pm pdt i think you'll find17:32
anteayaso today17:32
*** sressot has joined #openstack-infra17:32
*** dannywilson has quit IRC17:33
anteayathat time is a holdover from when elections were held in pst timeframes17:33
*** openstackgerrit has quit IRC17:33
*** dannywilson has joined #openstack-infra17:33
anteayaand nominations closed end of day thursday17:33
anteayaopenstackgerrit left again17:33
*** maxbit_ has quit IRC17:34
jeblairanteaya: oh, i have another for you; do you want to write a change to rm from config all the stuff that moved to project-config?17:34
anteayahttps://review.openstack.org/#/c/123921/17:35
*** MarkAtwood has quit IRC17:35
anteayabut jenkins doesn't like it17:35
*** maxbit has joined #openstack-infra17:35
fungiyeah, jenkins won't just yet17:35
fungibecause it'll need a rebase17:36
mordredjeblair: back17:36
jeblairbut moreover, AJaeger doesn't like it :)17:36
jeblairanteaya: but cool, i'll add that to the etherpad17:36
anteayajeblair: thanks17:36
fungioh, indeed. AJaeger hath spotted teh badness17:36
anteayayes I will address AJaeger's spotted badness17:36
*** sballe_ has joined #openstack-infra17:36
*** dannywilson has quit IRC17:37
SlickNikanteaya: Yes, I'm working on the email to the list — will be out in the next hour or two.17:37
anteayaSlickNik: good, no delays please17:38
jeblairif gerritbot were here, it would have told you i pushed a new series that fixes the gerrit dev thing17:38
*** dannywilson has joined #openstack-infra17:38
SlickNikanteaya: roger that.17:38
anteayaha ha ha17:38
*** afazekas has quit IRC17:38
* jeblair restarts gerritbot17:39
clarkbgah my tether thinks I should be hitting some 10 net address for everything why android why17:39
fungijogo: sounds like we're cheated out of an amusing opportunity to pointless debate as worthless alternative cantidates to an otherwise ptl-less project17:39
*** openstackgerrit has joined #openstack-infra17:40
fungipeople are actually on the ball with nominations this time around17:40
mtreinishfungi: it's not too late, throw your hat into the ring17:40
*** morgan_remote_ has joined #openstack-infra17:40
*** zz_dimtruck is now known as dimtruck17:40
fungimtreinish: nah, it's no fun when there are legitimate nominations. takes the joy out of it17:40
anteayafungi: you can still fight for glance17:40
*** koolhead17 has joined #openstack-infra17:41
*** koolhead17 has joined #openstack-infra17:41
anteayaI didn't mention markwash since he posted he wasn't running17:41
*** pcrews has quit IRC17:41
* anteaya gets something to eat before fixing the badness17:41
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for heat security group race bug 1374021  https://review.openstack.org/12409417:41
uvirtbotLaunchpad bug 1374021 in heat "test_server_cfn_init in check-tempest-dsvm-neutron-heat-slow fails with "BadRequest: Security group default already exists for project" (dup-of: 1370782)" [Undecided,New] https://launchpad.net/bugs/137402117:41
uvirtbotLaunchpad bug 1370782 in nova "SecurityGroupExists error when booting multiple instances concurrently" [Medium,In progress] https://launchpad.net/bugs/137078217:41
*** e0ne has joined #openstack-infra17:42
*** koolhead17 has quit IRC17:42
clarkbjeblair: considering the thing that fungi caught, should we be applying these changes as noop changes? more manually as we go?17:42
*** ramashri has quit IRC17:42
clarkbjeblair: not sure how paranoid you want to be17:42
*** pcrews has joined #openstack-infra17:42
fungii was considering making the same suggestion17:42
jeblairfungi: i say glance should store more data in swift!17:42
jeblairjogo: i say glance should store even more data in swift!17:42
fungitrying to think through whether --noop will actually do anything17:42
*** annegent_ has joined #openstack-infra17:42
*** koolhead17 has joined #openstack-infra17:43
jeblairwe'd probably have to do an initial checkout of the vcs repo manually17:43
fungijeblair: i say your glance data in switch doesn't go too far enough!17:43
fungis/switch/swift/17:43
notmynameEVERYTHING IN SWIFT!!!111!!!elevety!!!111!!17:44
jogojeblair: hehe17:44
*** koolhead17 has quit IRC17:44
fungijogo: i'm considering running on a platform which would dissolve glance and re-merge it back into nova. i just can't tolerate your unfettered love for expanding swift into an artifact mega-mart17:45
jogonotmyname: you know swift doesn't have anyone running yet  ...17:45
fungier, expanding glance17:45
fungisee, i care so little about the project i'm running for i can't even remember which name it has17:45
jogofungi: is it more like kmart or walmart17:46
notmynameI've actually been wondering what features would need to be in swift to allow nova to talk directly to swift for image management17:46
fungijogo: you should propose a rename as part of your platform. i wonder if anyone already has a trademark on either of those17:46
notmynamewe've had a _lot_ of requests for better ways to index/search (or hooks at least). and we've got some interesting thing in-progress around permissions that solve some problems around who can edit what17:47
jeblairjhesketh caught another error; i'm about to push another series in a minute17:47
jogonotmyname:  I like that idea a alot. just cut glance out all together17:48
fungii need to start a new cloud service which allows customer money to talk directly to your company's bank account... cut out all that needless software in the middle17:48
*** Ryan_Lane has joined #openstack-infra17:48
jogonotmyname: although some people use shared filesysystem backends for glance as well17:48
clarkbjogo: ya was just going to say17:48
clarkbglance makes sense in that context17:49
*** arnaud has joined #openstack-infra17:49
jeblair(especially shared filesystems with excellent client-side caching and invalidation protocols)17:49
notmynameyeah, I'm not sure how realistic "totally replace glance" is, but I strongly suspect there are some things we can do in swift to offload stuff from glance17:49
*** lttrl has quit IRC17:50
notmynameas in, solve hard problems for the app (glance) so the app devs can focus on the value they bring17:50
*** lttrl has joined #openstack-infra17:50
*** arnaud has quit IRC17:50
*** reed has joined #openstack-infra17:50
fungiclearly the answer is _artifacts_17:50
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch gerrit to project-config  https://review.openstack.org/12289417:51
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Doc updates for project-config  https://review.openstack.org/12290817:51
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch jenkins to project-config  https://review.openstack.org/12289317:51
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch specs to project-config  https://review.openstack.org/12290417:51
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch gerritbot to project-config  https://review.openstack.org/12290517:51
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch nodepool to project-config  https://review.openstack.org/12290217:51
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch accessbot to project-config  https://review.openstack.org/12290317:51
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch zuul and zuul-dev to project-config  https://review.openstack.org/12253317:51
clarkbjeblair: which change was the error in?17:51
jeblairclarkb: zuul and gerrit17:51
jeblairdev/ instead of /dev17:51
clarkbthanks17:52
fungiohh, my eyeballs totally glazed over on that17:52
clarkbalso ec2 is rebooting all the nodes17:52
clarkbshould we reboot all the oepnstack clouds too?17:52
jeblair??17:52
fungihttp://cheezburger.com/48661836817:52
clarkbjeblair: mostly a bad joke (we like doing what amazon does [or at least we often seem to do these things])17:53
clarkbgah my wifi hopped on PSUs network and was captive portalled that explains the tethering weirdness17:53
clarkbthat said I would not be surprised if rax had to reboot everything as soon as they patch17:54
clarkbapparently a xen bug17:54
clarkbcritical enough to reboot all ec2 instances17:54
fungiyeah, there was a heapin' helpin' of xen security bugs announced yesterday17:54
clarkbso I guess we should be prepared for the possibility17:55
anteayawas watching the 5 turkeys on my lawn17:55
jeblairyeah, you know, just the portion of our system in rax :)17:55
*** lttrl has quit IRC17:55
fungimostly lost in the noise of "shellshock" (because everyone's servers are like so totally pwnd by the bash environment parsing issue or something)17:56
clarkbworse than heartbleed17:56
fungiclearly. lock up your children and datas17:57
* fungi prepares the bunker17:57
clarkbjeblair: the dev/ vs /dev swaps seem to have flopped in the zuul and gerrit changes17:57
clarkbjeblair: so it isn't clear to me which one we actually want17:57
sdagueshellshock only impacts if you run bash cgi, right?17:57
jeblairoops17:57
*** lttrl has joined #openstack-infra17:57
clarkbsdague: sort of. it affects anything that has access to bash. so if you run a terminal box and someone ssh's to it they can pwn themselves17:58
sdagueok17:58
clarkbsdague: and python's popen apparently is vulnerable if you cgi that and subprocess to bash17:58
jogofungi: clarkb like gerrit?17:58
fungisdague: under fairly limited conditions. depends on how your cgi is invoked, what it does with its environment, what interpreter it's using, how it invokes subprocesses, what your default shell is in those circumstances...17:58
jeblairand ssh with forced commands17:58
*** zns has quit IRC17:59
*** ildikov has quit IRC17:59
fungiright. the forced ssh commands situation is actually a lot more insidious, but it's also something very few sites use17:59
clarkbjogo: gerrit runs a java ssh thing without letting you exec bash so gerrit is fine17:59
sdagueI was kind of annoyed by the "all the IoT devices are broken with this" ... because mostly those run busybox, so no17:59
jogoclarkb: java FTW17:59
jeblairfungi: right, just the ones really concerned about security :)17:59
fungijeblair: yep! ;)18:00
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch gerrit to project-config  https://review.openstack.org/12289418:00
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Doc updates for project-config  https://review.openstack.org/12290818:00
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch jenkins to project-config  https://review.openstack.org/12289318:00
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch specs to project-config  https://review.openstack.org/12290418:00
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch gerritbot to project-config  https://review.openstack.org/12290518:00
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch nodepool to project-config  https://review.openstack.org/12290218:00
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch accessbot to project-config  https://review.openstack.org/12290318:00
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch zuul and zuul-dev to project-config  https://review.openstack.org/12253318:00
JayFfungi: yeah, the bypassing of command= in authorized_keys is one of the nastiest expressions of that bug18:00
jeblairswappeded zuul's dev path again ^18:00
*** dustins has quit IRC18:00
*** markmcclain has quit IRC18:01
*** armax has quit IRC18:01
* clarkb rereviews18:01
fungiJayF: agreed. though generally, if you've done that you know you've done it, and can reason about the exposure risk. as opposed to the cgi-related means of exploiting it which tend to mean you get to dig around inside nebulous web app code you didn't write18:02
*** Longgeek_ has quit IRC18:02
mordredfungi: you should probably just stop running thigns as CGI ...18:03
mordred:)18:03
*** annegent_ has quit IRC18:03
clarkbmordred: you know what runs as cgi? mailman and our wiki18:03
clarkb:/18:03
JayFCGI is easy to exploit in this case; bash gets run everytime Apache tries to call a CGI18:03
*** annegent_ has joined #openstack-infra18:03
JayFyou don't even need vulnerable code in the CGI being run, just the fact you're running a cgi causes you to be vuln18:04
JayFaiui18:04
mordredclarkb: sigh18:04
clarkbJayF: I don't think that is true18:04
fungiJayF: that was not at all my takeaway from the thread on the oss-security ml18:04
JayFHmm.18:04
clarkbthe big issues were if you cgi scripts were bash or subshell to bash then you have problems18:04
clarkbmordred: its ok I used the power of ansible to update bash everywhere18:05
mordredwouldn't an exploit during CGI only get you the lower-priv apache user too?18:05
anteayasubmitting a new patch to https://review.openstack.org/#/c/123921/ requires a rebase, I am happier to checkout a new config, run my new-improved delete script and use the same change id18:05
mordredclarkb: NICE18:05
anteayaany objections?18:05
mordredclarkb: did it make you happy?18:05
*** dannywilson has quit IRC18:05
fungiJayF: apache doesn't invoke a shell unless you happen to have written your cgi *in* shell script. but your cgi may make system exec calls to run other executables and if it preserves the calling environment and if it's defaulting to bash as that user's shell (or something symlinked to bash) then your goose is cooked18:05
clarkbmordred: ansible '*' -m apt -a 'name=bash update_cache=yes state=latest' -f1018:05
JayFfungi: clarkb: gimme one sec to get the log from a buddy, I might have misinterpreted this...18:05
clarkbmordred: it still a bit clunky as far as figuring out how ansible wants me to do the things, but its amazing that I can do them18:05
*** dannywilson has joined #openstack-infra18:06
JayFfungi: clarkb: https://gist.github.com/aweeks/ab1926ca4c4272374b4f although looking at this I don't think he was successfully exploited, it was just an attempt18:06
AJaegeranteaya: go ahead18:06
anteayaAJaeger: thanks18:06
anteayaAJaeger: and thanks for the awesome review!!18:06
anteayaI can always count on you18:06
*** datsun180b has quit IRC18:06
fungiJayF: but of course if you get your security analysis from new stories, then the sky is falling. doomsday. armageddon. apocalypse. cats and dogs living together...18:06
fungis/new/news/18:07
clarkbevery news story I said made sure i knew this was worse than heartbleed18:07
anteayado you know that now?18:07
JayFfungi: I tend to err on the side of thinking that more things are vuln than they are, keeps me on my toes and adequately patched :P18:07
anteayawere the news people successful?18:07
JayFanteaya: yes18:07
fungiJayF: yes, i think we all err on the side of just patching anyway18:07
JayFanteaya: my English-teacher wife asked me last night if I had patched my "bash" on servers at work18:08
anteayaawesome18:08
anteayaglad someone is doing their job18:08
anteayawell at least they are learning the buzz words18:08
anteayamy mom figured out how to stop brute force password hacks18:08
anteayawhen I was telling her the iCloud thing18:08
fungilrwxrwxrwx 1 root root 4 Jan 10  2014 /bin/sh -> dash18:08
anteayashe said why don't they just stop them after 3 wrong password attempts18:09
anteayaway to go mom!18:09
*** dustins has joined #openstack-infra18:09
*** pcm_ has quit IRC18:09
AJaegeranteaya: that was pure luck - I wanted to figure out quickly were layout.yaml was and then was confused ;)18:09
*** dannywilson has quit IRC18:10
* AJaeger didn't check each and every file18:10
anteayaI'll take luck18:10
fungibetween my debian servers and my openbsd servers, i wasn't extremely worried about my personal systems... sane operating systems don't use a full-featured shell for non-interactive default environments18:10
anteayawhen I points out my errors18:10
*** pcm_ has joined #openstack-infra18:10
AJaeger;)18:10
*** _nadya_ has joined #openstack-infra18:10
clarkbjeblair: stack lgtm18:11
*** arosen1 has joined #openstack-infra18:11
*** datsun180b has joined #openstack-infra18:11
clarkbI need to afk now to find early lunch and my train. back as soon as I find a seat18:11
arosen1What do you guys thinking about adding support for nodepool and zuul to devstack?18:11
fungiclarkb: jeblair: agreed. i'm +2 on all of them now18:12
fungiarosen1: zuulstack!18:12
arosen1I think it would make it easier for people if they could deploy all their CI from that since it's a pretty well known tool18:12
jeblairfungi: do you want to go with manual apply / noop, or just do it?18:12
arosen1fungi: :)18:12
openstackgerritAnita Kuno proposed a change to openstack-infra/config: Removes content split into project-config  https://review.openstack.org/12392118:12
jeblairarosen1: so we're at the beginning of a very long process to make all of this more reusable18:13
anteayayay that worked18:13
anteayaAJaeger: have at it18:13
jeblairarosen1: if we start encoding the current practice into devstack, i think it will conflict with where we are going, and moreover, these are all production systems, which we won't be mananging with devstack18:13
fungijeblair: i think probably manual, to give at least one of us a chance to see what puppet actually wants to do. shall disable remove the cron entry on the master?18:14
jeblairarosen1: we do manage them with puppet, so that's why we're working on making that more useful for downstreams18:14
jeblairarosen1: the project-config move we are doing is part of this process18:14
fungier, shall i18:14
* anteaya returns to reviewing the jeblair stack18:14
*** Daviey has joined #openstack-infra18:15
anteayaAJaeger: do join in, lots to review: https://review.openstack.org/#/c/122532/618:15
arosen1jeblair:  yea i understand that running a production system from devstack isn't the best idea but i think it will at least allow people to see what they need to do to easily configure it.18:15
arosen1also will allow developers to contribe to nodepool more easily18:15
fungii think for testing out nodepool it might be interesting to install it on a devstack machine you can point it at, if you don't already have an openstack cloud to point it at instead, but meh18:16
dtroyerso -1 on zuulstack.sh?  ;)18:16
jeblairyeah, we have so little manpower working on this project as it is, i don't think it's helpful to start fragmenting the way we install it and set it up18:17
*** emagana has joined #openstack-infra18:17
jeblairwe can make a pushbutton deployment, and we can do it with the actual tools we use to deploy in production18:17
*** ramashri has joined #openstack-infra18:17
jeblairand then we won't have two teams of people playing catchup as it all changes18:17
*** esker has joined #openstack-infra18:18
jeblairfungi: yes, please disable the cron on master18:19
fungii have, though the run from 18:15 hasn't wrapped up yet18:20
*** marcoemorais has quit IRC18:20
*** marcoemorais has joined #openstack-infra18:20
jeblairk, i'm working on writing next steps18:20
*** marcoemorais has quit IRC18:20
AJaegeranteaya: I didn't review all that puppet stuff, just found some wrong docu in 122908 ;)18:20
*** rushiagr is now known as rushiagr_away18:20
*** marcoemorais has joined #openstack-infra18:21
arosen1fungi:  I was asking you about this yesterday and tried a bunch of random things last night and haven't gotten a working solution :(18:21
fungigiven stackalytics says AJaeger does more reviews than anyone else in all of openstack, we can probably cut him a little slack ;)18:21
jeblairarosen1: what's not working?18:22
arosen1fungi:  I'm working on this stackforge project call congress. I have tempest tests in this project that i copy over to tempest. Though the problem is tempest doesn't have the congressclient in it's requirement.txt;  Heres my jenkins job config for it: http://paste.openstack.org/show/115376/18:22
anteayaAJaeger: good work18:22
fungiarosen1: i have promptly forgotten most of last night, even without the assistance of alcohol. what were you asking me about18:22
fungi?18:22
fungiarosen1: ahh, yes, that18:22
anteayafungi: :D18:22
jeblairarosen1: i think we may have been talking about _very_ different things.  :)18:22
arosen1:)18:22
fungiarosen1: so if memory serves, you need congress installed from git into the devstack-gate workspace?18:23
fungier, python-congressclient i guess18:23
arosen1so tempest runs from devstackgate using: tox which creates a venv. If i put python-congressclient in requirements.txt i tries to download it from pip i think?18:24
arosen1fungi: nah different issue we got that one fixed :)18:24
fungiarosen1: so, yes, if you have it in requirements.txt and it's not already installed, it will try to install from pypi18:24
mikaany chance that we'll might get a new jenkins-job-builder release? :)18:24
*** dprince has quit IRC18:24
mtreinisharosen1: the tempest tox.ini has site-packages=True, so it'll use the git versions if they're installed18:24
fungiarosen1: one solution is to install it from source before the requirements.txt gets parsed18:25
fungiwhat mtreinish just said18:25
arosen1fungi: it is installed on the system though. I can import the congressclient from the cli18:25
fungiarosen1: what does the requirements entry look like?18:25
* AJaeger needs some slack ;)18:26
anteayamika: I think zaro and mgagne are the folks you want to ask on that one18:26
mikaanteaya: thanks for the pointer18:26
anteayamika: np18:26
fungiarosen1: is it unversioned?18:26
mikazaro: mgagne: any chance that we might get a new jenkins-job-builder release? it would be great if a recent version could make it into Debian/jessie18:26
mtreinisharosen1: oh, is the issue that congressclient isn't in the tempest requirements file so it's not getting put in the venv?18:26
fungiarosen1: oh, also it does look like you're doing pip install -U (upgrade) which will try to find and install the newest version of all requirements even if you already have them installed18:27
fungii'm guessing the full-serial section in tempest's tox.ini specifies to use -U18:28
arosen1mtreinish:  right but i put it in there as part of my jenkins job18:29
arosen1fungi:  right there are no git tags and it's not on pypi18:29
mtreinishfungi: yeah it's set on the testenv section: http://git.openstack.org/cgit/openstack/tempest/tree/tox.ini#n1018:29
arosen1fungi:  i'm just running the command that devstack-gate runs to run tempest.18:29
fungiarosen1: unrelated note, you may want to go ahead and register it on pypi just so someone else doesn't decide to squat it and force you to rename18:30
arosen1fungi:  k, will do that.18:30
anteayaAJaeger: if you offer a patch with those doc fixes I will review18:31
jeblairfungi: does the etherpad look correct for those puppet commands?18:31
jeblairfungi: (i'm in the process of testing them)18:31
fungijeblair: run_all.sh is done btw and commented out18:31
*** markmcclain has joined #openstack-infra18:31
AJaegeranteaya: yeah let me do those so that jeblair's stack can sail in easily...18:32
anteayathanks18:32
*** ramashri has quit IRC18:32
*** pcrews has quit IRC18:32
mtreinisharosen1: I'm curious why you're copying things into tempest. Instead of just importing the bits you need from tempest18:32
fungijeblair: yeah, we haven't tweaked install_modules.sh so what you have there is i think all we'll need to test18:32
*** john-davidge has quit IRC18:32
*** HeOS has joined #openstack-infra18:33
arosen1mtreinish:  I do import things from tempest in my code. But this code needs to get copied into tempest to run with tempest. Checkout: https://review.openstack.org/#/c/123839/18:33
*** jergerber has joined #openstack-infra18:34
*** lcheng has quit IRC18:34
*** ramashri has joined #openstack-infra18:35
*** sballe_ has quit IRC18:36
arosen1fungi:  so is the issue that site_package is not set to true in http://git.openstack.org/cgit/openstack/tempest/tree/tox.ini#n4318:36
mtreinisharosen1: why? you just need to call the test runner on that file (and specify a location for tempest config file with the env variables) and it should work just the same if you run it in that repo vs in tempest's18:36
mtreinishwell assuming you adjust the import path for the policy client18:36
AJaegeranteaya: "Job config-compare.xml not defined" in http://logs.openstack.org/03/124103/3/check/gate-config-layout/f831512/console.html18:37
jeblairfungi: okay, the apply command works18:37
*** sballe_ has joined #openstack-infra18:37
jeblairfungi: so i think we merge the stack now18:37
arosen1mtreinish: right i agree. It doesn't matter if i copy it into the tempest project or not if i do that. Though I'll still have the same problem with the python-congressclient18:37
fungijeblair: agreed18:38
mtreinisharosen1: well depending on how you run it. The issue is the venv right? why not just setup a post-test hook that directly calls the test runner for the congress tests without using a venv18:39
mtreinishI think barbican has an example of how to do that18:39
jeblairfungi: done18:39
jeblairwell, approved anyway18:39
anteayaAJaeger: yeah fixed in the patch that is the child of the failing patch18:39
zaromika: yes, when is the deadline?18:39
*** bswartz has quit IRC18:40
*** jcoufal has quit IRC18:40
*** tonytan4ever has joined #openstack-infra18:41
arosen1mtreinish: hrm, i kinda wanted to leverage all the goodness of devstack-gate. I think the problem with this is if i want to run additional tests from tempest too things will get more complicated as there'll be 2 results.html.gz files18:41
arosen1or i guess i can just have my own tempest tests output to a different file.18:42
mtreinisharosen1: oh, I assumed this was going to be separate from a full tempest run18:42
mikazaro: beginning of november is freeze, until then the package must be in Debian/testing, so ASAP would be good ;)18:42
arosen1mtreinish: I'd prefer to combine i think this will let me make things more flexable if i want.18:43
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Improve jeepyb.rst content  https://review.openstack.org/12412718:43
AJaegeranteaya: here's the patch you wanted to review ;)18:44
openstackgerritJames E. Blair proposed a change to openstack-infra/project-config: Make gate-config-bashate nonvoting  https://review.openstack.org/12412818:44
* anteaya reviews18:44
jeblairwe will need to force-merge https://review.openstack.org/124128 and maybe force-apply it too18:45
arosen1fungi:  actually this is what is running: http://git.openstack.org/cgit/openstack/tempest/tree/tox.ini#n30 tox -eall ; which has sitepackages=True18:45
fungijeblair: yep. i'm happy to take care of that18:45
anteayaAJaeger: thanks18:45
AJaegeranteaya: thansk for the review!18:46
jeblairfungi: cool, why don't you go ahead and do that then; that way it will be in the repo when the config changes merge to start using it18:46
*** emagana has quit IRC18:47
openstackgerritA change was merged to openstack-infra/project-config: Make gate-config-bashate nonvoting  https://review.openstack.org/12412818:47
*** habib has joined #openstack-infra18:47
*** emagana has joined #openstack-infra18:47
*** arnaud has joined #openstack-infra18:47
lifelesssdague: dhellmann any thoughts on my namespace proposal?18:48
dhellmannlifeless: is that on the ML? I haven't looked today18:48
fungidoes zuul need sighup to reread its config, or does it just take a moment to notice when the layout file is modified?18:48
lifelessdhellmann: yes couple days ago18:48
*** mikedillion has joined #openstack-infra18:48
*** esker has quit IRC18:49
*** dustins has quit IRC18:49
*** asettle has joined #openstack-infra18:50
notmynameanteaya: done18:50
dhellmannlifeless: let's talk in openstack-oslo, the infra team is busy in here today18:51
*** dannywilson has joined #openstack-infra18:51
*** otherwiseguy has quit IRC18:51
*** emagana has quit IRC18:52
*** cody-somerville has quit IRC18:52
notmynameanteaya: http://lists.openstack.org/pipermail/openstack-dev/2014-September/047180.html18:52
anteayanotmyname: thank you18:52
openstackgerritA change was merged to openstack-infra/config: Move zuul layout into /etc/zuul/layout  https://review.openstack.org/12253218:53
anteayaawesome, tristanC will confirm you when he has a moment18:53
anteayanotmyname: :D18:53
anteayaglad swift is not orphaned18:53
fungiaha, http://ci.openstack.org/zuul/zuul.html#starting-zuul says yes, sighup18:53
notmynameanteaya: "I'm running for Swift PTL. You may know me from my previous emails: 'I'm running for Swift PTL', 'I'm running for Swift PTL', 'I'm running for Swift PTL', and 'I'm running for Swift PTL'"18:54
anteayanotmyname: ha h aha, I sense a theme18:54
anteayaconsistency is important18:54
*** cody-somerville has joined #openstack-infra18:54
*** cody-somerville has quit IRC18:54
*** cody-somerville has joined #openstack-infra18:54
notmynamemaybe next time elections come around, we can use metacloud's card deck. "You may know me as the 10 of spades..."18:55
fungiokay, zuul's config is reloaded with 122532 manually applied18:55
morganfainbergnotmyname, LOL18:55
funginotmyname: useful as a future conference speaker headshot18:56
jeblairfungi: oh, er, what?18:56
*** dannywilson has quit IRC18:56
*** annegent_ has quit IRC18:56
anteayanotmyname: ha ha ha18:56
clarkbnotmyname: :P18:57
anteayaI don't always self-nominate for swift ptl, when when I do...18:57
jeblairfungi: i'm not sure what you're working on or how it fits into the sequence18:57
fungijeblair: was asking how to notify zuul the layout configuration changed since puppet was not the one doing it18:57
*** dustins has joined #openstack-infra18:57
jeblairfungi: yeah, i'm just wondering why puppet is not the one doing it18:57
clarkbjeblair: are the files different?18:58
nibalizeris the notify passed up from the vcsrepo?18:58
clarkbpuppet will md5sum them and won't copy if they match18:58
jeblairbasically, there's a process in https://etherpad.openstack.org/p/HTlrATUqXy18:58
jeblairit starts on line 1018:58
fungijeblair: oh, when you said manually apply you meant through puppet, not hand-patched and reloaded. sorry18:58
finchclarkb: https://github.com/puppetlabs/puppet/blob/master/ext/rack/example-passenger-vhost.conf#L12 - our default vhost config doesn't set up a max requests per ruby instance - you might want to set that to some low value and then tune up18:58
openstackgerritA change was merged to openstack/requirements: Update xstatic-jquery-ui for Horizon  https://review.openstack.org/11318418:58
jeblairi thought we were wating on line 11 to complete18:58
clarkbfinch: is that a bug in the package?18:59
clarkbbut that seems like a reasonable thing to do18:59
*** andreaf has quit IRC18:59
finchclarkb: it's a tuning parameter, it should probably be set by default but the specific value will depend on the site18:59
openstackgerritKhai Do proposed a change to stackforge/python-jenkins: Add a get_plugin_info method  https://review.openstack.org/12393218:59
fungijeblair: i thought you meant 122532 needed to go in before we approved those changes18:59
*** andreaf has joined #openstack-infra19:00
clarkbfinch: sure, I just expect defaults to work for small deployments :)19:00
fungior before they merged at any rate19:00
clarkbotherwise they are poor defaults19:00
fungibut thinking about it now, it doesn't seem like it needed to19:00
*** ociuhandu has quit IRC19:00
fungiit just wouldn't have been able to merge on its own19:00
arosen1dtroyer: have you seen this before: http://codepad.org/Kg89MPvN it's weird if i do: sudo pip install e . (in openstackclient/congressclient) everything works. But if i open a new terminal i get this same error.19:00
finchclarkb: ENOTENOUGHCAFFEINETODISCUSSMERITSBUTYOUHAVEAFAIRPOINT19:00
fungisince the other changes are to the openstack-infra/config repo not openstack-infra/project-config19:00
clarkbfinch: clearly you should drink more19:01
*** marcoemorais has quit IRC19:01
jeblairyeah, that's all i meant.  i did want it to be in the project-config repo before we ran puppet for real on zuul so that it would be applied, but that's about it19:01
jeblairanyway, we should be fine19:01
fungiokay, well no harm done19:01
*** marcoemorais has joined #openstack-infra19:01
fungiyep19:01
jeblairand, incidentally, we can recheck 124103 again now :)19:01
fungihappy accident19:01
finchclarkb: blrrrrfgh I just got back from puppetconf and don't want to do anything in particular19:01
openstackgerritA change was merged to openstack-infra/config: Switch zuul and zuul-dev to project-config  https://review.openstack.org/12253319:02
jeblairanteaya: 124103 has an error19:02
*** marcoemorais has quit IRC19:02
jeblairi need to add a 'favorites' screen to gertty19:02
*** marcoemorais has joined #openstack-infra19:03
*** SumitNaiksatam has quit IRC19:03
* anteaya looks at 12410319:03
zaromika: ok.  will do next week.19:03
fungiahh, yep, config-compare.xml19:03
*** marcoemorais has quit IRC19:04
*** marcoemorais has joined #openstack-infra19:04
zaroclarkb: we all go on this one? https://review.openstack.org/#/c/11634719:04
*** marcoemorais has quit IRC19:04
arosen1dtroyer:  odd; it seems like sudo pip install -e . ; unsets this variable which makes things work;19:04
arosen1unset os_PACKAGE19:04
arosen1i put that in my .profile and it lets me avoid it.19:05
fungijeblair: that's i think what the cryptic comment in the etherpad meant about the fix for it being in the child change 12411619:05
mikazaro: very much appreciated, thanks! \o/19:05
*** marcoemorais has joined #openstack-infra19:05
finchclarkb: anywho. try turning that on and see if something neat happens. if that solves your issue then we can do a PR to fix that in upstream.19:05
anteayajeblair: yes that is config-compare.xml not defined, which is defined in the child patch19:05
clarkbfinch: kk19:05
fungianteaya: jeblair: so maybe the order of those changes needs to be switched19:05
anteayaI can do that19:05
clarkbfungi: jeblair: I know you are busy. any opposition to me manually uncommenting # PassengerMaxRequests 1000 in the puppetmaster vhost?19:05
clarkbsee if that curbs the leaking19:06
jeblairclarkb: wfm19:06
fungiclarkb: please do19:06
fungiclarkb: i had to reboot it this morning... it had oom'd and stopped all puppetmastering yesterday evening19:06
fungiclarkb: thus about half of the config changes we merged yesterday had yet to be applied when i woke up19:07
jeblairanteaya, fungi: so project-config doesn't exist until the backport patch19:07
openstackgerritA change was merged to openstack-infra/config: Switch jenkins to project-config  https://review.openstack.org/12289319:07
clarkbI am going to restart apache for good measure (I hope a reload is sufficient bbut I never know)19:07
openstackgerritA change was merged to openstack-infra/config: Switch gerrit to project-config  https://review.openstack.org/12289419:07
jeblairanteaya: the error is in the backport patch, which adds config-compare.xml19:07
clarkball done19:07
openstackgerritA change was merged to openstack-infra/config: Switch nodepool to project-config  https://review.openstack.org/12290219:08
*** dannywilson has joined #openstack-infra19:08
openstackgerritA change was merged to openstack-infra/config: Switch accessbot to project-config  https://review.openstack.org/12290319:08
jeblairanteaya, fungi: i think the etherpad comment is wrong; the backport patch should correct the job, and the rename patch should still be based on it19:08
fungiahh, yep19:08
fungithat'd do it19:09
anteayaokay I will do that19:09
*** pcrews has joined #openstack-infra19:12
anteayaI'm staring at it and can't see the typo19:12
anteayaI'm an idiot19:12
lifelessmordred: ping - pbr semver versioning need your ack19:12
mordredlifeless: will look later - I'm very poorly attempting to keep up with the project-config split today19:14
fungianteaya: compare-xml not compare,xml19:14
fungii can see where finger memory would make -xml harder to type than .xml19:15
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Tell project-config about itself  https://review.openstack.org/12410319:15
*** Azher has quit IRC19:16
anteayafungi: thanks yeah I had to look at the infra.yaml jobs to see that19:16
anteayaI could have addressed AJaeger comment about upstream at teh same time and I didn't19:16
anteayasorry AJaeger19:16
fungianteaya: though you'll also need to update 12411619:16
*** annegent_ has joined #openstack-infra19:17
anteayayes and rebase19:17
*** amcrn has joined #openstack-infra19:17
*** emagana has joined #openstack-infra19:18
*** melwitt has joined #openstack-infra19:18
Sinclerhi. I'm having a problem and i wish if any one can help me, when i upgrade to gerrit 2.8.4 jenkins stop working, if i push a commit jenkins don evaluate it, i have the change in queue on zuul waiting for the jenkins vote, do you had this problem when you made this upgrade?, how do you solve it?19:18
*** dane_leblanc has quit IRC19:18
zaroSincler: what version are you upgrading from?19:20
*** Sukhdev has quit IRC19:20
Sinclerzaro: from 2.4.419:20
*** jbjohnso__ has quit IRC19:20
*** esker has joined #openstack-infra19:20
anteayafungi: if I want to rebase the second patch in a series on the first19:21
anteayafungi: git rebase -i sha-of-parent-patch?19:21
zaroSincler: what version of zuul? and gearman?19:21
AJaegeranteaya: no problem19:23
*** emagana has quit IRC19:23
anteayayep19:23
anteayaAJaeger: :D19:23
openstackgerritA change was merged to openstack-infra/config: Switch specs to project-config  https://review.openstack.org/12290419:23
*** Sukhdev has joined #openstack-infra19:23
Sinclerzaro: Zuul version: 2.0.0.250 and checking gearman19:24
*** asettle has quit IRC19:24
zaroSincler: zuul listens to gerrit events to do it's thing, so make sure that gerrit is still firing events for you. https://gerrit-review.googlesource.com/Documentation/cmd-stream-events.html19:24
SergeyLukjanovanteaya, done re ptl candidacy email19:25
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Update jenkins job names that test project-config  https://review.openstack.org/12411619:25
anteayaSergeyLukjanov: thank you19:25
anteayaboo, I rebased by gerrit still says it is outdated19:26
anteayaI had done git review -d 124116 in a new branch and rebased19:26
fungianteaya: yeah, without knowing the current state of your working dir, hard to say19:27
anteayaI had tried to cherry pick 124103 then 124116 in a new branch but I can't cherry pick 124116 it erros19:27
anteayaI deleted project-config and started with a fresh repo19:27
anteayas/erros/errors19:27
fungianteaya: right, there will be a merge conflict because you're changing a line in 124116 which is changed in your updated patchset for 12410319:28
anteayaokay19:28
Sinclerzaro:  gearman-plugin': version => '0.0.3' and if i run ssh -i /home/gerrit2/.ssh/gerrit2 -p 29418 gerrit2@localhost gerrit stream-events  there is no answer19:28
fungiwhen it complains that there are conflicts, edit layout.yaml to resolve those conflicts, git add that file, then git cherry-pick --continue19:28
anteayaah19:29
anteayalet me try again19:29
*** hashar has joined #openstack-infra19:32
zaroSincler: you won't get a response until there is an event in gerrit.  try pushing a change or updating an existing change.19:32
anteayafungi: do I edit layout.yaml to match the parent or the child?19:32
fungianteaya: you edit layout.yaml to match what you want it to look like19:33
anteayaso the child19:33
*** tsg has joined #openstack-infra19:33
fungithe conflict section there shows you what the parent commit and the attempted update look like, and you edit them for what you want to appear in the new patch you're trying to apply19:34
anteayathanks19:34
*** thedodd has joined #openstack-infra19:34
AJaegeranteaya: https://review.openstack.org/#/c/124116/ contains now an outdated dependency, something went wrong...19:35
openstackgerritA change was merged to openstack-infra/config: Switch gerritbot to project-config  https://review.openstack.org/12290519:36
fungiAJaeger: she didn't edit them at the same time, so she's in the process of updating that one now to be based on the other19:36
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Tell project-config about itself  https://review.openstack.org/12410319:36
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Update jenkins job names that test project-config  https://review.openstack.org/12411619:36
openstackgerritA change was merged to openstack-infra/config: Doc updates for project-config  https://review.openstack.org/12290819:36
anteayafungi: I understand now19:36
anteayaI'm used to doing the edit after the cherry pick but with conflicts I do the edit in the middle of the cherry pick19:37
clarkbso is glance the last project without a ptl candidate?19:38
fungianteaya: eventually you may want to get comfortable with using git rebase -i to edit changes in the middle of a series19:38
fungianteaya: once your familiarity with it increases, it makes that particular problem much easier to deal with19:38
clarkb++19:38
fungiclarkb: apparently19:38
anteayaclarkb: yes19:38
*** marcoemorais1 has joined #openstack-infra19:39
AJaegerfungi: ah - so I was too fast ;)19:39
anteayathough two different people have contacted me about being a last resort19:39
AJaegerclarkb: do you want to step up?19:39
anteayathey are chatting to decide between them who will post19:39
*** annegent_ has quit IRC19:39
anteayafungi: okay thank, I'm used to git rebase -i master19:39
fungianteaya: good. saves me and jogo embarrassing ourselves ;)19:39
clarkbAJaeger: fungi's platform sounded good to me :)19:39
anteayaand git rebase -i sha just worked19:40
AJaegerclarkb: can we propose him? ;)19:40
anteayafungi: well you anyway19:40
anteaya:D19:40
*** bswartz has joined #openstack-infra19:40
*** marcoemorais has quit IRC19:41
anteayaAJaeger: twist arms all you like but candidates have to self nominate19:41
arosen1fungi:  doh! I think i figured out why i'm getting this... I had: echo "enable_service congress" >> localrc ; so that gets put at the top of the localrc file. Though later in that file it does ENABLED_SERVICES=etc ; which over rides my: enable_service congress X.x19:41
*** JayJ_ has quit IRC19:41
anteayaclarkb: well SlickNik still hasn't posted but he said he will19:41
*** JayJ_ has joined #openstack-infra19:41
jeblairfungi, clarkb: i believe our changes have merged19:43
jeblairfungi: do you want to update the production config repo on puppetmaster?19:43
jeblairi'll get ready to test out the procedure on zuul-dev19:43
fungijeblair: yep, looks like it. doing that now19:43
*** _nadya_ has quit IRC19:43
jeblaironce confirmed, we can split up the rest of the hosts19:43
Sinclerzaro: i restored a change, and stream-events return this {"type":"change-restored","change":{"project":"tutorials","branch":"master","id":"Ica5a096b3d74a034fefafa38c7de7942c0018886","number":"1","subject":"Omar Test","owner":{"name":"Demo User","email":"demouser@localhost","username":"demouser"}......19:45
*** ramashri has quit IRC19:46
jeblairfungi: i think you can skip the install_modules.sh step19:46
fungijeblair: should be all set to try on zuul-dev now19:46
jeblairok19:46
mordredanteaya: oh, I was going to put in for glance as a last resort too :)19:47
anteayathe turkeys are still on my lawn, they usually don't stay this long19:47
anteayamordred: great, so three now19:47
mordredanteaya: maybe all three of us should submit and see how the voting goes :)19:47
anteaya:D19:47
fungianteaya: yeah, my folks have had a bunch this year. they installed a turkey feeder (elevated where the bears don't get to it) to keep them around19:48
jeblairmordred: istr you have strong feelings about glanceclient options19:48
anteayamordred: would be fun voting19:48
fungimordred: i for one welcome our rewritten glanceclient masters19:48
anteayafungi: nice19:48
mordredthat's my platform, btw19:49
anteayaapparently I have enough good eatn' on my lawn to keep them amused19:49
fungimordred: someone needs to run with a platform of dissolving glance and merging it back into nova19:49
*** dane_leblanc has joined #openstack-infra19:49
*** hashar has quit IRC19:49
mordredwell, that won't be me - I think it's fine as a thing19:49
mordredand don't think we should have a more complex nova ata ll19:49
fungi"Subject: well, i guess this multi-year experiment failed..."19:49
mordred:)19:49
anteayajust a thing with a better client19:49
*** gyee has quit IRC19:50
anteayathere are some good devs there19:50
anteayaI think the just need a flag in the ground to rally around19:50
*** morgan_remote_ has quit IRC19:50
anteayayou are good at sticking flags in the ground mordred19:51
jeblairfungi: we're not going to get detailed info for zuul without manually applying the change to move the layout dir (since it's an absent->present change as opposed to diff)19:51
jeblairfungi: however, you have, with great prescience, done that on prod ;)19:51
jeblairfungi: so i'm going to say the process looks like it works, run this for real on zuul dev, then move on to zuul19:51
fungijeblair: oh, i didn't move the file, i merely edited it to set that one job non-voting19:51
jeblairfungi: oh, i confused that with another change19:52
jeblairwell, do you think it's worth applying manually to get the detailed diff?19:52
fungibut yeah, i suspected the file moves were going to result in a fairly opaque --noop result unfortunately19:52
jeblairi kind of think we can just let it slide on zuul19:52
fungii agree. i was more worried about gerrit19:52
jeblairi think it's either going to work or not and we'll know pretty quick19:52
*** esker has quit IRC19:52
jeblairwe can save the queue before doing it just in case19:53
fungiand about manage-projects running roughshod over stuff19:53
*** juzuluag has quit IRC19:53
jeblairk, lemme run this for real on zuul-dev and see if there are any issues19:53
fungimordred: the joke was going to be that if nobody stepped up to lead the project, then someone should step up to dissolve it instead ;_19:53
*** unicell1 has quit IRC19:53
jeblairzuul-dev looks good; content looks correct and it ran the reload trigger19:54
jeblairi'll do zuul now19:55
jeblairfungi: you want to start on the jenkinses?19:55
*** emagana has joined #openstack-infra19:55
fungiyep, hitting jenkins.o.o first since it tends to not do things very often19:55
*** hashar has joined #openstack-infra19:56
jeblairfungi: er, did i completely miss the git backends?19:57
fungijeblair: oh, you know what? they use projects.yaml to create cgit repos don't they?19:58
jeblairi think they do19:58
*** dimtruck is now known as zz_dimtruck19:58
jeblairi'm going to finish up zuul, then i'll go write that change while you continue with jenkins19:58
*** datsun180b has quit IRC19:58
*** marcoemorais1 has quit IRC19:59
fungijeblair: as for jenkins, --noop wants to remove project-config from /etc/jenkins_jobs/config/projects.yaml but otherwise no relevant file content changes19:59
*** marcoemorais has joined #openstack-infra19:59
* fungi goible-checks that that's right19:59
*** marcoemorais has quit IRC19:59
fungidouble-checks19:59
jeblairhrm, wouldn't hurt to get 124103 in19:59
*** _nadya_ has joined #openstack-infra19:59
*** marcoemorais has joined #openstack-infra20:00
fungigood idea, that way the job updates from this make sense20:00
openstackgerritDavid Shrewsbury proposed a change to stackforge/dox: Better support for multiple commands  https://review.openstack.org/12056220:00
jeblairmaybe let's pause until that happens20:00
*** marcoemorais has quit IRC20:00
jeblairi'll go write the git change20:00
fungishall i approve 124103 or did someone else want to review that most recent patchset too?20:00
jeblairi just review/aprvd20:01
fungik20:01
*** marcoemorais has joined #openstack-infra20:01
*** marcoemorais has quit IRC20:01
*** marcoemorais has joined #openstack-infra20:01
zaroSincler: might be the events have changed.  have you checked your gerrit trigger in zuul layout.yaml file to see if they still match?20:02
*** marcoemorais has quit IRC20:02
*** marcoemorais has joined #openstack-infra20:02
*** yamahata_ has quit IRC20:02
mriedemwhen is the stable/havana branch going to be deleted?20:02
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: CORS support for StoryBoard API  https://review.openstack.org/12416320:03
fungiSincler: when we upgraded from gerrit 2.4 to 2.8 we had to modify our comment trigger patterns to account for changes in the comment formatting in the gerrit event stream20:03
fungimriedem: when i or someone writes the patches. i'm trying to work on them this week, but it may end up being next week20:03
*** rfolco has quit IRC20:03
Sinclerzaro :  gerrit:20:03
Sincler        - event: patchset-created20:03
Sincler        - event: change-restored20:03
Sinclerthe name  match20:03
mriedemfungi: ok, i knew it was coming up20:04
fungimriedem: regardless, we wanted to wait and not actually delete those branches until a week had passed after the last release (which was this past tuesday)20:04
mriedemgot some scripts using stable/havana tempest and wanted to plan for when those will break :)20:04
*** ramashri has joined #openstack-infra20:04
fungimriedem: tuesday of this coming week, at the earliest20:04
*** JayJ_ has quit IRC20:05
mriedemthanks20:05
*** JayJ_ has joined #openstack-infra20:05
fungimriedem: the one week grace is to account for if the stable release manager made a mistake tagging the last version and didn't notice it right away, or if some egregious bug was introduced between the last two versions which needed a brown-bag update20:06
mriedemyup20:06
*** marcoemorais has quit IRC20:07
*** zz_dimtruck is now known as dimtruck20:07
*** ociuhandu has joined #openstack-infra20:07
*** hashar has quit IRC20:07
*** radez is now known as radez_g0n320:08
*** ociuhandu has quit IRC20:09
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch git and storyboard to project-config  https://review.openstack.org/12416620:09
jeblairfungi, clarkb, mordred, anteaya: ^20:09
*** radez_g0n3 is now known as radez20:10
Sinclerfungi: i'm testing with your layout.yaml for example http://pastebin.com/ZEKb8nYR20:10
*** atiwari has quit IRC20:10
anteayajeblair: does that patch need to define those variables to match actual paths in the project-config repo?20:11
*** esker has joined #openstack-infra20:11
anteayajeblair: I didn't see paths in that patch20:11
*** esker has quit IRC20:11
*** ociuhandu has joined #openstack-infra20:11
*** radez is now known as radez_g0n320:11
*** esker has joined #openstack-infra20:12
jeblairanteaya: it's already defined (it was defined in the gerrit change).  though the gerrit change used the wrong name (gerrit -> jeepyb)20:12
fungiyeah, looks like we missed that in the earlier review20:12
*** pcm_ has quit IRC20:13
openstackgerritJames E. Blair proposed a change to openstack-infra/project-config: Update URL for projects.yaml  https://review.openstack.org/12417320:13
fungisort of odd that the puppet apply job didn't break20:13
anteayak20:13
jeblairfungi: what puppet apply job?20:13
*** _nadya_ has quit IRC20:14
*** resker has joined #openstack-infra20:14
jeblairoh that puppet apply job :)20:14
*** resker has quit IRC20:14
zaroSincler: zuul talks to your gearman server.  so now you should check whether zuul added a job to the gearman queue.20:14
jeblairyeah, i have a suspicion something is really wrong with those since the move to puppet320:14
*** esker has quit IRC20:14
jeblairif someone has time to look into that, that would be really great20:14
fungijeblair: the ones which ran on 122894 and its successors20:14
jeblairbecause i think they are throwing _lots_ of errors and then succeeding20:15
zaroSincler: ohh actually zuul runs it's own gearman server. so what you see in the zuul queue is the gearman queue.20:15
jeblairnibalizer, jesusaurus: ^ if you have a minute20:15
*** mikedillion has quit IRC20:15
zaroSincler: your version of the gearman plugin seems pretty old.  i would upgrade that to the latest release which is 0.0.7 i believe.20:15
zaroSincler: also what version of jenkins are you running?20:16
anteayajeblair: I question the paths in 12417320:16
Sinclerzaro: Jenkins ver. 1.565.220:16
jeblairanteaya: did you visit the url?20:16
anteayaI did not20:17
jesusaurusjeblair: whats going on with the apply jobs?20:17
jeblairanteaya: i would recommend doing that, and then comparing the output to the one you suggested20:17
fungianteaya: "plain" is what cgit changes the url to instead of "tree" if you ask to see the raw file content20:18
zaroSincler: ahh i think there are issues with later version of jenkins.  we are running LTS ver 1.532.220:18
anteayaah, I'm wrong20:18
jeblairjesusaurus: we're not sure they're working since the move to puppet320:18
anteayathanks20:18
zaroSincler: can you downgrade jenkins?20:18
*** arnaud__ has joined #openstack-infra20:19
jeblairjesusaurus: http://logs.openstack.org/94/122894/10/gate/gate-infra-puppet-apply-precise/28b952f/console.html20:19
Sinclerzaro:yes i'll try upgrade gearman and downgrade jenkins20:19
jeblairjesusaurus: that change used an undefined variable20:19
jesusaurusoh interesting20:20
anteayagrey squirrel on the deck 4 metres in front of the cat, looks like she will jump through the screen20:21
*** asettle has joined #openstack-infra20:21
jeblairjesusaurus: 'gerrit_projcet_file' was used instead of 'jeepyb_project_file'.  i don't see an error in the log relating to that20:22
anteayaso we want to keep going using the jeepyp variable rather than fixing it to gerrit?20:22
*** baoli has quit IRC20:22
annegentleI once worked at a software company that supported every software release they ever had, including the one on floppy disks.20:23
*** andreaf has quit IRC20:23
anteayaannegentle: ha ha ha20:23
annegentlesquirrel!20:23
*** andreaf has joined #openstack-infra20:23
*** tsg has quit IRC20:24
*** apmelton has quit IRC20:24
*** rfolco has joined #openstack-infra20:24
fungianteaya: jeepyb_project_file is a more accurate name for the puppet manifest20:25
fungianteaya: gerrit/projects.yaml is a more user-friendly path for the project-config repo20:25
anteayaokay as long as those addressing the puppet manifest are happy20:26
*** dkliban is now known as dkliban_afk20:29
zaroSincler: https://bugs.launchpad.net/gearman-plugin/+bug/135389120:29
uvirtbotLaunchpad bug 1353891 in gearman-plugin "Jobs generated by jenkins job builder are not registered" [Undecided,New]20:29
anteayaI'm looking at the gerrit patch again: https://review.openstack.org/#/c/122894/10 and I am not seeing the jeepyb reference20:29
*** dhellmann is now known as dhellmann_20:29
*** dimtruck is now known as zz_dimtruck20:30
Sinclerzaro: ok, i'll check it and downgrade to test, thanks20:32
*** habib has quit IRC20:32
*** ramashri has quit IRC20:33
jesusaurusjeblair: in that particular case, it looks like the incorrect 'gerrit_project_file' was use on the right-hand side, so it may have silently resolved to an empty string? its concerning that theres nothing in the log about that20:33
*** pcrews has quit IRC20:34
jesusaurusjeblair: but in general there are a lot of errors in that log, but puppet agent seems to have returned 0, so we may have to use --detailed-exit-codes and manually check each run for a return of 4 or 6 (the possible returns for an error)20:34
*** david-lyle has quit IRC20:37
*** boden has quit IRC20:37
anteayafound it: https://review.openstack.org/#/c/122894/10/modules/project_config/manifests/init.pp20:38
*** boden has joined #openstack-infra20:38
jeblairjesusaurus: that all makes sense (though i thought we were using detailed exit codes in the apply job)20:38
anteaya124103 is in the gate awaiting nodes20:40
jesusaurusjeblair: no, we aren't. i don't think we can with clarkb's `xargs -P $(nprocs)` trick, we will have to run them serially to check each exit code20:40
jeblairclarkb, mordred: around to review 124166?20:40
jeblairjesusaurus: the other thing is that a lot of those errors are not really errors, they are mostly due to noop20:41
clarkbI can try currently in dead spot of train ride20:41
jeblairclarkb, fungi: we're waiting on 124166 to merge to proceed20:41
*** ZZelle_ has joined #openstack-infra20:41
*** dkehnm has joined #openstack-infra20:41
jeblair(as well as 103, but that's imminent)20:42
*** e0ne has quit IRC20:42
*** boden has quit IRC20:43
jeblairclarkb: do you suspect the logstash queue second derivative is related to rax cinder performance?20:43
fungijeblair: yep, it's currently waiting on workers in the gate pipeline20:43
clarkbjeblair: yes20:44
*** ramashri has joined #openstack-infra20:44
jeblairthe line has been decidedly discongruous with our workload recently20:44
fungiand clouds are notoriously inconsistent for performance, _especially_ where i/o is concerned20:45
*** asettle has quit IRC20:45
fungimainly because of lack of good metering/quotas/enforcement around iops20:46
*** maxbit_ has joined #openstack-infra20:46
mordredjeblair: on it20:48
mordredjeblair: oh - nevermind :)20:49
jeblairmordred: feel free, you still have time :)20:49
jeblairanteaya: are you updating 124116?20:49
mordredjeblair: it looked approved to me ...20:49
anteayajeblair: ah, I wasn't working on it, should I be?20:50
jeblairmordred: yes, but not merged20:50
jeblairanteaya: it has an error20:50
mordredjeblair: nod20:50
clarkblgtm edge service so bad20:50
anteayait failed again :(20:50
*** maxbit has quit IRC20:50
mordredjeblair: lgtm - no need to panic -220:50
*** AJaeger has quit IRC20:51
anteayawell the errors are fixed in the child patch: 2014-09-25 20:23:09.714 | Job config-compare-xml not defined20:51
anteaya2014-09-25 20:23:09.714 | Job gate-config-irc-access not defined20:51
anteayaso am I squashing the patches?20:51
jeblairanteaya: there is no child patch20:52
*** gyee has joined #openstack-infra20:52
jeblairanteaya: that's the end of the line, and the error is in that patch20:52
jeblairerrors20:52
anteayakk20:52
*** HenryG has quit IRC20:53
jeblairi'll leave a pointer20:53
fungianteaya: you've renamed those jobs in the jenkins config but missed instances of them in the zuul config20:53
anteayaah yes okay20:53
*** thedodd has quit IRC20:54
*** zz_dimtruck is now known as dimtruck20:54
*** gyee has quit IRC20:54
*** gokrokve has quit IRC20:54
*** thedodd has joined #openstack-infra20:54
jeblairanteaya: i left pointers to the things that need changing20:54
*** gokrokve has joined #openstack-infra20:54
*** otherwiseguy has joined #openstack-infra20:55
*** thedodd has quit IRC20:55
fungihah, tristanC just showed me http://istheinternetonfire.com/20:55
fungihilarious20:56
*** thedodd has joined #openstack-infra20:56
*** thedodd has quit IRC20:56
*** thedodd has joined #openstack-infra20:56
*** comstud has quit IRC20:56
*** dannywilson has quit IRC20:57
notmynameis the merge-check queue deduped?20:57
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Update jenkins job names that test project-config  https://review.openstack.org/12411620:57
jeblairnotmyname: i hope so?20:57
*** bswartz has quit IRC20:58
anteayajeblair: I'm just not sure about line 3025: https://review.openstack.org/#/c/124116/5/zuul/layout.yaml20:59
*** mfer has quit IRC20:59
jeblairnotmyname: spot checking some nova numbers, i only see them showing up once20:59
notmynamejust curious. I only noticed that it exits just now20:59
*** andreykurilin_ has joined #openstack-infra21:00
anteayawe have a glance ptl candidate!! woooo21:00
jeblairyeah, it's been kinda busy lately21:00
*** thedodd has quit IRC21:00
fungiwhich is good news--means stuff is merging!21:00
*** thedodd has joined #openstack-infra21:00
*** gyee has joined #openstack-infra21:00
jeblairanteaya: what are you unsure about?21:00
*** thedodd has quit IRC21:01
jeblairoh, erm, why wasn't that job dropped?21:01
*** comstud has joined #openstack-infra21:01
jeblairanteaya: i think you can drop it.  right fungi?21:01
anteayaso drop that job?21:01
*** tsg has joined #openstack-infra21:01
jeblairyeah.  we are not currently running it; it must have been missed in the backport21:02
anteayakk21:02
anteayadropping21:02
fungijeblair: anteaya: yeah, the data and the script exercising it have nothing to do with that repo any longer21:02
openstackgerritAnita Kuno proposed a change to openstack-infra/project-config: Update jenkins job names that test project-config  https://review.openstack.org/12411621:03
jeblairmordred: 124102 has errors21:04
*** sdake_ has joined #openstack-infra21:05
*** pcrews has joined #openstack-infra21:05
*** mrmartin has joined #openstack-infra21:05
*** thedodd has joined #openstack-infra21:06
openstackgerritJames E. Blair proposed a change to openstack-infra/project-config: Add project-config to infra integration test  https://review.openstack.org/12418121:06
*** mmedvede has quit IRC21:06
mattoliveraumorning21:06
anteayamorning mattoliverau21:06
*** dannywilson has joined #openstack-infra21:06
*** david-lyle has joined #openstack-infra21:06
*** hashar has joined #openstack-infra21:07
jeblairmattoliverau: we're working from here on the project-config split: https://etherpad.openstack.org/p/HTlrATUqXy21:07
*** dustins has quit IRC21:08
*** david-ly_ has joined #openstack-infra21:08
*** dannywilson has quit IRC21:08
*** dannywilson has joined #openstack-infra21:09
*** krotscheck has quit IRC21:09
*** aysyanne has quit IRC21:10
fungiyesterday's review frenzy put a 40% wallop on my unreviewed config changes21:10
anteayajeblair: for 124181 do we need to add anything to jenkins/jobs/projects.yaml? here is what the state soon will be: https://review.openstack.org/#/c/124103/5/jenkins/jobs/projects.yaml21:11
anteaya40% is a lot21:11
*** dannywilson has quit IRC21:11
*** krotscheck has joined #openstack-infra21:11
*** dannywilson has joined #openstack-infra21:11
*** david-lyle has quit IRC21:11
*** denis_makogon has quit IRC21:13
*** emagana has quit IRC21:13
*** dmakogon_ has joined #openstack-infra21:14
*** emagana has joined #openstack-infra21:14
mordredjeblair: gah. fixing21:14
*** ociuhandu has quit IRC21:14
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add project-config to zuul cloner  https://review.openstack.org/12410221:14
jeblairanteaya: no21:15
jeblairanteaya: they already exist21:16
*** hashar has quit IRC21:16
anteayajeblair: kk21:16
*** kgiusti_lunch has left #openstack-infra21:16
openstackgerritA change was merged to openstack-infra/project-config: Tell project-config about itself  https://review.openstack.org/12410321:18
asselinis this correct? creates a directory 'layout', or should it be 'file'? https://github.com/openstack-infra/config/blob/master/modules/zuul/manifests/server.pp#L3421:21
asselinor /etc/zuul?21:21
*** dane_leblanc has quit IRC21:21
jeblairasselin: directory now; we put both the layout.yaml and the openstack_functions.py in that directory.  we have adjusted our zuul.conf to point one level deeper for its layout21:22
fungiokay, since that finally merged i'm retesting jenkins.o.o21:22
*** doug-fish has quit IRC21:22
jeblairasselin: that was to accomodate arbitrary include files in the project-config repo21:22
asselinok21:22
fungioh, right, need to update the puppetmaster again21:22
*** doug-fish has joined #openstack-infra21:22
jeblairfungi: should be good.  we'll just want to stop before we get to gerrit if 124166 doesn't merge before then.21:22
* fungi notes this21:23
*** SumitNaiksatam has joined #openstack-infra21:23
*** david-ly_ has quit IRC21:23
*** Sukhdev has quit IRC21:23
*** tsg has quit IRC21:24
fungiaha, right, needed to rerun the module apply21:24
*** asettle has joined #openstack-infra21:24
fungiokay, no relevant content changes that time21:25
fungijeblair: should be safe to proceed with a non --noop apply right?21:25
jeblairyep21:25
fungilooks good. i'll apply on the other masters21:26
*** MaxV has joined #openstack-infra21:26
*** mrmartin has quit IRC21:29
jeblairfungi: i'm going to save the zuul queue, cross my fingers, then apply on zuul21:29
*** ramashri has quit IRC21:29
*** ramashri has joined #openstack-infra21:30
fungijeblair: good plan21:30
*** Sukhdev has joined #openstack-infra21:30
*** ramashri has quit IRC21:30
lifelessdstufft: ping21:30
lifelessactually21:30
*** MaxV has quit IRC21:31
*** ramashri has joined #openstack-infra21:31
jeblairfungi: zuul is done21:31
jeblairi'll start on git backends21:31
fungik21:31
jeblairer no i won't21:32
jeblairthat's waiting on 124166 too21:32
fungiahh, right21:32
openstackgerritMatthew Treinish proposed a change to openstack-infra/config: Stop using deprecated TESTR_FULL option  https://review.openstack.org/12418921:32
openstackgerritMatthew Treinish proposed a change to openstack-infra/devstack-gate: Remove TESTR_FULL option for running tempest  https://review.openstack.org/12419021:32
fungimtreinish: i suspect 124189 has been proposed against teh wrong project21:33
mtreinishfungi: oh, was that part of the repo split?21:33
fungimtreinish: yep, the change to delete those files from the config repo just hasn't merged yet21:33
jeblair(it's still a bit dangerous to merge it)21:34
anteayamtreinish: https://review.openstack.org/#/c/123921/21:34
fungiright21:34
mtreinishok I can push it against the other repo instead21:34
mtreinishit's not super urgent21:34
anteayamtreinish: wait a few21:34
anteayamtreinish: since project-config isn't quite yet in the shape we would like it to be21:35
anteayayou might just be creating rebases for yourself21:35
anteayamy sense is jeblair will post to the ml once we have the all clear21:35
morganfainberganteaya, what if mtreinish really likes pointless rebasing? :P ok ok /me quiets up again21:35
*** mjturek has quit IRC21:36
anteayanever let me stand in the way of someone wanting do to useless work21:36
mtreinishmorganfainberg: sometimes I do. The monotony is calming, that's why I like doing giant hacking fix patches.21:36
anteayaunless of course it beeps when it backs up21:36
anteayathen I hate you21:36
jeblairanteaya: i am, in fact, drafting a message right now21:36
anteayajeblair: cool21:36
*** mmedvede has joined #openstack-infra21:38
*** Sincler has quit IRC21:40
*** sdake_ has quit IRC21:41
*** dims has quit IRC21:41
fungiokay, jenkins masters are all done21:41
*** dims has joined #openstack-infra21:42
fungii'm being told that it's time to go eat something21:42
jeblairfungi: okay, i'll take care of the rest when 124116 merges (in 2 mins)21:42
anteayamy dinner is ready as well21:42
jeblairalong with clarkb and mordred, i hope.  :)21:42
anteaya:D21:43
anteayaback in a bit21:43
*** koolhead17 has joined #openstack-infra21:43
clarkbI am approsching olympia and should have better internets once north of there21:43
jeblairis anyone else around?  i've got some independent project-config tasks that could be done21:43
fungijeblair: i'll return as quickly as i can and help with running down whatever fresh fun we've created21:43
clarkbjeblair what should I be doing?21:43
jeblairclarkb: want to add LICENSE, README, and CONTRIBUTING files to project-config?21:44
openstackgerritA change was merged to openstack-infra/config: Switch git and storyboard to project-config  https://review.openstack.org/12416621:45
clarkbjeblair yip can do21:45
*** nikhil_k is now known as nikhil_k|afk21:46
*** dims has quit IRC21:46
*** tonytan4ever has quit IRC21:46
openstackgerritMichael Krotscheck proposed a change to openstack-infra/puppet-storyboard: Added CORS configuration to puppet-storyboard  https://review.openstack.org/12419721:49
jeblairupdated puppetmaster config repo again21:51
clarkbjeblair: apache 2 license I assume?21:53
jeblairclarkb: hrm.  i guess so.  :)21:53
anteayaback21:54
anteayaI can do a README21:54
*** packet has quit IRC21:55
clarkbanteaya: oh I was going to do it all in one change.21:55
*** eharney has quit IRC21:55
anteayaah go you21:55
anteayawhat else can I do21:55
jeblairanteaya: i pasted my draft email in the etherpad; want to review?21:55
anteayaI can do that21:55
anteayaI am thinking I should update the spec to include the 5 files from the tools directory we moved over21:57
*** signed8bit has quit IRC21:57
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Expanded length of storytag name.  https://review.openstack.org/12360721:57
anteayado you think I should?21:57
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Sanitize title during launchpad import.  https://review.openstack.org/12360621:57
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Added catch for OpenID Failures in Launchpad Import  https://review.openstack.org/12360521:57
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: CORS support for StoryBoard API  https://review.openstack.org/12416321:57
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Manage autoincrement in migration  https://review.openstack.org/12382121:57
*** markmcclain has quit IRC21:57
jeblairanteaya: either way21:57
openstackgerritClark Boylan proposed a change to openstack-infra/project-config: Add license, readme, and contributing files  https://review.openstack.org/12420021:59
clarkbok I feel like my muse has taken a day off21:59
clarkbthe contributing file is bsaically from nova21:59
*** Sincler has joined #openstack-infra21:59
clarkband readme problably needs work. anteaya maybe you want to push a new patchset with better readme and we can coauthor the change?21:59
clarkbin any case feedback is very welcome22:00
jeblairi wonder if it might be useful to itemize the directories and what they are for with links to their documentation pages on ci.o.o?22:00
clarkbjeblair: maybe a per dir readme or just at the top level?22:01
jeblair(just very briefly -- not to duplicate the docs, but just to provide an intro and context)22:01
anteayajeblair: looks good, the links all work, the docs look accurate22:01
jeblairclarkb: heh, some of the directories get copied over wholesale; including a readme in them is probably okay but requires thought and double-checking :)22:01
clarkboh right22:01
jeblairclarkb: so i was thinking just at the top level22:02
clarkbjeblair: probably better just at the global level then22:02
clarkbI can add that to the readme22:02
jeblaira link to the stackforge howto might be good too22:02
*** doug-fish has quit IRC22:02
anteayathe contributing.rst22:02
anteayaare we ready to point to the infra/manual yet on docs.o.o?22:03
*** doug-fish has joined #openstack-infra22:03
jeblairi don't think so, i'm so far behind on reviewing that i don't know the current state22:03
anteayaokay also project-config bugs, still launchpad?22:03
anteayaclarkb: what are all the \r marks in the license? https://review.openstack.org/#/c/124200/1/LICENSE22:04
jeblairanteaya: yeah, i guess link to lp for now22:04
anteayaclarkb: the readme looks fine to me22:04
jeblairstarting review-dev now22:05
anteayaI honestly expect whoever contributes to do so based upon lurking in the channel and asking us questions22:05
anteayaI don't expect anyone to contribute to project-config based on the contents of the README file22:05
jeblairi would rather they read the readme than ask the sort of questions that can be answered in the readme :)22:06
anteayaread ci.o.o is in the readme22:06
openstackgerritMichael Krotscheck proposed a change to openstack-infra/config: Added CORS allowed domains to storyboard server config.  https://review.openstack.org/12420122:06
clarkbanteaya: apparently the nova readme has \r in it22:06
clarkbor was it zuul22:07
anteayawhat does \r mean or do?22:07
anteayaI have never seen it before22:07
boris-42jeblair hi22:07
boris-42jeblair seems like we found the issue with rally-designate job https://review.openstack.org/#/c/123983/22:08
boris-42jeblair wrong indent22:08
boris-42=)22:08
*** boden has joined #openstack-infra22:08
jeblairboris-42: you'll be able to fix that tomorrow when the project-config repo is finished.  those files are about to be deleted.22:09
boris-42jeblair ah okay22:09
boris-42jeblair so it will be openstack-infra/project-config ?22:09
*** ramashri has quit IRC22:09
clarkbanteaya: it is carriage return22:09
clarkbon windows you need \r\n for newlines22:09
jeblairboris-42: yep.  it's not ready for changes yet.  i'll send an email to the list when it is.22:09
boris-42jeblair ok, thank you22:10
*** ramashri has joined #openstack-infra22:10
jeblairanteaya: oops, we forgot something in the split.  patch incoming22:10
anteayaclarkb: ah22:10
*** ramashri has quit IRC22:10
* anteaya stands by to review22:10
*** ramashri has joined #openstack-infra22:11
*** gokrokve has quit IRC22:12
*** morgan_remote_ has joined #openstack-infra22:12
*** boden has quit IRC22:13
*** JayJ_ has quit IRC22:13
*** gokrokve has joined #openstack-infra22:13
openstackgerritJames E. Blair proposed a change to openstack-infra/project-config: Move review-dev acl file  https://review.openstack.org/12420322:13
*** gondoi is now known as zz_gondoi22:13
jeblairanteaya, clarkb: ^22:13
openstackgerritA change was merged to openstack-infra/project-config: Update jenkins job names that test project-config  https://review.openstack.org/12411622:14
anteayayeah, never knew that, so wouldn't even have thought to look22:14
*** dkehnm has quit IRC22:14
*** jaypipes has quit IRC22:15
*** marcoemorais has joined #openstack-infra22:15
*** dkehnm has joined #openstack-infra22:16
* asselin updated 3rd party ci repos to latest infra changes. So far so good.22:17
*** gokrokve has quit IRC22:18
jeblairasselin: neat.  i'm hoping this will be a big help (you can make your own project-config repo and just point to it).  still a long way to go, but it's the direction we're heading.22:18
asselinjeblair, is infra's stable enough to start looking at?22:19
jeblairasselin: not quite22:19
*** asettle has quit IRC22:19
*** pradk has quit IRC22:19
asselinjeblair, ok, let me know and I'll start migrating over. thanks!22:20
openstackgerritClark Boylan proposed a change to openstack-infra/project-config: Add license, readme, and contributing files  https://review.openstack.org/12420022:21
clarkbanteaya: jeblair ^ how is that?22:21
clarkbI wgetted that license file straight from apache22:21
jeblairclarkb: cool; can you aprv 124203 while i look at 200?22:21
clarkbjeblair: yup looking now22:21
lifelessclarkb: wgot ?22:21
clarkblifeless: english is hard22:22
jeblairclarkb: why md instead of rst?22:23
clarkbjeblair: it is what config used22:23
jeblairoh, heh.  i think we've switched most things to rst now, yeah?22:23
*** andreykurilin_ has quit IRC22:23
clarkbjeblair: yeah, but I figured consistency with config at this point is good?22:23
clarkbI can convert to rst if that is preferred22:23
finchclarkb: how's the memory usage looking?22:24
*** homeless has quit IRC22:24
jeblairclarkb: i think so, if you don't mind22:25
anteayaclarkb: readme looks very nice22:25
clarkbfinch: its still low but we are not running puppet at full power right now22:25
jeblairclarkb: left one correction in there22:25
*** homeless has joined #openstack-infra22:25
clarkbfinch: doing some major refactoring should know more tomorrow22:25
clarkbjeblair: ok22:25
finchclarkb: okiedokie, just thought I would check in22:25
jheskethMorning22:26
*** dangers is now known as dangers_away22:27
jeblairjhesketh: morning.  we're working from https://etherpad.openstack.org/p/HTlrATUqXy22:27
anteayamorning jhesketh22:28
*** mriedem has quit IRC22:29
openstackgerritClark Boylan proposed a change to openstack-infra/project-config: Add license, readme, and contributing files  https://review.openstack.org/12420022:30
*** flaper87 is now known as flaper87|afk22:30
clarkbnow with more rst22:30
*** thedodd has quit IRC22:30
jeblairclarkb: 1 thing inline22:31
*** bknudson has quit IRC22:31
arosen1almost got this working i think... Needed to add: --allow-external python-congressclient to the tempest requirements.txt22:31
clarkbgah missed that last markdown thing22:31
openstackgerritClark Boylan proposed a change to openstack-infra/project-config: Add license, readme, and contributing files  https://review.openstack.org/12420022:32
jeblairclarkb: i am getting my -1 count up! :)22:33
clarkbMORE!? I am so bad right now22:34
openstackgerritClark Boylan proposed a change to openstack-infra/project-config: Add license, readme, and contributing files  https://review.openstack.org/12420022:34
clarkbcomputers are hard22:34
*** andreaf has quit IRC22:38
jeblairyay +322:38
*** andreaf has joined #openstack-infra22:39
jeblairclarkb: are you in a position to login and run puppet on systems?22:39
clarkbjeblair: probably for about 10 more minutes22:40
clarkbgive me easy ones :)22:40
jeblairhrm, that's sketchy; let's pass :)22:40
clarkbkk22:40
*** emagana has quit IRC22:41
*** amotoki has quit IRC22:41
*** emagana has joined #openstack-infra22:41
*** amotoki has joined #openstack-infra22:42
*** dannywilson has quit IRC22:42
*** emagana has quit IRC22:42
*** gokrokve has joined #openstack-infra22:42
*** dannywilson has joined #openstack-infra22:43
*** mbacchi has quit IRC22:43
*** emagana has joined #openstack-infra22:43
*** dannywilson has quit IRC22:43
*** dannywilson has joined #openstack-infra22:44
*** leifmadsen has quit IRC22:44
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Autoconfiguration before bootstrap.  https://review.openstack.org/12421122:44
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Simplify API-base configuration.  https://review.openstack.org/12421222:44
*** jgrimm is now known as zz_jgrimm22:44
*** gokrokve_ has joined #openstack-infra22:44
*** gokrokve has quit IRC22:45
openstackgerritJames E. Blair proposed a change to openstack-infra/project-config: Update URL for projects.yaml  https://review.openstack.org/12417322:45
anteayawas reviewing and then had a phone call22:46
clarkbjeblair: can I approve 124173?22:46
*** camunoz has joined #openstack-infra22:46
jeblairclarkb: er yes.  but i just did :)22:46
clarkboh you got it :)22:46
jeblairclarkb: want to take over 124102?22:47
*** amotoki has quit IRC22:47
openstackgerritA change was merged to openstack-infra/project-config: Move review-dev acl file  https://review.openstack.org/12420322:47
*** dimtruck is now known as zz_dimtruck22:47
jeblairresuming work on review-dev now22:47
*** leifmadsen has joined #openstack-infra22:49
*** gokrokve_ has quit IRC22:50
clarkbjeblair: sure22:50
openstackgerritJames E. Blair proposed a change to openstack-infra/project-config: Add notify_mpact to dev gerrit  https://review.openstack.org/12421422:50
jeblairclarkb: actually, want to +3 that one first ^ :)22:50
clarkblooking22:52
clarkbjeblair: also what is (.*?) matching22:53
clarkbis the ? redundant and should be replaced with +?22:53
anteayajeblair: typo: the bugs it creates.22:53
jeblairanteaya: what?22:53
clarkboh I think that was a cp we should fix that in a different change22:54
anteayafirst line of that patch22:54
anteayalast word, you have create I think it should be creates22:54
jeblairyeah, i didn't write that file.  i'm trying not to change anything (i mentioned that in the commit msg)22:54
anteayaah sorry22:54
clarkbI have +3'd22:55
clarkblooking at regexes now22:55
asselinjeblair, if you have time. what does include do? why remove it? https://github.com/jaypipes/os-ext-testing-data/commit/3b49d923d343d15a40e1a95e830b55344f1008d2#diff-8eaa3be81ee98f46d860c073b13513f4L122:55
anteayathere is a typo in the commit message: Add notify_mpact to dev gerrit22:55
asselin(line 1 & 2)22:55
clarkbI am not going to overthink (.*?) that is what we matched in the first glob perviously22:56
clarkband the second part makes sense to me22:56
clarkbso pushing as suggested22:56
jeblairclarkb: current is "shortest string of anything up to a /".  it doesn't need to be the shortest since we only expect one /.  i think the alt would be (.+)/  i think either will work.22:56
jeblairanteaya: oops.  oh well.  :)22:57
clarkbjeblair: ya (.+)/ is how my brain works. do you have a preference?22:57
anteaya:D22:57
*** maxbit has joined #openstack-infra22:58
arosen1do you guys ever see this on your jenkins? http://paste.openstack.org/show/115438/22:58
clarkbjeblair: I will keep it as is, makes the diff smaller22:58
arosen1it's like the ssh connection got broken. Second time i've seen it :(22:58
openstackgerritClark Boylan proposed a change to openstack-infra/config: Add project-config to zuul cloner  https://review.openstack.org/12410222:59
clarkbarosen1: ya we see it too. zuul will detect some occurences of that type of bug and rerun the job22:59
jeblairasselin: it imports a file that should not be necessary22:59
*** HenryG has joined #openstack-infra22:59
jeblairasselin: in our setup, it switches node types based on jobs23:00
*** sdake_ has joined #openstack-infra23:00
asselinjeblair, ok, so for single use nodes, I should keep it, correct?23:00
jeblairasselin: yes, it does that too. if you are using nodepool and single use nodes, you will need a variant of it.23:01
*** maxbit_ has quit IRC23:02
*** dmsimard is now known as dmsimard_away23:02
*** _buzz_ has joined #openstack-infra23:02
*** maxbit has quit IRC23:02
jeblairasselin: probably shouldn't be called openstack_functions.py though :)23:03
*** HenryG has quit IRC23:04
*** maxbit has joined #openstack-infra23:04
openstackgerritJan Kundrát proposed a change to stackforge/gertty: Fix help string for --version  https://review.openstack.org/12422023:04
openstackgerritJan Kundrát proposed a change to stackforge/gertty: Unify small vs. capital letters in help output for consistency  https://review.openstack.org/12422123:04
asselinjeblair, ok thanks for the clarification23:05
*** _buzz has quit IRC23:05
jeblairbiab23:05
*** dannywilson has quit IRC23:06
*** gokrokve has joined #openstack-infra23:06
*** dannywilson has joined #openstack-infra23:06
*** HenryG has joined #openstack-infra23:07
*** maxbit has quit IRC23:08
*** baoli has joined #openstack-infra23:08
*** boden has joined #openstack-infra23:09
openstackgerritA change was merged to openstack-infra/project-config: Add license, readme, and contributing files  https://review.openstack.org/12420023:09
*** marcoemorais has quit IRC23:10
*** marcoemorais has joined #openstack-infra23:10
*** marcoemorais has quit IRC23:10
*** marcoemorais has joined #openstack-infra23:11
*** marcoemorais has quit IRC23:11
*** marcoemorais has joined #openstack-infra23:11
*** dannywilson has quit IRC23:12
*** sdake has quit IRC23:12
*** dannywilson has joined #openstack-infra23:13
*** sarob_away is now known as sarob23:13
*** ChuckC has quit IRC23:14
*** boden has quit IRC23:14
*** jpich has joined #openstack-infra23:16
*** dannywilson has quit IRC23:17
*** marcoemorais has quit IRC23:17
*** marcoemorais has joined #openstack-infra23:17
*** gyee has quit IRC23:21
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added configuration file for docs-draft.  https://review.openstack.org/12422223:21
openstackgerritMichael Krotscheck proposed a change to openstack-infra/config: Upload custom configuration for js-draft build.  https://review.openstack.org/12422323:22
openstackgerritA change was merged to openstack-infra/project-config: Update URL for projects.yaml  https://review.openstack.org/12417323:24
*** gyee has joined #openstack-infra23:25
*** bswartz has joined #openstack-infra23:26
reeddo openstack ntegrated and incubated programs map 1-1 to Gerrit groups? example Release Managers https://review.openstack.org/#/admin/groups/11  are "core" members of all RelMgmt program?23:28
openstackgerritA change was merged to openstack-infra/project-config: Add notify_mpact to dev gerrit  https://review.openstack.org/12421423:29
reedRel Mngmt is the wrong example... let's look at Telemetry program23:29
*** Sukhdev has quit IRC23:30
jeblairreed: not always, but it's close.   most programs have a core group and a release group, and all of their projects use the same groups (at least, for recent well structured groups in gerrit)23:31
jeblairreed: some larger programs (eg infra, oslo) have a global core group and also have individual projects with their own core groups that build on that23:31
reedjeblair, makes sense... how about the clients, like ceilometer-client?23:31
morganfainbergreed, jeblair, pycadf is that way in keystone as well. (separate core group)23:31
jeblairreed: clients usually used the shared core groups23:32
reedmorganfainberg, isn't pycadf in Common libraries?23:32
morganfainbergreed, no keystone took it over from oslo team23:32
* fungi checks for actual excitement, rather than the fabricated stuff23:32
reedok, thanks jeblair. So it's fair to say that Programs are made of people *and* git repositories23:32
*** marcoemorais has quit IRC23:33
reedmorganfainberg, thanks, I just saw the yaml file :)23:33
*** marcoemorais has joined #openstack-infra23:33
*** marcoemorais has quit IRC23:33
morganfainbergreed, yeah, we are doing a lot of audit work, makes sense for keystone folks to have a closer eye on it.23:33
*** marcoemorais has joined #openstack-infra23:33
* reed is editing the Programs page on the wiki23:34
*** sdake_ has quit IRC23:34
*** dims has joined #openstack-infra23:35
jeblairfungi: i believe i've corrected the review-dev related errors.  i then rm'd all of the _production_ acl files from review-dev and ran puppet and it now reports no changes23:35
jeblairfungi: (it was going to rm them itself and run manage-projects; i did not think it was necessary to have it do that)23:35
jeblairi ran it for real on review-dev; moving on to review now.23:35
*** tsg has joined #openstack-infra23:36
*** sdake has joined #openstack-infra23:36
jeblairfungi: similarly, i'm going to manually rumove /home/gerrit2/acls/test-manage-project.config from review.o.o to prevent a m-p run.  sound good?23:37
*** david-lyle has joined #openstack-infra23:37
*** HenryG has quit IRC23:37
*** david-lyle has quit IRC23:38
*** MarkAtwood has joined #openstack-infra23:39
*** sballe_ has quit IRC23:39
jeblairdone23:39
jeblairi'm going to re-enable the puppetmaster cron now23:39
*** sarob is now known as sarob_away23:40
fungiscrollback is full of patches which have already merged...23:40
*** dmsimard_away is now known as dmsimard23:41
*** bknudson has joined #openstack-infra23:41
*** SumitNaiksatam has quit IRC23:41
fungialso, i always use "mpact" as an abbrev for impact. doesn't everyone?23:41
jeblairsounds the same to me!23:41
anteayalike mkay23:41
*** nelsnelson has quit IRC23:43
fungimkay, so i'm all caught up now23:44
fungiwhat should i help break^H^H^H^H^Hfix?23:44
fungiI can start going through openstack-infra/config and -2'ing changes which only touch files that got moved to project-config, and -1 the ones which should touch both projects... or is it still early for that?23:45
fungiproofing the e-mail now23:46
*** Sukhdev has joined #openstack-infra23:47
*** boris-42 has quit IRC23:47
openstackgerritA change was merged to openstack-infra/config: Add project-config to zuul cloner  https://review.openstack.org/12410223:48
jeblairfungi: yeah, i think we're about ready for that.  when you're done reviewing the email, i'll send it, then i think we can aprv the remove files in config change23:48
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Improve jeepyb.rst content  https://review.openstack.org/12412723:48
fungii guess the jobs addition can get approved now? ^23:48
*** boris-42 has joined #openstack-infra23:48
jeblairfungi: yep23:48
jeblairfungi: then i'd say we're set to -2 related changes.  maybe keep a list of those and we can abandon them after a couple of weeks?23:48
jeblair(this seems like a good use of abandon to me)23:49
fungiwill totally add a running list of the change ids in the etherpad23:49
*** dmsimard is now known as dmsimard_away23:49
fungiapproved 12418123:50
jeblairpuppet is running for real globally and so far looks good23:50
jeblairfungi: email lgty?23:51
SukhdevFolks, I am hitting failure in jenkins -  http://logs.openstack.org/86/123886/2/check/check-tempest-dsvm-neutron-heat-slow/5f86a11/console.html23:51
*** gokrokve has quit IRC23:51
SukhdevI rechecked to kick off another jenkins run and now I get this failure https://jenkins05.openstack.org/job/check-tempest-dsvm-neutron-heat-slow/16693/console23:51
fungijeblair: just finished proofing, lgtm, no edits, send it asap, kthxbai23:52
jeblairSukhdev: those don't look like test infrastructure failures to me23:52
jeblairSukhdev: you might try asking in the #openstack-qa channel23:52
SukhdevIs this known issue? anything I can do to get past this issue?23:52
Sukhdevjeblair: I did - no body responded23:52
Sukhdevjeblair: I usually get better luck with these kinds of issues - hence, thought I ask in both channels23:53
anteayaSukhdev: or a neutron issue23:53
jeblairSukhdev: it's near or past the end of day in US, you could wait or try again at a different time23:53
anteayaBadRequest: Security group default already exists for project23:53
jeblairemail sent!23:54
fungiyay!23:54
fungifinish line in sight23:54
jeblairfungi: i think we're ready for you to aprv 123921 at will23:55
fungijeblair: now that it's sent, i can quote a url to it in the -2 and -1 comments i'm about to leave on changes ;)23:55
jeblairyay!23:55
fungiapproving23:56
*** bknudson1 has joined #openstack-infra23:59
clarkbwoo I am back to old place now23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!