Thursday, 2015-03-26

jeblair	clarkb: yeah, i'm with you, and feel much more strongly about this than => alignment. :) otoh, it's tough to say no to code already written, but there's ongoing maintenance and what we want to spend our time writing and reviewing to consider.	00:00
*** HeOS has joined #openstack-infra		00:01
jeblair	i'd like to see if we can get consensus if possible	00:01
openstackgerrit	Ian Wienand proposed openstack-infra/system-config: Split diskimage-builder built images out https://review.openstack.org/167837	00:02
clarkb	nibalizer: and question on 164845	00:02
*** annegentle has quit IRC		00:02
*** SumitNaiksatam has quit IRC		00:03
openstackgerrit	Merged openstack-infra/project-config: heat-dsvm-functional enable build pip mirror https://review.openstack.org/165227	00:04
*** HeOS has quit IRC		00:05
sdake	regarding rename happenign shortly	00:06
sdake	does anyone hae an example review of how this is done?	00:06
sdake	want to get magnum renamed	00:07
anteaya	sdake: if you look at the bottom of https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting	00:07
anteaya	in the section renames, you will notice a number of patches	00:08
*** krtaylor has joined #openstack-infra		00:08
anteaya	pick one, use it as a template and then add yours to the same section	00:08
openstackgerrit	James E. Blair proposed stackforge/gertty: Add held changes https://review.openstack.org/167838	00:08
sdake	anteaya thanks!	00:09
*** marun has quit IRC		00:09
anteaya	sdake: thanks for asking	00:09
*** ddieterly has joined #openstack-infra		00:10
*** tkelsey has joined #openstack-infra		00:10
*** dangers is now known as dangers_away		00:11
*** ddieterl_ has joined #openstack-infra		00:11
*** davide has joined #openstack-infra		00:11
*** otter768 has quit IRC		00:11
*** sdake__ has joined #openstack-infra		00:11
openstackgerrit	Monty Taylor proposed openstack-infra/ansible-puppet: Copy in matching hiera files https://review.openstack.org/165157	00:13
mordred	clarkb, nibalizer: ^^	00:13
asselin	nibalizer, clarkb (or other puppet expert) any idea why templates can't be found on my change here? http://paste.openstack.org/show/196760/ https://review.openstack.org/#/c/167425/2	00:13
*** tkelsey has quit IRC		00:14
*** ddieterly has quit IRC		00:14
*** sdake has quit IRC		00:15
*** davide has quit IRC		00:15
openstackgerrit	Steven Weston proposed openstack-infra/infra-specs: Third Party CI Dashboard Spec https://review.openstack.org/135170	00:15
*** ivar-lazzaro has quit IRC		00:16
fungi	sdake_: thanks for noticing the subtle nudge in the announcement ;)	00:16
*** gyee has quit IRC		00:16
*** ivar-lazzaro has joined #openstack-infra		00:16
*** ddieterly has joined #openstack-infra		00:17
*** ddieterl_ has quit IRC		00:17
clarkb	asselin: you need to use the module name template(openstackci/foo.erb) iirc	00:17
asselin	clarkb, ok will try	00:17
*** Longgeek has joined #openstack-infra		00:18
clarkb	nibalizer: still around? curious about your thoughts on my comments on the hiera changes	00:19
mordred	clarkb: the ones to add the other paths?	00:19
openstackgerrit	Steven Weston proposed openstack-infra/infra-specs: Third Party CI Dashboard Spec https://review.openstack.org/135170	00:19
openstackgerrit	Henry Gessau proposed openstack-infra/project-config: Add an IPv6 enabled job to neutron experimental pipeline https://review.openstack.org/167847	00:19
clarkb	mordred: yes	00:19
*** mtanino has quit IRC		00:19
openstackgerrit	Steven Dake proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack->magnum https://review.openstack.org/167848	00:19
* mordred goes to read clarkb's comment		00:19
sdake__	anteaya how does that look?	00:20
mordred	clarkb: I'm not sure I undersatnd your english	00:20
mordred	clarkb: but that might be the cachaca talking	00:20
openstackgerrit	Steven Weston proposed openstack-infra/infra-specs: Third Party CI Dashboard Spec https://review.openstack.org/135170	00:20
*** annegentle has joined #openstack-infra		00:20
mordred	clarkb: would you mind re-stating for my adleminded brainhole?	00:20
clarkb	mordred: first one is jenkins.o.o should be part of the jenkins group	00:20
mordred	clarkb: should it?	00:21
mordred	(asking)	00:21
mordred	nevermind - I agree	00:21
clarkb	mordred: second one is we have a comment there saying "don't scope this because its in node scope" why not just explicitly use the node scope then isntead of commetning why there is no explicit scope	00:21
clarkb	mordred: yes I am pretty sure we want that master to be in the same group	00:21
mordred	++	00:21
dstufft	+= 1	00:21
*** ghostpl_ has joined #openstack-infra		00:21
clarkb	BUT I don't actually know if node scope is a thing you can be explicit about in puppet	00:21
mordred	clarkb: yeah - I think that's the thing he's saying	00:22
clarkb	because puppet scoping is something one does not simply understand	00:22
mordred	but I agree, I'd like to learn more	00:22
mordred	from him	00:22
mordred	nibalizer: stop drinking and answer us!	00:22
*** HeOS has joined #openstack-infra		00:22
anteaya	sdake__: the tests will come back asking you to aphabetize your projects.yaml and layout.yaml entries	00:22
openstackgerrit	Merged stackforge/gertty: Add mouse wheel scrolling https://review.openstack.org/133550	00:22
anteaya	sdake__: you can wait or fix it now	00:22
asselin	clarkb, that was it thanks.	00:22
*** pblaho has quit IRC		00:22
sdake__	I thought I did :)	00:22
sdake__	apparently my abcs are off	00:22
mordred	clarkb: there is also one impdence mismatch - in that a node in ansible can be in more than one group, and in this layout it is one node per one group	00:23
mordred	clarkb: I think taht's fine, because we're humans	00:23
mordred	but it'sa thing to note	00:23
jeblair	mordred: speak for yourself	00:23
mordred	the ansible playbook will look for any group yaml file matching a group that the node is in	00:23
mordred	by ansible's reconning	00:23
mordred	jeblair: I may be dancer	00:23
jeblair	mordred: some of us may have been replaced by a small python script	00:24
*** spzala has quit IRC		00:24
anteaya	sdake__: gerritbot channels.yaml looks like it needs to change as well, plus you might as well do magnumclient in the same patchset	00:24
mordred	dude. that ahppened to me ages ago	00:24
jeblair	self.throw_wet_cat(random.choice(targets))	00:24
mordred	class WetAnimal(object): def throw(self, targets) class WetCat(WetAnimal): class WetLlama(WetAnimal): ...	00:25
jeblair	Monty's Moist Menagerie	00:26
*** HeOS has quit IRC		00:26
fungi	ew. just... ew	00:26
openstackgerrit	Steven Dake proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack->magnum https://review.openstack.org/167848	00:27
anteaya	yeah I'm with fungi	00:27
sdake__	anteaya yup apparently my brain was broken when I submitted that first revew - should be all fixed now :)	00:27
anteaya	ew	00:27
dstufft	mordred: get you some composition	00:27
anteaya	sdake__: it happens everytime	00:27
dstufft	Wet(Cat())	00:27
anteaya	sdake__: no worries	00:27
*** sarob has joined #openstack-infra		00:27
sdake__	unfortunately	00:28
mordred	dstufft: I could write a wet cat factory factory that returns factories that return wet cat factories	00:28
anteaya	sdake__: fortune and project-config don't hang out much I find	00:28
mordred	dstufft: and then I could use dependency injection to turn those factory factories into factories taht return different factoires	00:28
sdake__	its all good	00:29
mordred	dstufft: all driven, of course, by some simple xml that can be modified by xslt	00:29
dstufft	mordred: seems legit to me	00:29
* mordred has done enterprise development before		00:29
sdake__	projet config reviews are typically very fast	00:29
dstufft	how did anyone throw wet cats before they had xml	00:29
*** Longgeek has quit IRC		00:29
anteaya	sdake__: the gerritbot channels.yaml file? did you want to take a look at that file too, it appears to me it could use a refresh	00:29
anteaya	sdake__: yes	00:29
anteaya	sdake__: on both sides	00:29
mordred	dstufft: well, they had to do them by hand. why do that when you can write a system to generically throw things that might be cats or wet or animals and which you could also use to pitch or lob or hoist things with?	00:30
openstackgerrit	Steven Dake proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack->magnum https://review.openstack.org/167848	00:30
sdake__	thanks anteaya	00:30
anteaya	thank you	00:31
*** sarob has quit IRC		00:32
anteaya	sdake__: add in python-magnumclient to it and you are all set	00:32
sdake__	anteaya I was going to do a seprate patch for that one	00:32
anteaya	sdake__: just do what you did for the config file, projects.yaml and layout.yaml	00:32
sdake__	shoud I merge them into one?	00:32
anteaya	sdake__: we prefer one patch	00:32
anteaya	please	00:33
sdake__	sounds good	00:33
clarkb	mordred: I don't know if yo usaw but recently spearce wrote about how google runs gerrit on the gerrit ml. and now I know why gerrit uses dependency injection when the open code basically has no use for it	00:33
anteaya	as gerrit has limits about how many rename patches it can merge before it gets upset	00:33
anteaya	sdake__: so the more in one patch, the better for us	00:33
anteaya	I think the limit is 5?	00:33
clarkb	mordred: because google replaces significant portions of gerrit to use bigtable and megastore	00:33
mordred	clarkb: of course they do	00:34
mordred	funny thing is ...	00:34
mordred	our gerrit is bigger than theirs	00:34
mordred	and we run on one mysql	00:34
mordred	so phbpbpbpbpbpbpbpbpb	00:34
mordred	once again mysql triumphs over bigtable	00:34
* mordred looks over at AdSense and chuckles ...		00:35
clarkb	granted they are apparently still working hard on killing the need for any db	00:35
clarkb	which is good for us and them so thats nice	00:35
*** davide has joined #openstack-infra		00:35
*** ivar-laz_ has joined #openstack-infra		00:36
mordred	clarkb: you know - I used to think it was good for us	00:38
mordred	but actually, I think scaling git repos turns out to be kinda hard and the db has never really been a problem	00:38
mordred	in fact, if they did less in git and more in the db - we could do scale out with multiple gerrit servers earier	00:38
mordred	easier	00:38
mordred	but	00:38
mordred	you know	00:38
mordred	that would just require taking advantage of tried and true architecture	00:38
anteaya	what is our plan for scaling gerrit?	00:39
mordred	anteaya: prayer	00:39
anteaya	once we find out the limit	00:39
clarkb	mordred: so they have described how multimaster would work with only git and no db and its not that terrible	00:39
anteaya	yes that does seem to be the current strategy	00:39
*** ivar-lazzaro has quit IRC		00:39
mordred	clarkb: ok	00:39
anteaya	and prayer works well for me personally	00:39
mordred	clarkb: I'm sure it's great in theory	00:39
anteaya	doesn't seem to work quite so well for groups I find	00:39
mordred	and I'm sure that once it's in production it will fail in new ways that nobody has dealt with at scale in production yet	00:39
clarkb	mordred: you basically have paxos to keep your masters in sync with git then do specialness for knowing if an operation should block on an update	00:39
mordred	just like every other multi-master replication technology invented over the last 30 years	00:40
clarkb	mordred: yes but thats true of any multi master solution	00:40
*** woodster_ has quit IRC		00:40
mordred	clarkb: yes - this is why new multi master tech almost always is great at first and then fails hard	00:40
clarkb	mordred: put multiple masters in front of one mysql and watch them get cranky at each other over row locks	00:40
mordred	so you want to wait until it's been sorted out	00:40
mordred	clarkb: mysql doesn't do multi-master	00:40
clarkb	mordred: sorry galera	00:40
mordred	galera is exactly my point	00:41
mordred	it's new tech	00:41
mordred	and will take at least another 10 years to be actually good	00:41
mordred	nobody doing thing actually at scale uses it	00:41
*** davide has quit IRC		00:41
mordred	but it turns out you don't actually need multi-master for anything other than banking and telco	00:41
mordred	and those fokls can pay for the ram it takes to do it right :)	00:42
dstufft	big iron supremacy	00:42
openstackgerrit	Steven Dake proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack->magnum https://review.openstack.org/167848	00:42
*** woodster_ has joined #openstack-infra		00:42
clarkb	mordred: my favorite outages were always the ones where HA tooling explodes	00:42
mordred	yup	00:42
mordred	I'm not going to reference a recent SAN outage that happened in a company I know	00:43
clarkb	mordred: because then you have to explain to management that that fancy tool they spent tons of money on was the cause of the downtime and we would have been better off without it :)	00:43
dstufft	sometimes people want me to switch PyPI to some sort of magic nosql multi master thing	00:43
mordred	dstufft: nope	00:43
mordred	turns out it'sa terrible idea	00:43
mordred	dstufft: nosql will be ready for primetime when and only when Mark Callaghan and Domas Mitzuas say it is	00:43
mordred	they're looking at it. hard. and they want it to be ready	00:44
mordred	and they're still not using it	00:44
mordred	so once they do, I'll start paying attention	00:44
dstufft	mordred: made even more terrible by the fact that PyPI has a tiny number of writes	00:44
mordred	(they run MySQL for Facebook for anyone not familiar with this particular rant)	00:44
mordred	dstufft: hahahaha	00:44
mordred	yeah. you don't need anything fancy	00:44
mordred	you need front-loaded caching	00:44
dstufft	we get like	00:44
dstufft	~600 new uploads a day	00:45
mordred	hehe	00:45
mordred	now - if you wanted to go fancy	00:45
mordred	you should use mogilefs	00:45
dstufft	(that's obv not all the writes, but it's the major ones)	00:45
mordred	because that shit is SOLID	00:45
clarkb	dstufft: would you like us to significantly change that number?	00:45
openstackgerrit	Steven Dake proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack->magnum https://review.openstack.org/167848	00:45
mordred	clarkb: :)	00:45
clarkb	dstufft: beacuse we can totally help you out >_>	00:45
*** Somay has joined #openstack-infra		00:45
mordred	dstufft: if you're like to justify purchasing a san or something	00:45
mordred	we could publish a pypi package on every commit that lands	00:45
dstufft	lol "purchase", we're total beggars	00:46
*** sdake has joined #openstack-infra		00:46
dstufft	I'm not sure my @python.org email address is magical enough to get a free SAN tho	00:46
*** sdake__ has quit IRC		00:46
*** tsg has quit IRC		00:46
openstackgerrit	Steven Dake proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack/magnum https://review.openstack.org/167848	00:47
*** sdake__ has joined #openstack-infra		00:47
*** sdake has quit IRC		00:47
mordred	dstufft: I'm telling you - you should join python.org forces with openstack.org - we turn down racks of hardware regularly because we don't have a place to put them :)	00:47
anteaya	mordred: please don't swear in channel, I get so tired of it	00:47
mordred	anteaya: sorry	00:48
*** melwitt has quit IRC		00:48
anteaya	thank you	00:48
*** markvoelker has joined #openstack-infra		00:48
mordred	dstufft: of course, then you'd have to figure out what to do when docker takes over and we can't get a crust of bread	00:48
mordred	dstufft: so you're probably better off just remaining sane	00:49
*** aysyd has quit IRC		00:49
mordred	fungi: you round?	00:49
mordred	fungi: your email left os-client-config out ... should I send you a fruit-basket for bribery?	00:50
jeblair	mordred: now might be a good time to add it to the list here: https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting	00:51
*** sdake__ has quit IRC		00:51
*** HeOS has joined #openstack-infra		00:51
fungi	mordred: d'oh! i missed it	00:52
*** oomichi_ has joined #openstack-infra		00:52
jeblair	(because it wasn't there)	00:52
*** sdake has joined #openstack-infra		00:52
*** amitgandhinz has joined #openstack-infra		00:52
fungi	indeed. i often miss things that aren't there	00:53
*** markvoelker has quit IRC		00:53
fungi	almost as often as i see things that aren't there	00:53
dstufft	mordred: lol	00:53
fungi	i should probably see a doctor for that	00:53
* jeblair is not here		00:53
*** sdake__ has joined #openstack-infra		00:53
fungi	good call	00:53
*** annegentle has quit IRC		00:54
anteaya	fungi: do the things you see that aren't there talk to you?	00:54
dstufft	mordred: if we take the numbers at the Fastly edge (e.g. total traffic, our read:write ratio on just files themselves, not /simple/ or anything else is like 1:1000000	00:54
anteaya	sdake__: getting closer, we need the patch url added to https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting	00:54
fungi	anteaya: oddly no, it's the things i don't see that aren't there that usually talk to me	00:54
anteaya	sdake__: that is the place we go looking for patches	00:55
anteaya	fungi: cool	00:55
anteaya	fungi: I hope they say nice things	00:55
sdake__	anteaya will do	00:55
fungi	usually, yes	00:55
anteaya	that's good	00:55
anteaya	mine too	00:55
dstufft	if we take the ones that just hit the origin, it's like 3:1000 or so	00:55
openstackgerrit	Steven Dake proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack/magnum https://review.openstack.org/167848	00:56
*** HeOS has quit IRC		00:56
ianw	Cloning into 'devstack-gate'...	00:56
ianw	fatal: The remote end hung up unexpectedly	00:56
ianw	fatal: protocol error: bad pack header	00:56
ianw	^ not the first time i've seen this with --depth flag	00:56
sdake__	sorry for the sloppy work	00:56
sdake__	been long weeks prior	00:56
mordred	dstufft: you are the perfect ratio for success	00:56
sdake__	this is why gerrit rocks :)	00:56
fungi	ianw: yes, cloning from review.openstack.org/p with --depth is broken now that we're running git on trusty there	00:56
dstufft	mordred: our scaling plan is "cache everything in fastly and never think about it"	00:56
sdake__	anteaya ^^ :)	00:56
clarkb	ianw: git on ubuntu trusty is broken and does not support shallow clones. You can clone from the git mirror (which is whwere we want people to clone from)	00:56
mordred	dstufft: its not a terrible plan	00:57
anteaya	sdake__: good work	00:57
dstufft	turn out that works pretty well	00:57
*** sdake has quit IRC		00:57
dstufft	we're at ~2 billion requests a month and I didn't even notice it went up so much	00:57
anteaya	sdake__: what you just saw is typical of any patch of that type	00:57
mordred	ianw: git in upstream master git is also broken - so it's not trusty's fault	00:57
dstufft	(used to be way under a billion)	00:57
anteaya	sdake__: thanks for staying with it	00:57
mordred	basically, upstream git has decided to break shallow clones	00:57
ianw	fungi / clarkb / mordred : ok, good to know, thanks	00:57
fungi	ianw: for some reason current git releases expect the git-http-backend cgi to be able to write inside the git repos it's serving to support shallow cloning	00:57
openstackgerrit	Stephanie Miller proposed openstack-infra/puppet-zanata: Add apache configuration for zanata https://review.openstack.org/164011	00:57
clarkb	mordred: fungi oh is that what the issue is. Thats a bit silly	00:58
mordred	fungi: it's one of the most truly insane things I've ever seen	00:58
fungi	clarkb: yep	00:58
mordred	and that includes TI Basic	00:58
sdake__	anteaya is there any other way to do the job? :)	00:58
anteaya	sdake__: not currently	00:58
fungi	clarkb: you can see it plain as day in the apache error logs	00:58
dstufft	wait what	00:58
fungi	big fat "permission denied" errors	00:58
anteaya	sdake__: and yes persistence does get a job done	00:58
anteaya	:)	00:59
dstufft	why would git expect to be able to write for a clone	00:59
dstufft	that's crazy	00:59
fungi	dstufft: it wants to be able to create a temporary pack file to serve a shallow clone request	00:59
*** achanda has quit IRC		00:59
mordred	jeblair, fungi: I ahve fixed the wiki	00:59
clarkb	mordred: also can you enlighten us with your ccla idea? (don't need to do it here, but its a bit annoying to be teased like that :) )	00:59
dstufft	shouldn't a temporary file go inside of TMPDIR	00:59
fungi	dstufft: and apparently inside the git root is the perfect place to do that	00:59
mordred	clarkb: it's not my idea	00:59
mordred	clarkb: it's jonathans	00:59
*** ociuhandu has quit IRC		00:59
mordred	he ran it by me, sounded great	01:00
mordred	I've been hoping he'd follow up with a public plan	01:00
anteaya	sdake__: so just adding it to the meeting page is left to do	01:00
*** ghostpl_ has quit IRC		01:00
sdake__	in process	01:00
anteaya	thanks	01:00
mordred	clarkb: tl;dr - CCLA managed by foundation member data system - corporations that sign the CCLA get an accoutn to manage the people that it covers	01:00
mordred	clarkb: so developers don't have any need to be bothered by anything related to CCLA at all	01:01
ianw	fungi: alright, i'm running the cinder + full tempest just like in your example on that host, and i'm piping out a bunch of telemetry data via netcat to a remote host	01:01
clarkb	mordred: then what good is tracking the data? I thought the whole point was make gerrit aware of it	01:01
mordred	nope	01:01
clarkb	mordred: I mean if lawyers are happy with that I won't complain	01:01
clarkb	but that seems to be fixing a problem for no good reason	01:01
mordred	the best part is that it gets infra OUT of the game of caring	01:01
fungi	mordred: i don't think developers had to care about the ccla before? corporate lawyers were just doing stuff in echosign and sometimes submitting updated schedula a lists for it	01:01
anteaya	so how does that change the acl files?	01:01
ianw	fungi: my hope is that it all goes pear shaped and i capture what went crazy	01:01
mordred	the thing it fixes is that it makes the lawyers happier	01:01
sdake__	ok all done	01:02
anteaya	do we remove the cla required line?	01:02
sdake__	ping me if something else is needed :)	01:02
clarkb	anteaya: it doesn't. what mordred is saying is we don't have to chang anything	01:02
mordred	fungi: the thing it fixes is that we might not need devs who are covered by a CCLA to sign an ICLA	01:02
dstufft	if lawyers get cranky they start suing people	01:02
fungi	ianw: hope so. odds are it will go silent just before you get the good stuff	01:02
anteaya	sdake__: awesome thank you	01:02
anteaya	sdake__: will ping	01:02
anteaya	sdake__: also if you are around during the rename it never hurts	01:02
mordred	fungi: so if openstackid auth's you - you're good to go. nothing to see here	01:02
fungi	mordred: right, but for that we need some way for gerrit to know they're covered by a cla	01:02
anteaya	just being available in channel	01:02
clarkb	can I shop https://review.openstack.org/#/c/167382/ around for reviews? we need to be prepared before pypi breaks us	01:03
sdake__	i am always in this channel	01:03
mordred	fungi: we need gerrit to know they're allowed to upload changes	01:03
sdake__	just say magnum or kolla or sdake ;)	01:03
ianw	fungi: that's ok, i'm really hoping to see a process ramping up usage just before it explodes	01:03
sdake__	one of em will get me	01:03
*** doude has quit IRC		01:03
mordred	fungi: in any case- there are details to be worked out	01:03
anteaya	sdake__: cool, good to know	01:03
mordred	and I don't have enough to be useful here	01:03
clarkb	mordred: right the tricky bit is enforcement. If we don't need to enforce then cool. If we do need to enforce then this idea isn't so great	01:03
fungi	mordred: right, it's the details which will make or break it. clarkb has at least a proposal for how to do that	01:03
*** stevemar has joined #openstack-infra		01:04
mordred	I think the big key there is taht enforcement in the way _WE_ think about it isnt' what they're looking for	01:04
mordred	which is great	01:04
clarkb	my understanding was the lawyers wanted enforcement	01:04
fungi	but involves corporate lawyers getting control of specific per-company ccla groups in gerrit	01:04
mordred	and we can get out of the game	01:04
mordred	they want enforcement	01:04
dstufft	clarkb: fwiw I can't think of anything else we're likely to change that'll require a bandersnatch update beyond the last thing.	01:04
* anteaya will still expect to see needs cla in acl files of namespaces that currently have that		01:04
mordred	but they want lawyer-speak enforcement	01:04
*** doude has joined #openstack-infra		01:04
*** sdake has joined #openstack-infra		01:04
mordred	they want the paper trail that makes corporate lawyers happy	01:04
*** zul has quit IRC		01:04
clarkb	mordred: my only concern with that is 6 months from now someone will push code, someone that cares will go "wait you can't do that" then they will ask us to actually enforce	01:05
*** HeOS has joined #openstack-infra		01:05
clarkb	mordred: and if we had just solved the actual problem outright it would not be a problem	01:05
mordred	clarkb: I suggest talking to jonathan	01:05
mordred	I'm being a bad proxy	01:05
fungi	(actual problem being the need for a cla at all)	01:05
mordred	but I think the problem is being shifted	01:05
clarkb	ok	01:05
clarkb	dstufft: ya, its probably a reasonable thing to grab latest bandersnatch until it breaks then we can scream about it and install the last working version	01:06
mordred	and I think the actual problem fungi brings up will be addressed in a way that makes us happier	01:06
fungi	anyway, if there are lawyers saying they're okay with us turning off cla enforcement in gerrit, i'll throw them a party	01:06
mordred	dstufft: any chance you have any pull with the bandersnatch people?	01:06
fungi	and by party i mean stop complaining constantly	01:06
mordred	fungi: yah. that is EXACTLY what they're saying	01:06
dstufft	mordred: I know Christian and talk to him from time to time	01:07
clarkb	another set of changes that would be super handy for swift logs starts at https://review.openstack.org/#/c/167143/	01:08
*** sdake__ has quit IRC		01:08
clarkb	basically moves the d-g help footer into d-g then we can start adding it to the swift uploaded index files	01:08
mordred	dstufft: jeblair has this patch outstanding ...	01:08
mordred	that would help us out in our mirroring infrastructure	01:09
*** HeOS has quit IRC		01:09
mordred	dstufft: it seems to have been ignored since Dec 20	01:09
mordred	dstufft: https://bitbucket.org/pypa/bandersnatch/pull-request/12/add-option-to-dir-hash-index-files/diff	01:09
* mordred bribes dstufft with a fatted calf		01:10
*** HeOS has joined #openstack-infra		01:11
anteaya	clarkb: I see no dependencies or depends-on in that linked patch	01:12
dstufft	mordred: sorry was emailing my daughter's teacher	01:12
dstufft	mordred: looking	01:12
anteaya	clarkb: you said it was a set of changes?	01:12
*** achanda has joined #openstack-infra		01:13
*** jcoufal has joined #openstack-infra		01:14
*** HeOS has quit IRC		01:16
*** Longgeek has joined #openstack-infra		01:16
dstufft	mordred: there's a merge conflict on it, beyond that I'd just poke christian about it. Normally I tweet at him on twitter or email him	01:17
anteaya	clarkb: found it, this patch depends-on the patch you linked: https://review.openstack.org/#/c/167158/2	01:17
clarkb	anteaya: sorry, forgot it was a depends on elsewhere	01:17
clarkb	anteaya: if you review the topic:enable_swift you should see all of them	01:18
dstufft	mordred: if y'all fix the merge conflict I'm happy to toss christian an email	01:18
anteaya	clarkb: unfortunately there is no easy for for me to go backwards in that relationship	01:18
mordred	dstufft: woot!	01:18
*** SumitNaiksatam has joined #openstack-infra		01:18
mordred	jeblair: ^^	01:18
anteaya	clarkb: yeah that is how I found it, just wanted to make sure I understood you correctly	01:18
anteaya	thanks	01:18
mordred	jeblair: look at me bridging gaps	01:18
mordred	dstufft: if we get that in, we can start running our AFS mirror :)	01:18
* anteaya hands mordred a cookie		01:18
* mordred eats cookie. gets sugar rage. starts running through the streets with a pack of 72 alpacas and one lost goat		01:19
dstufft	mordred: what's the benefit of an AFS mirror	01:20
*** tonytan4ever has quit IRC		01:20
mordred	dstufft: AFS can do atomic publication to its volumes - so the plan is that we run bandersnatch into a readwrite volume	01:21
mordred	and then if everything is good and there are no issues, we'll do an AFS publish	01:21
dstufft	mordred: oh, so a mirror isn't in an inconsistent state?	01:22
fungi	and publish is atomic across the entire fs	01:22
*** Longgeek has quit IRC		01:22
mordred	dstufft: yah	01:22
fungi	oh, mordred said that	01:22
fungi	once we're through the gerrit upgrade i want to finish doing the same thing for our git push mirrors so we can drop the source affinity in the load balancers and get a more even request distribution	01:23
fungi	though i'll need to get gerrit running wrapped in kstart	01:23
fungi	dstufft: also afs is a great excuse to dust off your mad kerberos skilz	01:24
*** mixpix3ls has joined #openstack-infra		01:25
mixpix3ls	Hi everyone.	01:25
mixpix3ls	I'm trying to get GerritBot working on RHEL. Is there a RHEL-compatible gerritbot.init?	01:25
*** Longgeek has joined #openstack-infra		01:25
dstufft	fwiw an inconsistent state should probably almost never happen anymore	01:25
dstufft	I think the only thing that can cause it now is if someone deletes a file, which is rare (especially now that you can't re-upload)	01:26
mordred	dstufft: awesome	01:26
*** mark_gloshen has quit IRC		01:26
fungi	mixpix3ls: not that we've written	01:26
fungi	mixpix3ls: i think the initscript there was just us scratching an itch to be able to start it on our (ubuntu) server	01:27
openstackgerrit	Wayne Warren proposed openstack-infra/jenkins-job-builder: (hipchat) Accept list of hipchat rooms. https://review.openstack.org/153831	01:28
* anteaya reconsiders offering mordred a cookie in future		01:28
* anteaya thinks of the wildlife		01:29
mordred	anteaya: the lost goat seems happy	01:29
mixpix3ls	Hm, ya, it's pretty Ubuntu specific. I'm trying to convert it to systemd.	01:29
anteaya	well there's that	01:29
*** HeOS has joined #openstack-infra		01:30
*** Longgeek has quit IRC		01:30
pleia2	mixpix3ls: we'll certainly appreciate that work at some point, the Ubuntu LTS coming out next year will be using systemd (it landed in 15.04 a couple weeks ago)	01:31
pleia2	+by default	01:31
fungi	mixpix3ls: a systemd unit file would probably be best in that case	01:31
*** zul has joined #openstack-infra		01:31
fungi	instead of an initscript	01:31
fungi	though maybe that's what you meant	01:31
ianw	fungi: if you have a moment, could you log into a comparable ubuntu vm running tempest atm and do a "free -h" just so i can get an idea of the numbers	01:33
fungi	okay, the pending renames that i'm aware of are all using a consistent topic for their changes now https://review.openstack.org/#/q/status:open+topic:project-rename,n,z	01:33
fungi	ianw: sure, doing that now	01:33
*** tiswanso has joined #openstack-infra		01:33
anteaya	clarkb: do I read jhesketh's comment correctly here that he wanted to change the name of the job 'zuul-swift-devstack-logs' to 'zuul-swift-devstack-logs-with-console'? https://review.openstack.org/#/c/167158/1..2/jenkins/jobs/macros.yaml	01:34
*** HeOS has quit IRC		01:34
*** Longgeek has joined #openstack-infra		01:34
anteaya	because if that is the case, I don't see that that happened	01:35
jhesketh	anteaya: it's here: https://review.openstack.org/#/c/167378/	01:35
anteaya	jhesketh: so it is, thank you	01:35
fungi	ianw: http://paste.openstack.org/show/196768/ is taken from the host running https://jenkins06.openstack.org/job/check-tempest-dsvm-full/26604/ right now	01:36
anteaya	jhesketh: I did read your comment correctly, thank you	01:36
fungi	ianw: so most of the way through the job, well into tempest tests	01:36
fungi	ianw: thought that was in hpcloud. if it needs to be in rackspace it may take more effort to track down a comparable example	01:37
*** baoli has joined #openstack-infra		01:37
ianw	fungi: no, that's cool -- the numbers are in the same ballpark	01:37
ianw	that host has stopped though	01:38
mixpix3ls	fungi and pleia2 thanks for the input.	01:39
*** Longgeek has quit IRC		01:39
*** jcoufal has quit IRC		01:39
ianw	fungi: http://paste.openstack.org/show/196759/ is the last snapshot from the host before it stopped responding	01:40
fungi	ianw: so you're able to reproduce the behavior. excellent	01:41
ianw	fungi: http://paste.openstack.org/show/196769/	01:41
ianw	sorry	01:41
fungi	yeah, it was nearly out of memory for sure	01:42
ianw	there's no one process that seems to be out of line, its death by a thousand cuts	01:42
fungi	that's more or less what i get when glancing at the dumps from the oom killer in dmesg too	01:42
ianw	i'm not sure what to go with other than "need more memory"	01:42
fungi	which explains why it tends to randomly select something, but not necessarily the same something	01:43
*** jerryz has joined #openstack-infra		01:43
*** hashirabdi has quit IRC		01:43
ianw	but also from what you paste of the ubuntu vm, it's pretty close too	01:43
clarkb	the odd thing is its only centos7 and only on rax	01:43
clarkb	which points to bug not "use more memory"	01:43
fungi	or "bug uses more memory"	01:44
mordred	bugs use more memory	01:44
anteaya	jhesketh: if I approve this now, what jobs does it affect, after it gets into the images? https://review.openstack.org/#/c/167046/1	01:44
anteaya	all infra and python-jobs?	01:44
ianw	fungi / clarkb : did we fix the issues with hp hosts actually having 32gb ram due to missing limits?	01:45
clarkb	ianw: I think fungi fixed it. I can check a random trusty node I have held	01:45
clarkb	yup random trusty node i have held in hpcloud-b4 has 8GB ram exposed to it	01:45
ianw	fungi / clarkb: if you can get me an equivalent centos / hp node, i can re-run the test and we can compare the output ...	01:45
ianw	clarkb: but was it for sure fixed for centos too?	01:46
*** bhunter71 has joined #openstack-infra		01:46
ianw	because if it really had more ram, that would hide the problem	01:46
clarkb	ianw: I do not have any centos7 ndes in hpcloud currently to check	01:46
fungi	i recall checking one after the change went into effect and it was correct	01:47
fungi	no guarantees that it still is of course	01:48
*** markvoelker has joined #openstack-infra		01:49
clarkb	though we dump that info in devstack logs somewhere I think	01:49
clarkb	ifwe can find logs for it we should be able to tell	01:50
*** sputnik13 has quit IRC		01:50
ianw	fungi: if you still have that window open on ubuntu, could you run $ "ps --sort -pmem -eo pid:10,pmem:6,cputime:10,nlwp:8,wchan:25,comm:25,args:100"	01:52
mordred	lifeless, dstufft: I win the setup_requires counts!	01:52
ianw	i'd like to compare to the fedora output and see if ordering is similar	01:52
jhesketh	anteaya: yep, and some devstack-gate logs... I /think/ if it fails the job will fail	01:52
jhesketh	anteaya: so there is a risk the code has a bug and it breaks a lot of jobs until we fix it	01:53
jhesketh	so possibly one to do when people are around to help fix	01:53
jhesketh	which I am, but I don't have root if we need to force something through	01:53
dstufft	mordred: your prize is you get to stare at setuptools code	01:53
clarkb	according to logstash we have not run any of these tests on hpcloud over the last week	01:53
lifeless	dstufft: +1	01:53
ianw	i'm still getting periodic telemetry back from the poor dying rackspace host; nothing is going crazy still	01:53
clarkb	ianw: fungi so I think the issue is likely more due to some change that bumped us over a limit less reliably in the past but is now reliable	01:53
*** markvoelker has quit IRC		01:53
* mordred hands lifeless a cookie, braces for alpaca herd		01:54
anteaya	jhesketh: okay good to know	01:54
fungi	ianw: error: unknown user-defined format specifier "ps"	01:54
anteaya	jhesketh: what do you want to do?	01:54
mixpix3ls	mordred sure likes his cookies ;)	01:54
clarkb	ianw: fungi: so likely that hpcloud isn't involved at all	01:54
lifeless	mordred: I'd rather a steak	01:54
anteaya	mixpix3ls: and damp wildlife	01:54
mixpix3ls	lol!	01:54
fungi	ianw: it also dislikes multiple sort options	01:54
*** annegentle has joined #openstack-infra		01:55
*** Hunner has quit IRC		01:55
mordred	lifeless: you will eat the cookie and you will like it	01:55
anteaya	jhesketh: I'm not for breaking stuff taht is working	01:55
clarkb	going back a bit further to the limits of what logstash has stored we don't have any jobs on hpcloud	01:55
ianw	urgh, ps is never ps everywhere	01:55
mordred	mixpix3ls: :)	01:55
fungi	ianw: nevermind--my middle-click bounced	01:55
lifeless	mordred: poison. Nevah.	01:55
anteaya	jhesketh: at least not without a plan	01:55
mixpix3ls	mordred, haha, good stuff.	01:55
* mordred dangles a lovely port skirt steak in front of lifeles		01:55
mixpix3ls	Wow, Alpacas look like Llamas.	01:55
fungi	ianw: http://paste.openstack.org/show/196778	01:55
* lifeless salivates		01:56
lifeless	bb after lunch	01:56
anteaya	clarkb: what is your feeling on https://review.openstack.org/#/c/167046/1, if I merge and there is a bug it might break a fair few jobs	01:56
clarkb	anteaya: yes it may but only after the next image rebuild	01:56
mordred	lifeless: http://s7d9.scene7.com/is/image/LaTienda/ip-02?defaultImage=placeholder_460x460&$medium_460$	01:56
anteaya	clarkb: so you are okay if I merge?	01:56
jhesketh	anteaya: I think we see if clarkb or fungi are going to be around during the next image rebuilds?	01:57
lifeless	mordred: hell yes	01:57
anteaya	jhesketh: good plan	01:57
clarkb	anteaya: well I won't be in a great position the next couple days to babysit or fix any major problems. So I don't want to commit to anything	01:57
anteaya	they usually happen at what 0600?	01:57
clarkb	anteaya: 1400UTC	01:57
anteaya	clarkb: okay good to know	01:57
clarkb	7am PST	01:57
anteaya	ah	01:57
clarkb	instead I will be tlaking zuul with krotscheck	01:58
anteaya	fungi: are you willing to fix the world if I break it by merging https://review.openstack.org/#/c/167046/1 ?	01:58
anteaya	clarkb: cool	01:58
ianw	fungi: thanks; nothing seems totally out of order with http://paste.openstack.org/show/196769/	01:58
anteaya	I can't even picturing tlaking zuul, but I hope it is enjoyable	01:58
anteaya	and makes krotscheck happy	01:58
fungi	anteaya: jhesketh: sure, i'll be here and awake when they start	01:59
anteaya	fungi: great thank you	01:59
* anteaya approves		01:59
jhesketh	:-)	01:59
anteaya	my firefox keeps crashing	01:59
anteaya	no idea why yet	01:59
anteaya	it maybe a harbinger of tomorrow though I hope not	02:00
anteaya	at least for that patch	02:00
*** annegentle has quit IRC		02:00
pleia2	mixpix3ls: yeah, they're like small llamas :)	02:00
*** tonytan4ever has joined #openstack-infra		02:01
mordred	pleia2:	02:02
mordred	gah	02:02
anteaya	jhesketh: I didn't approve any of the rest of the project-config patches because the devstack-gate patch hasn't gone in yet: https://review.openstack.org/#/c/167143/	02:02
anteaya	I don't like to approve depends-on changes too early since sometimes one needs time to evaluate one merge at a time	02:03
anteaya	rather than a stampede	02:03
*** trey has quit IRC		02:03
jhesketh	anteaya: that's cool	02:03
anteaya	thought I'd share my thinking with you	02:04
jhesketh	it's good thinking ;-)	02:04
anteaya	thanks	02:04
anteaya	:)	02:04
*** trey has joined #openstack-infra		02:05
*** tonytan4ever has quit IRC		02:06
*** sdake__ has joined #openstack-infra		02:06
*** dalgaaf has quit IRC		02:07
openstackgerrit	Merged openstack-infra/project-config: zuul_swift_upload add support for appending help https://review.openstack.org/167046	02:07
*** sarob has joined #openstack-infra		02:08
*** patrickeast has quit IRC		02:09
*** achanda has quit IRC		02:09
*** spzala has joined #openstack-infra		02:10
*** ivar-laz_ has quit IRC		02:10
*** sdake has quit IRC		02:11
*** thingee has quit IRC		02:11
*** otter768 has joined #openstack-infra		02:12
*** sarob has quit IRC		02:13
*** HeOS has joined #openstack-infra		02:13
*** gokrokve has joined #openstack-infra		02:16
*** otter768 has quit IRC		02:17
*** sarob has joined #openstack-infra		02:17
*** Somay has quit IRC		02:17
*** HeOS has quit IRC		02:18
*** hdd has quit IRC		02:18
*** sigmavirus24 is now known as sigmavirus24_awa		02:22
*** kaisers1 has joined #openstack-infra		02:24
*** kaisers has quit IRC		02:24
*** sdake has joined #openstack-infra		02:25
*** sdake__ has quit IRC		02:28
*** HeOS has joined #openstack-infra		02:29
*** davide has joined #openstack-infra		02:30
*** sarob has quit IRC		02:31
*** mixpix3ls has quit IRC		02:33
*** HeOS has quit IRC		02:33
ianw	fungi / anyone who can log into a host ... i'm narrowing down on swift. i'd appreciate if i could get a /proc/pid/maps for swift on one of the running ubuntu hosts	02:34
*** davide has quit IRC		02:34
*** HeOS has joined #openstack-infra		02:34
fungi	ianw: i'll give it a shot	02:35
*** Longgeek has joined #openstack-infra		02:35
*** gokrokve has quit IRC		02:36
ianw	fungi: see https://etherpad.openstack.org/p/oom-in-rax-centos7-CI-job	02:37
ianw	everything on centos uses a bit more, but swift is double	02:37
ianw	i wonder if we've done some python import of some big library only on centos	02:38
*** HeOS has quit IRC		02:38
*** sdake__ has joined #openstack-infra		02:39
*** dims has quit IRC		02:39
fungi	ianw: ugh, i count 29 swift processes. which ones specifically?	02:39
fungi	object servers? account servers? container servers?	02:40
*** sdake__ has quit IRC		02:40
fungi	sync, replicator, auditor, updater...	02:40
ianw	fungi: yep; they're all bigger on centos	02:40
fungi	reaper (that's a great one)	02:40
*** Longgeek has quit IRC		02:41
ianw	maybe just pick two and paste maps & smaps	02:41
*** sdake has quit IRC		02:41
mordred	notmyname: ^^	02:42
mordred	notmyname: that seems like one of those things you'd be interested in knowing in your brain	02:42
*** annegentle has joined #openstack-infra		02:42
*** enikanorov__ has joined #openstack-infra		02:43
fungi	ianw: first randomly-selected process http://paste.openstack.org/show/196779 and http://paste.openstack.org/show/196780	02:43
fungi	ianw: second randomly-selected process http://paste.openstack.org/show/196781 and http://paste.openstack.org/show/196782	02:44
clarkb	ARG! why are libvirt feature flag names different than the flag names in /proc/cpuinfo	02:44
ianw	fungi: awesome; thanks. now have to figure out how to get centos comparison :)	02:44
*** gokrokve has joined #openstack-infra		02:45
*** enikanorov has quit IRC		02:45
*** gokrokve has quit IRC		02:45
ianw	fungi: if you've got time, maybe we can kill that running host and i can start a new one. i'll stop it before tempest and see if swift is looking twice as big still	02:45
fungi	ianw: sure, takes just a sec	02:46
mordred	clarkb: because	02:46
mordred	clarkb: the /proc/cpuinfo flags made lennart unhappy	02:46
clarkb	mordred: I want that to not be true but I am afraid to find out	02:47
clarkb	I am going to try major hacks	02:47
mordred	clarkb: you're still trying to deal with the cpu stuff aren't you?	02:47
lifeless	cpu stuff?	02:48
ianw	fungi: also if you remember which two pids they were maybe add to https://etherpad.openstack.org/p/oom-in-rax-centos7-CI-job so we can do 1:1 comparison	02:48
fungi	ianw: ssh jenkins@104.130.30.211	02:49
fungi	ianw: i don't recall but i'll see if i can spot them in the ps	02:49
*** markvoelker has joined #openstack-infra		02:49
clarkb	lifeless: yes I am trying to make nova migrations work with 64bit libvirt cpu	02:50
clarkb	lifeless: unfortunately our clouds are not homogenous hardware so we need to construct our own cpu model to do that	02:50
clarkb	lifeless: and it would be so incredibly helpful if libvirt used the same strings as /proc/cpuinfo	02:51
fungi	ianw: ps lines pasted into the etherpad corresponding to those	02:51
lifeless	clarkb: righto - considered asking on -ops ? Folk may know	02:51
clarkb	-ops?	02:52
lifeless	openstack-operators	02:52
clarkb	oh I mean libvirt is silly and does not do this	02:52
lifeless	you know, the folk that deploy openstack	02:52
clarkb	I know that	02:52
*** tiswanso has quit IRC		02:52
lifeless	:)	02:52
clarkb	anyways I will have code we can all laugh at shortly	02:52
fungi	ianw: heading to bed momentarily, but others in here may be able to help or i can pick this back up tomorrow if you need anything else	02:53
*** harlowja is now known as harlowja_away		02:53
openstackgerrit	Clark Boylan proposed openstack-infra/devstack-gate: Try to calculate common CPU feature set https://review.openstack.org/163514	02:53
ianw	fungi: thanks ... i think i've got enough. i'm getting devstack up on that new host and we can try to figure out why swift is taking up twice as much memory. i feel like there's been pretty good progress	02:54
clarkb	https://review.openstack.org/#/c/163514/2/functions.sh comment on lines 872/873 addresses this	02:54
clarkb	lifeless: ^	02:54
*** markvoelker has quit IRC		02:54
clarkb	mostly I just want to prove this is possible in some capacity, then I can bother with making the code less incredibly hackish	02:55
mordred	lifeless: to be fair to the operators, most of the probably would not attempt to do the thing that we need to do	02:55
lifeless	mordred: testing deployments in virt? Might be surprised :)	02:55
clarkb	https://jenkins07.openstack.org/job/check-tempest-dsvm-aiopcpu-full/25/ is the test that should tell us how broken my code is	02:56
*** hdd has joined #openstack-infra		02:56
*** dhp has quit IRC		02:57
*** dhp has joined #openstack-infra		02:57
clarkb	one thing I think we should consider putting in nova is autonegotiation of migration domains	02:58
clarkb	I should write a spec. But novas job is to figure this stuff out for the operator	02:58
*** HeOS has joined #openstack-infra		03:02
*** amitgandhinz has quit IRC		03:03
nibalizer	mordred: lol	03:04
nibalizer	there is no way to explicitly ask for node scope	03:04
nibalizer	lolfucked	03:04
lifeless	mordred: not helping!	03:06
*** tqtran has quit IRC		03:06
*** [HeOS] has joined #openstack-infra		03:06
*** HeOS has quit IRC		03:07
clarkb	oh hrm this isn't going to work because libvirt is installed by devstack later	03:07
*** achanda has joined #openstack-infra		03:10
*** [HeOS] has quit IRC		03:11
*** HeOS has joined #openstack-infra		03:12
*** zz_dimtruck is now known as dimtruck		03:13
*** xyang1 has quit IRC		03:14
openstackgerrit	Clark Boylan proposed openstack-infra/devstack-gate: Try to calculate common CPU feature set https://review.openstack.org/163514	03:14
*** [HeOS] has joined #openstack-infra		03:15
clarkb	looks like an alternative would be to have nova construct the model from scratch? but I don't think nova supports that today	03:16
*** HeOS has quit IRC		03:16
*** achanda has quit IRC		03:17
*** ddieterly has quit IRC		03:18
mordred	lifeless: aroo?	03:19
openstackgerrit	Jerry Zhao proposed openstack-infra/nodepool: add option to use ipv6 for image update and node launching https://review.openstack.org/156178	03:19
clarkb	I wonder if we could ask rackspace to give us homogenous cpus >_>	03:20
*** [HeOS] has quit IRC		03:20
*** HeOS has joined #openstack-infra		03:21
*** tkelsey has joined #openstack-infra		03:22
*** ajmiller has joined #openstack-infra		03:24
*** annegentle has quit IRC		03:24
*** HeOS has quit IRC		03:25
*** carl_baldwin has joined #openstack-infra		03:26
*** tkelsey has quit IRC		03:27
*** ivar-lazzaro has joined #openstack-infra		03:31
*** ivar-lazzaro has quit IRC		03:32
*** dhritishikhar_ has joined #openstack-infra		03:32
*** ivar-lazzaro has joined #openstack-infra		03:32
*** bhunter71 has quit IRC		03:33
*** HeOS has joined #openstack-infra		03:36
*** baoli has quit IRC		03:37
*** achanda has joined #openstack-infra		03:39
*** baoli has joined #openstack-infra		03:39
ianw	clarkb: if you've got a minute, could you do "ldd /usr/local/lib/python2.7/dist-packages/cryptography/_Cryptography_cffi_f3e4673fx399b1113.so" on a running ubuntu?	03:39
*** ivar-lazzaro has quit IRC		03:40
*** HeOS has quit IRC		03:40
clarkb	ianw: http://paste.openstack.org/show/196791/	03:41
clarkb	ianw: thats a dib built image on hpcloud	03:41
lifeless	mordred: the pip discussion, of course :)	03:41
*** tsg has joined #openstack-infra		03:42
ianw	clarkb: so i think i've found some of the problem here -> http://paste.openstack.org/show/196792/	03:42
ianw	on centos we're bringing in the kitchen sink	03:42
*** HeOS has joined #openstack-infra		03:43
mordred	lifeless: bah. I'm helping bring either clarity of rage or confusion or apathy	03:44
mordred	lifeless: I have to at least be helping with ONE of those things	03:44
*** HeOS has quit IRC		03:44
openstackgerrit	Steve Kowalik proposed openstack-infra/puppet-zanata: Add a zanata::client manifest https://review.openstack.org/167081	03:45
*** dannywilson has joined #openstack-infra		03:46
openstackgerrit	Clark Boylan proposed openstack-infra/devstack-gate: Try to calculate common CPU feature set https://review.openstack.org/163514	03:46
*** Sukhdev has joined #openstack-infra		03:47
clarkb	ianw: I wonder if the cffi build is super opportunistic and if all those libs exist it happily thrwos them into crytpography	03:48
ianw	clarkb: that's what i'm thinking...	03:48
clarkb	in particular kerberos things and selinux won't be on ubuntu iirc	03:48
ianw	also, it's no surprise that i think we started building our own crypto from pip recently, instead of packaged...	03:49
openstackgerrit	Steve Kowalik proposed openstack-infra/system-config: Use zanata::client in proposal slave https://review.openstack.org/166109	03:49
clarkb	ianw: oh good point, if that was removed from the distro package list we would do exactly that	03:49
*** HeOS has joined #openstack-infra		03:50
*** markvoelker has joined #openstack-infra		03:50
ianw	hmm looks like the packaged cryptography brings in about the same	03:51
*** sdake has joined #openstack-infra		03:52
*** dims has joined #openstack-infra		03:52
*** markvoelker has quit IRC		03:55
clarkb	cryptography has a severe lack of comments and its too late in the evening for me to parse its build setup	03:57
clarkb	however I don't think it directly wants to pull in krb5 and selinux things. May be transitive	03:57
*** dannywilson has quit IRC		03:58
*** dannywilson has joined #openstack-infra		03:59
ianw	yeah, looking closer at smaps, hardly any of it is mapped in	04:01
*** ghostpl_ has joined #openstack-infra		04:01
ianw	the difference in size is heap usage	04:01
*** sdake has quit IRC		04:01
*** nijaba has quit IRC		04:01
*** sdake has joined #openstack-infra		04:02
*** jerryz has quit IRC		04:04
*** otter768 has joined #openstack-infra		04:05
*** jerryz has joined #openstack-infra		04:05
*** ghostpl_ has quit IRC		04:06
*** Longgeek has joined #openstack-infra		04:07
*** davide has joined #openstack-infra		04:07
*** nijaba has joined #openstack-infra		04:08
clarkb	https://github.com/pyca/cryptography/blob/master/src/cryptography/hazmat/bindings/utils.py#L44-L100 is neat in its simplicity. Its basically a giant c file construced on the fly then built	04:10
lifeless	WCPGW	04:13
*** Longgeek has quit IRC		04:14
*** dims has quit IRC		04:14
openstackgerrit	Khai Do proposed openstack-infra/jenkins-job-builder: Allow JJB to set connection timeout to jenkins server https://review.openstack.org/151472	04:15
*** dhritishikhar_ has quit IRC		04:15
*** ddieterly has joined #openstack-infra		04:18
openstackgerrit	Clark Boylan proposed openstack-infra/devstack-gate: Try to calculate common CPU feature set https://review.openstack.org/163514	04:22
*** ajmiller has quit IRC		04:22
*** ddieterly has quit IRC		04:24
*** achanda has quit IRC		04:30
*** dimtruck is now known as zz_dimtruck		04:32
*** carl_baldwin has quit IRC		04:32
*** yamamoto_ has joined #openstack-infra		04:32
*** achanda has joined #openstack-infra		04:34
openstackgerrit	Thanh Ha proposed openstack-infra/nodepool: Clarify meaning of Jenkins master configuration settings in target https://review.openstack.org/167878	04:36
*** sdake has quit IRC		04:38
openstackgerrit	Thanh Ha proposed openstack-infra/nodepool: Clarify project-id meaning for Rackspace deployment https://review.openstack.org/167881	04:42
*** YorikSar has quit IRC		04:47
*** pal_ has joined #openstack-infra		04:48
openstackgerrit	Khai Do proposed openstack-infra/jenkins-job-builder: Added support for JClouds Blobstore https://review.openstack.org/107515	04:50
*** markvoelker has joined #openstack-infra		04:51
*** baoli has quit IRC		04:53
*** fifieldt has joined #openstack-infra		04:53
*** markvoelker has quit IRC		04:56
*** yamamoto_ has quit IRC		04:56
*** dannywilson has quit IRC		04:57
openstackgerrit	Steve Kowalik proposed openstack-infra/puppet-zanata: Add a zanata::client manifest https://review.openstack.org/167081	05:02
*** asettle has joined #openstack-infra		05:04
*** davide has quit IRC		05:04
*** craigbr has quit IRC		05:05
openstackgerrit	Merged openstack-infra/jenkins-job-builder: Adding more configurable options in Notifications plugin https://review.openstack.org/163137	05:08
*** Longgeek has joined #openstack-infra		05:11
*** ibiris_away is now known as ibiris		05:16
*** ddieterly has joined #openstack-infra		05:20
*** hdd has quit IRC		05:21
*** hdd has joined #openstack-infra		05:22
*** Longgeek has quit IRC		05:22
*** ddieterly has quit IRC		05:25
*** yfried\|afk is now known as yfried		05:25
*** esker has joined #openstack-infra		05:30
*** hichihara has quit IRC		05:31
*** sputnik13 has joined #openstack-infra		05:32
*** sarob has joined #openstack-infra		05:33
*** BharatK has joined #openstack-infra		05:36
openstackgerrit	Steve Kowalik proposed openstack-infra/puppet-zanata: Add a zanata::client manifest https://review.openstack.org/167081	05:36
*** r-daneel has joined #openstack-infra		05:36
*** doude has quit IRC		05:37
*** sarob has quit IRC		05:40
*** sarob has joined #openstack-infra		05:40
*** r-daneel has quit IRC		05:41
*** jerryz has quit IRC		05:43
*** ibiris is now known as ibiris_away		05:45
*** Sukhdev has quit IRC		05:47
*** YorikSar has joined #openstack-infra		05:47
*** sarob has quit IRC		05:48
*** markvoelker has joined #openstack-infra		05:52
*** markvoelker has quit IRC		05:56
*** yfried is now known as yfried\|afk		05:56
*** yamamoto_ has joined #openstack-infra		05:57
*** yfried\|afk is now known as yfried		05:58
*** reed has quit IRC		05:58
*** asettle has quit IRC		05:58
*** mrmartin has joined #openstack-infra		06:01
openstackgerrit	OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/167889	06:01
*** unicell has joined #openstack-infra		06:03
*** yamamoto_ has quit IRC		06:03
*** armax has quit IRC		06:07
*** stevemar has quit IRC		06:09
*** yfried has quit IRC		06:09
*** stevemar has joined #openstack-infra		06:09
*** yamamoto_ has joined #openstack-infra		06:11
*** mmedvede has quit IRC		06:12
*** dims has joined #openstack-infra		06:12
*** macjack has joined #openstack-infra		06:14
*** macjack has quit IRC		06:16
*** jamielennox is now known as jamielennox\|away		06:16
*** hdd has quit IRC		06:16
*** vigneshvar has joined #openstack-infra		06:17
*** sputnik13 has quit IRC		06:19
*** ddieterly has joined #openstack-infra		06:20
*** ddieterly has quit IRC		06:25
*** otter768 has quit IRC		06:29
*** achanda has quit IRC		06:31
*** dims has quit IRC		06:32
*** ibiris_away is now known as ibiris		06:36
*** achanda has joined #openstack-infra		06:39
*** thingee has joined #openstack-infra		06:39
*** chlong has quit IRC		06:45
*** ibiris is now known as ibiris_away		06:48
*** davide has joined #openstack-infra		06:52
*** markvoelker has joined #openstack-infra		06:52
*** yfried has joined #openstack-infra		06:53
*** alexpilotti has joined #openstack-infra		06:56
*** markvoelker has quit IRC		06:57
*** davide has quit IRC		06:57
*** yfried has quit IRC		06:58
*** yfried has joined #openstack-infra		06:58
*** stevemar has quit IRC		06:58
*** ibiris_away is now known as ibiris		07:05
openstackgerrit	Lukas Bednar proposed openstack-infra/jenkins-job-builder: publishers: groovy-postbuild changes between 1.x & 2.x https://review.openstack.org/134276	07:06
*** Longgeek has joined #openstack-infra		07:08
*** stevemar has joined #openstack-infra		07:09
*** achanda has quit IRC		07:13
*** HeOS has quit IRC		07:14
*** yfried is now known as yfried\|afk		07:14
*** yamamoto_ has quit IRC		07:14
*** achanda has joined #openstack-infra		07:15
*** scheuran has joined #openstack-infra		07:15
*** mrunge has joined #openstack-infra		07:19
*** spzala has quit IRC		07:19
*** Longgeek has quit IRC		07:20
*** tnovacik has joined #openstack-infra		07:20
*** ddieterly has joined #openstack-infra		07:21
*** stevemar has quit IRC		07:21
*** achanda has quit IRC		07:24
*** sandywalsh has quit IRC		07:25
*** ddieterly has quit IRC		07:26
*** dhritishikhar_ has joined #openstack-infra		07:26
*** sergsh has joined #openstack-infra		07:27
*** yamahata has quit IRC		07:34
*** sandywalsh has joined #openstack-infra		07:34
*** jyuso has joined #openstack-infra		07:35
*** pblaho has joined #openstack-infra		07:37
*** yfried\|afk is now known as yfried		07:41
*** jyuso1 has quit IRC		07:44
*** jyuso1 has joined #openstack-infra		07:44
*** tkelsey has joined #openstack-infra		07:46
*** Ala has joined #openstack-infra		07:48
*** jyuso1 has quit IRC		07:49
openstackgerrit	Timur Nurlygayanov proposed openstack-infra/project-config: Add HAOS project into Stackforge https://review.openstack.org/167813	07:50
*** thingee has quit IRC		07:51
*** Bsony has joined #openstack-infra		07:52
*** markvoelker has joined #openstack-infra		07:53
*** dannywilson has joined #openstack-infra		07:57
*** markvoelker has quit IRC		07:58
*** ekarlso has quit IRC		07:59
*** dannywilson has quit IRC		08:02
*** rm_work is now known as rm_work\|away		08:03
*** e0ne has joined #openstack-infra		08:09
*** sandywalsh_ has joined #openstack-infra		08:12
*** sandywalsh has quit IRC		08:12
*** markus_z has joined #openstack-infra		08:14
*** hashar has joined #openstack-infra		08:14
*** hashar_ has joined #openstack-infra		08:14
*** sobersabre has joined #openstack-infra		08:15
*** dims has joined #openstack-infra		08:17
*** fifieldt has quit IRC		08:20
*** woodster_ has quit IRC		08:20
*** ddieterly has joined #openstack-infra		08:22
*** yfried is now known as yfried\|afk		08:24
*** dims has quit IRC		08:25
*** e0ne has quit IRC		08:25
*** ddieterly has quit IRC		08:27
*** jistr has joined #openstack-infra		08:27
*** ibiris is now known as ibiris_away		08:27
*** e0ne has joined #openstack-infra		08:29
*** yfried\|afk is now known as yfried		08:30
*** ibiris_away is now known as ibiris		08:30
*** otter768 has joined #openstack-infra		08:30
*** HeOS has joined #openstack-infra		08:31
*** e0ne has quit IRC		08:33
*** arxcruz has joined #openstack-infra		08:34
*** otter768 has quit IRC		08:35
*** mpaolino has joined #openstack-infra		08:35
openstackgerrit	yolanda.robla proposed openstack-infra/system-config: Expose jenkins_gitfullname and jenkins_gitemail https://review.openstack.org/161695	08:37
*** dims has joined #openstack-infra		08:38
*** jlanoux has joined #openstack-infra		08:41
*** andreykurilin_ has joined #openstack-infra		08:46
*** pal_ has quit IRC		08:46
*** tnovacik has quit IRC		08:47
*** ociuhandu has joined #openstack-infra		08:47
*** oomichi_ has quit IRC		08:50
*** pal_ has joined #openstack-infra		08:50
*** bradjones has quit IRC		08:51
*** MaxV has joined #openstack-infra		08:53
*** ibiris is now known as ibiris_away		08:54
*** markvoelker has joined #openstack-infra		08:54
*** bradjones has joined #openstack-infra		08:55
*** bradjones has quit IRC		08:55
*** bradjones has joined #openstack-infra		08:55
*** ildikov has quit IRC		08:56
*** markvoelker has quit IRC		08:58
*** pal__ has joined #openstack-infra		09:00
*** zz_johnthetubagu is now known as johnthetubaguy		09:01
*** pal_ has quit IRC		09:02
*** Longgeek has joined #openstack-infra		09:02
*** ibiris_away is now known as ibiris		09:06
*** Longgeek has quit IRC		09:07
openstackgerrit	yolanda.robla proposed openstack-infra/puppet-cgit: Add configurable setting for httpd.conf https://review.openstack.org/160885	09:07
*** e0ne has joined #openstack-infra		09:08
*** ekarlso has joined #openstack-infra		09:09
openstackgerrit	yolanda.robla proposed openstack-infra/puppet-cgit: Add configurable settings for httpd.conf https://review.openstack.org/160885	09:10
*** andreykurilin_ has quit IRC		09:11
*** hashar_ has quit IRC		09:15
*** alexpilotti has quit IRC		09:16
*** ociuhandu has quit IRC		09:18
*** yfried is now known as yfried\|afk		09:19
*** e0ne has quit IRC		09:21
*** skolekonov has joined #openstack-infra		09:22
*** derekh has joined #openstack-infra		09:22
*** ddieterly has joined #openstack-infra		09:23
*** dhritishikhar_ has quit IRC		09:23
*** pal_ has joined #openstack-infra		09:23
*** enikanorov__ has quit IRC		09:26
*** pal__ has quit IRC		09:26
*** ominakov has joined #openstack-infra		09:26
*** dtantsur\|afk is now known as dtantsur		09:26
openstackgerrit	Lukas Bednar proposed openstack-infra/jenkins-job-builder: Fixed broken links in documentation https://review.openstack.org/167925	09:27
*** ddieterly has quit IRC		09:28
*** yfried\|afk is now known as yfried		09:31
*** jerryz has joined #openstack-infra		09:34
openstackgerrit	yolanda.robla proposed openstack-infra/puppet-cgit: Add configurable settings for httpd.conf https://review.openstack.org/160885	09:34
*** vigneshvar has quit IRC		09:39
*** ihrachyshka has joined #openstack-infra		09:40
*** doude has joined #openstack-infra		09:41
*** yfried is now known as yfried\|afk		09:43
*** vigneshvar has joined #openstack-infra		09:43
*** dhritishikhar_ has joined #openstack-infra		09:47
*** bradjones has quit IRC		09:47
*** bradjones has joined #openstack-infra		09:51
*** bradjones has quit IRC		09:52
*** bradjones has joined #openstack-infra		09:52
*** dizquierdo has joined #openstack-infra		09:52
*** pelix has joined #openstack-infra		09:54
*** markvoelker has joined #openstack-infra		09:55
*** lucasagomes has joined #openstack-infra		09:56
lucasagomes	hi, if you have a time please take a look at https://review.openstack.org/#/c/163344/ , this add a job (non-voting) in the openstack/ironic-python-agent project to test the IPA ramdisk with the pxe_* drivers. It's already has 1 +2	09:56
*** Hal has joined #openstack-infra		09:58
*** Hal is now known as Guest44815		09:58
*** yfried\|afk is now known as yfried		09:59
*** markvoelker has quit IRC		09:59
openstackgerrit	Serg Melikyan proposed openstack-infra/project-config: Rename stackforge/murano to openstack/murano https://review.openstack.org/167934	10:01
openstackgerrit	yolanda.robla proposed openstack-infra/puppet-cgit: Add configurable settings for httpd.conf https://review.openstack.org/160885	10:01
*** ghostpl_ has joined #openstack-infra		10:01
*** pal_ has quit IRC		10:02
*** mkj_wk has quit IRC		10:02
*** mkj_wk has joined #openstack-infra		10:03
*** imcsk8 is now known as imcsk8\|zZz		10:07
*** e0ne has joined #openstack-infra		10:10
*** ssam2 has joined #openstack-infra		10:11
openstackgerrit	Serg Melikyan proposed openstack-infra/project-config: Rename stackforge/murano to openstack/murano https://review.openstack.org/167934	10:17
*** Longgeek has joined #openstack-infra		10:17
*** mpaolino has quit IRC		10:19
*** yfried is now known as yfried\|afk		10:22
*** ddieterly has joined #openstack-infra		10:24
*** rlandy has joined #openstack-infra		10:24
*** e0ne is now known as e0ne_		10:28
*** Longgeek has quit IRC		10:28
*** ddieterly has quit IRC		10:28
*** otter768 has joined #openstack-infra		10:31
*** e0ne_ is now known as e0ne		10:32
*** arxcruz has quit IRC		10:32
*** fifieldt has joined #openstack-infra		10:36
*** otter768 has quit IRC		10:36
*** arxcruz has joined #openstack-infra		10:36
*** afazekas has joined #openstack-infra		10:39
*** samueldmq-away is now known as samueldmq		10:39
*** yfried\|afk is now known as yfried		10:41
*** Bsony_ has joined #openstack-infra		10:49
*** pc_m has joined #openstack-infra		10:50
*** yfried is now known as yfried\|afk		10:51
openstackgerrit	Merged openstack/requirements: Raise cap for Django to allow 1.7 https://review.openstack.org/155353	10:52
*** Bsony has quit IRC		10:53
*** yfried\|afk is now known as yfried		10:55
*** markvoelker has joined #openstack-infra		10:56
*** ominakov has quit IRC		10:58
*** sslypushenko has quit IRC		10:58
*** mikeit has joined #openstack-infra		10:59
*** markvoelker has quit IRC		11:00
*** pal_ has joined #openstack-infra		11:00
*** yfried is now known as yfried\|afk		11:05
*** pal_ has quit IRC		11:05
*** mikeit has quit IRC		11:07
*** yamamoto has joined #openstack-infra		11:08
*** ajo is now known as ajo\|lunch		11:09
*** yfried\|afk is now known as yfried		11:14
openstackgerrit	Nikki Heald proposed openstack-infra/puppet-cgit: Add configurable settings for httpd.conf https://review.openstack.org/160885	11:16
*** saper_ has joined #openstack-infra		11:16
*** niska` has joined #openstack-infra		11:16
openstackgerrit	Davanum Srinivas (dims) proposed openstack-infra/devstack-gate: [DO-NOT-MERGE] Enable libvirt debug logs and core dumps https://review.openstack.org/167691	11:16
*** crinkle_ has joined #openstack-infra		11:17
*** ChuckC_ has joined #openstack-infra		11:17
*** imcsk8\|z1z has joined #openstack-infra		11:17
*** dims_ has joined #openstack-infra		11:18
*** mikap has joined #openstack-infra		11:19
*** mikap has joined #openstack-infra		11:19
*** tsg_ has joined #openstack-infra		11:19
*** primemin1sterp has joined #openstack-infra		11:19
*** tteggel_ has joined #openstack-infra		11:19
*** pelix has quit IRC		11:19
*** mordred_ has joined #openstack-infra		11:20
*** omnibus7 has joined #openstack-infra		11:21
*** barnaby has joined #openstack-infra		11:21
*** lifeless1 has joined #openstack-infra		11:21
*** pelix has joined #openstack-infra		11:21
*** bodepd_ has joined #openstack-infra		11:21
*** tsg has quit IRC		11:22
*** vhoward- has joined #openstack-infra		11:23
*** _ianychoi_ has joined #openstack-infra		11:23
*** claudiub has joined #openstack-infra		11:23
*** dhritishikhar_ has quit IRC		11:23
*** dansmith_ has joined #openstack-infra		11:24
*** tnovacik has joined #openstack-infra		11:24
*** ddieterly has joined #openstack-infra		11:24
*** yfried is now known as yfried\|afk		11:24
*** notnownikki has joined #openstack-infra		11:25
*** pblaho has quit IRC		11:25
*** zul has quit IRC		11:25
*** gnuoy has quit IRC		11:25
*** ianychoi has quit IRC		11:25
*** niska has quit IRC		11:25
*** imcsk8\|zZz has quit IRC		11:25
*** davidlenwell has quit IRC		11:25
*** bodepd has quit IRC		11:25
*** lifeless has quit IRC		11:25
*** tteggel has quit IRC		11:25
*** derekh has quit IRC		11:25
*** sergsh has quit IRC		11:25
*** ghostpl has quit IRC		11:25
*** xnox has quit IRC		11:25
*** tlbr has quit IRC		11:25
*** peterstac has quit IRC		11:25
*** blitzrage has quit IRC		11:25
*** wolsen has quit IRC		11:25
*** tristanC has quit IRC		11:25
*** rcarrill` has joined #openstack-infra		11:25
*** chlong has joined #openstack-infra		11:26
*** mpavone has joined #openstack-infra		11:27
*** ajo\|lunch is now known as ajo		11:28
*** derekh has joined #openstack-infra		11:29
*** sergsh has joined #openstack-infra		11:29
*** ghostpl has joined #openstack-infra		11:29
*** xnox has joined #openstack-infra		11:29
*** peterstac has joined #openstack-infra		11:29
*** tlbr has joined #openstack-infra		11:29
*** blitzrage has joined #openstack-infra		11:29
*** wolsen has joined #openstack-infra		11:29
*** tristanC has joined #openstack-infra		11:29
*** jgriffit1 has joined #openstack-infra		11:29
*** xnox has quit IRC		11:29
*** ghostpl has quit IRC		11:29
*** yamamoto has quit IRC		11:29
*** ddieterly has quit IRC		11:29
*** xnox has joined #openstack-infra		11:29
*** ghostpl has joined #openstack-infra		11:30
*** dims has quit IRC		11:30
*** ChuckC has quit IRC		11:30
*** bcourt has quit IRC		11:30
*** praveenkumar has quit IRC		11:30
*** mika has quit IRC		11:30
*** dhellmann has quit IRC		11:30
*** neoice has quit IRC		11:30
*** saper has quit IRC		11:30
*** rcarrillocruz has quit IRC		11:30
*** crinkle has quit IRC		11:30
*** vhoward has quit IRC		11:30
*** tchaypo has quit IRC		11:30
*** dansmith has quit IRC		11:30
*** primeministerp has quit IRC		11:30
*** jgriffith has quit IRC		11:30
*** mordred has quit IRC		11:30
*** anteaya has quit IRC		11:30
*** pc_m has quit IRC		11:30
*** jerryz has quit IRC		11:30
*** ekarlso has quit IRC		11:30
*** esker has quit IRC		11:30
*** ZZelle has quit IRC		11:30
*** fabo has quit IRC		11:30
*** morgabra has quit IRC		11:30
*** nikhil_k\|eto has quit IRC		11:30
*** freerunner has quit IRC		11:30
*** enikanorov_ has quit IRC		11:30
*** ianw has quit IRC		11:30
*** sambetts has quit IRC		11:30
*** mrodden has quit IRC		11:30
*** michchap has quit IRC		11:30
*** johnthetubaguy has quit IRC		11:30
*** seeg has quit IRC		11:30
*** bradm has quit IRC		11:30
*** SergK has quit IRC		11:30
*** cschwede has quit IRC		11:30
*** dosaboy has quit IRC		11:30
*** mwhudson has quit IRC		11:30
*** grue_pm has quit IRC		11:30
*** spredzy has quit IRC		11:30
*** zz_avozza has quit IRC		11:30
*** s0nea has quit IRC		11:30
*** igorbelikov has quit IRC		11:30
*** naotokl has quit IRC		11:30
*** mordred_ is now known as mordred		11:30
*** dansmith_ is now known as dansmith		11:30
*** mikap is now known as mika		11:30
*** tchaypo has joined #openstack-infra		11:32
*** chuck_ has joined #openstack-infra		11:33
*** gnuoy` has joined #openstack-infra		11:33
*** pblaho_ has joined #openstack-infra		11:33
*** neoice_ has joined #openstack-infra		11:33
*** dhellmann has joined #openstack-infra		11:33
*** ominakov has joined #openstack-infra		11:33
*** pc_m has joined #openstack-infra		11:33
*** jerryz has joined #openstack-infra		11:33
*** ekarlso has joined #openstack-infra		11:33
*** esker has joined #openstack-infra		11:33
*** ZZelle has joined #openstack-infra		11:33
*** fabo has joined #openstack-infra		11:33
*** morgabra has joined #openstack-infra		11:33
*** seeg has joined #openstack-infra		11:33
*** nikhil_k\|eto has joined #openstack-infra		11:33
*** freerunner has joined #openstack-infra		11:33
*** enikanorov_ has joined #openstack-infra		11:33
*** ianw has joined #openstack-infra		11:33
*** sambetts has joined #openstack-infra		11:33
*** mrodden has joined #openstack-infra		11:33
*** michchap has joined #openstack-infra		11:33
*** johnthetubaguy has joined #openstack-infra		11:33
*** bradm has joined #openstack-infra		11:33
*** SergK has joined #openstack-infra		11:33
*** cschwede has joined #openstack-infra		11:33
*** spredzy has joined #openstack-infra		11:33
*** dosaboy has joined #openstack-infra		11:33
*** mwhudson has joined #openstack-infra		11:33
*** grue_pm has joined #openstack-infra		11:33
*** zz_avozza has joined #openstack-infra		11:33
*** s0nea has joined #openstack-infra		11:33
*** igorbelikov has joined #openstack-infra		11:33
*** naotokl has joined #openstack-infra		11:33
*** dkliban_afk is now known as dkliban		11:34
*** ldnunes has joined #openstack-infra		11:36
*** serg_melikyan has joined #openstack-infra		11:36
serg_melikyan	Hi folks, we have an issue with PyPI mirrors on OpenStack CI	11:36
serg_melikyan	https://review.openstack.org/167583 - here is change that bumps version requirements for newly released python-muranoclient	11:37
serg_melikyan	Patch is submitted yesterday at 4 pm UTC+3	11:37
seeg	hello, concerning https://review.openstack.org/#/c/167152/ -- someone should set up gerrit now, right? :)	11:38
*** anteaya has joined #openstack-infra		11:38
serg_melikyan	Client was released hours before this patch, but jobs are still failing due to missing python-muranoclient 0.5.6	11:38
*** praveenkumar has joined #openstack-infra		11:39
serg_melikyan	https://pypi.python.org/pypi/python-muranoclient/0.5.6 - link to package on pypi	11:39
*** Somay has joined #openstack-infra		11:40
*** _nadya_ has joined #openstack-infra		11:42
*** tnovacik has quit IRC		11:44
*** yfried\|afk is now known as yfried		11:45
*** dhritishikhar_ has joined #openstack-infra		11:46
*** ominakov has quit IRC		11:52
*** AJaeger has quit IRC		11:52
*** Somay has quit IRC		11:53
*** sdake has joined #openstack-infra		11:54
*** dhritishikhar_ has quit IRC		11:55
*** arxcruz has quit IRC		11:55
*** markvoelker has joined #openstack-infra		11:56
*** sdake has quit IRC		11:59
*** sdake has joined #openstack-infra		12:01
*** markvoelker has quit IRC		12:01
*** julim has joined #openstack-infra		12:01
*** pal_ has joined #openstack-infra		12:07
*** markvoelker has joined #openstack-infra		12:08
*** jcoufal has joined #openstack-infra		12:09
*** vigneshvar has quit IRC		12:11
*** sandywalsh_ has quit IRC		12:12
*** Longgeek has joined #openstack-infra		12:13
jroll	gerrit bot is dead, in case y'all weren't aware	12:14
*** sandywalsh has joined #openstack-infra		12:14
*** yfried is now known as yfried\|afk		12:15
*** woodster_ has joined #openstack-infra		12:15
*** ZZelle has quit IRC		12:15
*** johnthetubaguy is now known as zz_johnthetubagu		12:16
*** ZZelle has joined #openstack-infra		12:16
*** zz_johnthetubagu is now known as johnthetubaguy		12:16
*** ildikov has joined #openstack-infra		12:16
*** rcarrill` is now known as rcarrillocruz		12:17
seeg	ping concerning https://review.openstack.org/#/c/167152/ -- should someone on openstack-infra side set up gerrit?	12:18
serg_melikyan	seeg: what do you mean by setup gerrit?	12:19
*** davide has joined #openstack-infra		12:19
seeg	well git review throws me Exception: Could not connect to gerrit at ssh://PK@review.openstack.org:29418/openstack/fuel-dev-tools.git so I'm guessing this should be setup by someone?	12:20
seeg	I'm trying to submit the cookiecutter patch	12:20
serg_melikyan	Btw, you guys have empty core team for this project	12:21
serg_melikyan	https://review.openstack.org/#/admin/groups/uuid-af8f5d2dfa9f156717f2dc3f18ab6095c47d5c69,info	12:21
serg_melikyan	No one will be able to approve submitted patches	12:21
*** Longgeek has quit IRC		12:21
serg_melikyan	seeg: and you are using wrong URL	12:21
serg_melikyan	seeg: git clone https://review.openstack.org/stackforge/fuel-dev-tools	12:21
samueldmq	and .git at the end	12:22
serg_melikyan	seeg: You are using openstack instead of stackforge	12:22
samueldmq	serg_melikyan, seeg git clone https://review.openstack.org/stackforge/fuel-dev-tools.git ;)	12:22
*** mmedvede has joined #openstack-infra		12:22
seeg	pf, yeah	12:22
seeg	right :)	12:22
serg_melikyan	samueldmq: it works either way with .git or without	12:23
seeg	though I have stackforge in git config	12:23
samueldmq	serg_melikyan, nvm ... yeah it works, sorry :/	12:23
*** dhritishikhar_ has joined #openstack-infra		12:23
*** lucasagomes is now known as lucas-hungry		12:23
*** davide has quit IRC		12:24
serg_melikyan	seeg: clone this repo from scratch and execute git review -s	12:24
seeg	OK, works: https://review.openstack.org/#/c/167968/ :)	12:25
serg_melikyan	seeg: heh ) https://review.openstack.org/167967 - I tested too :)	12:25
seeg	cool	12:25
serg_melikyan	seeg: happy development ;)	12:26
seeg	heh, thanks :)	12:26
*** yfried\|afk is now known as yfried		12:27
*** dhritishikhar_ has joined #openstack-infra		12:28
*** zz_dimtruck is now known as dimtruck		12:28
*** kgiusti has joined #openstack-infra		12:30
*** bknudson has quit IRC		12:30
serg_melikyan	seeg: I am not sure how now days you request to create group in gerrit with initial member, but you must to have at least one member in the core group of your project	12:32
*** otter768 has joined #openstack-infra		12:32
seeg	serg_melikyan, I know -- I asked slukjanov from Mirantis to set this up as i heard he can do it	12:33
*** aysyd has joined #openstack-infra		12:33
serg_melikyan	seeg: ok :)	12:34
*** ominakov has joined #openstack-infra		12:35
*** serg_melikyan has quit IRC		12:36
*** cdent has joined #openstack-infra		12:37
*** otter768 has quit IRC		12:37
*** sslypushenko has joined #openstack-infra		12:37
*** gordc has joined #openstack-infra		12:38
*** mriedem_away is now known as mriedem		12:39
*** rfolco has joined #openstack-infra		12:39
*** dkranz has joined #openstack-infra		12:42
*** ominakov has quit IRC		12:43
*** ominakov has joined #openstack-infra		12:43
*** ociuhandu has joined #openstack-infra		12:43
*** spredzy is now known as spredzy\|afk		12:44
*** serg_melikyan has joined #openstack-infra		12:45
*** ociuhandu has quit IRC		12:45
*** doug-fish has joined #openstack-infra		12:46
fungi	zuul looks unhappy	12:47
cdent	yes	12:47
*** e0ne is now known as e0ne_		12:47
fungi	we seem to have ceased running jobs about an hour ago	12:47
fungi	checking its logs now for signs of badnez	12:47
*** baoli has joined #openstack-infra		12:47
*** shardy has joined #openstack-infra		12:49
*** toabctl has quit IRC		12:50
*** tnovacik has joined #openstack-infra		12:50
fungi	there's a few tracebacks in the log but i think most are benign. however, this one is different, roughly coincides with when jobs seem to have stopped being added, and just generally looks not great http://paste.openstack.org/show/196854/	12:52
*** dboik_ has quit IRC		12:52
*** smccully has joined #openstack-infra		12:53
mordred	fungi: I agree - that does not look good	12:54
*** toabctl has joined #openstack-infra		12:54
fungi	i'm going to get a couple stack dumps and then restart zuul	12:54
cdent	gearmans gears are grinding	12:54
sdague	fungi: yeh, that's the weird	12:54
sdague	fungi: so an hour ago I was approving come project-config changes	12:54
sdague	I wonder if one of those actually makes zuul explode, and we did not realize it	12:54
*** annegentle has joined #openstack-infra		12:55
fungi	sdague: great idea, could be a layout config change did something	12:55
fungi	i'll dig deeper into post-mortem once it's up and running again	12:55
*** alexpilotti has joined #openstack-infra		12:55
*** adalbas has joined #openstack-infra		12:56
*** e0ne_ is now known as e0ne		12:57
*** jaypipes has quit IRC		12:57
*** dprince has joined #openstack-infra		12:58
*** arxcruz has joined #openstack-infra		13:01
*** xyang1 has joined #openstack-infra		13:02
fungi	hrm, that traceback is actually a few hours earlier than when we saw jobs stop getting triggered, so possibly entirely unrelated	13:03
*** eharney has quit IRC		13:03
fungi	also last update to layout.yaml was 16 hours ago, so probably not triggered by a change to that	13:04
fungi	restarting zuul and then rechecking a change does not appear to have triggered anything	13:05
fungi	this issue may be on the gerrit end	13:05
fungi	checking its queues now	13:05
*** jaypipes has joined #openstack-infra		13:05
*** jaypipes_ has joined #openstack-infra		13:06
*** ociuhandu has joined #openstack-infra		13:07
*** dkranz has quit IRC		13:07
*** afazekas has quit IRC		13:07
fungi	there are a lot of waiting stream events tasks in the gerrit queue	13:08
fungi	with start times of ~11:36 utc	13:08
fungi	which does look like it corresponds to when the trouble started	13:08
fungi	not sure if those are a cause or an effect	13:08
*** e0ne is now known as e0ne_		13:09
fungi	i'm going to restart gerrit now, and then go digging in its ssh log to see if there's a misbehaving client we need to firewall	13:09
*** yfried_ has joined #openstack-infra		13:09
*** dtantsur is now known as dtantsur\|brb		13:09
cdent	fungi: when these kinds of things happen, do you guys do post mortems and post them anywhere? I’m always curious and it really helps to understand how it works and all the pieces fit together.	13:10
*** sigmavirus24_awa is now known as sigmavirus24		13:10
*** prmtl has joined #openstack-infra		13:10
fungi	#status notice gerrit stopped emitting stream events around 11:30 utc and has now been restarted. please recheck any changes currently missing results from jenkins	13:11
openstackstatus	fungi: sending notice	13:11
*** ociuhandu_ has joined #openstack-infra		13:11
fungi	cdent: depends on how protracted the incident was	13:11
*** ociuhandu has quit IRC		13:11
*** ociuhandu_ is now known as ociuhandu		13:11
*** annegentle has quit IRC		13:11
* cdent nods		13:12
-openstackstatus- NOTICE: gerrit stopped emitting stream events around 11:30 utc and has now been restarted. please recheck any changes currently missing results from jenkins		13:12
mordred	fungi: well that was exciting	13:12
*** yfried has quit IRC		13:12
*** dboik has joined #openstack-infra		13:13
*** dboik has quit IRC		13:13
*** AJaeger has joined #openstack-infra		13:13
*** AJaeger has joined #openstack-infra		13:13
openstackstatus	fungi: finished sending notice	13:13
*** dboik has joined #openstack-infra		13:13
fungi	mordred: morning's always fun. i never know what's waiting for me when i wake up	13:13
*** yfried__ has joined #openstack-infra		13:14
fungi	lookie there... jobs are starting now	13:14
*** ildikov has quit IRC		13:14
samueldmq	fungi, wow, this openstackstatus is nice, it notifies you when it finishes your request o/	13:14
fungi	samueldmq: i agree. want the source?	13:14
samueldmq	fungi, sure, I am curious	13:14
fungi	it can change channel topics too when things are really bad	13:14
*** tiswanso has joined #openstack-infra		13:14
*** dustins has joined #openstack-infra		13:15
fungi	we tend to just use notice to give people a heads up, but for longer outages and maintenances we use alert/ok to set and revert the channel topics too	13:15
samueldmq	fungi, nice, and who are authorized to do so ? anyone in the core?	13:15
*** annegentle has joined #openstack-infra		13:16
*** yfried_ has quit IRC		13:16
fungi	samueldmq: the source for statusbot and our puppet-statusbot module are linked at the top of http://ci.openstack.org/irc.html	13:17
*** krtaylor has quit IRC		13:17
fungi	samueldmq: there's also a section on that page with basic usage	13:17
*** Longgeek has joined #openstack-infra		13:18
*** e0ne_ has quit IRC		13:19
fungi	samueldmq: right now its config parameters are embedded at https://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n263	13:19
*** Bsony_ has quit IRC		13:20
serg_melikyan	folks, can you also check-up on OpenStack CI PyPI mirrors?	13:20
samueldmq	fungi, great	13:20
samueldmq	fungi, taking a look at that ...	13:20
fungi	samueldmq: so there you can see the list of nicks which are authorized to interact with it (as long as they're authenticated to the irc network) and the channels it lurks in	13:20
samueldmq	fungi, when do those config params are replaced?	13:20
serg_melikyan	We released python-muranoclient 0.5.6 day ago, but pep-8 jobs are still can't find this dependency	13:20
samueldmq	fungi, the passwd, etc	13:20
fungi	serg_melikyan: i'll take a look	13:20
*** dkranz has joined #openstack-infra		13:20
samueldmq	fungi, ah I see statusbot_auth_nicks, nice	13:21
serg_melikyan	fungi: https://review.openstack.org/167583 - example change-request	13:21
fungi	samueldmq: the ones which are calling out to hiera() actually get substituted from a private yaml file stored on our puppetmaster server	13:21
samueldmq	fungi, nice ... so actually if I try it would fail, right?	13:22
samueldmq	#status notice gerrit stopped emitting stream events around 11:30 utc and has now been restarted. please recheck any changes currently missing results from jenkins	13:22
fungi	samueldmq: i dunno about would, but should anyway	13:22
fungi	yep, it's ignoring you correctly ;)	13:22
samueldmq	fungi, yeah it is :-)	13:23
*** afazekas has joined #openstack-infra		13:23
openstackgerrit	Merged openstack-infra/project-config: project-config: Merge lint jobs (1/2) https://review.openstack.org/164531	13:24
*** serg_melikyan has quit IRC		13:25
fungi	serg_melikyan: the mirror that py27 job failed on seems to have the packages you want currently: http://pypi.iad.openstack.org/simple/python-muranoclient/	13:25
*** e0ne has joined #openstack-infra		13:26
*** hashar is now known as hasharAway		13:26
*** dboik_ has joined #openstack-infra		13:27
*** ildikov has joined #openstack-infra		13:27
*** spredzy\|afk is now known as spredzy		13:27
marcusvrn	hi guys! just a question....Zuul is now working on ubuntu14.04 or it still only working on ubuntu12.04?	13:28
fungi	well, if serg_melikyan comes back into the channel, it looks like bandersnatch didn't see the python-muranoclient 0.5.6 release files or index update until 12:00 utc today	13:29
*** tnovacik has quit IRC		13:29
openstackgerrit	Merged openstack-infra/project-config: Add new tox funxtional test and gate to swift3 https://review.openstack.org/166568	13:29
fungi	marcusvrn: we're still running it exclusively on 12.04 but i think others may have it working successfully on 14.04 and have upstreamed patches to the puppet-zuul module to support it too	13:29
*** afazekas has quit IRC		13:30
*** dboik has quit IRC		13:30
samueldmq	fungi, looking at this just merged patch ^	13:30
samueldmq	fungi, so this is how we add new patches to the gate ( https://review.openstack.org/#/c/166568/2/jenkins/jobs/projects.yaml )	13:31
*** spzala has joined #openstack-infra		13:31
*** sabeen1 has joined #openstack-infra		13:31
samueldmq	fungi, and then the specified project just need to have that env correctly declared into its tox.ini file, right?	13:31
fungi	samueldmq: well, the files in jenkins/jobs/ are how we configure the jenkins masters to know how to run specific jobs, the zuul/layout.yaml file actually tells zuul when to request that they be run and under what circumstances	13:32
*** weshay has joined #openstack-infra		13:33
fungi	samueldmq: _if_ the job is running tox, then yes a tox.ini in the project in question comes into play	13:33
fungi	samueldmq: but jobs are configured to mostly run arbitrary shell commands (often tox but just as often something else, like devstack-gate or some random shell script)	13:34
*** dangers_away is now known as dangers		13:35
*** serg_melikyan has joined #openstack-infra		13:35
*** pblaho_ has quit IRC		13:35
*** pblaho has joined #openstack-infra		13:36
samueldmq	fungi, k thanks, nice to know	13:36
*** peristeri has joined #openstack-infra		13:36
*** chris613 has joined #openstack-infra		13:37
*** mjturek1 has left #openstack-infra		13:38
*** nijaba_ has joined #openstack-infra		13:38
*** nijaba has quit IRC		13:39
*** ominakov has quit IRC		13:39
*** mrmartin has quit IRC		13:40
*** eharney has joined #openstack-infra		13:40
*** davide has joined #openstack-infra		13:40
*** davide has quit IRC		13:40
marcusvrn	fungi: thks for the answer...I'll try to update mine CI to use ubuntu14.04	13:41
*** Ala has quit IRC		13:41
marcusvrn	my CI*	13:41
*** dboik_ has quit IRC		13:42
*** cdent has quit IRC		13:43
*** tnovacik has joined #openstack-infra		13:43
*** nikhil_k\|eto is now known as nikhil_k		13:43
*** afazekas has joined #openstack-infra		13:43
openstackgerrit	Timur Nurlygayanov proposed openstack-infra/project-config: Add HAOS project into Stackforge https://review.openstack.org/167813	13:44
*** tiswanso has quit IRC		13:44
*** ddieterly has joined #openstack-infra		13:45
*** tiswanso_ has joined #openstack-infra		13:45
*** yfried__ has quit IRC		13:46
*** krtaylor has joined #openstack-infra		13:46
pabelanger	morning	13:46
*** sarob has joined #openstack-infra		13:47
*** dboik has joined #openstack-infra		13:48
*** samueldmq is now known as samueldmq-away		13:49
mrodden	is there someone i can ping/ask about when the last email set for ATC registration codes is going out and if i'm on that list...?	13:52
*** tnovacik has quit IRC		13:52
*** serg_melikyan has quit IRC		13:52
*** Longgeek has quit IRC		13:53
jroll	mrodden: email stefano@openstack.org, unless I'm mistaken	13:53
*** dkranz has quit IRC		13:53
mrodden	jroll, ok. assuming he doesn't mind being emailed directly?	13:53
openstackgerrit	Joe Gordon proposed openstack-infra/project-config: Add heat tempest job to heat and tempest https://review.openstack.org/167803	13:53
jroll	don't believe so :)	13:53
openstackgerrit	Joe Gordon proposed openstack-infra/project-config: Add check-tempest-dsvm-heat https://review.openstack.org/167802	13:53
mrodden	ok thanks	13:54
jroll	mrodden: doesn't seem he's online at the moment so I think that's the best route	13:54
mrodden	k	13:54
mrodden	is his irc nick just stefano?	13:54
anteaya	mrodden: reed	13:55
anteaya	mrodden: and the last patch of emails went out	13:55
*** miqui has joined #openstack-infra		13:55
anteaya	mrodden: so if you need to email him to ask for an ATC pass I suggest linking a merged patch of your that took place in the last 12 months (is it 12 for passes or 6 now?) as well as a link to your foundation membership	13:56
mrodden	anteaya, ok thanks	13:56
anteaya	mrodden: I know you need to have an individual foundation membership for voting I don't know about passes	13:56
mrodden	might have gotten lost in my email too	13:56
anteaya	mrodden: it might have, yes check your spam first	13:56
*** sarob has quit IRC		13:56
fungi	anteaya: it's since juno release day	13:56
AJaeger	anteaya, this release cycle - so last 6 months	13:56
anteaya	fungi: thanks	13:56
fungi	so actually a little less than 6 months	13:57
anteaya	AJaeger: yeah I was wondering, thanks	13:57
AJaeger	anteaya, mrodden https://ask.openstack.org/en/question/45531/atc-pass-for-the-openstack-summit/	13:57
mrodden	yeah i saw that	13:57
jroll	thanks for clearing that up anteaya :)	13:57
mrodden	but it hasn't been update in awhile	13:57
anteaya	jroll: sure	13:57
fungi	on or after october 16	13:57
anteaya	mrodden: have you a merged patch in an openstack projects since oct 16?	13:57
anteaya	project	13:58
mrodden	yep	13:58
*** ChuckC_ has quit IRC		13:58
mrodden	i can find it in a sec	13:58
*** ildikov has quit IRC		13:58
anteaya	well then you should be good	13:58
anteaya	fungi: new images kick off soon yeah?	13:58
fungi	mrodden: also make sure your foundation profile's e-mail address and gerrit preferred e-mail address are correct for you. it's always possible he sent your code to an address you no longer use	13:59
sdake	are the centos 6 images bused in some way?	13:59
AJaeger	mrodden, I don't find you on stackalytics for kilo	13:59
fungi	anteaya: in ~14 minutes (14:14 utc last time i looked at the nodepool.yaml anyway)	13:59
sdake	keep getting repeated gate faiures	13:59
anteaya	fungi: ah	13:59
sdake	timeout on git pull	13:59
AJaeger	mrodden, found it - http://stackalytics.com/?metric=commits&project_type=all&user_id=locke105 that was bashate	13:59
anteaya	sdake: logs?	13:59
AJaeger	mrodden, bashate is a stackforge project and that does not qualify for ATC	13:59
*** sandywalsh_ has joined #openstack-infra		13:59
sdake	http://logs.openstack.org/06/164806/9/check/gate-infra-puppet-apply-centos6/3f99bbc/console.html	14:00
mrodden	oh...	14:00
mrodden	its in openstack-dev	14:00
mrodden	?	14:00
fungi	AJaeger: confusing it with doc8?	14:00
sdake	anateaya ^	14:00
sdake	what is wierd is it passed the ccheck but failed two rechecks	14:00
jogo	clarkb: what failed in https://review.openstack.org/#/c/163514/5 ? I didn't see things in the logs	14:00
anteaya	mrodden: it has to be a project listed here: http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml	14:01
sdake	and the gate as well	14:01
*** mmedvede has quit IRC		14:01
fungi	sdake: ever since the glibc "ghost" vulnerability updates we've seen git timrout on remote operations a lot on centos 6	14:01
anteaya	mrodden: if it isn't listed there is doesn't count for ATC for passes or voting	14:01
mrodden	oko	14:01
fungi	er, timeout	14:01
sdake	fungi thanks i'll keep rechecking	14:01
AJaeger	fungi, you're right bashate is openstack-dev	14:01
fungi	sdake: we've still been unable to reliably reproduce the failure so that we can get a better idea of how and hwere it's actually breaking	14:01
anteaya	mrodden: you can still qualify to vote if you merge a patch to one of the projects in that file	14:02
mrodden	when is the cutoff date?	14:02
*** sandywalsh has quit IRC		14:02
sdake	28 march iirc	14:02
clarkb	jogo the file copy. because it happens after devstack runs permissions are funny. nees to remote copy to jenkins homedir not stacks	14:02
*** xyang1 has quit IRC		14:02
sdake	but dont quote me - that is when ptl elections open up	14:02
fungi	mrodden: what's your gerrit username? https://review.openstack.org/#/q/status:merged+owner:mrodden,n,z isn't showing anything	14:03
anteaya	mrodden: for voting? in the past it has been the last day of ptl nominations	14:03
*** ChuckC_ has joined #openstack-infra		14:03
*** ominakov has joined #openstack-infra		14:04
mrodden	locke105 is my gerrit username	14:04
sdake	https://wiki.openstack.org/wiki/PTL_Elections_April_2015	14:04
anteaya	tristanC: when you get a chance the latest election pages should be linked under https://wiki.openstack.org/wiki/Governance#Elections	14:05
anteaya	or pleia2 ^^	14:05
fungi	mrodden: ahh, yep https://review.openstack.org/#/q/status:merged+owner:locke105,n,z	14:06
*** bhunter71 has joined #openstack-infra		14:06
*** mattfarina has joined #openstack-infra		14:06
fungi	mrodden: so your merged commits were all before the juno release, except the one you merged on the 18th of this month. let me see when i last generated the list for reed	14:06
anteaya	trey: and actually I think you need to update the https://wiki.openstack.org/wiki/PTL_Elections_April_2015#Electorate section to April 9th cutoff not April 16th otherwise you can't generate the electoral roll until after the election period is over	14:07
*** lucas-hungry is now known as lucasagomes		14:07
mrodden	fungi, i think anteaya just said that it has to be an official project in the governance repo	14:07
mrodden	to get on the email list	14:07
anteaya	to vote certainly	14:08
mrodden	o	14:08
*** rm_work\|away is now known as rm_work		14:08
fungi	mrodden: right, wondering how openstack-dev/bashate didn't end up in the governance official list	14:08
mrodden	yeah its sort of a weird one	14:08
anteaya	I had thought the stipulateions for passes were the same	14:08
mrodden	we pulled it out of devstack	14:08
mrodden	and it kind of fell through the cracks	14:08
anteaya	it did	14:08
fungi	oh, so it's supposed to be in the qa project?	14:08
fungi	mtreinish: ^ do you recall?	14:08
anteaya	interestingly enough we were discussing projects falling through the cracks just yesterday	14:09
*** prmtl has quit IRC		14:09
anteaya	trey sorry tab complete fail	14:09
sdake	any folks on here have permissiosn to edit https://wiki.openstack.org/wiki/Project_Teams to add Magnum?	14:09
anteaya	tristanC: that was for you as well, not trey	14:09
anteaya	sdake: I think that would be ttx	14:09
mtreinish	fungi: you know we never really discussed it	14:09
anteaya	sdake: and possibly reed	14:09
mtreinish	but sure it's like hacking so we can put it there	14:09
sdake	thought so pinged him but he is away thanks anteaya i'll ping reed	14:10
*** cdent has joined #openstack-infra		14:10
anteaya	sdake: sure	14:10
*** tnovacik has joined #openstack-infra		14:10
fungi	mtreinish: got it. so right now it's sort of sitting there in an official openstack git namespace but not claimed by any projects in the governance repo	14:10
sdake	reed not on irc-- guess i'll have to wait :)	14:10
*** eharney has quit IRC		14:10
mtreinish	fungi: yeah I think that's what happened, it might have been part of devstack when it was a program	14:11
mrodden	i am pretty sure we were leaning towards qa	14:11
mtreinish	but I think we just kinda added it	14:11
anteaya	sdake: yeah I don't know his schedule, he travels a lot	14:11
mtreinish	sort of like sdague's devstack-vagrant repo	14:11
sdake	guess i'll hav eto catch ttx on his tz :)	14:11
mrodden	meanwhile 'm gonna go look for a typo i can fix... :)	14:11
anteaya	sdake: get up early	14:11
sdake	its 7am	14:11
anteaya	mrodden: good plan	14:11
sdake	more liek stay up late ;)	14:11
jroll	sdake: get up earlier >:)	14:11
anteaya	sdake: ah yes stay up late	14:12
*** r-daneel has joined #openstack-infra		14:12
anteaya	sdake: I've found 0800 is usually a good time for him	14:12
openstackgerrit	Merged openstack-infra/project-config: Add HAOS project into Stackforge https://review.openstack.org/167813	14:12
anteaya	he isn't rushed, has time to talk	14:12
fungi	sdake: i _can_ edit that page, but it looks like ttx has been the one to make modifications to it recently so would probably make sense to check with him before i go adding anything to Project_Teams	14:12
anteaya	sdake: not many other demands on his time	14:12
sdake	i think i'll just send him a mail	14:12
*** signed8bit has joined #openstack-infra		14:12
sdake	thanks folks	14:12
anteaya	sdake: there you go	14:13
mtreinish	fungi, mrodden: I'll push the change to add that to the qa program in governance	14:13
*** dkranz has joined #openstack-infra		14:13
mrodden	mtreinish, ok cool	14:13
mrodden	now i really need to show up for the summit :)	14:13
*** e0ne is now known as e0ne_		14:13
anteaya	mrodden: that is the problem with making a fuss, you have to follow through	14:13
fungi	sdake: i think that page was supposed to get ditched now that we generate http://governance.openstack.org/reference/projects/index.html	14:14
anteaya	see you there	14:14
*** dboik has quit IRC		14:14
sdake	fungi I mostly care about https://wiki.openstack.org/wiki/PTL_Elections_April_2015 for magnum election addition	14:15
mtreinish	mrodden: nah, you just need to vote for me :)	14:15
*** mmedvede has joined #openstack-infra		14:16
fungi	sdake: oh, sorry, i thought you were asking about the other page	14:16
fungi	sdake: yeah, looks like the election officials are handling that page. maybe they don't yet know that the Project_Teams wiki article is obsolete	14:17
*** annegentle has quit IRC		14:18
*** lucasagomes has left #openstack-infra		14:18
*** tonytan4ever has joined #openstack-infra		14:18
sdake	fungi thanks ;)	14:19
fungi	tristanC: pleia2: might want to consider updating the "official project team" link on the PTL_Elections_April_2015 wiki article to instead refer to http://governance.openstack.org/reference/projects	14:19
*** eharney has joined #openstack-infra		14:19
*** wenlock has joined #openstack-infra		14:19
fungi	since that's automatically updated when the tc approves changes, whereas the old Project_Teams wiki article is stale and probably about to get gutted and replaced with a brief note linking to governance.o.o	14:20
mtreinish	mrodden, fungi: https://review.openstack.org/#/c/168012/	14:20
fungi	thanks mtreinish	14:20
*** prad has joined #openstack-infra		14:21
mrodden	cool	14:22
*** dboik has joined #openstack-infra		14:23
*** e0ne_ has quit IRC		14:23
*** gokrokve has joined #openstack-infra		14:23
*** mrunge has quit IRC		14:24
*** stevemar has joined #openstack-infra		14:25
*** pblaho has quit IRC		14:26
*** thedodd has joined #openstack-infra		14:26
*** emagana has joined #openstack-infra		14:26
*** dhritishikhar_ has quit IRC		14:26
*** amitgandhinz has joined #openstack-infra		14:27
*** timcline has joined #openstack-infra		14:27
*** emagana has quit IRC		14:27
*** emagana has joined #openstack-infra		14:28
*** markvan has joined #openstack-infra		14:29
mordred	ok - I love our new thing where we have the puppet modules broken out	14:30
mordred	but wow it makes grepping for something that you're not sure where a good example of it is much harder	14:30
*** __TheDodd__ has joined #openstack-infra		14:30
jeblair	mordred: we need an "infra grep" tool which will git grep all the infra repos	14:31
jeblair	(i wonder if 'repo' has that)	14:31
anteaya	alex gaynor was working on an openstack grep	14:32
anteaya	it was awesome	14:32
mordred	yes - the livegrep server	14:32
*** thedodd has quit IRC		14:32
mordred	it woudl be gerat if that would be picked up and finished	14:32
*** e0ne has joined #openstack-infra		14:32
mordred	it would be awesome for us to have a livegrep	14:32
nibalizer	yea livegrep is great	14:33
*** otter768 has joined #openstack-infra		14:33
*** dtantsur\|brb is now known as dtantsur		14:33
mordred	jeblair: I hesitate to say this - but having an "infra-puppet" repo that has each of our puppet repos as a git submodule wouldn't be the most unuseful	14:33
clarkb	mordred install modules :)	14:34
mordred	I don't know that it would help our deploys in any way - but it would make it easier to keep track of all of the locally	14:34
mordred	in my source dirs	14:34
nibalizer	i was setting it up then got quite distracted	14:34
*** isviridov has joined #openstack-infra		14:34
nibalizer	with docker	14:35
mordred	nibalizer: yeah - I remember - I think the docker part of that was a rathole, tbh	14:35
mordred	easy to get caught in	14:35
*** sarob has joined #openstack-infra		14:35
mordred	not because one couldn't deploy things very sensibly with docker - but because it's a different paradigm and really would, I think, need to be something we wanted to adopt more systemically	14:36
anteaya	where is livegrep?	14:36
anteaya	I thought it was on stackforge	14:36
mordred	anteaya: there are two half-finished patches in gerrit	14:36
anteaya	what project name?	14:37
*** ajmiller has joined #openstack-infra		14:37
mordred	anteaya: https://review.openstack.org/#/q/message:livegrep,n,z	14:37
*** yamamoto_ has joined #openstack-infra		14:37
*** hodos has joined #openstack-infra		14:37
*** otter768 has quit IRC		14:37
mordred	anteaya: the problem with alex's	14:37
mordred	anteaya: is that he was trying to do the git repo management directly in puppet	14:38
mordred	but we already have this file, projects.yaml, that has the list of all of our repos	14:38
nibalizer	i suggested to the opw list that finishing livegrep could be a component of an infra project if we got a participant	14:38
mordred	and we have scripts in jeepyb to create git repos for cgit mirrors and to register things with gerrit	14:38
nibalizer	same with the 'run stackalytics in infra' project	14:38
mordred	so really, either re-using or copying and changing one of those commands so that there was a "make livegrep repos" jeepyb command	14:39
mordred	would be WAY saner	14:39
anteaya	nibalizer: oh well if you want to save it for opw	14:39
anteaya	go ahead	14:39
anteaya	I would just really like it is all	14:39
nibalizer	mordred: my problem with alex's was that it used exec to call make	14:39
*** zz_jgrimm is now known as jgrimm		14:39
mordred	nibalizer: yeah - we do that in etherpad_lite too - it's not the highest on our list to solve	14:40
mordred	anteaya: I would not let a theoretical opw person dissuade you from working on livegrep	14:40
anteaya	nibalizer: what do you want?	14:41
mordred	anteaya: if we get an opw person and you have finished livegrep, we can ALWAYS find something else for them to do	14:41
pabelanger	jeblair, any thoughts on how zuul could be made smarter to look for lost builds, when nodepool has a label with min-ready: 0? Currently, zuul always reports the build as lost, until I bring one node online.	14:41
pabelanger	We talked about this before last year, but cannot remember what was decided	14:41
mordred	anteaya: while I understand what nibalizer is saying about exec to call make - I think there are two potentially dangerous ratholes here	14:41
mordred	anteaya: one is solving that by doing distro packages, one is solving that by using docker	14:41
anteaya	unless the caliber of opw folks has skyrocketed since I last checked I don't think they would be able to do much with an idea with no design that we don't know how to build anyway	14:42
*** dims_ has quit IRC		14:42
anteaya	but taht is just my opinion	14:42
mordred	the problem with both is taht we do not currently have any system for dealing with either building and publishing docker containers or distro pacakges	14:42
anteaya	not sure how much I could do here anyway, but I am good at making noise	14:42
mordred	and while it would be neat to have systems for doing both things that fit with an infra workflow	14:42
jeblair	pabelanger: it should be reporting as 'not registered' rather than lost, right?	14:42
mordred	those are both large projects in their own right	14:42
*** dims has joined #openstack-infra		14:42
mordred	anteaya: you've been wanting a good project to dive in on	14:43
hodos	hello, can someone help me with the nodepool? I have a question regarding bootstrapServer method	14:43
jeblair	pabelanger: but i was thinking that we should add an option to skip the registration check at this point	14:43
mordred	anteaya: if livegrep is somethign you'd like to see accomplished - why not dive in on it?	14:43
anteaya	I would much rather get a thing working and worry about packaging and automated deployment _after_ it is working	14:43
pabelanger	jeblair, well, my version of zuul says not register. However, it is an older copy. I can test with my 2nd instance here shortly	14:43
anteaya	personally	14:43
mordred	anteaya: yes	14:43
openstackgerrit	Mathew Odden proposed openstack-infra/subunit2sql: Fix double PL::Python category typo https://review.openstack.org/168018	14:43
mordred	anteaya: I agree with you	14:43
jeblair	pabelanger: because the layout check job makes it fairly unlikely for us to mess that up at this point	14:43
mordred	I think the code that is there that just calls make is fine for now	14:43
*** annegentle has joined #openstack-infra		14:44
anteaya	why do we have docker as part of this conversation?	14:44
anteaya	what livegrep problem is docker solving?	14:44
*** tkelsey has quit IRC		14:44
*** amotoki has joined #openstack-infra		14:44
mordred	anteaya: it was nibalizer's stab at not having puppet call make	14:44
*** reed has joined #openstack-infra		14:44
hodos	can anyone tell me why bootstrapServer method doesn't use the key pair instead trying to log in with the 'root' user?	14:45
anteaya	I'm going to be a rebel here	14:45
pabelanger	jeblair, Ya, I'm still on the 2.0.0 release for these instances of zuul. I should at least upgrade one and retest	14:45
anteaya	and suggest that I would just have puppet call make	14:45
anteaya	but since I would just elminate the deployment step entirely until a thing worked	14:45
anteaya	I wouldn't even havae the conversation yet	14:45
anteaya	so if I follow alex has a github repo with livegrep?	14:46
pabelanger	only 366 commits behind master :)	14:46
mordred	anteaya: no	14:46
anteaya	so my first step would be seeing if we can move that to stackforge	14:46
anteaya	if he is open to the suggestion	14:46
*** sarob has quit IRC		14:46
mordred	anteaya: https://review.openstack.org/#/c/96877	14:46
anteaya	mordred: no to what part?	14:46
mordred	there is a patch to deploy it	14:46
*** dboik has quit IRC		14:46
*** dboik_ has joined #openstack-infra		14:46
mordred	there is nothing to move to stackforge	14:46
mordred	there is a patch to system-config that needs to be finished	14:46
*** sarob has joined #openstack-infra		14:47
anteaya	what are we deploying?	14:47
mordred	anteaya: livegrep	14:47
mordred	it'sa project	14:47
mordred	on it's own right	14:47
fungi	anteaya: https://github.com/livegrep/livegrep	14:47
*** pblaho has joined #openstack-infra		14:47
fungi	http://livegrep.com/about	14:47
*** dboik_ has quit IRC		14:47
mordred	the biggest problem in his patch is on line 84 of https://review.openstack.org/#/c/96877/38/modules/livegrep/manifests/init.pp	14:48
*** achanda has joined #openstack-infra		14:48
*** dboik has joined #openstack-infra		14:48
mordred	where he has this puppet thing called "create_resources" which takes an array of repos	14:48
anteaya	looks like development has tapered off: https://github.com/livegrep/livegrep/graphs/commit-activity	14:48
mordred	that is what I was suggesting needs to be replaced with a call to a jeepyb script	14:48
*** ildikov has joined #openstack-infra		14:48
mordred	that should be pretty much exactly like the create_cgitrepos script	14:49
anteaya	it is written in go	14:49
mordred	so line 84 should be "install the projects.yaml file onto the server, and then if the file has changed, run a jeepyb script"	14:49
*** YorikSar has quit IRC		14:49
anteaya	mordred: I'm not where you are yet	14:50
anteaya	that what you say makes any sense to me	14:50
anteaya	now I can just copy paste what you say	14:50
anteaya	but that doesn't mean I understand it	14:51
mordred	anteaya: okie	14:51
*** pblaho has quit IRC		14:51
hodos	I'm trying to use nodepool in my thirdparty CI setup and it seems like it has a problem with ssh connection during 'bootstrapServer'	14:51
hodos	it uses 'root' as a user and ignores the private key from nodepool config. Is it a correct behavior ?	14:51
*** sarob has quit IRC		14:51
*** pblaho has joined #openstack-infra		14:51
mordred	well, I'm more than happy to either start from a different place or to give you time to digest - whichever would be more useful	14:51
openstackgerrit	Christian Berendt proposed openstack-infra/project-config: Enable PyPi push for the stackforge/faafo project https://review.openstack.org/168022	14:51
anteaya	well I can't move at the speed you move at, and I don't have the history you do, so perhaps time to digest for me is the best thing at this very moment	14:52
anteaya	mordred: thanks	14:52
mordred	anteaya: cool! I recommend looking at the livegrep website that fungi pasted in and spend some time with that for a bit, and also to look through the patch from alex when you feel like that would be a useful thing to read through	14:53
anteaya	thank you	14:53
openstackgerrit	Merged openstack-infra/project-config: Add "heat-coe-templates" project https://review.openstack.org/164806	14:54
clarkb	hodos it uses root/ubuntu/fedora user on initial login then runs puppet to install whatever keys you want	14:54
*** e0ne has quit IRC		14:54
clarkb	*may not be puppet	14:54
*** dhritishikhar_ has joined #openstack-infra		14:54
fungi	_we_ use puppet (in our prep scripts)	14:55
clarkb	then snapshots that node and boots subsequent machines off of that image	14:55
fungi	but yeah, something in your image build configuration will need to explicitly set up an account and a ~/.ssh/authorized_keys file for it	14:56
*** e0ne has joined #openstack-infra		14:57
fungi	we get that question a lot. i wonder if our nodepool documentation is deficient on that point	14:57
*** AJaeger has quit IRC		14:57
*** imcsk8\|z1z has quit IRC		14:58
*** imcsk8\|zZz has joined #openstack-infra		14:58
*** bswartz has joined #openstack-infra		14:58
fungi	yeah, looks like we could stand to go into some detail in http://ci.openstack.org/nodepool/scripts.html#ready-script	14:59
*** dboik_ has joined #openstack-infra		14:59
*** tiswanso_ has quit IRC		15:00
*** jaypipes_ has quit IRC		15:00
*** baoli has quit IRC		15:00
fungi	oh, though nodepoold will try to ssh into new nodes to confirm they're reachable even if you don't specify a ready-script, so maybe not there	15:00
*** tiswanso has joined #openstack-infra		15:00
nibalizer	mordred: re projects yaml there is a puppet function called load_yaml	15:00
hodos	ok...but i get this i the log all the time	15:01
*** achanda has quit IRC		15:01
hodos	Traceback (most recent call last):	15:01
hodos	File "/usr/local/lib/python2.7/dist-packages/nodepool/nodeutils.py", line 49, in ssh_connect	15:01
hodos	client = SSHClient(ip, username, **connect_kwargs)	15:01
hodos	File "/usr/local/lib/python2.7/dist-packages/nodepool/sshclient.py", line 30, in __init__	15:01
hodos	key_filename=key_filename)	15:01
hodos	File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 251, in connect	15:01
hodos	retry_on_signal(lambda: sock.connect(addr))	15:01
hodos	File "/usr/local/lib/python2.7/dist-packages/paramiko/util.py", line 270, in retry_on_signal	15:01
hodos	return function()	15:01
hodos	File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 251, in <lambda>	15:01
hodos	retry_on_signal(lambda: sock.connect(addr))	15:01
hodos	File "/usr/lib/python2.7/socket.py", line 224, in meth	15:01
hodos	return getattr(self._sock,name)(*args)	15:01
hodos	error: [Errno 110] Connection timed out	15:01
*** baoli_ has joined #openstack-infra		15:01
fungi	hodos: in the future please paste multi-line content into http://paste.openstack.org/ and link the result here	15:01
hodos	and I can log into the instance using the private key	15:01
hodos	sorry	15:01
jeblair	hodos: is that the _complete_ traceback?	15:02
fungi	does nodepoold's debug log mention the ip address it's trying to reach, and is that address the one you're testing? when you log into that instance with the private key are you testing that from your nodepool server or elsewhere?	15:03
hodos	yep, that's the address	15:03
hodos	so I can log into this ip using the private key	15:03
hodos	and a different username	15:03
*** dboik has quit IRC		15:03
hodos	yes, nodepool server and nodepool user	15:04
*** sdake__ has joined #openstack-infra		15:04
*** erikmwilson has quit IRC		15:04
*** erikmwilson has joined #openstack-infra		15:04
fungi	strange that the socket library is raising a error: [Errno 110] Connection timed out	15:04
*** erikmwilson_ has joined #openstack-infra		15:04
jeblair	that traceback doesn't make sense. it appears to start in the middle.	15:04
hodos	so ssh -i /home/nodepool/.ssh/id_rsa ubuntu@_ip_ works	15:04
hodos	I know but that's the whole debug traceback	15:05
anteaya	have image builds finished?	15:05
fungi	unless that error is misleading, it implies a socket-layer timeout not an account credential issue (which would be coming from further up the stack if that were the problem)	15:05
anteaya	the spikes on the git farm are over	15:05
*** bknudson has joined #openstack-infra		15:06
fungi	anteaya: some have probably finished, but we still have 65 image updates going	15:06
anteaya	ah okay thanks	15:06
fungi	anteaya: they tend to mostly finish around 15:30-16:00	15:07
*** sdake has quit IRC		15:08
anteaya	okay thanks I'll check back	15:09
*** AJaeger has joined #openstack-infra		15:10
*** AJaeger has quit IRC		15:10
*** AJaeger has joined #openstack-infra		15:10
fungi	i'm going to have to pop out for a moment to pick up my car from the mechanic, right down the road so should only be a few minutes	15:10
*** dhritishikhar_ has quit IRC		15:11
clarkb	fungi I want to image that you drive a dune buggy everywhere	15:11
hodos	ok....maybe it's just a slow setup	15:11
fungi	clarkb: i WANT to get one. apparently people out here collect and restore old ones	15:11
hodos	i have nodepool running on the same machine as jenkins master	15:11
hodos	is it OK ?	15:11
openstackgerrit	Wayne Warren proposed openstack-infra/jenkins-job-builder: (hipchat) Accept list of hipchat rooms. https://review.openstack.org/153831	15:12
hodos	how much resourses should that machine have ?	15:12
fungi	clarkb: but no, it's a '95 ford escort hatchback, color is some combination of rust and primer with a bit of green paint in a few places where it hasn't flaked off yet	15:13
clarkb	yup need a dune buggy	15:13
fungi	hodos: it can vary widely, but for our (fairly large) deployment you can see the resource utilization on our servers tracked at http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1	15:13
fungi	okay, heading out, back in a few	15:14
anteaya	mordred: so how much would someone need to search all of openstack vs need to search all of infra or all of neutron?	15:15
*** crinkle_ is now known as crinkle		15:15
anteaya	mordred: if someone were searching for a fairly generic phrase they could get many useless results	15:15
anteaya	I'm imagining that scoped to project would be helpful	15:16
*** thingee has joined #openstack-infra		15:16
anteaya	and might address our need to constantly update the index, which currently isn't a feature in livegrep	15:16
anteaya	from the readme: "livegrep builds an index file of your source code, and then works entirely out of that index, with no further access to the original git repositories"	15:17
*** dannywilson has joined #openstack-infra		15:17
anteaya	given the speed with we we update out git repos that relationship will need to be addressed somehow	15:18
*** timcline has quit IRC		15:20
*** timcline has joined #openstack-infra		15:20
*** unicell1 has joined #openstack-infra		15:20
*** unicell has quit IRC		15:20
*** nelsnelson has joined #openstack-infra		15:20
*** dannywil_ has joined #openstack-infra		15:21
*** dannywilson has quit IRC		15:21
*** timcline_ has joined #openstack-infra		15:22
*** timcline_ has quit IRC		15:22
*** wenlock has quit IRC		15:22
*** wenlock has joined #openstack-infra		15:22
*** timcline_ has joined #openstack-infra		15:23
*** mwagner_lap has quit IRC		15:23
*** jistr is now known as jistr\|mtg		15:23
*** jistr\|mtg is now known as jistr		15:25
*** timcline has quit IRC		15:26
*** achanda has joined #openstack-infra		15:27
fungi	okay, back	15:28
openstackgerrit	Mathew Odden proposed openstack-infra/project-config: Fix TODO in tempest testr prep script https://review.openstack.org/168036	15:29
*** eharney has quit IRC		15:29
*** esker has quit IRC		15:30
anteaya	fungi: multicoloured car is speedy	15:30
fungi	mtreinish: you probably want to review https://review.openstack.org/168036	15:30
fungi	anteaya: yeah, it's surprisingly spry for being two decades old	15:31
mrodden	i didn't see any way to req. a minimum version for subunit2sql	15:31
mrodden	so i'm not sure if we can just assume that we'll get the latest?	15:31
*** BharatK has quit IRC		15:31
fungi	mrodden: i think we continuously deploy on our images, but i'll check	15:31
mrodden	during the nodepool builds	15:31
clarkb	see commwnt	15:31
mtreinish	mrodden: we do a pip install -U in a different script there	15:31
*** esker has joined #openstack-infra		15:31
mrodden	oh ok	15:31
clarkb	build will use latest	15:31
mrodden	so should be ok	15:31
clarkb	but see comment :)	15:31
mrodden	oh derp	15:32
mrodden	find replace	15:32
mrodden	:)	15:32
fungi	clarkb: good catch. i totally missed that	15:32
mordred	anteaya: so - your mention that livegrep seems to have tapered off led me to websearch some things	15:32
*** prmtl has joined #openstack-infra		15:32
mordred	anteaya: our friends at etsy, who wrote some of the software we run already to great success, have recently released a code search tool	15:33
mordred	anteaya: https://codeascraft.com/2015/01/27/announcing-hound-a-lightning-fast-code-search-tool/	15:33
fungi	mrodden: clarkb: i _hope_ gate-project-config-pep8 catches that	15:33
*** eharney has joined #openstack-infra		15:34
mordred	anteaya: with a few quick pokes - it seems like it at the very least solves one of the ugly parts of livegrep - namely, livegrep assumed your repos were on github	15:34
fungi	yuck	15:34
mordred	anteaya: and hound seems to undersatnd thatyou might want to configure your web links to point somewhere else	15:34
*** hdd has joined #openstack-infra		15:34
openstackgerrit	Mathew Odden proposed openstack-infra/project-config: Fix TODO in tempest testr prep script https://review.openstack.org/168036	15:35
*** armax has joined #openstack-infra		15:35
*** marun has joined #openstack-infra		15:36
*** sergsh has quit IRC		15:36
*** gyee has joined #openstack-infra		15:36
*** tnovacik has quit IRC		15:36
*** eharney_ has joined #openstack-infra		15:37
*** eharney has quit IRC		15:37
anteaya	mordred: so right now I just have a node up and upgraded	15:37
*** eharney_ is now known as eharney		15:37
anteaya	so put up hound?	15:37
*** Longgeek has joined #openstack-infra		15:38
*** ujuc has joined #openstack-infra		15:38
mordred	anteaya: I think so - I installed it on my laptop and it was very easy to follow their instructions and get one running	15:38
*** e0ne is now known as e0ne_		15:38
anteaya	both tools reference Russ Cox	15:38
mordred	I think they both use the same tech on the backend	15:38
anteaya	okay I will do so on a vm and see if I can confirm	15:38
anteaya	mordred: that is what I see as well, so far	15:39
mordred	anteaya: http://paste.openstack.org/show/196892 here is a small config file that will point it at three of our repos	15:39
*** Kiall has quit IRC		15:39
mordred	anteaya: feel free to use or ignore it	15:39
*** Kiall has joined #openstack-infra		15:39
anteaya	thank you	15:39
anteaya	I will evaluate first and then decide	15:39
mordred	++	15:39
anteaya	and thanks	15:39
*** dimtruck is now known as zz_dimtruck		15:39
nibalizer	cool	15:39
*** tsg_ has quit IRC		15:39
*** bswartz has quit IRC		15:40
*** achanda has quit IRC		15:40
mordred	anteaya: oh neat. in the search interface, it has a place where you can select to restrict your search to a specific repo already	15:40
mordred	anteaya: so we might be able to get away with just running one	15:41
anteaya	good	15:41
anteaya	it isn't very old: https://github.com/etsy/Hound/graphs/commit-activity	15:41
*** zz_dimtruck is now known as dimtruck		15:41
anteaya	however it does have 17 contributors, so perhaps they are open to contributions, should any happen to be offered	15:42
fungi	btw, circling back around to this morning's incident, i found no significant correlations in the gerrit logs	15:42
anteaya	fungi: hmmmm	15:42
anteaya	fungi: no explaination for it then?	15:42
fungi	so pretty sure those waiting stream events tasks were an effect, not a cause	15:42
mordred	anteaya: yeah - they just released it to the public	15:42
jeblair	fungi: istr this has happened once before; i'm not sure we ended up with a cause there except the idea that perhaps all of the available ssh-handling threads were stuck on zombie connections	15:43
mordred	but etsy has a good track record of being good stewards to open source projects	15:43
*** Kiall has quit IRC		15:43
*** Kiall has joined #openstack-infra		15:43
*** kir_ has joined #openstack-infra		15:44
fungi	jeblair: right, this looks basically like what we saw before. again i tried killing tasks that looked like they might be at fault, but it didn't help so i just restarted gerrit before activity started to pick up even more	15:44
anteaya	mordred: okay that is good, and it appears the index gets updated	15:44
arosen	mordred: I believe you said yesterday that raw image formatting was in the works for nodepool. I was thinking I could help implement this unless someone has already started on it.	15:44
arosen	mordred: i was thinking under diskimages: I can just add image-format: raw and have the default be qcow2	15:45
mrodden	i wonder what a livegrep TODO\|FIXME would look like across projects	15:45
anteaya	mordred: I need some help already, I've never used go, I don't know what the go tools are:	15:45
anteaya	Use the Go tools to install Hound.	15:46
mordred	anteaya: look slightly further down - they give you a command to run	15:46
anteaya	mordred: and searching gives me harbourfreight (love them) and the game of go	15:46
anteaya	mordred: okay thanks	15:46
*** ekarlso has quit IRC		15:46
mordred	anteaya: what they don't tell you is that you'll need to "sudo apt-get install golang"	15:46
*** e0ne_ is now known as e0ne		15:46
anteaya	ah thanks	15:46
mordred	anteaya: you'll also want to make a directory for things to go in and then set that as your "GODIR"	15:46
anteaya	perhaps taht might be a testing of teh waters of offering a patch	15:47
anteaya	lovely	15:47
jeblair	anteaya: do they call it 'harbourfreight' up there? or are you correcting it for them? :)	15:47
anteaya	I'm working on an hpcloud vm	15:47
anteaya	jeblair: ha ha ha	15:47
anteaya	good point	15:47
mordred	anteaya: I did "mkdir ~/src/gopath ; export GOPATH=/home/mordred/src/gopath"	15:47
mordred	anteaya: fwiw	15:47
mordred	anteaya: and then everyting it did it did in that path	15:47
anteaya	mordred: cool thanks	15:47
*** mattt has left #openstack-infra		15:48
anteaya	mordred: why do they use docker?	15:49
*** Swami has joined #openstack-infra		15:49
*** mtanino has joined #openstack-infra		15:50
*** hasharAway is now known as hashar		15:50
anteaya	I'll ask them	15:50
*** jlanoux has quit IRC		15:50
*** rkukura has quit IRC		15:50
mestery	Hi infra! I'm in need of a feature branch in neutron, and I can't seem to find instructions for creating this. Any pointers would be greatly appreciated! Thank you!	15:51
jeblair	mestery: i think we're asking openstack projects to ask the release team to create feature branches	15:51
*** tnovacik has joined #openstack-infra		15:52
mestery	jeblair: Thanks! I'll go and find ttx then :)	15:52
jeblair	mestery: mostly because there can be a cross-project aspect of these, and it's useful to make sure we have consistent names for the same effort	15:52
jeblair	mestery: (or purposefully inconsistent names for different efforts)	15:52
mestery	jeblair: Thsoe are all great reasons, thanks!	15:52
jeblair	mestery: once you get past that, the mechanics should be here: http://docs.openstack.org/infra/manual/drivers.html#feature-branches	15:53
mestery	jeblair: thanks!	15:53
*** dhritishikhar_ has joined #openstack-infra		15:54
fungi	and also, as of a couple days ago, requirements check jobs work correctly for them ;)	15:54
mestery	fungi: yay!	15:54
*** AJaeger has quit IRC		15:55
*** dulek has joined #openstack-infra		15:56
*** esker has quit IRC		15:56
*** timcline has joined #openstack-infra		15:57
mordred	anteaya: you don' thave to use docker	15:57
mordred	anteaya: they've just got that Dockerfile there if using docker is a thing that you're in to	15:57
mordred	anteaya: I did not use docker for anything locally	15:57
anteaya	cool thanks	15:58
openstackgerrit	Ilya Shakhat proposed openstack-infra/project-config: Enable upload to pypi for Shaker project https://review.openstack.org/168047	15:59
anteaya	and their irc channel was prompt and friendly in telling me the same thing	15:59
anteaya	:)	15:59
*** harlowja_at_home has joined #openstack-infra		16:00
*** timcline_ has quit IRC		16:01
mordred	woot	16:01
anteaya	heartening	16:01
mordred	just so everyone knows - I've been banging against booting nodes for teh last several days and I need a break from it	16:03
mordred	so I'm going to work on ridiculous and useless things today	16:03
anteaya	which I'm sure you'll tell us all about	16:03
anteaya	have fun	16:03
mordred	I may or may not mention any of them in the light of day	16:03
anteaya	as you see fit	16:04
*** alexpilotti has quit IRC		16:04
*** mrmartin has joined #openstack-infra		16:04
ttx	pleia2: around?	16:05
anteaya	ttx I think she said something about a conference on tuesday	16:05
anteaya	she was packing for something, not sure how long it lasts	16:06
jeblair	mordred: how is 140840 related to nodepool dib?	16:06
ttx	anteaya: I need an election official to add the recent additions (think:Magnum) to the PTL election page	16:06
anteaya	ttx yes	16:06
ttx	I think tristanC is away too	16:06
anteaya	ttx :(	16:06
*** claudiub has quit IRC		16:07
anteaya	I have to respect my boundaries and not volunteer to help	16:07
anteaya	sorry	16:07
*** timcline has quit IRC		16:07
ttx	sure, could you just ping her if she returns online ?	16:07
anteaya	ttx also I think the electorate patch contribution time needs to be changed from april 16 to april 9th	16:07
openstackgerrit	Merged openstack-infra/project-config: Simplify requirements check run_command invocation https://review.openstack.org/167445	16:08
anteaya	as the 16th is after the election is over	16:08
*** timcline has joined #openstack-infra		16:08
anteaya	ttx: will ping unless you get to her first	16:08
openstackgerrit	Merged openstack-infra/project-config: No need to run the puppet-unit tests on setproxy https://review.openstack.org/166976	16:08
*** mrmartin has quit IRC		16:08
ttx	thx! She might just read the backlog :)	16:08
anteaya	ttx: she is good like that	16:08
fungi	ttx: also should https://wiki.openstack.org/wiki/Project_Teams now just provide a link over to http://governance.openstack.org/reference/projects/	16:08
*** sdake has joined #openstack-infra		16:09
marcusvrn	hey....I'm trying to disable sahara service, but I can't... I tried to disable it in devstack-gate, exporting DEVSTACK_GATE_SAHARA=0, tried to force ENABLED_SERVICES in localrc, but both didn't work...	16:09
fungi	i worry that wiki article will grow more and more stale otherwise	16:09
ttx	fungi: yeah. Wanted to have the links to the wiki pages but meh	16:09
openstackgerrit	Merged openstack-infra/project-config: Remove some tests for Gnocchi https://review.openstack.org/164211	16:09
ttx	will cut down	16:09
*** rkukura has joined #openstack-infra		16:10
*** sdake__ has quit IRC		16:10
*** sdake___ has joined #openstack-infra		16:10
fungi	ttx: well, that list does still link to the wiki pages via the individual homepage links in each team's specific governance page	16:10
fungi	it's just one extra click now	16:11
ttx	yep	16:11
*** mwagner_lap has joined #openstack-infra		16:11
ttx	fixed	16:11
*** ominakov has quit IRC		16:12
*** Longgeek has quit IRC		16:12
*** weshay has quit IRC		16:12
*** yamahata has joined #openstack-infra		16:12
*** weshay has joined #openstack-infra		16:12
fungi	lgtm	16:12
*** tiswanso has quit IRC		16:13
*** unicell1 has quit IRC		16:14
*** tiswanso_ has joined #openstack-infra		16:14
*** sdake has quit IRC		16:14
*** aarefiev has quit IRC		16:17
*** mrmartin has joined #openstack-infra		16:17
*** aarefiev has joined #openstack-infra		16:17
*** pal_ has quit IRC		16:17
openstackgerrit	Merged openstack-infra/nodepool: Remove unnecessary path joining for dib filenames https://review.openstack.org/139704	16:17
*** hdd has quit IRC		16:18
*** hdd has joined #openstack-infra		16:19
*** Somay has joined #openstack-infra		16:19
*** harlowja_at_home has quit IRC		16:20
zaro	morning	16:20
*** carl_baldwin has joined #openstack-infra		16:22
*** tqtran has joined #openstack-infra		16:23
anteaya	morning zaro	16:25
*** arxcruz has quit IRC		16:25
*** jgriffit1 is now known as jgriffith		16:26
*** samueldmq-away is now known as samueldmq		16:26
*** jgriffith is now known as Guest40491		16:26
*** ChuckC_ has quit IRC		16:26
*** ekarlso has joined #openstack-infra		16:27
pleia2	good morning	16:27
openstackgerrit	Lucas Alvares Gomes proposed openstack-infra/project-config: Ironic: Create and enable pxe_ipa-src-nv on ironic-python-agent https://review.openstack.org/163344	16:28
pleia2	ttx: I'm around for most of the day, I'll take a look at what I need to do	16:29
*** Guest40491 is now known as jgriffith		16:29
openstackgerrit	Henry Gessau proposed openstack-infra/devstack-gate: WIP: (experiment) Test dual-stack without dependency https://review.openstack.org/168063	16:29
pleia2	fungi: thanks, I'll propose a change	16:29
openstackgerrit	Merged openstack-infra/subunit2sql: Fix double PL::Python category typo https://review.openstack.org/168018	16:29
*** pblaho has quit IRC		16:30
*** chlong has quit IRC		16:31
pleia2	fungi: wiki updated with link	16:31
anteaya	pleia2: morning	16:31
anteaya	pleia2: also take a look at the electorate elegbility times	16:31
pleia2	anteaya: where is that?	16:32
fungi	anteaya: most of the image updated completed. there's one straggler that's been running for over 2 hours in hpcloud, but if there are going to be job-related issues from the swift upload changes we ought to be seeing them now	16:32
*** mfmcdonagh has joined #openstack-infra		16:32
anteaya	pleia2: the elegibiiltiy period has to be over before the election takes place	16:32
anteaya	fungi: awesome thanks	16:32
derekh	ianw: how did the problem this fix manifest itself ? https://review.openstack.org/#/c/165681/	16:33
derekh	ianw: [ 1.224080] VFS: Cannot open root device "UUID=d355e4e0-5805-45a4-a04c-cd4675044207" or unknown-block(0,0): error -6	16:33
anteaya	pleia2: https://wiki.openstack.org/wiki/PTL_Elections_April_2015#Electorate	16:33
*** otter768 has joined #openstack-infra		16:34
*** timcline_ has joined #openstack-infra		16:34
anteaya	pleia2: the electorate has to be committers before April 9th	16:34
anteaya	in order to be able to generate the electoral roll	16:34
pleia2	anteaya: oh, I misunderstood, I thought we were still talking about eligibility of projects, not voters	16:35
pleia2	I think fungi is actually generating the list of voters	16:35
anteaya	he is	16:35
anteaya	but he has to do so according to the time frame you set	16:35
fungi	yep, just need to make sure that you tell me the right dates ;)	16:35
anteaya	and right now the time frame is such he can't generate the roll until after the election is over	16:36
fungi	if you give me dates which are wrong, there's a chance i might not notice	16:36
anteaya	pleia2: you have't to watch fungi	16:36
anteaya	have to watch	16:36
anteaya	:)	16:36
fungi	especially if i break out into dance	16:36
anteaya	especially then	16:36
pleia2	ok, I'll look back through tristanC's etherpad for these things then (I was really following his lead here and just helping out - and our vacations overlap!)	16:36
anteaya	pleia2: yes I know	16:37
*** timcline has quit IRC		16:37
anteaya	and this is one of the things that are tiny important details which I was hoping to support him in	16:37
*** e0ne is now known as e0ne_		16:37
pleia2	thanks for your help :)	16:37
anteaya	as he has been following my lead here and this is the first time he is the decision maker	16:37
anteaya	totally understand	16:37
openstackgerrit	Jordan Pittier proposed openstack-infra/system-config: Third party CI: document how to properly format message with Gerrit Trigger https://review.openstack.org/167716	16:37
anteaya	so all support here	16:37
anteaya	but we do need to get the dates right asap	16:38
anteaya	otherwise the electorate will be unhappy	16:38
pleia2	re: ttx's earlier comment, https://wiki.openstack.org/wiki/PTL_Elections_April_2015#Elected_Positions should probably match http://governance.openstack.org/reference/projects/ - yes?	16:38
anteaya	pleia2: yes	16:38
pleia2	ok, I'll focus on syncing this up as ttx requested, then I'll look at electorate stuff	16:38
anteaya	pleia2: the projects.yaml file is the canonical file	16:38
*** otter768 has quit IRC		16:38
pleia2	also, maybe finish first cup of coffee :)	16:39
anteaya	pleia2: if any document is out of sync with projects.yaml the governance file is the correct file	16:39
anteaya	pleia2: yes	16:39
*** yfried__ has joined #openstack-infra		16:39
pleia2	gotcha	16:39
anteaya	sorry to pile on here	16:39
anteaya	and we aren't finished yet but I'll stop unti lyou have coffee	16:39
pleia2	no worries, I appreciate the help, I'll ping you when I finish this and think I have my head around the correct dates to confirm	16:40
anteaya	yup	16:40
derekh	So the tripleo cloud should be back working again but the fedora vms are failing to boot	16:41
derekh	is there any way to trigger a new template build (not sure but I think I need a template with https://review.openstack.org/#/c/165681/ )	16:41
*** dulek has quit IRC		16:42
asselin	nibalizer, clarkb or other puppet expert: is there an easy way to try to use 'latest' and if that fails, use 'present' or '<version number>'? https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/static.pp#n117	16:43
greghaynes	derekh: \O/	16:43
greghaynes	derekh: well \O/ for its supposed to be working	16:43
derekh	greghaynes: ;-)	16:43
*** markus_z has quit IRC		16:44
*** e0ne_ is now known as e0ne		16:44
openstackgerrit	Merged openstack-infra/puppet-graphite: Add ability to configure retention policy https://review.openstack.org/161679	16:44
fungi	derekh: when was it brought back online?	16:44
asselin	i.e. trying to work issue where 'lastest' 'pip' doesn't work using a proxy after it's already installed....	16:44
openstackgerrit	Matthieu Huin proposed openstack-infra/reviewstats: Add --server option to openapproved script https://review.openstack.org/168067	16:44
fungi	if it was before 14:14 utc then we should have built new images there already	16:44
derekh	fungi: about an hour ago	16:45
derekh	fungi: so after 14:14 utc	16:45
fungi	derekh: okay, i'll delete your old ready images there	16:45
*** Guest44815 has quit IRC		16:45
derekh	fungi: cool, thanks, I'll be gone before its built but will check back later and see how things are going	16:46
ttx	pleia2: yes, should match projects file	16:47
ttx	basically we close the list on the day the election process starts	16:47
ttx	that's usually when we cut the voters list as well	16:47
ttx	so April 2	16:48
openstackgerrit	Merged openstack-infra/jenkins-job-builder: Fix jenkins-job-builder-compare-xml job by using zuul-cloner https://review.openstack.org/167821	16:48
*** unicell has joined #openstack-infra		16:49
fungi	derekh: images are updating for tripleo-f20 and tripleo-precise now	16:49
fungi	derekh: i'm heading out to lunch myself, but will check back on it when i return	16:49
*** yamamoto_ has quit IRC		16:49
*** sputnik13 has joined #openstack-infra		16:49
pleia2	ttx, anteaya - Murano seems to have just elected a PTL, so do they do it again? and I can't seem to figure out Magnum	16:50
*** rcarrillocruz has quit IRC		16:50
ttx	pleia2: if their process was thorough, you could vet hteir results, I guess...	16:50
derekh	fungi: k, thanks	16:50
marcusvrn	asselin: hi! have you ever disabled a service in devstack/devstack-gate? I tried to disable it in devstack-gate, exporting DEVSTACK_GATE_SAHARA=0, tried to force ENABLED_SERVICES in localrc, but both didn't work...	16:51
asselin	marcusvrn, I have not	16:51
*** prmtl has quit IRC		16:51
*** rcarrillocruz has joined #openstack-infra		16:52
*** rmcall has joined #openstack-infra		16:52
pleia2	ttx: I wouldn't even know where to begin vetting (ask SergeyLukjanov? what criteria means "thorough"?), all I see is the thread http://lists.openstack.org/pipermail/openstack-dev/2015-March/thread.html#58764	16:52
ttx	if you don't have visibility over the process they followed, probably easier to redo it :)	16:53
pleia2	ok	16:54
*** claudiub has joined #openstack-infra		16:54
openstackgerrit	Matthieu Huin proposed openstack-infra/reviewstats: Add GERRIT_KEY and GERRIT_SERVER options in scripts https://review.openstack.org/168073	16:55
pleia2	alright, wiki page updated with the assumption that everyone has one PTL spot open	16:55
*** medieval1 has joined #openstack-infra		16:56
*** achanda has joined #openstack-infra		16:56
*** scheuran has quit IRC		16:56
anteaya	pleia2: yes this is the part where the election official needs to make a decision	16:57
anteaya	pleia2: gather all evidence you can and then you make a decision (usually in concert with your co-official)	16:57
anteaya	pleia2: and what you decide goes	16:57
*** omnibus7 is now known as davidlenwell		16:57
pleia2	anteaya: what decision?	16:58
anteaya	any decision	16:58
anteaya	whether or not a newly included group needs to have an election or not, for example	16:58
pleia2	oh, ok	16:58
*** alexpilotti has joined #openstack-infra		16:59
pleia2	I need to jump on a call, hopefully won't take long, then back to looking at dates	16:59
anteaya	yup	16:59
*** jistr has quit IRC		17:01
*** tkelsey has joined #openstack-infra		17:01
*** ghostpl_ has quit IRC		17:01
*** tonytan4ever has quit IRC		17:01
*** davide_ has joined #openstack-infra		17:02
*** imcsk8\|zZz is now known as imcsk8		17:02
marcusvrn	asselin: hmmm ok!	17:03
openstackgerrit	Ramy Asselin proposed openstack-infra/puppet-openstackci: Create a log server like http://logs.openstack.org https://review.openstack.org/167425	17:04
*** gokrokve_ has joined #openstack-infra		17:05
*** gyee has quit IRC		17:07
*** krtaylor has quit IRC		17:08
*** gokrokve has quit IRC		17:08
*** isviridov has quit IRC		17:09
*** gokrokve_ has quit IRC		17:09
SpamapS	Ok, so I'm poking at adding some tests to diskimage-builder	17:09
SpamapS	One thing that comes up a lot with dib is downloading stuff.	17:10
*** annegentle has quit IRC		17:10
SpamapS	I'm wondering what approach we should take to avoiding massive amounts of downloading from fedora/ubuntu mirrors.	17:10
SpamapS	clarkb: thoughts?	17:10
SpamapS	fungi: ^	17:10
SpamapS	mordred: ^	17:10
SpamapS	jeblair: ^	17:10
SpamapS	greghaynes: ^	17:10
*** mhu has joined #openstack-infra		17:11
SpamapS	I basically just want to have a test that tries to build one of each supported OS with an element that exercises all of the interfaces we expect elements to make use of.	17:11
jeblair	SpamapS: we could run our own mirrors (we talk about doing that periodically but have not started).	17:11
SpamapS	so, source-repos, package-installs, element-deps, element-provides.	17:11
jeblair	SpamapS: we can pre-cache some things on images. it might make sense to do that after we finish the nodepool-dib + bindep work	17:11
SpamapS	jeblair: I have heard that somewhere out in the internets somebody made an apt-mirror -> swift thing.	17:11
SpamapS	Which doesn't seem like it would be hard.	17:12
jeblair	SpamapS: if we did os mirrors, we would probably want to use AFS so that we can solve the 'inconsistent debian mirror' problem	17:12
SpamapS	jeblair: AFS was what I recall as a solution too.	17:12
SpamapS	But that's still ongoing yes?	17:12
jeblair	SpamapS: we have a cell, we don't use it for any production things yet	17:13
*** melwitt has joined #openstack-infra		17:13
SpamapS	jeblair: also the incosistent mirror problem can be entirely avoided if you control Expires headers.	17:13
*** derekh has quit IRC		17:13
SpamapS	which is why archive.ubuntu.com no longer has it.	17:13
*** ibiris is now known as ibiris_away		17:13
SpamapS	Because they just make sure all of the mirrors return the same Expires:	17:13
jeblair	SpamapS: apparently, if i update my bandersnatch patch again, they might possibly consider merging it again. but then again, maybe not. then we could use bandersnatch on afs.	17:14
SpamapS	jeblair: my experience w/ bandersnatch devs was not positive. But I was asking that they reconsider its use behind corporate firewalls.. so maybe I was the unreasonable one. ;)	17:14
krotscheck	zigo_: Hey, are you around, and have some time to talk more about Javascript?	17:14
*** prmtl has joined #openstack-infra		17:14
*** sdake has joined #openstack-infra		17:15
*** SumitNaiksatam has quit IRC		17:15
*** HeOS has quit IRC		17:15
SpamapS	jeblair: anyway, if we did just have an AFS mirror of all the things, how would a check job access it? HTTP or do all the nodepools get it as a mount?	17:15
*** patrickeast has joined #openstack-infra		17:15
jeblair	SpamapS: http would probably be best	17:15
SpamapS	Ok good, that makes it pretty simple.	17:16
zigo_	krotscheck: If it's quick (got to take care of kids at this time).	17:16
*** sarob has joined #openstack-infra		17:16
zigo_	krotscheck: How may I help? :)	17:16
SpamapS	jeblair: btw this is actually in response to some major surgery going on over in diskimage-builder land as a result of nodepool-dib .. so I am not sure it can wait for that work to finish. We're getting slowed down because the only real testing of diskimage-builder right now is TripleO, which has just had a 4 day CI outage. :-P	17:16
*** cdent has quit IRC		17:16
krotscheck	zigo_: Well, with the assumption that javascript isn't going away, I want to talk to you about what tooling choices (and how we use those tools) will lead to reproducible builds.	17:17
*** tnovacik has quit IRC		17:17
zigo_	krotscheck: What's wrong with XStatic stuff, except that Horizon people have to "do the work"?	17:17
krotscheck	zigo_: I'm not talking about horizon, actually. I'm talking about javascript thick clients. No python.	17:18
jeblair	SpamapS: oh i see. yeah, i was mostly thinking that we don't have a great story for on-node caching aside from devstack atm, but that will get better with bindep.	17:18
zigo_	krotscheck: Like with Fuel?	17:18
jeblair	SpamapS: btw, are you mostly thinking about packages? or base image tarball thingies?	17:18
*** sdake___ has quit IRC		17:18
mordred	SpamapS: I'd love to figure out how to do functional testing of dib in normal openstack nodes	17:18
greghaynes	^ YES	17:19
SpamapS	mordred: that is my intention.	17:19
SpamapS	mordred: but I wonder if the sudo is too much to ask?	17:19
*** dtantsur is now known as dtantsur\|afk		17:19
jeblair	mordred: did you see my question from earlier?	17:19
mordred	SpamapS: aroo?	17:19
SpamapS	I guess we give devstack sudo already?	17:19
mordred	jeblair: i did not	17:19
anteaya	fungi: so far so good with swift logs, yes? https://jenkins07.openstack.org/job/gate-cinder-python27/2724/consoleFull	17:19
jeblair	SpamapS: yeah, give a job sudo is an established pattern now	17:19
krotscheck	zigo_: Let me look at what they're doing, educate myself, and I'll get back to you tomorrow?	17:19
* mordred looks at scrollback		17:19
SpamapS	jeblair: ok good.	17:19
*** ivar-lazzaro has joined #openstack-infra		17:19
*** Hefeweizen has joined #openstack-infra		17:20
jeblair	SpamapS: assume you can do anything you want with a node of any type	17:20
zigo_	krotscheck: Fuel uses a single web page full of Ajax, no python.	17:20
zigo_	krotscheck: They used Grunt, and now they switched to Gulp.	17:20
greghaynes	SpamapS: pre-caching on images could work out well since were using dib to make those images	17:20
*** tqtran has quit IRC		17:20
* SpamapS makes sweet jazz with a Fedora node and some 1993 bottle of Côte du Rhone		17:21
mordred	jeblair: oh! it no longer is - it WAS the commit where the process-config-drive element was	17:21
mordred	jeblair: but that has moved to dib now	17:21
*** bmwiedemann has joined #openstack-infra		17:21
SpamapS	So pre-caching is what I think would work well.	17:21
mordred	jeblair: it's still stuff that's needed for sane testing by infra humans	17:21
*** doug-fish has left #openstack-infra		17:21
mordred	jeblair: so work on dib-nodepool and the images winds up being based on that commit at the moment	17:21
greghaynes	SpamapS: what I wonder is how would you test the result?	17:21
SpamapS	Basically just build images, take what is in ~/.cache/image-create, and put it in the test node.	17:21
mordred	but landing it is not a pre-req for landing dib-nodepool	17:21
SpamapS	greghaynes: boot it	17:22
greghaynes	SpamapS: so qemu?	17:22
SpamapS	aye	17:22
jeblair	SpamapS: so in future-world, i imagine that you will put a bindep manifest in dib, and the bindep element on our db runs will pre-cache those packages so the dib tests can use them	17:22
krotscheck	zigo_: That's exactly what I'm trying to figure out.	17:22
SpamapS	greghaynes: devstack	17:22
*** e0ne is now known as e0ne_		17:22
jeblair	fungi: ^ bindep	17:22
krotscheck	zigo_: Do yo u konw what choices they've made to make it packageable?	17:22
krotscheck	(Off the top of your head)	17:22
zigo_	krotscheck: It's hardly packageable !!!	17:22
greghaynes	SpamapS: Sure, the issue is itll be non hardware virtualization for that, unsure how much of a hit it would be	17:22
zigo_	krotscheck: I had a hard time to do it, and it's not finished.	17:22
jeblair	SpamapS: that doesn't exist yet, so if we need to do something right now, i think we need to put something in the nodepool prep scripts + nodepool dib elements to cache what you need	17:22
mhu	Hi, I'd like to report a bug on reviewstats, should I use launchpad and/or storyboard to do so ?	17:23
SpamapS	jeblair: interesting, that does sound more deliberate and clear than just dumping ~/.cache/image-create into the node.	17:23
mordred	jeblair, SpamapS, greghaynes: I kind of think that once bindep is ready, the package-install infrastructure in dib shoudl consider using bindep	17:23
zigo_	krotscheck: I've packaged each libjs separately, and did a kind of symlink forest which I use instead of what they get with Grunt / Gulp.	17:23
krotscheck	zigo_: Ok, so I want to create a web ui project that does not give you a hernia.	17:23
*** ivar-lazzaro has quit IRC		17:23
krotscheck	zigo_: Happy middle ground.	17:23
*** prmtl has quit IRC		17:23
greghaynes	SpamapS: Maybe not a full devstack is better in that case? I think it might be sufficient to just test booting, get networking, ssh	17:23
jeblair	SpamapS: you could also decide that change frequency on dib is low enough right now that we don't need to do anything right this moment and can wait for future-world, especially if this dib test does not get run for all openstack changes	17:24
*** ivar-lazzaro has joined #openstack-infra		17:24
openstackgerrit	Merged openstack-infra/puppet-jenkins: Add extension settings in jjb https://review.openstack.org/162152	17:24
jeblair	mordred: nice idea	17:24
krotscheck	zigo_: And I have some ideas on how to make that work, but I need some of your guidance on what can or cannot be done.	17:24
SpamapS	jeblair: I only want this to run on diskimage-builder changes to allow us to refactor things.	17:24
krotscheck	zigo_: (For reference, I talked about this with a gentleman named... boris? who is also a package maintainer, and he had some good suggestions)	17:24
SpamapS	jeblair: I figure the use of dib in nodepool-dib will be a good enough test to make sure OpenStack doesn't break dib ;)	17:25
jeblair	krotscheck: ooh, what's the project?	17:25
krotscheck	jeblair: https://github.com/krotscheck/ironic-webclient	17:25
*** e0ne_ is now known as e0ne		17:25
jeblair	krotscheck: nice	17:26
openstackgerrit	Merged openstack-infra/system-config: Refactor o_p::base into o_p::template https://review.openstack.org/162830	17:26
krotscheck	jeblair: devananda presented me with the problem of: Can you build a UI for Ironic, that is packageable, that is not dependent on horizon, but that is packaged in a way that can be added to horizon easily.	17:26
greghaynes	mordred: how does bindep do package mapping per distro?	17:26
* mordred supports krotscheck figuring out how to do javascript apps that debian is happy with		17:26
mordred	greghaynes: yes	17:26
jeblair	clarkb, fungi, mordred, pleia2: ^ that's a major possibly break everything change: "Refactor o_p::base into o_p::template"	17:26
mordred	greghaynes: that is, in fact, exactly what it does	17:26
zigo_	krotscheck: Well, just store a list of non-minified javascript files in your project, and do a README.source telling where you got the files and what version it is. Put them in a well identified folder like for example "3rd-party-jslibs". Best would be if you make sure you use versions which are already in Debian, and not just the very latest (example: I can't upgrade Jquery).	17:26
zigo_	krotscheck: I think that's it ...	17:26
*** angela-s has joined #openstack-infra		17:27
zigo_	krotscheck: When packaging, I'll just replace the .js files with the symlinks.	17:27
greghaynes	heh, im asking how because thatll probably be more of the work than package-installs	17:27
greghaynes	mordred: ^	17:27
krotscheck	zigo_: So, wait, lemme get you a link.	17:27
zigo_	krotscheck: Avoid "automatic downloaders" ...	17:27
mordred	greghaynes: let me show you example	17:27
krotscheck	zigo_: So there's this: https://github.com/krotscheck/ironic-webclient/tree/master/app/js/lib	17:27
krotscheck	zigo_: This particular project actually commits all the dependencies to source, and is designed to run _from source_	17:28
greghaynes	mordred: IIRC in some places in tripleo we use package mapping to say X -> [package a, package b]	17:28
*** gokrokve has joined #openstack-infra		17:28
krotscheck	zigo_: Insofar as "Dump everything in the 'app' directory onto a webserver and it'll just work.	17:28
mordred	greghaynes: hrm ...	17:28
greghaynes	mordred: and so wed have to make sure bindep and tripleo pkg-map agree on that	17:28
jeblair	SpamapS: i don't have a handle on how many test runs we're talking about, but my feeling is that if it's not a huge number, just hit the mirrors for now and wait for bindep to land; adding it to both kinds of image builds for the moment will be a little bit of work (with a large test/debug cycle).	17:28
greghaynes	mordred: its not a big issue, but just something well have to fi	17:29
mordred	fungi: I was looking at a patch you made realted to bindep and now I can't find it	17:29
zigo_	krotscheck: Well, actually, many web projects are designed like this, then later, we want it as package ...	17:29
*** mrmartin has quit IRC		17:29
jeblair	greghaynes: oh, so pkg-map does an extra thing bindep dose not -- 1-to-many mappings?	17:29
greghaynes	yes	17:29
mordred	there we go	17:30
mordred	greghaynes: https://review.openstack.org/#/c/164447/1/jenkins/data/bindep-fallback.txt	17:30
mordred	greghaynes: that's a bindep file	17:30
zigo_	krotscheck: Why not integrating this project into Horizon?	17:30
SpamapS	jeblair: I'd guess it will be 10 / day, with about 300MB of download per test.	17:30
mordred	greghaynes: it's less featureful than pkg-map	17:31
zigo_	krotscheck: Also, what's the difference with something like tuskar-ui?	17:31
*** armax has quit IRC		17:31
*** bhuvan has joined #openstack-infra		17:31
greghaynes	mordred: hrmmm	17:32
*** ZZelle has quit IRC		17:32
krotscheck	zigo_: That's a decision that the Ironic team has made- they want the option to run standalone.	17:32
*** ZZelle has joined #openstack-infra		17:32
jeblair	nibalizer: base refactor change merged	17:32
*** ghostpl_ has joined #openstack-infra		17:32
zigo_	krotscheck: That's fine for me, I'm just curious ! :)	17:32
*** armax has joined #openstack-infra		17:32
krotscheck	zigo_: I figured you'd like things being independent and not-interdependent :)	17:33
zigo_	krotscheck: If you wish to authenticate against Keystone, there's a Javascript implementation of Keystone in Fuel.	17:33
nibalizer	jeblair: woooo	17:33
zigo_	krotscheck: I think it's fine to have independent stuff, yeah. Oh, just one thing, please make sure that your project can be installed somewhere else than on the root of the web server.	17:33
zigo_	Like for example /i-like-ironic-a-lot	17:33
zigo_	:)	17:33
krotscheck	zigo_: You got it.	17:34
openstackgerrit	Mathew Odden proposed openstack-infra/project-config: Fix TODO in tempest testr prep script https://review.openstack.org/168036	17:34
krotscheck	zigo_: Right now I'm test hosting it at krotscheck.github.io/ironic-webclient/#/ironic	17:34
krotscheck	So, yeah, that's already backed.	17:34
krotscheck	zigo_: Ok, so actually committing the libraries to source is a good thing?	17:34
zigo_	krotscheck: The usual way to package stuff in Debian is to just drop the web files in /usr/share/<project-name>, then write some nginx or apache Alias directives.	17:34
*** Hal has joined #openstack-infra		17:35
zigo_	krotscheck: I don't think it is a good thing, but I don't know any other ways. I'd prefer you just rely on libjs-* packages in Debian! :)	17:35
*** btran has joined #openstack-infra		17:36
zigo_	krotscheck: If you wish, I can do the packaging work for any libjs you may need.	17:36
*** Hal is now known as Guest32445		17:36
krotscheck	zigo_: Yeah, well, I can't exactly tie myself to debian.	17:36
krotscheck	zigo_: BUT	17:36
krotscheck	zigo_: I can talk to the npm/bower team to see if I can add deb resolution.	17:36
zigo_	krotscheck: Do you also care about RedHat?	17:37
zigo_	I can ask Haikel tomorrow, he does the work for Fedora.	17:37
krotscheck	zigo_: I care about "Things officially supported by OpenStack"	17:37
zigo_	Ah ok.	17:38
*** sdake__ has joined #openstack-infra		17:38
zigo_	And the gate + selenium, I guess...	17:38
krotscheck	zigo_: Basically	17:38
jeblair	(so that's a 'yes' on rhat)	17:38
mordred	greghaynes: if we can move pkg-map to bindep - or to be able to consume it or whatnot	17:39
mordred	greghaynes: then the script fungi is working on to pre-cache everything referred to in all of our repos that has a bindep manifest	17:40
mordred	greghaynes: can walk all of the element folders in the tripleo repos too	17:40
*** sdake has quit IRC		17:41
*** SumitNaiksatam has joined #openstack-infra		17:41
*** doude has quit IRC		17:42
*** ildikov has quit IRC		17:42
greghaynes	yea, it also wouldnt be hard to make a cache pass in DIB, which is something weve talked about for a while	17:42
jeblair	cinerama: totally unimportant question for you in 166073 when you have a minute	17:43
cinerama	hi jeblair	17:43
cinerama	oh i see your comment there	17:43
greghaynes	mordred: because packages are part of the issue, but theres also things like cloud images and random binary packages we grab that you really just need to defer to the elements on how to cache	17:44
rlucio	hey guys, i'm getting a ton of email, from every code reviewer on every review that my ci system posts results on... any idea how i can turn all of those notifications off?	17:44
mordred	greghaynes: yah - but also, we need to be able to inspect your repos and pre-cache things into the image without running code from your repo	17:44
krotscheck	zigo_: Ok, so that's something I can work with. I'm going to hack on the ironic webclient for a bit to incorporate your changes and we can pick up the discussion tomorrow or monday. Thanks!	17:44
zigo_	ok	17:44
cinerama	jeblair: i like the idea of fewer moving parts - will give it a spin when i'm done with what i'm working on for the apache part	17:44
zigo_	Works ! :)	17:44
mordred	greghaynes: nope. we can't really do that - we need to be able to know how to find the info from the caching script - we don't want to run your elements to know what pre-caching they need so that they can be run	17:44
jeblair	cinerama: cool :)	17:44
zigo_	krotscheck: FYI, at this time tomorrow, I'll be at the "Rohn Alpe meet-up" here in Grenoble, with devananda & ttx ! :)	17:45
mordred	greghaynes: that said - you've already got a good readable format in the source-repositories file for expressing things that aren't packages	17:45
krotscheck	zigo_: Neat! Say hi to devananda for me :)	17:45
greghaynes	mordred: yea, thats what I was just thinking	17:45
greghaynes	mordred: what if we allow you to run package-installs to generate a bindep yaml for packages that would be installed for a set of elements?	17:46
jeblair	yolanda: it wasn't clear from your comment in 166252 -- should that change be abandoned?	17:46
*** krtaylor has joined #openstack-infra		17:47
mordred	greghaynes: we'll want to pre-cache EVERYTHING	17:47
mordred	greghaynes: basically, our question of the repo would be "what things could you ever want to download from the internet if you were run on $distro-$release	17:47
krotscheck	zigo_: (Also say hi to ttx for me. I really appreciate that he doesn't think storyboard is dead yet)	17:47
*** YorikSar has joined #openstack-infra		17:48
*** yamahata has quit IRC		17:48
*** pal_ has joined #openstack-infra		17:48
krotscheck	So the next question for Javascript things is more of an infra thing. Yes, we want these things to be easily packageable by debian and centos/rhel. But we also want to make them available to the javascript world.	17:48
mordred	greghaynes: that way when we're doing image builds (yes, I recognize the tail-eating) we can say "I'm building an image for ubuntu-trusty, I have these 500 git repos that make up openstack, what is the entire sum of things that any of them could ever want to do with the internet"	17:48
mordred	krotscheck: ++	17:48
openstackgerrit	Merged openstack-infra/project-config: Add py27-scenario-unit tox non-voting job for Sahara https://review.openstack.org/163776	17:48
krotscheck	Which means bower and/or npm.	17:48
greghaynes	mordred: yep. As-is that isnt something we can just answer. We actually need for elements to specify things it is going to download but needs to do it on its own for both packages and source-repositories	17:48
mordred	krotscheck: we want debian people to be able to consume them in debian form, but javascript people in javascript form	17:49
mordred	and for netierh to feel like a second class citizen	17:49
mordred	yeah?	17:49
krotscheck	mordred: Yep, exactly	17:49
*** jerryz has quit IRC		17:49
jeblair	netierh: are you a second class citizen?	17:49
*** krotscheck is now known as netierh		17:50
netierh	yep	17:50
*** netierh is now known as krotscheck		17:50
mordred	gah	17:50
jeblair	:(	17:50
krotscheck	;D	17:50
krotscheck	Moving on.	17:50
larsks	Hello -infra folks. What do I need to do in order to be able to add people to a group for a newly-created stackforge project?	17:50
krotscheck	So I'm not even going to talk about npm packaging yet, because isolating problems is useful.	17:50
jeblair	larsks: tell us the group and we'll add one person to it and that person can add the rest	17:51
krotscheck	Bower uses a detached-branch approach. put release artifacts into a detached branch, push it, tag it with a semver version.	17:51
mordred	two sides to the bower problem - publication of things (like we publish to pypi) - and local caching/mirroring for jobs we want to run	17:51
mordred	hrm- that's an interesting problem from our side	17:51
larsks	jeblair: Thanks. The group is heat-coe-templates-core (and -release). Start by adding me (username "lars").	17:51
*** e0ne is now known as e0ne_		17:51
krotscheck	mordred: The second part of that is not a thing- if it turns out we have to commit our bower dependencies to source anyway.	17:52
*** e0ne_ is now known as e0ne		17:52
mordred	because of the sequence of actions that implies as well as what it implies about the actors performing them	17:52
mordred	awesome	17:52
jeblair	larsks: done	17:52
mordred	so - we don't really need to worry about the "make a detached branch" part	17:52
krotscheck	mordred: But if it does become a thing we want to do, there's this: https://github.com/krotscheck/puppet-bower	17:52
mordred	if the code is consumable - it's more the tag with semver part, yeah?	17:52
larsks	jeblair: Awesome, thanks!	17:52
mordred	krotscheck: oh - wait - I may have misunderstood words	17:53
krotscheck	mordred: Maaaybe. THere's the question on whether the source resembles the release artifacts. We may want to concatenate for instance.	17:53
mordred	krotscheck: let's step back for one second, I think we overlapped for a second and I've now lost the train	17:53
krotscheck	words are hard.	17:53
mordred	they are	17:53
*** pal_ has quit IRC		17:53
krotscheck	ok	17:53
krotscheck	Two problems.	17:53
krotscheck	Problem one: Infra builds reach out and resolve bower.	17:54
krotscheck	Assuming that bower is being run in infra.	17:54
krotscheck	Solution 1: Commit dependencies, make devs run bower, infra therefore never has to run bower, everyone wins.	17:54
*** ZZelle has quit IRC		17:54
krotscheck	Solution 2: Allow bower to run in infra, set up bower mirror as per the puppet module I just posted.	17:55
*** tnovacik has joined #openstack-infra		17:55
krotscheck	Solution 3: Fix bower (In reality, teach it to resolve a hierarchical tarball uri)	17:55
*** ZZelle has joined #openstack-infra		17:55
krotscheck	(Solution 3 is actually not that difficult.)	17:55
mordred	I have a question about 2	17:55
jeblair	(is "commit dependencies" something like "vendor copies of dependencies"?)	17:55
mordred	given the way bower works - where it expects the full thing at a semver tagged git ref	17:56
mordred	if we did not "vendor" the deps into the source tree	17:56
krotscheck	jeblair: Urrr... it's basically like oslo-incubator. Resolve it, copy relevant bits into my tree, commit those bits.	17:56
*** Longgeek has joined #openstack-infra		17:56
*** Ryan52 has joined #openstack-infra		17:56
mordred	would we have to come up with a publication thing that did but all the build artifacts into a git branch?	17:56
krotscheck	mordred: ...yeeess.	17:57
mordred	like, given current state of bower, they are going to expect a git repo ref with all the things in it - whether they were in the original "source" or not	17:57
* mordred is just repeating things to make sure he understands the state of the bower world		17:57
krotscheck	mordred: Taht depends on whether you have an app or a lib.	17:57
krotscheck	mordred: A lib can declare peer dependencies.	17:58
krotscheck	mordred: an app is assumed to be entirely self contained.	17:58
mordred	gotcha	17:58
mordred	impl is different, but the split is similar in rust, fwiw	17:58
krotscheck	Ultimately, libs never really package their dependencies, they assume the dev controls that.	17:58
mordred	and I support having a concept of apps vs. libs wrt dependency management - because the use cases are different	17:59
mordred	anywho	17:59
krotscheck	Righto.	17:59
mordred	krotscheck: 3 would, I think, make the infra pov the easiest	17:59
*** doug-fish has joined #openstack-infra		17:59
*** __TheDodd__ has quit IRC		17:59
mordred	because we know how to publish tarballs	17:59
mordred	a world in which 2 is the developer workflow seems the most like what devs hacking with bower expect the world to feel like, yeah?	18:00
krotscheck	mordred: Option 2 is.... well, people like it because it's easy, but nobody believes it's a good idea.	18:00
krotscheck	mordred: That all came out in the NPM SSL Key fiasco a year ago.	18:01
*** Longgeek has quit IRC		18:01
krotscheck	mordred: Hell, the NPM documentation even tells you to commit your dependencies.	18:01
mordred	I remember that. they told me that openstack sucked when I asked if they had an https download url	18:01
*** yamahata has joined #openstack-infra		18:01
mordred	which I thought was a strange response	18:01
*** esker has joined #openstack-infra		18:01
krotscheck	I agree.	18:01
krotscheck	I think they've fixed that...	18:02
mordred	committing dependencies though is considered a bad idea in other circles	18:02
*** claudiub has quit IRC		18:02
*** e0ne has quit IRC		18:02
krotscheck	Right. Because from the standpoint of someone like zigo_, it forces him to peel apart all the dependencies.	18:02
mordred	it was one of teh thigns that made me happy about bower's existence - since it meant people could stop doing that	18:02
mordred	it makes it harder to know what version of dependent libs are there - and to update the libs on your system in response to security things	18:03
mordred	the ops folks at the ops summit actually were very loud on a related point	18:03
jeblair	(and also is just confusing from a source control point of view)	18:03
mordred	which is that they didn't want more than one version of a given lib in their env, even if they were running services on separate servers and physically could	18:03
mordred	because they didnt' want to track depends across things	18:03
mordred	gah. sorry	18:03
mordred	not track depends	18:03
mordred	track security bugs	18:04
mordred	so they wanted to be able to know they had requests==1.2.5 in their custom-built local apt repo	18:04
mordred	and taht they should be paying attention to CVEs for it	18:04
*** nijaba_ is now known as nijaba		18:04
* krotscheck gets the idea that he's going to be doing way more talking to people and way less hacking as he pursues this.		18:04
sdague	can have devstack-gate log the cleanup into a separate file?	18:04
mordred	krotscheck: you are walking righ tin to one of the most difficult intersections right now	18:04
nibalizer	krotscheck: getting consensus is harder than hacking :)	18:04
sdague	actually figuring out that this was the error line in this file was pretty challenging - http://logs.openstack.org/14/163514/5/experimental/check-tempest-dsvm-aiopcpu-full/5602026/console.html#_2015-03-26_14_29_26_993	18:05
sdague	because there is so much post fail cleanup logging	18:05
mordred	krotscheck: but it's one that honestly really needs some answers in the industry	18:05
mordred	sdague: that's a good idea - I don't know how feasible it is	18:05
*** andreykurilin_ has joined #openstack-infra		18:05
mordred	sdague: but I do know that it's frequently a confusion point for userse	18:05
krotscheck	mordred: Is there a common established tarball archive URI structure out there?	18:05
SpamapS	mordred: been chasing that one since 2010. It's ugly.	18:05
SpamapS	Cassandra was the first thing I ran into that was so heavily vendored it _could not_ be un-vendored.	18:06
krotscheck	Also: If I publish a tarball, how is that 'securely' different than pushing a tag? I can --force overwrite the tag, I can replace the tarball.	18:06
mordred	krotscheck: not really - there are a few patterns that emerge, but not particularly anything consistent across different backgroudns	18:06
sdague	also, 50% of the console is now cpumap :)	18:06
anteaya	mordred: http://15.126.217.83:6080/?q=config&i=nope&files=&repos=	18:06
SpamapS	krotscheck: a signed tag and a signed tarball are the same in terms of authentication.	18:06
mordred	krotscheck: it's not	18:06
mordred	yeah	18:06
mordred	main thing is - we don't have a mechanism for a build job to commit and push to a repo	18:07
mordred	and we've been trying to keep it that way	18:07
mordred	beacuse of skynet	18:07
krotscheck	Hrm.....	18:07
krotscheck	skynet.	18:07
krotscheck	Right	18:07
nibalizer	mordred: so is this a degenerate form of 'create an artifact and store it somewhere' ?	18:07
mordred	so a build job can make an artifact and publish it	18:07
nibalizer	krotscheck: ?	18:07
krotscheck	nibalizer: Yep.	18:07
mordred	but we don't have a publication path that is to a git repo	18:07
mordred	which is why if we could establish a tarball convention that could be fetched by bower	18:08
jeblair	sdague: like http://logs.openstack.org/14/163514/5/experimental/check-tempest-dsvm-aiopcpu-full/5602026/logs/devstack-gate-cleanup-host.txt ?	18:08
jeblair	sdague: maybe there's a bug?	18:08
SpamapS	This debate has been going on in Debian too, since many teams are packaging things that don't actually publish tarballs by exporting git tags and then uploading that as a source package. They find that silly, and would like to just have the Debian source repositories host git repos.	18:08
mordred	SpamapS: ++	18:08
mordred	SpamapS: I, also, find it silly	18:08
SpamapS	mordred: I think it is under-the-hood meh	18:08
mordred	SpamapS: yah - except in the case of "I am an upstream and a packager and I don't want to create a tarball, import it back into the git repo, then use that to make the package"	18:09
jeblair	sdague: error in redirection or something?	18:09
*** alexpilotti has quit IRC		18:09
*** Somay has quit IRC		18:10
*** _nadya_ has quit IRC		18:11
SpamapS	mordred: the import-it-back-into-the-repo is not a thing anymore	18:11
SpamapS	mordred: I mean it can be if you want, but it's not necessary with git-buildpackage.	18:11
SpamapS	mordred: it's smart enough to match tags <-> tarballs	18:12
krotscheck	mordred: So, bower takes URI's, sees whether there's a resolver for it, and uses that resolver.	18:12
krotscheck	Here's a list of resolvers: https://github.com/bower/bower/tree/master/lib/core/resolvers	18:12
mordred	krotscheck: so - perfect from our world view here would be a) no vendored libraries checked in to source repos b) bower runs in test jobs against a local mirror and c) we publish artifacts to a tarball repo that have bower-fetched all of the depends that are appropriate and included them in the tarball	18:12
sdague	jeblair: oh, like the file might not actually have gotten written?	18:12
SpamapS	mordred: if you're an upstream and you want packaging, just make a branch w/ only ./debian in it, and git-buildpackage will use that to make source packages for you. Ergo "plumbing meh"	18:12
mordred	SpamapS: can you please point me to a simple write up of current modern best practice for gbp ?	18:12
sdague	jeblair: sorry, wrong thought	18:13
*** tsg has joined #openstack-infra		18:13
jeblair	sdague: well, i see some stuff in that file that's in console, but also some stuff in console that's not in that file....	18:13
mordred	SpamapS: because my choices so far are "read about everything I could possibly do with gbp" or "follow alberry's out of date set of notes"	18:13
*** alexpilotti has joined #openstack-infra		18:13
SpamapS	mordred: https://wiki.debian.org/PackagingWithGit is the canonical source of information for that lately.	18:13
*** alexpilotti has quit IRC		18:13
mordred	I've read it - it explains nothing	18:13
mordred	it's too "you could do this or this or this or maybe you wated to bake chicken"	18:13
SpamapS	mordred: https://wiki.debian.org/PackagingWithGit#Using_the_upstream_repo does	18:13
jeblair	sdague: so i wonder if there is an error that's causing those contents to also be displayed in console (or perhaps it's working and there's just something in console that's doing the same thing)	18:14
krotscheck	mordred: Well, for webapps anyway :). For libs I don't think we need to package those.	18:14
mordred	awesome - me reads	18:14
krotscheck	But I get what you're saying.	18:14
mordred	krotscheck: awesome	18:14
mordred	yeah	18:14
krotscheck	Let me go talk to the bower people, see if anyone there is receptive.	18:14
jeblair	sdague: and i also wonder if there's a step that has no current redirection that has simply gotten longer since the last time we looked at this; maybe that was your original point?	18:14
*** kgiusti has left #openstack-infra		18:14
mordred	krotscheck: so you could totally make a tar.gz resolver most likely	18:15
*** tsg_ has joined #openstack-infra		18:15
fungi	wow. my lunches are the worst time _ever_ for me to step away from the computer. that's something like 10 pings for me on this channel alone	18:15
mordred	and it would take you like a minute	18:15
mordred	fungi: stop eating	18:15
krotscheck	mordred: Yeah, but I'd like to add versioning.	18:15
SpamapS	mordred: agree that it's _way_ too Debian.. "we can do things any way we want ever because we don't prescribe anything even if everything else is insane."	18:15
* Clint chuckles.		18:15
pleia2	mordred: repeat to self: work/life balance	18:15
openstackgerrit	Stephanie Miller proposed openstack-infra/puppet-zanata: Add apache configuration for zanata https://review.openstack.org/164011	18:16
*** gyee has joined #openstack-infra		18:16
krotscheck	mordred: So that the bower uri might be someting like semver://http://foo.com/libname/{{version}}/tarball.tgz or something like that.	18:16
mordred	krotscheck: a common hosting pattern is foo.com/libname/libname-{{version}}.tar.gz ... and also foo.com/b/bar/bar-{{version}}.tar.gz	18:17
krotscheck	...aaaactually, if I just add variable substitution support to the fsresolver or urlresolver....	18:17
mordred	krotscheck: ooh, that's probably great	18:17
openstackgerrit	greghaynes proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927	18:17
greghaynes	clarkb: ^	18:17
krotscheck	mordred: Because that's thing I think they'd even be awesome with.	18:18
*** tsg has quit IRC		18:18
*** gokrokve has quit IRC		18:18
*** gokrokve_ has joined #openstack-infra		18:18
jeblair	clarkb, jhesketh: i'm missing why in 167158 we need to change anything for the scp case -- don't we still have apache handling that for static.o.o?	18:18
sdague	jeblair: yeh, jogo is looking now, looks like subnode filter	18:19
jeblair	sdague: ah, subnodes!	18:19
mordred	krotscheck: yah - it's good when we don't have to sell people on why our worldview differs	18:20
openstackgerrit	Joe Gordon proposed openstack-infra/devstack-gate: Redirect subnode cleanup to devstack-gate-cleanup-host.txt https://review.openstack.org/168102	18:20
jogo	jeblair: ^	18:20
jogo	sdague: ^ should fix it	18:20
*** ZZelle has quit IRC		18:20
* SpamapS goes into the cone of silence to finish working on slides for tonight		18:20
*** andreykurilin_ has quit IRC		18:20
openstackgerrit	Christian Berendt proposed openstack-infra/project-config: Enable PyPi push for the stackforge/faafo project https://review.openstack.org/168022	18:20
*** ZZelle has joined #openstack-infra		18:20
*** Sukhdev has joined #openstack-infra		18:21
fungi	jeblair: SpamapS: yes, my expectation is that nodepool dib image builds will scrape bindep manifests (the existing bindep default is other-requirements.txt but that's open to debate) on all branches of all git repos we cache (all those in our gerrit) to build a list of what to cache (along with the details we pull from devstack already)	18:21
*** neoice_ is now known as neoice		18:22
mordred	greghaynes: ^^ fungi didn't ping you	18:23
mordred	fungi: another thing that came up while you were eating	18:23
jeblair	ew	18:23
fungi	mordred: jeblair: SpamapS: if you're looking for a complex bindep example, https://review.openstack.org/164447 mostly turns our devstack-.* nodes into bare-.* nodes at the moment (i need to dig up a few more name patterns to add to it for the database server packages)	18:23
mordred	fungi: is that, like devstack, dib elements also can want to download more things than packages	18:23
*** jlanoux has joined #openstack-infra		18:23
*** gyee has quit IRC		18:24
mordred	fungi: unlike devstack, they do have a descriptive format for expressing this	18:24
mordred	that isn't just in the shell scripts	18:24
*** sdake_ has quit IRC		18:24
mordred	fungi: so we might want to learn to read those for generalized cache scraping	18:24
*** baoli_ has quit IRC		18:24
*** tnovacik has quit IRC		18:24
*** afazekas has quit IRC		18:25
*** baoli has joined #openstack-infra		18:25
jeblair	mordred: did you intentionally reverse the dependencies again in 167607 and 167612?	18:26
anteaya	I have to go for a walk now, back later	18:26
pleia2	anteaya: I'll catch up with you later this afternoon (reviewing dates now, then have dr appt+lunch)	18:27
*** gyee has joined #openstack-infra		18:28
openstackgerrit	Sean Dague proposed openstack-infra/devstack-gate: Try to calculate common CPU feature set https://review.openstack.org/163514	18:28
fungi	greghaynes: fair point wrt caching distro packages, all bindep does is tell you for a given current distro (based on what it can glean by running lsb_release) a list of packages you want. it doesn't actually take any action of any sort. think of it as a cross-distro package dependency listing tool and nothing more	18:30
openstackgerrit	Stephanie Miller proposed openstack-infra/puppet-zanata: Add OpenID login provider support to Zanata config https://review.openstack.org/166073	18:31
fungi	rlucio: in case nobody else has answered yet, the only way to avoid receiving e-mail for changes on which your account has commented is to not have an e-mail address associated with that account in gerrit	18:32
*** ibiris_away is now known as ibiris		18:32
*** andreykurilin_ has joined #openstack-infra		18:33
mordred	jeblair: nope. totally unintentional. I apologize that the policy confuses me and I keep getting it wrong - it's not intentional	18:34
*** otter768 has joined #openstack-infra		18:35
dprince	fungi: derekh poked me to check back on this: https://review.openstack.org/#/c/165681/1/install_puppet.sh,cm, how long before that is effective?	18:35
openstackgerrit	Monty Taylor proposed openstack-infra/project-config: Add puppet-diskimage_builder module https://review.openstack.org/167607	18:35
*** carl_baldwin has quit IRC		18:35
mordred	jeblair: fixed	18:35
fungi	in skimming, i keep seeing references to "commit your dependencies (source into your repository)" which makes me uneasy. i know it's a paradigm in very new programming circles, and from what i've seen they just haven't been around long enough to discover the eventual pain which results from that suggestion	18:37
openstackgerrit	Joe Gordon proposed openstack-infra/project-config: Add tempest-dsvm-sahara https://review.openstack.org/167779	18:37
openstackgerrit	Joe Gordon proposed openstack-infra/project-config: Add sahara tempest job to sahara and tempest https://review.openstack.org/167780	18:37
*** otter768 has quit IRC		18:39
*** Guest32445 has quit IRC		18:41
*** jlanoux has quit IRC		18:41
dims	sdague: anteaya: may i request help with adding a dsvm job for magnum? https://review.openstack.org/#/c/164553/	18:41
mordred	fungi: I think we're trying to work through possibilities that will make that not necessary	18:42
*** ssam2 has quit IRC		18:43
sdague	dims: +@	18:43
*** andreykurilin_ has quit IRC		18:43
dims	thanks for the elevated +2 sdague :)	18:43
openstackgerrit	Ramy Asselin proposed openstack-infra/puppet-openstackci: Create a log server like http://logs.openstack.org https://review.openstack.org/167425	18:43
sdague	:)	18:43
mordred	SpamapS:	18:43
mordred	If we're using the upstream repo, we can git fetch the new release tag into the repo, in the upstream branch. We need to git merge this branch into master, and to commit the new tarball with pristine-tar. The patches can be rebased with gbp-pq, and git itself can be used to resolve any conflicts that have been created.	18:43
greghaynes	fungi: mordred Yea, my thinking with DIB is that its actually not that important for us ATM to be able to cache based on elements since our main use of dib is in a case where network access is a requirement	18:43
*** tonytan4ever has joined #openstack-infra		18:43
*** __afazekas is now known as afazekas		18:44
mordred	greghaynes: but testing dib changes in the gate	18:44
mordred	or, in the normal gating infrastructure	18:44
greghaynes	mordred: aye	18:44
mordred	greghaynes: would be awesome if we did not have to touch the internets	18:44
greghaynes	mordred: good point	18:44
mordred	because the internets break	18:44
mordred	I tihnk mostly because of all the perl	18:44
mordred	SpamapS: can you explain either why I need to have an upstream branch or how I can get around the need for it?	18:45
fungi	mordred: greghaynes: so maybe bindep needs to learn a second input format so that it can interpret dib elements' package specification, or maybe the two should converge	18:45
mordred	SpamapS: it seems like what I want to do is have upstream's master be master, no upstream branch, have a debian branch, and have gbp figure out the tag	18:45
greghaynes	mordred: ok. So I think the path of least resistence is to just write a tool that can query package-installs and spit out a list (which package installs basically does already)	18:45
greghaynes	fungi: Im thinking why not have dib spit out bindep	18:45
mordred	I think you both just mainly said the same thing	18:46
openstackgerrit	Clark Boylan proposed openstack-infra/devstack-gate: Try to calculate common CPU feature set https://review.openstack.org/163514	18:46
greghaynes	yea, its a matter of which side the work goes in on	18:46
greghaynes	but yes, same page :)	18:46
mordred	if you're going to write a tool for dib that can spit out bindep - it seems like maybe just writing a patch to bindep that can read package-install files is easier?	18:46
mordred	otherwise we'd have to run a tool that's in the dib repo to output data that woudl then be concatted with output from a second tool we'd be running	18:47
fungi	dprince: it's in place on your current images. is it possible your tripleo-f20 node builds are broken by something different?	18:47
*** ihrachyshka has quit IRC		18:48
mordred	greghaynes: you'd still need the package-install element probably to deal with stuff in-chroot - but bindep could do the reading and consolidation for you on the outside waves arms	18:48
*** tgohad has joined #openstack-infra		18:48
clarkb	sdague: I see now that you removed a sudo from there? not sure why?	18:49
*** unicell has quit IRC		18:49
*** unicell has joined #openstack-infra		18:49
mordred	SpamapS: also: "Since the Debian APT repositories still use tarballs you still have to manage those with this setup, but the pristine-tar exists for that purpose."	18:50
sdague	clarkb: so there is a permission denied in scping the file	18:50
sdague	but the target dir looks fine	18:50
clarkb	sdague: its the target dir that fails	18:50
clarkb	sdague: the scp runs as jenkins but the dest side is owned by stack after devstack runs	18:50
clarkb	I think	18:50
sdague	we just scped to that directory a couple lines above	18:50
mordred	SpamapS: btw, I'm not trolling,so I hope it's not coming across that way - I'm actively confused	18:50
greghaynes	mordred: sure, if were fine adding an appendage like that to bindep then no problem with me	18:50
clarkb	sdague: before devstack runs	18:50
sdague	oh, it was late?	18:50
fungi	greghaynes: yes, i agree with mordred that running one thing to get the job done is better than running two things, unless this is a unix "do one thing and do it well" with pipes scenario, in which case i might be forced to concede because at&t used to have lots of smart people in the beforetimes	18:50
sdague	clarkb: ok, didn't realize that	18:51
clarkb	sdague: we have to do this after devstack so that libvirt is installed	18:51
*** HeOS has joined #openstack-infra		18:51
mordred	fungi: I think it's still one thing	18:51
sdague	clarkb: gotcha, so should I just put it in /tmp ?	18:51
mordred	fungi: it just ahs two file formats	18:51
*** ildikov has joined #openstack-infra		18:51
sdague	so we don't have to worry about that	18:51
clarkb	sdague: my patch uses jenkins home dir	18:51
greghaynes	mordred: fungi hah, How about I play around with bindep and see if it sticks	18:51
clarkb	but /tmp would work. we should see in ~20 minutes if I am correct	18:51
sdague	clarkb: oh, there is a new patch?	18:51
mordred	greghaynes: cool	18:51
clarkb	sdague: ya I just pushed one	18:51
sdague	ok	18:51
greghaynes	mordred: fungi (it being making it compile package-installs)	18:51
sdague	can you also integrate the xtrace change?	18:52
greghaynes	One issue I forsee - its going to need to know about pkg-map also	18:52
mordred	yup	18:52
*** tsg_ has quit IRC		18:52
jogo	clarkb: ahh I didn't realize how much the devstack run line changes	18:52
sdague	also, this is kind of wonky to debug to the point that I think for subnodes we should define a dropbox so we aren't trying to keep track of perm changes	18:52
greghaynes	mordred: fungi and actually, package-installs already knows about that, if we were able to make package installs just a python file that you point at and elements dir and it spits out a bindep file....	18:52
*** carl_baldwin has joined #openstack-infra		18:52
*** Somay has joined #openstack-infra		18:52
greghaynes	anyhow, ill play with it	18:53
fungi	greghaynes: also note the patches i have proposed to bindep to flesh it out a bit. and consider that you may need to add some more patches to get anywhere close to parity with whatever dib wants	18:53
greghaynes	maybe we can make package-installs depend on bindep so we dont duplicate that logic	18:53
fungi	greghaynes: yes, if there's something that can be libified there, i agree it should be	18:54
clarkb	sdague: yes the file stuff needs to be fixed, I was actually thinking maybe we use the swift install fo rthat	18:54
openstackgerrit	Andrew Boik proposed openstack-infra/devstack-gate: WIP: (experiment) Test dual-stack with dependency https://review.openstack.org/168063	18:54
clarkb	sdague: since we can get the primary node up with swift, then put files in swift and pull from that on subnodes but that might be too chicke and eggy	18:54
sdague	yeh, I think that's way too chicken / egg	18:55
sdague	honestly just /tmp/filedrop	18:55
sdague	or something	18:55
hodos	hey guys can you point me to a good nodepool documentation. to understand it workflow	18:57
fungi	hodos: i assume you've already seen http://ci.openstack.org/nodepool/ ?	18:57
hodos	fungi: yep	18:58
fungi	(which is just a rendering of the rst files from doc/source in the nodepool source tree)	18:58
hodos	i just have it runnig, creating and purgin instances but never get the nodes 'ready'	18:59
fungi	hodos: the first place to look is the logs from nodepoold (likely in /var/log/nodepool)	18:59
hodos	yes, doing that	19:00
hodos	fungi: the only error i get is this: http://paste.openstack.org/show/196928/ i don't know why i get it though	19:01
*** e0ne has joined #openstack-infra		19:01
*** sigmavirus24 is now known as sigmavirus24_awa		19:01
*** Somay has quit IRC		19:01
*** emagana has quit IRC		19:02
*** ujuc has quit IRC		19:02
*** e0ne is now known as e0ne_		19:02
hodos	i think it just hits the timeout and then removes the instance	19:03
fungi	hodos: that implies that your nodepool server is unable to ssh into the template instance it booted to try and make your initial instance	19:03
fungi	er, to try and make your initial image	19:03
*** emagana has joined #openstack-infra		19:03
hodos	fungi: yep. i guessed so too. but i don't understand why it can't ssh	19:04
*** dhritishikhar_ has quit IRC		19:04
hodos	fungi: i can do it manually using ssh -i /home/nodepool/.ssh/id_rsa	19:05
*** johnthetubaguy is now known as zz_johnthetubagu		19:06
*** e0ne_ is now known as e0ne		19:06
fungi	hodos: since it's using a snapshot-based workflow, nodepoold needs to know how to ssh into an instance booted from the base image. in your /etc/nodepool/nodepool.yaml does your providers->images->private-key have /home/nodepool/.ssh/id_rsa set (and does the username also match whatever you're testing as)?	19:07
*** yfried__ has quit IRC		19:07
*** carl_baldwin has quit IRC		19:07
*** yfried__ has joined #openstack-infra		19:07
openstackgerrit	Andrea Frittoli proposed openstack-infra/project-config: Add an experimental stress identity v3 test https://review.openstack.org/168113	19:07
hodos	fungi: yes	19:07
*** annegentle has joined #openstack-infra		19:08
fungi	er, sorry, actually that's for the node ssh access. this stage is dependent on providers->username and providers->password	19:08
fungi	no, sorry again, that's for nodepoold authenticating to the provider's api endpoint	19:09
hodos	hm, i thought providers-> username and providers->password should be for nova api access ?	19:10
fungi	mordred: jeblair: is nodepool relying on nova keypair functionality to initially log into template instances?	19:10
*** mmedvede has quit IRC		19:10
fungi	trying to remember how it obtains initial access to configure the template	19:10
*** carl_baldwin has joined #openstack-infra		19:11
hodos	why doesn't it use the keypair since it's injected on the boot by the openstack	19:11
*** e0ne is now known as e0ne_		19:12
fungi	hodos: i think it does. what username are you authenticating as? it tries several usernames since base images for some distros like to disable the root accouhnt	19:13
hodos	fungi: it just tries 'root' all the time. i see the code where it iterates through 'ubuntu', etc., but it never gets to that point...	19:13
hodos	fungi: and my nodepool config has 'ubuntu' as an image user	19:14
openstackgerrit	Joe Gordon proposed openstack-infra/project-config: Remove tempest-dsvm-ceilolmeter-mongodb job https://review.openstack.org/167695	19:14
fungi	yeah, if it gets an auth rejection for root it should try ubuntu. though as you said your log is showing a socket timeout exception so that implies your nodepool server is never reaching the ip address it thinks is associated with that template instance	19:15
mordred	fungi: yes	19:15
fungi	hrm, though in this case it's nodeutils.py raising the timeout rather than the socket library the last person today (i forget the nick) was running into	19:16
*** e0ne_ has quit IRC		19:17
*** crc32 has joined #openstack-infra		19:17
fungi	but i think they were having trouble getting nodepool to log into the booted nodes rather than the template instance, so nodeutils may just be masking that exception	19:17
fungi	hodos: so yes, as mordred confirmed, nodepoold should be trying to log into the instance via whatever keypair it supplied via the nova api	19:19
*** AJaeger has joined #openstack-infra		19:19
*** crc32 has quit IRC		19:19
hodos	fungi: this is not true, as there's a piece of code in bootstrapServer method: host = utils.ssh_connect(server['public_v4'], 'root', ssh_kwargs, timeout=CONNECT_TIMEOUT)	19:20
*** ghostpl_ has quit IRC		19:21
hodos	hodos: i think this is the place where i get the exception	19:21
*** kgiusti has joined #openstack-infra		19:22
mordred	hodos: so it's throwing an exception rather than connect timing out	19:23
mordred	or, rather, it's timining out and throwing an exception rather than just failing to log in as root	19:23
hodos	mordred: yes, i also looked into image log. there's an error that causes the issue? http://paste.openstack.org/show/196931/	19:24
fungi	right, this is coming from iterate_timeout() so it's basically hitting a retry max	19:24
mordred	oh! no	19:24
mordred	hodos: you have an error in your puppet	19:24
mordred	can you paste in just the snippet of your jenkinsuser.pp where your key is getting set?	19:25
*** lifeless1 is now known as lifeless		19:25
mordred	hodos: $key wants to just be the key itself, without ssh-rsa and without the comment at the end	19:26
fungi	yeah, looks like http://git.openstack.org/cgit/openstack-infra/puppet-jenkins/tree/manifests/jenkinsuser.pp#n58 expects just the ascii key encoding, and uses additional fields for key type and comment	19:26
fungi	hodos: ^	19:26
*** notnownikki has quit IRC		19:27
Shrews	mordred: shade question: what should be the "correct" behavior when deleting a resource by name, and multiple resources with that name may exist? Should we disallow deleting by name and only by ID, delete the first resource with that name, or raise an error?	19:27
Shrews	I lean toward raising an error, but much of our code does the 2nd thing and deletes the first it finds	19:27
fungi	Shrews: the current .*clients seem to raise an error when there's >1 match	19:28
Shrews	fungi: right	19:28
fungi	at least from what i've encountered	19:28
fungi	"pick the first hit when they're likely returned in an arbitrary/random order" seems very dangerous to me	19:28
hodos	mordred: so i have the same jenkinsuser.pp as the infra repo	19:28
*** yfried__ has quit IRC		19:29
fungi	hodos: right but where are you supplying that nexenta@ubuntu public key?	19:29
mordred	Shrews: I lean towards throwing an error if more than one host is matched	19:30
*** crc32 has joined #openstack-infra		19:30
fungi	hodos: it should just be the unbroken base64 encoded key without the "ssh-rsa" at the beginning and without the "nexenta@ubuntu" comment on the end	19:30
openstackgerrit	Marton Kiss proposed openstack-infra/project-config: Add puppet-pgsql_backup as a separate puppet module https://review.openstack.org/167631	19:31
mordred	Shrews: but allowing the input parameters for find, update and delete to all be the same, so that if we figure out that "get_server(name_or_id, some_other_parameter)" is a reasonable way to express uniqueness	19:31
mordred	Shrews: that shoudl carry over to update and delete too	19:31
hodos	fungi: ok, but where do i set it? right in this file?	19:32
mordred	Shrews: this will actually get nicer if we get that refactor for delete to be more like cloud.get_server(args).delete() or something	19:32
Shrews	mordred: i agree. so we have a bug. shade needs bug tracking, btw. I would set it up, but I'm not on the core team.	19:32
mordred	Shrews: https://storyboard.openstack.org/#!/project/760	19:33
fungi	hodos: there are several places you could be supplying it. what i asked is where you _are_ supplying it (NODEPOOL_SSH_KEY envvar for the prepare_node.sh calling environment maybe?)	19:33
Shrews	mordred: oh yay	19:33
fungi	hodos: wherever your puppet manifest is picking up your key content from, that's where you need to trim the string you're supplying	19:34
*** mriedem has quit IRC		19:34
*** e0ne has joined #openstack-infra		19:35
hodos	fungi: i do set NODEPOOL_SSH_KEY before launching nodepoold. i thought that's the place.	19:35
openstackgerrit	Merged openstack-infra/devstack-gate: Move tempest log documentation into devstack-gate https://review.openstack.org/167143	19:35
openstackgerrit	melanie witt proposed openstack-infra/project-config: Exclude the slow tag for the Nova Cells V1 job https://review.openstack.org/168121	19:35
openstackgerrit	Merged openstack-infra/project-config: Add coreos-image-builder project https://review.openstack.org/155868	19:36
*** timcline_ has quit IRC		19:36
mordred	fungi: it's possible that we haven't used that var since we changed how puppet does keys	19:36
mordred	fungi: and maybe the thing that sucks in that env var should do the trimming since it's a puppet detail that most folks won't grok	19:36
*** sandywalsh_ has quit IRC		19:36
*** pc_m has quit IRC		19:36
TheJulia	7	19:37
mordred	fungi: oh - nevermind	19:37
mordred	fungi: we document it properly in th eREADME	19:37
fungi	hodos: yep, so don't add "ssh-rsa" and "nexenta@ubuntu" when you're setting that variable	19:37
mordred	export NODEPOOL_SSH_KEY=`cat ~/.ssh/id_rsa.pub \| awk '{print $2}'`	19:37
yolanda	jeblair, i've set as WIP until i create the change for puppet-pip, not sure if all bits need to be abandoned yet	19:37
mordred	is the readme line	19:37
fungi	hodos: just the base64-encoded gibberish in the middle is all you should set in that variable	19:37
hodos	fungi: i understand and thats exactly what's there....	19:38
fungi	hodos: since jenkinsuser.pp (for right or wrong) is hard-coded to ssh-rsa type	19:38
*** timcline has joined #openstack-infra		19:38
*** masayukig has quit IRC		19:38
*** sandywalsh has joined #openstack-infra		19:39
*** mmedvede has joined #openstack-infra		19:39
fungi	hodos: and yet puppet is still throwing a "Key must not contain whitespace" error on it?	19:39
hodos	fungi: yes.	19:40
*** sandywalsh has quit IRC		19:40
mordred	hrm	19:41
*** achanda has quit IRC		19:41
hodos	mordred: echo $NODEPOOL_SSH_KEY	19:42
hodos	AAAAB3NzaC1yc2EAAAADAQABAAAAgQDTRQjbo1Yrmup9HFSKZuBHHo4kMwJ1jnuKunBbYyJWkTkx54mmP6EofBw7eX+9RJLkZJNEoOG4x/1C3ldPfZtUDgQaO4F6MPoH5rKRMnGif5BBdKKxOjWVg5+F7+mympIfXXRl1EVF0RXAGYIbF9cNVlzXzp+8foW2fgx7TlVckQ==	19:42
*** mestery has quit IRC		19:42
mordred	k. so somethign else is pulling in your full key	19:43
*** masayukig has joined #openstack-infra		19:43
*** carl_baldwin has quit IRC		19:43
fungi	it looks like it must be picking it up from somewhere else then, or or it wouldn't be seeing ssh-rsa and certainly not nexenta@ubuntu there	19:43
arosen	should pip freeze show that setuptools is installed? I'm a little confused that it isn't though pip install setuptools says it's installed.	19:43
fungi	did you maybe hardcode that into a prep script somewhere, or have a `cat ~/.ssh/id_rsa.pub` anywhere?	19:44
*** achanda has joined #openstack-infra		19:44
fungi	arosen: pip freeze doesn't mention setuptools, but modern pip versions have pip list which does	19:44
hodos	fungi: mordred: i'll try to find where it sits.	19:44
*** Longgeek has joined #openstack-infra		19:45
*** mestery has joined #openstack-infra		19:45
arosen	fungi: hrm i'm hitting this weird issue: http://paste.openstack.org/show/196933/ have you seen it?	19:45
yolanda	jeblair, thx for the reviews, i answered some of the comments	19:45
fungi	arosen: pip freeze traditionally tried not to show the syamdard packaging libraries	19:45
fungi	standard	19:45
openstackgerrit	Andrew Boik proposed openstack-infra/devstack-gate: WIP: (experiment) Test dual-stack with dependency https://review.openstack.org/168063	19:46
*** mriedem has joined #openstack-infra		19:46
*** btully has joined #openstack-infra		19:46
fungi	arosen: when i've seen that in the past, it's newer pip-installed setuptools importing older distro-installed pkg_resources	19:46
fungi	arosen: if this is a debian derivative, try `sudo apt-get remove python-pkg-resources python-setuptools python-pip` and see if that works	19:47
btully	not sure if this is the place to ask, but in Launchpad how do I reassign a bug to the original owner? I committed a patch and it automatically changed the bug to me as the owner, and there does not appear to be a way for me to set it back to the original owner	19:48
arosen	fungi: ah thanks that's it.	19:49
fungi	btully: you can't assign a bug to someone other than yourself unless you're a member of the bug-supervisor group for that project	19:50
arosen	mordred: do you have a sec to chat about raw image support for nodepool ?	19:50
fungi	btully: however you can unassign it if it's assigned to you	19:51
btully	ahh i see. thanks fungi	19:51
btully	shame that it autmatically changes assignee on commit	19:51
jeblair	fungi: yes nova keypair for first boot.	19:51
*** openstackgerrit has quit IRC		19:52
fungi	btully: usually it's assumed that if you're committing a new patchset to a proposed fix for a bug, that you're taking over the bug assignment. i agree that's suboptimal in some cases	19:52
*** openstackgerrit has joined #openstack-infra		19:52
btully	thanks for clarifying. i was scratching my head and just assumed the UI was broken	19:53
mordred	arosen: yes - but gimme just a sec ... I need to finish a braindump	19:54
stevebaker	hey, would I need an iptables rule somewhere for a CI guest vm to reach http://$HOST_IP:8899? http://logs.openstack.org/87/165287/11/check/check-heat-dsvm-functional-mysql/f1dec3e/console.html#_2015-03-26_03_02_44_680	19:56
*** annegentle has quit IRC		19:56
*** Longgeek has quit IRC		19:56
fungi	stevebaker: in devstack, yes	19:57
fungi	stevebaker: there are a few places, e.g. ironic support, where iptables manipulation happens in a pluggable way if you want to look at that for an example	19:58
*** annegentle has joined #openstack-infra		19:58
*** dkranz has quit IRC		19:59
stevebaker	fungi: ironic in the devstack repo?	19:59
*** achanda has quit IRC		19:59
stevebaker	ah found it	19:59
*** ominakov has joined #openstack-infra		20:00
fungi	stevebaker: yep	20:00
*** claudiub has joined #openstack-infra		20:00
*** ominakov has quit IRC		20:01
*** samueldmq is now known as samueldmq-away		20:01
*** achanda has joined #openstack-infra		20:02
*** MaxV has quit IRC		20:02
*** tnovacik has joined #openstack-infra		20:03
mordred	JayF: ping	20:04
mordred	arosen: hi!	20:04
arosen	mordred: yo	20:04
arosen	mordred: so I was thinking about adding a field here to nodepool.yaml called image-format: http://paste.openstack.org/show/196934/	20:05
mordred	JayF: reading the network_info.json patch - it looks like what's in the network_info dict in vendor_json will move to being top level when it lands in network_info.json, right?	20:05
mordred	arosen: looking	20:05
arosen	mordred: whoops i mean here one sec.	20:05
mordred	arosen: yes. I think that seems like a sane thing to add	20:05
arosen	http://paste.openstack.org/show/196936/	20:05
arosen	mordred: this way if you have a provider that doesn't support qcow images we can upload raw ones	20:06
arosen	and reuse the same qcow2 image.	20:06
arosen	So if we specify image-type: xxx then we'll call image-convert on the qcow2 image we built before we do glance-upload.	20:06
clarkb	so I think my cpu model didn't work	20:06
mordred	arosen: so - yeah - I thnk there need to be two things	20:07
clarkb	jogo: ^ tempest ran but 66 tests failed	20:07
openstackgerrit	Merged openstack-infra/project-config: Stop running jobs that don't install trove on trove changes. https://review.openstack.org/167679	20:07
mordred	arosen: there should be a parameter on the image description that is a list of image types to produce	20:07
openstackgerrit	Merged openstack-infra/puppet-ssh: Add the trusted source as class parameter https://review.openstack.org/167635	20:07
mordred	arosen: we need this ourselves - we know ahead of time that we want to output qcow2 and vhd	20:07
JayF	mordred: yes, that's my understanding. I'd also suspect we'll go a couple of releases with it in both places.	20:07
*** annegentle has quit IRC		20:07
mordred	arosen: then, as you point out, in the image-to-nodepool mapping like you have there	20:07
mordred	arosen: specifying which of the image formats should be uploaded is good	20:07
mordred	arosen: that way in the disk-image-create call, -t $comma_serparated_list_of_desired_output_formats can be added	20:08
*** dboik_ has quit IRC		20:08
mordred	arosen: that way greghaynes's patch to do more efficient things in dib in terms of outputting things can be leveraged	20:09
arosen	mordred: does disk-image-create already allow you to specify multiple image formats you wanted outputted?	20:09
mordred	arosen: since dib _starts_ with a raw	20:09
mordred	yes	20:09
mordred	there are two outstanding patches to make its support for that better	20:09
mordred	including one to optimize the raw outpout	20:09
mordred	output	20:09
mordred	since right now -t raw causes dib to convert frmo raw to raw	20:10
jlvillal	adam_g: I was thinking for 'stable' branch. What if on each project created a requirements.txt that capped each and every Python package. Including the transitive dependencies. Not sure if that is a good idea or not. But it is an idea :)	20:10
mordred	rather than just mv'ing the original raw file	20:10
mordred	:)	20:10
mordred	JayF: awesome. I was thinking I'd go ahead and add support to my script for looking in both locations	20:10
adam_g	jlvillal, projects already have a requirements.txt taht is a subset of global-requirements.txt	20:12
arosen	mordred: currently nodepool invokes disk-image-create like this: disk-image-create -x --no-tmpfs -o /root/nodepool_dib/devstack-trusty-dib-1427400704.qcow2 ubuntu vm openstack-repos puppet nodepool-base node-devstack	20:12
*** dkranz has joined #openstack-infra		20:13
arosen	err let me try the multiple -t's	20:13
mordred	arosen: its' a single t with a comma separated list	20:14
arosen	mordred: yup trying this: disk-image-create -x --no-tmpfs -t raw,qcow2 -o /root/nodepool_dib/devstack-trusty-dib-1427400704.qcow2 ubuntu	20:14
mordred	arosen: cool	20:14
*** timcline has quit IRC		20:14
mordred	arosen: it should work - although it will be mildly less efficient	20:15
arosen	mordred: just curious where it's going to write the file. I guess this will also make a : /root/nodepool_dib/devstack-trusty-dib-1427400704.raw	20:15
*** timcline has joined #openstack-infra		20:15
mordred	greghaynes: ^^ how do file extensions work with this ^^ ?	20:15
clarkb	jogo: libvirtError: internal error: Unknown CPU model gate64 so the cpumap didn't stick	20:15
*** esker has quit IRC		20:16
jogo	clarkb: :(	20:16
*** esker has joined #openstack-infra		20:17
*** hdd has quit IRC		20:17
*** kgiusti has left #openstack-infra		20:17
arosen	mordred: cool it looks like it worked.	20:17
mordred	arosen: woot!	20:18
arosen	it created it called: devstack-trusty-dib-1427400704.qcow2.raw	20:18
mordred	heh	20:18
mordred	arosen: what was the qcow called?	20:18
arosen	devstack-trusty-dib-1427400704.qcow2.qcow2	20:18
arosen	easy enough to fix that :)	20:18
mordred	:)	20:18
openstackgerrit	Andreas Jaeger proposed openstack-infra/project-config: Remove unneccessary noop jobs from zuul/layout.yaml https://review.openstack.org/168135	20:19
*** Somay has joined #openstack-infra		20:20
*** sandywalsh has joined #openstack-infra		20:20
*** dustins has quit IRC		20:20
jeblair	jogo: updated "topic:devstack-trove"	20:20
clarkb	jogo: I did restart libvirt-bin so not quite sure what else is missing	20:20
*** emagana has quit IRC		20:20
clarkb	jogo: sdague however, after talking to dansmith I think we should likely just use the i386 model	20:20
*** Sukhdev has quit IRC		20:21
clarkb	jogo: sdague: dansmith points out that if that model matters we aren't testing openstack anymore	20:21
*** ghostpl_ has joined #openstack-infra		20:21
clarkb	jogo: sdague: though for sanity we probably do want to use the same model across the baord but the specific model isn't very important	20:22
*** tqtran has joined #openstack-infra		20:22
sdague	iirc the guests run slower in i386 than x86_64 because we're doing instruction translations	20:23
clarkb	oh hrm	20:23
clarkb	I will continue to bang libvirt then	20:23
clarkb	eventually it will make a shape that works	20:23
sdague	I think starting with i386 in the multihost jobs so that we get it working but don't backslide would be fine though	20:24
sdague	and move to x86_64 later as an optimization	20:24
jeblair	clarkb is a test sculptor	20:24
*** Somay has quit IRC		20:24
openstackgerrit	Merged openstack-infra/project-config: Switch devstack-dsvm-f20 to centos7 for solum gate https://review.openstack.org/167413	20:25
fungi	chiseling away the bits	20:25
*** Somay has joined #openstack-infra		20:26
jogo	jeblair: nice I saw your grenade patch	20:27
clarkb	sdague: ok	20:27
clarkb	jogo: ^ I am much happier about doing that now	20:27
*** tqtran_ has joined #openstack-infra		20:27
*** tqtran has quit IRC		20:27
*** wenlock has quit IRC		20:27
jogo	clarkb: works for me	20:27
jeblair	jogo: i'm about to write what i _think_ are the last 2 patches in the series, then it's just seeing what broke	20:27
anteaya	pleia2: yup sorry I keep missing you	20:28
*** ghostpl_ has quit IRC		20:28
*** shardy has quit IRC		20:29
openstackgerrit	Ilya Shakhat proposed openstack-infra/project-config: Enable upload to pypi for Shaker project https://review.openstack.org/168047	20:29
*** annegentle has joined #openstack-infra		20:29
*** dprince has quit IRC		20:29
jogo	clarkb: so leave the main job on 64bit or change to i386 as well?	20:29
openstackgerrit	Spencer Krum proposed openstack-infra/system-config: Remove get-pip.py after using it in install_puppet https://review.openstack.org/168140	20:30
hodos	mordred: fungi: i think i solved the issue with the in puppet; there are no errors in the image.log. but i'm still experiencing the retry issue: http://paste.openstack.org/show/196938/	20:31
hodos	the !!! is my own debug string	20:31
anteaya	dims sdake__ this job has a 2 hour timeout for a functional job	20:31
*** samueldmq has joined #openstack-infra		20:31
anteaya	we are trying to have functional jobs finish in 1 hour	20:31
anteaya	https://review.openstack.org/#/c/164553/2/jenkins/jobs/magnum.yaml	20:31
dims	anteaya: will tweak it it's just experimental for now	20:32
sdake__	make sure its nonvoting ls dims	20:32
clarkb	jogo: sounds like sdague wants us to get to 64bit on this job too so likely leave main job 64bit for now	20:32
sdake__	pls	20:32
arosen	mordred: circling back on this... I think it might be better to just do this with one flag on the provider instead. Then on buildimage we could look at the image types we need for the providers and only build those.	20:32
dims	sdake__: it is nonvoting and experimental so won't run at all	20:32
dims	unless one of us triggers it	20:32
anteaya	sdake__: it is non-voting	20:32
sdake__	cool i didn't actually look at the patch :)	20:32
dims	sdake__: haha	20:32
*** SumitNaiksatam has quit IRC		20:33
anteaya	okay just be aware that we really need to pay attention to how long jobs take	20:33
dims	anteaya: yep thanks	20:33
anteaya	and repos copy pasting 120 minute timeouts is not a good habit	20:33
mordred	arosen: I'm also fine with that	20:33
arosen	mordred: k	20:33
anteaya	dims: can you comment on the patch that you have the intention of ensuring this job finishes in 60 minutes	20:33
anteaya	then we don't have runaway timeout issues	20:33
*** mattfarina has quit IRC		20:34
openstackgerrit	Spencer Krum proposed openstack-infra/system-config: Remove templatedir directive from puppet.conf https://review.openstack.org/168144	20:34
dims	anteaya: done	20:35
anteaya	dims: thank you	20:35
*** e0ne has quit IRC		20:35
*** otter768 has joined #openstack-infra		20:35
clarkb	nibalizer: https://etherpad.openstack.org/p/cloud-cpu-models	20:36
dansmith	sdague: I don't think i386 guests mean we're translating instructions	20:36
dansmith	sdague: may be other perf reasons, but I don't think it's that	20:36
clarkb	dansmith: ^ going to collect cpu models that we have there	20:36
jogo	clarkb: works for me	20:36
openstackgerrit	James E. Blair proposed openstack-infra/project-config: Enable trove plugin for trove tempest job https://review.openstack.org/168145	20:36
*** adalbas has quit IRC		20:37
clarkb	jogo: do you have links to jobs that failed because the cpus were different? trying to collect the info	20:38
samueldmq	anteaya, ping - you around? I had an idea and I'd like to share with you	20:38
openstackgerrit	James E. Blair proposed openstack-infra/devstack-gate: Revert "Temporarily add trove plugin configuration" https://review.openstack.org/168146	20:38
sdague	dansmith: some of the smaller word instructions don't exist in x86_64. I believe I saw a noticable speed difference when changing this in the gate.	20:38
openstackgerrit	Spencer Krum proposed openstack-infra/puppet-jeepyb: Modernize template syntax https://review.openstack.org/168147	20:38
*** carl_baldwin has joined #openstack-infra		20:38
*** eharney has quit IRC		20:38
*** emagana has joined #openstack-infra		20:38
dansmith	sdague: I don't doubt that there are some perf reasons	20:39
*** annegentle has quit IRC		20:39
clarkb	jogo: looks like it may be long enough that those logs were deleted :( I will be happy to have logs in swift	20:39
anteaya	samueldmq: share away	20:39
clarkb	anteaya: speaking of did new images cause any trouble?	20:39
anteaya	clarkb: not that I saw	20:39
jogo	clarkb: yeah would have to dig in a lot of logs to find it	20:40
fungi	hodos: were you the one earlier this week trying to figure out how to modify nodepoold to connect to an instance address labeled something other than "public"? your paste there shows you only have "private" returned in the addresses hash	20:40
*** annegentle has joined #openstack-infra		20:40
*** otter768 has quit IRC		20:40
anteaya	clarkb: fungi told me when images finished, except for one, and I didn't see any explosion on zuul with regard to python jobs	20:40
dansmith	sdague: https://rwmj.wordpress.com/2010/10/08/32-or-64-bit-virtual-cpu-in-kvm/	20:40
pelix	zaro: discovered that the sphinx linkcheck has a bug in it that will not result in it failing if links are broken https://github.com/sphinx-doc/sphinx/issues/1815, it will only output the failure. Once https://review.openstack.org/#/c/117478/ lands it might be possible to create a separate build job to run linkcheck as a separate job where we can look to scrap the output for errors until such time as sphinx has a release with the fix	20:41
dansmith	sdague: doesn't mention any such impact	20:41
anteaya	clarkb: here is a recent one: https://jenkins01.openstack.org/job/gate-ironic-python27/2473/console	20:41
anteaya	clarkb: looks good to me	20:41
samueldmq	anteaya, what about something like folding.stanford.edu to run tox jobs for projects	20:41
dansmith	sdague: but it doesn't matter why, if it's substantially slower, then that's fine	20:41
clarkb	anteaya: cool, so are we moving forward on the additional changes?	20:41
anteaya	samueldmq: what problem are we solving?	20:41
dansmith	sdague: but I'd hate to not get migration testing because the tests take 10% longer	20:41
samueldmq	anteaya, running tox tests still faster :)	20:41
anteaya	clarkb: I'm up for it, I like to do one change per image build run and evaluate myself	20:42
samueldmq	anteaya, and then jenkins would stay with gate jobs, etc	20:42
openstackgerrit	lifeless proposed openstack-dev/pbr: Fixup semver https://review.openstack.org/124651	20:42
openstackgerrit	lifeless proposed openstack-dev/pbr: Honour MANIFEST.in excludes https://review.openstack.org/123597	20:42
lifeless	dhellmann: mordred: also re pbr: I had these ^ pending unlanded for some reason	20:42
sdague	dansmith: I agree, which is why I said we should do i386 for the multihost case to prevent backslide	20:42
anteaya	samueldmq: okay well let's see some metrics	20:42
sdague	and move to 64bit as an optimization later	20:43
dansmith	sdague: okay, didn't hear that	20:43
anteaya	samueldmq: time a run of nova tests with tox now and then with your idea	20:43
*** sobersabre has quit IRC		20:43
*** dustins has joined #openstack-infra		20:43
sdague	yes, I've consistently said that, including in reviews :)	20:43
anteaya	samueldmq: I dno't know the framework you propose	20:43
*** dboik has joined #openstack-infra		20:43
openstackgerrit	Andreas Jaeger proposed openstack-infra/project-config: Cleanup jenkins/jobs/projects.yaml https://review.openstack.org/168148	20:43
anteaya	samueldmq: but if faster is what you are going for, there is one sure way to measure that	20:43
anteaya	samueldmq: make sure you are running tox in parallel when you test	20:43
anteaya	samueldmq: rather than serially	20:44
openstackgerrit	Merged openstack-infra/project-config: Add a DSVM job for Magnum for functional testing https://review.openstack.org/164553	20:44
samueldmq	anteaya, yes I am :)	20:44
*** rfolco has quit IRC		20:44
mordred	lifeless: does the semver path reflect what the fixed version will be?	20:44
samueldmq	anteaya, that would be something like keystone@home, did you see that link (folding.stanford.edu) ?	20:44
anteaya	samueldmq: great	20:44
*** dustins_ has joined #openstack-infra		20:44
fungi	anteaya: samueldmq: it's a distributed execution model (boinc et al). would pretty much need to distribute one job as a work unit, we wouldn't be able to break it up more granular than that because of the tight coupling in devstack integration/functional jobs	20:45
anteaya	samueldmq: I don't know what link you are thinking of	20:45
samueldmq	anteaya, but I agree, I dont' know how worth it can be without testing it	20:45
lifeless	mordred: TBH I don't recall. I just spotted the commits sitting around and was like WTF	20:45
*** emagana has quit IRC		20:45
lifeless	mordred: so I've rebased and pushed it up again	20:45
samueldmq	fungi, that would be just to the tox py27 tests, not functional ones	20:45
*** dustins_ has quit IRC		20:45
lifeless	mordred: they are old review #'s, so were pushed up in the past too	20:45
samueldmq	fungi, only for those unit/integration ones we already run in parallel locally	20:45
*** emagana has joined #openstack-infra		20:45
*** dustins_ has joined #openstack-infra		20:46
samueldmq	anteaya, folding.stanford.edu	20:46
fungi	samueldmq: ahh, yeah probably possible for the ones which already run under testr (which is most of them) but would need people donating resources by running a client to take on the work units	20:46
anteaya	clarkb: how many of the new stack could we merge before it affects what builds on the image	20:46
samueldmq	fungi, ++	20:46
samueldmq	fungi, we're a big community, arent we?	20:46
AJaeger	fungi, anteaya, clarkb: The shaker repo wants to upload to pypi as pyshaker - is this possible? See https://review.openstack.org/#/c/168047	20:47
mordred	lifeless: kk	20:47
samueldmq	fungi, donate you machine and get your tests running faster when you need as well	20:47
fungi	samueldmq: though it's probably easier to just ask our cloud provider member companies to donate more resources	20:47
anteaya	AJaeger: do they own pyshaker?	20:47
lifeless	mordred: I think you or dhellmann or anyone working on pbr trunk should review those first though	20:48
clarkb	anteaya: I thought the rest of the changes don't affect the image builds. Just the jobs	20:48
lifeless	mordred: if not take them over	20:48
AJaeger	anteaya: seems to be the case	20:48
*** dustins has quit IRC		20:48
samueldmq	fungi, but not as fun as this	20:48
samueldmq	fungi, each contributor may feel part of the infra for OS	20:48
samueldmq	:)	20:48
fungi	samueldmq: sure, fun _is_ important	20:48
clarkb	AJaeger: I think it iwll upload what the resulting sdist/whl ends up being	20:48
*** mwagner_lap has quit IRC		20:49
samueldmq	fungi, I'm just arguing about the ideas, etc , not exactly sure about how hard is to make it happen	20:49
AJaeger	clarkb: So, we need to check the repo whether it does it correctly?	20:49
anteaya	clarkb: cool, yes then I'm for merging the stack and holding back on the last one	20:49
clarkb	AJaeger: yes I believe so	20:49
anteaya	clarkb: the devstack-gate one has to go first	20:49
AJaeger	clarkb: so, "name = pyshaker" in setup.cfg should be all that's needed, correct?	20:50
*** erikmwilson has quit IRC		20:50
openstackgerrit	Merged openstack-infra/gear: Move running check in getJob https://review.openstack.org/159899	20:50
*** emagana has quit IRC		20:50
fungi	samueldmq: one thing you'll discover is that good ideas aren't in short supply here. available time to implement them is where we tend to get stuck	20:50
anteaya	samueldmq: and if you care os means operating system to me	20:51
*** dustins has joined #openstack-infra		20:51
fungi	ideas go here https://review.openstack.org/#/q/status:open+project:openstack-infra/infra-specs,n,z	20:51
samueldmq	fungi, yeah I understand	20:52
*** tgohad has quit IRC		20:52
samueldmq	fungi, I will dig a bit more on this idea and maybe write a spec later :)	20:52
fungi	cool!	20:52
clarkb	AJaeger: I think so, if you do a python setup.py sdist in that repo what do you end up with at the tarball?	20:52
*** dustins has quit IRC		20:52
samueldmq	fungi, maybe there are already frameworks to help doing so	20:52
*** tgohad has joined #openstack-infra		20:52
samueldmq	anteaya, ah sorry, I meant OpenStack, is there any abbreviation?	20:53
AJaeger	clarkb: ./dist/pyshaker-79073bc.tar.gz - so looks correct	20:53
*** dustins_ has quit IRC		20:53
greghaynes	fungi: mordred bindep question - Are we always going to be bindep'ing on the same platform as were targetting?	20:53
AJaeger	another question in this context as raised in the review: If publish-to-pypi template is used in zuul/layout.yaml, we can remove the release/pre-release tarball jobs, correct?	20:54
greghaynes	fungi: mordred basically, its fine to trust lsb_release to get platform info, I dont need to allow user specified platform info?	20:54
fungi	greghaynes: we're going to be running it either on a job worker at the start of a job to determine what to install, or inside a chroot when pre-caching things in dib nodepool images (or on template instances for the snapshot nodepool variant if that's still in place)	20:54
greghaynes	ah, ok. So yes I can trust lsb_release	20:55
samueldmq	anteaya, did you get the principle behind the idea I said, right? just to make sure you understood me	20:55
anteaya	samueldmq: o.o	20:55
fungi	greghaynes: though a command-line option to override lsb_release seems like a reasonable addition if we need it at some point	20:55
anteaya	samueldmq: means openstack.org usually in reference to a url at openstack.org like docs.o.o	20:55
anteaya	samueldmq: I honestly don't find it that difficult to type openstack	20:56
samueldmq	anteaya, hmm, I already saw this here in this channel :)	20:56
anteaya	yup	20:56
samueldmq	anteaya, no it isnt	20:56
fungi	OpSt	20:56
fungi	OpnStk?	20:56
samueldmq	haha	20:56
fungi	yeah, there's not a lot of great OpenStack abbrevs	20:56
anteaya	AJaeger: that is a good question	20:56
*** bswartz has joined #openstack-infra		20:56
anteaya	AJaeger: I don't know the answer but would also like to know what it is	20:57
AJaeger	sdake__: could you abandon https://review.openstack.org/#/c/149334, please? Or let's fix the real problem...	20:57
sdake__	done	20:57
AJaeger	anteaya: my understanding is that it's not needed but confirmation would be great indeed	20:57
clarkb	AJaeger: ya I think you can remove the explicit release and pre releaes jobs	20:57
sdake__	ajeager abandoned sorry I have so much crap in my jenkins screen I can't find anything :(	20:57
fungi	AJaeger: but also i believe the parsed list of jobs ends up in the log for the layout check. i might be mistaken about that	20:58
openstackgerrit	Spencer Krum proposed openstack-infra/system-config: Turn off implicit hiera lookups https://review.openstack.org/168156	20:58
AJaeger	fungi: yes, it does - let me check there	20:58
AJaeger	sdake__: thanks!	20:58
*** hdd has joined #openstack-infra		20:58
*** dustins has joined #openstack-infra		20:58
anteaya	sdake__: do you know how to use gerrit queries?	20:58
AJaeger	sdake__: before it gets lost: you need to rebase https://review.openstack.org/#/c/167848/	20:59
sdake__	anteaya yes	20:59
clarkb	jogo: fwiw I cannot find the second model of cpu on rax anymore, I have looked at ~20 rax job logs now and they are consistent.	20:59
clarkb	jogo: going to check logstash now	20:59
anteaya	sdake__: ah okay great, not everyone does	20:59
*** hogepodge has quit IRC		20:59
*** gokrokve_ has quit IRC		21:00
anteaya	nova meeting	21:00
*** hdd has quit IRC		21:00
clarkb	jogo: we don't seem to index thta file	21:00
openstackgerrit	Steven Dake proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack/magnum https://review.openstack.org/167848	21:01
sdake__	anteaya ^^ enjoy :)	21:01
clarkb	jogo: but ya best I can tell the CPUs are pretty consistent now	21:02
AJaeger	fungi: indeed, the setting from publish-to-pypi is used, so the extra tarballs can be remved...	21:02
* AJaeger is doing spring cleaning today...		21:02
jogo	clarkb: hrm, I don't remember which patch it failed on	21:03
*** resker has joined #openstack-infra		21:04
anteaya	sdake__: thanks will do after nova meeting	21:04
jogo	clarkb: here is a example of the error http://logs.openstack.org/30/141530/7/experimental/check-tempest-dsvm-aiopcpu-full/13c9f37/logs/screen-n-cpu.txt.gz?level=TRACE	21:06
jogo	clarkb: but that was on a patch trying to fix it	21:06
*** MaxV has joined #openstack-infra		21:07
*** esker has quit IRC		21:07
*** fifieldt has quit IRC		21:07
*** esker has joined #openstack-infra		21:07
*** MaxV has quit IRC		21:09
*** erikmwilson has joined #openstack-infra		21:09
*** erikmwilson has quit IRC		21:09
jogo	and may be slightly different	21:09
*** erikmwilson_ is now known as erikmwilson		21:09
*** resker has quit IRC		21:10
*** resker has joined #openstack-infra		21:10
hodos	jogo: nope, that wasn't me	21:10
hodos	fungi: nope	21:10
*** devlaps has joined #openstack-infra		21:11
openstackgerrit	Andreas Jaeger proposed openstack-infra/project-config: Remove extra tarball lines https://review.openstack.org/168159	21:11
AJaeger	anteaya: ^ 41 lines deleted ;)	21:11
*** dustins has quit IRC		21:11
anteaya	AJaeger: nice	21:11
*** SumitNaiksatam has joined #openstack-infra		21:13
jogo	clarkb: yeah I don't think we have the logs anymore :(	21:13
*** AJaeger has quit IRC		21:13
*** MaxV_ has joined #openstack-infra		21:13
*** esker has quit IRC		21:13
*** mtanino has quit IRC		21:14
*** esker has joined #openstack-infra		21:15
anteaya	I'm away tomorrow for most of the day but should be back for 2200	21:15
*** ldnunes has quit IRC		21:15
fungi	ianw: the image update log breakout patch seems to have worked great http://nodepool.openstack.org/	21:15
*** aysyd has quit IRC		21:16
*** mtanino has joined #openstack-infra		21:16
fungi	ianw: oh, except it's creating empty logs for anything built via dib	21:16
waynr	anyone have any opinions about whether or not it should be considered a bug that parameters from JJB job-group override parameters from JJB project?	21:16
ianw	fungi: yep ... also a follow-on for dib images in https://review.openstack.org/#/c/167837/	21:16
fungi	hah!	21:16
fungi	read my mind	21:16
ianw	fungi: i figured there would be bike-shedding over the output name of that one so left it to be split out	21:17
*** resker has quit IRC		21:17
waynr	hierarchically, a group can never contain a project so it seems strange to me that it can override settings at the project level	21:17
waynr	makes the job-group less re-usable	21:18
fungi	ianw: you mean as far as the dib. filename prefix?	21:18
*** weshay has quit IRC		21:18
ianw	yep	21:18
fungi	i would have skipped the prefixing, but i have a personal no-bikeshedding policy so fine by me	21:19
mordred	fungi: I like your policy	21:19
*** jcoufal has quit IRC		21:19
*** esker has quit IRC		21:19
openstackgerrit	Clark Boylan proposed openstack-infra/devstack-gate: Try to calculate common CPU feature set https://review.openstack.org/163514	21:19
clarkb	jogo: ^ thats a hand curated list, we shall see how this goes	21:20
*** esker has joined #openstack-infra		21:20
*** ibiris is now known as ibiris_away		21:20
*** jgrimm is now known as zz_jgrimm		21:20
fungi	mordred: though if you think i should rename my no-bikeshedding policy, we can certainly discuss it at length ;)	21:20
jogo	bespoke cpus	21:21
ianw	fungi: not sure if you saw my email to -dev ... to me it appears swift on centos uses a bunch more heap; it accounts for most of the memory usage difference	21:21
clarkb	jogo: all natural hand crafted silicon goodness	21:21
*** jamielennox\|away is now known as jamielennox		21:21
fungi	ianw: yep, your evidence is compelling	21:21
waynr	clarkb zaro pelix: where should I report JJB bugs?	21:21
clarkb	waynr: https://storyboard.openstack.org	21:22
jeblair	waynr: i've read your question, and honestly would probably need some examples to figure out the answer	21:22
jeblair	waynr: it's not immediately obvious to me	21:23
mordred	fungi: maybe it's a worldview and not a policy?	21:23
waynr	jeblair: okay, i'll try to include simple examples in my report	21:23
fungi	mordred: way of life	21:23
fungi	mordred: or perhaps a religion	21:23
waynr	I know what needs to be done to fix it in JJB source but figured I should seek consensus on whether or not this is a bug first	21:24
*** fifieldt has joined #openstack-infra		21:24
openstackgerrit	Ramy Asselin proposed openstack-infra/system-config: Use common log server implementation https://review.openstack.org/167426	21:25
*** stevemar has quit IRC		21:25
*** ghostpl_ has joined #openstack-infra		21:26
*** tonytan4ever has quit IRC		21:27
fungi	waynr: so you mean with a project instantiating a job group, if the job group has specific parameters set in it then you can't override those from the project? i can see the reasoning that allowing it is unlikely to offend the principle of least surprise	21:28
fungi	and increases flexibility	21:29
*** ociuhandu has quit IRC		21:29
*** ZZelle_ has joined #openstack-infra		21:29
jeblair	waynr, fungi: one question i have is that if the job-group does not specify those parameters, they are passed from project -> job(-template), right? so the only time this is an issue is if both the group and project supply them, and i'm not sure why someone would do that.	21:30
fungi	unless they see the job-group as a means to set a default value	21:31
jeblair	waynr, fungi: it seems like an 'override' situation, and i'm not sure whether something lower or higher in the stack should have priority, and off the top of my head, i don't know if we have an established pattern in jjb for that	21:31
jeblair	fungi: yeah, that's a reasonable interpretation. though i suspect the inverse may also be reasonable.	21:31
fungi	but yeah, that presupposes that you're including the same job-templates in multiple job-groups	21:31
jeblair	so it might come down to picking one	21:31
fungi	or else there would be no point	21:32
fungi	it does seem like a very corner case. i'd want to see a use case at least	21:32
openstackgerrit	Ramy Asselin proposed openstack-infra/system-config: Add puppet-openstackci to included modules https://review.openstack.org/167444	21:32
openstackgerrit	Ramy Asselin proposed openstack-infra/system-config: Use common log server implementation https://review.openstack.org/167426	21:32
*** gordc has quit IRC		21:32
fungi	in our situation we only ever (afaik) have a given job-template in a single job-group	21:33
waynr	fungi: i use the same job-group in multiple projects and want to set defaults at the job-group level that can be overridden by the project	21:33
*** ghostpl_ has quit IRC		21:33
*** tiswanso_ has quit IRC		21:33
waynr	so in my use cases it seems like the priority order should be defaults -> job-template -> job-group -> project	21:33
*** dkranz has quit IRC		21:33
waynr	in order of lowest to hights priority	21:34
waynr	my reasoning for this is that a project can't be referred in multiple places so it seems strange that the values it sets can be overridden by the job-group, which can be used in multiple places	21:34
waynr	i also have job-templates that are reused over and over again in many different job-groups	21:35
fungi	project is an artificial construct in the first place though right?	21:36
*** oomichi_ has joined #openstack-infra		21:36
fungi	i guess it's a defined key, but beyond that it just serves as a place to collect parameters	21:36
*** dkranz has joined #openstack-infra		21:37
fungi	so i guess the problem is that it can be argued that the reverse of your priority order could be someone else's preference	21:37
*** ivar-laz_ has joined #openstack-infra		21:37
*** ivar-laz_ has quit IRC		21:37
*** rlandy has quit IRC		21:37
fungi	and we seem to have that as an implicit (if somewhat arbitrary) choice in the current behavior	21:38
waynr	someone might argue that, but the limitation i mentioned is actually very real and one-way...you can specify a job-group within a project, but you can't specify a project within a job-group	21:38
fungi	i agree that your expressed preference makes more sense to me, but not necessarily enough to break the current behavior someone else might already be relying on	21:39
openstackgerrit	Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525	21:40
waynr	yeah, i understand someone else might be relying on this behavior...that's very unfortunate	21:40
fungi	maybe "relying on" is a strong word	21:40
*** ivar-lazzaro has quit IRC		21:40
fungi	the end result of their current configuration may change if that ordering is switched	21:41
waynr	what do you think about queuing the kind of override re-prioritization for the next major version change?	21:41
*** signed8bit has quit IRC		21:41
waynr	i could implement the reordering in such a way that the choice of override priority is configurable	21:41
*** bhuvan has quit IRC		21:42
*** Longgeek has joined #openstack-infra		21:42
*** carl_baldwin has quit IRC		21:42
waynr	s/kind of//	21:42
*** peristeri has quit IRC		21:42
jeblair	i'm not certain the original intent was cascading levels of override -- i believe our config usually explicitly sets all parameters, usually in the project definition	21:42
fungi	if it's something others are in favor of, maybe a global configuration switch to change that, defaulting to the current behavior, then in the next release switch the default for it, then in the release after that remove the option and clean up the cruft code associated with it	21:42
waynr	that sounds reasonable to me	21:43
jeblair	so at least with the provided example, i don't think we would have noticed it because we would have set the variable on each project explicitly	21:43
*** prad has quit IRC		21:43
fungi	and yes, i agree we (openstack infra) don't really set parameters on job groups in the first place	21:44
openstackgerrit	Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525	21:44
*** andreykurilin_ has joined #openstack-infra		21:45
openstackgerrit	Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525	21:47
*** bhuvan has joined #openstack-infra		21:48
pleia2	anteaya: still about? as I understand it, Apr 16 2014-Apr 16 2015 is fine for TC elections, but PTL should be Apr 1 2014 - Apr 1 2015, since the nominations period for PTL starts on Apr 2nd	21:48
*** annegentle has quit IRC		21:49
*** pelix has quit IRC		21:50
clarkb	my handcrafted cpu isn't workingeither	21:50
waynr	fungi, jeblair, clarkb: i'll go ahead and write out the storyboard/bug and work on a patch with the global configuration switch for review	21:50
clarkb	jogo: ^ I think maybe the bug is in how I am making the cpu	21:50
*** mpavone has quit IRC		21:50
jeblair	waynr: thanks	21:50
*** andreykurilin_ has quit IRC		21:53
*** andreykurilin_ has joined #openstack-infra		21:54
waynr	i wonder if there would be any value in more general global configuration switch that enables yamlparser features for a specific upcoming version of JJB, like "yamlparser-version: 2.x"	21:54
openstackgerrit	Stephanie Miller proposed openstack-infra/puppet-zanata: Add SMTP configuration options for Zanata https://review.openstack.org/168167	21:54
waynr	s/features/behavior/	21:54
pleia2	cinerama: you're on a roll :) I best get to testing+reviewing!	21:54
*** doug-fish has left #openstack-infra		21:55
cinerama	pleia2: good idea	21:55
*** jcoufal has joined #openstack-infra		21:55
anteaya	pleia2: here	21:56
pleia2	anteaya: hey	21:56
anteaya	pleia2: well it is your call	21:56
cinerama	pleia2: lots of progress, we just need to get things landed and deal with the actual functionality of zanata more	21:56
anteaya	the way I did it before, you can look at the prior wikipages	21:56
*** dkranz has quit IRC		21:56
anteaya	main wikipage > governance > prior elections	21:57
pleia2	ok	21:57
anteaya	is have one eligibility period for both, easier for the electorate to get their heads around	21:57
anteaya	easier for fungi to generate both sets of rolls at once	21:57
openstackgerrit	K Jonathan Harker proposed openstack-infra/nodepool: Reset the client object after proxy timeouts https://review.openstack.org/74943	21:57
waynr	i guess maybe it's not really necessary to set defaults at the job group level	21:57
anteaya	and the eligibility period ends at the same time ast the ptl nomination period ends	21:57
waynr	(and then override them at the project)	21:58
anteaya	but you and tristanC can do anything you like	21:58
fungi	and i'm happy to accommodate whatever you decide	21:58
anteaya	large groups tend to do what they did before, they don't adapt to change quickly	21:58
*** asettle has joined #openstack-infra		21:59
anteaya	but the big thing is the elgibility period has to end in time for the electoral rolls to be generated to run the elections	21:59
*** dkranz has joined #openstack-infra		21:59
pleia2	anteaya: I see, so the problem with April 16th is that's after the voting ends for PTL, so it needs to be at least before voting starts	21:59
anteaya	right	21:59
anteaya	you got it	21:59
anteaya	having two different elegibiltiy periods is mainly going to cause headaches for you	21:59
*** tqtran has joined #openstack-infra		22:00
pleia2	ok, I'll leave a msg for tristanC, in spite of vacationing he did just say he'd be "slow to reply" so hopefully he'll pop in before tomorrow and we can get this sorted	22:00
anteaya	as you try to explain to hte people who email you why they are unable to vote when they insist they can	22:00
pleia2	anteaya: thanks for your help	22:00
anteaya	sure	22:00
* pleia2 nods		22:00
anteaya	you will have people email you	22:00
anteaya	every election	22:00
anteaya	so whatever you do, you have to be able to go to sleep at night when you tell them they can't vote	22:01
anteaya	so any communication is for you to be comfortable with your decision	22:01
anteaya	since some folks will always insist they can vote when they can't	22:01
anteaya	so making them happy isn't possible	22:01
*** tqtran_ has quit IRC		22:01
openstackgerrit	Brant Knudson proposed openstack-infra/project-config: Experimental bandit job for keystone https://review.openstack.org/157595	22:02
*** MaxV_ has quit IRC		22:02
*** Longgeek has quit IRC		22:02
*** MaxV has joined #openstack-infra		22:02
*** asettle has quit IRC		22:03
anteaya	pleia2: also if you look at the governance wikipage linked from the main wikipage	22:04
pleia2	anteaya: yeah, I'm not so worried about saying no, rules are good :)	22:04
anteaya	pleia2: your wikipages for these elections should be linked in current elections	22:04
*** mwagner_lap has joined #openstack-infra		22:04
anteaya	pleia2: good, you will be forced to say no, glad you are comfortable with it	22:04
anteaya	some ballots legitimate go missing	22:04
anteaya	I can't remember the last time someone was legitamitely not on the voter's list	22:05
pleia2	yeah, I've run votes for the Ubuntu community before	22:05
anteaya	okay great	22:05
pleia2	fair number of "that's the wrong email address" and simple "I didn't get it/deleted it/have no control over email organization" :)	22:05
*** tkelsey has quit IRC		22:06
anteaya	ah	22:06
anteaya	we fixed the wrong email address issue when fungi fixed the scripts to collect the gerrit preferred email address, and included a warning to folks to check their gerrit preferred email address	22:07
anteaya	that helped a lot	22:07
*** MaxV has quit IRC		22:07
*** Swami has quit IRC		22:07
* pleia2 nods		22:07
*** annegentle has joined #openstack-infra		22:08
*** tsg has joined #openstack-infra		22:09
openstackgerrit	Khai Do proposed openstack-infra/jenkins-job-builder: Add ability to configure releaseEnvVar for maven-deploy publisher https://review.openstack.org/163626	22:10
anteaya	pleia2: but yeah a few everytime from corp email servers that eat the ballot	22:11
pleia2	fun times	22:11
*** tgohad has quit IRC		22:11
*** asettle has joined #openstack-infra		22:12
*** asettle has quit IRC		22:12
anteaya	not so bad once I learned that while condorcet doesn't keep a record you can obtain of voter's you input, it does know when you resend a ballot to the same email address and won't let both ballots be used	22:13
* pleia2 nods		22:13
*** asettle has joined #openstack-infra		22:13
anteaya	you just track down what email you used for that voter on the voter's list and send it again	22:13
*** sdake has joined #openstack-infra		22:15
*** andreykurilin_ has quit IRC		22:15
*** andreykurilin_ has joined #openstack-infra		22:16
pleia2	cinerama: good plan, thanks, I think I'll be focusing on desired behavior (use of variables, etc) rather than strict functionality, we'll knock it into working order in subsequent changes as needed	22:16
openstackgerrit	Brant Knudson proposed openstack-infra/project-config: Experimental bandit job for keystone https://review.openstack.org/157595	22:16
*** yfried__ has joined #openstack-infra		22:16
*** tnovacik has quit IRC		22:17
cinerama	pleia2: we need to make sure we can log in and stuff :)	22:18
*** rmcall has quit IRC		22:18
*** sdake has quit IRC		22:18
pleia2	cinerama: hah, yeah, basics are good! just will try to be less picky for now	22:18
*** sdake has joined #openstack-infra		22:18
*** nelsnelson has quit IRC		22:19
*** sdake__ has quit IRC		22:19
*** melwitt has quit IRC		22:19
pleia2	and I'm out all next week (family stuff back east), so I want to push this as much as possible so I don't hold things up	22:19
cinerama	pleia2: sounds good	22:19
openstackgerrit	Brant Knudson proposed openstack-infra/project-config: Experimental bandit job for keystone https://review.openstack.org/157595	22:19
openstackgerrit	Eric Larson proposed openstack-infra/project-config: Create oslo.cache project https://review.openstack.org/165224	22:20
*** marcusvrn has quit IRC		22:20
*** amitgandhinz has quit IRC		22:20
cinerama	pleia2: i might be out for a little bit in a few...have been waiting for a guy to finish up here	22:20
cinerama	pleia2: our doors actually shut now :)	22:20
*** erlon has quit IRC		22:21
pleia2	cinerama: stunning weather today, should enjoy it some :)	22:21
*** carl_baldwin has joined #openstack-infra		22:22
openstackgerrit	K Jonathan Harker proposed openstack-infra/nodepool: Reset the client object after proxy timeouts https://review.openstack.org/74943	22:22
*** timcline has quit IRC		22:23
elarson	pleia2: thanks for the comments on the review.	22:23
pleia2	elarson: sure thing, hopefully that'll get you to the finish line :)	22:23
david-lyle	did something change in the set up for the nodes running gate-horizon-dsvm-integration ?	22:24
anteaya	david-lyle: what are you seeing?	22:24
david-lyle	jobs have been failing all day, I put a skip on the offending integration test and now getting another job failure in the gate on another integrated test	22:25
david-lyle	https://review.openstack.org/#/c/168124/	22:25
david-lyle	is said commented out test	22:25
david-lyle	that job alone took over 50 minutes, on a local system, maybe 20	22:26
david-lyle	without the same setup time overhead of course	22:26
*** dims_ has joined #openstack-infra		22:27
*** dims_ has quit IRC		22:27
*** dims_ has joined #openstack-infra		22:27
*** dkranz has quit IRC		22:27
david-lyle	are the nodes smaller, slower, have a vendetta against horizon?	22:28
*** dimtruck is now known as zz_dimtruck		22:28
*** dims has quit IRC		22:28
*** ghostpl_ has joined #openstack-infra		22:29
openstackgerrit	Khai Do proposed openstack-infra/jenkins-job-builder: Added support for JClouds Blobstore https://review.openstack.org/107515	22:29
anteaya	david-lyle: yes we have a vendetta against horizon	22:31
anteaya	david-lyle: you found us out	22:31
anteaya	:)	22:31
jeblair	david-lyle: they change daily to pick up newer caches and update to the latest os, but i don't beleive any of the things you list have changed (also, the actual test run took just over 20 mins)	22:31
david-lyle	as long as it's out in the open	22:31
anteaya	ha ha ha	22:31
*** annegentle has quit IRC		22:31
*** lhcheng has joined #openstack-infra		22:31
*** Somay has quit IRC		22:31
*** rmcall has joined #openstack-infra		22:32
david-lyle	jeblair: watched the hover over on the job, mis-assumed what the time was measuring	22:32
jeblair	david-lyle: yeah, that's start to finish including devstack setup and teardown	22:33
jeblair	the tempest part was 1300 seconds	22:33
jeblair	er, testr	22:33
*** bknudson has quit IRC		22:33
jeblair	not tempest	22:33
*** ghostpl_ has quit IRC		22:34
jeblair	jhesketh: 121528 promises a next commit; is that ready?	22:34
david-lyle	jeblair, anteaya: thanks for the sanity check, will go back to applying head to wall :)	22:35
*** andreykurilin_ has quit IRC		22:35
anteaya	david-lyle: well enjoy that too	22:35
anteaya	david-lyle: come back if we can offer any more support/dose of reality	22:36
anteaya	:)	22:36
david-lyle	anteaya: will do	22:36
openstackgerrit	Merged openstack-infra/jenkins-job-builder: Fixed broken links in documentation https://review.openstack.org/167925	22:36
*** otter768 has joined #openstack-infra		22:36
*** tqtran_ has joined #openstack-infra		22:37
*** asettle has quit IRC		22:37
*** tqtran has quit IRC		22:40
*** otter768 has quit IRC		22:41
*** ddieterly has quit IRC		22:43
*** derekh has joined #openstack-infra		22:46
*** dizquierdo has quit IRC		22:47
derekh	fungi: looks like fedora image still failing to boot on rh1	22:47
derekh	ianw: did you see my question earlier	22:47
derekh	2015-03-26T16:33:01 <derekh> ianw: how did the problem this fix manifest itself ? https://review.openstack.org/#/c/165681/	22:47
*** gokrokve has joined #openstack-infra		22:47
derekh	2015-03-26T16:33:20 <derekh> ianw: [ 1.224080] VFS: Cannot open root device "UUID=d355e4e0-5805-45a4-a04c-cd4675044207" or unknown-block(0,0): error -6	22:47
fungi	derekh: what we were seeing (and only on rackspace afaik) was the bootloader saying it couldn't find a boot configuration over and over	22:48
*** marun has quit IRC		22:48
fungi	so that looks decidedly different	22:48
ianw	derekh: sorry, missed that. as fungi said, system wouldn't boot	22:49
ianw	derekh: but ... i would check that your extlinux is not corrupted	22:49
fungi	looks like your bootloader config is passing a block device uuid to the kernel as the root device, which is almost certainly not valid. should probably use a device name instead	22:49
derekh	fungi: ok, something is borked with the image being built, will look into it a bit tonight and pick it back up tomorrow	22:49
ianw	derekh: i have seen it do things before like try to boot new kernels with old initrd's because it didn't understand the layout of extlinux.conf	22:50
fungi	unless we really are able to reliably predict/set device uuids for the images we're building	22:50
ianw	(assuming you're using that)	22:50
*** carl_baldwin has quit IRC		22:50
*** jcoufal has quit IRC		22:50
derekh	fungi: yup, going to grab an image now and look, in the meantime want to just let it continue failing or disable something?	22:51
*** cdent has joined #openstack-infra		22:51
derekh	ianw: yup, ok thanks will take a look	22:51
anteaya	jhesketh: so https://review.openstack.org/#/c/167143/ merged, how many of the project-config stack should we merge before we stop to evaluate?	22:51
jeblair	greghaynes: if you have a minute (well, likely more than a minute unless you are just extra amazing), can you give 121528 another look?	22:52
fungi	derekh: as long as it's not causing you an issue by repeatedly trying and failing to boot there, i'd just leave nodepool configuration as-is for now	22:52
derekh	fungi: ok, its no issue with me, hopefully I'll get a solution fairly quickly	22:53
*** angela-s has left #openstack-infra		22:53
*** Somay has joined #openstack-infra		22:54
jhesketh	jeblair: re 121528, no it's not ready sorry. Completely fell off my radar, I should swing back to that next week (on leave for a few days at the moment sorry)	22:59
openstackgerrit	David Lyle proposed openstack-infra/project-config: Temp - making horizon integration tests non-voting https://review.openstack.org/168180	22:59
jhesketh	anteaya: it's pretty safe... We should be able to merge https://review.openstack.org/#/c/167158/ and check that any of the current devstack logs still hvae the right footers	23:00
jhesketh	anteaya: then the rest are about adding the footers to the swift copies of the files so it's extra features	23:01
asselin	\o/ jenkins passed my downstream-puppet "common log server" patch....through all the depends on AND cross-project depends on, etc. amazing feature.	23:02
*** asettle has joined #openstack-infra		23:02
*** sabeen1 has quit IRC		23:03
*** bookwar has joined #openstack-infra		23:03
ianw	fungi / clarkb : see the heap profiles at the bottom of -> https://etherpad.openstack.org/p/oom-in-rax-centos7-CI-job ; packaged pyOpenSSL v pip-installed seems to be at the root of the increased memory usage for swift	23:05
bookwar	if i want two groups OneGroup and AnotherGroup to have the same acls in gerrit, can i write rules as "create = group OneGroup AnotherGroup" or should i create two lines, one per each group?	23:05
SpamapS	hm	23:06
SpamapS	mordred: is there a digital form of the stuff we whiteboarded in Seattle anywhere?	23:06
SpamapS	mordred: I'm talking to some openstack operators and sysadminy types tonight and I think they'd like to see the nuts and bolts.	23:07
mordred	SpamapS: uhm - can you say more words about "stuff we whiteboarded"	23:07
mordred	SpamapS: do you mean the dog and pony show about current infra? or the plans for infra-cloud?	23:08
mordred	or smethign else	23:08
mordred	?	23:08
SpamapS	mordred: the dog and pony	23:08
SpamapS	mordred: I'm talking to people about how a bill becomes a law... I mean.. a patch becomes an OpenStack.	23:08
bookwar	now i need to see the dog and the pony too..	23:08
mordred	SpamapS: we have several slide decks	23:09
mordred	SpamapS: http://docs.openstack.org/infra/publications/ is a good index	23:10
mordred	SpamapS: I think you may want to look at http://docs.openstack.org/infra/publications/overview/#%281%29 or	23:10
mordred	SpamapS: http://docs.openstack.org/infra/publications/overview/#%2813%29 and http://docs.openstack.org/infra/publications/overview/#%2814%29 are good talking points slides	23:11
*** hichihara has joined #openstack-infra		23:11
mordred	http://docs.openstack.org/infra/publications/overview/#%2829%29 is also worth looking at and stepping through	23:11
jeblair	jhesketh: okay cool. talk to you next week :)	23:11
mordred	SpamapS: on the last one, clicking the mouse button will advance through the steps of teh animation	23:12
*** hichihar_ has joined #openstack-infra		23:13
*** timcline has joined #openstack-infra		23:14
*** timcline has quit IRC		23:14
*** timcline has joined #openstack-infra		23:14
*** hichihara has quit IRC		23:15
*** julim has quit IRC		23:16
anteaya	bookwar: why do you need two different groups to have the same permissions on the same repo?	23:17
anteaya	bookwar: you can include one group within another, example: https://review.openstack.org/#/admin/groups/459,members	23:17
openstackgerrit	James E. Blair proposed stackforge/gertty: Add held changes https://review.openstack.org/167838	23:17
openstackgerrit	James E. Blair proposed stackforge/gertty: Highlight starred changes in list https://review.openstack.org/166572	23:17
openstackgerrit	James E. Blair proposed stackforge/gertty: Sync own changes regardless of subscription https://review.openstack.org/168181	23:17
*** julim has joined #openstack-infra		23:18
anteaya	so the group that has permissions on the project-config repo is the project-config-core group, and infra-core is a member of that group	23:18
jeblair	that last change would have been helpful today (working on projects i'm not subscribed to)	23:18
jeblair	clarkb, fungi, mordred: ^ since you probably do that a lot you may be interested in 168181	23:19
SpamapS	mordred: OH! the mouse click is NOT obvious	23:19
jeblair	(oh, it's a slide with a dot. that's great.)	23:20
bookwar	anteaya: ah, that's actually what i need. I want to have the core "core group" - the one for the whole project, and additional core groups for each repository which is the small subpart of this project	23:20
anteaya	yup	23:20
anteaya	so one acl file creates the core core group	23:20
jeblair	bookwar: oslo does this too	23:20
anteaya	and then the project acl file creates teh project group	23:21
jeblair	bookwar: what's the project you're working on?	23:21
mordred	jeblair: nice!	23:21
*** timcline has quit IRC		23:21
anteaya	jhesketh: jeblair has a question about https://review.openstack.org/#/c/167158/, I'm holding off merging until the question is answered	23:21
bookwar	jeblair: stackforge/fuel-*	23:21
SpamapS	mordred: still, I like the idea of showing it as it is deployed, not conceptually.	23:21
anteaya	bookwar: fuel has quite a few groups already: https://review.openstack.org/#/admin/groups/?filter=fuel	23:22
SpamapS	mordred: I think that makes it easier for systems engineers to understand.	23:22
jeblair	anteaya, jhesketh, clarkb: if jhesketh is away on leave clarkb might be able to answer the question on 167158	23:22
anteaya	jeblair: ah okay	23:22
jhesketh	jeblair, anteaya: writing a response now ;-)	23:22
*** julim has quit IRC		23:22
jeblair	cool	23:22
anteaya	jhesketh: thanks	23:22
jeblair	bookwar, mordred, SpamapS: when we do the whiteboard thing, we do it progressively. i think that any written or recorded form should probably also do that	23:23
*** julim has joined #openstack-infra		23:23
jeblair	bookwar, mordred, SpamapS: because i think the end product would just make your eyes glaze over if that's the first thing you saw	23:23
jeblair	bookwar, mordred, SpamapS: but narrating it i think is very helpful in understanding it	23:24
bookwar	anteaya: yes, these are plugins groups, which were created recently, but we currently have one fuel-core group for all 'main' repos, like fuel-web,fuel-library,fuel-qa,fuel-docs and so on, and i need to split them too	23:25
* jeblair acquires strawberries		23:25
jhesketh	jeblair, anteaya: responded, let me know if it makes sense https://review.openstack.org/#/c/167158/	23:25
mordred	SpamapS: yeah - I agree with jhesketh - and I would love to put that presentation together - but we don't have it yet	23:25
openstackgerrit	Khai Do proposed openstack-infra/jenkins-job-builder: Allow JJB to set connection timeout to jenkins server https://review.openstack.org/151472	23:25
anteaya	bookwar: okay	23:25
anteaya	jeblair: mmmmm strawberries	23:26
anteaya	jhesketh: I understand your response, I'll wait for jeblair to take a look	23:26
*** lhcheng has left #openstack-infra		23:28
anteaya	savouring strawberries should never be rushed	23:28
*** smccully has quit IRC		23:28
*** cdent has quit IRC		23:29
*** ddieterly has joined #openstack-infra		23:29
*** melwitt has joined #openstack-infra		23:29
*** ghostpl_ has joined #openstack-infra		23:30
*** annegentle has joined #openstack-infra		23:30
*** amitgandhinz has joined #openstack-infra		23:31
zaro	waynr: would you like to comment on Di's reponse? https://review.openstack.org/#/c/150730/7/jenkins_jobs/builder.py	23:32
*** chlong has joined #openstack-infra		23:33
*** amitgandhinz has quit IRC		23:35
openstackgerrit	Ken'ichi Ohmichi proposed openstack-infra/devstack-gate: Add DEVSTACK_GATE_NOVA_API for switching Nova API https://review.openstack.org/163725	23:36
openstackgerrit	Ken'ichi Ohmichi proposed openstack-infra/devstack-gate: Add DEVSTACK_GATE_NOVA_API for switching Nova API https://review.openstack.org/163725	23:37
*** r-daneel has quit IRC		23:38
fungi	strawberries sound good, but i found a single-malt scotch aged in rum casks. that has stolen my attention for the night i think	23:38
fungi	only 14 years, but still remarkably smooth	23:39
*** dmellado has quit IRC		23:40
*** annegentle has quit IRC		23:40
*** ghostpl_ has quit IRC		23:40
*** baoli has quit IRC		23:42
*** dmellado has joined #openstack-infra		23:43
anteaya	ha ha ha	23:43
anteaya	sounds like you are all set	23:43
openstackgerrit	Steven Weston proposed openstack-infra/infra-specs: Third Party CI Dashboard Spec https://review.openstack.org/135170	23:44
waynr	zaro: sure, thanks for the heads up	23:44
openstackgerrit	Aleksandra Fedorova proposed openstack-infra/project-config: Create core groups for main fuel-* projects https://review.openstack.org/168182	23:45
*** dboik has quit IRC		23:45
bookwar	i wonder what kind of confirmation i need for this patch ^^	23:46
bookwar	going to add some of our core members to review	23:46
*** Longgeek has joined #openstack-infra		23:47
openstackgerrit	Swaminathan Vasudevan proposed openstack-infra/project-config: Add Neutron DVR aiopcpu job for multi-node testing https://review.openstack.org/158415	23:49
*** claudiub has quit IRC		23:50
*** Hunner has joined #openstack-infra		23:50
*** ajmiller has quit IRC		23:51
*** achanda has quit IRC		23:52
anteaya	why does fuel have so many repos	23:52
anteaya	I'm really not comfortable reviewing that patch	23:53
bookwar	anteaya: fuel gives you the ready to use deployment tool _and_ basically the framework to build your own openstack distribution. So there are deployment scenarios (fuel-library), build system (fuel-main), web interface(fuel-web), test subsystem (fuel-qa+fuel-devops) and so on, so yes, it is quite big	23:57
*** achanda has joined #openstack-infra		23:57
*** ZZelle_ has quit IRC		23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!