Tuesday, 2013-12-10

mroddenclarkb: i found a driver for straight lxc and one for lxc through libvirt as well, but that doesn't appear to offer much benefit over the plain LXC one00:02
mroddenclarkb: is there a reason you recommend against vagrant?00:02
mroddenits handy for the extra driver/platforms it offers (and the packaging for image retrieval/reuse)00:03
*** ryanpetrello has quit IRC00:03
clarkbmrodden: bceause it is another dependency00:03
clarkblibvirt is everywhere00:03
mroddentrue00:03
mroddenexcept windows and mac(?)00:03
mroddeni'm probably wrong00:03
*** herndon has quit IRC00:03
clarkbwe don't run our tests on those platforms00:03
clarkblxc doesn't exist on them either00:03
mroddenyep00:04
fungiin this case "everywhere" on our slaves00:04
*** HenryG has quit IRC00:04
clarkbmrodden: with libvrit I think you can just propose a change and it *should* just work00:04
fungiubuntu lts and latest centos6 specifically00:04
*** HenryG has joined #openstack-infra00:04
*** senk has joined #openstack-infra00:04
fungi(not everywhere in the known universe)00:04
clarkbunless we need lxc specific packages on top of our cgroup packages00:05
mroddeni didn't need to install anything special except lxc on ubuntu precise00:06
mroddenlxc, ruby1.9 and vagrant00:06
mroddenbut i can probalby get around vagrant00:06
mroddenand then it was just stuff that devstack brings in as well, like libxml2-dev and friends00:06
fungiworth noting, lxc's cgroups configuration may conflict with the cgroups limits we add for the jenkins account, yes?00:07
clarkblooks like we don't have that package installed yet00:07
clarkbfungi: I don't expect them too, cgroups are hierarchical and namespaced00:07
clarkbfungi: in theory they will do the right thing00:07
mroddenwhich package?00:07
clarkbmrodden: lxc00:07
fungibut we do set up a root namespace explicitly in our cgroups configs right?00:07
mroddenok00:07
mroddenmore worried about ruby1.9 TBH00:08
clarkbfungi: we mount the filesystems and create a jenkins namespace under that00:08
fungibeen a while since i had to fix anything related to it, so i'm probably just misremembering00:08
clarkb/sys/fs/cgroup/*/jenkins00:08
fungimrodden: i think we already install ruby 1.9.3 on precise00:09
mroddennice00:09
clarkbwe do, it is in the 1.9.1 pacakge because packaging00:09
mroddenyeah lol00:09
mroddenapt-get install ruby1.9.1 and you get 1.9.3 currently...00:09
mroddenmight need -dev too00:09
mroddenfor nokogiri or whatever00:10
mrodden ruby equivalent of lxml (python)00:10
fungi(the packages are named ruby1.9.1 for hysterical raisins)00:10
clarkbI need some of those raisins00:11
fungibasically, ruby decided to change api/abi compat between 1.9.0 and 1.9.1 if memory serves00:11
* mrodden is hungry now00:11
* mgagne is eating chocolate covered raisins00:12
fungiif they'd changed between 1.8.x and 1.9.0 then the packages would just be ruby1.900:12
mordredI'm going to pretend I did not see the recent scrollback00:12
fungimordred: good call00:12
*** oubiwann-lambda has quit IRC00:12
clarkbmordred: you don't enjoy vagrant and hysterical ruby raisins?00:12
funginothing to see here, move along00:12
mroddeni hate how vagrant decided to stop maintaining their ruby gem and opt for platform specific packages00:13
mroddenhow does that make any sense...00:13
mgagnemrodden: IMO, they needed a recent version of ruby and platforms couldn't provide it.00:15
mroddenanyways, since we don't want to use vagrant (which i'm fine with) i guess i'm going to look into the straight LXC stuff, since i dont want to write and maintain a libvirt/qemu driver for cookbook testing of openstack00:15
openstackgerritA change was merged to openstack/requirements: Add django_openstack_auth to projects  https://review.openstack.org/6092100:16
clarkbjeblair: ^00:16
mroddenmgagne: so they package a ruby interpreter with the .msi / .deb / .rpm i bet00:16
mordredclarkb: woot00:16
mgagnemrodden: https://groups.google.com/forum/#!msg/vagrant-up/kX_wvn7wcds/luwNur4kgDEJ00:16
mordrednicely done00:16
zaroclarkb: back to git submodules.00:17
fungieek submodules, by the way00:17
clarkbzaro: I didn;t manage to update g-g-p yesterday but I don't think it iwll be hard. Easier than the cloning idea00:17
mordredoy00:17
jeblairzaro: what?00:17
fungi(seriously, gerrit? distribute submodules in your source?)00:17
mordredwait - why are we submodulesing?00:17
clarkbzaro: and I thiink I convinced myself that zuul doesn't need to know about them explictly00:17
clarkbmordred: Gerrit00:17
*** jcooley_ has joined #openstack-infra00:17
zaroclarkb: gerrit plugins use submodules00:17
zarojeblair: ^00:18
jeblairzaro: gotcha.  :(00:18
fungidouble sad face00:18
mordredoh. so - if we want to track upstream gerrit repo and build it - we need to do a git submodule update00:18
clarkbmordred: yes00:18
mordredI think that's not a ton of stuff zuul or g-g-p need to learn about - just that one call00:18
*** rnirmal has quit IRC00:19
zaromore than just the call right?  the submodule init and update don't work on git.o.o00:19
clarkbzaro: ?00:19
clarkbgit.o.o doesn't need them to do anything00:19
clarkbg-g-p would do git submodule update --init00:19
fungiit seems not too hard to integrate into ggp. i just think gerrit's making a lot of assumptions about the git savvy of their downstreams. on the other hand, it's a git front-end, so maybe reasonable expectations of gerrit admins00:19
zarowe are building from git.o.o/openstack-infra/gerrit right?00:20
mordredconsumption of submodules isn't bad - it's hacking from it00:20
* fungi nods00:20
clarkbzaro: yes, we can either host the submodules too or just deal with them being elsewhere00:20
zarowhen i manually update submodule from the gerrit fork in git.o.o it doesn't work.00:20
jeblairanother reason we should move to tex for docs: http://xkcd.com/1301/00:20
*** sdake_ has joined #openstack-infra00:21
clarkbzaro: the gerrit fork doesn't have submodules00:21
mordredzaro: hrm. can you paste me the commands you're doing so I can watch it break?00:21
mroddenjeblair: so true; tex +100:21
fungii think we rebuild gerrit infrequently enough that unless we need to carry patches to plugins themselves, we probably don't need them imported as additional gerrit projects00:21
clarkboh nevermind we have a new branch in the fork00:21
mordredfungi: I agree- BUT00:22
zarosubmodules should be here:  http://git.openstack.org/cgit/openstack-infra/gerrit/tree/plugins?h=openstack/2.800:22
zaroit's in that folder in upstream00:22
mordredzaro: what if we make our own plugin? does that need to go into a submodule00:22
zaromordred: nope, i don't think so.00:23
clarkbmordred: no we can carry it out of tree, these are just plugins that are core and come with the war00:23
fungijeblair: i find their conclusion non-dubious, but wonder at the lack of citation of sources00:23
mordredthere's the issue00:23
mordredSubmodule 'plugins/replication' (ssh://mordred@review.openstack.org:29418/openstack-infra/plugins/replication) registered for path 'plugins/replication'00:23
zaroi suggested to git clone modules into plugin folder00:23
mordredit's doing relative submodule paths00:23
mordred    url = ../plugins/replication00:24
*** datsun180b has quit IRC00:24
funginot very forking friendly00:24
clarkbresizing servers takes forever00:24
fungiclarkb: or longer00:24
annegentlejeblair: that xkcd is hilarious00:24
fungithough i'd have placed .ppt at the very bottom there00:25
fungii've been lied to by almost every .ppt i've encountered00:25
fungiseems to be their primary purpose00:25
annegentlefungi: bwah ha00:25
clarkbif I ever write a .ppt I will put a slide in it with false info just for you fungi00:25
zaromordred, clarkb : have a suggestion?00:25
clarkbzaro: change your origin00:26
fungialso, pdf is getting a bit too much credit there. lots of "whitepapers" get dsitributed as pdfs these days00:26
clarkbsee if that fixes the relativeness00:26
zaroclarkb: not understanding.00:26
fungizaro: origin remote in .git/config00:26
fungipoint it to the upstream repository00:27
clarkbor we add all of those plugins to our git too00:27
clarkbs/git/gerrit/00:27
*** AaronGr is now known as AaronGr_AFK00:27
fungiseems messy00:27
openstackgerritElizabeth Krumbach Joseph proposed a change to openstack-infra/config: Configure pretty formatting for README files  https://review.openstack.org/6037500:27
* mordred poking at it00:27
clarkbalso add this to the list of reasosn why we should not use submodules00:27
mordredthis: http://paste.openstack.org/show/5474000:27
mordreddoes not work00:28
zaroclarkb, fungi : will give it a try.00:28
fungiwell, mordred just did00:28
fungioh, he tried overriding .gitmodules00:28
clarkbmordred: that isn't a git repo00:28
clarkbat least the one I just tried, I think your paths may be wrong00:29
clarkbor the name of the gerrit host00:29
mordredgot it00:31
clarkbmordred: https://review.openstack.org/#/c/60509/ btw :)00:31
* mordred cries00:31
mordredjd__: why do you hate me?00:31
clarkbI am going to approve that now00:31
clarkbunless you want to -2 so badly that you beat me to it00:32
mordredhttp://paste.openstack.org/show/5474100:32
mordredzaro: ^^00:32
mordredthat00:32
mordredplus "git submodule sync ; git submodule update --init"00:32
mordredworks00:32
*** rongze has quit IRC00:34
mordredhttps://review.openstack.org/6098400:34
mordredhttps://review.openstack.org/6098500:34
mordredzaro: ^^00:34
fungigerrit unit tests00:35
fungiwe've got another challenge ahead of us there00:35
fungi./tools/version.sh: No such file or directory00:35
*** bknudson has left #openstack-infra00:35
openstackgerritA change was merged to openstack-infra/config: Install ZooKeeper on slaves  https://review.openstack.org/6050900:36
*** senk has quit IRC00:37
*** EntropyWorks has quit IRC00:37
*** senk has joined #openstack-infra00:37
clarkbjeblair: btw https://review.openstack.org/#/c/60579/ you indicated you would approve it00:38
*** ilyashakhat_ has joined #openstack-infra00:38
*** ilyashakhat has quit IRC00:38
*** dripton has quit IRC00:38
openstackgerritA change was merged to openstack-infra/config: Add #heat to eavesdrop  https://review.openstack.org/6029800:38
*** dripton has joined #openstack-infra00:39
zarofungi, mordred : what's that failure all about?00:39
*** yamahata_ has joined #openstack-infra00:39
mordredzaro: the unittests job is still expecting the old maven semantics I think00:40
mordredzaro: did you ever get that ...00:40
mordredoh, you know what?00:40
*** EntropyWorks has joined #openstack-infra00:40
*** fifieldt has quit IRC00:40
fungizaro: our unit tests job for gerrit source changes tries to run ./tools/version.sh which presumably no longer exists in 2.8 land00:40
clarkbmgagne: Hunner https://review.openstack.org/#/c/60582/4/modules/openstack_project/manifests/eavesdrop.pp does the multiline string variable there work like that?00:40
fungior it's something we added in 2.4.x00:40
mordredclarkb, zaro we don't really need to update ggp - we can just put git submodule init ; git submuodule update  in the gerrit unittest job for the 2.8 branch00:40
clarkbmordred: zaro and I discussed that yesterday, I think we should do it once in g-g-p00:41
clarkbthen we never have to worry about it again00:41
*** herndon has joined #openstack-infra00:41
mordredk. I'm fine with that too00:41
fungiyou know, for kids00:41
mordredso I think, to be safe, we shoudl do "git submodule init ; git submodule sync ; git submodule update --init"00:41
mgagneclarkb: it "should" work: http://docs.puppetlabs.com/puppet/3/reference/lang_datatypes.html#line-breaks never tested it00:42
*** gothicmindfood has joined #openstack-infra00:42
mordredAaronGr_AFK: ^^ any opinions?00:42
*** davidlenwell has joined #openstack-infra00:42
clarkbmgagne: I think you are correct since we want to preserve the newlines. The problems we have had have to do with wanting to escape the newlines00:43
*** dcramer_ has joined #openstack-infra00:44
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Update submodules if they're there  https://review.openstack.org/6098600:44
mordredclarkb: there you go00:44
mordred(we're going to need to re-order those gerrit patches too)00:45
mordredbecause the submodule patch needs to come before the gitreview patch, else I believe the gitrveview patch will not pass tests00:45
clarkbis .submodules always a regular file eg not a dir?00:45
*** ianw has quit IRC00:46
*** davidlenwell_ has quit IRC00:46
jesusaurusclarkb: im pretty sure it is00:46
clarkbjesusaurus: thanks00:47
*** jcooley_ has quit IRC00:47
clarkbfungi: mordred: if I approve that will you guys be able to babysit?00:47
zarocool.  gotta take off for now.  back later to check up.00:47
fungiclarkb: i'm here all night--don't forget to tip your waiters00:47
clarkbapproved00:48
*** GheRiver1 has joined #openstack-infra00:49
*** openstack has joined #openstack-infra00:49
*** SpamapS has quit IRC00:49
* fungi is an eating machine00:49
openstackgerritA change was merged to openstack-infra/config: Update submodules if they're there  https://review.openstack.org/6098600:50
*** SpamapS has joined #openstack-infra00:50
*** dstanek has joined #openstack-infra00:50
*** jcooley_ has joined #openstack-infra00:51
fungiunless i missed some really terrible shell syntax error though, i don't expect that to cause any issues at all except maybe on a project with a .submodules file, which in our gerrit is probably only gerrit currentl;y00:51
fungier, .gitmodules file00:51
*** jcooley_ has quit IRC00:51
*** fifieldt has joined #openstack-infra00:52
*** ianw has joined #openstack-infra00:56
Hunnerclarkb: It will work, but you'll have extra newlines and spaces at the beginning and end of the variable string. As long as you don't care about the resulting output much...00:56
clarkbHunner: perfect that should work fine00:57
clarkbthanks00:57
*** changbl has quit IRC00:57
*** GheRivero has quit IRC00:57
clarkbfungi: https://review.openstack.org/#/c/60610/1 have an opinion on that?00:57
fungiclarkb: lgtm00:58
clarkbthanks00:58
fungimy choice of varied screen sizes and computing systems shouldn't get in the way of pragmatism, in my opinion00:59
fungimy beef is more in terms of poor/wasteful ui design00:59
clarkbfungi: sure, I just think you are a good sanity check for people less fortunate that I00:59
clarkbjust because i have 8GB of ram and a 24" display doesn't mean everyone does00:59
fungiright now i'm standing in front of a pair of 1440x2560 (portrait) displays01:00
openstackgerritA change was merged to openstack-infra/config: Bump up results per page in Kibana  https://review.openstack.org/6061001:00
fungieach is a 27" diagonal01:00
*** jorisroovers has quit IRC01:00
fungii just also happen to work from a 640x480 set of headmount goggles. i pick my battles01:01
clarkbpaste seems to be going through its I am down period of the resize right now01:01
*** yongli_away is now known as yongli01:01
* dims is staring a old laptop :)01:01
*** changbl has joined #openstack-infra01:02
fungidepending on what room i'm in, i also find myself working from netbooks with 7" displays and resolutions in the 600x480 or 1024x800 range01:02
fungier, i made up those last couple resolutions, but it's something like that (never can remember them precisely)01:03
fungi1024x600 and 800x48001:04
clarkbsdague: you don't happen to be around do you?01:07
*** fifieldt has quit IRC01:07
* clarkb took the plunge, approved several e-r dashboard changes01:10
openstackgerritA change was merged to openstack-infra/elastic-recheck: move the graph generating code over to facets  https://review.openstack.org/6060901:12
openstackgerritA change was merged to openstack-infra/elastic-recheck: sort the graph list by number of fails  https://review.openstack.org/6061301:12
openstackgerritA change was merged to openstack-infra/elastic-recheck: set y axis on failure rate graph  https://review.openstack.org/6063001:12
clarkbfungi: could I bug you to do independent paste.o.o verification?01:12
fungiclarkb: you bet01:12
clarkbresize is asking me to confirm now and best I can tell it is happy01:12
fungii'll poke around a few places01:12
clarkbfree -mo reports 2GB of memory, service responds to force refreshes01:13
fungiyep, pulled up and reloaded some recent pastes from my history, resource consumption looks good01:14
fungii say confirm it01:14
*** jhesketh_ has quit IRC01:17
clarkbk doing that now01:18
*** melwitt has quit IRC01:18
*** adalbas has quit IRC01:22
clarkball done01:22
*** gothicmindfood has quit IRC01:22
*** mrodden has quit IRC01:26
clarkbI want to point out https://review.openstack.org/#/c/59787/ according to e-r it hit 5 bugs01:26
*** AaronGr_AFK is now known as AaronGr01:26
clarkbacross 4 test failures01:27
fungii *hope* that's a high score01:28
*** nosnos has joined #openstack-infra01:28
*** senk1 has joined #openstack-infra01:28
*** senk has quit IRC01:29
clarkbwoot sdague's e-r changes that I approved seem happy01:31
clarkbnotice the graphs on the elastic-recheck status page are sorted no01:31
dimsclarkb, one of the bugs has a "SUCCESS" in the legend with blue line, others don't seem to.01:33
clarkbdims: yeah that means that a job that ran successfully matched the signature01:34
clarkbwe do our best to write signatures that only match failures whcih is why most don't have that01:34
dimsah. ok.01:34
openstackgerritPranesh Pandurangan proposed a change to openstack-infra/config: Added new project, entropy  https://review.openstack.org/6013201:34
clarkbthe signature for that bug should probably be tightened up01:34
*** thedodd has quit IRC01:37
dimsclarkb, thanks for the explanation. i can see that query with successful builds01:37
*** ryanpetrello has joined #openstack-infra01:42
*** ljjjustin has joined #openstack-infra01:43
*** senk1 has quit IRC01:43
*** mrodden has joined #openstack-infra01:43
krotscheckWho runs gerritt? I'd like a sample of current browser/OS usage so we've got a good sense of what we can build storyboard in.01:45
mordredwe do!01:45
krotscheckOh goody!01:45
krotscheckSo if I want google analytics added, I would do that a soon as I have the necessary permissions on monday?01:46
lifelesssdague: hi, around? https://review.openstack.org/#/c/59303/01:46
lifelesssdague: I want to make sure I am not thoroughly confused :)01:46
clarkbkrotscheck: anyone can propose the cahnge (you could do it tonight), not sure how jeblair would feel about google analytics though01:46
mordredkrotscheck: well... you'll have a hard time selling jeblair on using the google - do you know of any good post-processing things we can run on the apache logs?01:46
lifelesskrotscheck: uhm, adding google analytics is entirely separate to answering your question :)01:47
fungikrotscheck: you would be strung up by your toes and beaten until you recant01:47
krotscheck….because google is… evil?01:47
mordredkrotscheck: well, that. and non-free01:47
lifelesskrotscheck: because Google is non-free01:47
clarkbnot to mention the overhead, iirc most of the dosc load time is google analytics01:48
fungiwe could probably use google analytics as long as they'd agree to free up all their source code for it01:48
lifelessfungi: see, we need some actual policy around this01:48
krotscheckWow, new world.01:48
AaronGrapache -> lumberjack -> logstash -> es+kibana3 ?01:48
fungibut i'm not keen on my browser spinning on broken google analytics privacy-invading spider callbacks on every page refresh either01:48
lifelessfungi: because while I can get behind it, it's more than a little out of kilter with the rest of OpenStacks vendor stuff01:48
mordredAaronGr: lumberjack?01:48
mordredwe could certainly run awstats on our access logs01:49
AaronGrflat file shipper that ties into logstash/etc01:49
lifelesskrotscheck: but more importantly01:49
AaronGrfor when rsyslog isn't realistic for whatever reason01:49
mordreda - well, shipping log files into logstash we've got01:49
mordredthat's actually how we get all the build logs into it01:49
fungithere are a number of log post-processing tools we could manually run to get some spot checks too if it's not something we want to install and leave running long-term01:49
lifelesskrotscheck: gerrit is slow enough already, I'd -far- prefer a server side analyser than a web-dot one01:49
clarkbmordred: sort of, I wouldn't put our server logs into the public logstash/elastisearch (simple enough to build a different backend for that data)01:49
AaronGrmordred: oh. cool01:50
krotscheckHonestly I don't care how we do it, I just want to know who's using what.01:50
clarkbkrotscheck: ++01:50
lifelesskrotscheck: erm, no... you want to know how many people use what.01:50
* fungi can | cut | sort | uniq -c01:50
lifelesskrotscheck: I don't think you have the right, nor should you have interest, in who is using what :)01:50
*** senk has joined #openstack-infra01:50
krotschecksemantics. You know what I mean.01:50
clarkbis this a who is on first joke?01:50
lifelesskrotscheck: I do now.01:51
lifelesskrotscheck: I've known folk that would have meant what you wrote, so I have learnt not to assume.01:51
*** rongze has joined #openstack-infra01:51
clarkboh you meant it literally, carry on01:51
*** yamahata_ has quit IRC01:51
*** rongze has quit IRC01:51
krotscheckFact is that I want to know whether I have to render all my HTML server side or whether I can do it via an all JS client.01:52
krotscheckyadda yadda yadda web tech selection etc.01:52
lifelessyup, got that. Fastest way to do it is probably to zvengali clarkb or fungi into doing a quick spot check on the logs to get a point in time aggregate for you.01:52
* mordred copying gerrit access log to the side and will run awstats on it for you01:52
*** rongze has joined #openstack-infra01:52
krotscheckThanks.01:53
lifelesshow often do you think we'd want to revisit the choice?01:53
*** julim has quit IRC01:53
*** rongze is now known as 36DABWUXY01:53
*** rongze has joined #openstack-infra01:53
*** rongze is now known as 45PAAS1G101:53
lifelessif it's like every 6-12 months, I suggest just doing spot checks as needed01:53
fungiwe have users on a variety of limited-resource platforms all over the world, and server resources are effectively unlimited for us, so anything which can sanely happen server-side should01:53
mordredugh. vomit01:53
lifelessfungi: mmm01:53
lifelessfungi: How resource limited?01:53
mordredwhy do people make tools that require you to make a config file???01:53
fungireduce round-trips as much as possible too, because lifeless browses from tin cans and string on an island somewhere in the pacific01:53
krotschecklifeless: Depends on how far back we have to set our "These are the browsers that storyboard will support" stake. If it's somewhere around IE6, I'm going to want daily updates.01:54
lifelessfungi: I'm hugely in favour of client side rendering, because it's faster for me :001:54
clarkbkrotscheck: no no, I don't think we need to support IE at all, but I may lose that argument :)01:54
*** dkranz has joined #openstack-infra01:54
mordredI don't think we do either01:54
lifelesskrotscheck: IE6 should be taken out back and shot:)01:54
fungii would be strongly in favor of breaking ie entirely, as long as we could come up with a passable excuse01:54
lifelessI'm betting on some huge proportion MAC, the Linux, and a small IE group01:54
lifelessfungi: 'something better is free' ? :)01:55
*** senk has quit IRC01:55
fungi"anything free is better"01:55
lifelessDoes gerrit support IE6 ?01:55
clarkblifeless: no01:55
*** senk has joined #openstack-infra01:55
lifelessI'm not sure LP does anymore01:55
lifelessso01:55
krotscheckWe could just go with what Google does, which is "current for auto updating browsers, 2 version back for everything else"01:55
fungii haven't seen ie6 in longer than i like to remember, so no idea01:55
lifelesswe clearly don't need to support anything that gerrit + LP - the incubment system - do not.01:55
clarkbthey "support" IE that google supports which is either 7 or 8 now, but it breaks all the time because tehy don'y test it01:55
fungii'm more a fan of web applications coded to some industry standard specification, with as few browser-specific workarounds as possible01:56
*** julim has joined #openstack-infra01:56
*** ^d is now known as ^demon|away01:56
krotscheckIndustry standard specification? In browsers? Really?01:56
*** 45PAAS1G1 has quit IRC01:57
lifelessfungi: so seriously, how resource constrained? The reason I like client side rendering is that less data == smaller pipe needed to render == faster.01:57
lifelessfungi: As you note, I'm basically your acid test for high latency low(ish) bandwidth link.01:57
*** jerryz has quit IRC01:58
lifeless+ incremental updates to the page by pushing a few hundred bytes of JSON is massively better than pushing an entire new HTML page out.01:58
krotschecklifeless: Urm- where are you?01:58
fungilifeless: i'm a fan of client-side rendering under those circumstances. my beef is that a lot of apps get written with half the webserver reimplemented on the client because the devs don't know any better01:58
lifelesskrotscheck: NZ01:58
krotschecklifeless: Color me jealous01:58
lifelesskrotscheck: :)01:58
lifelesskrotscheck: Thats a shade of Puce, right?01:58
krotscheckMore mauve, really01:58
fungikrotscheck: but point is our dev effort is distributed to all points on the globe, so assumptions about available bandwidth, round-trip times et cetera are best assumed to be worst-case01:59
krotscheckfungi: Gotcha. So sideloading references and heavy use of caching.01:59
lifelessall your basic 'we've learnt to write things better' lessons01:59
lifelessyeah01:59
krotscheckWe can adhere to the 1/3/10 second rule.02:00
fungiand if a machine with, say, a browser fullscreened on 1024x768 display is painful to use with it, or when that's the only page you've loaded, with nothing else running, and your browser starts eating more than 1gib of ram... not very considerate02:01
fungiwell-written web application, in other words. as opposed to most proprietary crap out there02:01
funginot the "i wrote my own gui from scratch on top of recursive xml/soap calls02:03
clarkbfungi: don't worry the new gerrit change screen is breaking that :) to be fair it is a user toggleable option to enable and disable thankfully02:03
mordredkrotscheck: yesterday...02:03
mordred1 129961 67.84% Mozilla/502:03
mordred2 16280 8.50% git/1.7.9.502:03
mordred3 11697 6.11% python-requests/1.2.3 CPython/2.7.3 Linux/3.2.0-49-virtual02:03
mordredfunny ones:02:03
mordred0.88% Browser: Netscape 4.802:03
fungiw00t!02:03
mordred0.31% Browser: Internet Explorer 3.0 (win95)02:04
krotscheckThat's AOL02:04
mordredthat's the top IE stat, btw02:04
krotscheckwhaaaaa02:04
fungiteehee02:04
mordred0.31%02:04
*** 36DABWUXY has quit IRC02:04
krotscheckWow, all firefox? Fascinating.02:04
fungisomeone with a browser string plugin trolling02:04
clarkbmordred: any way you can ignore git and python-requests since git is well our CI system02:04
*** rongze has joined #openstack-infra02:04
*** ^demon|away has quit IRC02:04
clarkbpython-requests is chmouel or whoever scraping json02:04
krotscheckMozilla is often overreported, because chrome identifies as that as well02:05
krotscheckAs does IE02:05
* fungi doesn't use firefox at all. totally not firefox. iceweasel isn't firefox, right? ;)02:05
clarkbfungi: how far behind is iceweasel typically?02:05
clarkbthat would actually bug me quite a bit if/when I switch to debian02:06
mordredah02:06
fungiclarkb: depends on what suite you run. on unstable i've got 17.0.902:06
mordredI had bad config - slightly different numbers when I clean it a bit02:06
mordred1 80214 41.87% Browser: Safari (OSX)02:06
mordred2 49582 25.88% Browser: Firefox02:06
fungiclarkb: and i haven't updated in a week or so02:06
clarkbfungi: so only about 8 releases behind :/02:06
fungiclarkb: nevermind--i'm on testing. unstable has 24.1.0 and 25.0 is in experimental right now02:07
krotscheckmordred: Cool, that works for me.02:07
mordredfor the record, i'm running:02:07
mordredawffull -a 'git' -a python-requests -a Python-urllib -v -v gerrit-ssl-access.log02:07
*** mrodden has quit IRC02:07
* fungi forgot he only had the kernel and a handful of xorg packages pinned form unstable trying to get new hardware going02:08
mordredfor the one file I processed, which is a single day file, we have:02:08
mordredTotal Hits 22488002:08
mordredTotal Volume 16.39 GB02:08
mordredHits per Hour 3123 1504502:09
mordred (that's avg and max)02:09
clarkbthe next ubuntu lts won't have mir, so I can use it02:09
mordredkrotscheck: IE reports as Firefox?02:10
fungiclarkb: wat? next ubuntu is going back to xorg?02:10
krotscheckmordred: All browsers prefix with Mozilla/5.002:10
mordredkrotscheck: heh02:10
clarkbfungi: ya mir isn't ready or some such02:10
krotscheckIn case you're curious: http://webaim.org/blog/user-agent-string-history/02:11
funginetscape did win the browser wars. whaddyaknow?02:11
*** rwsu has quit IRC02:12
lifelessfungi: Ubuntu hasn't moved to Mir02:13
mordredkrotscheck: that is fantastic02:13
lifelessfungi: the Ubuntu phone is the only bit that ships Mir today [as default]02:13
krotscheckYea verily02:13
mordredAnd thus Chrome used WebKit, and pretended to be Safari, and WebKit pretended to be KHTML, and KHTML pretended to be Gecko, and all browsers pretended to be Mozilla, and Chrome called itself Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13, and the user agent string was a complete mess, and near useless, and everyone pretended to02:14
mordredbe everyone else, and confusion abounded.02:14
*** notmyname has quit IRC02:14
fungithis has made me want to change the user agent string in my broswer to "NCSA Mosaic/1.0 (X11;SunOS 4.1.3 sun4m)"02:14
lifelessjeblair: https://review.openstack.org/#/c/60875/2..3/modules/openstack_project/files/gerrit/acls/stackforge/chef-cookbooks.config02:15
krotscheckAnd that's basically the reason people stop doing browser detection, and start doing feature detection instead.02:15
*** michchap_ has joined #openstack-infra02:15
lifelessjeblair: is that so that the heads/* permissions don't recurse into .../stable/ ?02:15
*** notmyname has joined #openstack-infra02:15
krotscheckAnyway, long story short is that we can do client side JS rendering if we so choose, which is nice.02:16
*** zhiyan has joined #openstack-infra02:16
lifelesskrotscheck: \o/02:16
*** michchap has quit IRC02:17
krotscheckI'll see what I can do about benchmarking the memory vs bandwidth tradeoff.02:17
fungilifeless: yes, otherwise it will inherit from the parent container and be cumulative, but also that's to shadow the refs/heads/stable/* entry in the all-projects global acl which grants the same exclusive perms to the stable-maint group instead02:17
krotscheckOff to go to yoga, see y'all in the morning.02:17
lifelessok02:17
*** krotscheck has quit IRC02:17
fungiso this allows it to be overridden02:17
lifelessso kicks two separate groups out02:17
lifelesswithout it would you get just the global, or the global + the local?02:18
*** mrodden has joined #openstack-infra02:18
*** dkranz has quit IRC02:18
fungiwithout it you would be getting the one from all-projects applied, so only the stable-maint group could approve on those branches02:18
lifelessfungi: without that one line02:18
lifelessfungi: or without the section?02:18
fungimmm, good question. i haven't tried with that section existing but empty02:19
lifelessfungi: I mean present and non-empty02:19
lifelessjust without the exclusive line02:19
lifelesslike02:19
lifelessI"m not clear on the interactions there02:19
lifelessbut if they specify the same path02:19
lifelesswon't they union ?02:19
fungiin this case it would still inherit the one from all-projects, almost certain (because i think we caught one in that state not long ago and needed to fix it)02:20
lifelessok02:20
*** senk has quit IRC02:20
lifelessthanks02:20
lifelessTIL02:20
fungiso stable-maint would still have exclusive approve on it02:20
*** sarob has quit IRC02:22
funginot 100% sure though. would want to test it to convince myself02:22
*** senk has joined #openstack-infra02:22
*** sarob has joined #openstack-infra02:22
*** oubiwan__ has joined #openstack-infra02:22
*** sarob has quit IRC02:23
*** sarob has joined #openstack-infra02:24
*** yaguang has joined #openstack-infra02:25
*** rwsu has joined #openstack-infra02:26
*** yaguang has quit IRC02:29
*** sarob has quit IRC02:29
*** senk has quit IRC02:31
*** marun has quit IRC02:32
*** AaronGr is now known as AaronG_Zzz02:33
*** marun has joined #openstack-infra02:33
*** fungi has quit IRC02:34
*** rongze_ has joined #openstack-infra02:35
*** rongze has quit IRC02:35
*** weshay has quit IRC02:36
mordredclarkb: re: some of the above stuff, AaronG_Zzz said words about log files into logstash02:37
mordredclarkb: which made me think - should we run a second logstash cluster to handle our sysadmin logs?02:37
clarkbmordred: I think it is a good idea02:37
clarkbdefinitely don't use the public logstash02:37
*** senk has joined #openstack-infra02:40
*** senk has quit IRC02:42
openstackgerritMonty Taylor proposed a change to openstack-infra/pypi-mirror: Clean up the copyright header  https://review.openstack.org/5678302:49
*** vkozhukalov has joined #openstack-infra02:49
StevenKmordred: Did you just remove the callsite for main(), but not main() itself?02:55
clarkbStevenK: main is probably an entrypoint02:56
*** loq_mac has joined #openstack-infra02:56
StevenKclarkb: Nothing else in pypi-mirror calls it02:59
clarkbStevenK: http://git.openstack.org/cgit/openstack-infra/pypi-mirror/tree/setup.cfg#n2503:00
StevenKI knew grepping for 'main()' was a terrible idea03:01
*** masayukig has joined #openstack-infra03:04
*** zhiyan has quit IRC03:05
*** zhiyan has joined #openstack-infra03:05
*** michchap has joined #openstack-infra03:07
*** dcramer_ has quit IRC03:09
*** michchap_ has quit IRC03:09
*** fifieldt has joined #openstack-infra03:10
*** fifieldt has quit IRC03:14
mordredclarkb: we still have unhappy mirror builds03:14
lifelessmordred: can't sleep?03:15
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Raise the timeout on 2.6 mirror  https://review.openstack.org/6099403:16
mordredlifeless: what's sleep?03:16
lifelessthat thing where you get better at things03:16
mordredlifeless: did you see the lolcommits thing from earlier?03:17
lifelessprobably not03:17
clarkbmordred: timeouts?03:17
mordredclarkb: yeah. where we stop running a thing after a period of time03:18
clarkbbump the timeout?03:18
mordredclarkb: https://jenkins.openstack.org/job/periodic-mirror-python26/03:18
mordredclarkb: it hasn't run successfully since Nov 1603:19
clarkbI say go to 90 minutes03:19
clarkbthen sort out making it faster03:19
mordredyup03:19
mordredhttps://jenkins.openstack.org/job/post-mirror-python26/03:20
mordredsame thing there03:20
*** michchap_ has joined #openstack-infra03:20
*** yaguang has joined #openstack-infra03:20
clarkbmordred: if you propose the change I will approve it03:23
*** michchap has quit IRC03:23
mordredclarkb: that's the change above03:23
mordredclarkb: https://review.openstack.org/6099403:23
clarkbclearly I should read03:23
Alex_Gaynormordred: if you've got a sec, would love your input on https://github.com/wal-e/wal-e/issues/6503:23
mordredhehe03:23
mordredAlex_Gaynor: looking03:23
Alex_Gaynormordred: thanks03:23
*** fungi has joined #openstack-infra03:24
*** fungi has quit IRC03:25
*** slong- has joined #openstack-infra03:26
*** slong has quit IRC03:27
clarkbhrm change merged but no annoucement, and fungi flapped, please don't netsplit us freenode03:27
openstackgerritA change was merged to openstack-infra/config: Raise the timeout on 2.6 mirror  https://review.openstack.org/6099403:29
clarkbthere it is03:30
dkehnclarkb: I see a review for the bump of the stevedore=0.12, why would we go to 0.13, just curious03:31
clarkbdkehn: ?03:31
mordredAlex_Gaynor: sigh03:31
dkehnclarkb: https://review.openstack.org/#/c/60974/103:31
mordredAlex_Gaynor: tl;dr - pbr exposes underlying design deficiencies in easy_install03:32
dkehnclarkb: just curious if there is a prerequisite on 0.12 that we would go to .1303:32
clarkbdkehn: are you asking why it is not capped?03:32
mordredAlex_Gaynor: it's also possibleit's a pbr bug03:32
mordredAlex_Gaynor: but the platform on which life is happneing there is such a giant pile of poop03:32
Alex_Gaynormordred: pff, a dog rooting around in the backyard exposes design deficiencies in easy_install03:32
dkehnclarkb: yeah I guess03:32
mordredAlex_Gaynor: yup03:32
clarkbdkehn: https://github.com/openstack/requirements/commit/4d7e2b84e8a6a74edd6a132d465fc2642d141792 is why we bumped to 0.1203:33
dkehnbut I over looked the >= was thinking it was a =03:33
Alex_Gaynormordred: so grr, is there a way to solve this? It sucks that depending on swiftclient changes how a package needs to be installed03:33
clarkbdkehn: typically we don't cap03:33
dkehnclarkb: gotit03:33
clarkbbecause we want ot know when an upstream breaks us03:33
dkehnclarkb: thx03:33
mordredAlex_Gaynor: I mean, easy_install is broken. I don't understand why we, as python, insist on supporting it03:33
clarkbAlex_Gaynor: to be fair, you should only install things with pip because https and all that03:33
mordredI don't think it's depending on swiftclient03:33
mordredit's that you shoul dNEVER use easy_install because of what clarkb just said03:34
mordredand you can't avoid using easy_install when you run pyhton setup.py install03:34
* Alex_Gaynor wonders if he can bribe mordred or clarkb to explain that on the ticket ;)03:34
mordredso the python community needs to learn to replace setup.py install with pip install or pip install -e .03:34
mordredme goes to type03:34
*** fungi has joined #openstack-infra03:34
Alex_GaynorI can do it if you prefer, just hate feeling like I'm imposing on someone.03:35
Alex_GaynorOn the plus side, wal-e totally supports swift now :-)03:35
*** yamahata_ has joined #openstack-infra03:36
*** saschpe_ has joined #openstack-infra03:37
*** jgrimm has quit IRC03:37
*** saschpe has quit IRC03:37
mordredawesome03:38
*** amotoki has joined #openstack-infra03:39
mordredAlex_Gaynor: is this to snarky? http://paste.openstack.org/show/54740/03:40
*** gyee has quit IRC03:40
*** yamahata_ has quit IRC03:40
Alex_Gaynormordred: a little :-)03:41
mordredAlex_Gaynor: damn. lemme rewrite03:41
Alex_Gaynorsorry03:41
Alex_Gaynorthanks for the assist on this03:41
clarkbjust a little snarky03:42
*** chandankumar__ has quit IRC03:43
lifelessis it?03:43
fungiit seemed on the nose to me03:44
fungibut maybe i'm calloused from spending so many years on alt.flame03:45
clarkbits very mordred, if youare used to mordred prose its fine, if not I could see it being a problem03:45
mordredI rewrote it03:45
fungiit's not as if it's theo prose03:45
*** chandankumar has joined #openstack-infra03:52
clarkbAlex_Gaynor always on top of the requirementy thing03:52
clarkbs03:52
Alex_Gaynorclarkb: trying to make myself useful :-)03:52
clarkbAlex_Gaynor: it was useful for me, I thought theer was a backport package but couldn't remember what it was called03:52
fungione of the typical "reverify no bug" gems we'll come to miss after wednesday... https://review.openstack.org/5911304:11
clarkbha nice04:12
*** herndon has quit IRC04:15
*** mrodden1 has joined #openstack-infra04:16
*** slong has joined #openstack-infra04:16
*** slong- has quit IRC04:16
*** mrodden has quit IRC04:16
*** talluri has joined #openstack-infra04:17
*** ryanpetrello has quit IRC04:18
*** yamahata_ has joined #openstack-infra04:19
*** dcramer_ has joined #openstack-infra04:19
*** zhiyan has quit IRC04:20
*** ryanpetrello has joined #openstack-infra04:22
*** ryanpetrello has quit IRC04:24
*** rcleere has joined #openstack-infra04:25
zaroyo! why is this still failing? https://review.openstack.org/#/c/6098404:26
zarothought it was fixed with mordred patch.04:26
clarkbdid they run after that change merged and got applied?04:27
zarowait. wrong test is running for this job.04:27
*** fifieldt has joined #openstack-infra04:27
fungialso, i haven't started poking at review-dev again yet. one of my xen hosts here crashed and i've burned a couple hours getting everything back into a sane state04:28
zaroohh, no hold up.  that is the right test. lets try recheck.04:29
*** ryanpetrello has joined #openstack-infra04:37
*** SergeyLukjanov has joined #openstack-infra04:38
*** hartsocks has quit IRC04:39
*** senk has joined #openstack-infra04:41
*** senk1 has joined #openstack-infra04:41
*** senk has quit IRC04:41
*** nati_ueno has quit IRC04:42
*** julim has quit IRC04:44
*** senk1 has quit IRC04:46
*** senk has joined #openstack-infra04:46
*** ryanpetrello has quit IRC04:48
*** boris-42 has joined #openstack-infra04:50
*** fifieldt has joined #openstack-infra04:50
*** yamahata_ has quit IRC04:50
*** senk has quit IRC04:51
*** talluri has quit IRC04:51
*** ryanpetrello has joined #openstack-infra04:54
openstackgerritMark T. Voelker proposed a change to openstack-infra/config: Add puppet_openstack_builder to gerritbot config  https://review.openstack.org/6101004:57
*** paul-- has joined #openstack-infra04:58
*** dstanek has quit IRC05:07
*** colinmc__ has joined #openstack-infra05:08
*** loq_mac has quit IRC05:08
*** dcramer_ has quit IRC05:11
*** yamahata_ has joined #openstack-infra05:11
*** talluri has joined #openstack-infra05:11
*** rcleere has quit IRC05:12
*** colinmc__ has quit IRC05:12
*** pcrews has quit IRC05:17
*** yamahata_ has quit IRC05:23
*** yamahata_ has joined #openstack-infra05:23
*** gothicmindfood has joined #openstack-infra05:24
*** gothicmindfood has quit IRC05:26
*** dstanek has joined #openstack-infra05:27
*** gothicmindfood has joined #openstack-infra05:31
*** gothicmindfood has left #openstack-infra05:31
*** gothicmindfood has quit IRC05:31
*** rongze_ has quit IRC05:34
*** paul-- has quit IRC05:34
openstackgerritKhai Do proposed a change to openstack-infra/config: Setup tooling and jobs for openstack Gerrit ver 2.8 and beyond  https://review.openstack.org/6034805:35
*** nicedice has quit IRC05:38
*** dstanek has quit IRC05:43
*** rongze has joined #openstack-infra05:44
*** dstanek has joined #openstack-infra05:46
*** oubiwan__ has quit IRC05:49
*** rongze has quit IRC05:50
*** senk has joined #openstack-infra05:59
*** rongze has joined #openstack-infra05:59
*** NikitaKonovalov has joined #openstack-infra05:59
*** michchap_ has quit IRC06:00
*** masayukig has quit IRC06:06
*** senk has quit IRC06:06
*** fifieldt has quit IRC06:10
*** senk has joined #openstack-infra06:10
*** SergeyLukjanov is now known as _SergeyLukjanov06:10
fungizaro: http://paste.openstack.org/show/5474306:10
*** _SergeyLukjanov has quit IRC06:10
fungithat was after restoring the db from yesterday06:11
fungiwith the gerrit service stopped06:11
*** vkozhukalov has quit IRC06:12
*** jcoufal has joined #openstack-infra06:13
*** GheRivero has joined #openstack-infra06:13
*** GheRiver1 has quit IRC06:14
*** senk has quit IRC06:14
clarkbthat looks like a lolbug06:14
zarofungi: ohh i see ait's a failure doing the schema conversion.06:14
*** GheRivero has joined #openstack-infra06:14
zaroor maybe a messed up db entry?06:15
*** GheRivero is now known as Ghe_HPDiscover06:15
fungiyeah, i suspect there could be db cruft it's stumbling over06:16
*** senk has joined #openstack-infra06:16
clarkbright but the constraint should never let that in in the first place06:16
fungiat any rate, i'm guessing that exception is why the upgrade broke06:16
zarofungi: hmmm, can you zip the gerrit site up and give me to me somehow?  that's the only  way i can debug.06:17
zarofungi: hope you backed it up before doing the upgrade.06:18
zarofungi: i would need the original.06:19
fungiwhat else would it touch besides the db, out of curiosity?06:19
*** senk has quit IRC06:21
zarofungi: probably nothing else.  but i'm not certain.06:21
fungii think everything else is added/replaced by puppet, or within the git repos06:21
*** cody-somerville has quit IRC06:21
fungiif there's a risk it mangled other things, i can dig into the bup history for it tomorrow06:23
zarofungi: you can run the same test on the prod site to see if there are any errors.  well probably need to do that anyway06:23
fungibut yeah, we have daily backups, so we can put it back and retry as many times as we like06:26
zarofungi: ohh one thing is different when i ran test on my laptop.  i just use the H3 in memory db, not mysql so even if i have the site i would need the mysql db as well right?06:27
fungiyep06:27
zaroyikes!06:28
fungii can redact passwords/keys for external systems (github and so on) and give you a tarball of that homedir and the mysqldump06:29
zaroi guees that's what i would need to debug.  i will need to setup a mysql db and figure out how to impor that dump :(06:31
fungiimporting is dead simple06:31
zarohow about configuring the mysql db?06:32
*** jcooley_ has joined #openstack-infra06:33
fungiuse the mysql command-line client to connect to the db as root, then 'source gerrit.sql;'06:33
zarowhile i'm debugging it would be nice to try the same upgrade with the review gerrit site to see if we get any errors.06:33
*** senk has joined #openstack-infra06:33
*** paul-- has joined #openstack-infra06:33
zaroi would hate to go thru all this trouble and find out that it doesn't occur on review06:34
fungiit includes the db schema and everything in the dump. even drop statements06:34
zaroor that review-dev db is just messed up.06:34
zaroyou think that's possible?06:35
*** rongze has quit IRC06:35
zaroalright. it's late and i need my beauty sleep.  catch you in the morning.06:36
fungii think it's possible there's data in the db which the upgrade is unable to deal with, though i'd consider it a bug in the upgrade process not to cope with that06:36
funginight!06:36
fungii should start thinking about going into standby mode for a few hours as well06:38
*** rongze has joined #openstack-infra06:38
*** ladquin has joined #openstack-infra06:39
*** dstanek has quit IRC06:39
*** paul-- has quit IRC06:41
*** senk has quit IRC06:43
*** ladquin is now known as ladquin_afk06:52
*** dstanek has joined #openstack-infra07:02
*** denis_makogon has joined #openstack-infra07:10
*** senk has joined #openstack-infra07:14
*** dstanek has quit IRC07:16
*** odyssey4me has joined #openstack-infra07:25
chmouelclarkb: what do I scrap?07:28
clarkbthe gerrit json07:28
chmoueloh yeah the openstackwatch rss07:29
chmouellet me know if the polling is bad for the servers and i'll disable that07:29
chmouelthe cron launch hourly07:30
chmouelclarkb: just checked the code I use urllib tho https://github.com/chmouel/openstackwatch.rss/blob/master/openstackwatch.py#L7007:31
*** thomasbiege has joined #openstack-infra07:33
clarkbI dont think it is a problem07:35
*** SushilKM has joined #openstack-infra07:37
*** boris-42 has quit IRC07:39
*** yolanda has joined #openstack-infra07:42
*** thomasbiege1 has joined #openstack-infra07:46
*** thomasbiege has quit IRC07:47
*** vkozhukalov has joined #openstack-infra07:52
*** SergeyLukjanov has joined #openstack-infra07:53
*** SushilKM has quit IRC07:53
*** DinaBelova has joined #openstack-infra07:54
*** nati_ueno has joined #openstack-infra07:56
*** rongze has quit IRC07:59
*** avishayb has joined #openstack-infra08:02
*** rongze has joined #openstack-infra08:05
jd__mordred: ;-)08:05
*** thomasbiege1 has quit IRC08:21
*** ljjjustin has quit IRC08:26
*** fbo_away is now known as fbo08:29
*** rossella_s has joined #openstack-infra08:34
openstackgerritMatthew Mosesohn proposed a change to openstack-infra/config: Added Fuel Library and DevOps to StackForge  https://review.openstack.org/6104008:36
mattymoSergeyLukjanov, thanks for +1)08:39
mattymoclarkb, are you still available?08:39
SergeyLukjanovmattymo, you're welcome :)08:39
SergeyLukjanovmattymo, are you planning to add some gate jobs?08:39
*** rossella_s has quit IRC08:39
mattymoSergeyLukjanov, we gate from our fuel CI user08:40
SergeyLukjanovmattymo, what's about pep8 / hacking, smth else?08:40
mattymowe do flake8 + smoke tests08:40
mattymofor fuel we will run pylint08:40
mattymooops puppet-lint08:40
SergeyLukjanovcool, jobs for it could be easily added08:41
mattymowe're already prepared for it. there were some reservations about merging fuel puppet code before08:41
mattymofuelweb, fuel main, ostf, and astute are already in stackforge08:42
*** jcooley_ has quit IRC08:45
*** flaper87|afk is now known as flaper8708:50
*** denis_makogon has quit IRC08:56
*** sdake_ has quit IRC08:58
*** dizquierdo has joined #openstack-infra08:59
*** yassine has joined #openstack-infra08:59
openstackgerritChangBo Guo proposed a change to openstack-dev/hacking: Add check for remvoed mouldes in Python 3  https://review.openstack.org/6104909:07
*** derekh has joined #openstack-infra09:09
openstackgerritChangBo Guo proposed a change to openstack-dev/hacking: Add check for removed mouldes in Python 3  https://review.openstack.org/6104909:10
*** DinaBelova has quit IRC09:10
*** jpich has joined #openstack-infra09:10
*** rongze has quit IRC09:11
*** rongze has joined #openstack-infra09:15
*** thomasbiege has joined #openstack-infra09:22
*** zhiyan_ has joined #openstack-infra09:24
*** zhiyan_ is now known as zhiyan09:25
*** hashar has joined #openstack-infra09:30
*** thomasbiege has quit IRC09:31
*** SergeyLukjanov is now known as _SergeyLukjanov09:34
*** _SergeyLukjanov has quit IRC09:34
*** thomasbiege has joined #openstack-infra09:41
*** markmcclain has joined #openstack-infra09:42
*** DinaBelova has joined #openstack-infra09:45
*** SergeyLukjanov has joined #openstack-infra09:45
*** che-arne has joined #openstack-infra09:52
*** nprivalova has joined #openstack-infra09:55
*** chandankumar has quit IRC10:00
*** jcoufal has quit IRC10:03
*** senk1 has joined #openstack-infra10:06
*** senk1 has quit IRC10:06
*** senk has quit IRC10:07
*** chandankumar has joined #openstack-infra10:07
*** jorisroovers has joined #openstack-infra10:13
*** denis_makogon has joined #openstack-infra10:15
*** avishayb has quit IRC10:24
*** yaguang has quit IRC10:25
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Stories and Tasks search  https://review.openstack.org/6051510:26
openstackgerritDenis M. proposed a change to openstack-infra/config: Create new repository for trove-image-elements  https://review.openstack.org/6106410:28
*** talluri has quit IRC10:29
*** avishayb has joined #openstack-infra10:30
*** markmcclain has quit IRC10:30
*** talluri has joined #openstack-infra10:32
*** markmcclain has joined #openstack-infra10:33
*** adalbas has joined #openstack-infra10:35
openstackgerritDenis M. proposed a change to openstack-infra/config: Create new repository for trove-image-elements  https://review.openstack.org/6106410:41
openstackgerritDenis M. proposed a change to openstack-infra/config: Create new repository for trove-image-elements  https://review.openstack.org/6106410:44
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: cloverphp publisher  https://review.openstack.org/5294010:47
openstackgerritDenis M. proposed a change to openstack-infra/config: Create new repository for trove-image-elements  https://review.openstack.org/6106410:49
*** denis_makogon has left #openstack-infra10:49
*** yamahata_ has quit IRC10:49
*** cyeoh has quit IRC10:50
*** avishayb has quit IRC10:51
*** rongze has quit IRC10:52
*** rongze has joined #openstack-infra10:52
*** rongze has quit IRC10:55
*** markmc has joined #openstack-infra10:55
*** thomasbiege has quit IRC10:59
*** thomasbiege has joined #openstack-infra11:01
*** boris-42 has joined #openstack-infra11:05
*** markmcclain has quit IRC11:05
*** lcestari has joined #openstack-infra11:06
*** jorisroovers has quit IRC11:14
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: make path a global option (--path)  https://review.openstack.org/6108411:17
*** hashar has quit IRC11:18
*** rfolco has joined #openstack-infra11:18
*** thomasbiege has quit IRC11:19
*** mdenny has quit IRC11:23
*** hashar has joined #openstack-infra11:28
*** jcooley_ has joined #openstack-infra11:28
*** heyongli has joined #openstack-infra11:29
*** senk has joined #openstack-infra11:30
openstackgerritAntoine Musso proposed a change to openstack-infra/config: zuul: support 'recheck nobug'  https://review.openstack.org/6108611:31
*** jcooley_ has quit IRC11:33
*** jcoufal has joined #openstack-infra11:34
*** odyssey4me has quit IRC11:35
*** zhiyan has quit IRC11:39
*** rongze has joined #openstack-infra11:41
*** persia has quit IRC11:42
*** persia has joined #openstack-infra11:42
*** persia is now known as Guest9977411:43
*** jorisroovers has joined #openstack-infra11:46
*** SushilKM has joined #openstack-infra11:47
*** ArxCruz has joined #openstack-infra11:48
*** rongze has quit IRC11:51
*** amotoki has quit IRC11:53
*** SushilKM has quit IRC11:54
*** Guest99774 has quit IRC11:55
*** Guest99774 has joined #openstack-infra11:55
*** Guest99774 is now known as persia11:55
*** pabelanger has quit IRC11:55
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: list jobs in config and on target Jenkins  https://review.openstack.org/6088111:55
*** HenryG has quit IRC11:55
*** pabelanger has joined #openstack-infra11:56
*** rongze has joined #openstack-infra11:56
openstackgerrityolanda.robla proposed a change to openstack-infra/jenkins-job-builder: Enable ignore_cache flag on jenkins_jobs.ini  https://review.openstack.org/5951511:59
*** zhiyan has joined #openstack-infra12:01
*** senk has quit IRC12:02
*** hashar is now known as hasharLunch12:03
matelfungi: do you know if prepare_node_devstack.sh is executed synchronously? I am writing a script, that will do the same functionality, but the VM will be restarted.12:13
matelIs my understanding correct, that prepare_node_devstack.sh will be executed on the instance?12:14
mattymomatel, it's still a bit early in the US12:17
*** hasharLunch has quit IRC12:20
*** weshay has joined #openstack-infra12:21
*** SergeyLukjanov is now known as _SergeyLukjanov12:21
*** _SergeyLukjanov has quit IRC12:21
*** cyeoh_ has joined #openstack-infra12:27
*** cyeoh_ is now known as cyeoh12:27
*** nosnos has quit IRC12:29
*** jcooley_ has joined #openstack-infra12:29
*** thomasbiege has joined #openstack-infra12:30
*** tma996 has joined #openstack-infra12:32
*** jcooley_ has quit IRC12:35
*** SergeyLukjanov has joined #openstack-infra12:38
*** cyeoh has quit IRC12:38
romchegHi all12:41
romchegSorry, wrong chat :)12:41
*** smarcet has joined #openstack-infra12:42
matelmattymo: ok.12:46
mattymomatel, wait maybe 90 minutes12:47
*** romcheg has left #openstack-infra12:48
*** boris-42 has quit IRC12:50
*** boris-42 has joined #openstack-infra12:52
*** Grishkin has quit IRC12:52
*** markmc has quit IRC12:53
*** Grishkin has joined #openstack-infra12:53
*** markmc has joined #openstack-infra12:54
*** thomasbiege has quit IRC12:55
*** thomasbiege has joined #openstack-infra12:56
*** SushilKM has joined #openstack-infra12:57
mordredmorning matel12:57
*** dstanek has joined #openstack-infra12:58
matelmordred: hi, I wanted to ask about prepare_node_devstack.sh12:59
matelSo I have a script, that converts a standard RS image to a XenServer13:00
mordredmatel: it's executed when constructing the image that we then freeze and use as a base13:00
mordredmatel: awesome13:00
matelBut during the process, the instance will be rebooted13:00
mordredmatel: why?13:00
*** yassine_ has joined #openstack-infra13:00
mordredlike, do you reboot it as the final step before using the instance?13:00
matelIt's rebooted to the xenserver installer13:00
mordredoh, hrm13:00
matelI'm basically siping the underlying Ubuntu, and installing a XenServer, and in the end, a new ubuntu VM will run inside the XenServer.13:01
matelsiping = wiping13:01
mordredI think you'd have to extend nodepool to handle that13:01
mordredBUT13:01
mordredthere is another thing you should potentially consider13:01
*** xchu has joined #openstack-infra13:02
*** nprivalova has quit IRC13:02
*** yassine has quit IRC13:02
mordredmatel: first though - let me ask more questions - how does one talk to the ubuntu vm in the xenserver - is that what you expect the CI system to talk to?13:03
*** bingbu has joined #openstack-infra13:04
matelmordred: So the idea, is that we'll end up with an Ubuntu VM accessible through the public IP, so from there, the usual devstack scripts can do their job.13:04
openstackgerritChangBo Guo proposed a change to openstack-dev/pbr: Fix typos in document  https://review.openstack.org/6111013:04
*** SushilKM has quit IRC13:04
*** sandywalsh_ has joined #openstack-infra13:05
*** SushilKM has joined #openstack-infra13:05
mordredmatel: if you ran your xenserver script and then did a nova image-create13:05
matelYes, I have that kind of setup as well, so we can have a snapshot.13:05
mordredmatel: and then booted a copy of that - would the new machines you booted from the image created be configured already to have ssh to the new machine go to the ubuntu vm?13:05
matelyep13:06
mattymomordred, are you in a good mood today? https://review.openstack.org/#/c/61040/13:06
mordredmatel: k. I _think_ that what we want is essentially chaining13:06
matelmodrded: https://github.com/citrix-openstack/xenapi-in-the-cloud13:06
*** avishayb has joined #openstack-infra13:06
matelchaining - is it a nodepool related concept?13:06
mordredmatel: we want to have the image that is the result of snapshotting the results of your script13:07
mordredmatel: and we want to configure nodepool to use _that_ as a base image13:07
mordredfor a set of devstack-xenserver images it will mkae13:07
matelmordred - That sounds good, the question is, how can I "publish" the images - so that they are available for the infra cloud user.13:07
mordrednodepool does not know how to do the first thing I don't think13:08
mordredso I think we need to talk about how to do that - and we should wait for jeblair for that part13:08
mordredbut if booting the image made from your script gets us a thing that nodepool can treat normally, tat's pretty cool and I _think_ should Just Work13:08
matelYes, that would be fine.13:08
matelWe'll have some localrc tweak, but other than that, it should be OK.13:09
*** SushilKM has quit IRC13:09
matelI'm about to try it out, fungi pointed me to: https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/README.rst#n10013:09
mordredawesome13:10
mordredmattymo: I have +2'd it - but I'd like to wait for fungi or clarkb to get up before +A - I think we fixed all of the manage-projects issues, but I want to make sure13:10
matelmordred: is there a mechanism for periodically generating these base images?13:10
mordredmatel: well, nodepool does that for us currently13:11
mordredmatel: but I don't think it understands two concepts that might be needed13:11
mordredmatel: one is an image prep script that might need a reboot13:11
mordredmatel: and the other is that a base image produced by one image building config might be the basis for another image13:11
*** nprivalova has joined #openstack-infra13:12
mattymomordred++13:12
mattymomordred, it worked just fine yesterday when we moved fuel-docs13:12
matelI see.13:12
*** dstanek has quit IRC13:12
mordredmattymo: ah - ok. good. I'll pop it in then so that you can go ahead and migrate13:12
mattymowe do need branch pruning of course13:13
mattymowhich I commented in the request13:13
mattymothat can get handled whenever13:13
openstackgerritChangBo Guo proposed a change to openstack-dev/pbr: Fix typos in document  https://review.openstack.org/6111013:13
mordredmatel: and jeblair will be the best person to discuss exactly what that might look like13:13
matelmordred: I think that's two actions for me: 1.) look at nodepool - how to generate golden images 2.) Try out devstack scripts according to https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/README.rst#n10013:14
mordredmatel: alternately - we've poked at the idea of having base images defined by diskiage-builder - and then hav ea generic capability in nodeppol to make its images locally using diskimage-builder and uploading them13:14
openstackgerritA change was merged to openstack-infra/config: Added Fuel Library and DevOps to StackForge  https://review.openstack.org/6104013:14
mordredbut poke at 1 and 2 first - once you play with those for a bit, it sohudl all make good sense13:15
matelYes, that makes sense as well.13:15
matelwhen is jeblair usually online?13:15
openstackgerritChangBo Guo proposed a change to openstack-dev/hacking: Fix typos of comment in module core  https://review.openstack.org/6111113:16
*** dstanek has joined #openstack-infra13:16
matelmordred: Are there any blueprints/logs/etc for the diskimage-builder integration to nodepool?13:17
mordredmatel: there are not - I made an intial patch though...13:19
mordredmatel: https://review.openstack.org/#/c/46482/13:20
*** mriedem has joined #openstack-infra13:22
*** hashar has joined #openstack-infra13:22
*** oubiwan__ has joined #openstack-infra13:22
mattymomordred, any idea how long it takes for changes to take effect?13:23
mordredmattymo: could be as long as 30 minutes ish13:23
*** salv-orlando has quit IRC13:23
*** jooools has joined #openstack-infra13:24
matelmordred: Thanks for the help!13:27
*** thomasem has joined #openstack-infra13:27
*** sandywalsh__ has joined #openstack-infra13:29
*** HenryG has joined #openstack-infra13:29
*** rongze_ has joined #openstack-infra13:29
*** jcooley_ has joined #openstack-infra13:31
*** sandywalsh_ has quit IRC13:32
*** rongze has quit IRC13:33
*** jcooley_ has quit IRC13:36
*** openstackgerrit has quit IRC13:37
*** openstackgerrit has joined #openstack-infra13:37
*** dims has quit IRC13:39
sdaguemordred: https://review.openstack.org/#/c/53879/ if you have a second13:40
sdagueI'd like to get those grenade-forward jobs running so we can see how bad a state they are in13:41
*** thomasbiege has quit IRC13:41
*** dims has joined #openstack-infra13:41
*** bingbu has quit IRC13:43
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Improve storyboard gate jobs  https://review.openstack.org/6111913:44
*** oubiwan__ has quit IRC13:45
mordredsdague: lgtm13:45
sdaguethanks13:46
sdagueI'll annoy fungi when he gets in13:46
sdagueI think maurosr's patches for making grenade move forward are all ready. I pushed the first 2 to the gate this morning13:46
*** rongze_ has quit IRC13:48
maurosrcool13:48
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Initial test set added  https://review.openstack.org/6112013:49
*** colinmcnamara has joined #openstack-infra13:49
*** dkranz has joined #openstack-infra13:49
sdagueI'll also need to bug someone to figure out why one of the landed er changes didn't seem to take13:51
sdaguethough it is nice that the bugs are sorted by hits - http://status.openstack.org/elastic-recheck/13:51
*** dcramer_ has joined #openstack-infra13:52
*** colinmcnamara has quit IRC13:54
*** ryanpetrello has quit IRC13:54
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Add Savanna d-g jobs to tempest exp pipeline  https://review.openstack.org/6112513:55
*** dcramer_ has quit IRC13:56
sdaguelifeless: whenever day time is for you - we should discuss https://review.openstack.org/#/c/59303/13:58
*** rcleere has joined #openstack-infra13:58
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Improve storyboard gate jobs  https://review.openstack.org/6111913:59
*** dprince has joined #openstack-infra14:00
*** jcoufal has quit IRC14:00
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Improve storyboard gate jobs  https://review.openstack.org/6111914:01
*** talluri has quit IRC14:06
*** senk has joined #openstack-infra14:06
*** yaguang has joined #openstack-infra14:07
*** talluri has joined #openstack-infra14:07
*** pleia2 has quit IRC14:07
*** sandywalsh_ has joined #openstack-infra14:07
*** dkranz has quit IRC14:07
*** SushilKM has joined #openstack-infra14:07
*** sandywalsh__ has quit IRC14:08
*** tma996 has quit IRC14:08
*** luis_ has joined #openstack-infra14:08
*** luisg has quit IRC14:08
*** SlickNik has quit IRC14:09
*** tma996 has joined #openstack-infra14:09
*** pleia2 has joined #openstack-infra14:09
*** dripton has quit IRC14:09
*** dripton has joined #openstack-infra14:09
*** saschpe_ has quit IRC14:10
*** basic` has quit IRC14:10
*** talluri has quit IRC14:11
*** saschpe has joined #openstack-infra14:11
*** ladquin_afk has quit IRC14:12
*** basic` has joined #openstack-infra14:13
*** jd__ has quit IRC14:13
*** jd__ has joined #openstack-infra14:13
*** weshay has quit IRC14:13
*** ryanpetrello has joined #openstack-infra14:14
SergeyLukjanovhi guys14:14
*** SlickNik has joined #openstack-infra14:14
SergeyLukjanovI've added simple tempest api test for savanna14:14
*** ladquin_afk has joined #openstack-infra14:14
*** ladquin_afk has joined #openstack-infra14:14
SergeyLukjanovand I think that we need savanna d-g jobs for tempest in exp pipeline to be able to check tests14:15
SergeyLukjanovwhat do you think?14:15
*** wayneeseguin has quit IRC14:15
*** zhiyan has quit IRC14:16
sdagueSergeyLukjanov: sounds right to me14:18
SergeyLukjanovsdague, great!14:19
SergeyLukjanovsdague, thank you :)14:20
*** wayneeseguin has joined #openstack-infra14:22
openstackgerritMatthew Mosesohn proposed a change to openstack-infra/jeepyb: add fuel-library fuel-devops fuel-docs to jeepyb  https://review.openstack.org/6112914:22
*** rongze has joined #openstack-infra14:23
*** oubiwan__ has joined #openstack-infra14:24
*** bpokorny has joined #openstack-infra14:24
*** avishayb has quit IRC14:24
*** loq_mac has joined #openstack-infra14:24
*** loq_mac has quit IRC14:24
*** weshay has joined #openstack-infra14:25
*** loq_mac has joined #openstack-infra14:27
*** avishayb has joined #openstack-infra14:27
*** julim has joined #openstack-infra14:28
*** sandywalsh__ has joined #openstack-infra14:30
*** jcooley_ has joined #openstack-infra14:32
*** sandywalsh_ has quit IRC14:33
*** rcleere has quit IRC14:34
*** jcooley_ has quit IRC14:36
*** salv-orlando has joined #openstack-infra14:39
*** zhiyan has joined #openstack-infra14:41
SergeyLukjanovit looks like gate-ci-docs failing14:42
*** thomasbiege has joined #openstack-infra14:42
SergeyLukjanovhttp://logs.openstack.org/25/61125/1/check/gate-ci-docs/7b6226c/console.html14:42
SergeyLukjanoverror: 'source_dir' must be a directory name (got `/home/jenkins/workspace/gate-ci-docs/doc/source`)14:42
*** sandywalsh_ has joined #openstack-infra14:43
*** salv-orlando_ has joined #openstack-infra14:44
*** sandywalsh__ has quit IRC14:44
*** beagles is now known as beagles_brb14:45
openstackgerritSalvatore Orlando proposed a change to openstack-infra/devstack-gate: Set ovs-vsctl timeout to 10 seconds  https://review.openstack.org/6113714:45
*** salv-orlando has quit IRC14:46
*** salv-orlando_ is now known as salv-orlando14:46
*** jcoufal has joined #openstack-infra14:48
*** talluri has joined #openstack-infra14:50
*** senk1 has joined #openstack-infra14:53
*** senk has quit IRC14:54
*** alcabrera has joined #openstack-infra14:55
*** salv-orlando has quit IRC14:56
*** senk1 has quit IRC14:58
*** senk has joined #openstack-infra14:58
*** sandywalsh__ has joined #openstack-infra15:01
*** thomasbiege1 has joined #openstack-infra15:01
chmouelwho takes care of checking the "recheck bug ??" in gerrit?15:01
*** sandywalsh_ has quit IRC15:01
chmoueli would imagine it was in jeepyb but it doesnt seem so15:01
chmouelis that hook specific on the server15:01
*** prad has joined #openstack-infra15:01
chmouel(I want to add the ability to add comment after a break line explaining why the recheck)15:02
*** thomasbiege has quit IRC15:05
*** Shrews has quit IRC15:05
*** julim has quit IRC15:05
*** zhiyan has quit IRC15:06
*** zhiyan has joined #openstack-infra15:06
*** zhiyan has joined #openstack-infra15:07
*** Shrews has joined #openstack-infra15:07
*** julim has joined #openstack-infra15:07
fungisdague: why ever would you want to annoy me anyway?15:08
fungioh, 5387915:08
mgagnechmouel: recheck is handled by zuul15:09
mgagnechmouel: there is a comment filter triggering the check pipeline15:09
mgagnechmouel: https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/layout.yaml#L1515:10
chmouelmgagne: oh cool thank you15:11
*** dkranz has joined #openstack-infra15:11
*** jcoufal has quit IRC15:11
*** jgrimm has joined #openstack-infra15:12
*** reaper has joined #openstack-infra15:12
*** reaper has quit IRC15:12
*** heyongli has quit IRC15:13
*** rnirmal has joined #openstack-infra15:16
*** loq_mac has quit IRC15:18
fungimattymo: for future reference, it's better to remove branches and tags you're not wanting prior to import (or to have us import from a sanitized clone which has them removed) so that you don't need us to delete things afterward15:18
mattymofungi, I'm in the awkward situation where our core reviewers are unavailable for our old github account to delete15:20
*** loq_mac has joined #openstack-infra15:20
mattymoone can't be disturbed and the other is on a train15:20
fungimattymo: oh, ew. that's fun15:20
mattymoand you (infra team) have different availability15:20
fungiwell, we'll get it sorted out15:20
mattymoit's no rush of course15:21
mattymoextra branches doesn't break anything. they're just unsavory15:21
*** ywu has joined #openstack-infra15:21
*** jgrimm has quit IRC15:22
*** beagles_brb is now known as beagles15:23
*** senk has quit IRC15:23
*** loq_mac has quit IRC15:24
*** loq_mac has joined #openstack-infra15:25
*** zhiyan_ has joined #openstack-infra15:26
*** loq_mac has quit IRC15:27
openstackgerritA change was merged to openstack-infra/config: Add grenade-dsvm-forward job template  https://review.openstack.org/5387915:27
fungisdague: ^15:28
*** zhiyan has quit IRC15:28
*** zhiyan_ is now known as zhiyan15:28
*** boris-42 has quit IRC15:28
EmilienMfungi: i also did some updates / rebase from sdague & clarkb recommandations > https://review.openstack.org/#/c/59039/15:28
sdaguefungi: thanks15:29
fungiEmilienM: yep, looks like it follows the same pattern with the -forward jobs now. just need to give it a quick once-over before i approve15:30
*** sandywalsh__ has quit IRC15:31
EmilienMfungi: cool, thx15:32
fungiEmilienM: actually, it's going to need a rebase now that 53879 is merged, but also you got your branch expressions a little wrong. i'll update the review with specific comments15:32
*** mfer has joined #openstack-infra15:32
mordredlike this: extra branches doesn't break anything. they're just unsavory15:34
fungisame for extra limbs15:34
*** jgrimm has joined #openstack-infra15:35
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Remove Stale Bugs from elastic-check page  https://review.openstack.org/6114915:35
fungisdague: i just got to thinking... with the branch matches we're doing on the forward jobs, we'll be skipping them for changes to milestone-proposed. is that intentional, or oversight?15:37
EmilienMfungi: ack15:37
*** xchu has quit IRC15:38
hasharhey there :-]  Jenkins jobs building sphinx documentation are failing on multiple projects 'source_dir' must be a directory name.  I filled a bug for it https://bugs.launchpad.net/openstack-ci/+bug/125951115:39
uvirtbotLaunchpad bug 1259511 in openstack-ci "sphinx docs gating jobs are broken (source_dir must be a directory)" [Undecided,New]15:39
hasharand did some recheck that confirms it15:39
sdaguefungi: accidental at this point. I want to see if they run at all first :)15:39
*** rongze_ has joined #openstack-infra15:39
*** salv-orlando has joined #openstack-infra15:40
*** rongze has quit IRC15:40
fungifair enough. let's just not forget we've got a hole in that sock15:41
*** herndon has joined #openstack-infra15:42
*** rongze_ has quit IRC15:43
*** datsun180b has joined #openstack-infra15:43
openstackgerritA change was merged to openstack-infra/config: Adding chef-manager-core to stable maintainers acl  https://review.openstack.org/6087515:44
*** jergerber has joined #openstack-infra15:48
*** salv-orlando has quit IRC15:48
*** nati_ueno has quit IRC15:48
*** NikitaKonovalov has quit IRC15:52
*** changbl has quit IRC15:54
*** SergeyLukjanov has quit IRC15:55
*** changbl has joined #openstack-infra15:55
*** AaronG_Zzz is now known as AaronGr15:57
*** DinaBelova has quit IRC15:57
*** loq_mac has joined #openstack-infra15:58
*** rcleere has joined #openstack-infra16:00
*** yaguang has quit IRC16:00
*** loq_mac has quit IRC16:01
*** randallburt has joined #openstack-infra16:02
*** randallburt has quit IRC16:02
*** randallburt has joined #openstack-infra16:02
*** jorisroovers has quit IRC16:02
*** jorisroovers has joined #openstack-infra16:02
*** sarob has joined #openstack-infra16:04
openstackgerritEmilien Macchi proposed a change to openstack-infra/config: Neutron support in Grenade jobs  https://review.openstack.org/5903916:04
*** HenryG_ has joined #openstack-infra16:05
EmilienMfungi: i hope i did well ^16:05
*** vkozhukalov has quit IRC16:05
*** HenryG has quit IRC16:05
*** sarob has quit IRC16:05
*** sarob has joined #openstack-infra16:06
*** jcooley_ has joined #openstack-infra16:06
*** jroovers has joined #openstack-infra16:06
EmilienMactually jenkins's not happy16:07
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add e-r query for nova-network bug 1258848  https://review.openstack.org/6115816:08
uvirtbotLaunchpad bug 1258848 in nova "vif doesn't exist Error message in tempest runs" [Critical,Triaged] https://launchpad.net/bugs/125884816:08
*** SushilKM has quit IRC16:09
EmilienMsdague: it's weird, I already did a rebase 2 min ago16:09
*** jorisroovers has quit IRC16:10
EmilienMi retry16:10
*** sarob has quit IRC16:10
*** sarob has joined #openstack-infra16:11
fungiEmilienM: yeah, you added in a bunch of previously removed lines when you rebased that. see my comments on that last patchset16:11
*** sarob has quit IRC16:12
*** thomasbiege1 has quit IRC16:12
*** reed has joined #openstack-infra16:12
*** sarob has joined #openstack-infra16:13
*** sarob has quit IRC16:14
*** senk has joined #openstack-infra16:15
*** ^d has joined #openstack-infra16:15
*** ^d has joined #openstack-infra16:15
*** sarob has joined #openstack-infra16:16
*** jcooley_ has quit IRC16:16
*** jroovers has quit IRC16:16
*** jcooley_ has joined #openstack-infra16:17
*** sarob has quit IRC16:17
*** loq_mac has joined #openstack-infra16:18
*** senk has quit IRC16:19
openstackgerritEmilien Macchi proposed a change to openstack-infra/config: Neutron support in Grenade jobs  https://review.openstack.org/5903916:20
*** sarob has joined #openstack-infra16:20
*** HenryG_ has quit IRC16:21
jeblairhashar: the sphinx thing is a version incompatability16:21
*** sarob has quit IRC16:22
*** HenryG has joined #openstack-infra16:23
fungierror: 'source_dir' must be a directory name (got `/home/jenkins/workspace/gate-ci-docs/doc/source`)16:23
*** sarob has joined #openstack-infra16:23
jeblairhashar: looks link sphinx 1.2 was just released16:23
fungithat?16:23
jeblairfungi: yes16:23
fungiwhee16:23
jeblairfungi: it's a unicode string error, bug in sphinx16:23
hasharnicee16:24
jeblairfungi: it's what i was helping lcheng with yesterday, by switching django_openstack_auth to use the mirror16:24
* fungi looks for upstream bug tracker16:24
hasharjeblair: thank you :-]16:24
fungii guess it's just started to bite gate-ci-docs16:24
jeblairhowever, now that 1.2 is released, i expect it's going to hit the mirror soon and start affecting everything else.16:24
*** sarob has quit IRC16:24
sdagueso, can we get this landed as soon as people can take a look - https://review.openstack.org/#/c/57066/16:26
sdagueit's the d-g change to move grenade from upgrade from grizzly to upgrade from havana16:26
sdaguewhich is finally working16:26
*** sarob has joined #openstack-infra16:26
sdaguethanks to maurosr's grenade patches16:26
*** zhiyan has quit IRC16:26
EmilienMfungi: looks like gate-ci-docs is not happy16:27
jeblairfungi: looks like this tracker: https://bitbucket.org/birkenfeld/sphinx/issues/16:27
*** avishayb has quit IRC16:27
fungijeblair: yep, already there, browsing for recently filed issues16:27
jeblairsdague: we're putting out a smoldering fire that's about to spread.  :)16:28
fungiEmilienM: yeah, that's what we're troubleshooting16:28
sdaguejeblair: no worries :)16:28
sdagueI'll let the fire fighting happen first, about to head to lunch anyway16:28
funginone of the recently filed issues look relevant. do we have a traceback captured? if not i'll reproduce locally16:29
*** jergerber has quit IRC16:29
jeblairfungi: gimme a sec, let me see if i have something16:29
*** thomasbiege has joined #openstack-infra16:30
fungioh... https://bitbucket.org/birkenfeld/sphinx/pull-request/193/builddoc-shouldnt-fail-on-unicode-paths/diff ?16:30
*** sarob has quit IRC16:30
fungipoints to http://bugs.python.org/issue1957016:30
*** sarob has joined #openstack-infra16:31
openstackgerritJames E. Blair proposed a change to openstack/requirements: Pin Sphinx to <1.2  https://review.openstack.org/6116416:31
EmilienMfungi: oops, sorry.16:31
jeblairfungi: that issue touches on all the relevant functions16:31
jeblairfungi: and yes, the distutils bug exactly describes the error16:32
mordred+A16:32
fungihttps://bitbucket.org/birkenfeld/sphinx/issue/114216:32
mordredwhat? there's a bug in distutils?16:32
jeblairmordred: 1 sec16:33
fungiseems to be the corresponding sphinx end issue16:33
* mordred reading it16:33
*** boris-42 has joined #openstack-infra16:33
jeblairmordred: i'm going to re-push with links in the commit msg to these bugs16:33
jeblairmordred: i -2'd it16:33
mordredjeblair: kk16:33
mordredwow16:33
mordredAlex_Gaynor: "This is more a feature request than a bug. You should use an encoded path, or upgrade to Python 3 which handles Unicode correctly."16:33
openstackgerritJames E. Blair proposed a change to openstack/requirements: Pin Sphinx to <1.2  https://review.openstack.org/6116416:34
mordredAlex_Gaynor: when people ask why I stopped trying to support easy_install in pbr - it's because the people who run easy_install are divorced from reality16:34
jeblairmordred: ^ re-pushed16:34
mordredjeblair: +216:35
fungijeblair: feel free to approve when you remove your -216:35
jeblairdone16:35
fungias for "upstream, i guess we can chime in loudly in the sphinx pull request and the distutils issue16:35
*** sarob has quit IRC16:36
*** jooools has quit IRC16:36
fungibut looks like it's already well documented since a month16:36
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Pin Sphinx to <1.2  https://review.openstack.org/6116716:36
*** sarob has joined #openstack-infra16:38
*** jcooley_ has quit IRC16:40
openstackgerritJames E. Blair proposed a change to openstack-infra/jenkins-job-builder: Pin Sphinx to <1.2  https://review.openstack.org/6117116:40
*** nprivalova has quit IRC16:40
*** prad has quit IRC16:40
*** mrodden1 has quit IRC16:40
*** jcooley_ has joined #openstack-infra16:40
*** tma996 has quit IRC16:40
openstackgerritJames E. Blair proposed a change to openstack-infra/zuul: Pin Sphinx to <1.2  https://review.openstack.org/6117216:41
openstackgerritA change was merged to openstack-infra/config: Pin Sphinx to <1.2  https://review.openstack.org/6116716:41
ttxmordred: can I remove milestone-proposed branches from icehouse-1 ? I assume you merged the tags back to master ?16:41
fungiinteresting that we have gate-ci-docs voting in check but not gating (just rerun in post)16:42
mordredttx: lemme check first16:42
funginot sure whether that's intentional or an oversight16:42
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Pin Sphinx to <1.2  https://review.openstack.org/6117316:42
*** loq_mac has quit IRC16:42
mordredttx: oh - actually, I don't need m-p branch for the merge jobs anyway16:42
*** sarob has quit IRC16:43
ttxmordred: ok, will clean them up. Do you think we could get them auto-deleted by the pre-release or release pipe jobs ?16:44
ttxit's a bit tricky since you don't want prerelease to remove MP branch on RC, and what you actually want on release is renaming MP to stable/*16:45
ttxbut that's like the last bit of the release process which is manual :)16:45
*** jcooley_ has quit IRC16:45
*** sarob has joined #openstack-infra16:45
*** jergerber has joined #openstack-infra16:45
mordredyeah. I don'thave good answers for that yet16:46
*** hashar has quit IRC16:46
ttxno hurry16:46
*** marun has quit IRC16:46
*** sarob_ has joined #openstack-infra16:47
*** HenryG has quit IRC16:47
*** sarob_ has quit IRC16:48
notmynamejog0: where do you get the source numbers to graph for http://status.openstack.org/elastic-recheck/ ?16:48
*** sarob has quit IRC16:49
pleia2we have bug day planning to start in ~10 minutes16:51
* pleia2 prepping etherpad16:51
jeblairnotmyname: for the top (job) graph or the bug graphs?16:51
mordrednotmyname: graphite.openstack.org16:51
mordrednotmyname: is the top one16:52
notmynamejeblair: sorry, the top job graph. thanks16:52
*** gyee has joined #openstack-infra16:52
*** MarkAtwood has joined #openstack-infra16:52
*** hashar has joined #openstack-infra16:53
*** hogepodge has joined #openstack-infra16:53
*** hashar is now known as hasharCall16:54
*** pcrews has joined #openstack-infra16:55
fungiit may also not be reasily apparent that the top graph there is a percentage of the jobs which ran along the vertical axis16:55
fungier, readily16:55
*** mrodden has joined #openstack-infra16:55
fungirather than a straight tally16:55
*** SushilKM has joined #openstack-infra16:56
mordredttx: mordred@camelot:~/src/openstack/nova$ git describe16:57
*** nati_ueno has joined #openstack-infra16:57
mordred2014.1.b1-116-gffe6bb516:57
mordredttx: ^^ that means that 2014.1.b1 has been merged back into mainline nova16:57
ttxmordred: ok16:57
mordredttx: so I beliee the "merge tags back to master" jobs have worked16:58
mordredI love it when I forget abot things because they just work16:58
fungiuntil centuries later when our civilazation collapses because nobody remembers how to fix all this automation we've set in motion16:58
zarogood morning16:58
*** nicedice has joined #openstack-infra16:58
Alex_Gaynormordred: :(16:59
openstackgerritA change was merged to openstack-infra/config: Neutron support in Grenade jobs  https://review.openstack.org/5903916:59
*** prad has joined #openstack-infra16:59
fungiEmilienM: ^17:00
*** flaper87 is now known as flaper87|afk17:00
ttxfungi: I think that's called Judgement Day17:00
zaromordred: your patches for the git submodules depends on this merging first -> https://review.openstack.org/#/c/6034817:01
*** NikitaKonovalov has joined #openstack-infra17:01
EmilienMfungi: cool, thank you for your help on that. I feel i was between huge work and trying to send stuff between :)17:01
mordredzaro: yes. I totally agree with you17:01
*** SushilKM has quit IRC17:01
pleia2jeblair, fungi, clarkb, mordred, zaro - happy bug day! https://etherpad.openstack.org/p/cibugreview-december201317:01
zarooohh yah! bugs!17:01
fungipleia2: thanks! many happy returns of the bugday to you as well17:01
pleia2I'm still adding notes to medium bugs17:01
fungiAaronGr: ^17:02
*** loq_mac has joined #openstack-infra17:02
fungigreat way to get an early view of all the things you're going to fix ;)17:02
jeblairhow about i triage new bugs?17:02
AaronGrfungi: thanks -- reading.17:03
fungijeblair: go for it17:03
*** ChanServ sets mode: +o jeblair17:03
fungii'm going to do what i should have been doing all along and make sure my bugs are in sane shape before i dive into the rest17:03
*** jeblair changes topic to "Bug Day: https://etherpad.openstack.org/p/cibugreview-december2013"17:03
pleia2AaronGr: we do this every 6 weeks or so (typically week after a milestone) so our bugs don't get (more) out of control17:03
*** NikitaKonovalov has quit IRC17:07
AaronGrgood to know, i can see a few items i might be able to add value with after my intro crash course next week.17:07
* mordred imagines a day in the future where he interacts with the bug system more so that teh list doesn't get out of hand17:07
jeblairmordred: ++17:08
fungii've been trying to stay on top of most of the ones which get opened day-to-day and close them out quickly if they're not too involved17:08
*** NikitaKonovalov has joined #openstack-infra17:08
anteayapleia2: feel free to shame me, I have basically made no progress on any of my -infra work since jumping into -neutron, bad bad17:09
mordredooh!17:09
*** NikitaKonovalov has quit IRC17:09
*** SergeyLukjanov has joined #openstack-infra17:10
jeblairwhoah, i didn't know about this: https://bugs.launchpad.net/openstack-ci/+bug/124411517:10
uvirtbotLaunchpad bug 1244115 in openstack-ci "PostgreSQL server failed to start" [Low,Triaged]17:10
jeblairapparently dpkg starting pgsql in devstack sometimes just fails :(17:10
*** SergeyLukjanov is now known as _SergeyLukjanov17:11
fungii've updated long-standing private security bug 914432 and switched it to public now that i think it's basically resolved. can i get one more long-timer to look over my assertions in the comment and confirm?17:11
uvirtbotLaunchpad bug 914432 in openstack-ci "Remove sensitive files from build slaves" [High,Fix committed] https://launchpad.net/bugs/91443217:11
*** _SergeyLukjanov is now known as SergeyLukjanov17:11
*** moted has quit IRC17:12
*** NikitaKonovalov has joined #openstack-infra17:12
jeblairfungi: lgtm17:13
*** NikitaKonovalov has quit IRC17:13
*** moted has joined #openstack-infra17:13
fungijeblair: thanks!17:14
jeblairfungi, clarkb, mordred: what if we tagged gate-failing bugs with something like 'gate-fail' or 'gate' ?17:14
mordredzaro: https://review.openstack.org/#/c/60348/17:14
jeblair(like the postgres bug, etc...)17:14
mordredjeblair: wfm17:14
pleia2anteaya: if you could just eyeball the bugs you have and see if there are any updates or anything to add to your priority list, that would be helpful :)17:14
fungijeblair: i think there was a proposal on the dev ml to do just that. we should align with whichever format the other projects settled on17:14
anteayapleia2: can do17:15
fungi(i just don't remember what got decided)17:15
reedjeblair, fungi: I'll need your help today to complete the CI/CD of openid server17:15
jeblairsdague: ^ did you read that thread?17:15
reedjeblair, fungi: do you have time today?17:15
mordredAaronGr: oh - another thing - you may want to log in to gerrit and go to your preferences to 'watched projects' and add openstack-infra/config - which will make all of the outstanding changes to the config repo show up in your review list17:15
fungireed: yes, i was going to put together the infra change to import the openid project today, but smarcet needed to get me the github url i think? (unless i missed it in scrollback)17:15
reedfungi, I'll get that URL for you17:16
*** SushilKM has joined #openstack-infra17:16
fungireed: too awesome. you rock17:16
reedI have smarcet on the phone :)17:16
fungireed: he did at least confirm yesterday that the openstackid-dev server was accessible to him, so we're getting close17:16
jeblairfungi: jgriffith started using 'gate-failure' for cinder17:17
jeblairhttp://lists.openstack.org/pipermail/openstack-dev/2013-November/020631.html17:17
fungijeblair: wfm then17:17
AaronGrmordred: on it now, thanks17:17
smarcetfungi: https://github.com/smarcet/openstackid17:17
jeblairclarkb, fungi, mordred, pleia2: tag gate-failure bugs with 'gate-failure'17:17
*** loq_mac has quit IRC17:17
*** NikitaKonovalov has joined #openstack-infra17:17
reedfungi, yeah, the server is reachable, we've got a question regarding the mysql instance...17:17
fungismarcet: thanks! i'll whip that up after the bug triage and infra meeting, so probably closer to 20:00-21:00 utc17:18
smarcetfungi: np tnx a lot :)17:18
reedfungi, in modules/openstack_project/manifests/openid_dev.pp the mysql instance is on localhost... not sure it's correct17:18
fungireed: oh, i think that's overridden in site.pp though (it's just there as a default value)17:19
fungireed: that same variable should be getting stuffed for that server in the top-level manifests/site.pp (not under modules/anything)17:19
reedfungi, oh, ok, so those are in hiera... meaning you'll have to tell the values to smarcet?17:19
fungireed: well, we can inject those values into any file he needs too, by using variable substitution in puppet17:20
smarcetreed: thing is that was overlooked, i mean should be removed as per comment on https://review.openstack.org/#/c/53644/17/modules/openstack_project/manifests/openid_dev.pp17:20
smarcetYou shouldn't need this, since the database server is already provided  externally to this system.17:20
pleia2mordred: can you have a look at this one? https://bugs.launchpad.net/openstack-ci/+bug/122470517:21
uvirtbotLaunchpad bug 1224705 in openstack-ci "pypi + --offline is still accessing tarballs on the internet" [Medium,Triaged]17:21
jeblairanteaya: are you still working on https://bugs.launchpad.net/openstack-ci/+bug/1245905 ?17:21
uvirtbotLaunchpad bug 1245905 in openstack-ci "manage-projects needs to write its logging somewhere" [Undecided,New]17:21
reedsmarcet, right, the database is served as a separate service, the openid script doesn't need to install it nor to manage it17:21
smarcetalso reed: check this https://review.openstack.org/#/c/53644/20/modules/openstack_project/manifests/openid_dev.pp17:21
jeblairfungi: manage-projects still needs logging work, right?17:22
smarcetreed: ok in that case i should correct script to remove all config variables related to mysql17:22
anteayajeblair: I want to do the work, but no, I haven't done one thing to address that bug17:22
*** yassine_ has quit IRC17:22
jeblairanteaya: can you unassign yourself if you aren't working on it?17:22
smarcetshould i create a new branch for this change , rite?17:22
anteayaif someone else needs to take it, they can take it17:22
fungijeblair: um, yes i think it still needs a log config or we need to use the new-fangled option in puppet to have it regurgitate stdout from that exec17:22
anteayajeblair: I will17:22
fungijeblair: i'll update the bug17:22
anteayaunassigned17:22
jeblairok.  i also tagged it lhf because most work can be done independently, and it's not too hard.17:23
jeblairAaronGr, yolanda: ^ we should get some new low-hanging-fruit bugs out of this for you to look at.17:23
fungismarcet: reed: but anyway, it's fine that site_mysql_host is '127.0.0.1' there since it gets overridden when instantiated in manifests.site.pp with our hiera value instead. it could just as well be defaulted to '' there, but same end result17:23
zaromordred: your questions answered https://review.openstack.org/#/c/60348/17:23
yolandajeblair, nice17:24
yolandai pushed a change today but got failures with doc17:24
fungiyolanda: recheck no bug, it should be fixed now17:24
yolandahttps://review.openstack.org/#/c/59515/17:24
yolandaok17:24
jeblairyolanda: i have patches up to fix that17:24
fungiyolanda: sphinx 1.2 caused issues...17:24
jeblairfungi: did those merge?17:24
jeblairfungi: no, jjb fix hasn't been approved yet17:24
fungijeblair: oh, the one for config did. the zuul/jjb/nodepool ones may not have yet. rebase on the appropriate one if so17:25
jeblairyolanda: for jjb: https://review.openstack.org/#/c/61171/17:25
fungiyolanda: ^17:25
zarowould an openstack-infra mind weighing in on this change? https://review.openstack.org/#/c/5834917:26
*** NikitaKonovalov has quit IRC17:26
smarcetfungi: thing is that $site_mysql_host is not used on any place rite now17:26
jeblairzaro: we would love to, after the bug triage event.17:26
zaroohh yeah, pesky bugs!17:27
*** jergerber has quit IRC17:27
*** NikitaKonovalov has joined #openstack-infra17:27
jeblairsmarcet, fungi, reed: can this wait until after our bug triage and team meeting?17:27
smarcetsure np17:28
fungismarcet: right, the host, username and password would need to get added to, for example, a mysql-defaults.config which you can load17:28
smarcetnot hurry17:28
*** sdake_ has joined #openstack-infra17:28
smarcetfungi: roger that :)17:28
*** sandywalsh_ has joined #openstack-infra17:29
yolandajeblair, still getting failure17:29
*** loq_mac has joined #openstack-infra17:30
jeblairyolanda: did you rebase on that change?17:31
jeblairyolanda: 'recheck no bug' will not work because it hasn't landed; but rebasing on that change should work17:31
yolandajeblair, ok, let me try iut17:31
pleia2added a note in the pad too, but I think the Low bugs didn't get many eyeballs last time, many need to be retargeted to icehouse17:32
*** atiwari has joined #openstack-infra17:32
anteayapleia2: I've unassigned myself on a few other bugs, may as well free them up so someone can work on them17:32
pleia2anteaya: thanks!17:32
anteayak17:32
*** sandywalsh has quit IRC17:33
devanandajeblair: on client releases, are there any guidelines for version numbering across projects?17:34
jeblairdevananda: there is no coordination17:35
*** elasticio has joined #openstack-infra17:35
mateljeblair: Hi, mordred told me you are the nodepool expert: I would like to create a bootstrap script, that converts an official RS image to XenServer. The issue that I have, is I need to restart the server - what do you think, what is the best way to handle this?17:35
*** blamar has joined #openstack-infra17:35
*** dizquierdo has quit IRC17:35
jeblairmatel: it will need a code change to nodepool itself; the script won't be able to handle it alone.  i can give you more help later this afternoon if you like (we're doing bug cleanup right nowe)17:36
fungianteaya: i added a good example for you on 1245905. should be a really trivial way out17:36
mateljeblair: thanks, unfortunately I am about to end my day - I will send a patch tomorrow to nodepool, so that we can discuss that, what do you think?17:37
jeblairmatel: sounds great17:37
mateljeblair: cool, thanks.17:37
*** sarob has joined #openstack-infra17:37
openstackgerritA change was merged to openstack/requirements: Pin Sphinx to <1.2  https://review.openstack.org/6116417:37
fungianteaya: though bug 1220620 is related, so you might want to grab it too?17:38
uvirtbotLaunchpad bug 1220620 in openstack-ci "manage-projects is inscrutable on gerrit failures" [Low,Triaged] https://launchpad.net/bugs/122062017:38
fungianteaya: it might even be reasonable to mark one that one as a dupe17:38
fungijeblair: clarkb: i noticed the other day that pypi.slave (in the rs openstackjenkins account) is still a legacy server. i'm adding it to the list in bug 108278717:41
uvirtbotLaunchpad bug 1082787 in openstack-ci "Migrate remaining servers from rackspace classic to rackspace nova" [High,Triaged] https://launchpad.net/bugs/108278717:41
anteayafungi: thanks I'll look17:42
*** loq_mac has quit IRC17:42
*** hasharCall has quit IRC17:42
jeblairsdague: i need help understanding if https://bugs.launchpad.net/tempest/+bug/1248924 is a problem17:42
uvirtbotLaunchpad bug 1248924 in tempest "Ceilometer is not installed on check-grenade-devstack-vm environment" [Undecided,New]17:42
SergeyLukjanovoh, sphinx<1.2 propose req update job failed :( https://jenkins.openstack.org/job/propose-requirements-updates/35/17:43
*** nati_ueno has quit IRC17:45
*** NikitaKonovalov has quit IRC17:46
jeblairSergeyLukjanov: wow, interesting17:46
SergeyLukjanov2013-12-10 17:38:46.931 | Downloading refs/changes/69/60969/1 from gerrit into review/jenkins/openstack/requirements17:46
SergeyLukjanov2013-12-10 17:38:46.931 | fatal: unresolved deltas left after unpacking17:46
SergeyLukjanov2013-12-10 17:38:46.931 | fatal: unpack-objects failed17:46
jeblairSergeyLukjanov: any chance you want to try to figure that out?17:47
*** jcooley_ has joined #openstack-infra17:48
SergeyLukjanovjeblair, trying atm, but have no ideas currently17:48
SergeyLukjanovjeblair, will digg into it17:48
jeblairSergeyLukjanov: cool, thanks!17:48
anteayafungi: I agree 1220620 is a dupe of 1245905, I don't see a status setting to indicate it is a duplicate. Do I just state so in the comments with a link to 1245905?17:49
jeblairanteaya: search for "Mark as duplicate" on the page17:49
*** jcooley_ has quit IRC17:50
anteayajeblair: thanks, that worked17:50
SergeyLukjanovjeblair, it looks like CR code ref isn't accessible17:53
SergeyLukjanovhttps://github.com/openstack/requirements/tree/refs/changes/69/60969/117:53
jeblairSergeyLukjanov: it should be getting that from gerrit17:55
clarkbmorning17:55
*** NikitaKonovalov has joined #openstack-infra17:56
SergeyLukjanovjeblair, but if everythin is ok such refs are always available in github17:56
SergeyLukjanovclarkb, o/17:56
*** HenryG has joined #openstack-infra17:56
jeblairclarkb: welcome to bug day17:57
jeblairhttps://etherpad.openstack.org/p/cibugreview-december201317:57
jeblairclarkb: i'm triaging new bugs17:57
SergeyLukjanovgit review -d I2d8f240009d2f847b321baa011880f753783be83 => http://paste.openstack.org/show/54775/17:57
SergeyLukjanovjeblair, fatal: Couldn't find remote ref refs/changes/69/60969/117:57
SergeyLukjanovwith ssh://slukjanov@review.openstack.org:29418/openstack/requirements.git as remote17:58
jeblairSergeyLukjanov: ah.  that change is not in the requirements repo; it's in cinder.17:58
jeblairhttps://review.openstack.org/#/c/60969/17:58
SergeyLukjanovjeblair, oh... incorrect CR id, my bad, excuse me17:59
jeblairclarkb: can you update (and triage as appropriate) this bug: https://bugs.launchpad.net/openstack-ci/+bug/125202417:59
uvirtbotLaunchpad bug 1252024 in tempest "gate-tempest-devstack-vm-neutron failing on stable/havana" [Undecided,New]17:59
clarkbsure17:59
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Soft Delete Stale Bugs from elastic-check page  https://review.openstack.org/6114918:04
jeblairclarkb: also https://bugs.launchpad.net/openstack-ci/+bug/125454618:05
uvirtbotLaunchpad bug 1254546 in openstack-ci "missing/ corrupt history for https://etherpad.openstack.org/p/tripleo-icehouse-ha-production-configuration" [Undecided,New]18:05
jeblairRyan_Lane: any chance you could look at https://bugs.launchpad.net/openstack-ci/+bug/1254911 ?18:06
uvirtbotLaunchpad bug 1254911 in openstack-ci "UniversalLanguageSelector breaks wiki skin" [Undecided,New]18:06
*** SushilKM has quit IRC18:07
*** nati_ueno has joined #openstack-infra18:08
fungiSergeyLukjanov: seems to work for me... http://paste.openstack.org/show/54776/18:09
SergeyLukjanovfungi, yup, I've used incorrect repo :(18:10
fungiSergeyLukjanov: just wondering why the jenkins job failed that way though18:10
* Ryan_Lane looking18:11
*** randallburt has quit IRC18:11
Ryan_Laneah, right18:11
Ryan_LaneULS is required for translate18:11
Ryan_LaneI need to discuss this with nikerabbit18:11
SergeyLukjanovfungi, and it seems like that's time to upgrade git-review in slaves ;)18:12
*** loq_mac has joined #openstack-infra18:12
clarkbmordred: jeblair: fungi: re sphinx release brokeness see my comments on https://review.openstack.org/#/c/57473/ I noticed several problems with the new release but they had bugs open for all of them18:12
fungiSergeyLukjanov: nah--we've removed that warning entirely in later git-review anyway. mostly just new features between what's running there and later releases18:12
clarkbdidn't figure they would release with those bugs open18:12
clarkbthis makes me unhappy18:12
fungiclarkb: well, technically it wasn't a bug *in* sphinx18:13
fungimerely exposed by it18:13
fungithough there does seem to be some fingerpointing from the distutils bug report back to sphinx's choice of behavior18:14
clarkbregardless, sphinx doesn't work18:14
jeblairwe should switch to tex for dev docs too.18:14
clarkband there are bugs reporting this behavior to them that have chosen to ignore?18:14
clarkbfungi: it isn't just the path issue, see my othe rcomment18:14
*** AaronGr is now known as AaronGr_afk18:15
fungiclarkb: ahh yes. well the path issue didn't so much have an issue open in their tracker (just some comments on a resolved one and an open bitbucket pull-request to work around distutils)18:17
*** vkozhukalov has joined #openstack-infra18:18
jeblairmordred: can you update https://bugs.launchpad.net/openstack-ci/+bug/1257905 ?18:18
uvirtbotLaunchpad bug 1257905 in openstack-ci "mirror building broken requirements testing" [Undecided,New]18:18
SergeyLukjanovI've executed propose-requirements-update.sh and after it clones repo, I've exec git review -s in it18:18
SergeyLukjanovhttp://paste.openstack.org/show/54777/18:18
SergeyLukjanovbut after re-clone it works ok again18:19
*** vipuls is now known as vipuls-away18:20
*** pabelanger has quit IRC18:21
openstackgerritSergey Lukjanov proposed a change to openstack-infra/release-tools: Initial setup of python project layout  https://review.openstack.org/6084018:23
*** Abhishek_ has joined #openstack-infra18:23
lifelesssdague: it is dayish now18:23
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Ability to Soft Delete Stale Bugs from elastic-recheck  https://review.openstack.org/6114918:23
*** pabelanger has joined #openstack-infra18:23
*** krotscheck has joined #openstack-infra18:24
*** adalbas has quit IRC18:24
*** vipuls-away is now known as vipuls18:25
jeblairfungi, pleia2, clarkb, mordred: i've triaged the new bugs18:25
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Delete stale bugs from elastic-recheck page  https://review.openstack.org/6118618:25
pleia2jeblair: thank you!18:26
fungijeblair: i saw--thanks!18:26
*** AaronGr_afk is now known as AaronGr18:28
jeblairclarkb: i think we should close https://bugs.launchpad.net/openstack-ci/+bug/117238818:30
uvirtbotLaunchpad bug 1172388 in openstack-ci "automatically run logstash bug-queries to identify known causes of failures" [Medium,In progress]18:30
fungisounds like elastic-recheck to me!18:30
*** klrmn has left #openstack-infra18:30
clarkbjeblair: ++18:30
jeblairmordred, clarkb: is https://bugs.launchpad.net/openstack-ci/+bug/1172409 done?18:31
uvirtbotLaunchpad bug 1172409 in openstack-ci "better feedback for check jobs with new reqs" [Medium,Triaged]18:31
*** klrmn has joined #openstack-infra18:31
*** klrmn has left #openstack-infra18:32
fungii tend to say yes. the failure message from zuul is pretty clear, and the log is reasonably easy to pick a useful mismatch error out of18:34
funginot like back when that job was first introduced18:34
clarkband we are now testing requirement update proposals against pypi.o.o for all tests run under requirements18:34
clarkber pypi.python.org18:34
jeblairmordred, ttx: how do we update https://bugs.launchpad.net/openstack-ci/+bug/1172444 to say fix released for openstack-ci?18:34
uvirtbotLaunchpad bug 1172444 in python-neutronclient "migrate all projects to flake8" [Undecided,Fix committed]18:34
clarkbjeblair: I say close it18:35
mordredjeblair: yes18:35
*** mrmartin has joined #openstack-infra18:36
jeblairttx, mordred: https://bugs.launchpad.net/openstack-ci/+bug/1179008 can we mark that as fix released for openstack-ci too?18:37
uvirtbotLaunchpad bug 1179008 in python-neutronclient "rename requires files to standard names" [Medium,In progress]18:37
*** shardy is now known as shardy_afk18:37
fungii think i was going to do that a while back, but was having trouble with the lp webui timeouts and didn't know about the smtp api yet18:38
*** adalbas has joined #openstack-infra18:38
*** jerryz has joined #openstack-infra18:43
*** loq_mac has quit IRC18:44
*** randallburt has joined #openstack-infra18:44
*** jcooley_ has joined #openstack-infra18:47
*** MarkAtwood has quit IRC18:47
*** jcooley_ has quit IRC18:49
*** MarkAtwood has joined #openstack-infra18:49
*** jcooley_ has joined #openstack-infra18:49
jeblair[Low] only replicate some gerrit repos https://bugs.launchpad.net/openstack-ci/+bug/94939018:51
jeblairI'm not sure what we want to do in this bug.  Which projects do we want to include and exclude?  More description would be nice.18:51
uvirtbotLaunchpad bug 949390 in openstack-ci "only replicate some gerrit repos" [Low,Triaged]18:51
jeblairclarkb: ^ did you write that?18:51
jeblairzaro: ^ or was that you?18:51
jeblairyou have almost the same color18:51
clarkblooking18:52
clarkbjeblair: I am confused, lp says you reported the bug18:52
jeblairclarkb: i'm asking who wrote the note in the etherpad18:52
jeblair"I'm not sure what we want to do in this bug.  Which projects do we want to include and exclude?  More description would be nice."18:52
*** jcooley_ has quit IRC18:53
clarkbjeblair: I think that must be zaro because I don't remember writing it18:53
jeblairzaro: ^ ?18:53
sdagueok, going through backlog post lunch now18:53
zarojeblair: i was me.18:53
jeblairzaro: see etherpad.18:54
sdaguejeblair: which thread is this that you were referencing?18:54
*** jcooley_ has joined #openstack-infra18:54
zarojeblair: ok. thanks for the clarity.18:54
jeblairsdague: we were looking for a "standard" gate failure tag.  we found "gate-failure" mentioned in http://lists.openstack.org/pipermail/openstack-dev/2013-November/020631.html18:54
jeblairsdague: that sound right to you?  do you remember any other discussion or any other proposed tag names?18:55
jeblairsdague: (launchpad bug tag, that is)18:55
jeblairnow i know why i don't remember that.  that was right after i left for thanksgiving, and of course i had to declare mailing list bankruptcy after returning.18:56
*** salv-orlando has joined #openstack-infra18:56
sdaguejeblair: I didn't remember anything else. It only makes sense to tag these things if we can get a global report on that tag18:56
jeblairsdague: it will help us in openstack-ci to find/identify currently known infra breakage causes18:57
clarkbshould https://bugs.launchpad.net/openstack-ci/+bug/1242569 be marked fix released?18:57
uvirtbotLaunchpad bug 1242569 in openstack-ci "manage-projects error on new project creation" [Critical,Fix committed]18:57
sdaguejeblair: sure18:57
jeblairsdague: especially since we expect ours to stay open longer (a network error is not something we can fix with a patch)18:57
sdaguehttps://bugs.launchpad.net/openstack/+bugs?field.tag=gate-failure actually seems to work18:57
jeblairsdague: nice18:58
jeblairclarkb: yes; i've been marking similar bugs fixed18:58
fungiclarkb: i think it's probably safe to close 124256918:59
clarkbdone18:59
clarkbjeblair: do we want to track https://bugs.launchpad.net/openstack-ci/+bug/1172400 under both openstack-ci and zuul? is that intentional so that people see it?19:00
uvirtbotLaunchpad bug 1172400 in zuul "dequeue changes on -2 events" [High,Triaged]19:00
jeblairclarkb: yes, it's something that needs to be fixed in zuul that is affecting openstack ci19:01
clarkbk19:01
jeblairi think it's meeting time19:01
*** che-arne has quit IRC19:01
*** jeblair changes topic to "Discussion of OpenStack Project Infrastructure | Docs http://ci.openstack.org/ | Bugs https://launchpad.net/openstack-ci | Code https://git.openstack.org/cgit/openstack-infra/"19:02
*** jcooley_ has quit IRC19:02
*** esker has joined #openstack-infra19:04
*** thomasbiege has quit IRC19:05
*** Abhishek_ has quit IRC19:07
*** jcooley_ has joined #openstack-infra19:07
*** sarob has quit IRC19:11
*** sarob has joined #openstack-infra19:11
SergeyLukjanovabout the propose-requirements-update - can't find any issues, it works ok locally for me19:12
*** salv-orlando has quit IRC19:13
SergeyLukjanovwhat's about re-triggering job to check the behaviour?19:13
*** jcoufal has joined #openstack-infra19:13
*** hogepodge has quit IRC19:14
fungiSergeyLukjanov: we can try that after the meeting--i agree chances are there's some sort of race at work there19:14
*** adalbas has quit IRC19:15
*** sarob has quit IRC19:16
sdagueif fires are properly out, post meeting landing this - https://review.openstack.org/#/c/57066/ would be really good19:17
*** jcooley_ has quit IRC19:18
clarkbsdague: did maurosr's changes to grenade merge then?19:18
sdagueclarkb: yes19:18
sdaguethat's why that passes now19:18
clarkbwoot, will review 57066 after meeting19:18
SergeyLukjanovfungi, cool19:18
maurosrclarkb: yup, waiting  57066 to get one more change on grenade and it's finished19:19
*** salv-orlando has joined #openstack-infra19:19
*** weshay has quit IRC19:20
*** yolanda has quit IRC19:24
*** mrmartin has quit IRC19:26
*** mordred has quit IRC19:26
*** mordred has joined #openstack-infra19:26
openstackgerritA change was merged to openstack-infra/config: Updates to static.o.o volume management docs  https://review.openstack.org/5961519:27
ttxjeblair: will send email to LP to close them19:29
jeblairttx: thank you19:31
*** weshay has joined #openstack-infra19:31
*** jcoufal has quit IRC19:31
*** jcoufal has joined #openstack-infra19:32
*** salv-orlando has quit IRC19:34
openstackgerritSteve Martinelli proposed a change to openstack/requirements: Add oauthlib library to replace oauth2  https://review.openstack.org/6119819:34
ttxwow, got an erreor even on the email. Nice19:37
ttxwill retry19:37
SergeyLukjanov:)19:37
* SergeyLukjanov thinking about what to do if "can't report a bug because of a bug in bug reporting system"19:38
*** klrmn has joined #openstack-infra19:38
fungiSergeyLukjanov: we already have the answer. give up and write your own bug tracker19:39
SergeyLukjanovfungi, yup, I'm already involved into it :)19:39
* fungi nods19:40
*** AaronGr is now known as AaronGr_afk19:40
pleia2ttx: mind if I jump in on this bug? https://bugs.launchpad.net/openstack-ci/+bug/117244019:43
uvirtbotLaunchpad bug 1172440 in openstack-ci "bug day stats should be in openstack-infra" [Medium,Triaged]19:43
pleia2ttx: I wanted to add some things to it and I believe it still needs to move off an old server, so I'm happy to nudge it along19:44
pleia2might grab https://bugs.launchpad.net/openstack-ci/+bug/1172441 too once that's done19:44
uvirtbotLaunchpad bug 1172441 in openstack-ci "rc countdown should be in openstack-infra" [Medium,Triaged]19:44
*** olaph has joined #openstack-infra19:46
ttxpleia2: no, please do19:47
clarkbzaro: https://bugs.launchpad.net/openstack-ci/+bug/979227 is that something you can take on with the gerrit upgrade?19:47
uvirtbotLaunchpad bug 979227 in openstack-ci "convert gerrit's mysql tables to utf8" [Medium,Triaged]19:47
ttxthe rc countdown one is slightly more complex because it's not running all the time19:47
pleia2ttx: ok, I'll touch base with you on that one after I handle bugs19:47
fungiclarkb: zaro: i think it's somethign we want to test on review-dev as part of the upgrade testing, so we can limit the impact of the multiple outages we'll need for production improvements to gerrit19:49
clarkbmaurosr: 57066 is the dependency of another grenade change?19:51
ttxjeblair: looks like even my EmailInterface Launchpad trick has its limits. Got errors in response to email. My suggestion is to remove the openstack-ci task altogether (click on the "-" icon next to the project name)... I think that's not subject to timeouts19:51
*** randallburt has quit IRC19:52
lifelessttx: it will still notify19:52
*** hashar has joined #openstack-infra19:52
maurosrclarkb: yup, for this one https://review.openstack.org/#/c/53874/19:53
ttx"less subject to timeout" maybe19:53
maurosrclarkb: which is pretty much only a clean up19:53
*** jcooley_ has joined #openstack-infra19:53
maurosrand soon will be used to put havana=>icehouse new config for each project19:54
clarkbwoot sounds good19:54
*** weshay has quit IRC19:57
ttxfungi: could you add me to release-tools-core ? Currently the team is empty19:57
fungittx: sure thing19:58
jeblairsmarcet: hi!  since your ssh key is on an infra server, can you review our ssh access policy and make sure you're okay with it?19:58
jeblairsmarcet: http://ci.openstack.org/sysadmin.html#ssh-access19:58
fungittx: added19:58
ttxfungi: thx19:59
smarcetjeblair: sure on it19:59
SpamapShrm19:59
fungijeblair: i think he already did. i'll dig that up for the record19:59
SpamapSwhy can I search gerrit for 'boo' but not 'ephemeral' ?19:59
jeblairfungi: oh, sorry19:59
*** hashar has quit IRC19:59
fungijeblair: though i may be confusing that with the key we added for groups-dev20:00
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Use openstack-doc-tools for validation of operations-guide  https://review.openstack.org/6120420:00
SergeyLukjanovthere was not enough time on meeting, but as for about moving git2lp mappings out of the jeepyb, here's an email with links to all reviews http://lists.openstack.org/pipermail/openstack-infra/2013-December/000491.html20:01
SergeyLukjanovand I'm planning to add docs about it tomorrow20:02
SergeyLukjanovand I'm thinking about how to test direct release option ;)20:02
*** hashar has joined #openstack-infra20:02
smarcetjeblair: yeah its ok20:02
fungijeblair: smarcet: looks like i finally got involved with https://review.openstack.org/53644 so far into the review process that i forgot to ask him to comment there agreeing to the ssh access policy. smarcet can you add a review comment on that stating you agree to it?20:03
SergeyLukjanovfungi, could you please re-trigger https://jenkins.openstack.org/job/propose-requirements-updates/35/ ?20:03
fungiSergeyLukjanov: yep--trying that now20:03
SergeyLukjanovfungi, thx!20:04
bnemecHey, it kind of looks like all of the docs jobs are failing right now.20:04
bnemecExample: https://review.openstack.org/#/c/58993/20:04
*** jcooley_ has quit IRC20:04
fungijeblair: we still need to use trigger-job.py for rerunning jobs from the post pipeline, right?20:04
bnemecEvery recent job I see in the zuul status page also has a failed docs job.20:05
fungibnemec: projects need to update their Sphinx version spec to match what's in openstacl/requirements20:05
oubiwan__jeblair: I've got some folks who want to do some dev testing against trystack, and they don't have FB accounts20:05
oubiwan__jeblair: who should I ping?20:05
smarcetfungi: ok i will20:05
fungismarcet: thanks!20:06
bnemecfungi: Okay, thanks.20:06
oubiwan__jeblair: also "hi" -- it's been ages ;-)20:06
clarkbfungi: https://review.openstack.org/#/c/57066/ when you have a minute20:06
*** jcoufal has quit IRC20:07
clarkboubiwan__: trystack is special. It is volunteer run by people not us. I think there is an email address on trystack.org that you can try20:07
smarcetfungi: i think that is not possible, i mean , review option its gone rite now20:07
fungismarcet: if you're not logged into gerrit, there won't be a review button20:07
smarcetok let me try it20:08
smarcetone sec20:08
fungismarcet: but if you log in you should get a review button even though the change is in a merged status20:08
SergeyLukjanovfungi, oh, it was failed prev. time too... https://jenkins.openstack.org/job/propose-requirements-updates/34/consoleFull20:08
pleia2woo, under 200 bugs!20:08
smarcetfungi: ok checking on it20:08
*** rfolco has quit IRC20:08
SergeyLukjanovfungi, with error: ssh://review.openstack.org:29418/openstack/python-neutronclient.git did not send all necessary objects20:08
fungiSergeyLukjanov: yep, failed again when rerun... https://jenkins.openstack.org/job/propose-requirements-updates/36/console20:09
*** weshay has joined #openstack-infra20:09
zaroclarkb, fungi : zaro has assigned 979227 to himself20:09
smarcetfungi: done20:09
fungismarcet: thanks so much!20:10
clarkbzaro: woot thank you20:10
* zaro meant bug 97922720:10
uvirtbotLaunchpad bug 979227 in openstack-ci "convert gerrit's mysql tables to utf8" [Medium,Triaged] https://launchpad.net/bugs/97922720:10
*** jcooley_ has joined #openstack-infra20:10
SergeyLukjanovfungi, i have a bad feeling about using --depth=1, I had some problems with it several years ago, but can't remember the actual reason20:10
*** NikitaKonovalov has quit IRC20:10
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Pin Sphinx to <1.2  https://review.openstack.org/6117120:11
lifelessSergeyLukjanov: shallow clones? Can't update them with gerrit20:11
lifelesslet me find the bug20:11
*** jcooley_ has quit IRC20:11
oubiwan__clarkb: great -- thanks for the tip; found an email at the bottom of the page and giving it a try :-)20:11
lifelessSergeyLukjanov: https://bugs.launchpad.net/openstack-ci/+bug/119566620:12
uvirtbotLaunchpad bug 1195666 in openstack-ci "poor interaction with thin clones" [Medium,Triaged]20:12
*** denis_makogon has joined #openstack-infra20:12
clarkboubiwan__: good luck, and if you don't get a response you can probably try sending mail to the openstack mail list20:12
openstackgerritA change was merged to openstack-infra/nodepool: Pin Sphinx to <1.2  https://review.openstack.org/6117320:12
SergeyLukjanovlifeless, oh, thank you for catching that!20:13
SergeyLukjanovlifeless, looks like we could replace clone from gerrit with git.o.o20:13
*** jcooley_ has joined #openstack-infra20:13
fungilifeless: yes, seems it works if you hit the apache mirror of the repo but not the jetty built-in20:13
lifelessright20:13
lifelesswe tried thin clones in tripleo, it's how we uncovered this bug20:14
fungilifeless: though it also works via git+ssh, which is what that job is doing20:14
mtreinishfungi: was this you again?: http://logs.openstack.org/04/58604/5/gate/gate-tempest-docs/b2104df/console.html20:14
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Use openstack-doc-tools for validation of operations-guide  https://review.openstack.org/6120420:15
fungimtreinish: no, projects need to update their Sphinx version spec to match what's in openstack/requirements20:16
*** jcooley_ has quit IRC20:16
pabelangerfungi, okay, so you don't need to do a complete sync to pull it in?20:16
bnemecmtreinish: https://bugs.launchpad.net/openstack-ci/+bug/125951120:16
fungimtreinish: see the commit message from the recent emergency change to the reqs repo for details20:16
uvirtbotLaunchpad bug 1259511 in openstack-ci "sphinx docs gating jobs are broken (source_dir must be a directory)" [Critical,In progress]20:16
fungipabelanger: i believe only the lines you change in your reqs files are checked against the globa list20:17
fungiso it doesn't need to be a full sync with global reqs in one change20:17
openstackgerritKhai Do proposed a change to openstack-infra/config: fix passing of params to nodepool module  https://review.openstack.org/6121320:17
zaroclarkb: ^20:18
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add e-r query for nova-network bug 1258848  https://review.openstack.org/6115820:18
uvirtbotLaunchpad bug 1258848 in nova "vif doesn't exist Error message in tempest runs" [Critical,Triaged] https://launchpad.net/bugs/125884820:18
clarkbfungi: pretty sure you need to do a full sync20:18
clarkbzaro: thanks, I will leave a note about the scoping on that change20:18
mtreinishfungi, bnemec: ok thanks20:19
fungiclarkb: oh? i can see how that could be a blocker for some projects when they're fairly out of date20:19
*** jcooley_ has joined #openstack-infra20:19
clarkbfungi: why?20:20
fungisince it would be hard to land unrelated changes involving support for different reqs adjustments20:20
pabelangerfungi, thanks20:20
clarkbremember all integration testing is done with that list anyways. Projects should just accept it imo20:20
fungieh, good point20:20
fungiclarkb: well, not entirely20:21
fungiclarkb: reqs changes which expose issues in integration testing, yes20:21
fungireqs changes which only impact unit testing, doc builds, et cetera not so much20:21
openstackgerritIlya Kharin proposed a change to openstack-infra/config: Initial configuration for python-rallyclient  https://review.openstack.org/6121520:21
clarkbright there is still the potential for hurt in the tests and doc builds and so on20:22
clarkbbut if we start allowing piecemeal syncing we run into the oslo problem20:23
clarkbit will get ignored until you have a major problem then everything hurts more20:23
clarkbthe automatic proposal of requirements updates should help this20:23
fungiyup. so more eyeballs on http://logs.openstack.org/04/58604/5/gate/gate-tempest-docs/b2104df/console.html20:23
fungire-ran it with the same params, got the same end result20:24
fungino, wait, wrong link again20:24
clarkbfungi: wrong log I think :)20:24
fungihttps://jenkins.openstack.org/job/propose-requirements-updates/36/console20:24
* fungi has too many irons in one fire20:24
fungias lifeless suggests, it *could* be shallow clone issues20:26
fungithat error certainly can arise when you try to update one20:26
SergeyLukjanovfungi, yup, checking it locally now20:26
zaromgagne: ping20:26
mgagnezaro: hi20:26
zaromgagne: take a look? https://review.openstack.org/#/c/6121320:27
clarkbfungi: that works locally for me. The biggest thing that stands out to me is git-review is old on that host according to the banner, but I have up to date git-review20:27
SergeyLukjanovclarkb, ++20:27
clarkboh wait nevermind error is in review -d20:27
*** jerryz has quit IRC20:27
*** ywu has quit IRC20:28
SergeyLukjanovhm, here is an error I have when cloning with --depth=1 and git review -s20:29
SergeyLukjanovhttp://paste.openstack.org/show/54780/20:29
*** vipuls is now known as vipuls-away20:30
SergeyLukjanovand it works ok if clone w/o --depth=120:30
clarkbSergeyLukjanov: weird, git review -s works for me but git review -d fails like it does in the job log20:30
clarkbwhen using --depth=120:30
zaromgagne: actually just read the last comment in the review.20:31
*** jcooley_ has quit IRC20:31
mgagnezaro: yes, just got interrupt, 2s20:31
*** jcooley_ has joined #openstack-infra20:31
clarkbI need to grab lunch, running on fumes back in a bit20:32
openstackgerritIlya Kharin proposed a change to openstack-infra/config: Gerritbot: watch master on python-climateclient  https://review.openstack.org/6122020:33
* zaro finds food with clarkb20:35
*** jcooley_ has quit IRC20:36
* fungi is going heads-down to knock out the openid project import change20:37
dkranzmordred: It's kind of unfortunate that we are making tempest tests a requirement for incubation but telling people they can't actually have tests in tempest unless they are incubated... Or are we not saying that?20:37
SergeyLukjanovfungi, what do you think about removing --depth=1?20:38
SergeyLukjanovfungi, due to the local testing it raises errors in git-review ...20:38
fungiSergeyLukjanov: oh, you were able to get an error trying to check that change out onto a shallow clone?20:39
SergeyLukjanovfungi, [00:30:29] <clarkb> SergeyLukjanov: weird, git review -s works for me but git review -d fails like it does in the job log20:39
SergeyLukjanovfungi, git review -s isn't working for me in case --depth=1 enabled20:40
SergeyLukjanovfungi, even if I'm using git.o.o for initial clone20:40
fungioh, i did miss those comments above. so yes, we need full clones there i guess20:40
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: cloverphp publisher  https://review.openstack.org/5294020:40
SergeyLukjanovfungi, so, I'll go to add the corresponding patch, are you ok with it?20:41
fungiSergeyLukjanov: yes, i think we can get it rushed through to fix that20:42
openstackgerritAntoine Musso proposed a change to openstack-infra/config: zuul: support 'recheck nobug'  https://review.openstack.org/6108620:42
SergeyLukjanovfungi, k20:42
*** klrmn has left #openstack-infra20:43
mgagnezaro: commented20:43
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Use full clone in propose requirements updates  https://review.openstack.org/6122420:47
fungiSergeyLukjanov: thanks20:47
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Ability to Soft Delete Stale Bugs from elastic-recheck  https://review.openstack.org/6114920:47
SergeyLukjanovfungi, np, I'm glad to help in infra area when it could be done by me20:48
fungiclarkb: if you're cool with https://review.openstack.org/61224 we should try to get that in asap and retrigger the proposal job again20:48
fungi(once you're done lunching)20:48
*** dstanek has quit IRC20:49
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Delete stale bugs from elastic-recheck page  https://review.openstack.org/6118620:51
mgagnehow can I sanely uninstall an egg? :O20:51
fungimgagne: delete the virtualenv you installed it into?20:51
fungipip uninstall sort of works, but has a tendency to still leave cruft20:51
mgagnefungi: but wait, there is more! it's on windows and I don't have venv20:52
fungimgagne: reformat? (clearly i'm lacking good recommendations for this)20:52
mgagnefungi: alright, will try pip, don't know how badly it will crash ^^'20:53
openstackgerritIlya Kharin proposed a change to openstack-infra/config: Initial configuration for python-rallyclient  https://review.openstack.org/6121520:53
mgagnefungi: you are the king, it surprisingly worked well20:53
*** DennyZhang has joined #openstack-infra20:54
fungimgagne: install several versions of something on top of distro-provided versions of the same and then try to pip uninstall and see how smoothly it goes :(20:55
mgagnefungi: fortunately, I don't have that problem (yet)20:55
annegentlefungi: do you have links to the sphinx bugs that are preventing our builds? Just want one more level of detail if I can get to it, on the breakage20:59
fungiannegentle: see the urls in this commit message... http://git.openstack.org/cgit/openstack/requirements/commit/?id=0e8862e958813a031fe7398440e8a3866b42a8e421:00
annegentlefungi: and it was .../doc/source that sphinx doesn't like? the relative path or some such?21:01
pleia2heading out for a bit for lunch with some mozillians, then dr appointment (I've had a cold since Hong Kong, bored of that now :))21:02
*** melwitt has joined #openstack-infra21:02
annegentlepleia2: blerg! That's a long cold!21:02
pleia2annegentle: yeah, I think it's a sinus infection, time to get some antibiotics21:02
*** ryanpetrello has quit IRC21:02
*** smarcet has quit IRC21:03
fungiannegentle: i'm not clear on the details there. i think it's that distutils doesn't like sphinx referencing a path which isn't an encoded string type21:03
*** ryanpetrello has joined #openstack-infra21:03
*** HenryG has quit IRC21:05
*** dstanek has joined #openstack-infra21:06
*** afazekas has quit IRC21:06
*** AaronGr_afk is now known as AaronGr21:07
mordredmgagne: what are you trying to uninstall?21:08
mgagnemordred: an egg to I can replace it with my own version with a different name =)21:08
mordredmgagne: just delete the egg file or directory21:09
mordredmgagne: you _might_ have to also delete a line from easy_install.pth21:09
mgagnemordred: the egg installs stuff in scripts too21:09
mordredwell, those will likely get overwritten properly by the next install21:09
mgagnemordred: pip uninstall looks to have removed everything21:09
mordredawesome21:09
mordredjust saying - it's totally possible to sledgehammer if needed21:09
mgagnemordred: I'm already using a sledgehammer21:10
fungiyeah, when it also adds entrypoints, manpages and so on, it can get messy21:10
*** fbo is now known as fbo_away21:10
*** atiwari has quit IRC21:10
mgagnemordred: and I'm not so proud of it21:10
* mordred afk's for dinner21:11
mgagnemordred: otherwise, if you know how to uncompile an .msi and replace stuff by your own... =)21:11
*** markmc has quit IRC21:11
*** julim has quit IRC21:12
*** DennyZhang has quit IRC21:13
*** DennyZhang has joined #openstack-infra21:13
*** vipuls-away is now known as vipuls21:19
*** lcestari has quit IRC21:19
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Alphabetize projects.yaml yet still again  https://review.openstack.org/6123321:22
* fungi sighs again \^21:22
*** ladquin_afk is now known as ladquin21:24
*** sharwell has joined #openstack-infra21:25
openstackgerritA change was merged to openstack-infra/devstack-gate: Bump grenade master to use stable/havana.  https://review.openstack.org/5706621:25
fungiclarkb: sdague: maurosr: ^21:26
sdaguew00t21:26
SergeyLukjanovfungi, oh, it'll probably breake my git2lp patches21:26
clarkbfungi: ty21:27
clarkbfungi: looking at 61149 now21:27
fungiSergeyLukjanov: ah, you're right, the alphabetizing might conflict some (hopefully not much)21:27
sharwellzaro: can you explain the deployment process you use for Java projects in Maven Central?21:27
clarkbfungi: SergeyLukjanov and approved21:27
fungiwe need to be better about checking whether entries are being added out of alphabetic order in that file, probably with an automated check job, or else give up on keeping it sorted21:28
jeblairfungi: ++check job21:29
*** jeblair sets mode: -o jeblair21:29
SergeyLukjanovfungi, ++21:29
SergeyLukjanovfungi, that's the only way to keep them sorted21:29
zarosharwell: hiho21:29
zaro!21:29
openstackgerritA change was merged to openstack-infra/config: Use full clone in propose requirements updates  https://review.openstack.org/6122421:29
*** jcooley_ has joined #openstack-infra21:30
SergeyLukjanovfungi, I have an idea about writing simple grep for lists project names, sort them and check that nothing changed21:30
*** alcabrera has quit IRC21:30
zarosharwell: ok.  we don't use the maven release plugin. instead what we do is deploy with curl.21:30
fungii'll give 61224 a few minutes to make it onto proposal.slave and then retrigger again21:30
morganfainbergout of curiosity, is there a known issue going on with docs and gate/check?  i'm seeing a ton of failures that look like https://jenkins01.openstack.org/job/gate-nova-docs/14677/console (`source_dir` must be a directory name in doc building)21:30
zarosharwell: here are a few reasons.. https://wiki.openstack.org/wiki/Deployed_to_a_repository21:30
fungiSergeyLukjanov: yeah, what i did was grep the projects bullets to a file, then sort them to another file and diff them21:31
jeblairSergeyLukjanov: you probably don't have to check that nothing has changed -- i think you could just make sure they are sorted21:31
clarkbmorganfainberg: yes, sphinx 1.2 released and broke everything. You need to update your requirements21:31
morganfainberg*reading scroll back but not seeing any comment on it*21:31
morganfainbergclarkb, ah, woot21:31
annegentlemorganfainberg: yes, we needed to pin to a version of Sphinx <1.221:31
sdaguemaurosr: what's next that needs landing?21:31
annegentlemorganfainberg: mostly looks like just use recheck no bug?21:31
zarosharwell: we deploy our java artifacts just like we deploy our python stuff.21:31
morganfainbergannegentle, sure it would do, just don't want to be issuing that if it's an endless cycle :)21:31
jeblairmorganfainberg: what project?21:32
jeblairannegentle: there's a bug, but there's no point rechecking unless it's been fixed in the associated project21:32
morganfainbergjeblair, looks like ... uhm.. all of them? sphinx 1.2 is broken ^21:32
SergeyLukjanovfungi, what's about landing partially reviewed git2lp patches before sorting? it could be really useful to avoid two places where mappings could be specified21:32
*** elasticio has quit IRC21:32
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Ability to Soft Delete Stale Bugs from elastic-recheck  https://review.openstack.org/6114921:32
*** herndon has quit IRC21:33
annegentlejeblair: morganfainberg: oh ok, got it21:33
jeblairSergeyLukjanov: i think we should go ahead and land the git2lp patches, then add a check, then resort21:33
morganfainbergI'll make sure we get the pinning into keystone's stuff then asap :)21:33
morganfainbergthanks for the info!21:33
fungijeblair: SergeyLukjanov: works for me21:33
jeblairmorganfainberg: yeah, it's in requirements, but the requirements proposal job doesn't quite work yet.  it's almost there.21:33
annegentlejeblair: so the project itself has to pin sphinx <1.2?21:33
annegentlejeblair: ok21:34
SergeyLukjanovfungi, jeblair, thx!21:34
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Delete stale bugs from elastic-recheck page  https://review.openstack.org/6118621:34
fungijust trying to add an entry for another infra project, and got annoyed they were out of order again21:34
jeblairannegentle: yes21:34
SergeyLukjanovfungi, jeblair, I can try to add a simple job to check the order21:34
jeblairSergeyLukjanov: that would be great, thanks!21:34
morganfainbergjeblair, np.  easy to to.  i'll do a quick scan of other projects that don't have it yet since it's a very quick fix.21:34
sharwellzaro: it sounds like what you want is to use the maven-release-plugin to update the project version information and create the tag, but not perform the build/test/release steps. then another system detects the tag, builds, and releases.21:34
jeblairmorganfainberg: can you hang on for just a few mins?21:34
jeblairmorganfainberg: we might be able to retrigger the requirements proposal job21:34
morganfainbergjeblair, sure thing, let me know :)21:34
* fungi will check to see if that's made it to the proposal slave21:35
morganfainbergjeblair, happy to wait for the automated job21:35
clarkbfungi: should be on puppetmaster, if nothing else you can trigger a puppet run21:35
jeblairfungi, clarkb, mordred: do you know why sphinx 1.2 made it to the mirror?21:35
sharwellzaro: i haven't seen anything explaining why this would be useful: https://review.openstack.org/5834921:35
clarkbjeblair: because the mirror job does not install documentation21:35
clarkbjeblair: sphinx 1.2 installs just fine, it doesn't build the docs properly21:36
zarosharwell: actually we don't like the release plugin workflow.  it's doing too much.21:36
fungiclarkb: i think the question is more why was 1.2 downloaded into the mirror even though by then (presumably?) the reqs list was <1.221:36
jeblairfungi: ++21:36
sharwellzaro: yes, i saw on the wiki concerning it doing too much.21:37
zarosharwell: when we wan to make a release we just tag it.  then we get the tag and then we pass the tag to project-version param21:37
zarosharwell: here's the info on tagging. https://wiki.openstack.org/wiki/GerritJenkinsGit#Tagging_a_Release21:37
clarkbjeblair: fungi: sphinx tarball last modified at 18:53 today, I am looking at requirements timestamps now21:38
zarosharwell: so we don't save the version into a file, we just retreive it from git.21:38
sharwellzaro: we need to find a solution that does not alter the "version" property in the pom, even if that means this is the only Java project that avoids it. It is a major headache for developers and simply unnecessary.21:38
jeblairclarkb: is it because of branches?21:38
jeblairclarkb, fungi: openstack/requirements branches?21:38
clarkbchange merged at 9:37 PST /me does maths21:39
fungijeblair: oh, yep. if pypi-mirror also processed stable/havana, we did not pin it there21:39
jeblairfungi: mirror-build jobs run on requirements changes landing, right?21:40
clarkbok it merged first, yes must be the stable branches21:40
fungijeblair: right21:40
fungiwhich means this is also going to impact last-minute merges for the 2013.2.1 release21:40
* mgagne grabs some popcorn21:40
jeblairfungi, mordred, clarkb: so that means there is no way to land a fix like this to requirements without triggering a mirror update that causes the breakage it's designed to fix?21:40
zarosharwell: what do you think about my suggestion to give project-version a default value?21:40
fungiincluding any handful of security fixes i might theoretically need to be pushing tomorrow21:41
sharwellzaro: the release process would thus be 1) commit an update that removes "-SNAPSHOT" from the version property in the pom, 2) tag that commit with the release number, then 3) commit an update that increments the version number and adds -SNAPSHOT back21:41
clarkbsharwell: this was one of the things dwcramer and jeblair and I had discussed at the Portland summit. There wasn't any indication it would be problem then. What specifically is the problem with git tag based versioning?21:41
jeblairfungi, mordred, clarkb: i will propose to other branches, and then delete the sphinx 1.2 tarball from the mirror.  that may need to be repeated.21:41
fungijeblair: agreed. that seems like the only workable solution for the moment21:41
*** jergerber has joined #openstack-infra21:41
*** flaper87|afk is now known as flaper8721:42
sharwellclarkb: users need to be able to clone the source, then do "mvn clean install". Behavior: If they cloned a tag, it will install with the release version, otherwise it will install a snapshot with a version matching a later release appended with -SNAPSHOT21:42
clarkbjeblair: fungi: the only problem with that process I can come up with is we may release stable releases that don't work against pypi21:42
clarkbassuming the pins don't affect all of the projects before that happens but the mirror has been cleaned21:43
jeblairclarkb: that's a good point; we should rope in stable maintainers21:43
clarkbsharwell: ok, so why don't we tackle solving that problem without using the horrid 3 step git commit process21:43
sharwellzaro: clarkb: These builds for release are automated. It is well worth getting the 3 steps I mentioned exactly correct so that all developers trying to get involved with the project find themselves with a normal build process.21:44
*** eharney has joined #openstack-infra21:44
fungitoo bad it's late in apevec's timezone. he needs to be made aware21:44
fungipost to the stable-maint ml would be the best thing, i think21:44
clarkbsharwell: but they cannot be automated with gerrit, all code changes must be reviewed21:44
mgagneclarkb: this is relevant to my interest. we have a very similar (identical) system here and haven't find a way to avoid this commit intensive process when releasing.21:44
clarkbmgagne: we have a way around it21:44
clarkbmgagne: we make the version based on git tags and make the version variable based which is updated before making releases, Seems like the release process is covered but not local dev iterations21:45
jeblairfungi: okay, i'll send a msg once we've determined whether requirements update jobs will run21:45
clarkbI think solving this for local dev iterations is worthwhile rather than giving up and perpetuating this horrible process that makes life hard for everyone21:45
zaromgagne: ohh yeah, didn't know you had the same problem.21:45
mgagneclarkb: I know the way around it. We don't use git tag based release. We rely on a plethora of pom.xml for dependencies management and people expect the version to be included in pom.xml for the reason mentioned by sharwell.21:45
zaromgagne: yeah, just do what we do.21:46
mgagnezaro: don't start me on that one. I have yet to come up with a reasonable alternative to this system.21:46
mgagnezaro: that pleases everyone.21:47
*** dprince has quit IRC21:47
sharwellzaro: clarkb: it also works if you change the order of operations. 1) commit the change to remove -SNAPSHOT, 2) commit the change to increment the version and add -SNAPSHOT back, 3) post the pair of commits as linked changes to code review, 4) after Jenkins merges the pair of commits, tag the first of the 221:47
openstackgerritSolly Ross proposed a change to openstack-infra/os-loganalyze: Use JS to Up-Filter  https://review.openstack.org/5778621:48
openstackgerritSolly Ross proposed a change to openstack-infra/os-loganalyze: Introduce Generic Parsing/Filtering Framework  https://review.openstack.org/5778421:48
openstackgerritSolly Ross proposed a change to openstack-infra/os-loganalyze: Introduce Console Version, Move Common Code  https://review.openstack.org/5778521:48
jeblairfungi: is the propose script deployed?21:49
fungijeblair: clarkb: SergeyLukjanov: https://jenkins.openstack.org/job/propose-requirements-updates/37/console21:49
fungiit's running21:49
openstackgerritSolly Ross proposed a change to openstack-infra/os-loganalyze: Use JS to Up-Filter  https://review.openstack.org/5778621:49
openstackgerritSolly Ross proposed a change to openstack-infra/os-loganalyze: Introduce Generic Parsing/Filtering Framework  https://review.openstack.org/5778421:49
openstackgerritSolly Ross proposed a change to openstack-infra/os-loganalyze: Introduce Console Version, Move Common Code  https://review.openstack.org/5778521:49
*** boris-42 has quit IRC21:49
sharwellzaro: clarkb: that preserves local dev cycle behavior and tagged release behavior that users expect, and if gerrit ever supports better automation you can update that side of it to ease the process. we shouldn't make things harder for developers to get involved just because gerrit wants things done in a different order.21:49
jeblairfungi: cool, that's a manual retrigger?21:49
fungijeblair: yep. getting a lot further21:49
fungiso i think at least *that* bug is addressed by SergeyLukjanov's patch21:50
fungiyay!21:50
clarkbsharwell: I think if we go down that route then we just give up on any tag based release mechanism all together since the automagic simplicity of the process is completely gone21:50
jeblairsharwell: i missed the beginning of your conversation, what project are you discussing?21:50
*** jooools has joined #openstack-infra21:50
zarojeblair: maven-clouddocs-plugin or basically any java project21:51
zarobasically this https://review.openstack.org/#/c/5834921:52
sharwellclarkb: the tag would trigger a release, but the text of the tag would not be part of the build process. but that's the goal here.21:52
clarkbsharwell: right I am saying why bother, just do a mvn release at that point21:52
jeblairi think we should continue with the the plan we discussed with dwcramer; iirc correctly you can control the version built locally with an env var.  perhaps there is an opportunity for improvement to the local build automation there.21:53
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Ability to Soft Delete Stale Bugs from elastic-recheck  https://review.openstack.org/6114921:53
fungigah! failed on a new problem21:53
clarkbfungi: NOOOO21:53
jeblairsharwell: does dwcramer still work on this?21:53
sharwellclarkb: mvn release involves other steps that they'd like to avoid. i'm suggesting that even if we skip those steps, we shouldn't change the particular aspects regarding the way maven-release-plugin introduces the two commits and the tag21:54
fungiclarkb: git-review trying to prompt interactively for a multi-commit change (which it shouldn't be generating, right?)21:54
clarkbfungi: oh hmm there might be a missing --amend21:54
sharwelljeblair: environment variables are not an acceptable solution to any problem on Windows machines21:54
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/elastic-recheck: Delete stale bugs from elastic-recheck page  https://review.openstack.org/6118621:54
jeblairmorganfainberg: the proposal job is not going to immediately work21:54
morganfainbergjeblair. no problem, I'll toss up reviews for the projects i have cloned.21:55
sharwelljeblair: they are a necessary evil in some legacy code, but they never need to be added to a new project21:55
morganfainbergjeblair, i have it mostly ready except for the commit/review call :)21:55
jeblairsharwell: i realize i'm coming in late to this, but i'm surprised at this -- we're at the end of a very long road that we started working with dwcramer on about 8 months ago21:55
jeblairsharwell: so the things you are saying are taking us back to square one21:55
SergeyLukjanovfungi, clarkb, thanks for re-triggering propose-req job, looks like one more problem is here...21:55
fungiSergeyLukjanov: clarkb: yes, i think i see it21:56
*** jooools has quit IRC21:56
SergeyLukjanovwe need anyone to sometimes type 'yes' :)21:56
fungigit commit -a -F-21:56
fungishouldn't there be an --amend>21:56
clarkbfungi: right that owrked with the translation updates because we never git review -d the old change, but now we git review -d the old change21:56
clarkbfungi: maybe we should go back to not using git review -d at all? I am trying to remember why we did that21:56
fungii'm sure it seemed like a simple choice at the time21:57
jeblairsharwell: so is dwcramer still involved in the project?  i think having someone who has been working on this from the beginning in the conversation may make it more productive.21:57
sharwelljeblair: I'm extremely concerned with preserving the ability to attract new developers, and minimize their pain getting involved with the project. The solution I proposed involves a few additional steps during the release process itself, but everyone trying to work with the code base benefits from it.21:57
SergeyLukjanovfungi, looks like it should be amend, https://review.openstack.org/#/c/60980/21:57
clarkbfungi: I remember, it was to make sure we didn't propose code without a diff21:58
sharwelljeblair: yes, dwcramer is still working on the project. my primary role was supporting the initial migration to maven central21:58
clarkbfungi: because gerrit refuses no change changes21:58
fungiclarkb: good point21:58
clarkbfungi: SergeyLukjanov give me a minute will push a quick fix21:58
jeblairsharwell: they are good goals, but again, you're asking zaro to throw away 8 months of work that we clarkb, and i did with dwcramer.21:58
sharwelljeblair: i was not aware that discussions were taking place. had i known, i would have become involved much earlier.22:00
*** jcoufal has joined #openstack-infra22:00
*** 17WAAJ2SG has joined #openstack-infra22:01
openstackgerritClark Boylan proposed a change to openstack-infra/config: Use amend when updating reqs proposal change.  https://review.openstack.org/6124422:02
*** 17WAAJ2SG is now known as jhesketh_22:02
clarkbfungi: SergeyLukjanov ^ that should be better22:02
SergeyLukjanovclarkb, lgtm :)22:03
maurosrsdague: yay it passed =)  https://review.openstack.org/#/c/53874/ now it's almost done.22:03
clarkbwith the translation jobs we ignore individual git review failures because the jobs run for specific projects. A failure in that way is a noop. However requirements proposal jobs run for all projects together so we need to handle that failure case. We could deal with it in a different way as well22:03
jeblairsharwell: so now that you appreciate the position -- this is not a random idea -- this is something we have been working on with dwcramer for a long time and with his approval.  what would be most productive for you?22:04
fungiclarkb: agreed22:04
jeblairsharwell: do you want zaro and clarkb to continue to explain their work?  or would you like to talk to dwcramer and get more background?22:05
jeblairsharwell: or something else?22:05
SergeyLukjanovI think that propose-requirements-update is the really important job and so maybe it should send emails about fails to somewhere?22:06
openstackgerritA change was merged to openstack-infra/config: Use amend when updating reqs proposal change.  https://review.openstack.org/6124422:06
sharwelljeblair: let me formalize my alternative proposal so that we can see if it's workable from the automation side22:06
pabelangerhashar, BTW: not sure if you seen: https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul-packaging,n,z22:07
clarkbSergeyLukjanov: we can add email notifications to the job once we have it working, if there is a group of folks willing to receive them (maybe -infra?)22:07
pabelangerhashar, Would love some reviews :)22:07
SergeyLukjanovclarkb, I'd like to keep eyes on it too if possible22:08
clarkbSergeyLukjanov: we do this with the periodic devstack gate jobs and send mail to the qa team22:08
SergeyLukjanovclarkb, -infra + volunteers group sounds interesting to receive emails about some important jobs22:08
SergeyLukjanovclarkb, yep, I see it now in zuul layout22:09
sharwelljeblair: (and anyone else interested) where should I send a description (since it's longer than would be good here)?22:10
clarkbsharwell: paste.openstack.org or etherpad.openstack.org are good for this sort of thing. (I prefer etherpad)22:11
jeblairsharwell: the openstack-infra@lists.openstack.org mailing list22:12
clarkbor there22:12
hasharpabelanger: haven't seen it that is awessome22:12
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Import the openstackid authentication project  https://review.openstack.org/6125022:13
fungireed: ^22:13
pabelangerhashar, Ya, we don't have any gates yet. I need to see what is needed to get into zuul-dev and jenkins-dev, but have a few ideas on simple gates right now22:13
hasharpabelanger: I have been using jenkins-debian-glue , that is a bunch of shell wrappers around git build package. Works nicely if you respect git buildpackage conventions (aka pristine-tar, upstream and debian branches)22:14
hasharpabelanger: related site http://jenkins-debian-glue.org/  , the author is in Europe and there is #jenkins-debian-glue channel22:15
sdaguemaurosr: nice22:16
fungihashar: is that being used for jenkins.debian.org?22:16
hasharpabelanger: I only have one package passing right now. It shows you lintian / piuparts reports are clean : https://integration.wikimedia.org/ci/view/Ops-DebGlue/job/operations-debs-pybal-debian-glue/ :-]22:16
sdagueone quick update needed on that22:16
maurosrchecking22:16
hasharfungi: never heard of jenkins.debian.org before :(22:16
fungihashar: sorry, i think it's .net actually. /me checks22:16
hasharyeah .net22:16
fungihashar: yep, http://jenkins.debian.net/22:17
*** DennyZhang has quit IRC22:17
pabelangerhashar, Ya, the first gate is to actually build the packaging, however there are some technical discussions needed about it.  I'm going to write an email to the list and see how getting some resources for testing22:17
pabelangerhashar, Sadly, I have to run now. Maybe we can talk later tonight or tomorrow morning22:17
hasharpabelanger: in bed right now so maybe tomorrow :-]22:17
fungiclarkb: SergeyLukjanov: next try... https://jenkins.openstack.org/job/propose-requirements-updates/38/console22:17
pabelangerlater22:17
hasharpabelanger: or by email! spam as much as you want :]22:17
SergeyLukjanovfungi, heh ;)22:18
hasharfungi: nice to see debian getting some CI. Have to sold them the Gerrit/Zuul/Gearman/Jenkins quadruple offer22:18
SergeyLukjanovfungi, clarkb, how many patches will be landed today to fix it?:)22:18
hasharfungi: s/sold/propose/22:18
clarkbSergeyLukjanov: I hope this is the last one :)22:18
fungiSergeyLukjanov: i've lost count, but proposal.slave.openstack.org is getting more of a workout than it has in quite a while22:19
SergeyLukjanovclarkb, fungi, it's already Dec 11 02:19 AM for me and so I hope it'll start working 'today' :)22:20
fungiheh22:20
*** MarkAtwood has quit IRC22:21
SergeyLukjanovlooks like it works ok22:21
SergeyLukjanovpython-glanceclient pass22:21
SergeyLukjanovit fails on glance client last time22:22
fungiyup, that was the latest bit of broken, so hopefully the last issue22:22
SergeyLukjanovand I see some new sync requests from jenkins - https://review.openstack.org/#/q/owner:jenkins,n,z22:22
clarkbwoot22:23
morganfainbergoh very nice22:23
hasharfungi: thanks for the jenkins.debian.net link :]  off to sleep now. *wave*22:24
fungisuccess!!!22:24
SergeyLukjanovclarkb, fungi, we do it :) https://jenkins.openstack.org/job/propose-requirements-updates/38/?22:24
fungigood news, everyone!22:25
clarkbwe made our delivery on time22:25
hasharthat job is crazy :-]  i love the idea22:25
SergeyLukjanovclarkb, fungi, thank you guys, going to sleep ;)22:25
fungiSergeyLukjanov: thank you!22:26
*** AaronGr is now known as AaronGr_afk22:27
*** esker has quit IRC22:28
*** hashar has quit IRC22:30
*** syerrapragada has quit IRC22:32
fungii'm wondering if our stable branch hypothesis is actually correct. i seem to remember we're running with -b so that it doesn't iterate over every branch. digging into the job log to confirm22:32
*** syerrapragada has joined #openstack-infra22:32
*** dizquierdo has joined #openstack-infra22:32
*** jcooley_ has quit IRC22:34
*** jcooley_ has joined #openstack-infra22:34
jeblairfungi, clarkb: https://jenkins.openstack.org/job/post-mirror-python27/327/consoleFull <- i think that's the build that placed sphinx 1.222:34
fungisame one i'm reading through22:34
*** michchap has joined #openstack-infra22:35
fungiand yep, it does run for each branch22:35
jeblairfungi: where's the bit where it downloads stuff? :)22:35
*** michchap has quit IRC22:35
*** derekh has quit IRC22:36
jeblairalso, it didn't seem to actually upload anything since it timed out, so how'd it get to the mirror?22:36
*** michchap has joined #openstack-infra22:36
fungii'm guessing it's in that dead space between 17:38:29 and 18:18:2922:37
fungirun-mirror must be logging somewhere...22:37
jeblairfungi: https://jenkins.openstack.org/job/post-mirror-python26/327/console22:37
jeblairfungi: it's actually the 2.6 build that uploaded it22:38
fungiyup, that one did *not* time out22:38
fungibecause we upped the timeout to 90 minutes for it22:38
*** jcooley_ has quit IRC22:38
sharwellzaro: clarkb: jeblair: here is my initial document regarding the java artifact release process: https://etherpad.openstack.org/p/java-release-process22:39
*** oubiwan__ has quit IRC22:39
jeblairfungi: so stable/grizzly is broken so we can't land the change to the grizzly branch of requirements22:39
jeblairfungi: which means that the mirror runs will continue to pick up 1.222:39
jeblairfungi: this is causing me to be more in favor of not deleting it from the mirror and counting on individual project merges to fix things.22:40
clarkb++22:41
*** SergeyLukjanov has quit IRC22:41
lifelessthe mirror is a poor place to enforce compat; because devstack on developers machines etc will pickup whats on pypi, not our mirror.22:42
lifelessSo +1 to individual fixes.22:42
jeblairlifeless: yeah, that's the long range plan; avoiding a window of lost productivity would have been nice, but with the proposal job now working, we're in a better position to lean more on individual fixes.22:43
jeblairso i guess that makes the long range plan now the nearly-complete short-range plan.  :)22:44
*** mriedem has quit IRC22:44
*** herndon has joined #openstack-infra22:44
fungithough this does point out that the --branch option to pypi-mirror could use fixing22:44
jeblairfungi: what's broken about it?22:45
fungii think we could couple that with the mechanism clarkb introduced to the proposal job for branch-awareness to only update the mirror for the particular branch of requirements which got a change landed22:45
fungijeblair: i think we're calling run-mirror -b for the post job, but it's iterating through all branches regardless22:45
fungii need to dig back through and confirm22:45
fungii believe when the logic got reworked to base the mirror run off the requirements repo rather than the various project repos, the single-branch logic didn't get applied to the requirements repo logic22:47
jeblairfungi: ok.22:47
*** AaronGr_afk is now known as AaronGr22:47
*** mfer has quit IRC22:49
fungimmm, well, the script looks like it should work the way we're wanting. maybe the job needs tweaking22:49
jeblairfungi: stable-maint mail sent.  i'm also going to send an email to -dev.22:50
fungijeblair: thanks22:51
sdagueman, I feel so disconnected that I didn't realize the gate was busted :)22:51
fungiyeah, so that's what it is... i thought we were passing -b but i guess we aren't. so in theory we should be able to -b $ZUUL_REF22:51
clarkbsdague: well it should work fine for any project with the sphinx pin in22:52
sdagueclarkb: right22:52
clarkbfungi: no you can't do that22:52
fungii'll see if i can work up a patch after the meeting tonight22:52
fungiclarkb: no?22:52
clarkbfungi: -b tries to be too smart and actually looks for the argument in a branch listing22:52
sdagueclarkb: but you can't land the sphinx pin until it's in g-r, right?22:52
clarkbfungi: see my comment on the other job to see a long comment about it22:52
jeblairsdague: it's been there since this morning.  :)22:52
fungiclarkb: so we need to refactor around that logic?22:53
clarkbfungi: I worked around it22:53
clarkbfungi: pulling up a link22:53
sdaguejeblair: oh... that's the stable/havana change22:53
fungisdague: yeah, needs stable branch manager approval22:53
jeblairsdague: yeah, that's in testing22:53
sdagueyep22:54
clarkbfungi: https://git.openstack.org/cgit/openstack-dev/pbr/tree/tools/integration.sh#n11622:54
jeblairsdague: req sync changes for projects are up: https://review.openstack.org/#/q/topic:openstack/requirements,n,z22:54
*** dizquierdo has quit IRC22:54
clarkbfungi: so you use -b master, pass --no-update and make sure that the ref you want is already checked out22:54
clarkbfungi: we could make pypi-mirror smarter, would need to update pbr:tools/integration.sh if we do that22:55
fungiclarkb: awesome--i remembered bits of that comment but couldn't recall where i'd seen it22:55
sdaguecool, I went and +2ed everything I had perms on22:56
fungiclarkb: but it feels like we're not too far from being able to have the post mirror jobs only run against the global reqs for the branch being changed at least22:56
*** dcramer_ has joined #openstack-infra22:56
sdaguethe gate queue looks super funny under current conditions22:56
clarkbyup22:56
fungii'll see what i can figure out in a little while22:57
sdagueare we considering doing g-r sync for the docs jobs?22:57
sdaguethen it could have been centrally fixed22:57
clarkbsdague: no22:57
clarkbthe non integration testing should use per project reqs imo22:58
clarkbotherwise per project reqs make little sense22:58
*** bpokorny has quit IRC22:58
jeblairsdague: i believe g-r sync for unit tests (and related single-project jobs) was deemed not desirable because it contradicts what the project says it requires22:58
jeblairas clark said.  :)22:58
sdaguesure22:58
*** thomasem has quit IRC22:58
sdaguethough the docs job is a little different22:58
jeblairsdague: project should be able to build their own docs, i think22:58
sdaguesure22:59
jeblairi sent an openstack-dev email22:59
clarkbjeblair: thanks22:59
sdagueso... nova already had an outstanding requirements patch23:00
sdaguewhich means it didn't take the sphinx one23:00
sdagueknown issue?23:00
jeblairsdague: i think the intent is that it should have been updated23:01
jeblairsdague: so possibly unknown issue23:01
sdaguejeblair: ok, well it didn't23:02
sdaguehttps://review.openstack.org/#/c/60975/23:03
sdagueI was about to fix it manually23:03
*** weshay has quit IRC23:03
jeblairsdague: yeah, if i'm reading https://jenkins.openstack.org/job/propose-requirements-updates/38/consoleFull right, i'm guessing "2013-12-10 22:21:06.032 | + git diff --quiet" for nova didn't do what was expected.23:03
lifelesssdague: oh hai23:04
lifelesssdague: we had a thing to talk aboot23:04
sdaguewe did23:04
sdagued-g metadata server change23:04
lifelessya so my concern about ironic is that ironic + config drive doesn't work23:04
sdaguelifeless: sure23:05
*** jcoufal has quit IRC23:05
sdagueso set that in the ironic job23:05
lifelessoh23:05
clarkbjeblair: looks like it git review -d'd the right change, but the diff didn't find anything different23:05
lifelesssure, I guess - ok23:05
anteayajeblair: I'm trying to find a good time to ask, but there isn't one, have you decided on a date and location for the -infra mid-cycle meet up?23:05
*** jpich has quit IRC23:05
sdaguelifeless: basically that d-g change just means the job definition gets to decide23:05
*** jcooley_ has joined #openstack-infra23:05
anteaya*she asked hoping it isn't in January23:05
clarkbjeblair: I am looking at the workspace for that job now23:07
*** mattray has joined #openstack-infra23:07
*** jcooley_ has quit IRC23:08
clarkbgit diff --quiet exits with a 123:08
*** jcooley_ has joined #openstack-infra23:08
clarkb! git diff --quiet returns 0. job script runs in bash23:08
openstackclarkb: Error: "git" is not a valid command.23:08
clarkbsorry openstack23:09
sdaguelifeless: so if that revises your -1 on this - https://review.openstack.org/#/c/59303/23:10
clarkbI have no idea why that wouldn't work as expected. maybe git diff didn't return 1 for some reason at that point in time?23:10
sdaguewe can get someone else to land it23:10
sdagueclarkb: where is the sync code?23:10
*** jgrimm has quit IRC23:10
sdaguemaybe another set of eyes will see something23:10
clarkbsdague: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/propose_requirements_update.sh23:10
*** chuck__ has joined #openstack-infra23:11
*** yamahata_ has joined #openstack-infra23:12
*** jcooley_ has quit IRC23:12
anteayaclarkb: where does the projects.txt file reside, I want to check on neutron23:13
clarkbanteaya: if that is the file I think it is it is in the requirements repo23:14
anteayaclarkb: k, I will check23:14
*** denis_makogon has quit IRC23:15
*** pmathews has joined #openstack-infra23:16
anteayayes, funny neutron is listed and we were included on Dec. 9th for a change to stevedore but didn't get a patch auto-magically this time, morganfainberg gave us one23:16
*** Bada has joined #openstack-infra23:17
*** jcooley_ has joined #openstack-infra23:18
*** dstanek has quit IRC23:18
*** slong has quit IRC23:20
clarkbanteaya: looking at the job log neutron seems to have hit the same thing that nova hit. change is git review -d'd but git diff shows no diff23:20
anteayainteresting23:20
clarkbbut a manual git diff in the workspace shows a diff23:20
clarkbwe shouldn't need to do a sync between those changes and hte diff right?23:21
anteayado you just use `git review -d` manually?23:21
clarkbanteaya: no, the job did the git review -d23:21
*** dims has quit IRC23:21
* anteaya wonders what could be the difference in diff creation23:21
anteayawhat command do you run manually?23:22
clarkbgit diff23:22
*** jcooley_ has quit IRC23:22
clarkbI am in the workspace double checking that the requirements changes were applied in the workspace23:22
*** jcooley_ has joined #openstack-infra23:22
anteayais it a git review bug, mayhap?23:22
clarkbno, git review is putting the right change in place.23:22
anteayahmmm23:22
clarkbI think it is a script bug or a git weirdness23:22
morganfainbergclarkb, i think heat and keystone saw similar issues23:23
anteayayou could sell me on either23:23
morganfainbergactually, not sure if heat subscribes23:23
anteayamorganfainberg: http://git.openstack.org/cgit/openstack/requirements/tree/projects.txt#n6 they do23:23
morganfainberganteaya, aha so they do23:24
anteayamorganfainberg: I got an all hands on your patch, it is in the gate queue, thank you23:24
morganfainberganteaya, of course, happy to help23:24
morganfainberganteaya, i lurk in -infra for a reason ;)23:24
anteaya:D23:24
clarkbI wonder if we need to be more expicit and `git diff --quiet HEAD`23:25
morganfainbergclarkb, you'rer doing just a git diff --quiet?23:25
morganfainbergclarkb, i've seen some oddities if you're not explicit with jenkins workspaces.  not really sure why though23:25
sdagueclarkb: it should function as is, but that couldn't hurt23:26
clarkbmorganfainberg: yes, just git diff --quiet, but this is a self cloned repo23:26
morganfainbergsdague, agreed it should "work" but sometimes things just get odd w/o specification23:26
clarkbnot setup by any java jenkins plugins23:26
morganfainbergsdague, and i can't pinpoint why23:26
morganfainbergclarkb, hm. ok the plugins might be where i've seen the oddities before23:26
morganfainbergclarkb, i never hurts to be more specific though (imo)23:27
morganfainbergs/i/it23:27
sdagueclarkb: why not make the git diff non quiet23:27
*** HenryG has joined #openstack-infra23:27
clarkbsdague: because the return code is meaningless if you do that23:27
sdagueah23:27
clarkb--quiet implies --exit-code23:27
sdaguegit diff --exit-code ?23:28
morganfainbergclarkb, and git diff can't be called otherwise with --exit-code?23:28
sdaguethen you'd get the change in the output23:28
clarkbit can be but why bother at that point23:28
sdagueso we could see why it worked or not23:28
sdagueclarkb: mostly for debug23:28
clarkboh just to see the output, ya we can do that23:28
*** flaper87 is now known as flaper87|afk23:28
morganfainbergclarkb, ^ yep that23:28
sdaguethen in the trace logs we'll see what's up23:29
pmathewssdague: Can I get you to take a look at https://review.openstack.org/#/c/59875/23:29
*** jcooley_ has quit IRC23:30
openstackgerritClark Boylan proposed a change to openstack-infra/config: Be explicit and verbose when using git diff.  https://review.openstack.org/6126823:30
*** jcooley_ has joined #openstack-infra23:30
clarkbfungi: do you want to try another round of running the proposal job23:30
fungiyou bet23:31
fungijust caught up on scrollback since our call ended early for a change23:31
clarkbneed to get that merged and on the slave first, but it should give us better dta23:31
*** dkranz has quit IRC23:32
*** heyongli has joined #openstack-infra23:33
*** heyongli has quit IRC23:33
*** rnirmal has quit IRC23:33
sdagueoh, hey, the neutron grenade job is running (not working yet) but running23:33
anteaya\o/23:34
anteayathanks for helping EmilienM with his patch, he is very happy23:35
*** jcooley_ has quit IRC23:35
fungimanually tested some of the git diff return code behaviors just to be sure23:36
clarkbfungi: yeah I was playing with them in the workspace23:36
anteayafungi: what did you learn?23:36
clarkbI should get out of that workspace so that the cleaning goes without problems23:36
clarkbI am out now23:37
fungianteaya: learned that i have no idea why it's broken ;)23:37
openstackgerritA change was merged to openstack-infra/config: Be explicit and verbose when using git diff.  https://review.openstack.org/6126823:37
fungii'll drag that ^ through the puppet muck ahead of schedule23:37
*** jcooley_ has joined #openstack-infra23:37
anteayafungi: well that is something to know anyway23:38
*** melwitt has quit IRC23:39
pleia2are we done with bug day? going to update the stats23:39
*** mattray has quit IRC23:39
clarkbpleia2: I fizzled out on updating bugs with the requirements things :/23:40
fungipleia2: i feel like i need a do-over (or a bug make-up day)23:40
fungiso much broken23:40
pleia2hehe23:40
pleia2ok, we can close down for the day23:40
pleia2went from 213 open to 186, go us23:40
pmathewssdague: thanks!23:40
anteayapleia2: way to go, you23:41
anteayathanks for all that work23:41
anteayaand yay us23:41
SpamapShm23:42
SpamapSI thought <=1.2 fixed the sphinx problem23:42
SpamapShttps://review.openstack.org/#/c/61274/23:42
fungiclarkb: running for a few minutes now... https://jenkins.openstack.org/job/propose-requirements-updates/39/console23:43
fungiit's on nova, so we should be able to see something there23:43
SpamapSerr23:43
SpamapSn/m23:43
SpamapS<1.2, not <=1.223:43
SpamapSduh23:43
*** jcooley_ has quit IRC23:44
fungiclarkb: look at the full log and you can see a hit for cielo23:44
fungiceilo23:44
fungi-sphinx>=1.1.223:44
*** jcooley_ has joined #openstack-infra23:44
fungi+sphinx>=1.1.2,<1.223:44
anteayaand for neutron23:44
clarkbhttps://review.openstack.org/#/c/60974/ for neutron23:44
anteayayay23:44
*** jcooley_ has quit IRC23:44
anteayathanks23:44
clarkbfungi: I bet the non explicit thing to diff against was breaking it for some reason23:45
*** jcooley_ has joined #openstack-infra23:45
fungii wonder why though. doesn't make much sense23:45
pleia2and I tried to use the least creepy bug for my telling-the-world-about it post http://princessleia.com/journal/?p=882723:45
*** fbo_away is now known as fbo23:45
*** dstanek has joined #openstack-infra23:45
fungiisn't HEAD implicit when not explicit?23:45
*** chuck__ has quit IRC23:45
*** prad has quit IRC23:46
clarkbfungi: I think it is more complicated than that because it looks at non staged stuff23:46
fungianteaya: you want moths if you're going for historical accuracy. they were attracted to the glow tubes and would short out the leads23:46
pleia2then again, in a photo of 12 of them I think any bug is creepy23:46
clarkbpleia2: oh pictures of real bugs23:46
fungier, not anteaya, pleia223:46
openstackgerritJenkins proposed a change to openstack-dev/hacking: Updated from global requirements  https://review.openstack.org/6128623:46
pleia2fungi: indeed!23:46
*** rcleere has quit IRC23:46
clarkbI didn't think our bug discriptions were that creepy :)23:46
* fungi is talking to too many people at once23:46
pleia2clarkb: haha23:46
fungi(and in a channel with 199 of his closest friends)23:47
fungi(at least several of whom are bots)23:47
fungi(so sad)23:47
anteayaon your goggles?23:48
fungianteaya: nope. in my laboratory23:48
anteayaah23:48
fungiwith basement cat23:48
* anteaya pictures test tubes and steam23:48
clarkbfungi: I now have the theme song from dexter's lab stuck in my head23:48
fungiclarkb: i always have that song stuck in my head23:48
anteayabasement cat is a must have in a laboratory23:48
fungiclarkb: also, if you enjoy dester's lab, rick and morty is similar and hilarious23:49
fungiand not as much targeted and young children23:49
*** dstanek has quit IRC23:50
clarkbI keep meaning to watch invader zim23:50
fungiclarkb: if you share my tastes in comedy and cartoons, you will like zim23:50
fungii have the boxed set. they needed to not get cancelled. oh well23:50
*** jhesketh_ has quit IRC23:51
clarkbalso adventure time. I enjoy them but typically not in a way that I can marathon them. More when bored ntohing to do watch cartoons because that is what I did long ago and it worked well :)23:51
StevenKWhere is the code that comments on LP bugs? It still uses github URLs23:52
fungiadventure time is not too bad. i've found it amusing but i have only seen a few23:52
fungiStevenK: ooh! good catch! it's in openstack-infra/jeepyb23:52
fungichecking for that now23:52
*** michchap_ has joined #openstack-infra23:53
openstackgerritChangBo Guo proposed a change to openstack-dev/pbr: Pin sphinx to <1.2  https://review.openstack.org/6128823:53
*** jcooley_ has quit IRC23:53
anteayapleia2: nice ladybugs23:53
*** jcooley_ has joined #openstack-infra23:54
pleia2anteaya: thank you23:54
fungiStevenK: found, fixing23:54
*** dstanek has joined #openstack-infra23:55
anteayapleia2: I got your holiday card today, very pretty23:55
pleia2anteaya: yay!23:55
anteaya:D23:55
*** michchap has quit IRC23:57
*** jcooley_ has quit IRC23:58
openstackgerritJeremy Stanley proposed a change to openstack-infra/jeepyb: Link git.o.o in bug updates  https://review.openstack.org/6129023:58
fungiStevenK: ^23:58
*** jcooley_ has joined #openstack-infra23:58
*** mrodden has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!