Wednesday, 2016-06-15

*** piet_ has quit IRC00:00
*** kzaitsev_mb has quit IRC00:01
*** kzaitsev_mb has joined #openstack-infra00:03
*** jamesmcarthur has joined #openstack-infra00:06
*** thorst has joined #openstack-infra00:06
*** zz_dimtruck is now known as dimtruck00:07
*** xarses has joined #openstack-infra00:10
openstackgerritMerged openstack-infra/system-config: Upgrade infracloud modules to mitaka
*** jamesmcarthur has quit IRC00:10
*** thorst has quit IRC00:10
*** yamahata has quit IRC00:14
openstackgerritMerged openstack-infra/puppet-infracloud: Upgrade to Mitaka
*** rcernin has joined #openstack-infra00:17
*** ddieterly has joined #openstack-infra00:17
*** sdake has quit IRC00:18
*** bhavik1 has joined #openstack-infra00:18
*** bhavik has quit IRC00:21
*** bhavik1 is now known as bhavik00:21
*** rcernin has quit IRC00:22
*** sarob has quit IRC00:30
clarkblifeless: have you seen it is on the general ml not sure if yo usubscribe00:31
lifelessclarkb: I put some filters in place - ~= unsubscribed00:32
*** david-lyle has quit IRC00:32
lifelessclarkb: that version support was removed from pbr in uhm 0.11 ?00:32
lifelessclarkb: when PEP-440 because enforced by pip/pypi00:32
lifelessclarkb: pbr puts the hash in the pbr.json00:32
*** tqtran has quit IRC00:33
*** bhavik has quit IRC00:33
*** sputnik13 has quit IRC00:34
*** ddieterly has quit IRC00:35
clarkbyes we know and that was explained00:35
clarkbbut supposedly pep440 supports +things suffixes and pbr doesn't? I haven't paged all that in00:35
lifelessclarkb: we used + for prefix versions during the period where 0.11 wasn't being used by everyone and point releases of 0.10 had to adjust to PEP-440 things00:36
lifelessclarkb: or maybe that wasn't it.00:36
lifelessclarkb: anyhow, I do want to support + local versions, but really want to make sure its tied into nova's custom version overrides00:37
lifelessclarkb: because we only get one shot at doing it.00:37
lifelessclarkb: I've asked repeatedly for someone interested to write a spec :)00:37
clarkbnova's custom version overrides?00:37
*** lvdongbing has joined #openstack-infra00:37
lifelessclarkb: its a thing which allows changing the version nova reports by setting a config setting00:39
lifelessclarkb: which distro's apparently use to make their openstacks report that they are from them00:39
lifelessclarkb: yes00:39
*** tkelsey has joined #openstack-infra00:41
*** dimtruck is now known as zz_dimtruck00:41
*** mtanino has quit IRC00:41
clarkbon rereading pep440 there is support for local version specifiers but they are not public00:42
lifelessso its fine for local builds00:42
clarkbmaybe pbr should accept them if the version is set directly as in that thread but not generate them automagically00:43
lifelessnot for redhat or ubuntu or whatever00:43
lifelessfundamentally though, its gnarly, we need to have documentation and use cases and clearly articulate how these things interact - or not00:43
lifelessI don't want to mess around with backwards compat because we haven't actually had the discussion - which is what I want from a spec here00:44
lifelesssomething we can walk around all the distros00:44
lifelessand operators00:44
*** Sukhdev has joined #openstack-infra00:44
*** Sukhdev has quit IRC00:45
*** tkelsey has quit IRC00:45
*** amitgandhinz has joined #openstack-infra00:45
*** csomerville has quit IRC00:46
dstufftlifeless: clarkb local versions are fine for Red Hat and such too, it's generally for "things which are not upstream to indicate they're not really version X", without it you're stuck either pretending upstream "1.0" and downstream/local "1.0" are the same thing (which they aren't) or doing something like "" and hoping upstream never releases a version of that00:47
*** csomerville has joined #openstack-infra00:47
dstufftconceptually similar to -ubuntu1 or w/e in an ubuntu version00:47
*** zul_ has joined #openstack-infra00:47
lifelessdstufft: well, fine if they don't put them in pypi00:48
lifelessdstufft: so context is pretty important00:48
dstufft(PyPI won't allow you to upload a +version)00:48
dstufftyea that's true00:48
*** rcernin has joined #openstack-infra00:48
lifelessdstufft: and thus why I want that broad discussion, rather than 3-4 people deciding without knowing what people do with e.g. the nova version override00:48
*** roxanagh_ has joined #openstack-infra00:49
dstufftI was assuming Redhat in the "redhat took something from PyPI and shipped it as an rpm, maybe with some patches"00:49
dstufftlifeless: yea, reasonable.00:49
lifelessI do try ;)00:49
*** csomerville has quit IRC00:50
*** amitgandhinz has quit IRC00:50
*** thorst has joined #openstack-infra00:53
*** roxanagh_ has quit IRC00:53
*** haaj has quit IRC00:53
*** rcernin has quit IRC00:53
clarkblifeless: any chance you are able to respond to that thread with a summary of this "need spec, nova does funny things"00:53
lifelessclarkb: oh, there's a whole thread? Let me read through00:54
*** weshay_mtg has quit IRC00:55
*** bpokorny_ has joined #openstack-infra00:57
*** ddieterly has joined #openstack-infra00:59
*** bpokorny has quit IRC01:00
*** bpokorny_ has quit IRC01:01
lifelessclarkb: no, I cannot respond01:04
lifelessYou have to be a subscriber to post to this mailing list, so your01:04
lifelessmessage has been automatically rejected. You can subscribe at:01:04
*** zhurong has joined #openstack-infra01:06
*** rcernin has joined #openstack-infra01:06
*** weshay_mtg has joined #openstack-infra01:07
*** pvaneck has quit IRC01:08
*** rcernin has quit IRC01:10
*** bhavik has joined #openstack-infra01:12
*** claudiub has quit IRC01:14
*** ddieterly has quit IRC01:15
*** kzaitsev_mb has quit IRC01:16
lifelessclarkb: I have replied on in more etail though01:17
*** Daisy has joined #openstack-infra01:22
*** EricGonczer_ has joined #openstack-infra01:22
*** kushal has quit IRC01:25
*** Apoorva has quit IRC01:25
*** sdake has joined #openstack-infra01:28
*** Apoorva has joined #openstack-infra01:28
*** Apoorva has quit IRC01:28
*** tqtran has joined #openstack-infra01:30
*** tphummel has quit IRC01:32
*** sdake has quit IRC01:32
clarkbjhesketh: I am looking at because github emailed me and I am01:33
clarkbconfu At first I wondered if we started01:33
clarkbreviewing gerrit changes in github but then trying to view that change on gerrit doesn't show anything01:33
clarkbwow fail copy paste01:33
clarkblet me try again01:33
clarkb is the thing01:34
*** tqtran has quit IRC01:34
jheskethclarkb: tonyb and I are doing a thing where we fail in private before we push it up to gerrit to fail more publicly01:34
jheskethclarkb: in other words, it's like pre-review :p01:34
jheskethinteresting that it pinged you01:34
clarkbbut how do you get the thing on openstack/diskimage-builder?01:35
clarkbjhesketh: it pinged me because ^ that repo was updated01:35
jheskeththat part is interesting, I just noticed that too01:35
jheskethclarkb: so we should have been working on
jheskethbut somehow through the github ui the comments made it onto openstack/diskimage-builder01:36
*** thorst has quit IRC01:36
notmorganclarkb: had a nice chat with lifeless, going to hopefully have more fixes for capturing logs from zuul (etc) soon. *and* do some "threads didn't die nicely" warnings so we can fix the "threads dont exit right" problem we have [cc mordred jeblair ]01:36
clarkbjhesketh: well the commit made it there too01:36
notmorganalso jhesketh o/ Mornin :)01:37
clarkbjhesketh: which I think is my biggest concern, unless github is being really weird01:37
jheskethwhy it exists there is a weird githubism01:37
clarkbjhesketh: so that commit isn't actually pushed into the repo?01:37
jheskethclarkb: it's not in the repo, no01:37
jheskethwell not that I can see having just checked01:37
*** thorst has joined #openstack-infra01:37
jheskethnotmorgan: o/01:37
clarkbgit fetch confirms, /me adds this to additional thing to dislike about github01:39
notmorganclarkb: there are so many these days....01:39
notmorganclarkb: i stoped "tracking" all of mine.01:40
clarkbI don't think I would care if it didn't mean I got emailed01:40
clarkbbut since I do get emailed about it ...01:40
notmorganclarkb: fair enough01:40
jheskethclarkb: so you can view the commit on any fork it seems (eg - even when it's behind master01:41
*** baoli has joined #openstack-infra01:41
jheskethsomehow I browsed onto the commit in the openstack repo while doing a diff between branches01:41
clarkband do all fork owners get emails too  Iwonder01:41
*** wc2912 has quit IRC01:41
jheskethand inadvertently commented there sorry :-s01:41
clarkbah ok01:41
* clarkb understands now01:41
mordredyah. that makes sense01:41
mordredother than being able to do it01:41
mordredwhich not so much with the making sense01:42
notmorganmordred: lol01:42
clarkbexcept for the part where refs not in a repo show up in that repo01:42
mordredthat part not so much01:42
jheskethclarkb: I suspect it's just the particular fork that gets notified01:42
notmorganclarkb: welcome to github. it's magical! :P01:42
mordredbut given that - jhesketh commenting there emailing you does make sense01:42
clarkbI wonder if this si exploitable01:42
mordredclarkb: probably in some manner01:42
clarkbto see commits in an eg private fork of a repo01:42
notmorganclarkb: you can't private fork anymore01:42
mordredoh. that's an interesting question01:42
greghaynesclarkb: I was certainly wondering why we merged a WIP patvh01:43
greghaynesPatch, even01:43
mordrednotmorgan: no?01:43
notmorganyou have to fork and then reupload into an unconnected repo.01:43
jheskethnotmorgan: really? I think I did last night... but it was private in my org too01:43
clarkbnotmorgan: maybe this is why :)01:43
notmorganmordred: nope. private/not-private is dictated by the original repo01:43
notmorganmordred: and if you delete the original private repo the private forks go away iirc.01:43
clarkbprobably every repo family has a single repo on their backend and forks are just views onto  arepo01:43
jheskethclarkb: I can't access that commit sha on a not diskimage-builder repo fwiw01:43
lifelessmordred: you didn't think github stored a copy of the repo for each fork did you?01:43
notmorganmordred: it used to be you could make a fork private etc. but that was exploitable iirc.01:44
clarkblifeless: no but I didn't expect them to leak commits across them in the ui01:44
notmorganand was just a terrible ui.01:44
mordredlifeless: yeah. what clarkb said01:44
lifelessclarkb: PR's do this somewhat as a feature I think01:44
notmorganlifeless: ++01:44
lifelessbut yeah, I fully expect its just a unique set of refs01:44
notmorganit's to help make PRs easier to push around.01:44
notmorganor somesuch01:45
lifelessthe git protocol doesn't permit access to things that are not reachable from a ref01:45
* notmorgan goes back to not liking PR workflows.01:45
*** furlongm_ is now known as furlongm01:45
*** thorst has quit IRC01:46
prometheanfireianw: well, found a bindep error, which is nice01:46
prometheanfireianw: also, I think you mentioned that we might be able to proceed if I got to the chown piece?01:47
prometheanfirechown: invalid user: 'jenkins:jenkins'01:47
*** amitgandhinz has joined #openstack-infra01:47
*** kushal has joined #openstack-infra01:48
tonybclarkb: Sorry I had no idea that would happen01:48
clarkbI was mostly just worried that somehow commits were going into github tat weren't in gerrit, but apparently its just github buggyness01:49
anteayatonyb: jhesketh and you thought you could fail in private01:49
notmorgananteaya: colour update: you're a lovely shade of green again.01:50
anteayanotmorgan: oh how nice01:50
anteayanotmorgan: thank you01:50
anteayanotmorgan: can I see?01:50
tonybanteaya: pretty much01:50
anteayatonyb: ha ha ha01:50
jheskethanteaya: welcome to our public art performance where we face-plant01:50
anteayajhesketh: you do it so well01:50
jheskethand often!01:50
anteayaand with such a nice finish01:50
anteayajhesketh: you really stuck the landing there01:50
anteayaI'm expecting high points from the judges01:51
*** amitgandhinz has quit IRC01:52
*** haaj has joined #openstack-infra01:54
anteayanotmorgan: thank you01:54
anteayaI do like that green, thank you01:54
*** links has joined #openstack-infra01:54
prometheanfiretonyb: or anyone that can +2 bindep stuff...
anteayawould jhesketh be taupe, I wonder?01:55
tonybprometheanfire: not me :(01:55
notmorganprometheanfire: hehe glad you approve of my change ;)01:55
jheskethanteaya: that's a very specific colour I just had to google to figure out what it was01:55
* jhesketh is not a smart man01:56
anteayajhesketh: I looked too, the colour of french moles01:56
prometheanfirenotmorgan: ya, needed to be able to reproduce the error I got locally, it was pointing to another line oddly01:56
anteayajhesketh: sure you are, and bonus is you are a kind person too, like Forrest01:56
notmorganprometheanfire: i think bindep core is lifeless and uhm infra-core01:56
anteayajhesketh: wouldn't you say taupe there:
prometheanfirenotmorgan: ya :(01:57
anteayajhesketh: now on mine you are a lovely blue01:57
*** ddieterly has joined #openstack-infra01:57
*** ddieterly is now known as ddieterly[away]01:57
*** ddieterly[away] is now known as ddieterly01:57
notmorganprometheanfire: well i might have been on a py3 "fix the things" kick lately...01:58
prometheanfirenotmorgan: I'm happy, because I needs that01:59
*** haaj has quit IRC02:00
lifelessnotmorgan: prometheanfire: I started bindep, but really, its infra-core now. I'll review if someone pings me for one02:03
prometheanfireianw: I've tested and left a note as to the results, I think it's ready to move forward02:03
notmorganlifeless: i did mox->mock patch and ~2 lines to fix py3 issues02:03
prometheanfirethe jenkins user is set up via puppet right?02:03
prometheanfireand group02:04
notmorganlifeless: you're welcome to review it, or it can hold until infra-core has bandwidth02:04
* notmorgan isn't in a huge rush there.02:04
prometheanfireError: Evaluation Error: Error while evaluating a Function Call, Could not find class ::ntp for gentoo-dib.openstacklocal at /opt/build_git/openstack-infra/system-config/modules/openstack_project/manifests/template.pp:29:3 on node gentoo-dib.openstacklocal02:06
ianwprometheanfire: yes02:07
ianwprometheanfire: what i've found helpful when working on similar stuff is having the good logs to compare to from http://nodepool.openstack.org02:07
prometheanfireianw: looks like next step is fixing puppet modules?02:07
ianwprometheanfire: yep ... that's kind of what i predicted wouldn't work :)02:07
prometheanfireyep, but good to know it got that far02:08
*** gongysh has quit IRC02:08
prometheanfireadding a new distro is becoming a neverending rabbit hole02:08
ianwprometheanfire: if it was easy, everyone would do it :)02:08
prometheanfireand that'd be bad?02:08
ianwno, i mean in general making things more portable and flexible makes things better02:09
prometheanfireis there a quick way I can run the puppet stuff without waiting an hour each run?02:09
ianwbut it's often hard work02:10
ianwyou can just run the puppet by hand, start with the system-config script02:10
prometheanfirethat already works02:10
ianwthat brings in the modules, then you can run the puppet apply slave by hand02:10
prometheanfireI fixed that a month ago :P02:11
openstackgerritChaoyi Huang proposed openstack-infra/project-config: [Tricircle]add functional test job
prometheanfireguess I'll set up a dib image just without puppet and work from there02:12
*** ddieterly has quit IRC02:12
*** Daisy has quit IRC02:13
prometheanfireianw: cache-bindep didn't work because it's not python3 yet though, but that'll be fixed in review (poke poke, nudge nudge)02:13
*** Daisy has joined #openstack-infra02:14
*** gongysh has joined #openstack-infra02:16
*** zz_dimtruck is now known as dimtruck02:17
*** Daisy has quit IRC02:18
*** bhavik has quit IRC02:19
*** ddieterly has joined #openstack-infra02:20
*** PollyZ has joined #openstack-infra02:20
*** docaedo2 has left #openstack-infra02:21
*** PollyZ has quit IRC02:25
*** thorst has joined #openstack-infra02:28
*** weshay_mtg has quit IRC02:28
*** thorst has quit IRC02:28
*** thorst has joined #openstack-infra02:29
*** baoli has quit IRC02:32
*** ddieterly has quit IRC02:35
*** ddieterly has joined #openstack-infra02:35
*** ddieterly has quit IRC02:35
*** tphummel has joined #openstack-infra02:35
eliqiaohi, can anyone help to check . I need some help on irc renaming.02:36
*** thorst has quit IRC02:37
*** r-mibu has quit IRC02:37
*** julim has joined #openstack-infra02:38
*** kzaitsev_mb has joined #openstack-infra02:41
*** PollyZ has joined #openstack-infra02:42
*** zul_ has quit IRC02:43
*** grassass has quit IRC02:47
*** r-mibu has joined #openstack-infra02:47
*** amitgandhinz has joined #openstack-infra02:48
*** yuanying has quit IRC02:50
*** roxanaghe has joined #openstack-infra02:50
*** EricGonczer_ has quit IRC02:51
*** amitgandhinz has quit IRC02:53
*** yuanying has joined #openstack-infra02:53
*** gongysh has quit IRC02:53
*** Daisy has joined #openstack-infra02:53
*** yanyanhu has joined #openstack-infra02:54
*** PollyZ has quit IRC02:55
*** roxanaghe has quit IRC02:55
*** PollyZ has joined #openstack-infra02:55
*** haaj has joined #openstack-infra02:56
*** zhurong has quit IRC02:56
*** zhurong has joined #openstack-infra02:56
*** links has quit IRC02:56
*** PollyZ has quit IRC02:58
*** gyee has quit IRC02:59
*** elynn has joined #openstack-infra03:00
openstackgerritDi XiaoLi proposed openstack-infra/project-config: Add a heat job to use devstack plugin
*** haaj has quit IRC03:01
*** ramishra has joined #openstack-infra03:02
*** Daisy has quit IRC03:05
*** Daisy has joined #openstack-infra03:05
*** Daisy_ has joined #openstack-infra03:08
*** Daisy has quit IRC03:10
*** tphummel has quit IRC03:12
*** hongbin has joined #openstack-infra03:13
*** asselin_ has joined #openstack-infra03:13
*** cody-somerville has joined #openstack-infra03:13
*** grassass has joined #openstack-infra03:14
*** sflanigan has quit IRC03:15
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: WIP: Add web-based console log streaming
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: Python3 Fix: tools use six.moves.urllib
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: Bundled Fixes
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: Python 3 fix: GC final state inspection
*** kzaitsev_mb has quit IRC03:24
*** cody-somerville has quit IRC03:25
*** erlon has quit IRC03:26
*** sarob has joined #openstack-infra03:28
*** PollyZ has joined #openstack-infra03:30
*** tqtran has joined #openstack-infra03:31
*** yamada-h has joined #openstack-infra03:32
*** sarob has quit IRC03:35
*** thorst has joined #openstack-infra03:35
*** tqtran has quit IRC03:36
*** sflanigan has joined #openstack-infra03:36
*** yamada-h has quit IRC03:37
*** ramishra has quit IRC03:38
*** PollyZ has quit IRC03:41
*** tkelsey has joined #openstack-infra03:42
*** thorst has quit IRC03:43
*** Daisy_ has quit IRC03:43
*** Daisy has joined #openstack-infra03:44
*** tkelsey has quit IRC03:47
*** hongbin has quit IRC03:47
*** yuanying_ has joined #openstack-infra03:48
*** Daisy has quit IRC03:48
*** amitgandhinz has joined #openstack-infra03:49
*** binbincong has quit IRC03:49
*** yuanying has quit IRC03:49
*** Sukhdev has joined #openstack-infra03:50
*** amitgandhinz has quit IRC03:53
openstackgerritwatanabe isao proposed openstack-infra/zuul: WIP: Improve zuul proxy feature
*** haaj has joined #openstack-infra03:57
*** yamada-h has joined #openstack-infra03:57
*** ramishra has joined #openstack-infra03:57
*** njohnston|afk is now known as njohnston03:59
*** dimtruck is now known as zz_dimtruck04:00
*** Wei_Liu has quit IRC04:02
*** haaj has quit IRC04:02
*** Wei_Liu has joined #openstack-infra04:03
*** ramishra has quit IRC04:03
*** julim has quit IRC04:04
*** links has joined #openstack-infra04:05
*** binbincong has joined #openstack-infra04:06
*** david-lyle has joined #openstack-infra04:06
*** Daisy has joined #openstack-infra04:09
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: Capture Logs when Timeout is hit
*** rkukura has quit IRC04:10
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: DNM: Synthetic Failure to show proper log capture
*** rkukura has joined #openstack-infra04:12
notmorganmordred, jeblair, clarkb: even better form, this now captures all the data "correctly". Thanks lifeless! Next step will be doing thread introspection and killing threads / checking daemon status. so we can move to a proper "gentle" form.04:12
*** furlongm has quit IRC04:13
*** furlongm has joined #openstack-infra04:13
*** ramishra has joined #openstack-infra04:13
mordrednotmorgan: looks good04:16
notmorganmordred: it's way better than just dumping to stderr. i'm learning a lot about subunit :P04:17
* notmorgan was hesitant to say that too loudly as people might start asking questions.04:17
notmorganoh i should probably add the logger things to it so we see that it was actually a timeout that occured.04:18
notmorganthe ascii art was lost :(04:19
*** tphummel has joined #openstack-infra04:19
notmorganmordred: how important is the ascii art saying it was actually a timeout (or an explicit log) vs just a failure?04:19
*** PollyZ has joined #openstack-infra04:19
*** kzaitsev_mb has joined #openstack-infra04:21
notmorganmordred: bah. and it still didn't show up in the testrepository stuff in the gate...04:21
* notmorgan goes and glares some more.04:21
mordrednotmorgan: you are now officially the subunit expert. congrats04:22
lifelessnotmorgan: it probably wants to write to stdout, not err04:24
lifelessnotmorgan: since testr passes err straight to the user.04:24
notmorganlifeless: ahh ok.04:24
notmorganlifeless: i'll circle back on that. interestingly it was captured locally under tox, must be some wonkyness04:25
notmorganlifeless: so do i need to write to both stdout and err in this case?04:25
notmorganlifeless: or is stdout going to still hit the console as well?04:25
*** kuromagi has quit IRC04:25
* notmorgan can actually test that but ... has a local lifeless to ask :P04:25
*** kuromagi has joined #openstack-infra04:25
* notmorgan runs to get food.04:26
lifelessnotmorgan: well, tox mangles stderr to stdout04:26
*** oanson has joined #openstack-infra04:28
notmorgani'll poke at this so much more once i get back from some dinner04:29
* notmorgan also realizes a nice lamp for this new desk is a must04:29
*** gouthamr has quit IRC04:34
*** elynn has quit IRC04:34
*** _nadya_ has joined #openstack-infra04:35
*** PollyZ has quit IRC04:35
*** Daisy has quit IRC04:39
*** psachin has joined #openstack-infra04:41
*** thorst has joined #openstack-infra04:42
*** thorst has quit IRC04:48
*** mrmartin has joined #openstack-infra04:49
*** amitgandhinz has joined #openstack-infra04:50
*** ramishra has quit IRC04:50
*** ramishra has joined #openstack-infra04:50
*** roxanaghe has joined #openstack-infra04:51
*** rkukura has quit IRC04:52
*** rkukura has joined #openstack-infra04:54
*** amitgandhinz has quit IRC04:54
*** kzaitsev_mb has quit IRC04:54
*** salv-orl_ has quit IRC04:54
*** roxanaghe has quit IRC04:54
*** salv-orlando has joined #openstack-infra04:54
*** ramishra has quit IRC04:54
*** roxanaghe has joined #openstack-infra04:55
*** elynn has joined #openstack-infra04:56
openstackgerritIan Wienand proposed openstack/diskimage-builder: Clear up "already provided" message
openstackgerritIan Wienand proposed openstack/diskimage-builder: Convert element_dependencies to logging
openstackgerritYanyan Hu proposed openstack-infra/project-config: Revise gerritbot config for senlin channel
*** haaj has joined #openstack-infra04:58
*** njohnston is now known as njohnston|sleep04:59
*** ramishra has joined #openstack-infra05:00
openstackgerritYanyan Hu proposed openstack-infra/project-config: Revise gerritbot config of senlin channel
*** elynn has quit IRC05:00
*** elynn has joined #openstack-infra05:01
*** yamamot__ has joined #openstack-infra05:01
*** haaj has quit IRC05:04
*** jaosorior has joined #openstack-infra05:04
*** Daisy has joined #openstack-infra05:05
*** Daisy has quit IRC05:05
*** Daisy has joined #openstack-infra05:05
*** pilgrimstack has joined #openstack-infra05:11
*** pilgrimstack has quit IRC05:13
*** salv-orlando has quit IRC05:17
*** salv-orlando has joined #openstack-infra05:17
*** ramishra has quit IRC05:18
*** sree has joined #openstack-infra05:18
*** mrmartin has quit IRC05:21
*** _nadya_ has quit IRC05:25
*** salv-orl_ has joined #openstack-infra05:29
*** sarob has joined #openstack-infra05:31
*** salv-orlando has quit IRC05:32
gmannAJaeger:  on LP, is there any way to open the bug which is Fix_released ?05:34
*** sarob has quit IRC05:36
*** gongysh has joined #openstack-infra05:36
AJaegergmann: I think you can just reopen it - but never tried...05:38
*** elynn has left #openstack-infra05:38
gmannAJaeger: i tried but all option are disabled seems05:39
gmanntried to change the status05:39
AJaegergmann: then you don't have the right permissions ;) It works for me on those where I'm in the bug group05:39
gmannah i see.05:39
AJaegerstatus chane is only possible for those in the proper group - so ask one of the bug czars for the project05:40
*** gongysh has quit IRC05:40
reduxioGood morning. I'm having trouble setting up and testing a third party CI system. Is this the right place to ask about it?05:40
gmannAJaeger: sure, Thanks.05:40
*** abregman has joined #openstack-infra05:41
AJaegerreduxio: there's a third-party IRC channel and a meeting for that one.05:41
AJaegerreduxio: #openstack-third-party-ci05:42
reduxioAjaeger: Thanks!05:42
*** mrodden has quit IRC05:43
openstackgerritMerged openstack-infra/project-config: translations: Fix error abort
*** ramishra has joined #openstack-infra05:45
*** claudiub has joined #openstack-infra05:45
*** pgadiya has joined #openstack-infra05:46
*** thorst has joined #openstack-infra05:47
*** netsin has quit IRC05:47
*** netsin has joined #openstack-infra05:48
*** ramishra has quit IRC05:48
*** Wei_Liu has quit IRC05:49
*** Jeffrey4l has quit IRC05:49
*** Wei_Liu has joined #openstack-infra05:49
*** kzaitsev_mb has joined #openstack-infra05:50
*** amitgandhinz has joined #openstack-infra05:50
*** cody-somerville has joined #openstack-infra05:50
*** cody-somerville has joined #openstack-infra05:50
*** mrodden has joined #openstack-infra05:51
*** yamahata has joined #openstack-infra05:53
*** thorst has quit IRC05:53
*** Jeffrey4l has joined #openstack-infra05:54
openstackgerritYanyan Hu proposed openstack-infra/project-config: Remove two inactive channels from gerritbot list
*** garyk has joined #openstack-infra05:54
*** kzaitsev_mb has quit IRC05:55
*** amitgandhinz has quit IRC05:55
*** cody-somerville has quit IRC05:55
*** sarob has joined #openstack-infra05:57
openstackgerritMerged openstack-infra/project-config: Added new repos for fuel plugins
*** ramishra has joined #openstack-infra05:58
*** rlandy has quit IRC05:58
reduxioAJaeger: the #openstack-third-party-ci channel is quiet like a graveyard. Any other channels you can point me to?05:59
openstackgerritYanyan Hu proposed openstack-infra/project-config: Remove two inactive channels
*** haaj has joined #openstack-infra06:00
AJaegerreduxio: might be the wrong time of day for this. There's nobody here either right now to help AFAIK. Feel free to ask anteaya during American morning for advise.06:00
reduxioAJaeger: I see. Thank you.06:01
*** sarob has quit IRC06:02
*** yuanying has joined #openstack-infra06:03
*** ramishra has quit IRC06:03
*** yuanying_ has quit IRC06:05
*** haaj has quit IRC06:05
*** flwang has quit IRC06:05
snarwadehello, is there any way to write JJB if I have kubernetes plugin in jenkins06:06
openstackgerritYanyan Hu proposed openstack-infra/project-config: Remove two inactive channels
*** yolanda has joined #openstack-infra06:13
*** yolanda_ has joined #openstack-infra06:13
*** roxanaghe has quit IRC06:14
*** yolanda_ has quit IRC06:14
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml
*** ramishra has joined #openstack-infra06:16
yolandagood morning06:16
*** openstackgerrit has quit IRC06:18
*** openstackgerrit has joined #openstack-infra06:18
*** Daisy has quit IRC06:19
*** pilgrimstack has joined #openstack-infra06:19
*** Daisy has joined #openstack-infra06:19
*** andreas_s has joined #openstack-infra06:20
*** yuanying has quit IRC06:21
*** pilgrimstack has quit IRC06:21
*** rook has quit IRC06:22
*** tphummel has quit IRC06:23
*** salv-orlando has joined #openstack-infra06:24
notmorganyolanda: mornin!06:27
*** yuanying has joined #openstack-infra06:28
*** salv-orl_ has quit IRC06:28
*** salv-orlando has quit IRC06:29
*** rcernin has joined #openstack-infra06:30
*** flwang has joined #openstack-infra06:30
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: Capture Logs when Timeout is hit
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: DNM: Synthetic Failure to show proper log capture
*** aviau has quit IRC06:37
openstackgerritSachi King proposed openstack-dev/pbr: Restore warnerrors behavior
*** aviau has joined #openstack-infra06:39
*** tkelsey has joined #openstack-infra06:45
*** tkelsey has quit IRC06:49
*** ifarkas has joined #openstack-infra06:49
*** rook has joined #openstack-infra06:49
*** e0ne has joined #openstack-infra06:51
*** kzaitsev_mb has joined #openstack-infra06:51
*** amitgandhinz has joined #openstack-infra06:51
*** thorst has joined #openstack-infra06:51
notmorganlifeless: found a bug in subunit - in py27 seems to raise ValueError for not open for "read" instead of (_UnsupportedOperation, IOError)06:55
AJaegermorning, yolanda06:56
*** amitgandhinz has quit IRC06:56
*** kzaitsev_mb has quit IRC06:56
notmorganor... ... i dunno *why* this time it's raising a value error...06:56
*** thorst has quit IRC06:58
*** zhurong has quit IRC06:58
*** maestro has joined #openstack-infra06:59
*** amoralej|off is now known as amoralej06:59
*** kushal has quit IRC06:59
*** nmagnezi has joined #openstack-infra07:01
*** nmagnezi has quit IRC07:01
*** nmagnezi has joined #openstack-infra07:01
*** haaj has joined #openstack-infra07:01
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml
*** rossella_s has joined #openstack-infra07:02
*** tesseract has joined #openstack-infra07:03
*** rcernin has quit IRC07:04
*** Daisy_ has joined #openstack-infra07:04
*** rcernin has joined #openstack-infra07:04
*** yuanying has quit IRC07:05
*** maestro has left #openstack-infra07:05
*** netsin has quit IRC07:05
*** haaj has quit IRC07:07
*** Daisy_ has quit IRC07:07
*** Daisy_ has joined #openstack-infra07:07
*** flepied has quit IRC07:08
*** yuanying has joined #openstack-infra07:08
*** Daisy has quit IRC07:08
*** matrohon has joined #openstack-infra07:10
*** Daisy has joined #openstack-infra07:10
lifelessnotmorgan: how is that a bug?07:10
lifelessnotmorgan: a not-open stream isn't usable...07:10
*** zhurong has joined #openstack-infra07:11
*** acoles_ is now known as acoles07:13
*** dtardivel has joined #openstack-infra07:14
*** Daisy_ has quit IRC07:14
*** sree has quit IRC07:17
openstackgerritMerged openstack-infra/project-config: Remove two inactive channels
openstackgerritMerged openstack-infra/project-config: Revise gerritbot config of senlin channel
*** dhellmann has quit IRC07:20
*** sree has joined #openstack-infra07:20
*** dhellmann has joined #openstack-infra07:20
openstackgerritMerged openstack-infra/system-config: Add compute004.uswest metadata for better tracking
*** winggundamth has quit IRC07:22
*** javeriak_ has joined #openstack-infra07:23
*** shardy has joined #openstack-infra07:24
*** jlanoux has joined #openstack-infra07:25
*** vincentll has joined #openstack-infra07:27
openstackgerritMerged openstack-infra/project-config: zuul/layout: switch beaker/xenial jobs in gate
AJaegeryolanda: gerritbot was not accessing all channels, we have a limit of 120 - and after 329768 merged we're exactly at 120. So, let's see whether it works now before adding more - I blocked the charm addition.07:27
*** sree has quit IRC07:28
yolandaoh, didn't know about that limit. Why 120 is the number, cannot handle more?07:28
*** esikachev has joined #openstack-infra07:28
*** tlian has quit IRC07:28
*** Sukhdev has quit IRC07:29
AJaegeryolanda: we're not sure - there was some discussion last evening with fungi whether this is the problem.07:29
*** ikhudoshyn has joined #openstack-infra07:29
AJaegerSo, I suggest to see whether this is really the issue.07:30
AJaegeryolanda: yesterday discussion started around 14:01 UTC - for senlin07:30
*** yaume has joined #openstack-infra07:30
*** abregman_ has joined #openstack-infra07:31
ikhudoshynhi everybody! I'm about to add one more job for Rally. Kindly ask to take a look Thanks in advance.07:31
ikhudoshynAJaeger: I updated the patch ^^07:31
openstackgerritMerged openstack-infra/project-config: Remove gate-grenade-dsvm-partial-ironic
openstackgerritMerged openstack-infra/project-config: Add check-requirements check job for nova-lxd
*** yaume has quit IRC07:32
*** abregman has quit IRC07:32
*** openstackgerrit has quit IRC07:33
*** openstackgerrit has joined #openstack-infra07:33
*** dizquierdo has joined #openstack-infra07:36
*** amotoki has joined #openstack-infra07:37
*** Daisy has quit IRC07:37
yolandaAJaeger, looking at the conversation. So there is no confirmation of that 120 limit, right?07:37
*** Daisy has joined #openstack-infra07:37
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: Capture Logs when Timeout is hit
AJaegeryolanda: no confirmation - but I counted and we had 122 before we removed those 2 - so want to see whether it works now...07:39
*** links has quit IRC07:39
*** abregman_ is now known as abregman07:39
AJaegerwe can add charm later after if needed.07:39
AJaegerLet's see what fungi things...07:40
AJaegerI meant: later if we see that this fixes the problem...07:40
*** pgadiya is now known as pgadiya|lunch07:40
*** salv-orlando has joined #openstack-infra07:41
openstackgerritMorgan Fainberg proposed openstack-infra/zuul: DNM: Synthetic Failure to show proper log capture
openstackgerritwatanabe isao proposed openstack-infra/zuul: Add ssh timeout
yolandaAJaeger, ok07:42
*** Daisy has quit IRC07:42
*** shardy has quit IRC07:43
*** flepied has joined #openstack-infra07:43
*** flepied has quit IRC07:44
*** flepied has joined #openstack-infra07:44
openstackgerritIan Wienand proposed openstack/diskimage-builder: Clear up "already provided" message
openstackgerritIan Wienand proposed openstack/diskimage-builder: Convert element_dependencies to logging
openstackgerritwatanabe isao proposed openstack-infra/nodepool: Add ssh timeout
*** zeih has joined #openstack-infra07:50
*** dtantsur|afk is now known as dtantsur07:50
*** sree has joined #openstack-infra07:51
*** kzaitsev_mb has joined #openstack-infra07:52
*** amitgandhinz has joined #openstack-infra07:52
*** darvon has quit IRC07:53
*** Daisy has joined #openstack-infra07:53
*** jaosorior is now known as jaosorior_brb07:53
*** jtomasek_ has joined #openstack-infra07:53
*** thorst has joined #openstack-infra07:55
*** darvon has joined #openstack-infra07:55
*** links has joined #openstack-infra07:56
*** ccamacho has joined #openstack-infra07:56
*** amitgandhinz has quit IRC07:56
*** kzaitsev_mb has quit IRC07:57
*** zhs_s has joined #openstack-infra07:58
*** sarob has joined #openstack-infra07:58
*** zzzeek has quit IRC08:00
*** sree has quit IRC08:00
*** dizquierdo has quit IRC08:00
*** zeih has quit IRC08:01
*** zhs has quit IRC08:01
*** thorst has quit IRC08:02
*** sarob has quit IRC08:02
*** zeih has joined #openstack-infra08:02
*** zzzeek has joined #openstack-infra08:03
*** sree has joined #openstack-infra08:06
*** winggundamth has joined #openstack-infra08:07
openstackgerritHisashi Osanai proposed openstack-infra/project-config: Add jobs for Monasca-Analytics
*** jtomasek_ has quit IRC08:10
*** shardy has joined #openstack-infra08:10
*** sorantis has joined #openstack-infra08:10
*** sorantis has quit IRC08:10
*** eezhova has joined #openstack-infra08:11
*** sorantis has joined #openstack-infra08:12
*** kzaitsev_mb has joined #openstack-infra08:13
*** abregman has quit IRC08:15
*** dmk0202 has joined #openstack-infra08:22
*** yaume has joined #openstack-infra08:25
*** mhickey has joined #openstack-infra08:25
*** abregman has joined #openstack-infra08:28
openstackgerritTomoki Sekiyama proposed openstack-infra/project-config: Create new functional job for glance_store-cinder
*** yaume_ has joined #openstack-infra08:30
*** yaume has quit IRC08:30
openstackgerritwatanabe isao proposed openstack-infra/zuul: Add ssh timeout
*** jaosorior_brb has quit IRC08:40
*** jaosorior_brb has joined #openstack-infra08:40
*** jaosorior_brb is now known as jaosorior08:41
openstackgerritFerenc Horváth proposed openstack-infra/project-config: Add job which builds test images with watchdog support
*** sree has quit IRC08:44
*** hferenc has joined #openstack-infra08:44
*** sree has joined #openstack-infra08:45
*** derekh has joined #openstack-infra08:45
*** tqtran has joined #openstack-infra08:45
*** Daisy has quit IRC08:46
*** Daisy has joined #openstack-infra08:46
hferenchello all!08:46
*** Daisy_ has joined #openstack-infra08:47
*** Daisy_ has quit IRC08:47
*** Daisy_ has joined #openstack-infra08:48
*** tqtran has quit IRC08:50
*** sree has quit IRC08:50
*** linuxmaniac_ has joined #openstack-infra08:50
hferenci'd like to publish a test image to tarballs.openstack.org08:50
*** Daisy has quit IRC08:51
hferencbased on your previous advices i created a job which intends to do this08:51
*** ihrachys has joined #openstack-infra08:51
hferencbut i'm not sure that i found the right way at first08:52
*** amitgandhinz has joined #openstack-infra08:53
hferencso could you please give a glimpse at this:
*** tkelsey has joined #openstack-infra08:55
*** ramishra has quit IRC08:56
*** amitgandhinz has quit IRC08:57
*** jaosorior has quit IRC08:57
AJaegerhferenc: the triggered monthyl will not work AFAIK, we trigger with zuul, so you need to add the job to zuul/layout.yaml. I don't know whether we have setup for monthly jobs, you might need to discuss this again with the team here...08:59
*** snarwade has quit IRC08:59
*** electrofelix has joined #openstack-infra08:59
*** hashar has joined #openstack-infra09:00
*** hashar has left #openstack-infra09:00
*** thorst has joined #openstack-infra09:00
*** ramishra has joined #openstack-infra09:02
*** Daisy has joined #openstack-infra09:02
*** esikachev has quit IRC09:03
hferencAJaeger: thx! i was looking at the zuul triggers, but if i understand it correctly the least frequent is periodic, however i think that it is unnecessary to build this image that often09:05
*** pgadiya|lunch has quit IRC09:06
*** Daisy_ has quit IRC09:06
*** thorst has quit IRC09:07
*** jtomasek_ has joined #openstack-infra09:08
*** arxcruz has joined #openstack-infra09:08
hferencbecause it won't change a lot so i gave a try to @monthly09:09
*** jtomasek_ has quit IRC09:13
*** javeriak_ has quit IRC09:13
*** snarwade has joined #openstack-infra09:14
*** esikachev has joined #openstack-infra09:16
*** sambetts|afk is now known as sambetts09:16
*** yamada-h has quit IRC09:17
AJaegerhferenc: I don't think that we interpret the triggers word at all, you need to add it to a periodic-queue in zuul instead09:17
AJaegerhferenc: we only have the perodic pipeline that runs every day09:18
*** hdaniel has joined #openstack-infra09:19
hferencAJaeger: ok, i'll change the triggers in a sec09:19
*** jaosorior has joined #openstack-infra09:21
*** javeriak has joined #openstack-infra09:22
*** penguinolog has quit IRC09:24
*** mikelk has joined #openstack-infra09:26
*** javeriak has quit IRC09:26
*** Daisy has quit IRC09:26
*** penguinolog has joined #openstack-infra09:27
*** Daisy has joined #openstack-infra09:27
*** pgadiya|lunch has joined #openstack-infra09:27
*** esikachev has quit IRC09:28
*** esikachev has joined #openstack-infra09:29
*** Daisy has quit IRC09:31
*** markus__ has joined #openstack-infra09:33
*** kzaitsev_mb has quit IRC09:33
*** hferenc has quit IRC09:34
*** Na3iL has joined #openstack-infra09:36
*** hdaniel has quit IRC09:39
*** pilgrimstack has joined #openstack-infra09:40
*** ociuhandu has quit IRC09:42
*** M-docaedo_vector has quit IRC09:42
*** kushal has joined #openstack-infra09:43
eliqiaohi AJaeger09:43
eliqiaoAJaeger: could you give some advice on "irc channel renaming"09:44
*** eliqiao has left #openstack-infra09:46
*** eliqiao has joined #openstack-infra09:46
*** salv-orlando has quit IRC09:47
*** salv-orlando has joined #openstack-infra09:48
*** M-docaedo_vector has joined #openstack-infra09:49
*** cinerama has quit IRC09:51
*** dizquierdo has joined #openstack-infra09:51
*** pilgrimstack has quit IRC09:51
*** amitgandhinz has joined #openstack-infra09:54
*** cinerama has joined #openstack-infra09:55
*** amitgandhinz has quit IRC09:58
*** Na3iL has quit IRC09:58
*** Na3iL has joined #openstack-infra09:58
*** permalac has joined #openstack-infra09:58
*** sarob has joined #openstack-infra09:59
*** _nadya_ has joined #openstack-infra09:59
*** zhurong has quit IRC10:00
*** zeih has quit IRC10:00
AJaegereliqiao: only the link I gave - best talk to pleia2, she will be around at US west coast time10:01
*** ajmiller_ has quit IRC10:02
*** ajmiller has joined #openstack-infra10:02
*** sarob has quit IRC10:03
*** ajmiller_ has joined #openstack-infra10:04
*** ajmiller has quit IRC10:04
*** thorst has joined #openstack-infra10:06
*** mescanef has joined #openstack-infra10:06
*** M-docaedo_vector has quit IRC10:08
mescanefhello! please add to the following gerrit acl groups:,members;,members;,members;,members (ref.
*** pilgrimstack has joined #openstack-infra10:10
*** sorantis has quit IRC10:11
*** thorst has quit IRC10:12
*** markus__ has quit IRC10:13
*** javeriak has joined #openstack-infra10:13
*** pgadiya|lunch has quit IRC10:13
*** snarwade has quit IRC10:13
yolandahi mescanef i can help10:14
yanyanhuhi, AJaeger, yolanda, gerritbot has come back to senlin channel, thanks10:14
yanyanhuI guess removing idle channels works10:14
mescanefyolanda: :-)10:15
*** javeriak_ has joined #openstack-infra10:16
*** markus__ has joined #openstack-infra10:16
openstackgerritIan Wienand proposed openstack/diskimage-builder: [WIP] install pip/virtualenv packages
AJaegeryanyanhu: glad to hear.10:17
*** markus__ has quit IRC10:18
*** javeriak has quit IRC10:18
AJaegeryolanda, fungi: Could you check log files whether everything works - and then we could consider merging the charm request to check whether we have a missing channel again...10:18
*** HeOS has quit IRC10:19
yolandaAJaeger, i don't see errors on gerritbot10:20
mescanefyolanda: oh one group missed:,members10:20
yolandabut i think we better wait for fungi10:20
openstackgerritIan Wienand proposed openstack/diskimage-builder: Clear up "already provided" message
openstackgerritIan Wienand proposed openstack/diskimage-builder: Convert element_dependencies to logging
*** yanyanhu has quit IRC10:23
*** sorantis has joined #openstack-infra10:24
*** javeriak_ has quit IRC10:24
*** pgadiya|lunch has joined #openstack-infra10:26
*** markus__ has joined #openstack-infra10:27
*** kzaitsev_mb has joined #openstack-infra10:27
*** eezhova has quit IRC10:31
*** eezhova has joined #openstack-infra10:32
*** ociuhandu has joined #openstack-infra10:36
*** snarwade has joined #openstack-infra10:36
*** zeih has joined #openstack-infra10:37
*** hashar has joined #openstack-infra10:38
*** M-docaedo_vector has joined #openstack-infra10:40
*** markus__ has quit IRC10:41
*** markus__ has joined #openstack-infra10:43
*** shahid_ has joined #openstack-infra10:45
*** markus__ has quit IRC10:45
*** weshay has joined #openstack-infra10:51
*** YorikSar_ has quit IRC10:51
*** hdaniel has joined #openstack-infra10:51
*** pilgrimstack has quit IRC10:52
*** sdague has joined #openstack-infra10:52
*** markus__ has joined #openstack-infra10:53
*** lvdongbing has quit IRC10:53
*** amitgandhinz has joined #openstack-infra10:54
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/ansible-role-puppet: Log puppet output on Ansible control machine
*** zeih has quit IRC10:56
*** salv-orlando has quit IRC10:56
*** salv-orlando has joined #openstack-infra10:57
*** zeih has joined #openstack-infra10:57
*** YorikSar has joined #openstack-infra10:58
rcarrillocruznibalizer , clarkb : suggestions implemented on ansible-puppet role ^10:58
*** tlian has joined #openstack-infra10:59
*** amitgandhinz has quit IRC10:59
*** zeih_ has joined #openstack-infra11:00
*** hashar has quit IRC11:00
*** pradk has quit IRC11:01
*** zeih has quit IRC11:02
*** baoli has joined #openstack-infra11:05
*** salv-orlando has quit IRC11:06
*** baoli has quit IRC11:06
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Bump PuppetDB version to 4.1.0 to get ssl related params
*** thorst has joined #openstack-infra11:10
*** yamamot__ has quit IRC11:10
*** markus__ has quit IRC11:11
*** markus__ has joined #openstack-infra11:11
*** javeriak has joined #openstack-infra11:12
*** dprince has joined #openstack-infra11:14
*** xenogear has quit IRC11:15
openstackgerritMadhuri Kumari proposed openstack-infra/project-config: Add new project Zunclient to openstack repo
*** markus__ has quit IRC11:15
*** ccamacho is now known as ccamacho|lunch11:16
*** hparekh has quit IRC11:16
*** thorst has quit IRC11:17
*** javeriak has quit IRC11:18
*** hparekh has joined #openstack-infra11:19
*** hparekh_ has joined #openstack-infra11:19
*** hparekh_ has quit IRC11:20
*** pgadiya|lunch is now known as pgadiya11:20
*** hferenc has joined #openstack-infra11:28
rcarrillocruznibalizer , clarkb : as the role will log stuff on /var/log/ansible-puppet/<hostname> , we want to logrotate those11:33
rcarrillocruzi think that should be put in the role logic for more self-contained logic, let me know if you agree or prefer to set that on the puppet classes managing puppetmaster.openstack.org11:33
prometheanfireI'm trying to make an image for a slave, it should just be puppet apply slave from system config right?11:34
*** eezhova has quit IRC11:34
rcarrillocruzprometheanfire: what kind of slave, a jenkins slave or ... ?11:35
prometheanfirethis is for a nodepool image11:36
*** kushal has quit IRC11:37
rcarrillocruzthe images are built with dib (disk image builder)11:37
rcarrillocruzso, you want to check this:11:37
prometheanfireright, I'm in the process of adding a gentoo image to nodepool and am on the stage where puppet fails :P11:38
rcarrillocruzin there, you can see which elements is a ubuntu-trusty made of11:38
prometheanfireI've already done some work, I need to be able to run it manually so that I can submit reviews for the puppet modules themselves11:39
prometheanfireI know the ntp one failed, but I'd rather not wait an hour to test each time11:39
*** ldnunes has joined #openstack-infra11:39
*** lvdongbing has joined #openstack-infra11:39
prometheanfire(to run dib with the elements)11:40
prometheanfirelooks like it's here11:41
*** lvdongbing has quit IRC11:42
rcarrillocruzthen yeah11:42
rcarrillocruzget a gentoo vm11:43
rcarrillocruzbash -x install_puppet.sh11:43
rcarrillocruzbash -x install_modules.sh11:43
prometheanfireya, those two already complete, did that work a month ago, now it's module time11:43
rcarrillocruzthen something like 'sudo puppet apply --modulepath /etc/puppet/modules:whereveris/system-config -e "include ntp"'11:43
*** javeriak has joined #openstack-infra11:44
rcarrillocruzthat, or just create a dummy local.pp and pass on to puppet apply11:44
prometheanfirepuppet apply --detailed-exitcodes --color=false         --modulepath=/root/system-config/modules:/etc/puppetlabs/code/environments/production/modules11:44
rcarrillocruzi've seen ntp fail typically when the install_modules did not work11:44
rcarrillocruzor i wasn't passing the right --modulepath11:44
prometheanfirethis is puppet-agent, so the path differs slightly11:44
prometheanfireanyway, got the first error11:45
rcarrillocruzah, i believe does install it on /etc/puppet/modules11:45
prometheanfireError: Facter: error while resolving custom fact "system_python_version": Execution of '/usr/bin/eix --nocolor --pure-packages --format '<category> <name> [<installedversions:LASTVERSION>] [<bestversion:LASTVERSION>] <homepage> <description>11:45
prometheanfire' --exact --name python-pip' returned 1:11:45
openstackgerritMerged openstack-infra/irc-meetings: Correction to meeting ID for Common Flow Classifier meeting
prometheanfireMODULE_PATH=`puppet config print modulepath | cut -d ':' -f 1`11:45
*** ddieterly has joined #openstack-infra11:45
prometheanfirenope, it's whatever the version of puppet says it should be11:45
prometheanfirethat script should probably be updated to call that whenever it needs to edit the module path11:46
*** javeriak has quit IRC11:46
prometheanfirethat's the first line from install_modules11:46
*** zhurong has joined #openstack-infra11:46
*** thorst has joined #openstack-infra11:47
rcarrillocruzyeah, saw it11:48
*** yamamoto has joined #openstack-infra11:48
prometheanfirethis is puppet-4 as well (what puppet-agent is based off of11:49
*** amitgandhinz has joined #openstack-infra11:55
*** rhallisey has joined #openstack-infra11:57
openstackgerritMerged openstack-infra/release-tools: unset custom GREP_OPTIONS
*** javeriak has joined #openstack-infra11:58
prometheanfirenow I need to figure out how to tell puppet to use diferent package names :(11:59
*** jcoufal has joined #openstack-infra11:59
*** amitgandhinz has quit IRC11:59
*** sarob has joined #openstack-infra12:00
rcarrillocruztypical pattern is a selector12:00
*** prometheanfire has quit IRC12:01
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Implement Xenial/CentOS jobs for OpenStack-Ansible
*** yaume_ has quit IRC12:01
*** prometheanfire has joined #openstack-infra12:01
*** e0ne has quit IRC12:01
sdagueis there someone that can help with a docs cleanup?12:01
bkeroprometheanfire: likely there's already a package selector in params.pp for the modules12:02
sdaguethere is a bunch of nova dev docs that are all out of date that I'd like cleaned up off the cloudfiles server12:02
*** e0ne has joined #openstack-infra12:02
prometheanfirebkero: the python modules's facter stuff all calls python-pip, for python, pip and virtualenv12:02
*** javeriak has quit IRC12:02
prometheanfireI think that's wrong12:02
bkerofacter does?12:02
bkeroFacter core shouldn't, I wonder which facts are using that12:03
prometheanfireno, the python module's custom facter facts12:03
bkeroAh ok12:03
bkeroSounds like that needs some patches for extensibility then12:03
prometheanfireI'm trying to find the code now12:04
*** sarob has quit IRC12:04
prometheanfirefor instance though
prometheanfireoh, looks like we may be a few commits back12:05
prometheanfirequite a few12:06
*** ddieterly is now known as ddieterly[away]12:08
*** zhurong has quit IRC12:09
*** ccamacho|lunch is now known as ccamacho12:10
*** zhurong has joined #openstack-infra12:10
*** sree has joined #openstack-infra12:11
AJaegersdague: I can help12:12
*** aysyd has joined #openstack-infra12:12
*** erlon has joined #openstack-infra12:13
*** sree has quit IRC12:16
*** daemontool has joined #openstack-infra12:16
*** furlongm_ has joined #openstack-infra12:16
*** furlongm has quit IRC12:16
*** ramishra has quit IRC12:16
sdagueAJaeger: so shouldn't exist, that was deleted in Oct 201512:16
sdagueI wonder if we could do a full delete / regen of
*** ramishra has joined #openstack-infra12:18
*** baoli has joined #openstack-infra12:19
AJaegersdague: I can delete all old files in there (those that weren't published today;)12:19
AJaegerShould i deletel folder runnova from 2012 as well?12:19
*** baoli has quit IRC12:20
*** baoli has joined #openstack-infra12:20
*** eezhova has joined #openstack-infra12:21
AJaegersdague: v2 was last updated in January. that directory looks outdated..12:21
*** yaume has joined #openstack-infra12:21
*** hferenc_ has joined #openstack-infra12:21
AJaegersdague: If you want, I can do a delete of everything that's not current - with the exception of old releases/branches12:22
*** baoli_ has joined #openstack-infra12:22
*** shahid_ has quit IRC12:22
*** trown|outtypewww is now known as trown12:23
sdagueAJaeger: that would be great12:24
sdaguethe whole v2 directory I think was deleted in Oct12:24
AJaegersdague: ok, I start deletion now...12:25
sdaguebut yeh, anything that looks old, should go12:25
sdaguewe moved a lot around in the last year12:25
*** baoli has quit IRC12:25
sdague was the merge where it went away12:25
*** hferenc_ has quit IRC12:26
*** hferenc has quit IRC12:26
*** Goneri has joined #openstack-infra12:26
*** hferenc has joined #openstack-infra12:26
*** raildo-afk is now known as raildo12:26
*** eezhova has quit IRC12:27
AJaegersdague: yeah, a lot of moved... I'm deleted devref directory next...12:27
*** ddieterly[away] is now known as ddieterly12:27
*** winggundamth_ has joined #openstack-infra12:28
*** lvdongbing has joined #openstack-infra12:28
*** sree has joined #openstack-infra12:29
*** rlandy has joined #openstack-infra12:29
*** winggundamth_ has joined #openstack-infra12:29
openstackgerritFerenc Horváth proposed openstack-infra/project-config: Add job which builds test images with watchdog support
*** sarob has joined #openstack-infra12:32
*** eezhova has joined #openstack-infra12:32
*** ddieterly has quit IRC12:35
*** sarob has quit IRC12:36
AJaegersdague: all gone...12:38
*** psilvad has joined #openstack-infra12:40
*** edmondsw has joined #openstack-infra12:43
*** tqtran has joined #openstack-infra12:47
*** esikachev has quit IRC12:48
sdagueAJaeger: thanks12:49
AJaegersdague: you're welcome12:51
*** tqtran has quit IRC12:51
*** jsavak has joined #openstack-infra12:51
*** matt-borland has joined #openstack-infra12:52
*** _nadya_ has quit IRC12:53
*** HeOS has joined #openstack-infra12:54
*** _nadya_ has joined #openstack-infra12:54
*** salv-orlando has joined #openstack-infra12:56
*** amitgandhinz has joined #openstack-infra12:56
*** salv-orlando has quit IRC12:56
*** esikachev has joined #openstack-infra12:56
*** salv-orlando has joined #openstack-infra12:56
*** _nadya_ has quit IRC12:56
*** _nadya__ has joined #openstack-infra12:57
*** eharney has joined #openstack-infra12:58
*** rfolco has joined #openstack-infra12:58
*** yujunz has joined #openstack-infra12:58
*** julim has joined #openstack-infra12:59
*** markusry has joined #openstack-infra12:59
*** amitgandhinz has quit IRC13:00
*** eezhova has quit IRC13:00
*** mriedem_away is now known as mriedem13:01
*** yujunz has quit IRC13:01
*** woodster_ has joined #openstack-infra13:02
*** sorantis has quit IRC13:02
*** afred312 has quit IRC13:02
*** dprince has quit IRC13:02
*** inc0 has joined #openstack-infra13:03
*** psachin has quit IRC13:03
*** jcoufal has quit IRC13:04
*** EricGonczer_ has joined #openstack-infra13:05
anteayareduxio: you can ask your third party set up questions here13:06
*** eezhova has joined #openstack-infra13:06
*** amitgandhinz has joined #openstack-infra13:07
*** amitgandhinz has quit IRC13:08
*** amitgandhinz has joined #openstack-infra13:08
*** lvdongbing has quit IRC13:08
*** gouthamr has joined #openstack-infra13:09
*** bhavik1 has joined #openstack-infra13:10
*** sorantis has joined #openstack-infra13:11
*** nmagnezi has quit IRC13:12
*** mdrabe has joined #openstack-infra13:13
*** rcernin has quit IRC13:14
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE TESTING
*** EricGonczer_ has quit IRC13:15
*** ddieterly has joined #openstack-infra13:16
*** xyang1 has joined #openstack-infra13:16
*** amoralej is now known as amoralej|lunch13:16
*** psachin has joined #openstack-infra13:16
*** kgiusti has joined #openstack-infra13:16
*** kgiusti has quit IRC13:16
*** kgiusti has joined #openstack-infra13:16
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Update ironic-multitenant network vairable names
*** _nadya__ has quit IRC13:18
*** _nadya_ has joined #openstack-infra13:18
*** _nadya_ has quit IRC13:18
anteayamordred: Shrews the dox project is collecting stories in storyboard:!/story/200014813:19
openstackgerritAlina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job
anteayamordred: Shrews yet there is no dox repo in storyboard, from the path openstack/dox I don't intuit this is a repo that is part of infra13:20
anteayamordred: Shrews where should folks file dox bugs for the project at this time?13:20
*** ramishra has quit IRC13:20
*** YorikSar has quit IRC13:20
mordredanteaya: I don't think we're really doing anything with dox so much anymore13:21
openstackgerritAlina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job
mordredanteaya: but i'll defer to Shrews on that13:22
Shrewsmordred: i was going to defer to you. but yeah, it's pretty dead13:22
anteayamordred: okay would you like to help triage!/story/2000148 and!/story/200014713:22
*** YorikSar has joined #openstack-infra13:22
mordredflaper87: what about you?13:22
anteayathe storyboard folks are trying to clean things up and would appreciate some guidance here13:23
anteayanow if you want bugs on storyboard, that is fine, it can happen13:23
anteayabut what do you want here13:23
mordredI don't want bugs at all :)13:23
anteayajust saying the project is dead doesn't answer the where to file bugs question13:23
anteayamordred: can you close off those stories then please? with some guidance on how to storyboard triagers should address more dox bugs should they be filed13:24
mordredwell, I think that maybe we should push up a commit to git to retire the project. then close these stories with "sorry, we stopped workig on this"13:24
anteayamordred: that sounds reasonable to me13:24
anteayamordred: so I'll leave this with you, and storyboard folks can move on with their work?13:25
*** yamamoto has quit IRC13:25
anteayathank you13:26
openstackgerritHenrique Truta proposed openstack-infra/project-config: Run identity v3 only job in ceilometer
openstackgerritHenrique Truta proposed openstack-infra/project-config: Run identity v3 only job in heat
*** afred312 has joined #openstack-infra13:28
*** yamamoto has joined #openstack-infra13:28
*** akshai has joined #openstack-infra13:28
*** rcernin has joined #openstack-infra13:29
*** sdake has joined #openstack-infra13:29
*** eharney has quit IRC13:30
*** xenogear has joined #openstack-infra13:31
*** yamamoto has quit IRC13:32
*** yamamoto has joined #openstack-infra13:32
openstackgerritMaciej Relewicz proposed openstack-infra/project-config: New repo for murano plugin
*** jlanoux has quit IRC13:33
*** bhavik1 has quit IRC13:35
*** ddieterly is now known as ddieterly[away]13:39
*** jcoufal has joined #openstack-infra13:39
openstackgerritAlina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job
asselin_rcarrillocruz, hi, do you have some time to discuss infra-ansible? There's lots of files there I didn't expect to exist.13:40
asselin_rcarrillocruz, e.g. bootstrap gerrit, jjb:
*** jcoufal_ has joined #openstack-infra13:41
*** sdake_ has joined #openstack-infra13:42
openstackgerritMaciej Relewicz proposed openstack-infra/project-config: New repo for murano plugin
asselin_rcarrillocruz, also, with masterless puppet, why are there still so many refs to puppetmaster?13:42
openstackgerritAlina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job
rcarrillocruzbecause there's no puppetmaster-less in infra ansible yet13:43
*** jamesmcarthur has joined #openstack-infra13:43
openstackgerritMerged openstack-infra/jenkins-job-builder: Add support for testfairy plugin
asselin_ok, is that going to be the same repo? or a new one created?13:44
rcarrillocruzit will be in the same repo13:44
*** jcoufal has quit IRC13:45
*** sdake has quit IRC13:45
asselin_ok cool13:46
*** ddieterly[away] is now known as ddieterly13:46
*** yamamoto has quit IRC13:46
openstackgerritMateusz Matuszkowiak proposed openstack-infra/project-config: Updated two projects in gerrit/projects.yaml
*** PollyZ has joined #openstack-infra13:47
*** rodrigods has quit IRC13:47
*** rodrigods has joined #openstack-infra13:48
*** _ari_ has joined #openstack-infra13:48
*** yamamoto has joined #openstack-infra13:48
*** eharney has joined #openstack-infra13:49
*** Marquis42 has joined #openstack-infra13:49
*** zeih_ has quit IRC13:50
*** pgadiya has quit IRC13:51
*** pradk has joined #openstack-infra13:51
*** flepied1 has joined #openstack-infra13:51
rcarrillocruzasselin_: i've paused a bit, since i'm watching how zuul v3 is going13:52
rcarrillocruzcos that will need a bigger change13:52
rcarrillocruzsetting up launchers, getting rid of jenkins, what not13:52
openstackgerritPaul Belanger proposed openstack-infra/project-config: Fix upload-to-docs-site publisher
rcarrillocruzwe shall see13:52
pabelangerAJaeger: We'll need to properly test ^, as zuul-launcher exposed the issue13:52
asselin_rcarrillocruz, i thought it would just orchestrate, but use puppet to define the servers themselves13:52
*** yamamoto has quit IRC13:53
rcarrillocruzit does, but puppet modules doesn't do discovery or bootsrapping13:53
*** yamamoto has joined #openstack-infra13:53
*** flepied has quit IRC13:53
rcarrillocruzi.e. you run 'puppet apply zuul'13:53
rcarrillocruzand it will config with however you put it on site.pp13:53
rcarrillocruzthat needs work, guessing what goes where13:53
rcarrillocruzit's not trivial13:53
asselin_what's not trivial? we have zuul setup this way for 3rd party ci13:54
*** rcernin has quit IRC13:54
rcarrillocruzcos it's not the same thing of having a an openstack infra where you can define your services in X machines13:54
rcarrillocruzthan having an all in one/single node13:54
*** njohnston|sleep is now known as njohnston13:55
asselin_no, I mean, with infra ansbile, you define your servers w/ names, and then puppet apply all of them using site.pp which tells each one how to 'become' based on fqdn.13:55
*** amotoki has quit IRC13:55
*** amoralej|lunch is now known as amoralej13:56
AJaegerpabelanger: oh, interesting...13:56
*** links has quit IRC13:56
rcarrillocruzright, but the site.pp is generated per the data the playbooks discover... if you know all the data upfront, it's trivial to just put your final site.pp and run puppet apply13:56
rcarrillocruzinfra-ansible templatizes a site.pp based on your cloud config13:57
rcarrillocruzand other services files per your cloud config13:57
asselin_oh I see, I missed that part13:57
AJaegerpabelanger: same problem twice in jenkins/jobs/api-jobs.yaml according to a quick grep13:57
rcarrillocruzthen when zuul is deployed13:57
rcarrillocruzit makes sure that zuul.conf has a config pointing to mygerrit.lol13:57
rcarrillocruzso on and so forth13:57
*** rbrndt has joined #openstack-infra13:57
*** hashar has joined #openstack-infra13:59
jrollhey, you all run the mailing lists, right? rackspace people are trying to fix up all the bounces we generate, want to make sure I put them in touch with the right people13:59
asselin_rcarrillocruz, I see...I was assuming site.pp already existed. I need to see where that gets templated.13:59
jrollfungi: ^13:59
AJaegerpabelanger: could you check again today that nodepool uploads images everywhere, please? we've merged a change on Monday morning and it's still not everywhere (missing for sure from rax-ord)13:59
rcarrillocruzasselin_: i warn you, upstream follows a different pattern than gozer fork13:59
anteayajroll: morning, yes fungi was working on that last night13:59
*** dprince has joined #openstack-infra13:59
rcarrillocruzgozer fork does have a complete site.pp , ala system-config site.pp13:59
jrollanteaya: great, thanks :)13:59
*** xarses has quit IRC14:00
rcarrillocruzupstream i use hiera to do node classification14:00
*** _nadya_ has joined #openstack-infra14:00
rcarrillocruzthe site.pp is literally:14:00
krotscheckCan someone add me as a core to,access14:00
krotscheckThanks :)14:00
asselin_rcarrillocruz, I'm using upstream right now, but for masterless puppet, using pre-defined site.pp14:00
rcarrillocruzhiera_include('roles', [])14:00
*** jed56 has quit IRC14:01
rcarrillocruzsuper cute14:01
*** jed56 has joined #openstack-infra14:01
*** hashar has quit IRC14:01
fungiyolanda: AJaeger: the challenge is that gerritbot does not (currently) log messages it receives from the irc network/server so we have no idea if it's being rejected when it tries to join additional channels. it doesn't seem to actually even confirm whether its /joins work. i want to try out patch 287056 and see if we can get a clearer picture of what's actually going on, but given that `/quote14:01
fungiversion` reports CHANLIMIT=#:120 for freenode and the bot isn't appearing in subsequent channels past the first 120 it's a pretty safe bet14:01
*** zhurong_ has joined #openstack-infra14:01
fungithe 242/244 count i gave yesterday from parsing the logs was me stupidly not spotting there was more than one bot restart in there14:02
AJaegerfungi: and right now we have 120 (exactly) channels in gerritbot14:02
*** zhurong has quit IRC14:02
AJaegerfungi, I've blocked a new channel to be used:
openstackgerritMerged openstack-infra/system-config: Private embargoed vulnerability announcement list
fungiprobably a "simple" solution would be to run multiple gerritbot instances with distinct freenode accounts and split the channels between them14:03
AJaegerfungi, please go ahead - and tell me once I should unblock 32912914:03
anteayafungi: maybe they won't both die at the same time?14:04
jrollanteaya: thank you :)14:04
fungijroll: i believe elarson was updating an internal rackspace ticket about the timeouts and failure to defer for some fireeye scanner there?14:05
openstackgerritMaciej Relewicz proposed openstack-infra/project-config: New repo for murano plugin
jrollfungi: yep, that's how that log reads to me, I guess some people internally are out of sync, hopefully I straightened them out :)14:05
fungijroll: i haven't caught up on my overnight e-mail yet this morning, so not sure if i have additional details yet14:05
fungijroll: adrian otto was also pinging ttx about the situation, so there may be multiple parties out of sync over there14:06
*** mtanino has joined #openstack-infra14:06
jrollfungi: well, seems to be a few people coordinating well, and everyone else independently getting unsubscribed and freaking out :)14:06
*** flepied1 has quit IRC14:07
*** flepied has joined #openstack-infra14:07
anteayasounds like fun14:08
elarsonjroll: I sent the ticket to the dev list. I'll send it to you on internal chat14:09
jrollelarson: I see it on the dev list14:09
elarsonok, great14:09
jrollI replied at the same time as you :)14:09
*** rcernin has joined #openstack-infra14:09
elarsongreat minds think alike :)14:09
* jroll thinks everyone should just use personal email for this14:09
* jroll realizes he's off topic and steps away :P14:10
anteayajroll: no, I think confirming using personal email for open source work is very much on topic14:10
jrollanteaya: :)14:10
anteayait saves a host of problems/work14:11
*** ddieterly is now known as ddieterly[away]14:11
*** zeih has joined #openstack-infra14:12
fungijroll: it's been my solution since... well since always14:13
*** cdent has joined #openstack-infra14:14
fungii only use my employer-provided e-mail address for a couple of very specific tasks, which do not include git/gerrit, mailing lists, authentication, et cetera14:14
*** hongbin has joined #openstack-infra14:15
jeblairJayF: do you know about ?14:15
electrofelixzxiiro: quick question in just about how you tested the extensions XML merging, wasn't fully sure from your response, but sounds like it works as desired (if not quite as expected based on previous interesting behaviour :p)14:17
electrofelixzxiiro: just checking before I approve it.14:17
*** zz_dimtruck is now known as dimtruck14:17
*** zeih has quit IRC14:17
zxiiroelectrofelix: yeah I tested it with several of the extensions (not all) but I think enough to understand the behaviour.14:17
jeblairJayF: you can define new dashboards in git here:
zxiiroelectrofelix: I configured several plugins and then commented them out to see if tehy got removed. and it did14:18
electrofelixso extensions replaces the XML sub tags14:18
electrofelixsounds good, approving so14:18
odyssey4meyolanda AJaeger (or any other project-config cores) if you have a moment I'd appreciate a review of
*** rcernin has quit IRC14:18
*** sigmavirus24_awa is now known as sigmavirus2414:18
zxiiroelectrofelix: yeah extensions just uses a null tag <whatever.feature/> for enabling and if it doesn't exist it clears it14:18
*** trown is now known as trown|mtg14:18
*** ddieterly[away] is now known as ddieterly14:19
*** nmagnezi has joined #openstack-infra14:19
*** oanson has quit IRC14:19
hongbinAJaeger: there?14:22
AJaegerhongbin: yes14:22
hongbinAJaeger: Have time to discuss this patch:
*** xarses has joined #openstack-infra14:23
AJaegerhongbin: let me first review odyssey4me and then I'll look at yours...14:24
openstackgerritMerged openstack-infra/zuul: Refactor local_action SCP task into function
*** Hal has joined #openstack-infra14:24
openstackgerritMerged openstack-infra/zuul: Ansible launcher: Remove unused plugins dir
*** Hal is now known as Guest7764314:25
*** dprince has quit IRC14:25
openstackgerritMerged openstack-infra/zuul: Add log_path to ansible.cfg
openstackgerritMerged openstack-infra/jenkins-job-builder: Update Git SCM so that extensions are configured correctly.
AJaegerhongbin: ok, reviewed and ready to discuss14:26
openstackgerritMerged openstack-infra/zuul: Ansible launcher: move ftp/scp staging out of ansible dir
openstackgerritMerged openstack-infra/zuul: Upload ansible playbook directory to logs server
*** winggundamth_ has quit IRC14:26
hongbinAJaeger: I just want to know how to proceed14:26
AJaegerhongbin: but better to talk with pleia2, she has done the last IRC renames.14:26
jeblairi'm gracefully stopping zl01 and zl02.14:27
openstackgerritMerged openstack-infra/zuul: Ansible launcher: ensure that console/ansible publishing is last
AJaegerhongbin: since you need to change system-config as well, we need to do this in the proper order, so perhaps (but let pleia2 confirm): 1) create new channels, 2) rename in system-config; 3) remove old channels14:27
AJaegerAnd this together with the redirect of your old channel if you want it...14:28
hongbinI see14:28
*** pt_15 has joined #openstack-infra14:28
*** yujunz has joined #openstack-infra14:29
rcarrillocruzasselin_: putting puppetmaster-less should be easy, i thought about it a while ago, but haven't tackled it because of $WORK. I'd add a flag 'puppetmaster_less', if true, do not run bootstrap_puppetamaster playbook and run in all hosts. Also, branch the puppet.conf.j2 accordingly, so nodes are not pointing to a master14:29
rcarrillocruzworking on puppetdb stuff now14:29
rcarrillocruzansible-puppet should work on puppetdb 3x14:30
asselin_rcarrillocruz, ok yeah, right now I just use the default puppet.conf after install_puppet.sh14:30
rcarrillocruzso testing stuff , but also decoupling stuff as i find it, e.g. adding a puppetdb class on puppet_openstackci14:30
rcarrillocruzcos right now, it's bundled all the logic on openstack_project::puppetdb14:30
*** vhosakot has joined #openstack-infra14:31
*** rcernin has joined #openstack-infra14:31
pabelangerjeblair: ack14:33
*** jsavak has quit IRC14:34
pabelangerAJaeger: Sure, I can confirm uploads worked14:34
pabelangerAJaeger: will also update the review14:34
jeblairpabelanger, AJaeger: would you like me to refresh ?14:35
*** jsavak has joined #openstack-infra14:35
*** yujunz has quit IRC14:35
pabelangerjeblair: Yes, please. On my list to test, but never did14:36
*** burgerk has joined #openstack-infra14:37
rcarrillocruznibalizer: heya, in infra's setup, how's ssl for puppetdb? Are nodes using the previous puppet agent certs or those were removed and is apache being a proxy?14:38
rcarrillocruzunless there's no SSL at all, but running puppet apply on system-config/site.pp does set ssl on jetty.ini14:39
nibalizeryep puppet agent certs14:40
rcarrillocruzi'll set up a puppetmaster to issue certs to do my tests14:40
rcarrillocruzbtw, pushed patchset for ansible-puppet thingy14:40
AJaegerpabelanger: - this is still the old content, so rax-ord is not uptodate14:41
*** sree has quit IRC14:42
AJaegerjeblair: yes, please!14:42
jeblairAJaeger, pabelanger, fungi: updated -- remote: Add a simple status webapp14:44
AJaegerpabelanger: if you say uploads worked, why did I get an old image for that?14:44
pabelangerAJaeger: uploads don't always work. It is possible they timed out, I'll check shortly when we last uploaded14:45
openstackgerritPaul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher
openstackgerritPaul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher
openstackgerritPaul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher
AJaegerpabelanger: thanks14:46
*** hdaniel has quit IRC14:46
*** yamamoto has quit IRC14:46
openstackgerritMateusz Matuszkowiak proposed openstack-infra/project-config: New repo for murano plugin
AJaegerpabelanger: looking at your patch series, is there a way to enforce this? Or document it at least? Perhaps in the REVIEW document?14:48
openstackgerritMerged openstack-infra/jenkins-job-builder: Added amazon web services to credentials bindings
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add console-log validator
openstackgerritMateusz Matuszkowiak proposed openstack-infra/project-config: New repo for murano plugin
*** gordc has joined #openstack-infra14:50
AJaegerpabelanger: thanks for 329988!14:51
*** YorikSar has quit IRC14:51
JayFjeblair: now I do :D thanks!14:51
pabelangerAJaeger: Still failing, but will be easier to find the console-logs now14:51
*** YorikSar has joined #openstack-infra14:53
*** zhurong_ has quit IRC14:54
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add console-log validator
*** Julien-zte has joined #openstack-infra14:57
*** croelandt is now known as croelandt|afk14:58
*** jsavak has quit IRC14:58
*** tonytan4ever has joined #openstack-infra14:58
*** jsavak has joined #openstack-infra14:59
*** yamahata has quit IRC14:59
*** yamahata has joined #openstack-infra15:00
*** hashar has joined #openstack-infra15:00
*** inc0 has quit IRC15:00
anteayanotmorgan: you about yet?15:00
anteayanotmorgan: I'm wondering if you can help me figure out a thing with api auth in #storyboard15:01
*** psachin has quit IRC15:04
*** afred312_ has joined #openstack-infra15:04
*** abregman has quit IRC15:04
AJaegeranteaya: do you have time to review and approve, please? Once that's in and on all servers, we can merge the dependent changes - so, one after each other to not overwhelm Jenkins ;)15:05
*** afred312 has quit IRC15:05
anteayaAJaeger: right after the storyboard meeting, sure15:05
*** _nadya_ has quit IRC15:07
*** inc0 has joined #openstack-infra15:07
*** sdake_ has quit IRC15:08
*** sdake has joined #openstack-infra15:10
mordredAJaeger: you know - soon we will no longer have to worry about overwhelming jenkins15:11
*** sdake has quit IRC15:11
*** yamamoto has joined #openstack-infra15:12
jeblairwe could just wait until jenkins is gone to approve those15:12
AJaegermordred: I look forward to that ;)15:14
AJaegerjeblair: let's get them in to help us get better logs...15:15
jeblairAJaeger: well, they aren't a pre-requisite for zuulv2.5 -- we have a thing in zuul that will automatically make that transformation for us.  getting those changes in just normalizes the config with what zuul does automatically15:17
*** dimtruck is now known as zz_dimtruck15:17
*** kzaitsev_mb has quit IRC15:17
*** xwizard has quit IRC15:18
AJaegerjeblair: interesting!15:18
*** zz_dimtruck is now known as dimtruck15:18
AJaegerjeblair: you're doing far too much magic for me to follow ;)15:19
*** xwizard has joined #openstack-infra15:19
*** esikachev has quit IRC15:19
openstackgerritHongbin Lu proposed openstack-infra/project-config: Zun: rename higgins to zun (1)
openstackgerritHongbin Lu proposed openstack-infra/project-config: Zun: rename higgins to zun (2)
*** bhavik has joined #openstack-infra15:20
*** skraynev has quit IRC15:20
*** arxcruz has quit IRC15:21
*** oanson has joined #openstack-infra15:21
*** abregman has joined #openstack-infra15:22
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add console-log validator
openstackgerritPaul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher
openstackgerritPaul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher
openstackgerritPaul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher
openstackgerritPaul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher
pabelangerwith gate job working too15:23
*** ifarkas has quit IRC15:23
jeblairpabelanger: i hesitate to say this, but if we continue with the block/always approach, i'm not sure if we can remove all of the transformPublishers bit -- we at least still need to split publishers into block/always, even if we don't reorder them15:24
*** tonytan4ever has quit IRC15:24
*** Na3iL has quit IRC15:24
jeblairnotmorgan: we haven't merged the timeout thing yet have we?15:26
*** YorikSar has quit IRC15:26
*** andreas_s has quit IRC15:26
pabelangerjeblair: Right, I do want us to land the copy zuul logs and upload with console-logs publisher stuff, because we need the ansible logs. But moving forward, I do think breaking that upload out in to a dedicated zuul_logs rsync process is something we need.15:27
*** xarses_ has joined #openstack-infra15:28
pabelangerthe main reason, today we won't have ansible logs for our publisher tasks. So there is a little bit of a blind spot in our ansible.log15:28
*** flepied has quit IRC15:29
*** xarses has quit IRC15:29
AJaegerpabelanger: ovh-bhs1 seems very old, the jenkins slave_script does not even exist on an image that my job just run on ;( and rax-ord looks outdated but not that old15:30
openstackgerritHongbin Lu proposed openstack-infra/system-config: Rename IRC channel of container service
openstackgerritMerged openstack-infra/zuul: Ansible launcher: ensure log publishing always runs
*** amitgandhinz has quit IRC15:33
jeblairpabelanger: we do have those logs15:33
*** amitgandhinz has joined #openstack-infra15:34
openstackgerritMerged openstack-infra/zuul: Ansible launcher: set keep-hierarchy on log copying
jeblairpabelanger: the logs go right up to "copy the ansible directory into the upload staging dir" -- so the only thing we are missing is the results of that copy and the final upload.  i don't think we will ever get the logs of the final upload itself, more or less by definition :)15:34
openstackgerritMerged openstack-infra/jenkins-job-builder: Add flaky test report and allow empty options to JUnit publisher
*** shardy has quit IRC15:35
*** sorantis has quit IRC15:36
*** hashar has quit IRC15:36
*** yamamoto has quit IRC15:36
*** yamamoto has joined #openstack-infra15:36
pabelangerAJaeger: possible, I haven't looked yet15:37
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Implement Xenial/CentOS jobs for OpenStack-Ansible
*** dizquierdo has quit IRC15:37
dasmhey folks. i'm trying to use logstash but it doesn't cooperate with me.15:38
pabelangerjeblair: I don't want to distract on the current effort for publishing logs, but have an idea how to maybe fix that.  Happy to talk about it more once jenkins have been turned off :D15:38
dasmi opened private browser, with clean history, and nothing happens15:38
dasmin Network tab (Chrome) I see:
dasmis it known problem?15:38
odyssey4meAJaeger anteaya thanks for the reviews - has been updated15:39
pabelangerAJaeger: Wow, we haven't updated ovh-bhs1 in 22 days15:39
*** ddieterly is now known as ddieterly[away]15:39
pabelangersomething going on there15:39
*** garyk has quit IRC15:39
*** kzaitsev_mb has joined #openstack-infra15:39
AJaegeroops ;(15:39
pabelangerAJaeger: reuploading ubuntu-trusty now15:39
pabelangerchecking debug logs15:40
dasmprobably, problem is around: "[FIELDDATA] Data too large, data for [@timestamp] would be larger than limit of [22469214208/20.9gb]];"15:40
jeblairpabelanger: here's the log from my local test:
*** yamamoto has quit IRC15:40
jeblairpabelanger: the other changes have landed, i'm going to start zl01 now15:41
*** YorikSar has joined #openstack-infra15:42
pabelangerHmm, Unauthorized. Check username, password and tenant name/id. Found that in nodepool-builder.log
pabelangerback on 06.1215:42
*** markusry has quit IRC15:42
*** vincentll has quit IRC15:43
*** markusry has joined #openstack-infra15:45
*** mikelk has quit IRC15:45
*** markusry has quit IRC15:46
*** markusry has joined #openstack-infra15:47
*** markusry has quit IRC15:48
*** yamamoto has joined #openstack-infra15:48
*** ddieterly[away] is now known as ddieterly15:49
jeblairpabelanger, Shrews:
*** markusry has joined #openstack-infra15:50
*** pcaruana has quit IRC15:50
pabelangerjeblair: very nice15:50
*** roxanaghe has joined #openstack-infra15:51
pabelangershould make debugging a little easier15:51
Shrewsjeblair: those zuul_runner parameters are funky15:51
jeblairwe should make '.log' a text/plain extension on that server15:52
pabelangermordred: clarkb: I think something odd is happening in ovh-bhs1:
*** trown|mtg is now known as trown15:52
*** dmk0202 has quit IRC15:52
*** Guest77643 has quit IRC15:52
jeblairShrews: yeah, i think we may want to use the 'safe' yaml enecoder ?15:52
pabelangermordred: clarkb: if reading right, shade says the images is ready in ovh-bhs1, but nodepool still reports is over 23 days old15:52
mordredyes. you want to use the safe one15:52
*** croelandt|afk is now known as croelandt15:53
*** rcernin has quit IRC15:53
*** Sukhdev has joined #openstack-infra15:53
*** markusry has quit IRC15:54
jeblairbut i really like that it uses the &/* reference15:54
*** gsagie has joined #openstack-infra15:54
jeblairbecause otherwise that playbook would be HUGE15:54
gsagieI am not sure where this is related to, but i think the 1000 chars restriction in the talks submission is not working right (it complains for much less then 1000 chars)15:54
Shrewsjeblair: nothing in the 'block' block in the post-playbook?
jeblairShrews: correct -- that means that job had no other publishers than the console log15:55
gsagienot sure if its related here :)15:55
jeblairShrews: lemme look for another job with maybe a more interesting playbook15:55
Shrewsjeblair: i'm surprised that always:block: (not block:always:) works15:55
*** tesseract has quit IRC15:55
*** netsin has joined #openstack-infra15:55
gsagieanteaya: thanks!15:56
anteayagsagie: I suggest you file a bug with them, the folks in charge of that decision aren't in this channel (much) and do most of their intake via launchpad15:56
jeblairShrews: ya, i wasn't sure it would, so i tested it.15:56
anteayagsagie: welcome15:56
*** amitgandhinz has quit IRC15:56
*** tonytan4ever has joined #openstack-infra15:56
jeblairShrews: i've become comfortable with it by thinking about the inverse -- how would you explain to someone that the thing you put in the 'always' block actually *never* ran.  *that* would be even more counterintuitive.  :)15:56
*** acabot has quit IRC15:57
*** amitgandhinz has joined #openstack-infra15:57
openstackgerritLukas Bednar proposed openstack-infra/git-review: Fix AttributeError when can not connect to Gerrit
*** tbarron is now known as tbarron|afk15:57
Shrewsjeblair: as long as ansible never changes to enforce the order, good by me  :)15:58
jeblairShrews: oh, you're talking about the order15:58
*** Rockyg has joined #openstack-infra15:58
jeblairShrews: i thought you were talking about empty blocks -- that's what i was yapping on about15:58
jeblairShrews: i think it's a dictionary, so order doesn't matter15:58
*** fernnest__ has joined #openstack-infra15:59
jeblairtask: [ {always:...; block:...} ]15:59
jeblairrather -- tasks: [ {always:...; block:...} ]15:59
Shrewsjeblair: i wonder if it would make the log easier to read if we started using names on the tasks16:00
mordredjeblair: fwiw, we've got an OrderedDict yaml serializer in flamel - so although order doesn't matter, if we think that a certain order would be more readable, it's not hard to achieve16:00
*** oanson has quit IRC16:01
jeblairShrews: that's a good idea, and naming them by their jjb macro name would be useful, however, i think we lose that information by the time jjb has fully parsed the job...  we might need some jjb modification to do that, and that may not be worth the effort.16:01
*** jcoufal has joined #openstack-infra16:01
jeblair(also, if you don't use macros, then the publishers are completely anonymous anyway)16:02
*** javeriak has joined #openstack-infra16:02
*** afred312 has joined #openstack-infra16:03
*** jcoufal_ has quit IRC16:03
jeblairShrews, mordred, pabelanger: a more interesting post playbook:
*** jsavak has quit IRC16:04
*** jsavak has joined #openstack-infra16:04
jeblairif you look at the log for that one, you can see the 'block' running first16:04
*** sabari has joined #openstack-infra16:04
*** afred312_ has quit IRC16:04
*** sdake has joined #openstack-infra16:05
*** Julien-zte has quit IRC16:06
*** javeriak_ has joined #openstack-infra16:06
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul: Turn off Ansible host fact gathering
*** javeriak has quit IRC16:07
jeblairi'm happy enough with this to go aheand and continue the rollout to the other zl hosts -- sound good?16:07
Shrewsjeblair: pabelanger: ^^^16:07
*** abregman has quit IRC16:07
*** ramishra has joined #openstack-infra16:07
jeblairShrews: ++.  is that a config file option by any chance?16:07
*** ramishra has quit IRC16:07
Shrewsjeblair: mmm, sort of? it's a bit wonky16:07
*** Julien-zte has joined #openstack-infra16:09
Shrewsjeblair: we could set it to 'explicit' i guess16:09
*** krotscheck is now known as krotscheck_dcm16:09
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Add a simple status webapp
*** jsavak has quit IRC16:09
*** jsavak has joined #openstack-infra16:10
Shrewsjeblair: i can change it to set that, if you prefer16:10
* rcarrillocruz thinks that explicit should be the default on ansible/ansible, but oh well...16:10
*** thiagop has joined #openstack-infra16:10
jeblairShrews: your call16:11
*** YorikSar has quit IRC16:12
*** mdrabe has quit IRC16:12
Shrewsjeblair: for consistencies sake, i'll move it to config16:12
*** YorikSar has joined #openstack-infra16:13
*** ramishra has joined #openstack-infra16:13
*** tonytan4ever has quit IRC16:14
anteayaAJaeger: so I said I would review and consider approving 329983, then there was discussion, do we still want me to review and merge it?16:14
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul: Turn off Ansible host fact gathering
rcarrillocruznibalizer: the puppetdb routes.yaml, do we manage it on git? or that's set by hand?16:15
Shrewsjeblair: there ya go16:15
rcarrillocruzcan't seem to get hits for it on system-config16:15
anteayaAJaeger: I'm +2, I'll leave it up to you to merge it that is still the concensous16:15
jeblairShrews: cool -- missing a final \n16:15
*** ddieterly is now known as ddieterly[away]16:16
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul: Turn off Ansible host fact gathering
*** piet has joined #openstack-infra16:17
*** tonytan4ever has joined #openstack-infra16:19
jeblairi have started zl02 and gracefulededed zl0316:20
*** jaosorior has quit IRC16:20
jeblairalso graceful zlstatic0116:21
*** ddieterly[away] is now known as ddieterly16:21
jeblairand restarted zlstatic0116:21
jeblairmordred: want to +3
*** derekh has quit IRC16:22
*** mescanef has quit IRC16:22
mordredjeblair: done16:22
openstackgerritMonty Taylor proposed openstack-infra/zuul: Add custom serializer that supports OrderedDicts
mordredjeblair: there is handling ordered dicts and also rendering the unicode strings as normal strings and not the python/unicode things16:23
zarosnarwade: i don’t believe jjb supports jenkins kubernets plugin yet.16:23
jeblairmordred: do we care about the ordered dicts?16:24
*** kmartin has joined #openstack-infra16:24
mordredjeblair: dunno. that's mostly a "if we think that block then always is easier to read" patch16:24
anteayamorning zaro16:25
mordredjeblair: since I had the code to implement it already somewhere else16:25
*** tonytan4ever has quit IRC16:26
jeblairmordred: okay.  i'm going to vote a decided +0 on that patch.  :)16:27
jeblair(basically, it doesn't bother me, but if others like it, great!)16:28
*** mdrabe has joined #openstack-infra16:28
openstackgerritJames Page proposed openstack-infra/project-config: Add specifications repository for charms
openstackgerritMerged openstack-infra/zuul: Turn off Ansible host fact gathering
nibalizerrcarrillocruz: I don't futz with routes.yaml16:29
nibalizeras a general rule16:29
nibalizerdoes puppetdb even have one? I think without a puppetmaster routes.yaml doesn't really do anything16:30
*** sarob has joined #openstack-infra16:30
*** dprince has joined #openstack-infra16:31
*** sarob has quit IRC16:31
*** piet has quit IRC16:32
*** sarob has joined #openstack-infra16:32
*** matrohon has quit IRC16:33
*** piet has joined #openstack-infra16:33
* notmorgan wakes up and glares at subunit16:33
notmorganyou know... it's bad when you have a dream ... in python :(16:33
mordrednotmorgan: hahaha16:33
notmorganalso, i lost my computer-fixing screw driver set. it's going to make it hard to build this system up tonight.16:34
*** ihrachys has quit IRC16:34
*** hongbin has quit IRC16:34
notmorganbut tomorrow... or the next day... actually having a system i can run openstack on locally for zuul/nodepool testing = win.16:34
Shrewsnotmorgan: that would be nice16:35
* notmorgan needs to ask red hat for a workstation too so the ambient temperature of my place can be raised by 5 degrees *and* I have more CPU capacity for things.16:35
*** sarob has quit IRC16:35
*** zeih has joined #openstack-infra16:35
*** sarob has joined #openstack-infra16:36
notmorganShrews: yeah, well i figured i needed something I can actually mirror some infra workloads on- so yay 64GB machine [also, I kindof want to play OverWatch w/ friends... so DUAL PURPOSE!]16:36
*** piet has quit IRC16:36
notmorganplus it's fun to make my friends in the game industry's heads explode when I describe insane virtualization setups with PCI passthrough and processor affinity16:36
*** piet has joined #openstack-infra16:37
pabelangerjust catching up on backscroll16:37
jeblairnotmorgan: i don't want to throw water on your building a supercomputer or anything -- so go right ahead -- i just want to say that it's pretty important to me that zuul and nodepool be runnable without actual clouds ---16:38
notmorganjeblair: i can run them here on my laptop16:38
jeblairnotmorgan: so with nodepool, you can run it with a completely fake provider system, and zuul i mostly exercise through its unit/functional tests (which have the gerrit/jenkins stuff faked out)16:38
notmorganjeblair: the issue is when i want to put more workload than 16GB of ram will allow.16:38
*** Apoorva has joined #openstack-infra16:39
notmorganjeblair: testing is great, but sometimes you want to cycle through a lot more than the laptop will allow [docker, etc on this laptop has proven painful]16:39
*** trown is now known as trown|lunch16:39
*** eezhova has quit IRC16:39
*** infra-red has joined #openstack-infra16:41
mordredI think it's very important for all of my testing to run on jeblair's laptop16:41
*** infra-red has quit IRC16:41
*** zeih has quit IRC16:41
notmorganjeblair: fake providers are good, but also real providers are important too. if you only ever test the fake provider except when you drop it in production, i'm going to argue you're doing it wrong16:41
notmorganand i'd like to not need to lean on devstack/infra for every single test that exceeds my capabilities locally :)16:41
* notmorgan always runs this stuff locally and lean first.16:42
*** sdake has quit IRC16:42
jeblairnotmorgan: then i will happily do it wrong16:42
jeblairnotmorgan: this method of testing has been *extremely* good at keeping us from dropping broken code into production16:42
jeblaira change that passes zuul's unit tests is absolutely fit for production16:43
notmorganjeblair: i'm not advocating changing the testing model ;) I'm advocating adding more rigerous testing in :)16:43
notmorganfor me at least.16:44 seems to be stuck, unable to load events16:44
* notmorgan shrugs.16:44
jeblairbecause zuul's interactions with other systems are so thoroughly replicated, and we're able to test completx series of events that are very difficult to replicate manually16:44
*** jlanoux has joined #openstack-infra16:44
jeblairnotmorgan: like i said, i'm not trying to discourage you from anything -- but i am resistant to the idea that you need a cloud to test nodepool or zuul16:45
*** _nadya_ has joined #openstack-infra16:45
mordredI think zuul's unit tests should be comprehensive and sufficient. however, I think that at times depending on working style, pointing things at real resources might be the thing that a human needs to do to poke at something, and that should also be totally possible16:46
jeblairyep, i'm not saying otherwise16:46
*** yolanda has quit IRC16:46
notmorganmordred: ++16:46
notmorganjeblair: it's def. more my workflow :)16:46
anteayamelwitt: confirmed16:47
notmorganjeblair: and unit/functoional/acceptance tests are always where it starts16:47
jeblairit is, however, *extremely* important to me that we not *require* that.  that is a huge barrier to contribution, and if it is not necessary, which it is not, we should not erect it16:47
*** bpokorny has joined #openstack-infra16:47
notmorganjeblair: and if those SUCK - we're wrong16:47
notmorganjeblair: or insufficient16:47
anteayajesusaur: clarkb do either of you have time to take a peek at our logstash status?16:47
*** bpokorny has quit IRC16:47
notmorganjeblair: i think what i'm saying is i'm in very strong agreement, but want more flexibility personally to poke at things.16:47
*** ddieterly is now known as ddieterly[away]16:47
anteayait doesn't seem to be logstashing as expected16:48
*** bpokorny has joined #openstack-infra16:48
jeblairnotmorgan: good -- i only mentioned it because i don't know that you know that i don't think it's a technical requirement.  :)16:48
*** BobBall is now known as BobBall_AWOL16:48
jeblairand i don't want there to be a meme that you need a cloud to work on this stuff16:48
jeblairif you want one, go right ahead.  if you don't have or want one, you can be just as effective without it16:49
*** jlanoux has quit IRC16:50
*** raunak has joined #openstack-infra16:50
openstackgerritPaul Belanger proposed openstack-infra/zuul: Add task name of builder for zuul_runner
notmorganjeblair: i also frankly can't keep long running vms in clouds [it adds up stupid fast] for faster/more concurrent tests. I've had exactly one computer (a laptop) to do everything for ~3yrs. I'm happy to be able to fire off a series of tests on a local machine [esp. when the gate is busy] and not have to walk away to look at something else/coffee [or like when i was doing the zuul loop test, that was16:51
notmorgan2 days of being able to do almost nothing but irc since the laptop was busy]16:51
jesusauranteaya: in what way is it not working as expected?16:51
notmorganjeblair: so this has been on my todo list for a long time :)16:51
jeblairnotmorgan: oh definitely16:52
anteayajesusaur: melwitt reports and I confirm that logstash.o.o is not loading events16:52
anteayajesusaur: just the round spinning thing, which is entertaining but not as productive as it could be16:52
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/system-config: Apt mirror for MariaDB
jesusauranteaya: hm, yeah I'm seeing that too :(16:53
*** _nadya_ has quit IRC16:53
jesusaurbut shows that elasticsearch is happy16:53
odyssey4mepabelanger ^ I'm not entirely sure that I have the link path right for - as I recall there was also some discussion about naming with more from mordred, but that feedback never made it into the review :/16:53
openstackgerritLukas Bednar proposed openstack-infra/jenkins-job-builder: Builders: Add ansible-playbook builder
*** mtanin___ has joined #openstack-infra16:54
odyssey4meit'd be really nice to have a search mechanism for eavesdrop :)16:54
anteayajesusaur: interesting thanks for confirming16:54
jesusauranteaya: clarkb: logstash looks like it needs a rooter to poke it16:55
*** mtanino has quit IRC16:55
odyssey4meoh neat, google actually works for searching eavesdrop16:55
*** yamahata has quit IRC16:56
pabelangerodyssey4me: sure, I'll try to review it later today16:57
jeblairpabelanger: i don't understand
*** _nadya_ has joined #openstack-infra16:58
notmorganodyssey4me: it's pretty cool :)16:58
pabelangerjeblair: I'll update the commit message. We are seeing Exception: Target path / is not below site root from zuul-launcher16:59
*** woodster_ has quit IRC16:59
*** NobodyCam has quit IRC16:59
*** mgkwill has quit IRC16:59
*** petems has quit IRC16:59
jeblairpabelanger: right, so maybe we need to think a bit more about what zuul-launcher is doing16:59
pabelangerjeblair: I haven't actually tested . however16:59
*** igorbelikov has quit IRC16:59
*** johnny_ has joined #openstack-infra17:00
*** cody-somerville has joined #openstack-infra17:00
pabelangerjeblair: I don't mind the revisi17:00
*** erlon has quit IRC17:00
jeblairpabelanger: basically, at the moment, errors are zuul-launcher's fault until proven otherwise :)17:00
*** dtantsur is now known as dtantsur|afk17:00
*** NobodyCam has joined #openstack-infra17:00
*** woodster_ has joined #openstack-infra17:00
pabelangerjeblair: sure. In that case, I think jenkins will silently strip the first char, if it is '/'17:00
*** mgkwill has joined #openstack-infra17:01
pabelangerAJaeger: ovh-bhs1 ubuntu-trusty uploaded correctly17:01
*** erlon has joined #openstack-infra17:01
*** igorbelikov has joined #openstack-infra17:01
*** mtanino has joined #openstack-infra17:01
*** petems has joined #openstack-infra17:01
*** tphummel has joined #openstack-infra17:01
openstackgerritPaul Belanger proposed openstack-infra/project-config: Fix upload-to-docs-site publisher
jeblairpabelanger: so maybe in the ftp publisher, we need to strip leading '/' before joining on to the site root?17:02
*** sree has joined #openstack-infra17:02
*** mtanin___ has quit IRC17:03
mordredjeblair: that sounds plausible17:05
pabelangerjeblair: we can. I do like the functionality in zuul-launcher today. Having zuul-launcher silently doing things to our jobs doesn't still well with me. That said, I think we could add a warning to the process and I'd be on board17:05
*** Jeffrey4l has quit IRC17:05
jeblairpabelanger: the intent for the scp and ftp publishers in zuul launcher is to be transparently compatible with jenkins17:06
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/system-config: Apt mirror for MariaDB
*** PollyZ_ has joined #openstack-infra17:07
*** Rockyg has quit IRC17:07
openstackgerritJames E. Blair proposed openstack-infra/zuul: Ansible launcher: strip leading / from scp/ftp targets
jeblairpabelanger, mordred: ^17:08
jeblairplease double check me on the assertions in the commit message :)17:08
AJaegerthanks, pabelanger !17:08
*** ddieterly[away] is now known as ddieterly17:08
AJaegerpabelanger: are you doing rax-ord as well?17:09
*** amoralej is now known as amoralej|off17:10
*** PollyZ has quit IRC17:10
pabelangerjeblair: +317:10
*** yaume has quit IRC17:11
*** abregman has joined #openstack-infra17:12
jeblairi'll start up zl04 (and maybe zl03 if it has stopped) with that when it lands17:13
*** bswartz has quit IRC17:13
*** rossella_s has quit IRC17:13
*** yamahata has joined #openstack-infra17:15
*** jsavak has quit IRC17:16
*** jsavak has joined #openstack-infra17:16
*** tonytan4ever has joined #openstack-infra17:17
*** sputnik13 has joined #openstack-infra17:17
notmorganianw: ping re if you see the use of the fixture as a blocker I'll swap it out, but I prefer the fixture since it ensures the object remains patched through the whole test run rather than just through the context manager.17:18
notmorganianw: but i understand that it's a slight bit easier to read with mock.patch.object directly :)17:19
openstackgerritMerged openstack-infra/project-config: Ensure console-log is our last publisher
*** PollyZ has joined #openstack-infra17:21
*** pcaruana has joined #openstack-infra17:21
*** eezhova has joined #openstack-infra17:22
openstackgerritMerged openstack-infra/zuul: Ansible launcher: strip leading / from scp/ftp targets
*** PollyZ_ has quit IRC17:24
*** ramishra has quit IRC17:25
anteayafungi: I have confirmed I can POST to storyboard-dev when manually curling from the command line of my gerrit instance, and have found out that the storyboard plugin is designed to take a long lived token, however the plugin is expecting ssl on the storyboard server, which -dev does not have17:26
anteayafungi: what is involved in getting the storyboard-dev server real certs instead of self-signed ones?17:26
*** Sukhdev has quit IRC17:26
anteayathe plugin is designed to not like self signed certs, so sayth zaro in the #storyboard channel17:27
fungianteaya: that seems mighty inflexible17:27
fungizaro: ^'17:27
fungii mean, we _can_ put a cert obtained from a third-party ca on there, but it seems like a pretty significant bug if people can't test that plugin with a self-signed cert17:28
anteayaI don't have enough knowledge here to have an opinion17:28
fungiand i'd rather not get a different cert on there just because some piece of software has a bug17:28
anteayajust trying to get enough proof the plugin works to have agreement it can get insalled on review-dev17:29
*** piet has quit IRC17:29
zaroanteaya: i think you can get around that by doing something to your local instance17:29
zaroanteaya: you shouldn’t need to change sb-dev in anyway17:29
*** afred312 has quit IRC17:29
anteayawhat needs doing, and in your estimation am I cabable of doing it?17:30
anteayathe plugin can talk to storyboard-dev which was what I was trying to prove17:30
anteayahave we got enough it can be deployed on review-dev yet?17:31
*** tqtran has joined #openstack-infra17:31
zaroanteaya: something with x509 i think which i remember needing to do. need to look for it again though17:31
anteayaplease if you could17:31
anteayaI'd rather not have this work held up by my ineptitude any longer than necessary17:31
zaroanteaya: i think we need to verify post before install on review-dev17:31
anteayazaro: I did17:32
zaroanteaya: i mean from storyboard17:32
anteayaI don't understand17:32
zaroanteaya: i mean from gerrit sb plugin17:32
anteayaright, well standing by on how to get that to work on my local instance17:32
*** jsavak has quit IRC17:32
anteayaI just don't want to agree to have to do something that will take me a long time to set up17:33
electrofelixappreciate reviews to enable py3 jobs for openstack/git-upstream -
*** electrofelix has quit IRC17:34
*** pilgrimstack has joined #openstack-infra17:34
*** kzaitsev_mb has quit IRC17:35
*** tqtran has quit IRC17:35
*** kzaitsev_mb has joined #openstack-infra17:37
*** HeOS has quit IRC17:37
*** roxanaghe has quit IRC17:41
*** ddieterly is now known as ddieterly[away]17:41
*** roxanaghe has joined #openstack-infra17:42
*** kzaitsev_mb has quit IRC17:42
*** kzaitsev_mb has joined #openstack-infra17:42
*** hongbin has joined #openstack-infra17:42
jeblairstarting zl0417:43
*** jamesmcarthur has quit IRC17:44
*** jsavak has joined #openstack-infra17:45
*** mhickey has quit IRC17:45
*** jamesmcarthur has joined #openstack-infra17:45
*** pilgrimstack has quit IRC17:45
*** akshai has quit IRC17:47
*** akshai has joined #openstack-infra17:47
*** mtanino has quit IRC17:48
openstackgerritJames E. Blair proposed openstack-infra/puppet-openstackci: Logs: serve .log and .sh files as text/plain
jeblairnotmorgan: that look right to you ^ ?17:48
jeblairpabelanger: ^17:49
pabelangerjeblair: looking17:50
*** mriedem1 has joined #openstack-infra17:50
notmorganjeblair: assuming we have mod_mime enabled?17:50
*** kzaitsev_mb has quit IRC17:50
jeblairgood question!  lemme check17:50
notmorganjeblair: because you don't want to do that w/o it enabled :)17:51
notmorgani'm guessing so because of the ForceMatch below17:51
jeblairyep, it is17:51
notmorganbut good to be sure.17:51
*** trown|lunch is now known as trown17:51
pabelangerjeblair: WFM17:51
notmorganand you want addtype not a forcetype? [i prefer addtype]17:52
notmorganand this wfm17:52
dasmo/ anyone looked at this issue?
*** degorenko is now known as _degorenko|afk17:52
*** mriedem1 has quit IRC17:52
notmorgandasm logstash was having issues earlier iirc. not sure what the status was.17:52
jeblairnotmorgan: yeah, i *think* we only need addtype here17:52
notmorgandasm: s/was/currently is17:53
notmorganjeblair: [pretty sure addtype should be used... unless it doesn't work.. then forcetype17:53
jeblairnotmorgan: ++17:53
dasmnotmorgan: mhm. so it's not usable? :/ eh. any ETA for change of logstash status? :)17:53
*** mriedem has quit IRC17:53
*** mriedem1 has joined #openstack-infra17:54
notmorgandasm: like i said, i heard someone tlaking about it earlier. not sure what the status is now.17:54
dasmnotmorgan: thanks.17:54
* notmorgan has to defer to other -infra-y type folks on that.17:54
notmorgananteaya: ^ you said something about logstash i think?17:54
anteayanotmorgan: yes I did and jesusaur replied17:54
*** tbarron|afk is now known as tbarron17:54
anteayaI was waiting to hear more back from jesusaur17:55
notmorgananteaya: perfect. thnx, dasm ^ cc17:55
notmorgandasm: if you lurk here and watch, you'll see updates as they come through :)17:55
anteayanotmorgan: welcome17:55
jeblairis there an infra-root looking into logstash?17:55
dasmnotmorgan: yeah. i'm idling here :)17:55
dasmnotmorgan: thanks gain17:55
anteayaI have not heard of one doing so17:55
notmorganjeblair: also, thanks for asking on that.17:55
jeblairelasticsearch says it is 'green' with 5 nodes17:56
jesusauranteaya: I don't have access to the systems, but I suspect that the kibana host is having connectivity issues connecting to elasticsearch17:56
fungia quick glimpse at the job queue graph suggests it's caught up processing at the moment, but kibana still doesn't pull up any results for me17:56
jeblairthat seems like 1 fewer nodes than we expect?17:56
fungielasticsearch is _supposed_ to have 8 i think?17:57
fungiso may be 3 fewer than we expect?17:57
*** eezhova has quit IRC17:57
anteayajesusaur: yes, that is fine, I had just thought you would follow up17:57
*** PollyZ has quit IRC17:57
jeblairi thought it was 6 -- elasticsearch02 through elasticsearch0717:57
anteayajesusaur: to find someone with access17:57
fungii'll take a peek at the kibana logs17:58
jeblairfungi: where are they?17:58
pabelangerjeblair: I am not17:58
fungijeblair: that's the first thing i'm trying to figure out17:58
jesusaurjeblair: they will be apache logs17:58
fungii assume it has some17:58
jeblairi can tell you one place they are not17:59
fungiheh, /var/log/kibana dne huh? ;)17:59
*** daemontool has quit IRC17:59
openstackgerritRyan Hallisey proposed openstack-infra/project-config: Shorten the name of the kolla-kubernetes gates
fungi[Tue Jun 14 12:43:27.090786 2016] [proxy_http:error] [pid 29690:tid 140337871787776] (20014)Internal error: [client] AH01102: error reading status line from remote server, referer:
fungialso we have...18:00
jeblairthat was a while ago, and i did the status curn on es02 and it responded18:00
fungi[Mon Jun 13 19:57:20.447396 2016] [proxy:error] [pid 29690:tid 140337980892928] [client] AH00898: Error reading from remote server returned by /elasticsearch/logstash-2016.06.13/_search, referer:
fungiyeah, those are from >24 hours ago18:01
fungiaccess logs are current, so apache is logging18:01
jesusaurdoes the elasticsearch log on es02 have anything useful to say?18:01
fungijust nothing recent for apache error logs18:01
*** PollyZ has joined #openstack-infra18:02
fungi[2016-06-15 18:00:50,330][WARN ][indices.breaker          ] [elasticsearch02] [FIELDDATA] New used memory 22480787131 [20.9gb] from field [@timestamp] would be larger than configured breaker: 22469214208 [20.9gb], breaking18:02
*** pvaneck has joined #openstack-infra18:02
fungisame thing dasm said was in the page source when he pulled it18:02
zaroanteaya: try this
*** mtanino has joined #openstack-infra18:02
*** jsavak has quit IRC18:03
*** hashar has joined #openstack-infra18:03
*** bswartz has joined #openstack-infra18:04
anteayazaro: thank you18:04
*** tonytan4ever has quit IRC18:04
*** mriedem1 is now known as mriedem18:05
fungijesusaur: suggestions seem to be increasing heap space or reducing stored field data?18:05
jeblairi'm wondering if something is broken with expiration again18:05
jeblairthe curator logs seem to be empty18:05
anteayazaro: does it matter what user creates the cert? should the gerrit2 user create it?18:05
jeblairanyone know how to use bigdesk?  it's not in the docs18:06
*** eezhova has joined #openstack-infra18:07
jeblairoh got it18:07
jeblairthat wasn't too hard18:07
jesusaurfungi: seems reasonable, but how many data nodes should there be? is 5 a low number? if so then getting the others back online may relieve the memory pressure18:07
jeblairthe cluster does not know about elasticsearch0618:08
jeblairthe cluster does not know about elasticsearch0518:08
jeblairsorry ^ correction.  05 is correct18:08
*** oanson has joined #openstack-infra18:08
jeblairfungi: you want to check out 05 while i look for indices?18:08
anteayapabelanger: do you remember how to use bigdesk? clarkb showed us how, I'm trying to find the instructions18:08
*** ccamacho is now known as ccamacho|out18:08
*** csomerville has joined #openstack-infra18:09
fungi[2016-06-10 11:36:46,231][WARN ][indices.breaker          ] [elasticsearch05] [FIELDDATA] New used memory 22866534892 [21.2gb] from field [message] would be larger than configured breaker: 22469214208 [20.9gb], breaking18:09
jesusaurI see nodes 03 02 06 07 04 listed in
jeblairthat whas a while ago18:10
jeblairjesusaur: oh neat!  i didn't have to run bigdesk :)18:10
*** pvaneck has quit IRC18:10
*** akshai_ has joined #openstack-infra18:10
jesusaurjeblair: nope, bigdesk does the same thing I did, then makes a pretty page out of it :)18:10
jeblairjesusaur: where do i see what indexes there are?18:11
*** ajmiller_ is now known as ajmiller18:11
fungimost recent es log on 05 is from 2016-06-10 11:36:46,23118:12
fungithat seems older than i would expect?18:12
jesusauror for the shards18:12
jesusaurbut that doesn't tell you which node a shard is on18:12
fungilooks like there's a java process running on other es servers but not on 0518:13
*** akshai has quit IRC18:13
fungishould i try starting the elasticsearch initscript?18:13
*** cody-somerville has quit IRC18:13
*** vhosakot has quit IRC18:13
jesusaurfungi: that's what I would try18:13
fungithe log doesn't seem to indicate why it stopped, it just went silent18:13
jeblairthe indexes appear to be rotating correctly...18:13
*** tqtran has joined #openstack-infra18:13
fungiokay, i can see the expected process running on 05 now18:14
pabelangeranteaya: I usually run ssh -L elasticsearch02 -L 9200:localhost:9200 from a terminal then open file:///home/pabelanger/git/pabelanger/bigdesk/index.html#nodes in my browse18:14
jeblairso the only thing that seems wrong is that we're one server down, which es doesn't seem particularly unhappy about18:14
jeblairnow the cluster is red18:14
*** hashar has quit IRC18:14
jeblairwith 6 nodes18:14
*** hashar has joined #openstack-infra18:15
jesusaurthe cluster is likely looking for the "missing" shards of the indices that have been deleted from the other nodes18:15
*** hongbin has quit IRC18:15
jesusaurrunning the curator cron command will delete the old shards from 0518:15
anteayapabelanger: thanks18:16
*** hongbin has joined #openstack-infra18:16
jeblairjesusaur: i'll do that now18:16
anteayazaro: somehow creating that cert has affected how my vm is able to interpret keyboard input18:16
*** tonytan4ever has joined #openstack-infra18:16
anteayazaro: I can't paste into the terminal anymore and I have to hit every key twice to get a character to show up18:17
*** akshai_ has quit IRC18:17
jeblairjesusaur, fungi: ^18:17
*** notnownikki has joined #openstack-infra18:17
anteayafor instance getting it to accept https:// requires me to type hhttttppss::////18:17
jeblairdeleting 6.15 is slightly concerning...18:17
fungii've double-checked all the es servers and they're all running the same java process now18:17
*** akshai has joined #openstack-infra18:17
jeblairthe cluster is 'green'18:17
fungianteaya: your terminal might have just gotten in a strange state. try running `reset` and `stty sane`18:18
anteayafungi: thanks I will try18:18
*** tonytan4ever has quit IRC18:19
jeblairlogstash.o.o/_search is returning 500 errors18:19
jeblairbut nothing in the apache error log18:19
anteayafungi: I've tried the two above commands twice, no change in the situation18:19
jesusaurjeblair: yeah, that delete is odd18:19
anteayawell reset cleared my screen but other than that no change18:20
jeblairi'm inclined to restart apache18:20
*** sarob_ has joined #openstack-infra18:20
*** vhosakot has joined #openstack-infra18:21
*** _nadya_ has quit IRC18:21
fungi[2016-06-15 18:16:33,841][WARN ][indices.cluster          ] [elasticsearch05] [[logstash-2016.06.02][4]] marking and sending shard failed due to [failed recovery]18:21
fungiwith a lengthy java traceback on 05 now18:22
anteayaexiting the user and then sudo suing back to being gerrit2 apparently fixed the issue18:22
anteayahow odd18:22
jeblairfungi: that's one of the old indexes curator deleted, so probably okay?18:22
fungijeblair: that's my hope. i mean the service seems to have stayed running and that's just a warning18:23
*** sarob has quit IRC18:23
*** chem``` has joined #openstack-infra18:23
jeblairjesusaur: oh! i just noticed, the thing it deleted was "0016.06.15"18:25
jeblairas in -- the yer 16.  not 2016.18:25
zaroI would use the same user to create the cert as the one you plan to use in gerrit sb plugin18:25
*** chem`` has quit IRC18:25
zaroanteaya: ^18:25
jesusaurjeblair: ohhh18:25
jesusaurjeblair: that probably indicates a y2k bug in the logstash parsing18:26
fungipossible we have some bad timestamps in logs or bad timestamp parsing in our grok set?18:26
* jesusaur wonders which log18:26
jeblairanyway, that shard is gone now, so that's okay18:26
jeblair(maybe we can track it down the next time it shows up)18:26
anteayazaro: oh I created the cert with gerrit2 but my user for storyboard is my account, which is my email address18:27
jeblairso it seems like the cluster is fine, we just literally have too much data in the 06.15 index?18:27
jeblair(so it's refusing to run queries on it)18:27
fungijeblair: jesusaur: fwiw, we're still logging that indices.breaker warning about new used memory would be larger than configured breaker18:27
fungifor the @timestamp field18:28
jeblairfungi: yeah, and that's what i get back with the 500 error if i curl the search url directly18:28
jeblairfungi: so i think that's the issue18:28
fungii found one reference saying the search cache can fill up and trigger this too18:29
funginot sure how credible18:29
jeblair'we blew a breaker'18:29
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: promote: add upgrade job part of voting
clarkbsorry I have caught something and havent really computered today but thats the we have too many timestamps to fit in memory issue. if you delete the oldest index it goes away18:29
* fungi grabs a flashlight and heads down into the basement18:29
fungiwhat's the easiest way to delete the oldest index?18:29
*** e0ne has quit IRC18:30
jeblairclarkb: oh, okay -- i was thrown off by the fact that it was including today's index in the error.... but i guess that's just when it hit the error -- like, it collected all the data from the previous indices and then right near the end went over the limit?18:30
jesusaurfungi: you can run the curator command with a different number-of-days (or whatever the argument is called)18:30
jeblairfungi: curator, i'd guess?  tell it to delete days +118:30
clarkbjeblair: ya I think its the aggregate18:31
jeblairi will curate18:31
anteayazaro: so did this work or do I try again with sudo?
fungithanks jesusaur, clarkb, jeblair18:31
anteayazaro: certificate added, keytool error is kind of contradictory18:31
anteayaclarkb: :(18:32
*** sambetts is now known as sambetts|afk18:32
anteayaclarkb: I hope you are doing the things you need to feel better18:32
jeblaircurator --logfile /var/log/curator.log delete indices --older-than 9 --time-unit days --timestring \%Y.\%m.\%d18:32
jeblair2016-06-15 18:31:26,138 INFO      ---deleting index logstash-2016.06.0618:32
jeblairfungi, jesusaur, clarkb: ^18:32
jeblairlogstash.o.o is making pretty graphs now.18:32
fungithat looks perfect18:33
anteayamelwitt: ^^18:33
jeblairincluding a spike in events today18:33
fungidasm: ^18:33
notmorganjeblair: i like pretty graphs!18:33
dasmthanks jeblair fungi jesusaur18:33
dasmand all involved18:33
jeblairwhich i bet if we understood might explain the issue18:33
*** sarob_ has quit IRC18:34
fungithere was a sizeable spike on the job queue graph as well18:34
*** sdague has quit IRC18:34
jeblairoops, the spike i was looking at is too recent to be the cause, probably a symptom18:35
*** pt_15 has quit IRC18:36
melwittawesome, thanks everyone! (logstash.o.o)18:36
jeblairstarting zl03; gracefully stopping zl01 ond zl0218:37
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: Update current repo setup includepkgs
*** PollyZ_ has joined #openstack-infra18:38
*** aimeeu has quit IRC18:39
*** PollyZ has quit IRC18:39
*** akshai has quit IRC18:39
*** akshai has joined #openstack-infra18:39
pabelangerjeblair: Looks like jenkins07 is out of service (crash maybe). Should we restart it or leave it stopped?18:40
jeblair#status log jenkins03 and jenkins04 are in prepare-for-shutdown mode in preparation for decomissioning18:40
openstackstatusjeblair: finished logging18:40
notmorganjeblair: ^ that is pretty darn awesome i might add18:40
jeblairpabelanger: hrm.  restart, i think, under the circumstances :)18:40
anteayazaro: so I completed your instructions, restarted my local gerrit instance and added another comment to my patch which has Story: 11 in the commit message, yet I see no activity on!/story/1118:40
pabelangerjeblair: sure, let me clean it up18:41
notnownikkijeblair, that's fantastic stuff with 2.5 :)18:41
jeblairnotmorgan: i'm glad we have so many to do, because each one is fun :)18:41
*** sdague has joined #openstack-infra18:41
anteayazaro: I did confirm I can use the cert I created to POST to the storyboard-dev api manually18:41
pleia2anteaya: great!18:42
jeblairpabelanger: if you feel like restarting zl01 and zl02 after they stop, go right ahead; i'm going to break for lunch now18:42
jeblairnotnownikki: thanks! :)18:42
*** bhavik has quit IRC18:42
jeblairnotnownikki, notmorgan: two negative nicks!18:43
anteayapleia2: do you have any desire to take a peek at the storyboard-dev logs and see what it thinks I am trying to do?18:43
pabelangerjeblair: Sure, I'll keep an eye out18:43
notnownikkijeblair, mine is an inspector morse reference18:43
anteayapleia2: if not fungi suggested I use tcpdump, but it will take some work on his part to guide me how18:43
*** salv-orlando has quit IRC18:43
*** ddieterly[away] has quit IRC18:43
fungianteaya: well, you'll need your own storyboard instance for starters, since you'll need to use http instead of https18:44
anteayaoh my18:44
*** ddieterly has joined #openstack-infra18:44
*** salv-orlando has joined #openstack-infra18:44
*** jaosorior has joined #openstack-infra18:44
fungior else figure out some sort of http to https proxy between your gerrit and storyboard-dev18:44
pleia2our storyboard forces to https18:44
openstackgerritChristian Fetzer proposed openstack-infra/jenkins-job-builder: Add support for Clang Scan-Build plugin version 1.7
fungiassuming the plugin supports not using https18:44
jeblairnotnownikki: oh that's cool18:44
anteayayeah, I don't know how to do that18:44
anteayafungi: I don't know about that18:45
*** jed56 has quit IRC18:45
anteayazaro: does the plugin support not using https?18:45
*** ihrachys has joined #openstack-infra18:45
*** eharney has quit IRC18:45
zaroanteaya: i think the directions i gave you will work now.18:47
anteayazaro: so I completed your instructions, restarted my local gerrit instance and added another comment to my patch which has Story: 11 in the commit message, yet I see no activity on!/story/1118:47
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/system-config: OpenStack-CI Apt mirror for Ubuntu Cloud Archive
anteayazaro: and I don't have access to storyboard-dev to check the logs to see what it sent to the -dev server18:48
zaroanteaya: take a look at the gerrit log to see if you are getting any errors18:48
pleia2fungi: I see some channel cleanup got #senlin's gerrit bot back, still able to make some time to look at 287056 for bot debugging?18:49
fungimordred: we still have openstack-infra/ansible-build-image listed in governance with no accompanying repo created... do you happen to know if that's still meant to be a thing or if it should be removed from governance?18:49
fungipleia2: working my way down to it. trying to make sure first that we're all set to send out barcelona summit invites later this week18:50
anteaya[2016-06-15 18:38:37,992] WARN : Cannot load plugin its-storyboard18:50
zarowell there you go.  any other message why it can’t load?18:51
*** dimtruck is now known as zz_dimtruck18:51
fungipleia2: also AJaeger made a direct count of channels in the config and it was actually 122 not 244 (i was grepping the log and forgetting to see if there was more than one restart) so this seems almost certain to be freenode's channel maximum at work18:51
*** sree has quit IRC18:51
pleia2AJaeger, hongbin - we document channel renames at so if there's something we're missing, please add it :)18:51
pleia2fungi: aha! we should probably see about getting that lifted18:52
pabelanger#status log jenkins07 back online. Will manually cleanup used nodes moving forward18:52
openstackstatuspabelanger: finished logging18:52
zaroanteaya: yeah, i see the problem.18:52
anteayafunny the plugin was able to find the storyboard-dev api before18:52
fungipleia2: i don't know whether that's even something they can adjust for individual accounts18:52
pleia2fungi: I had my personal limit adjusted18:52
fungipleia2: oh? perfect!18:52
zaroit’s because that jar file is built against latest gerrit.  i’ll need to build one against 2.1118:52
fungii've never tried to exceed 120 actual channels18:52
anteayazaro: ah thank you18:52
odyssey4mepabelanger another mirror request to queue up when you have a moment
zarowill get it to you in a few18:53
pleia2the limit was lower a few years ago18:53
fungipleia2: in that case, almost certainly preferable to sharding across multiple gerritbot processes18:53
*** sarob has joined #openstack-infra18:53
anteayazaro: yup, that's fine, good time for me to take a walk18:53
anteayazaro: be back in about an hour, thank you18:53
anteayaI guess the plugin wasn't able to find the api before, I wonder how the -dev logs timed with my comment then18:54
*** cdent has quit IRC18:55
*** salv-orl_ has joined #openstack-infra18:55
zaroanteaya: your curl requests would do it.18:56
*** salv-orlando has quit IRC18:58
anteayaI guess I miss read the timestamps18:58
anteayayet another reason I am not a great person to test things locally18:59
*** sarob has quit IRC19:01
*** Sukhdev has joined #openstack-infra19:02
*** jsavak has joined #openstack-infra19:02
openstackgerritMerged openstack-infra/project-config: Import Panko
*** piet has joined #openstack-infra19:05
*** afred312 has joined #openstack-infra19:05
*** sarob has joined #openstack-infra19:05
*** tonytan4ever has joined #openstack-infra19:09
*** pvaneck has joined #openstack-infra19:10
*** sarob has quit IRC19:10
*** zz_dimtruck is now known as dimtruck19:12
*** sarob has joined #openstack-infra19:12
*** EricGonczer_ has joined #openstack-infra19:13
*** ihrachys has quit IRC19:14
*** dimtruck is now known as zz_dimtruck19:14
openstackgerritJames Page proposed openstack-infra/project-config: Add specifications repository for charms
*** tonytan4ever has quit IRC19:14
*** mhickey has joined #openstack-infra19:15
openstackgerritJames Page proposed openstack-infra/project-config: Add specifications repository for charms
pabelangerodyssey4me: thanks19:16
*** dtardivel has quit IRC19:17
*** sarob has quit IRC19:18
jbryceover the last few weeks i’ve seen an increase in the number of emails who are getting removed from different lists i administer for excessive bounces. separately i had someone from rax reach out to me asking me if we had any of the bounce headers we could send along to their email admins to try to track down what’s happening. anyone seen or heard anything about this?19:20
therveclarkb, If you have a few minutes to look at again that'd be cool19:20
therveI just fixed the tests compared the your +219:20
*** dprince has quit IRC19:21
*** eharney has joined #openstack-infra19:22
*** esikachev has joined #openstack-infra19:22
* zaro afk for a hours19:24
*** EricGonc_ has joined #openstack-infra19:25
*** piet has quit IRC19:25
*** EricGonczer_ has quit IRC19:26
*** bhavik has joined #openstack-infra19:26
fungijbryce: yes, they have a malfunctioning filter device. ttx and i were providing feedback to them yesterday and today19:26
*** piet has joined #openstack-infra19:26
jbrycefungi: cool. thanks for the update19:26
fungijbryce: there's supposedly an internal support ticket at rackspace for it, but elarson and jroll can probably provide more recent details19:27
jrollfungi: I can, if that's helpful19:27
fungijroll: probably only helpful if there's further assistance or info you need from our end19:28
jrollfungi: oh, I'll let the people on the ticket do that, I'm staying out of it as much as possible19:28
fungiheh, good call19:28
elarsonjbryce: feel free to ping me on internal chat if you wanted to watch the ticket19:29
jrollelarson: jbryce isn't a racker :)19:30
JayFjroll: I still think my workaround is best workaround :D19:30
mordredjroll: jbryce has access to all things at all companies19:30
jrollJayF: I agree, and said as much this morning in this channel :P19:30
jrollmordred: oh, cool, where do I get that job19:30
mordredjroll: I think you have to kill jbryce first19:30
notmorganjroll: you don't want that job... you get called for all the things then19:30
mordredand then yes - what notmorgan said19:31
notmorganmordred: there can be only one?19:31
notmorgan /highlander!19:31
mordrednotmorgan:  I will assume you said that in proper sean connery accent19:31
*** Apoorva has quit IRC19:31
fungimordred: somehow you ended up +o in channel19:31
*** ddieterly has quit IRC19:31
mordredfungi: oh - yeah. I kicked the person who likes to tell us about allah the other day19:32
notmorganmordred: that and the theme song from the tv series was also playing in the background19:32
*** ChanServ sets mode: -o mordred19:32
openstackgerritgreghaynes proposed openstack/diskimage-builder: Move hook generation in to python
mordredfungi: fixed19:32
fungimordred: oh, yep. worth noting you can akick them with /msg chanserv too and don't even need to op yourself19:33
elarsonjroll: well, then he is out of luck ;)19:33
*** openstackgerrit has quit IRC19:33
jrollmordred: interestingly enough, I watched that person do that to a different network for a while before it showed up here, must have switched over after the numerous g-lines19:33
*** openstackgerrit has joined #openstack-infra19:33
*** piet has quit IRC19:33
*** akshai_ has joined #openstack-infra19:33
fungijroll: the one i saw was coming in via freenode's web client19:34
*** jsavak has quit IRC19:34
fungiwith a random guest nick19:34
*** akshai has quit IRC19:35
*** raunak has quit IRC19:35
*** oanson has quit IRC19:35
*** jsavak has joined #openstack-infra19:35
*** sdake has joined #openstack-infra19:36
*** sarob has joined #openstack-infra19:37
cbaderhello, I am trying to bring up a new commonnci system and on the first run of the sudo puppet I am see this error anyone see this before on a new install. Error: Unknown function load_module_metadata at /etc/puppet/modules/nodepool/manifests/mysql.pp:14 on node openstackci-2.rose.rdlabs.hpecorp.net19:38
*** sdake_ has joined #openstack-infra19:40
rcarrillocruzhmm, iirc there was a change lately for managing mysql on the nodepool class19:41
rcarrillocruzit was decoupled into its own class or something like that19:41
*** sarob has quit IRC19:41
cbaderrcarrillocruz, is there somewhere I can look for fix.19:41
*** sdake has quit IRC19:42
cbaderrcarrillocruz, I have the latest dump.19:42
*** ociuhandu has quit IRC19:44
pabelangerjeblair: notmorgan: zl01 and zl02 restarted19:44
notmorganpabelanger: good ot know thnx19:45
*** salv-orl_ has quit IRC19:45
*** salv-orlando has joined #openstack-infra19:45
pabelanger4 zuul-launcher > 3 jenkins masters!19:46
*** e0ne has joined #openstack-infra19:48
mordredpabelanger: we're getting quite close19:49
pabelangermordred: Yup, I think we are over the hump now. Just a matter up standing up the replacements I think19:52
krotscheck_dcmfungi: Were you ever able to dig into those segfaults?19:56
*** tkelsey has quit IRC19:56
*** jaosorior has quit IRC19:57
*** acoles is now known as acoles_19:58
*** jcoufal_ has joined #openstack-infra19:59
fungikrotscheck_dcm: ahh, nope. worth noting they seem not to have continued in dfw or iad and are now only appearing in ord19:59
*** krotscheck_dcm is now known as krotscheck20:00
*** ihrachys has joined #openstack-infra20:00
krotscheckfungi: Odd, that.20:00
krotscheckfungi: Alright- I'm spinning up a mirror vm to see if I can reproduce the package truncation we saw.20:00
openstackgerritChristian Fetzer proposed openstack-infra/jenkins-job-builder: Add support for GitLab notifier and connection property
fungikrotscheck: also, they seem to have gone away for several weeks in ord and only started back up again over the past week20:01
*** jcoufal has quit IRC20:02
krotscheckfungi: Well, the npm mirrors are still disabled.20:02
*** Sukhdev has quit IRC20:02
openstackgerritChristian Fetzer proposed openstack-infra/jenkins-job-builder: Add support for GitLab notifier and connection property
*** sarob has joined #openstack-infra20:02
*** thorst has quit IRC20:02
*** zeih has joined #openstack-infra20:03
*** asselin_ has quit IRC20:04
fungikrotscheck: here's the match frequency for logged segfaults on the ord mirror
jeblairthe only jobs running on jenkins03 and 04 are tripleo jobs.  i am going to abort them.20:05
*** gouthamr has quit IRC20:05
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: promote: add upgrade job part of voting
krotscheckfungi: Am I supposed to be getting an Internal Server Error on that link?20:05
*** thorst has joined #openstack-infra20:05
openstackgerritMerged openstack-infra/tripleo-ci: promote: add upgrade job part of voting
rookIf I want to add a Jenkins job -- is there a specific method of having that happen? I had triggers when I was on GerritHub..20:06
*** ihrachys_ has joined #openstack-infra20:06
*** zz_dimtruck is now known as dimtruck20:06
fungikrotscheck: um, nope. usually that means lodgeit lost contact wit trove20:06
fungikrotscheck: does a refresh of the page "fix" it?20:06
jeblairrook: you may want to start here:
krotscheckfungi: Huhn. It's been happening to me for a few weeks now. Clearing my cookies helped.20:06
*** somerville32 has joined #openstack-infra20:06
*** dprince has joined #openstack-infra20:07
*** sdake_ has quit IRC20:07
krotscheckfungi: Those are daily logs, yes? I wonder what happened a week ago...20:07
*** sarob has quit IRC20:07
*** sdake has joined #openstack-infra20:07
pabelangerjeblair: is there any way we can clean up /var/lib/zuul/launcher.socket on zuul-launcher stop / graceful?20:07
*** zeih has quit IRC20:07
*** Apoorva has joined #openstack-infra20:07
fungikrotscheck: oh, nope good question and i'm wrong. those are rotated arbitrarily it seems20:07
*** xarses_ has quit IRC20:08
pabelangerjeblair: thinking about an ansible playbook to restart zuul-launcher and might be able to use wait_for out of the box20:08
jeblairpabelanger: oh, yeah, we should be able to do that as part of the stop20:08
pabelangerjeblair: neat!20:08
fungikrotscheck: so krotscheck so it looks like the segfaults started up in late april20:08
krotscheckfungi: That's... right around when xenial was released.20:09
*** ihrachys has quit IRC20:09
krotscheckfungi: But well before we turned on NPM mirrors.20:09
fungikrotscheck: i wonder if that's around when we turned on xenial nodes for a few jobs though20:09
fungiclarkb: ^ if you happen to be around and remember20:09
krotscheckfungi: Could be, I was pretty quick on the xenial trigger when I found out they were available.20:10
krotscheckMight be a red herring though...?20:10
*** sarob has joined #openstack-infra20:10
*** csomerville has quit IRC20:10
openstackgerritJames E. Blair proposed openstack-infra/project-config: Remove jenkins03 and jenkins04
*** sdake_ has joined #openstack-infra20:11
*** raunak has joined #openstack-infra20:11
jeblairpabelanger, fungi, mordred: ^20:11
*** tlian has quit IRC20:11
mordredjeblair: +220:11
krotscheckfungi: On a totally unrelated note, could you add me to the ACL groups on js-openstack-lib?,access20:12
*** sdake has quit IRC20:12
openstackgerritJames E. Blair proposed openstack-infra/system-config: Decomission jenkins03 and jenkins04
*** kgiusti has left #openstack-infra20:14
pabelangerjeblair: +320:14
*** sarob has quit IRC20:15
*** matrohon has joined #openstack-infra20:16
*** ihrachys_ has quit IRC20:16
openstackgerritPaul Belanger proposed openstack-infra/system-config: Add playbook to graceful restart zuul-launchers
openstackgerritJames E. Blair proposed openstack-infra/system-config: Decomission
pabelangermordred: jeblair: I was thinking something like ^ to restart zuul-launchers20:18
pabelangerstill needs testing20:18
fungikrotscheck: done20:20
krotscheckfungi: Thanks!20:20
timothyb89krotscheck: on the topic of acls, do you want +2 for karma-subunit-reporter?20:20
*** flepied has joined #openstack-infra20:20
openstackgerritJames E. Blair proposed openstack-infra/system-config: Decomission
krotschecktimothyb89: MOAAAR POWER... but meh, sure? I'll add it to my watch list.20:21
timothyb89krotscheck: cool, thanks :)20:21
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove unused nodepool-dev manifest
*** EricGonc_ has quit IRC20:21
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/system-config: OpenStack-CI Apt mirror for Ubuntu Cloud Archive
*** EricGonczer_ has joined #openstack-infra20:23
jeblairzaro: do you need anything from
jeblairzaro: or is it okay to delete that server now20:23
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Project renames may need governance updates
jeblairinfra-root: ^ same question for jenkins-dev and jenkins.o.o20:23
*** xarses_ has joined #openstack-infra20:24
mordredjeblair: I have no use for them20:24
*** EricGonc_ has joined #openstack-infra20:24
*** eezhova has quit IRC20:24
pleia2nothing I need on those20:25
nibalizerjeblair: delete away20:25
pabelangerjeblair: neither do I20:26
openstackgerritMerged openstack-dev/pbr: Remember the insertion order for pbr.json
*** sdake_ has quit IRC20:27
*** EricGonczer_ has quit IRC20:27
anteayazaro: okay so here is the latest entry in my httpd_log on my local gerrit instance: the comment I created that triggered the httpd_log events: yet sill no activity on the story:!/story/1120:30
*** zeih has joined #openstack-infra20:30
*** dimtruck is now known as zz_dimtruck20:30
anteayazaro: I don't think the GETs and POST correspond to actual storyboard api commands20:31
anteayais the httpd_log for my local gerrit instance tracking api commands for my instance?20:32
fungijeblair: i have no files on any of those i care about20:32
*** matrohon has quit IRC20:34
*** zeih has quit IRC20:34
*** sabari has quit IRC20:36
krotschecktimothyb89: What's the project's governance? 2 x +2 to land, or....20:37
mtreinishkrotscheck: for the karma subunit thing?20:37
krotscheckmtreinish: Yep20:37
*** banix has joined #openstack-infra20:37
*** abregman is now known as abregman|afk20:38
timothyb89krotscheck: uh, good question :)20:38
*** esikachev has quit IRC20:38
mtreinishkrotscheck: it gets fast tracked with a PTL's +1. After a week ttx will just merge it assuming there aren't any objections20:38
krotscheckmtreinish: You're talking about new projects, right? I'm talking about
*** julim has quit IRC20:39
timothyb89krotscheck: heh, the actual governance patch is always waiting
openstackgerritPaul Belanger proposed openstack-infra/project-config: Fix typo with build_env
*** FallenPegasus has joined #openstack-infra20:39
timothyb89*also waiting, I mean20:39
*** FallenPegasus has quit IRC20:39
krotscheckAhhh gotcha20:39
pabelangerfungi: AJaeger: embarrassing fix ^ for wheel-build20:39
banixhad a question about a gate job in project Kuryr; The Kuryr process runs with sudo in these tests; At some point the ipdb is used to create a veth pair in the code; this operation fails at the gate; works fine locally; what is it that we are missing? Any suggestions is appreciated20:40
krotschecktimothyb89: What timezone are you in?20:40
*** esikachev has joined #openstack-infra20:40
mtreinishkrotscheck: ah different patch. For someting in the project repo I'd assume 2 +2s like everything else20:40
timothyb89krotscheck: mountain, utc-6 I think?20:40
krotschecktimothyb89: What are you doing every wednesday morning at 8AM then? ;D20:41
krotscheckTRICK QUESTION!20:41
timothyb89ooh. I will add that to my calendar20:41
* krotscheck has a shiny new meeting.20:41
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove Jenkins documentation; replace with zuul-launcher
apuimedobanix: maybe some sort of rootwrap?20:42
*** mtanino has quit IRC20:43
banixapuimedo: what are you doing here? :)20:43
apuimedobanix: spying on you20:43
apuimedoor it could be some selinux/apparmor thingie20:43
apuimedowho knows20:43
banixapuimedo: ellaborate pls; on the rootwrap not the spying20:43
banixthat’s why i am here; i suppose there are people here who would know20:44
apuimedowell, the thing is, having sudo doesn't mean you can run anything you want20:44
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove jenkins playbooks and tools
*** kzaitsev_mb has joined #openstack-infra20:45
apuimedoit depends on the sudoers configuration and the execution context20:45
apuimedoso in el7 selinux could kick your ass, for example20:45
apuimedoif you try to perform ipdb stuff on a process with the wrong label20:45
apuimedobanix: the best course of action would be to get to SSH into the jenkins slave20:46
apuimedoor obtain its image to reproduce locally20:46
fungiapuimedo: banix: by default we grant the "jenkins" or "zuul" user who runs the job ability to sudo any command they want as root, and provide a helper macro called "revoke-sudo" which jobs can include at any point in their list of builders if they want part of it to run with no sudo privs20:46
anteayazaro: so pleia2 was kind enough to look at the storyboard-dev logs again, still no POSTs20:47
apuimedofungi: and no audit tool?20:47
fungibanix: what's the job name?20:47
fungibanix: if it runs on centos 7 i believe we have selinux enabled because centos enables it by default20:47
banixfungi: gate-kuryr-dsvm-fullstack-nv20:47
fungibanix: if it runs on ubuntu trusty then i don't think there's any selinux or apparmor happening (not 100% sure on the apparmor front)20:48
jeblairapuimedo: a job can run any audit tool its authors want20:48
apuimedojeblair: I meant if the jobs are harnessed by apparmor/selinux20:48
fungibanix: job definition seems to be at
*** zz_dimtruck is now known as dimtruck20:50
fungibanix: i don't see it explicitly revoking sudo privs, but it's running in the context of devstack/devstack-gate which may differ depending on whether the command is being run by the stack user or the jenkins user or the tempest user20:50
EmilienMawesome ^20:51
fungibanix: as for which node type, it's inheriting ubuntu-trusty from so i wouldn't expect selinux to come into the picture20:51
jeblairEmilienM: woo!20:51
EmilienMreally helpful to debug jobs20:51
jeblairEmilienM: cool -- what specifically is helpful?20:51
EmilienMjeblair: well everything: you can follow how job ran, the script it used (actual code) etc20:52
*** ociuhandu has joined #openstack-infra20:52
banixfungi: and we start the Kuryr process here:
openstackgerritMerged openstack-infra/project-config: Remove jenkins03 and jenkins04
*** jamesmcarthur has quit IRC20:53
jeblairEmilienM: cool; i have a change pending to serve the .sh and .log files as text/plain20:53
mordredjeblair: are the decomission patches safe? like, can I +A them whenever or shoudl I wait for any manual tasks you're performing?20:53
jeblairmordred: all safe20:54
fungibanix: okay, so it's presumably running as the stack user. i believe there are sudo restrictions that happen in devstack but i don't recall the specifics. you might get better answers asking the devstack devs in #openstack-qa20:54
fungibanix: though also, if you have the failure logs, there may be some included system logs which could indicate whether sudo was called and refused20:54
openstackgerritJames Page proposed openstack-infra/project-config: Add specifications repository for charms
banixfungi: thanks for the help; i’ll ask there; will look into the system logs first.20:55
anteayajroll: did you realize you booked the openstack sprint channel when the openstack sprint channel was already booked?20:55
jeblairawesome!  #openstack-sprint-220:56
jeblairwhy did we not see that one coming and number the first one?20:56
anteayajroll: storyboard is having a bug sprint in #openstack-sprint on the 22nd and 23rd20:56
anteayajeblair: silly us20:56
*** acoles_ is now known as acoles20:57
anteayajroll: I mean I'm glad you are using the channel, that is the intended purpose, thank you20:57
anteayajroll: but we seem to have missed a step with the whole booking step20:57
jrollanteaya: oh no20:57
* jroll looks20:57
anteayajroll: it is okay we can work it out20:57
*** bpokorny has quit IRC20:57
anteayajroll: any thougts on how it happened?20:57
jeblairi think that means all the ironic devs have to fix storyboard bugs?20:58
anteayahey now we're talking20:58
jrollanteaya: probably me not checking things well or assuming those are in chronological order20:58
anteayaI had thought 'check existing prior to booking' was a gimme20:58
anteayayeah I can't keep cleaning up after glance20:59
jrollanteaya: I notice they were booked a day apart, maybe I only checked before I put our poll up20:59
anteayajroll: ah perhaps in any case20:59
anteayajroll: what would you like to do20:59
jrollanteaya: so, sorry for that, I'll just move it to the ironic channel instead, we'll be primarily in voice20:59
*** aysyd has quit IRC20:59
anteayano no, that is okay20:59
fungiwell, having additional sprint channels isn't a terrible idea. especially if teams doing in-person sprints also want to leverage the channel for remote participants, given that there is frequent overlap with mid-cycles and the like20:59
anteayawe can move to storyboard, which is where storyboard devs are most comfy20:59
jrollironic devs are most comfy in the ironic channel :)21:00
anteayaI'm fine with another sprint channel21:00
anteayajroll: that's fair21:00
fungii'm most comfy in pajamas but whatever suits!21:00
anteayafungi: I thought your pajamas were your suit21:00
jrollhonestly, last cycle was a bit confusing, as some people were only in ironic channel and not -sprint21:00
jrollso it won't hurt to move it21:00
*** raildo is now known as raildo-afk21:01
anteayawell I don't want you to feel displaced or anything, mostly I wanted to talk to you about it21:01
fungibenefit to using a separate channel for a sprint is if you have a lot of unrelated (support, et cetera) traffic in your usual channel21:01
jrollanteaya: it's not a problem, really :)21:01
fungimakes it easier to focus21:01
anteayanikhil: ^^ in future can you try to follow the chronologicial format of the virtual sprints page?21:01
jrollfungi: indeed, I don't think it will be a problem for us21:01
anteayanikhil: that would help things21:02
fungifor infra sprints we usually update our channel topic directing sprinters to the sprint channel and post periodic reminders when people wonder why it's so quiet21:02
anteayaI'm all for creating a new sprint channel21:02
*** rbrndt has quit IRC21:03
openstackgerritMerged openstack-infra/puppet-openstackci: Logs: serve .log and .sh files as text/plain
pabelangerjeblair: I had a question on 330082 for your comment21:03
pabelangerjeblair: I left it on the review21:04
anteayanaming? #openstack-sprint-2, #openstack-sprint-too, #openstack-sprint-better-than-the-other-one, #openstack-sprint-also, #openstack-sprint-as-well21:04
*** jsavak has quit IRC21:04
jrollI think the meeting channels has taught us that where there is a 2, there may be a 3, and so having something other than -2 is confusing :)21:05
*** jsavak has joined #openstack-infra21:05
openstackgerritJames E. Blair proposed openstack-infra/zuul: Ansible launcher: use .txt as logfile extension
*** jsavak has quit IRC21:05
*** jsavak has joined #openstack-infra21:05
*** javeriak_ has quit IRC21:05
anteayajroll: okay I count that as a +1 for -221:05
*** rfolco has quit IRC21:06
jeblairpabelanger: replied21:06
*** ociuhandu has quit IRC21:06
*** ociuhandu_ has joined #openstack-infra21:06
*** ociuhandu_ is now known as ociuhandu21:06
dmsimardjeblair: Hey, curious about something for zuul v2.5/v3. When a shell script is run in jenkins, you can see the output streamed in real time in the console. However, Ansible does not stream the output of a task, it only prints the results when it is complete.21:06
pabelangerjeblair: Thanks, make sense now21:06
dmsimardDo you still have that limitation around the task output not being streamed ?21:06
openstackgerritMerged openstack-infra/system-config: Decomission jenkins03 and jenkins04
jrollanteaya: indeed :)21:07
*** FallenPegasus has joined #openstack-infra21:07
openstackgerritMerged openstack-infra/system-config: Decomission
dmsimardjeblair: this is especially a problem with long running tasks (say, installing openstack takes >30 minutes and could be inside just one task)21:08
openstackgerritMerged openstack-infra/system-config: Decomission
*** esikachev has quit IRC21:08
nikhilanteaya: when I posted my first entry it was on the right plaec (iirc). I agree that the next one is just a follow on. someone is moving the proposed topic to previous sprints too. would help if you've a history of the edits.21:08
jeblairdmsimard: run "nc 8088" to see something cool :)21:08
*** thorst has quit IRC21:08
phschwartzHas anyone seen an issue with puppet-diskimage_builder? When attempting to update my nodepool hosts I am geting a HTTP-Error 403 you must use HTTPS instead of HTTP. It was working till this morning.21:09
jeblair(or "telnet 8088")21:09
openstackgerritMerged openstack-infra/system-config: Remove unused nodepool-dev manifest
nikhilanteaya: also the top of the wiki says 'reverse chronological order'21:10
jeblairdmsimard: we lost that ability when we firewalled our jenkins servers for security reasons.  getting that back was one of the motivating factors for writing zuulv2.521:10
jeblairdmsimard: we accomplish it with a special ansible module that we use instead of 'shell' -- it captures stdout/stderr and streams it to a file which another process serves out like that21:11
jeblairdmsimard: once all of the jenkins masters are gone, we will be able to expose that connection information so that anyone can stream the logs for their jobs in a terminal21:11
* fungi is particularly thrilled to be able to check in-progress job console logs without needlessly involving a web browser21:12
jeblairdmsimard: and mordred and notmorgan are working on building on top of that and getting us websocket based streaming for folks who, unlike fungi, do prefer the browser :)21:12
dmsimardjeblair: that's sexy, I can definitely see a zuul cli client to see without crashing my browser21:12
jeblairbut we'll have both :)21:12
*** fernnest__ has quit IRC21:12
*** dprince has quit IRC21:13
jeblairdmsimard: the status page is built from  so it's pretty easy to automatically parse it21:13
dmsimardjeblair: this is the module
* fungi still maintains that "the internet" is more than just "the world-wide web"21:13
dmsimardjeblair: yeah, I came close to writing a CLI interface to status.json a couple times when zuul was too busy21:13
jeblairdmsimard: yeah, that's the server part of it, and is the 'shell' part of it21:13
fungidmsimard: i think we all have21:13
dmsimardjeblair: That's very clever. What does it look like on multi node ? Is the output intertwined ?21:14
fungidmsimard: i have a hacky one with some ansi coloring i can use to check what changes are in which pipelines and how long they're estimated to take (useful if you're trying to pick a good time to do disruptive things to the ci)21:14
jeblairdmsimard: i do not know exactly how we will fit this into zuul v3 yet.21:14
*** fernnest has quit IRC21:14
*** e0ne has quit IRC21:14
jeblairdmsimard: that too is a better question for v3, since there's no inherent multinode in v221:14
*** bpokorny has joined #openstack-infra21:14
dmsimardI think the whole "streaming output of a task" was brought up a couple times in upstream Ansible but iirc they are against it because it doesn't make sense on more than one host21:15
*** fernnest_ has quit IRC21:15
*** ldnunes has quit IRC21:15
*** trown is now known as trown|outtypewww21:15
fungidmsimard: current multi-node design in devstack-gate just dumps the subnode ansible stdout to another file if memory serves, so the situation would still be the same for zuul v2.521:15
jeblairbut i know that i want something like this in v3, and not only support for multiple nodes, but also for multiple files (not just the 'console' log)21:15
nikhilanteaya: in either case, I did my part and moved the glance items. thanks.21:16
dmsimardyeah, I guess some compromise were required to be able to transition away from jenkins21:16
dmsimardgood work :)21:16
dmsimardthose are the upstream issues I was talking about: &
asselinhey folks, introduced load_module_metadata which is causing Error: Unknown function load_module_metadata at /etc/puppet/modules/nodepool/manifests/mysql.pp:1421:17
fungidmsimard: i would be happy to pitch in on a more robust console-based zuul client once v3 is in production21:17
asselinanyone konw how to get that to work?21:17
*** rhallisey has quit IRC21:17
*** eezhova has joined #openstack-infra21:17
*** mhickey has quit IRC21:18
notmorganjeblair: ++21:18
*** rockyg has joined #openstack-infra21:19
mordredjeblair, dmsimard: for streaming for multi-host - we _might_ consider encoding the stream from each server into subunit - since it is designed to do interleaved streaming21:19
mordredbut I obviously agree, that's a v3 design chat21:20
jeblairmordred: i don't think the encoding system is the problem, i think the problem is that ansible has no mechanism to obtain the data21:20
*** thorst has joined #openstack-infra21:20
*** abregman|afk has quit IRC21:20
mordredoh, indeed21:20
jeblairand yeah, i seriously doubt we will solve it right now21:20
*** rbrndt has joined #openstack-infra21:21
*** Sukhdev has joined #openstack-infra21:22
asselinnevermind, that was just moved from
nibalizerasselin: you need a newish stdlib21:24
nibalizerto have load_module_metadata21:24
asselinnibalizer, how 'new'? I'm using from system-config21:25
*** thorst has quit IRC21:25
nibalizeri don't know exactly....21:25
nibalizerim kinda chasing this latest pip/puppet weirdness21:25
openstackgerritMerged openstack-infra/project-config: Fix typo with build_env
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove jenkins playbooks and tools
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove Jenkins documentation; replace with zuul-launcher
*** sarob has joined #openstack-infra21:26
pabelangerjeblair: too bad we didn't have access to the name of the builder element in JJB, would have been nice to add that information to our ansible output21:27
jeblairpabelanger: i agree.  *macros* have names, but they lose them when applied, and not all builders are macros21:28
jeblairwe could add it, but that's nontrivial.  better to zuulv3 instead.21:28
*** bpokorny has quit IRC21:28
*** jamesmcarthur has joined #openstack-infra21:29
*** mdrabe has quit IRC21:29
*** bpokorny has joined #openstack-infra21:29
*** banix has quit IRC21:29
pabelangerjeblair: Ya, zuulv3 removes the need for it21:29
asselincrinkle, do you know which puppet-stdlib is needed for load_module_metadata?
*** jamesmcarthur has quit IRC21:31
nibalizerasselin: all you have to do is look at the git history or changelog21:31
nibalizerto determine that21:31
*** jamesmcarthur has joined #openstack-infra21:31
asselinI see: depends on:
anteayanikhil: yes, I move expired sprints to the previous sprints section21:31
*** bpokorny_ has joined #openstack-infra21:31
*** sarob has quit IRC21:32
*** Sukhdev has quit IRC21:32
*** thorst has joined #openstack-infra21:32
anteayanikhil: thank you for editing the page21:32
anteayanikhil: anyone can move expired sprints to the previous sprints section21:33
*** ccamacho|out has quit IRC21:33
anteayajroll: so if I make a new sprint channel well you use #openstack-sprint?21:33
*** Sukhdev has joined #openstack-infra21:33
jrollanteaya: idk, I'm okay with just using the ironic channel to avoid confusion21:34
*** bpokorny has quit IRC21:34
anteayajroll: thanks for chatting with me about this21:35
jrollanteaya: you're welcome21:35
nibalizerso I think all our puppet runs are failing?21:35
nibalizercan someone who has root go look at the logs on puppetmaster.o.o ?21:35
nibalizer(I lack the ssh key at the moment)21:36
*** rossella_s has joined #openstack-infra21:36
*** thorst has quit IRC21:36
*** Sukhdev has quit IRC21:36
funginibalizer: failing how recently?21:36
nibalizerlast couple hours?21:36
nibalizer a lot of unreported nodes, which I think we decided was code for 'failing'21:37
nibalizeralso phschwartz discovered an issue downstream where the pip provider for puppet jus straight stopped functioning21:37
nibalizerfortunately crinkle 's provider just works21:37
funginibalizer: i saw a merged project creation patch take an hour or more to complete the circuit and show up, so we may have ansible timeouts to something unreachable again21:37
nibalizer is the error i see21:38
phschwartznibalizer: fungi: my issue appears to be puppet 3.8.7 with the puppet-jenkins, puppet-diskimage_builder, and a few others that use provider => pip21:39
nibalizereasliy reproduced by just trying to puppet apply -e 'package { "diskimage-builder": ensure => latest, provider => pip } '21:39
*** bpokorny_ has quit IRC21:39
funginibalizer: it looks (tailing the run_all log) to be working21:40
*** bpokorny has joined #openstack-infra21:40
funginibalizer: oh, i wonder if recently deleted jenkins masters are slowing things up because of ansible timeouts trying to reach them?21:40
fungipresumably refreshing the inventory will solve that21:41
jeblairpossibly, i'm in the process of that21:41
*** acoles is now known as acoles_21:41
jeblairand also launching zl05-zl0721:41
fungiin the process of refreshing the inventory cache, or just in the process of deleting servers? want me to hold off refreshing the inventory?21:42
asselinnibalizer, ok, forced a resync of system-config and all the modules and it's happy now. I guess there was a clone error or something that broke first time around.21:42
jeblairfungi: all of the above21:42
fungioh, launching new instances will refresh the cache anyway21:42
*** burgerk has quit IRC21:42
*** gyee has joined #openstack-infra21:42
fungithough i am seeing plenty of failures21:42
fungithings like...21:42
openstackgerritAndrea Frittoli proposed openstack-infra/project-config: Start using pre-prov creds in dsvm-neutron-full
jeblairthe launches are complete now, so the inventory should be ok.  we may need to kill stuck processes though21:42
fungi2016-06-15 21:41:56,828 p=25059 u=root |  fatal: []: FAILED! => {"changed": false, "failed": true, "msg": "/usr/bin/timeout -s 9 30m /usr/bin/puppet apply --detailed-exitcodes --logdest syslog --no-noop /opt/system-config/production/manifests/site.pp failed with return code: 6", "rc": 6, "stderr": "", "stdout": "", "stdout_lines": []}21:42
fungii'll take a look on that server to see what ended up in its syslog21:43
fungioh, it's the same thing asselin was just asking about21:43
fungiJun 15 21:41:48 proposal puppet-user[28182]: Could not get latest version: HTTP-Error: 403 Must access using HTTPS instead of HTTP21:44
openstackgerritAndrea Frittoli proposed openstack-infra/project-config: Start using pre-prov creds in dsvm-layer4
*** thorst has joined #openstack-infra21:44
jeblairit looks like the only active ansible proc is on static.o.o21:44
phschwartzfungi: that is the exact error I am getting21:44
nibalizerasselin: cool21:44
phschwartzfungi: those HTTP-Error: 403's are caused by provider => pip21:45
fungiso we've come full-circle to nibalizer's comment about the upstream pip provider vs crinkle's working one21:46
jeblairi'm all done with the server launch/deletion now21:46
nibalizerso i think the fix is to spam provider openstack_pip everywhere21:46
phschwartzfungi: yup. I have to run to get dinner for the prego wife, when I am home I will put up reviews swapping all the prover => pip to use crinkle's21:46
nibalizeralso my suspicion is that pypi changed how it's remote api works21:47
phschwartznibalizer: it looks like they just 403 now instead of a redirect21:47
*** bauzas has quit IRC21:47
funginibalizer: dstufft could probably fill us in on any api changes on pypi21:47
*** sputnik13 has quit IRC21:47
nibalizerfungi: yea i just want to avoid being like 'explain what you changed today'21:48
openstackgerritAndrea Frittoli proposed openstack-infra/project-config: Start using pre-prov creds in dsvm-layer4
crinklenot that it helps right now but in puppet 4 they fixed the upstream pip provider to not use the silly xml api21:48
nibalizerwhen I don't really understand what broke21:48
*** eezhova has quit IRC21:48
*** PollyZ_ has quit IRC21:48
*** thorst has quit IRC21:48
asselinsetting up 3rd party ci has this 403 error:
dstufftfungi: unless you're using a new enough puppet it uses XMLRPC, for various reasons we're blocking HTTP access to APIs21:49
dstufftfungi: going to go finish dinner, but can talk more in a bit later21:49
nibalizerdstufft: ok so puppet-pip is broken for anyone who is on puppet 3 then21:49
fungidstufft: oh, because of the urllib/urllib2 issue posted today. got it21:49
*** sputnik13 has joined #openstack-infra21:49
*** matt-borland has quit IRC21:49
phschwartzdstufft: sounds about right21:49
phschwartzfungi: do you want me to go ahead and do all the reviews to swap to openstack_pip to fix the issue for the forseable future?21:50
*** thorst has joined #openstack-infra21:50
fungi for reference21:50
nibalizeri don't really understand21:50
openstackgerritAndrea Frittoli proposed openstack-infra/devstack-gate: Throw-away change to test test-accounts
*** bauzas has joined #openstack-infra21:51
nibalizerthe apis we are using in puppet are not turned off though21:52
*** _ari_ has quit IRC21:52
nibalizerby changing puppet to use https, it works21:52
phschwartznibalizer: yeah, it looks like the redirector that was being used on pypi swapped to deny http instread of redirecting it like it did before related to that urllib exploit21:53
nibalizeroh awesome21:53
fungior at least that's my completely uninformed assumption21:53
phschwartznibalizer: that exploit was fully disclosed mid day today so it matches when we saw the issue start. (at least time wise)21:53
phschwartzfungi: I think you are close on it. Looking at old logs the HTTP call redirected which it now just 403's21:54
*** thorst_ has joined #openstack-infra21:54
*** thorst has quit IRC21:54
phschwartzwell off to dinner, back soon.21:54
*** nikeshm has quit IRC21:54
phschwartzwhen back if we want it, I will put the reviews up to move to crinkle's provider21:55
fungii'm happy with that as a solution. i think it exists because we wanted to use it anyway?21:55
fungii mean, it's a lot of change churn along with a review scramble, but it's changes we would eventually have wanted anyway?21:55
zarojeblair: i don’t need anything from jenkins-dev.o.o21:57
crinklemostly we wanted it because we wanted beaker to be using the mirrors21:57
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add zl05 - zl07
jeblaircrinkle: yeah.  i suppose that's still generally useful though (like, if someone wanted to use a site-local mirror for their pip installs)21:57
crinklejeblair: agreed21:58
*** thorst_ has quit IRC21:59
*** weshay has quit IRC22:00
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove jenkins playbooks and tools
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove Jenkins documentation; replace with zuul-launcher
nibalizerI had expected us to use the openstack_pip provider everywhere eventually22:01
*** hashar has quit IRC22:01
nibalizersince i'm sure it will be a while before we are on puppet 422:01
*** esikachev has joined #openstack-infra22:01
*** mriedem has quit IRC22:01
zaroanteaya: so is the storyboard plugin i provided loaded ok?22:02
*** inc0 has quit IRC22:02
*** jsavak has quit IRC22:02
*** openstackgerrit has quit IRC22:02
*** xyang1 has quit IRC22:03
*** notnownikki has quit IRC22:04
*** yamahata has quit IRC22:04
anteayathere were no errors in the error log22:04
zarodo you see it as loaded in the UI?22:05
anteayahang on22:05
*** openstackgerrit has joined #openstack-infra22:05
anteayawhat do you mean loaded in the UI22:05
anteayawhere do I look?22:05
fungipleia2: bad news. i missed it looking over 287056, but we'll need to convert our config before we can run that22:06
anteayazaro: I found it in admin plugings22:06
pabelangerjeblair: zl01 seems to be growing ready nodes, but I don't understand why at the moment22:06
*** esikachev has quit IRC22:06
anteayait says it is enabled, but it gets that from the config22:06
anteayazaro: I don't see loaded on that page22:06
pleia2fungi: ah, fun22:07
fungipleia2: i don't have time to translate the config from ini to yaml this evening, but can take a stab at it tomorrow unless you wanted to hack on that22:07
pleia2fungi: I'll see what I can do22:07
pleia2I'm out most of the day tomorrow, but I have some time now22:07
dstufftfungi: nibalizer so basically, in a not entirely public yet thing we discovered something that's going to require us to do a password reset at some point in the near future as a precaution, we decided to use this password reset to block HTTP access to "apis" (python2.6 for instance will upload to PyPI over plaintext if you don't use twine) and fold in reseting authors who might have uploaded over HTTP in that same reset22:08
nibalizerdstufft: i'd love to know what was changed22:08
nibalizeroh cool22:08
fungiokay, cool. i'm around... sort of. as long as my wife doesn't catch me getting back on the computer this evening when i meant to work on stuff around the house ;)22:08
*** jcoufal_ has quit IRC22:09
zaroanteaya: i think that lets us know that it’s installed ok.22:09
*** Sukhdev has joined #openstack-infra22:09
nibalizerso maybe the problem is just as simple as the ruby xml client library not following the redirect?22:09
fungidstufft: thanks for the update!22:09
*** Sukhdev has quit IRC22:09
pleia2fungi: haha, enjoy your evening ;)22:09
nibalizerbecuase I do get a redirect when I curl ""22:09
anteayazaro: here is the contents of the error log since I installed the new plugin build:
zaroanteaya: hmm, do you see anything in the log when you comment on a change?22:10
dstufftnibalizer: we redirect "Web UI" pages from HTTP to HTTPS but we hard fail on "API" responses22:10
anteayazaro: which log?22:10
*** Sukhdev has joined #openstack-infra22:10
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/system-config: OpenStack-CI Apt mirror for Ubuntu Cloud Archive
anteayaI see activity in the httpd log as I linked above22:10
anteayazaro: here it is again:
dstufftbecause the former are generally humans which might be typing into a browser (and once they get redirected once, their browser will get our HSTS setting and force them onto hTTPS in the future), but the latter are generally hard baked into programs (like puppet!) and we want them to update their URLs from http://.../ to https://../22:11
nibalizerokay so if Im setting accept-headers to content-xml or whatever then im classified as an api and I get the boot?22:11
zaroanteaya: looking for activity in the error log22:11
anteayazaro: I just gave you the error log22:11
nibalizerdstufft: is a really easy fix for puppet core22:11
dstufftnibalizer: yea, I think it's content-xml that triggers that specificlly22:11
anteayazaro: and here is the activity from the storyboard-dev logs:
dstufftnibalizer: yea, without looking at what exactly they're doing with the details from XMLRPC that's likely fixing a security bug too :)22:12
*** thiagop has quit IRC22:12
*** Sukhdev has quit IRC22:15
dstufftfungi: nibalizer we're going to be giving X number of days for people to change their own password before we force a reset, most like X will be 30 days once we actually do the notification. Anyone who changes their PW between now and then won't be reset.22:17
zaroanteaya: can you please provide gerri.config and actions.config ?22:17
openstackgerritMerged openstack-infra/system-config: Add zl05 - zl07
nibalizerdstufft: ok, thanks for the heads up22:18
nibalizerchanging now is ok though?22:18
dstufftnibalizer: correct22:19
nibalizerdstufft: i r bad at curl22:19
nibalizerI can't actually get it to xml: curl -L -H "Accept: application/xml" -H "Content-Type: application/xml" -X GET  ""22:19
fungidstufft: thanks! we'll make sure to get our openstackci account updated asap to avert breakage22:20
*** jamesmcarthur has quit IRC22:20
dstufftI added a password_date field with all existing accounts set to NULL with an UPDATE trigger to set that to NOW()22:20
dstufftwe'll reset anyone who is still NULL22:20
fungi(and my personal account for sure)22:20
dstufftnibalizer: might have to be POST22:20
*** thorst_ has joined #openstack-infra22:21
dstufftnibalizer: oh, it's also text/xml I guess?22:22
dstufftthis code is double plus ungood22:22
openstackgerritBen Nemec proposed openstack-infra/project-config: Add stable branch jobs for tripleo-ci
*** rlandy has quit IRC22:24
*** salv-orlando has quit IRC22:25
zaroanteaya: does the commit message in your change contain a comment link to the SB story?22:25
*** thorst_ has quit IRC22:25
*** roxanaghe has quit IRC22:26
*** salv-orlando has joined #openstack-infra22:26
anteayazaro: the commit message on my local gerrit patch I am commenting on:
*** yamahata has joined #openstack-infra22:27
*** roxanaghe has joined #openstack-infra22:28
zaroanteaya: i assume the link takes you directly to the story?22:28
anteayaoh it doesn't link22:28
anteayaI don't know what functionality makes Story: <story id> link to a storyboard story22:29
*** jamesmcarthur has joined #openstack-infra22:29
zarocolon probably not needed. try ‘Story 11'22:29
anteayanew patch or edit this commit message?22:29
anteayaand colon is used in revew.o.o and it links22:29
*** roxanaghe has quit IRC22:30
anteayamgagne: thanks22:30
*** roxanaghe has joined #openstack-infra22:30
zaroreview.o.o might have different matcher22:30
mgagnewhich is used by
*** vhosakot has quit IRC22:31
anteayaokay it links now22:31
anteayaand the link takes me to the right story22:31
anteayahowever a new comment still shows no new activity in the story that I can see22:32
*** sergmelikyan has joined #openstack-infra22:34
anteayano new activity in the error log on the local gerrit instance22:34
*** sergmelikyan has quit IRC22:34
anteayazaro: I'm assuming my gerrit.config and action.config look okay22:39
*** edmondsw has quit IRC22:40
*** Sukhdev has joined #openstack-infra22:42
*** rossella_s has quit IRC22:44
*** Sukhdev has quit IRC22:44
*** banix has joined #openstack-infra22:44
*** Sukhdev has joined #openstack-infra22:46
anteayamorning jhesketh22:47
*** FallenPegasus has quit IRC22:49
openstackgerritThanh Ha proposed openstack-infra/jenkins-job-builder: Allow update to provide plugin_info via yaml
openstackgerritElizabeth K. Joseph proposed openstack-infra/puppet-gerritbot: Migrate gerritbot config from ini to yaml
pleia2fungi: ^^ that gerritbot config should work22:51
*** rockyg has quit IRC22:52
*** mriedem has joined #openstack-infra22:52
openstackgerritElizabeth K. Joseph proposed openstack-infra/puppet-gerritbot: Migrate gerritbot config from ini to yaml
openstackgerritJames E. Blair proposed openstack-infra/zuul: Only update the time database if the build has a duration
*** dimtruck is now known as zz_dimtruck22:53
*** Sukhdev has quit IRC22:53
*** gouthamr has joined #openstack-infra22:54
*** jcoufal has joined #openstack-infra22:55
*** hongbin has quit IRC22:56
*** EricGonczer_ has joined #openstack-infra22:57
jeblairnotmorgan: comments on
*** EricGonc_ has quit IRC23:00
zaroanteaya: sorry, had a phone call come in.23:02

Generated by 2.14.0 by Marius Gedminas - find it at!