| *** piet_ has quit IRC | 00:00 | |
| *** kzaitsev_mb has quit IRC | 00:01 | |
| *** kzaitsev_mb has joined #openstack-infra | 00:03 | |
| *** jamesmcarthur has joined #openstack-infra | 00:06 | |
| *** thorst has joined #openstack-infra | 00:06 | |
| *** zz_dimtruck is now known as dimtruck | 00:07 | |
| *** xarses has joined #openstack-infra | 00:10 | |
| openstackgerrit | Merged openstack-infra/system-config: Upgrade infracloud modules to mitaka https://review.openstack.org/304946 | 00:10 |
|---|---|---|
| *** jamesmcarthur has quit IRC | 00:10 | |
| *** thorst has quit IRC | 00:10 | |
| *** yamahata has quit IRC | 00:14 | |
| openstackgerrit | Merged openstack-infra/puppet-infracloud: Upgrade to Mitaka https://review.openstack.org/312319 | 00:17 |
| *** rcernin has joined #openstack-infra | 00:17 | |
| *** ddieterly has joined #openstack-infra | 00:17 | |
| *** sdake has quit IRC | 00:18 | |
| *** bhavik1 has joined #openstack-infra | 00:18 | |
| *** bhavik has quit IRC | 00:21 | |
| *** bhavik1 is now known as bhavik | 00:21 | |
| *** rcernin has quit IRC | 00:22 | |
| *** sarob has quit IRC | 00:30 | |
| clarkb | lifeless: have you seen http://lists.openstack.org/pipermail/openstack/2016-June/016430.html it is on the general ml not sure if yo usubscribe | 00:31 |
| lifeless | clarkb: I put some filters in place - ~= unsubscribed | 00:32 |
| *** david-lyle has quit IRC | 00:32 | |
| lifeless | clarkb: that version support was removed from pbr in uhm 0.11 ? | 00:32 |
| lifeless | clarkb: when PEP-440 because enforced by pip/pypi | 00:32 |
| lifeless | clarkb: pbr puts the hash in the pbr.json | 00:32 |
| *** tqtran has quit IRC | 00:33 | |
| *** bhavik has quit IRC | 00:33 | |
| *** sputnik13 has quit IRC | 00:34 | |
| *** ddieterly has quit IRC | 00:35 | |
| clarkb | yes we know and that was explained | 00:35 |
| clarkb | but supposedly pep440 supports +things suffixes and pbr doesn't? I haven't paged all that in | 00:35 |
| lifeless | clarkb: we used + for prefix versions during the period where 0.11 wasn't being used by everyone and point releases of 0.10 had to adjust to PEP-440 things | 00:36 |
| lifeless | clarkb: or maybe that wasn't it. | 00:36 |
| lifeless | clarkb: anyhow, I do want to support + local versions, but really want to make sure its tied into nova's custom version overrides | 00:37 |
| lifeless | clarkb: because we only get one shot at doing it. | 00:37 |
| lifeless | clarkb: I've asked repeatedly for someone interested to write a spec :) | 00:37 |
| clarkb | nova's custom version overrides? | 00:37 |
| *** lvdongbing has joined #openstack-infra | 00:37 | |
| lifeless | clarkb: http://git.openstack.org/cgit/openstack/nova/tree/nova/version.py#n84 | 00:38 |
| lifeless | clarkb: its a thing which allows changing the version nova reports by setting a config setting | 00:39 |
| lifeless | clarkb: which distro's apparently use to make their openstacks report that they are from them | 00:39 |
| clarkb | wow | 00:39 |
| lifeless | clarkb: yes | 00:39 |
| *** tkelsey has joined #openstack-infra | 00:41 | |
| *** dimtruck is now known as zz_dimtruck | 00:41 | |
| *** mtanino has quit IRC | 00:41 | |
| clarkb | on rereading pep440 there is support for local version specifiers but they are not public | 00:42 |
| lifeless | right | 00:42 |
| lifeless | so its fine for local builds | 00:42 |
| clarkb | maybe pbr should accept them if the version is set directly as in that thread but not generate them automagically | 00:43 |
| lifeless | not for redhat or ubuntu or whatever | 00:43 |
| clarkb | right | 00:43 |
| lifeless | possibly | 00:43 |
| lifeless | fundamentally though, its gnarly, we need to have documentation and use cases and clearly articulate how these things interact - or not | 00:43 |
| lifeless | I don't want to mess around with backwards compat because we haven't actually had the discussion - which is what I want from a spec here | 00:44 |
| lifeless | something we can walk around all the distros | 00:44 |
| lifeless | and operators | 00:44 |
| *** Sukhdev has joined #openstack-infra | 00:44 | |
| *** Sukhdev has quit IRC | 00:45 | |
| *** tkelsey has quit IRC | 00:45 | |
| *** amitgandhinz has joined #openstack-infra | 00:45 | |
| *** csomerville has quit IRC | 00:46 | |
| dstufft | lifeless: clarkb local versions are fine for Red Hat and such too, it's generally for "things which are not upstream to indicate they're not really version X", without it you're stuck either pretending upstream "1.0" and downstream/local "1.0" are the same thing (which they aren't) or doing something like "1.0.0.0.0.1" and hoping upstream never releases a version of that | 00:47 |
| *** csomerville has joined #openstack-infra | 00:47 | |
| dstufft | conceptually similar to -ubuntu1 or w/e in an ubuntu version | 00:47 |
| *** zul_ has joined #openstack-infra | 00:47 | |
| lifeless | dstufft: well, fine if they don't put them in pypi | 00:48 |
| dstufft | yea | 00:48 |
| lifeless | dstufft: so context is pretty important | 00:48 |
| dstufft | (PyPI won't allow you to upload a +version) | 00:48 |
| dstufft | yea that's true | 00:48 |
| *** rcernin has joined #openstack-infra | 00:48 | |
| lifeless | dstufft: and thus why I want that broad discussion, rather than 3-4 people deciding without knowing what people do with e.g. the nova version override | 00:48 |
| *** roxanagh_ has joined #openstack-infra | 00:49 | |
| dstufft | I was assuming Redhat in the "redhat took something from PyPI and shipped it as an rpm, maybe with some patches" | 00:49 |
| dstufft | lifeless: yea, reasonable. | 00:49 |
| lifeless | I do try ;) | 00:49 |
| *** csomerville has quit IRC | 00:50 | |
| *** amitgandhinz has quit IRC | 00:50 | |
| *** thorst has joined #openstack-infra | 00:53 | |
| *** roxanagh_ has quit IRC | 00:53 | |
| *** haaj has quit IRC | 00:53 | |
| *** rcernin has quit IRC | 00:53 | |
| clarkb | lifeless: any chance you are able to respond to that thread with a summary of this "need spec, nova does funny things" | 00:53 |
| lifeless | clarkb: oh, there's a whole thread? Let me read through | 00:54 |
| *** weshay_mtg has quit IRC | 00:55 | |
| *** bpokorny_ has joined #openstack-infra | 00:57 | |
| *** ddieterly has joined #openstack-infra | 00:59 | |
| *** bpokorny has quit IRC | 01:00 | |
| *** bpokorny_ has quit IRC | 01:01 | |
| lifeless | clarkb: no, I cannot respond | 01:04 |
| lifeless | You have to be a subscriber to post to this mailing list, so your | 01:04 |
| lifeless | message has been automatically rejected. You can subscribe at: | 01:04 |
| lifeless | http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack | 01:04 |
| *** zhurong has joined #openstack-infra | 01:06 | |
| *** rcernin has joined #openstack-infra | 01:06 | |
| *** weshay_mtg has joined #openstack-infra | 01:07 | |
| *** pvaneck has quit IRC | 01:08 | |
| *** rcernin has quit IRC | 01:10 | |
| *** bhavik has joined #openstack-infra | 01:12 | |
| *** claudiub has quit IRC | 01:14 | |
| *** ddieterly has quit IRC | 01:15 | |
| *** kzaitsev_mb has quit IRC | 01:16 | |
| lifeless | clarkb: I have replied on https://review.openstack.org/#/c/327424/1 in more etail though | 01:17 |
| *** Daisy has joined #openstack-infra | 01:22 | |
| *** EricGonczer_ has joined #openstack-infra | 01:22 | |
| *** kushal has quit IRC | 01:25 | |
| *** Apoorva has quit IRC | 01:25 | |
| *** sdake has joined #openstack-infra | 01:28 | |
| *** Apoorva has joined #openstack-infra | 01:28 | |
| *** Apoorva has quit IRC | 01:28 | |
| *** tqtran has joined #openstack-infra | 01:30 | |
| *** tphummel has quit IRC | 01:32 | |
| *** sdake has quit IRC | 01:32 | |
| clarkb | jhesketh: I am looking at https://github.com/openstack/diskimage-builder/commit/f40749f71c7eebb1190f78c5489e48f39c909c63#commitcomment-17871241 because github emailed me and I am | 01:33 |
| clarkb | confuhttps://github.com/openstack/diskimage-builder/commit/f40749f71c7eebb1190f78c5489e48f39c909c63#commitcomment-17871241https://github.com/openstack/diskimage-builder/commit/f40749f71c7eebb1190f78c5489e48f39c909c63#commitcomment-17871241sed.https://github.com/openstack/diskimage-builder/commit/f40749f71c7eebb1190f78c5489e48f39c909c63#commitcomment-17871241 At first I wondered if we started | 01:33 |
| clarkb | reviewing gerrit changes in github but then trying to view that change on gerrit doesn't show anything | 01:33 |
| clarkb | wow fail copy paste | 01:33 |
| clarkb | let me try again | 01:33 |
| clarkb | https://github.com/openstack/diskimage-builder/commit/f40749f71c7eebb1190f78c5489e48f39c909c63#commitcomment-17871241 is the thing | 01:34 |
| *** tqtran has quit IRC | 01:34 | |
| jhesketh | clarkb: tonyb and I are doing a thing where we fail in private before we push it up to gerrit to fail more publicly | 01:34 |
| jhesketh | clarkb: in other words, it's like pre-review :p | 01:34 |
| jhesketh | interesting that it pinged you | 01:34 |
| clarkb | but how do you get the thing on openstack/diskimage-builder? | 01:35 |
| clarkb | jhesketh: it pinged me because ^ that repo was updated | 01:35 |
| jhesketh | that part is interesting, I just noticed that too | 01:35 |
| jhesketh | clarkb: so we should have been working on https://github.com/tbreeds/diskimage-builder/commit/f40749f71c7eebb1190f78c5489e48f39c909c63 | 01:36 |
| jhesketh | but somehow through the github ui the comments made it onto openstack/diskimage-builder | 01:36 |
| *** thorst has quit IRC | 01:36 | |
| notmorgan | clarkb: had a nice chat with lifeless, going to hopefully have more fixes for capturing logs from zuul (etc) soon. *and* do some "threads didn't die nicely" warnings so we can fix the "threads dont exit right" problem we have [cc mordred jeblair ] | 01:36 |
| clarkb | jhesketh: well the commit made it there too | 01:36 |
| notmorgan | also jhesketh o/ Mornin :) | 01:37 |
| clarkb | jhesketh: which I think is my biggest concern, unless github is being really weird | 01:37 |
| jhesketh | why it exists there is a weird githubism | 01:37 |
| clarkb | jhesketh: so that commit isn't actually pushed into the repo? | 01:37 |
| jhesketh | clarkb: it's not in the repo, no | 01:37 |
| jhesketh | well not that I can see having just checked | 01:37 |
| *** thorst has joined #openstack-infra | 01:37 | |
| mordred | _weird_ | 01:37 |
| jhesketh | notmorgan: o/ | 01:37 |
| clarkb | git fetch confirms, /me adds this to additional thing to dislike about github | 01:39 |
| notmorgan | clarkb: there are so many these days.... | 01:39 |
| notmorgan | clarkb: i stoped "tracking" all of mine. | 01:40 |
| clarkb | I don't think I would care if it didn't mean I got emailed | 01:40 |
| clarkb | but since I do get emailed about it ... | 01:40 |
| notmorgan | clarkb: fair enough | 01:40 |
| jhesketh | clarkb: so you can view the commit on any fork it seems (eg https://github.com/redhat-openstack/diskimage-builder/commit/f40749f71c7eebb1190f78c5489e48f39c909c63#commitcomment-17871241) - even when it's behind master | 01:41 |
| clarkb | wow | 01:41 |
| *** baoli has joined #openstack-infra | 01:41 | |
| jhesketh | somehow I browsed onto the commit in the openstack repo while doing a diff between branches | 01:41 |
| clarkb | and do all fork owners get emails too Iwonder | 01:41 |
| *** wc2912 has quit IRC | 01:41 | |
| jhesketh | and inadvertently commented there sorry :-s | 01:41 |
| clarkb | ah ok | 01:41 |
| * clarkb understands now | 01:41 | |
| mordred | yah. that makes sense | 01:41 |
| mordred | other than being able to do it | 01:41 |
| clarkb | well | 01:41 |
| mordred | which not so much with the making sense | 01:42 |
| notmorgan | mordred: lol | 01:42 |
| clarkb | except for the part where refs not in a repo show up in that repo | 01:42 |
| mordred | yah | 01:42 |
| mordred | that part not so much | 01:42 |
| jhesketh | clarkb: I suspect it's just the particular fork that gets notified | 01:42 |
| notmorgan | clarkb: welcome to github. it's magical! :P | 01:42 |
| mordred | but given that - jhesketh commenting there emailing you does make sense | 01:42 |
| clarkb | I wonder if this si exploitable | 01:42 |
| mordred | clarkb: probably in some manner | 01:42 |
| clarkb | to see commits in an eg private fork of a repo | 01:42 |
| notmorgan | clarkb: you can't private fork anymore | 01:42 |
| mordred | oh. that's an interesting question | 01:42 |
| greghaynes | clarkb: I was certainly wondering why we merged a WIP patvh | 01:43 |
| greghaynes | Patch, even | 01:43 |
| mordred | notmorgan: no? | 01:43 |
| notmorgan | you have to fork and then reupload into an unconnected repo. | 01:43 |
| jhesketh | notmorgan: really? I think I did last night... but it was private in my org too | 01:43 |
| clarkb | notmorgan: maybe this is why :) | 01:43 |
| notmorgan | mordred: nope. private/not-private is dictated by the original repo | 01:43 |
| notmorgan | mordred: and if you delete the original private repo the private forks go away iirc. | 01:43 |
| clarkb | probably every repo family has a single repo on their backend and forks are just views onto arepo | 01:43 |
| jhesketh | clarkb: I can't access that commit sha on a not diskimage-builder repo fwiw | 01:43 |
| lifeless | mordred: you didn't think github stored a copy of the repo for each fork did you? | 01:43 |
| notmorgan | mordred: it used to be you could make a fork private etc. but that was exploitable iirc. | 01:44 |
| clarkb | lifeless: no but I didn't expect them to leak commits across them in the ui | 01:44 |
| notmorgan | and was just a terrible ui. | 01:44 |
| notmorgan | s/ui/ux/ | 01:44 |
| mordred | lifeless: yeah. what clarkb said | 01:44 |
| lifeless | clarkb: PR's do this somewhat as a feature I think | 01:44 |
| notmorgan | lifeless: ++ | 01:44 |
| lifeless | but yeah, I fully expect its just a unique set of refs | 01:44 |
| notmorgan | it's to help make PRs easier to push around. | 01:44 |
| notmorgan | or somesuch | 01:45 |
| lifeless | the git protocol doesn't permit access to things that are not reachable from a ref | 01:45 |
| * notmorgan goes back to not liking PR workflows. | 01:45 | |
| *** furlongm_ is now known as furlongm | 01:45 | |
| *** thorst has quit IRC | 01:46 | |
| prometheanfire | ianw: well, found a bindep error, which is nice | 01:46 |
| prometheanfire | ianw: also, I think you mentioned that we might be able to proceed if I got to the chown piece? | 01:47 |
| prometheanfire | chown: invalid user: 'jenkins:jenkins' | 01:47 |
| *** amitgandhinz has joined #openstack-infra | 01:47 | |
| *** kushal has joined #openstack-infra | 01:48 | |
| tonyb | clarkb: Sorry I had no idea that would happen | 01:48 |
| clarkb | I was mostly just worried that somehow commits were going into github tat weren't in gerrit, but apparently its just github buggyness | 01:49 |
| anteaya | tonyb: jhesketh and you thought you could fail in private | 01:49 |
| notmorgan | anteaya: colour update: you're a lovely shade of green again. | 01:50 |
| anteaya | notmorgan: oh how nice | 01:50 |
| anteaya | notmorgan: thank you | 01:50 |
| anteaya | notmorgan: can I see? | 01:50 |
| tonyb | anteaya: pretty much | 01:50 |
| anteaya | tonyb: ha ha ha | 01:50 |
| jhesketh | anteaya: welcome to our public art performance where we face-plant | 01:50 |
| anteaya | jhesketh: you do it so well | 01:50 |
| jhesketh | and often! | 01:50 |
| anteaya | and with such a nice finish | 01:50 |
| anteaya | jhesketh: you really stuck the landing there | 01:50 |
| anteaya | I'm expecting high points from the judges | 01:51 |
| *** amitgandhinz has quit IRC | 01:52 | |
| notmorgan | anteaya: http://i.imgur.com/lFPBUuc.png | 01:53 |
| *** haaj has joined #openstack-infra | 01:54 | |
| anteaya | notmorgan: thank you | 01:54 |
| anteaya | I do like that green, thank you | 01:54 |
| *** links has joined #openstack-infra | 01:54 | |
| prometheanfire | tonyb: or anyone that can +2 bindep stuff... https://review.openstack.org/326144 | 01:55 |
| anteaya | would jhesketh be taupe, I wonder? | 01:55 |
| tonyb | prometheanfire: not me :( | 01:55 |
| notmorgan | prometheanfire: hehe glad you approve of my change ;) | 01:55 |
| jhesketh | anteaya: that's a very specific colour I just had to google to figure out what it was | 01:55 |
| * jhesketh is not a smart man | 01:56 | |
| anteaya | jhesketh: I looked too, the colour of french moles | 01:56 |
| prometheanfire | notmorgan: ya, needed to be able to reproduce the error I got locally, it was pointing to another line oddly | 01:56 |
| anteaya | jhesketh: sure you are, and bonus is you are a kind person too, like Forrest | 01:56 |
| notmorgan | prometheanfire: i think bindep core is lifeless and uhm infra-core | 01:56 |
| anteaya | jhesketh: wouldn't you say taupe there: http://i.imgur.com/lFPBUuc.png | 01:56 |
| prometheanfire | notmorgan: ya :( | 01:57 |
| anteaya | jhesketh: now on mine you are a lovely blue | 01:57 |
| *** ddieterly has joined #openstack-infra | 01:57 | |
| *** ddieterly is now known as ddieterly[away] | 01:57 | |
| *** ddieterly[away] is now known as ddieterly | 01:57 | |
| jhesketh | :-) | 01:58 |
| notmorgan | prometheanfire: well i might have been on a py3 "fix the things" kick lately... | 01:58 |
| anteaya | :) | 01:58 |
| prometheanfire | notmorgan: I'm happy, because I needs that | 01:59 |
| *** haaj has quit IRC | 02:00 | |
| lifeless | notmorgan: prometheanfire: I started bindep, but really, its infra-core now. I'll review if someone pings me for one | 02:03 |
| prometheanfire | ianw: I've tested https://review.openstack.org/#/c/310865/3 and left a note as to the results, I think it's ready to move forward | 02:03 |
| notmorgan | lifeless: i did mox->mock patch and ~2 lines to fix py3 issues | 02:03 |
| prometheanfire | the jenkins user is set up via puppet right? | 02:03 |
| prometheanfire | and group | 02:04 |
| notmorgan | lifeless: you're welcome to review it, or it can hold until infra-core has bandwidth | 02:04 |
| * notmorgan isn't in a huge rush there. | 02:04 | |
| prometheanfire | Error: Evaluation Error: Error while evaluating a Function Call, Could not find class ::ntp for gentoo-dib.openstacklocal at /opt/build_git/openstack-infra/system-config/modules/openstack_project/manifests/template.pp:29:3 on node gentoo-dib.openstacklocal | 02:06 |
| ianw | prometheanfire: yes | 02:07 |
| ianw | prometheanfire: what i've found helpful when working on similar stuff is having the good logs to compare to from http://nodepool.openstack.org | 02:07 |
| prometheanfire | ianw: looks like next step is fixing puppet modules? | 02:07 |
| ianw | prometheanfire: yep ... that's kind of what i predicted wouldn't work :) | 02:07 |
| prometheanfire | yep, but good to know it got that far | 02:08 |
| *** gongysh has quit IRC | 02:08 | |
| prometheanfire | adding a new distro is becoming a neverending rabbit hole | 02:08 |
| ianw | prometheanfire: if it was easy, everyone would do it :) | 02:08 |
| prometheanfire | and that'd be bad? | 02:08 |
| ianw | no, i mean in general making things more portable and flexible makes things better | 02:09 |
| prometheanfire | is there a quick way I can run the puppet stuff without waiting an hour each run? | 02:09 |
| prometheanfire | ah | 02:09 |
| ianw | but it's often hard work | 02:10 |
| ianw | you can just run the puppet by hand, start with the system-config install_puppet.sh script | 02:10 |
| prometheanfire | that already works | 02:10 |
| ianw | that brings in the modules, then you can run the puppet apply slave by hand | 02:10 |
| prometheanfire | I fixed that a month ago :P | 02:11 |
| prometheanfire | ok | 02:11 |
| openstackgerrit | Chaoyi Huang proposed openstack-infra/project-config: [Tricircle]add functional test job https://review.openstack.org/329740 | 02:12 |
| prometheanfire | guess I'll set up a dib image just without puppet and work from there | 02:12 |
| *** ddieterly has quit IRC | 02:12 | |
| *** Daisy has quit IRC | 02:13 | |
| prometheanfire | ianw: cache-bindep didn't work because it's not python3 yet though, but that'll be fixed in review https://review.openstack.org/326144 (poke poke, nudge nudge) | 02:13 |
| *** Daisy has joined #openstack-infra | 02:14 | |
| *** gongysh has joined #openstack-infra | 02:16 | |
| *** zz_dimtruck is now known as dimtruck | 02:17 | |
| *** Daisy has quit IRC | 02:18 | |
| *** bhavik has quit IRC | 02:19 | |
| *** ddieterly has joined #openstack-infra | 02:20 | |
| *** PollyZ has joined #openstack-infra | 02:20 | |
| *** docaedo2 has left #openstack-infra | 02:21 | |
| *** PollyZ has quit IRC | 02:25 | |
| *** thorst has joined #openstack-infra | 02:28 | |
| *** weshay_mtg has quit IRC | 02:28 | |
| *** thorst has quit IRC | 02:28 | |
| *** thorst has joined #openstack-infra | 02:29 | |
| *** baoli has quit IRC | 02:32 | |
| *** ddieterly has quit IRC | 02:35 | |
| *** ddieterly has joined #openstack-infra | 02:35 | |
| *** ddieterly has quit IRC | 02:35 | |
| *** tphummel has joined #openstack-infra | 02:35 | |
| eliqiao | hi, can anyone help to check https://review.openstack.org/#/c/326306/ . I need some help on irc renaming. | 02:36 |
| *** thorst has quit IRC | 02:37 | |
| *** r-mibu has quit IRC | 02:37 | |
| *** julim has joined #openstack-infra | 02:38 | |
| *** kzaitsev_mb has joined #openstack-infra | 02:41 | |
| *** PollyZ has joined #openstack-infra | 02:42 | |
| *** zul_ has quit IRC | 02:43 | |
| *** grassass has quit IRC | 02:47 | |
| *** r-mibu has joined #openstack-infra | 02:47 | |
| *** amitgandhinz has joined #openstack-infra | 02:48 | |
| *** yuanying has quit IRC | 02:50 | |
| *** roxanaghe has joined #openstack-infra | 02:50 | |
| *** EricGonczer_ has quit IRC | 02:51 | |
| *** amitgandhinz has quit IRC | 02:53 | |
| *** yuanying has joined #openstack-infra | 02:53 | |
| *** gongysh has quit IRC | 02:53 | |
| *** Daisy has joined #openstack-infra | 02:53 | |
| *** yanyanhu has joined #openstack-infra | 02:54 | |
| *** PollyZ has quit IRC | 02:55 | |
| *** roxanaghe has quit IRC | 02:55 | |
| *** PollyZ has joined #openstack-infra | 02:55 | |
| *** haaj has joined #openstack-infra | 02:56 | |
| *** zhurong has quit IRC | 02:56 | |
| *** zhurong has joined #openstack-infra | 02:56 | |
| *** links has quit IRC | 02:56 | |
| *** PollyZ has quit IRC | 02:58 | |
| *** gyee has quit IRC | 02:59 | |
| *** elynn has joined #openstack-infra | 03:00 | |
| openstackgerrit | Di XiaoLi proposed openstack-infra/project-config: Add a heat job to use devstack plugin https://review.openstack.org/317811 | 03:00 |
| *** haaj has quit IRC | 03:01 | |
| *** ramishra has joined #openstack-infra | 03:02 | |
| *** Daisy has quit IRC | 03:05 | |
| *** Daisy has joined #openstack-infra | 03:05 | |
| *** Daisy_ has joined #openstack-infra | 03:08 | |
| *** Daisy has quit IRC | 03:10 | |
| *** tphummel has quit IRC | 03:12 | |
| *** hongbin has joined #openstack-infra | 03:13 | |
| *** asselin_ has joined #openstack-infra | 03:13 | |
| *** cody-somerville has joined #openstack-infra | 03:13 | |
| *** grassass has joined #openstack-infra | 03:14 | |
| *** sflanigan has quit IRC | 03:15 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: WIP: Add web-based console log streaming https://review.openstack.org/320563 | 03:24 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python3 Fix: tools use six.moves.urllib https://review.openstack.org/327459 | 03:24 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: Bundled Fixes https://review.openstack.org/325661 | 03:24 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 fix: GC final state inspection https://review.openstack.org/327436 | 03:24 |
| *** kzaitsev_mb has quit IRC | 03:24 | |
| *** cody-somerville has quit IRC | 03:25 | |
| *** erlon has quit IRC | 03:26 | |
| *** sarob has joined #openstack-infra | 03:28 | |
| *** PollyZ has joined #openstack-infra | 03:30 | |
| *** tqtran has joined #openstack-infra | 03:31 | |
| *** yamada-h has joined #openstack-infra | 03:32 | |
| *** sarob has quit IRC | 03:35 | |
| *** thorst has joined #openstack-infra | 03:35 | |
| *** tqtran has quit IRC | 03:36 | |
| *** sflanigan has joined #openstack-infra | 03:36 | |
| *** yamada-h has quit IRC | 03:37 | |
| *** ramishra has quit IRC | 03:38 | |
| *** PollyZ has quit IRC | 03:41 | |
| *** tkelsey has joined #openstack-infra | 03:42 | |
| *** thorst has quit IRC | 03:43 | |
| *** Daisy_ has quit IRC | 03:43 | |
| *** Daisy has joined #openstack-infra | 03:44 | |
| *** tkelsey has quit IRC | 03:47 | |
| *** hongbin has quit IRC | 03:47 | |
| *** yuanying_ has joined #openstack-infra | 03:48 | |
| *** Daisy has quit IRC | 03:48 | |
| *** amitgandhinz has joined #openstack-infra | 03:49 | |
| *** binbincong has quit IRC | 03:49 | |
| *** yuanying has quit IRC | 03:49 | |
| *** Sukhdev has joined #openstack-infra | 03:50 | |
| *** amitgandhinz has quit IRC | 03:53 | |
| openstackgerrit | watanabe isao proposed openstack-infra/zuul: WIP: Improve zuul proxy feature https://review.openstack.org/240131 | 03:55 |
| *** haaj has joined #openstack-infra | 03:57 | |
| *** yamada-h has joined #openstack-infra | 03:57 | |
| *** ramishra has joined #openstack-infra | 03:57 | |
| *** njohnston|afk is now known as njohnston | 03:59 | |
| *** dimtruck is now known as zz_dimtruck | 04:00 | |
| *** Wei_Liu has quit IRC | 04:02 | |
| *** haaj has quit IRC | 04:02 | |
| *** Wei_Liu has joined #openstack-infra | 04:03 | |
| *** ramishra has quit IRC | 04:03 | |
| *** julim has quit IRC | 04:04 | |
| *** links has joined #openstack-infra | 04:05 | |
| *** binbincong has joined #openstack-infra | 04:06 | |
| *** david-lyle has joined #openstack-infra | 04:06 | |
| *** Daisy has joined #openstack-infra | 04:09 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Capture Logs when Timeout is hit https://review.openstack.org/328516 | 04:10 |
| *** rkukura has quit IRC | 04:10 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: DNM: Synthetic Failure to show proper log capture https://review.openstack.org/329755 | 04:11 |
| *** rkukura has joined #openstack-infra | 04:12 | |
| notmorgan | mordred, jeblair, clarkb: https://review.openstack.org/328516 even better form, this now captures all the data "correctly". Thanks lifeless! Next step will be doing thread introspection and killing threads / checking daemon status. so we can move to a proper "gentle" form. | 04:12 |
| *** furlongm has quit IRC | 04:13 | |
| *** furlongm has joined #openstack-infra | 04:13 | |
| *** ramishra has joined #openstack-infra | 04:13 | |
| mordred | notmorgan: looks good | 04:16 |
| notmorgan | mordred: it's way better than just dumping to stderr. i'm learning a lot about subunit :P | 04:17 |
| * notmorgan was hesitant to say that too loudly as people might start asking questions. | 04:17 | |
| notmorgan | oh i should probably add the logger things to it so we see that it was actually a timeout that occured. | 04:18 |
| notmorgan | the ascii art was lost :( | 04:19 |
| *** tphummel has joined #openstack-infra | 04:19 | |
| notmorgan | mordred: how important is the ascii art saying it was actually a timeout (or an explicit log) vs just a failure? | 04:19 |
| *** PollyZ has joined #openstack-infra | 04:19 | |
| *** kzaitsev_mb has joined #openstack-infra | 04:21 | |
| notmorgan | mordred: bah. and it still didn't show up in the testrepository stuff in the gate... | 04:21 |
| notmorgan | grrr... | 04:21 |
| * notmorgan goes and glares some more. | 04:21 | |
| mordred | notmorgan: you are now officially the subunit expert. congrats | 04:22 |
| lifeless | notmorgan: it probably wants to write to stdout, not err | 04:24 |
| lifeless | notmorgan: since testr passes err straight to the user. | 04:24 |
| notmorgan | lifeless: ahh ok. | 04:24 |
| notmorgan | lifeless: i'll circle back on that. interestingly it was captured locally under tox, must be some wonkyness | 04:25 |
| notmorgan | lifeless: so do i need to write to both stdout and err in this case? | 04:25 |
| notmorgan | lifeless: or is stdout going to still hit the console as well? | 04:25 |
| *** kuromagi has quit IRC | 04:25 | |
| * notmorgan can actually test that but ... has a local lifeless to ask :P | 04:25 | |
| *** kuromagi has joined #openstack-infra | 04:25 | |
| * notmorgan runs to get food. | 04:26 | |
| lifeless | notmorgan: well, tox mangles stderr to stdout | 04:26 |
| notmorgan | ahhhhhhhhhhhhharghhhh | 04:27 |
| lifeless | yeah | 04:27 |
| *** oanson has joined #openstack-infra | 04:28 | |
| notmorgan | i'll poke at this so much more once i get back from some dinner | 04:29 |
| * notmorgan also realizes a nice lamp for this new desk is a must | 04:29 | |
| *** gouthamr has quit IRC | 04:34 | |
| *** elynn has quit IRC | 04:34 | |
| *** _nadya_ has joined #openstack-infra | 04:35 | |
| *** PollyZ has quit IRC | 04:35 | |
| *** Daisy has quit IRC | 04:39 | |
| *** psachin has joined #openstack-infra | 04:41 | |
| *** thorst has joined #openstack-infra | 04:42 | |
| *** thorst has quit IRC | 04:48 | |
| *** mrmartin has joined #openstack-infra | 04:49 | |
| *** amitgandhinz has joined #openstack-infra | 04:50 | |
| *** ramishra has quit IRC | 04:50 | |
| *** ramishra has joined #openstack-infra | 04:50 | |
| *** roxanaghe has joined #openstack-infra | 04:51 | |
| *** rkukura has quit IRC | 04:52 | |
| *** rkukura has joined #openstack-infra | 04:54 | |
| *** amitgandhinz has quit IRC | 04:54 | |
| *** kzaitsev_mb has quit IRC | 04:54 | |
| *** salv-orl_ has quit IRC | 04:54 | |
| *** roxanaghe has quit IRC | 04:54 | |
| *** salv-orlando has joined #openstack-infra | 04:54 | |
| *** ramishra has quit IRC | 04:54 | |
| *** roxanaghe has joined #openstack-infra | 04:55 | |
| *** elynn has joined #openstack-infra | 04:56 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Clear up "already provided" message https://review.openstack.org/290968 | 04:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Convert element_dependencies to logging https://review.openstack.org/328072 | 04:58 |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Revise gerritbot config for senlin channel https://review.openstack.org/329761 | 04:58 |
| *** haaj has joined #openstack-infra | 04:58 | |
| *** njohnston is now known as njohnston|sleep | 04:59 | |
| *** ramishra has joined #openstack-infra | 05:00 | |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Revise gerritbot config of senlin channel https://review.openstack.org/329761 | 05:00 |
| *** elynn has quit IRC | 05:00 | |
| *** elynn has joined #openstack-infra | 05:01 | |
| *** yamamot__ has joined #openstack-infra | 05:01 | |
| *** haaj has quit IRC | 05:04 | |
| *** jaosorior has joined #openstack-infra | 05:04 | |
| *** Daisy has joined #openstack-infra | 05:05 | |
| *** Daisy has quit IRC | 05:05 | |
| *** Daisy has joined #openstack-infra | 05:05 | |
| *** pilgrimstack has joined #openstack-infra | 05:11 | |
| *** pilgrimstack has quit IRC | 05:13 | |
| *** salv-orlando has quit IRC | 05:17 | |
| *** salv-orlando has joined #openstack-infra | 05:17 | |
| *** ramishra has quit IRC | 05:18 | |
| *** sree has joined #openstack-infra | 05:18 | |
| *** mrmartin has quit IRC | 05:21 | |
| *** _nadya_ has quit IRC | 05:25 | |
| *** salv-orl_ has joined #openstack-infra | 05:29 | |
| *** sarob has joined #openstack-infra | 05:31 | |
| *** salv-orlando has quit IRC | 05:32 | |
| gmann | AJaeger: on LP, is there any way to open the bug which is Fix_released ? | 05:34 |
| *** sarob has quit IRC | 05:36 | |
| *** gongysh has joined #openstack-infra | 05:36 | |
| AJaeger | gmann: I think you can just reopen it - but never tried... | 05:38 |
| *** elynn has left #openstack-infra | 05:38 | |
| gmann | AJaeger: i tried but all option are disabled seems | 05:39 |
| gmann | tried to change the status | 05:39 |
| AJaeger | gmann: then you don't have the right permissions ;) It works for me on those where I'm in the bug group | 05:39 |
| gmann | ah i see. | 05:39 |
| AJaeger | status chane is only possible for those in the proper group - so ask one of the bug czars for the project | 05:40 |
| *** gongysh has quit IRC | 05:40 | |
| reduxio | Good morning. I'm having trouble setting up and testing a third party CI system. Is this the right place to ask about it? | 05:40 |
| gmann | AJaeger: sure, Thanks. | 05:40 |
| *** abregman has joined #openstack-infra | 05:41 | |
| AJaeger | reduxio: there's a third-party IRC channel and a meeting for that one. | 05:41 |
| AJaeger | reduxio: #openstack-third-party-ci | 05:42 |
| reduxio | Ajaeger: Thanks! | 05:42 |
| *** mrodden has quit IRC | 05:43 | |
| openstackgerrit | Merged openstack-infra/project-config: translations: Fix error abort https://review.openstack.org/329642 | 05:44 |
| *** ramishra has joined #openstack-infra | 05:45 | |
| *** claudiub has joined #openstack-infra | 05:45 | |
| *** pgadiya has joined #openstack-infra | 05:46 | |
| *** thorst has joined #openstack-infra | 05:47 | |
| *** netsin has quit IRC | 05:47 | |
| *** netsin has joined #openstack-infra | 05:48 | |
| *** ramishra has quit IRC | 05:48 | |
| *** Wei_Liu has quit IRC | 05:49 | |
| *** Jeffrey4l has quit IRC | 05:49 | |
| *** Wei_Liu has joined #openstack-infra | 05:49 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:50 | |
| *** amitgandhinz has joined #openstack-infra | 05:50 | |
| *** cody-somerville has joined #openstack-infra | 05:50 | |
| *** cody-somerville has joined #openstack-infra | 05:50 | |
| *** mrodden has joined #openstack-infra | 05:51 | |
| *** yamahata has joined #openstack-infra | 05:53 | |
| *** thorst has quit IRC | 05:53 | |
| *** Jeffrey4l has joined #openstack-infra | 05:54 | |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Remove two inactive channels from gerritbot list https://review.openstack.org/329768 | 05:54 |
| *** garyk has joined #openstack-infra | 05:54 | |
| *** kzaitsev_mb has quit IRC | 05:55 | |
| *** amitgandhinz has quit IRC | 05:55 | |
| *** cody-somerville has quit IRC | 05:55 | |
| *** sarob has joined #openstack-infra | 05:57 | |
| openstackgerrit | Merged openstack-infra/project-config: Added new repos for fuel plugins https://review.openstack.org/328223 | 05:58 |
| *** ramishra has joined #openstack-infra | 05:58 | |
| *** rlandy has quit IRC | 05:58 | |
| reduxio | AJaeger: the #openstack-third-party-ci channel is quiet like a graveyard. Any other channels you can point me to? | 05:59 |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Remove two inactive channels https://review.openstack.org/329768 | 06:00 |
| *** haaj has joined #openstack-infra | 06:00 | |
| AJaeger | reduxio: might be the wrong time of day for this. There's nobody here either right now to help AFAIK. Feel free to ask anteaya during American morning for advise. | 06:00 |
| reduxio | AJaeger: I see. Thank you. | 06:01 |
| *** sarob has quit IRC | 06:02 | |
| *** yuanying has joined #openstack-infra | 06:03 | |
| *** ramishra has quit IRC | 06:03 | |
| *** yuanying_ has quit IRC | 06:05 | |
| *** haaj has quit IRC | 06:05 | |
| *** flwang has quit IRC | 06:05 | |
| snarwade | hello, is there any way to write JJB if I have kubernetes plugin in jenkins | 06:06 |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Remove two inactive channels https://review.openstack.org/329768 | 06:07 |
| *** yolanda has joined #openstack-infra | 06:13 | |
| *** yolanda_ has joined #openstack-infra | 06:13 | |
| *** roxanaghe has quit IRC | 06:14 | |
| *** yolanda_ has quit IRC | 06:14 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/329773 | 06:15 |
| *** ramishra has joined #openstack-infra | 06:16 | |
| yolanda | good morning | 06:16 |
| *** openstackgerrit has quit IRC | 06:18 | |
| *** openstackgerrit has joined #openstack-infra | 06:18 | |
| *** Daisy has quit IRC | 06:19 | |
| *** pilgrimstack has joined #openstack-infra | 06:19 | |
| *** Daisy has joined #openstack-infra | 06:19 | |
| *** andreas_s has joined #openstack-infra | 06:20 | |
| *** yuanying has quit IRC | 06:21 | |
| *** pilgrimstack has quit IRC | 06:21 | |
| *** rook has quit IRC | 06:22 | |
| *** tphummel has quit IRC | 06:23 | |
| *** salv-orlando has joined #openstack-infra | 06:24 | |
| notmorgan | yolanda: mornin! | 06:27 |
| *** yuanying has joined #openstack-infra | 06:28 | |
| *** salv-orl_ has quit IRC | 06:28 | |
| *** salv-orlando has quit IRC | 06:29 | |
| *** rcernin has joined #openstack-infra | 06:30 | |
| *** flwang has joined #openstack-infra | 06:30 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Capture Logs when Timeout is hit https://review.openstack.org/328516 | 06:32 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: DNM: Synthetic Failure to show proper log capture https://review.openstack.org/329755 | 06:32 |
| *** aviau has quit IRC | 06:37 | |
| openstackgerrit | Sachi King proposed openstack-dev/pbr: Restore warnerrors behavior https://review.openstack.org/229951 | 06:38 |
| *** aviau has joined #openstack-infra | 06:39 | |
| *** tkelsey has joined #openstack-infra | 06:45 | |
| *** tkelsey has quit IRC | 06:49 | |
| *** ifarkas has joined #openstack-infra | 06:49 | |
| *** rook has joined #openstack-infra | 06:49 | |
| *** e0ne has joined #openstack-infra | 06:51 | |
| *** kzaitsev_mb has joined #openstack-infra | 06:51 | |
| *** amitgandhinz has joined #openstack-infra | 06:51 | |
| *** thorst has joined #openstack-infra | 06:51 | |
| notmorgan | lifeless: found a bug in subunit - in py27 stream.read(0) seems to raise ValueError for not open for "read" instead of (_UnsupportedOperation, IOError) | 06:55 |
| AJaeger | morning, yolanda | 06:56 |
| *** amitgandhinz has quit IRC | 06:56 | |
| *** kzaitsev_mb has quit IRC | 06:56 | |
| notmorgan | or... ... i dunno *why* this time it's raising a value error... | 06:56 |
| *** thorst has quit IRC | 06:58 | |
| *** zhurong has quit IRC | 06:58 | |
| *** maestro has joined #openstack-infra | 06:59 | |
| *** amoralej|off is now known as amoralej | 06:59 | |
| *** kushal has quit IRC | 06:59 | |
| *** nmagnezi has joined #openstack-infra | 07:01 | |
| *** nmagnezi has quit IRC | 07:01 | |
| *** nmagnezi has joined #openstack-infra | 07:01 | |
| *** haaj has joined #openstack-infra | 07:01 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/329773 | 07:02 |
| *** rossella_s has joined #openstack-infra | 07:02 | |
| *** tesseract has joined #openstack-infra | 07:03 | |
| *** rcernin has quit IRC | 07:04 | |
| *** Daisy_ has joined #openstack-infra | 07:04 | |
| *** rcernin has joined #openstack-infra | 07:04 | |
| *** yuanying has quit IRC | 07:05 | |
| *** maestro has left #openstack-infra | 07:05 | |
| *** netsin has quit IRC | 07:05 | |
| *** haaj has quit IRC | 07:07 | |
| *** Daisy_ has quit IRC | 07:07 | |
| *** Daisy_ has joined #openstack-infra | 07:07 | |
| *** flepied has quit IRC | 07:08 | |
| *** yuanying has joined #openstack-infra | 07:08 | |
| *** Daisy has quit IRC | 07:08 | |
| *** matrohon has joined #openstack-infra | 07:10 | |
| *** Daisy has joined #openstack-infra | 07:10 | |
| lifeless | notmorgan: how is that a bug? | 07:10 |
| lifeless | notmorgan: a not-open stream isn't usable... | 07:10 |
| *** zhurong has joined #openstack-infra | 07:11 | |
| *** acoles_ is now known as acoles | 07:13 | |
| *** dtardivel has joined #openstack-infra | 07:14 | |
| *** Daisy_ has quit IRC | 07:14 | |
| *** sree has quit IRC | 07:17 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove two inactive channels https://review.openstack.org/329768 | 07:18 |
| openstackgerrit | Merged openstack-infra/project-config: Revise gerritbot config of senlin channel https://review.openstack.org/329761 | 07:19 |
| *** dhellmann has quit IRC | 07:20 | |
| *** sree has joined #openstack-infra | 07:20 | |
| *** dhellmann has joined #openstack-infra | 07:20 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute004.uswest metadata for better tracking https://review.openstack.org/329602 | 07:21 |
| *** winggundamth has quit IRC | 07:22 | |
| *** javeriak_ has joined #openstack-infra | 07:23 | |
| *** shardy has joined #openstack-infra | 07:24 | |
| *** jlanoux has joined #openstack-infra | 07:25 | |
| *** vincentll has joined #openstack-infra | 07:27 | |
| openstackgerrit | Merged openstack-infra/project-config: zuul/layout: switch beaker/xenial jobs in gate https://review.openstack.org/329108 | 07:27 |
| AJaeger | yolanda: gerritbot was not accessing all channels, we have a limit of 120 - and after 329768 merged we're exactly at 120. So, let's see whether it works now before adding more - I blocked the charm addition. | 07:27 |
| *** sree has quit IRC | 07:28 | |
| yolanda | oh, didn't know about that limit. Why 120 is the number, cannot handle more? | 07:28 |
| *** esikachev has joined #openstack-infra | 07:28 | |
| *** tlian has quit IRC | 07:28 | |
| *** Sukhdev has quit IRC | 07:29 | |
| AJaeger | yolanda: we're not sure - there was some discussion last evening with fungi whether this is the problem. | 07:29 |
| *** ikhudoshyn has joined #openstack-infra | 07:29 | |
| AJaeger | So, I suggest to see whether this is really the issue. | 07:30 |
| AJaeger | yolanda: yesterday discussion started around 14:01 UTC - for senlin | 07:30 |
| *** yaume has joined #openstack-infra | 07:30 | |
| *** abregman_ has joined #openstack-infra | 07:31 | |
| ikhudoshyn | hi everybody! I'm about to add one more job for Rally. Kindly ask to take a look https://review.openstack.org/#/c/328249/. Thanks in advance. | 07:31 |
| ikhudoshyn | AJaeger: I updated the patch ^^ | 07:31 |
| openstackgerrit | Merged openstack-infra/project-config: Remove gate-grenade-dsvm-partial-ironic https://review.openstack.org/329553 | 07:31 |
| openstackgerrit | Merged openstack-infra/project-config: Add check-requirements check job for nova-lxd https://review.openstack.org/329331 | 07:31 |
| *** yaume has quit IRC | 07:32 | |
| *** abregman has quit IRC | 07:32 | |
| *** openstackgerrit has quit IRC | 07:33 | |
| *** openstackgerrit has joined #openstack-infra | 07:33 | |
| *** dizquierdo has joined #openstack-infra | 07:36 | |
| *** amotoki has joined #openstack-infra | 07:37 | |
| *** Daisy has quit IRC | 07:37 | |
| yolanda | AJaeger, looking at the conversation. So there is no confirmation of that 120 limit, right? | 07:37 |
| *** Daisy has joined #openstack-infra | 07:37 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Capture Logs when Timeout is hit https://review.openstack.org/328516 | 07:39 |
| AJaeger | yolanda: no confirmation - but I counted and we had 122 before we removed those 2 - so want to see whether it works now... | 07:39 |
| *** links has quit IRC | 07:39 | |
| *** abregman_ is now known as abregman | 07:39 | |
| AJaeger | we can add charm later after if needed. | 07:39 |
| AJaeger | Let's see what fungi things... | 07:40 |
| AJaeger | I meant: later if we see that this fixes the problem... | 07:40 |
| *** pgadiya is now known as pgadiya|lunch | 07:40 | |
| *** salv-orlando has joined #openstack-infra | 07:41 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: DNM: Synthetic Failure to show proper log capture https://review.openstack.org/329755 | 07:41 |
| openstackgerrit | watanabe isao proposed openstack-infra/zuul: Add ssh timeout https://review.openstack.org/329796 | 07:41 |
| yolanda | AJaeger, ok | 07:42 |
| *** Daisy has quit IRC | 07:42 | |
| *** shardy has quit IRC | 07:43 | |
| *** flepied has joined #openstack-infra | 07:43 | |
| *** flepied has quit IRC | 07:44 | |
| *** flepied has joined #openstack-infra | 07:44 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Clear up "already provided" message https://review.openstack.org/290968 | 07:45 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Convert element_dependencies to logging https://review.openstack.org/328072 | 07:45 |
| openstackgerrit | watanabe isao proposed openstack-infra/nodepool: Add ssh timeout https://review.openstack.org/329799 | 07:48 |
| *** zeih has joined #openstack-infra | 07:50 | |
| *** dtantsur|afk is now known as dtantsur | 07:50 | |
| *** sree has joined #openstack-infra | 07:51 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:52 | |
| *** amitgandhinz has joined #openstack-infra | 07:52 | |
| *** darvon has quit IRC | 07:53 | |
| *** Daisy has joined #openstack-infra | 07:53 | |
| *** jaosorior is now known as jaosorior_brb | 07:53 | |
| *** jtomasek_ has joined #openstack-infra | 07:53 | |
| *** thorst has joined #openstack-infra | 07:55 | |
| *** darvon has joined #openstack-infra | 07:55 | |
| *** links has joined #openstack-infra | 07:56 | |
| *** ccamacho has joined #openstack-infra | 07:56 | |
| *** amitgandhinz has quit IRC | 07:56 | |
| *** kzaitsev_mb has quit IRC | 07:57 | |
| *** zhs_s has joined #openstack-infra | 07:58 | |
| *** sarob has joined #openstack-infra | 07:58 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** sree has quit IRC | 08:00 | |
| *** dizquierdo has quit IRC | 08:00 | |
| *** zeih has quit IRC | 08:01 | |
| *** zhs has quit IRC | 08:01 | |
| *** thorst has quit IRC | 08:02 | |
| *** sarob has quit IRC | 08:02 | |
| *** zeih has joined #openstack-infra | 08:02 | |
| *** zzzeek has joined #openstack-infra | 08:03 | |
| *** sree has joined #openstack-infra | 08:06 | |
| *** winggundamth has joined #openstack-infra | 08:07 | |
| openstackgerrit | Hisashi Osanai proposed openstack-infra/project-config: Add jobs for Monasca-Analytics https://review.openstack.org/329376 | 08:09 |
| *** jtomasek_ has quit IRC | 08:10 | |
| *** shardy has joined #openstack-infra | 08:10 | |
| *** sorantis has joined #openstack-infra | 08:10 | |
| *** sorantis has quit IRC | 08:10 | |
| *** eezhova has joined #openstack-infra | 08:11 | |
| *** sorantis has joined #openstack-infra | 08:12 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:13 | |
| *** abregman has quit IRC | 08:15 | |
| *** dmk0202 has joined #openstack-infra | 08:22 | |
| *** yaume has joined #openstack-infra | 08:25 | |
| *** mhickey has joined #openstack-infra | 08:25 | |
| *** abregman has joined #openstack-infra | 08:28 | |
| openstackgerrit | Tomoki Sekiyama proposed openstack-infra/project-config: Create new functional job for glance_store-cinder https://review.openstack.org/329827 | 08:29 |
| *** yaume_ has joined #openstack-infra | 08:30 | |
| *** yaume has quit IRC | 08:30 | |
| openstackgerrit | watanabe isao proposed openstack-infra/zuul: Add ssh timeout https://review.openstack.org/329796 | 08:34 |
| *** jaosorior_brb has quit IRC | 08:40 | |
| *** jaosorior_brb has joined #openstack-infra | 08:40 | |
| *** jaosorior_brb is now known as jaosorior | 08:41 | |
| openstackgerrit | Ferenc Horváth proposed openstack-infra/project-config: Add job which builds test images with watchdog support https://review.openstack.org/329832 | 08:44 |
| *** sree has quit IRC | 08:44 | |
| *** hferenc has joined #openstack-infra | 08:44 | |
| *** sree has joined #openstack-infra | 08:45 | |
| *** derekh has joined #openstack-infra | 08:45 | |
| *** tqtran has joined #openstack-infra | 08:45 | |
| *** Daisy has quit IRC | 08:46 | |
| *** Daisy has joined #openstack-infra | 08:46 | |
| hferenc | hello all! | 08:46 |
| *** Daisy_ has joined #openstack-infra | 08:47 | |
| *** Daisy_ has quit IRC | 08:47 | |
| *** Daisy_ has joined #openstack-infra | 08:48 | |
| *** tqtran has quit IRC | 08:50 | |
| *** sree has quit IRC | 08:50 | |
| *** linuxmaniac_ has joined #openstack-infra | 08:50 | |
| hferenc | i'd like to publish a test image to tarballs.openstack.org | 08:50 |
| *** Daisy has quit IRC | 08:51 | |
| hferenc | based on your previous advices i created a job which intends to do this | 08:51 |
| *** ihrachys has joined #openstack-infra | 08:51 | |
| hferenc | but i'm not sure that i found the right way at first | 08:52 |
| *** amitgandhinz has joined #openstack-infra | 08:53 | |
| hferenc | so could you please give a glimpse at this: https://review.openstack.org/#/c/329832/ | 08:54 |
| *** tkelsey has joined #openstack-infra | 08:55 | |
| *** ramishra has quit IRC | 08:56 | |
| *** amitgandhinz has quit IRC | 08:57 | |
| *** jaosorior has quit IRC | 08:57 | |
| AJaeger | hferenc: the triggered monthyl will not work AFAIK, we trigger with zuul, so you need to add the job to zuul/layout.yaml. I don't know whether we have setup for monthly jobs, you might need to discuss this again with the team here... | 08:59 |
| *** snarwade has quit IRC | 08:59 | |
| *** electrofelix has joined #openstack-infra | 08:59 | |
| *** hashar has joined #openstack-infra | 09:00 | |
| *** hashar has left #openstack-infra | 09:00 | |
| *** thorst has joined #openstack-infra | 09:00 | |
| *** ramishra has joined #openstack-infra | 09:02 | |
| *** Daisy has joined #openstack-infra | 09:02 | |
| *** esikachev has quit IRC | 09:03 | |
| hferenc | AJaeger: thx! i was looking at the zuul triggers, but if i understand it correctly the least frequent is periodic, however i think that it is unnecessary to build this image that often | 09:05 |
| *** pgadiya|lunch has quit IRC | 09:06 | |
| *** Daisy_ has quit IRC | 09:06 | |
| *** thorst has quit IRC | 09:07 | |
| *** jtomasek_ has joined #openstack-infra | 09:08 | |
| *** arxcruz has joined #openstack-infra | 09:08 | |
| hferenc | because it won't change a lot so i gave a try to @monthly | 09:09 |
| *** jtomasek_ has quit IRC | 09:13 | |
| *** javeriak_ has quit IRC | 09:13 | |
| *** snarwade has joined #openstack-infra | 09:14 | |
| *** esikachev has joined #openstack-infra | 09:16 | |
| *** sambetts|afk is now known as sambetts | 09:16 | |
| *** yamada-h has quit IRC | 09:17 | |
| AJaeger | hferenc: I don't think that we interpret the triggers word at all, you need to add it to a periodic-queue in zuul instead | 09:17 |
| AJaeger | hferenc: we only have the perodic pipeline that runs every day | 09:18 |
| *** hdaniel has joined #openstack-infra | 09:19 | |
| hferenc | AJaeger: ok, i'll change the triggers in a sec | 09:19 |
| *** jaosorior has joined #openstack-infra | 09:21 | |
| *** javeriak has joined #openstack-infra | 09:22 | |
| *** penguinolog has quit IRC | 09:24 | |
| *** mikelk has joined #openstack-infra | 09:26 | |
| *** javeriak has quit IRC | 09:26 | |
| *** Daisy has quit IRC | 09:26 | |
| *** penguinolog has joined #openstack-infra | 09:27 | |
| *** Daisy has joined #openstack-infra | 09:27 | |
| *** pgadiya|lunch has joined #openstack-infra | 09:27 | |
| *** esikachev has quit IRC | 09:28 | |
| *** esikachev has joined #openstack-infra | 09:29 | |
| *** Daisy has quit IRC | 09:31 | |
| *** markus__ has joined #openstack-infra | 09:33 | |
| *** kzaitsev_mb has quit IRC | 09:33 | |
| *** hferenc has quit IRC | 09:34 | |
| *** Na3iL has joined #openstack-infra | 09:36 | |
| *** hdaniel has quit IRC | 09:39 | |
| *** pilgrimstack has joined #openstack-infra | 09:40 | |
| *** ociuhandu has quit IRC | 09:42 | |
| *** M-docaedo_vector has quit IRC | 09:42 | |
| *** kushal has joined #openstack-infra | 09:43 | |
| eliqiao | hi AJaeger | 09:43 |
| eliqiao | AJaeger: could you give some advice on https://review.openstack.org/#/c/326306/ "irc channel renaming" | 09:44 |
| *** eliqiao has left #openstack-infra | 09:46 | |
| *** eliqiao has joined #openstack-infra | 09:46 | |
| *** salv-orlando has quit IRC | 09:47 | |
| *** salv-orlando has joined #openstack-infra | 09:48 | |
| *** M-docaedo_vector has joined #openstack-infra | 09:49 | |
| *** cinerama has quit IRC | 09:51 | |
| *** dizquierdo has joined #openstack-infra | 09:51 | |
| *** pilgrimstack has quit IRC | 09:51 | |
| *** amitgandhinz has joined #openstack-infra | 09:54 | |
| *** cinerama has joined #openstack-infra | 09:55 | |
| *** amitgandhinz has quit IRC | 09:58 | |
| *** Na3iL has quit IRC | 09:58 | |
| *** Na3iL has joined #openstack-infra | 09:58 | |
| *** permalac has joined #openstack-infra | 09:58 | |
| *** sarob has joined #openstack-infra | 09:59 | |
| *** _nadya_ has joined #openstack-infra | 09:59 | |
| *** zhurong has quit IRC | 10:00 | |
| *** zeih has quit IRC | 10:00 | |
| AJaeger | eliqiao: only the link I gave - best talk to pleia2, she will be around at US west coast time | 10:01 |
| *** ajmiller_ has quit IRC | 10:02 | |
| *** ajmiller has joined #openstack-infra | 10:02 | |
| *** sarob has quit IRC | 10:03 | |
| *** ajmiller_ has joined #openstack-infra | 10:04 | |
| *** ajmiller has quit IRC | 10:04 | |
| *** thorst has joined #openstack-infra | 10:06 | |
| *** mescanef has joined #openstack-infra | 10:06 | |
| *** M-docaedo_vector has quit IRC | 10:08 | |
| mescanef | hello! please add pbrzozowski@mirantis.com to the following gerrit acl groups: https://review.openstack.org/#/admin/groups/1430,members; https://review.openstack.org/#/admin/groups/1431,members; https://review.openstack.org/#/admin/groups/1428,members; https://review.openstack.org/#/admin/groups/1429,members (ref. https://review.openstack.org/#/c/328223/) | 10:10 |
| *** pilgrimstack has joined #openstack-infra | 10:10 | |
| *** sorantis has quit IRC | 10:11 | |
| *** thorst has quit IRC | 10:12 | |
| *** markus__ has quit IRC | 10:13 | |
| *** javeriak has joined #openstack-infra | 10:13 | |
| *** pgadiya|lunch has quit IRC | 10:13 | |
| *** snarwade has quit IRC | 10:13 | |
| yolanda | hi mescanef i can help | 10:14 |
| yanyanhu | hi, AJaeger, yolanda, gerritbot has come back to senlin channel, thanks | 10:14 |
| yanyanhu | I guess removing idle channels works | 10:14 |
| mescanef | yolanda: :-) | 10:15 |
| yolanda | done | 10:15 |
| mescanef | thanks! | 10:15 |
| *** javeriak_ has joined #openstack-infra | 10:16 | |
| *** markus__ has joined #openstack-infra | 10:16 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: [WIP] install pip/virtualenv packages https://review.openstack.org/327472 | 10:17 |
| AJaeger | yanyanhu: glad to hear. | 10:17 |
| *** markus__ has quit IRC | 10:18 | |
| *** javeriak has quit IRC | 10:18 | |
| AJaeger | yolanda, fungi: Could you check log files whether everything works - and then we could consider merging the charm request to check whether we have a missing channel again... | 10:18 |
| *** HeOS has quit IRC | 10:19 | |
| yolanda | AJaeger, i don't see errors on gerritbot | 10:20 |
| mescanef | yolanda: oh one group missed: https://review.openstack.org/#/admin/groups/1431,members | 10:20 |
| yolanda | but i think we better wait for fungi | 10:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Clear up "already provided" message https://review.openstack.org/290968 | 10:21 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Convert element_dependencies to logging https://review.openstack.org/328072 | 10:21 |
| *** yanyanhu has quit IRC | 10:23 | |
| *** sorantis has joined #openstack-infra | 10:24 | |
| *** javeriak_ has quit IRC | 10:24 | |
| *** pgadiya|lunch has joined #openstack-infra | 10:26 | |
| *** markus__ has joined #openstack-infra | 10:27 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:27 | |
| *** eezhova has quit IRC | 10:31 | |
| *** eezhova has joined #openstack-infra | 10:32 | |
| *** ociuhandu has joined #openstack-infra | 10:36 | |
| *** snarwade has joined #openstack-infra | 10:36 | |
| *** zeih has joined #openstack-infra | 10:37 | |
| *** hashar has joined #openstack-infra | 10:38 | |
| *** M-docaedo_vector has joined #openstack-infra | 10:40 | |
| *** markus__ has quit IRC | 10:41 | |
| *** markus__ has joined #openstack-infra | 10:43 | |
| *** shahid_ has joined #openstack-infra | 10:45 | |
| *** markus__ has quit IRC | 10:45 | |
| *** weshay has joined #openstack-infra | 10:51 | |
| *** YorikSar_ has quit IRC | 10:51 | |
| *** hdaniel has joined #openstack-infra | 10:51 | |
| *** pilgrimstack has quit IRC | 10:52 | |
| *** sdague has joined #openstack-infra | 10:52 | |
| *** markus__ has joined #openstack-infra | 10:53 | |
| *** lvdongbing has quit IRC | 10:53 | |
| *** amitgandhinz has joined #openstack-infra | 10:54 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/ansible-role-puppet: Log puppet output on Ansible control machine https://review.openstack.org/327789 | 10:56 |
| *** zeih has quit IRC | 10:56 | |
| *** salv-orlando has quit IRC | 10:56 | |
| *** salv-orlando has joined #openstack-infra | 10:57 | |
| *** zeih has joined #openstack-infra | 10:57 | |
| *** YorikSar has joined #openstack-infra | 10:58 | |
| rcarrillocruz | nibalizer , clarkb : suggestions implemented on ansible-puppet role ^ | 10:58 |
| *** tlian has joined #openstack-infra | 10:59 | |
| *** amitgandhinz has quit IRC | 10:59 | |
| *** zeih_ has joined #openstack-infra | 11:00 | |
| *** hashar has quit IRC | 11:00 | |
| *** pradk has quit IRC | 11:01 | |
| *** zeih has quit IRC | 11:02 | |
| *** baoli has joined #openstack-infra | 11:05 | |
| *** salv-orlando has quit IRC | 11:06 | |
| *** baoli has quit IRC | 11:06 | |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job https://review.openstack.org/229789 | 11:08 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Bump PuppetDB version to 4.1.0 to get ssl related params https://review.openstack.org/329875 | 11:08 |
| *** thorst has joined #openstack-infra | 11:10 | |
| *** yamamot__ has quit IRC | 11:10 | |
| *** markus__ has quit IRC | 11:11 | |
| *** markus__ has joined #openstack-infra | 11:11 | |
| *** javeriak has joined #openstack-infra | 11:12 | |
| *** dprince has joined #openstack-infra | 11:14 | |
| *** xenogear has quit IRC | 11:15 | |
| openstackgerrit | Madhuri Kumari proposed openstack-infra/project-config: Add new project Zunclient to openstack repo https://review.openstack.org/317699 | 11:15 |
| *** markus__ has quit IRC | 11:15 | |
| *** ccamacho is now known as ccamacho|lunch | 11:16 | |
| *** hparekh has quit IRC | 11:16 | |
| *** thorst has quit IRC | 11:17 | |
| *** javeriak has quit IRC | 11:18 | |
| *** hparekh has joined #openstack-infra | 11:19 | |
| *** hparekh_ has joined #openstack-infra | 11:19 | |
| *** hparekh_ has quit IRC | 11:20 | |
| *** pgadiya|lunch is now known as pgadiya | 11:20 | |
| *** hferenc has joined #openstack-infra | 11:28 | |
| rcarrillocruz | nibalizer , clarkb : as the role will log stuff on /var/log/ansible-puppet/<hostname> , we want to logrotate those | 11:33 |
| rcarrillocruz | i think that should be put in the role logic for more self-contained logic, let me know if you agree or prefer to set that on the puppet classes managing puppetmaster.openstack.org | 11:33 |
| prometheanfire | I'm trying to make an image for a slave, it should just be puppet apply slave from system config right? | 11:34 |
| *** eezhova has quit IRC | 11:34 | |
| rcarrillocruz | prometheanfire: what kind of slave, a jenkins slave or ... ? | 11:35 |
| prometheanfire | this is for a nodepool image | 11:36 |
| *** kushal has quit IRC | 11:37 | |
| rcarrillocruz | ah | 11:37 |
| rcarrillocruz | the images are built with dib (disk image builder) | 11:37 |
| rcarrillocruz | so, you want to check this: | 11:37 |
| rcarrillocruz | http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n668 | 11:38 |
| prometheanfire | right, I'm in the process of adding a gentoo image to nodepool and am on the stage where puppet fails :P | 11:38 |
| rcarrillocruz | in there, you can see which elements is a ubuntu-trusty made of | 11:38 |
| prometheanfire | https://review.openstack.org/#/c/315865/ | 11:38 |
| prometheanfire | I've already done some work, I need to be able to run it manually so that I can submit reviews for the puppet modules themselves | 11:39 |
| prometheanfire | I know the ntp one failed, but I'd rather not wait an hour to test each time | 11:39 |
| *** ldnunes has joined #openstack-infra | 11:39 | |
| *** lvdongbing has joined #openstack-infra | 11:39 | |
| prometheanfire | (to run dib with the elements) | 11:40 |
| prometheanfire | looks like it's here | 11:41 |
| prometheanfire | https://github.com/openstack-infra/project-config/blob/master/nodepool/scripts/prepare_node.sh#L98-L118 | 11:41 |
| *** lvdongbing has quit IRC | 11:42 | |
| rcarrillocruz | then yeah | 11:42 |
| rcarrillocruz | essentially | 11:42 |
| rcarrillocruz | get a gentoo vm | 11:43 |
| rcarrillocruz | bash -x install_puppet.sh | 11:43 |
| rcarrillocruz | bash -x install_modules.sh | 11:43 |
| prometheanfire | ya, those two already complete, did that work a month ago, now it's module time | 11:43 |
| rcarrillocruz | then something like 'sudo puppet apply --modulepath /etc/puppet/modules:whereveris/system-config -e "include ntp"' | 11:43 |
| *** javeriak has joined #openstack-infra | 11:44 | |
| rcarrillocruz | that, or just create a dummy local.pp and pass on to puppet apply | 11:44 |
| prometheanfire | puppet apply --detailed-exitcodes --color=false --modulepath=/root/system-config/modules:/etc/puppetlabs/code/environments/production/modules | 11:44 |
| prometheanfire | etc | 11:44 |
| rcarrillocruz | i've seen ntp fail typically when the install_modules did not work | 11:44 |
| rcarrillocruz | or i wasn't passing the right --modulepath | 11:44 |
| prometheanfire | this is puppet-agent, so the path differs slightly | 11:44 |
| prometheanfire | anyway, got the first error | 11:45 |
| rcarrillocruz | ah, i believe install_modules.sh does install it on /etc/puppet/modules | 11:45 |
| prometheanfire | Error: Facter: error while resolving custom fact "system_python_version": Execution of '/usr/bin/eix --nocolor --pure-packages --format '<category> <name> [<installedversions:LASTVERSION>] [<bestversion:LASTVERSION>] <homepage> <description> | 11:45 |
| prometheanfire | ' --exact --name python-pip' returned 1: | 11:45 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Correction to meeting ID for Common Flow Classifier meeting https://review.openstack.org/329616 | 11:45 |
| prometheanfire | MODULE_PATH=`puppet config print modulepath | cut -d ':' -f 1` | 11:45 |
| *** ddieterly has joined #openstack-infra | 11:45 | |
| prometheanfire | nope, it's whatever the version of puppet says it should be | 11:45 |
| prometheanfire | that script should probably be updated to call that whenever it needs to edit the module path | 11:46 |
| *** javeriak has quit IRC | 11:46 | |
| prometheanfire | that's the first line from install_modules | 11:46 |
| *** zhurong has joined #openstack-infra | 11:46 | |
| *** thorst has joined #openstack-infra | 11:47 | |
| rcarrillocruz | yeah, saw it | 11:48 |
| *** yamamoto has joined #openstack-infra | 11:48 | |
| prometheanfire | this is puppet-4 as well (what puppet-agent is based off of | 11:49 |
| *** amitgandhinz has joined #openstack-infra | 11:55 | |
| *** rhallisey has joined #openstack-infra | 11:57 | |
| openstackgerrit | Merged openstack-infra/release-tools: unset custom GREP_OPTIONS https://review.openstack.org/324605 | 11:57 |
| *** javeriak has joined #openstack-infra | 11:58 | |
| prometheanfire | now I need to figure out how to tell puppet to use diferent package names :( | 11:59 |
| *** jcoufal has joined #openstack-infra | 11:59 | |
| *** amitgandhinz has quit IRC | 11:59 | |
| *** sarob has joined #openstack-infra | 12:00 | |
| rcarrillocruz | typical pattern is a selector | 12:00 |
| rcarrillocruz | prometheanfire: http://www.puppetcookbook.com/posts/packages-with-different-name-per-distro.html | 12:00 |
| *** prometheanfire has quit IRC | 12:01 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Implement Xenial/CentOS jobs for OpenStack-Ansible https://review.openstack.org/329892 | 12:01 |
| *** yaume_ has quit IRC | 12:01 | |
| *** prometheanfire has joined #openstack-infra | 12:01 | |
| *** e0ne has quit IRC | 12:01 | |
| sdague | is there someone that can help with a docs cleanup? | 12:01 |
| bkero | prometheanfire: likely there's already a package selector in params.pp for the modules | 12:02 |
| sdague | there is a bunch of nova dev docs that are all out of date that I'd like cleaned up off the cloudfiles server | 12:02 |
| *** e0ne has joined #openstack-infra | 12:02 | |
| prometheanfire | probably | 12:02 |
| prometheanfire | bkero: the python modules's facter stuff all calls python-pip, for python, pip and virtualenv | 12:02 |
| *** javeriak has quit IRC | 12:02 | |
| prometheanfire | I think that's wrong | 12:02 |
| bkero | facter does? | 12:02 |
| bkero | Facter core shouldn't, I wonder which facts are using that | 12:03 |
| prometheanfire | no, the python module's custom facter facts | 12:03 |
| bkero | Ah ok | 12:03 |
| bkero | Sounds like that needs some patches for extensibility then | 12:03 |
| prometheanfire | I'm trying to find the code now | 12:04 |
| *** sarob has quit IRC | 12:04 | |
| prometheanfire | ah https://github.com/stankevich/puppet-python | 12:04 |
| prometheanfire | for instance though https://github.com/stankevich/puppet-python/blob/master/lib/facter/virtualenv_version.rb | 12:05 |
| prometheanfire | oh, looks like we may be a few commits back | 12:05 |
| prometheanfire | quite a few | 12:06 |
| *** ddieterly is now known as ddieterly[away] | 12:08 | |
| *** zhurong has quit IRC | 12:09 | |
| *** ccamacho|lunch is now known as ccamacho | 12:10 | |
| *** zhurong has joined #openstack-infra | 12:10 | |
| *** sree has joined #openstack-infra | 12:11 | |
| AJaeger | sdague: I can help | 12:12 |
| *** aysyd has joined #openstack-infra | 12:12 | |
| *** erlon has joined #openstack-infra | 12:13 | |
| *** sree has quit IRC | 12:16 | |
| *** daemontool has joined #openstack-infra | 12:16 | |
| *** furlongm_ has joined #openstack-infra | 12:16 | |
| *** furlongm has quit IRC | 12:16 | |
| *** ramishra has quit IRC | 12:16 | |
| sdague | AJaeger: so http://docs.openstack.org/developer/nova/v2/extensions.html shouldn't exist, that was deleted in Oct 2015 | 12:16 |
| sdague | I wonder if we could do a full delete / regen of http://docs.openstack.org/developer/nova/ | 12:17 |
| *** ramishra has joined #openstack-infra | 12:18 | |
| *** baoli has joined #openstack-infra | 12:19 | |
| AJaeger | sdague: I can delete all old files in there (those that weren't published today;) | 12:19 |
| AJaeger | Should i deletel folder runnova from 2012 as well? | 12:19 |
| *** baoli has quit IRC | 12:20 | |
| *** baoli has joined #openstack-infra | 12:20 | |
| *** eezhova has joined #openstack-infra | 12:21 | |
| AJaeger | sdague: v2 was last updated in January. that directory looks outdated.. | 12:21 |
| *** yaume has joined #openstack-infra | 12:21 | |
| *** hferenc_ has joined #openstack-infra | 12:21 | |
| AJaeger | sdague: If you want, I can do a delete of everything that's not current - with the exception of old releases/branches | 12:22 |
| *** baoli_ has joined #openstack-infra | 12:22 | |
| *** shahid_ has quit IRC | 12:22 | |
| *** trown|outtypewww is now known as trown | 12:23 | |
| sdague | AJaeger: that would be great | 12:24 |
| sdague | the whole v2 directory I think was deleted in Oct | 12:24 |
| AJaeger | sdague: ok, I start deletion now... | 12:25 |
| sdague | but yeh, anything that looks old, should go | 12:25 |
| sdague | we moved a lot around in the last year | 12:25 |
| *** baoli has quit IRC | 12:25 | |
| sdague | https://review.openstack.org/#/c/230186/ was the merge where it went away | 12:25 |
| *** hferenc_ has quit IRC | 12:26 | |
| *** hferenc has quit IRC | 12:26 | |
| *** Goneri has joined #openstack-infra | 12:26 | |
| *** hferenc has joined #openstack-infra | 12:26 | |
| *** raildo-afk is now known as raildo | 12:26 | |
| *** eezhova has quit IRC | 12:27 | |
| AJaeger | sdague: yeah, a lot of moved... I'm deleted devref directory next... | 12:27 |
| *** ddieterly[away] is now known as ddieterly | 12:27 | |
| *** winggundamth_ has joined #openstack-infra | 12:28 | |
| *** lvdongbing has joined #openstack-infra | 12:28 | |
| *** sree has joined #openstack-infra | 12:29 | |
| *** rlandy has joined #openstack-infra | 12:29 | |
| *** winggundamth_ has joined #openstack-infra | 12:29 | |
| openstackgerrit | Ferenc Horváth proposed openstack-infra/project-config: Add job which builds test images with watchdog support https://review.openstack.org/329832 | 12:31 |
| *** sarob has joined #openstack-infra | 12:32 | |
| *** eezhova has joined #openstack-infra | 12:32 | |
| *** ddieterly has quit IRC | 12:35 | |
| *** sarob has quit IRC | 12:36 | |
| AJaeger | sdague: all gone... | 12:38 |
| *** psilvad has joined #openstack-infra | 12:40 | |
| *** edmondsw has joined #openstack-infra | 12:43 | |
| *** tqtran has joined #openstack-infra | 12:47 | |
| *** esikachev has quit IRC | 12:48 | |
| sdague | AJaeger: thanks | 12:49 |
| AJaeger | sdague: you're welcome | 12:51 |
| *** tqtran has quit IRC | 12:51 | |
| *** jsavak has joined #openstack-infra | 12:51 | |
| *** matt-borland has joined #openstack-infra | 12:52 | |
| *** _nadya_ has quit IRC | 12:53 | |
| *** HeOS has joined #openstack-infra | 12:54 | |
| *** _nadya_ has joined #openstack-infra | 12:54 | |
| *** salv-orlando has joined #openstack-infra | 12:56 | |
| *** amitgandhinz has joined #openstack-infra | 12:56 | |
| *** salv-orlando has quit IRC | 12:56 | |
| *** esikachev has joined #openstack-infra | 12:56 | |
| *** salv-orlando has joined #openstack-infra | 12:56 | |
| *** _nadya_ has quit IRC | 12:56 | |
| *** _nadya__ has joined #openstack-infra | 12:57 | |
| *** eharney has joined #openstack-infra | 12:58 | |
| *** rfolco has joined #openstack-infra | 12:58 | |
| *** yujunz has joined #openstack-infra | 12:58 | |
| *** julim has joined #openstack-infra | 12:59 | |
| *** markusry has joined #openstack-infra | 12:59 | |
| *** amitgandhinz has quit IRC | 13:00 | |
| *** eezhova has quit IRC | 13:00 | |
| *** mriedem_away is now known as mriedem | 13:01 | |
| *** yujunz has quit IRC | 13:01 | |
| *** woodster_ has joined #openstack-infra | 13:02 | |
| *** sorantis has quit IRC | 13:02 | |
| *** afred312 has quit IRC | 13:02 | |
| *** dprince has quit IRC | 13:02 | |
| *** inc0 has joined #openstack-infra | 13:03 | |
| *** psachin has quit IRC | 13:03 | |
| *** jcoufal has quit IRC | 13:04 | |
| *** EricGonczer_ has joined #openstack-infra | 13:05 | |
| anteaya | reduxio: you can ask your third party set up questions here | 13:06 |
| *** eezhova has joined #openstack-infra | 13:06 | |
| *** amitgandhinz has joined #openstack-infra | 13:07 | |
| *** amitgandhinz has quit IRC | 13:08 | |
| *** amitgandhinz has joined #openstack-infra | 13:08 | |
| *** lvdongbing has quit IRC | 13:08 | |
| *** gouthamr has joined #openstack-infra | 13:09 | |
| *** bhavik1 has joined #openstack-infra | 13:10 | |
| *** sorantis has joined #openstack-infra | 13:11 | |
| *** nmagnezi has quit IRC | 13:12 | |
| *** mdrabe has joined #openstack-infra | 13:13 | |
| *** rcernin has quit IRC | 13:14 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE TESTING https://review.openstack.org/316436 | 13:14 |
| *** EricGonczer_ has quit IRC | 13:15 | |
| *** ddieterly has joined #openstack-infra | 13:16 | |
| *** xyang1 has joined #openstack-infra | 13:16 | |
| *** amoralej is now known as amoralej|lunch | 13:16 | |
| *** psachin has joined #openstack-infra | 13:16 | |
| *** kgiusti has joined #openstack-infra | 13:16 | |
| *** kgiusti has quit IRC | 13:16 | |
| *** kgiusti has joined #openstack-infra | 13:16 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config: Update ironic-multitenant network vairable names https://review.openstack.org/329919 | 13:17 |
| *** _nadya__ has quit IRC | 13:18 | |
| *** _nadya_ has joined #openstack-infra | 13:18 | |
| *** _nadya_ has quit IRC | 13:18 | |
| anteaya | mordred: Shrews the dox project is collecting stories in storyboard: https://storyboard.openstack.org/#!/story/2000148 | 13:19 |
| openstackgerrit | Alina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job https://review.openstack.org/329497 | 13:19 |
| anteaya | mordred: Shrews yet there is no dox repo in storyboard, from the path openstack/dox I don't intuit this is a repo that is part of infra | 13:20 |
| anteaya | mordred: Shrews where should folks file dox bugs for the project at this time? | 13:20 |
| *** ramishra has quit IRC | 13:20 | |
| *** YorikSar has quit IRC | 13:20 | |
| mordred | anteaya: I don't think we're really doing anything with dox so much anymore | 13:21 |
| openstackgerrit | Alina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job https://review.openstack.org/329497 | 13:21 |
| mordred | anteaya: but i'll defer to Shrews on that | 13:22 |
| Shrews | mordred: i was going to defer to you. but yeah, it's pretty dead | 13:22 |
| anteaya | mordred: okay would you like to help triage https://storyboard.openstack.org/#!/story/2000148 and https://storyboard.openstack.org/#!/story/2000147 | 13:22 |
| *** YorikSar has joined #openstack-infra | 13:22 | |
| mordred | flaper87: what about you? | 13:22 |
| anteaya | the storyboard folks are trying to clean things up and would appreciate some guidance here | 13:23 |
| anteaya | now if you want bugs on storyboard, that is fine, it can happen | 13:23 |
| anteaya | but what do you want here | 13:23 |
| mordred | I don't want bugs at all :) | 13:23 |
| anteaya | just saying the project is dead doesn't answer the where to file bugs question | 13:23 |
| anteaya | mordred: can you close off those stories then please? with some guidance on how to storyboard triagers should address more dox bugs should they be filed | 13:24 |
| mordred | well, I think that maybe we should push up a commit to git to retire the project. then close these stories with "sorry, we stopped workig on this" | 13:24 |
| anteaya | mordred: that sounds reasonable to me | 13:24 |
| anteaya | mordred: so I'll leave this with you, and storyboard folks can move on with their work? | 13:25 |
| *** yamamoto has quit IRC | 13:25 | |
| mordred | yup | 13:25 |
| anteaya | thank you | 13:26 |
| openstackgerrit | Henrique Truta proposed openstack-infra/project-config: Run identity v3 only job in ceilometer https://review.openstack.org/315224 | 13:26 |
| openstackgerrit | Henrique Truta proposed openstack-infra/project-config: Run identity v3 only job in heat https://review.openstack.org/315214 | 13:26 |
| *** afred312 has joined #openstack-infra | 13:28 | |
| *** yamamoto has joined #openstack-infra | 13:28 | |
| *** akshai has joined #openstack-infra | 13:28 | |
| *** rcernin has joined #openstack-infra | 13:29 | |
| *** sdake has joined #openstack-infra | 13:29 | |
| *** eharney has quit IRC | 13:30 | |
| *** xenogear has joined #openstack-infra | 13:31 | |
| *** yamamoto has quit IRC | 13:32 | |
| *** yamamoto has joined #openstack-infra | 13:32 | |
| openstackgerrit | Maciej Relewicz proposed openstack-infra/project-config: New repo for murano plugin https://review.openstack.org/329931 | 13:32 |
| *** jlanoux has quit IRC | 13:33 | |
| *** bhavik1 has quit IRC | 13:35 | |
| *** ddieterly is now known as ddieterly[away] | 13:39 | |
| *** jcoufal has joined #openstack-infra | 13:39 | |
| openstackgerrit | Alina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job https://review.openstack.org/329497 | 13:39 |
| asselin_ | rcarrillocruz, hi, do you have some time to discuss infra-ansible? There's lots of files there I didn't expect to exist. | 13:40 |
| asselin_ | rcarrillocruz, e.g. bootstrap gerrit, jjb: http://git.openstack.org/cgit/openstack-infra/infra-ansible/tree/roles | 13:41 |
| *** jcoufal_ has joined #openstack-infra | 13:41 | |
| *** sdake_ has joined #openstack-infra | 13:42 | |
| openstackgerrit | Maciej Relewicz proposed openstack-infra/project-config: New repo for murano plugin https://review.openstack.org/329931 | 13:42 |
| asselin_ | rcarrillocruz, also, with masterless puppet, why are there still so many refs to puppetmaster? | 13:42 |
| openstackgerrit | Alina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job https://review.openstack.org/329497 | 13:43 |
| rcarrillocruz | because there's no puppetmaster-less in infra ansible yet | 13:43 |
| *** jamesmcarthur has joined #openstack-infra | 13:43 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add support for testfairy plugin https://review.openstack.org/261587 | 13:44 |
| asselin_ | ok, is that going to be the same repo? or a new one created? | 13:44 |
| rcarrillocruz | it will be in the same repo | 13:44 |
| *** jcoufal has quit IRC | 13:45 | |
| *** sdake has quit IRC | 13:45 | |
| asselin_ | ok cool | 13:46 |
| *** ddieterly[away] is now known as ddieterly | 13:46 | |
| *** yamamoto has quit IRC | 13:46 | |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: Updated two projects in gerrit/projects.yaml https://review.openstack.org/329943 | 13:47 |
| *** PollyZ has joined #openstack-infra | 13:47 | |
| *** rodrigods has quit IRC | 13:47 | |
| *** rodrigods has joined #openstack-infra | 13:48 | |
| *** _ari_ has joined #openstack-infra | 13:48 | |
| *** yamamoto has joined #openstack-infra | 13:48 | |
| *** eharney has joined #openstack-infra | 13:49 | |
| *** Marquis42 has joined #openstack-infra | 13:49 | |
| *** zeih_ has quit IRC | 13:50 | |
| *** pgadiya has quit IRC | 13:51 | |
| *** pradk has joined #openstack-infra | 13:51 | |
| *** flepied1 has joined #openstack-infra | 13:51 | |
| rcarrillocruz | asselin_: i've paused a bit, since i'm watching how zuul v3 is going | 13:52 |
| rcarrillocruz | cos that will need a bigger change | 13:52 |
| rcarrillocruz | setting up launchers, getting rid of jenkins, what not | 13:52 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Fix upload-to-docs-site publisher https://review.openstack.org/329947 | 13:52 |
| rcarrillocruz | we shall see | 13:52 |
| pabelanger | AJaeger: We'll need to properly test ^, as zuul-launcher exposed the issue | 13:52 |
| asselin_ | rcarrillocruz, i thought it would just orchestrate, but use puppet to define the servers themselves | 13:52 |
| *** yamamoto has quit IRC | 13:53 | |
| rcarrillocruz | it does, but puppet modules doesn't do discovery or bootsrapping | 13:53 |
| *** yamamoto has joined #openstack-infra | 13:53 | |
| *** flepied has quit IRC | 13:53 | |
| rcarrillocruz | i.e. you run 'puppet apply zuul' | 13:53 |
| rcarrillocruz | and it will config with however you put it on site.pp | 13:53 |
| rcarrillocruz | that needs work, guessing what goes where | 13:53 |
| rcarrillocruz | it's not trivial | 13:53 |
| rcarrillocruz | also | 13:54 |
| asselin_ | what's not trivial? we have zuul setup this way for 3rd party ci | 13:54 |
| *** rcernin has quit IRC | 13:54 | |
| rcarrillocruz | cos it's not the same thing of having a an openstack infra where you can define your services in X machines | 13:54 |
| rcarrillocruz | than having an all in one/single node | 13:54 |
| *** njohnston|sleep is now known as njohnston | 13:55 | |
| asselin_ | no, I mean, with infra ansbile, you define your servers w/ names, and then puppet apply all of them using site.pp which tells each one how to 'become' based on fqdn. | 13:55 |
| *** amotoki has quit IRC | 13:55 | |
| *** amoralej|lunch is now known as amoralej | 13:56 | |
| AJaeger | pabelanger: oh, interesting... | 13:56 |
| *** links has quit IRC | 13:56 | |
| rcarrillocruz | right, but the site.pp is generated per the data the playbooks discover... if you know all the data upfront, it's trivial to just put your final site.pp and run puppet apply | 13:56 |
| rcarrillocruz | infra-ansible templatizes a site.pp based on your cloud config | 13:57 |
| rcarrillocruz | and other services files per your cloud config | 13:57 |
| asselin_ | oh I see, I missed that part | 13:57 |
| rcarrillocruz | mygerrit.lol | 13:57 |
| AJaeger | pabelanger: same problem twice in jenkins/jobs/api-jobs.yaml according to a quick grep | 13:57 |
| rcarrillocruz | then when zuul is deployed | 13:57 |
| rcarrillocruz | it makes sure that zuul.conf has a config pointing to mygerrit.lol | 13:57 |
| rcarrillocruz | so on and so forth | 13:57 |
| *** rbrndt has joined #openstack-infra | 13:57 | |
| *** hashar has joined #openstack-infra | 13:59 | |
| jroll | hey, you all run the mailing lists, right? rackspace people are trying to fix up all the bounces we generate, want to make sure I put them in touch with the right people | 13:59 |
| asselin_ | rcarrillocruz, I see...I was assuming site.pp already existed. I need to see where that gets templated. | 13:59 |
| jroll | fungi: ^ | 13:59 |
| AJaeger | pabelanger: could you check again today that nodepool uploads images everywhere, please? we've merged a change on Monday morning and it's still not everywhere (missing for sure from rax-ord) | 13:59 |
| rcarrillocruz | asselin_: i warn you, upstream follows a different pattern than gozer fork | 13:59 |
| anteaya | jroll: morning, yes fungi was working on that last night | 13:59 |
| *** dprince has joined #openstack-infra | 13:59 | |
| rcarrillocruz | gozer fork does have a complete site.pp , ala system-config site.pp | 13:59 |
| jroll | anteaya: great, thanks :) | 13:59 |
| *** xarses has quit IRC | 14:00 | |
| rcarrillocruz | upstream i use hiera to do node classification | 14:00 |
| *** _nadya_ has joined #openstack-infra | 14:00 | |
| rcarrillocruz | the site.pp is literally: | 14:00 |
| krotscheck | Can someone add me as a core to https://review.openstack.org/#/admin/projects/openstack/js-openstack-lib,access | 14:00 |
| krotscheck | Thanks :) | 14:00 |
| asselin_ | rcarrillocruz, I'm using upstream right now, but for masterless puppet, using pre-defined site.pp | 14:00 |
| anteaya | jroll: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-06-14.log.html#t2016-06-14T21:39:31 | 14:00 |
| rcarrillocruz | hiera_include('roles', []) | 14:00 |
| *** jed56 has quit IRC | 14:01 | |
| rcarrillocruz | http://git.openstack.org/cgit/openstack-infra/infra-ansible/tree/roles/bootstrap_puppetmaster/files/site.pp | 14:01 |
| rcarrillocruz | super cute | 14:01 |
| *** jed56 has joined #openstack-infra | 14:01 | |
| *** hashar has quit IRC | 14:01 | |
| fungi | yolanda: AJaeger: the challenge is that gerritbot does not (currently) log messages it receives from the irc network/server so we have no idea if it's being rejected when it tries to join additional channels. it doesn't seem to actually even confirm whether its /joins work. i want to try out patch 287056 and see if we can get a clearer picture of what's actually going on, but given that `/quote | 14:01 |
| fungi | version` reports CHANLIMIT=#:120 for freenode and the bot isn't appearing in subsequent channels past the first 120 it's a pretty safe bet | 14:01 |
| *** zhurong_ has joined #openstack-infra | 14:01 | |
| fungi | the 242/244 count i gave yesterday from parsing the logs was me stupidly not spotting there was more than one bot restart in there | 14:02 |
| AJaeger | fungi: and right now we have 120 (exactly) channels in gerritbot | 14:02 |
| *** zhurong has quit IRC | 14:02 | |
| AJaeger | fungi, I've blocked a new channel to be used: https://review.openstack.org/329129 | 14:02 |
| openstackgerrit | Merged openstack-infra/system-config: Private embargoed vulnerability announcement list https://review.openstack.org/327699 | 14:03 |
| fungi | probably a "simple" solution would be to run multiple gerritbot instances with distinct freenode accounts and split the channels between them | 14:03 |
| AJaeger | fungi, please go ahead - and tell me once I should unblock 329129 | 14:03 |
| anteaya | fungi: maybe they won't both die at the same time? | 14:04 |
| jroll | anteaya: thank you :) | 14:04 |
| fungi | jroll: i believe elarson was updating an internal rackspace ticket about the timeouts and failure to defer for some fireeye scanner there? | 14:05 |
| openstackgerrit | Maciej Relewicz proposed openstack-infra/project-config: New repo for murano plugin https://review.openstack.org/329931 | 14:05 |
| jroll | fungi: yep, that's how that log reads to me, I guess some people internally are out of sync, hopefully I straightened them out :) | 14:05 |
| fungi | jroll: i haven't caught up on my overnight e-mail yet this morning, so not sure if i have additional details yet | 14:05 |
| jroll | nod | 14:05 |
| fungi | jroll: adrian otto was also pinging ttx about the situation, so there may be multiple parties out of sync over there | 14:06 |
| *** mtanino has joined #openstack-infra | 14:06 | |
| jroll | fungi: well, seems to be a few people coordinating well, and everyone else independently getting unsubscribed and freaking out :) | 14:06 |
| *** flepied1 has quit IRC | 14:07 | |
| *** flepied has joined #openstack-infra | 14:07 | |
| anteaya | sounds like fun | 14:08 |
| elarson | jroll: I sent the ticket to the dev list. I'll send it to you on internal chat | 14:09 |
| jroll | elarson: I see it on the dev list | 14:09 |
| elarson | ok, great | 14:09 |
| jroll | I replied at the same time as you :) | 14:09 |
| *** rcernin has joined #openstack-infra | 14:09 | |
| elarson | great minds think alike :) | 14:09 |
| * jroll thinks everyone should just use personal email for this | 14:09 | |
| * jroll realizes he's off topic and steps away :P | 14:10 | |
| anteaya | jroll: no, I think confirming using personal email for open source work is very much on topic | 14:10 |
| jroll | anteaya: :) | 14:10 |
| anteaya | it saves a host of problems/work | 14:11 |
| anteaya | :) | 14:11 |
| *** ddieterly is now known as ddieterly[away] | 14:11 | |
| *** zeih has joined #openstack-infra | 14:12 | |
| fungi | jroll: it's been my solution since... well since always | 14:13 |
| *** cdent has joined #openstack-infra | 14:14 | |
| fungi | i only use my employer-provided e-mail address for a couple of very specific tasks, which do not include git/gerrit, mailing lists, authentication, et cetera | 14:14 |
| jroll | indeed | 14:14 |
| *** hongbin has joined #openstack-infra | 14:15 | |
| jeblair | JayF: do you know about http://grafana.openstack.org/ ? | 14:15 |
| electrofelix | zxiiro: quick question in https://review.openstack.org/#/c/324975/2 just about how you tested the extensions XML merging, wasn't fully sure from your response, but sounds like it works as desired (if not quite as expected based on previous interesting behaviour :p) | 14:17 |
| electrofelix | zxiiro: just checking before I approve it. | 14:17 |
| *** zz_dimtruck is now known as dimtruck | 14:17 | |
| *** zeih has quit IRC | 14:17 | |
| zxiiro | electrofelix: yeah I tested it with several of the extensions (not all) but I think enough to understand the behaviour. | 14:17 |
| jeblair | JayF: you can define new dashboards in git here: http://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana | 14:18 |
| zxiiro | electrofelix: I configured several plugins and then commented them out to see if tehy got removed. and it did | 14:18 |
| electrofelix | so extensions replaces the XML sub tags | 14:18 |
| electrofelix | sounds good, approving so | 14:18 |
| odyssey4me | yolanda AJaeger (or any other project-config cores) if you have a moment I'd appreciate a review of https://review.openstack.org/329892 | 14:18 |
| *** rcernin has quit IRC | 14:18 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:18 | |
| zxiiro | electrofelix: yeah extensions just uses a null tag <whatever.feature/> for enabling and if it doesn't exist it clears it | 14:18 |
| *** trown is now known as trown|mtg | 14:18 | |
| *** ddieterly[away] is now known as ddieterly | 14:19 | |
| *** nmagnezi has joined #openstack-infra | 14:19 | |
| *** oanson has quit IRC | 14:19 | |
| hongbin | AJaeger: there? | 14:22 |
| AJaeger | hongbin: yes | 14:22 |
| hongbin | AJaeger: Have time to discuss this patch: https://review.openstack.org/#/c/326306/? | 14:23 |
| *** xarses has joined #openstack-infra | 14:23 | |
| AJaeger | hongbin: let me first review odyssey4me and then I'll look at yours... | 14:24 |
| hongbin | ack | 14:24 |
| openstackgerrit | Merged openstack-infra/zuul: Refactor local_action SCP task into function https://review.openstack.org/325601 | 14:24 |
| *** Hal has joined #openstack-infra | 14:24 | |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: Remove unused plugins dir https://review.openstack.org/329581 | 14:25 |
| *** Hal is now known as Guest77643 | 14:25 | |
| *** dprince has quit IRC | 14:25 | |
| openstackgerrit | Merged openstack-infra/zuul: Add log_path to ansible.cfg https://review.openstack.org/329525 | 14:25 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Update Git SCM so that extensions are configured correctly. https://review.openstack.org/324975 | 14:25 |
| AJaeger | hongbin: ok, reviewed and ready to discuss | 14:26 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: move ftp/scp staging out of ansible dir https://review.openstack.org/329607 | 14:26 |
| openstackgerrit | Merged openstack-infra/zuul: Upload ansible playbook directory to logs server https://review.openstack.org/325602 | 14:26 |
| *** winggundamth_ has quit IRC | 14:26 | |
| hongbin | AJaeger: I just want to know how to proceed | 14:26 |
| AJaeger | hongbin: but better to talk with pleia2, she has done the last IRC renames. | 14:26 |
| jeblair | i'm gracefully stopping zl01 and zl02. | 14:27 |
| hongbin | k | 14:27 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: ensure that console/ansible publishing is last https://review.openstack.org/329608 | 14:27 |
| AJaeger | hongbin: since you need to change system-config as well, we need to do this in the proper order, so perhaps (but let pleia2 confirm): 1) create new channels, 2) rename in system-config; 3) remove old channels | 14:27 |
| AJaeger | And this together with the redirect of your old channel if you want it... | 14:28 |
| hongbin | I see | 14:28 |
| *** pt_15 has joined #openstack-infra | 14:28 | |
| *** yujunz has joined #openstack-infra | 14:29 | |
| rcarrillocruz | asselin_: putting puppetmaster-less should be easy, i thought about it a while ago, but haven't tackled it because of $WORK. I'd add a flag 'puppetmaster_less', if true, do not run bootstrap_puppetamaster playbook and run install_modules.sh in all hosts. Also, branch the puppet.conf.j2 accordingly, so nodes are not pointing to a master | 14:29 |
| rcarrillocruz | working on puppetdb stuff now | 14:29 |
| rcarrillocruz | ansible-puppet should work on puppetdb 3x | 14:30 |
| asselin_ | rcarrillocruz, ok yeah, right now I just use the default puppet.conf after install_puppet.sh | 14:30 |
| rcarrillocruz | so testing stuff , but also decoupling stuff as i find it, e.g. adding a puppetdb class on puppet_openstackci | 14:30 |
| rcarrillocruz | cos right now, it's bundled all the logic on openstack_project::puppetdb | 14:30 |
| *** vhosakot has joined #openstack-infra | 14:31 | |
| *** rcernin has joined #openstack-infra | 14:31 | |
| pabelanger | jeblair: ack | 14:33 |
| *** jsavak has quit IRC | 14:34 | |
| pabelanger | AJaeger: Sure, I can confirm uploads worked | 14:34 |
| pabelanger | AJaeger: will also update the review | 14:34 |
| jeblair | pabelanger, AJaeger: would you like me to refresh https://review.openstack.org/293706 ? | 14:35 |
| *** jsavak has joined #openstack-infra | 14:35 | |
| *** yujunz has quit IRC | 14:35 | |
| pabelanger | jeblair: Yes, please. On my list to test, but never did | 14:36 |
| *** burgerk has joined #openstack-infra | 14:37 | |
| rcarrillocruz | nibalizer: heya, in infra's setup, how's ssl for puppetdb? Are nodes using the previous puppet agent certs or those were removed and is apache being a proxy? | 14:38 |
| rcarrillocruz | https://docs.puppet.com/puppetdb/4.1/connect_puppet_apply.html | 14:38 |
| rcarrillocruz | unless there's no SSL at all, but running puppet apply on system-config/site.pp does set ssl on jetty.ini | 14:39 |
| nibalizer | yep puppet agent certs | 14:40 |
| rcarrillocruz | k | 14:40 |
| rcarrillocruz | i'll set up a puppetmaster to issue certs to do my tests | 14:40 |
| rcarrillocruz | btw, pushed patchset for ansible-puppet thingy | 14:40 |
| AJaeger | pabelanger: http://logs.openstack.org/23/327523/1/check/gate-heat-install-guide/87db28e/console.html - this is still the old content, so rax-ord is not uptodate | 14:41 |
| *** sree has quit IRC | 14:42 | |
| AJaeger | jeblair: yes, please! | 14:42 |
| jeblair | AJaeger, pabelanger, fungi: updated -- remote: https://review.openstack.org/293706 Add a simple status webapp | 14:44 |
| AJaeger | pabelanger: if you say uploads worked, why did I get an old image for that? | 14:44 |
| pabelanger | AJaeger: uploads don't always work. It is possible they timed out, I'll check shortly when we last uploaded | 14:45 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher https://review.openstack.org/329983 | 14:46 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher https://review.openstack.org/329984 | 14:46 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher https://review.openstack.org/329985 | 14:46 |
| AJaeger | pabelanger: thanks | 14:46 |
| *** hdaniel has quit IRC | 14:46 | |
| *** yamamoto has quit IRC | 14:46 | |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: New repo for murano plugin https://review.openstack.org/329931 | 14:47 |
| AJaeger | pabelanger: looking at your patch series, is there a way to enforce this? Or document it at least? Perhaps in the REVIEW document? | 14:48 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Added amazon web services to credentials bindings https://review.openstack.org/326106 | 14:49 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add console-log validator https://review.openstack.org/329988 | 14:49 |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: New repo for murano plugin https://review.openstack.org/329931 | 14:49 |
| *** gordc has joined #openstack-infra | 14:50 | |
| AJaeger | pabelanger: thanks for 329988! | 14:51 |
| *** YorikSar has quit IRC | 14:51 | |
| JayF | jeblair: now I do :D thanks! | 14:51 |
| pabelanger | AJaeger: Still failing, but will be easier to find the console-logs now | 14:51 |
| AJaeger | ;) | 14:52 |
| *** YorikSar has joined #openstack-infra | 14:53 | |
| *** zhurong_ has quit IRC | 14:54 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add console-log validator https://review.openstack.org/329988 | 14:56 |
| *** Julien-zte has joined #openstack-infra | 14:57 | |
| *** croelandt is now known as croelandt|afk | 14:58 | |
| *** jsavak has quit IRC | 14:58 | |
| *** tonytan4ever has joined #openstack-infra | 14:58 | |
| *** jsavak has joined #openstack-infra | 14:59 | |
| *** yamahata has quit IRC | 14:59 | |
| *** yamahata has joined #openstack-infra | 15:00 | |
| *** hashar has joined #openstack-infra | 15:00 | |
| *** inc0 has quit IRC | 15:00 | |
| anteaya | notmorgan: you about yet? | 15:00 |
| anteaya | notmorgan: I'm wondering if you can help me figure out a thing with api auth in #storyboard | 15:01 |
| *** psachin has quit IRC | 15:04 | |
| *** afred312_ has joined #openstack-infra | 15:04 | |
| *** abregman has quit IRC | 15:04 | |
| AJaeger | anteaya: do you have time to review and approve https://review.openstack.org/#/c/329983, please? Once that's in and on all servers, we can merge the dependent changes - so, one after each other to not overwhelm Jenkins ;) | 15:05 |
| *** afred312 has quit IRC | 15:05 | |
| anteaya | AJaeger: right after the storyboard meeting, sure | 15:05 |
| *** _nadya_ has quit IRC | 15:07 | |
| *** inc0 has joined #openstack-infra | 15:07 | |
| *** sdake_ has quit IRC | 15:08 | |
| *** sdake has joined #openstack-infra | 15:10 | |
| mordred | AJaeger: you know - soon we will no longer have to worry about overwhelming jenkins | 15:11 |
| *** sdake has quit IRC | 15:11 | |
| *** yamamoto has joined #openstack-infra | 15:12 | |
| jeblair | we could just wait until jenkins is gone to approve those | 15:12 |
| AJaeger | mordred: I look forward to that ;) | 15:14 |
| AJaeger | jeblair: let's get them in to help us get better logs... | 15:15 |
| jeblair | AJaeger: well, they aren't a pre-requisite for zuulv2.5 -- we have a thing in zuul that will automatically make that transformation for us. getting those changes in just normalizes the config with what zuul does automatically | 15:17 |
| jeblair | AJaeger: https://review.openstack.org/329608 | 15:17 |
| *** dimtruck is now known as zz_dimtruck | 15:17 | |
| *** kzaitsev_mb has quit IRC | 15:17 | |
| *** xwizard has quit IRC | 15:18 | |
| AJaeger | jeblair: interesting! | 15:18 |
| *** zz_dimtruck is now known as dimtruck | 15:18 | |
| AJaeger | jeblair: you're doing far too much magic for me to follow ;) | 15:19 |
| *** xwizard has joined #openstack-infra | 15:19 | |
| *** esikachev has quit IRC | 15:19 | |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config: Zun: rename higgins to zun (1) https://review.openstack.org/326306 | 15:20 |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config: Zun: rename higgins to zun (2) https://review.openstack.org/329247 | 15:20 |
| *** bhavik has joined #openstack-infra | 15:20 | |
| *** skraynev has quit IRC | 15:20 | |
| *** arxcruz has quit IRC | 15:21 | |
| *** oanson has joined #openstack-infra | 15:21 | |
| *** abregman has joined #openstack-infra | 15:22 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add console-log validator https://review.openstack.org/329988 | 15:23 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher https://review.openstack.org/330013 | 15:23 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher https://review.openstack.org/330014 | 15:23 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher https://review.openstack.org/330015 | 15:23 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Ensure console-log is our last publisher https://review.openstack.org/330016 | 15:23 |
| pabelanger | there | 15:23 |
| pabelanger | with gate job working too | 15:23 |
| *** ifarkas has quit IRC | 15:23 | |
| jeblair | pabelanger: i hesitate to say this, but if we continue with the block/always approach, i'm not sure if we can remove all of the transformPublishers bit -- we at least still need to split publishers into block/always, even if we don't reorder them | 15:24 |
| *** tonytan4ever has quit IRC | 15:24 | |
| *** Na3iL has quit IRC | 15:24 | |
| jeblair | notmorgan: we haven't merged the timeout thing yet have we? | 15:26 |
| *** YorikSar has quit IRC | 15:26 | |
| *** andreas_s has quit IRC | 15:26 | |
| pabelanger | jeblair: Right, I do want us to land the copy zuul logs and upload with console-logs publisher stuff, because we need the ansible logs. But moving forward, I do think breaking that upload out in to a dedicated zuul_logs rsync process is something we need. | 15:27 |
| *** xarses_ has joined #openstack-infra | 15:28 | |
| pabelanger | the main reason, today we won't have ansible logs for our publisher tasks. So there is a little bit of a blind spot in our ansible.log | 15:28 |
| *** flepied has quit IRC | 15:29 | |
| *** xarses has quit IRC | 15:29 | |
| AJaeger | pabelanger: ovh-bhs1 seems very old, the jenkins slave_script does not even exist on an image that my job just run on ;( and rax-ord looks outdated but not that old | 15:30 |
| openstackgerrit | Hongbin Lu proposed openstack-infra/system-config: Rename IRC channel of container service https://review.openstack.org/330017 | 15:30 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: ensure log publishing always runs https://review.openstack.org/329612 | 15:32 |
| *** amitgandhinz has quit IRC | 15:33 | |
| jeblair | pabelanger: we do have those logs | 15:33 |
| *** amitgandhinz has joined #openstack-infra | 15:34 | |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: set keep-hierarchy on log copying https://review.openstack.org/329712 | 15:34 |
| jeblair | pabelanger: the logs go right up to "copy the ansible directory into the upload staging dir" -- so the only thing we are missing is the results of that copy and the final upload. i don't think we will ever get the logs of the final upload itself, more or less by definition :) | 15:34 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add flaky test report and allow empty options to JUnit publisher https://review.openstack.org/321305 | 15:34 |
| *** shardy has quit IRC | 15:35 | |
| *** sorantis has quit IRC | 15:36 | |
| *** hashar has quit IRC | 15:36 | |
| *** yamamoto has quit IRC | 15:36 | |
| *** yamamoto has joined #openstack-infra | 15:36 | |
| pabelanger | AJaeger: possible, I haven't looked yet | 15:37 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Implement Xenial/CentOS jobs for OpenStack-Ansible https://review.openstack.org/329892 | 15:37 |
| *** dizquierdo has quit IRC | 15:37 | |
| dasm | hey folks. i'm trying to use logstash but it doesn't cooperate with me. | 15:38 |
| pabelanger | jeblair: I don't want to distract on the current effort for publishing logs, but have an idea how to maybe fix that. Happy to talk about it more once jenkins have been turned off :D | 15:38 |
| dasm | i opened private browser, with clean history, and nothing happens | 15:38 |
| dasm | in Network tab (Chrome) I see: http://paste.openstack.org/show/516300/ | 15:38 |
| dasm | is it known problem? | 15:38 |
| odyssey4me | AJaeger anteaya thanks for the reviews - https://review.openstack.org/329892 has been updated | 15:39 |
| pabelanger | AJaeger: Wow, we haven't updated ovh-bhs1 in 22 days | 15:39 |
| *** ddieterly is now known as ddieterly[away] | 15:39 | |
| pabelanger | something going on there | 15:39 |
| *** garyk has quit IRC | 15:39 | |
| *** kzaitsev_mb has joined #openstack-infra | 15:39 | |
| AJaeger | oops ;( | 15:39 |
| pabelanger | AJaeger: reuploading ubuntu-trusty now | 15:39 |
| pabelanger | checking debug logs | 15:40 |
| dasm | probably, problem is around: "[FIELDDATA] Data too large, data for [@timestamp] would be larger than limit of [22469214208/20.9gb]];" | 15:40 |
| jeblair | pabelanger: here's the log from my local test: http://paste.openstack.org/show/516301/ | 15:40 |
| *** yamamoto has quit IRC | 15:40 | |
| jeblair | pabelanger: the other changes have landed, i'm going to start zl01 now | 15:41 |
| *** YorikSar has joined #openstack-infra | 15:42 | |
| pabelanger | Hmm, Unauthorized. Check username, password and tenant name/id. Found that in nodepool-builder.log http://paste.openstack.org/show/516303/ | 15:42 |
| pabelanger | back on 06.12 | 15:42 |
| *** markusry has quit IRC | 15:42 | |
| *** vincentll has quit IRC | 15:43 | |
| *** markusry has joined #openstack-infra | 15:45 | |
| *** mikelk has quit IRC | 15:45 | |
| AJaeger | ;( | 15:45 |
| *** markusry has quit IRC | 15:46 | |
| *** markusry has joined #openstack-infra | 15:47 | |
| *** markusry has quit IRC | 15:48 | |
| *** yamamoto has joined #openstack-infra | 15:48 | |
| *** ddieterly[away] is now known as ddieterly | 15:49 | |
| jeblair | pabelanger, Shrews: http://logs.openstack.org/66/325966/8/check/gate-charm-tempest-pep8/64c8cb7/ | 15:49 |
| AJaeger | bbl | 15:50 |
| Shrews | nice | 15:50 |
| *** markusry has joined #openstack-infra | 15:50 | |
| *** pcaruana has quit IRC | 15:50 | |
| pabelanger | jeblair: very nice | 15:50 |
| *** roxanaghe has joined #openstack-infra | 15:51 | |
| pabelanger | should make debugging a little easier | 15:51 |
| Shrews | jeblair: those zuul_runner parameters are funky | 15:51 |
| jeblair | we should make '.log' a text/plain extension on that server | 15:52 |
| pabelanger | mordred: clarkb: I think something odd is happening in ovh-bhs1: http://paste.openstack.org/show/516307/ | 15:52 |
| *** trown|mtg is now known as trown | 15:52 | |
| *** dmk0202 has quit IRC | 15:52 | |
| *** Guest77643 has quit IRC | 15:52 | |
| jeblair | Shrews: yeah, i think we may want to use the 'safe' yaml enecoder ? | 15:52 |
| pabelanger | mordred: clarkb: if reading right, shade says the images is ready in ovh-bhs1, but nodepool still reports is over 23 days old | 15:52 |
| mordred | yes. you want to use the safe one | 15:52 |
| *** croelandt|afk is now known as croelandt | 15:53 | |
| *** rcernin has quit IRC | 15:53 | |
| *** Sukhdev has joined #openstack-infra | 15:53 | |
| *** markusry has quit IRC | 15:54 | |
| jeblair | but i really like that it uses the &/* reference | 15:54 |
| *** gsagie has joined #openstack-infra | 15:54 | |
| jeblair | because otherwise that playbook would be HUGE | 15:54 |
| gsagie | I am not sure where this is related to, but i think the 1000 chars restriction in the talks submission is not working right (it complains for much less then 1000 chars) | 15:54 |
| Shrews | jeblair: nothing in the 'block' block in the post-playbook? http://logs.openstack.org/66/325966/8/check/gate-charm-tempest-pep8/64c8cb7/_zuul_ansible/post_playbook | 15:54 |
| jeblair | Shrews: correct -- that means that job had no other publishers than the console log | 15:55 |
| gsagie | not sure if its related here :) | 15:55 |
| anteaya | gsagie: https://bugs.launchpad.net/openstack-org | 15:55 |
| jeblair | Shrews: lemme look for another job with maybe a more interesting playbook | 15:55 |
| Shrews | jeblair: i'm surprised that always:block: (not block:always:) works | 15:55 |
| *** tesseract has quit IRC | 15:55 | |
| *** netsin has joined #openstack-infra | 15:55 | |
| gsagie | anteaya: thanks! | 15:56 |
| anteaya | gsagie: I suggest you file a bug with them, the folks in charge of that decision aren't in this channel (much) and do most of their intake via launchpad | 15:56 |
| jeblair | Shrews: ya, i wasn't sure it would, so i tested it. | 15:56 |
| anteaya | gsagie: welcome | 15:56 |
| *** amitgandhinz has quit IRC | 15:56 | |
| *** tonytan4ever has joined #openstack-infra | 15:56 | |
| jeblair | Shrews: i've become comfortable with it by thinking about the inverse -- how would you explain to someone that the thing you put in the 'always' block actually *never* ran. *that* would be even more counterintuitive. :) | 15:56 |
| *** acabot has quit IRC | 15:57 | |
| *** amitgandhinz has joined #openstack-infra | 15:57 | |
| openstackgerrit | Lukas Bednar proposed openstack-infra/git-review: Fix AttributeError when can not connect to Gerrit https://review.openstack.org/318181 | 15:57 |
| *** tbarron is now known as tbarron|afk | 15:57 | |
| Shrews | jeblair: as long as ansible never changes to enforce the order, good by me :) | 15:58 |
| jeblair | Shrews: oh, you're talking about the order | 15:58 |
| Shrews | yeah | 15:58 |
| *** Rockyg has joined #openstack-infra | 15:58 | |
| jeblair | Shrews: i thought you were talking about empty blocks -- that's what i was yapping on about | 15:58 |
| jeblair | Shrews: i think it's a dictionary, so order doesn't matter | 15:58 |
| *** fernnest__ has joined #openstack-infra | 15:59 | |
| jeblair | task: [ {always:...; block:...} ] | 15:59 |
| jeblair | rather -- tasks: [ {always:...; block:...} ] | 15:59 |
| Shrews | jeblair: i wonder if it would make the log easier to read if we started using names on the tasks | 16:00 |
| mordred | jeblair: fwiw, we've got an OrderedDict yaml serializer in flamel - so although order doesn't matter, if we think that a certain order would be more readable, it's not hard to achieve | 16:00 |
| *** oanson has quit IRC | 16:01 | |
| jeblair | Shrews: that's a good idea, and naming them by their jjb macro name would be useful, however, i think we lose that information by the time jjb has fully parsed the job... we might need some jjb modification to do that, and that may not be worth the effort. | 16:01 |
| *** jcoufal has joined #openstack-infra | 16:01 | |
| jeblair | (also, if you don't use macros, then the publishers are completely anonymous anyway) | 16:02 |
| *** javeriak has joined #openstack-infra | 16:02 | |
| *** afred312 has joined #openstack-infra | 16:03 | |
| *** jcoufal_ has quit IRC | 16:03 | |
| jeblair | Shrews, mordred, pabelanger: a more interesting post playbook: http://logs.openstack.org/2e/2e557ad97ce4b44fa18e51f31188d5b5bf8915fc/post/rally-branch-tarball/cc15a2f/_zuul_ansible/post_playbook | 16:03 |
| *** jsavak has quit IRC | 16:04 | |
| *** jsavak has joined #openstack-infra | 16:04 | |
| jeblair | if you look at the log for that one, you can see the 'block' running first | 16:04 |
| *** sabari has joined #openstack-infra | 16:04 | |
| *** afred312_ has quit IRC | 16:04 | |
| *** sdake has joined #openstack-infra | 16:05 | |
| *** Julien-zte has quit IRC | 16:06 | |
| *** javeriak_ has joined #openstack-infra | 16:06 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul: Turn off Ansible host fact gathering https://review.openstack.org/329688 | 16:06 |
| *** javeriak has quit IRC | 16:07 | |
| jeblair | i'm happy enough with this to go aheand and continue the rollout to the other zl hosts -- sound good? | 16:07 |
| Shrews | jeblair: pabelanger: ^^^ | 16:07 |
| *** abregman has quit IRC | 16:07 | |
| *** ramishra has joined #openstack-infra | 16:07 | |
| jeblair | Shrews: ++. is that a config file option by any chance? | 16:07 |
| *** ramishra has quit IRC | 16:07 | |
| Shrews | jeblair: mmm, sort of? it's a bit wonky | 16:07 |
| Shrews | jeblair: http://docs.ansible.com/ansible/intro_configuration.html#gathering | 16:08 |
| *** Julien-zte has joined #openstack-infra | 16:09 | |
| Shrews | jeblair: we could set it to 'explicit' i guess | 16:09 |
| *** krotscheck is now known as krotscheck_dcm | 16:09 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Add a simple status webapp https://review.openstack.org/293706 | 16:09 |
| *** jsavak has quit IRC | 16:09 | |
| *** jsavak has joined #openstack-infra | 16:10 | |
| Shrews | jeblair: i can change it to set that, if you prefer | 16:10 |
| * rcarrillocruz thinks that explicit should be the default on ansible/ansible, but oh well... | 16:10 | |
| *** thiagop has joined #openstack-infra | 16:10 | |
| jeblair | Shrews: your call | 16:11 |
| *** YorikSar has quit IRC | 16:12 | |
| *** mdrabe has quit IRC | 16:12 | |
| Shrews | jeblair: for consistencies sake, i'll move it to config | 16:12 |
| *** YorikSar has joined #openstack-infra | 16:13 | |
| *** ramishra has joined #openstack-infra | 16:13 | |
| *** tonytan4ever has quit IRC | 16:14 | |
| anteaya | AJaeger: so I said I would review and consider approving 329983, then there was discussion, do we still want me to review and merge it? | 16:14 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul: Turn off Ansible host fact gathering https://review.openstack.org/329688 | 16:14 |
| rcarrillocruz | nibalizer: the puppetdb routes.yaml, do we manage it on git? or that's set by hand? | 16:15 |
| Shrews | jeblair: there ya go | 16:15 |
| rcarrillocruz | can't seem to get hits for it on system-config | 16:15 |
| anteaya | AJaeger: I'm +2, I'll leave it up to you to merge it that is still the concensous | 16:15 |
| jeblair | Shrews: cool -- missing a final \n | 16:15 |
| Shrews | gah | 16:16 |
| *** ddieterly is now known as ddieterly[away] | 16:16 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul: Turn off Ansible host fact gathering https://review.openstack.org/329688 | 16:16 |
| *** piet has joined #openstack-infra | 16:17 | |
| *** tonytan4ever has joined #openstack-infra | 16:19 | |
| jeblair | i have started zl02 and gracefulededed zl03 | 16:20 |
| *** jaosorior has quit IRC | 16:20 | |
| jeblair | also graceful zlstatic01 | 16:21 |
| zaro | morning | 16:21 |
| *** ddieterly[away] is now known as ddieterly | 16:21 | |
| jeblair | and restarted zlstatic01 | 16:21 |
| jeblair | mordred: want to +3 https://review.openstack.org/329688 | 16:22 |
| *** derekh has quit IRC | 16:22 | |
| *** mescanef has quit IRC | 16:22 | |
| mordred | jeblair: done | 16:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul: Add custom serializer that supports OrderedDicts https://review.openstack.org/330052 | 16:22 |
| mordred | jeblair: there is handling ordered dicts and also rendering the unicode strings as normal strings and not the python/unicode things | 16:23 |
| zaro | snarwade: i don’t believe jjb supports jenkins kubernets plugin yet. | 16:23 |
| jeblair | mordred: do we care about the ordered dicts? | 16:24 |
| *** kmartin has joined #openstack-infra | 16:24 | |
| mordred | jeblair: dunno. that's mostly a "if we think that block then always is easier to read" patch | 16:24 |
| anteaya | morning zaro | 16:25 |
| mordred | jeblair: since I had the code to implement it already somewhere else | 16:25 |
| *** tonytan4ever has quit IRC | 16:26 | |
| jeblair | mordred: okay. i'm going to vote a decided +0 on that patch. :) | 16:27 |
| jeblair | (basically, it doesn't bother me, but if others like it, great!) | 16:28 |
| *** mdrabe has joined #openstack-infra | 16:28 | |
| openstackgerrit | James Page proposed openstack-infra/project-config: Add specifications repository for charms https://review.openstack.org/330054 | 16:29 |
| openstackgerrit | Merged openstack-infra/zuul: Turn off Ansible host fact gathering https://review.openstack.org/329688 | 16:29 |
| nibalizer | rcarrillocruz: I don't futz with routes.yaml | 16:29 |
| nibalizer | as a general rule | 16:29 |
| nibalizer | does puppetdb even have one? I think without a puppetmaster routes.yaml doesn't really do anything | 16:30 |
| *** sarob has joined #openstack-infra | 16:30 | |
| *** dprince has joined #openstack-infra | 16:31 | |
| *** sarob has quit IRC | 16:31 | |
| *** piet has quit IRC | 16:32 | |
| *** sarob has joined #openstack-infra | 16:32 | |
| *** matrohon has quit IRC | 16:33 | |
| *** piet has joined #openstack-infra | 16:33 | |
| * notmorgan wakes up and glares at subunit | 16:33 | |
| notmorgan | you know... it's bad when you have a dream ... in python :( | 16:33 |
| mordred | notmorgan: hahaha | 16:33 |
| notmorgan | also, i lost my computer-fixing screw driver set. it's going to make it hard to build this system up tonight. | 16:34 |
| *** ihrachys has quit IRC | 16:34 | |
| *** hongbin has quit IRC | 16:34 | |
| notmorgan | but tomorrow... or the next day... actually having a system i can run openstack on locally for zuul/nodepool testing = win. | 16:34 |
| Shrews | notmorgan: that would be nice | 16:35 |
| * notmorgan needs to ask red hat for a workstation too so the ambient temperature of my place can be raised by 5 degrees *and* I have more CPU capacity for things. | 16:35 | |
| *** sarob has quit IRC | 16:35 | |
| *** zeih has joined #openstack-infra | 16:35 | |
| *** sarob has joined #openstack-infra | 16:36 | |
| notmorgan | Shrews: yeah, well i figured i needed something I can actually mirror some infra workloads on- so yay 64GB machine [also, I kindof want to play OverWatch w/ friends... so DUAL PURPOSE!] | 16:36 |
| *** piet has quit IRC | 16:36 | |
| notmorgan | plus it's fun to make my friends in the game industry's heads explode when I describe insane virtualization setups with PCI passthrough and processor affinity | 16:36 |
| *** piet has joined #openstack-infra | 16:37 | |
| pabelanger | just catching up on backscroll | 16:37 |
| jeblair | notmorgan: i don't want to throw water on your building a supercomputer or anything -- so go right ahead -- i just want to say that it's pretty important to me that zuul and nodepool be runnable without actual clouds --- | 16:38 |
| notmorgan | jeblair: i can run them here on my laptop | 16:38 |
| jeblair | notmorgan: so with nodepool, you can run it with a completely fake provider system, and zuul i mostly exercise through its unit/functional tests (which have the gerrit/jenkins stuff faked out) | 16:38 |
| notmorgan | jeblair: the issue is when i want to put more workload than 16GB of ram will allow. | 16:38 |
| *** Apoorva has joined #openstack-infra | 16:39 | |
| notmorgan | jeblair: testing is great, but sometimes you want to cycle through a lot more than the laptop will allow [docker, etc on this laptop has proven painful] | 16:39 |
| *** trown is now known as trown|lunch | 16:39 | |
| *** eezhova has quit IRC | 16:39 | |
| *** infra-red has joined #openstack-infra | 16:41 | |
| mordred | I think it's very important for all of my testing to run on jeblair's laptop | 16:41 |
| *** infra-red has quit IRC | 16:41 | |
| *** zeih has quit IRC | 16:41 | |
| notmorgan | jeblair: fake providers are good, but also real providers are important too. if you only ever test the fake provider except when you drop it in production, i'm going to argue you're doing it wrong | 16:41 |
| notmorgan | and i'd like to not need to lean on devstack/infra for every single test that exceeds my capabilities locally :) | 16:41 |
| * notmorgan always runs this stuff locally and lean first. | 16:42 | |
| *** sdake has quit IRC | 16:42 | |
| jeblair | notmorgan: then i will happily do it wrong | 16:42 |
| jeblair | notmorgan: this method of testing has been *extremely* good at keeping us from dropping broken code into production | 16:42 |
| jeblair | a change that passes zuul's unit tests is absolutely fit for production | 16:43 |
| notmorgan | jeblair: i'm not advocating changing the testing model ;) I'm advocating adding more rigerous testing in :) | 16:43 |
| notmorgan | for me at least. | 16:44 |
| melwitt | logstash.openstack.org seems to be stuck, unable to load events | 16:44 |
| * notmorgan shrugs. | 16:44 | |
| jeblair | because zuul's interactions with other systems are so thoroughly replicated, and we're able to test completx series of events that are very difficult to replicate manually | 16:44 |
| *** jlanoux has joined #openstack-infra | 16:44 | |
| jeblair | notmorgan: like i said, i'm not trying to discourage you from anything -- but i am resistant to the idea that you need a cloud to test nodepool or zuul | 16:45 |
| *** _nadya_ has joined #openstack-infra | 16:45 | |
| mordred | I think zuul's unit tests should be comprehensive and sufficient. however, I think that at times depending on working style, pointing things at real resources might be the thing that a human needs to do to poke at something, and that should also be totally possible | 16:46 |
| jeblair | yep, i'm not saying otherwise | 16:46 |
| *** yolanda has quit IRC | 16:46 | |
| notmorgan | mordred: ++ | 16:46 |
| notmorgan | jeblair: it's def. more my workflow :) | 16:46 |
| anteaya | melwitt: confirmed | 16:47 |
| notmorgan | jeblair: and unit/functoional/acceptance tests are always where it starts | 16:47 |
| jeblair | it is, however, *extremely* important to me that we not *require* that. that is a huge barrier to contribution, and if it is not necessary, which it is not, we should not erect it | 16:47 |
| *** bpokorny has joined #openstack-infra | 16:47 | |
| notmorgan | jeblair: and if those SUCK - we're wrong | 16:47 |
| notmorgan | jeblair: or insufficient | 16:47 |
| anteaya | jesusaur: clarkb do either of you have time to take a peek at our logstash status? | 16:47 |
| *** bpokorny has quit IRC | 16:47 | |
| notmorgan | jeblair: i think what i'm saying is i'm in very strong agreement, but want more flexibility personally to poke at things. | 16:47 |
| *** ddieterly is now known as ddieterly[away] | 16:47 | |
| anteaya | it doesn't seem to be logstashing as expected | 16:48 |
| *** bpokorny has joined #openstack-infra | 16:48 | |
| jeblair | notmorgan: good -- i only mentioned it because i don't know that you know that i don't think it's a technical requirement. :) | 16:48 |
| *** BobBall is now known as BobBall_AWOL | 16:48 | |
| jeblair | and i don't want there to be a meme that you need a cloud to work on this stuff | 16:48 |
| jeblair | if you want one, go right ahead. if you don't have or want one, you can be just as effective without it | 16:49 |
| *** jlanoux has quit IRC | 16:50 | |
| *** raunak has joined #openstack-infra | 16:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Add task name of builder for zuul_runner https://review.openstack.org/330082 | 16:50 |
| notmorgan | jeblair: i also frankly can't keep long running vms in clouds [it adds up stupid fast] for faster/more concurrent tests. I've had exactly one computer (a laptop) to do everything for ~3yrs. I'm happy to be able to fire off a series of tests on a local machine [esp. when the gate is busy] and not have to walk away to look at something else/coffee [or like when i was doing the zuul loop test, that was | 16:51 |
| notmorgan | 2 days of being able to do almost nothing but irc since the laptop was busy] | 16:51 |
| jesusaur | anteaya: in what way is it not working as expected? | 16:51 |
| notmorgan | jeblair: so this has been on my todo list for a long time :) | 16:51 |
| jeblair | notmorgan: oh definitely | 16:52 |
| anteaya | jesusaur: melwitt reports and I confirm that logstash.o.o is not loading events | 16:52 |
| anteaya | jesusaur: just the round spinning thing, which is entertaining but not as productive as it could be | 16:52 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/system-config: Apt mirror for MariaDB https://review.openstack.org/307831 | 16:52 |
| jesusaur | anteaya: hm, yeah I'm seeing that too :( | 16:53 |
| *** _nadya_ has quit IRC | 16:53 | |
| jesusaur | but http://logstash.openstack.org/elasticsearch/_cluster/health?pretty shows that elasticsearch is happy | 16:53 |
| odyssey4me | pabelanger ^ I'm not entirely sure that I have the link path right for https://review.openstack.org/307831 - as I recall there was also some discussion about naming with more from mordred, but that feedback never made it into the review :/ | 16:53 |
| openstackgerrit | Lukas Bednar proposed openstack-infra/jenkins-job-builder: Builders: Add ansible-playbook builder https://review.openstack.org/322243 | 16:54 |
| *** mtanin___ has joined #openstack-infra | 16:54 | |
| odyssey4me | it'd be really nice to have a search mechanism for eavesdrop :) | 16:54 |
| anteaya | jesusaur: interesting thanks for confirming | 16:54 |
| jesusaur | anteaya: clarkb: logstash looks like it needs a rooter to poke it | 16:55 |
| *** mtanino has quit IRC | 16:55 | |
| odyssey4me | oh neat, google actually works for searching eavesdrop | 16:55 |
| *** yamahata has quit IRC | 16:56 | |
| pabelanger | odyssey4me: sure, I'll try to review it later today | 16:57 |
| jeblair | pabelanger: i don't understand https://review.openstack.org/329947 | 16:57 |
| *** _nadya_ has joined #openstack-infra | 16:58 | |
| notmorgan | odyssey4me: it's pretty cool :) | 16:58 |
| pabelanger | jeblair: I'll update the commit message. We are seeing Exception: Target path / is not below site root from zuul-launcher | 16:59 |
| *** woodster_ has quit IRC | 16:59 | |
| *** NobodyCam has quit IRC | 16:59 | |
| *** mgkwill has quit IRC | 16:59 | |
| *** petems has quit IRC | 16:59 | |
| jeblair | pabelanger: right, so maybe we need to think a bit more about what zuul-launcher is doing | 16:59 |
| pabelanger | jeblair: I haven't actually tested . however | 16:59 |
| *** igorbelikov has quit IRC | 16:59 | |
| *** johnny_ has joined #openstack-infra | 17:00 | |
| *** cody-somerville has joined #openstack-infra | 17:00 | |
| pabelanger | jeblair: I don't mind the revisi | 17:00 |
| pabelanger | revisit* | 17:00 |
| *** erlon has quit IRC | 17:00 | |
| jeblair | pabelanger: basically, at the moment, errors are zuul-launcher's fault until proven otherwise :) | 17:00 |
| *** dtantsur is now known as dtantsur|afk | 17:00 | |
| *** NobodyCam has joined #openstack-infra | 17:00 | |
| *** woodster_ has joined #openstack-infra | 17:00 | |
| pabelanger | jeblair: sure. In that case, I think jenkins will silently strip the first char, if it is '/' | 17:00 |
| *** mgkwill has joined #openstack-infra | 17:01 | |
| pabelanger | AJaeger: ovh-bhs1 ubuntu-trusty uploaded correctly | 17:01 |
| *** erlon has joined #openstack-infra | 17:01 | |
| *** igorbelikov has joined #openstack-infra | 17:01 | |
| *** mtanino has joined #openstack-infra | 17:01 | |
| *** petems has joined #openstack-infra | 17:01 | |
| *** tphummel has joined #openstack-infra | 17:01 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Fix upload-to-docs-site publisher https://review.openstack.org/329947 | 17:02 |
| jeblair | pabelanger: so maybe in the ftp publisher, we need to strip leading '/' before joining on to the site root? | 17:02 |
| *** sree has joined #openstack-infra | 17:02 | |
| *** mtanin___ has quit IRC | 17:03 | |
| mordred | jeblair: that sounds plausible | 17:05 |
| pabelanger | jeblair: we can. I do like the functionality in zuul-launcher today. Having zuul-launcher silently doing things to our jobs doesn't still well with me. That said, I think we could add a warning to the process and I'd be on board | 17:05 |
| *** Jeffrey4l has quit IRC | 17:05 | |
| jeblair | pabelanger: the intent for the scp and ftp publishers in zuul launcher is to be transparently compatible with jenkins | 17:06 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/system-config: Apt mirror for MariaDB https://review.openstack.org/307831 | 17:07 |
| *** PollyZ_ has joined #openstack-infra | 17:07 | |
| *** Rockyg has quit IRC | 17:07 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: strip leading / from scp/ftp targets https://review.openstack.org/330090 | 17:08 |
| jeblair | pabelanger, mordred: ^ | 17:08 |
| jeblair | please double check me on the assertions in the commit message :) | 17:08 |
| AJaeger | thanks, pabelanger ! | 17:08 |
| *** ddieterly[away] is now known as ddieterly | 17:08 | |
| AJaeger | pabelanger: are you doing rax-ord as well? | 17:09 |
| *** amoralej is now known as amoralej|off | 17:10 | |
| *** PollyZ has quit IRC | 17:10 | |
| pabelanger | jeblair: +3 | 17:10 |
| *** yaume has quit IRC | 17:11 | |
| *** abregman has joined #openstack-infra | 17:12 | |
| jeblair | i'll start up zl04 (and maybe zl03 if it has stopped) with that when it lands | 17:13 |
| *** bswartz has quit IRC | 17:13 | |
| *** rossella_s has quit IRC | 17:13 | |
| *** yamahata has joined #openstack-infra | 17:15 | |
| *** jsavak has quit IRC | 17:16 | |
| *** jsavak has joined #openstack-infra | 17:16 | |
| *** tonytan4ever has joined #openstack-infra | 17:17 | |
| *** sputnik13 has joined #openstack-infra | 17:17 | |
| notmorgan | ianw: ping re https://review.openstack.org/#/c/326144/4 if you see the use of the fixture as a blocker I'll swap it out, but I prefer the fixture since it ensures the object remains patched through the whole test run rather than just through the context manager. | 17:18 |
| notmorgan | ianw: but i understand that it's a slight bit easier to read with mock.patch.object directly :) | 17:19 |
| openstackgerrit | Merged openstack-infra/project-config: Ensure console-log is our last publisher https://review.openstack.org/329983 | 17:20 |
| *** PollyZ has joined #openstack-infra | 17:21 | |
| *** pcaruana has joined #openstack-infra | 17:21 | |
| *** eezhova has joined #openstack-infra | 17:22 | |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: strip leading / from scp/ftp targets https://review.openstack.org/330090 | 17:23 |
| *** PollyZ_ has quit IRC | 17:24 | |
| *** ramishra has quit IRC | 17:25 | |
| anteaya | fungi: I have confirmed I can POST to storyboard-dev when manually curling from the command line of my gerrit instance, and have found out that the storyboard plugin is designed to take a long lived token, however the plugin is expecting ssl on the storyboard server, which -dev does not have | 17:26 |
| anteaya | fungi: what is involved in getting the storyboard-dev server real certs instead of self-signed ones? | 17:26 |
| *** Sukhdev has quit IRC | 17:26 | |
| anteaya | the plugin is designed to not like self signed certs, so sayth zaro in the #storyboard channel | 17:27 |
| fungi | anteaya: that seems mighty inflexible | 17:27 |
| fungi | zaro: ^' | 17:27 |
| fungi | i mean, we _can_ put a cert obtained from a third-party ca on there, but it seems like a pretty significant bug if people can't test that plugin with a self-signed cert | 17:28 |
| anteaya | I don't have enough knowledge here to have an opinion | 17:28 |
| fungi | and i'd rather not get a different cert on there just because some piece of software has a bug | 17:28 |
| anteaya | just trying to get enough proof the plugin works to have agreement it can get insalled on review-dev | 17:29 |
| *** piet has quit IRC | 17:29 | |
| zaro | anteaya: i think you can get around that by doing something to your local instance | 17:29 |
| zaro | anteaya: you shouldn’t need to change sb-dev in anyway | 17:29 |
| *** afred312 has quit IRC | 17:29 | |
| anteaya | what needs doing, and in your estimation am I cabable of doing it? | 17:30 |
| anteaya | the plugin can talk to storyboard-dev which was what I was trying to prove | 17:30 |
| anteaya | have we got enough it can be deployed on review-dev yet? | 17:31 |
| *** tqtran has joined #openstack-infra | 17:31 | |
| zaro | anteaya: something with x509 i think which i remember needing to do. need to look for it again though | 17:31 |
| anteaya | please if you could | 17:31 |
| anteaya | I'd rather not have this work held up by my ineptitude any longer than necessary | 17:31 |
| zaro | anteaya: i think we need to verify post before install on review-dev | 17:31 |
| anteaya | zaro: I did | 17:32 |
| anteaya | http://paste.openstack.org/show/516319/ | 17:32 |
| zaro | anteaya: i mean from storyboard | 17:32 |
| anteaya | I don't understand | 17:32 |
| zaro | anteaya: i mean from gerrit sb plugin | 17:32 |
| anteaya | right, well standing by on how to get that to work on my local instance | 17:32 |
| *** jsavak has quit IRC | 17:32 | |
| anteaya | I just don't want to agree to have to do something that will take me a long time to set up | 17:33 |
| electrofelix | appreciate reviews to enable py3 jobs for openstack/git-upstream - https://review.openstack.org/329475 | 17:33 |
| *** electrofelix has quit IRC | 17:34 | |
| *** pilgrimstack has joined #openstack-infra | 17:34 | |
| *** kzaitsev_mb has quit IRC | 17:35 | |
| *** tqtran has quit IRC | 17:35 | |
| *** kzaitsev_mb has joined #openstack-infra | 17:37 | |
| *** HeOS has quit IRC | 17:37 | |
| *** roxanaghe has quit IRC | 17:41 | |
| *** ddieterly is now known as ddieterly[away] | 17:41 | |
| *** roxanaghe has joined #openstack-infra | 17:42 | |
| *** kzaitsev_mb has quit IRC | 17:42 | |
| *** kzaitsev_mb has joined #openstack-infra | 17:42 | |
| *** hongbin has joined #openstack-infra | 17:42 | |
| jeblair | starting zl04 | 17:43 |
| *** jamesmcarthur has quit IRC | 17:44 | |
| *** jsavak has joined #openstack-infra | 17:45 | |
| *** mhickey has quit IRC | 17:45 | |
| *** jamesmcarthur has joined #openstack-infra | 17:45 | |
| *** pilgrimstack has quit IRC | 17:45 | |
| *** akshai has quit IRC | 17:47 | |
| *** akshai has joined #openstack-infra | 17:47 | |
| *** mtanino has quit IRC | 17:48 | |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-openstackci: Logs: serve .log and .sh files as text/plain https://review.openstack.org/330119 | 17:48 |
| jeblair | notmorgan: that look right to you ^ ? | 17:48 |
| jeblair | pabelanger: ^ | 17:49 |
| notmorgan | looking | 17:49 |
| pabelanger | jeblair: looking | 17:50 |
| notmorgan | sec. | 17:50 |
| *** mriedem1 has joined #openstack-infra | 17:50 | |
| notmorgan | jeblair: assuming we have mod_mime enabled? | 17:50 |
| *** kzaitsev_mb has quit IRC | 17:50 | |
| jeblair | good question! lemme check | 17:50 |
| notmorgan | jeblair: because you don't want to do that w/o it enabled :) | 17:51 |
| notmorgan | i'm guessing so because of the ForceMatch below | 17:51 |
| jeblair | yep, it is | 17:51 |
| notmorgan | but good to be sure. | 17:51 |
| *** trown|lunch is now known as trown | 17:51 | |
| pabelanger | jeblair: WFM | 17:51 |
| notmorgan | and you want addtype not a forcetype? [i prefer addtype] | 17:52 |
| notmorgan | and this wfm | 17:52 |
| dasm | o/ anyone looked at this issue? http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-06-15.log.html#t2016-06-15T15:38:12 | 17:52 |
| *** degorenko is now known as _degorenko|afk | 17:52 | |
| *** mriedem1 has quit IRC | 17:52 | |
| notmorgan | dasm logstash was having issues earlier iirc. not sure what the status was. | 17:52 |
| jeblair | notmorgan: yeah, i *think* we only need addtype here | 17:52 |
| notmorgan | dasm: s/was/currently is | 17:53 |
| notmorgan | jeblair: [pretty sure addtype should be used... unless it doesn't work.. then forcetype | 17:53 |
| jeblair | notmorgan: ++ | 17:53 |
| dasm | notmorgan: mhm. so it's not usable? :/ eh. any ETA for change of logstash status? :) | 17:53 |
| *** mriedem has quit IRC | 17:53 | |
| *** mriedem1 has joined #openstack-infra | 17:54 | |
| notmorgan | dasm: like i said, i heard someone tlaking about it earlier. not sure what the status is now. | 17:54 |
| dasm | notmorgan: thanks. | 17:54 |
| * notmorgan has to defer to other -infra-y type folks on that. | 17:54 | |
| notmorgan | anteaya: ^ you said something about logstash i think? | 17:54 |
| anteaya | notmorgan: yes I did and jesusaur replied | 17:54 |
| *** tbarron|afk is now known as tbarron | 17:54 | |
| anteaya | I was waiting to hear more back from jesusaur | 17:55 |
| notmorgan | anteaya: perfect. thnx, dasm ^ cc | 17:55 |
| notmorgan | dasm: if you lurk here and watch, you'll see updates as they come through :) | 17:55 |
| anteaya | notmorgan: welcome | 17:55 |
| jeblair | is there an infra-root looking into logstash? | 17:55 |
| dasm | notmorgan: yeah. i'm idling here :) | 17:55 |
| dasm | notmorgan: thanks gain | 17:55 |
| anteaya | I have not heard of one doing so | 17:55 |
| notmorgan | jeblair: also, thanks for asking on that. | 17:55 |
| jeblair | elasticsearch says it is 'green' with 5 nodes | 17:56 |
| jesusaur | anteaya: I don't have access to the systems, but I suspect that the kibana host is having connectivity issues connecting to elasticsearch | 17:56 |
| fungi | a quick glimpse at the job queue graph suggests it's caught up processing at the moment, but kibana still doesn't pull up any results for me | 17:56 |
| jeblair | that seems like 1 fewer nodes than we expect? | 17:56 |
| fungi | elasticsearch is _supposed_ to have 8 i think? | 17:57 |
| fungi | so may be 3 fewer than we expect? | 17:57 |
| *** eezhova has quit IRC | 17:57 | |
| anteaya | jesusaur: yes, that is fine, I had just thought you would follow up | 17:57 |
| *** PollyZ has quit IRC | 17:57 | |
| jeblair | i thought it was 6 -- elasticsearch02 through elasticsearch07 | 17:57 |
| anteaya | jesusaur: to find someone with access | 17:57 |
| fungi | i'll take a peek at the kibana logs | 17:58 |
| jeblair | fungi: where are they? | 17:58 |
| pabelanger | jeblair: I am not | 17:58 |
| fungi | jeblair: that's the first thing i'm trying to figure out | 17:58 |
| jesusaur | jeblair: they will be apache logs | 17:58 |
| fungi | i assume it has some | 17:58 |
| jeblair | i can tell you one place they are not | 17:59 |
| fungi | aha | 17:59 |
| fungi | heh, /var/log/kibana dne huh? ;) | 17:59 |
| jeblair | yup | 17:59 |
| *** daemontool has quit IRC | 17:59 | |
| openstackgerrit | Ryan Hallisey proposed openstack-infra/project-config: Shorten the name of the kolla-kubernetes gates https://review.openstack.org/330123 | 17:59 |
| fungi | [Tue Jun 14 12:43:27.090786 2016] [proxy_http:error] [pid 29690:tid 140337871787776] (20014)Internal error: [client 203.99.198.240:23247] AH01102: error reading status line from remote server elasticsearch02.openstack.org:9200, referer: http://logstash.openstack.org/ | 18:00 |
| fungi | also we have... | 18:00 |
| jeblair | that was a while ago, and i did the status curn on es02 and it responded | 18:00 |
| fungi | [Mon Jun 13 19:57:20.447396 2016] [proxy:error] [pid 29690:tid 140337980892928] [client 89.237.15.12:51570] AH00898: Error reading from remote server returned by /elasticsearch/logstash-2016.06.13/_search, referer: http://logstash.openstack.org/ | 18:00 |
| fungi | yeah, those are from >24 hours ago | 18:01 |
| fungi | access logs are current, so apache is logging | 18:01 |
| jesusaur | does the elasticsearch log on es02 have anything useful to say? | 18:01 |
| fungi | just nothing recent for apache error logs | 18:01 |
| *** PollyZ has joined #openstack-infra | 18:02 | |
| fungi | [2016-06-15 18:00:50,330][WARN ][indices.breaker ] [elasticsearch02] [FIELDDATA] New used memory 22480787131 [20.9gb] from field [@timestamp] would be larger than configured breaker: 22469214208 [20.9gb], breaking | 18:02 |
| *** pvaneck has joined #openstack-infra | 18:02 | |
| fungi | same thing dasm said was in the page source when he pulled it | 18:02 |
| zaro | anteaya: try this http://paste.openstack.org/show/516325/ | 18:02 |
| *** mtanino has joined #openstack-infra | 18:02 | |
| *** jsavak has quit IRC | 18:03 | |
| *** hashar has joined #openstack-infra | 18:03 | |
| *** bswartz has joined #openstack-infra | 18:04 | |
| anteaya | zaro: thank you | 18:04 |
| *** tonytan4ever has quit IRC | 18:04 | |
| *** mriedem1 is now known as mriedem | 18:05 | |
| fungi | jesusaur: suggestions seem to be increasing heap space or reducing stored field data? | 18:05 |
| jeblair | i'm wondering if something is broken with expiration again | 18:05 |
| jeblair | the curator logs seem to be empty | 18:05 |
| anteaya | zaro: does it matter what user creates the cert? should the gerrit2 user create it? | 18:05 |
| jeblair | anyone know how to use bigdesk? it's not in the docs | 18:06 |
| *** eezhova has joined #openstack-infra | 18:07 | |
| jeblair | oh got it | 18:07 |
| jeblair | that wasn't too hard | 18:07 |
| jesusaur | fungi: seems reasonable, but how many data nodes should there be? is 5 a low number? if so then getting the others back online may relieve the memory pressure | 18:07 |
| jeblair | the cluster does not know about elasticsearch06 | 18:08 |
| jeblair | the cluster does not know about elasticsearch05 | 18:08 |
| jeblair | sorry ^ correction. 05 is correct | 18:08 |
| *** oanson has joined #openstack-infra | 18:08 | |
| jeblair | fungi: you want to check out 05 while i look for indices? | 18:08 |
| anteaya | pabelanger: do you remember how to use bigdesk? clarkb showed us how, I'm trying to find the instructions | 18:08 |
| *** ccamacho is now known as ccamacho|out | 18:08 | |
| fungi | looking | 18:08 |
| *** csomerville has joined #openstack-infra | 18:09 | |
| fungi | [2016-06-10 11:36:46,231][WARN ][indices.breaker ] [elasticsearch05] [FIELDDATA] New used memory 22866534892 [21.2gb] from field [message] would be larger than configured breaker: 22469214208 [20.9gb], breaking | 18:09 |
| jesusaur | I see nodes 03 02 06 07 04 listed in http://logstash.openstack.org/elasticsearch/_nodes?pretty | 18:09 |
| jeblair | that whas a while ago | 18:10 |
| jeblair | jesusaur: oh neat! i didn't have to run bigdesk :) | 18:10 |
| *** pvaneck has quit IRC | 18:10 | |
| *** akshai_ has joined #openstack-infra | 18:10 | |
| jesusaur | jeblair: nope, bigdesk does the same thing I did, then makes a pretty page out of it :) | 18:10 |
| jeblair | jesusaur: where do i see what indexes there are? | 18:11 |
| *** ajmiller_ is now known as ajmiller | 18:11 | |
| fungi | most recent es log on 05 is from 2016-06-10 11:36:46,231 | 18:12 |
| jesusaur | jeblair: http://logstash.openstack.org/elasticsearch/_cluster/health?pretty&level=indices | 18:12 |
| fungi | that seems older than i would expect? | 18:12 |
| jesusaur | or http://logstash.openstack.org/elasticsearch/_cluster/health?pretty&level=shards for the shards | 18:12 |
| jesusaur | but that doesn't tell you which node a shard is on | 18:12 |
| fungi | looks like there's a java process running on other es servers but not on 05 | 18:13 |
| *** akshai has quit IRC | 18:13 | |
| fungi | should i try starting the elasticsearch initscript? | 18:13 |
| *** cody-somerville has quit IRC | 18:13 | |
| *** vhosakot has quit IRC | 18:13 | |
| jesusaur | fungi: that's what I would try | 18:13 |
| fungi | the log doesn't seem to indicate why it stopped, it just went silent | 18:13 |
| jeblair | the indexes appear to be rotating correctly... | 18:13 |
| *** tqtran has joined #openstack-infra | 18:13 | |
| fungi | okay, i can see the expected process running on 05 now | 18:14 |
| pabelanger | anteaya: I usually run ssh -L elasticsearch02 -L 9200:localhost:9200 from a terminal then open file:///home/pabelanger/git/pabelanger/bigdesk/index.html#nodes in my browse | 18:14 |
| jeblair | so the only thing that seems wrong is that we're one server down, which es doesn't seem particularly unhappy about | 18:14 |
| jeblair | now the cluster is red | 18:14 |
| *** hashar has quit IRC | 18:14 | |
| jeblair | with 6 nodes | 18:14 |
| *** hashar has joined #openstack-infra | 18:15 | |
| fungi | hah | 18:15 |
| jesusaur | the cluster is likely looking for the "missing" shards of the indices that have been deleted from the other nodes | 18:15 |
| *** hongbin has quit IRC | 18:15 | |
| jesusaur | running the curator cron command will delete the old shards from 05 | 18:15 |
| anteaya | pabelanger: thanks | 18:16 |
| *** hongbin has joined #openstack-infra | 18:16 | |
| jeblair | jesusaur: i'll do that now | 18:16 |
| anteaya | zaro: somehow creating that cert has affected how my vm is able to interpret keyboard input | 18:16 |
| *** tonytan4ever has joined #openstack-infra | 18:16 | |
| anteaya | zaro: I can't paste into the terminal anymore and I have to hit every key twice to get a character to show up | 18:17 |
| *** akshai_ has quit IRC | 18:17 | |
| jeblair | http://paste.openstack.org/show/516326/ | 18:17 |
| jeblair | jesusaur, fungi: ^ | 18:17 |
| *** notnownikki has joined #openstack-infra | 18:17 | |
| anteaya | for instance getting it to accept https:// requires me to type hhttttppss:://// | 18:17 |
| jeblair | deleting 6.15 is slightly concerning... | 18:17 |
| fungi | i've double-checked all the es servers and they're all running the same java process now | 18:17 |
| *** akshai has joined #openstack-infra | 18:17 | |
| jeblair | the cluster is 'green' | 18:17 |
| fungi | anteaya: your terminal might have just gotten in a strange state. try running `reset` and `stty sane` | 18:18 |
| anteaya | fungi: thanks I will try | 18:18 |
| *** tonytan4ever has quit IRC | 18:19 | |
| jeblair | logstash.o.o/_search is returning 500 errors | 18:19 |
| jeblair | but nothing in the apache error log | 18:19 |
| anteaya | fungi: I've tried the two above commands twice, no change in the situation | 18:19 |
| jesusaur | jeblair: yeah, that delete is odd | 18:19 |
| anteaya | well reset cleared my screen but other than that no change | 18:20 |
| jeblair | i'm inclined to restart apache | 18:20 |
| *** sarob_ has joined #openstack-infra | 18:20 | |
| *** vhosakot has joined #openstack-infra | 18:21 | |
| *** _nadya_ has quit IRC | 18:21 | |
| fungi | [2016-06-15 18:16:33,841][WARN ][indices.cluster ] [elasticsearch05] [[logstash-2016.06.02][4]] marking and sending shard failed due to [failed recovery] | 18:21 |
| fungi | with a lengthy java traceback on 05 now | 18:22 |
| anteaya | exiting the user and then sudo suing back to being gerrit2 apparently fixed the issue | 18:22 |
| anteaya | how odd | 18:22 |
| jeblair | fungi: that's one of the old indexes curator deleted, so probably okay? | 18:22 |
| fungi | jeblair: that's my hope. i mean the service seems to have stayed running and that's just a warning | 18:23 |
| *** sarob has quit IRC | 18:23 | |
| *** chem``` has joined #openstack-infra | 18:23 | |
| jeblair | jesusaur: oh! i just noticed, the thing it deleted was "0016.06.15" | 18:25 |
| jeblair | as in -- the yer 16. not 2016. | 18:25 |
| jeblair | year | 18:25 |
| zaro | I would use the same user to create the cert as the one you plan to use in gerrit sb plugin | 18:25 |
| *** chem`` has quit IRC | 18:25 | |
| zaro | anteaya: ^ | 18:25 |
| jesusaur | jeblair: ohhh | 18:25 |
| jesusaur | jeblair: that probably indicates a y2k bug in the logstash parsing | 18:26 |
| jeblair | ya | 18:26 |
| mordred | wow | 18:26 |
| anteaya | zaro: https://storyboard-dev.openstack.org/#!/story/16 | 18:26 |
| fungi | possible we have some bad timestamps in logs or bad timestamp parsing in our grok set? | 18:26 |
| * jesusaur wonders which log | 18:26 | |
| jeblair | anyway, that shard is gone now, so that's okay | 18:26 |
| jeblair | (maybe we can track it down the next time it shows up) | 18:26 |
| anteaya | zaro: oh I created the cert with gerrit2 but my user for storyboard is my account, which is my email address | 18:27 |
| jeblair | so it seems like the cluster is fine, we just literally have too much data in the 06.15 index? | 18:27 |
| jeblair | (so it's refusing to run queries on it) | 18:27 |
| fungi | jeblair: jesusaur: fwiw, we're still logging that indices.breaker warning about new used memory would be larger than configured breaker | 18:27 |
| fungi | for the @timestamp field | 18:28 |
| jeblair | fungi: yeah, and that's what i get back with the 500 error if i curl the search url directly | 18:28 |
| jeblair | fungi: so i think that's the issue | 18:28 |
| fungi | i found one reference saying the search cache can fill up and trigger this too | 18:29 |
| fungi | not sure how credible | 18:29 |
| jeblair | 'we blew a breaker' | 18:29 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: promote: add upgrade job part of voting https://review.openstack.org/330139 | 18:29 |
| clarkb | sorry I have caught something and havent really computered today but thats the we have too many timestamps to fit in memory issue. if you delete the oldest index it goes away | 18:29 |
| * fungi grabs a flashlight and heads down into the basement | 18:29 | |
| fungi | what's the easiest way to delete the oldest index? | 18:29 |
| *** e0ne has quit IRC | 18:30 | |
| jeblair | clarkb: oh, okay -- i was thrown off by the fact that it was including today's index in the error.... but i guess that's just when it hit the error -- like, it collected all the data from the previous indices and then right near the end went over the limit? | 18:30 |
| jesusaur | fungi: you can run the curator command with a different number-of-days (or whatever the argument is called) | 18:30 |
| jeblair | fungi: curator, i'd guess? tell it to delete days +1 | 18:30 |
| clarkb | jeblair: ya I think its the aggregate | 18:31 |
| jeblair | i will curate | 18:31 |
| anteaya | zaro: so did this work or do I try again with sudo? http://paste.openstack.org/show/516329/ | 18:31 |
| fungi | thanks jesusaur, clarkb, jeblair | 18:31 |
| anteaya | zaro: certificate added, keytool error is kind of contradictory | 18:31 |
| anteaya | clarkb: :( | 18:32 |
| *** sambetts is now known as sambetts|afk | 18:32 | |
| anteaya | clarkb: I hope you are doing the things you need to feel better | 18:32 |
| jeblair | curator --logfile /var/log/curator.log delete indices --older-than 9 --time-unit days --timestring \%Y.\%m.\%d | 18:32 |
| jeblair | 2016-06-15 18:31:26,138 INFO ---deleting index logstash-2016.06.06 | 18:32 |
| jeblair | fungi, jesusaur, clarkb: ^ | 18:32 |
| jeblair | logstash.o.o is making pretty graphs now. | 18:32 |
| anteaya | yay | 18:33 |
| fungi | that looks perfect | 18:33 |
| anteaya | melwitt: ^^ | 18:33 |
| jeblair | including a spike in events today | 18:33 |
| fungi | dasm: ^ | 18:33 |
| dasm | \o/ | 18:33 |
| notmorgan | jeblair: i like pretty graphs! | 18:33 |
| dasm | thanks jeblair fungi jesusaur | 18:33 |
| dasm | and all involved | 18:33 |
| jeblair | which i bet if we understood might explain the issue | 18:33 |
| *** sarob_ has quit IRC | 18:34 | |
| fungi | there was a sizeable spike on the job queue graph as well | 18:34 |
| *** sdague has quit IRC | 18:34 | |
| jeblair | oops, the spike i was looking at is too recent to be the cause, probably a symptom | 18:35 |
| *** pt_15 has quit IRC | 18:36 | |
| melwitt | awesome, thanks everyone! (logstash.o.o) | 18:36 |
| jeblair | starting zl03; gracefully stopping zl01 ond zl02 | 18:37 |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: Update current repo setup includepkgs https://review.openstack.org/330148 | 18:37 |
| *** PollyZ_ has joined #openstack-infra | 18:38 | |
| *** aimeeu has quit IRC | 18:39 | |
| *** PollyZ has quit IRC | 18:39 | |
| *** akshai has quit IRC | 18:39 | |
| *** akshai has joined #openstack-infra | 18:39 | |
| pabelanger | jeblair: Looks like jenkins07 is out of service (crash maybe). Should we restart it or leave it stopped? | 18:40 |
| jeblair | #status log jenkins03 and jenkins04 are in prepare-for-shutdown mode in preparation for decomissioning | 18:40 |
| openstackstatus | jeblair: finished logging | 18:40 |
| notmorgan | jeblair: ^ that is pretty darn awesome i might add | 18:40 |
| jeblair | pabelanger: hrm. restart, i think, under the circumstances :) | 18:40 |
| anteaya | zaro: so I completed your instructions, restarted my local gerrit instance and added another comment to my patch which has Story: 11 in the commit message, yet I see no activity on https://storyboard-dev.openstack.org/#!/story/11 | 18:40 |
| pabelanger | jeblair: sure, let me clean it up | 18:41 |
| notnownikki | jeblair, that's fantastic stuff with 2.5 :) | 18:41 |
| jeblair | notmorgan: i'm glad we have so many to do, because each one is fun :) | 18:41 |
| *** sdague has joined #openstack-infra | 18:41 | |
| anteaya | zaro: I did confirm I can use the cert I created to POST to the storyboard-dev api manually | 18:41 |
| pleia2 | anteaya: great! | 18:42 |
| jeblair | pabelanger: if you feel like restarting zl01 and zl02 after they stop, go right ahead; i'm going to break for lunch now | 18:42 |
| jeblair | notnownikki: thanks! :) | 18:42 |
| *** bhavik has quit IRC | 18:42 | |
| jeblair | notnownikki, notmorgan: two negative nicks! | 18:43 |
| anteaya | pleia2: do you have any desire to take a peek at the storyboard-dev logs and see what it thinks I am trying to do? | 18:43 |
| notnownikki | haha | 18:43 |
| pabelanger | jeblair: Sure, I'll keep an eye out | 18:43 |
| notnownikki | jeblair, mine is an inspector morse reference | 18:43 |
| anteaya | pleia2: if not fungi suggested I use tcpdump, but it will take some work on his part to guide me how | 18:43 |
| *** salv-orlando has quit IRC | 18:43 | |
| *** ddieterly[away] has quit IRC | 18:43 | |
| fungi | anteaya: well, you'll need your own storyboard instance for starters, since you'll need to use http instead of https | 18:44 |
| anteaya | oh my | 18:44 |
| *** ddieterly has joined #openstack-infra | 18:44 | |
| *** salv-orlando has joined #openstack-infra | 18:44 | |
| *** jaosorior has joined #openstack-infra | 18:44 | |
| fungi | or else figure out some sort of http to https proxy between your gerrit and storyboard-dev | 18:44 |
| pleia2 | our storyboard forces to https | 18:44 |
| openstackgerrit | Christian Fetzer proposed openstack-infra/jenkins-job-builder: Add support for Clang Scan-Build plugin version 1.7 https://review.openstack.org/330150 | 18:44 |
| fungi | assuming the plugin supports not using https | 18:44 |
| jeblair | notnownikki: oh that's cool | 18:44 |
| anteaya | yeah, I don't know how to do that | 18:44 |
| anteaya | fungi: I don't know about that | 18:45 |
| *** jed56 has quit IRC | 18:45 | |
| anteaya | zaro: does the plugin support not using https? | 18:45 |
| *** ihrachys has joined #openstack-infra | 18:45 | |
| *** eharney has quit IRC | 18:45 | |
| zaro | anteaya: i think the directions i gave you will work now. | 18:47 |
| anteaya | zaro: so I completed your instructions, restarted my local gerrit instance and added another comment to my patch which has Story: 11 in the commit message, yet I see no activity on https://storyboard-dev.openstack.org/#!/story/11 | 18:47 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/system-config: OpenStack-CI Apt mirror for Ubuntu Cloud Archive https://review.openstack.org/330152 | 18:47 |
| anteaya | zaro: and I don't have access to storyboard-dev to check the logs to see what it sent to the -dev server | 18:48 |
| zaro | anteaya: take a look at the gerrit log to see if you are getting any errors | 18:48 |
| pleia2 | fungi: I see some channel cleanup got #senlin's gerrit bot back, still able to make some time to look at 287056 for bot debugging? | 18:49 |
| fungi | mordred: we still have openstack-infra/ansible-build-image listed in governance with no accompanying repo created... do you happen to know if that's still meant to be a thing or if it should be removed from governance? | 18:49 |
| fungi | pleia2: working my way down to it. trying to make sure first that we're all set to send out barcelona summit invites later this week | 18:50 |
| anteaya | [2016-06-15 18:38:37,992] WARN com.google.gerrit.server.plugins.PluginLoader : Cannot load plugin its-storyboard | 18:50 |
| zaro | well there you go. any other message why it can’t load? | 18:51 |
| *** dimtruck is now known as zz_dimtruck | 18:51 | |
| fungi | pleia2: also AJaeger made a direct count of channels in the config and it was actually 122 not 244 (i was grepping the log and forgetting to see if there was more than one restart) so this seems almost certain to be freenode's channel maximum at work | 18:51 |
| *** sree has quit IRC | 18:51 | |
| pleia2 | AJaeger, hongbin - we document channel renames at http://docs.openstack.org/infra/system-config/irc.html#renaming-an-irc-channel so if there's something we're missing, please add it :) | 18:51 |
| anteaya | zaro: http://paste.openstack.org/show/516332/ | 18:51 |
| pleia2 | fungi: aha! we should probably see about getting that lifted | 18:52 |
| pabelanger | #status log jenkins07 back online. Will manually cleanup used nodes moving forward | 18:52 |
| openstackstatus | pabelanger: finished logging | 18:52 |
| zaro | anteaya: yeah, i see the problem. | 18:52 |
| anteaya | funny the plugin was able to find the storyboard-dev api before | 18:52 |
| anteaya | great | 18:52 |
| fungi | pleia2: i don't know whether that's even something they can adjust for individual accounts | 18:52 |
| pleia2 | fungi: I had my personal limit adjusted | 18:52 |
| fungi | pleia2: oh? perfect! | 18:52 |
| zaro | it’s because that jar file is built against latest gerrit. i’ll need to build one against 2.11 | 18:52 |
| fungi | i've never tried to exceed 120 actual channels | 18:52 |
| anteaya | zaro: ah thank you | 18:52 |
| odyssey4me | pabelanger another mirror request to queue up when you have a moment https://review.openstack.org/330152 | 18:53 |
| zaro | will get it to you in a few | 18:53 |
| pleia2 | the limit was lower a few years ago | 18:53 |
| fungi | pleia2: in that case, almost certainly preferable to sharding across multiple gerritbot processes | 18:53 |
| *** sarob has joined #openstack-infra | 18:53 | |
| anteaya | zaro: yup, that's fine, good time for me to take a walk | 18:53 |
| anteaya | zaro: be back in about an hour, thank you | 18:53 |
| anteaya | I guess the plugin wasn't able to find the api before, I wonder how the -dev logs timed with my comment then | 18:54 |
| *** cdent has quit IRC | 18:55 | |
| *** salv-orl_ has joined #openstack-infra | 18:55 | |
| zaro | anteaya: your curl requests would do it. | 18:56 |
| *** salv-orlando has quit IRC | 18:58 | |
| anteaya | I guess I miss read the timestamps | 18:58 |
| anteaya | yet another reason I am not a great person to test things locally | 18:59 |
| *** sarob has quit IRC | 19:01 | |
| *** Sukhdev has joined #openstack-infra | 19:02 | |
| *** jsavak has joined #openstack-infra | 19:02 | |
| openstackgerrit | Merged openstack-infra/project-config: Import Panko https://review.openstack.org/318677 | 19:05 |
| *** piet has joined #openstack-infra | 19:05 | |
| *** afred312 has joined #openstack-infra | 19:05 | |
| *** sarob has joined #openstack-infra | 19:05 | |
| *** tonytan4ever has joined #openstack-infra | 19:09 | |
| *** pvaneck has joined #openstack-infra | 19:10 | |
| *** sarob has quit IRC | 19:10 | |
| *** zz_dimtruck is now known as dimtruck | 19:12 | |
| *** sarob has joined #openstack-infra | 19:12 | |
| *** EricGonczer_ has joined #openstack-infra | 19:13 | |
| *** ihrachys has quit IRC | 19:14 | |
| *** dimtruck is now known as zz_dimtruck | 19:14 | |
| openstackgerrit | James Page proposed openstack-infra/project-config: Add specifications repository for charms https://review.openstack.org/330054 | 19:14 |
| *** tonytan4ever has quit IRC | 19:14 | |
| *** mhickey has joined #openstack-infra | 19:15 | |
| openstackgerrit | James Page proposed openstack-infra/project-config: Add specifications repository for charms https://review.openstack.org/330054 | 19:16 |
| pabelanger | odyssey4me: thanks | 19:16 |
| *** dtardivel has quit IRC | 19:17 | |
| *** sarob has quit IRC | 19:18 | |
| jbryce | over the last few weeks i’ve seen an increase in the number of rackspace.com emails who are getting removed from different lists.openstack.org lists i administer for excessive bounces. separately i had someone from rax reach out to me asking me if we had any of the bounce headers we could send along to their email admins to try to track down what’s happening. anyone seen or heard anything about this? | 19:20 |
| therve | clarkb, If you have a few minutes to look at https://review.openstack.org/#/c/286136/ again that'd be cool | 19:20 |
| therve | I just fixed the tests compared the your +2 | 19:20 |
| *** dprince has quit IRC | 19:21 | |
| *** eharney has joined #openstack-infra | 19:22 | |
| *** esikachev has joined #openstack-infra | 19:22 | |
| * zaro afk for a hours | 19:24 | |
| *** EricGonc_ has joined #openstack-infra | 19:25 | |
| *** piet has quit IRC | 19:25 | |
| *** EricGonczer_ has quit IRC | 19:26 | |
| *** bhavik has joined #openstack-infra | 19:26 | |
| fungi | jbryce: yes, they have a malfunctioning filter device. ttx and i were providing feedback to them yesterday and today | 19:26 |
| *** piet has joined #openstack-infra | 19:26 | |
| jbryce | fungi: cool. thanks for the update | 19:26 |
| fungi | jbryce: there's supposedly an internal support ticket at rackspace for it, but elarson and jroll can probably provide more recent details | 19:27 |
| jroll | fungi: I can, if that's helpful | 19:27 |
| fungi | jroll: probably only helpful if there's further assistance or info you need from our end | 19:28 |
| jroll | fungi: oh, I'll let the people on the ticket do that, I'm staying out of it as much as possible | 19:28 |
| fungi | heh, good call | 19:28 |
| elarson | jbryce: feel free to ping me on internal chat if you wanted to watch the ticket | 19:29 |
| jroll | elarson: jbryce isn't a racker :) | 19:30 |
| JayF | jroll: I still think my workaround is best workaround :D | 19:30 |
| mordred | jroll: jbryce has access to all things at all companies | 19:30 |
| jroll | JayF: I agree, and said as much this morning in this channel :P | 19:30 |
| jroll | mordred: oh, cool, where do I get that job | 19:30 |
| mordred | jroll: I think you have to kill jbryce first | 19:30 |
| notmorgan | jroll: you don't want that job... you get called for all the things then | 19:30 |
| jroll | hrm | 19:30 |
| mordred | and then yes - what notmorgan said | 19:31 |
| notmorgan | mordred: there can be only one? | 19:31 |
| notmorgan | /highlander! | 19:31 |
| mordred | notmorgan: I will assume you said that in proper sean connery accent | 19:31 |
| *** Apoorva has quit IRC | 19:31 | |
| fungi | mordred: somehow you ended up +o in channel | 19:31 |
| *** ddieterly has quit IRC | 19:31 | |
| mordred | fungi: oh - yeah. I kicked the person who likes to tell us about allah the other day | 19:32 |
| notmorgan | mordred: that and the theme song from the tv series was also playing in the background | 19:32 |
| *** ChanServ sets mode: -o mordred | 19:32 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Move hook generation in to python https://review.openstack.org/271139 | 19:32 |
| mordred | fungi: fixed | 19:32 |
| fungi | mordred: oh, yep. worth noting you can akick them with /msg chanserv too and don't even need to op yourself | 19:33 |
| elarson | jroll: well, then he is out of luck ;) | 19:33 |
| *** openstackgerrit has quit IRC | 19:33 | |
| jroll | mordred: interestingly enough, I watched that person do that to a different network for a while before it showed up here, must have switched over after the numerous g-lines | 19:33 |
| *** openstackgerrit has joined #openstack-infra | 19:33 | |
| *** piet has quit IRC | 19:33 | |
| *** akshai_ has joined #openstack-infra | 19:33 | |
| fungi | jroll: the one i saw was coming in via freenode's web client | 19:34 |
| *** jsavak has quit IRC | 19:34 | |
| fungi | with a random guest nick | 19:34 |
| *** akshai has quit IRC | 19:35 | |
| *** raunak has quit IRC | 19:35 | |
| *** oanson has quit IRC | 19:35 | |
| *** jsavak has joined #openstack-infra | 19:35 | |
| *** sdake has joined #openstack-infra | 19:36 | |
| *** sarob has joined #openstack-infra | 19:37 | |
| cbader | hello, I am trying to bring up a new commonnci system and on the first run of the sudo puppet I am see this error anyone see this before on a new install. Error: Unknown function load_module_metadata at /etc/puppet/modules/nodepool/manifests/mysql.pp:14 on node openstackci-2.rose.rdlabs.hpecorp.net | 19:38 |
| *** sdake_ has joined #openstack-infra | 19:40 | |
| rcarrillocruz | hmm, iirc there was a change lately for managing mysql on the nodepool class | 19:41 |
| rcarrillocruz | it was decoupled into its own class or something like that | 19:41 |
| *** sarob has quit IRC | 19:41 | |
| cbader | rcarrillocruz, is there somewhere I can look for fix. | 19:41 |
| *** sdake has quit IRC | 19:42 | |
| cbader | rcarrillocruz, I have the latest dump. | 19:42 |
| *** ociuhandu has quit IRC | 19:44 | |
| pabelanger | jeblair: notmorgan: zl01 and zl02 restarted | 19:44 |
| notmorgan | pabelanger: good ot know thnx | 19:45 |
| *** salv-orl_ has quit IRC | 19:45 | |
| *** salv-orlando has joined #openstack-infra | 19:45 | |
| pabelanger | 4 zuul-launcher > 3 jenkins masters! | 19:46 |
| pleia2 | nice | 19:46 |
| *** e0ne has joined #openstack-infra | 19:48 | |
| mordred | pabelanger: we're getting quite close | 19:49 |
| pabelanger | mordred: Yup, I think we are over the hump now. Just a matter up standing up the replacements I think | 19:52 |
| Shrews | \o/ | 19:53 |
| krotscheck_dcm | fungi: Were you ever able to dig into those segfaults? | 19:56 |
| *** tkelsey has quit IRC | 19:56 | |
| *** jaosorior has quit IRC | 19:57 | |
| *** acoles is now known as acoles_ | 19:58 | |
| *** jcoufal_ has joined #openstack-infra | 19:59 | |
| fungi | krotscheck_dcm: ahh, nope. worth noting they seem not to have continued in dfw or iad and are now only appearing in ord | 19:59 |
| *** krotscheck_dcm is now known as krotscheck | 20:00 | |
| *** ihrachys has joined #openstack-infra | 20:00 | |
| krotscheck | fungi: Odd, that. | 20:00 |
| krotscheck | fungi: Alright- I'm spinning up a mirror vm to see if I can reproduce the package truncation we saw. | 20:00 |
| openstackgerrit | Christian Fetzer proposed openstack-infra/jenkins-job-builder: Add support for GitLab notifier and connection property https://review.openstack.org/330178 | 20:00 |
| fungi | krotscheck: also, they seem to have gone away for several weeks in ord and only started back up again over the past week | 20:01 |
| *** jcoufal has quit IRC | 20:02 | |
| krotscheck | fungi: Well, the npm mirrors are still disabled. | 20:02 |
| *** Sukhdev has quit IRC | 20:02 | |
| openstackgerrit | Christian Fetzer proposed openstack-infra/jenkins-job-builder: Add support for GitLab notifier and connection property https://review.openstack.org/330178 | 20:02 |
| *** sarob has joined #openstack-infra | 20:02 | |
| *** thorst has quit IRC | 20:02 | |
| *** zeih has joined #openstack-infra | 20:03 | |
| *** asselin_ has quit IRC | 20:04 | |
| fungi | krotscheck: here's the match frequency for logged segfaults on the ord mirror http://paste.openstack.org/show/516350/ | 20:05 |
| jeblair | the only jobs running on jenkins03 and 04 are tripleo jobs. i am going to abort them. | 20:05 |
| *** gouthamr has quit IRC | 20:05 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: promote: add upgrade job part of voting https://review.openstack.org/330139 | 20:05 |
| krotscheck | fungi: Am I supposed to be getting an Internal Server Error on that link? | 20:05 |
| *** thorst has joined #openstack-infra | 20:05 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: promote: add upgrade job part of voting https://review.openstack.org/330139 | 20:05 |
| rook | If I want to add a Jenkins job -- is there a specific method of having that happen? I had triggers when I was on GerritHub.. | 20:06 |
| *** ihrachys_ has joined #openstack-infra | 20:06 | |
| *** zz_dimtruck is now known as dimtruck | 20:06 | |
| fungi | krotscheck: um, nope. usually that means lodgeit lost contact wit trove | 20:06 |
| fungi | krotscheck: does a refresh of the page "fix" it? | 20:06 |
| jeblair | rook: you may want to start here: http://docs.openstack.org/infra/manual/drivers.html#running-jobs-with-zuul | 20:06 |
| krotscheck | fungi: Huhn. It's been happening to me for a few weeks now. Clearing my cookies helped. | 20:06 |
| *** somerville32 has joined #openstack-infra | 20:06 | |
| *** dprince has joined #openstack-infra | 20:07 | |
| *** sdake_ has quit IRC | 20:07 | |
| krotscheck | fungi: Those are daily logs, yes? I wonder what happened a week ago... | 20:07 |
| *** sarob has quit IRC | 20:07 | |
| *** sdake has joined #openstack-infra | 20:07 | |
| pabelanger | jeblair: is there any way we can clean up /var/lib/zuul/launcher.socket on zuul-launcher stop / graceful? | 20:07 |
| *** zeih has quit IRC | 20:07 | |
| *** Apoorva has joined #openstack-infra | 20:07 | |
| fungi | krotscheck: oh, nope good question and i'm wrong. those are rotated arbitrarily it seems | 20:07 |
| *** xarses_ has quit IRC | 20:08 | |
| pabelanger | jeblair: thinking about an ansible playbook to restart zuul-launcher and might be able to use wait_for out of the box | 20:08 |
| jeblair | pabelanger: oh, yeah, we should be able to do that as part of the stop | 20:08 |
| pabelanger | jeblair: neat! | 20:08 |
| fungi | krotscheck: so krotscheck so it looks like the segfaults started up in late april | 20:08 |
| krotscheck | fungi: That's... right around when xenial was released. | 20:09 |
| *** ihrachys has quit IRC | 20:09 | |
| krotscheck | fungi: But well before we turned on NPM mirrors. | 20:09 |
| fungi | krotscheck: i wonder if that's around when we turned on xenial nodes for a few jobs though | 20:09 |
| fungi | clarkb: ^ if you happen to be around and remember | 20:09 |
| krotscheck | fungi: Could be, I was pretty quick on the xenial trigger when I found out they were available. | 20:10 |
| krotscheck | Might be a red herring though...? | 20:10 |
| *** sarob has joined #openstack-infra | 20:10 | |
| *** csomerville has quit IRC | 20:10 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Remove jenkins03 and jenkins04 https://review.openstack.org/330181 | 20:11 |
| *** sdake_ has joined #openstack-infra | 20:11 | |
| *** raunak has joined #openstack-infra | 20:11 | |
| jeblair | pabelanger, fungi, mordred: ^ | 20:11 |
| *** tlian has quit IRC | 20:11 | |
| mordred | jeblair: +2 | 20:11 |
| krotscheck | fungi: On a totally unrelated note, could you add me to the ACL groups on js-openstack-lib? https://review.openstack.org/#/admin/projects/openstack/js-openstack-lib,access | 20:12 |
| *** sdake has quit IRC | 20:12 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Decomission jenkins03 and jenkins04 https://review.openstack.org/330182 | 20:14 |
| *** kgiusti has left #openstack-infra | 20:14 | |
| pabelanger | jeblair: +3 | 20:14 |
| *** sarob has quit IRC | 20:15 | |
| *** matrohon has joined #openstack-infra | 20:16 | |
| *** ihrachys_ has quit IRC | 20:16 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add playbook to graceful restart zuul-launchers https://review.openstack.org/330184 | 20:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Decomission jenkins.openstack.org https://review.openstack.org/330185 | 20:18 |
| pabelanger | mordred: jeblair: I was thinking something like ^ to restart zuul-launchers | 20:18 |
| pabelanger | still needs testing | 20:18 |
| fungi | krotscheck: done | 20:20 |
| krotscheck | fungi: Thanks! | 20:20 |
| timothyb89 | krotscheck: on the topic of acls, do you want +2 for karma-subunit-reporter? | 20:20 |
| *** flepied has joined #openstack-infra | 20:20 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Decomission jenkins-dev.openstack.org https://review.openstack.org/330186 | 20:21 |
| krotscheck | timothyb89: MOAAAR POWER... but meh, sure? I'll add it to my watch list. | 20:21 |
| timothyb89 | krotscheck: cool, thanks :) | 20:21 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Remove unused nodepool-dev manifest https://review.openstack.org/330187 | 20:21 |
| *** EricGonc_ has quit IRC | 20:21 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/system-config: OpenStack-CI Apt mirror for Ubuntu Cloud Archive https://review.openstack.org/330152 | 20:23 |
| *** EricGonczer_ has joined #openstack-infra | 20:23 | |
| jeblair | zaro: do you need anything from jenkins-dev.openstack.org? | 20:23 |
| jeblair | zaro: or is it okay to delete that server now | 20:23 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Project renames may need governance updates https://review.openstack.org/330190 | 20:23 |
| jeblair | infra-root: ^ same question for jenkins-dev and jenkins.o.o | 20:23 |
| *** xarses_ has joined #openstack-infra | 20:24 | |
| mordred | jeblair: I have no use for them | 20:24 |
| *** EricGonc_ has joined #openstack-infra | 20:24 | |
| *** eezhova has quit IRC | 20:24 | |
| pleia2 | nothing I need on those | 20:25 |
| nibalizer | jeblair: delete away | 20:25 |
| pabelanger | jeblair: neither do I | 20:26 |
| openstackgerrit | Merged openstack-dev/pbr: Remember the insertion order for pbr.json https://review.openstack.org/307405 | 20:27 |
| *** sdake_ has quit IRC | 20:27 | |
| *** EricGonczer_ has quit IRC | 20:27 | |
| anteaya | zaro: okay so here is the latest entry in my httpd_log on my local gerrit instance: http://paste.openstack.org/show/516358/ the comment I created that triggered the httpd_log events: http://imgur.com/PRbDgtY yet sill no activity on the story: https://storyboard-dev.openstack.org/#!/story/11 | 20:30 |
| *** zeih has joined #openstack-infra | 20:30 | |
| *** dimtruck is now known as zz_dimtruck | 20:30 | |
| anteaya | zaro: I don't think the GETs and POST correspond to actual storyboard api commands | 20:31 |
| anteaya | is the httpd_log for my local gerrit instance tracking api commands for my instance? | 20:32 |
| fungi | jeblair: i have no files on any of those i care about | 20:32 |
| *** matrohon has quit IRC | 20:34 | |
| *** zeih has quit IRC | 20:34 | |
| *** sabari has quit IRC | 20:36 | |
| krotscheck | timothyb89: What's the project's governance? 2 x +2 to land, or.... | 20:37 |
| mtreinish | krotscheck: for the karma subunit thing? | 20:37 |
| krotscheck | mtreinish: Yep | 20:37 |
| *** banix has joined #openstack-infra | 20:37 | |
| *** abregman is now known as abregman|afk | 20:38 | |
| timothyb89 | krotscheck: uh, good question :) | 20:38 |
| *** esikachev has quit IRC | 20:38 | |
| mtreinish | krotscheck: it gets fast tracked with a PTL's +1. After a week ttx will just merge it assuming there aren't any objections | 20:38 |
| krotscheck | mtreinish: You're talking about new projects, right? I'm talking about https://review.openstack.org/#/c/330055/1 | 20:39 |
| *** julim has quit IRC | 20:39 | |
| timothyb89 | krotscheck: heh, the actual governance patch is always waiting https://review.openstack.org/#/c/327947/ | 20:39 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Fix typo with build_env https://review.openstack.org/330198 | 20:39 |
| *** FallenPegasus has joined #openstack-infra | 20:39 | |
| timothyb89 | *also waiting, I mean | 20:39 |
| *** FallenPegasus has quit IRC | 20:39 | |
| krotscheck | Ahhh gotcha | 20:39 |
| pabelanger | fungi: AJaeger: embarrassing fix ^ for wheel-build | 20:39 |
| banix | had a question about a gate job in project Kuryr; The Kuryr process runs with sudo in these tests; At some point the ipdb is used to create a veth pair in the code; this operation fails at the gate; works fine locally; what is it that we are missing? Any suggestions is appreciated | 20:40 |
| krotscheck | timothyb89: What timezone are you in? | 20:40 |
| *** esikachev has joined #openstack-infra | 20:40 | |
| mtreinish | krotscheck: ah different patch. For someting in the project repo I'd assume 2 +2s like everything else | 20:40 |
| timothyb89 | krotscheck: mountain, utc-6 I think? | 20:40 |
| krotscheck | timothyb89: What are you doing every wednesday morning at 8AM then? ;D | 20:41 |
| krotscheck | TRICK QUESTION! | 20:41 |
| krotscheck | timothyb89: http://eavesdrop.openstack.org/#JavaScript_SDK_&_Tooling_Meeting | 20:41 |
| timothyb89 | ooh. I will add that to my calendar | 20:41 |
| * krotscheck has a shiny new meeting. | 20:41 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Remove Jenkins documentation; replace with zuul-launcher https://review.openstack.org/330200 | 20:42 |
| apuimedo | banix: maybe some sort of rootwrap? | 20:42 |
| *** mtanino has quit IRC | 20:43 | |
| banix | apuimedo: what are you doing here? :) | 20:43 |
| apuimedo | banix: spying on you | 20:43 |
| apuimedo | or it could be some selinux/apparmor thingie | 20:43 |
| apuimedo | who knows | 20:43 |
| banix | apuimedo: ellaborate pls; on the rootwrap not the spying | 20:43 |
| banix | that’s why i am here; i suppose there are people here who would know | 20:44 |
| apuimedo | well, the thing is, having sudo doesn't mean you can run anything you want | 20:44 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Remove jenkins playbooks and tools https://review.openstack.org/330202 | 20:44 |
| *** kzaitsev_mb has joined #openstack-infra | 20:45 | |
| apuimedo | it depends on the sudoers configuration and the execution context | 20:45 |
| apuimedo | so in el7 selinux could kick your ass, for example | 20:45 |
| apuimedo | if you try to perform ipdb stuff on a process with the wrong label | 20:45 |
| apuimedo | banix: the best course of action would be to get to SSH into the jenkins slave | 20:46 |
| apuimedo | or obtain its image to reproduce locally | 20:46 |
| fungi | apuimedo: banix: by default we grant the "jenkins" or "zuul" user who runs the job ability to sudo any command they want as root, and provide a helper macro called "revoke-sudo" which jobs can include at any point in their list of builders if they want part of it to run with no sudo privs | 20:46 |
| anteaya | zaro: so pleia2 was kind enough to look at the storyboard-dev logs again, http://paste.openstack.org/show/516364/ still no POSTs | 20:47 |
| apuimedo | fungi: and no audit tool? | 20:47 |
| fungi | banix: what's the job name? | 20:47 |
| fungi | banix: if it runs on centos 7 i believe we have selinux enabled because centos enables it by default | 20:47 |
| banix | fungi: gate-kuryr-dsvm-fullstack-nv | 20:47 |
| fungi | banix: if it runs on ubuntu trusty then i don't think there's any selinux or apparmor happening (not 100% sure on the apparmor front) | 20:48 |
| jeblair | apuimedo: a job can run any audit tool its authors want | 20:48 |
| apuimedo | jeblair: I meant if the jobs are harnessed by apparmor/selinux | 20:48 |
| fungi | banix: job definition seems to be at http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/kuryr.yaml#n40 | 20:49 |
| *** zz_dimtruck is now known as dimtruck | 20:50 | |
| fungi | banix: i don't see it explicitly revoking sudo privs, but it's running in the context of devstack/devstack-gate which may differ depending on whether the command is being run by the stack user or the jenkins user or the tempest user | 20:50 |
| EmilienM | jeblair: http://logs.openstack.org/88/330088/3/check/gate-puppet-keystone-puppet-beaker-rspec-ubuntu-xenial/d9e1d2c/_zuul_ansible/ | 20:51 |
| EmilienM | awesome ^ | 20:51 |
| fungi | banix: as for which node type, it's inheriting ubuntu-trusty from http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/projects.yaml#n3093 so i wouldn't expect selinux to come into the picture | 20:51 |
| jeblair | EmilienM: woo! | 20:51 |
| EmilienM | really helpful to debug jobs | 20:51 |
| jeblair | EmilienM: cool -- what specifically is helpful? | 20:51 |
| EmilienM | jeblair: well everything: you can follow how job ran, the script it used (actual code) etc | 20:52 |
| *** ociuhandu has joined #openstack-infra | 20:52 | |
| banix | fungi: and we start the Kuryr process here: https://github.com/openstack/kuryr/blob/master/devstack/plugin.sh#L132 | 20:52 |
| openstackgerrit | Merged openstack-infra/project-config: Remove jenkins03 and jenkins04 https://review.openstack.org/330181 | 20:52 |
| *** jamesmcarthur has quit IRC | 20:53 | |
| jeblair | EmilienM: cool; i have a change pending to serve the .sh and .log files as text/plain | 20:53 |
| mordred | jeblair: are the decomission patches safe? like, can I +A them whenever or shoudl I wait for any manual tasks you're performing? | 20:53 |
| jeblair | mordred: all safe | 20:54 |
| fungi | banix: okay, so it's presumably running as the stack user. i believe there are sudo restrictions that happen in devstack but i don't recall the specifics. you might get better answers asking the devstack devs in #openstack-qa | 20:54 |
| fungi | banix: though also, if you have the failure logs, there may be some included system logs which could indicate whether sudo was called and refused | 20:54 |
| openstackgerrit | James Page proposed openstack-infra/project-config: Add specifications repository for charms https://review.openstack.org/330054 | 20:55 |
| banix | fungi: thanks for the help; i’ll ask there; will look into the system logs first. | 20:55 |
| anteaya | jroll: did you realize you booked the openstack sprint channel when the openstack sprint channel was already booked? | 20:55 |
| jeblair | awesome! #openstack-sprint-2 | 20:56 |
| jeblair | why did we not see that one coming and number the first one? | 20:56 |
| anteaya | jroll: storyboard is having a bug sprint in #openstack-sprint on the 22nd and 23rd | 20:56 |
| anteaya | jeblair: silly us | 20:56 |
| *** acoles_ is now known as acoles | 20:57 | |
| anteaya | jroll: I mean I'm glad you are using the channel, that is the intended purpose, thank you | 20:57 |
| anteaya | jroll: but we seem to have missed a step with the whole booking step | 20:57 |
| jroll | anteaya: oh no | 20:57 |
| * jroll looks | 20:57 | |
| anteaya | jroll: it is okay we can work it out | 20:57 |
| *** bpokorny has quit IRC | 20:57 | |
| anteaya | jroll: any thougts on how it happened? | 20:57 |
| jeblair | i think that means all the ironic devs have to fix storyboard bugs? | 20:58 |
| pleia2 | haha | 20:58 |
| anteaya | hey now we're talking | 20:58 |
| jroll | anteaya: probably me not checking things well or assuming those are in chronological order | 20:58 |
| anteaya | I had thought 'check existing prior to booking' was a gimme | 20:58 |
| anteaya | yeah I can't keep cleaning up after glance | 20:59 |
| jroll | anteaya: I notice they were booked a day apart, maybe I only checked before I put our poll up | 20:59 |
| anteaya | jroll: ah perhaps in any case | 20:59 |
| anteaya | jroll: what would you like to do | 20:59 |
| jroll | anteaya: so, sorry for that, I'll just move it to the ironic channel instead, we'll be primarily in voice | 20:59 |
| *** aysyd has quit IRC | 20:59 | |
| anteaya | no no, that is okay | 20:59 |
| fungi | well, having additional sprint channels isn't a terrible idea. especially if teams doing in-person sprints also want to leverage the channel for remote participants, given that there is frequent overlap with mid-cycles and the like | 20:59 |
| anteaya | we can move to storyboard, which is where storyboard devs are most comfy | 20:59 |
| jroll | ironic devs are most comfy in the ironic channel :) | 21:00 |
| anteaya | I'm fine with another sprint channel | 21:00 |
| anteaya | jroll: that's fair | 21:00 |
| fungi | i'm most comfy in pajamas but whatever suits! | 21:00 |
| anteaya | :) | 21:00 |
| anteaya | fungi: I thought your pajamas were your suit | 21:00 |
| jroll | honestly, last cycle was a bit confusing, as some people were only in ironic channel and not -sprint | 21:00 |
| jroll | so it won't hurt to move it | 21:00 |
| *** raildo is now known as raildo-afk | 21:01 | |
| anteaya | well I don't want you to feel displaced or anything, mostly I wanted to talk to you about it | 21:01 |
| fungi | benefit to using a separate channel for a sprint is if you have a lot of unrelated (support, et cetera) traffic in your usual channel | 21:01 |
| jroll | anteaya: it's not a problem, really :) | 21:01 |
| fungi | makes it easier to focus | 21:01 |
| anteaya | nikhil: ^^ in future can you try to follow the chronologicial format of the virtual sprints page? | 21:01 |
| jroll | fungi: indeed, I don't think it will be a problem for us | 21:01 |
| anteaya | nikhil: that would help things | 21:02 |
| fungi | for infra sprints we usually update our channel topic directing sprinters to the sprint channel and post periodic reminders when people wonder why it's so quiet | 21:02 |
| anteaya | I'm all for creating a new sprint channel | 21:02 |
| *** rbrndt has quit IRC | 21:03 | |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Logs: serve .log and .sh files as text/plain https://review.openstack.org/330119 | 21:03 |
| pabelanger | jeblair: I had a question on 330082 for your comment | 21:03 |
| pabelanger | jeblair: I left it on the review | 21:04 |
| anteaya | naming? #openstack-sprint-2, #openstack-sprint-too, #openstack-sprint-better-than-the-other-one, #openstack-sprint-also, #openstack-sprint-as-well | 21:04 |
| *** jsavak has quit IRC | 21:04 | |
| jroll | I think the meeting channels has taught us that where there is a 2, there may be a 3, and so having something other than -2 is confusing :) | 21:05 |
| *** jsavak has joined #openstack-infra | 21:05 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: use .txt as logfile extension https://review.openstack.org/330213 | 21:05 |
| *** jsavak has quit IRC | 21:05 | |
| *** jsavak has joined #openstack-infra | 21:05 | |
| *** javeriak_ has quit IRC | 21:05 | |
| anteaya | jroll: okay I count that as a +1 for -2 | 21:05 |
| *** rfolco has quit IRC | 21:06 | |
| jeblair | pabelanger: replied | 21:06 |
| *** ociuhandu has quit IRC | 21:06 | |
| *** ociuhandu_ has joined #openstack-infra | 21:06 | |
| *** ociuhandu_ is now known as ociuhandu | 21:06 | |
| dmsimard | jeblair: Hey, curious about something for zuul v2.5/v3. When a shell script is run in jenkins, you can see the output streamed in real time in the console. However, Ansible does not stream the output of a task, it only prints the results when it is complete. | 21:06 |
| pabelanger | jeblair: Thanks, make sense now | 21:06 |
| dmsimard | Do you still have that limitation around the task output not being streamed ? | 21:06 |
| openstackgerrit | Merged openstack-infra/system-config: Decomission jenkins03 and jenkins04 https://review.openstack.org/330182 | 21:07 |
| jroll | anteaya: indeed :) | 21:07 |
| *** FallenPegasus has joined #openstack-infra | 21:07 | |
| openstackgerrit | Merged openstack-infra/system-config: Decomission jenkins.openstack.org https://review.openstack.org/330185 | 21:08 |
| dmsimard | jeblair: this is especially a problem with long running tasks (say, installing openstack takes >30 minutes and could be inside just one task) | 21:08 |
| openstackgerrit | Merged openstack-infra/system-config: Decomission jenkins-dev.openstack.org https://review.openstack.org/330186 | 21:08 |
| *** esikachev has quit IRC | 21:08 | |
| nikhil | anteaya: when I posted my first entry it was on the right plaec (iirc). I agree that the next one is just a follow on. someone is moving the proposed topic to previous sprints too. would help if you've a history of the edits. | 21:08 |
| jeblair | dmsimard: run "nc 158.69.69.196 8088" to see something cool :) | 21:08 |
| *** thorst has quit IRC | 21:08 | |
| phschwartz | Has anyone seen an issue with puppet-diskimage_builder? When attempting to update my nodepool hosts I am geting a HTTP-Error 403 you must use HTTPS instead of HTTP. It was working till this morning. | 21:09 |
| jeblair | (or "telnet 158.69.69.196 8088") | 21:09 |
| openstackgerrit | Merged openstack-infra/system-config: Remove unused nodepool-dev manifest https://review.openstack.org/330187 | 21:09 |
| nikhil | anteaya: also the top of the wiki says 'reverse chronological order' | 21:10 |
| jeblair | dmsimard: we lost that ability when we firewalled our jenkins servers for security reasons. getting that back was one of the motivating factors for writing zuulv2.5 | 21:10 |
| jeblair | dmsimard: we accomplish it with a special ansible module that we use instead of 'shell' -- it captures stdout/stderr and streams it to a file which another process serves out like that | 21:11 |
| jeblair | dmsimard: once all of the jenkins masters are gone, we will be able to expose that connection information so that anyone can stream the logs for their jobs in a terminal | 21:11 |
| * fungi is particularly thrilled to be able to check in-progress job console logs without needlessly involving a web browser | 21:12 | |
| jeblair | dmsimard: and mordred and notmorgan are working on building on top of that and getting us websocket based streaming for folks who, unlike fungi, do prefer the browser :) | 21:12 |
| dmsimard | jeblair: that's sexy, I can definitely see a zuul cli client to see zuul.openstack.org without crashing my browser | 21:12 |
| fungi | heh | 21:12 |
| jeblair | but we'll have both :) | 21:12 |
| *** fernnest__ has quit IRC | 21:12 | |
| *** dprince has quit IRC | 21:13 | |
| jeblair | dmsimard: the status page is built from http://zuul.openstack.org/status.json so it's pretty easy to automatically parse it | 21:13 |
| dmsimard | jeblair: this is the module https://github.com/openstack-infra/zuul/blob/master/zuul/ansible/library/zuul_console.py? | 21:13 |
| * fungi still maintains that "the internet" is more than just "the world-wide web" | 21:13 | |
| dmsimard | jeblair: yeah, I came close to writing a CLI interface to status.json a couple times when zuul was too busy | 21:13 |
| jeblair | dmsimard: yeah, that's the server part of it, and https://github.com/openstack-infra/zuul/blob/master/zuul/ansible/library/zuul_runner.py is the 'shell' part of it | 21:13 |
| fungi | dmsimard: i think we all have | 21:13 |
| dmsimard | jeblair: That's very clever. What does it look like on multi node ? Is the output intertwined ? | 21:14 |
| fungi | dmsimard: i have a hacky one with some ansi coloring i can use to check what changes are in which pipelines and how long they're estimated to take (useful if you're trying to pick a good time to do disruptive things to the ci) | 21:14 |
| jeblair | dmsimard: i do not know exactly how we will fit this into zuul v3 yet. | 21:14 |
| *** fernnest has quit IRC | 21:14 | |
| *** e0ne has quit IRC | 21:14 | |
| jeblair | dmsimard: that too is a better question for v3, since there's no inherent multinode in v2 | 21:14 |
| dmsimard | okay. | 21:14 |
| *** bpokorny has joined #openstack-infra | 21:14 | |
| dmsimard | I think the whole "streaming output of a task" was brought up a couple times in upstream Ansible but iirc they are against it because it doesn't make sense on more than one host | 21:15 |
| *** fernnest_ has quit IRC | 21:15 | |
| *** ldnunes has quit IRC | 21:15 | |
| *** trown is now known as trown|outtypewww | 21:15 | |
| fungi | dmsimard: current multi-node design in devstack-gate just dumps the subnode ansible stdout to another file if memory serves, so the situation would still be the same for zuul v2.5 | 21:15 |
| jeblair | but i know that i want something like this in v3, and not only support for multiple nodes, but also for multiple files (not just the 'console' log) | 21:15 |
| nikhil | anteaya: in either case, I did my part and moved the glance items. thanks. | 21:16 |
| dmsimard | yeah, I guess some compromise were required to be able to transition away from jenkins | 21:16 |
| dmsimard | good work :) | 21:16 |
| dmsimard | those are the upstream issues I was talking about: https://github.com/ansible/ansible/issues/4870 & https://github.com/ansible/ansible/issues/3887 | 21:16 |
| asselin | hey folks, https://review.openstack.org/#/c/299661/22/manifests/mysql.pp introduced load_module_metadata which is causing Error: Unknown function load_module_metadata at /etc/puppet/modules/nodepool/manifests/mysql.pp:14 | 21:17 |
| fungi | dmsimard: i would be happy to pitch in on a more robust console-based zuul client once v3 is in production | 21:17 |
| asselin | anyone konw how to get that to work? | 21:17 |
| *** rhallisey has quit IRC | 21:17 | |
| *** eezhova has joined #openstack-infra | 21:17 | |
| *** mhickey has quit IRC | 21:18 | |
| notmorgan | jeblair: ++ | 21:18 |
| *** rockyg has joined #openstack-infra | 21:19 | |
| mordred | jeblair, dmsimard: for streaming for multi-host - we _might_ consider encoding the stream from each server into subunit - since it is designed to do interleaved streaming | 21:19 |
| mordred | but I obviously agree, that's a v3 design chat | 21:20 |
| jeblair | mordred: i don't think the encoding system is the problem, i think the problem is that ansible has no mechanism to obtain the data | 21:20 |
| *** thorst has joined #openstack-infra | 21:20 | |
| *** abregman|afk has quit IRC | 21:20 | |
| mordred | oh, indeed | 21:20 |
| jeblair | and yeah, i seriously doubt we will solve it right now | 21:20 |
| *** rbrndt has joined #openstack-infra | 21:21 | |
| *** Sukhdev has joined #openstack-infra | 21:22 | |
| asselin | nevermind, that was just moved from https://review.openstack.org/#/c/299661/22/manifests/init.pp | 21:23 |
| nibalizer | asselin: you need a newish stdlib | 21:24 |
| nibalizer | to have load_module_metadata | 21:24 |
| asselin | nibalizer, how 'new'? I'm using install_puppet.sh from system-config | 21:25 |
| *** thorst has quit IRC | 21:25 | |
| nibalizer | i don't know exactly.... | 21:25 |
| nibalizer | im kinda chasing this latest pip/puppet weirdness | 21:25 |
| openstackgerrit | Merged openstack-infra/project-config: Fix typo with build_env https://review.openstack.org/330198 | 21:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Remove jenkins playbooks and tools https://review.openstack.org/330202 | 21:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Remove Jenkins documentation; replace with zuul-launcher https://review.openstack.org/330200 | 21:26 |
| *** sarob has joined #openstack-infra | 21:26 | |
| pabelanger | jeblair: too bad we didn't have access to the name of the builder element in JJB, would have been nice to add that information to our ansible output | 21:27 |
| jeblair | pabelanger: i agree. *macros* have names, but they lose them when applied, and not all builders are macros | 21:28 |
| pabelanger | ya | 21:28 |
| jeblair | we could add it, but that's nontrivial. better to zuulv3 instead. | 21:28 |
| *** bpokorny has quit IRC | 21:28 | |
| *** jamesmcarthur has joined #openstack-infra | 21:29 | |
| *** mdrabe has quit IRC | 21:29 | |
| *** bpokorny has joined #openstack-infra | 21:29 | |
| *** banix has quit IRC | 21:29 | |
| pabelanger | jeblair: Ya, zuulv3 removes the need for it | 21:29 |
| asselin | crinkle, do you know which puppet-stdlib is needed for load_module_metadata? https://review.openstack.org/#/c/258193/2/manifests/init.pp | 21:30 |
| *** jamesmcarthur has quit IRC | 21:31 | |
| nibalizer | asselin: all you have to do is look at the git history or changelog | 21:31 |
| nibalizer | to determine that | 21:31 |
| *** jamesmcarthur has joined #openstack-infra | 21:31 | |
| asselin | I see: depends on: https://review.openstack.org/#/c/235005/ | 21:31 |
| anteaya | nikhil: yes, I move expired sprints to the previous sprints section | 21:31 |
| asselin | thanks | 21:31 |
| *** bpokorny_ has joined #openstack-infra | 21:31 | |
| *** sarob has quit IRC | 21:32 | |
| *** Sukhdev has quit IRC | 21:32 | |
| *** thorst has joined #openstack-infra | 21:32 | |
| anteaya | nikhil: thank you for editing the page | 21:32 |
| anteaya | nikhil: anyone can move expired sprints to the previous sprints section | 21:33 |
| *** ccamacho|out has quit IRC | 21:33 | |
| anteaya | jroll: so if I make a new sprint channel well you use #openstack-sprint? | 21:33 |
| *** Sukhdev has joined #openstack-infra | 21:33 | |
| anteaya | will* | 21:33 |
| jroll | anteaya: idk, I'm okay with just using the ironic channel to avoid confusion | 21:34 |
| *** bpokorny has quit IRC | 21:34 | |
| anteaya | okay | 21:35 |
| anteaya | jroll: thanks for chatting with me about this | 21:35 |
| jroll | anteaya: you're welcome | 21:35 |
| anteaya | :) | 21:35 |
| nibalizer | so I think all our puppet runs are failing? | 21:35 |
| nibalizer | can someone who has root go look at the logs on puppetmaster.o.o ? | 21:35 |
| nibalizer | (I lack the ssh key at the moment) | 21:36 |
| *** rossella_s has joined #openstack-infra | 21:36 | |
| *** thorst has quit IRC | 21:36 | |
| *** Sukhdev has quit IRC | 21:36 | |
| fungi | nibalizer: failing how recently? | 21:36 |
| nibalizer | last couple hours? | 21:36 |
| nibalizer | http://puppetboard.openstack.org/ a lot of unreported nodes, which I think we decided was code for 'failing' | 21:37 |
| nibalizer | also phschwartz discovered an issue downstream where the pip provider for puppet jus straight stopped functioning | 21:37 |
| nibalizer | fortunately crinkle 's provider just works | 21:37 |
| fungi | nibalizer: i saw a merged project creation patch take an hour or more to complete the circuit and show up, so we may have ansible timeouts to something unreachable again | 21:37 |
| nibalizer | ok | 21:38 |
| nibalizer | http://paste.openstack.org/show/516372/ is the error i see | 21:38 |
| phschwartz | nibalizer: fungi: my issue appears to be puppet 3.8.7 with the puppet-jenkins, puppet-diskimage_builder, and a few others that use provider => pip | 21:39 |
| nibalizer | easliy reproduced by just trying to puppet apply -e 'package { "diskimage-builder": ensure => latest, provider => pip } ' | 21:39 |
| *** bpokorny_ has quit IRC | 21:39 | |
| fungi | nibalizer: it looks (tailing the run_all log) to be working | 21:40 |
| nibalizer | hrm | 21:40 |
| nibalizer | impressive | 21:40 |
| *** bpokorny has joined #openstack-infra | 21:40 | |
| fungi | nibalizer: oh, i wonder if recently deleted jenkins masters are slowing things up because of ansible timeouts trying to reach them? | 21:40 |
| fungi | presumably refreshing the inventory will solve that | 21:41 |
| jeblair | possibly, i'm in the process of that | 21:41 |
| *** acoles is now known as acoles_ | 21:41 | |
| jeblair | and also launching zl05-zl07 | 21:41 |
| fungi | in the process of refreshing the inventory cache, or just in the process of deleting servers? want me to hold off refreshing the inventory? | 21:42 |
| asselin | nibalizer, ok, forced a resync of system-config and all the modules and it's happy now. I guess there was a clone error or something that broke first time around. | 21:42 |
| jeblair | fungi: all of the above | 21:42 |
| fungi | oh, launching new instances will refresh the cache anyway | 21:42 |
| jeblair | right | 21:42 |
| *** burgerk has quit IRC | 21:42 | |
| *** gyee has joined #openstack-infra | 21:42 | |
| fungi | though i am seeing plenty of failures | 21:42 |
| fungi | things like... | 21:42 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config: Start using pre-prov creds in dsvm-neutron-full https://review.openstack.org/329723 | 21:42 |
| jeblair | the launches are complete now, so the inventory should be ok. we may need to kill stuck processes though | 21:42 |
| fungi | 2016-06-15 21:41:56,828 p=25059 u=root | fatal: [proposal.slave.openstack.org]: FAILED! => {"changed": false, "failed": true, "msg": "/usr/bin/timeout -s 9 30m /usr/bin/puppet apply --detailed-exitcodes --logdest syslog --no-noop /opt/system-config/production/manifests/site.pp failed with return code: 6", "rc": 6, "stderr": "", "stdout": "", "stdout_lines": []} | 21:42 |
| fungi | i'll take a look on that server to see what ended up in its syslog | 21:43 |
| fungi | oh, it's the same thing asselin was just asking about | 21:43 |
| fungi | Jun 15 21:41:48 proposal puppet-user[28182]: Could not get latest version: HTTP-Error: 403 Must access using HTTPS instead of HTTP | 21:44 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config: Start using pre-prov creds in dsvm-layer4 https://review.openstack.org/329723 | 21:44 |
| *** thorst has joined #openstack-infra | 21:44 | |
| jeblair | it looks like the only active ansible proc is on static.o.o | 21:44 |
| phschwartz | fungi: that is the exact error I am getting | 21:44 |
| nibalizer | asselin: cool | 21:44 |
| phschwartz | fungi: those HTTP-Error: 403's are caused by provider => pip | 21:45 |
| fungi | so we've come full-circle to nibalizer's comment about the upstream pip provider vs crinkle's working one | 21:46 |
| nibalizer | yep | 21:46 |
| jeblair | i'm all done with the server launch/deletion now | 21:46 |
| nibalizer | so i think the fix is to spam provider openstack_pip everywhere | 21:46 |
| phschwartz | fungi: yup. I have to run to get dinner for the prego wife, when I am home I will put up reviews swapping all the prover => pip to use crinkle's | 21:46 |
| nibalizer | also my suspicion is that pypi changed how it's remote api works | 21:47 |
| phschwartz | nibalizer: it looks like they just 403 now instead of a redirect | 21:47 |
| *** bauzas has quit IRC | 21:47 | |
| fungi | nibalizer: dstufft could probably fill us in on any api changes on pypi | 21:47 |
| *** sputnik13 has quit IRC | 21:47 | |
| nibalizer | fungi: yea i just want to avoid being like 'explain what you changed today' | 21:48 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config: Start using pre-prov creds in dsvm-layer4 https://review.openstack.org/329723 | 21:48 |
| crinkle | not that it helps right now but in puppet 4 they fixed the upstream pip provider to not use the silly xml api | 21:48 |
| nibalizer | when I don't really understand what broke | 21:48 |
| *** eezhova has quit IRC | 21:48 | |
| *** PollyZ_ has quit IRC | 21:48 | |
| *** thorst has quit IRC | 21:48 | |
| asselin | setting up 3rd party ci has this 403 error: http://paste.openstack.org/show/516374/ | 21:48 |
| dstufft | fungi: unless you're using a new enough puppet it uses XMLRPC, for various reasons we're blocking HTTP access to APIs | 21:49 |
| dstufft | fungi: going to go finish dinner, but can talk more in a bit later | 21:49 |
| nibalizer | dstufft: ok so puppet-pip is broken for anyone who is on puppet 3 then | 21:49 |
| fungi | dstufft: oh, because of the urllib/urllib2 issue posted today. got it | 21:49 |
| *** sputnik13 has joined #openstack-infra | 21:49 | |
| *** matt-borland has quit IRC | 21:49 | |
| phschwartz | dstufft: sounds about right | 21:49 |
| phschwartz | fungi: do you want me to go ahead and do all the reviews to swap to openstack_pip to fix the issue for the forseable future? | 21:50 |
| *** thorst has joined #openstack-infra | 21:50 | |
| nibalizer | issue? | 21:50 |
| fungi | http://blog.blindspotsecurity.com/2016/06/advisory-http-header-injection-in.html for reference | 21:50 |
| nibalizer | i don't really understand | 21:50 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/devstack-gate: Throw-away change to test test-accounts https://review.openstack.org/330237 | 21:51 |
| *** bauzas has joined #openstack-infra | 21:51 | |
| nibalizer | the apis we are using in puppet are not turned off though | 21:52 |
| *** _ari_ has quit IRC | 21:52 | |
| nibalizer | by changing puppet to use https, it works | 21:52 |
| phschwartz | nibalizer: yeah, it looks like the redirector that was being used on pypi swapped to deny http instread of redirecting it like it did before related to that urllib exploit | 21:53 |
| nibalizer | oh awesome | 21:53 |
| nibalizer | ok | 21:53 |
| fungi | or at least that's my completely uninformed assumption | 21:53 |
| phschwartz | nibalizer: that exploit was fully disclosed mid day today so it matches when we saw the issue start. (at least time wise) | 21:53 |
| phschwartz | fungi: I think you are close on it. Looking at old logs the HTTP call redirected which it now just 403's | 21:54 |
| *** thorst_ has joined #openstack-infra | 21:54 | |
| *** thorst has quit IRC | 21:54 | |
| phschwartz | well off to dinner, back soon. | 21:54 |
| *** nikeshm has quit IRC | 21:54 | |
| phschwartz | when back if we want it, I will put the reviews up to move to crinkle's provider | 21:55 |
| fungi | i'm happy with that as a solution. i think it exists because we wanted to use it anyway? | 21:55 |
| fungi | i mean, it's a lot of change churn along with a review scramble, but it's changes we would eventually have wanted anyway? | 21:55 |
| zaro | jeblair: i don’t need anything from jenkins-dev.o.o | 21:57 |
| crinkle | mostly we wanted it because we wanted beaker to be using the mirrors | 21:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add zl05 - zl07 https://review.openstack.org/330241 | 21:57 |
| jeblair | crinkle: yeah. i suppose that's still generally useful though (like, if someone wanted to use a site-local mirror for their pip installs) | 21:57 |
| crinkle | jeblair: agreed | 21:58 |
| *** thorst_ has quit IRC | 21:59 | |
| *** weshay has quit IRC | 22:00 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Remove jenkins playbooks and tools https://review.openstack.org/330202 | 22:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Remove Jenkins documentation; replace with zuul-launcher https://review.openstack.org/330200 | 22:00 |
| nibalizer | I had expected us to use the openstack_pip provider everywhere eventually | 22:01 |
| *** hashar has quit IRC | 22:01 | |
| nibalizer | since i'm sure it will be a while before we are on puppet 4 | 22:01 |
| *** esikachev has joined #openstack-infra | 22:01 | |
| *** mriedem has quit IRC | 22:01 | |
| zaro | anteaya: so is the storyboard plugin i provided loaded ok? | 22:02 |
| *** inc0 has quit IRC | 22:02 | |
| *** jsavak has quit IRC | 22:02 | |
| *** openstackgerrit has quit IRC | 22:02 | |
| *** xyang1 has quit IRC | 22:03 | |
| *** notnownikki has quit IRC | 22:04 | |
| *** yamahata has quit IRC | 22:04 | |
| anteaya | yes | 22:04 |
| anteaya | there were no errors in the error log | 22:04 |
| zaro | do you see it as loaded in the UI? | 22:05 |
| anteaya | hang on | 22:05 |
| *** openstackgerrit has joined #openstack-infra | 22:05 | |
| anteaya | what do you mean loaded in the UI | 22:05 |
| anteaya | where do I look? | 22:05 |
| fungi | pleia2: bad news. i missed it looking over 287056, but we'll need to convert our config before we can run that | 22:06 |
| anteaya | zaro: I found it in admin plugings | 22:06 |
| pabelanger | jeblair: zl01 seems to be growing ready nodes, but I don't understand why at the moment | 22:06 |
| *** esikachev has quit IRC | 22:06 | |
| anteaya | it says it is enabled, but it gets that from the config | 22:06 |
| anteaya | zaro: I don't see loaded on that page | 22:06 |
| pleia2 | fungi: ah, fun | 22:07 |
| fungi | pleia2: i don't have time to translate the config from ini to yaml this evening, but can take a stab at it tomorrow unless you wanted to hack on that | 22:07 |
| pleia2 | fungi: I'll see what I can do | 22:07 |
| pleia2 | I'm out most of the day tomorrow, but I have some time now | 22:07 |
| dstufft | fungi: nibalizer so basically, in a not entirely public yet thing we discovered something that's going to require us to do a password reset at some point in the near future as a precaution, we decided to use this password reset to block HTTP access to "apis" (python2.6 for instance will upload to PyPI over plaintext if you don't use twine) and fold in reseting authors who might have uploaded over HTTP in that same reset | 22:08 |
| nibalizer | dstufft: i'd love to know what was changed | 22:08 |
| nibalizer | oh cool | 22:08 |
| fungi | okay, cool. i'm around... sort of. as long as my wife doesn't catch me getting back on the computer this evening when i meant to work on stuff around the house ;) | 22:08 |
| *** jcoufal_ has quit IRC | 22:09 | |
| zaro | anteaya: i think that lets us know that it’s installed ok. | 22:09 |
| *** Sukhdev has joined #openstack-infra | 22:09 | |
| nibalizer | so maybe the problem is just as simple as the ruby xml client library not following the redirect? | 22:09 |
| fungi | dstufft: thanks for the update! | 22:09 |
| *** Sukhdev has quit IRC | 22:09 | |
| pleia2 | fungi: haha, enjoy your evening ;) | 22:09 |
| nibalizer | becuase I do get a redirect when I curl "http://pypi.python.org/pypi" | 22:09 |
| anteaya | zaro: here is the contents of the error log since I installed the new plugin build: http://paste.openstack.org/show/516377/ | 22:10 |
| zaro | anteaya: hmm, do you see anything in the log when you comment on a change? | 22:10 |
| dstufft | nibalizer: we redirect "Web UI" pages from HTTP to HTTPS but we hard fail on "API" responses | 22:10 |
| anteaya | zaro: which log? | 22:10 |
| *** Sukhdev has joined #openstack-infra | 22:10 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/system-config: OpenStack-CI Apt mirror for Ubuntu Cloud Archive https://review.openstack.org/330152 | 22:10 |
| anteaya | I see activity in the httpd log as I linked above | 22:10 |
| anteaya | zaro: here it is again: http://paste.openstack.org/show/516358/ | 22:11 |
| dstufft | because the former are generally humans which might be typing into a browser (and once they get redirected once, their browser will get our HSTS setting and force them onto hTTPS in the future), but the latter are generally hard baked into programs (like puppet!) and we want them to update their URLs from http://.../ to https://../ | 22:11 |
| nibalizer | okay so if Im setting accept-headers to content-xml or whatever then im classified as an api and I get the boot? | 22:11 |
| zaro | anteaya: looking for activity in the error log | 22:11 |
| anteaya | zaro: I just gave you the error log | 22:11 |
| nibalizer | dstufft: https://github.com/puppetlabs/puppet/pull/5024 is a really easy fix for puppet core | 22:11 |
| dstufft | nibalizer: yea, I think it's content-xml that triggers that specificlly | 22:11 |
| anteaya | zaro: and here is the activity from the storyboard-dev logs: http://paste.openstack.org/show/516364/ | 22:12 |
| dstufft | nibalizer: yea, without looking at what exactly they're doing with the details from XMLRPC that's likely fixing a security bug too :) | 22:12 |
| nibalizer | cool | 22:12 |
| *** thiagop has quit IRC | 22:12 | |
| *** Sukhdev has quit IRC | 22:15 | |
| dstufft | fungi: nibalizer we're going to be giving X number of days for people to change their own password before we force a reset, most like X will be 30 days once we actually do the notification. Anyone who changes their PW between now and then won't be reset. | 22:17 |
| zaro | anteaya: can you please provide gerri.config and actions.config ? | 22:17 |
| anteaya | sure | 22:17 |
| openstackgerrit | Merged openstack-infra/system-config: Add zl05 - zl07 https://review.openstack.org/330241 | 22:17 |
| nibalizer | dstufft: ok, thanks for the heads up | 22:18 |
| nibalizer | changing now is ok though? | 22:18 |
| anteaya | zaro: http://paste.openstack.org/show/516379/ | 22:19 |
| dstufft | nibalizer: correct | 22:19 |
| nibalizer | dstufft: i r bad at curl | 22:19 |
| nibalizer | I can't actually get it to xml: curl -L -H "Accept: application/xml" -H "Content-Type: application/xml" -X GET "http://pypi.python.org/pypi" | 22:19 |
| fungi | dstufft: thanks! we'll make sure to get our openstackci account updated asap to avert breakage | 22:20 |
| *** jamesmcarthur has quit IRC | 22:20 | |
| dstufft | I added a password_date field with all existing accounts set to NULL with an UPDATE trigger to set that to NOW() | 22:20 |
| dstufft | we'll reset anyone who is still NULL | 22:20 |
| fungi | (and my personal account for sure) | 22:20 |
| dstufft | nibalizer: might have to be POST | 22:20 |
| nibalizer | ah | 22:21 |
| *** thorst_ has joined #openstack-infra | 22:21 | |
| nibalizer | yep! | 22:21 |
| dstufft | nibalizer: oh, it's also text/xml I guess? | 22:22 |
| dstufft | this code is double plus ungood | 22:22 |
| openstackgerrit | Ben Nemec proposed openstack-infra/project-config: Add stable branch jobs for tripleo-ci https://review.openstack.org/297390 | 22:22 |
| *** rlandy has quit IRC | 22:24 | |
| *** salv-orlando has quit IRC | 22:25 | |
| zaro | anteaya: does the commit message in your change contain a comment link to the SB story? | 22:25 |
| *** thorst_ has quit IRC | 22:25 | |
| *** roxanaghe has quit IRC | 22:26 | |
| *** salv-orlando has joined #openstack-infra | 22:26 | |
| anteaya | zaro: the commit message on my local gerrit patch I am commenting on: http://imgur.com/J6IvRXo | 22:27 |
| *** yamahata has joined #openstack-infra | 22:27 | |
| *** roxanaghe has joined #openstack-infra | 22:28 | |
| zaro | anteaya: i assume the link takes you directly to the story? | 22:28 |
| anteaya | oh it doesn't link | 22:28 |
| anteaya | I don't know what functionality makes Story: <story id> link to a storyboard story | 22:29 |
| *** jamesmcarthur has joined #openstack-infra | 22:29 | |
| zaro | colon probably not needed. try ‘Story 11' | 22:29 |
| anteaya | new patch or edit this commit message? | 22:29 |
| zaro | either | 22:29 |
| mgagne | anteaya: https://github.com/openstack-infra/system-config/blob/master/modules/openstack_project/manifests/gerrit.pp#L156-L160 | 22:29 |
| anteaya | and colon is used in revew.o.o and it links | 22:29 |
| *** roxanaghe has quit IRC | 22:30 | |
| anteaya | mgagne: thanks | 22:30 |
| *** roxanaghe has joined #openstack-infra | 22:30 | |
| zaro | review.o.o might have different matcher | 22:30 |
| mgagne | which is used by https://github.com/openstack-infra/puppet-gerrit/blob/master/templates/gerrit.config.erb#L129-L138 | 22:30 |
| *** vhosakot has quit IRC | 22:31 | |
| anteaya | okay it links now | 22:31 |
| anteaya | and the link takes me to the right story | 22:31 |
| anteaya | however a new comment still shows no new activity in the story that I can see | 22:32 |
| *** sergmelikyan has joined #openstack-infra | 22:34 | |
| anteaya | no new activity in the error log on the local gerrit instance | 22:34 |
| *** sergmelikyan has quit IRC | 22:34 | |
| anteaya | zaro: I'm assuming my gerrit.config and action.config look okay | 22:39 |
| *** edmondsw has quit IRC | 22:40 | |
| *** Sukhdev has joined #openstack-infra | 22:42 | |
| jhesketh | Morning | 22:43 |
| *** rossella_s has quit IRC | 22:44 | |
| *** Sukhdev has quit IRC | 22:44 | |
| *** banix has joined #openstack-infra | 22:44 | |
| *** Sukhdev has joined #openstack-infra | 22:46 | |
| anteaya | morning jhesketh | 22:47 |
| *** FallenPegasus has quit IRC | 22:49 | |
| openstackgerrit | Thanh Ha proposed openstack-infra/jenkins-job-builder: Allow update to provide plugin_info via yaml https://review.openstack.org/326722 | 22:50 |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/puppet-gerritbot: Migrate gerritbot config from ini to yaml https://review.openstack.org/330260 | 22:51 |
| pleia2 | fungi: ^^ that gerritbot config should work | 22:51 |
| *** rockyg has quit IRC | 22:52 | |
| *** mriedem has joined #openstack-infra | 22:52 | |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/puppet-gerritbot: Migrate gerritbot config from ini to yaml https://review.openstack.org/330260 | 22:53 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Only update the time database if the build has a duration https://review.openstack.org/330263 | 22:53 |
| *** dimtruck is now known as zz_dimtruck | 22:53 | |
| *** Sukhdev has quit IRC | 22:53 | |
| *** gouthamr has joined #openstack-infra | 22:54 | |
| *** jcoufal has joined #openstack-infra | 22:55 | |
| *** hongbin has quit IRC | 22:56 | |
| *** EricGonczer_ has joined #openstack-infra | 22:57 | |
| jeblair | notmorgan: comments on https://review.openstack.org/325661 | 22:59 |
| *** EricGonc_ has quit IRC | 23:00 | |
| zaro | anteaya: sorry, had a phone call come in. | 23:02 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
