| fungi | i've been wondering but never dug into it | 00:00 |
|---|---|---|
| fungi | ahh, yeah there goes ansible | 00:00 |
| mtreinish | fungi: well I have a ruby script I'm running that does it | 00:00 |
| mtreinish | but iirc -v for verbose will print '$topic $payload' | 00:00 |
| fungi | oh, yep so it does! | 00:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Ensure ref-updated jobs run with their ref https://review.openstack.org/488216 | 00:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix null ref in github tests https://review.openstack.org/490257 | 00:01 |
| *** rloo has quit IRC | 00:01 | |
| fungi | man mosquitto_sub ftw. i should have done that before | 00:01 |
| mtreinish | clarkb: I assume we'd have had a logstash even by now | 00:01 |
| mtreinish | fungi: it is kinda cool to see a stream of all the work the infra machines do all the time | 00:04 |
| mtreinish | it's very busy | 00:04 |
| fungi | kinda cool for you, kinda scary for me ;) | 00:05 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Revert "Create publish-openstack-python-tarball trusted job" https://review.openstack.org/490258 | 00:05 |
| mtreinish | clarkb: is there anything in the logstash worker logs? because I'm still not seeing anything | 00:06 |
| jeblair | mtreinish: it's going to be a blast when we add this to zuul :) | 00:06 |
| mtreinish | hahaha, yeah. We might have to move it to a bigger node, just for more bandwidth | 00:07 |
| openstackgerrit | Merged openstack-infra/system-config master: Don't use subunit user on logstash workers https://review.openstack.org/490249 | 00:07 |
| mtreinish | fungi: do you remember what the msg rate we were hitting the bandwidth limits on for firehose.o.o | 00:08 |
| * mtreinish checks the graph | 00:08 | |
| fungi | i do not, no | 00:09 |
| clarkb | mtreinish: I'm just seeing it happily talk to logstash | 00:09 |
| clarkb | not seeing tracebacks like I was before | 00:09 |
| clarkb | what is the mqtt port? | 00:09 |
| *** kaisers2 has joined #openstack-infra | 00:09 | |
| fungi | mtreinish: i remember some big numbers | 00:09 |
| clarkb | 8883? | 00:09 |
| mtreinish | clarkb: yeah that's the port it should publish on | 00:10 |
| *** kaisers1 has quit IRC | 00:10 | |
| mtreinish | it's the ssl mqtt port | 00:10 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix github dependent pipeline with merge https://review.openstack.org/489925 | 00:11 |
| mtreinish | fungi: yeah from https://github.com/mtreinish/firehose/blob/boston-summit/pub_throughput.png it's with >1024 pubs and subs | 00:11 |
| mtreinish | that was with 1k msg size iirc | 00:12 |
| *** pfalleno1 has quit IRC | 00:12 | |
| *** pfallenop has joined #openstack-infra | 00:12 | |
| *** pfallenop has joined #openstack-infra | 00:12 | |
| clarkb | mtreinish: watching netstat it appears to be making new connections constantly. Possibly due to the CA cert subunit user problem? | 00:12 |
| fungi | sounds right | 00:12 |
| clarkb | mtreinish: guessing once that is fixed it will connect properly? | 00:12 |
| fungi | the 1k/msg i mean | 00:12 |
| fungi | clarkb: seems likely | 00:13 |
| mtreinish | clarkb: maybe, it might reconnects for each message | 00:13 |
| mtreinish | clarkb: yeah I think it will connect fresh on each message. It calls paho.mqtt.publish.single() to send the message | 00:14 |
| *** jamesmcarthur has joined #openstack-infra | 00:15 | |
| mtreinish | which will connect once per call: https://github.com/eclipse/paho.mqtt.python/blob/master/src/paho/mqtt/publish.py | 00:15 |
| *** baoli has joined #openstack-infra | 00:16 | |
| *** gmann has quit IRC | 00:17 | |
| *** baoli has quit IRC | 00:17 | |
| *** gmann has joined #openstack-infra | 00:17 | |
| *** jamesmcarthur has quit IRC | 00:17 | |
| *** s-shiono has joined #openstack-infra | 00:18 | |
| *** jamesmcarthur has joined #openstack-infra | 00:18 | |
| mtreinish | clarkb: maybe check the firehose.o.o mosquitto logs, maybe the connections are getting rejected on the ssl handshake again | 00:21 |
| clarkb | 1501719724: OpenSSL Error: error:140F3042:SSL routines:ssl_undefined_const_function:called a function you should not call | 00:22 |
| clarkb | I see a lot of that | 00:22 |
| mtreinish | yeah that was the cert thing we were seeing before | 00:22 |
| *** baoli has joined #openstack-infra | 00:22 | |
| clarkb | mtreinish: and should be fixed if the file is in place with proper ownership? | 00:24 |
| mtreinish | I think so | 00:24 |
| clarkb | ansible still appears to be out to lunch | 00:24 |
| clarkb | I've got to go herd children but can look at things in the morning | 00:25 |
| *** veebers2 has joined #openstack-infra | 00:25 | |
| mtreinish | it's been ~3 months since we hit that on the subunit gearman and/or ansible stuff | 00:25 |
| mtreinish | so I'm a bit foggy on the details | 00:25 |
| clarkb | I think we get the ca file sorted out and go from there | 00:25 |
| mtreinish | haha, although googling the error found: https://mgysel.ch/mosquitto-tls-instability/ | 00:26 |
| *** blznblzn2 has quit IRC | 00:26 | |
| mtreinish | but I don't buy that as a solution | 00:26 |
| mtreinish | (although I might try it for fixing the websockets stuff) | 00:26 |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Create publish-openstack-python-tarball trusted job" https://review.openstack.org/490258 | 00:26 |
| ianw | pabelanger: fedora 26 built and booted on the first attempt. i don't recall that ever happening before, i guess we're getting better at this :) | 00:27 |
| * mtreinish steps away to find food | 00:27 | |
| ianw | (insert everything coming crashing down now i've said that) | 00:27 |
| clarkb | ianw: if you want to have more logo fun https://review.openstack.org/#/c/458200/ should update cgit | 00:28 |
| clarkb | ianw: mnaser was kind enough to get a screenshot of the eidt | 00:29 |
| ianw | clarkb: cool. at least after that other fire-drill, if everything starts going haywire as gitweb fails out randomly we can debug it quickly | 00:30 |
| *** jamesmcarthur has quit IRC | 00:30 | |
| ianw | actually that wasn't a fire drill, that was just a regular fire :) | 00:31 |
| *** jamesmcarthur has joined #openstack-infra | 00:32 | |
| *** jamesmcarthur has quit IRC | 00:36 | |
| openstackgerrit | Merged openstack-infra/system-config master: Use new openstack logo on git.o.o https://review.openstack.org/458200 | 00:37 |
| mnaser | ianw borrow you for this one too https://review.openstack.org/#/c/490171/ :D | 00:38 |
| *** rwsu has quit IRC | 00:38 | |
| *** jamesmcarthur has joined #openstack-infra | 00:44 | |
| *** zhurong has joined #openstack-infra | 00:46 | |
| *** jamesmcarthur has quit IRC | 00:46 | |
| *** jamesmcarthur has joined #openstack-infra | 00:47 | |
| *** jamesmcarthur has quit IRC | 00:49 | |
| openstackgerrit | Rico Lin proposed openstack-infra/project-config master: Avoid py35 and docs ocata gate for heat-agents https://review.openstack.org/490092 | 00:49 |
| openstackgerrit | Merged openstack-infra/system-config master: Fix OpenStack header logo in Gitweb https://review.openstack.org/490171 | 00:51 |
| *** LindaWang has joined #openstack-infra | 00:55 | |
| *** tinwood has quit IRC | 00:56 | |
| *** Adri2000 has quit IRC | 00:56 | |
| *** tinwood has joined #openstack-infra | 00:57 | |
| openstackgerrit | Chaoyi Huang proposed openstack-infra/project-config master: Convert coverage and multiregion test to voting jobs https://review.openstack.org/490261 | 00:57 |
| *** Adri2000 has joined #openstack-infra | 00:57 | |
| *** ym200 has joined #openstack-infra | 00:58 | |
| *** camunoz has quit IRC | 00:59 | |
| *** jamesmcarthur has joined #openstack-infra | 00:59 | |
| *** hongbin has joined #openstack-infra | 00:59 | |
| *** jamesmcarthur has quit IRC | 01:00 | |
| openstackgerrit | Rico Lin proposed openstack-infra/project-config master: Avoid py35 and docs ocata gate for heat-agents https://review.openstack.org/490092 | 01:01 |
| *** jamesmcarthur has joined #openstack-infra | 01:02 | |
| *** jamesmcarthur has quit IRC | 01:06 | |
| *** jamesmcarthur has joined #openstack-infra | 01:06 | |
| *** EricGonczer_ has joined #openstack-infra | 01:08 | |
| *** mat128 has joined #openstack-infra | 01:08 | |
| *** Apoorva has quit IRC | 01:11 | |
| *** DFFlanders has quit IRC | 01:11 | |
| *** Apoorva has joined #openstack-infra | 01:12 | |
| openstackgerrit | Merged openstack-infra/project-config master: Initial Fedora 26 testing https://review.openstack.org/489820 | 01:15 |
| *** Apoorva has quit IRC | 01:17 | |
| *** yamahata has quit IRC | 01:20 | |
| *** lewo has quit IRC | 01:20 | |
| *** lewo has joined #openstack-infra | 01:25 | |
| *** jamesdenton has joined #openstack-infra | 01:25 | |
| *** jamesmcarthur has quit IRC | 01:25 | |
| *** jamesmcarthur has joined #openstack-infra | 01:26 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add project-config to zuulv3.o.o https://review.openstack.org/490268 | 01:28 |
| *** gongysh has joined #openstack-infra | 01:30 | |
| *** jamesmcarthur has quit IRC | 01:31 | |
| *** namnh has joined #openstack-infra | 01:31 | |
| *** mat128 has quit IRC | 01:31 | |
| *** jamesmcarthur has joined #openstack-infra | 01:32 | |
| *** liujiong has joined #openstack-infra | 01:33 | |
| *** armax has joined #openstack-infra | 01:34 | |
| *** esikachev has joined #openstack-infra | 01:36 | |
| *** mat128 has joined #openstack-infra | 01:36 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config master: tap-as-a-service: Add grafana dashboard https://review.openstack.org/490272 | 01:36 |
| *** thorst_afk has quit IRC | 01:37 | |
| *** jamesmcarthur has quit IRC | 01:37 | |
| *** esikachev has quit IRC | 01:40 | |
| *** jamesmcarthur has joined #openstack-infra | 01:40 | |
| *** rkukura has quit IRC | 01:42 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Check change owners against OSF Individual Members https://review.openstack.org/483974 | 01:43 |
| fungi | clarkb: ^ hopefully that addresses your concerns? | 01:43 |
| fungi | i'll adjust the rest of the stack tomorrow | 01:43 |
| *** jamesmcarthur has quit IRC | 01:44 | |
| *** mat128 has quit IRC | 01:44 | |
| *** yamamoto has quit IRC | 01:45 | |
| fungi | i also just finished building the member/nonmember contributor lists i mentioned and will follow up to the ml with those tomorrow as well | 01:45 |
| *** liujiong has quit IRC | 01:45 | |
| *** jamesmcarthur has joined #openstack-infra | 01:45 | |
| *** yamamoto has joined #openstack-infra | 01:45 | |
| *** mriedem has quit IRC | 01:47 | |
| *** EricGonczer_ has quit IRC | 01:51 | |
| *** ramishra has joined #openstack-infra | 01:51 | |
| *** EricGonczer_ has joined #openstack-infra | 01:51 | |
| *** EricGonczer_ has quit IRC | 01:52 | |
| *** cuongnv has joined #openstack-infra | 01:56 | |
| openstackgerrit | Rico Lin proposed openstack-infra/project-config master: Avoid py35 and docs ocata gate for heat-agents https://review.openstack.org/490092 | 01:56 |
| *** mtanino has quit IRC | 01:59 | |
| *** ihrachys has quit IRC | 01:59 | |
| *** ihrachys has joined #openstack-infra | 02:00 | |
| tonyb | fungi while you're workign on owners.py can you look at : https://review.openstack.org/#/c/483188/ ? | 02:00 |
| tonyb | We have projects that don't want to be EOLd that are still consuming the old stable branches (for some value of consuming) What I'd like to do is setup the jobs so that for ocata,newton,mpike and master they run what ever jobs they want to but for anything else they just get the noop job | 02:02 |
| tonyb | Is there anything like a 'nice' way to do that if not I'll settle for something that works with big fat comments | 02:03 |
| openstackgerrit | Rico Lin proposed openstack-infra/project-config master: Avoid py35 and docs ocata gate for heat-agents https://review.openstack.org/490092 | 02:04 |
| *** mat128 has joined #openstack-infra | 02:04 | |
| fungi | tonyb: is 483188 urgent for the coming election? or should we try to work it in between ptl and tc elections? | 02:05 |
| tonyb | fungi: It isn't urgent there is a work around for the commin election | 02:05 |
| fungi | i'm cool with updating one-off scripts like this to python 3, just not sure if anyone is stuck without access to python 3 | 02:05 |
| tonyb | fungi: when I wrote it I thought it was urgent but then I got a clue :) | 02:05 |
| fungi | er without access to python 2 | 02:06 |
| fungi | /usr/bin/python is python2 according to the psf, and /usr/bin/python3 is what we should put in the shebang if we want it run on python 3 | 02:06 |
| tonyb | fungi: Yeah the election repo is python3 first but we try to keep things working on both | 02:07 |
| fungi | so we should do one or the other, but not much point in making a script like this support both | 02:07 |
| tonyb | Oh okay I didn't know the PSF had said that thing | 02:07 |
| fungi | also i question why there's a wrapper around owners.py. can't we just move it into whatever repo the election officials want to stick with and collapse them into a single script? | 02:08 |
| tonyb | I kinda thought python2 and python3 were what you expect and the dirsto got to pick what python is | 02:08 |
| *** liujiong has joined #openstack-infra | 02:08 | |
| tonyb | fungi: Yup we can do that. wrapping it was an interim solution. | 02:09 |
| fungi | and yeah, archlinux i think went a little wild and decided to point python at python3 but the python community has documented that's not a great idea | 02:09 |
| tonyb | fungi: we had talked about moving owners into the election repo but we didn't do it and I thought owners is used by infra for other things. | 02:09 |
| fungi | it's used by the foundation for generating lists of contributors for certain timeframes to send event registration discounts, or at least has been in the past, but that's not incompatible with putting it in the election officials' care | 02:10 |
| *** jrist has joined #openstack-infra | 02:11 | |
| tonyb | okay. Perhaps we can move it between PTL and TC elections | 02:11 |
| tonyb | fungi: heck we could even just do it at the PTG. | 02:12 |
| tonyb | fungi: we'd refactor is slightly to have to cli's to suit both use cases but that isn't very interesting | 02:12 |
| fungi | right. seems like a good idea to me! | 02:13 |
| fungi | anyway, trying to finish getting these ml updates sent out with all the names of the people on the electoral rolls so people can make sure to double-check their member profiles | 02:14 |
| *** lrossetti has quit IRC | 02:14 | |
| *** gcb has quit IRC | 02:15 | |
| tonyb | fungi: cool. Thanks for letting me distract you | 02:15 |
| *** lrossetti has joined #openstack-infra | 02:15 | |
| *** gcb has joined #openstack-infra | 02:16 | |
| *** lrossetti has quit IRC | 02:16 | |
| *** lrossetti has joined #openstack-infra | 02:17 | |
| fungi | always a pleasure | 02:17 |
| *** lrossetti has quit IRC | 02:27 | |
| *** yushb has joined #openstack-infra | 02:34 | |
| *** baoli has quit IRC | 02:36 | |
| *** esberglu has quit IRC | 02:39 | |
| *** esberglu has joined #openstack-infra | 02:39 | |
| *** yamamoto has quit IRC | 02:41 | |
| *** esberglu has quit IRC | 02:47 | |
| *** skelso has joined #openstack-infra | 02:47 | |
| *** sekelso has joined #openstack-infra | 02:49 | |
| *** yamamoto has joined #openstack-infra | 02:51 | |
| *** skelso has quit IRC | 02:52 | |
| *** ihrachys has quit IRC | 02:58 | |
| *** lrossetti has joined #openstack-infra | 02:59 | |
| *** jamesmcarthur has quit IRC | 03:01 | |
| *** lrossetti has quit IRC | 03:01 | |
| *** lrossetti has joined #openstack-infra | 03:02 | |
| *** sekelso has quit IRC | 03:03 | |
| *** gouthamr has quit IRC | 03:03 | |
| ianw | fungi: where did we end up with osic (before i start digging too much ... it seems we have one fedora-26 osic node up and ready, but zuul doesn't want to put a job on it for some reason) | 03:04 |
| *** esberglu has joined #openstack-infra | 03:05 | |
| *** lrossetti has quit IRC | 03:05 | |
| *** lrossetti has joined #openstack-infra | 03:06 | |
| fungi | ianw: osic should be working again. no guess as to why we're not scheduling jobs onto f26 | 03:07 |
| *** sekelso has joined #openstack-infra | 03:07 | |
| fungi | maybe the zuul-launcher it thinks it's registered to lost it in the shuffle? | 03:07 |
| *** EricGonczer_ has joined #openstack-infra | 03:07 | |
| ianw | hmm, maybe | 03:07 |
| fungi | could try deleting that node and seeing what happens with the next one | 03:07 |
| ianw | just doing that :) | 03:08 |
| fungi | we were a little backlogged and so slammed osic _hard_ when i brought it back online earlier | 03:08 |
| fungi | so i wouldn't be entirely surprised if it's something like that | 03:08 |
| *** skelso has joined #openstack-infra | 03:09 | |
| *** lrossetti has quit IRC | 03:10 | |
| *** lrossetti has joined #openstack-infra | 03:11 | |
| *** sekelso has quit IRC | 03:13 | |
| ianw | fungi: yep, a new node seems to have now taken the job, thanks | 03:14 |
| *** nicolasbock__ has joined #openstack-infra | 03:14 | |
| fungi | yeah, we basically tried to build 768 nodes simultaneously. it's a beautiful spike on the test nodes graph | 03:18 |
| tonyb | fungi: and such a nice number unlike say 769 or 766 :D | 03:18 |
| *** masuberu has joined #openstack-infra | 03:18 | |
| fungi | 3x2^8? | 03:19 |
| fungi | the two you mentioned have much more interesting factors | 03:20 |
| *** lrossetti has quit IRC | 03:20 | |
| tonyb | fungi: Yeah basically | 03:20 |
| fungi | 769 is prime, in fact | 03:20 |
| *** lrossetti has joined #openstack-infra | 03:21 | |
| fungi | and half 766 is prime | 03:21 |
| fungi | so depends on your idea of beauty ;) | 03:21 |
| tonyb | fungi: okay that would have been cool but I didn't knwo that until just now | 03:21 |
| tonyb | fungi: doesn't everything? | 03:21 |
| fungi | touché | 03:22 |
| *** masber has quit IRC | 03:22 | |
| fungi | time for me to ponder the beauty of the inside of my eyelids for a bit anyway | 03:22 |
| fungi | later, all | 03:22 |
| tonyb | fungi: o/ | 03:22 |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config master: tap-as-a-service: Add a coverage job https://review.openstack.org/490307 | 03:23 |
| *** blznblzn2 has joined #openstack-infra | 03:28 | |
| *** hongbin has quit IRC | 03:29 | |
| *** tuanluong has joined #openstack-infra | 03:30 | |
| *** lrossetti has quit IRC | 03:30 | |
| *** jamesmcarthur has joined #openstack-infra | 03:31 | |
| *** lrossetti has joined #openstack-infra | 03:31 | |
| *** yamamoto has quit IRC | 03:33 | |
| *** xarses_ has quit IRC | 03:36 | |
| *** thorst_afk has joined #openstack-infra | 03:38 | |
| *** veebers2 has quit IRC | 03:38 | |
| *** skelso has quit IRC | 03:38 | |
| *** ykarel has joined #openstack-infra | 03:39 | |
| *** xarses has joined #openstack-infra | 03:41 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Use caching reverse proxy for pypi https://review.openstack.org/483761 | 03:42 |
| *** yamamoto has joined #openstack-infra | 03:42 | |
| *** lrossetti has quit IRC | 03:42 | |
| *** lrossetti has joined #openstack-infra | 03:43 | |
| *** EricGonczer_ has quit IRC | 03:45 | |
| *** ykarel has quit IRC | 03:46 | |
| *** ykarel has joined #openstack-infra | 03:46 | |
| *** yamamoto has quit IRC | 03:48 | |
| *** jamesmcarthur has quit IRC | 03:49 | |
| *** yamamoto has joined #openstack-infra | 03:49 | |
| *** jamesmcarthur has joined #openstack-infra | 03:50 | |
| *** thorst_afk has quit IRC | 03:52 | |
| *** yamamoto has quit IRC | 03:54 | |
| *** links has joined #openstack-infra | 03:54 | |
| *** EricGonczer_ has joined #openstack-infra | 03:56 | |
| *** lrossetti has quit IRC | 03:57 | |
| *** lrossetti has joined #openstack-infra | 03:57 | |
| *** adisky__ has joined #openstack-infra | 04:03 | |
| *** lrossetti has quit IRC | 04:07 | |
| *** EricGonc_ has joined #openstack-infra | 04:07 | |
| *** EricGonczer_ has quit IRC | 04:08 | |
| *** gildub has quit IRC | 04:11 | |
| *** EricGonc_ has quit IRC | 04:13 | |
| *** gongysh has quit IRC | 04:15 | |
| *** lrossetti has joined #openstack-infra | 04:16 | |
| *** stakeda has joined #openstack-infra | 04:17 | |
| *** yushb has quit IRC | 04:19 | |
| *** lrossetti has quit IRC | 04:21 | |
| *** ramishra has quit IRC | 04:22 | |
| *** ramishra has joined #openstack-infra | 04:24 | |
| *** jamesmcarthur has quit IRC | 04:25 | |
| *** jamesmcarthur has joined #openstack-infra | 04:25 | |
| *** stakeda has quit IRC | 04:30 | |
| *** lrossetti has joined #openstack-infra | 04:30 | |
| *** jamesmcarthur has quit IRC | 04:30 | |
| *** lrossetti has quit IRC | 04:31 | |
| *** lrossetti has joined #openstack-infra | 04:31 | |
| *** yamahata has joined #openstack-infra | 04:32 | |
| *** esikachev has joined #openstack-infra | 04:36 | |
| *** stakeda has joined #openstack-infra | 04:38 | |
| *** jamesmcarthur has joined #openstack-infra | 04:45 | |
| *** mat128 has quit IRC | 04:48 | |
| *** lrossetti has quit IRC | 04:49 | |
| *** esikachev has quit IRC | 04:53 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate master: Strip trailing null in env var output https://review.openstack.org/490333 | 04:59 |
| ianw | clarkb: ^ in today's episode of "problems you didn't even know you had" | 05:01 |
| *** cuongnv has quit IRC | 05:04 | |
| *** lrossetti has joined #openstack-infra | 05:05 | |
| *** lrossetti has quit IRC | 05:07 | |
| *** lrossetti has joined #openstack-infra | 05:07 | |
| *** armax has quit IRC | 05:13 | |
| *** armax has joined #openstack-infra | 05:13 | |
| *** armax has quit IRC | 05:13 | |
| *** armax has joined #openstack-infra | 05:14 | |
| *** lrossetti has quit IRC | 05:14 | |
| *** armax has quit IRC | 05:14 | |
| *** lrossetti has joined #openstack-infra | 05:14 | |
| *** pgadiya has joined #openstack-infra | 05:15 | |
| *** armax has joined #openstack-infra | 05:15 | |
| *** armax has quit IRC | 05:15 | |
| *** armax has joined #openstack-infra | 05:16 | |
| *** armax has quit IRC | 05:16 | |
| *** lrossetti has quit IRC | 05:17 | |
| *** lrossetti has joined #openstack-infra | 05:17 | |
| *** jascott1 has quit IRC | 05:18 | |
| *** jascott1 has joined #openstack-infra | 05:19 | |
| *** lrossetti has quit IRC | 05:22 | |
| *** lrossetti has joined #openstack-infra | 05:22 | |
| *** esikachev has joined #openstack-infra | 05:22 | |
| *** esberglu has quit IRC | 05:23 | |
| *** jascott1 has quit IRC | 05:23 | |
| *** cuongnv has joined #openstack-infra | 05:27 | |
| *** lrossetti has quit IRC | 05:29 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Make timer tests less racy https://review.openstack.org/490208 | 05:30 |
| *** lrossetti has joined #openstack-infra | 05:30 | |
| *** prateek has joined #openstack-infra | 05:31 | |
| *** jamesmcarthur has quit IRC | 05:33 | |
| *** lrossetti has quit IRC | 05:34 | |
| *** lrossetti has joined #openstack-infra | 05:34 | |
| *** lrossetti has quit IRC | 05:38 | |
| *** lrossetti has joined #openstack-infra | 05:38 | |
| *** robcresswell has quit IRC | 05:41 | |
| *** ym200 has quit IRC | 05:41 | |
| *** mandre has joined #openstack-infra | 05:43 | |
| *** lrossetti has quit IRC | 05:43 | |
| *** lrossetti has joined #openstack-infra | 05:43 | |
| *** esikachev has quit IRC | 05:44 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add topology flag to Heat job template https://review.openstack.org/482048 | 05:47 |
| *** lrossetti_ has joined #openstack-infra | 05:48 | |
| *** esikachev has joined #openstack-infra | 05:49 | |
| *** thorst_afk has joined #openstack-infra | 05:49 | |
| *** lrossetti has quit IRC | 05:50 | |
| *** yushb has joined #openstack-infra | 05:50 | |
| *** psachin has joined #openstack-infra | 05:51 | |
| *** dims has quit IRC | 05:52 | |
| *** cshastri has joined #openstack-infra | 05:53 | |
| *** thorst_afk has quit IRC | 05:53 | |
| *** dims has joined #openstack-infra | 05:54 | |
| *** lrossetti_ has quit IRC | 05:55 | |
| *** lrossetti has joined #openstack-infra | 05:55 | |
| *** larainema has quit IRC | 05:56 | |
| *** slaweq has joined #openstack-infra | 06:00 | |
| *** aeng has quit IRC | 06:00 | |
| *** lrossetti has quit IRC | 06:00 | |
| *** slaweq has quit IRC | 06:02 | |
| *** mwhahaha has quit IRC | 06:02 | |
| *** jamesmcarthur has joined #openstack-infra | 06:04 | |
| *** jamesmcarthur has quit IRC | 06:12 | |
| *** wolverineav has joined #openstack-infra | 06:13 | |
| *** wolverineav has quit IRC | 06:17 | |
| *** seongsoocho has quit IRC | 06:17 | |
| *** lrossetti has joined #openstack-infra | 06:17 | |
| *** seongsoocho has joined #openstack-infra | 06:18 | |
| *** pgadiya has quit IRC | 06:18 | |
| *** rcernin has joined #openstack-infra | 06:19 | |
| *** veebers has joined #openstack-infra | 06:23 | |
| *** alexchadin has joined #openstack-infra | 06:24 | |
| *** jascott1 has joined #openstack-infra | 06:24 | |
| *** kjackal_ has quit IRC | 06:25 | |
| *** lrossetti has quit IRC | 06:26 | |
| *** gildub has joined #openstack-infra | 06:26 | |
| *** lrossetti has joined #openstack-infra | 06:26 | |
| *** andreas_s has joined #openstack-infra | 06:27 | |
| *** lrossetti has quit IRC | 06:27 | |
| *** lrossetti has joined #openstack-infra | 06:27 | |
| *** ralonsoh has joined #openstack-infra | 06:29 | |
| *** kjackal_ has joined #openstack-infra | 06:29 | |
| *** lrossetti_ has joined #openstack-infra | 06:32 | |
| *** lrossetti has quit IRC | 06:32 | |
| *** lrossetti_ has quit IRC | 06:34 | |
| *** pgadiya has joined #openstack-infra | 06:34 | |
| *** lrossetti_ has joined #openstack-infra | 06:35 | |
| *** rlandy has joined #openstack-infra | 06:37 | |
| *** lrossetti_ has quit IRC | 06:37 | |
| *** rlandy is now known as rlandy|ci_mtgs | 06:37 | |
| *** lrossetti_ has joined #openstack-infra | 06:38 | |
| openstackgerrit | Mark Korondi proposed openstack-infra/project-config master: Bringing upstream training virtual environment over here https://review.openstack.org/490202 | 06:38 |
| *** martinkopec has joined #openstack-infra | 06:38 | |
| *** makowals has joined #openstack-infra | 06:39 | |
| *** yushb has quit IRC | 06:39 | |
| *** yushb has joined #openstack-infra | 06:40 | |
| *** lrossetti_ has quit IRC | 06:40 | |
| *** lrossetti_ has joined #openstack-infra | 06:41 | |
| *** lrossetti_ has quit IRC | 06:46 | |
| *** pgadiya has quit IRC | 06:48 | |
| *** pcaruana has joined #openstack-infra | 06:48 | |
| *** lindsayk has quit IRC | 06:51 | |
| *** gildub has quit IRC | 06:51 | |
| *** coolsvap has joined #openstack-infra | 06:53 | |
| *** arif-ali has quit IRC | 06:54 | |
| *** lrossetti has joined #openstack-infra | 06:55 | |
| *** pgadiya has joined #openstack-infra | 07:00 | |
| *** arif-ali has joined #openstack-infra | 07:01 | |
| *** jamesmcarthur has joined #openstack-infra | 07:01 | |
| *** slaweq has joined #openstack-infra | 07:03 | |
| *** gcb has quit IRC | 07:05 | |
| *** jamesmcarthur has quit IRC | 07:06 | |
| *** slaweq has quit IRC | 07:09 | |
| *** jtomasek has joined #openstack-infra | 07:09 | |
| *** lrossetti has quit IRC | 07:09 | |
| *** lrossetti has joined #openstack-infra | 07:09 | |
| *** Dinesh_Bhor has quit IRC | 07:10 | |
| *** lrossetti has quit IRC | 07:14 | |
| *** Dinesh_Bhor has joined #openstack-infra | 07:16 | |
| *** tesseract has joined #openstack-infra | 07:17 | |
| *** aviau has quit IRC | 07:19 | |
| *** aviau has joined #openstack-infra | 07:19 | |
| *** ccamacho has joined #openstack-infra | 07:20 | |
| *** markus_z has joined #openstack-infra | 07:21 | |
| *** thorst_afk has joined #openstack-infra | 07:22 | |
| *** gtrxcb has quit IRC | 07:24 | |
| *** yushb has quit IRC | 07:25 | |
| *** gildub has joined #openstack-infra | 07:25 | |
| *** thorst_afk has quit IRC | 07:27 | |
| *** esikachev has quit IRC | 07:30 | |
| AJaeger | yolanda: could you review https://review.openstack.org/489854 as well, please? | 07:31 |
| yolanda | yep | 07:31 |
| *** lrossetti has joined #openstack-infra | 07:32 | |
| openstackgerrit | Merged openstack-infra/project-config master: [Kuryr] Setup a non-voting py35 job for libnetwork https://review.openstack.org/490169 | 07:33 |
| openstackgerrit | Merged openstack-infra/project-config master: tap-as-a-service: Add grafana dashboard https://review.openstack.org/490272 | 07:34 |
| openstackgerrit | Merged openstack-infra/project-config master: Avoid py35 and docs ocata gate for heat-agents https://review.openstack.org/490092 | 07:35 |
| *** aarefiev_afk is now known as aarefiev | 07:35 | |
| *** jamesmcarthur has joined #openstack-infra | 07:37 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Add internal link for job naming https://review.openstack.org/489854 | 07:37 |
| openstackgerrit | Merged openstack-infra/project-config master: [Zun] Disable Swift for dsvm py35 job https://review.openstack.org/490165 | 07:38 |
| *** lrossetti has quit IRC | 07:38 | |
| AJaeger | thanks, yolanda | 07:38 |
| *** priteau has joined #openstack-infra | 07:39 | |
| openstackgerrit | Merged openstack-infra/project-config master: [Fuxi] Disable Swift for py35 fullstack job https://review.openstack.org/490166 | 07:39 |
| *** jpena|off is now known as jpena | 07:39 | |
| openstackgerrit | Merged openstack-infra/project-config master: tap-as-a-service: Add a coverage job https://review.openstack.org/490307 | 07:40 |
| *** robcresswell has joined #openstack-infra | 07:42 | |
| *** ym200 has joined #openstack-infra | 07:42 | |
| *** arif-ali has quit IRC | 07:43 | |
| *** egonzalez has joined #openstack-infra | 07:43 | |
| *** arif-ali has joined #openstack-infra | 07:44 | |
| *** ykarel has quit IRC | 07:45 | |
| *** dizquierdo has joined #openstack-infra | 07:45 | |
| openstackgerrit | Merged openstack-infra/project-config master: Keystone CI job using ldap domain specific driver https://review.openstack.org/483576 | 07:46 |
| openstackgerrit | Merged openstack-infra/project-config master: [magnum] make legacy swarm -nv https://review.openstack.org/487858 | 07:46 |
| openstackgerrit | Merged openstack-infra/project-config master: tripleo/oooq/cleanup: remove unused jobs in gate https://review.openstack.org/490148 | 07:46 |
| *** ym200 has quit IRC | 07:46 | |
| *** lindsayk has joined #openstack-infra | 07:50 | |
| openstackgerrit | Merged openstack-infra/project-config master: tripleo/quickstart: use zuul template for scenarios https://review.openstack.org/490163 | 07:50 |
| openstackgerrit | Merged openstack-infra/project-config master: Add Python3.5 job to masakari-* projects https://review.openstack.org/489950 | 07:50 |
| openstackgerrit | Merged openstack-infra/project-config master: Add bits to deploy TripleO with stable/pike https://review.openstack.org/490093 | 07:50 |
| openstackgerrit | Merged openstack-infra/project-config master: Add gate-watcherclient-dsvm-functional-ubuntu-xenial to check and gate pipelines https://review.openstack.org/489504 | 07:51 |
| *** michaelxin has joined #openstack-infra | 07:53 | |
| openstackgerrit | Merged openstack-infra/project-config master: Notify changes in OSC feature/osc4 branch to IRC https://review.openstack.org/489703 | 07:54 |
| openstackgerrit | Merged openstack-infra/project-config master: Add ansible-role-bindep to zuulv3.o.o https://review.openstack.org/484526 | 07:54 |
| *** larainema has joined #openstack-infra | 07:54 | |
| openstackgerrit | Alexander Chadin proposed openstack-infra/project-config master: Add test job for watcher with split tempest plugin https://review.openstack.org/490375 | 07:57 |
| *** nunchuck has quit IRC | 07:57 | |
| *** sflanigan has quit IRC | 07:59 | |
| *** ramishra has quit IRC | 08:01 | |
| *** efoley has joined #openstack-infra | 08:02 | |
| *** mwhahaha has joined #openstack-infra | 08:03 | |
| *** slaweq has joined #openstack-infra | 08:04 | |
| *** ramishra has joined #openstack-infra | 08:08 | |
| *** slaweq has quit IRC | 08:09 | |
| *** jamesmca_ has joined #openstack-infra | 08:10 | |
| *** hashar has joined #openstack-infra | 08:12 | |
| *** sileht has joined #openstack-infra | 08:12 | |
| *** Hal has joined #openstack-infra | 08:13 | |
| *** sree_ has joined #openstack-infra | 08:13 | |
| *** Hal is now known as Guest12100 | 08:13 | |
| *** jamesmcarthur has quit IRC | 08:13 | |
| *** sree_ is now known as Guest52077 | 08:14 | |
| *** esikachev has joined #openstack-infra | 08:14 | |
| *** Guest52077 has quit IRC | 08:17 | |
| *** derekh has joined #openstack-infra | 08:18 | |
| *** openstackgerrit has quit IRC | 08:18 | |
| *** dingyichen has quit IRC | 08:22 | |
| *** wolverineav has joined #openstack-infra | 08:25 | |
| *** yushb has joined #openstack-infra | 08:25 | |
| *** lucas-afk is now known as lucasagomes | 08:28 | |
| *** wolverineav has quit IRC | 08:29 | |
| *** markvoelker has quit IRC | 08:39 | |
| *** openstackgerrit has joined #openstack-infra | 08:41 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate master: Strip trailing null in env var output https://review.openstack.org/490333 | 08:41 |
| *** electrofelix has joined #openstack-infra | 08:45 | |
| *** gildub has quit IRC | 08:49 | |
| *** gildub has joined #openstack-infra | 08:50 | |
| *** apetrich has quit IRC | 08:58 | |
| *** jamesmca_ has quit IRC | 08:59 | |
| *** yushb has quit IRC | 08:59 | |
| *** ykarel has joined #openstack-infra | 09:00 | |
| *** e0ne has joined #openstack-infra | 09:01 | |
| *** isq_ has joined #openstack-infra | 09:02 | |
| *** isq has quit IRC | 09:03 | |
| *** dims has quit IRC | 09:04 | |
| *** slaweq has joined #openstack-infra | 09:05 | |
| *** yushb has joined #openstack-infra | 09:06 | |
| *** dims has joined #openstack-infra | 09:08 | |
| openstackgerrit | Alexander Chadin proposed openstack-infra/project-config master: Register watcher-tempest-plugin jobs https://review.openstack.org/490400 | 09:08 |
| openstackgerrit | Alexander Chadin proposed openstack-infra/project-config master: Register watcher-tempest-plugin jobs https://review.openstack.org/490400 | 09:10 |
| *** slaweq has quit IRC | 09:10 | |
| *** ykarel has quit IRC | 09:12 | |
| *** alexchadin has quit IRC | 09:15 | |
| *** thorst_afk has joined #openstack-infra | 09:23 | |
| *** sdague has joined #openstack-infra | 09:25 | |
| *** thorst_afk has quit IRC | 09:28 | |
| *** ym200 has joined #openstack-infra | 09:28 | |
| *** hashar has quit IRC | 09:30 | |
| *** hashar has joined #openstack-infra | 09:32 | |
| *** ym200 has quit IRC | 09:33 | |
| openstackgerrit | Ifat Afek proposed openstack-infra/project-config master: Vitrage tempests now use Mistral. https://review.openstack.org/490406 | 09:33 |
| *** s-shiono has quit IRC | 09:33 | |
| *** sambetts|afk is now known as sambetts | 09:38 | |
| *** yamamoto has joined #openstack-infra | 09:41 | |
| *** apetrich has joined #openstack-infra | 09:44 | |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 09:45 |
| *** tosky has joined #openstack-infra | 09:47 | |
| *** lrossetti has joined #openstack-infra | 09:49 | |
| *** strigazi_OFF has quit IRC | 09:49 | |
| *** strigazi_OFF has joined #openstack-infra | 09:50 | |
| *** ramishra has quit IRC | 09:50 | |
| *** ramishra has joined #openstack-infra | 09:53 | |
| *** nunchuck has joined #openstack-infra | 09:55 | |
| *** yamahata has quit IRC | 09:55 | |
| *** ramishra has quit IRC | 09:57 | |
| *** stakeda has quit IRC | 09:59 | |
| *** ramishra has joined #openstack-infra | 10:00 | |
| *** cuongnv has quit IRC | 10:02 | |
| *** slaweq has joined #openstack-infra | 10:07 | |
| *** katkapilatova has joined #openstack-infra | 10:09 | |
| *** dtantsur|afk is now known as dtantsur | 10:10 | |
| *** slaweq has quit IRC | 10:12 | |
| *** ramishra has quit IRC | 10:13 | |
| *** ramishra has joined #openstack-infra | 10:15 | |
| dtantsur | fungi: nice, thanks! now I see my correct name at https://governance.openstack.org/election/#queens-ptl-candidates and I still seem to be able to use gerrit :) | 10:17 |
| *** gildub has quit IRC | 10:20 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Optionally limit github to protected branches https://review.openstack.org/490134 | 10:21 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Remove status handling from FakeGithubConnection https://review.openstack.org/490413 | 10:21 |
| *** jascott1 has quit IRC | 10:27 | |
| *** veebers has quit IRC | 10:27 | |
| *** jascott1 has joined #openstack-infra | 10:28 | |
| *** jascott1 has quit IRC | 10:33 | |
| seanhandley | AJaeger: I don't understand your latest comment on https://review.openstack.org/#/c/489548/ | 10:35 |
| seanhandley | the line you're referring to has already been removed | 10:35 |
| seanhandley | *lines | 10:35 |
| *** wolverineav has joined #openstack-infra | 10:35 | |
| openstackgerrit | Sean Handley proposed openstack-infra/project-config master: Add Public Cloud WG project. https://review.openstack.org/489548 | 10:37 |
| seanhandley | oh wait, I see | 10:37 |
| AJaeger | seanhandley: split the change in two parts. You add a new project - so don't clean up something else. And if you do, it deserves some comments. | 10:37 |
| seanhandley | I mistakenly edited the api-wg file | 10:37 |
| seanhandley | my bad | 10:37 |
| AJaeger | ;) | 10:38 |
| *** zhurong has quit IRC | 10:38 | |
| seanhandley | Good spot :) | 10:38 |
| AJaeger | seanhandley: will review later again... | 10:38 |
| seanhandley | Thanks | 10:38 |
| *** liujiong has quit IRC | 10:38 | |
| *** markvoelker has joined #openstack-infra | 10:40 | |
| *** yushb has quit IRC | 10:42 | |
| openstackgerrit | Merged openstack-infra/yaml2ical master: Add adhoc to the README https://review.openstack.org/490072 | 10:42 |
| *** jkilpatr has quit IRC | 10:44 | |
| rakhmerov | hi, is this a known failure? http://logs.openstack.org/89/488789/1/check/gate-python-mistralclient-devstack-dsvm-ubuntu-xenial/dbb6be7/console.html | 10:54 |
| *** nicolasbock__ has quit IRC | 10:54 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Remove status handling from FakeGithubConnection https://review.openstack.org/490413 | 10:54 |
| *** Guest12100 has quit IRC | 10:54 | |
| AJaeger | rakhmerov: do you mean http://logs.openstack.org/89/488789/1/check/gate-python-mistralclient-devstack-dsvm-ubuntu-xenial/dbb6be7/console.html#_2017-08-02_06_20_05_977928 - linking to the line will help ;) | 10:55 |
| rakhmerov | yes, sorry | 10:55 |
| rakhmerov | I mean this line | 10:55 |
| rakhmerov | AJaeger: it was sent by the release bot so I wonder if that's something that's just not ready in infra etc. | 10:56 |
| AJaeger | rakhmerov: normally means that upperconstraints is broken, perhaps a network issue? | 10:56 |
| rakhmerov | I rechecked several times | 10:56 |
| rakhmerov | doesn't help | 10:56 |
| rakhmerov | AJaeger: should I try again? | 10:57 |
| rakhmerov | ooh, interesting.. I guess it's just because stable/pike branch doesn't exist yet for openstack/requirements ? | 10:58 |
| rakhmerov | at least I don't see it | 10:59 |
| AJaeger | rakhmerov: exactly that's your problem | 11:00 |
| rakhmerov | ok | 11:00 |
| rakhmerov | so just waiting till it's finally created.. | 11:00 |
| AJaeger | we only branch that one once the server projects are branched | 11:00 |
| rakhmerov | yeah, got it | 11:01 |
| AJaeger | rakhmerov: yes, patience, please ;) | 11:01 |
| rakhmerov | always get confused by that branching stuff.. | 11:01 |
| rakhmerov | AJaeger: thanks | 11:01 |
| AJaeger | no worries, it's all magic ;) | 11:02 |
| *** slaweq has joined #openstack-infra | 11:07 | |
| rakhmerov | true statement ) | 11:10 |
| openstackgerrit | Alexander Chadin proposed openstack-infra/project-config master: Register watcher-tempest-plugin jobs https://review.openstack.org/490400 | 11:11 |
| *** slaweq has quit IRC | 11:12 | |
| *** markvoelker has quit IRC | 11:13 | |
| *** jkilpatr has joined #openstack-infra | 11:17 | |
| *** esikachev has quit IRC | 11:18 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add zuul:var directive and role https://review.openstack.org/490246 | 11:21 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/publications master: Fix glossary URL, use https https://review.openstack.org/490430 | 11:27 |
| *** ym200 has joined #openstack-infra | 11:34 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/puppet-ptgbot master: Add a site index https://review.openstack.org/484798 | 11:34 |
| ttx | fungi: I pushed the patchset with the Queens URL in ^ | 11:35 |
| *** ldnunes has joined #openstack-infra | 11:38 | |
| *** ym200 has quit IRC | 11:38 | |
| *** apetrich has quit IRC | 11:41 | |
| *** lucasagomes is now known as lucas-hungry | 11:48 | |
| *** dizquierdo is now known as dizquierdo_afk | 11:55 | |
| *** eharney has joined #openstack-infra | 11:55 | |
| *** namnh has quit IRC | 12:01 | |
| *** thorst_afk has joined #openstack-infra | 12:03 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 12:04 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Allow use of the uri module from localhost https://review.openstack.org/490215 | 12:04 |
| openstackgerrit | Antonio Ojea proposed openstack-infra/project-config master: Add a p35 tempest job for networking midonet https://review.openstack.org/490447 | 12:04 |
| openstackgerrit | Antonio Ojea proposed openstack-infra/project-config master: Add a py35 tempest job for networking midonet https://review.openstack.org/490447 | 12:05 |
| *** camunoz has joined #openstack-infra | 12:06 | |
| *** david-lyle has quit IRC | 12:07 | |
| *** makowals_ has joined #openstack-infra | 12:07 | |
| *** makowals has quit IRC | 12:07 | |
| *** slaweq has joined #openstack-infra | 12:08 | |
| *** markvoelker has joined #openstack-infra | 12:10 | |
| *** blznblzn2 has quit IRC | 12:11 | |
| *** dprince has joined #openstack-infra | 12:12 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add some lookup plugin tests https://review.openstack.org/454396 | 12:12 |
| *** slaweq has quit IRC | 12:13 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Don't ignore inexistent jobs in config https://review.openstack.org/488758 | 12:13 |
| *** tuanluong has quit IRC | 12:14 | |
| *** makowals_ has quit IRC | 12:16 | |
| *** makowals has joined #openstack-infra | 12:17 | |
| *** skelso has joined #openstack-infra | 12:18 | |
| *** sekelso has joined #openstack-infra | 12:20 | |
| *** adisky__ has quit IRC | 12:21 | |
| *** makowals_ has joined #openstack-infra | 12:22 | |
| *** skelso has quit IRC | 12:23 | |
| *** makowals has quit IRC | 12:23 | |
| *** sree has joined #openstack-infra | 12:28 | |
| *** tinwood is now known as tinwood-afk | 12:30 | |
| *** baoli has joined #openstack-infra | 12:30 | |
| *** jamesmcarthur has joined #openstack-infra | 12:31 | |
| *** markvoelker has quit IRC | 12:33 | |
| *** markvoelker has joined #openstack-infra | 12:33 | |
| *** makowals_ has quit IRC | 12:36 | |
| *** makowals has joined #openstack-infra | 12:36 | |
| *** jamesmcarthur has quit IRC | 12:36 | |
| *** dosaboy has quit IRC | 12:38 | |
| *** dosaboy has joined #openstack-infra | 12:38 | |
| *** pgadiya has quit IRC | 12:38 | |
| *** esberglu has joined #openstack-infra | 12:40 | |
| *** LindaWang has quit IRC | 12:40 | |
| *** gcb has joined #openstack-infra | 12:41 | |
| *** jpena is now known as jpena|lunch | 12:44 | |
| *** esberglu has quit IRC | 12:45 | |
| *** zhurong has joined #openstack-infra | 12:47 | |
| *** gordc has joined #openstack-infra | 12:50 | |
| *** apetrich has joined #openstack-infra | 12:52 | |
| *** Goneri has joined #openstack-infra | 12:54 | |
| *** lucas-hungry is now known as lucasagomes | 12:54 | |
| *** fmccrthy has quit IRC | 12:55 | |
| *** fmccrthy has joined #openstack-infra | 12:58 | |
| *** ykarel has joined #openstack-infra | 12:58 | |
| *** markus_z has quit IRC | 12:59 | |
| *** skelso has joined #openstack-infra | 13:01 | |
| *** sekelso has quit IRC | 13:01 | |
| mnaser | morning everyone. https://review.openstack.org/#/c/490171/ merged yesterday to fix gitweb logo size (see https://review.openstack.org/gitweb?p=openstack-infra/system-config.git;a=commitdiff;h=096d80ae718c631b5d9e17d1f317d8f03699698c for example). however, it seems that gerrit might require a restart for it to "reload" the css for gitweb | 13:05 |
| mnaser | so im just poking if that is scheduled anytime soon to fix it | 13:05 |
| *** Goneri has quit IRC | 13:05 | |
| *** yamamoto has quit IRC | 13:06 | |
| *** links has quit IRC | 13:06 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Set OVERCLOUD_HOSTS for deployed server ssh admin creation https://review.openstack.org/490468 | 13:06 |
| *** gibi is now known as gibi_chaos-monke | 13:07 | |
| *** gibi_chaos-monke is now known as gibi_chaosmonkey | 13:07 | |
| *** slaweq has joined #openstack-infra | 13:09 | |
| *** esberglu has joined #openstack-infra | 13:10 | |
| *** ym200 has joined #openstack-infra | 13:12 | |
| *** Goneri has joined #openstack-infra | 13:12 | |
| *** kgiusti has joined #openstack-infra | 13:12 | |
| *** ramishra has quit IRC | 13:13 | |
| *** psachin has quit IRC | 13:13 | |
| *** LindaWang has joined #openstack-infra | 13:14 | |
| *** slaweq has quit IRC | 13:14 | |
| *** EricGonczer_ has joined #openstack-infra | 13:14 | |
| *** ramishra has joined #openstack-infra | 13:15 | |
| *** markus_z has joined #openstack-infra | 13:16 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 13:20 |
| *** pblaho has quit IRC | 13:22 | |
| *** jcoufal has joined #openstack-infra | 13:23 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 13:23 |
| *** psachin has joined #openstack-infra | 13:24 | |
| *** mriedem has joined #openstack-infra | 13:24 | |
| *** dizquierdo_afk is now known as dizquierdo | 13:24 | |
| *** ym200 has quit IRC | 13:28 | |
| *** EricGonc_ has joined #openstack-infra | 13:29 | |
| *** EricGonczer_ has quit IRC | 13:31 | |
| *** kjackal__ has joined #openstack-infra | 13:32 | |
| *** kjackal_ has quit IRC | 13:32 | |
| openstackgerrit | Antonio Ojea proposed openstack-infra/project-config master: Add a py35 tempest job for networking midonet https://review.openstack.org/490447 | 13:32 |
| *** ihrachys has joined #openstack-infra | 13:35 | |
| *** pblaho1 has joined #openstack-infra | 13:36 | |
| *** yushb has joined #openstack-infra | 13:39 | |
| *** skelso has quit IRC | 13:40 | |
| *** skelso has joined #openstack-infra | 13:40 | |
| *** tinwood-afk is now known as tinwood | 13:41 | |
| *** pblaho1 is now known as pblaho | 13:41 | |
| *** yushb has quit IRC | 13:44 | |
| *** jpena|lunch is now known as jpena | 13:46 | |
| *** felipemonteiro has joined #openstack-infra | 13:48 | |
| *** felipemonteiro_ has joined #openstack-infra | 13:49 | |
| *** sree has quit IRC | 13:49 | |
| *** jaypipes has joined #openstack-infra | 13:49 | |
| pabelanger | so, noticed that zl08 and zl09 are not processing jobs any more. I don't believe they are connected into gearman properly | 13:51 |
| *** felipemonteiro has quit IRC | 13:53 | |
| *** LindaWang has quit IRC | 13:54 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Set OVERCLOUD_HOSTS for deployed server ssh admin creation https://review.openstack.org/490468 | 13:56 |
| *** spzala has joined #openstack-infra | 13:56 | |
| pabelanger | Ya, don't appear to be seeing the onStarted events | 13:56 |
| *** bobh has joined #openstack-infra | 13:57 | |
| *** LindaWang has joined #openstack-infra | 13:58 | |
| *** krtaylor has quit IRC | 14:02 | |
| openstackgerrit | Antonio Ojea proposed openstack-infra/project-config master: Add a py35 tempest job for networking midonet https://review.openstack.org/490447 | 14:03 |
| *** xyang1 has joined #openstack-infra | 14:04 | |
| AJaeger | pabelanger: could you review https://review.openstack.org/490174 , please? (nothing urgent) - and tell me whether we should really remove those graphs or have a suggestion for replacement. | 14:05 |
| *** pblaho has quit IRC | 14:05 | |
| *** hongbin has joined #openstack-infra | 14:06 | |
| *** yamamoto has joined #openstack-infra | 14:06 | |
| *** mrunge has quit IRC | 14:07 | |
| openstackgerrit | Attila Darazs proposed openstack-infra/tripleo-ci master: Add adarazs to tripleo-cd-admins https://review.openstack.org/490493 | 14:09 |
| *** annegentle has joined #openstack-infra | 14:09 | |
| *** slaweq has joined #openstack-infra | 14:10 | |
| *** yamamoto has quit IRC | 14:11 | |
| *** sree has joined #openstack-infra | 14:12 | |
| *** irenab has left #openstack-infra | 14:14 | |
| *** mrunge has joined #openstack-infra | 14:15 | |
| *** slaweq has quit IRC | 14:15 | |
| *** zhurong has quit IRC | 14:17 | |
| *** markus_z has quit IRC | 14:17 | |
| *** ykarel has quit IRC | 14:20 | |
| fungi | thanks ttx! reviewing in a bit | 14:21 |
| *** felipemonteiro_ has quit IRC | 14:22 | |
| fungi | pabelanger: odd that the two newest additions would be the only ones impacted. seems like more than coincidence | 14:25 |
| pabelanger | fungi: agree | 14:26 |
| *** jascott1 has joined #openstack-infra | 14:27 | |
| *** marst has joined #openstack-infra | 14:28 | |
| fungi | ttx: i've started ptgbot back up... looks like when it was started originally on the 12th it bounced back out of the server a few times, then died not log after without leaving a traceback or anything in the logs | 14:30 |
| *** markus_z has joined #openstack-infra | 14:30 | |
| *** thorst_afk has quit IRC | 14:30 | |
| fungi | if it dies again i'll try running it in the foreground so i have a better chance of catching tracebacks | 14:30 |
| *** jascott1 has quit IRC | 14:31 | |
| *** skelso has quit IRC | 14:32 | |
| *** skelso has joined #openstack-infra | 14:32 | |
| *** thorst_afk has joined #openstack-infra | 14:32 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add accessbot to #openstack-ptg https://review.openstack.org/490069 | 14:33 |
| *** apetrich has quit IRC | 14:34 | |
| *** sree has quit IRC | 14:36 | |
| *** thorst_afk has quit IRC | 14:37 | |
| *** jonesn has joined #openstack-infra | 14:40 | |
| *** jonesn has left #openstack-infra | 14:41 | |
| *** xarses_ has joined #openstack-infra | 14:41 | |
| *** thorst_afk has joined #openstack-infra | 14:42 | |
| *** esikachev has joined #openstack-infra | 14:44 | |
| *** armax has joined #openstack-infra | 14:47 | |
| *** yamamoto has joined #openstack-infra | 14:47 | |
| *** yamamoto has quit IRC | 14:47 | |
| *** jamesden_ has joined #openstack-infra | 14:48 | |
| *** slagle has quit IRC | 14:48 | |
| *** jamesdenton has quit IRC | 14:49 | |
| *** slagle has joined #openstack-infra | 14:50 | |
| *** felipemonteiro has joined #openstack-infra | 14:52 | |
| *** felipemonteiro_ has joined #openstack-infra | 14:53 | |
| *** eharney has quit IRC | 14:53 | |
| *** mat128 has joined #openstack-infra | 14:53 | |
| *** slaweq has joined #openstack-infra | 14:55 | |
| *** felipemonteiro has quit IRC | 14:56 | |
| *** rbrndt has joined #openstack-infra | 14:59 | |
| *** krtaylor has joined #openstack-infra | 14:59 | |
| *** Swami has joined #openstack-infra | 15:00 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 15:00 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add some ansible plugin tests https://review.openstack.org/454396 | 15:00 |
| *** thorst_afk has quit IRC | 15:00 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Allow and document use of the uri module from localhost https://review.openstack.org/490215 | 15:00 |
| *** thorst_afk has joined #openstack-infra | 15:01 | |
| *** mat128 has quit IRC | 15:02 | |
| ttx | thanks fungi | 15:07 |
| mtreinish | fungi, clarkb: did puppet push the cert permissions change to the logstash workers? I'm still not seeing any mqtt event from the workers | 15:08 |
| *** eharney has joined #openstack-infra | 15:09 | |
| *** priteau has quit IRC | 15:09 | |
| *** esikachev has quit IRC | 15:11 | |
| *** cshastri has quit IRC | 15:17 | |
| *** coolsvap has quit IRC | 15:22 | |
| *** krtaylor has quit IRC | 15:22 | |
| *** annegentle has quit IRC | 15:23 | |
| *** annegentle has joined #openstack-infra | 15:24 | |
| fungi | mtreinish: i suspect they need another restart so the import older paho-mqtt? | 15:25 |
| clarkb | mtreinish: ya looks like the cert permissions are applied | 15:25 |
| clarkb | fungi: I did the restart for older paho-mqtt but that was before cert fix went in. We may need to restart for that too so that perms are correct? | 15:25 |
| clarkb | mtreinish: fungi I just restarted the D process on worker02 | 15:26 |
| fungi | oh, i misunderstood what the cert fix was in that case | 15:27 |
| *** dprince has quit IRC | 15:27 | |
| clarkb | fungi: we were attempting to write out the file with a non existant user and group | 15:28 |
| fungi | thought you were talking about cert verification in newer paho-mqtt needing to be rolled back | 15:28 |
| fungi | right, so that's the one you were talking about then | 15:28 |
| fungi | it might need a restart to pick up the file? no clue really | 15:28 |
| mtreinish | clarkb: still not seeing anything :/ | 15:28 |
| clarkb | mtreinish: also on worker01 the A process crashed with http://paste.openstack.org/show/617451/ possibly another corner case here? | 15:30 |
| clarkb | and we are seeing the connection to port 8883 so I think it is trying | 15:30 |
| mtreinish | clarkb: hmm, a gearman job without a job_id? | 15:32 |
| mtreinish | which is the build_change field from the metadata | 15:33 |
| clarkb | looks like that error may have caused quite a few workers to crash | 15:33 |
| mtreinish | oh, periodic jobs... | 15:33 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for live migration failure bug 1706377 https://review.openstack.org/490565 | 15:34 |
| openstack | bug 1706377 in OpenStack Compute (nova) "(libvirt) live migration fails on source host due to "Assertion `!(bs->open_flags & BDRV_O_INACTIVE)' failed."" [Undecided,Confirmed] https://launchpad.net/bugs/1706377 | 15:34 |
| mtreinish | they don't have a triggering change, so that probably won't be in the metadata | 15:34 |
| *** hashar has quit IRC | 15:34 | |
| clarkb | mtreinish: yes nto every job has a change associated to it | 15:34 |
| *** katkapilatova has left #openstack-infra | 15:34 | |
| clarkb | periodic jobs and post jobs and release jobs aren't tied to changes | 15:35 |
| clarkb | you could change_id or "None" | 15:35 |
| ianychoi | clarkb, good morning - if you have time, would you have comments on 1) database change for some duplicated ids from openstackid & openstack-dev in translate-dev.o.o (e.g., my ID logins to ianychoi_dev but it needs to be deleted and my login needs to be linked to Zanata ianychoi id) and 2) I am not still sure but would you guide how https://review.openstack.org/#/c/487936/ can get more attention from infra team? | 15:35 |
| mtreinish | clarkb: post jobs dont? It's after merge, but there was still a triggering change | 15:35 |
| *** jamesmcarthur has joined #openstack-infra | 15:36 | |
| clarkb | mtreinish: nope, its the ref update that triggers those (same as release jobs) whcih isn't tied to a change | 15:36 |
| *** baoli has quit IRC | 15:36 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Add documentation jobs for monasca-api https://review.openstack.org/490569 | 15:38 |
| *** baoli has joined #openstack-infra | 15:38 | |
| clarkb | ianychoi: for 1) I think someone needs to write down a plan (an etherpad is fine) that describes the problem and how we want to fix it. So that everyone is on the same page about it beacuse we have gone back and forth with 3 different groups of people on this config and it has resulted in this confusing mess | 15:38 |
| fungi | mtreinish: ref updated gives you the commit, but not a change id | 15:38 |
| clarkb | ianychoi: then we can get i18n, infra, and smarcet/openstackid-dev devs to look at it and agree that the direction we are taking is the one we want to take going forward | 15:39 |
| *** skelso has quit IRC | 15:39 | |
| *** skelso has joined #openstack-infra | 15:39 | |
| *** dprince has joined #openstack-infra | 15:40 | |
| clarkb | ianychoi: for 2) I would bring that up in the infra meeting if you are able to make it and ask for volunteer root help. Even if we get reviews the big hurdle will be getting a root to deploy that server. Even if you aren't able to make the meeting can you add the topic to the meeting agenda https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting and the meeting chair can act in proxy of you? | 15:41 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-log_processor master: Handle cases without a build_change field https://review.openstack.org/490570 | 15:41 |
| mtreinish | clarkb, fungi: ^^^ I think that should fix the failure | 15:41 |
| clarkb | mtreinish: and it has to be a strict prefix right? is that why you don't include action in the last case? perhaps we want to do project, action, $id instead so we can always have at least project and action? | 15:43 |
| *** camunoz has quit IRC | 15:43 | |
| ianychoi | clarkb, thanks a lot - I will do your suggestion of 1) - writing into Etherpad and share once it is ready and 2) Yep I think eumel8 would happily attend to infra meeting (unfortunately 4 am in here for the meeting fails me to attend the meeting) => Thank u :) | 15:45 |
| fungi | mtreinish: i see a bug | 15:45 |
| mtreinish | clarkb: well I could do that, there's nothing stopping it. But my thinking would make dynamic querying a bit more difficult | 15:45 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for live migration failure bug 1706377 https://review.openstack.org/490565 | 15:45 |
| openstack | bug 1706377 in OpenStack Compute (nova) "(libvirt) live migration fails on source host due to "Assertion `!(bs->open_flags & BDRV_O_INACTIVE)' failed."" [Undecided,Confirmed] https://launchpad.net/bugs/1706377 | 15:45 |
| mtreinish | clarkb: because you would have a subscription like gearman-logstash/nova/#/action-foo | 15:45 |
| *** sambetts is now known as sambetts|afk | 15:45 | |
| mtreinish | fungi: ooh, good catch | 15:46 |
| *** ccamacho has quit IRC | 15:47 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-log_processor master: Handle cases without a build_change field https://review.openstack.org/490570 | 15:47 |
| mtreinish | fungi: ^^^ ok fixed that | 15:47 |
| clarkb | ianychoi: please include a link to that change in the meeting agenda as well (it will help people understand what is done and what needs to be done next) | 15:47 |
| *** e0ne has quit IRC | 15:47 | |
| *** yamamoto has joined #openstack-infra | 15:48 | |
| *** LindaWang has quit IRC | 15:48 | |
| *** andreas_s has quit IRC | 15:48 | |
| ianychoi | clarkb, yep will do - I totally understand it since I saw how the deployment is difficult when we experienced Zanata upgrade (although I am planning to one more but it should be easier since Java and Wildfly version is the same) | 15:49 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add some ansible plugin tests https://review.openstack.org/454396 | 15:50 |
| *** aarefiev is now known as aarefiev_afk | 15:53 | |
| *** yamamoto has quit IRC | 15:53 | |
| *** sree has joined #openstack-infra | 15:55 | |
| *** spzala has quit IRC | 15:56 | |
| *** esikachev has joined #openstack-infra | 15:58 | |
| *** jonesn has joined #openstack-infra | 15:59 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: WIP: Create openstack-python-tarball job https://review.openstack.org/490576 | 15:59 |
| jonesn | Is anyone around willing to take a look at a patch that seems to be stuck in "Needs Verified" for me? I haven't seen any of the obvious signs, like a stuck dependency. | 16:00 |
| *** wolverineav has quit IRC | 16:01 | |
| AJaeger | jonesn: which change? | 16:01 |
| jonesn | https://review.openstack.org/#/c/487956/ It's not showing up on zuul/status at all, which I've never had to deal with before. | 16:01 |
| *** wolverineav has joined #openstack-infra | 16:01 | |
| *** psachin has quit IRC | 16:01 | |
| clarkb | mtreinish: ok I've approved that change. I don't think that will address the lack of events though | 16:01 |
| clarkb | mtreinish: is older paho mqtt compatible with our mosquitto server? | 16:02 |
| AJaeger | jonesn: your problem :+1A - and later +2. The state machine does not start | 16:02 |
| jonesn | Would having a core redo +1A fix it? | 16:02 |
| AJaeger | Remove the +A and reapply. The logic is: Merge once it transitions to +A if there is at least a +2 | 16:02 |
| AJaeger | jonesn: yes | 16:02 |
| AJaeger | not +1A - +2A ! | 16:02 |
| AJaeger | A core should only approve if there is a +2. | 16:03 |
| jonesn | I had assumed the reverify would've done that. Is that not the case? | 16:03 |
| AJaeger | jonesn: no, zuul will not see a change state | 16:03 |
| AJaeger | and reverify is a nope, don't use it. | 16:03 |
| *** eeiden is now known as eeiden_ | 16:03 | |
| mtreinish | clarkb: yeah it is | 16:03 |
| jonesn | Thanks AJaeger. | 16:03 |
| *** egonzalez has quit IRC | 16:03 | |
| mtreinish | clarkb: we're using it on all the trusty servers | 16:03 |
| mtreinish | which includes all the ansible messages | 16:04 |
| *** eeiden_ is now known as eeiden | 16:04 | |
| *** LindaWang has joined #openstack-infra | 16:04 | |
| *** wolverineav has quit IRC | 16:05 | |
| clarkb | mtreinish: ok I guess next step after that fix is tcpdump then | 16:05 |
| openstackgerrit | Alex Schultz proposed openstack-infra/irc-meetings master: Update Puppet OpenStack meeting information https://review.openstack.org/489662 | 16:06 |
| openstackgerrit | Alex Schultz proposed openstack-infra/irc-meetings master: Update Puppet OpenStack meeting information https://review.openstack.org/489662 | 16:07 |
| *** david-lyle has joined #openstack-infra | 16:07 | |
| *** Apoorva has joined #openstack-infra | 16:08 | |
| *** thorst_afk has quit IRC | 16:08 | |
| *** jonesn has left #openstack-infra | 16:08 | |
| *** apetrich has joined #openstack-infra | 16:10 | |
| openstackgerrit | Merged openstack-infra/puppet-log_processor master: Handle cases without a build_change field https://review.openstack.org/490570 | 16:11 |
| *** krtaylor has joined #openstack-infra | 16:12 | |
| *** rcernin has quit IRC | 16:12 | |
| *** rlandy|ci_mtgs has quit IRC | 16:12 | |
| *** dave-mccowan has joined #openstack-infra | 16:14 | |
| *** thorst_afk has joined #openstack-infra | 16:15 | |
| fungi | infra-root: anybody happen to know a particular nodepool donor provider who gives us bandwidth utilization graphs, by any chance? | 16:17 |
| fungi | i've got another potential donor asking for estimated bandwidth utilization | 16:18 |
| mtreinish | clarkb: hmm, yeah I'm not sure. My other thought is I screwed something up in the cert config (or config handling) and it's not actually doing things properly | 16:19 |
| clarkb | fungi: I don't but I think the vast majority of bw use is internal due to our caching mirrors. git repos will be the exception as well as anything hitting eg github | 16:19 |
| mtreinish | but you're right the most recent fix probably isn't why we're not seeing any messages | 16:19 |
| clarkb | fungi: so you may be able to get a decent idea just by profiling the mirror (which cacti does) | 16:20 |
| fungi | yeah, i suppose so. i've checked rax and ovh dashboards thus far, to no avail | 16:21 |
| *** camunoz has joined #openstack-infra | 16:22 | |
| jeblair | fungi, clarkb: any chance we can get that info from infra-cloud? | 16:24 |
| *** ccamacho has joined #openstack-infra | 16:24 | |
| clarkb | jeblair: we don't have network gear access there so we'd ahve to aggregate it all ourselves on the hypervisors I think | 16:25 |
| *** jtomasek has quit IRC | 16:25 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Add documentation jobs for monasca-api https://review.openstack.org/490569 | 16:25 |
| clarkb | jeblair: might also make differentiating internal traffic from external painful. But we may be able to sort something out | 16:25 |
| fungi | jeblair: unfortunately i thought about that, but yeah as clarkb says we don't have access to the router there | 16:25 |
| *** ccamacho has left #openstack-infra | 16:25 | |
| jeblair | clarkb, fungi: we do have a lot of interface graphs for the hypervisors in cacti | 16:25 |
| jeblair | i don't know what the interfaces are though | 16:25 |
| jeblair | and i don't think any of that ended up in our documentation? https://docs.openstack.org/infra/system-config/infra-cloud.html | 16:26 |
| fungi | right, and i'm looking more for data transfer to/from the internet at the environment's border rather than between our systems | 16:26 |
| openstackgerrit | Alex Schultz proposed openstack-infra/project-config master: Update puppet jobs to 4.8 https://review.openstack.org/490582 | 16:26 |
| *** electrofelix has quit IRC | 16:26 | |
| jeblair | fungi: yeah, but is there an iface on the compute host that is only outward facing? | 16:27 |
| fungi | good question. i thought we virtualized traffic flow by using 802.1q vlans | 16:28 |
| jeblair | fungi: we have graphs for those too | 16:28 |
| fungi | so physical interface counters are going to aggregate traffic for each vlan together anyway | 16:28 |
| jeblair | i just don't see any docs for what they are | 16:28 |
| fungi | oh, we do? | 16:28 |
| fungi | maybe in that case | 16:28 |
| fungi | though i expect we're still communicating between the mirror server and the compute nodes over the internet-facing interfaces | 16:29 |
| jeblair | br-vlan2551, eth0, eth1, eth2, eth2.2551, lo, veth1, veth2, vibr0, virb0-nic | 16:29 |
| jeblair | yeah, i'm lost without a diagram | 16:30 |
| fungi | i'll jump onto one in a sec and see what interface has the default egress route | 16:30 |
| fungi | just about done poking around inap's dashboards and no network traffic graphing there either it seems | 16:30 |
| clarkb | jeblair: fungi no there is no iface that is only external facing | 16:30 |
| clarkb | we put everything on vlan whatever it is and then that gets routed internally and externally | 16:30 |
| clarkb | basically VMs have a single IP that is for talking to other VMs and the internet and its all routed over the same interface | 16:31 |
| jeblair | ok | 16:31 |
| mgagne | fungi: the region you are in (mtl01) is kind of a no man's land =) | 16:32 |
| clarkb | the networking is convoluted in infracloud because we could only get a single working interface per hypervisor | 16:32 |
| *** Goneri has quit IRC | 16:33 | |
| fungi | mgagne: heh, good to know! ;) | 16:33 |
| mgagne | fungi: so it's more or less not linked to our usual systems | 16:33 |
| fungi | mgagne: got it. i figured of all providers at least inap would have customer traffic graphs. when i worked at an isp we went to extreme lengths to tie netflow reporting per client address back into our traditional port-based bandwidth tracking so we could do things like contracted 95th-percentile billing for customers | 16:34 |
| mgagne | fungi: yes, fully agree with you. that's the case for our other regions. ;) | 16:34 |
| fungi | even for "cloud" environments which had no actual independent ports per customer | 16:34 |
| fungi | mgagne: makes more sense now ;) | 16:35 |
| *** Swami has quit IRC | 16:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move configure-mirror role into base/pre https://review.openstack.org/490586 | 16:40 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: WIP: Create local-site variables for configure-mirror role https://review.openstack.org/490587 | 16:40 |
| *** esikachev has quit IRC | 16:41 | |
| *** oomichi has quit IRC | 16:41 | |
| *** lucasagomes is now known as lucas-afk | 16:42 | |
| *** sree has quit IRC | 16:42 | |
| *** annegentle has quit IRC | 16:42 | |
| *** sree has joined #openstack-infra | 16:43 | |
| *** esikachev has joined #openstack-infra | 16:44 | |
| *** oomichi has joined #openstack-infra | 16:44 | |
| *** dizquierdo has quit IRC | 16:44 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor master: else needs to be else: https://review.openstack.org/490589 | 16:45 |
| clarkb | fungi: mtreinish ^ that is running on worker01 now | 16:45 |
| * mtreinish hangs his head in shame | 16:45 | |
| mtreinish | clarkb: we really need to get python ci running on these things | 16:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Don't request empty nodesets https://review.openstack.org/487243 | 16:46 |
| clarkb | mtreinish: well there is a separate python project we'll want to port all of this to then switch to using that does have that iirc | 16:46 |
| clarkb | mtreinish: problem is we never pushed code to it after we extracted it or switched deployment to it | 16:46 |
| clarkb | maybe I should see where that ended up and just push it till its done | 16:47 |
| *** sree has quit IRC | 16:48 | |
| *** esikachev has quit IRC | 16:48 | |
| clarkb | fungi: can you look at https://review.openstack.org/#/c/490589/ really quickly? | 16:48 |
| *** efoley has quit IRC | 16:49 | |
| clarkb | mtreinish: on the mqtt front I'm definitely not seeing anything that would indicate our python scripts are aware of an error. Nothing in logs etc | 16:49 |
| fungi | clarkb: already did | 16:49 |
| clarkb | mtreinish: and tcpdump seems to see traffic going both ways and it is indecipherable so assuming that tls is working | 16:49 |
| mtreinish | yeah, that sounds like it's working | 16:50 |
| mtreinish | clarkb: maybe I'm not looking for the right topic | 16:50 |
| mtreinish | clarkb: what is the base topic in the worker yaml config? | 16:50 |
| mtreinish | I thought it was gearman-logstash | 16:50 |
| pabelanger | clarkb: fungi: jeblair: mordred: Do we have any examples of taking yaml format from hiera and generating a yaml file with puppet? | 16:51 |
| clarkb | mtreinish: I don't see one | 16:51 |
| clarkb | mtreinish: also no username or password | 16:51 |
| pabelanger | been so long that I forget how to do it in puppet | 16:51 |
| pabelanger | hash to yaml I guess | 16:51 |
| mtreinish | clarkb: oh, that would explain why it's not working | 16:52 |
| mtreinish | it needs auth to have publish permissions | 16:52 |
| clarkb | oh wit there is the topic | 16:52 |
| clarkb | mtreinish: gearman-logstash/logstash-worker01 | 16:52 |
| clarkb | but ya no username or password | 16:52 |
| mtreinish | clarkb: I see the bug, one sec patch incoming | 16:53 |
| clarkb | pabelanger: if puppet doesn't need to be aware of the yaml couldn't you just pass it straight through? | 16:53 |
| clarkb | pabelanger: eg don't interpret it as a hash at all, just read in string, write it out | 16:53 |
| pabelanger | clarkb: ya, thats basically the step I am doing now. I think that might be better in the long run | 16:53 |
| pabelanger | just dump the file into project-config and have puppet-zuul place in on disk | 16:54 |
| clarkb | ya | 16:54 |
| clarkb | or even put it in yaml but as a literal string | 16:54 |
| *** pcaruana has quit IRC | 16:54 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Fix mqtt username and password vars in logstash worker config https://review.openstack.org/490593 | 16:54 |
| mtreinish | clarkb: ^^^ | 16:54 |
| mtreinish | it's username and password in the puppet: https://github.com/openstack-infra/system-config/blob/master/modules/openstack_project/manifests/logstash_worker.pp#L25 | 16:55 |
| *** Douhet has quit IRC | 16:56 | |
| clarkb | mtreinish: well I don't even have the username set to infra | 16:56 |
| *** spzala has joined #openstack-infra | 16:56 | |
| clarkb | oh varnames didn't match | 16:56 |
| *** Goneri has joined #openstack-infra | 16:56 | |
| *** Douhet has joined #openstack-infra | 16:57 | |
| *** dtantsur is now known as dtantsur|afk | 16:57 | |
| mtreinish | clarkb: yeah, it was a variable mismatch | 16:57 |
| *** derekh has quit IRC | 16:58 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: WIP: Create local-site variables for configure-mirror role https://review.openstack.org/490587 | 16:58 |
| fungi | pabelanger: right, sticking with clarkb's suggestion you get the full yaml protocol rather than filtered through a least-common-denominator of what limited bits hiera supports | 17:01 |
| *** spzala has quit IRC | 17:01 | |
| *** tosky has quit IRC | 17:01 | |
| *** rcernin has joined #openstack-infra | 17:02 | |
| *** armax_ has joined #openstack-infra | 17:03 | |
| *** armax has quit IRC | 17:03 | |
| *** armax_ is now known as armax | 17:03 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: WIP: Create local-site variables for configure-mirror role https://review.openstack.org/490587 | 17:04 |
| pabelanger | clarkb: fungi: jeblair: mordred: ^what about something like this for our site-variables? Then I can just update puppet-zuul to know about it | 17:04 |
| pabelanger | even zuul-executor folder | 17:05 |
| openstackgerrit | Merged openstack-infra/puppet-log_processor master: else needs to be else: https://review.openstack.org/490589 | 17:07 |
| openstackgerrit | Major Hayden proposed openstack-infra/project-config master: Enable minimum CentOS repositories https://review.openstack.org/490596 | 17:08 |
| jeblair | pabelanger: that works for me; you could do that via puppet-project-config | 17:08 |
| *** spzala has joined #openstack-infra | 17:09 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move configure-mirror role into base-test/pre https://review.openstack.org/490586 | 17:10 |
| pabelanger | wow | 17:10 |
| pabelanger | https://review.openstack.org/gitweb?p=openstack-infra/project-config.git;a=commitdiff;h=ee803be4072954b26a943461c8216d515c4bcc98 | 17:10 |
| *** baoli has quit IRC | 17:10 | |
| pabelanger | that is a large openstack logo | 17:11 |
| *** EricGonc_ has quit IRC | 17:11 | |
| *** iyamahat_ has joined #openstack-infra | 17:11 | |
| clarkb | pabelanger: ya its because it went in as unconstrainted svg, mnaser has a patch we will liekly need a gerrit restart for | 17:11 |
| * clarkb checks if that merged | 17:11 | |
| *** EricGonczer_ has joined #openstack-infra | 17:11 | |
| clarkb | ya it did so we should plan to get a gerrit restart in sometime soon | 17:12 |
| jeblair | pabelanger: openstack is a *big deal* | 17:12 |
| pabelanger | ha | 17:12 |
| fungi | logos are kind of a big deal in openstack? | 17:13 |
| *** Swami has joined #openstack-infra | 17:13 | |
| pabelanger | jeblair: ya, puppet-project-config make sense too | 17:13 |
| pabelanger | doing that now | 17:13 |
| clarkb | (we used to use a png with a fixed size, so when adding unconstrained svg it went big as possible there | 17:13 |
| *** markus_z has quit IRC | 17:17 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-project_config master: Create $zuul_executor_site_variables_yaml variable https://review.openstack.org/490602 | 17:18 |
| pabelanger | bikeshed ready^ | 17:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move configure-mirror role into base-test/pre https://review.openstack.org/490586 | 17:20 |
| jeblair | pabelanger: why zuul-executor directory instead of zuul? | 17:20 |
| pabelanger | jeblair: at the moment we don't copy layout_dir to zuul-executor | 17:20 |
| pabelanger | so, we could | 17:21 |
| pabelanger | but wasnt' sure we wanted to do that | 17:21 |
| mnaser | aha yes the logo was fixed, just pending a restart.. or so we think. :p | 17:21 |
| pabelanger | actually, it would be zuulv3.o.o that would also get the site-variables.yaml file | 17:21 |
| pabelanger | jeblair: are we okay with adding site-variables onto zuulv3.o.o, and not being used? | 17:22 |
| jeblair | pabelanger: i am | 17:22 |
| pabelanger | okay | 17:22 |
| pabelanger | changing | 17:22 |
| jeblair | pabelanger: though i don't think /etc/zuul/layout makes sense as a destination | 17:23 |
| jeblair | pabelanger: the zuul-executor approach may be the best thing for now | 17:23 |
| pabelanger | jeblair: okay | 17:24 |
| *** jpena is now known as jpena|off | 17:25 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move configure-mirror role into base-test/pre https://review.openstack.org/490586 | 17:25 |
| *** yamahata has joined #openstack-infra | 17:26 | |
| *** vhosakot has joined #openstack-infra | 17:27 | |
| clarkb | mtreinish: worker01 where I manually applied the syntax error fix is happy so far. I am optimistic that both these changes togehter will fix everything | 17:28 |
| *** jamesmcarthur has quit IRC | 17:28 | |
| *** manjeets has joined #openstack-infra | 17:29 | |
| *** baoli has joined #openstack-infra | 17:29 | |
| *** jamesmcarthur has joined #openstack-infra | 17:29 | |
| EmilienM | pabelanger: any idea when I can start using buildlogs mirror? | 17:31 |
| openstackgerrit | Merged openstack-infra/system-config master: Fix mqtt username and password vars in logstash worker config https://review.openstack.org/490593 | 17:31 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 17:32 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Allow and document use of the uri module from localhost https://review.openstack.org/490215 | 17:32 |
| *** jamesmcarthur has quit IRC | 17:34 | |
| pabelanger | EmilienM: should be live now | 17:34 |
| mtreinish | clarkb: I'm still not seeing logstash events on firehose.o.o | 17:38 |
| *** thorre has quit IRC | 17:39 | |
| clarkb | mtreinish: ya puppet hasnt applied the username and password fix | 17:39 |
| mtreinish | ah, ok. I thought you meant you applied that manually | 17:40 |
| mtreinish | I just misread it | 17:40 |
| *** thorre has joined #openstack-infra | 17:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move configure-mirror role into base-test/pre https://review.openstack.org/490586 | 17:42 |
| clarkb | mtreinish: I applied the else: fix manually not the other one, sorry for confusion | 17:43 |
| *** rkukura has joined #openstack-infra | 17:43 | |
| EmilienM | pabelanger: ok thx | 17:44 |
| EmilienM | pabelanger: testing it now | 17:44 |
| pabelanger | EmilienM: sure, which review. I can monitor apache logs to make sure | 17:45 |
| EmilienM | pabelanger: https://review.openstack.org/#/c/490206 | 17:46 |
| pabelanger | EmilienM: will all scenarios hit that logic? | 17:47 |
| pabelanger | EmilienM: going to monitor 002, that is on internap right now | 17:48 |
| *** tesseract has quit IRC | 17:49 | |
| *** tosky has joined #openstack-infra | 17:50 | |
| EmilienM | pabelanger: all yes | 17:52 |
| EmilienM | pabelanger: except ubuntu ;-) | 17:52 |
| pabelanger | EmilienM: Oh, you need to source /etc/ci/mirror_info.sh | 17:52 |
| pabelanger | looks like you are still using /etc/nodepool/provider | 17:52 |
| EmilienM | I never did that before | 17:53 |
| EmilienM | and? not good? | 17:53 |
| pabelanger | EmilienM: ya, /etc/ci/mirror_info.sh is a new file that contains all the new variables | 17:53 |
| EmilienM | ok cool | 17:53 |
| EmilienM | let me change that | 17:53 |
| *** gouthamr has joined #openstack-infra | 17:53 | |
| pabelanger | EmilienM: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/scripts/configure_mirror.sh#n84 | 17:53 |
| pabelanger | so you don't have the build them yourself now | 17:53 |
| pabelanger | just source, and they are setup | 17:54 |
| *** SumitNaiksatam has joined #openstack-infra | 17:54 | |
| *** martinkopec has quit IRC | 17:54 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move configure-mirror role into base-test/pre https://review.openstack.org/490586 | 17:57 |
| *** baoli has quit IRC | 17:58 | |
| *** annegentle has joined #openstack-infra | 17:58 | |
| clarkb | waiting for ansible to run puppet is like watching paint dry | 17:58 |
| openstackgerrit | Lin Yang proposed openstack-infra/project-config master: Change all tests of python-rsdclient and rsd-lib to voting https://review.openstack.org/490614 | 17:59 |
| inc0 | hey, do we have proxy for quay.io? | 17:59 |
| clarkb | inc0: no | 17:59 |
| inc0 | I'm looking at kolla-k8s job and pulling containers from quay+gcr.io took 40min | 18:00 |
| clarkb | not unless their images are hosted in the same s3 bucket thing as dockerhub | 18:00 |
| inc0 | right, probably not | 18:00 |
| inc0 | any chances we could get that up? | 18:00 |
| clarkb | inc0: you can propose it same as anyone else :) | 18:01 |
| inc0 | http://logs.openstack.org/99/489299/5/check/gate-kolla-kubernetes-deploy-centos-source-4-microchart-ansible-nv/e80d338/console.html#_2017-08-02_02_39_03_747096 look at times there | 18:01 |
| inc0 | right, sure I can, just point me where | 18:01 |
| clarkb | inc0: https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/mirror.vhost.erb to configure the proxy and https://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/scripts/configure_mirror.sh to add it to the mirror_info.sh file | 18:02 |
| clarkb | inc0: you'll want to hit it with curl or similar to see where it fetches from as the dockerhub one isn't obvious and I'm guessing quay does similar | 18:03 |
| *** sekelso has joined #openstack-infra | 18:03 | |
| *** skelso has quit IRC | 18:04 | |
| *** spzala has quit IRC | 18:05 | |
| *** baoli has joined #openstack-infra | 18:07 | |
| *** ralonsoh has quit IRC | 18:12 | |
| mtreinish | clarkb: there is a pretty constant stream of logstash events now | 18:12 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul master: Add $site_variables_yaml_file to zuul::executor https://review.openstack.org/490617 | 18:12 |
| mtreinish | it looks like everything is working | 18:13 |
| clarkb | mtreinish: cool I was just noticing that the config updated | 18:13 |
| clarkb | mtreinish: I'm going to make sure all the daemones are running again but we should be in the clear now | 18:13 |
| mtreinish | clarkb: you can really see the spike on the stats page too: http://grafana.openstack.org/dashboard/db/mosquitto-status | 18:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add site-variable yaml file to ze01.o.o https://review.openstack.org/490618 | 18:18 |
| clarkb | mtreinish: nice | 18:18 |
| *** dabukalam has quit IRC | 18:18 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Support longer pkcs1-oaep secrets https://review.openstack.org/490620 | 18:22 |
| *** slaweq_ has joined #openstack-infra | 18:25 | |
| *** slaweq has quit IRC | 18:27 | |
| pabelanger | infra-root: project-config-core: when you have time, please review https://review.openstack.org/#/q/topic:zuulv3-site-variables it plumbs thought a new zuul-executor site-variable.yaml file to be used on our zuul-executors | 18:29 |
| *** mat128 has joined #openstack-infra | 18:33 | |
| clarkb | pabelanger: I think the depends on on https://review.openstack.org/#/c/490586/ goes in the wrong direction but also don't think it will matter much in practice | 18:34 |
| clarkb | pabelanger: we shouldn't expose the new site-variables file via puppet-project_config until it is in project-config | 18:35 |
| openstackgerrit | Numan Siddique proposed openstack-infra/project-config master: Add TripleO scenario007-container for OVN https://review.openstack.org/490622 | 18:35 |
| pabelanger | clarkb: ya, I can flip that | 18:36 |
| clarkb | pabelanger: also why do we care what the mirror host's domain is if we also have the fqdn? | 18:36 |
| pabelanger | clarkb: that's how configure-mirror role is setup today. It is possible we want to refactor that a bit | 18:38 |
| pabelanger | mirror_domain is currently used by pydistutils.cfg for all_hosts | 18:39 |
| pabelanger | moving forward, I think I'm going to split configure_mirror into more specific roles | 18:39 |
| pabelanger | eg: configure_apt, configure_rpm_mirror, configure_pypi | 18:40 |
| pabelanger | since it is possible other zuulv3 users don't care about everything we mirror | 18:40 |
| clarkb | pabelanger: I left comments on that change but +2'd I don't think those are major issues but may be worth udpating if you have to push new patchset for some reason | 18:40 |
| *** jamesmcarthur has joined #openstack-infra | 18:41 | |
| clarkb | pabelanger: reading that I think you can rewrite pydistutils.cfg to have allow_hosts = mirror fqdn | 18:43 |
| clarkb | pabelanger: since it shouldn't talk to any other node | 18:43 |
| clarkb | pabelanger: then we can drop the redundant domain entirely | 18:43 |
| mordred | clarkb: ah - good call | 18:43 |
| pabelanger | okay | 18:43 |
| pabelanger | let me see if I can stack that change | 18:44 |
| *** slaweq_ has quit IRC | 18:47 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Remove mirror_domain from configure-mirror https://review.openstack.org/490625 | 18:50 |
| pabelanger | mordred: clarkb: ^like that for pydistutils? | 18:50 |
| pabelanger | I'll do the fqdn rename in the next patch | 18:50 |
| *** prateek has quit IRC | 18:51 | |
| *** jamesmcarthur has quit IRC | 18:52 | |
| clarkb | pabelanger: yup +2 | 18:52 |
| *** jamesmcarthur has joined #openstack-infra | 18:53 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Rename mirror_host to mirror_fqdn in configure-mirrors https://review.openstack.org/490628 | 18:55 |
| pabelanger | and FQDN change | 18:55 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove getProjectBranches from FakeGithubConnection https://review.openstack.org/490043 | 18:56 |
| *** masber has joined #openstack-infra | 18:57 | |
| EmilienM | pabelanger: buildlogs mirror seems to work fine! | 18:58 |
| EmilienM | pabelanger: doing tripleo now, I'll let you know :) | 18:58 |
| pabelanger | EmilienM: great | 18:59 |
| *** masuberu has quit IRC | 18:59 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move configure-mirror role into base-test/pre https://review.openstack.org/490586 | 19:03 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-project_config master: Create $zuul_executor_site_variables_yaml variable https://review.openstack.org/490602 | 19:03 |
| *** rhallisey has quit IRC | 19:10 | |
| *** Sukhdev has joined #openstack-infra | 19:13 | |
| *** jcoufal has quit IRC | 19:15 | |
| *** slaweq has joined #openstack-infra | 19:17 | |
| *** jianghuaw has quit IRC | 19:17 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Remove mirror_domain from configure-mirror https://review.openstack.org/490625 | 19:19 |
| jeblair | this may briefly break the unittest jobs ^ | 19:19 |
| *** sekelso has quit IRC | 19:19 | |
| *** sekelso has joined #openstack-infra | 19:20 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Rename mirror_host to mirror_fqdn in configure-mirrors https://review.openstack.org/490628 | 19:20 |
| openstackgerrit | Swaminathan Vasudevan proposed openstack/diskimage-builder master: Failed to open parameter YAML error while trying to unmount imagedir https://review.openstack.org/490637 | 19:20 |
| jeblair | pabelanger: what's the story with the directory name in 490602? the patch content doesn't match the commit message. | 19:21 |
| *** EricGonczer_ has quit IRC | 19:21 | |
| *** EricGonczer_ has joined #openstack-infra | 19:22 | |
| *** eharney has quit IRC | 19:22 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Optionally limit github to protected branches https://review.openstack.org/490134 | 19:22 |
| pabelanger | jeblair: Oh, I must have incorrectly pushed up something | 19:22 |
| pabelanger | 1 se | 19:22 |
| jeblair | pabelanger: i think your content is correct but commit message is wrong, based on other patches i'm looking at | 19:22 |
| pabelanger | so, do we want https://review.openstack.org/#/c/490586/7/zuul-executor/site-variables.yaml | 19:23 |
| pabelanger | or zuul/site-variables.yaml | 19:23 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Remove status handling from FakeGithubConnection https://review.openstack.org/490413 | 19:23 |
| pabelanger | if zuul/site-varaibles.yaml, it will be installed into zuulv3.o.o too | 19:23 |
| jeblair | pabelanger: i still think that's fine. | 19:23 |
| pabelanger | okay | 19:23 |
| pabelanger | let me fix | 19:23 |
| jeblair | pabelanger: the only thing i don't want is for it to be installed on the executors in /etc/zuul/layout/site-vars.yaml. but it looks like that won't be the case. | 19:24 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-project_config master: Create $zuul_executor_site_variables_yaml variable https://review.openstack.org/490602 | 19:24 |
| pabelanger | jeblair: right | 19:24 |
| jeblair | pabelanger: (it will be installed on zuul.o.o in /etc/zuul/layout/.... but that's fine, i don't care) | 19:24 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move configure-mirror role into base-test/pre https://review.openstack.org/490586 | 19:25 |
| *** jianghuaw has joined #openstack-infra | 19:25 | |
| jeblair | mordred, clarkb: 490586 and 490602 are ready for re-review | 19:25 |
| *** spzala has joined #openstack-infra | 19:27 | |
| *** spzala has quit IRC | 19:27 | |
| *** felipemonteiro_ has quit IRC | 19:27 | |
| *** spzala has joined #openstack-infra | 19:27 | |
| *** felipemonteiro_ has joined #openstack-infra | 19:27 | |
| *** baoli has quit IRC | 19:27 | |
| *** spzala has quit IRC | 19:27 | |
| *** baoli has joined #openstack-infra | 19:28 | |
| *** skelso has joined #openstack-infra | 19:29 | |
| *** gyee has joined #openstack-infra | 19:29 | |
| *** mat128 has quit IRC | 19:30 | |
| *** felipemonteiro_ has quit IRC | 19:30 | |
| *** felipemonteiro_ has joined #openstack-infra | 19:30 | |
| *** sekelso has quit IRC | 19:31 | |
| *** spzala has joined #openstack-infra | 19:35 | |
| *** Sukhdev has quit IRC | 19:36 | |
| *** esberglu has quit IRC | 19:38 | |
| *** ijw has joined #openstack-infra | 19:38 | |
| *** jamesmcarthur has quit IRC | 19:42 | |
| *** jamesmcarthur has joined #openstack-infra | 19:42 | |
| *** felipemonteiro_ has quit IRC | 19:47 | |
| *** esberglu has joined #openstack-infra | 19:53 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Doc improvements on user guide https://review.openstack.org/490640 | 19:56 |
| *** harlowja has quit IRC | 19:57 | |
| *** annegentle has quit IRC | 19:57 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul master: Add $site_variables_yaml_file to zuul::executor https://review.openstack.org/490617 | 19:58 |
| *** puiterwijk is now known as basset | 19:58 | |
| *** jianghuaw has quit IRC | 19:58 | |
| *** basset is now known as puiterwijk | 19:58 | |
| *** puiterwijk is now known as basset | 19:58 | |
| *** basset is now known as puiterwijk | 19:59 | |
| *** bobh has quit IRC | 19:59 | |
| *** annegentle has joined #openstack-infra | 20:01 | |
| *** eharney has joined #openstack-infra | 20:02 | |
| *** jianghuaw has joined #openstack-infra | 20:04 | |
| *** camunoz has quit IRC | 20:10 | |
| *** slaweq has quit IRC | 20:18 | |
| *** slaweq has joined #openstack-infra | 20:18 | |
| *** slaweq has quit IRC | 20:20 | |
| *** slaweq has joined #openstack-infra | 20:20 | |
| *** baoli has quit IRC | 20:22 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Doc improvements on user guide https://review.openstack.org/490640 | 20:22 |
| *** mriedem has quit IRC | 20:25 | |
| *** hamzy has quit IRC | 20:37 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Create fetch-tox-output role https://review.openstack.org/490643 | 20:38 |
| *** kgiusti has left #openstack-infra | 20:39 | |
| *** Sukhdev_ has joined #openstack-infra | 20:42 | |
| *** veebers has joined #openstack-infra | 20:42 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove status handling from FakeGithubConnection https://review.openstack.org/490413 | 20:44 |
| *** mriedem has joined #openstack-infra | 20:44 | |
| *** felipemonteiro has joined #openstack-infra | 20:47 | |
| *** jkilpatr has quit IRC | 20:48 | |
| *** slaweq has quit IRC | 20:48 | |
| *** slaweq has joined #openstack-infra | 20:49 | |
| *** rhallisey has joined #openstack-infra | 20:51 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Update OpenStack-Helm gates https://review.openstack.org/490647 | 20:52 |
| *** felipemonteiro_ has joined #openstack-infra | 20:52 | |
| *** slaweq has quit IRC | 20:53 | |
| *** bobh has joined #openstack-infra | 20:55 | |
| *** SumitNaiksatam has quit IRC | 20:56 | |
| *** skelso has quit IRC | 20:56 | |
| *** felipemonteiro has quit IRC | 20:56 | |
| *** Apoorva_ has joined #openstack-infra | 21:00 | |
| *** krtaylor has quit IRC | 21:02 | |
| openstackgerrit | David Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids https://review.openstack.org/489013 | 21:02 |
| *** Sukhdev_ has quit IRC | 21:03 | |
| *** Apoorva has quit IRC | 21:04 | |
| *** ijw has quit IRC | 21:04 | |
| *** skelso has joined #openstack-infra | 21:05 | |
| *** rtjure has joined #openstack-infra | 21:05 | |
| *** ijw has joined #openstack-infra | 21:05 | |
| *** slaweq has joined #openstack-infra | 21:10 | |
| *** thorst_afk has quit IRC | 21:10 | |
| *** Sukhdev_ has joined #openstack-infra | 21:12 | |
| *** thorst_afk has joined #openstack-infra | 21:12 | |
| *** slaweq has quit IRC | 21:14 | |
| *** harlowja has joined #openstack-infra | 21:15 | |
| *** ldnunes has quit IRC | 21:15 | |
| *** Apoorva_ has quit IRC | 21:16 | |
| *** skelso has quit IRC | 21:16 | |
| *** Apoorva has joined #openstack-infra | 21:16 | |
| *** skelso has joined #openstack-infra | 21:17 | |
| *** thorst_afk has quit IRC | 21:17 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: add initial documentation style guide https://review.openstack.org/490654 | 21:20 |
| *** efoley has joined #openstack-infra | 21:20 | |
| openstackgerrit | Julia Kreger proposed openstack/diskimage-builder master: Add netbase to ensure /etc/protocols is placed for debian https://review.openstack.org/490656 | 21:20 |
| *** yamamoto has joined #openstack-infra | 21:21 | |
| openstackgerrit | David Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids https://review.openstack.org/489013 | 21:22 |
| openstackgerrit | David Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids https://review.openstack.org/489013 | 21:24 |
| openstackgerrit | David Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids https://review.openstack.org/489013 | 21:26 |
| *** yamamoto has quit IRC | 21:26 | |
| openstackgerrit | David Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids https://review.openstack.org/489013 | 21:27 |
| *** slaweq has joined #openstack-infra | 21:28 | |
| *** thorst_afk has joined #openstack-infra | 21:30 | |
| *** jamesmcarthur has quit IRC | 21:31 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 21:32 |
| *** thorst_afk has quit IRC | 21:34 | |
| clarkb | jeblair: pabelanger reviewed, I think https://review.openstack.org/#/c/490586/8 just needs a second reviewer then that whole stack should go into gating | 21:37 |
| *** gyee has quit IRC | 21:38 | |
| jeblair | done | 21:40 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Don't do pre-pingtest cleanup with --skip-pingtest-cleanup https://review.openstack.org/490661 | 21:40 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 21:42 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add some ansible plugin tests https://review.openstack.org/454396 | 21:42 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Allow and document use of the uri module from localhost https://review.openstack.org/490215 | 21:42 |
| *** annegentle has quit IRC | 21:45 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: convert some component config to zuul directives https://review.openstack.org/490662 | 21:48 |
| openstackgerrit | Merged openstack-infra/project-config master: Move configure-mirror role into base-test/pre https://review.openstack.org/490586 | 21:49 |
| *** rhallisey has quit IRC | 21:50 | |
| *** thorst_afk has joined #openstack-infra | 21:50 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 21:51 |
| *** dprince has quit IRC | 21:54 | |
| *** thorst_afk has quit IRC | 21:55 | |
| *** yamamoto has joined #openstack-infra | 21:58 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add some ansible plugin tests https://review.openstack.org/454396 | 22:01 |
| *** andreww has joined #openstack-infra | 22:02 | |
| *** bobh has quit IRC | 22:02 | |
| *** iyamahat_ has quit IRC | 22:02 | |
| *** iyamahat has joined #openstack-infra | 22:02 | |
| *** xarses_ has quit IRC | 22:03 | |
| *** gordc has quit IRC | 22:05 | |
| *** esberglu has quit IRC | 22:06 | |
| *** esberglu has joined #openstack-infra | 22:06 | |
| *** Apoorva_ has joined #openstack-infra | 22:07 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: reformat merger and executor config docs https://review.openstack.org/490665 | 22:09 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Include retired projects/deliverables in elections https://review.openstack.org/488543 | 22:09 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Use foundation member IDs in invitations list https://review.openstack.org/488555 | 22:09 |
| *** annegentle has joined #openstack-infra | 22:09 | |
| *** esberglu has quit IRC | 22:10 | |
| *** Apoorva has quit IRC | 22:11 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: reformat web section https://review.openstack.org/490666 | 22:12 |
| *** spzala has quit IRC | 22:13 | |
| *** annegentle has quit IRC | 22:15 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Ensure ref-updated jobs run with their ref https://review.openstack.org/488216 | 22:16 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix null ref in github tests https://review.openstack.org/490257 | 22:17 |
| *** krtaylor has joined #openstack-infra | 22:17 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add some ansible plugin tests https://review.openstack.org/454396 | 22:17 |
| fungi | clarkb: does the last revision on 483974 adequately address your concerns from patchset 2? | 22:17 |
| *** jascott1 has joined #openstack-infra | 22:21 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add project related type hints to gerritconnection https://review.openstack.org/488593 | 22:21 |
| clarkb | fungi: yup thanks | 22:21 |
| clarkb | mtreinish: logstash indexing "feels" slow. | 22:21 |
| fungi | i kept the refactoring to a parent patch for added clarity | 22:21 |
| clarkb | mtreinish: I notice that we we will record a mqtt event for every log file even if for aborted jobs or if the log file doesn't exist. Do you think we could avoid sending events for those (just to cut down on total io?) | 22:22 |
| *** krtaylor has quit IRC | 22:23 | |
| clarkb | mtreinish: also I don't think we want to send an event for every log line do we? | 22:24 |
| fungi | oh, right, it tries to grab all sorts of files even though it doesn't know whether they'll exist | 22:24 |
| clarkb | I have a hunch all this additional io (basically doubled io on system) is slowing throughput | 22:24 |
| fungi | probably makes sense to only emit an event for actual logs it retrieves | 22:24 |
| *** xyang1 has quit IRC | 22:25 | |
| clarkb | ya | 22:25 |
| *** jascott1 has quit IRC | 22:25 | |
| clarkb | and not for each log line | 22:25 |
| *** jascott1 has joined #openstack-infra | 22:25 | |
| clarkb | I'm working on a patch so we can look at it with concrete data | 22:25 |
| *** Douhet has quit IRC | 22:25 | |
| *** Douhet has joined #openstack-infra | 22:26 | |
| *** krtaylor has joined #openstack-infra | 22:26 | |
| fungi | oh, is it doing every log line? i thought it had been rewritten to do one event per file | 22:26 |
| jeblair | yes please. i think actual retreieved files for completed jobs is the max amount of detail that makes sense. | 22:27 |
| fungi | emitting an event for every line of a thousands-of-lines-long file is not very useful | 22:27 |
| jeblair | fungi: there are indeed better protocols for that | 22:28 |
| jeblair | we support them | 22:28 |
| *** skelso has quit IRC | 22:28 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor master: Only send mqtt events for processed files https://review.openstack.org/490668 | 22:29 |
| fungi | i thought we had reverted this implementation once before because it doubled the network bandwidth consumption of the logstash workers | 22:29 |
| clarkb | there we go | 22:29 |
| fungi | which is why i expected it wasn't per-line any longer | 22:30 |
| jeblair | if it's worth reverting once, it's worth reverting again | 22:30 |
| clarkb | fungi: the previous one we reverted had the mqtt events in logstash itself but that plugin wasn't able to keep up and was crashing logstash | 22:30 |
| clarkb | in any case there is a patch that I think does what we just atlked about | 22:30 |
| *** krtaylor has quit IRC | 22:31 | |
| clarkb | of course that has no testing so maybe I will apply it lcoally to worker01 really qucik to make sure it doesn't break :) | 22:31 |
| *** Apoorva_ has quit IRC | 22:31 | |
| fungi | heh, wow... | 22:31 |
| fungi | mtreinish: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=4952&rra_id=all | 22:31 |
| *** Apoorva has joined #openstack-infra | 22:31 | |
| jeblair | and that's clearly hitting a rate limit | 22:32 |
| fungi | challenge: see if you can spot where we started sending logstash events to the service | 22:32 |
| fungi | you might have to look really hard | 22:33 |
| fungi | (to see what the traffic levels had been like before it started) | 22:33 |
| clarkb | no immediate crashes on lgostash-worker01 with that patch | 22:33 |
| clarkb | so I think its good if it passes code review | 22:33 |
| *** andreww has quit IRC | 22:34 | |
| fungi | clarkb: thanks! looking at it now | 22:34 |
| *** Sukhdev_ has quit IRC | 22:34 | |
| fungi | lgtm and passed check, so approved just now | 22:38 |
| clarkb | thanks | 22:39 |
| clarkb | I'll make sure that the services get restarted with that code update once applied | 22:39 |
| *** felipemonteiro_ has quit IRC | 22:41 | |
| *** Sukhdev_ has joined #openstack-infra | 22:43 | |
| openstackgerrit | Merged openstack-infra/puppet-log_processor master: Only send mqtt events for processed files https://review.openstack.org/490668 | 22:45 |
| *** Apoorva_ has joined #openstack-infra | 22:47 | |
| *** Apoorva has quit IRC | 22:47 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Docs: add initial documentation style guide https://review.openstack.org/490654 | 22:50 |
| *** dave-mccowan has quit IRC | 22:51 | |
| *** slaweq has quit IRC | 22:51 | |
| *** slaweq has joined #openstack-infra | 22:51 | |
| *** EricGonczer_ has quit IRC | 22:54 | |
| *** aeng has joined #openstack-infra | 22:54 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: update format in connections.rst https://review.openstack.org/490672 | 22:55 |
| *** slaweq has quit IRC | 22:56 | |
| *** sdague has quit IRC | 22:56 | |
| *** pbourke has quit IRC | 22:57 | |
| *** rcernin has quit IRC | 22:58 | |
| *** pbourke has joined #openstack-infra | 22:59 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Docs: convert some component config to zuul directives https://review.openstack.org/490662 | 23:00 |
| *** ijw has quit IRC | 23:04 | |
| *** ijw has joined #openstack-infra | 23:05 | |
| *** ihrachys_ has joined #openstack-infra | 23:06 | |
| *** ihrachys has quit IRC | 23:06 | |
| *** tosky has quit IRC | 23:07 | |
| *** ijw has quit IRC | 23:09 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add some ansible plugin tests https://review.openstack.org/454396 | 23:11 |
| openstackgerrit | Merged openstack-infra/shade master: Add support for get details of available QoS rule type https://review.openstack.org/489751 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 23:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Allow and document use of the uri module from localhost https://review.openstack.org/490215 | 23:12 |
| *** spzala has joined #openstack-infra | 23:13 | |
| *** ijw has joined #openstack-infra | 23:14 | |
| *** ijw has quit IRC | 23:15 | |
| *** spzala has quit IRC | 23:17 | |
| *** Goneri has quit IRC | 23:20 | |
| *** mriedem1 has joined #openstack-infra | 23:23 | |
| *** EricGonczer_ has joined #openstack-infra | 23:24 | |
| openstackgerrit | Merged openstack-infra/shade master: Use valid_kwargs decorator in QoS related functions https://review.openstack.org/489759 | 23:25 |
| *** mriedem has quit IRC | 23:25 | |
| *** ijw has joined #openstack-infra | 23:25 | |
| *** thorst_afk has joined #openstack-infra | 23:26 | |
| *** Goneri has joined #openstack-infra | 23:27 | |
| *** baoli has joined #openstack-infra | 23:27 | |
| *** tosky has joined #openstack-infra | 23:31 | |
| *** baoli has quit IRC | 23:32 | |
| *** thorst_afk has quit IRC | 23:32 | |
| clarkb | ok I just restarted all of the worker daemons to pick up that code change | 23:32 |
| clarkb | we should see io fall off | 23:33 |
| *** Goneri has quit IRC | 23:33 | |
| *** hamzy has joined #openstack-infra | 23:33 | |
| *** hongbin has quit IRC | 23:34 | |
| yamamoto | is there any recommendation for py3 job names? there seems we have many variants. python3, python35, py3, py3x, etc | 23:34 |
| clarkb | yamamoto: python3X is what we have been using for unittest type jobs to line up with eg python26 and python27. I think those have been around for quite a while and following that format likely makes sense. Unless you don't care what version of python3 you test with and maybe just call that python3? | 23:36 |
| clarkb | jeblair: AJaeger ^ does the test naming guide have an opinion? | 23:36 |
| *** rbrndt has quit IRC | 23:37 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: reformat gerrit driver docs https://review.openstack.org/490681 | 23:37 |
| jeblair | clarkb, yamamoto: i would call it python35 for now, and expect the name to change (possibly automatically) to py35 when we switch to zuulv3. | 23:38 |
| *** tosky has quit IRC | 23:38 | |
| jeblair | it's better not to switch now though, as that will may make the zuulv3 migration more difficult | 23:38 |
| jeblair | s/will may/may/ :) | 23:39 |
| yamamoto | i'm considering how to name a new job. | 23:39 |
| yamamoto | clarkb: it's a tempsest job with DEVSTACK_GATE_USE_PYTHON3=True | 23:41 |
| clarkb | yamamoto: the existing python3 tempest jobs that use devstack-gate use "py35" | 23:42 |
| clarkb | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=4952&rra_id=all shows we have indeed fallen back to pre change graph heights | 23:43 |
| clarkb | mnaser: I think I will try to sneak a gerrit restart in tomorrow | 23:44 |
| clarkb | mnaser: now that it looks like logstash is happy again | 23:44 |
| mnaser | clarkb no worries, probably better than possibly having other things break tonight :P | 23:44 |
| *** bobh has joined #openstack-infra | 23:44 | |
| ianw | we've got a few 1tb unattached volumes --> graphite.openstack.org/main01 nb01.openstack.org/main01 nb02.openstack.org/main01 | 23:45 |
| ianw | given the price it quotes me / month that's non-trivial, should we remove them? | 23:46 |
| clarkb | pabelanger: ^ those are for builders taht don't exist anymore right | 23:46 |
| clarkb | ianw: I think its safe to remove them assuming pabelanger confirms ^ | 23:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: update format in connections.rst https://review.openstack.org/490672 | 23:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: reformat web section https://review.openstack.org/490666 | 23:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: reformat merger and executor config docs https://review.openstack.org/490665 | 23:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: reformat gerrit driver docs https://review.openstack.org/490681 | 23:47 |
| ianw | yeah, they were the trusty builders. nb01 still responds, but maybe isn't puppeted because i can't log in? | 23:47 |
| ianw | hmm, yeah puppet is also unhappy about a changed rsa key for nb01 | 23:47 |
| pabelanger | clarkb: ianw: ya, we rotated to nb03 / nb04. Likely was never cleaned up properly after they were moved into production | 23:48 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove state_dir requirement from merger https://review.openstack.org/490682 | 23:48 |
| ianw | pabelanger: ok, i'm in the rax volume stuff now, so i'll remove those | 23:49 |
| jeblair | ianw: do we have two graphite main01 volumes? | 23:49 |
| ianw | jeblair: we seem to have graphite/main01 & graphite/main02 ... only 02 is attached | 23:50 |
| *** ym200 has joined #openstack-infra | 23:50 | |
| jeblair | ianw: hrm. graphite.o.o has two 1tb physical volumes active in lvm. | 23:50 |
| yamamoto | clarkb: thank you | 23:50 |
| jeblair | ianw: i recommend not deleting the graphite one at the moment. :) | 23:51 |
| ianw | jeblair: interesting ... the status of 01 is "Status: Error Deleting" | 23:51 |
| *** stakeda has joined #openstack-infra | 23:51 | |
| *** marst has quit IRC | 23:51 | |
| jeblair | ianw: i'm unsure how to reconcile this... :/ | 23:52 |
| *** rkukura has quit IRC | 23:52 | |
| ianw | there is also a 50gb "eavesdrop.openstack.org/main01" that is unattached | 23:52 |
| ianw | or so it says | 23:52 |
| jeblair | we have one 2tb lv on graphite which spans both of those pv | 23:53 |
| ianw | /dev/mapper/main-graphite 2.0T 987G 1.1T | 23:54 |
| ianw | i wonder if this will fail in 13gb or so | 23:54 |
| ianw | also ... [5666553.022979] Buffer I/O error on device dm-0, logical block 0 | 23:54 |
| jeblair | it's currently mounted read-write, though there are many errors for xvdb in dmesg | 23:54 |
| *** ym200 has quit IRC | 23:55 | |
| ianw | i get the feeling it's not there and graphite just hasn't noticed :) | 23:55 |
| jeblair | is there a reliable way to determine which device is which? | 23:55 |
| jeblair | is which volume i mean | 23:55 |
| clarkb | jeblair: yes | 23:56 |
| clarkb | jeblair: with kvm you look in /dev/disk/by-uuid or something for xen you chcek the mount point as reported back to you by nova | 23:56 |
| jeblair | ianw: what does nova say ^? | 23:56 |
| ianw | the volume id is 4b5d9f10-0427-4bfb-b45b-a4d682ac4ba5 ... | 23:57 |
| clarkb | ianw: it should give you an attach point if you show the volume iirc | 23:57 |
| clarkb | this is accurate on xen but not on kvm | 23:57 |
| ianw | clarkb: well it's unattached | 23:57 |
| jeblair | ianw: how about the other one? | 23:57 |
| jeblair | main02 | 23:58 |
| ianw | yes, that has the attachment recorded as /dev/xvdc | 23:58 |
| jeblair | okay, then it's all in the same order. xvdb/main01 is the first half of the lv, and xvdc/main02 is the second half | 23:59 |
| jeblair | (so says pvdisplay -m) | 23:59 |
| ianw | which matches the errors from xvdb in the logs | 23:59 |
| jeblair | right, but we're generally pretty sucessfully using xvdb. | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
