| *** bhunter71 has quit IRC | 00:00 | |
| *** dannywil_ has quit IRC | 00:02 | |
| *** markvoelker has quit IRC | 00:02 | |
| *** unicell has quit IRC | 00:03 | |
| anteaya | yeah, I know all that | 00:03 |
|---|---|---|
| anteaya | still not comfortable | 00:03 |
| *** unicell has joined #openstack-infra | 00:03 | |
| *** unicell has quit IRC | 00:03 | |
| *** unicell has joined #openstack-infra | 00:03 | |
| bookwar | anteaya: sorry for that :) | 00:03 |
| *** bknudson has joined #openstack-infra | 00:04 | |
| *** annegentle has joined #openstack-infra | 00:06 | |
| *** mmedvede has quit IRC | 00:06 | |
| *** yamamoto has joined #openstack-infra | 00:06 | |
| *** dangers is now known as dangers_away | 00:09 | |
| *** rmcall has quit IRC | 00:12 | |
| *** sdake_ has joined #openstack-infra | 00:14 | |
| *** bhuvan has quit IRC | 00:14 | |
| *** Longgeek has quit IRC | 00:15 | |
| *** sdake has quit IRC | 00:17 | |
| openstackgerrit | Brant Knudson proposed openstack/requirements: Update keystoneclient requirement https://review.openstack.org/168187 | 00:18 |
| *** sputnik13 has quit IRC | 00:20 | |
| *** dboik has joined #openstack-infra | 00:21 | |
| *** gokrokve has quit IRC | 00:24 | |
| *** mtanino has quit IRC | 00:24 | |
| *** davide_ has quit IRC | 00:26 | |
| *** reed has quit IRC | 00:29 | |
| *** reed has joined #openstack-infra | 00:29 | |
| *** reed has quit IRC | 00:30 | |
| *** reed has joined #openstack-infra | 00:31 | |
| *** ddieterly has quit IRC | 00:31 | |
| *** yamamoto has quit IRC | 00:34 | |
| mordred | bookwar: looks sane to me | 00:35 |
| ianw | so, pyOpenSSL rewrote itself in python ... which may be better for many reasons, but memory usage is not one of them | 00:35 |
| *** otter768 has joined #openstack-infra | 00:37 | |
| morganfainberg | ianw: :( | 00:38 |
| *** baoli has joined #openstack-infra | 00:41 | |
| *** hodos has quit IRC | 00:42 | |
| *** otter768 has quit IRC | 00:43 | |
| *** baoli has quit IRC | 00:48 | |
| derekh | ianw: that grubby patch looks like its causing the problems with fedora images on the tripleo cloud | 00:48 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 00:48 | |
| ianw | derekh: my workaround you mean? | 00:49 |
| *** tiswanso has joined #openstack-infra | 00:49 | |
| derekh | ianw: yup, if I take a clean F20 instance, install that grubby and the yum update. it wont boot and console shows the same error | 00:50 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack-infra/project-config: Specify Nova API versions clearly https://review.openstack.org/168193 | 00:50 |
| ianw | derekh: so can you pastebin the /boot/extlinux.conf before & after? | 00:51 |
| *** ddieterly has joined #openstack-infra | 00:51 | |
| derekh | ianw: will do, give me a few minutes and I'll redo it, also I noticed a grubby core dump http://nodepool.openstack.org/tripleo-test-cloud-rh1.tripleo-f20.log | 00:52 |
| derekh | ianw: search for "invalid pointer", not sure if its related or not | 00:52 |
| ianw | derekh: urgh; it probably is because i'm grabbing it from rawhide | 00:53 |
| derekh | ianw: yup, possibly | 00:53 |
| ianw | derekh: so i have a series of patches in-flight to remove f20 nodes alltogether | 00:53 |
| ianw | the solution here is probably to get rid of the f20 nodes in main testing and get rid of the work-around, instead of layering on more hacks | 00:53 |
| derekh | ianw: are your serries of patches to run the tripleo tests on f21 ? or are you talking about other tests? | 00:55 |
| *** SumitNaiksatam has quit IRC | 00:57 | |
| *** amitgandhinz has joined #openstack-infra | 00:57 | |
| ianw | derekh: no, just removing the devstack testing nodes https://review.openstack.org/#/c/167443/ | 00:58 |
| *** tqtran_ has quit IRC | 00:58 | |
| ianw | derekh: or as a quick work-around, we can add a "is_rackspace" test to that existing call and then it won't trigger? | 01:00 |
| derekh | ianw: ok, so that will still leave us using F20 for the tripleo tests, although we could probably move that at the same time | 01:00 |
| ianw | derekh: right, but that's not using the rax / hpcloud images is it? | 01:01 |
| ianw | that being tripleo | 01:01 |
| derekh | ianw: I'm all for a quick workaround, our test have been down all week and want to get something running again, once we have it working I'll look into moving to F21 also | 01:01 |
| derekh | ianw: correct its on the tripleo cloud | 01:01 |
| *** esker has quit IRC | 01:02 | |
| *** tiswanso has quit IRC | 01:03 | |
| *** esker has joined #openstack-infra | 01:03 | |
| *** tiswanso has joined #openstack-infra | 01:03 | |
| *** Longgeek has joined #openstack-infra | 01:03 | |
| ianw | derekh: right, so let me do a quick "is_rackspace" check thing, and when we've moved out f20 nodes we can revert the whole lot | 01:04 |
| derekh | ianw: ok, cool | 01:04 |
| *** resker has joined #openstack-infra | 01:04 | |
| *** dtantsur|afk has quit IRC | 01:05 | |
| *** samueldmq has quit IRC | 01:07 | |
| *** esker has quit IRC | 01:08 | |
| *** dkliban is now known as dkliban_afk | 01:08 | |
| *** marcusvrn has joined #openstack-infra | 01:09 | |
| *** dtantsur has joined #openstack-infra | 01:10 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Apply F20 gruby work-around only on RAX https://review.openstack.org/168196 | 01:10 |
| ianw | derekh: ^ | 01:10 |
| *** salv-orlando has quit IRC | 01:10 | |
| derekh | ianw: befor yum update http://paste.openstack.org/show/196966/ and after http://paste.openstack.org/show/196967/ | 01:11 |
| derekh | ianw: looking | 01:11 |
| ianw | derekh: not sure there's anyone around to approve it, so you might want to keep up with it tomorrow | 01:11 |
| *** crinkle has quit IRC | 01:11 | |
| ianw | derekh: yeah, in the "after" shot the 3.18.9 kernel doesn't have an initrd line | 01:12 |
| *** taron has quit IRC | 01:13 | |
| derekh | ianw: ahh yes, | 01:13 |
| ianw | my guess is that's the problem ... if you want to dig into grubby good luck :) | 01:13 |
| *** annegentle has quit IRC | 01:14 | |
| derekh | ianw: ok bed time for me, I'll leave digging into grubby for another day ;-) | 01:14 |
| *** ujuc has joined #openstack-infra | 01:14 | |
| ianw | derekh: cool ... sorry i broke it | 01:15 |
| *** esker has joined #openstack-infra | 01:15 | |
| *** crinkle has joined #openstack-infra | 01:15 | |
| *** taron has joined #openstack-infra | 01:15 | |
| derekh | ianw: no probs, its a pain of a things to verify befor hand | 01:16 |
| *** crc32 has quit IRC | 01:16 | |
| derekh | fungi: ^^ that patch from ianw should fix the fedora image builds on rh1 if you have a chance to look at it | 01:16 |
| *** annegentle has joined #openstack-infra | 01:17 | |
| derekh | night all | 01:17 |
| *** resker has quit IRC | 01:17 | |
| *** derekh has quit IRC | 01:17 | |
| *** yamamoto has joined #openstack-infra | 01:17 | |
| *** resker has joined #openstack-infra | 01:19 | |
| *** asettle has quit IRC | 01:20 | |
| *** esker has quit IRC | 01:21 | |
| *** dims_ has quit IRC | 01:23 | |
| *** ghostpl_ has joined #openstack-infra | 01:25 | |
| *** crinkle has quit IRC | 01:25 | |
| *** ChuckC has joined #openstack-infra | 01:26 | |
| *** taron has quit IRC | 01:26 | |
| *** ghostpl_ has quit IRC | 01:32 | |
| *** chris6131 has joined #openstack-infra | 01:34 | |
| *** SumitNaiksatam has joined #openstack-infra | 01:35 | |
| *** ChuckC has quit IRC | 01:36 | |
| *** Somay has quit IRC | 01:39 | |
| *** baoli has joined #openstack-infra | 01:43 | |
| *** dkranz has joined #openstack-infra | 01:47 | |
| *** annegentle has quit IRC | 01:48 | |
| *** melwitt has quit IRC | 01:48 | |
| *** yamahata has quit IRC | 01:49 | |
| *** amotoki has quit IRC | 01:50 | |
| *** Longgeek has quit IRC | 01:52 | |
| *** Longgeek has joined #openstack-infra | 01:53 | |
| *** dank_ has quit IRC | 01:55 | |
| *** otter768 has joined #openstack-infra | 01:56 | |
| *** bhunter71 has joined #openstack-infra | 02:05 | |
| *** sarob has quit IRC | 02:07 | |
| *** rmcall has joined #openstack-infra | 02:08 | |
| *** salv-orlando has joined #openstack-infra | 02:08 | |
| *** rmcall_ has joined #openstack-infra | 02:11 | |
| *** rmcall has quit IRC | 02:12 | |
| *** rmcall_ is now known as rmcall | 02:12 | |
| *** asettle has joined #openstack-infra | 02:14 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 02:14 | |
| *** oomichi_ has quit IRC | 02:17 | |
| *** jerryz has joined #openstack-infra | 02:19 | |
| *** sarob has joined #openstack-infra | 02:20 | |
| *** sarob has quit IRC | 02:20 | |
| *** sarob has joined #openstack-infra | 02:20 | |
| *** kaisers has joined #openstack-infra | 02:22 | |
| *** rmcall has quit IRC | 02:22 | |
| *** kaisers1 has quit IRC | 02:23 | |
| *** patrickeast has quit IRC | 02:25 | |
| *** taron has joined #openstack-infra | 02:25 | |
| *** achanda has quit IRC | 02:27 | |
| *** ghostpl_ has joined #openstack-infra | 02:28 | |
| *** tsg has quit IRC | 02:29 | |
| *** crinkle has joined #openstack-infra | 02:29 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525 | 02:33 |
| *** ghostpl_ has quit IRC | 02:35 | |
| *** Somay has joined #openstack-infra | 02:42 | |
| *** pal_ has joined #openstack-infra | 02:43 | |
| *** salv-orlando has quit IRC | 02:45 | |
| *** annegentle has joined #openstack-infra | 02:48 | |
| *** Somay has quit IRC | 02:49 | |
| *** gyee has quit IRC | 02:50 | |
| *** annegentle has quit IRC | 02:54 | |
| *** devlaps has quit IRC | 02:54 | |
| *** unicell1 has joined #openstack-infra | 02:58 | |
| *** unicell has quit IRC | 02:58 | |
| *** sdake has joined #openstack-infra | 03:00 | |
| *** amitgandhinz has quit IRC | 03:02 | |
| *** sdake_ has quit IRC | 03:04 | |
| *** tiswanso has quit IRC | 03:04 | |
| *** asettle has quit IRC | 03:09 | |
| *** unicell1 has quit IRC | 03:10 | |
| *** asettle has joined #openstack-infra | 03:10 | |
| *** asettle has quit IRC | 03:10 | |
| *** Alexandra_ has joined #openstack-infra | 03:10 | |
| *** Alexandra_ has quit IRC | 03:10 | |
| *** asettle has joined #openstack-infra | 03:11 | |
| *** asettle has quit IRC | 03:11 | |
| *** asettle has joined #openstack-infra | 03:11 | |
| *** asettle has quit IRC | 03:11 | |
| *** Alexandra_ has joined #openstack-infra | 03:12 | |
| *** Alexandra_ has quit IRC | 03:13 | |
| *** asettle has joined #openstack-infra | 03:13 | |
| *** achanda has joined #openstack-infra | 03:13 | |
| *** emagana has joined #openstack-infra | 03:18 | |
| *** liusheng has joined #openstack-infra | 03:21 | |
| *** spzala has quit IRC | 03:26 | |
| *** tsg has joined #openstack-infra | 03:26 | |
| *** bhunter71 has quit IRC | 03:26 | |
| *** ajmiller has joined #openstack-infra | 03:28 | |
| openstackgerrit | Steve Kowalik proposed openstack-infra/puppet-zanata: Add a zanata::client manifest https://review.openstack.org/167081 | 03:33 |
| *** sputnik13 has joined #openstack-infra | 03:33 | |
| *** Longgeek has quit IRC | 03:34 | |
| *** sputnik13 has quit IRC | 03:35 | |
| *** sputnik13 has joined #openstack-infra | 03:38 | |
| *** sputnik13 has quit IRC | 03:38 | |
| *** ddieterly has quit IRC | 03:39 | |
| *** pal__ has joined #openstack-infra | 03:44 | |
| *** pal_ has quit IRC | 03:45 | |
| *** salv-orlando has joined #openstack-infra | 03:46 | |
| *** baoli has quit IRC | 03:49 | |
| *** annegentle has joined #openstack-infra | 03:50 | |
| *** emagana has quit IRC | 03:51 | |
| *** resker has quit IRC | 03:52 | |
| *** emagana has joined #openstack-infra | 03:52 | |
| *** baoli has joined #openstack-infra | 03:52 | |
| *** salv-orlando has quit IRC | 03:52 | |
| *** pal__ has quit IRC | 03:55 | |
| *** annegentle has quit IRC | 03:55 | |
| *** emagana has quit IRC | 03:56 | |
| *** mitchell has joined #openstack-infra | 03:58 | |
| *** sputnik13 has joined #openstack-infra | 03:58 | |
| *** sdake has quit IRC | 03:59 | |
| *** asettle has quit IRC | 04:01 | |
| *** sdake has joined #openstack-infra | 04:02 | |
| *** baoli has quit IRC | 04:02 | |
| StevenK | raise TypeError, "dist must be a Distribution instance" | 04:02 |
| StevenK | TypeError: dist must be a Distribution instance | 04:02 |
| StevenK | Sigh | 04:02 |
| lifeless | StevenK: context? | 04:03 |
| mordred | StevenK: that usually means that someone tried to easy_install or setup.py install something and it screwed up setuptools | 04:03 |
| *** marcusvrn has quit IRC | 04:03 | |
| mordred | StevenK: WAY too vague of an answer from me - but 95% of the time that's what it means | 04:03 |
| StevenK | lifeless: Running sbuild against python-cliff | 04:03 |
| lifeless | StevenK: make sure you have pbr as a build-depends | 04:03 |
| mordred | also - don't forget to set the right env vars if it's important | 04:04 |
| StevenK | steven@undermined:~% grep -c pbr python-cliff_1.9.0-0ubuntu2_amd64.build | 04:04 |
| StevenK | 23 | 04:04 |
| StevenK | Clearly, doing something with pbr | 04:04 |
| mordred | StevenK: http://docs.openstack.org/developer/pbr/packagers.html | 04:04 |
| mordred | fwiw, if you hasn't seen that doc | 04:04 |
| mordred | StevenK: does sbuild do python setup.py install behind the scenes? | 04:05 |
| * mordred no know what all sbuild does | 04:05 | |
| StevenK | mordred: sbuild installs build depends and then runs dpkg-buildpackage | 04:05 |
| mordred | kk | 04:05 |
| mordred | well, it's possible that the packaging for cliff does not turn off pip processing | 04:06 |
| StevenK | I didn't actually package this, I've just taken the packaging from Vivid | 04:06 |
| mordred | nod | 04:06 |
| StevenK | Maybe the version of pbr in Trusty has fossilized or something | 04:06 |
| mordred | shouldn't matter | 04:06 |
| mordred | the interface should be the same | 04:06 |
| mordred | StevenK: I've been meaning to do a debian package for pbr project | 04:06 |
| mordred | so that I can point to a "Do this" | 04:07 |
| mordred | but I have not done that yet | 04:07 |
| StevenK | mordred: I'd be delighted to help you out with it | 04:07 |
| mordred | StevenK: cool! | 04:07 |
| *** unicell has joined #openstack-infra | 04:07 | |
| mordred | StevenK: because, honestly, there are two possible outcomes: | 04:07 |
| mordred | StevenK: a) a repo that shows there is a very straightforward way to do it right | 04:07 |
| mordred | StevenK: b) a bug in pbr that makes a) harder than it should be | 04:08 |
| *** stevemar has joined #openstack-infra | 04:08 | |
| *** Longgeek has joined #openstack-infra | 04:08 | |
| *** annegentle has joined #openstack-infra | 04:08 | |
| lifeless | its already packaged.... | 04:08 |
| lifeless | rmadison python-pbr | 04:08 |
| lifeless | python-pbr | 0.7.0-0ubuntu2 | trusty | source, all | 04:08 |
| lifeless | python-pbr | 0.10.0-0ubuntu1 | utopic | source, all | 04:08 |
| lifeless | python-pbr | 0.10.0-0ubuntu1 | vivid | source, all | 04:08 |
| clarkb | ya the version on pbr is pretty broken too iirc | 04:09 |
| clarkb | 0.7? | 04:09 |
| clarkb | er on trusty | 04:09 |
| *** sdake has quit IRC | 04:10 | |
| clarkb | we have to work around that in devstack | 04:11 |
| StevenK | Hmmm | 04:12 |
| *** amitgandhinz has joined #openstack-infra | 04:13 | |
| StevenK | This build seems to be downloading stevedore from pypi ... | 04:13 |
| *** annegentle has quit IRC | 04:13 | |
| *** amitgandhinz has quit IRC | 04:17 | |
| *** ghostpl_ has joined #openstack-infra | 04:19 | |
| *** _ianychoi_ is now known as ianychoi | 04:22 | |
| mordred | StevenK: that means the packaging is wrong | 04:23 |
| mordred | StevenK: export SKIP_PIP_INSTALL=1 | 04:24 |
| StevenK | It uses dh-python, it should be sensible | 04:24 |
| mordred | StevenK: export SKIP_PIP_INSTALL=1 | 04:26 |
| mordred | StevenK: see my b) above | 04:26 |
| mordred | if it's downloading stevedore from pypi, it means there is a bug in the version of pbr being used | 04:27 |
| *** gokrokve has joined #openstack-infra | 04:27 | |
| *** ghostpl_ has quit IRC | 04:27 | |
| mordred | StevenK: it's possible that our handling of how we're operating in a dh_python context is wrong | 04:27 |
| mordred | but up until now nobody has been willing to give feedback or file bugs | 04:27 |
| mordred | other than "pbr sucks" | 04:27 |
| mordred | which I ignore | 04:27 |
| mordred | StevenK: I agree, just using dh_python should be all you need to do | 04:28 |
| *** gokrokve has quit IRC | 04:32 | |
| *** stevemar2 has joined #openstack-infra | 04:32 | |
| *** gokrokve has joined #openstack-infra | 04:32 | |
| *** stevemar has quit IRC | 04:35 | |
| *** sarob has quit IRC | 04:36 | |
| *** gokrokve has quit IRC | 04:39 | |
| *** sputnik13 has quit IRC | 04:39 | |
| *** sputnik13 has joined #openstack-infra | 04:39 | |
| *** markvoelker has joined #openstack-infra | 04:39 | |
| *** chris6132 has joined #openstack-infra | 04:39 | |
| *** unicell has quit IRC | 04:39 | |
| *** ddieterly has joined #openstack-infra | 04:39 | |
| *** unicell has joined #openstack-infra | 04:40 | |
| *** chris613 has quit IRC | 04:41 | |
| *** markvoelker has quit IRC | 04:43 | |
| *** ddieterly has quit IRC | 04:44 | |
| *** sputnik13 has quit IRC | 04:44 | |
| *** hdd has joined #openstack-infra | 04:45 | |
| *** sputnik13 has joined #openstack-infra | 04:45 | |
| greghaynes | jeblair: Added to my review queue, will definitely take me a bit more than a few mins to review ;) | 04:48 |
| *** sarob has joined #openstack-infra | 04:48 | |
| *** YorikSar has quit IRC | 04:48 | |
| *** ujuc has quit IRC | 04:48 | |
| *** sputnik13 has quit IRC | 04:49 | |
| *** esker has joined #openstack-infra | 04:53 | |
| *** stevemar2 is now known as stevemar | 04:53 | |
| *** sputnik13 has joined #openstack-infra | 04:56 | |
| *** esker has quit IRC | 04:57 | |
| *** achanda has quit IRC | 05:00 | |
| *** reed has quit IRC | 05:03 | |
| *** ajmiller has quit IRC | 05:05 | |
| openstackgerrit | Clark Laughlin proposed openstack-infra/puppet-jenkins: Changes installation of jdk to use the architecture of the host https://review.openstack.org/168222 | 05:08 |
| *** zz_dimtruck is now known as dimtruck | 05:09 | |
| *** sputnik13 has quit IRC | 05:12 | |
| *** pal_ has joined #openstack-infra | 05:16 | |
| *** hdd has quit IRC | 05:16 | |
| *** annegentle has joined #openstack-infra | 05:19 | |
| *** annegentle has quit IRC | 05:24 | |
| *** tsg has quit IRC | 05:30 | |
| *** prmtl has joined #openstack-infra | 05:31 | |
| *** Somay has joined #openstack-infra | 05:36 | |
| *** achanda has joined #openstack-infra | 05:37 | |
| *** markvoelker has joined #openstack-infra | 05:39 | |
| *** ddieterly has joined #openstack-infra | 05:41 | |
| *** jamielennox is now known as jamielennox|away | 05:41 | |
| *** markvoelker has quit IRC | 05:43 | |
| *** salv-orlando has joined #openstack-infra | 05:44 | |
| *** ddieterly has quit IRC | 05:45 | |
| *** chris6131 has quit IRC | 05:46 | |
| *** dims has joined #openstack-infra | 05:46 | |
| *** salv-orlando has quit IRC | 05:47 | |
| *** sputnik13 has joined #openstack-infra | 05:53 | |
| *** esker has joined #openstack-infra | 05:53 | |
| *** YorikSar has joined #openstack-infra | 05:55 | |
| *** sputnik13 has quit IRC | 05:57 | |
| *** esker has quit IRC | 05:58 | |
| *** sputnik13 has joined #openstack-infra | 06:02 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/167889 | 06:03 |
| *** sarob has quit IRC | 06:07 | |
| *** ghostpl_ has joined #openstack-infra | 06:11 | |
| *** armax has quit IRC | 06:13 | |
| *** ghostpl_ has quit IRC | 06:16 | |
| openstackgerrit | Andrew Boik proposed openstack-infra/devstack-gate: WIP: (experiment) Test dual-stack with dependency https://review.openstack.org/168063 | 06:19 |
| *** BharatK has joined #openstack-infra | 06:19 | |
| *** woodster_ has quit IRC | 06:20 | |
| *** annegentle has joined #openstack-infra | 06:20 | |
| *** dims has quit IRC | 06:21 | |
| *** Somay has quit IRC | 06:24 | |
| *** annegentle has quit IRC | 06:25 | |
| *** afazekas is now known as __afazekas | 06:28 | |
| *** achanda has quit IRC | 06:32 | |
| *** salv-orlando has joined #openstack-infra | 06:33 | |
| *** salv-orlando has quit IRC | 06:37 | |
| *** bmwiedemann has left #openstack-infra | 06:39 | |
| *** markvoelker has joined #openstack-infra | 06:40 | |
| *** ddieterly has joined #openstack-infra | 06:41 | |
| *** tnovacik has joined #openstack-infra | 06:42 | |
| *** markvoelker has quit IRC | 06:44 | |
| openstackgerrit | Clark Laughlin proposed openstack-infra/puppet-jenkins: Changes installation of jdk to use the architecture of the host https://review.openstack.org/168222 | 06:44 |
| *** AJaeger has joined #openstack-infra | 06:45 | |
| *** ddieterly has quit IRC | 06:46 | |
| *** yamamoto has quit IRC | 06:48 | |
| *** mmedvede has joined #openstack-infra | 06:49 | |
| *** Longgeek has quit IRC | 06:50 | |
| *** Longgeek has joined #openstack-infra | 06:53 | |
| *** dimtruck is now known as zz_dimtruck | 06:53 | |
| *** esker has joined #openstack-infra | 06:55 | |
| *** bradjones has quit IRC | 06:55 | |
| *** bradjones has joined #openstack-infra | 06:56 | |
| *** bradjones has quit IRC | 06:57 | |
| *** bradjones has joined #openstack-infra | 06:57 | |
| *** ajo has quit IRC | 06:58 | |
| *** esker has quit IRC | 06:59 | |
| *** sarob has joined #openstack-infra | 07:07 | |
| *** scheuran has joined #openstack-infra | 07:08 | |
| *** dhritishikhar_ has joined #openstack-infra | 07:12 | |
| *** sarob has quit IRC | 07:12 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/167889 | 07:12 |
| *** afazekas has joined #openstack-infra | 07:13 | |
| *** chlong has quit IRC | 07:16 | |
| openstackgerrit | Sebastian Kalinowski proposed openstack-infra/project-config: Add CI jobs for fuel-tasklib https://review.openstack.org/167539 | 07:17 |
| *** HeOS has quit IRC | 07:19 | |
| *** ujuc has joined #openstack-infra | 07:31 | |
| openstackgerrit | Sebastian Kalinowski proposed openstack-infra/project-config: Add CI jobs for fuel-tasklib https://review.openstack.org/167539 | 07:32 |
| *** otter768 has quit IRC | 07:37 | |
| *** markvoelker has joined #openstack-infra | 07:40 | |
| *** dhritishikhar_ has quit IRC | 07:41 | |
| *** ghostpl_ has joined #openstack-infra | 07:42 | |
| *** ddieterly has joined #openstack-infra | 07:42 | |
| *** markvoelker has quit IRC | 07:45 | |
| *** ghostpl_ has quit IRC | 07:46 | |
| *** ddieterly has quit IRC | 07:47 | |
| *** MaxV has joined #openstack-infra | 07:48 | |
| *** MaxV has joined #openstack-infra | 07:48 | |
| *** ildikov has quit IRC | 07:51 | |
| *** esker has joined #openstack-infra | 07:55 | |
| *** stevemar has quit IRC | 07:59 | |
| *** esker has quit IRC | 08:00 | |
| *** ujuc has quit IRC | 08:02 | |
| *** dulek has joined #openstack-infra | 08:04 | |
| *** mpavone has joined #openstack-infra | 08:06 | |
| *** unicell has quit IRC | 08:07 | |
| *** SumitNaiksatam has quit IRC | 08:08 | |
| *** sarob has joined #openstack-infra | 08:08 | |
| *** markus_z has joined #openstack-infra | 08:09 | |
| *** unicell has joined #openstack-infra | 08:11 | |
| *** Ala has joined #openstack-infra | 08:11 | |
| *** tnovacik has quit IRC | 08:14 | |
| *** salv-orlando has joined #openstack-infra | 08:15 | |
| *** sarob has quit IRC | 08:16 | |
| *** arxcruz has joined #openstack-infra | 08:17 | |
| *** salv-orlando has quit IRC | 08:20 | |
| prmtl | hi all | 08:21 |
| prmtl | is project renaming already scheduled? | 08:21 |
| prmtl | or it still waits for some projects? | 08:22 |
| *** annegentle has joined #openstack-infra | 08:22 | |
| AJaeger | prmtl: scheduled for later today... | 08:22 |
| AJaeger | prmtl: why do you ask? | 08:23 |
| prmtl | ah, thiss a great news | 08:23 |
| prmtl | I wasn't sure from the last infra meeting logs about the date | 08:23 |
| *** MaxV has quit IRC | 08:23 | |
| *** MaxV has joined #openstack-infra | 08:24 | |
| *** btran has quit IRC | 08:24 | |
| AJaeger | prmtl: http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html | 08:24 |
| *** unicell1 has joined #openstack-infra | 08:25 | |
| prmtl | apparently I missed that e-mail, sorry | 08:25 |
| AJaeger | prmtl: no problem | 08:26 |
| *** ajo has joined #openstack-infra | 08:27 | |
| *** unicell has quit IRC | 08:27 | |
| *** annegentle has quit IRC | 08:27 | |
| *** MaxV has quit IRC | 08:28 | |
| *** tkelsey has joined #openstack-infra | 08:30 | |
| *** marun has joined #openstack-infra | 08:31 | |
| *** ujuc has joined #openstack-infra | 08:31 | |
| *** doude has joined #openstack-infra | 08:31 | |
| *** JoshNang has quit IRC | 08:32 | |
| openstackgerrit | Sebastian Kalinowski proposed openstack-infra/project-config: Add CI jobs for fuel-tasklib https://review.openstack.org/167539 | 08:32 |
| *** ihrachyshka has joined #openstack-infra | 08:32 | |
| *** tkelsey has quit IRC | 08:34 | |
| *** marun has quit IRC | 08:35 | |
| *** enikanorov has joined #openstack-infra | 08:36 | |
| *** jlanoux has joined #openstack-infra | 08:37 | |
| *** jistr has joined #openstack-infra | 08:40 | |
| *** markvoelker has joined #openstack-infra | 08:41 | |
| *** ominakov has joined #openstack-infra | 08:42 | |
| *** ghostpl_ has joined #openstack-infra | 08:42 | |
| *** ddieterly has joined #openstack-infra | 08:43 | |
| *** andreykurilin_ has joined #openstack-infra | 08:43 | |
| *** ihrachyshka has quit IRC | 08:45 | |
| *** jerryz_ has joined #openstack-infra | 08:45 | |
| *** markvoelker has quit IRC | 08:46 | |
| *** ddieterly has quit IRC | 08:48 | |
| *** tkelsey has joined #openstack-infra | 08:48 | |
| *** HeOS has joined #openstack-infra | 08:48 | |
| *** jerryz has quit IRC | 08:48 | |
| *** ghostpl_ has quit IRC | 08:50 | |
| *** Hal has joined #openstack-infra | 08:52 | |
| *** Hal is now known as Guest97266 | 08:53 | |
| *** tnovacik has joined #openstack-infra | 08:53 | |
| *** MaxV has joined #openstack-infra | 08:55 | |
| *** Guest97266 has quit IRC | 08:55 | |
| openstackgerrit | Michal Dulko proposed openstack-dev/hacking: Fix string formatting in example https://review.openstack.org/168269 | 08:55 |
| *** esker has joined #openstack-infra | 08:56 | |
| *** esker has quit IRC | 09:01 | |
| *** hashar is now known as hasharCall | 09:06 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/system-config: Use "$puppetmaster_server" as parameter to the ssh class https://review.openstack.org/167646 | 09:12 |
| *** abhi_ has joined #openstack-infra | 09:15 | |
| *** ssam2 has joined #openstack-infra | 09:16 | |
| abhi_ | hi all, can anyone suggest me on how to add the vendor_name in tempest.conf using devstack's localrc file | 09:16 |
| *** ildikov has joined #openstack-infra | 09:18 | |
| *** andreykurilin_ has quit IRC | 09:19 | |
| *** andreykurilin_ has joined #openstack-infra | 09:19 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 09:23 | |
| *** ibiris_away is now known as ibiris | 09:27 | |
| *** _nadya_ has joined #openstack-infra | 09:31 | |
| *** derekh has joined #openstack-infra | 09:33 | |
| derekh | Hi, is there anybody else around who can take a look at https://review.openstack.org/#/c/168196/1 , I'd like to get the tripleo fedora images fixed as soon as possible | 09:34 |
| *** abhi_ has quit IRC | 09:36 | |
| *** otter768 has joined #openstack-infra | 09:38 | |
| *** AJaeger has quit IRC | 09:41 | |
| *** markvoelker has joined #openstack-infra | 09:42 | |
| *** otter768 has quit IRC | 09:43 | |
| *** ddieterly has joined #openstack-infra | 09:44 | |
| *** Hal has joined #openstack-infra | 09:45 | |
| *** Hal is now known as Guest25866 | 09:45 | |
| *** ghostpl_ has joined #openstack-infra | 09:46 | |
| *** markvoelker has quit IRC | 09:46 | |
| *** ddieterly has quit IRC | 09:49 | |
| *** andreykurilin_ has quit IRC | 09:49 | |
| *** pblaho has joined #openstack-infra | 09:51 | |
| *** dims_ has joined #openstack-infra | 09:51 | |
| *** rlandy has joined #openstack-infra | 09:55 | |
| *** ibiris is now known as ibiris_away | 09:56 | |
| *** ghostpl_ has quit IRC | 09:56 | |
| *** esker has joined #openstack-infra | 09:57 | |
| *** hichihar_ has quit IRC | 09:58 | |
| *** pelix has joined #openstack-infra | 10:00 | |
| *** ibiris_away is now known as ibiris | 10:00 | |
| *** esker has quit IRC | 10:02 | |
| *** hasharCall is now known as hashar | 10:03 | |
| *** pc_m has joined #openstack-infra | 10:11 | |
| *** amotoki has joined #openstack-infra | 10:13 | |
| *** AJaeger has joined #openstack-infra | 10:15 | |
| *** marcusvrn has joined #openstack-infra | 10:17 | |
| *** e0ne has joined #openstack-infra | 10:18 | |
| *** marun has joined #openstack-infra | 10:20 | |
| *** annegentle has joined #openstack-infra | 10:23 | |
| *** marun has quit IRC | 10:24 | |
| *** annegentle has quit IRC | 10:29 | |
| *** salv-orlando has joined #openstack-infra | 10:30 | |
| *** Longgeek has quit IRC | 10:31 | |
| *** salv-orlando has quit IRC | 10:34 | |
| *** marcusvrn has quit IRC | 10:34 | |
| *** markvoelker has joined #openstack-infra | 10:43 | |
| *** ddieterly has joined #openstack-infra | 10:44 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/reviewstats: Update docs team https://review.openstack.org/168290 | 10:46 |
| *** markvoelker has quit IRC | 10:48 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/project-config: Add stackforge/puppet-autossh https://review.openstack.org/167553 | 10:48 |
| *** ddieterly has quit IRC | 10:49 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/reviewstats: Update docs and security-docs teams https://review.openstack.org/168290 | 10:50 |
| tnurlygayanov | hi there | 10:52 |
| tnurlygayanov | I need help with the access to new project on stackforge: https://review.openstack.org/#/c/167813/ could you please add me to the core reviewers/admins lists for this project? | 10:53 |
| *** amotoki has quit IRC | 10:56 | |
| *** amotoki has joined #openstack-infra | 10:57 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/reviewstats: Update docs and security-docs teams https://review.openstack.org/168290 | 10:57 |
| *** esker has joined #openstack-infra | 10:58 | |
| *** esker has quit IRC | 11:02 | |
| *** dtantsur is now known as dtantsur|brb | 11:02 | |
| *** macjack has joined #openstack-infra | 11:05 | |
| *** macjack has quit IRC | 11:07 | |
| *** vlaza has joined #openstack-infra | 11:07 | |
| *** Longgeek has joined #openstack-infra | 11:08 | |
| kaisers | Hi Infra! Is anybody seeing connection problems to github? I've two ci run fails due to unreachable messages for github and saw a jenkins fail in stackforge/puppet-cinder with similiar issues this morning. I also think i had a browser hanging on something similiar. Is it only me? :) | 11:08 |
| *** dprince has joined #openstack-infra | 11:08 | |
| *** ominakov has quit IRC | 11:08 | |
| *** baoli has joined #openstack-infra | 11:11 | |
| *** ihrachyshka has joined #openstack-infra | 11:11 | |
| *** pal_ has quit IRC | 11:12 | |
| *** pal_ has joined #openstack-infra | 11:13 | |
| *** mitchell has quit IRC | 11:15 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/reviewstats: Update docs and security-docs teams https://review.openstack.org/168290 | 11:17 |
| *** erlon has joined #openstack-infra | 11:18 | |
| *** ajo has quit IRC | 11:18 | |
| *** BharatK has quit IRC | 11:19 | |
| *** marcusvrn has joined #openstack-infra | 11:20 | |
| *** baoli has quit IRC | 11:22 | |
| *** marcusvrn1 has joined #openstack-infra | 11:23 | |
| AJaeger | kaisers: we have occasional failures, please retry | 11:23 |
| AJaeger | tnurlygayanov: which project is this and what's your gerrit login? | 11:23 |
| kaisers | Github is under DDOS | 11:24 |
| *** annegentle has joined #openstack-infra | 11:24 | |
| kaisers | https://status.github.com/ | 11:24 |
| tnurlygayanov | AJaeger, the project is https://github.com/stackforge/haos , my gerrit login is 'tnurlygayanov' | 11:24 |
| *** salv-orlando has joined #openstack-infra | 11:24 | |
| tnurlygayanov | AJaeger Thank you! | 11:25 |
| *** marcusvrn has quit IRC | 11:25 | |
| AJaeger | let's ask infro roots like clarkb and fungi to add you ^. They usually scroll back and will respond once they are awake | 11:25 |
| dprince | AJaeger: morning, would you be comfortable sending this https://review.openstack.org/#/c/168196/1/install_puppet.sh,cm | 11:25 |
| AJaeger | tnurlygayanov: I can't do it myself | 11:25 |
| dprince | AJaeger: we've been down for a week and this is one of the remaining issues | 11:25 |
| tnurlygayanov | AJaeger, ok | 11:26 |
| AJaeger | dprince: sorry, I'm not looking at system-config | 11:26 |
| dprince | AJaeger: ah, right. thanks | 11:27 |
| AJaeger | dprince: looks fine to me but my +1 won't help much ;) | 11:27 |
| tnurlygayanov | clarkb, fungi, we need help with the access to new project on stackforge: https://review.openstack.org/#/c/167813/ could you please add me to the core reviewers/admins lists for this project? Project on github: https://github.com/stackforge/haos , my gerrit login is 'tnurlygayanov' | 11:27 |
| dprince | fungi: when you come online would you have a look at https://review.openstack.org/#/c/168196/1/install_puppet.sh,cm. It is one of our issues... | 11:28 |
| AJaeger | dprince: ping fungi about https://review.openstack.org/#/c/168196 once he's awake and had his coffee | 11:28 |
| AJaeger | ;) | 11:28 |
| AJaeger | kaisers: you can clone from git.openstack.org instead of github | 11:29 |
| *** tnovacik has quit IRC | 11:29 | |
| *** ddieterly has joined #openstack-infra | 11:29 | |
| *** annegentle has quit IRC | 11:29 | |
| *** ociuhandu has joined #openstack-infra | 11:31 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/project-config: Add stackforge/puppet-autossh https://review.openstack.org/167553 | 11:33 |
| *** dizquierdo has joined #openstack-infra | 11:36 | |
| *** cdent has joined #openstack-infra | 11:37 | |
| *** dkliban_afk is now known as dkliban | 11:38 | |
| *** mwagner_lap has quit IRC | 11:38 | |
| *** ghostpl_ has joined #openstack-infra | 11:38 | |
| *** otter768 has joined #openstack-infra | 11:39 | |
| *** ldnunes has joined #openstack-infra | 11:41 | |
| *** dprince has quit IRC | 11:42 | |
| *** Longgeek has quit IRC | 11:42 | |
| *** markvoelker has joined #openstack-infra | 11:43 | |
| *** otter768 has quit IRC | 11:44 | |
| *** tiswanso has joined #openstack-infra | 11:44 | |
| *** dprince has joined #openstack-infra | 11:44 | |
| *** tsufiev has joined #openstack-infra | 11:44 | |
| *** ildikov has quit IRC | 11:45 | |
| *** ghostpl_ has quit IRC | 11:45 | |
| *** tiswanso has quit IRC | 11:45 | |
| *** tiswanso has joined #openstack-infra | 11:46 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/puppet-zuul: Set zuul server and merger services with ensure running https://review.openstack.org/168306 | 11:46 |
| *** e0ne is now known as e0ne_ | 11:48 | |
| *** markvoelker has quit IRC | 11:48 | |
| *** e0ne_ is now known as e0ne | 11:49 | |
| *** vigneshvar has joined #openstack-infra | 11:49 | |
| *** ricolin has joined #openstack-infra | 11:49 | |
| *** claudiub has joined #openstack-infra | 11:50 | |
| *** samueldmq-away is now known as samueldmq | 11:52 | |
| *** vigneshvar_ has joined #openstack-infra | 11:56 | |
| *** ominakov has joined #openstack-infra | 11:56 | |
| *** esker has joined #openstack-infra | 11:58 | |
| *** tiswanso has quit IRC | 11:59 | |
| *** pal_ has quit IRC | 11:59 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/puppet-zuul: Set zuul server and merger services with ensure running https://review.openstack.org/168306 | 12:00 |
| *** woodster_ has joined #openstack-infra | 12:01 | |
| *** resker has joined #openstack-infra | 12:02 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-pip: Ad the ability to set pip.conf and configure it https://review.openstack.org/168309 | 12:03 |
| *** esker has quit IRC | 12:04 | |
| *** _nadya_ has quit IRC | 12:05 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 12:06 | |
| *** pal_ has joined #openstack-infra | 12:10 | |
| *** dtantsur|brb is now known as dtantsur | 12:13 | |
| *** baoli has joined #openstack-infra | 12:13 | |
| *** bswartz has quit IRC | 12:14 | |
| *** baoli has quit IRC | 12:14 | |
| *** baoli has joined #openstack-infra | 12:15 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 12:15 | |
| *** pal_ has quit IRC | 12:16 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-pip: Ad the ability to set pip.conf and configure it https://review.openstack.org/168309 | 12:16 |
| *** dboik has quit IRC | 12:16 | |
| openstackgerrit | Clark Laughlin proposed openstack-infra/puppet-jenkins: Changes installation of jdk to use the architecture of the host https://review.openstack.org/168222 | 12:17 |
| *** lucasagomes has joined #openstack-infra | 12:17 | |
| lucasagomes | hi if you have a time, can you please look at https://review.openstack.org/163344 ? It was approved but needed to be rebased | 12:18 |
| *** markvoelker has joined #openstack-infra | 12:18 | |
| *** ociuhandu has quit IRC | 12:18 | |
| *** mwagner_lap has joined #openstack-infra | 12:22 | |
| *** annegentle has joined #openstack-infra | 12:22 | |
| tsufiev | Hello! could anyone advice me where the specific jenkins jobs are described? | 12:22 |
| tsufiev | to be more specific: I'm looking how to define test jobs for web frontend project, like https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L4142 | 12:23 |
| tsufiev | but it is not clear to me where I can found, say, gate-storyboard-webclient-js-test-unit | 12:23 |
| *** annegentle has quit IRC | 12:23 | |
| *** radez_g0n3 is now known as radez | 12:26 | |
| AJaeger | tsufiev: everyting is under jenkins/jobs/ | 12:28 |
| *** ociuhandu has joined #openstack-infra | 12:29 | |
| igorbelikov | tsufiev: gate-storyboard-webclient-js-test-unit is using template from https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/javascript.yaml | 12:31 |
| AJaeger | tsufiev: the file projects.yaml instantiates the template and file javascript.yaml contains gate-{name}-js-test-{test-type} | 12:31 |
| tsufiev | AJaeger, igorbelikov: thanks! it was a bit tricky for me, since grep didn't find it by full name :) | 12:32 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 12:33 | |
| *** adalbas has joined #openstack-infra | 12:33 | |
| *** yamamoto has joined #openstack-infra | 12:33 | |
| *** acfleury has joined #openstack-infra | 12:33 | |
| *** gordc has joined #openstack-infra | 12:35 | |
| *** sputnik13 has quit IRC | 12:39 | |
| *** yamamoto has quit IRC | 12:40 | |
| *** bknudson has quit IRC | 12:40 | |
| *** aysyd has joined #openstack-infra | 12:40 | |
| *** jcoufal has joined #openstack-infra | 12:46 | |
| kaisers | AJaeger: ok, will use that, thanks | 12:49 |
| *** bswartz has joined #openstack-infra | 12:49 | |
| *** amotoki has quit IRC | 12:50 | |
| *** rfolco has joined #openstack-infra | 12:50 | |
| *** doug-fish has joined #openstack-infra | 12:51 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 12:51 | |
| *** ildikov has joined #openstack-infra | 12:51 | |
| *** sdake has joined #openstack-infra | 12:51 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 12:57 | |
| *** dboik has joined #openstack-infra | 12:58 | |
| *** amotoki has joined #openstack-infra | 12:58 | |
| *** prmtl has quit IRC | 13:00 | |
| *** _nadya_ has joined #openstack-infra | 13:01 | |
| *** bknudson has joined #openstack-infra | 13:01 | |
| *** lucasagomes is now known as lucas-hungry | 13:02 | |
| *** e0ne is now known as e0ne_ | 13:03 | |
| *** pal_ has joined #openstack-infra | 13:04 | |
| *** jaypipes is now known as leakypipes | 13:05 | |
| *** tnovacik has joined #openstack-infra | 13:06 | |
| *** kgiusti has joined #openstack-infra | 13:07 | |
| *** e0ne_ is now known as e0ne | 13:11 | |
| *** spzala has joined #openstack-infra | 13:13 | |
| *** tnovacik has quit IRC | 13:13 | |
| mordred | tsufiev, AJaeger: you know - maybe we should make a "jjb-grep" tool or something "where is this job actually defined" | 13:15 |
| mordred | to be clear, I'm not actually going to do that - I'm just going to spout the idea here and then walk away :) | 13:15 |
| *** sarob has joined #openstack-infra | 13:15 | |
| *** pal_ has quit IRC | 13:16 | |
| kaisers | Q: I have issues with tempest volume tests where tempest is unable to find a fixed network. My guess is some network element in devstack is not configured correctly, i'm running nova-network default config in devstack. Does somebody have a hint on this for me? Log: http://176.9.127.22:8081/refs-changes-35-155735-218/console.log.out | 13:17 |
| kaisers | Weird thing is the tests have been running for a few days and i've not seen this issue before. | 13:17 |
| tsufiev | mordred, I've already got a small bit of tribal knowledge and that's enough for me :) | 13:19 |
| mordred | hehe :) | 13:19 |
| AJaeger | mordred: good diea. let's volunteer somebody else ;) | 13:19 |
| *** sarob has quit IRC | 13:20 | |
| *** acfleury has quit IRC | 13:21 | |
| *** dustins has joined #openstack-infra | 13:21 | |
| *** ddieterly has quit IRC | 13:21 | |
| *** mrmartin has joined #openstack-infra | 13:23 | |
| *** annegentle has joined #openstack-infra | 13:24 | |
| *** baoli has quit IRC | 13:25 | |
| *** bhunter71 has joined #openstack-infra | 13:27 | |
| *** ihrachyshka has quit IRC | 13:27 | |
| *** ihrachyshka has joined #openstack-infra | 13:27 | |
| *** baoli has joined #openstack-infra | 13:28 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/system-config: Move some hardcoded values as parameters for zuul manifests https://review.openstack.org/168330 | 13:28 |
| *** ilyashakhat has quit IRC | 13:28 | |
| *** ghostpl_ has joined #openstack-infra | 13:29 | |
| *** annegentle has quit IRC | 13:29 | |
| *** ghostpl_ has quit IRC | 13:36 | |
| openstackgerrit | Merged openstack-infra/system-config: Apply F20 gruby work-around only on RAX https://review.openstack.org/168196 | 13:39 |
| bookwar | hi, i have a question about git.openstack.org - is there an infrastructure of public third-party mirrors around it? | 13:39 |
| *** otter768 has joined #openstack-infra | 13:40 | |
| *** yamamoto has joined #openstack-infra | 13:40 | |
| bookwar | we are thinking about making mirrors in our locations (ru and cz probably). And i am wondering wouldn't it be better to make it publicly available for everyone? | 13:40 |
| *** tiswanso has joined #openstack-infra | 13:41 | |
| AJaeger | bookwar: github is mirroring git.openstack.org and that's it AFAIK | 13:42 |
| *** tiswanso has quit IRC | 13:42 | |
| *** tiswanso has joined #openstack-infra | 13:42 | |
| *** annegentle has joined #openstack-infra | 13:43 | |
| AJaeger | bookwar: better ask again in an hour or two when all infra admins are around... | 13:43 |
| *** Somay has joined #openstack-infra | 13:44 | |
| *** otter768 has quit IRC | 13:44 | |
| bookwar | AJaeger: ok, thanks | 13:44 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Bump puppetdb version https://review.openstack.org/168334 | 13:44 |
| openstackgerrit | Andrew Boik proposed openstack-infra/devstack-gate: WIP: (experiment) Test dual-stack with dependency https://review.openstack.org/168063 | 13:45 |
| *** yamamoto has quit IRC | 13:46 | |
| *** sabeen1 has joined #openstack-infra | 13:47 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/project-config: Add stackforge/puppet-autossh https://review.openstack.org/167553 | 13:49 |
| *** dims_ is now known as dimsum__ | 13:49 | |
| *** sdake_ has joined #openstack-infra | 13:50 | |
| *** peristeri has joined #openstack-infra | 13:51 | |
| *** ominakov has quit IRC | 13:52 | |
| *** sdake has quit IRC | 13:54 | |
| *** ihrachyshka has quit IRC | 13:56 | |
| fungi | bookwar: they're push mirrors, so we have to actively configure them in gerrit. each one increases the load on our gerrit, so we can't really add arbitrary external mirrors | 13:57 |
| *** ominakov has joined #openstack-infra | 13:57 | |
| *** ihrachyshka has joined #openstack-infra | 13:57 | |
| fungi | at least not with the mechanism we're using currently | 13:57 |
| *** r-daneel has joined #openstack-infra | 13:58 | |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config: Add tempest experimental job running tempest-lib from source https://review.openstack.org/168337 | 13:58 |
| *** ominakov has quit IRC | 13:58 | |
| *** AJaeger has quit IRC | 13:59 | |
| bookwar | fungi: i see, thanks | 13:59 |
| *** ddieterly has joined #openstack-infra | 14:00 | |
| *** salv-orl_ has joined #openstack-infra | 14:00 | |
| *** ddieterly has quit IRC | 14:00 | |
| *** ddieterly has joined #openstack-infra | 14:00 | |
| *** dimsum__ has quit IRC | 14:01 | |
| *** hodos has joined #openstack-infra | 14:01 | |
| david-lyle | would appreciate a approval of https://review.openstack.org/#/c/168180/ all Horizon changes are currently dead in the water due to horizon integration test framework failings | 14:02 |
| *** dimsum__ has joined #openstack-infra | 14:02 | |
| *** tiswanso has quit IRC | 14:02 | |
| hodos | how often thirdparty CI should rebuild the images? | 14:03 |
| *** salv-orlando has quit IRC | 14:03 | |
| *** signed8bit has joined #openstack-infra | 14:04 | |
| fungi | hodos: i don't think there are any specific requirements around that unless the project you're planning on testing has decreed something | 14:04 |
| fungi | we (openstack infra) update our images daily | 14:04 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 14:05 | |
| *** tiswanso has joined #openstack-infra | 14:06 | |
| *** tiswanso has quit IRC | 14:06 | |
| *** sputnik13 has joined #openstack-infra | 14:06 | |
| *** tiswanso has joined #openstack-infra | 14:06 | |
| *** lucas-hungry is now known as lucasagomes | 14:07 | |
| *** annegentle has quit IRC | 14:07 | |
| *** mattfarina has joined #openstack-infra | 14:07 | |
| *** samueldmq is now known as samueldmq-away | 14:08 | |
| *** baoli has quit IRC | 14:10 | |
| *** trey has quit IRC | 14:11 | |
| *** notnownikki has joined #openstack-infra | 14:11 | |
| *** rlandy has quit IRC | 14:11 | |
| *** trey has joined #openstack-infra | 14:12 | |
| *** ianw has quit IRC | 14:12 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-pip: Ad the ability to set pip.conf and configure it https://review.openstack.org/168309 | 14:12 |
| *** ianw has joined #openstack-infra | 14:12 | |
| ameade | Hey i named the 'netapp-fc-ci' account full name slightly wrong and cant change it, it needs to be 'NetApp FC CI'. Can someone with gerrit admin powers fix that for me? I'll buy you a beer at the summit | 14:13 |
| *** gokrokve has joined #openstack-infra | 14:13 | |
| fungi | ameade: i don't understand. can't you change it at https://review.openstack.org/#/settings/contact ? | 14:14 |
| ameade | fungi: apparently not, and I cant do a gerrit ssh command either | 14:14 |
| fungi | ameade: the full name has nothing to do with ssh | 14:15 |
| fungi | are you confusing that with your gerrit username? | 14:15 |
| fungi | also can you expand on "apparently not?" | 14:15 |
| ameade | fungi: ah that does work, didnt realize there were two pages | 14:15 |
| ameade | profile & contact | 14:15 |
| fungi | yep | 14:15 |
| ameade | fungi: lol nvm you can still have a beer though | 14:15 |
| fungi | cool, it works the way i expected ;) | 14:16 |
| *** mtanino has joined #openstack-infra | 14:16 | |
| *** sarob has joined #openstack-infra | 14:16 | |
| * ameade facepalm | 14:16 | |
| morganfainberg | fungi: the method to stockpile beers for the summit? | 14:16 |
| morganfainberg | :P | 14:17 |
| fungi | morganfainberg: surprisingly, there has never been a shortage | 14:17 |
| morganfainberg | fungi: I'd say unsurprisingly ;) | 14:17 |
| fungi | granted, paris was lacking in great beer, it did still have plenty on hand | 14:17 |
| morganfainberg | Now good beer... That is a horse of a different color. | 14:17 |
| fungi | good beer is whatever they keep bringing to the table | 14:18 |
| morganfainberg | Lol!! | 14:18 |
| * morganfainberg had a thought to bug infra folks about last night. And now it disappeared. | 14:19 | |
| morganfainberg | :( | 14:19 |
| openstackgerrit | Dane LeBlanc proposed openstack-infra/devstack-gate: WIP: (experiment) Test dual-stack with dependency https://review.openstack.org/168063 | 14:19 |
| morganfainberg | Must not be important. | 14:19 |
| morganfainberg | Anyway. Mornin :) | 14:19 |
| fungi | this morning is improving with each successive cup of coffee | 14:20 |
| *** ianw has quit IRC | 14:21 | |
| *** sarob has quit IRC | 14:21 | |
| *** ianw has joined #openstack-infra | 14:21 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 14:21 | |
| *** eharney has joined #openstack-infra | 14:21 | |
| *** timcline has joined #openstack-infra | 14:22 | |
| *** amitgandhinz has joined #openstack-infra | 14:23 | |
| *** gokrokve has quit IRC | 14:23 | |
| *** gokrokve has joined #openstack-infra | 14:24 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-pip: Ad the ability to set pip.conf and configure it https://review.openstack.org/168309 | 14:25 |
| *** gokrokve has quit IRC | 14:29 | |
| *** prad has joined #openstack-infra | 14:29 | |
| *** ayoung is now known as ayoung-afk | 14:29 | |
| *** vigneshvar_ has quit IRC | 14:31 | |
| *** vigneshvar has quit IRC | 14:31 | |
| *** annegentle has joined #openstack-infra | 14:32 | |
| *** ddieterly has quit IRC | 14:32 | |
| *** reed has joined #openstack-infra | 14:33 | |
| clarkb | adam_g you have successfully managed to get me to think far too much about stable requirements now | 14:33 |
| clarkb | adam_g still dont have a good answer though | 14:33 |
| *** ghostpl_ has joined #openstack-infra | 14:33 | |
| *** achanda has joined #openstack-infra | 14:35 | |
| *** e0ne is now known as e0ne_ | 14:36 | |
| *** dhritishikhar_ has joined #openstack-infra | 14:37 | |
| *** unicell has joined #openstack-infra | 14:38 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-pip: Ad the ability to set pip.conf and configure it https://review.openstack.org/168309 | 14:38 |
| *** unicell1 has quit IRC | 14:38 | |
| pelix | can anyone familiar with zuul/gearman comment on how long since zuul was migrated to gearman, relevant for https://review.openstack.org/167254 (I'm presuming it's quite some time ago?) | 14:39 |
| *** wznoinsk has quit IRC | 14:39 | |
| * mkj_wk begs for review of https://review.openstack.org/#/c/158877/ ☺ | 14:41 | |
| *** zz_jgrimm is now known as jgrimm | 14:41 | |
| *** wenlock has joined #openstack-infra | 14:41 | |
| fungi | pelix: hrm... there was a feature branch and merge. might have been around the 2.0 release tag... checking | 14:41 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config: Add an experimental stress identity v3 test https://review.openstack.org/168113 | 14:42 |
| *** achanda has quit IRC | 14:42 | |
| *** ghostpl_ has quit IRC | 14:42 | |
| *** MaxV has quit IRC | 14:42 | |
| pelix | fungi: is this it https://review.openstack.org/gitweb?p=openstack-infra/zuul.git;a=commitdiff;h=1f4c2bb1046d428e67cdb089942f687ae78ad0e1 ? | 14:43 |
| *** thingee has quit IRC | 14:43 | |
| *** wznoinsk has joined #openstack-infra | 14:44 | |
| fungi | pelix: c225b4d 2013-07-18 Merge branch 'feature/gearman' https://review.openstack.org/37777 | 14:44 |
| fungi | pelix: that's when it first landed in the master branch anyway | 14:45 |
| *** mfisch has joined #openstack-infra | 14:45 | |
| fungi | pelix: 2.0.0 was the first tag to include it though | 14:45 |
| mfisch | is there an issue today with gerrit not showing a rebase button? | 14:45 |
| pelix | fungi: much obliged | 14:45 |
| fungi | mfisch: i see a rebase button, picking an arbitrary change | 14:46 |
| *** hashar has quit IRC | 14:46 | |
| fungi | mfisch: more likely there's something about the specific change you're looking at that gerrit doesn't believe it needs a rebase? | 14:46 |
| *** dhritishikhar_ has quit IRC | 14:46 | |
| *** dansmith is now known as superdan | 14:47 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Don't hardcode pip.conf values https://review.openstack.org/166252 | 14:47 |
| *** e0ne_ is now known as e0ne | 14:48 | |
| mfisch | fungi: yep my fault, two people did the same code in 2 different places | 14:49 |
| *** ominakov has joined #openstack-infra | 14:49 | |
| *** Somay has quit IRC | 14:49 | |
| morganfainberg | fungi: I shall be getting coffee soooooon | 14:50 |
| *** Somay has joined #openstack-infra | 14:51 | |
| morganfainberg | mfisch: yay no more downward sql migrations in keystone!! ;) | 14:51 |
| mfisch | +100 | 14:51 |
| sdague | clarkb: so, can we just approve from - https://review.openstack.org/#/c/135768 down? | 14:51 |
| sdague | because these things are catching useful nova regressions, and we'd like them in play asap for the release | 14:51 |
| sdague | then work out the cpu_map optimizations later | 14:52 |
| *** Somay has quit IRC | 14:52 | |
| *** Somay has joined #openstack-infra | 14:53 | |
| *** Somay has quit IRC | 14:54 | |
| *** emagana has joined #openstack-infra | 14:54 | |
| openstackgerrit | Merged openstack-infra/project-config: Temp - making horizon integration tests non-voting https://review.openstack.org/168180 | 14:57 |
| *** nelsnelson has joined #openstack-infra | 14:58 | |
| *** zz_dimtruck is now known as dimtruck | 14:58 | |
| clarkb | sdague done. as another data point on a held node I havent been able to convince virsh to boot using core2duo less pse36 and monitor (the two flags that we dont have on these cpus) | 14:58 |
| *** MaxV has joined #openstack-infra | 14:58 | |
| sdague | clarkb: a cirros image won't boot? I did it with virt-manager locally | 14:59 |
| clarkb | libvirt says hypervisor cant support the cpu | 14:59 |
| clarkb | I think something else is affecting this not just cpu flags | 15:00 |
| sdague | clarkb: ok, interesting, I thought I had that working | 15:00 |
| clarkb | sdague did you use kvm? | 15:00 |
| sdague | yes | 15:00 |
| clarkb | I wonder if it is something to do with qemu | 15:00 |
| *** Somay has joined #openstack-infra | 15:00 | |
| clarkb | but kvm works because proper virt | 15:00 |
| *** carl_baldwin has joined #openstack-infra | 15:01 | |
| sdague | yeh, it could be. | 15:01 |
| sdague | I think we might need danpb in the thread then to figure it out | 15:02 |
| sdague | which means an ML thread, because he doesn't join irc | 15:02 |
| clarkb | I also tried pentiumpro + lm feature | 15:03 |
| *** doude has quit IRC | 15:04 | |
| *** annegentle has quit IRC | 15:04 | |
| *** annegentle has joined #openstack-infra | 15:05 | |
| *** julim has quit IRC | 15:07 | |
| *** julim has joined #openstack-infra | 15:08 | |
| *** weshay has joined #openstack-infra | 15:09 | |
| *** devlaps has joined #openstack-infra | 15:10 | |
| *** marun has joined #openstack-infra | 15:10 | |
| *** ghostpl_ has joined #openstack-infra | 15:11 | |
| *** julim has quit IRC | 15:13 | |
| *** ddieterly has joined #openstack-infra | 15:13 | |
| *** dannywilson has joined #openstack-infra | 15:14 | |
| *** julim has joined #openstack-infra | 15:14 | |
| *** dannywilson has quit IRC | 15:14 | |
| *** ZZelle has quit IRC | 15:14 | |
| *** dannywilson has joined #openstack-infra | 15:14 | |
| *** reed has quit IRC | 15:14 | |
| *** marun has quit IRC | 15:15 | |
| clarkb | sdague the other thing I noticed was rax VMs are a lot more consistent now. I was unable to find the second cpu type yesterday so we may be able to go back to nova defaults | 15:17 |
| *** sarob has joined #openstack-infra | 15:17 | |
| clarkb | but need to investigate that more. likely just coincidence | 15:17 |
| *** markus_z has quit IRC | 15:17 | |
| fungi | rackspace consistency? must be a bug | 15:19 |
| yolanda | here, i have a problem with that change | 15:20 |
| yolanda | https://review.openstack.org/#/c/168309/ | 15:20 |
| yolanda | i'm trying to move pip.conf out of openstack_project to pip module | 15:20 |
| yolanda | but the puppet apply tests are complaining because a duplicate file now | 15:20 |
| hodos | fungi: yesterday I've bugged you guys about ssh timeout; i think it happened because the image was beign built. right now i have an image built, which shows up as a 'snapshot' in the horizon image list | 15:20 |
| yolanda | because it's declared on puppet-pip | 15:20 |
| yolanda | and in the old code of openstack_project | 15:21 |
| yolanda | how could make the tests pass? | 15:21 |
| *** fifieldt has quit IRC | 15:21 | |
| hodos | fungi: but now when the node is instaciated from this image for some reason the private key is not injected there | 15:21 |
| clarkb | yolanda you will need a single definition of the file. pick one to remove | 15:21 |
| yolanda | clarkb, i did | 15:21 |
| yolanda | but tests are using the old code | 15:22 |
| hodos | fungi: but now when the node is instanciated from this image for some reason the private key is not injected there | 15:22 |
| yolanda | so it's related to https://review.openstack.org/168309 and https://review.openstack.org/166252 | 15:22 |
| *** sarob has quit IRC | 15:22 | |
| fungi | hodos: whatever configures the operating system on your images needs to set up an account with a ~/.ssh/authorized_keys file. for example we use a puppet manifest to do that for us | 15:22 |
| yolanda | so clarkb, problem i have is that my change for puppet-pip is gating against the old code of system-config. In a new change i'm removing pip.conf from there, but this is causing that change on puppet-pip don't pass | 15:24 |
| hodos | fungi: i think i have the same; can you point me to that manifest ? | 15:24 |
| clarkb | yolanda 309 is the parebt so it should test with the older code | 15:24 |
| yolanda | mm, shall i switch the depends-on? | 15:25 |
| *** ajo has joined #openstack-infra | 15:25 | |
| clarkb | yolanda you may need to make that file definition optional in 309, then enable it in 252 | 15:25 |
| clarkb | no I think the order is correct we just have to do this in a forward and backward compatible manner | 15:25 |
| yolanda | file definition is optional in 309 | 15:26 |
| yolanda | i used an if ! defined there | 15:26 |
| yolanda | clarkb, do you mean that? | 15:26 |
| clarkb | that only works if both sides have that | 15:26 |
| clarkb | I was thinking more of if install_pip_conf { file { pip.conf } } | 15:27 |
| yolanda | i see, because it first executes code of puppet-pip then fails in the openstack_project side | 15:27 |
| clarkb | yup | 15:27 |
| yolanda | ah, and don't install by default? | 15:28 |
| openstackgerrit | Nikki Heald proposed openstack-infra/puppet-cgit: Add configurable settings for httpd.conf https://review.openstack.org/160885 | 15:28 |
| clarkb | ya then we change that later | 15:28 |
| *** armax has joined #openstack-infra | 15:28 | |
| yolanda | ok | 15:28 |
| yolanda | going to file it | 15:28 |
| sdague | can we get reviews of the project-config patches here - https://etherpad.openstack.org/p/kilo-qa-code-sprint-nyc - I've got +2 on the 3 of them | 15:29 |
| sdague | and it would be nice to land while at the sprint | 15:29 |
| derekh | fungi: clarkb we have a working F20 image again, thanks for you help | 15:29 |
| derekh | *your | 15:29 |
| *** gyee has joined #openstack-infra | 15:29 | |
| *** baoli has joined #openstack-infra | 15:30 | |
| *** baoli has quit IRC | 15:31 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-pip: Add the ability to set pip.conf and configure it https://review.openstack.org/168309 | 15:33 |
| *** stevemar has joined #openstack-infra | 15:34 | |
| openstackgerrit | Clark Laughlin proposed openstack-infra/puppet-jenkins: Change installation of the JDK to use the architecture of the host https://review.openstack.org/168222 | 15:34 |
| *** baoli has joined #openstack-infra | 15:35 | |
| *** JoshNang has joined #openstack-infra | 15:36 | |
| *** vponomaryov has quit IRC | 15:38 | |
| *** _nadya_ has quit IRC | 15:39 | |
| *** vponomaryov has joined #openstack-infra | 15:39 | |
| *** otter768 has joined #openstack-infra | 15:41 | |
| *** Somay has quit IRC | 15:42 | |
| *** _shaps_ has joined #openstack-infra | 15:43 | |
| *** Somay has joined #openstack-infra | 15:43 | |
| *** Ala has quit IRC | 15:44 | |
| *** otter768 has quit IRC | 15:45 | |
| *** Somay has quit IRC | 15:48 | |
| *** sarob has joined #openstack-infra | 15:48 | |
| *** radez is now known as radez_g0n3 | 15:50 | |
| fungi | hodos: we set it via the openstack_project::single_use_slave class from the openstack-infra/system-config repo https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/single_use_slave.pp | 15:51 |
| *** dboik_ has joined #openstack-infra | 15:51 | |
| fungi | hodos: we call into that from this script, where it can also be overridden via a $NODEPOOL_SSH_KEY variable in the calling environment http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/scripts/prepare_node.sh#n60 | 15:52 |
| hodos | fungi: thank you! | 15:53 |
| *** sarob has quit IRC | 15:54 | |
| *** dboik has quit IRC | 15:54 | |
| zaro | morning | 15:55 |
| *** dboik_ has quit IRC | 15:57 | |
| *** scheuran has quit IRC | 15:57 | |
| *** dboik has joined #openstack-infra | 15:57 | |
| *** tsg has joined #openstack-infra | 15:57 | |
| *** ayoung-afk is now known as ayoung | 15:59 | |
| *** YorikSar has quit IRC | 15:59 | |
| *** spzala has quit IRC | 16:00 | |
| *** spzala has joined #openstack-infra | 16:01 | |
| *** carl_baldwin has quit IRC | 16:03 | |
| *** thedodd has joined #openstack-infra | 16:04 | |
| *** annegentle has quit IRC | 16:06 | |
| *** ominakov has quit IRC | 16:07 | |
| *** gokrokve has joined #openstack-infra | 16:08 | |
| *** mpavone has quit IRC | 16:08 | |
| *** ominakov has joined #openstack-infra | 16:08 | |
| *** unicell has quit IRC | 16:09 | |
| *** thingee has joined #openstack-infra | 16:10 | |
| *** ChuckC has joined #openstack-infra | 16:11 | |
| *** btran has joined #openstack-infra | 16:13 | |
| *** _nadya_ has joined #openstack-infra | 16:13 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Add methods for logical router management https://review.openstack.org/167371 | 16:14 |
| *** mriedem is now known as mriedem_lunch | 16:17 | |
| *** patrickeast has joined #openstack-infra | 16:17 | |
| *** prad has quit IRC | 16:17 | |
| *** Swami has joined #openstack-infra | 16:17 | |
| *** dhritishikhar_ has joined #openstack-infra | 16:19 | |
| *** AJaeger has joined #openstack-infra | 16:19 | |
| *** chuck_ is now known as zul | 16:19 | |
| *** zul has quit IRC | 16:19 | |
| *** zul has joined #openstack-infra | 16:19 | |
| *** e0ne is now known as e0ne_ | 16:20 | |
| *** vlaza has quit IRC | 16:20 | |
| superdan | clarkb: https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt | 16:21 |
| superdan | clarkb: 1:qemu_monitor | 16:21 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Allow root user to ssh in as stack https://review.openstack.org/137176 | 16:22 |
| *** crc32 has joined #openstack-infra | 16:22 | |
| *** gokrokve has quit IRC | 16:23 | |
| *** dulek has quit IRC | 16:24 | |
| *** e0ne_ is now known as e0ne | 16:26 | |
| *** e0ne is now known as e0ne_ | 16:27 | |
| *** e0ne_ is now known as e0ne | 16:27 | |
| *** pal_ has joined #openstack-infra | 16:28 | |
| *** claudiub has quit IRC | 16:29 | |
| *** teran has joined #openstack-infra | 16:30 | |
| clarkb | jogo: sdague those changes need a rebase so I am working on that now | 16:30 |
| *** weshay has quit IRC | 16:31 | |
| *** ominakov has quit IRC | 16:31 | |
| *** dkranz has quit IRC | 16:32 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Set custom cpu_model for live_migrate https://review.openstack.org/141530 | 16:32 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Enable live block migration https://review.openstack.org/135768 | 16:32 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Use gate64 cpu type. https://review.openstack.org/168408 | 16:32 |
| *** ominakov has joined #openstack-infra | 16:35 | |
| *** ildikov has quit IRC | 16:35 | |
| *** SumitNaiksatam has joined #openstack-infra | 16:35 | |
| *** AJaeger has quit IRC | 16:36 | |
| clarkb | sdague: I was able to locally get pentiumpro + lm working locally after dan pointed out my fallback was wrong | 16:36 |
| clarkb | and I have reapproved the two changes that needed a rebase | 16:37 |
| *** tsg has quit IRC | 16:39 | |
| *** alaski is now known as lascii | 16:39 | |
| *** unicell has joined #openstack-infra | 16:39 | |
| *** afazekas has quit IRC | 16:40 | |
| *** ominakov has quit IRC | 16:41 | |
| *** esker has joined #openstack-infra | 16:41 | |
| *** resker has quit IRC | 16:41 | |
| *** ominakov has joined #openstack-infra | 16:42 | |
| *** yamamoto has joined #openstack-infra | 16:45 | |
| rcarrillocruz | clarkb: hiya, what you think about removing the rm workspace-cache on https://review.openstack.org/#/c/165234/ ? if it is not used at all, we shouldn't bother deleting | 16:47 |
| rcarrillocruz | besides changing the devstack-gate doc to make sure the project tested in question is put on /opt, instead of workspace-cache | 16:48 |
| openstackgerrit | Merged openstack-infra/storyboard: Session per request https://review.openstack.org/155751 | 16:48 |
| *** yamamoto has quit IRC | 16:49 | |
| *** alexpilotti has joined #openstack-infra | 16:50 | |
| *** weshay has joined #openstack-infra | 16:51 | |
| openstackgerrit | Merged openstack-infra/storyboard: Change Story to StorySummary in search https://review.openstack.org/163852 | 16:51 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Use gate64 cpu type https://review.openstack.org/168408 | 16:51 |
| *** bknudson has quit IRC | 16:51 | |
| clarkb | rcarrillocruz: let me refresh my memory on thta change | 16:52 |
| openstackgerrit | Merged openstack-infra/storyboard: Remove search from branches, milestones and tags https://review.openstack.org/167155 | 16:52 |
| openstackgerrit | Merged openstack-infra/python-storyboardclient: Added subscription events controller https://review.openstack.org/167223 | 16:52 |
| *** rfolco has quit IRC | 16:52 | |
| openstackgerrit | Merged openstack-infra/storyboard: Fix docs https://review.openstack.org/166803 | 16:53 |
| *** deepakcs has joined #openstack-infra | 16:53 | |
| clarkb | rcarrillocruz: I think the only place where it is attempted to be used is the -dev script, but then d-g itself doesn't use it | 16:54 |
| sdague | clarkb: awesome, great | 16:54 |
| clarkb | rcarrillocruz: so we can probably clean it up | 16:54 |
| rcarrillocruz | cool | 16:54 |
| rcarrillocruz | so i'll abandon and up two changes, one for docs and another to clean up the rm, thx | 16:54 |
| clarkb | rcarrillocruz: sounds good | 16:55 |
| *** rfolco has joined #openstack-infra | 16:56 | |
| *** bhuvan has joined #openstack-infra | 16:57 | |
| *** jogo is now known as flashgordon | 16:57 | |
| *** Somay has joined #openstack-infra | 16:58 | |
| *** marun has joined #openstack-infra | 16:58 | |
| *** davide has joined #openstack-infra | 17:00 | |
| *** signed8b_ has joined #openstack-infra | 17:01 | |
| *** signed8bit has quit IRC | 17:02 | |
| rcarrillocruz | so, i'm guessing you must have needed manual intervention for setting puppetdb in infra | 17:02 |
| rcarrillocruz | cos the class does not have any params to set the ssl cert and keys | 17:02 |
| rcarrillocruz | unless you set it up without ssl? | 17:02 |
| * rcarrillocruz goes check | 17:02 | |
| *** tqtran has joined #openstack-infra | 17:03 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 17:03 | |
| *** annegentle has joined #openstack-infra | 17:04 | |
| *** dangers_away is now known as dangers | 17:05 | |
| nibalizer | clarkb: https://review.openstack.org/#/c/139598/ | 17:06 |
| *** jlanoux has quit IRC | 17:06 | |
| *** yamahata has joined #openstack-infra | 17:07 | |
| *** marcusvrn has joined #openstack-infra | 17:07 | |
| rcarrillocruz | the upstream puppetdb has by default SSL enabled (the version you use), I assume you disabled it manually | 17:08 |
| rcarrillocruz | this is all related to https://review.openstack.org/#/c/168334/ btw, we run an old version that doesn't have ssl params available in the class signature | 17:09 |
| *** ominakov has quit IRC | 17:09 | |
| *** barnaby is now known as bcourt | 17:09 | |
| clarkb | rcarrillocruz: I believe we use the puppetmaster as the CA | 17:10 |
| *** aysyd has quit IRC | 17:10 | |
| clarkb | rcarrillocruz: and it gives puppetdb the certs? | 17:10 |
| clarkb | nibalizer would know | 17:10 |
| mordred | nibalizer knows all | 17:10 |
| *** dhritishikhar_ has quit IRC | 17:11 | |
| *** marcusvrn1 has quit IRC | 17:11 | |
| rcarrillocruz | ok, this is what you use in upstream: | 17:12 |
| rcarrillocruz | http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/puppetdb.pp | 17:12 |
| rcarrillocruz | no ssl certs at all | 17:12 |
| *** jerryz_ has quit IRC | 17:12 | |
| rcarrillocruz | and that's due to puppetdb module is 3.01 does not expose any: | 17:12 |
| rcarrillocruz | https://github.com/puppetlabs/puppetlabs-puppetdb/blob/3.0.1/manifests/init.pp | 17:12 |
| pleia2 | I know in general our puppet bootstrapping isn't the best | 17:13 |
| rcarrillocruz | whereas latest stable release does: | 17:13 |
| rcarrillocruz | https://github.com/puppetlabs/puppetlabs-puppetdb/blob/4.1.0/manifests/init.pp | 17:13 |
| lucasagomes | hi if you have a time, please take a look at https://review.openstack.org/163344 . It's adding a check on openstack/ironic-python-agent (non-voting) to test the IPA ramdisk with the PXE drivers in Ironic. It was approved but needed to be rebased (needed another +2 +a again) | 17:13 |
| *** jistr has quit IRC | 17:13 | |
| *** gokrokve has joined #openstack-infra | 17:13 | |
| rcarrillocruz | that's very painful to automate puppetdb, it requires manual intervention | 17:13 |
| rcarrillocruz | we should bump the module to 4.1.0 (as my change tries to do) and then later on have the openstack_project::puppetdb have those params exposed | 17:14 |
| rcarrillocruz | pleia2: oh yeah, they're not :/ | 17:14 |
| *** weshay has quit IRC | 17:15 | |
| *** MaxV has quit IRC | 17:16 | |
| *** achanda has joined #openstack-infra | 17:16 | |
| *** achanda has quit IRC | 17:16 | |
| *** achanda has joined #openstack-infra | 17:17 | |
| *** dboik_ has joined #openstack-infra | 17:17 | |
| *** dtantsur is now known as dtantsur|afk | 17:19 | |
| *** dboik has quit IRC | 17:20 | |
| *** deepakcs has quit IRC | 17:21 | |
| *** aysyd has joined #openstack-infra | 17:22 | |
| fungi | jeblair: mordred: clarkb: SergeyLukjanov: jhesketh: pleia2: proposed maintenance plan for today's project renaming at 22:00 https://etherpad.openstack.org/p/repo-renaming-2015-03-27 | 17:22 |
| mkj_wk | fungi: thanks for the testing and review! Since I'm new here, is there anything I need to do, or does it just need another +2? | 17:22 |
| jeblair | fungi: ack, thx, will review | 17:23 |
| *** _nadya_ has quit IRC | 17:23 | |
| fungi | mkj_wk: a second +2 in its current state and it will generally get approved by that second reviewer | 17:23 |
| mkj_wk | ok, cool | 17:23 |
| *** _nadya_ has joined #openstack-infra | 17:24 | |
| leakypipes | jeblair: I nominate you "Chief Steward of OpenStack" :) | 17:25 |
| leakypipes | jeblair: although that does sound very Game of Thrones... | 17:26 |
| *** e0ne is now known as e0ne_ | 17:26 | |
| clarkb | I might be derping locally, but is anyone else not able to ssh onto a nodepool built slave? | 17:26 |
| clarkb | ssh -vvv shows it offering the correct key | 17:27 |
| *** jeblair is now known as denethor | 17:27 | |
| * denethor leaps from minis tirith wreathed in flame | 17:27 | |
| *** denethor is now known as jeblair | 17:27 | |
| *** MaxV has joined #openstack-infra | 17:27 | |
| jeblair | well that didn't go well for the steward did it? | 17:28 |
| *** sdake_ has quit IRC | 17:28 | |
| *** sdake has joined #openstack-infra | 17:28 | |
| *** yamahata has quit IRC | 17:28 | |
| *** e0ne_ is now known as e0ne | 17:28 | |
| *** HeOS has quit IRC | 17:28 | |
| clarkb | I am able to log into long lived nodes | 17:29 |
| jeblair | clarkb: i sshd into 104.130.130.6 corvus@bare-trusty-rax-dfw-1617550 | 17:29 |
| jeblair | clarkb: what image are you trying? and what node? | 17:29 |
| clarkb | ok I did get on that one | 17:29 |
| fungi | clarkb: i cannot ssh into 162.242.166.73 (a devstack-trusty in rax-ord) | 17:29 |
| fungi | Permission denied (publickey). | 17:29 |
| clarkb | ssh clarkb@104.130.229.95 bare-precise on rax-ad | 17:30 |
| *** pal_ has quit IRC | 17:30 | |
| clarkb | but also devstack-trusty-2-node (where I started, held the node that my latest batch of qemu cpu map edits got onto) | 17:30 |
| *** mriedem_lunch is now known as mriedem | 17:30 | |
| *** cebruns has quit IRC | 17:30 | |
| fungi | i similarly can't log into 104.130.229.95 | 17:31 |
| fungi | wonder what's going on there | 17:31 |
| fungi | able to log into 104.130.130.6 fine | 17:31 |
| fungi | so, yeah, there's something wrong with some of these servers at least | 17:31 |
| *** MaxV has quit IRC | 17:31 | |
| *** ptomka has joined #openstack-infra | 17:31 | |
| *** pblaho has quit IRC | 17:32 | |
| clarkb | I feel both good and scared that its not just me :) | 17:33 |
| *** aysyd has quit IRC | 17:34 | |
| clarkb | I am reading the rax-ord devstack-trusty image build log now | 17:35 |
| *** kashyap has joined #openstack-infra | 17:35 | |
| *** ildikov has joined #openstack-infra | 17:37 | |
| *** ricolin has quit IRC | 17:37 | |
| morganfainberg | jeblair: so I have a question regarding quality of life for us PTL types. For feature freeze and the like we use procedural -2s. Would it make sense to make it a -2 workflow+sticky instead? The code review -2 really blocks up review eyes even though we don't want to say "this can't merge because it's bad code" | 17:37 |
| morganfainberg | jeblair: just a random thought. | 17:37 |
| clarkb | fungi: jeblair the puppet output shows it adding a jenkins user but not any others | 17:38 |
| clarkb | so I think we should look at puppet changes | 17:38 |
| *** aysyd has joined #openstack-infra | 17:38 | |
| fungi | clarkb: yep, already digging there | 17:38 |
| jeblair | morganfainberg: we've looked into the procedural workflow issue, and i think we may have a workable solution when we can use the 'work in progress' gerrit plugin, which i think may become feasible with gerrit 2.10 | 17:38 |
| morganfainberg | jeblair: awesome!! I look forward to it. | 17:39 |
| morganfainberg | :) | 17:39 |
| fungi | clarkb: https://review.openstack.org/162152 is the only thing to land in puppet-jenkins in the past few days, so i don't think it's there | 17:39 |
| *** e0ne is now known as e0ne_ | 17:39 | |
| *** ajmiller has joined #openstack-infra | 17:39 | |
| clarkb | fungi: if I had to guess nibalizers system-config change "Refactor o_p::base into o_p::template" is at fault | 17:40 |
| fungi | clarkb: strongly suspecting https://review.openstack.org/162830 | 17:40 |
| fungi | yeah, that one | 17:40 |
| jeblair | morganfainberg: me too -- we don't so much have the freeze problem, but we also have the "do not land this change which will cause major production changes" problem, and we use procedural -2s for that with the same attendant problems | 17:40 |
| *** YorikSar has joined #openstack-infra | 17:40 | |
| morganfainberg | jeblair: yeah figured it would benefit your team in similar / subtly different words/ways | 17:40 |
| fungi | morganfainberg: jeblair: yep, like the half dozen project rename changes which need to be reviewed but not merged before 22:00 today | 17:40 |
| fungi | which i should go -2 while i'm thinking about it | 17:41 |
| *** otter768 has joined #openstack-infra | 17:42 | |
| pleia2 | big rename list | 17:42 |
| jeblair | clarkb, fungi: do you think we're seeing problems with all the images as they go into production? | 17:42 |
| clarkb | jeblair: ya, I think so | 17:42 |
| fungi | that's what it's looking like | 17:43 |
| * jeblair starts launching nodes | 17:43 | |
| *** SumitNaiksatam has quit IRC | 17:44 | |
| clarkb | I think I see the problem | 17:44 |
| clarkb | give me a couple minutes to confirm | 17:44 |
| *** amotoki has quit IRC | 17:45 | |
| *** arxcruz has quit IRC | 17:46 | |
| *** _shaps_ has quit IRC | 17:46 | |
| * nibalizer is here | 17:47 | |
| *** otter768 has quit IRC | 17:47 | |
| pleia2 | nibalizer: how goes the event? | 17:47 |
| *** MaxV has joined #openstack-infra | 17:48 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Realize users in template.pp https://review.openstack.org/168433 | 17:48 |
| clarkb | fungi: jeblair ^ I believe that to be the fix | 17:48 |
| *** pelix has quit IRC | 17:49 | |
| *** mwagner_lap has quit IRC | 17:49 | |
| *** bknudson has joined #openstack-infra | 17:49 | |
| *** cdent has left #openstack-infra | 17:49 | |
| *** e0ne_ has quit IRC | 17:50 | |
| fungi | clarkb: i agree, we're no longer doing that anywhere | 17:51 |
| jeblair | so, er, would new servers not come up with our users either? | 17:51 |
| fungi | that's what i'm expecting | 17:51 |
| nibalizer | clarkb: fungi jeblair i agree | 17:52 |
| clarkb | fungi: jeblair correct | 17:53 |
| clarkb | old ones are ok because not having the realiez won | 17:53 |
| clarkb | 't delete other users | 17:53 |
| jeblair | so should we try to include that fix as well? | 17:53 |
| fungi | i'm still a little unclear why we don't use the openstack_project::users_install class instead | 17:53 |
| nibalizer | I specifically remember dealing with that block in one of the rebases of my template/base patch | 17:53 |
| fungi | since we have some duplication there | 17:54 |
| nibalizer | fungi: i think that is a good suggestion | 17:54 |
| *** dhritishikhar_ has joined #openstack-infra | 17:55 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Use gate64 cpu type during multinode testing https://review.openstack.org/168408 | 17:55 |
| clarkb | fungi: does that install all the possible users or just the root set? | 17:56 |
| *** aysyd has quit IRC | 17:56 | |
| *** MaxV has quit IRC | 17:56 | |
| fungi | just the root set | 17:56 |
| *** derekh has quit IRC | 17:56 | |
| *** aysyd has joined #openstack-infra | 17:57 | |
| *** yamahata has joined #openstack-infra | 17:57 | |
| clarkb | fungi: then ya that is likely a good improvement. Want to update my change or should I update it? | 17:57 |
| jeblair | clarkb, fungi, nibalizer: commented on 168433 | 17:57 |
| clarkb | I will make the change now | 17:58 |
| nibalizer | jeblair: you are correct | 17:58 |
| clarkb | there is a slight danger in that change though and that is the user disable | 17:59 |
| clarkb | we didn't hose the world here because no users were disabled, getting the true/false wrong there would break a bunc of stuff though | 17:59 |
| *** lucasagomes is now known as lucas-beer | 17:59 | |
| jeblair | i have a node ready to test a template run; we can do some noop runs (and maybe even real runs) in a non-production environment on production servers | 18:00 |
| clarkb | yup | 18:00 |
| fungi | clarkb: well, at the moment we have the ansible root backdoor | 18:00 |
| nibalizer | jeblair: ++ | 18:00 |
| *** e0ne has joined #openstack-infra | 18:00 | |
| fungi | if worse comes to worse | 18:00 |
| jeblair | true | 18:00 |
| nibalizer | pleia2: stuff is going pretty well | 18:01 |
| nibalizer | er the event | 18:01 |
| pleia2 | nibalizer: glad to hear it :) | 18:01 |
| fungi | even if we manage to lock ourselves out of puppetmaster.o.o, we can connect to the virtual console and reboot it into single-user mode, then root it, then use ansible to get into anything else form there. not perfect, but possible | 18:01 |
| *** tnovacik has joined #openstack-infra | 18:02 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Realize users in template.pp https://review.openstack.org/168433 | 18:02 |
| fungi | er, from there | 18:02 |
| *** andreykurilin_ has joined #openstack-infra | 18:02 | |
| *** otter768 has joined #openstack-infra | 18:03 | |
| *** e0ne has quit IRC | 18:03 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/project-config: Add Neutron DVR aiopcpu job for multi-node testing https://review.openstack.org/158415 | 18:04 |
| jeblair | clarkb, fungi, nibalizer: i'm running PS2 on a test trusty node now | 18:05 |
| *** _nadya_ has quit IRC | 18:05 | |
| fungi | jeblair: thanks! | 18:06 |
| *** dizquierdo has quit IRC | 18:06 | |
| *** MaxV has joined #openstack-infra | 18:08 | |
| *** teran has quit IRC | 18:09 | |
| *** bhunter71 has quit IRC | 18:09 | |
| yolanda | so jeblair, i've been looking at what you said about splitting ProviderManager in two, and sounds not too complicated, i'd like to give a try. Shall i create an spec or is ok to start with a change? | 18:10 |
| jeblair | yolanda: i think for that you can just make a change. i'd still rather not merge it if we can get the hpcloud indexes fixed. but if you're okay with the idea that we may not merge it if the indexes are fixed, it may make sense for you to start on it so we have a backup plan. | 18:11 |
| *** tkelsey has quit IRC | 18:12 | |
| *** AJaeger has joined #openstack-infra | 18:12 | |
| yolanda | jeblair, i will be ok. But even with indexes fixed, shall not it improve performance ? as it will be isolating different tasks, we will guarantee that poller tasks are run even if providers are slow | 18:13 |
| AJaeger | fungi, clarkb, jeblair: looking at https://etherpad.openstack.org/p/repo-renaming-2015-03-27 - one question: What about ninja-approving the gerrit review as well? | 18:13 |
| fungi | AJaeger: what gerrit review? | 18:14 |
| AJaeger | fungi, the ".gitreview" change | 18:14 |
| openstackgerrit | greghaynes proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 18:14 |
| AJaeger | doesn't this avoid problems with our bot jobs like proposal jobs? | 18:14 |
| fungi | AJaeger: those tend to be useful to test that the projects are renamed correctly | 18:14 |
| fungi | and that core review groups can still approve changes for them | 18:15 |
| *** hdd has joined #openstack-infra | 18:15 | |
| fungi | and that jobs run, et cetera | 18:15 |
| *** ssam2 has quit IRC | 18:15 | |
| AJaeger | but if they are wrong, the requirements job might fail... So, other idea: What about asking in your all-clear email to approve the .gitreview change quickly? | 18:16 |
| *** MaxV has quit IRC | 18:16 | |
| fungi | AJaeger: oh, right we probably need to rethink having check/gate jobs using .gitreview files in projects | 18:16 |
| jeblair | yolanda: rate limits make that complicated -- we will have to lower the rate limit of each of the providers in order for the overall rate limit not to exceed the actual limit. we will have to be conservative about that because the two providers won't be communicating with each other. | 18:16 |
| fungi | AJaeger: post/periodic jobs are fine to do that i think | 18:17 |
| jeblair | yolanda: (also, it makes the code more complex which would be good to avoid in nodepool if we can) | 18:17 |
| *** ptomka has quit IRC | 18:17 | |
| AJaeger | fungi: post/periodic are the only ones that fail if .gitreview is wrong/non-existant - AFAIK | 18:17 |
| greghaynes | flashgordon: I think https://review.openstack.org/164927 is good to go now | 18:17 |
| AJaeger | I don't think we invoke git review from infra anywhere else than post and periodic | 18:17 |
| fungi | AJaeger: yep, i agree that we should ask for them to be approved rapidly for that reason, and i'll make sure to include a list of them in the maintenance completion e-mail and keep an eye on them. i'll also link them all in the etherpad | 18:18 |
| mordred | Shrews: ^^ we shoudl think about rate-limits ina shade context too - as it relates to how nodepool is and needs to do them | 18:18 |
| *** Guest25866 has quit IRC | 18:19 | |
| AJaeger | fungi: Thanks! | 18:19 |
| flashgordon | greghaynes: woot! | 18:19 |
| mordred | Shrews: that being, for actions shade models that have more than one api - how do we model that in a way that doesn't cause complete insanity? | 18:19 |
| greghaynes | clarkb: https://review.openstack.org/#/c/165240/ | 18:19 |
| yolanda | jeblair, ah, i understand, so if we split poller tasks + creation tasks, then it's not as easy to track rate usage | 18:19 |
| mordred | s/one api/one api call/ | 18:19 |
| *** dhritishikhar_ has quit IRC | 18:19 | |
| yolanda | anyway, i think it's worth the effort to do a try and do an stress test there, so we can have a b plan if hpcloud doesn't improve | 18:20 |
| *** weshay has joined #openstack-infra | 18:20 | |
| mtreinish | anteaya: there are 3 project config patches on: https://etherpad.openstack.org/p/kilo-qa-code-sprint-nyc if you get a sec could you take a look? | 18:22 |
| *** afazekas has joined #openstack-infra | 18:23 | |
| jeblair | clarkb, fungi, nibalizer: i think that change will work well enough on the test nodes | 18:25 |
| clarkb | jeblair: ok, should we also test it on review-dev/jenkins-dev et al? | 18:25 |
| jeblair | clarkb, fungi, nibalizer: (my test run was not completely successful, but i think i missed a step before running puppet -- it worked well enough to create users and do quite a bit of other stuff) | 18:25 |
| openstackgerrit | greghaynes proposed openstack-infra/system-config: Add wheel_mirror_slaves that use pypi-mirror https://review.openstack.org/165240 | 18:26 |
| fungi | has anyone looked at node build errors today? i'm guessing hpcloud is basically offline for us again | 18:26 |
| fungi | i can dig in real quick if nobody has done so yet | 18:26 |
| jeblair | clarkb: yeah, i think we should do that. are you able to? | 18:26 |
| clarkb | jeblair: I am | 18:26 |
| *** esker has quit IRC | 18:26 | |
| jeblair | cool, thx | 18:26 |
| marun | anteaya: As part of the move of neutron api test development to the neutron repo, I'd like to make our api job voting. We already have a +2 and mestery, armax and HenryG have +1'd: https://review.openstack.org/#/c/164886/ | 18:26 |
| clarkb | fungi: I think nibalizer mentioned similar behavior fo rtheir tenant | 18:27 |
| clarkb | but that was IRL | 18:27 |
| marun | anteaya: oh, damn. It has a problem. | 18:27 |
| *** esker has joined #openstack-infra | 18:27 | |
| *** sdake_ has joined #openstack-infra | 18:27 | |
| *** sdake has quit IRC | 18:28 | |
| *** sdake_ has quit IRC | 18:28 | |
| AJaeger | marun: good catch, thanks! | 18:28 |
| *** sdake_ has joined #openstack-infra | 18:29 | |
| openstackgerrit | Maru Newby proposed openstack-infra/project-config: Configure neutron to gate on neutron-dsvm-api job https://review.openstack.org/164886 | 18:30 |
| openstackgerrit | greghaynes proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 18:30 |
| marun | anteaya, clarkb: fixed ^^ | 18:30 |
| nibalizer | fungi: yea at hpcloud we have errors in nodepool log | 18:31 |
| nibalizer | more successes than failures but yeaaa | 18:31 |
| clarkb | review-dev has puppet administratively disabled | 18:31 |
| clarkb | zaro: ^ is it unsafe for me to go around that? | 18:31 |
| nibalizer | jeblair: fungi clarkb also puppet 3.7.5 dropped yesterday I think... and i guess we'll be getting a 3.8 pretty soonish | 18:31 |
| jeblair | yaaaay! | 18:31 |
| *** dkehn has quit IRC | 18:31 | |
| fungi | 2015-03-27 18:31:10,771 ERROR nodepool.NodeLauncher: OverLimit launching node id: 1644841 in provider: hpcloud-b3 | 18:31 |
| jeblair | clarkb: i think it is probably unsafe | 18:31 |
| *** enikanorov has quit IRC | 18:32 | |
| *** gyee has quit IRC | 18:32 | |
| *** Sukhdev has joined #openstack-infra | 18:32 | |
| jeblair | clarkb: let's just do a jenkins or jenkins-dev. i think any server will do. | 18:32 |
| fungi | i'll look for cruft which could be embiggening this | 18:32 |
| *** ZZelle has joined #openstack-infra | 18:33 | |
| clarkb | jeblair: ok | 18:33 |
| *** ptomka has joined #openstack-infra | 18:33 | |
| *** MaxV has joined #openstack-infra | 18:34 | |
| *** dkehn has joined #openstack-infra | 18:34 | |
| fungi | clarkb: also jenkins02 is still offline for analysis i guess? | 18:34 |
| clarkb | fungi: ya, I haven't heard anything bac kfrom jenkins. they were all responsive when I didn't have any concrete data but as soon as I narrowed down the cause they have been quiet... | 18:34 |
| clarkb | http://paste.openstack.org/show/197209/ which looks ok, the changes there are due to puppet using my env | 18:34 |
| *** e0ne has joined #openstack-infra | 18:35 | |
| *** thingee has quit IRC | 18:35 | |
| clarkb | and my system-config repo has the change locally applied there | 18:36 |
| *** sdake has joined #openstack-infra | 18:36 | |
| clarkb | I think we are good to go | 18:36 |
| *** afazekas has quit IRC | 18:37 | |
| clarkb | fungi: if we are getting 50X errors from hpcloud again we may be leaking nodes pretty quickly | 18:39 |
| *** sdake_ has quit IRC | 18:39 | |
| fungi | clarkb: i suspect as much, which is what i'm trying to quantify | 18:39 |
| fungi | did we get the "ignore 5xx errors" change merged and nodepoold restarted with it? the past couple weeks has been a blur | 18:40 |
| *** annegentle has quit IRC | 18:40 | |
| clarkb | I dont think so, it didn't have any tests | 18:40 |
| clarkb | and appears to break existing tests | 18:41 |
| clarkb | ya | 18:41 |
| *** annegentle has joined #openstack-infra | 18:41 | |
| fungi | ahh, right, poop | 18:41 |
| *** MaxV has quit IRC | 18:42 | |
| *** smccully has joined #openstack-infra | 18:43 | |
| mordred | yeah - sorry - I didn't get a chance to go back and fix that patch yet | 18:43 |
| mordred | and it seemed less urgent at the time :( | 18:43 |
| fungi | 398 disassociated floating-ips in hpcloud. deleting those now | 18:44 |
| yolanda | fungi, are you experiencing floating ips not released properly in nodepool as well? | 18:45 |
| fungi | yolanda: from time to time. probably a symptom of something on the provider end | 18:46 |
| pleia2 | fungi: I don't know what that means yet for us, mind giving me a quick tutorial? | 18:46 |
| *** crc32 has quit IRC | 18:46 | |
| yolanda | ok, as we 've found that regularly as well | 18:46 |
| pleia2 | seems like a good friday thing to learn :) | 18:46 |
| fungi | pleia2: hpcloud (unlike rackspace) assigns rfc-1918 addresses to the "public" interfaces of instances, which prevents them from being globally routable. to make them reachable we also bind a "floating ip address" to each | 18:47 |
| jeblair | clarkb, fungi, mordred: i'm going to aprv 168433 unless you object | 18:47 |
| pleia2 | fungi: right, so this happens with VMs disappear but the floating addresses are still being shown as used? | 18:47 |
| fungi | pleia2: nodepool tries to track these fip assignments and delete them when deleting the associated instance | 18:47 |
| zaro | clarkb: go ahead, i must have forgotten to re-enable it. | 18:47 |
| *** ihrachyshka has quit IRC | 18:47 | |
| fungi | jeblair: go fot iy | 18:48 |
| pleia2 | fungi: aha | 18:48 |
| fungi | go for it too | 18:48 |
| mordred | jeblair: ++ | 18:48 |
| *** thingee has joined #openstack-infra | 18:48 | |
| jeblair | to late i already fot iy | 18:48 |
| fungi | jeblair: sounds like an inslut | 18:48 |
| timrc | We could just not use floating ips and have things move through a jump host :) I totally didn't think what I just said out at all though. | 18:48 |
| fungi | pleia2: anyway, yes sometimes we delete things in our providers and for whatever reason they don't delete (we may at times even get a confirmation back that the delete command was accepted) | 18:49 |
| *** med_ has joined #openstack-infra | 18:49 | |
| fungi | pleia2: so, for a variety of reasons which could either be corner cases in nodepool's resource state tracking or in the provider's nova/neutron or both, we end up with floating ips which aren't associated with any instance | 18:50 |
| pleia2 | fungi: lovely, `nova floating-ip-list` to investigate? | 18:50 |
| yolanda | fungi, and do you have some monitor/automation to clean that? | 18:50 |
| fungi | pleia2: and because floating ips are in relatively limited supply, we have a quota, which we can overrun | 18:50 |
| *** e0ne is now known as e0ne_ | 18:50 | |
| fungi | pleia2: yep | 18:50 |
| * pleia2 nods | 18:50 | |
| *** SumitNaiksatam has joined #openstack-infra | 18:50 | |
| *** MaxV has joined #openstack-infra | 18:50 | |
| pleia2 | yolanda: good question! | 18:50 |
| fungi | yolanda: we do not | 18:51 |
| yolanda | mm, not sure if that's on the scope of nodepol, or general monitoring | 18:51 |
| pleia2 | I don't think we have a monitor, but a script/command for cleanup? | 18:51 |
| pleia2 | for each... | 18:51 |
| pleia2 | :) | 18:51 |
| yolanda | pleia2, i ask you because we've done manual checks and cleanups as well :) | 18:51 |
| fungi | just manual client calls grepped for certain patterns, then filtered through more client calls to mop them up | 18:51 |
| * pleia2 nods | 18:51 | |
| *** sdake_ has joined #openstack-infra | 18:52 | |
| fungi | it tends to not happen unless something's going wrong somewhere | 18:52 |
| yolanda | fungi, and that's not unusual :) | 18:52 |
| pleia2 | fungi: maybe worthwhile to toss some notes into nodepool.rst for now? (I'm happy to do so) | 18:52 |
| jeblair | fungi, yolanda, pleia2, mordred: i _think_ we may be expecting that to get a little better with shade because it will use neutron directly | 18:53 |
| mordred | jeblair: this is true | 18:53 |
| cinerama | pleia2: oh cool, you're here. just wanted to make sure i have everything i need for the next little while | 18:53 |
| yolanda | ah cool, so it won't be relying on nova anymore? | 18:53 |
| fungi | pleia2: can't hurt. i just look at `nova floating-ip-list` output a few minutes apart, and anything with a - for the instance uuid which was in the first run and shows up again in the second is likely not going to get cleaned up automagically | 18:53 |
| mordred | yolanda: that's right - if the neutron endpoint is available, we'll use it instaed | 18:54 |
| yolanda | ah great, how is the status of the patchf or that? | 18:54 |
| mordred | it's my next week task | 18:54 |
| *** tsg has joined #openstack-infra | 18:55 | |
| mordred | I've been focused on the dib part this week | 18:55 |
| yolanda | i left you alone with that :( | 18:55 |
| *** xnox has quit IRC | 18:55 | |
| mordred | yolanda: it's all good - I've been having a good week of hacking :) | 18:55 |
| *** sdake has quit IRC | 18:55 | |
| timrc | So for hpcloud why not just rejig it to connect directly to the private vlan addresses through a jump host... it being jenkins or zuul or whatever. Then you only need one floating ip and it's always assigned. | 18:55 |
| *** e0ne_ has quit IRC | 18:55 | |
| timrc | I think I'm totally understating the easiness of doing that... but floating ips... bleh :) | 18:56 |
| timrc | s/understating/overstating/ | 18:56 |
| mordred | timrc: that would require probably as much rework of systems as the other things, and would add a bit rube-goldberg | 18:56 |
| timrc | mordred, Haha. | 18:56 |
| pleia2 | cinerama: I plan on doing another pass through zanata topic patches this afternoon, other thoughts? | 18:57 |
| *** tsg_ has joined #openstack-infra | 18:57 | |
| fungi | timrc: yes, we'd rather just have globally-routable ip addresses bound to the instance interfaces and be done with it actually. maybe if hpcloud ever gets ipv6 working we'll cease to care about hackish always-nat ipv4 craziness | 18:57 |
| timrc | fungi, Aye. | 18:57 |
| mordred | floating ips are the worst thing | 18:57 |
| timrc | mordred, Agreed. | 18:57 |
| cinerama | pleia2: so i have a big list of stuff here as possible...things...and i was wondering if we could go through them | 18:57 |
| timrc | I learned not to trust them while working at Canonical. | 18:58 |
| *** imcsk8 has quit IRC | 18:58 | |
| *** imcsk8 has joined #openstack-infra | 18:58 | |
| *** MaxV has quit IRC | 18:58 | |
| *** e0ne has joined #openstack-infra | 18:58 | |
| cinerama | pleia2: unless you are busy | 18:58 |
| mordred | timrc: it's like someone decided "hey, let's make these cloud computers behave liek someone's apartment" | 18:58 |
| yolanda | timrc, you were finding floating ips changed with someone else in Canonical? | 18:59 |
| pleia2 | cinerama: sounds good, give me like 5 minutes? | 18:59 |
| timrc | mordred, I don't like that the guest has no idea wtf it is. | 18:59 |
| yolanda | i had the funny surprise of entering into my host with floating ip and reaching someone elses host | 18:59 |
| *** tsg has quit IRC | 19:00 | |
| fungi | "funny" | 19:00 |
| mordred | timrc: neither do I | 19:00 |
| cinerama | pleia2, sure, ping me when you are ready | 19:00 |
| timrc | yolanda, Wat? | 19:01 |
| timrc | yolanda, You were prompted for a password I hope? | 19:01 |
| *** ptomka has quit IRC | 19:02 | |
| yolanda | yes, but i just got my job switched with someone else | 19:02 |
| *** xnox has joined #openstack-infra | 19:02 | |
| timrc | yolanda, I could never depend on my floating IPs at Canonical. I couldn't even depend on one being available... that's how oversubscribed they were. | 19:02 |
| yolanda | so you could enter into a browser and see complete different things than the ones that i had | 19:02 |
| *** e0ne is now known as e0ne_ | 19:02 | |
| *** MaxV has joined #openstack-infra | 19:02 | |
| yolanda | we had our own tests servers after that canonistack tries so i cannot complain after that | 19:03 |
| fungi | yeeowch 611 alien nodes in hpcloud | 19:06 |
| fungi | cleaning those up now too | 19:06 |
| *** e0ne_ is now known as e0ne | 19:07 | |
| *** alexpilotti has quit IRC | 19:07 | |
| pleia2 | cinerama: ok! have an etherpad or something to browse through, or..? | 19:07 |
| cinerama | pleia2: i was just going to haphazardly talk through things, but if you think that's better we can do that too | 19:08 |
| *** emagana has quit IRC | 19:08 | |
| pleia2 | cinerama: haphazard works for me | 19:08 |
| cinerama | pleia2: so basically i'm just going through the zanata docs, plus random stuff i thought of, to improve the module as we have it | 19:09 |
| * pleia2 nods | 19:09 | |
| *** MaxV has quit IRC | 19:10 | |
| cinerama | pleia2: so we can do some improvement on the existing stuff we have in place, like make the apache config a bit more...configurable | 19:11 |
| cinerama | pleia2: and obviously any cleanups you folks have in mind | 19:11 |
| openstackgerrit | Min Pae proposed openstack-infra/project-config: add cue-dashboard project to openstack-cue channel https://review.openstack.org/168455 | 19:11 |
| *** ujuc has quit IRC | 19:11 | |
| cinerama | pleia2: i also need to figure out what the management port does and whether i can just turn that off | 19:12 |
| *** HeOS has joined #openstack-infra | 19:12 | |
| timrc | yolanda, Lucky. | 19:13 |
| cinerama | pleia2: also wondering if we should try to pre-approve the admin accounts. right now we pre-list okayed username(s) for admin in the standalone.xml and then they have to login (via openid) with the correct username. they then get a magic email link | 19:13 |
| pleia2 | cinerama: yeah, so is this the admin port on wildfly, or zanata? | 19:13 |
| *** ujuc has joined #openstack-infra | 19:13 | |
| cinerama | pleia2: we can do sql magic to pre-approve the accounts but doing that in puppet would mean doing it via exec and that might be bad | 19:14 |
| *** melwitt has joined #openstack-infra | 19:14 | |
| *** tonytan4ever has joined #openstack-infra | 19:14 | |
| cinerama | pleia2: i think it's for wildfly but it's not emitting anything informative at the moment so i'm looking at the docs | 19:14 |
| cinerama | pleia2: obviously if we don't need the port open i'd rather not have it :) | 19:14 |
| cinerama | pleia2: what else....probably i need to try out StevenK's client stuff and add any puppet stuff we need to make sure the client can connect | 19:15 |
| pleia2 | cinerama: afaik we shouldn't need to mess with wildfly at all, so I think we keep the wildfly port that has a web interface firewalled off (aside from ssh, we block all by default, and just let specific ports through) | 19:16 |
| jeblair | cinerama, pleia2: just something to keep in the back of your minds -- the more admin stuff we can do via code review, the fewer actual admin accounts we need to have and the more like the rest of our systems it will be, and i think the more happy we will be | 19:16 |
| pleia2 | jeblair: ++ | 19:16 |
| *** MaxV has joined #openstack-infra | 19:17 | |
| pleia2 | we do manage admins and groups in gerrit manually, but it's not optimal | 19:17 |
| jeblair | obviously we may have to compromise on that to get something up and running but maybe can improve it over time | 19:17 |
| pleia2 | seems to make for a lot of silly work for us | 19:17 |
| jeblair | pleia2: yep. but we managed to make the acl changes and project creation, eventually, an automated code-review thing | 19:17 |
| cinerama | pleia2, jeblair: indeed. i haven't really played with what the admin powers actually do for us yet on the server. this also addresses zaro's question in that if we have a single non-openid admin account i believe we need the internal security module as well as the openid one then | 19:17 |
| jeblair | which got rid of lots of silly work we used to do | 19:17 |
| pleia2 | jeblair: yeah :) | 19:18 |
| jeblair | there was a time when _i_ created projects in gerrit | 19:18 |
| jeblair | i don't even know how to do that now | 19:18 |
| cinerama | pleia2, jeblair: also easier to manage things like joiners & leavers probably | 19:18 |
| pleia2 | yeah, audit trail is <3 | 19:18 |
| jeblair | cinerama: yep. we have _no_ automation around launchpad, and it's all just random whatever people remember to do. and it bitrots and... | 19:19 |
| cinerama | small group w/access, if one joins or leaves group who need it, just change passwd | 19:19 |
| yolanda | 19:19 | |
| pleia2 | cinerama: your intuition has been spot on and StevenK has a great grasp of the client, so between you, him and AJaeger I don't need to block on any of the client stuff | 19:19 |
| jeblair | obviously we have decided that the solution to that is not to create more automation around launchpad :) | 19:19 |
| cinerama | jeblair: i think we will be using openstackid here so launchpad is moot, but certainly similar issues | 19:20 |
| *** hodos|2 has joined #openstack-infra | 19:20 | |
| *** xnox has quit IRC | 19:20 | |
| cinerama | pleia2: while i'm thinking of it if you have other contacts that would be useful pass them on to me | 19:20 |
| pleia2 | cinerama: yeah, AJaeger does translations and has probably the best grasp of all our current scripts used for transifex (which are what will use the client), and StevenK has been picking that up on the Zanata side | 19:21 |
| pleia2 | so they're the two you can sync up with | 19:21 |
| sputnik13 | question... https://review.openstack.org/#/c/166033/ has 2x +2 and +1 workflow, but it's not being merged... what might be preventing this? | 19:21 |
| jeblair | cinerama: oh, i was just citing it as our last system that is run purely manually -- there's a whole messy structure of projects and groups and permissions just to have it manage our bugs | 19:21 |
| pleia2 | cinerama: clarkb also knows about translations scripts, but I know he's been very busy lately | 19:21 |
| jeblair | cinerama: it's not a shining example of our work :) | 19:21 |
| AJaeger | cinerama: yeah, for the current scripts feel free to ask me. I do bit of translation (10 strings every release or so), so I might answer questions or give you pointers ;) | 19:22 |
| jeblair | anyway, that was food for thought, now i need food for calories | 19:22 |
| cinerama | pleia2: my notes here are a bit vague and situationally dependent. but i have the feeling generally that we need to start playing inside zanata rather than around it and making sure we have what we need for folks to actually do work. then see if we can clean up extra stuff in the standalone that we don't want or need just to keep it simple | 19:22 |
| AJaeger | cinerama: if you have questions, feel free to email me - I'm Europe based and not always online. Use jaegerandi at gmail | 19:22 |
| cinerama | pleia2: then there are the cosmetic finishing touches like whether we can do a retheme on the web ui to make it openstacky and stuff like that | 19:22 |
| sdague | so... what's up with the nodepool delete backup? | 19:23 |
| pleia2 | cinerama: that makes sense to me, I had some trouble with my local.pp the other day when I was testing so we also need to make sure things brings up a working zanata right now | 19:23 |
| *** hodos has quit IRC | 19:23 | |
| zaro | sputnik13: you need someone to re-approve because of the depends-on. | 19:23 |
| *** ujuc has quit IRC | 19:23 | |
| cinerama | pleia2: would it be worth adding a sample local.pp or whatever to the module? what is the usual thing? | 19:24 |
| AJaeger | zaro, sputnik13 : recheck should work as well | 19:24 |
| sputnik13 | zaro: oh? | 19:24 |
| *** afazekas has joined #openstack-infra | 19:24 | |
| sputnik13 | AJaeger: ok I will try that | 19:24 |
| AJaeger | sputnik13: It's different repos and the second patch was approved before the first one | 19:24 |
| pleia2 | cinerama: you could propose a patch to system-config for zanata-dev.pp in the openstack_project module that has the contents of our former local.pp | 19:24 |
| AJaeger | We do not handle that | 19:24 |
| zaro | AJaeger: no, unfortunately not because recheck won't send the correct event to zuul | 19:24 |
| flashgordon | so the -postgres job | 19:24 |
| sputnik13 | AJaeger: ah ok, so in instances like that just request a recheck and it kicks off the flow again? | 19:24 |
| pleia2 | cinerama: actually, s/zanata-dev/translations-dev | 19:25 |
| pleia2 | .pp | 19:25 |
| *** mwagner_lap has joined #openstack-infra | 19:25 | |
| pleia2 | application named servers bad :) | 19:25 |
| flashgordon | I want to drop it from the integrated gate, since making sure postgres works is not nearly as important as making sure multi node works etc. | 19:25 |
| AJaeger | zaro: I think it would work since recheck starts from scratch. But asking for a system-config core to reapprove would be better ;) | 19:25 |
| cinerama | pleia2: yeah that was another idea. i might WIP it though cause we are probably not ready to deploy on a 'real' system that isn't safely tucked away in a padded vagrant cell | 19:25 |
| pleia2 | cinerama: deployment from our .pp files isn't automatic anyway, so it's safe | 19:26 |
| zaro | AJaeger: recheck won't work, we need this gerrit change to get it to work: https://gerrit-review.googlesource.com/#/c/65530/ | 19:26 |
| pleia2 | cinerama: one of us has to launch a server and attach the manifest to it | 19:26 |
| sdague | or I guess more importantly the fact that 70% of nodes in nodepool are in building state | 19:26 |
| cinerama | pleia2: oh ok. this has been an awesome way to learn infra puppet from the ground up | 19:27 |
| zaro | AJaeger, sputnik13: recheck won't send vote information because there's no vote transistion. | 19:27 |
| cinerama | pleia2: with plenty of -1s on the way :) | 19:27 |
| cinerama | pleia2: oh what else. i think the other thing is how much we care about doing upgrade without manual intervention | 19:27 |
| pleia2 | cinerama: yeah, I'm glad I had this project to make you do^W^W^W^Whelp you learn infra with | 19:27 |
| flashgordon | sdague: that does sounds really bad | 19:28 |
| pleia2 | cinerama: alas, upgrades will have to be manual, just like gerrit and jenkins | 19:28 |
| cinerama | pleia2: i am kind of leery of upgrading anything with a database & etc etc on the fly | 19:28 |
| pleia2 | cinerama: we'll start with doing an upgrade on a translations-dev server, and then find a strategy for production | 19:28 |
| AJaeger | zaro: Does it really need a vote information? Doesn't it check after passing the check queue for all precondietions including the depends-on and then will work? | 19:29 |
| cinerama | pleia2: that is probably best...also if we nuke the cached war/jar files, the module should DTRT | 19:29 |
| *** vigneshvar has joined #openstack-infra | 19:29 | |
| pleia2 | cinerama: this is why I hate these java things :) | 19:29 |
| AJaeger | zaro: I tried a recheck in the past and AFAIR it worked - but can't point out a patch, so might be I remember wrongly | 19:29 |
| *** vigneshvar_ has joined #openstack-infra | 19:29 | |
| *** gokrokve has quit IRC | 19:30 | |
| *** gokrokve has joined #openstack-infra | 19:31 | |
| cinerama | pleia2: wait until i start suggesting we use bugzilla..you will be running back to the java mountains of madness | 19:31 |
| zaro | AJaeger: i believe so. the gate pipeline in layout.yaml seems to say it does | 19:31 |
| *** gokrokve has quit IRC | 19:31 | |
| *** afazekas has quit IRC | 19:32 | |
| pleia2 | cinerama: the ansible email I forwarded you from cmunoz suggested signing up for the redhat bugzilla for zanata, so I did have to do that x_x | 19:32 |
| pleia2 | cinerama: *using* bugzilla is bad enough! | 19:32 |
| *** weshay has quit IRC | 19:34 | |
| cinerama | pleia2: i have used bugzilla, developed customizations for it, *and* ported them between major releases. we have a very special relationship :) | 19:34 |
| fungi | sdague: if nobody's answered you yet, hpcloud is basically unusable for us again | 19:35 |
| *** andreykurilin__ has joined #openstack-infra | 19:35 | |
| mordred | cinerama: oh my | 19:35 |
| *** andreykurilin_ has quit IRC | 19:35 | |
| sdague | fungi: no one has, bummer | 19:35 |
| sdague | any idea why? | 19:35 |
| fungi | sdague: i'm cleaning up massive piles of cruft in it now, but chances are that's only going to help me determine whether the cruft comes back quickly vs instantly | 19:36 |
| fungi | sdague: best guess is lots of 5xx errors from the api again | 19:36 |
| sdague | gotcha | 19:36 |
| cinerama | mordred: perhaps there is some kind of 12 step group for that | 19:36 |
| mordred | fungi: I'll work on finishing that "treat 5xx errors as retry requests" patch this weekend | 19:36 |
| flashgordon | fungi: is it all from HP? | 19:36 |
| mordred | cinerama: I bet you could start one | 19:36 |
| fungi | sdague: once i get enough cruft deleted so that we're not getting overlimit errors instead, i'll have a better idea of what and how quickly | 19:36 |
| fungi | flashgordon: yep | 19:36 |
| *** devlaps has quit IRC | 19:36 | |
| sdague | fungi: ok | 19:37 |
| sdague | mordred: any idea if the team actually looked at the db optimization? | 19:37 |
| mordred | flashgordon, sdague: we have this fun symptom where nova will 5xx on a request, but it'll get actioned, and eventually we'll get an active node, except we never knew about it because the create got a 5xx | 19:37 |
| mordred | sdague: it's complicated | 19:37 |
| mordred | sdague: but I can say that conversations are ongoing | 19:37 |
| sdague | mordred: ok | 19:38 |
| pleia2 | cinerama: brave soul :) | 19:39 |
| *** angela-s has joined #openstack-infra | 19:39 | |
| *** aisha_7337 has joined #openstack-infra | 19:39 | |
| cinerama | might have to settle for some 12yr laphroaig | 19:39 |
| aisha_7337 | hi..just a question about jenkins-job-builder... i updated JJB to latest version and i see even for testing my jobs , i need to pass creds file (.ini) ..why is that so? even for testing jobs , why need to pass the creds file... | 19:40 |
| *** dustins has quit IRC | 19:41 | |
| aisha_7337 | this is sad and frustrating.. | 19:41 |
| fungi | cinerama: this week i'm enjoying 14-year balvenie aged in rum casks. it's good | 19:41 |
| *** emagana has joined #openstack-infra | 19:41 | |
| aisha_7337 | like i cant simply run **jenkins-jobs test <jobs-folder> ** ...i have to do **jenkins-jobs conf <cred-file> test <jobs-dir>** .... | 19:42 |
| aisha_7337 | weird | 19:42 |
| cinerama | fungi: oh yum. starting to turn to margarita weather here though. | 19:42 |
| flashgordon | mordred: yeah I looked at that with fungi a few days ago and it looks like something in front of nova is failing | 19:42 |
| *** leakypipes has quit IRC | 19:42 | |
| flashgordon | mordred: what happened with booting with a server tag | 19:43 |
| mordred | flashgordon: nothing | 19:43 |
| flashgordon | and periodically checking for things with that tag that you don't know about and killing 'em | 19:43 |
| fungi | agreed. i'm unconvinced it's the openstack api endpoint itself. entirely likely hpcloud have some sort of proxy in front ti "protect" the endpoint | 19:43 |
| flashgordon | mordred: nova boot --meta | 19:43 |
| cinerama | pleia2: anyway just to wrap up i think i will get a proposal going to system config and play around more with The Actual Application rather than install/config puppet issues | 19:44 |
| pleia2 | cinerama: perfect :) | 19:44 |
| fungi | mordred: we're getting 502 and 503 errors with no analogue in the source. of course i guess hpcloud may also have modified source there | 19:44 |
| cinerama | pleia2: also with making sure the client can do the things | 19:44 |
| flashgordon | mordred: that way you can differentiate between nodepool booted servers and hand booted ones | 19:44 |
| flashgordon | so you don't kill the wrong thing | 19:44 |
| fungi | flashgordon: well, and more importantly between nodepool-booted servers and nodepool-booted servers from another nodepool | 19:45 |
| fungi | flashgordon: since some deployers very well may share a tenant with multiple nodepool daemons | 19:45 |
| pleia2 | cinerama: yeah, I think StevenK is doing most of the work there, so just check in to see where he is on it, it's so much fun dealing with a 4 person team that's EU, US, AU ;D | 19:45 |
| *** emagana has quit IRC | 19:46 | |
| cinerama | pleia2: probably just means i should get up earlier and have a siesta | 19:46 |
| flashgordon | fungi: yup! | 19:46 |
| mordred | fungi: when we say that ... do we know that's a thing that happens? | 19:46 |
| fungi | pleia2: at least they're all on the same planet! (i think?) | 19:46 |
| mordred | fungi: or we just know that there is nothing to prevent it? | 19:46 |
| mordred | fungi: hpcloud also may have an API proxy service in front of it | 19:47 |
| fungi | mordred: i'd like to be able to run nodepool-dev.o.o without a dedicated tenant for it. i don't know for sure that anybody does it, but also seems likely useful | 19:47 |
| pleia2 | cinerama: StevenK is around during our afternoons (well, not today, because he's in the future), he usually syncs up with AJaeger when I'm sleeping and I should read backscroll but I don't, I sync up with AJaeger in our morning | 19:47 |
| *** yamamoto has joined #openstack-infra | 19:47 | |
| AJaeger | pleia2: we haven't done really much syncing yet, I just pointed stevenK in the right direction | 19:48 |
| pleia2 | cinerama: I have had to do 1AM translations teams meetings, but I try to avoid makin them a regular thing :) | 19:48 |
| pleia2 | AJaeger: yeah, and that's appreciated! | 19:48 |
| fungi | and yes, StevenK is definitely from the future. you can tell by his fashion sense | 19:48 |
| pleia2 | hehe | 19:49 |
| cinerama | i thought we were all going to wear spandex jumpsuits | 19:49 |
| fungi | he's like an extra out of bladerunner or something | 19:49 |
| flashgordon | fungi: using the --meta stuff should be a fairly minor change to implement if you want to go that route | 19:51 |
| flashgordon | figure out what the key and value should be | 19:51 |
| flashgordon | in the delete logic, scan the full list of servers for rogue servers with the key value | 19:51 |
| fungi | flashgordon: yes and no. it's a piece to a bigger solution, which is some periodic task to find and delete the lost nodes | 19:51 |
| mordred | the meta part is not the hard bit | 19:51 |
| fungi | right, that second part | 19:51 |
| flashgordon | fungi: ahh, and no periodic task like that is in place | 19:51 |
| flashgordon | gotcha | 19:52 |
| *** xnox has joined #openstack-infra | 19:52 | |
| *** yamamoto has quit IRC | 19:52 | |
| *** mpaolino has joined #openstack-infra | 19:52 | |
| *** dkliban is now known as dkliban_brb | 19:52 | |
| fungi | i mean, the metadata is probably just going to be a random string with a name in the nodepool.yaml so that the nodepoold can identify its own nodes. getting that injected at boot time will be the trivial part | 19:53 |
| *** MaxV has quit IRC | 19:53 | |
| clarkb | pleia2: cinerama just got back from lunch, I am happy to ansewr questions about translations scripts | 19:55 |
| cinerama | clarkb: cool. i haven't quite gotten to that bit yet but good to know | 19:56 |
| *** andreykurilin__ has quit IRC | 19:57 | |
| *** xnox has quit IRC | 19:57 | |
| flashgordon | fungi: if you get that in can you run a periodic thing manually to clean it out | 19:58 |
| flashgordon | or are you able to do that now? | 19:58 |
| fungi | flashgordon: we can basically already do that manually without the metadata | 19:58 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Fixed import https://review.openstack.org/168467 | 19:59 |
| flashgordon | fungi: ahh | 19:59 |
| flashgordon | fungi: how many zombies are there at the moment btw? | 19:59 |
| *** mpaolino has quit IRC | 19:59 | |
| * fungi is in the process of monitoring that as it runs right now in fact | 19:59 | |
| fungi | flashgordon: 611 in hplcoud a little while ago when i started deleting | 19:59 |
| *** gokrokve has joined #openstack-infra | 19:59 | |
| flashgordon | wow | 20:00 |
| flashgordon | that is really aweful | 20:00 |
| fungi | also 398 disassociated floating ips | 20:00 |
| flashgordon | wowwww | 20:00 |
| *** e0ne is now known as e0ne_ | 20:00 | |
| *** e0ne_ is now known as e0ne | 20:00 | |
| flashgordon | any HP instances actually working? | 20:00 |
| fungi | yeah, we end up with a bit of a mess when the provider starts misbehaving. nodepool can cope with a lot, but it's still mostly designed around some assumptions about api calls behaving in deterministic ways | 20:01 |
| *** emagana has joined #openstack-infra | 20:01 | |
| flashgordon | turns out with multiple clouds that isn't always true I guess | 20:01 |
| fungi | flashgordon: zero nodes in hpcloud in a used state right now. all are either building or delete | 20:02 |
| *** xnox has joined #openstack-infra | 20:02 | |
| fungi | flashgordon: basically that constant-width blue stripe on the test nodes graph is our rackspace quota | 20:03 |
| flashgordon | lolz | 20:03 |
| flashgordon | this explains why things are soo backed up all the time | 20:03 |
| flashgordon | over half the quota is gone at any given time | 20:03 |
| fungi | it's surprising we've got only a 3.5 hour delay in the check pipeline, but i guess we can chalk that up to it being friday | 20:04 |
| flashgordon | and post K3 | 20:04 |
| fungi | yeah | 20:04 |
| mordred | flashgordon: getting a "give me a server and a floating-ip" create parameter that's similar to the "give me a server and a volume" would clean up a non-trivial amount of issues | 20:05 |
| mordred | fwiw | 20:05 |
| *** baoli has quit IRC | 20:05 | |
| mordred | flashgordon: like block_device_mapping_v2 | 20:06 |
| *** notnownikki has quit IRC | 20:06 | |
| mordred | "make me an ip and also make it go away when the server goes away" is _exceptionally_ hard | 20:06 |
| timrc | sigh | 20:07 |
| mordred | but all the semantics we'd want are described for volumes in block_device_mapping_V2 | 20:07 |
| *** sputnik13 has quit IRC | 20:07 | |
| *** afazekas has joined #openstack-infra | 20:07 | |
| *** baoli has joined #openstack-infra | 20:07 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Use gate64 cpu type during multinode testing https://review.openstack.org/168408 | 20:08 |
| clarkb | flashgordon: ^ I am pretty hopeful about that now | 20:08 |
| *** achanda has quit IRC | 20:08 | |
| flashgordon | fungi mordred: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Run_Instance_Idempotency.html | 20:09 |
| flashgordon | AFAIK nova supports that | 20:09 |
| flashgordon | mtreinish: 6 | 20:09 |
| flashgordon | mtreinish: ^ | 20:09 |
| flashgordon | mtreinish: ^* | 20:10 |
| mtreinish | flashgordon: what is that supposed to mean :) | 20:10 |
| mordred | flashgordon: yes. the things they say there are easy | 20:10 |
| flashgordon | mordred: ++ | 20:10 |
| mordred | flashgordon: none of those are the hard thing | 20:10 |
| flashgordon | mordred: more pointing that out as as an interesting side note | 20:11 |
| mordred | flashgordon: http://docs.rackspace.com/servers/api/v2/cs-devguide/content/one_step_create_vol_boot.html is what I'm talking about - except for floating ips and not for block devices | 20:11 |
| mordred | flashgordon: nod | 20:12 |
| flashgordon | sc68cal: ^ | 20:12 |
| *** achanda has joined #openstack-infra | 20:12 | |
| andreaf | fungi: what are you seeing in nodepool? 5xx on delete? | 20:13 |
| mordred | andreaf: on create, actually | 20:13 |
| mordred | andreaf: the difficult part is that we see 5xx on create ... and then later on the create succeeds | 20:14 |
| mordred | andreaf: except that we do not have a handle for that server, because it 5xx'd on the create | 20:14 |
| fungi | (after nodepoold has given up on any expectation that could actually have not been a fatal error) | 20:14 |
| mordred | andreaf: so it is effectively a leak - since we don't have a context for the server | 20:14 |
| flashgordon | mordred: and no request ID to right? | 20:14 |
| clarkb | flashgordon: right, it appears to not be nova's fault | 20:14 |
| clarkb | its whatever is in front of nova | 20:14 |
| andreaf | mordred: ok I see | 20:15 |
| mordred | andreaf: I've got a patch I need to finish to start assuming that 5xx's are lies and that we should check back later | 20:15 |
| *** ibiris is now known as ibiris_away | 20:15 | |
| mordred | andreaf: https://review.openstack.org/#/c/166383/ | 20:15 |
| mtreinish | mordred: sounds safe :) | 20:15 |
| andreaf | mordred: I though you had a patch for the cloud | 20:15 |
| flashgordon | Deal with failures that succeed | 20:15 |
| mordred | andreaf: they won't let me land patches to the cloud | 20:15 |
| mordred | andreaf: :( | 20:15 |
| flashgordon | sounds like the title of a motivational speech | 20:15 |
| fungi | mordred: though as flashgordon and clarkb point out, if this is actually the behavior of some random network proxy hpcloud has implemented and not the openstack api endpoint itself, working around that might be mostly self-serving | 20:16 |
| mordred | I disagree | 20:16 |
| fungi | but as long as it's really just extra checking, i guess it can't hurt | 20:16 |
| *** ociuhandu has quit IRC | 20:16 | |
| mordred | both of our clouds are not really fully openstack and a large portion of our automation is to deal with their weirdness | 20:16 |
| mordred | this is another weirdness we've discovered in the wild | 20:16 |
| mordred | we can deal with it, understand it, and work around it | 20:17 |
| fungi | it's potentially an instance of us encoding a random non-openstack behavior into our tools because of a deployment decision on the part of one provider | 20:17 |
| fungi | but yeah, that's acceptable i think | 20:17 |
| *** mjturek1 has joined #openstack-infra | 20:17 | |
| flashgordon | fungi: I think proxies in front of APIs is somewhat expected no? | 20:17 |
| fungi | and we do certainly have plenty of precedent for that | 20:17 |
| fungi | flashgordon: it is, but this particular proxy and it's specific failure mode within that one environment might be a globally unique situation | 20:18 |
| *** timcline has quit IRC | 20:19 | |
| *** emagana has quit IRC | 20:19 | |
| *** timcline has joined #openstack-infra | 20:20 | |
| sdague | mordred: are you fixing the test fails? | 20:20 |
| flashgordon | fungi: very possible | 20:20 |
| mordred | sdague: I will - although it'll likely be tomorrow when I do | 20:20 |
| mordred | if someone else wants to adopt that and fix it before then, I will not be offended | 20:20 |
| *** baoli has quit IRC | 20:21 | |
| sdague | mordred: ok | 20:21 |
| clarkb | tests on what? | 20:22 |
| clarkb | btw sdague flashgordon the thing that was likely causing trouble with the cpu model is that restart libvirt-bin was not sufficient | 20:22 |
| mordred | clarkb: the nodepool 5xx patch | 20:22 |
| clarkb | you have to stop start it | 20:22 |
| sdague | clarkb: oh, interesting | 20:22 |
| clarkb | one of those things that you spend far too much time debugging and its a silly init problem | 20:22 |
| clarkb | sdague: so I expect the latest patch to work, we shall see | 20:23 |
| sdague | I almost wonder if we should go full crazy pants and instead install it as a deb with a trigger | 20:23 |
| clarkb | sdague: the updated file? | 20:24 |
| *** lucas-beer has quit IRC | 20:24 | |
| sdague | the updater script | 20:24 |
| sdague | so on any libvirt package change it makes sure it's there | 20:24 |
| sdague | it's probably overkill, but was a random friday crazy pants idea | 20:24 |
| clarkb | one thing that would simplify this is if nova let us pass through feature adds and removals | 20:25 |
| clarkb | but its after featur efreeze so not helpful right now | 20:25 |
| flashgordon | clarkb: doh | 20:25 |
| sdague | clarkb: also, it's probably a ton more complicated than that | 20:26 |
| sdague | because I'm not sure what that pass mechanism is to libvirt | 20:26 |
| clarkb | sdague: I think you can put <feature> tags inside the <cpu> block but this is all new to me | 20:27 |
| jeblair | fungi, mordred: know why the monasca cookbooks are going to the stackforge attic? | 20:27 |
| clarkb | sdague: kashyap and danpb may want to weigh in on that, but if this is the issue we can point at concreteness | 20:27 |
| sdague | clarkb: how would you specify it to nova? | 20:27 |
| mordred | jeblair: I do not | 20:28 |
| jeblair | " | 20:28 |
| jeblair | Monasca has moved to using Ansible instead of Chef so the | 20:28 |
| jeblair | cookbook-monasca-* projects are no longer being maintained." | 20:28 |
| jeblair | https://review.openstack.org/#/c/167387/ | 20:28 |
| jeblair | interesting | 20:28 |
| * fungi was about to paste that commit message | 20:28 | |
| jeblair | fungi: you probably would have done a better job of it too :) | 20:29 |
| clarkb | sdague: as a list of +feature,-feature flags? | 20:29 |
| clarkb | sdague: then nova could xmlify that? | 20:29 |
| sdague | to what? | 20:29 |
| clarkb | sdague: the nova config | 20:29 |
| clarkb | sdague: the same place that takes the cpu model | 20:29 |
| fungi | jeblair: probably. my paste button lights up in multiple colors, after all | 20:29 |
| sdague | clarkb: gotcha | 20:29 |
| * jeblair starts installing more lights on his model m | 20:30 | |
| clarkb | jeblair: sparkfun has led legos | 20:30 |
| clarkb | jeblair: I am pretty sur that all led installations must not be led lego pieces | 20:30 |
| sdague | clarkb: you want to post a spec for what you'd want / expect there? | 20:30 |
| fungi | well, that's my 3" trackball and arcade buttons, though my ducky keyboard does have all the keycaps individually illuminated too | 20:31 |
| *** hashar has joined #openstack-infra | 20:31 | |
| fungi | and cherry mx blue mechanical switches for a nice satisfying click | 20:31 |
| cinerama | i need a model m again | 20:32 |
| * pleia2 has the blues too | 20:32 | |
| fungi | cinerama: i just found one over the holidays when visiting my parents in fact | 20:32 |
| *** Sukhdev has quit IRC | 20:32 | |
| sdague | yeh, feature enable / disable pass through probably would be fine as part of the model fine tuning, looking at the libvirt docs now | 20:32 |
| fungi | cinerama: need to clean it up, but it was just in a pile of old ibm hardware in their library | 20:32 |
| fungi | saw it and was, like, you planning to use this? O_o | 20:33 |
| openstackgerrit | Joe Gordon proposed openstack-infra/project-config: Drop neutron-pg in favor of a voting neutron-pg-full https://review.openstack.org/168478 | 20:33 |
| cinerama | fungi: lucky! i do have an old-timey 3 button optical mouse but last time i tried to get it going with the db9->usb thingy it didn't work super good | 20:33 |
| clarkb | sdague: ya I can write one up | 20:33 |
| cinerama | daskeyboard is here at least | 20:33 |
| clarkb | though I am currently also trying to figre out why mordreds nodepool chnage fails so hard in tests | 20:33 |
| *** dkliban_brb is now known as dkliban | 20:34 | |
| clarkb | I think it may be that the fakes are not sufficient | 20:36 |
| anteaya | sdake_: I missed this last night I guess, my fault, but you lost your experimental job on magnum: https://review.openstack.org/#/c/167848/8/zuul/layout.yaml | 20:38 |
| anteaya | sdake_: did you want to keep that? | 20:38 |
| sdake_ | I think that was dims_ that added that | 20:38 |
| sdake_ | I guess he can add it back :) | 20:39 |
| sdake_ | The patch isn't in merge conflict? | 20:39 |
| sdake_ | dims ?? | 20:39 |
| *** e0ne is now known as e0ne_ | 20:40 | |
| openstackgerrit | Merged openstack-infra/project-config: Add docs jobs for openstack-infra/openstackid repo https://review.openstack.org/165661 | 20:40 |
| *** e0ne has joined #openstack-infra | 20:41 | |
| *** achanda has quit IRC | 20:43 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Add fallback bindep manifest https://review.openstack.org/164447 | 20:43 |
| openstackgerrit | Joe Gordon proposed openstack-infra/project-config: Drop neutron-pg in favor of a voting neutron-pg-full https://review.openstack.org/168478 | 20:45 |
| clarkb | nope fakes have server ids and mark nodes active | 20:45 |
| *** Somay has quit IRC | 20:45 | |
| *** tiswanso has quit IRC | 20:45 | |
| mordred | clarkb: as well they should :) | 20:46 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Add methods for logical router management https://review.openstack.org/167371 | 20:47 |
| *** afazekas has quit IRC | 20:47 | |
| Shrews | mordred: i know you already +2'd that ^^^ but i figured we should actually test stuff | 20:47 |
| mordred | Shrews: it looked god | 20:48 |
| mordred | good | 20:48 |
| Shrews | mordred: yup. just added some unit tests | 20:48 |
| anteaya | sdake_: no, nothing is wrong with teh patch | 20:48 |
| anteaya | sdake_: you can add that back after the move, no problem | 20:48 |
| anteaya | sdake_: just sharing that I noticed it now and missed it last night | 20:49 |
| anteaya | fresh eyes and all that | 20:49 |
| sdake_ | cool | 20:49 |
| sdake_ | i'll let dims know | 20:49 |
| openstackgerrit | Andrew Boik proposed openstack-infra/devstack-gate: WIP: (experiment) Test dual-stack with dependency https://review.openstack.org/168063 | 20:49 |
| *** emagana has joined #openstack-infra | 20:50 | |
| fungi | do we want to send out a one-hour warning notice for the maintenance window via statusbot in about 10 minutes? | 20:50 |
| fungi | status notice Gerrit maintenance commences in 1 hour at 22:00 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html | 20:50 |
| clarkb | sdague: flashgordon well tests are failing so maybe that wasn't the only thing | 20:50 |
| *** achanda has joined #openstack-infra | 20:50 | |
| *** otter768 has quit IRC | 20:50 | |
| anteaya | sdake_: thanks | 20:51 |
| jeblair | fungi: lgtm | 20:51 |
| anteaya | fungi: sounds good | 20:52 |
| sdake_ | anteaya thank you :) | 20:52 |
| anteaya | :) | 20:52 |
| timrc | nodepool gave me a good laugh today... | 20:53 |
| timrc | chavezti@supersecrethostname:~$ sudo nodepool list -h | 20:53 |
| timrc | usage: nodepool list [-h] | 20:53 |
| fungi | sometimes argparse can be a bit literal | 20:54 |
| *** emagana has quit IRC | 20:54 | |
| timrc | That command needs someone to add bells and whistles to it :) | 20:55 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Add methods for logical router management https://review.openstack.org/167371 | 20:56 |
| *** mfisch has quit IRC | 20:56 | |
| *** e0ne is now known as e0ne_ | 20:57 | |
| *** Guest90957 has joined #openstack-infra | 20:58 | |
| *** e0ne_ is now known as e0ne | 20:58 | |
| clarkb | I think I know why mordred's patch is failing | 20:58 |
| mordred | clarkb: what did I do? | 20:58 |
| fungi | oh, found a short/fuzzy video of the illuminated keyboard and trackball http://fungi.yuggoth.org/tmp/trackball.mp4 (the cardboard mounting panel for the trackball is temporary while i finish the tray i'm building out of chopped-up 12" disk platters) | 20:58 |
| mordred | timrc: that's the best | 20:59 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Deal with failures that succeed https://review.openstack.org/166383 | 20:59 |
| fungi | #status notice Gerrit maintenance commences in 1 hour at 22:00 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html | 21:00 |
| openstackstatus | fungi: sending notice | 21:00 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Add methods for logical router management https://review.openstack.org/167371 | 21:00 |
| clarkb | mordred: ^ server returned by waitForServer is a dict not a novaclient server object | 21:00 |
| clarkb | mordred: why? I do not know | 21:00 |
| fungi | it's probably just to keep you on your toes | 21:00 |
| mordred | clarkb: wow | 21:00 |
| -openstackstatus- NOTICE: Gerrit maintenance commences in 1 hour at 22:00 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html | 21:00 | |
| openstackstatus | fungi: finished sending notice | 21:02 |
| *** sputnik13 has joined #openstack-infra | 21:02 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/project-config: Drop neutron-pg in favor of a voting neutron-pg-full https://review.openstack.org/168478 | 21:03 |
| openstackgerrit | Joe Gordon proposed openstack-infra/project-config: Move neutron-pg-full to periodic https://review.openstack.org/168486 | 21:03 |
| jeblair | fungi: how about i turn the project-config changes into a stack and rebase them for ease of gerrit merging? | 21:03 |
| *** amitgandhinz has quit IRC | 21:04 | |
| fungi | jeblair: that sounds great. i'll hold off final review for a few minutes | 21:04 |
| fungi | hopefully there aren't many conflicts, but with this many project moves/renames there are bound to be some | 21:05 |
| pleia2 | anteaya: fyi, haven't heard back from tristanC, but I've updated https://wiki.openstack.org/wiki/Governance#Elections and the voter eligibility window for both elections on their respective wiki pages | 21:05 |
| pleia2 | fungi: ^^ | 21:05 |
| fungi | pleia2: great | 21:05 |
| jeblair | fungi: neat, there already are some. :) this will be extra helpful then. | 21:05 |
| *** dprince has quit IRC | 21:06 | |
| *** _nadya_ has joined #openstack-infra | 21:06 | |
| fungi | jeblair: ahh, yeah at least one was already claiming a merge conflict | 21:06 |
| *** achanda has quit IRC | 21:07 | |
| anteaya | pleia2: good call | 21:07 |
| anteaya | pleia2: good to try to include tristanC | 21:07 |
| anteaya | but i'm getting questions | 21:07 |
| pleia2 | he has a whole pile of PMs to come back to :) | 21:07 |
| anteaya | so accuracy is important | 21:07 |
| *** peristeri has quit IRC | 21:08 | |
| * pleia2 nods | 21:08 | |
| anteaya | yup | 21:08 |
| *** rkukura has quit IRC | 21:08 | |
| *** aisha_7337 has quit IRC | 21:08 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Move Gnocchi from Stackforge to OpenStack https://review.openstack.org/162146 | 21:09 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Rename stackforge/bindep -> openstack-infra/bindep https://review.openstack.org/160963 | 21:09 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Renaming stackforge/xstatic-angular-irdragndrop https://review.openstack.org/167798 | 21:09 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack/magnum https://review.openstack.org/167848 | 21:09 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Move coobook-monasca-* to stackforge-attic https://review.openstack.org/167387 | 21:09 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Rename fuel-tasks-validator to fuel-tasklib https://review.openstack.org/162952 | 21:09 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Move os-client-config from Stackforge to OpenStack https://review.openstack.org/167213 | 21:09 |
| jeblair | 2 of those had conflicts, so a double check would be good. | 21:09 |
| fungi | pleia2: based on what you've got there, looks like i'll generate the voter rolls early in the day on april 9 for everything | 21:09 |
| pleia2 | fungi: yep! | 21:09 |
| fungi | jeblair: awesome. reviewing them all now | 21:09 |
| * anteaya also reviews the stack | 21:10 | |
| *** _nadya_ has quit IRC | 21:10 | |
| *** annegentle has quit IRC | 21:10 | |
| *** timcline has quit IRC | 21:11 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Added commandline commands to package.json https://review.openstack.org/168488 | 21:11 |
| *** marcusvrn has quit IRC | 21:12 | |
| *** gyee has joined #openstack-infra | 21:14 | |
| *** ldnunes has quit IRC | 21:14 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Removed grunt:server https://review.openstack.org/163165 | 21:15 |
| *** flaper87 has quit IRC | 21:15 | |
| openstackgerrit | Merged openstack-infra/system-config: Realize users in template.pp https://review.openstack.org/168433 | 21:16 |
| anteaya | jeblair: did gnocchi lose some gate jobs in zuul/layout.yaml? | 21:16 |
| fungi | i'm going to start building a replacement image to see if that's ^ doing what our manual tests indicate it should | 21:16 |
| anteaya | jeblair: it just has bashate in the gate on my diff | 21:16 |
| *** vigneshvar_ has quit IRC | 21:16 | |
| *** vigneshvar has quit IRC | 21:16 | |
| anteaya | I'm expecting its 3 functional jobs as well, am I wrong? | 21:17 |
| jeblair | anteaya: anteaya you are correct | 21:17 |
| jeblair | anteaya: i focused on the docs jobs (they changed rtfd to docs.o.o which i think is expected) but missed the gate change | 21:18 |
| *** baoli has joined #openstack-infra | 21:18 | |
| anteaya | okay | 21:18 |
| *** e0ne has quit IRC | 21:18 | |
| jeblair | wasn't my rebase that did it though... looking through patchset history | 21:18 |
| *** flaper87 has joined #openstack-infra | 21:18 | |
| *** flaper87 has quit IRC | 21:18 | |
| *** flaper87 has joined #openstack-infra | 21:18 | |
| jeblair | it was ps7 -> 8 where they wer elost | 21:18 |
| openstackgerrit | Timothy R. Chavez proposed openstack-infra/shade: Add 'rebuild' to shade https://review.openstack.org/161067 | 21:19 |
| sputnik13 | hello, can someone with +2 for system-config please re-approve this for us? https://review.openstack.org/#/c/166033/ | 21:19 |
| jeblair | anteaya: i see no comments about that, so i bet it was an accident | 21:19 |
| jeblair | anteaya: i think you should -1 and we should add it back | 21:19 |
| jeblair | jd__: ping if you are around | 21:19 |
| anteaya | jeblair: ah okay | 21:19 |
| sputnik13 | ^ already has 2x +2 and +1 workflow | 21:19 |
| anteaya | I will do so | 21:20 |
| jd__ | jeblair: pong | 21:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Use gate64 cpu type during multinode testing https://review.openstack.org/168408 | 21:20 |
| sputnik13 | but the dependent patch get approved after it, so it's stuck | 21:20 |
| fungi | sputnik13: doesn't look like it's stuck | 21:20 |
| fungi | sputnik13: check http://status.openstack.org/zuul/ | 21:20 |
| jeblair | jd__: can you confirm anteaya's comment in 162146 ? | 21:20 |
| sputnik13 | fungi: waaa, it's been 2 hours | 21:21 |
| sputnik13 | wow | 21:21 |
| sputnik13 | ok, I'll wait for that to complete | 21:21 |
| sputnik13 | someone was saying earlier a recheck won't actually resolve this | 21:21 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Move os-client-config to OpenStack section https://review.openstack.org/168490 | 21:21 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Set custom cpu_model for live_migrate https://review.openstack.org/141530 | 21:21 |
| openstackgerrit | Julien Danjou proposed openstack-infra/project-config: Move Gnocchi from Stackforge to OpenStack https://review.openstack.org/162146 | 21:22 |
| jd__ | jeblair: yes, fixed | 21:22 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add CI jobs for fuel-tasklib https://review.openstack.org/167539 | 21:22 |
| anteaya | that removed the bindep dependency | 21:22 |
| anteaya | jeblair: let me know when you get them restacked | 21:22 |
| anteaya | jd__: thanks | 21:22 |
| jeblair | yeah, i'll restack them | 21:23 |
| anteaya | brb | 21:23 |
| *** rkukura has joined #openstack-infra | 21:24 | |
| *** dkehn has quit IRC | 21:24 | |
| *** AJaeger has quit IRC | 21:25 | |
| *** chris6132 has quit IRC | 21:25 | |
| *** achanda has joined #openstack-infra | 21:27 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Enable live block migration https://review.openstack.org/135768 | 21:27 |
| *** doug-fish has left #openstack-infra | 21:28 | |
| *** kgiusti has left #openstack-infra | 21:30 | |
| *** ildikov has quit IRC | 21:32 | |
| *** saper_ is now known as saper | 21:33 | |
| *** saper has quit IRC | 21:33 | |
| *** saper has joined #openstack-infra | 21:33 | |
| fungi | i assume when 2200 rolls around we're going to dump the check and gate pipelines and then requeue them all once we bring zuul back online. massive backlog for a friday afternoon | 21:35 |
| *** baoli has quit IRC | 21:36 | |
| *** baoli has joined #openstack-infra | 21:36 | |
| anteaya | why do we have so many nodes in build status, is that covered in the backscroll and I missed it? | 21:36 |
| *** angela-s has left #openstack-infra | 21:36 | |
| fungi | anteaya: the blue stripe is our rackspace quota. hpcloud is effectively offline for us | 21:37 |
| fungi | has been all day | 21:37 |
| anteaya | ah | 21:37 |
| anteaya | :( | 21:37 |
| *** tonytan4ever has quit IRC | 21:38 | |
| fungi | i'm still slowly deleting alien nodes in hpcloud, but i fully expect by the time i finish there will be a bunch of new ones and still no usable workers there | 21:38 |
| *** tonytan4ever has joined #openstack-infra | 21:38 | |
| *** ddieterly has quit IRC | 21:38 | |
| anteaya | boo | 21:39 |
| anteaya | have they asked us to shut ourselves off from them yet? | 21:39 |
| anteaya | jeblair: did you rebase and openstackgerrit missed it? | 21:39 |
| jeblair | anteaya: not yet | 21:40 |
| anteaya | ah okay | 21:40 |
| anteaya | 20 minutes left | 21:40 |
| jeblair | anteaya: i'll make it! | 21:40 |
| anteaya | you will, was hoping for another pass myself | 21:41 |
| *** jcoufal has quit IRC | 21:41 | |
| jeblair | i'm still reviewing the first stack | 21:41 |
| anteaya | ah | 21:41 |
| *** marcusvrn has joined #openstack-infra | 21:42 | |
| anteaya | there really isn't anything I can do to help regarding the etherpad tasks, is there? | 21:42 |
| *** mrmartin has quit IRC | 21:42 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack/magnum https://review.openstack.org/167848 | 21:42 |
| jeblair | that's a commit message change ^ | 21:42 |
| *** dimtruck is now known as zz_dimtruck | 21:42 | |
| *** stevemar has quit IRC | 21:42 | |
| *** tonytan4ever has quit IRC | 21:42 | |
| anteaya | good | 21:43 |
| *** mattfarina has quit IRC | 21:44 | |
| mordred | jeblair: you're a commit message change face | 21:45 |
| openstackgerrit | K Jonathan Harker proposed stackforge/gertty: Add is:watched to p_is_term() https://review.openstack.org/168495 | 21:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Rename stackforge/murano to openstack/murano https://review.openstack.org/167934 | 21:45 |
| fungi | same defect, it seems | 21:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Move Gnocchi from Stackforge to OpenStack https://review.openstack.org/162146 | 21:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Rename stackforge/murano to openstack/murano https://review.openstack.org/167934 | 21:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Renaming stackforge/xstatic-angular-irdragndrop https://review.openstack.org/167798 | 21:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Rename stackforge/magnum -> openstack/magnum https://review.openstack.org/167848 | 21:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Move coobook-monasca-* to stackforge-attic https://review.openstack.org/167387 | 21:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Rename fuel-tasks-validator to fuel-tasklib https://review.openstack.org/162952 | 21:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Move os-client-config from Stackforge to OpenStack https://review.openstack.org/167213 | 21:46 |
| clarkb | I have etherpad open now, though am mostly distracted by portland things | 21:46 |
| clarkb | anything I can do to help? | 21:46 |
| mordred | clarkb: is portland just portlanding? | 21:47 |
| jeblair | (i am really glad i have the thread view in gertty right now) | 21:47 |
| greghaynes | portlant gonna portland | 21:47 |
| greghaynes | portland, even | 21:47 |
| anteaya | clarkb: jeblair created a stack we are reviewing it https://review.openstack.org/#/c/160963/4 | 21:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Move os-client-config to OpenStack section https://review.openstack.org/168490 | 21:47 |
| anteaya | clarkb: lots of room for you if you want | 21:47 |
| jeblair | so... | 21:49 |
| jeblair | i have a -1 on 167934 | 21:49 |
| *** yamamoto has joined #openstack-infra | 21:49 | |
| *** aysyd has quit IRC | 21:49 | |
| jeblair | left comment | 21:51 |
| fungi | is it something we need the author's input on, or can we fix it ourselves? | 21:51 |
| fungi | ahh, you just posted it | 21:51 |
| pleia2 | also, should they still publish docs-on-rtfd ? | 21:52 |
| jeblair | probably not | 21:52 |
| pleia2 | I'll leave a comment | 21:52 |
| jeblair | though that's easy to fix after the rename | 21:52 |
| fungi | serg_mel isn't in here apparently | 21:52 |
| *** tnovacik has quit IRC | 21:53 | |
| jeblair | this was only posted yesterday | 21:53 |
| jeblair | we could excutive decide to go with option two and create a new group for the other project and if they feel that it is part of the project, move it in a later rename. | 21:53 |
| fungi | i'm fine with that | 21:54 |
| *** teran has joined #openstack-infra | 21:54 | |
| fungi | it's not like whatever we do now is etched in stone | 21:54 |
| *** yamamoto has quit IRC | 21:54 | |
| jeblair | (i realize this may sound like a minor point, but i'm actually just trying to reflect reality -- either this is one project or two, but the current patch muddies the water) | 21:54 |
| pleia2 | oh no, the i to l one is kind of funny and sad (fonts!) | 21:54 |
| fungi | hrm, there's another problem with it though | 21:54 |
| *** chris613 has joined #openstack-infra | 21:55 | |
| fungi | they're moving murano-docs to stackforge-attic it seems | 21:55 |
| *** ociuhandu has joined #openstack-infra | 21:55 | |
| fungi | which is unrelated to their projects moving into openstack namespace | 21:55 |
| *** ChuckC has quit IRC | 21:56 | |
| jeblair | fungi: should they not do that? | 21:56 |
| *** mriedem has quit IRC | 21:56 | |
| fungi | well, they can. except it needs a read-only acl in that case | 21:56 |
| jeblair | (i do not understand the implications there, which is a red flag but i can't tell if it's a blocker) | 21:56 |
| *** dkehn has joined #openstack-infra | 21:57 | |
| fungi | and also i'll need to add it to the etherpad since i was only expecting them to rename the projects mentioned in the governance change | 21:57 |
| fungi | that's on me of course | 21:57 |
| jeblair | fungi: i think we should declare we're out of time to fix this and bump murano to the next one | 21:57 |
| fungi | agreed. i'll strip them from the etherpad commandsets | 21:58 |
| *** ajmiller_ has joined #openstack-infra | 22:00 | |
| * mordred is stuck on a phone call ... will attempt to get off before this is completed | 22:00 | |
| jeblair | when zuul is down, i will stop nodepool and delete all nodes to help us have enough nodes to finish the work | 22:00 |
| fungi | okay, everything related to the murano change is removed from https://etherpad.openstack.org/p/repo-renaming-2015-03-27 now | 22:01 |
| *** sambetts has quit IRC | 22:01 | |
| anteaya | +1 bump | 22:02 |
| *** ajmiller has quit IRC | 22:02 | |
| fungi | status alert Gerrit is offline for maintenance, ETA 22:30 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html | 22:02 |
| fungi | look good? | 22:02 |
| jeblair | fungi: lgtm | 22:02 |
| fungi | #status alert Gerrit is offline for maintenance, ETA 22:30 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html | 22:02 |
| openstackstatus | fungi: sending alert | 22:02 |
| anteaya | also I noted the attic change has a bots file change that has nothing to do with the rename | 22:02 |
| anteaya | it doesn't hurt anything, just unrelated | 22:02 |
| jeblair | anteaya: yeah, agreed | 22:02 |
| anteaya | I +2'd with a comment | 22:03 |
| fungi | yeah, there are several things that need cleaning up on the murano change, so best to save for another window | 22:03 |
| jeblair | it was mentioned in the commit though, so at least intentional | 22:03 |
| fungi | oh, by attic change you mean the cookbooks | 22:03 |
| anteaya | true | 22:03 |
| anteaya | fungi: yes the cookbooks | 22:03 |
| jeblair | fungi, pleia2: zuul is stopped | 22:03 |
| anteaya | their bot file change has nothing to do with the attci | 22:03 |
| anteaya | attic | 22:03 |
| *** dboik_ has quit IRC | 22:03 | |
| anteaya | tell me what i can do to help | 22:03 |
| fungi | i'' go ahead and stop gerrit now | 22:04 |
| -openstackstatus- NOTICE: Gerrit is offline for maintenance, ETA 22:30 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html | 22:04 | |
| *** ChanServ changes topic to "Gerrit is offline for maintenance, ETA 22:30 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html" | 22:04 | |
| *** annegentle has joined #openstack-infra | 22:04 | |
| jeblair | fungi: have you stopped puppetbaster ansnible? | 22:04 |
| *** smccully has quit IRC | 22:04 | |
| *** sambetts has joined #openstack-infra | 22:04 | |
| *** Swami has quit IRC | 22:04 | |
| nibalizer | not sure if typo | 22:05 |
| nibalizer | puppetblaster ansible, it is named | 22:05 |
| clarkb | should I do the github tasks? I can do those despite distractions locally | 22:05 |
| jeblair | nibalizer: it is now | 22:05 |
| crinkle | ansnibble | 22:05 |
| nibalizer | or puppetbaster is good too | 22:05 |
| anteaya | let's leave channel room for the work | 22:05 |
| nibalizer | kk | 22:05 |
| fungi | yeah, puppetmaster is stopped and no run_all going | 22:05 |
| anteaya | thanks | 22:05 |
| clarkb | of course github is being DDoS'd | 22:06 |
| jeblair | nodepool is mass-deleting. or, you know, trying. | 22:06 |
| jeblair | not our fault! | 22:06 |
| clarkb | but I will do my best with github | 22:06 |
| openstackstatus | fungi: finished sending alert | 22:06 |
| fungi | i'm grabbing the other local gerrit mods | 22:06 |
| anteaya | clarkb: thanks | 22:06 |
| fungi | starting on gerrit db changes now | 22:07 |
| clarkb | I am logged into github and starting the renames first | 22:07 |
| * anteaya nods | 22:07 | |
| anteaya | pleia2: do you have a task? or want one? | 22:07 |
| pleia2 | anteaya: I think I'm going to just follow along in the pad+channel this time | 22:08 |
| anteaya | pleia2: okay good plan | 22:08 |
| *** dimsum__ has quit IRC | 22:09 | |
| * clarkb confirms that the xstatic repo is just getting a case change of one letter? | 22:09 | |
| anteaya | clarkb: that is what I see | 22:09 |
| clarkb | oh thats an l not an I | 22:09 |
| anteaya | from i to l | 22:09 |
| clarkb | ya ok | 22:09 |
| jeblair | clarkb: incorrect | 22:09 |
| jeblair | anteaya: right | 22:10 |
| clarkb | ya the etherpad font makes it look like i to I but its an i to l I copy pastad to get it right | 22:10 |
| jeblair | old and now are both lowercase. the letter is changing from "i" (eye) to "l" (ell) | 22:10 |
| clarkb | yup | 22:10 |
| *** dkehn has quit IRC | 22:10 | |
| clarkb | the readme also confused me, in any case its correct now so yay | 22:10 |
| anteaya | yay | 22:11 |
| jeblair | droid sans is "mostly" okay, right? :) | 22:11 |
| clarkb | jeblair: mostly | 22:11 |
| clarkb | droid mono gets this right but no O and 0 | 22:11 |
| clarkb | maybe? anyways enough font derping off to transfers | 22:11 |
| jeblair | probably best to just copy/paste whatever thai tran submitted :) | 22:12 |
| clarkb | and we decided that gerrit should be added to the owner list of the transferred repos right? | 22:12 |
| anteaya | looks like fungi is done with gerritdb | 22:12 |
| clarkb | or the access list | 22:13 |
| fungi | lucene reindexing underway | 22:13 |
| jeblair | clarkb: isn't it already? | 22:13 |
| anteaya | who wants to do step 8? | 22:13 |
| fungi | anybody working on the git server farm? | 22:13 |
| clarkb | jeblair: ya, its just a thing github makes you confirm during the move | 22:14 |
| jeblair | clarkb: oh new to me | 22:14 |
| jeblair | fungi: nak | 22:14 |
| fungi | clarkb: yeah, keep gerrit in the owner list | 22:14 |
| fungi | i'll go ahead and work on the 5 git servers | 22:15 |
| anteaya | fungi: thanks | 22:15 |
| fungi | lucene reindex is ~half finished | 22:17 |
| anteaya | fungi: do we need clark to finish (after reindexing is done) before restarting gerrit? | 22:18 |
| fungi | yeah, but there's time | 22:18 |
| anteaya | kk | 22:18 |
| mordred | ok. off phone. anything left for me to do? | 22:18 |
| jeblair | mordred: ansible step 16 | 22:18 |
| fungi | right, that should be safe to do now | 22:18 |
| *** pc_m has quit IRC | 22:18 | |
| mordred | jeblair: I hae that ability | 22:18 |
| *** ayoung has quit IRC | 22:19 | |
| jeblair | it's not in the critical section to minimize downtime, but can be done now. | 22:19 |
| jeblair | also, rax-dfw is not having a great day today either; we have quite a few servers stuck in delete there for several hours | 22:20 |
| anteaya | :( | 22:20 |
| *** smccully has joined #openstack-infra | 22:20 | |
| anteaya | unhappy clouds today | 22:20 |
| fungi | i guess i'll also take approving the config changes (14) since i have -2 on most of them i need to reverse anyway | 22:20 |
| *** ChuckC has joined #openstack-infra | 22:20 | |
| mordred | done | 22:20 |
| *** dimsum__ has joined #openstack-infra | 22:20 | |
| anteaya | mordred: thanks | 22:20 |
| anteaya | fungi: nod | 22:20 |
| fungi | reindex almost complete | 22:20 |
| anteaya | great | 22:21 |
| fungi | looks like clarkb's finishing just in time | 22:21 |
| anteaya | clarkb is almost done | 22:21 |
| anteaya | yup | 22:21 |
| anteaya | so once reindex is finished we can restart gerrit? | 22:21 |
| anteaya | clarkb: is done | 22:21 |
| jeblair | get a clear from me before restarting gerrit | 22:21 |
| clarkb | ya I am done | 22:21 |
| fungi | jeblair: will do | 22:21 |
| anteaya | jeblair: very good | 22:21 |
| anteaya | clarkb: thank you | 22:21 |
| fungi | reindex just finished, so awaiting jeblair | 22:22 |
| *** mdenny has quit IRC | 22:22 | |
| anteaya | clarkb: can you put done beside your name for step 9, please? | 22:22 |
| anteaya | it keeps the etherpad tidy | 22:22 |
| jeblair | fungi: okay to proceed | 22:23 |
| fungi | gerrit's on its way up now | 22:23 |
| *** dkehn has joined #openstack-infra | 22:23 | |
| jeblair | (i wanted to verify nodepool had all node types available) | 22:23 |
| fungi | great idea | 22:23 |
| fungi | gerrit looks, good, so okay to proceed starting zuul | 22:23 |
| jeblair | starting zuul | 22:23 |
| jeblair | fungi: i will delay re-enqueuing so you can approve the config changes... | 22:24 |
| jeblair | fungi: zuul is up, you can do that now | 22:24 |
| fungi | on it | 22:25 |
| *** ChuckC has quit IRC | 22:25 | |
| *** ChuckC has joined #openstack-infra | 22:26 | |
| *** thedodd has quit IRC | 22:27 | |
| *** dkehn has quit IRC | 22:27 | |
| sputnik13 | fungi: yeah the recheck seems to have not worked | 22:28 |
| *** ajmiller has joined #openstack-infra | 22:29 | |
| sputnik13 | https://review.openstack.org/#/c/166033/ <-- is still "stuck" on "merge if necessary" despite having +1 wf | 22:29 |
| fungi | sputnik13: it will, we're in the process of a maintenance | 22:29 |
| sputnik13 | ahhh ok | 22:29 |
| sputnik13 | :-D | 22:29 |
| *** ajmiller has quit IRC | 22:29 | |
| jeblair | re-enqueing | 22:29 |
| fungi | sputnik13: we've had gerrit/zuul offline so we could perform a slew of project renames | 22:29 |
| fungi | anyway, i had to recheck the 8 approved config changes since they were all uploaded too soon before zuul going offline to have any check results, so approving them wasn't workingh | 22:30 |
| *** ajmiller_ has quit IRC | 22:30 | |
| anteaya | ah | 22:30 |
| jeblair | fungi: oh, i thought you were just doing it backwards. | 22:30 |
| fungi | they would have gotten reenqueued with jeblair's dump, but we want them to go in first anyway | 22:31 |
| *** achanda_ has joined #openstack-infra | 22:31 | |
| anteaya | yes | 22:31 |
| *** erlon has quit IRC | 22:31 | |
| jeblair | fungi: can probably status ok now | 22:32 |
| fungi | #status ok | 22:32 |
| openstackstatus | fungi: sending ok | 22:32 |
| fungi | yep | 22:32 |
| anteaya | can steps 12, 14 and 15 be done in any order? | 22:32 |
| anteaya | well 14 is happening now | 22:32 |
| jeblair | 14 must complete before 15 | 22:32 |
| fungi | i'm doing 12 now while we wait | 22:33 |
| *** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/manual/ http://ci.openstack.org/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/" | 22:33 | |
| jeblair | 12 must also complete before 15 | 22:33 |
| anteaya | kk | 22:33 |
| jeblair | 12 and 14 are not dependent as long as they both happen before 15 | 22:33 |
| *** sabeen1 has quit IRC | 22:33 | |
| anteaya | great thank you | 22:33 |
| openstackstatus | fungi: finished sending ok | 22:34 |
| *** bswartz has quit IRC | 22:34 | |
| *** achanda has quit IRC | 22:34 | |
| jeblair | anteaya: the logic is basically that we need to complete things that jeepyb might get wrong before it is run again (which appens with step 15) | 22:34 |
| anteaya | ah | 22:34 |
| anteaya | thank you | 22:34 |
| *** nelsnelson has quit IRC | 22:35 | |
| fungi | yep, don't want manage_projects creating new repos or new groups or... | 22:35 |
| anteaya | yeah, that wouldn't help | 22:35 |
| anteaya | makes sense | 22:35 |
| fungi | except for the ones we intend (for example it's going to add bindep-release for us) | 22:35 |
| anteaya | right | 22:35 |
| *** achanda_ has quit IRC | 22:36 | |
| *** jcoufal has joined #openstack-infra | 22:38 | |
| jeblair | i will leave another message on 167934 | 22:38 |
| *** baoli has quit IRC | 22:38 | |
| *** markvoelker has quit IRC | 22:40 | |
| anteaya | might take a bit before step 14 is complete | 22:41 |
| jeblair | yep | 22:42 |
| fungi | i'm taking this opportunity to quickly stuff my face but will start puppet back up once the rename changes merge | 22:42 |
| *** gordc has quit IRC | 22:43 | |
| openstackgerrit | Merged openstack-infra/project-config: Rename stackforge/bindep -> openstack-infra/bindep https://review.openstack.org/160963 | 22:43 |
| *** baoli has joined #openstack-infra | 22:43 | |
| jeblair | fungi: if you're not around, someone else can get it :) | 22:43 |
| fungi | i plan to be around anyway to upload the .gitreview updates so i can include the urls in the completion e-mail | 22:44 |
| anteaya | k | 22:45 |
| anteaya | happy stuffing | 22:45 |
| openstackgerrit | Merged openstack-infra/project-config: Move Gnocchi from Stackforge to OpenStack https://review.openstack.org/162146 | 22:45 |
| *** patrickeast has quit IRC | 22:46 | |
| clarkb | here come the merges | 22:46 |
| anteaya | yay | 22:46 |
| * mordred images merges running down the street | 22:46 | |
| fungi | not really... they trickling in as we get sufficient nodes to test them | 22:46 |
| * mordred imagines merges slowly sauntering aimlessly down the street | 22:47 | |
| fungi | or perhaps staggering drunkenly | 22:47 |
| asselin | does nodepool do any kind of load balancing between providers? | 22:47 |
| jeblair | asselin: yes. it tries to balance everything! | 22:48 |
| asselin | jeblair, ok thanks | 22:48 |
| *** thingee has quit IRC | 22:49 | |
| anteaya | I imagine them as 20 somethings crawling in the grass in the darkness trying to figure out where the next clue is in the city wide scavenger hunt | 22:50 |
| jeblair | anteaya: are you talking about portland? | 22:51 |
| *** otter768 has joined #openstack-infra | 22:51 | |
| *** signed8b_ is now known as signed8bit_ZZZzz | 22:51 | |
| pleia2 | bwahaha | 22:51 |
| anteaya | actually san antonio | 22:51 |
| clarkb | jeblair: not sure we crawl. its more like stumbling after the vast quantities of beer we hvae | 22:51 |
| anteaya | but I can imagine it would happen in portland too | 22:52 |
| anteaya | was walking home along the river walk and saw all these people with flashlights crawling in the grass | 22:52 |
| anteaya | figured they lost a dog or something | 22:52 |
| anteaya | it is a club that does scavenger hunts | 22:52 |
| jeblair | either a dog or an earring | 22:52 |
| anteaya | that's what I thought | 22:52 |
| anteaya | precious earring to bring out crowds like that | 22:53 |
| anteaya | 4 in check, 2 in the gate | 22:55 |
| * anteaya makes tea | 22:55 | |
| *** dkliban is now known as dkliban_afk | 22:55 | |
| *** otter768 has quit IRC | 22:55 | |
| sigmavirus24 | anteaya++ | 22:56 |
| anteaya | actually I have to collect sap, the sun is going down, then tea | 22:56 |
| *** baoli has quit IRC | 22:57 | |
| pleia2 | like, from a tree? | 22:57 |
| pleia2 | canada <3 | 22:57 |
| *** nelsnelson has joined #openstack-infra | 22:58 | |
| *** gyee has quit IRC | 22:58 | |
| openstackgerrit | James E. Blair proposed stackforge/gertty: Sync starred changes regardless of subscription https://review.openstack.org/168508 | 22:58 |
| jeblair | i thought that was going to take longer | 22:59 |
| fungi | that's another feature i'll definitely use | 22:59 |
| *** baoli has joined #openstack-infra | 23:00 | |
| *** salv-orl_ has quit IRC | 23:01 | |
| *** hashar has quit IRC | 23:02 | |
| *** tnovacik has joined #openstack-infra | 23:02 | |
| *** salv-orlando has joined #openstack-infra | 23:03 | |
| *** Somay has joined #openstack-infra | 23:04 | |
| fungi | i think that top project-config change in the gate is going to fail on a centos6 git timeout https://jenkins01.openstack.org/job/gate-infra-puppet-apply-centos6/745/console | 23:05 |
| fungi | i'll see if i can snipe the worker so it'll get requeued and another assigned quickly | 23:05 |
| jeblair | fungi: ooh, thanks | 23:06 |
| clarkb | fungi is the node sniper | 23:07 |
| fungi | i nova-deleted the instance out from under it | 23:07 |
| fungi | jenkins should notice momentarily | 23:08 |
| fungi | i wonder what the actual jenkins slave agent heartbeat timeout is | 23:11 |
| clarkb | fungi: ya I typically kill the job in jenkins directly | 23:11 |
| clarkb | not sure how fast a node going away will be notice | 23:11 |
| anteaya | pleia2: yes | 23:11 |
| fungi | oh, will that work too without failing the job? i'll just do that | 23:12 |
| *** marun has quit IRC | 23:12 | |
| fungi | nope, that causes it to get aborted | 23:12 |
| fungi | unfortunately | 23:12 |
| anteaya | :( | 23:13 |
| anteaya | now we know | 23:13 |
| fungi | so i'll reenqueue these | 23:13 |
| clarkb | oh ya, I kill it then rerun | 23:13 |
| *** baoli has quit IRC | 23:13 | |
| openstackgerrit | James E. Blair proposed stackforge/gertty: Add checkout and cherry-pick to change list https://review.openstack.org/168510 | 23:13 |
| fungi | i was trying to avoid that since getting new workers is taking a really, really long time | 23:13 |
| clarkb | the node went away now rerun detection is something I try not t orely on as it seems to be an exceptional case and not expected behavior at least for jenkins | 23:13 |
| jeblair | fungi, clarkb: ^ that's a change i wish i had _before_ today's maint | 23:14 |
| *** nelsnelson has quit IRC | 23:15 | |
| fungi | okay, the 6 that were already in the gate are back in there now | 23:16 |
| clarkb | \o/ finally got live migration running with 64bit guests | 23:16 |
| anteaya | clarkb: yay | 23:16 |
| *** annegentle has quit IRC | 23:19 | |
| *** dangers is now known as dangers_away | 23:21 | |
| *** ajmiller has joined #openstack-infra | 23:23 | |
| openstackgerrit | Merged openstack-infra/project-config: Rename fuel-tasks-validator to fuel-tasklib https://review.openstack.org/162952 | 23:27 |
| *** cebruns has joined #openstack-infra | 23:33 | |
| *** bswartz has joined #openstack-infra | 23:33 | |
| openstackgerrit | Merged stackforge/gertty: Expand sample keymaps to ameliorate OSX features https://review.openstack.org/157488 | 23:33 |
| asselin | clarkb, congrats! | 23:34 |
| openstackgerrit | Merged openstack-infra/project-config: Move os-client-config from Stackforge to OpenStack https://review.openstack.org/167213 | 23:35 |
| clarkb | the tests fail but live migration works | 23:35 |
| jeblair | oops, the os-client-config config file section move change failed (it was not based on the end of the stack, so the file changed from under it) | 23:36 |
| jeblair | that's not particularly urgent though | 23:36 |
| jeblair | (also, yay gate; this was not caught in check :) | 23:36 |
| dimsum__ | looks like we are in the middle of the renames.. | 23:36 |
| *** timcline has joined #openstack-infra | 23:37 | |
| jeblair | dimsum__: the maintenance is over except for the renamed projects; there's some more work to do for them | 23:37 |
| dimsum__ | thanks jeblair | 23:38 |
| *** markvoelker has joined #openstack-infra | 23:41 | |
| *** ajmiller has quit IRC | 23:41 | |
| *** markvoelker has quit IRC | 23:46 | |
| *** tnovacik has quit IRC | 23:47 | |
| *** nelsnelson has joined #openstack-infra | 23:47 | |
| *** andreykurilin_ has joined #openstack-infra | 23:47 | |
| openstackgerrit | Merged openstack-infra/project-config: Move coobook-monasca-* to stackforge-attic https://review.openstack.org/167387 | 23:48 |
| *** signed8bit_ZZZzz is now known as signed8b_ | 23:49 | |
| *** btran has quit IRC | 23:50 | |
| *** ociuhandu has quit IRC | 23:52 | |
| *** dannywilson has quit IRC | 23:53 | |
| jeblair | we're waiting on bare-precise nodes and those are only being attempted in hpcloud :/ | 23:55 |
| anteaya | :( | 23:56 |
| *** wenlock has quit IRC | 23:56 | |
| *** spzala has quit IRC | 23:57 | |
| pleia2 | I need to run out to the pharmacy before they close, bbiab | 23:59 |
| fungi | and as previously, we still actually have no nodes in use in hpcloud | 23:59 |
| fungi | all are either building or delete | 23:59 |
| *** dimsum__ has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!