| fungi | jeblair: sdague: first i forced replication, then noticed that several of the git servers had no head at all (similar to what we've seen sometimes on zuul) so... now what did i do? | 00:03 |
|---|---|---|
| * fungi checks his shell history | 00:03 | |
| sdague | fungi: so we think some of the jenkins slaves had stale bad trees | 00:03 |
| sdague | that were unrecoverable | 00:03 |
| fungi | ahh, that seems likely | 00:04 |
| sdague | so hopefully it was really just that one slave | 00:04 |
| fungi | because yeah, what i did after forcing replication was noticed not missing heads but entire missing repos because puppet was failing to complete/update jeepyb because setuptools/pip was broken due to trying to upgrade itself, so i embarked on fixing that on all the git servers | 00:05 |
| fungi | what is what spurred mordred on to the new changes in review/post on the -infra ml which i still need to respond to | 00:06 |
| fungi | er, which is what | 00:07 |
| fungi | anyway, since reverifying sdague's failing change worked, i assumed i'd fixed whatever the issue was. guess not :/ | 00:07 |
| *** fifieldt_ has joined #openstack-infra | 00:24 | |
| *** sarob has joined #openstack-infra | 00:25 | |
| *** senk has joined #openstack-infra | 00:26 | |
| *** sarob has quit IRC | 00:29 | |
| *** david_h has joined #openstack-infra | 00:33 | |
| *** mriedem has joined #openstack-infra | 00:36 | |
| *** david_h has quit IRC | 00:52 | |
| *** dcramer_ has joined #openstack-infra | 00:59 | |
| *** sarob has joined #openstack-infra | 01:09 | |
| *** yaguang has joined #openstack-infra | 01:12 | |
| *** sarob has quit IRC | 01:14 | |
| *** che-arne has quit IRC | 01:22 | |
| openstackgerrit | Qiu Yu proposed a change to openstack-infra/pypi-mirror: Print help message and exit if no config file by default https://review.openstack.org/51501 | 01:26 |
| *** guohliu has joined #openstack-infra | 01:51 | |
| *** jhesketh has quit IRC | 01:53 | |
| *** jhesketh has joined #openstack-infra | 01:54 | |
| *** sileht has quit IRC | 02:05 | |
| *** sdake has joined #openstack-infra | 02:08 | |
| *** sileht has joined #openstack-infra | 02:10 | |
| *** dims has joined #openstack-infra | 02:25 | |
| *** markmcclain has joined #openstack-infra | 02:34 | |
| *** rcleere has joined #openstack-infra | 02:35 | |
| *** rcleere has quit IRC | 02:37 | |
| *** dhouck_ has joined #openstack-infra | 02:49 | |
| *** jtzl has quit IRC | 02:57 | |
| *** jtzl has joined #openstack-infra | 02:57 | |
| *** david_h has joined #openstack-infra | 03:00 | |
| *** dims has quit IRC | 03:03 | |
| *** harlowja_at_home has joined #openstack-infra | 03:05 | |
| harlowja_at_home | does anyone know if the etherpad switchover worked out? one of my etherpads seems totally busted. | 03:05 |
| harlowja_at_home | "Error: Attempt to load undefined module. in https://etherpad.openstack.org/static/js/require-kernel.js (line 1)" | 03:05 |
| clarkb | harlowja_at_home: yes, our testing indicated it was fine. You may need to hard refresh to pull in new js | 03:06 |
| clarkb | harlowja_at_home: do you have a link to the etherpad? | 03:06 |
| harlowja_at_home | https://etherpad.openstack.org/p/BrainstormFlowConditions | 03:06 |
| harlowja_at_home | let me try on a different browser that shouldn't have any cache | 03:06 |
| harlowja_at_home | ah, seems like u are right, maybe just have to clear my cache or something | 03:07 |
| clarkb | it loads fine for me. try ctrl + f5 for chromium | 03:07 |
| clarkb | harlowja_at_home: yes, this is a known issue with the upgrade and was noted in the announcement | 03:08 |
| harlowja_at_home | k | 03:08 |
| harlowja_at_home | np | 03:08 |
| harlowja_at_home | thx clarkb | 03:08 |
| clarkb | np | 03:09 |
| harlowja_at_home | all better, woot | 03:10 |
| *** D30 has joined #openstack-infra | 03:10 | |
| *** harlowja_at_home has quit IRC | 03:11 | |
| *** senk has quit IRC | 03:14 | |
| *** markmcclain has quit IRC | 03:16 | |
| *** markmcclain has joined #openstack-infra | 03:17 | |
| *** dhouck_ has quit IRC | 03:21 | |
| *** xchu has joined #openstack-infra | 03:23 | |
| *** david_h has quit IRC | 03:26 | |
| *** nati_ueno has joined #openstack-infra | 03:35 | |
| *** mriedem has quit IRC | 03:40 | |
| *** kong has left #openstack-infra | 03:50 | |
| openstackgerrit | Zhongyue Luo proposed a change to openstack/requirements: Adds byteformat https://review.openstack.org/51517 | 04:05 |
| fifieldt_ | clarkb, are you able to take another look at https://review.openstack.org/#/c/51112/ ? I updated one more path | 04:13 |
| clarkb | fifieldt_: done (found one extra set of things that need updating | 04:15 |
| fifieldt_ | actually, that's not true :) | 04:15 |
| fifieldt_ | hang on | 04:15 |
| fifieldt_ | half true :D | 04:15 |
| fifieldt_ | the PDF name still follows the old format | 04:16 |
| fifieldt_ | but dammit, there's still one | 04:16 |
| clarkb | imo you should fix the pdf name | 04:16 |
| clarkb | its confusing to have so many idfferently named things | 04:16 |
| fifieldt_ | oh, wtf - I forgot to commit the previous patch that fixced 43 | 04:17 |
| fifieldt_ | clarkb, yes, that's in plan - but it needs to be done to everything | 04:17 |
| openstackgerrit | Tom Fifield proposed a change to openstack-infra/config: Fix Doc Location for Transifex https://review.openstack.org/51112 | 04:18 |
| fifieldt_ | so much cleanup still to do clarkb, it' | 04:18 |
| fifieldt_ | s | 04:18 |
| fifieldt_ | driving me nuts :) | 04:18 |
| clarkb | fifieldt_: https://review.openstack.org/#/c/51112/3/modules/jenkins/files/slave_scripts/propose_translation_update_manuals.sh is line 63 wrong? | 04:18 |
| fifieldt_ | checking | 04:19 |
| fifieldt_ | that one wasn't | 04:21 |
| openstackgerrit | Tom Fifield proposed a change to openstack-infra/config: Fix Doc Location for Transifex https://review.openstack.org/51112 | 04:21 |
| fifieldt_ | but the other file was | 04:21 |
| fifieldt_ | it's so damn confusion - the pom.xml file stuff | 04:21 |
| fifieldt_ | there are about 4 different variables in there that do the same thing | 04:21 |
| fifieldt_ | with slightly different values | 04:21 |
| * fifieldt_ rage | 04:21 | |
| fifieldt_ | apologies for the shoddy patching, clarkb | 04:21 |
| clarkb | np +2 from me | 04:22 |
| clarkb | as someone that doesn't write docs professionally I have no idea how docbook is a tool of choice | 04:22 |
| fifieldt_ | docbook itself isn't a problem | 04:22 |
| fifieldt_ | it's maven | 04:22 |
| clarkb | I mean markdown, rst, tex, html, etc would all be simpler and have better renderers | 04:23 |
| clarkb | fifieldt_: right, but docbook forces you to use maven to render/compile stuff | 04:23 |
| *** xchu has quit IRC | 04:23 | |
| fifieldt_ | for me, docbook has two killer features: xi:include, and conditional formatting (os="ubuntu") | 04:23 |
| fifieldt_ | other than that, don't care | 04:23 |
| fifieldt_ | it doesn't get in my way | 04:24 |
| fifieldt_ | personally I don't find markdown, rst, tex or html any simpler :) | 04:24 |
| fifieldt_ | but maybe I'm crazy | 04:24 |
| clarkb | really? I can edit all of those in vim and not feel insane :) | 04:24 |
| * fifieldt_ uses vim to edit docbook | 04:24 | |
| clarkb | which is what I am basing my "simpler" judgement on. editing docbook in vim makes me feel crazy | 04:25 |
| fifieldt_ | oh, strange - I've never had an issue :) | 04:25 |
| fifieldt_ | probably again proving my insanity | 04:25 |
| fifieldt_ | I think one of the problems people had was that openstack was their first exposure to docbook, and the docs had sooooo much cruft and crappy layout | 04:26 |
| clarkb | that could be part of it. bitrot is hard to deal with anywhere | 04:27 |
| * fifieldt_ is currently screwing over his havana test vm every which way trying to write the network install docs :) | 04:29 | |
| fifieldt_ | things are so much simpler in puppet ;) | 04:29 |
| *** xchu has joined #openstack-infra | 04:36 | |
| *** nicedice has quit IRC | 04:40 | |
| *** yaguang has quit IRC | 04:41 | |
| *** jcoufal has joined #openstack-infra | 04:43 | |
| *** yaguang has joined #openstack-infra | 04:46 | |
| *** Canaimero-e64b8 has joined #openstack-infra | 04:46 | |
| Canaimero-e64b8 | cdvd | 04:46 |
| Canaimero-e64b8 | mm, | 04:46 |
| *** Canaimero-e64b8 has quit IRC | 04:47 | |
| *** dmakogon_ has joined #openstack-infra | 05:13 | |
| D30 | hi all, good day.. may i know if this is the right room to ask regarding openstack related issue? | 05:15 |
| *** __afazekas is now known as afazekas | 05:15 | |
| fifieldt_ | D30, this room is for talking about the infrastructure powering the openstack project | 05:16 |
| fifieldt_ | if you have an issue with running openstack | 05:16 |
| fifieldt_ | you should try #openstack or #openstack-101 | 05:16 |
| fifieldt_ | or http://ask.openstack.org | 05:16 |
| D30 | ahh okay fifieldt_ thank you for pointing that out.. | 05:16 |
| *** dmakogon_ has quit IRC | 05:25 | |
| *** dafter has joined #openstack-infra | 05:34 | |
| *** dafter has quit IRC | 05:40 | |
| *** dafter has joined #openstack-infra | 05:41 | |
| *** dafter has quit IRC | 05:45 | |
| *** yaguang has quit IRC | 05:48 | |
| *** SergeyLukjanov has joined #openstack-infra | 05:48 | |
| *** jecarey has quit IRC | 05:56 | |
| *** yaguang has joined #openstack-infra | 06:03 | |
| *** michchap has quit IRC | 06:23 | |
| *** oubiwann is now known as oubiwann-lambda | 06:27 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 06:29 | |
| *** SergeyLukjanov has joined #openstack-infra | 06:30 | |
| *** beagles has joined #openstack-infra | 06:33 | |
| *** dkehn has quit IRC | 06:41 | |
| *** dkehn has joined #openstack-infra | 06:42 | |
| *** odyssey4me has joined #openstack-infra | 06:47 | |
| *** DinaBelova has joined #openstack-infra | 07:00 | |
| *** dizquierdo has joined #openstack-infra | 07:07 | |
| *** pblaho has joined #openstack-infra | 07:26 | |
| *** DinaBelova has quit IRC | 07:30 | |
| bauzas | morning all | 07:30 |
| bauzas | any chance someone could help me on a Jenkins issue ? | 07:30 |
| bauzas | http://logs.openstack.org/70/50970/1/check/gate-climate-python27/5ede61d/console.html | 07:31 |
| bauzas | my own tox -r -epy27 is fine | 07:31 |
| bauzas | but Jenkins is building a venv having both oslo-config and oslo.config | 07:31 |
| bauzas | as a result, it doesn't work as oslo-config is too old | 07:31 |
| bauzas | I haven't asked for oslo-config, only asking oslo.config>1.2.0 | 07:32 |
| bauzas | any idea why Jenkins is pulling oslo-config ? | 07:32 |
| bauzas | and how can I fix it ? | 07:32 |
| *** markmc has joined #openstack-infra | 07:33 | |
| *** DinaBelova has joined #openstack-infra | 07:34 | |
| *** afazekas is now known as __afazekas | 07:35 | |
| *** osanchez has joined #openstack-infra | 07:41 | |
| *** dafter has joined #openstack-infra | 07:46 | |
| *** bauzas_ has joined #openstack-infra | 07:52 | |
| *** thomasbiege has joined #openstack-infra | 07:53 | |
| *** thomasbiege has quit IRC | 07:54 | |
| *** afazekas has joined #openstack-infra | 07:57 | |
| *** nati_ueno has quit IRC | 08:02 | |
| *** nati_ueno has joined #openstack-infra | 08:02 | |
| *** bauzas has quit IRC | 08:04 | |
| *** yaguang has quit IRC | 08:05 | |
| *** bauzas_ has quit IRC | 08:05 | |
| *** nati_ueno has quit IRC | 08:07 | |
| *** hashar has joined #openstack-infra | 08:14 | |
| *** yassine has joined #openstack-infra | 08:14 | |
| *** yaguang has joined #openstack-infra | 08:17 | |
| *** dizquierdo has quit IRC | 08:19 | |
| *** derekh has joined #openstack-infra | 08:20 | |
| *** SergeyLukjanov has quit IRC | 08:20 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:21 | |
| *** jpich has joined #openstack-infra | 08:22 | |
| *** mihgen has quit IRC | 08:24 | |
| *** aspiers has joined #openstack-infra | 08:25 | |
| *** SergeyLukjanov has quit IRC | 08:25 | |
| *** yaguang has quit IRC | 08:28 | |
| *** rakhmerov has joined #openstack-infra | 08:33 | |
| *** matsuhashi has joined #openstack-infra | 08:43 | |
| *** qba73 has joined #openstack-infra | 08:46 | |
| *** yaguang has joined #openstack-infra | 08:46 | |
| *** qba73 has left #openstack-infra | 08:47 | |
| *** fbo_away is now known as fbo | 08:48 | |
| *** amotoki has joined #openstack-infra | 08:50 | |
| *** amotoki has quit IRC | 08:51 | |
| *** amotoki has joined #openstack-infra | 08:51 | |
| *** mihgen_ has joined #openstack-infra | 08:58 | |
| *** hashar has quit IRC | 09:02 | |
| *** dizquierdo has joined #openstack-infra | 09:04 | |
| openstackgerrit | Petr Blaho proposed a change to openstack-infra/config: Remove python33 jobs for python-tuskarclient https://review.openstack.org/51552 | 09:12 |
| *** bauzas has joined #openstack-infra | 09:12 | |
| *** hashar has joined #openstack-infra | 09:18 | |
| *** hashar_ has joined #openstack-infra | 09:23 | |
| *** hashar has quit IRC | 09:24 | |
| *** hashar_ is now known as hashar | 09:24 | |
| rakhmerov | Hi, we've sent a change request to add a new project (Mistral): https://review.openstack.org/#/c/51205 . Now we've got a question about acl. Could someone find a couple of minutes to discuss that more detailed? Thanks in advance | 09:25 |
| *** salv-orlando has quit IRC | 09:39 | |
| *** salv-orlando has joined #openstack-infra | 09:41 | |
| *** DinaBelova has quit IRC | 09:48 | |
| *** jtzl has quit IRC | 09:53 | |
| *** wannatest has quit IRC | 09:53 | |
| *** michchap has joined #openstack-infra | 09:54 | |
| *** xchu has quit IRC | 09:56 | |
| *** salv-orlando has quit IRC | 09:56 | |
| *** salv-orlando has joined #openstack-infra | 10:05 | |
| *** pcm_ has joined #openstack-infra | 10:06 | |
| *** pcm_ has joined #openstack-infra | 10:07 | |
| *** erfanian has quit IRC | 10:11 | |
| *** erfanian has joined #openstack-infra | 10:11 | |
| *** DinaBelova has joined #openstack-infra | 10:15 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Add optional option to copyartifact plugin https://review.openstack.org/50392 | 10:17 |
| *** branen has quit IRC | 10:18 | |
| *** salv-orlando has quit IRC | 10:19 | |
| *** openstackstatus has quit IRC | 10:20 | |
| *** salv-orlando has joined #openstack-infra | 10:21 | |
| ekarlso | https://review.openstack.org/#/c/49850/ < how can I re-trigger a build of it ? | 10:27 |
| *** salv-orlando_ has joined #openstack-infra | 10:28 | |
| *** salv-orlando has quit IRC | 10:29 | |
| *** salv-orlando_ is now known as salv-orlando | 10:29 | |
| *** salv-orlando_ has joined #openstack-infra | 10:32 | |
| *** salv-orlando has quit IRC | 10:34 | |
| *** salv-orlando_ is now known as salv-orlando | 10:34 | |
| *** DinaBelova has quit IRC | 10:39 | |
| *** boris-42 has quit IRC | 10:42 | |
| *** guohliu has quit IRC | 10:42 | |
| *** michchap has quit IRC | 10:45 | |
| *** michchap has joined #openstack-infra | 10:48 | |
| *** mkerrin has joined #openstack-infra | 10:51 | |
| *** salv-orlando has quit IRC | 10:53 | |
| *** salv-orlando has joined #openstack-infra | 10:56 | |
| *** qba73_ has joined #openstack-infra | 11:00 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: switch over to os-loganalyze on logs.o.o https://review.openstack.org/51571 | 11:02 |
| *** dafter has quit IRC | 11:04 | |
| *** rickerc has quit IRC | 11:05 | |
| *** michchap has quit IRC | 11:06 | |
| *** yaguang has quit IRC | 11:07 | |
| *** dims has joined #openstack-infra | 11:08 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/jenkins-job-builder: missing parameter error are no more meaningfull https://review.openstack.org/49336 | 11:08 |
| *** hashar has quit IRC | 11:09 | |
| *** D30 has quit IRC | 11:09 | |
| *** hashar has joined #openstack-infra | 11:12 | |
| *** jhesketh__ has quit IRC | 11:18 | |
| *** fifieldt_ has quit IRC | 11:21 | |
| *** odyssey4me has quit IRC | 11:26 | |
| *** jcoufal has quit IRC | 11:35 | |
| BobBallAway | lifeless / fungi: Euither of you (or anyone) know if it's possible to boot in HP cloud with virtio drivers blacklisted / missing? A quick test failed to bring up networking - but I could easily be missing something... | 11:38 |
| *** lcestari has joined #openstack-infra | 11:39 | |
| *** BobBallAway is now known as BobBall | 11:41 | |
| openstackgerrit | afazekas proposed a change to openstack-infra/elastic-recheck: Add query for 1239637 https://review.openstack.org/51579 | 11:47 |
| *** rakhmerov has quit IRC | 11:49 | |
| *** adalbas has joined #openstack-infra | 11:49 | |
| *** rakhmerov has joined #openstack-infra | 11:53 | |
| *** hashar has quit IRC | 11:59 | |
| *** hashar has joined #openstack-infra | 12:03 | |
| *** dafter has joined #openstack-infra | 12:06 | |
| *** bauzas has quit IRC | 12:07 | |
| *** boris-42 has joined #openstack-infra | 12:12 | |
| *** dafter has quit IRC | 12:12 | |
| *** beagles has quit IRC | 12:12 | |
| *** bauzas has joined #openstack-infra | 12:12 | |
| *** beagles has joined #openstack-infra | 12:14 | |
| *** tvb|afk has joined #openstack-infra | 12:14 | |
| *** tvb|afk has quit IRC | 12:14 | |
| *** tvb|afk has joined #openstack-infra | 12:14 | |
| *** matsuhashi has quit IRC | 12:18 | |
| *** matsuhashi has joined #openstack-infra | 12:18 | |
| *** matsuhashi has quit IRC | 12:22 | |
| *** rakhmerov has left #openstack-infra | 12:24 | |
| *** bauzas has quit IRC | 12:24 | |
| *** dprince has joined #openstack-infra | 12:24 | |
| *** ArxCruz has joined #openstack-infra | 12:27 | |
| *** jcoufal has joined #openstack-infra | 12:28 | |
| *** weshay has joined #openstack-infra | 12:36 | |
| fungi | BobBall: can't say i've ever tried. how did you go about it (lkm blacklist file, bootloader configuration, renaming the module on disk)? | 12:38 |
| BobBall | /etc/modules.d blacklist | 12:39 |
| BobBall | Blacklist seemed to work in that it didn't load the modules, but networking didn't come up so it couldn't access cloudinit | 12:40 |
| BobBall | I thought it should have been possible from my knowledge of KVM (and that we've also seen VMs running without virtio drivers outside of HP cloud) | 12:40 |
| sdague | BobBall: I'd expect the way the device model is setup by the libvirt driver, your only option is virtio | 12:40 |
| *** jaypipes has joined #openstack-infra | 12:40 | |
| sdague | BobBall: but remember the nova libvirt driver is pretty explicit on what it sets up | 12:41 |
| *** tvb|afk has quit IRC | 12:41 | |
| BobBall | I've not played with the nova libvirt driver at all... admittedly this was outside of openstack we had it running | 12:41 |
| sdague | there was some metadata tags about device models added to glance images at some point, but I don't know of HP cloud is up to that level | 12:41 |
| BobBall | *sigh* | 12:41 |
| jcoufal | hey everybody, is here somebody who was dealing with customization of askbot for ask.openstack.org? | 12:41 |
| BobBall | Another angle blocked... | 12:41 |
| sdague | BobBall: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L110 is the only possible way around it that I know, but again, I have no idea if HP is on code level that supports that | 12:43 |
| sdague | it came in late grizzly or early havana IIRC | 12:43 |
| BobBall | ohhh thanks sdague | 12:43 |
| sdague | that's going to require you getting metadata onto your glance image | 12:44 |
| BobBall | I imagine that's possible... :) | 12:44 |
| *** dafter has joined #openstack-infra | 12:44 | |
| *** dafter has quit IRC | 12:44 | |
| *** dafter has joined #openstack-infra | 12:44 | |
| *** krtaylor has joined #openstack-infra | 12:45 | |
| BobBall | yay it is | 12:45 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix Doc Location for Transifex https://review.openstack.org/51112 | 12:45 |
| BobBall | probably :P | 12:45 |
| sdague | well, it's at least something to explore | 12:45 |
| BobBall | very much so, thanks. | 12:45 |
| *** julim has joined #openstack-infra | 12:46 | |
| *** dims has quit IRC | 12:48 | |
| *** johnthetubaguy has joined #openstack-infra | 12:49 | |
| *** dims has joined #openstack-infra | 12:51 | |
| *** jcoufal has quit IRC | 12:51 | |
| *** dcramer_ has quit IRC | 12:55 | |
| fungi | ekarlso: looking at 49850 i assume you got an answer to your question earlier (recheck no bug or recheck bug somenumber) | 12:57 |
| mordred | morning | 12:59 |
| mordred | fungi: how's your nodepool grokking? | 12:59 |
| fungi | mordred: increasing okay in absence of documentation i guess (rtfs!) but still not as solid as i'd like, why? | 13:00 |
| mordred | fungi: so, I spun up a nodepool on my laptop so that I could test the tripleo cloud without just continuing to retry landing config changes on infra | 13:00 |
| mordred | fungi: here is a snippet of local config: http://paste.openstack.org/show/48417/ | 13:01 |
| fungi | yeah, i caught some of that yesterday. didn't work completely i guess? | 13:01 |
| mordred | DEBUG:nodepool.NodePool:No current image for tripleo-test-cloud on tripleo-precise | 13:01 |
| mordred | I get that | 13:01 |
| mordred | which, may mean I have something configured backwards, or that the message is backwards | 13:01 |
| mordred | in either case - nodepool image-list shows that I do have a tripleo-precise image | 13:01 |
| fungi | are you getting that when it's trying to launch a fresh vm to configure as a template for the pool? | 13:02 |
| * fungi needs to trace where that error can originate | 13:02 | |
| mordred | INFO:nodepool.NodePool:Need to launch 2 tripleo-precise nodes for fake-jenkins on tripleo-test-cloud | 13:02 |
| mordred | so, it wants to launch 2 vms | 13:02 |
| mordred | but thinks it doesn't have any available images, best I can tell | 13:02 |
| ttx | mordred: what share of the US is expected to observe Columbus day ? | 13:02 |
| *** thomasm has joined #openstack-infra | 13:02 | |
| mordred | ttx: I do not expect many people to have it off | 13:02 |
| fungi | ttx: banks | 13:02 |
| mordred | it's not a real holiday | 13:02 |
| *** afazekas has quit IRC | 13:03 | |
| ttx | mordred: ah, great | 13:03 |
| fungi | bankers love to take random holidays | 13:03 |
| ttx | fungi: "we do because we can and f* you" | 13:03 |
| mordred | fungi: oh - I'm sorry - I have an image in 'building' | 13:03 |
| fungi | ttx: absolutely. it's like you've been to a bank before | 13:03 |
| mordred | fungi: lemme poke a little more | 13:03 |
| fungi | mordred: okay. i was thinking that might be benign since it's only debug level | 13:04 |
| ttx | fungi: bankers in france also happen to have the best holiday packages of all. | 13:04 |
| fungi | amazing how that works | 13:04 |
| mordred | O M G | 13:05 |
| *** senk has joined #openstack-infra | 13:05 | |
| mordred | I really really really hate things when they don't respond to ctrl-c all the way | 13:05 |
| *** freyes has quit IRC | 13:05 | |
| sdague | heh | 13:05 |
| mordred | but I'm really impatient too | 13:05 |
| sdague | fungi: when you get a chance - https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:os_loganalyze,n,z the first one should be a no brainer, the second one will probably need baby sitting when landing. | 13:06 |
| ekarlso | can I bother anyone to approve https://review.openstack.org/#/c/51066/ and https://review.openstack.org/#/c/51069/ | 13:06 |
| fungi | sdague: awesome! i'll have a look | 13:06 |
| sdague | given you and clarkb are the folks I've most worked with on the log servers, I figured I'd hit you up | 13:06 |
| *** ryanpetrello has joined #openstack-infra | 13:08 | |
| *** mestery_ has joined #openstack-infra | 13:09 | |
| ekarlso | Hmm, what's up with it failing to download taskflow from here? https://review.openstack.org/#/c/50591/ | 13:10 |
| *** rustlebee is now known as russellb | 13:10 | |
| *** pblaho has quit IRC | 13:11 | |
| *** osanchez has quit IRC | 13:12 | |
| *** gaelL has quit IRC | 13:12 | |
| *** OlivierSanchez has joined #openstack-infra | 13:12 | |
| *** mestery has quit IRC | 13:12 | |
| *** gaelL has joined #openstack-infra | 13:13 | |
| fungi | ekarlso: where is it failing to download taskflow there? i don't see it | 13:15 |
| *** matty_dubs|gone is now known as matty_dubs | 13:17 | |
| * olaph does not respond to ctrl-c all the way | 13:17 | |
| openstackgerrit | Mehdi Abaakouk proposed a change to openstack-infra/jenkins-job-builder: Allow use template variables in dictionnary key https://review.openstack.org/51159 | 13:18 |
| *** mriedem has joined #openstack-infra | 13:20 | |
| *** jpeeler has quit IRC | 13:21 | |
| *** clayg has quit IRC | 13:21 | |
| *** jpeeler has joined #openstack-infra | 13:21 | |
| *** clayg has joined #openstack-infra | 13:21 | |
| *** rakhmerov has joined #openstack-infra | 13:23 | |
| * mordred realy really hates olaph | 13:23 | |
| Shrews | olaph only responds to beatings about his head | 13:25 |
| fungi | and beer | 13:25 |
| *** afazekas has joined #openstack-infra | 13:25 | |
| fungi | Shrews: olaph: bier, ja? | 13:25 |
| olaph | ja wohl! | 13:26 |
| mordred | fungi, Shrews, olaph: we're getting bier next week, ja? | 13:26 |
| Shrews | ewww, beer, gross | 13:26 |
| fungi | though the evening after the openstack meetup next monday isn't an option for me. visitors in town early in the week | 13:26 |
| Shrews | mordred: you staying downtown during ATO? | 13:27 |
| Shrews | olaph, fungi: you guys going to ATO? | 13:27 |
| fungi | what's an ato? | 13:27 |
| Shrews | all things open conference | 13:28 |
| olaph | still gotta ask my boss... | 13:28 |
| fungi | huh. how do i miss these announcements? | 13:28 |
| *** bauzas has joined #openstack-infra | 13:28 | |
| mordred | Shrews: I'm actually staying in Durham | 13:28 |
| mordred | Shrews: (well, closer to RTP, but whatev) | 13:28 |
| mordred | fungi: I get in Wed evening, will be sticking around through the weekend because my 20 year high school reunion is saturday | 13:29 |
| * fungi is reading the ato manifesto now | 13:29 | |
| mordred | olaph: the hotel here in buenos aires has the best power strip ever in the desk drawer | 13:29 |
| fungi | mordred: i'm absolutely certain we can work out a time late next week or that weekend (i'll be free pretty much the whole time) | 13:29 |
| mordred | https://www.facebook.com/photo.php?fbid=10152315629924741&set=a.10150364889994741.433581.501289740&type=1&theater | 13:30 |
| mordred | (people should be able to view that without a facebook account) | 13:30 |
| mordred | handles agentina, us and eu power plugs | 13:30 |
| mordred | argentina | 13:30 |
| Shrews | geeenyus | 13:30 |
| mordred | right? | 13:30 |
| mordred | I'm thinking about getting a couple and starting to bring them to conferences | 13:31 |
| olaph | nice! | 13:31 |
| fungi | oh, too nifty | 13:31 |
| mordred | it won't help folks from the uk - but DUDE, those plugs are WAY too big | 13:31 |
| yolanda | hi, i'm trying to setup jenkins-job-builder, but i have an issue with authentication. Is jenkins-job-builder prepared to work with SSO? | 13:31 |
| yolanda | my admin user is logged through SSO, so i'm having problems setting the credentials | 13:32 |
| mordred | yolanda: so, there is a trick | 13:32 |
| mordred | yolanda: fungi might remember the steps better than me - but if you log in to jenkins via sso as the user you want to be the api user | 13:32 |
| mordred | yolanda: then in the users.xml file on the filesystem (I thnk) there is an api key | 13:33 |
| mordred | it's a terrible interface | 13:33 |
| fungi | ahh, yeah. i haven't had to generate one in a while but that matches my recollection | 13:33 |
| yolanda | mordred, but .. what about if i need to configure the jobs before admin ever logs? | 13:33 |
| mordred | yolanda: I do not believe we have solved that problem yet (thank you for working on solving our initial setup problems though!) | 13:34 |
| mordred | we just do that part of the jenkins install by hand | 13:34 |
| yolanda | mordred, bad news... :( | 13:34 |
| fungi | yolanda: this is probably straying into what might better be a "what's possible?" conversation in #jenkins | 13:34 |
| * mordred has REALLY been meaning to do more work on scripting the initial creation of the jenkins | 13:35 | |
| fungi | because you're likely going to have bootstrapping problems if you want to automatically/programatically interact as an admin with a jenkins which has just been set up and nobody has ever logged into | 13:35 |
| mordred | it honestly should not be terrible to do | 13:35 |
| mordred | fungi: I _think_ everything could completely be done by generating a config.xml and then starting jenkins | 13:35 |
| mordred | jenkins | 13:35 |
| mordred | there is completely nothing that's not configured through that file | 13:35 |
| fungi | probably, similar to bootstrapping gerrit access, you'll need to inject some initial credentials, yeah | 13:36 |
| yolanda | maybe generating a fake user xml file... | 13:36 |
| mordred | indeed - but I believe the jenkins bootstrap might be saner | 13:36 |
| fungi | yeah, no db involved | 13:36 |
| yolanda | if we have the name of the user, we could create an entry in the users dir, makes sense? | 13:36 |
| *** lascii is now known as alaski | 13:36 | |
| mordred | yup | 13:36 |
| mordred | yolanda: if you look in users/$adminusername/config.xml - you'll see an apiToken entry | 13:37 |
| fungi | making a temporary password-authenticated user might be simplest to bootstrap that, but you'd still need to be able to accurately recreate the password hashing algorithm jenkins uses | 13:37 |
| mordred | that's the thing you need for jjb | 13:37 |
| fungi | oh, or if the api token is in there in the clear, that could work | 13:37 |
| mordred | yup. that's where it comes from | 13:37 |
| fungi | is there nothing else magical about the api token? is it just a random key blob? | 13:38 |
| * mordred is not sure - guessing random key blob | 13:38 | |
| mordred | and then you need an entry in <authorizationStrategy class="hudson.security.ProjectMatrixAuthorizationStrategy"> | 13:38 |
| mordred | in config.xml | 13:38 |
| mordred | which references that user | 13:38 |
| mordred | seriously. we should puppet these files - OH - I remember why we donm't do that now | 13:39 |
| yolanda | we have an admin user there, but when i setup the SSO strategy for security, admin becomes unusable | 13:39 |
| mordred | yolanda: ah - we have an SSO account for our jjb user | 13:39 |
| mordred | and we just logged in to jenkins with it once, which made jenkins create the api key, and then we copied it out | 13:40 |
| yolanda | mordred, yes, we have one... but i was wondering on how to reuse the old initial admin account, doesn't look possible, right? | 13:40 |
| mordred | I don't believe you can once you've turned on SSO | 13:40 |
| yolanda | for example in gerrit you can have users comming from SSO, but you can create users that are non-interactive | 13:40 |
| mordred | yeah. gerrit is smarter about this | 13:40 |
| mordred | jenkins doesn't seem to understand users from multiple auth sources | 13:41 |
| *** mestery_ is now known as mestery | 13:41 | |
| fungi | looks like our api token is a 384-bit keyblob base64-encoded | 13:42 |
| fungi | not sure if jenkins cares whether it can base64 decode it | 13:42 |
| *** michchap has joined #openstack-infra | 13:42 | |
| yolanda | so the trick should be how to generate a fake api token ... if admin has logged once i just could grab it, but it not i need to generate a fake one | 13:43 |
| bauzas | fungi: thanks for your comment on https://review.openstack.org/#/c/50970/ | 13:43 |
| bauzas | fungi: I found that the pypi mirror is different in betwen my own tox env and Jenkins one | 13:44 |
| bauzas | fungi: Jenkins is using http://pypi.openstack.org/openstack/ | 13:44 |
| fungi | yolanda: dd if=/dev/random bs=48 count=1 | base64 | 13:44 |
| bauzas | fungi: while I'm using https://pypi.python.org/simple/ | 13:44 |
| *** qba73 has joined #openstack-infra | 13:44 | |
| yolanda | fungi, thx! | 13:44 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Show total age stats last https://review.openstack.org/51607 | 13:45 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Show total age stats last https://review.openstack.org/51607 | 13:45 |
| bauzas | fungi: so, I'll swith from python.org to openstack.org and check what is pulling oslo-config | 13:45 |
| fungi | bauzas: yes, i think for unofficial projects our tests will use pypi.openstack.org and then fall back on pypi.python.org if the dependency can't be satisfied from the first | 13:45 |
| bauzas | fungi: well, that's not cool at all :( | 13:46 |
| bauzas | any chance to fix this for Climate ? | 13:46 |
| bauzas | (a stackforge project) | 13:46 |
| *** qba73_ has quit IRC | 13:47 | |
| fungi | bauzas: well, that allows projects to depend on unreleased openstack libraries while under development. i'd rather just fix whatever is causing the issue. what was the difference in your pip freeze besides oslo.config and oslo-config? | 13:49 |
| bauzas | I have to dig further | 13:50 |
| *** michchap has quit IRC | 13:51 | |
| *** SergeyLukjanov has joined #openstack-infra | 13:51 | |
| fungi | you should be able to cut and paste the pip freeze output from jenkins and from your local run, use sort to make sure they're in the same order (may not even be necessary) and then just run diff on them | 13:52 |
| bauzas | yup, just done that actually | 13:52 |
| fungi | i do that sort of comparison all the time when we see issues which might be introduced by new releases of dependencies | 13:53 |
| bauzas | sounds like my nova egg is the faulty one | 13:53 |
| *** thedodd has joined #openstack-infra | 13:53 | |
| bauzas | pasting int | 13:53 |
| fungi | you're depending on a master branch tarball of openstack/nova from tarballs.openstack.org right? that should depend on oslo.config normally | 13:54 |
| * fungi checks to see if he might just be insane | 13:54 | |
| fungi | yeah, nova's master branch has oslo.config>=1.2.0 in requirements.txt | 13:55 |
| bauzas | argh, connection timed out while pasting the diff | 13:57 |
| bauzas | basically, there was a 41c43 | 13:57 |
| bauzas | < nova==2014.1.a265.gfd24923 | 13:57 |
| bauzas | --- | 13:57 |
| bauzas | > nova==2013.1.g3 | 13:57 |
| bauzas | http://paste.openstack.org/show/48423/ | 13:57 |
| *** DinaBelova has joined #openstack-infra | 13:57 | |
| bauzas | we're pulling from a tarball | 13:58 |
| bauzas | https://review.openstack.org/#/c/50652/5/test-requirements.txt | 13:58 |
| bauzas | when using pypi.openstack.org, we got a Grizzly release | 13:59 |
| fungi | oh, i wonder if your nova>=master line is the issue... | 13:59 |
| fungi | mordred: ^ | 13:59 |
| mordred | yes. the nova>=master line is a problem | 13:59 |
| * mordred needs to think for a second about how to do that right ... | 14:00 | |
| mordred | speaking of ... | 14:00 |
| fungi | i think that will tell pip to pull in the most recent prerelease version it can find from pypi | 14:00 |
| bauzas | well, speak to jd__ then :D | 14:00 |
| mordred | ttx: I think we might have reached the point where we might need to consider actually releasing nova and friends to pypi on tags | 14:00 |
| mordred | ttx: we now have _multiple_ projects in our own ecosystem which are trying to consume nova in their test-requirements.txt in order to test themsleves against it | 14:01 |
| mordred | which seems fair | 14:01 |
| mordred | but we're making them do crazy tarball urls to accomplish it | 14:01 |
| bauzas | mordred: we actually need to test our own Nova scheduler filtrer | 14:01 |
| mordred | yup | 14:01 |
| *** prad_ has joined #openstack-infra | 14:01 | |
| bauzas | s/filtrer/filter | 14:01 |
| mordred | ttx: I'm wondering if we shouldn't start publishing the actual releases to pypi - and then, since we're going to add the code to be able to publish pre-releases to pypi but only as wheels so that pip >=1.4 will do the right thing for oslo.config - we should be able to use the same tooling for server projects too | 14:02 |
| mordred | ttx: thoughts - does that thought make you want to kill me? | 14:02 |
| *** pblaho has joined #openstack-infra | 14:03 | |
| *** sdake_ has joined #openstack-infra | 14:03 | |
| *** sdake_ has quit IRC | 14:03 | |
| *** sdake_ has joined #openstack-infra | 14:03 | |
| *** rwsu has joined #openstack-infra | 14:03 | |
| fungi | bauzas: so this makes sense. on our test infrastructure the dep on the tarball url gets satisfied, then the dep on >=master gets checked and the master tarball is updated with the most recent nova version on our pypi mirror (which is a grizzly milestone). when not using our mirror the >=master fails to download anything else because there is no nova released to real pypi | 14:04 |
| bauzas | fungi: yup | 14:04 |
| mordred | fungi: why is our mirror picking up a nova? | 14:04 |
| fungi | mordred: because we have one on there from circa grizzly g3 | 14:04 |
| mordred | fungi: ah - hysterical raisins you mean | 14:04 |
| mordred | makes sense to me now | 14:04 |
| fungi | mordred: http://pypi.openstack.org/openstack/nova/ | 14:05 |
| bauzas | fungi: mordred: I'm actually getting a nova package | 14:05 |
| bauzas | fungi: nova==2014.1.a265.gfd24923 | 14:05 |
| fungi | bauzas: yeah that's getting built from the tarball pip retrieves. i think if the nova>=master line were omitted it would cease to bet replaced with the old grizzly one | 14:06 |
| fungi | er, cease to be replaced | 14:06 |
| bauzas | fungi: mmm, you're right | 14:06 |
| fungi | bauzas: you can try that by amending your patch to take that nova>=master line out of test-requirements.txt and see if the tests pass | 14:07 |
| bauzas | we actually only need to test against Nova for the filter | 14:07 |
| *** prad__ has joined #openstack-infra | 14:07 | |
| *** jecarey has joined #openstack-infra | 14:07 | |
| bauzas | fungi: yup, doing it right now | 14:07 |
| *** prad_ has quit IRC | 14:07 | |
| *** afazekas has quit IRC | 14:08 | |
| bauzas | I'll rebuild the tox env for sanity checking before submitting to Gerrit | 14:09 |
| *** pblaho has quit IRC | 14:09 | |
| mordred | jeblair: when you get up - nodepool is assuming that the cloud account has a secgroup with 22 open | 14:10 |
| bauzas | fungi: btw, I guess that I could test the Jenkins behaviour by providing the Openstack pypi mirror when building thru tox ? | 14:10 |
| mordred | jeblair: this is apparently not a thing that can be counted on | 14:10 |
| ttx | mordred: hmm | 14:10 |
| mordred | bauzas: if you just put the openstack mirror in your pip config, your local tox will behave as if it's on our system | 14:10 |
| fungi | bauzas: yes, what we do is basically https://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/select-mirror.sh | 14:10 |
| yolanda | fungi, mordred, i checked token from the xml file but it should be encrypted in some way. Token is different from the one shown in the UI to the one in the xml file. Using the one in XML doesn't work, but the one from the UI does | 14:11 |
| mordred | yolanda: wow, really? | 14:11 |
| yolanda | yes... | 14:11 |
| mordred | yolanda: Im guessing that perhaps jenkins has changed behavior but ours is still generated/stored using old behavior | 14:11 |
| bauzas | mordred: fungi: thanks | 14:11 |
| mordred | :( | 14:11 |
| yolanda | i'm with jenkins 1.53 version | 14:12 |
| ttx | mordred: I'd rather not do it automatically this time around. We can certainly upload something manually for havana if needed and then discuss at summit what to do for icehouse ? | 14:12 |
| mordred | ttx: yes | 14:12 |
| mordred | ttx: totally don't want to to automatically this roud - it's too close | 14:12 |
| fungi | yolanda: yes, so my guess is that this is a sha384 hash base64 encoded | 14:12 |
| mordred | ttx: but I thnk it might make some of the things people want to do have a more defined set of semantics | 14:12 |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 14:13 | |
| fungi | yolanda: can you confirm that what it generates is 64 printable ascii text characters in length? | 14:13 |
| fungi | (in the config file) | 14:13 |
| ttx | mordred: I want to wrap that discussion with the "where to publish our tarballs" storyboard/tarballs.o.o/pypi discussion we need to have anyway | 14:13 |
| mordred | jeblair: do you think the 'right' thing to do is add code to nodepool to detect secgroup extension, then if it's there, ensure there is a port 22 rule? | 14:13 |
| *** dcramer_ has joined #openstack-infra | 14:13 | |
| mordred | ttx: oh - I didn't realize that was a discussion topic | 14:13 |
| mordred | but if you think it is, then we should _definitely_ have that discussion :) | 14:14 |
| ttx | mordred: there is no reason why we should duplicate LP "tarball downloading" feature if we can have something clean at tarballs.o.o and/or pypi | 14:14 |
| fungi | yolanda: that's what i see in ours anyway. when i base64 decode it i get 48 seemingly-random bytes, and the only common hash function which makes output of that length is sha384 | 14:14 |
| mordred | ttx: +10000 | 14:14 |
| yolanda | fungi, it looks like base64 encoded | 14:14 |
| ttx | mordred: the only reason we use LP for that is that it gives you convenient sig upload | 14:14 |
| mordred | ttx: I do not think that we need to have tarball downloading in storyboard. at most, I could see having a pointer from a release in storyboard to a pypi/tarball.o.o location | 14:14 |
| mordred | ttx: yah | 14:15 |
| bauzas | fungi: that doesn't work, as I'm not pulling the nova tarball | 14:15 |
| mordred | ttx: well, I think that pypi has sig uploading that we could figure out how to use | 14:15 |
| ttx | mordred: in the case of storyboard, less is more | 14:15 |
| fungi | yolanda: unfortunately, if they're using sha384 then that's probably only the *last* step in their algorithm, which is almost certainly doing multiple rounds with key stretching of some variety | 14:15 |
| mordred | ttx: have you put in a topic for this? | 14:15 |
| mordred | ttx: ++ yes | 14:15 |
| mordred | ttx: I do not think storyboard wants to be launchpad | 14:15 |
| mordred | I think it wants to be exactly what we need out of it | 14:16 |
| ttx | mordred: we could wrap that discussion into the "release cycle" one... Or just make another one | 14:16 |
| ttx | mordred: actuially would make sense as another one | 14:16 |
| mordred | let's make anotuher one | 14:16 |
| mordred | I think it's a different topic | 14:16 |
| ttx | mordred: where we'd also discuss stremalining the client lib release process | 14:16 |
| yolanda | fungi, doesn't like as good news then | 14:16 |
| annegentle_ | ttx: when you have a minute, I have some questions re: release timing | 14:16 |
| yolanda | doesn't look as good news | 14:16 |
| mordred | ttx: what's the short story on that one? | 14:16 |
| fungi | yolanda: yes, it's probably something you'd need to read the jenkins source to figure out, or chat with the devs on #jenkins | 14:17 |
| ttx | mordred: I'd like people go through the same steps as far as announcing is concerned | 14:17 |
| ttx | and/or changelogging | 14:17 |
| mordred | ah. nod | 14:17 |
| *** rakhmerov1 has joined #openstack-infra | 14:17 | |
| *** rakhmerov has quit IRC | 14:17 | |
| ttx | so it's a genberal discussion on the future of release artifacts in general | 14:17 |
| ttx | mordred: i'll let you file it | 14:17 |
| ttx | you can push to "release management" topic. If you keep "storyboard" in infra, I have one slot left | 14:18 |
| *** rakhmerov1 has quit IRC | 14:18 | |
| yolanda | fungi, i'll try to talk with jenkins, i'll leave that as a manual step now, that people login with admin user and set the api token manually | 14:18 |
| *** rakhmerov has joined #openstack-infra | 14:18 | |
| *** senk has quit IRC | 14:18 | |
| fungi | bauzas: you get an error about the -f http:// line in test-requirements.txt or you get an error about an unsatisfied dependency on nova? | 14:18 |
| bauzas | atm, I can't test my filter due to the nova master tarball missing | 14:18 |
| bauzas | fungi: I'm getting an ImportError as nova is missing | 14:19 |
| mordred | ttx: submitted | 14:19 |
| bauzas | when running the tests | 14:19 |
| mordred | ttx: I included one more question ... | 14:19 |
| *** gaelL has quit IRC | 14:19 | |
| ttx | mordred: more question, more better | 14:19 |
| mordred | ttx: which is "if we base things around releasing to pypi, what happens if we had a non-python thing" | 14:19 |
| ttx | mordred: ++ | 14:20 |
| mordred | ttx: which, while not in scope right now - infra does release a few different non-python things | 14:20 |
| *** gaelL has joined #openstack-infra | 14:20 | |
| mordred | andI'd like to be able to track that in storyboard | 14:20 |
| mordred | at the very least | 14:20 |
| fungi | for that matter, there are tripleo projects which include python boilerplate only so they can build release tarballs, but otherwise contain no python at all | 14:20 |
| *** beagles has quit IRC | 14:21 | |
| *** beagles has joined #openstack-infra | 14:21 | |
| fungi | which does allow us to release them to pypi, but that's still a little odd | 14:21 |
| *** alcabrera has joined #openstack-infra | 14:21 | |
| mordred | fungi: ooh! | 14:21 |
| mordred | I seem to be building nodepool things now on the tripleo grizzly cloud | 14:21 |
| fungi | mordred: congrats! | 14:22 |
| mordred | Ng, SpamapS, lifeless ^^ | 14:22 |
| *** SergeyLukjanov has joined #openstack-infra | 14:22 | |
| mordred | three patches that are currently pending need to land | 14:22 |
| mordred | and the security-group has to be added | 14:22 |
| mordred | (although that is now done by hand) | 14:22 |
| mordred | but I'm getting puppet executions happening | 14:22 |
| mordred | INFO:nodepool.image.build.tripleo-test-cloud.tripleo-precise:notice: /Stage[main]/Openstack_project::Base/File[/etc/puppet/puppet.conf]/mode: mode changed '0644' to '0444' | 14:22 |
| mordred | which means we're probably past the "nodepool interacts weirdly with this cloud" stage | 14:23 |
| bauzas | fungi: http://paste.openstack.org/show/48426/ | 14:23 |
| bauzas | I do understand the problem, I'm just looking for a workaround | 14:23 |
| mordred | remove the -f | 14:23 |
| mordred | just put the url | 14:24 |
| mordred | see if tha tworks | 14:24 |
| bauzas | what is doing the -f flag ? (sorry for that question if obvious) | 14:24 |
| Ng | mordred: \o/ | 14:24 |
| rakhmerov | mordred: hey, sorry for being a bother :) could you pls tell some details on you comment in https://review.openstack.org/#/c/51205 ? We'll fix that once you let us know | 14:25 |
| mordred | bauzas: in your file - each line is basically an argument/param to pip | 14:25 |
| mordred | rakhmerov: looking | 14:25 |
| rakhmerov | thanks! | 14:25 |
| bauzas | mordred: uh oh :) | 14:25 |
| *** dcramer_ has quit IRC | 14:26 | |
| fungi | bauzas: http://www.pip-installer.org/en/0.8.3/requirement-format.html | 14:26 |
| mordred | rakhmerov: you have a section for refs/heads/release-* which is a different pattern than other openstack-y things | 14:26 |
| mordred | that's fine - I just wanted to check in with you and make sure that was on purpose and was going to do what you thought it was going to do | 14:26 |
| *** tstevenson has joined #openstack-infra | 14:26 | |
| fungi | bauzas: "if you want to install from a tarball or zip file with a direct link, you don’t need -f option, you just need to pass the absolute url" | 14:26 |
| rakhmerov | well, yeah. I thought it shouldn't be necessary a 100% match to other projects' structure | 14:27 |
| rakhmerov | mordred: yes, that's what we wanted | 14:27 |
| mordred | rakhmerov: great! | 14:27 |
| rakhmerov | :) thx | 14:27 |
| bauzas | fungi: mordred: thanks, we actually copied from ceilometer :) | 14:27 |
| *** gaelL has quit IRC | 14:27 | |
| mordred | bauzas: well, they're slightly different - but I think that the weirdness here is an indication that we need to do something different | 14:28 |
| *** erfanian has quit IRC | 14:29 | |
| fungi | mordred: is -e needed to parse the egg comment at the end of the url? | 14:29 |
| mordred | fungi: no | 14:30 |
| mordred | foo/bin/pip install 'http://tarballs.openstack.org/nova/nova-master.tar.gz#egg=nova-master' | 14:30 |
| mordred | works fine | 14:30 |
| *** gaelL has joined #openstack-infra | 14:30 | |
| *** CaptTofu has joined #openstack-infra | 14:30 | |
| *** gaelL has quit IRC | 14:30 | |
| bauzas | mordred: fungi: thanks a lot, removing the #egg tag plus the -f is enough for working :) | 14:31 |
| bauzas | last step before submitting, I'll change my pip mirror to openstack | 14:31 |
| bauzas | and check if still ok | 14:31 |
| *** tstevenson has quit IRC | 14:33 | |
| *** dafter has quit IRC | 14:33 | |
| *** dafter has joined #openstack-infra | 14:33 | |
| *** dafter has joined #openstack-infra | 14:33 | |
| *** gaelL has joined #openstack-infra | 14:35 | |
| *** markmcclain has joined #openstack-infra | 14:36 | |
| *** michchap has joined #openstack-infra | 14:37 | |
| *** tvb|afk has joined #openstack-infra | 14:38 | |
| *** tvb|afk has quit IRC | 14:38 | |
| *** tvb|afk has joined #openstack-infra | 14:38 | |
| *** dafter has quit IRC | 14:38 | |
| *** swann has joined #openstack-infra | 14:40 | |
| *** gaelL has quit IRC | 14:41 | |
| *** gaelL has joined #openstack-infra | 14:42 | |
| *** datsun180b has joined #openstack-infra | 14:42 | |
| *** wenlock has joined #openstack-infra | 14:44 | |
| *** michchap has quit IRC | 14:45 | |
| *** pentameter has joined #openstack-infra | 14:46 | |
| *** rcleere has joined #openstack-infra | 14:47 | |
| *** rnirmal has joined #openstack-infra | 14:48 | |
| *** zojoncj has joined #openstack-infra | 14:49 | |
| *** tvb|afk has quit IRC | 14:50 | |
| *** senk has joined #openstack-infra | 14:51 | |
| *** jcoufal has joined #openstack-infra | 14:54 | |
| *** flaper87|afk is now known as flaper87 | 14:57 | |
| openstackgerrit | A change was merged to openstack-infra/config: add os-loganalyze to #openstack-infra https://review.openstack.org/51489 | 14:57 |
| *** yassine has quit IRC | 15:03 | |
| *** dcramer_ has joined #openstack-infra | 15:04 | |
| *** senk has quit IRC | 15:04 | |
| openstackgerrit | David Shrewsbury proposed a change to openstack-infra/gear: Add SSL error checking https://review.openstack.org/46741 | 15:08 |
| sdague | fungi: did you have an opinion on the switch over to os-loganalyze as the wsgi filter? | 15:10 |
| fungi | sdague: just now getting around to reading it | 15:10 |
| sdague | coolio | 15:10 |
| jeblair | mordred: haha. re: secgroup -- you could rephrase that as "nodepool is assuming the cloud provides vms with a working connection to the internet". :) | 15:10 |
| *** mrodden has quit IRC | 15:10 | |
| fungi | we just have a secgroup which is wide open and then filter with iptables on the individual servers there, right? | 15:11 |
| jeblair | mordred: to seriously think about your question -- the secgroup thing isn't something that needs to be done for each host, so it's a little strange to put it in nodepool | 15:11 |
| jeblair | fungi: yep | 15:11 |
| mordred | jeblair: fair | 15:11 |
| mordred | jeblair: so we should just consider that potentially as part of setting up the cloud accout itself | 15:12 |
| mordred | "note to self, configure secgroup on new cloud account" | 15:12 |
| fungi | also, i believe the last time we tried to auto-update security groups with the level of churn in that tenant we brought hpcloud to its knees and then clarkb got called in the middle of the night | 15:12 |
| mordred | jeblair: I'm much closer - I got it running puppet on the nodes, but then something got unhappy and I wasn't looking and the scrollback was gone from y buffer by the time I looked | 15:13 |
| jeblair | mordred: we could have it check something like that at startup, but i'm currently leaning toward what you just said; partly for simplicity and bitrot reasons. | 15:13 |
| mordred | ++ | 15:13 |
| jeblair | mordred: you could set up a logging config that goes to console and file | 15:14 |
| mordred | jeblair: I'm about to send a mailing list message to tripleo and infra about "how I set up a nodepool and pointed it at this cloud for testing" | 15:14 |
| mordred | jeblair: I could indeed - but I'm pretty fast and loose with this setup | 15:14 |
| mordred | I'm thkning >file & tail -f file | 15:14 |
| jeblair | mordred: cool, could it be a howto doc for nodepool? | 15:14 |
| mordred | jeblair: might be - I think there are a couple of steps which a bit wonky and specific | 15:15 |
| mordred | jeblair: but certainly could be the start of one | 15:15 |
| *** yassine has joined #openstack-infra | 15:16 | |
| mordred | jeblair: if I wasn't running fake-server.py - would nodepool still try to make images and spin up the min amount? | 15:16 |
| mordred | jeblair: also, this: | 15:18 |
| mordred | self.log.debug("No current image for %s on %s" | 15:18 |
| mordred | % (provider.name, image.name)) | 15:18 |
| *** yassine has quit IRC | 15:19 | |
| *** yassine has joined #openstack-infra | 15:19 | |
| jeblair | mordred: yes it would | 15:20 |
| mordred | is giving me DEBUG:nodepool.NodePool:No current image for tripleo-test-cloud on tripleo-precise | 15:21 |
| mordred | which scans weirdly in english | 15:21 |
| jeblair | mordred: i believe the interpolation is backwards | 15:21 |
| *** senk has joined #openstack-infra | 15:21 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/nodepool: Change image debug interpolation order https://review.openstack.org/51641 | 15:23 |
| *** mrodden has joined #openstack-infra | 15:23 | |
| *** beagles has quit IRC | 15:27 | |
| *** alexpilotti has joined #openstack-infra | 15:29 | |
| bauzas | fungi: I'm still stuck with my Jenkins issue | 15:30 |
| bauzas | I provided the workaround you mentioned about nova | 15:30 |
| bauzas | and now the whole gates are failing... | 15:30 |
| bauzas | fungi: https://review.openstack.org/#/c/50970/2 | 15:30 |
| bauzas | fungi: error in setup command: 'tests_require' must be a string or list of strings containing valid project/version requirement specifiers | 15:31 |
| bauzas | fungi: http://logs.openstack.org/70/50970/2/check/gate-climate-pep8/50b370c/console.html | 15:31 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/nodepool: Add gear to requirements https://review.openstack.org/51644 | 15:33 |
| annegentle_ | ttx: have any time to chat about docs release? | 15:33 |
| jcoufal | clarkb: hey Clark, is it you who put together customizations for ask.openstack.org? | 15:35 |
| jcoufal | clarkb: mostly skin, layout, etc | 15:36 |
| fungi | jcoufal: the askbot developers did all that work for us, i think. reed would know when he's around | 15:36 |
| fungi | bauzas: i'm posing around in other projects to find working examples | 15:36 |
| annegentle_ | ttx: mostly want to know exact timing and mechanism for cutting a stable/havana branch for openstack-manuals (anyone else in -infra know?) | 15:37 |
| bauzas | fungi: oki doki, that's good point | 15:37 |
| fungi | jcoufal: keep in mind that we basically contracted the askbot devs to set up, configure and manage that for us | 15:37 |
| jcoufal | fungi: I know, I know, just basic deployment is pretty terrible, so I wanted to just take the skin and layout customization from already existing openstack solution | 15:38 |
| jcoufal | will ask reed when he's around | 15:39 |
| jcoufal | fungi: thanks | 15:39 |
| fungi | jcoufal: https://git.openstack.org/cgit/openstack-infra/askbot-theme/ | 15:39 |
| jcoufal | fungi: awesome | 15:40 |
| fungi | jcoufal: those *may* be the droids you're looking for | 15:40 |
| fungi | bauzas: ceilo uses http://tarballs.openstack.org/nova/nova-master.tar.gz#egg=nova | 15:40 |
| fungi | (not #egg=nova-master) | 15:41 |
| fungi | bauzas: could that be the issue? | 15:41 |
| * fungi double checks whether they still do that or if his checkout is stale | 15:41 | |
| fungi | bauzas: yep, they still do it that way... https://git.openstack.org/cgit/openstack/ceilometer/tree/test-requirements.txt#n12 | 15:42 |
| *** rfolco has joined #openstack-infra | 15:42 | |
| *** dafter has joined #openstack-infra | 15:43 | |
| *** amotoki has quit IRC | 15:43 | |
| bauzas | fungi: lookign | 15:44 |
| bauzas | fungi: ok, putting #egg=nova at the end | 15:45 |
| ttx | annegentle_: yes | 15:45 |
| bauzas | and testing | 15:45 |
| *** UtahDave has joined #openstack-infra | 15:45 | |
| annegentle_ | ttx: ok, in the past I just gave a SHA to either clarkb or fungi and he made a stable/relname from it, does that sound right? What time of day does that need to be done? | 15:46 |
| fungi | annegentle_: that sounds right, and i can do it whenever you need as long as that sha is already merged | 15:47 |
| *** flaper87 is now known as flaper87|afk | 15:47 | |
| ttx | annegentle_: we can do it anytime between now and Thursday | 15:47 |
| ttx | annegentle_: (we did oslo-incubator already, for example) | 15:48 |
| clarkb | morning | 15:48 |
| clarkb | jcoufal: fungi is correct, I didn't do any customization. I had only tried to deploy it with mysql | 15:48 |
| jcoufal | clarkb: ok, thanks | 15:49 |
| ttx | annegentle_: sorry, got distracted | 15:49 |
| clarkb | fungi: mordred: yeah security groups are bad | 15:49 |
| ttx | annegentle_: I can probably do it for you as well | 15:50 |
| mordred | clarkb: well, yeah | 15:50 |
| fungi | mordred: Shrews: olaph: i went ahead and registered for ato since early bird pricing is cheap and ends on wednesday. looks like fun | 15:50 |
| *** guohliu has joined #openstack-infra | 15:50 | |
| fungi | i may not attend all of it, but at that price it's not like i have to squeeze every ounce from the conference | 15:50 |
| ttx | annegentle_: so the moment when you are done (hopefully on Thursday or before) give me a SHA and i'll create stable/havana for you (or infra will) | 15:50 |
| clarkb | there are still people wandering around wiht power tools. I wonder if I should've taken this call at home... | 15:51 |
| annegentle_ | ttx: okay sounds good | 15:51 |
| ttx | annegentle_: no precise time -- you can actually cut now if that's how you roll :) | 15:51 |
| annegentle_ | ttx: I'm working on a patch that I'll rebase | 15:51 |
| annegentle_ | ttx: but it takes something like 2 days to get a sitemap.xml, heh | 15:51 |
| ttx | annegentle_: huh | 15:51 |
| annegentle_ | ttx: crazy I know | 15:52 |
| *** hashar has quit IRC | 15:52 | |
| fungi | mordred: are you the one giving the openstack talk at ato? or do you know who is? | 15:53 |
| fungi | their site didn't seem to say | 15:53 |
| clarkb | ato? | 15:53 |
| fungi | clarkb: http://www.allthingsopen.org/ (local conference) | 15:54 |
| mordred | fungi: atwood and I both are giving one | 15:54 |
| fungi | mordred: oh, great! | 15:54 |
| annegentle_ | clarkb: or fungi: Do you have a minute to check into why DocImpact flags aren't working any more? Looks like since this went in: https://review.openstack.org/#/c/46721/1/jeepyb/cmd/notify_impact.py we haven't seen DocImpact | 15:55 |
| fungi | annegentle_: i'll check gerrit's logs | 15:55 |
| fungi | annegentle_: do you have an example change which should have opened a bug and didn't? | 15:55 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: switch over to os-loganalyze on logs.o.o https://review.openstack.org/51571 | 15:56 |
| sdague | fungi: ok, should be fixed now | 15:56 |
| *** oubiwann has joined #openstack-infra | 15:57 | |
| *** qba73 has quit IRC | 15:57 | |
| annegentle_ | fungi: yeah I think https://review.openstack.org/#/c/48385/ should have? | 15:57 |
| *** jcoufal has quit IRC | 15:59 | |
| *** markmc has quit IRC | 16:03 | |
| fungi | annegentle_: oh, i wonder if we're not calling the impact script from the change-merged hook | 16:03 |
| * fungi checks | 16:03 | |
| *** beekneemech is now known as bnemec | 16:05 | |
| *** qba73 has joined #openstack-infra | 16:05 | |
| *** alcabrera has quit IRC | 16:06 | |
| fungi | yep, only on patchset-created, looks like | 16:06 |
| fungi | i'll propose a fix | 16:06 |
| fungi | annegentle_: the problem was actually introduced in the previous patch, https://review.openstack.org/#/c/45907/2/jeepyb/cmd/notify_impact.py | 16:07 |
| *** gyee has joined #openstack-infra | 16:07 | |
| fungi | we were never running the script on that event | 16:07 |
| *** dafter has quit IRC | 16:08 | |
| *** reed has joined #openstack-infra | 16:09 | |
| *** alcabrera has joined #openstack-infra | 16:09 | |
| *** yaguang has joined #openstack-infra | 16:09 | |
| *** markmcclain has quit IRC | 16:11 | |
| *** mihgen_ has quit IRC | 16:11 | |
| *** yassine has quit IRC | 16:13 | |
| *** DinaBelova has quit IRC | 16:14 | |
| *** boris-42 has quit IRC | 16:15 | |
| Shrews | fungi: awesome. c ya there | 16:16 |
| *** david-lyle_ has joined #openstack-infra | 16:17 | |
| *** matty_dubs is now known as matty_dubs|lunch | 16:17 | |
| annegentle_ | fungi: oh whew, glad you found it. Too bad we lost a month of docimpact. sigh. | 16:18 |
| annegentle_ | fungi: ok maybe just 3 weeks. argh. any way to know how many we missed? | 16:19 |
| jeblair | annegentle_, fungi: i think you could use a gerrit search | 16:19 |
| annegentle_ | jeblair: oh I wondered, what is the search incantation? | 16:19 |
| annegentle_ | jeblair: (I don't know how to search within commit messages) | 16:19 |
| jeblair | message:DocImpact status:merged | 16:20 |
| annegentle_ | jeblair: thanks, looks like only a dozen or so, whew | 16:21 |
| annegentle_ | oh crap it hasn't been working since Sept 12? | 16:21 |
| annegentle_ | more like 2 dozen | 16:21 |
| jeblair | annegentle_: yeah, i think it goes onto the next page then for a total of 27 | 16:22 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move docimpact from created hook to merged https://review.openstack.org/51658 | 16:23 |
| fungi | annegentle_: ^ | 16:23 |
| *** senk has quit IRC | 16:24 | |
| *** fbo is now known as fbo_away | 16:26 | |
| *** yaguang has quit IRC | 16:27 | |
| *** branen has joined #openstack-infra | 16:28 | |
| annegentle_ | fungi: maybe I don't understand, but didn't we stop sending emails as notification and instead directly log a doc bug? | 16:28 |
| clarkb | that was the intent, but it had a bug | 16:29 |
| annegentle_ | fungi: clarkb oh ok, cuz Sept 13 was the last time our mailing list had a report in it | 16:29 |
| fungi | annegentle_: yeah, in my opinion it got changed in the wrong place, or at least not enough of the right places | 16:30 |
| annegentle_ | fungi: clarkb: so will we see mails again? | 16:30 |
| fungi | annegentle_: no, the change to only do e-mails wasn't what broke it | 16:30 |
| annegentle_ | fungi: ok | 16:30 |
| fungi | annegentle_: as i said earlier, the change before that one to only update on change-merged events is what broke it | 16:30 |
| fungi | annegentle_: because the script was looking for a condition which would never occur (we weren't calling it on change-merged events) | 16:31 |
| *** SergeyLukjanov has quit IRC | 16:31 | |
| *** ryanpetrello has quit IRC | 16:31 | |
| annegentle_ | fungi: ok then I'll +1 (but can't really test it) | 16:31 |
| annegentle_ | fungi: ok | 16:31 |
| *** ryanpetrello has joined #openstack-infra | 16:31 | |
| annegentle_ | clarkb: ah yeah removing the dest-address would've helped me read it :) | 16:32 |
| *** ryanpetrello has quit IRC | 16:32 | |
| fungi | annegentle_: a simpler approach to only opening the bug on change-merged would have been the patch i just posted, which stopped calling the script on patchset-created events and started calling it on change-merged instead | 16:32 |
| *** SergeyLukjanov has joined #openstack-infra | 16:32 | |
| *** ryanpetrello has joined #openstack-infra | 16:32 | |
| *** SergeyLukjanov has quit IRC | 16:32 | |
| annegentle_ | fungi: calling it on changed-merged is our desired state | 16:32 |
| fungi | annegentle_: good, since that's what my patch does | 16:33 |
| annegentle_ | fungi: well, then WOOHOO! :) | 16:33 |
| fungi | heh ;) | 16:33 |
| annegentle_ | fungi: are we all loopy this week or what? :) | 16:33 |
| * fungi is loopy every week | 16:34 | |
| *** rakhmerov has quit IRC | 16:34 | |
| annegentle_ | for once I get to go to a summit with the release done | 16:34 |
| annegentle_ | heh | 16:34 |
| fungi | that slightly increases the chances that you might be able to keep up with mordred's liver while we're there | 16:34 |
| clarkb | annegentle_: I bet that is a nice feeling | 16:34 |
| *** moted has joined #openstack-infra | 16:35 | |
| *** OlivierSanchez has quit IRC | 16:35 | |
| annegentle_ | fungi: no chance of that, I'm a lightweight drinker :) | 16:35 |
| annegentle_ | clarkb: yeah I'm giddy with anticipation. Now if only the install guide were done. gaaaahh | 16:36 |
| fungi | anybody know if there's a gerrit query syntax which would allow me to find changes that i specifically voted a certain way on (for example, changes where i voted <=-1)? | 16:36 |
| fungi | trying to update my review workflow with a step to look for reviews where people replied to my comments without uploading a new patchset | 16:37 |
| jeblair | fungi: i don't see it | 16:38 |
| fungi | didn't think so. best i can do is filter for watched changes with a negative vote by any reviewer | 16:38 |
| fungi | that'll have to do i guess | 16:38 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add a new ssh key for jeblair https://review.openstack.org/51661 | 16:38 |
| *** mriedem has quit IRC | 16:39 | |
| jeblair | fungi, clarkb, mordred: ^ that's a key i'm going to use on a device while traveling | 16:39 |
| *** zaro0508 has joined #openstack-infra | 16:40 | |
| *** dizquierdo has quit IRC | 16:41 | |
| *** osanchez has joined #openstack-infra | 16:41 | |
| *** aspiers has quit IRC | 16:43 | |
| *** rakhmerov has joined #openstack-infra | 16:43 | |
| *** rfolco has quit IRC | 16:44 | |
| annegentle_ | Ok, so we have 27 more DocImpact bugs since Sept 12 that weren't picked up by the automation. https://review.openstack.org/#/q/message:DocImpact+status:merged,n,z finds them all | 16:51 |
| annegentle_ | er sorry, wrong channel :) | 16:51 |
| *** markmcclain has joined #openstack-infra | 16:52 | |
| *** beagles has joined #openstack-infra | 16:56 | |
| *** SergeyLukjanov has joined #openstack-infra | 16:56 | |
| *** nicedice has joined #openstack-infra | 16:56 | |
| wenlock | whats clark's last name? | 16:57 |
| wenlock | miss type | 16:57 |
| wenlock | sorry | 16:57 |
| sdague | fungi: no, you can't do that level of detailed query | 16:59 |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 17:00 | |
| sdague | sadly... once we get to newer gerrit we've got a rest api though, right? | 17:00 |
| fungi | sdague: a pity :( | 17:00 |
| clarkb | there has been work upstream to use a lucene indexes for more complex queries | 17:00 |
| clarkb | but I think they found that has problems scaling (indexes are expensive) | 17:00 |
| fungi | that would rock | 17:00 |
| clarkb | not sure if they sorted the problems they ran into with projects that have a lot of history | 17:00 |
| fungi | and yeah, i imagine something on the order of what we have now for elasticsearch infrastructure to make that feasible | 17:01 |
| pleia2 | good morning | 17:01 |
| clarkb | pleia2: morning | 17:01 |
| clarkb | sdague: how are those etherpad style headings working out? | 17:01 |
| clarkb | sdague: :) we finally have those for you. sorry about the wait | 17:01 |
| sdague | clarkb: good, though I haven't done much more in etherpad since they went live | 17:01 |
| sdague | this week though, I need to start planning the qa sessions out | 17:02 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: switch over to os-loganalyze on logs.o.o https://review.openstack.org/51571 | 17:03 |
| sdague | fungi: commit message fixed | 17:03 |
| sdague | clarkb: no worries on the wait, it's all good | 17:03 |
| *** ben_duyujie has joined #openstack-infra | 17:04 | |
| *** derekh has quit IRC | 17:04 | |
| sdague | clarkb: if you want to check that out, the current os-loganalyze tree should fix all the missing log issues | 17:05 |
| *** guohliu has quit IRC | 17:05 | |
| fungi | sdague: you're improving the negativity of my review stats. thanks! ;) | 17:05 |
| sdague | hehe | 17:05 |
| clarkb | sdague: awesome. I will poke at that as soon as I have finished settling into this office move | 17:06 |
| sdague | I was +Aing a bunch of stuff in myself this weekend to get to functioning state, I'll stop doing that now to get eyes on it as it's going into prod at some point. So you might want to look at the os-loganalyze history just to familiarize with changes as well | 17:06 |
| clarkb | sdague: ok | 17:06 |
| *** SergeyLukjanov has joined #openstack-infra | 17:07 | |
| *** matty_dubs|lunch is now known as matty_dubs | 17:08 | |
| *** _david_ has joined #openstack-infra | 17:12 | |
| * fungi pops out for a quick errand and will brb | 17:13 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/reviewstats: Make openreviews generate quartiles instead of just median. https://review.openstack.org/51665 | 17:17 |
| jog0 | russellb: ^ | 17:17 |
| *** nati_ueno has joined #openstack-infra | 17:21 | |
| *** thomasbiege has joined #openstack-infra | 17:22 | |
| russellb | jog0: cool | 17:23 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Make openreviews generate quartiles instead of just median. https://review.openstack.org/51665 | 17:23 |
| russellb | jog0: regenerating stats now | 17:24 |
| *** thomasbiege has quit IRC | 17:24 | |
| jog0 | russellb: neato thanks | 17:24 |
| *** vipul is now known as vipul-away | 17:28 | |
| russellb | jog0: updated | 17:30 |
| *** senk has joined #openstack-infra | 17:30 | |
| clarkb | sdague: does os-loganalyze handle syslog and swift too? | 17:32 |
| *** vipul-away is now known as vipul | 17:33 | |
| clarkb | sdague: looks like ti skips them maybe? (this is fine, trying to look at history like you mentioed) | 17:33 |
| sdague | clarkb: yep, so now it has a stricter check of what it supports - https://github.com/openstack-infra/os-loganalyze/blob/master/os_loganalyze/wsgi.py#L148 | 17:33 |
| sdague | based on this regex on files - https://github.com/openstack-infra/os-loganalyze/blob/master/os_loganalyze/wsgi.py#L26 | 17:34 |
| *** markmcclain has quit IRC | 17:34 | |
| *** dprince has quit IRC | 17:34 | |
| sdague | so the passthrough filter will just ignore the level param if the file name isn't known to be supported | 17:35 |
| sdague | it will still do timestamp linking in the html though if it can | 17:35 |
| sdague | so console.html.gz will have timestamp links with this | 17:36 |
| clarkb | I see | 17:37 |
| *** SergeyLukjanov has quit IRC | 17:37 | |
| *** markmcclain has joined #openstack-infra | 17:37 | |
| *** DennyZhang has joined #openstack-infra | 17:37 | |
| *** rakhmerov has left #openstack-infra | 17:38 | |
| *** jerryz has joined #openstack-infra | 17:38 | |
| *** sarob has joined #openstack-infra | 17:45 | |
| *** markmcclain has quit IRC | 17:48 | |
| *** markmcclain has joined #openstack-infra | 17:48 | |
| clarkb | sdague: the config change to use os-loganalzyse lgtm. I am not able to babysit right now or I would approve. Hopefully can babysit after lunch | 17:49 |
| *** blamar has quit IRC | 17:50 | |
| *** oubiwann_ has joined #openstack-infra | 17:51 | |
| sdague | clarkb: sounds good | 17:52 |
| sdague | just let me know | 17:52 |
| *** ben_duyujie has quit IRC | 17:54 | |
| *** oubiwann has quit IRC | 17:54 | |
| hub_cap | so now that elastic recheck is telling me that i hit a particular bug, do i get to blindly reverify bug XXX and not feel bad? | 17:55 |
| *** miqui has joined #openstack-infra | 17:55 | |
| clarkb | hub_cap: I think we are hoping that some human interaction will still happen (this is why we aren't auto rechecking for you) | 17:56 |
| clarkb | hub_cap: but we should be giving you good enough bread crumbs most of the time | 17:56 |
| hub_cap | clarkb: kk thx | 17:56 |
| * hub_cap blindly clicks review (i kid i kid) | 17:56 | |
| *** DennyZhang has quit IRC | 17:57 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Increase timeout for neutron jobs https://review.openstack.org/51673 | 17:57 |
| *** pblaho has joined #openstack-infra | 17:59 | |
| *** senk has quit IRC | 18:01 | |
| *** boris-42 has joined #openstack-infra | 18:02 | |
| clarkb | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=116&rra_id=all etherpad.o.o is holding up | 18:03 |
| *** sarob has quit IRC | 18:05 | |
| *** lcestari has quit IRC | 18:07 | |
| *** qba73 has quit IRC | 18:07 | |
| *** senk has joined #openstack-infra | 18:08 | |
| *** mriedem has joined #openstack-infra | 18:09 | |
| ekarlso | does one need to register a package before it can be uploaded to pypi ? (python-libraclient) | 18:11 |
| clarkb | ekarlso: yes, and you need to make the openstackci account a maintainer | 18:11 |
| *** johnthetubaguy has quit IRC | 18:11 | |
| clarkb | ekarlso: eg https://pypi.python.org/pypi/gear/0.4.0 | 18:12 |
| *** senk has quit IRC | 18:12 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Convert test_classifier to be a unit test https://review.openstack.org/50733 | 18:12 |
| *** senk has joined #openstack-infra | 18:12 | |
| mtreinish | clarkb: want to enable gating unit tests for elastic-recheck?: https://review.openstack.org/#/c/50743/ | 18:13 |
| clarkb | mtreinish: looking. for https://review.openstack.org/#/c/51673/ do we expect the tests to take more than an hour? I ask because now that we have parallel testing with testr for the other jobs it might cause a bit of a revolt if we go back to >1hour test times :) | 18:15 |
| sdague | clarkb: neutron tests take > 1 hr | 18:16 |
| mtreinish | clarkb: it's only for neutron which isn't parallel yet | 18:16 |
| sdague | because they can't run parallel yet | 18:16 |
| mtreinish | we can drop it back down once we get parallel working on neutron | 18:16 |
| *** nicedice_ has joined #openstack-infra | 18:16 | |
| *** pblaho has quit IRC | 18:17 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Remove query for bug 1227915 https://review.openstack.org/51682 | 18:18 |
| uvirtbot | Launchpad bug 1227915 in cinder "logical volume "volume-x" already exists in volume group "y" (dup-of: 1223469)" [High,Confirmed] https://launchpad.net/bugs/1227915 | 18:18 |
| uvirtbot | Launchpad bug 1223469 in cinder "Volume create tgtadmin fails with error code 22" [High,Fix released] https://launchpad.net/bugs/1223469 | 18:18 |
| mtreinish | jog0: nice you got a triple ^^^ | 18:19 |
| jog0 | mtreinish: ! | 18:20 |
| *** nicedice has quit IRC | 18:20 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add unit tests to gating jobs on elastic-recheck https://review.openstack.org/50743 | 18:20 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for 1239637 https://review.openstack.org/51579 | 18:21 |
| *** sarob has joined #openstack-infra | 18:21 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Remove query for bug 1227915 https://review.openstack.org/51682 | 18:22 |
| uvirtbot | Launchpad bug 1227915 in cinder "logical volume "volume-x" already exists in volume group "y" (dup-of: 1223469)" [High,Confirmed] https://launchpad.net/bugs/1227915 | 18:22 |
| uvirtbot | Launchpad bug 1223469 in cinder "Volume create tgtadmin fails with error code 22" [High,Fix released] https://launchpad.net/bugs/1223469 | 18:22 |
| *** senk has quit IRC | 18:23 | |
| lifeless | moted: it takes a few seconds for floating ip associations to take effect under neutron | 18:33 |
| lifeless | bah | 18:33 |
| lifeless | mordred: ^ | 18:33 |
| lifeless | moted: sorry | 18:33 |
| moted | lifeless: np | 18:33 |
| *** annegentle_ has quit IRC | 18:33 | |
| *** annegentle has joined #openstack-infra | 18:34 | |
| *** senk has joined #openstack-infra | 18:34 | |
| *** jerryz has quit IRC | 18:37 | |
| mordred | lifeless: awesome | 18:38 |
| mordred | lifeless: is there any way to check to see if they're actualy in effect? | 18:38 |
| clarkb | fungi: I have dug through the jenkins changelog and bug tracker but haven't found anything like what we are experiencing. We may want to try and collect more info then submit a bug ourselves | 18:38 |
| *** dhouck_ has joined #openstack-infra | 18:38 | |
| lifeless | mordred: no, you need to know if the neutron agent on the network node (which is where the NAT rules live) has done it's thing | 18:38 |
| mordred | sigh | 18:39 |
| fungi | clarkb: the load issue? is it happening again? | 18:39 |
| clarkb | fungi: it isn't happening now | 18:39 |
| mordred | and I suppose it's too much to ask for the cloud ot actually know that status and report it | 18:39 |
| jeblair | lifeless: you can't query it for state like you can a server? | 18:39 |
| clarkb | fungi: but looking at graphs it seems to steadily ramp up (we have a ways to go before it falls over) | 18:39 |
| mordred | lifeless: have any of the people designing this ever tried to use a cloud? | 18:39 |
| clarkb | fungi: also comparing to jenkins01, 02 seems so much busier | 18:39 |
| lifeless | BobBall: I haven't looked into what drivers HPCloud uses at all | 18:39 |
| lifeless | BobBall: sorry | 18:40 |
| mordred | how do I tell if I'm still waiting for the floating ip to take effect, or if something has failed? | 18:40 |
| mordred | jeblair: client.floating_ips.delete(self.args['ip_id']) | 18:40 |
| mordred | KeyError: 'ip_id' | 18:40 |
| lifeless | jeblair: not as far as I know | 18:40 |
| mordred | jeblair: I'm hitting that on teardown - any idea where I should look? | 18:40 |
| *** jerryz has joined #openstack-infra | 18:40 | |
| mordred | lifeless: ERROR:nodepool.image.build.tripleo-test-cloud.tripleo-precise:tripleo-incubator/scripts/pull-tools: line 27: TRIPLEO_ROOT: unbound variable | 18:41 |
| lifeless | mordred: I suspect all of them have run a cloud in devstack style | 18:41 |
| mordred | lifeless: this is where I'm getting in nodepool + tripeo | 18:41 |
| lifeless | mordred: and are used to grvelling in log files / being patient | 18:41 |
| lifeless | mordred: rather than treating things as 'if I were a user, how would I ...' | 18:41 |
| jeblair | mordred: return self.submitTask(DeleteFloatingIPTask(floating_ip=ip_id)) | 18:42 |
| lifeless | mordred: source refresh-env $wherever_you_put_your_tripleo_root | 18:42 |
| jeblair | mordred: that should probably be ip_id=ip_id | 18:42 |
| fungi | clarkb: i think jeblair's nodepool changes may solve some of the load disparity between 01 and 02? | 18:42 |
| pleia2 | lifeless, mordred - looking at how toci does some of this waiting stuff might help, since that's all automated | 18:42 |
| mordred | jeblair: ok. I'll go look for htat line - it didn't look lke id_id existed in the current context | 18:42 |
| jeblair | fungi: i don't think i've fixed that yet. | 18:42 |
| mordred | pleia2: we do that a lot in our current stuff like nodepool | 18:43 |
| pleia2 | ah, ok | 18:43 |
| jerryz | lifeless: is there any progress on this bug: https://bugs.launchpad.net/openstack-ci/+bug/1217722 | 18:43 |
| mordred | pleia2: the problem is - in production, when we aska cloud we don't control to do something and then we have to just poll over and over again until it works | 18:43 |
| uvirtbot | Launchpad bug 1217722 in openstack-ci "'rake' or 'rake lint' blows up" [Undecided,New] | 18:43 |
| fungi | jeblair: oh, right, you were still philosophizing over that this weekend | 18:43 |
| mordred | pleia2: it's really tough to know when something's actually failed | 18:43 |
| pleia2 | mordred: yeah | 18:43 |
| * mordred stabs humans | 18:43 | |
| pleia2 | I see | 18:43 |
| pleia2 | hah | 18:43 |
| jeblair | yeah, the wait-for-resource paradigm is standardized in nodepool | 18:43 |
| jeblair | mordred, pleia2: but it assumes that there's actually some kind of success condition | 18:44 |
| jeblair | which doesn't seem to be the case here? | 18:44 |
| mordred | jeblair: I think the success condition is "your IP works" | 18:44 |
| mordred | lifeless: btw - the actual error was that we didn't have a secgroup defined | 18:44 |
| pleia2 | I could do with some catching up for a couple of days I was gone, what's the status of the tripleo nodepool thing I wrote last week and had to be disabled? | 18:44 |
| mordred | pleia2: I'm working on debugging it right now | 18:45 |
| pleia2 | mordred: ok, great | 18:45 |
| mordred | pleia2: see email I just sent to mailing list... | 18:45 |
| jeblair | mordred: have we seen this problem yet? | 18:45 |
| mordred | pleia2: but the tl;dr is that spinning up a local nodepool with the config for tripleo is a needed thing | 18:45 |
| lifeless | mordred: oh, well not having a sensible default security group | 18:45 |
| pleia2 | mordred: oh, haven't caught up on -dev yet :) | 18:45 |
| lifeless | mordred: or a custom one - yearh, that 'll fail | 18:45 |
| mordred | pleia2: so that we can track down it working before we re-proposed to infra | 18:46 |
| jeblair | mordred: nodepool currently waits for an ip to be reported as associated with a server | 18:46 |
| lifeless | jerryz: no idea | 18:46 |
| lifeless | jerryz: I just reported it :> | 18:46 |
| mordred | jeblair: yah. I do not think we've seen the problem lifeless is saying might be there in the wild | 18:46 |
| lifeless | jerryz: when I was told to use it to see if a patch was good. | 18:46 |
| jerryz | jerryz: just curious how precise slaves work with that. | 18:47 |
| jeblair | mordred: ok, let's see if it's a problem then. perhaps it's covered by the existing check. | 18:47 |
| jerryz | lifeless: did you do it locally or it ran on precise slaves? | 18:47 |
| fungi | jerryz: i think you have to gem install puppetlabs_spec_helper (we may already do that on our slaves) | 18:48 |
| fungi | looking | 18:48 |
| lifeless | fungi: jeblair: I have an unmirrored revision on git.o.o | 18:48 |
| lifeless | jerryz: locally | 18:48 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/nodepool: Make ip_id arg match calling and consuming https://review.openstack.org/51687 | 18:48 |
| jerryz | lifeless: thanks | 18:49 |
| mordred | jerryz: agree | 18:49 |
| lifeless | jeblair: de1dc315539f0c69c30c9101d1db93735bc78c26 is in gerrit as merged, but not present in https://git.openstack.org/cgit/openstack/tripleo-image-elements when I pull it from 138.35.77.3 | 18:49 |
| mordred | lifeless, pleia2: in any case, I believe I'm down to nodepool works with grizzly cloud now (once we land the changes I've submitted, plus jeblair's plus pabelanger's) | 18:50 |
| mordred | lifeless, pleia2: and the tripleo script themselves are now unhappy | 18:50 |
| mordred | which is great | 18:50 |
| lifeless | git.o.o is resolving to 192.237.223.224 for me | 18:50 |
| pleia2 | mordred: yeah, so right now in this first step I don't think we want any real tripleo stuff running | 18:50 |
| *** mindjive1 is now known as mindjiver | 18:50 | |
| pleia2 | mordred: we just want it to run a simple shell script as POC and I'm working on actually running tripleo | 18:50 |
| fungi | jerryz: yeah, we install it on the slaves... http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/manifests/slave.pp#n123 | 18:50 |
| pleia2 | mordred: see iterations 1 & 2 https://etherpad.openstack.org/p/tripleo-test-cluster | 18:51 |
| mordred | pleia2: totally. but the script in openstack-infra/config that is used to make the image | 18:52 |
| mordred | pleia2: for tripleo nodes | 18:52 |
| mordred | pleia2: is unhappy and fails | 18:52 |
| pleia2 | mordred: oh! I see | 18:52 |
| mordred | when the image creation process runs it | 18:52 |
| mordred | pleia2: yah! we're to the good stuff :) | 18:52 |
| fungi | lifeless: i'll have a look on the git server farm. maybe one of them has that repo in a bad state | 18:52 |
| mordred | pleia2: lemme send a follow up to that email about the nodepool patches that need to be applied or land for it to work | 18:52 |
| pleia2 | mordred: cool, thanks | 18:52 |
| lifeless | fungi: tripleo-image-elements | 18:52 |
| lifeless | pleia2: pull-tools is failing | 18:54 |
| lifeless | pleia2: perhaps using refresh-env would make sense | 18:54 |
| fungi | lifeless: i see that commit on all four of the git servers | 18:55 |
| lifeless | git log origin/master | head -n1 | 18:55 |
| lifeless | commit 98d3195d2d18851686652af81acbafee2a529844 | 18:55 |
| mordred | jeblair, pleia2, clarkb: these are the patches I've been running with locally in order to make things work: http://paste.openstack.org/show/48445/ | 18:56 |
| lifeless | git remote show -n origin | 18:56 |
| lifeless | * remote origin | 18:56 |
| lifeless | Fetch URL: https://git.openstack.org/cgit/openstack/tripleo-image-elements | 18:56 |
| clarkb | lifeless: did you fetch origin/master ? | 18:56 |
| lifeless | clarkb: yes | 18:56 |
| jeblair | fungi, lifeless: i cloned using the cgit urls on all 4 servers and the commit only shows up in the clone from 03 | 18:56 |
| lifeless | git fetch origin | 18:56 |
| lifeless | root@undercloud-notcompute-jws3awlsb2kh:/opt/stack/tripleo-image-elements# | 18:56 |
| *** hashar has joined #openstack-infra | 18:56 | |
| fungi | lifeless: does git show 98d3195d2d18851686652af81acbafee2a529844 work? it worked for me in the bare repo on all 4 servrers | 18:57 |
| lifeless | fungi: it does for me but that is because I fetched it from gerrit | 18:57 |
| lifeless | fungi: see jeblair's comment | 18:57 |
| clarkb | so maybe the ref for master didn't get updated? | 18:57 |
| jerryz | fungi: thanks! | 18:57 |
| fungi | lifeless: oh, sorry, i was looking at the other ref you pasted, de1dc315539f0c69c30c9101d1db93735bc78c26 | 18:57 |
| lifeless | fungi: de1dc315539f0c69c30c9101d1db93735bc78c26 is on top of 98d3195d2d18851686652af81acbafee2a529844 | 18:58 |
| lifeless | fungi: 98 is what I get when I fetch, de is what I should | 18:58 |
| jeblair | lifeless, fungi: i repeated that using the git:// protocol, and it's the same -- only on git03 | 18:58 |
| pleia2 | mordred: great, thanks | 18:58 |
| fungi | weird. it's definitely in the repos themselves | 18:58 |
| *** oubiwann_ is now known as oubiwann | 18:58 | |
| fungi | i wonder why pull doesn't find it | 18:58 |
| lifeless | fungi: what does git log master | head -n1 get you? | 18:59 |
| *** senk has quit IRC | 18:59 | |
| *** SergeyLukjanov has joined #openstack-infra | 18:59 | |
| jeblair | i see no related errors in the gerrit log | 18:59 |
| jeblair | fungi, lifeless: ^ | 18:59 |
| jeblair | fungi, lifeless: i have to run now, sorry | 18:59 |
| *** alcabrera has quit IRC | 19:00 | |
| fungi | lifeless: 98d3195d2d18851686652af81acbafee2a529844 on git02 and git04, 2b2aa7cb887a9cce08f7dc0d2a81520bcd283a3a on 01 and de1dc315539f0c69c30c9101d1db93735bc78c26 on 03 | 19:00 |
| fungi | so clearly replication is not keeping some of those in sync | 19:01 |
| fungi | trying to figure out why | 19:02 |
| lifeless | fungi: those are head refs? | 19:02 |
| clarkb | fungi: it is syncing the actual contents but not updating the master ref? | 19:03 |
| fungi | lifeless: yes, git log master | head -n1 | 19:03 |
| *** jpich has left #openstack-infra | 19:03 | |
| lifeless | fungi: so yeah, this is borken :) | 19:03 |
| fungi | clarkb: it's a possibility, though i'm not seeing that behavior on repos i've been using | 19:03 |
| *** david_h has joined #openstack-infra | 19:03 | |
| fungi | lifeless: yes indeed | 19:04 |
| lifeless | I only spotted this because we had a monkey patch in place in prod | 19:04 |
| lifeless | and I went to undo it now the commit had landed | 19:04 |
| lifeless | and ... things that make you go hmmm | 19:05 |
| fungi | http://paste.openstack.org/show/48447/ | 19:06 |
| fungi | i'm going to see if there's something coincident about the vintage of those stale head refs | 19:06 |
| *** nati_uen_ has joined #openstack-infra | 19:07 | |
| fungi | huh, all from the past few days | 19:10 |
| *** nati_ueno has quit IRC | 19:10 | |
| fungi | frustrating. gerrit show-queue showed nothing going on, but gerrit replicate openstack/tripleo-image-elements caused them all to sync up | 19:13 |
| fungi | hunting around now for any similar bugs filed on gerrit | 19:14 |
| *** che-arne has joined #openstack-infra | 19:18 | |
| *** luhrs1 has joined #openstack-infra | 19:18 | |
| *** luhrs1 has quit IRC | 19:19 | |
| lifeless | fungi: it's flipping back and forth ? :) | 19:20 |
| lifeless | fungi: wonder if this is related to the 'X showed unmerged but was actually merged' thing | 19:21 |
| *** _david_ has quit IRC | 19:21 | |
| fungi | it's possible, though that was a while back right? | 19:21 |
| lifeless | yeah | 19:21 |
| *** julim has quit IRC | 19:22 | |
| *** aspiers has joined #openstack-infra | 19:22 | |
| fungi | wow, 24 counts of pushreplication errors in the gerrit error log from today | 19:23 |
| fungi | i hate the way java apps like to fill up their logs with multiline stack traces | 19:23 |
| *** DennyZhang has joined #openstack-infra | 19:25 | |
| *** eharney has joined #openstack-infra | 19:27 | |
| fungi | most recent one for tripleo-image-elements was on thursday... http://paste.openstack.org/show/48448/ | 19:28 |
| *** dcramer_ has quit IRC | 19:29 | |
| fungi | not sure if gerrit expects to recover gracefully or not, but it's odd that the later commits did get replicated, just not the state of the head ref | 19:30 |
| *** sarob has quit IRC | 19:30 | |
| *** sarob has joined #openstack-infra | 19:31 | |
| fungi | also, head refs got updated on all of the git servers for a couple days after that error was logged, which makes me think it's unrelated | 19:31 |
| sdague | fungi: I should point you to our openstack logs some time :) | 19:32 |
| sdague | reed: multiline stack traces | 19:32 |
| fungi | sdague: no need ;) | 19:32 |
| fungi | sdague: in a debug log they're not so bad, but elsewhere... annoying | 19:33 |
| *** nati_uen_ has quit IRC | 19:34 | |
| *** nati_ueno has joined #openstack-infra | 19:34 | |
| fungi | https://groups.google.com/forum/#!msg/repo-discuss/Y71yLwhC6hk/fex3cEgLcvEJ is interesting (circa 2.2 though) | 19:35 |
| *** sarob has quit IRC | 19:36 | |
| *** atiwari has joined #openstack-infra | 19:36 | |
| *** hashar has quit IRC | 19:38 | |
| *** jaypipes has quit IRC | 19:41 | |
| *** beagles has quit IRC | 19:45 | |
| *** blamar has joined #openstack-infra | 19:45 | |
| *** dmakogon has joined #openstack-infra | 19:46 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 19:46 | |
| *** _SergeyLukjanov has quit IRC | 19:46 | |
| reed | sdague, not sure I follow :) | 19:46 |
| *** hashar has joined #openstack-infra | 19:48 | |
| sdague | reed: oh, that was supposed to be 're:' | 20:00 |
| sdague | :) | 20:00 |
| reed | now it makes more sense :) | 20:00 |
| clarkb | jeblair: mordred: fungi: I am going to approve https://review.openstack.org/#/c/51571/3 did anyone else want to review it before I do that? | 20:01 |
| clarkb | this is the change that uses the new os-loganalyze project | 20:01 |
| mordred | nope | 20:01 |
| mordred | love it | 20:02 |
| *** marun has joined #openstack-infra | 20:02 | |
| clarkb | ok approved | 20:02 |
| *** lcestari has joined #openstack-infra | 20:02 | |
| fungi | ahh, yep. already reviewed that one anyway | 20:03 |
| *** qba73 has joined #openstack-infra | 20:04 | |
| openstackgerrit | A change was merged to openstack-infra/config: switch over to os-loganalyze on logs.o.o https://review.openstack.org/51571 | 20:04 |
| clarkb | the other thing I am going to do today is get bup backups running on new etherpad.openstack.org | 20:04 |
| clarkb | before I do that I want to sort out what I should do with the old backups on old etherpad.openstack.org. first thought is to tar them up and stash them on new etherpad.openstack.org but then they do not get rotated out (I could manually remove them in a month) | 20:05 |
| clarkb | any opinions on that? | 20:05 |
| clarkb | or we could just let them live on the old server and die with the old server when it is removed | 20:06 |
| fungi | i'm not too concerned about them sitting on the new server collecting dust until someone thinks to delete them, but leaving the old server up for a month can't hurt either | 20:06 |
| *** dcramer_ has joined #openstack-infra | 20:07 | |
| *** qba73 has quit IRC | 20:09 | |
| *** SergeyLukjanov has joined #openstack-infra | 20:13 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Change image debug interpolation order https://review.openstack.org/51641 | 20:13 |
| clarkb | fungi: re Gerrit repo replication are you seeing the failure to lock exceptions in the error log? | 20:17 |
| *** DennyZhang has quit IRC | 20:17 | |
| fungi | clarkb: i hadn't spotted one, no | 20:19 |
| *** david_h has quit IRC | 20:19 | |
| fungi | i'll look again | 20:19 |
| fungi | nada | 20:20 |
| *** blamar has quit IRC | 20:20 | |
| *** nati_uen_ has joined #openstack-infra | 20:23 | |
| *** dizquierdo has joined #openstack-infra | 20:25 | |
| *** nati_ueno has quit IRC | 20:26 | |
| mtreinish | fungi, mordred, jeblair: can one of you nudge https://review.openstack.org/#/c/51673/ through? It's a prereq for enabling more tests on neutron.(I need it to confirm my exclude list) The current list is looking like a full tempest run -10 tests. | 20:29 |
| jog0 | mordred: why 3 change-ids https://review.openstack.org/#/c/51039/ | 20:32 |
| mordred | jog0: bug in script :( | 20:33 |
| mtreinish | mordred: did clarkb's fix for that work? | 20:33 |
| jog0 | mordred: -1 for you | 20:34 |
| clarkb | mtreinish: I think it should work but we haven't updated requirements since | 20:34 |
| clarkb | we might also want to abandon the existing derped changes | 20:34 |
| mtreinish | clarkb: you'd have to login as Jenkins to do that right? | 20:36 |
| mtreinish | the tempest one won't pass jenkins right now so you can just pull that one out if you want to test | 20:36 |
| mtreinish | (or wait until thurs for it to autoexpire) | 20:36 |
| openstackgerrit | A change was merged to openstack-infra/config: Increase timeout for neutron jobs https://review.openstack.org/51673 | 20:36 |
| mtreinish | fungi: thanks for the +A, if you want to watch the slow pseudo full neutron tempest runs: https://review.openstack.org/#/c/51275/ | 20:38 |
| mtreinish | it's got a fun regex | 20:38 |
| clarkb | mtreinish: Gerrit admins can abandon them. I haven't done that yet, but I think I will if no one disagrees it should be done | 20:38 |
| *** vipul is now known as vipul-away | 20:39 | |
| mtreinish | clarkb: I guess being an admin has it's advantages. :) | 20:39 |
| clarkb | mtreinish: there are a few :) | 20:39 |
| clarkb | mtreinish: sdague: with os-loganalyze in place we are indexing keystone, swift and syslog again so you can use those files in queries | 20:40 |
| clarkb | jog0: ^ | 20:40 |
| mtreinish | clarkb: https://review.openstack.org/#/c/51041/ you can go ahead an abandon it, since build_sphinx doesn't work | 20:40 |
| mtreinish | clarkb: ok cool | 20:40 |
| jog0 | clarkb: awesome | 20:41 |
| clarkb | mordred: is abandoning the list of changes here with mutliple change ids something that hsould be done? https://review.openstack.org/#/q/status:open+topic:openstack/requirements,n,z | 20:42 |
| *** sarob has joined #openstack-infra | 20:42 | |
| *** sarob has quit IRC | 20:46 | |
| mordred | clarkb: sure - except I would like to actually sync to global requirements before final release - but the cat is probably out of the box now | 20:47 |
| *** SergeyLukjanov has quit IRC | 20:47 | |
| clarkb | mordred: I think syncing is a good thing before release. I just think those change are a bit messy | 20:47 |
| clarkb | mordred: newly generated ones should be fine. maybe we should try manually triggering the job after the existing changes are abandoned? | 20:47 |
| mordred | clarkb: totes. I'm just saying I think most projects aren't expecting additional changes now | 20:47 |
| clarkb | mordred: up to you (we could just go with the slightly dirty changes) | 20:47 |
| mordred | I could go either way | 20:48 |
| *** SergeyLukjanov has joined #openstack-infra | 20:48 | |
| hub_cap | mordred: lifeless clarkb image caching question. I think it migth make sense for me to run the 10*cache scripts that are in dib (or have a --cache-image flag or something in dib) for this caching. im not sure just curling the images is the thing to do because dib does a _lot_ more than caching the image in the 10* step and we can bypass it all if we run it ahead of time and pass --offline to dib in the actual jobs | 20:48 |
| mtreinish | clarkb, sdague: found a small issue with os-loganalyze: http://logs.openstack.org/96/51696/2/check/check-tempest-devstack-vm-neutron-pg-isolated/60bf1a1/testr_results.html.gz | 20:48 |
| clarkb | mtreinish: that is awesome :) | 20:49 |
| clarkb | sdague: maybe we shouldn't touch the .html files for now? | 20:49 |
| clarkb | hub_cap: right, iirc actual building of some stuff needs to happen or there will be many cache misses | 20:49 |
| hub_cap | clarkb: ok good. i think originally there was thought that the 10* script was too much _stuff_ to run | 20:50 |
| sdague | mtreinish: awesome :) | 20:51 |
| sdague | ok, I'll provide a fix in a few for that | 20:51 |
| *** nati_uen_ has quit IRC | 20:52 | |
| lifeless | sorry, on a call, will drop in later | 20:52 |
| *** nati_ueno has joined #openstack-infra | 20:53 | |
| hub_cap | lifeless: no worries we can chat later | 20:53 |
| *** yamahata has quit IRC | 20:53 | |
| *** yamahata has joined #openstack-infra | 20:54 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Backup etherpad.o.o with bup. https://review.openstack.org/51703 | 20:55 |
| *** michchap has joined #openstack-infra | 20:55 | |
| *** michchap has quit IRC | 20:56 | |
| clarkb | mordred: I won't abandon any changed yet then | 20:57 |
| clarkb | mordred: I don't think there is a rush to change anyhting right now so we can mull it over | 20:57 |
| *** michchap has joined #openstack-infra | 20:59 | |
| *** emagana has joined #openstack-infra | 20:59 | |
| *** emagana has quit IRC | 20:59 | |
| *** emagana has joined #openstack-infra | 21:00 | |
| *** amotoki has joined #openstack-infra | 21:01 | |
| *** HenryG has joined #openstack-infra | 21:01 | |
| *** dmakogon_ has joined #openstack-infra | 21:02 | |
| clarkb | fungi: https://review.openstack.org/#/c/50189/1 may interest you as you have kicked the irc bots a bit | 21:03 |
| *** dmakogon has quit IRC | 21:05 | |
| *** adalbas has quit IRC | 21:05 | |
| *** julim has joined #openstack-infra | 21:06 | |
| openstackgerrit | A change was merged to openstack-infra/gerritbot: Reconnect to IRC on send error https://review.openstack.org/50189 | 21:07 |
| *** miqui has quit IRC | 21:10 | |
| *** sarob has joined #openstack-infra | 21:11 | |
| *** lcestari has quit IRC | 21:13 | |
| *** alexpilotti has quit IRC | 21:13 | |
| *** vipul-away is now known as vipul | 21:14 | |
| sdague | mordred: ... pbr error in keystone? - http://logs.openstack.org/76/51676/2/check/check-tempest-devstack-vm-full/0aeac34/console.html | 21:15 |
| *** sarob has quit IRC | 21:17 | |
| *** sarob has joined #openstack-infra | 21:18 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: only run the console.html logs through os-loganalyzer https://review.openstack.org/51707 | 21:20 |
| sdague | clarkb: that should fix the testr issue | 21:20 |
| sdague | but keep things for the console.html logs | 21:20 |
| clarkb | sdague: thanks | 21:21 |
| mordred | sdague: why do yo uthink it's pbr? | 21:21 |
| mordred | oh. | 21:21 |
| mordred | I see it | 21:21 |
| *** esheffield has joined #openstack-infra | 21:21 | |
| *** sarob has quit IRC | 21:22 | |
| pleia2 | mordred: I don't know how you travel so much, I just went across the country last night and I'm beat today :) | 21:23 |
| esheffield | clarkb: it was suggested that I ping you about a problem I've encountered | 21:25 |
| clarkb | esheffield: ok | 21:25 |
| clarkb | pleia2: if you get mordred to share his secret you should share it with the rest of us | 21:25 |
| esheffield | clarkb: I have a patch failing in jenkins | 21:26 |
| esheffield | clarkb: but it passes locally | 21:26 |
| esheffield | clarkb: the patch is https://review.openstack.org/#/c/46507/ | 21:26 |
| esheffield | clarkb: it seems to be the same as bug https://bugs.launchpad.net/nova/+bug/1180670 which was marked invalid at some point | 21:27 |
| uvirtbot | Launchpad bug 1180670 in nova "XenAPIVMTestCase.test_instance_snapshot_fails_with_no_primary_vdi sometimes fails" [Medium,Invalid] | 21:27 |
| mordred | sdague: why the heck is that only doing that in that one patch? | 21:27 |
| mordred | pleia2: you just learn to leave in pain constantly :) | 21:27 |
| pleia2 | mordred: haha | 21:27 |
| clarkb | esheffield: how old is the parent commit of that patch? I wonder if maybe rebasing on latest master would pull in the fix noted in that bug report | 21:29 |
| *** aspiers has quit IRC | 21:29 | |
| mordred | sdague: omg. so - | 21:29 |
| esheffield | clarkb: I just rebased it this past friday | 21:29 |
| clarkb | esheffield: in general test failures like that are a result of inter test conflicts. We use testr to run tests in parallel, testr does this by partitioning the set of tests then running each partition in its own process. This partitioning results in different test orders across runs which can cause conflicts | 21:30 |
| clarkb | esheffield: typical conflicts arise when test A requires test B to run first or if test A requires test B to not run first | 21:30 |
| mordred | sdague: we're running update.py on the pbr repo | 21:30 |
| mordred | sdague: which is copying in a setup.py that setup_requires on pbr | 21:30 |
| mordred | sdague: in devstack | 21:31 |
| mordred | how has this eer worked? | 21:31 |
| esheffield | clarkb: interesting - is there a good approach for fixing that or just have to retry the tests later and hope for a different ordering | 21:31 |
| clarkb | esheffield: to hunt those down load http://logs.openstack.org/07/46507/6/check/gate-nova-python27/dfc5885/subunit_log.txt.gz into testr as the most recent test run, then do `testr run --analyze-isolation` this causes testr to do test bisection looking for conflicts | 21:31 |
| esheffield | clarkb: also, that test worked on friday after the rebase, but I had a different failure which I pushed up today and encountered this problem | 21:32 |
| *** fifieldt_ has joined #openstack-infra | 21:32 | |
| clarkb | esheffield: you may need to convert subunit_log.txt to subunit v2 before testr loading the file. `subunit-1to2 < subunit_log.txt > v2.txt && testr load v2.txt && testr run --analyze-isolation` | 21:32 |
| sdague | mordred: interesting.... ? | 21:32 |
| esheffield | clarkb: ok, will try that. thanks | 21:32 |
| clarkb | esheffield: right the test order is not deterministic. So if a test breaks we may not see it consistently break (which is both a good and a bad thing, good that it fails, bad that it doesn't fail 100% of the time) | 21:33 |
| *** hashar has quit IRC | 21:33 | |
| esheffield | clarkb: thanks for the suggestions - I'll try them out and see if I can find anything! | 21:34 |
| *** mriedem has quit IRC | 21:34 | |
| clarkb | esheffield: good luck. let me know if you have more testr/unittest framework questions | 21:34 |
| *** thomasm has quit IRC | 21:35 | |
| openstackgerrit | Monty Taylor proposed a change to openstack/requirements: Only sync things that have the bullet https://review.openstack.org/51710 | 21:36 |
| mordred | sdague: ^^ | 21:36 |
| *** michchap has quit IRC | 21:38 | |
| *** michchap has joined #openstack-infra | 21:38 | |
| *** matty_dubs is now known as matty_dubs|gone | 21:41 | |
| *** aspiers has joined #openstack-infra | 21:42 | |
| jog0 | mordred: NOVA_RAX_AUTH http://docs.rackspace.com/servers/api/v2/cs-gettingstarted/content/section_gs_install_nova.html http://git.openstack.org/cgit/openstack/python-novaclient/tree/novaclient/client.py#n322 | 21:44 |
| *** jhesketh has quit IRC | 21:44 | |
| jog0 | I think we should kill that block of code but don't think that will go over so well | 21:44 |
| mordred | jog0: O M G I thought I deleted that 6 months ago | 21:45 |
| mordred | jog0: I don't care. it's completely inappropriate. I raised a stink about it a while ago | 21:45 |
| mordred | chmouel and I wrote a set of auth plugin drivers | 21:45 |
| mordred | also | 21:45 |
| mordred | you don't need it to connect to rax anymore | 21:45 |
| * mordred is angry | 21:45 | |
| jog0 | thats not what the random doc I Found on teh Internets said | 21:46 |
| mordred | it's wrong | 21:46 |
| mordred | you only need that if you want to use rax's proprietary api key extension | 21:46 |
| *** vipul is now known as vipul-away | 21:46 | |
| mordred | which is not our just to support | 21:46 |
| hub_cap | quick run, mordred is turning gren | 21:46 |
| hub_cap | *green | 21:46 |
| hub_cap | and his pants are ripping | 21:46 |
| mordred | if you use username/password, it works without us carrying code | 21:46 |
| mordred | jog0: oh - so that whole system is deprecated I thought | 21:47 |
| mordred | the auth_system | 21:47 |
| jog0 | mordred: want to push out this patch? | 21:47 |
| mordred | well, I mean, I'm the maintainer of the rax auth plugin :) | 21:47 |
| jog0 | chmouel's comment implies it is deprecated | 21:47 |
| mordred | it is | 21:48 |
| *** dhouck_ has quit IRC | 21:48 | |
| jog0 | mordred: heh so who better to remove it | 21:48 |
| sdague | mordred: unit tests? | 21:48 |
| jeblair | that code looks like it's a backwards-compat shim for the env variable, so that it uses the plugin | 21:48 |
| * jog0 likes seeing code get killed | 21:48 | |
| jeblair | mordred, jog0: ^ | 21:48 |
| jeblair | mordred, jog0: (so it's deprecating the env var, not deprecating the plugin) | 21:49 |
| jeblair | mordred, jog0: (which i agree does not seem to be necessary, though tbh, does provide something you can't get by using a password) | 21:49 |
| sdague | mordred: also, is there a bug for that? | 21:49 |
| mordred | jeblair: last I talked to chmoel, I thought we were ditching the whole auth plugin approach | 21:49 |
| mordred | chmouel. not chmoel | 21:50 |
| clarkb | jeblair: https://review.openstack.org/#/c/51703/ I think you will like that change | 21:50 |
| jog0 | mordred: we should get rid of auth plugins | 21:50 |
| jeblair | mordred: wfm, i don't have any input there; just trying to 'help' clarify the conversation. :) | 21:50 |
| *** hemna is now known as hemnafk | 21:50 | |
| sdague | the pbr thing, so I can recheck with it | 21:50 |
| jog0 | OpenStaack uses keystone for auth not 3rd party plugin | 21:50 |
| mordred | jeblair: :) | 21:50 |
| *** vipul-away is now known as vipul | 21:50 | |
| mordred | sdague: nope. I just wrote the patch - filing bugs is too hard | 21:50 |
| sdague | heh | 21:51 |
| mordred | sdague: you want me ot file a bug real quick? | 21:51 |
| jeblair | clarkb: i love that change | 21:51 |
| jog0 | jeblair: I am trying to find the actual rackspace auth code it may not be in trunk | 21:51 |
| jeblair | jog0: i believe it's in the plugin | 21:51 |
| sdague | mordred: if you think it's fixed, I'm fine, I'll just recheck no bug it | 21:51 |
| mordred | I think it should be | 21:51 |
| mordred | I mean- if my patch above clear, and it still doesn't work, there is _another_ bug we'll need to track down | 21:52 |
| chmouel | jog0: i think we can't remove it just yet | 21:52 |
| mordred | chmouel: why not? | 21:52 |
| jog0 | chmouel: :( | 21:52 |
| mordred | chmouel: it's not needed | 21:52 |
| chmouel | some peps use it for other stuff than rax or hp | 21:52 |
| mordred | oh - the plugin system? | 21:52 |
| mordred | they do? | 21:52 |
| chmouel | yep | 21:52 |
| mordred | gah | 21:52 |
| chmouel | i have a review somewhere which was explained | 21:53 |
| hub_cap | if you file a bug for that RAX_AUTH_JUNK, plz file a bug in trove/cinder too jog0 | 21:53 |
| jog0 | chmouel: what about just NOVA_RAX_AUTH | 21:53 |
| jog0 | hub_cap: :) | 21:53 |
| * jog0 files a bug | 21:53 | |
| chmouel | the env variable compa? | 21:53 |
| mordred | chmouel: we were discussing making novaclient consume keystoneclient | 21:53 |
| mordred | chmouel: instead of doing its own thing | 21:53 |
| jog0 | chmouel: http://git.openstack.org/cgit/openstack/python-novaclient/tree/novaclient/client.py#n322 | 21:53 |
| chmouel | mordred: oh yeah but first we wanted to bring all the functionalities | 21:54 |
| chmouel | that novaclient had | 21:54 |
| chmouel | which was the auth system | 21:54 |
| hub_cap | https://github.com/openstack/python-cinderclient/blob/873bed99a8019f87bbce3edf883153e7f67e875e/cinderclient/client.py#L299 | 21:54 |
| chmouel | i have a review somewhere that moves it to keystoneclient | 21:54 |
| mordred | ah | 21:54 |
| chmouel | but i think jamie lennox was doing ina different way | 21:54 |
| chmouel | there is a review for that | 21:54 |
| mordred | of course | 21:54 |
| jog0 | hub_cap: gahhh | 21:54 |
| chmouel | i'm on a plane atm so i can't check | 21:54 |
| mordred | chmouel: :) | 21:54 |
| jog0 | hub_cap: so you either need the plugin auth supprt or we nuke plugin auth all together | 21:55 |
| chmouel | but will come back to you tomo with the stats of all of this | 21:55 |
| clarkb | jeblair: mordred: you both +2'd 51703 at about the same time, should I +A it? | 21:55 |
| mordred | clarkb: yes | 21:55 |
| * fungi is going to grab some dinner and then brb | 21:55 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Fix another wrong use of server_id https://review.openstack.org/51465 | 21:55 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Add gear to requirements https://review.openstack.org/51644 | 21:55 |
| hub_cap | jog0: yes... its everywhere (god why isint this code in some common place) | 21:55 |
| jog0 | hub_cap: yeah ... sigh | 21:55 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Make ip_id arg match calling and consuming https://review.openstack.org/51687 | 21:55 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Add the ability to pass pool for AddPublicIP https://review.openstack.org/49833 | 21:55 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Don't use max_overflow with sqlite https://review.openstack.org/49639 | 21:56 |
| chmouel | mordred, jog0: https://review.openstack.org/#/c/36427/ | 21:56 |
| chmouel | so yeah there isa ctually three people trying to move the auth system mechanism to keystoneclient :) | 21:56 |
| jog0 | chmouel: nice I still don't think clients should support plugin auth | 21:56 |
| jog0 | but that is me ignoring reality | 21:57 |
| chmouel | jog0: i think there is some use cases for security that may be needed | 21:57 |
| jog0 | chmouel: then we fix keystone | 21:57 |
| mordred | ++ | 21:57 |
| openstackgerrit | A change was merged to openstack-infra/config: Backup etherpad.o.o with bup. https://review.openstack.org/51703 | 21:57 |
| jog0 | mordred: want to start a fun ML thread? | 21:57 |
| chmouel | jog0: i had one started some time ago about it | 21:57 |
| mordred | the auth system plugin is helping people run non-interoperable clouds | 21:57 |
| chmouel | let me check | 21:57 |
| * mordred helped write it - so hopefully it's not terrible if I trash it | 21:58 | |
| hub_cap | is present life mordred shaking his fist at past life mordred | 21:58 |
| clarkb | hub_cap: yes, we should get popcorn | 21:58 |
| jog0 | https://bugs.launchpad.net/python-novaclient/+bug/966329 | 21:58 |
| uvirtbot | Launchpad bug 966329 in python-novaclient "RAX-specific auth in novaclient" [Low,Fix released] | 21:58 |
| jog0 | mordred: haha | 21:59 |
| mordred | :) | 21:59 |
| jog0 | mordred: looks like you didn't actually dothat one | 21:59 |
| chmouel | jog0, mordred: http://www.mail-archive.com/openstack-dev@lists.openstack.org/msg00318.html | 21:59 |
| chmouel | the story was brian removed it | 22:00 |
| chmouel | infra needed it at that time | 22:00 |
| chmouel | and rax as well | 22:00 |
| chmouel | so we kind of made it in a plugged way | 22:00 |
| chmouel | but sill not nice | 22:00 |
| mordred | since then, we've stoped needing it | 22:01 |
| jog0 | "IMHO, as long as keystone allows for external authentication (as it does), | 22:01 |
| jog0 | the auth plugin system on the client side should exist." | 22:01 |
| jog0 | http://www.mail-archive.com/openstack-dev@lists.openstack.org/msg00319.html | 22:01 |
| mordred | clarkb, jeblair: if you're bored, https://review.openstack.org/#/c/51403/ and its dependency chain are fun | 22:01 |
| *** sarob has joined #openstack-infra | 22:01 | |
| mordred | I quasi-agree with that | 22:02 |
| mordred | although I'm pretty sure that the code to implement it sohuld go in keystone client | 22:02 |
| jog0 | mordred: for sure | 22:02 |
| chmouel | mordred: yep | 22:02 |
| mordred | I also think that the auth extension system is broken and misdesigned | 22:02 |
| chmouel | yeah totally | 22:02 |
| mordred | because it does not allow for discoverability of what auth plugin one needs to use to connect | 22:02 |
| mordred | you have to know before you connect what auth plugin to use | 22:03 |
| mordred | instead, like the other registries, you should be able to ask your cloud "hey man, what plugin do I need to talk to you" | 22:03 |
| clarkb | mordred: re 51403. Part of the reason that code exists is that tox's depend on virtualenv doesn't give us new virtualenvs | 22:03 |
| mordred | and it should say "dude, you need x509 auth" | 22:03 |
| mordred | clarkb: no? | 22:03 |
| *** jpeeler has quit IRC | 22:03 | |
| clarkb | mordred: and you want to keep virtualenv up to date so that it syncs up with setuptools | 22:03 |
| mordred | clarkb: I believe that has changed | 22:03 |
| clarkb | mordred: let me look at tox | 22:03 |
| mordred | clarkb: current tox should pull in current virtualenv - because it depends on pip 1.4 being present | 22:04 |
| mordred | in the virtualenv | 22:04 |
| chmouel | I think i have a version of cinderclient that moves it to use keystoneclient | 22:04 |
| mordred | tox actually uses pip 1.4-only command lines | 22:04 |
| mordred | chmouel: neat | 22:05 |
| chmouel | not sure why i didn't go on with that and submitted for review https://github.com/chmouel/python-cinderclient/tree/keystoneclient | 22:05 |
| jog0 | mordred chmouel: this is why this belongs in keystoneclient http://docs.openstack.org/developer/keystone/external-auth.html | 22:05 |
| clarkb | mordred: install_requires = ['virtualenv>=1.9.1', 'py>=1.4.15', ] so we will get at least 1.9.1 | 22:05 |
| mordred | clarkb: yup. which is what we need | 22:06 |
| clarkb | mordred: but nothing newer once installed. With that puppet resource we keep things in sync and up to date | 22:06 |
| clarkb | without it we may drift | 22:06 |
| *** jhesketh has joined #openstack-infra | 22:06 | |
| *** jhesketh__ has joined #openstack-infra | 22:06 | |
| mordred | clarkb: I believe we are now at a point where that is fine | 22:06 |
| mordred | clarkb: because 1.9 is post distribute/setuptools re-merge | 22:07 |
| clarkb | mordred: until the next round of crazy setuptools ness | 22:07 |
| mordred | and post pip1.4 | 22:07 |
| mordred | if setuptools goes crazy again, I will kill them | 22:07 |
| *** dizquierdo has left #openstack-infra | 22:07 | |
| clarkb | mordred: I think I am more worried about drift | 22:07 |
| mordred | I'm less, actually | 22:07 |
| clarkb | mordred: old slaves will have 1.9.1, new slaves will have 1.9.5 and others 2.0 an dso on | 22:07 |
| mordred | I'm more worried, tbh, that we drift virtualenv up without tox | 22:07 |
| mordred | lemme think about that for a sec | 22:08 |
| mordred | jog0: ++ | 22:09 |
| jog0 | mordred hub_cap: https://bugs.launchpad.net/python-troveclient/+bug/966329 | 22:09 |
| uvirtbot | Launchpad bug 966329 in python-novaclient "RAX-specific auth in novaclient" [Low,Fix released] | 22:09 |
| jog0 | I brought that one back | 22:09 |
| mordred | clarkb: ok. I'm willing to stew on that one longer | 22:09 |
| mordred | clarkb: how about the two in the dep chain before it | 22:09 |
| hub_cap | <3 jog0 | 22:09 |
| clarkb | mordred: fwiw I don't feel too strongly about it. If it becomes a problem we upgrade virtualenv again | 22:10 |
| clarkb | or pin the version | 22:10 |
| mordred | clarkb: yeah | 22:10 |
| clarkb | just trying to make sure we understand why it did exist in that state though | 22:10 |
| mordred | I was mostly trying to clean up extra invocations of pip in our puppet | 22:10 |
| mordred | so that we could see the places we were actually needing to directly have puppet manage it | 22:10 |
| mordred | clarkb: btw - I think that our dance around python2 and python3 pip may be obsolete | 22:11 |
| mordred | clarkb: but I need to go do some testing | 22:11 |
| clarkb | nice | 22:11 |
| mordred | clarkb: now that we can do the tox 1.6 use_develop - the pyhton _inside_ of the virtualenv should always be the python used for all things | 22:12 |
| mordred | unless there was another issue I'm forgetting | 22:12 |
| jog0 | mordred chmouel: https://review.openstack.org/51715 | 22:12 |
| *** syerrapragada has joined #openstack-infra | 22:12 | |
| clarkb | mordred: there is the slight issues of trying to isntall stuff globally for both | 22:12 |
| clarkb | mordred: but that isn't a unittest slave problem so much as a server problem | 22:13 |
| mordred | clarkb: right - but why do we need to do that? | 22:13 |
| clarkb | mordred: initially I wanted it so that infra could be leading hte way in python3 support | 22:13 |
| * mordred was trying to remember what it was we wanted to accomlish when we wanted to do that | 22:13 | |
| clarkb | mordred: the logstash stuff was all python3 at one point | 22:13 |
| mordred | yah. I remember that | 22:13 |
| clarkb | but daeling with that is a bit derpy for other reasons | 22:13 |
| mordred | yup | 22:14 |
| mordred | such as being told we should never install things globally with pip :) | 22:14 |
| clarkb | mordred: I have been thinking about that, and I think it is wrong | 22:14 |
| mordred | yeah? | 22:14 |
| clarkb | if your package management system can't coexist with other package managers then it is broken | 22:15 |
| clarkb | cabal and maven seem to deal with it alrgith (with their pseudo virtualenv things) | 22:15 |
| mordred | so, the current state is that it is broken then | 22:15 |
| clarkb | I don't think it is a global vs venv problem. I see it more of a `pip install foo` should just work | 22:15 |
| mordred | I agree. but it doesn't | 22:15 |
| mordred | python and the distros have an intractible disagreement about where things should install | 22:15 |
| mordred | and who should own them | 22:15 |
| mordred | and how it should work | 22:16 |
| clarkb | the globalness is an implementation detail | 22:16 |
| mordred | with each side thinking the other is dead wrong | 22:16 |
| clarkb | mordred: right | 22:16 |
| mordred | clarkb: so I agree with you in principle | 22:16 |
| *** datsun180b has quit IRC | 22:16 | |
| mordred | but in practice, it turns out that global installation of software via pip mixed with global installation of software via package managers is unsupported and undefined | 22:16 |
| mordred | that this is the case is sad | 22:17 |
| *** CaptTofu has quit IRC | 22:17 | |
| mordred | and makes me want to ditch python and move to java or c++ | 22:17 |
| clarkb | mordred: right so pip should instead do a thing where it installs to /usr/local/pip (or /opt or wherever outside of apt and yums pervue) | 22:17 |
| *** CaptTofu has joined #openstack-infra | 22:17 | |
| clarkb | then deal with paths | 22:17 |
| mordred | well... funny story | 22:17 |
| clarkb | pip sort of does this on debian | 22:17 |
| mordred | it MOSTLY works on debian | 22:17 |
| mordred | but python is mad at debian | 22:17 |
| clarkb | but still thinks it should nuke globally installed things | 22:18 |
| mordred | for doing this | 22:18 |
| *** gyee has quit IRC | 22:18 | |
| clarkb | its "global" but doesn't interfere | 22:18 |
| clarkb | this is good for dedup of data on multiuser systems and sanity | 22:18 |
| mordred | well... you know | 22:19 |
| *** psedlak has quit IRC | 22:19 | |
| clarkb | I think we sometimes forget that these are multiuser systems | 22:19 |
| mordred | you could always make a global virtualenv, say /opt/python, put 'source /opt/python/bin/activate' into /etc/profile | 22:19 |
| mordred | and go from there | 22:19 |
| mordred | clarkb: I think they are becomming less and less multi-user with the advent of clouds and containers | 22:20 |
| mordred | separation is happening via containers or entire special-purpose VMs | 22:20 |
| mordred | of course, what was really awesome was the add and attach commands back in jeblair's and my AFS past :) | 22:21 |
| mordred | now _there_ was a user-space way of dealing with this problem | 22:21 |
| clarkb | I think containers and VMs address a particular subset of needs. There is still a lot of reasons for mutliuser friendliness | 22:21 |
| clarkb | hell I run multiple desktops on my desktop at home concurrently | 22:22 |
| mordred | well, but you're smart | 22:22 |
| *** arosen has quit IRC | 22:23 | |
| *** vipul is now known as vipul-away | 22:23 | |
| *** vipul-away is now known as vipul | 22:24 | |
| clarkb | that said, I don't think thin clients are going to happen. They tried and tried and we don't seem to be headed that direction (I think the cheapness of actual usable hardware killed thin clients) | 22:25 |
| *** psedlak has joined #openstack-infra | 22:26 | |
| clarkb | mordred: https://review.openstack.org/#/c/51710/1 is GLOBAL_REQUIREMENTS a string in setup.py now? | 22:28 |
| *** syerrapragada has left #openstack-infra | 22:29 | |
| pleia2 | can this review get nudged along? https://review.openstack.org/#/c/46015/ | 22:29 |
| *** prad__ has quit IRC | 22:29 | |
| pleia2 | (very reviewed, just no +A) | 22:30 |
| pleia2 | and useful ;) | 22:30 |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: - remove syntax issues with pip3.rb - remove syntax issues with python2/3.pp so that class names are pip:python2/3 - change python pp file requirements on bootstrap so we use a define - added require for install_pip2 to occur after install https://review.openstack.org/51716 | 22:31 |
| *** Protux has quit IRC | 22:31 | |
| *** gyee has joined #openstack-infra | 22:31 | |
| *** dcramer_ has quit IRC | 22:31 | |
| *** Protux has joined #openstack-infra | 22:32 | |
| *** marun has quit IRC | 22:33 | |
| mordred | wenlock: if I was you - I'd just squash the two commits and push up over top of mine | 22:34 |
| *** Protux has quit IRC | 22:34 | |
| jog0 | http://logs.openstack.org/66/50966/1/gate/gate-tempest-devstack-vm-postgres-full/672f3c0/testr_results.html.gz? | 22:34 |
| mordred | wenlock: (we don't consider that sort of thing rude in these parts) | 22:34 |
| jog0 | WAT | 22:34 |
| jog0 | where is the HTML? | 22:34 |
| wenlock | ahh , i wasn't sure | 22:34 |
| jog0 | clarkb: ^ | 22:34 |
| wenlock | can i actually do that after i've commited? | 22:34 |
| jog0 | http://logs.openstack.org/66/50966/1/gate/gate-tempest-devstack-vm-postgres-full/672f3c0/testr_results.html? works | 22:34 |
| mordred | wenlock: also - two things - a) congrats on your first patch! and b) next time you do produce a new commit, try to keep the first line to under 60 chars | 22:35 |
| wenlock | ok, np | 22:35 |
| *** mrodden has quit IRC | 22:35 | |
| reed | damn, jog0 that page is killing my firefox | 22:35 |
| mordred | wenlock: totally! for now, do 'git rebase -i origin/master' and then select s instead of pick on the second one | 22:35 |
| mordred | then, when you squash them, make sure that the Change-Id from the first commit is the only change-id line in the commit message and that it's the last line | 22:36 |
| mordred | then do a git review again | 22:36 |
| * jog0 is glad he uses chrome | 22:36 | |
| reed | jog0, don't be glad to use chrome, google is evil | 22:36 |
| *** CaptTofu has quit IRC | 22:36 | |
| clarkb | jog0: sdague broke it, fix is in review | 22:36 |
| *** CaptTofu has joined #openstack-infra | 22:36 | |
| clarkb | jog0: mordred jog0 fungi https://review.openstack.org/#/c/51707/ | 22:37 |
| clarkb | pleia2: looking | 22:37 |
| clarkb | pleia2: I think we were hoping fungi would review it. I will go ahead and approve it since we have test snow | 22:38 |
| *** thedodd has quit IRC | 22:38 | |
| mordred | clarkb: I approved | 22:38 |
| clarkb | mordred: danke | 22:38 |
| jog0 | thanks | 22:39 |
| openstackgerrit | A change was merged to openstack-infra/config: only run the console.html logs through os-loganalyzer https://review.openstack.org/51707 | 22:39 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Consider existing nodes when allocating to a target https://review.openstack.org/51719 | 22:43 |
| *** thomasm has joined #openstack-infra | 22:43 | |
| *** dmakogon_ has left #openstack-infra | 22:44 | |
| jeblair | clarkb, fungi, mordred: ^ that should even out the load on jenkins01 and 02 | 22:45 |
| *** rnirmal has quit IRC | 22:45 | |
| *** alexpilotti has joined #openstack-infra | 22:45 | |
| *** SergeyLukjanov has quit IRC | 22:46 | |
| sdague | clarkb: that change go through yet? | 22:46 |
| clarkb | sdague: doesn't look like it | 22:48 |
| clarkb | I think it will get aplpied in ~4 minutes | 22:48 |
| sdague | ok | 22:48 |
| sdague | so, new crazy idea... | 22:48 |
| sdague | how would you propose I should set up the test to run this - https://review.openstack.org/#/c/51711/ ? | 22:49 |
| jeblair | clarkb, fungi: i just put that nodepool change into prod; current levels are jenkins01=20, jenkins02=71. we'll see how it levels out. | 22:49 |
| sdague | this is basic syntax checking for bash for devstack | 22:49 |
| clarkb | sdague: I would do something similar to the existing pep8 jobs but make it not so pythony | 22:50 |
| clarkb | jeblair: I am working my way through that change | 22:50 |
| sdague | clarkb: ok, so just run them on the unit test nodes? | 22:50 |
| clarkb | sdague: ya | 22:51 |
| *** SergeyLukjanov has joined #openstack-infra | 22:53 | |
| *** pentameter has quit IRC | 22:53 | |
| sdague | modules/jenkins/files/slave_scripts/run-pep8.sh? | 22:53 |
| jeblair | clarkb, fungi: http://graphite.openstack.org/render/?width=586&height=308&_salt=1381791295.272&target=sumSeries%28stats.gauges.nodepool.target.jenkins01.devstack-precise.*.*%29&target=sumSeries%28stats.gauges.nodepool.target.jenkins02.devstack-precise.*.*%29 | 22:54 |
| *** branen has quit IRC | 22:54 | |
| jeblair | clarkb, fungi: and we're already at 48 and 66 | 22:55 |
| clarkb | sdague: yeah, in this case you don't need the special pythonstuff you can probably just embed it in the JJB configs without it looking nasty | 22:55 |
| clarkb | jeblair: the change looked good. I did have one comment but it doesn't need to be addressed | 22:55 |
| *** sarob_ has joined #openstack-infra | 22:56 | |
| *** mrodden has joined #openstack-infra | 22:58 | |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 22:59 |
| *** sarob has quit IRC | 22:59 | |
| *** senk has joined #openstack-infra | 23:00 | |
| *** david-lyle_ has quit IRC | 23:00 | |
| *** branen has joined #openstack-infra | 23:01 | |
| *** sarob_ has quit IRC | 23:01 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add bash8 to devstack tests https://review.openstack.org/51726 | 23:02 |
| *** sarob has joined #openstack-infra | 23:02 | |
| sdague | clarkb: so.... that's part of this, but I'm a little lost on where to add the layout definitions correctly | 23:02 |
| *** zojoncj has quit IRC | 23:02 | |
| clarkb | sdague: misc.yaml for JJB and under openstack/devstack in zuul/layout.yaml is where I would put it | 23:03 |
| *** sarob has quit IRC | 23:04 | |
| *** sarob has joined #openstack-infra | 23:05 | |
| *** thomasm has quit IRC | 23:06 | |
| clarkb | jog0: sdague: testr html files are good now | 23:07 |
| sdague | cool | 23:07 |
| jog0 | clarkb sdague: thanks | 23:09 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add bash8 to devstack tests https://review.openstack.org/51726 | 23:09 |
| *** oubiwann has quit IRC | 23:09 | |
| sdague | clarkb: ok, still not sure I got everything, but maybe it's close | 23:09 |
| *** pcm_ has quit IRC | 23:09 | |
| *** amotoki has quit IRC | 23:10 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add query for bug 1239856 https://review.openstack.org/51728 | 23:11 |
| uvirtbot | Launchpad bug 1239856 in glance ""TimeoutException: Request timed out" on tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestXML " [Undecided,New] https://launchpad.net/bugs/1239856 | 23:11 |
| *** nati_uen_ has joined #openstack-infra | 23:18 | |
| fungi | eek scrollback | 23:20 |
| SpamapS | http://logs.openstack.org/85/51685/3/check/check-tempest-devstack-vm-full/80644d4/console.html .. there's a fail in there because six is not >= 1.4.1 but the change I'm testing adds '>=1.4.1' to python-keystoneclient's requirements.txt .. is there just lag in the devstack gate nodes getting updated to the latest global reqs? | 23:20 |
| jeblair | SpamapS: can you link to the line you're talking about? | 23:21 |
| *** nati_ueno has quit IRC | 23:22 | |
| *** jecarey has quit IRC | 23:22 | |
| clarkb | SpamapS: jeblair: http://logs.openstack.org/85/51685/3/check/check-tempest-devstack-vm-full/80644d4/logs/pip-freeze.txt.gz six==1.4.1 was installed | 23:23 |
| *** nati_uen_ has quit IRC | 23:23 | |
| mordred | SpamapS: | 23:23 |
| mordred | 2013-10-14 22:40:12.351 | 2013-10-14 22:40:12 Downloading/unpacking six (from oslo.config) | 23:23 |
| mordred | 2013-10-14 22:40:12.357 | 2013-10-14 22:40:12 Downloading six-1.4.1.tar.gz | 23:23 |
| mordred | 2013-10-14 22:40:12.360 | 2013-10-14 22:40:12 Storing download in cache at /var/cache/pip/http%3A%2F%2Fpypi.openstack.org%2Fopenstack%2Fsix%2Fsix-1.4.1.tar.gz | 23:23 |
| *** eharney has quit IRC | 23:23 | |
| jeblair | sdague: the console logs seem to be escaping html -- so there's stuff like "<pre>" and "<a href='https://review.openstack.org/51685'>" in there | 23:23 |
| clarkb | jeblair: sdague: yes jenkins escapes the html for us when we write it out as html | 23:24 |
| mordred | jeblair: there was a processing bug that we've APRV'd a change for - but I don't think it's rolled out yet? clarkb ? | 23:24 |
| clarkb | it went out, it was for the non console.html files | 23:25 |
| mordred | ah. gotcha | 23:25 |
| SpamapS | https://review.openstack.org/#/c/51685/ | 23:26 |
| SpamapS | jeblair: ^^ re the change that is failing | 23:26 |
| jeblair | SpamapS: i read that in the logs, thanks. :) | 23:26 |
| jeblair | clarkb: jenkins doesn't escape the html | 23:26 |
| jeblair | clarkb: looking at the file on disk, it looks like perfectly normal html | 23:27 |
| clarkb | jeblair: I thought it did | 23:27 |
| clarkb | because it is embedding it in <pre> tags. jeblair is there a particular console.html that is having problems/ | 23:27 |
| jeblair | clarkb: the one SpamapS linked | 23:27 |
| SpamapS | jeblair: oh sorry I read your request wrong | 23:27 |
| clarkb | oh I see | 23:27 |
| jeblair | clarkb: http://paste.openstack.org/show/48454/ | 23:28 |
| clarkb | jeblair: yeah that is a bug I think | 23:28 |
| SpamapS | 2013-10-14 22:48:10.193 | File "/opt/stack/new/python-keystoneclient/keystoneclient/httpclient.py", line 26, in <module> | 23:28 |
| SpamapS | 2013-10-14 22:48:10.193 | from six.moves.urllib import parse as urlparse | 23:28 |
| SpamapS | 2013-10-14 22:48:10.194 | ImportError: No module named urllib | 23:28 |
| jeblair | SpamapS: (and everyone else) you can link to a line using the anchor on the timestamp: http://logs.openstack.org/85/51685/3/check/check-tempest-devstack-vm-full/80644d4/console.html#_2013-10-14_22_48_10_189 | 23:28 |
| SpamapS | Oh, neato | 23:29 |
| clarkb | jeblair: sdague: we have nested <pre>s I think that is the problem | 23:29 |
| SpamapS | did not even notice they were clickable | 23:29 |
| *** nati_ueno has joined #openstack-infra | 23:29 | |
| jeblair | clarkb: not to mention <module> | 23:29 |
| *** senk has quit IRC | 23:30 | |
| clarkb | SpamapS: the import is wrong | 23:31 |
| *** senk has joined #openstack-infra | 23:31 | |
| jeblair | clarkb: yeah, the log analyzer shouldn't <pre> an html file. | 23:31 |
| clarkb | jeblair: yup | 23:32 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Consider existing nodes when allocating to a target https://review.openstack.org/51719 | 23:32 |
| clarkb | SpamapS: hrm in a venv I can import that line | 23:33 |
| SpamapS | clarkb: yeah I can't get that import to fail as long as six == 1.4.1 | 23:34 |
| clarkb | I wonder, it is only failing in the context of the tempest tests and tempest runs with tox | 23:35 |
| clarkb | perhaps site pacakges isn't doing what we expect there | 23:35 |
| SpamapS | yes it is failing in that context | 23:35 |
| clarkb | ahahahahahahha | 23:37 |
| openstackgerrit | A change was merged to openstack-infra/git-review: Prefer .gitconfig username https://review.openstack.org/46015 | 23:37 |
| *** atiwari has quit IRC | 23:37 | |
| clarkb | tempest six<1.4.0 | 23:37 |
| SpamapS | _doh_ | 23:37 |
| SpamapS | wtf? | 23:37 |
| clarkb | sitepackage=True must be smart enough to find versions it needs if that don't exist system wide (it didn't do this before) | 23:37 |
| clarkb | SpamapS: I think we can blame ceilometer for that. They pinned six, then we made them unpin because it caused all sorts of trouble | 23:38 |
| clarkb | SpamapS: so I think fixing that in tempest will fix it for you | 23:38 |
| clarkb | upper bounds on six are bad | 23:38 |
| * SpamapS was just trying to throw 30 minutes on the pile of py3 porting effort | 23:39 | |
| SpamapS | getting dizzy having tumbled down this rabbit hole for a while now | 23:39 |
| SpamapS | clarkb: thanks for the analysis | 23:39 |
| clarkb | SpamapS: np, and yeah py3k is a hole you jump in and you quickly find that the bottom isn't where you expected it to be | 23:40 |
| SpamapS | usually I smack my head on the eventlet rock early on | 23:40 |
| sdague | jeblair: yeh, I'll build an exclusion for that tomorrow | 23:41 |
| jeblair | sdague: cool, thanks | 23:41 |
| fungi | i fell down a py3k hole 'round ought-six i reckon. haven't managed to climb back up yet | 23:42 |
| sdague | back to cooking dinner.... | 23:42 |
| clarkb | I am freezing at my desk (going to walk home and find warmth) | 23:42 |
| fungi | aught-six i means to say | 23:42 |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 23:43 |
| sdague | I might get to it later tonight, it's a simple enough change. Will be good to put unit tests in place for it as well | 23:43 |
| SpamapS | heh, two of my sons can at least say "well, I 'as born in aught-nine" or "I 'member you 'us born in aught-three" | 23:44 |
| * SpamapS channeling his inner okie | 23:44 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 23:53 | |
| *** _SergeyLukjanov has quit IRC | 23:53 | |
| openstackgerrit | Edward Raigosa proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 23:53 |
| *** SergeyLukjanov has joined #openstack-infra | 23:54 | |
| *** SergeyLukjanov has quit IRC | 23:54 | |
| *** CaptTofu has quit IRC | 23:55 | |
| *** CaptTofu has joined #openstack-infra | 23:56 | |
| *** yamahata has quit IRC | 23:56 | |
| openstackgerrit | Dan Nguyen proposed a change to openstack/requirements: Add flufl.password to requirements https://review.openstack.org/51068 | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!