| *** gokrokve_ has quit IRC | 00:01 | |
| *** msabramo has quit IRC | 00:03 | |
| *** todd_dsm has quit IRC | 00:03 | |
| *** xchu has joined #openstack-infra | 00:04 | |
| *** unicell has joined #openstack-infra | 00:04 | |
| *** xchu has quit IRC | 00:04 | |
| *** kirscht has joined #openstack-infra | 00:04 | |
| *** kgiusti has left #openstack-infra | 00:04 | |
| *** kgiusti has quit IRC | 00:04 | |
| *** zaphod42 has quit IRC | 00:05 | |
| *** zaphod42 has joined #openstack-infra | 00:05 | |
| *** zigo has quit IRC | 00:07 | |
| *** zaphod42 has quit IRC | 00:10 | |
| mordred | anteaya: you _don't_ need auth to do anythign with it, actually | 00:11 |
|---|---|---|
| anteaya | oh | 00:11 |
| mordred | anteaya: you can totally access resources marked as available for system:anyuser | 00:11 |
| anteaya | ah | 00:11 |
| anteaya | the documentation talked about a puppet password | 00:11 |
| anteaya | I don't yet understand how that fits in with the flow | 00:12 |
| anteaya | but I'll work on figuring out the pieces | 00:12 |
| mordred | in fact, right now, I'm pretty sure if you did "sudo apt-get install openafs-client ; ls /afs/openstack.org/user" you'd see three files ... but there would be no use or purpose in you doing that | 00:12 |
| *** david-lyle has quit IRC | 00:12 | |
| mordred | anteaya: so, practically, yeah, there isn't much point in people installing the client broadly | 00:12 |
| *** alexandra has joined #openstack-infra | 00:12 | |
| mordred | so I don't expect much of it | 00:13 |
| *** todd_dsm has joined #openstack-infra | 00:13 | |
| *** annegent_ has quit IRC | 00:13 | |
| *** emagana_ has quit IRC | 00:13 | |
| *** zigo has joined #openstack-infra | 00:13 | |
| mordred | anteaya: I mention that mainly for the purposes of sharing information :) | 00:14 |
| *** emagana has joined #openstack-infra | 00:14 | |
| *** asettle has quit IRC | 00:15 | |
| anteaya | thank you | 00:15 |
| anteaya | i appreciate the shared information | 00:15 |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack-infra/jenkins-job-builder: Support additional events for HipChat plugin https://review.openstack.org/129760 | 00:15 |
| *** david-lyle has joined #openstack-infra | 00:15 | |
| *** emagana has quit IRC | 00:18 | |
| *** dimsum_ has joined #openstack-infra | 00:21 | |
| *** Ryan_Lane has quit IRC | 00:22 | |
| *** weshay has joined #openstack-infra | 00:22 | |
| openstackgerrit | Angus Lees proposed a change to openstack-infra/project-config: Promote gate-neutron-pylint check to voting https://review.openstack.org/129762 | 00:25 |
| *** Ryan_Lane has joined #openstack-infra | 00:25 | |
| *** Ryan_Lane has quit IRC | 00:27 | |
| *** tonytan4ever has quit IRC | 00:27 | |
| *** annegent_ has joined #openstack-infra | 00:29 | |
| *** annegent_ has quit IRC | 00:30 | |
| *** bdpayne has quit IRC | 00:31 | |
| *** nelsnels_ has joined #openstack-infra | 00:36 | |
| *** andreaf has quit IRC | 00:37 | |
| *** nelsnelson has quit IRC | 00:38 | |
| *** andreaf has joined #openstack-infra | 00:38 | |
| *** Ryan_Lane has joined #openstack-infra | 00:40 | |
| *** gokrokve has joined #openstack-infra | 00:42 | |
| *** markvan_ has quit IRC | 00:43 | |
| *** markvan has joined #openstack-infra | 00:44 | |
| asselin | nibalizer, jesusaurus what do you think of this? https://github.com/rasselin/puppet-jenkins | 00:44 |
| *** baoli has joined #openstack-infra | 00:45 | |
| *** jerryz has joined #openstack-infra | 00:48 | |
| *** dangers is now known as dangers_away | 00:49 | |
| openstackgerrit | Christopher Yeoh proposed a change to openstack-infra/project-config: Add API Working Group project https://review.openstack.org/128466 | 00:49 |
| *** mmedvede has quit IRC | 00:51 | |
| *** aysyd has quit IRC | 00:51 | |
| *** hdd has quit IRC | 00:57 | |
| *** arnaud has quit IRC | 00:57 | |
| *** arnaud__ has quit IRC | 00:57 | |
| *** alexandra is now known as asettle | 00:57 | |
| *** mmaglana has quit IRC | 00:57 | |
| *** patrickeast has quit IRC | 01:00 | |
| *** sdake_ has joined #openstack-infra | 01:01 | |
| *** gokrokve has quit IRC | 01:02 | |
| *** r1chardj0n3s_afk is now known as r1chardj0n3s | 01:02 | |
| *** gokrokve has joined #openstack-infra | 01:03 | |
| cyeoh | anyone know if something weird is going on with jenkins? I had a bunch of changesets fail jenkins with merge conflict errors this morning, but when I got to rebase them there is no actual merge conflict. Its been around 5 or 6 patches across different projects | 01:03 |
| *** amotoki has joined #openstack-infra | 01:04 | |
| *** yamahata has joined #openstack-infra | 01:04 | |
| anteaya | cyeoh: hi chris | 01:06 |
| anteaya | yes | 01:06 |
| anteaya | we were doing work | 01:06 |
| cyeoh | anteaya: hi! | 01:06 |
| anteaya | and the mass rebase happened as a result | 01:06 |
| anteaya | just a recheck should do it | 01:06 |
| anteaya | we tried to get all affected patches but might have missed some | 01:07 |
| anteaya | sorry not a mass rebase | 01:07 |
| *** msabramo has joined #openstack-infra | 01:07 | |
| cyeoh | anteaya: ok, thanks! | 01:07 |
| anteaya | a mass -1 with a merge conflict message | 01:07 |
| anteaya | with no actual merge conflict | 01:07 |
| *** gokrokve has quit IRC | 01:07 | |
| anteaya | cyeoh: how are you doing? | 01:07 |
| anteaya | been a long time since we chatted | 01:07 |
| anteaya | any idea when you can travel again? | 01:07 |
| cyeoh | anteaya: am doing pretty well! the new chemo is much easier to handle so its generally only a few days after that I'm affected really badly. And I only have it every 3 weeks. | 01:08 |
| anteaya | oh thank goodness | 01:08 |
| cyeoh | so I'm going to try to get to the nova mid-cycle - was a bit too late to organise for Paris | 01:09 |
| anteaya | I'm glad you have found something taht is gentler on your system | 01:09 |
| anteaya | oh yay!! | 01:09 |
| anteaya | congratulations!! | 01:09 |
| cyeoh | and given the distance will be bringing along Kelly just in case. | 01:09 |
| anteaya | that would inspire me to try to go | 01:09 |
| anteaya | good idea | 01:09 |
| *** yaguang has joined #openstack-infra | 01:10 | |
| cyeoh | it'll be good to be at a face to face meeting again. Has been too long! | 01:10 |
| anteaya | would you bring your daughter too? | 01:10 |
| anteaya | or do you have a grandmother looking forward to taking her? | 01:10 |
| anteaya | cyeoh: no kidding | 01:10 |
| anteaya | I can't speak for everyone but *I've* been missing you | 01:11 |
| cyeoh | Alyssa would be coming too :-) Would probably have to do a stop over at disneyland ;-) | 01:11 |
| anteaya | yay | 01:11 |
| anteaya | good idea | 01:11 |
| anteaya | that sound awesome | 01:11 |
| cyeoh | anteaya: thanks - I've really missed not being able to attend the summits and mid-cycle | 01:11 |
| anteaya | yeah | 01:11 |
| anteaya | man I feel ya | 01:11 |
| *** sdake_ has quit IRC | 01:11 | |
| anteaya | so glad you have some good things happening now | 01:11 |
| anteaya | and the nova mid-cycle to look forward to | 01:12 |
| anteaya | :D | 01:12 |
| cyeoh | yea its good :-) And Kira is having kittens! | 01:12 |
| anteaya | Kira? | 01:12 |
| * anteaya hopes Kira is a cat | 01:12 | |
| cyeoh | our Bengal kitten | 01:12 |
| anteaya | oh nice | 01:12 |
| anteaya | when are you expecting the sharp little pockets of fuzzy love? | 01:13 |
| cyeoh | the kittens belong to the original breeder (its the deal that was made when we got her), but if there is a snow bengal we'll probably keep one | 01:13 |
| *** yaguang has quit IRC | 01:13 | |
| anteaya | what is a snow bengal? | 01:13 |
| cyeoh | not sure, but probably 5-6 weeks away. | 01:13 |
| * anteaya hopes for pictures | 01:13 | |
| *** weshay has quit IRC | 01:13 | |
| *** yaguang has joined #openstack-infra | 01:13 | |
| anteaya | I hope she delivers safely | 01:13 |
| *** todd_dsm has quit IRC | 01:13 | |
| cyeoh | so this is what they can look like: http://www.google.com.au/imgres?imgurl=http%3A%2F%2Fwww.snowbengalkittensforsale.com%2Fimages%2FslpSide.jpg&imgrefurl=http%3A%2F%2Fwww.snowbengalkittensforsale.com%2F&h=300&w=400&tbnid=SbzJLk5iy2MEkM%3A&zoom=1&docid=yiVr2cuzIDhLdM&ei=TrNFVKyqGcHDmQWKxYG4AQ&tbm=isch&ved=0CCUQMygEMAQ&iact=rc&uact=3&dur=3108&page=1&start=0&ndsp=30 | 01:14 |
| cyeoh | Kira is a brown bengal - https://pbs.twimg.com/media/B0bNrahCIAAGqSz.jpg:large - that's her with her pregnant belly | 01:15 |
| anteaya | wow | 01:15 |
| anteaya | she looks like she is doing pretty well | 01:16 |
| *** mfer has joined #openstack-infra | 01:16 | |
| anteaya | she still has 6 weeks to go though eh? | 01:16 |
| cyeoh | yep. So she should get quite a bit bigger. Hasn't stopped her from hunting the other cats at night yet though | 01:17 |
| *** otter768 has joined #openstack-infra | 01:18 | |
| anteaya | great | 01:20 |
| anteaya | bet the other cats are really happy about that | 01:20 |
| StevenK | They might be able to hear her coming up on them, then ... | 01:21 |
| cyeoh | heh yea :-) And it does slow her down a bit. The other two cats are old grumpy things and just want to sleep though while she just wants to play all the time | 01:21 |
| *** Longgeek has joined #openstack-infra | 01:21 | |
| anteaya | the swish of the big belly | 01:21 |
| anteaya | those young cats | 01:22 |
| anteaya | so playful | 01:22 |
| cyeoh | yea and Bengals remain very playful for years. | 01:22 |
| *** dane_leblanc has joined #openstack-infra | 01:23 | |
| anteaya | cyeoh: yeah so I'm -1 on the api-wg repo mostly since it doesn't fit the structure of the creation of other openstack/ repos | 01:23 |
| anteaya | and the tc hasn't had a chance to chat about it | 01:23 |
| anteaya | since not everyone can create an openstack/ repo | 01:23 |
| anteaya | what do you envision going into the repo? | 01:23 |
| cyeoh | anteaya: yea that's fine - I agree we need the tc to approve it first | 01:23 |
| anteaya | docs mostly? | 01:23 |
| anteaya | cyeoh: thanks | 01:23 |
| anteaya | actually it appears there are more repos taht fit this model | 01:24 |
| anteaya | so this is just one more example to demonstrate the conversation needs to happen | 01:24 |
| *** radez_g0n3 is now known as radez | 01:24 | |
| cyeoh | anteaya: yes, only documentation - process for how the API WG works and and then the API guidelines themsleves | 01:24 |
| *** rushiagr_away is now known as rushiagr | 01:24 | |
| anteaya | cyeoh: cool | 01:24 |
| anteaya | so we will let the tc have a chew | 01:24 |
| anteaya | and then we will have a process *hopefully* | 01:24 |
| anteaya | so I can know how to address more patches of this nature | 01:25 |
| cyeoh | yep, I think Thierry said they would talk about it at the next TC meeting | 01:25 |
| anteaya | awesome | 01:25 |
| anteaya | so hopefully by your tomorrow we can have some sort of guidance | 01:25 |
| anteaya | and go from there | 01:25 |
| cyeoh | yep! I'm getting called to go to lunch so I'd better head off. Been great talking to you again :-) | 01:25 |
| anteaya | same here | 01:26 |
| anteaya | miss you | 01:26 |
| anteaya | glad to see you around | 01:26 |
| anteaya | enjoy lunch | 01:26 |
| anteaya | and say hi to kelly when you get a chance | 01:26 |
| anteaya | :D | 01:26 |
| *** rushiagr is now known as rushiagr_away | 01:26 | |
| cyeoh | will do - good to be around more again :-) | 01:26 |
| anteaya | :D | 01:26 |
| *** andreaf has quit IRC | 01:27 | |
| *** Longgeek has quit IRC | 01:27 | |
| *** andreaf has joined #openstack-infra | 01:27 | |
| *** todd_dsm has joined #openstack-infra | 01:28 | |
| *** dane_leblanc has quit IRC | 01:29 | |
| *** gokrokve has joined #openstack-infra | 01:29 | |
| *** dane_leblanc has joined #openstack-infra | 01:30 | |
| openstackgerrit | Ramy Asselin proposed a change to openstack-infra/infra-specs: Update puppet module git subtree instructions https://review.openstack.org/129768 | 01:30 |
| *** gokrokve has quit IRC | 01:31 | |
| *** gokrokve has joined #openstack-infra | 01:31 | |
| asselin | nibalizer, jesusaurus ^^ I updated the spec with the steps I took to do the git subtree here: https://github.com/rasselin/puppet-jenkins | 01:32 |
| anteaya | asselin: why the move to # instead of numbers? | 01:33 |
| asselin | b/c it's easier to add steps :) | 01:33 |
| asselin | they get generated with the actual 1,2,3 | 01:34 |
| anteaya | oh do they | 01:34 |
| * anteaya waits for the check docs to return | 01:34 | |
| anteaya | all this .rst magic that I don't know | 01:34 |
| asselin | yes, i tested it locally | 01:34 |
| anteaya | great | 01:34 |
| asselin | I'm learning it too | 01:34 |
| anteaya | you are doing a good job | 01:35 |
| anteaya | keep it up | 01:35 |
| asselin | going for dinner. see you tomorrow anteaya | 01:35 |
| anteaya | asselin: enjoy dinner, thanks for your hard work | 01:36 |
| jeblair | cyeoh: oh hi! :) sorry about the -1s (my fault); the rechecks for all the errors will still probably take another 18 hours from now (it's a queue, so 0-18 hours for any given change) | 01:36 |
| anteaya | see you tomorrow | 01:36 |
| jeblair | anteaya: that's some rough math based on the zuul job queue graph ^ | 01:37 |
| anteaya | jeblair: thanks | 01:40 |
| *** otter768 has quit IRC | 01:40 | |
| anteaya | wow | 01:40 |
| *** sdake_ has joined #openstack-infra | 01:40 | |
| anteaya | I hadn't looked before | 01:40 |
| anteaya | but the line just goes straight up | 01:41 |
| anteaya | with 607 still in check | 01:41 |
| anteaya | so yeah, having the 18 hour guesstimate is helpful for answering tomorrow's questions too | 01:41 |
| *** otter768 has joined #openstack-infra | 01:42 | |
| *** SumitNaiksatam has joined #openstack-infra | 01:42 | |
| jeblair | clarkb, fungi, mordred, anteaya: http://paste.openstack.org/show/122421/ | 01:45 |
| jeblair | that's an afs client install/config on a new precise vm | 01:45 |
| mordred | jeblair: ++ | 01:46 |
| clarkb | neat | 01:46 |
| jeblair | oh, and this is the coolest thing about afs: http://paste.openstack.org/show/122422/ | 01:46 |
| mordred | jeblair: we could pre-seed the two questions | 01:46 |
| jeblair | mordred: well, our puppet config will answer them; i mostly documented it for folks who want to install the client locally | 01:47 |
| *** chianingwang has joined #openstack-infra | 01:47 | |
| mordred | jeblair: sure. I'm just saing I think we could write a 3 line non-interactive shell script :) | 01:47 |
| jeblair | mordred: oh, ok cool. | 01:47 |
| jeblair | mordred: wasn't sure how hard that was :) | 01:47 |
| clarkb | you should use expect | 01:48 |
| jeblair | fungi, clarkb, mordred, SergeyLukjanov: so afstest.openstack.org exists and has the client running if you want to log in and poke around | 01:48 |
| clarkb | >_> <_> <_< | 01:48 |
| *** hdd has joined #openstack-infra | 01:48 | |
| anteaya | jeblair: awesome | 01:48 |
| jeblair | i figure we can do some tests with it tomorrow. i think now i will eat some brunswick stew. | 01:49 |
| krotscheck | Urm… ok, so when mordred said “AFS == Andrew File System” he was actually telling the truth? | 01:49 |
| clarkb | jeblair: that sounds good | 01:49 |
| clarkb | krotscheck: yes | 01:49 |
| * krotscheck never really knows when to take him seriously. | 01:49 | |
| *** SumitNaiksatam has quit IRC | 01:49 | |
| krotscheck | Talk about a blast from the past for me. Go go Alma Mater :) | 01:50 |
| jeblair | krotscheck: it's named after BOTH andrews! :) | 01:50 |
| mordred | jeblair: did I mention krotscheck went to CMU? | 01:50 |
| mordred | jeblair: http://paste.openstack.org/show/122424 | 01:50 |
| krotscheck | \o/ | 01:50 |
| mordred | jeblair: I believe that's what you need to do | 01:50 |
| jeblair | mordred: i don't recall hearing that, but he just told me :) | 01:51 |
| *** esker has joined #openstack-infra | 01:51 | |
| mordred | for the preseed stuff | 01:51 |
| jeblair | cmu is awesome | 01:51 |
| jeblair | afs _and_ self-driving cars | 01:51 |
| mordred | krotscheck: jeblair and I went to NCSU, which was also an AFS place | 01:51 |
| clarkb | mordred: that doesn't use expect | 01:51 |
| mordred | clarkb: no need. turns out this stuff just works :) | 01:51 |
| krotscheck | jeblair: I loved it so much I stayed there for FIVE YEARS! | 01:52 |
| mordred | clarkb: in fact, I KEEP meaning to make more of our puppet stuff just drive debconf where it's possible | 01:52 |
| mordred | clarkb: but then I don't | 01:52 |
| clarkb | mordred: I mean debconf is cool but expect is so much more 1990s like AFS | 01:52 |
| mordred | clarkb: you're trying to troll, by making fun of the age of AFS | 01:54 |
| mordred | clarkb: I see what you're doing there | 01:54 |
| anteaya | asselin: the split repo appears to have all the files and directories I would expect | 01:54 |
| anteaya | asselin: good job | 01:54 |
| jeblair | oops, forgot 'apt-get install krb5-user' | 01:54 |
| jeblair | just ran that | 01:54 |
| mordred | jeblair: oops :) | 01:54 |
| jeblair | this, however, is not 1990s: http://paste.openstack.org/show/122426/ | 01:55 |
| *** VijayTripathi has joined #openstack-infra | 01:55 | |
| mordred | jeblair: thats so awesome | 01:55 |
| jeblair | fungi: this is the first afs cell i've used with an aes token! | 01:55 |
| jeblair | okay, brunswick stew for realz | 01:56 |
| clarkb | mordred: yes sorry :P I figured you would enjoy that slight troll | 01:56 |
| mordred | clarkb: I did | 01:56 |
| anteaya | troll lite | 01:56 |
| mordred | clarkb: you should test my shell script snipped by running it as root on your laptop | 01:56 |
| *** VijayTripathi has quit IRC | 01:57 | |
| mordred | I'd test it - but I have afs already installed and working | 01:57 |
| mordred | ON A PLANE | 01:57 |
| clarkb | nice | 01:57 |
| clarkb | probably not tonight. I am about to find dinner then find something other than work to do | 01:57 |
| mordred | you should do that | 01:57 |
| clarkb | don't I need to add my principal somewhere too? | 01:57 |
| mordred | yup | 01:57 |
| mordred | I mean | 01:58 |
| mordred | you don't need that to install AFS or verify you can see the openstack stuff | 01:58 |
| dstufft | clarkb: lifeless wat | 01:58 |
| clarkb | oh do I get read only in that case? | 01:58 |
| mordred | yup | 01:58 |
| dstufft | I got highlights but too old to see them | 01:58 |
| mordred | we've got /afs/openstack.org set rl for system:anyuser | 01:58 |
| clarkb | dstufft: so get-pip.py if you have two indexes in your pip.conf or passed via -i | 01:58 |
| lifeless | dstufft: get-pip.py failing when the main index doesn't contain pip but secondary indices do. | 01:58 |
| clarkb | dstufft: and the first one is a dead index then the secondary index is completely unused | 01:58 |
| clarkb | and pip fails | 01:58 |
| lifeless | clarkb: its not a dead index | 01:59 |
| dstufft | well you can only pass one index via -i | 01:59 |
| dstufft | -i replaces the default index | 01:59 |
| dstufft | it's not additive | 01:59 |
| lifeless | dstufft: oh so my test was broken. | 01:59 |
| dstufft | --extra-index-url adds an extra index | 01:59 |
| lifeless | dstufft: but still - we have two indexes, one local. | 01:59 |
| clarkb | lifeless: in greghaynes' case there was no file there | 01:59 |
| dstufft | or --find-links adds a flat directory | 01:59 |
| clarkb | lifeless: index was file:///tmp/pypi but that dir did not exist | 02:00 |
| dstufft | lifeless: can you tell me exactly what you're invoking | 02:00 |
| clarkb | at least that is how I parsed | 02:00 |
| lifeless | clarkb: http://paste.openstack.org/show/122432/ does indeed behave differently | 02:01 |
| lifeless | dstufft: ^ | 02:01 |
| dstufft | file://$(pwd)/foo is nont existent yes? | 02:01 |
| *** mfer has quit IRC | 02:02 | |
| lifeless | dstufft: yes | 02:02 |
| lifeless | dstufft: but I just tried | 02:02 |
| dstufft | so yea, ``pip install testtools -i file://$(pwd)/foo`` is setting the primary index to file://$(pwd)/foo, overiding the default or any config file/env var | 02:02 |
| lifeless | dstufft: http://paste.openstack.org/show/122434/ | 02:02 |
| *** nosnos has joined #openstack-infra | 02:02 | |
| lifeless | dstufft: this is an artificial test | 02:02 |
| lifeless | dstufft: greghaynes has more detail | 02:03 |
| *** alexandra has joined #openstack-infra | 02:04 | |
| *** todd_dsm has quit IRC | 02:04 | |
| dstufft | I don't think I understand the problem | 02:05 |
| dstufft | I would expect pip install testtools -i file://$(pwd)/foo to fail | 02:05 |
| lifeless | dstufft: yes, but the second pip invocation fails too | 02:05 |
| *** Ryan_Lane has quit IRC | 02:05 | |
| *** mfer has joined #openstack-infra | 02:06 | |
| crinkle | there seem to be a lot of stuck jobs in zuul.openstack.org, is that related to the mass -1's that I see mentioned in scrollback? | 02:06 |
| mordred | jeblair: http://paste.openstack.org/show/122433 does the krb5-config default realm t | 02:06 |
| dstufft | lifeless: the second one probably fails becasue the URL is wrong | 02:06 |
| dstufft | lifeless: try https://pypi.python.org/simple/ | 02:06 |
| lifeless | dstufft: its not that, since I had -i there before and it works | 02:07 |
| *** asettle has quit IRC | 02:07 | |
| lifeless | dstufft: confirmed - http://paste.openstack.org/show/122435/ | 02:07 |
| clarkb | crinkle: yup, zuul is currently chewing through ~500 changes to update those -1s | 02:08 |
| dstufft | lifeless: name's still wrong | 02:08 |
| dstufft | imple != simple | 02:08 |
| lifeless | oh | 02:08 |
| clarkb | crinkle: and we are basically maxed out on the number of nodes we have right now | 02:08 |
| lifeless | that snuck in somewhere | 02:08 |
| crinkle | okie i'll wait, thanks clarkb | 02:08 |
| lifeless | dstufft: http://paste.openstack.org/show/122436/ still fails | 02:08 |
| dstufft | lifeless: what's the error? | 02:08 |
| lifeless | OSError: [Errno 2] No such file or directory: '/home/robertc/personal/testrepository/foo/testtools/' | 02:09 |
| lifeless | dstufft: it appears to be not consulting the extra index | 02:09 |
| lifeless | dstufft: its possible this is the file:// urls don't raise 404 that tchaypo fixed - has that actually gotten into a release? | 02:09 |
| *** koolhead17 has joined #openstack-infra | 02:09 | |
| dstufft | oh | 02:10 |
| dstufft | I see | 02:10 |
| dstufft | yea that's what it is | 02:10 |
| dstufft | and no it hasn't | 02:10 |
| tchaypo | yo ho ho | 02:10 |
| dstufft | pip install --upgrade git+https://github.com/pypa/pip.git@develop | 02:10 |
| dstufft | fixes it | 02:10 |
| lifeless | dstufft: please sir can I have the fix released please please please | 02:10 |
| lifeless | dstufft: please | 02:10 |
| tchaypo | and introduces 3 other bugs :) | 02:10 |
| dstufft | there are new bugs with the develop branch? | 02:10 |
| lifeless | dstufft: tchaypo was teasing. | 02:11 |
| tchaypo | (I have no knowledge of specific bugs, I'm just teasing) | 02:11 |
| dstufft | oh | 02:11 |
| tchaypo | sorry dstufft | 02:11 |
| lifeless | dstufft: So seriously, what blocks a release of pip ? | 02:11 |
| dstufft | I mean it wouldn't surprise me | 02:11 |
| lifeless | dstufft: its what 6 months ago we fixed this? | 02:11 |
| dstufft | lifeless: I wanted to get PEP 440 into the next release is the primary blocker, let me see how much more I need to do on that and see if it makes sense to just pushj that off into the next next release | 02:12 |
| greghaynes | ohai | 02:16 |
| * greghaynes reads backscroll | 02:16 | |
| greghaynes | So, using -i multiple times wasnt the actual issue, the issue is when we specify multiple indexes in .pip/pip.conf | 02:17 |
| lifeless | greghaynes: no | 02:17 |
| lifeless | greghaynes: the issue is that the index is a file index | 02:17 |
| lifeless | greghaynes: not an http index | 02:17 |
| lifeless | greghaynes: we either need a pip release (easy, if we can convince dstufft to do it, like today) | 02:18 |
| greghaynes | oh? so that works over http? | 02:18 |
| lifeless | greghaynes: or we need to export the local index over http | 02:18 |
| greghaynes | fun | 02:18 |
| tchaypo | greghaynes: when the index is http://, it catches a 404 and tries variant spellings | 02:18 |
| greghaynes | well, devpi solves this :) | 02:18 |
| lifeless | greghaynes: get-pip might be broken some other way, but pip itself is broken | 02:18 |
| tchaypo | greghaynes: when the index is file:// it just does sys.exit() instead | 02:18 |
| greghaynes | great | 02:18 |
| dstufft | the problem is when we switched to using requests I didn't catch all the exceptions we needed | 02:18 |
| dstufft | when I implemented file:// support | 02:18 |
| dstufft | so blame me | 02:18 |
| greghaynes | haha | 02:19 |
| dstufft | I'm bad at software what can I say | 02:19 |
| *** bdpayne has joined #openstack-infra | 02:19 | |
| tchaypo | greghaynes: my patch catches the exception and creates a fake 404 that then bubbles up and triggers the try-harder code | 02:19 |
| greghaynes | I have never seen a bug around not catching all the exceptions from requests ;) | 02:19 |
| lifeless | dstufft: well, file:/// should be in requests, no ? | 02:19 |
| greghaynes | (not really) | 02:19 |
| dstufft | lifeless: no, requests is a HTTP library not a access random files on your filesystem library | 02:19 |
| lifeless | dstufft: blah :) | 02:20 |
| dstufft | so I wrote a connection adapter that mounts a thing that adds that particular thing to requests for file:// urls | 02:20 |
| dstufft | I just didn't write it good enough | 02:20 |
| *** VijayTripathi has joined #openstack-infra | 02:21 | |
| dstufft | A full release isn't going to hapen today, I might be able to cut a RC once I take a minute to look at the current state of things | 02:21 |
| greghaynes | yep, so sounds like I should poke at devpi for now | 02:21 |
| greghaynes | which I was considering anyways | 02:21 |
| *** signed8bit_ZZZzz is now known as signed8bit | 02:21 | |
| *** signed8b_ has joined #openstack-infra | 02:23 | |
| dstufft | or ``twistd -n web --path $(pwd)/foo`` | 02:23 |
| dstufft | ora ny other web server | 02:23 |
| lifeless | SimpleServer can do it | 02:23 |
| dstufft | does simpleserver create index.html's automatically? | 02:23 |
| dstufft | I can't remember | 02:23 |
| clarkb | yes | 02:23 |
| dstufft | oh | 02:23 |
| dstufft | welp | 02:23 |
| dstufft | then just use that if you want | 02:24 |
| greghaynes | yes, we just already have some echo "blah" > index.html so I figure if were going to also run a http server why not use the tool that does all of this | 02:24 |
| greghaynes | oh, wat | 02:24 |
| greghaynes | it does auto index | 02:24 |
| clarkb | does for me | 02:24 |
| *** stevemar has quit IRC | 02:25 | |
| greghaynes | ok, SGTM :) | 02:25 |
| dstufft | I hate the simple protocol, but the one really good thing about it | 02:25 |
| *** stevemar has joined #openstack-infra | 02:25 | |
| dstufft | it makes it really easy to stand up a custom repo | 02:25 |
| *** signed8bit has quit IRC | 02:26 | |
| *** sdake_ has quit IRC | 02:32 | |
| *** signed8b_ has quit IRC | 02:34 | |
| *** pcm_ has quit IRC | 02:35 | |
| *** salv-orlando has quit IRC | 02:40 | |
| *** Ryan_Lane has joined #openstack-infra | 02:42 | |
| *** david-lyle_ has joined #openstack-infra | 02:42 | |
| *** david-lyle has quit IRC | 02:44 | |
| *** yaguang has quit IRC | 02:47 | |
| *** yaguang has joined #openstack-infra | 02:47 | |
| *** dimsum_ has quit IRC | 02:58 | |
| *** dimsum_ has joined #openstack-infra | 02:59 | |
| *** otherwiseguy has joined #openstack-infra | 03:00 | |
| *** radez is now known as radez_g0n3 | 03:01 | |
| *** amotoki_ has joined #openstack-infra | 03:01 | |
| *** dimsum_ has quit IRC | 03:03 | |
| *** mmaglana has joined #openstack-infra | 03:03 | |
| openstackgerrit | David Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Fix command to build docs in installation documentation https://review.openstack.org/129498 | 03:04 |
| openstackgerrit | David Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Ignore testr_subunit_log https://review.openstack.org/129503 | 03:04 |
| *** dane_leblanc has quit IRC | 03:06 | |
| *** sdake_ has joined #openstack-infra | 03:06 | |
| *** mfer has quit IRC | 03:07 | |
| *** VijayTripathi has quit IRC | 03:07 | |
| *** amotoki has quit IRC | 03:09 | |
| *** amotoki_ is now known as amotoki | 03:10 | |
| *** Longgeek has joined #openstack-infra | 03:10 | |
| *** yjiang5 has quit IRC | 03:13 | |
| *** r1chardj0n3s is now known as r1chardj0n3s_afk | 03:13 | |
| *** KanagarajM has joined #openstack-infra | 03:14 | |
| *** yjiang5 has joined #openstack-infra | 03:15 | |
| *** Longgeek has quit IRC | 03:15 | |
| *** greghaynes has quit IRC | 03:15 | |
| *** greghaynes has joined #openstack-infra | 03:18 | |
| *** ivar-laz_ has joined #openstack-infra | 03:21 | |
| *** otter768 has quit IRC | 03:22 | |
| *** otter768 has joined #openstack-infra | 03:23 | |
| *** pcrews has quit IRC | 03:23 | |
| openstackgerrit | David Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Add support for "Override Build Parameters" option in EnvInject https://review.openstack.org/129189 | 03:24 |
| openstackgerrit | David Pursehouse proposed a change to openstack-infra/jenkins-job-builder: Add tests for properties inject https://review.openstack.org/129502 | 03:24 |
| *** esker has quit IRC | 03:25 | |
| *** ivar-lazzaro has quit IRC | 03:25 | |
| *** ivar-laz_ has quit IRC | 03:26 | |
| *** nosnos has quit IRC | 03:26 | |
| *** bdpayne has quit IRC | 03:27 | |
| *** nosnos has joined #openstack-infra | 03:27 | |
| *** yaguang has quit IRC | 03:28 | |
| *** VijayTripathi has joined #openstack-infra | 03:29 | |
| *** armax has quit IRC | 03:30 | |
| *** nosnos has quit IRC | 03:32 | |
| *** otter768 has quit IRC | 03:33 | |
| *** ayoung has quit IRC | 03:35 | |
| *** ChuckC has quit IRC | 03:35 | |
| *** spzala has quit IRC | 03:36 | |
| *** koolhead17 has quit IRC | 03:40 | |
| *** mmaglana has quit IRC | 03:42 | |
| *** mmaglana has joined #openstack-infra | 03:42 | |
| *** hdd has quit IRC | 03:53 | |
| *** r1chardj0n3s_afk is now known as r1chardj0n3s | 03:53 | |
| *** gokrokve_ has joined #openstack-infra | 04:02 | |
| *** jamespd has quit IRC | 04:04 | |
| *** jamespd has joined #openstack-infra | 04:04 | |
| *** bradjones has quit IRC | 04:05 | |
| *** alexandra is now known as asettle | 04:05 | |
| *** gokrokve has quit IRC | 04:06 | |
| *** gokrokve_ has quit IRC | 04:07 | |
| *** bradjones has joined #openstack-infra | 04:07 | |
| *** garyh has quit IRC | 04:09 | |
| *** nosnos has joined #openstack-infra | 04:16 | |
| *** emagana has joined #openstack-infra | 04:18 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/system-config: Add HP Cloud Centos7 images https://review.openstack.org/128498 | 04:29 |
| *** mbacchi has quit IRC | 04:32 | |
| *** otherwiseguy has quit IRC | 04:40 | |
| *** garyh has joined #openstack-infra | 04:40 | |
| *** timrc is now known as timrc-afk | 04:49 | |
| *** changbl has joined #openstack-infra | 04:49 | |
| *** armax has joined #openstack-infra | 04:50 | |
| *** jerryz has quit IRC | 04:54 | |
| *** jerryz has joined #openstack-infra | 05:01 | |
| *** wendar_ is now known as wendar | 05:03 | |
| *** yfried has joined #openstack-infra | 05:16 | |
| *** chandankumar has joined #openstack-infra | 05:18 | |
| *** yfried is now known as yfried|afk | 05:24 | |
| *** gokrokve has joined #openstack-infra | 05:25 | |
| *** yfried has joined #openstack-infra | 05:27 | |
| *** yfried|afk has quit IRC | 05:29 | |
| *** emagana has quit IRC | 05:29 | |
| *** emagana has joined #openstack-infra | 05:30 | |
| *** Sukhdev has joined #openstack-infra | 05:30 | |
| *** harlowja is now known as harlowja_away | 05:30 | |
| *** otherwiseguy has joined #openstack-infra | 05:31 | |
| *** yfried_ has joined #openstack-infra | 05:32 | |
| *** yfried_ is now known as yfried|afk | 05:32 | |
| *** yfried has quit IRC | 05:32 | |
| *** gokrokve has quit IRC | 05:34 | |
| *** gokrokve has joined #openstack-infra | 05:34 | |
| *** emagana has quit IRC | 05:34 | |
| *** ildikov has joined #openstack-infra | 05:37 | |
| *** BharatK has joined #openstack-infra | 05:38 | |
| *** Longgeek has joined #openstack-infra | 05:41 | |
| *** Longgeek has quit IRC | 05:47 | |
| *** reed has quit IRC | 05:47 | |
| *** david-lyle_ has quit IRC | 05:48 | |
| *** k4n0 has joined #openstack-infra | 05:50 | |
| *** ryanpetrello has quit IRC | 05:55 | |
| *** ryanpetrello has joined #openstack-infra | 05:55 | |
| *** gokrokve has quit IRC | 05:57 | |
| *** gokrokve has joined #openstack-infra | 05:58 | |
| *** andreaf has quit IRC | 05:58 | |
| *** andreaf has joined #openstack-infra | 05:58 | |
| *** yfried|afk is now known as yfried_ | 06:02 | |
| *** gokrokve has quit IRC | 06:02 | |
| *** otherwiseguy has quit IRC | 06:02 | |
| *** maxbit has joined #openstack-infra | 06:13 | |
| *** r1chardj0n3s is now known as r1chardj0n3s_afk | 06:14 | |
| *** yolanda has joined #openstack-infra | 06:15 | |
| *** Longgeek has joined #openstack-infra | 06:15 | |
| *** chandankumar has quit IRC | 06:15 | |
| *** bo_sh has joined #openstack-infra | 06:19 | |
| *** oramraz has joined #openstack-infra | 06:19 | |
| openstackgerrit | A change was merged to openstack-infra/system-config: Fix one missed afs->openafs rename https://review.openstack.org/129729 | 06:19 |
| *** sileht has quit IRC | 06:19 | |
| *** armax has quit IRC | 06:19 | |
| *** gokrokve has joined #openstack-infra | 06:28 | |
| *** yfried_ has quit IRC | 06:29 | |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/project-config: Use beaker tests on puppet storyboard https://review.openstack.org/126093 | 06:29 |
| *** gokrokve has quit IRC | 06:33 | |
| *** yfried_ has joined #openstack-infra | 06:36 | |
| *** chandankumar has joined #openstack-infra | 06:38 | |
| *** flaper87|afk is now known as flaper87 | 06:39 | |
| *** VijayTripathi has quit IRC | 06:40 | |
| *** Adri2000 has quit IRC | 06:54 | |
| *** _nadya_ has joined #openstack-infra | 06:56 | |
| *** _nadya_ has quit IRC | 06:59 | |
| *** _nadya_ has joined #openstack-infra | 07:00 | |
| *** _nadya_ has quit IRC | 07:00 | |
| *** Adri2000 has joined #openstack-infra | 07:01 | |
| *** hashar has joined #openstack-infra | 07:02 | |
| *** luqas has joined #openstack-infra | 07:02 | |
| *** ihrachyshka has joined #openstack-infra | 07:06 | |
| *** Sukhdev has quit IRC | 07:08 | |
| *** _nadya_ has joined #openstack-infra | 07:08 | |
| *** dizquierdo has joined #openstack-infra | 07:08 | |
| *** andreykurilin_ has joined #openstack-infra | 07:11 | |
| *** e0ne has joined #openstack-infra | 07:12 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/project-config: Unbreak translation proposals https://review.openstack.org/129827 | 07:13 |
| *** ihrachyshka has quit IRC | 07:13 | |
| openstackgerrit | Denis A. Kochetkov proposed a change to openstack-infra/jenkins-job-builder: Ability to restrict where job can be executed. https://review.openstack.org/129828 | 07:13 |
| *** jyuso has joined #openstack-infra | 07:16 | |
| *** jyuso has quit IRC | 07:17 | |
| *** HeOS has quit IRC | 07:18 | |
| *** amcrn has quit IRC | 07:20 | |
| *** MaxV has joined #openstack-infra | 07:23 | |
| *** changbl has quit IRC | 07:24 | |
| *** mmaglana has quit IRC | 07:26 | |
| *** Guest59036 has joined #openstack-infra | 07:27 | |
| *** Ryan_Lane has quit IRC | 07:28 | |
| openstackgerrit | Denis A. Kochetkov proposed a change to openstack-infra/jenkins-job-builder: Review issues fixed. https://review.openstack.org/129287 | 07:28 |
| *** gokrokve has joined #openstack-infra | 07:28 | |
| *** jgallard has joined #openstack-infra | 07:28 | |
| *** skolekonov has joined #openstack-infra | 07:30 | |
| *** mrmartin has joined #openstack-infra | 07:32 | |
| StevenK | dstufft: When you're around, I've found that my bandersnatch mirror is not quite consistent. There are a number of files that are referenced in simple/*/index.html that are not under packages at all. Is there any way I can tell bandersnatch, "You're not done with this packages, try harder" ? | 07:32 |
| dstufft | StevenK: you can reset the state of the bandersnatch mirror completely and it'll do a full sync again instead of a partial sync | 07:33 |
| dstufft | it won't redownload files it already has though | 07:33 |
| dstufft | so it's not super bad | 07:33 |
| StevenK | ls: cannot access ../../packages/py2/o/os-collect-config/os_collect_config-0.1.29-py2-none-any.whl: No such file or directory | 07:33 |
| *** gokrokve has quit IRC | 07:33 | |
| StevenK | ls: cannot access ../../packages/source/c/crossbar/crossbar-0.9.9.zip: No such file or directory | 07:33 |
| StevenK | Things like that | 07:33 |
| dstufft | delete the state file and rerun sync | 07:34 |
| StevenK | dstufft: As in <pypi root>/status ? | 07:34 |
| dstufft | and the todo file | 07:34 |
| StevenK | No todo file | 07:34 |
| dstufft | https://pypi.python.org/pypi/bandersnatch#maintenance | 07:35 |
| StevenK | dstufft: Neither state nor todo exist in /srv/pypi | 07:35 |
| StevenK | The cloest I have is status | 07:35 |
| dstufft | hrm | 07:36 |
| StevenK | steven@thrashed:~$ cat /srv/pypi/status ; echo | 07:36 |
| StevenK | 1276859 | 07:36 |
| dstufft | try that then | 07:36 |
| StevenK | dstufft: Oh, the other thing I wanted to whine about, is can you have bandersnatch log the filesize when it's downloading something | 07:37 |
| *** dimsum_ has joined #openstack-infra | 07:37 | |
| StevenK | Then you can get some idea of the churn per day/time period | 07:37 |
| *** afazekas has joined #openstack-infra | 07:38 | |
| dstufft | StevenK: I don't actually have commit or much to do with bandersantch :) I'm sure Christian would be happy to have a PR for that though | 07:38 |
| *** andreaf has quit IRC | 07:38 | |
| * StevenK stabs Bitbucket for not allowing new openid signups | 07:41 | |
| *** andreykurilin_ has quit IRC | 07:41 | |
| *** dimsum_ has quit IRC | 07:41 | |
| openstackgerrit | yolanda.robla proposed a change to openstack-infra/system-config: Install puppet modules in right order https://review.openstack.org/122132 | 07:42 |
| *** mgrygoriev has joined #openstack-infra | 07:47 | |
| *** yfried_ is now known as yfried|afk | 07:48 | |
| *** stevemar has quit IRC | 07:50 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 07:50 | |
| *** jcoufal has joined #openstack-infra | 07:50 | |
| *** yfried|afk is now known as yfried_ | 07:52 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 07:53 | |
| *** Hal has joined #openstack-infra | 07:54 | |
| *** Hal is now known as Guest57939 | 07:54 | |
| *** Guest59036 has quit IRC | 07:54 | |
| *** alexpilotti has joined #openstack-infra | 07:54 | |
| BharatK | Hi, | 07:55 |
| BharatK | I am following https://raw.github.com/jaypipes/os-ext-testing/master/puppet/install_master.sh script to install Jenkins Mater Node | 07:55 |
| BharatK | But facing so many issues with puppet | 07:55 |
| BharatK | Is that the correct link? | 07:56 |
| *** yamamoto has quit IRC | 07:56 | |
| *** jpich has joined #openstack-infra | 07:56 | |
| *** Guest57939 has quit IRC | 07:57 | |
| *** Hal_ has joined #openstack-infra | 08:00 | |
| *** jistr has joined #openstack-infra | 08:02 | |
| *** jedimike has joined #openstack-infra | 08:03 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 08:04 | |
| *** mpaolino has joined #openstack-infra | 08:09 | |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/system-config: Test failure of apply test https://review.openstack.org/129837 | 08:09 |
| *** berendt has joined #openstack-infra | 08:10 | |
| *** arxcruz has joined #openstack-infra | 08:11 | |
| *** Hal_ has quit IRC | 08:13 | |
| *** mancdaz_away is now known as mancdaz | 08:14 | |
| *** jp_at_hp has joined #openstack-infra | 08:18 | |
| *** yamahata has quit IRC | 08:21 | |
| openstackgerrit | yolanda.robla proposed a change to openstack-infra/storyboard-webclient: Allow assigning tasks without expanding the edition box https://review.openstack.org/125643 | 08:24 |
| openstackgerrit | A change was merged to openstack-infra/project-config: Remove confusing access "refs/meta/config" ACL def https://review.openstack.org/129754 | 08:24 |
| *** koolhead17 has joined #openstack-infra | 08:27 | |
| *** mmaglana has joined #openstack-infra | 08:27 | |
| *** jlibosva has joined #openstack-infra | 08:27 | |
| openstackgerrit | A change was merged to openstack-infra/project-config: Adding puppet-pip as split out module https://review.openstack.org/126096 | 08:27 |
| *** koolhead_ has joined #openstack-infra | 08:27 | |
| *** koolhead17 has quit IRC | 08:27 | |
| *** _nadya_ has quit IRC | 08:28 | |
| *** gokrokve has joined #openstack-infra | 08:28 | |
| *** camunoz has quit IRC | 08:28 | |
| *** e0ne has quit IRC | 08:30 | |
| *** gokrokve has quit IRC | 08:33 | |
| *** lttrl has quit IRC | 08:33 | |
| *** openstackgerrit has quit IRC | 08:34 | |
| *** openstackgerrit has joined #openstack-infra | 08:35 | |
| *** sileht has joined #openstack-infra | 08:35 | |
| *** vdo has joined #openstack-infra | 08:37 | |
| *** Hal_ has joined #openstack-infra | 08:38 | |
| *** mrmartin has quit IRC | 08:41 | |
| *** HeOS has joined #openstack-infra | 08:42 | |
| *** talluri has joined #openstack-infra | 08:42 | |
| *** davidhadas has quit IRC | 08:43 | |
| *** cdent has joined #openstack-infra | 08:44 | |
| *** Hal_ has quit IRC | 08:46 | |
| *** hashar has quit IRC | 08:47 | |
| *** Hal_ has joined #openstack-infra | 08:51 | |
| *** ihrachyshka has joined #openstack-infra | 08:51 | |
| *** bradjones_ has joined #openstack-infra | 08:53 | |
| *** bradjones has quit IRC | 08:53 | |
| *** bradjones_ is now known as bradjones | 08:53 | |
| *** bradjones has quit IRC | 08:53 | |
| *** bradjones has joined #openstack-infra | 08:53 | |
| *** Hal_ has quit IRC | 08:56 | |
| *** lttrl has joined #openstack-infra | 09:00 | |
| *** koolhead_ has quit IRC | 09:08 | |
| *** pelix has joined #openstack-infra | 09:09 | |
| *** yamamoto has joined #openstack-infra | 09:09 | |
| *** andreaf_ is now known as andreaf | 09:10 | |
| openstackgerrit | A change was merged to openstack-infra/project-config: puppet-sahara: update Gerrit permissions https://review.openstack.org/129655 | 09:13 |
| *** yamamoto has quit IRC | 09:15 | |
| *** hashar has joined #openstack-infra | 09:16 | |
| *** mpaolino has quit IRC | 09:21 | |
| *** rlandy has joined #openstack-infra | 09:21 | |
| hashar | hello! any idea how I can have a change in Gerrit to be linked with a launchpad bug? I proposed https://review.openstack.org/#/c/128921/ with the magic header 'Closes-Bug: #1358517' but I had to manually update the bug on launchpad https://bugs.launchpad.net/zuul/+bug/1358517 | 09:26 |
| uvirtbot | Launchpad bug 1358517 in zuul "Change stuck in queue when Gearman errors during merge:merge submit" [Undecided,In progress] | 09:27 |
| hashar | I guess I am doing something wrong :D | 09:27 |
| *** talluri has quit IRC | 09:27 | |
| *** mugsie has quit IRC | 09:27 | |
| rcarrillocruz | hey | 09:28 |
| rcarrillocruz | do you still use Launchpad for filing bugs for infra? | 09:28 |
| rcarrillocruz | or should I check storyboard? | 09:28 |
| *** gokrokve has joined #openstack-infra | 09:28 | |
| *** mugsie has joined #openstack-infra | 09:29 | |
| rcarrillocruz | AJaeger: morning :-), if you are around ^ | 09:29 |
| AJaeger | rcarrillocruz, use launchpad | 09:29 |
| rcarrillocruz | thx | 09:30 |
| *** salv-orlando has joined #openstack-infra | 09:32 | |
| *** gokrokve has quit IRC | 09:33 | |
| *** e0ne has joined #openstack-infra | 09:37 | |
| *** KanagarajM has quit IRC | 09:38 | |
| *** 1JTAAQZNO has joined #openstack-infra | 09:40 | |
| *** pblaho has joined #openstack-infra | 09:40 | |
| *** teran has quit IRC | 09:42 | |
| *** kingia has joined #openstack-infra | 09:51 | |
| *** kingia is now known as iain_king | 09:51 | |
| *** talluri has joined #openstack-infra | 09:52 | |
| *** yamamoto has joined #openstack-infra | 09:54 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Fix command to build docs in installation documentation https://review.openstack.org/129498 | 09:56 |
| *** hashar_ has joined #openstack-infra | 09:56 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Ignore testr_subunit_log https://review.openstack.org/129503 | 09:58 |
| *** hashar has quit IRC | 09:59 | |
| *** teran has joined #openstack-infra | 10:00 | |
| *** unicell has quit IRC | 10:01 | |
| *** nosnos has quit IRC | 10:02 | |
| *** nosnos has joined #openstack-infra | 10:03 | |
| yolanda | here, can i ask a question about how pypi publish works? i'm interested in picking a change i did for gerritlib, but i see that the latest version published in the mirror is 0.4.0 from 16 april | 10:04 |
| *** teran has quit IRC | 10:04 | |
| *** yfried_ is now known as yfried|afk | 10:05 | |
| *** koolhead17 has joined #openstack-infra | 10:05 | |
| *** koolhead17 has joined #openstack-infra | 10:05 | |
| *** _nadya_ has joined #openstack-infra | 10:05 | |
| *** yfried|afk is now known as yfried_ | 10:07 | |
| *** nosnos has quit IRC | 10:08 | |
| AJaeger | yolanda, if a release gets tagged, we publish automatically to pypi | 10:09 |
| *** jgallard has quit IRC | 10:09 | |
| AJaeger | yolanda, 0.4.0 is the latest release - so ask jeblair to release again if there'S a need for it. | 10:10 |
| *** jcoufal has quit IRC | 10:10 | |
| yolanda | i added some changes to gerritlib, to allow managing descriptions in projects, so i need that code to be in the mirror in order to use it from jeepyb | 10:11 |
| *** jp_at_hp has quit IRC | 10:13 | |
| yolanda | AJaeger, thx, i'll ping jeblair about it | 10:13 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/system-config: Update comment in logs.vhost https://review.openstack.org/129861 | 10:13 |
| *** yamahata has joined #openstack-infra | 10:13 | |
| *** jp_at_hp has joined #openstack-infra | 10:14 | |
| *** teran has joined #openstack-infra | 10:15 | |
| *** teran_ has joined #openstack-infra | 10:19 | |
| openstackgerrit | Denis A. Kochetkov proposed a change to openstack-infra/jenkins-job-builder: Added description for node attribute. https://review.openstack.org/129287 | 10:19 |
| *** koolhead17 has quit IRC | 10:20 | |
| *** teran has quit IRC | 10:21 | |
| *** teran_ has quit IRC | 10:25 | |
| *** hashar_ is now known as hashar | 10:28 | |
| *** gokrokve has joined #openstack-infra | 10:28 | |
| *** gokrokve has quit IRC | 10:29 | |
| *** gokrokve has joined #openstack-infra | 10:30 | |
| *** gokrokve has quit IRC | 10:35 | |
| *** koolhead17 has joined #openstack-infra | 10:37 | |
| *** koolhead17 has quit IRC | 10:37 | |
| *** koolhead17 has joined #openstack-infra | 10:37 | |
| *** jcoufal has joined #openstack-infra | 10:38 | |
| *** jcoufal has quit IRC | 10:39 | |
| *** teran has joined #openstack-infra | 10:39 | |
| *** jcoufal has joined #openstack-infra | 10:39 | |
| *** teran has quit IRC | 10:39 | |
| *** teran has joined #openstack-infra | 10:40 | |
| bo_sh | fungi: hi there. | 10:41 |
| *** pc_m has joined #openstack-infra | 10:42 | |
| *** bradjones has quit IRC | 10:45 | |
| *** bradjones has joined #openstack-infra | 10:47 | |
| *** koolhead17 has quit IRC | 10:53 | |
| *** danpb has joined #openstack-infra | 10:53 | |
| danpb | is anyone aware of problems with jenkins git merges today ? it has marked a whole bunch of patches as having merge conflicts, when in fact no conflicts exist ?!?! | 10:55 |
| *** ociuhandu has joined #openstack-infra | 10:56 | |
| openstackgerrit | A change was merged to openstack-infra/project-config: Use beaker tests on puppet storyboard https://review.openstack.org/126093 | 10:58 |
| *** baoli has quit IRC | 11:02 | |
| AJaeger | danpb, all patches were requeued and thus we have a very long queue... | 11:02 |
| hashar | danpb: there was an issue yesterday, around 19:50 UTC iirc | 11:03 |
| *** jamielennox_ has joined #openstack-infra | 11:04 | |
| hashar | though it is kind of slow processing jobs | 11:05 |
| hashar | http://tinyurl.com/k9nfovw | 11:06 |
| *** rcarrillocruz has quit IRC | 11:06 | |
| *** jamielennox_ is now known as jamielennox | 11:06 | |
| *** rcarrillocruz has joined #openstack-infra | 11:08 | |
| danpb | why would re-queuing jobs result in them all being marked as in merge conflict though ? | 11:08 |
| hashar | there was an issue with a Zuul merger that could not connect to Gerrit | 11:08 |
| hashar | so it would fail the merge of the proposed patchset on tip of the target branch and report back a merge error | 11:09 |
| danpb | ah, ok | 11:09 |
| hashar | since that marked a bunch of changes with Verified -1 , I guess that all affected patch have been manually reenqueued | 11:09 |
| hashar | and seems the rate at which jobs are being processed as slowed down a few hours ago | 11:09 |
| hashar | which is what the tiny url link show | 11:10 |
| *** teran_ has joined #openstack-infra | 11:10 | |
| *** yamamoto has quit IRC | 11:11 | |
| *** BharatK has quit IRC | 11:12 | |
| *** habib has joined #openstack-infra | 11:13 | |
| *** teran has quit IRC | 11:13 | |
| *** mwagner_lap has quit IRC | 11:14 | |
| *** ildikov has quit IRC | 11:20 | |
| *** dimsum_ has joined #openstack-infra | 11:21 | |
| *** dmakogon_ is now known as denis_makogon | 11:22 | |
| *** miqui has quit IRC | 11:24 | |
| *** hashar has quit IRC | 11:25 | |
| *** BharatK has joined #openstack-infra | 11:26 | |
| dimsum_ | SergeyLukjanov: regarding docker-py, it's fine to say that we don't need it since it is used by a stack forge project. but the practical problem is that gate-nova-docker-requirements, check-tempest-dsvm-docker and check-tempest-dsvm-f20-docker fail miserably | 11:26 |
| *** jistr is now known as jistr|english | 11:27 | |
| *** gokrokve has joined #openstack-infra | 11:28 | |
| *** weshay has joined #openstack-infra | 11:28 | |
| *** aysyd has joined #openstack-infra | 11:29 | |
| *** mmedvede has joined #openstack-infra | 11:29 | |
| *** danpb has left #openstack-infra | 11:30 | |
| *** isviridov_away is now known as isviridov | 11:31 | |
| *** gokrokve has quit IRC | 11:32 | |
| *** _nadya_ has quit IRC | 11:37 | |
| *** BharatK has quit IRC | 11:37 | |
| *** pelix has quit IRC | 11:37 | |
| *** afazekas has quit IRC | 11:37 | |
| *** k4n0 has quit IRC | 11:37 | |
| *** asettle has quit IRC | 11:37 | |
| *** rwsu has quit IRC | 11:37 | |
| *** sunil_ has quit IRC | 11:37 | |
| *** larsks has quit IRC | 11:37 | |
| *** sunil_ has joined #openstack-infra | 11:37 | |
| *** asettle has joined #openstack-infra | 11:37 | |
| *** larsks has joined #openstack-infra | 11:37 | |
| *** afazekas has joined #openstack-infra | 11:37 | |
| *** BharatK has joined #openstack-infra | 11:37 | |
| *** rwsu has joined #openstack-infra | 11:38 | |
| *** pelix has joined #openstack-infra | 11:38 | |
| *** k4n0 has joined #openstack-infra | 11:38 | |
| *** flaper87 has quit IRC | 11:40 | |
| *** flaper87 has joined #openstack-infra | 11:40 | |
| *** mfink has quit IRC | 11:41 | |
| *** Longgeek has quit IRC | 11:42 | |
| *** luqas has quit IRC | 11:43 | |
| *** mrmartin has joined #openstack-infra | 11:44 | |
| *** jp_at_hp has quit IRC | 11:45 | |
| *** jp_at_hp has joined #openstack-infra | 11:45 | |
| *** afazekas has quit IRC | 11:46 | |
| *** afazekas has joined #openstack-infra | 11:46 | |
| *** k4n0 has quit IRC | 11:46 | |
| *** k4n0 has joined #openstack-infra | 11:46 | |
| *** BharatK has quit IRC | 11:46 | |
| *** BharatK has joined #openstack-infra | 11:46 | |
| *** pelix has quit IRC | 11:46 | |
| *** pelix has joined #openstack-infra | 11:46 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed a change to openstack-infra/system-config: Expose the vhost_name parameter on openstack_project::nodepool https://review.openstack.org/129876 | 11:48 |
| *** unicell has joined #openstack-infra | 11:50 | |
| *** Longgeek has joined #openstack-infra | 11:50 | |
| *** ZZelle has quit IRC | 11:51 | |
| *** yamamoto has joined #openstack-infra | 11:51 | |
| *** ZZelle has joined #openstack-infra | 11:51 | |
| *** dprince has joined #openstack-infra | 11:52 | |
| *** baoli has joined #openstack-infra | 11:53 | |
| *** HeOS_ has quit IRC | 11:53 | |
| *** HeOS_ has joined #openstack-infra | 11:54 | |
| *** HeOS_ has joined #openstack-infra | 11:55 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/project-config: allow for NON_STANDARD_REQS override https://review.openstack.org/129880 | 11:56 |
| dimsum_ | SergeyLukjanov: Here's a counter proposal :) ^^^ | 11:56 |
| *** yamamoto has quit IRC | 11:57 | |
| *** maxbit has quit IRC | 12:01 | |
| *** hdd has joined #openstack-infra | 12:05 | |
| *** BharatK has quit IRC | 12:07 | |
| openstackgerrit | A change was merged to openstack/requirements: Add demjson to requirements https://review.openstack.org/114818 | 12:08 |
| *** jgallard has joined #openstack-infra | 12:09 | |
| *** mwagner_lap has joined #openstack-infra | 12:11 | |
| *** mrmartin has quit IRC | 12:12 | |
| *** jedimike has quit IRC | 12:13 | |
| *** mbacchi has joined #openstack-infra | 12:14 | |
| *** hashar has joined #openstack-infra | 12:15 | |
| *** mrmartin has joined #openstack-infra | 12:16 | |
| *** mancdaz is now known as mancdaz_away | 12:18 | |
| *** mancdaz_away has quit IRC | 12:18 | |
| *** dkranz has joined #openstack-infra | 12:19 | |
| *** ildikov has joined #openstack-infra | 12:20 | |
| *** mancdaz_away has joined #openstack-infra | 12:21 | |
| *** BharatK has joined #openstack-infra | 12:21 | |
| *** mancdaz_away is now known as mancdaz | 12:21 | |
| *** rkukura has quit IRC | 12:22 | |
| *** koolhead17 has joined #openstack-infra | 12:22 | |
| *** koolhead17 has joined #openstack-infra | 12:22 | |
| *** mancdaz has quit IRC | 12:23 | |
| *** mancdaz_away has joined #openstack-infra | 12:23 | |
| *** mancdaz_away is now known as mancdaz | 12:24 | |
| *** mancdaz has quit IRC | 12:26 | |
| *** yamamoto has joined #openstack-infra | 12:28 | |
| *** habib has quit IRC | 12:28 | |
| *** k4n0 has quit IRC | 12:28 | |
| *** gokrokve has joined #openstack-infra | 12:28 | |
| *** mancdaz_away has joined #openstack-infra | 12:29 | |
| *** mancdaz_away is now known as mancdaz | 12:29 | |
| *** yamamoto has quit IRC | 12:29 | |
| *** yamamoto has joined #openstack-infra | 12:30 | |
| *** gokrokve has quit IRC | 12:33 | |
| *** yamamoto has quit IRC | 12:35 | |
| *** chandankumar has quit IRC | 12:37 | |
| *** dimsum_ has quit IRC | 12:38 | |
| *** dimsum_ has joined #openstack-infra | 12:39 | |
| *** kgiusti has joined #openstack-infra | 12:39 | |
| *** mfink has joined #openstack-infra | 12:43 | |
| *** timrc-afk is now known as timrc | 12:46 | |
| *** spzala has joined #openstack-infra | 12:47 | |
| *** kgiusti has quit IRC | 12:53 | |
| *** kgiusti has joined #openstack-infra | 12:54 | |
| *** miqui has joined #openstack-infra | 12:56 | |
| *** julim has joined #openstack-infra | 12:57 | |
| *** jistr|english is now known as jistr | 12:57 | |
| AJaeger | looking at the zuul queue, I see that the top jobs in check and gate queue are not moving forward. Is there a reason why those jobs are not started? | 13:04 |
| *** yfried_ has quit IRC | 13:04 | |
| *** Sukhdev has joined #openstack-infra | 13:04 | |
| *** iain_king has quit IRC | 13:05 | |
| fungi | hashar: you only get a fix-proposed/in-progress update to the bug if you correctly link it in the commit message of your first patchset (this is a protection against spamming the bug on every patchset). if you want to look at improving the script which updates bugs, it's http://git.openstack.org/cgit/openstack-infra/jeepyb/tree/jeepyb/cmd/update_bug.py | 13:06 |
| fungi | (but as we're soon migrating to storyboard, may not be worth the headache) | 13:06 |
| *** bcourt has quit IRC | 13:06 | |
| *** luqas has joined #openstack-infra | 13:06 | |
| hashar | fungi: that is what I eventually figured out :_D | 13:07 |
| *** dkranz has quit IRC | 13:07 | |
| hashar | i.e. only on first patchset | 13:07 |
| fungi | AJaeger: taking a look now that i'm caught up on scrollback | 13:07 |
| hashar | fungi: thanks for the follow up | 13:07 |
| AJaeger | good morning, fungi! | 13:08 |
| AJaeger | fungi, the openstack/cinder patch in check is there for ages but one was before it this morning - and the system-config in post has not moved at all. | 13:08 |
| hashar | fungi: also Zuul seems to no more be triggering jobs since 6am UTC , at least the queue of jobs barely progress http://tinyurl.com/k9nfovw | 13:08 |
| *** salv-orlando has quit IRC | 13:09 | |
| *** sdake_ has quit IRC | 13:09 | |
| AJaeger | hashar, there are jobs triggered, I got some out of check and gate... | 13:09 |
| *** sdake_ has joined #openstack-infra | 13:09 | |
| *** sdake_ has quit IRC | 13:09 | |
| *** sdake_ has joined #openstack-infra | 13:09 | |
| hashar | yeah for some new ones | 13:09 |
| AJaeger | the post queue might be due to the low priority... | 13:09 |
| hashar | I have the issue on my setup as well | 13:10 |
| hashar | from time to time jobs refuse to be retriggered | 13:10 |
| hashar | possibly because Zuul consider them as already being running | 13:10 |
| hashar | I unleash it by promoting a change in the pipeline, but that causes every changes to be retriggered :/ | 13:10 |
| hashar | or maybe some of those jobs are no more registered in Zuul gearman daemon :/ | 13:11 |
| *** yamahata has quit IRC | 13:11 | |
| *** amotoki has quit IRC | 13:11 | |
| *** yamahata has joined #openstack-infra | 13:12 | |
| AJaeger | the check queue total length is osciallating around 600 - this morning (7 hours ago) 590 then up to 620, now at 604 again. | 13:12 |
| *** koolhead17 has quit IRC | 13:13 | |
| fungi | okay, so for starters... we're maxxed out at quota in our providers, thus we're only assigning new nodes as they get recycled. we currently seem to be able to run between 200 and 300 jobs concurrently | 13:13 |
| fungi | any activity in gate, release, pre-release or tag pipelines will take resources first, then check pipeline gets whatever's available after that | 13:13 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Add a --version command line flag to show JJB version https://review.openstack.org/127844 | 13:14 |
| jroll | ughhh | 13:14 |
| *** mancdaz is now known as mancdaz_away | 13:14 | |
| jroll | fungi: have y'all asked rackspace to bump quota? | 13:14 |
| fungi | jroll: they reduce it on us regularly | 13:14 |
| jroll | ... | 13:14 |
| fungi | seems the churn we create by booting and deleting nodes is significant compared to most tenants | 13:15 |
| jroll | god forbid you use the cloud like a cloud | 13:15 |
| jroll | is that churn actually a problem? | 13:15 |
| AJaeger | glad that I screwed up the translation periodic jobs - that's 10 jobs less for the check pipeline ;( https://review.openstack.org/129827 | 13:15 |
| jaypipes | BharatK: pong... | 13:15 |
| fungi | it has been described to us as creating negative performance impact within the regions we're using, yes | 13:15 |
| jaypipes | BharatK: unfortunately, the Puppet modules for setting up CI systems has changed a bit, and I have not had the time yet to adjust my blog post and the os-ext-testing repo... | 13:16 |
| AJaeger | So, post will not get anything. Should we stop approving patches? | 13:16 |
| jroll | fungi: interesting. | 13:16 |
| fungi | i think a lot of that may be that because we're booting from snapshots and snapshot storage is somewhere other than normal image storage, that plays into the performance issues somewhat | 13:16 |
| jroll | this saddens me greatly | 13:16 |
| jroll | mmmm | 13:16 |
| joa | jaypipes: I think someone cloned it and made an updated version | 13:16 |
| jroll | so the solution is to throttle infra | 13:16 |
| *** koolhead17 has joined #openstack-infra | 13:16 | |
| *** dane_leblanc has joined #openstack-infra | 13:16 | |
| *** jedimike has joined #openstack-infra | 13:17 | |
| *** radez_g0n3 is now known as radez | 13:17 | |
| fungi | jroll: well, anyway, the capacity is a fixed amount... for the most part i'm looking right now to see what we're able to do within those confines. there are potentially other issues unrelated to capacity | 13:17 |
| *** ZZelle has quit IRC | 13:17 | |
| joa | jaypipes: rasselin, that was his github handle :) | 13:17 |
| jroll | right | 13:18 |
| jroll | fungi: but I don't see why the quota can't be bumped | 13:18 |
| jroll | but that's just me | 13:18 |
| fungi | jroll: the other issue described to us arising from instance churn is fixed ip assignment... apparently those can't be turned over instantaneously, so sometimes we have boot requests wait an hour for an ip address before nova reports them as complete | 13:18 |
| *** cpowell has joined #openstack-infra | 13:19 | |
| fungi | where "sometimes" is up to 50% of our boot requests at peak load in some regions | 13:19 |
| *** freyes has quit IRC | 13:19 | |
| jroll | fungi: yeah, letting arp clear | 13:19 |
| anteaya | joa: which I believe he is frustrated at maintaining, which is why he is working on helping infra implement some of our specs, like the puppet module split spec | 13:19 |
| *** dmsimard_away has quit IRC | 13:19 | |
| anteaya | since he realizes his time is better spent helping infra | 13:19 |
| jroll | fungi: thanks for the info :) | 13:19 |
| erlon_ | guys, we have requested our CI account to be enable but we got not response so far | 13:20 |
| fungi | jroll: you're welcome--and thanks for your support! | 13:20 |
| jroll | :) | 13:20 |
| *** jistr has quit IRC | 13:20 | |
| joa | anteaya: I think so, I'm not surprised ^^' | 13:21 |
| *** freyes has joined #openstack-infra | 13:21 | |
| erlon_ | Im looking at the topics in the meeting wiki but I dont know if a topic is needed for this | 13:21 |
| fungi | AJaeger: hashar: so i think the changes which have been waiting longest in the check pipeline are likely to be from one or more jobs aborting prematurely (something going wrong with the jenkins slave agent connection for example) which causes that job to get requeued and wait for a new worker to be assigned | 13:21 |
| joa | anteaya: which is why I made my propositions of having something more tied with infra for newcomers :) | 13:21 |
| jroll | fungi: fwiw, I'd put in a ticket to bump the quota and ask that it doesn't get lowered again, not sure if that will work but worth a try | 13:21 |
| fungi | jroll: oh, thanks! | 13:22 |
| *** erlon_ is now known as erlon | 13:22 | |
| *** bcourt has joined #openstack-infra | 13:22 | |
| anteaya | jroll: thank you | 13:22 |
| jroll | anteaya: of course :) | 13:22 |
| anteaya | erlon: we address requests for new accounts about once a week | 13:22 |
| *** jistr has joined #openstack-infra | 13:23 | |
| anteaya | sorry tab complete fail, I was addressing joa | 13:23 |
| anteaya | and thank you as well jroll, :D | 13:23 |
| anteaya | erlon: no topic is needed for third party account requests | 13:23 |
| *** nelsnels_ has quit IRC | 13:23 | |
| jroll | oh lol | 13:23 |
| anteaya | jroll: :D | 13:24 |
| AJaeger | jroll, thanks a lot! | 13:24 |
| jroll | hehehe | 13:24 |
| *** nelsnelson has joined #openstack-infra | 13:24 | |
| *** bswartz has quit IRC | 13:24 | |
| anteaya | erlon: now if you are requesting a ci account to test stackforge and the only information in the request is the name of the stackforge project, that is a problem | 13:24 |
| anteaya | and mordred is taking care of that problem | 13:24 |
| jedimike | quick question: is CI running on trusty, or is partly migrated to trusty, or is it all still running precise? I'm looking at how we can migrate gozer to trusty | 13:25 |
| erlon | anteaya: hmm, ok | 13:25 |
| anteaya | jedimike: the slaves are on trusty | 13:25 |
| anteaya | the infrastructure nodes are on precise | 13:25 |
| *** dustins has joined #openstack-infra | 13:26 | |
| *** jp_at_hp has quit IRC | 13:26 | |
| erlon | https://www.irccloud.com/pastebin/9jy406lz | 13:26 |
| jedimike | anteaya, is there any problem going to trusty? (I mean apart from finding the time to migrate!) | 13:26 |
| fungi | well, slaves are on a combination of trusty, precise, centos 6, centos 7, fedora 20... | 13:26 |
| *** dmsimard_away has joined #openstack-infra | 13:27 | |
| *** dmsimard_away is now known as dmsimard | 13:27 | |
| erlon | o_O | 13:27 |
| erlon | anteaya: its been a week so I assume you guys will enable it in this next days right? | 13:27 |
| erlon | anteaya: another thing, is there a way to disable the CI and keep the access to the sandbox? | 13:27 |
| fungi | jedimike: and *some* infrastructure nodes are running ubuntu trusty (our puppet 3 master in particular) | 13:28 |
| *** _nadya_ has joined #openstack-infra | 13:28 | |
| *** yamamoto has joined #openstack-infra | 13:28 | |
| *** bknudson has joined #openstack-infra | 13:29 | |
| openstackgerrit | Denis A. Kochetkov proposed a change to openstack-infra/jenkins-job-builder: Review issues fixed. https://review.openstack.org/129287 | 13:29 |
| AJaeger | anteaya, could you review https://review.openstack.org/129827 today, please? | 13:31 |
| anteaya | erlon: you posted a paste of questions to me | 13:33 |
| *** yamamoto has quit IRC | 13:33 | |
| anteaya | AJaeger: sure | 13:33 |
| *** 1JTAAQZNO has quit IRC | 13:33 | |
| *** hdd has quit IRC | 13:34 | |
| *** r-daneel has joined #openstack-infra | 13:34 | |
| erlon | anteaya: yep, disregard the paste, irccloud tricked me, I post the questions right after | 13:34 |
| AJaeger | anteaya, since the load is high, consider not approving it, just +2 if you agree - without it all translation proposal jobs to fail. We can wait another day with those if needed ;) | 13:35 |
| *** ZZelle has joined #openstack-infra | 13:35 | |
| AJaeger | anteaya, and I can approve once the queue is short enough... | 13:35 |
| anteaya | AJaeger: +# | 13:35 |
| *** pblaho has quit IRC | 13:35 | |
| anteaya | +3 | 13:35 |
| AJaeger | Ok, fine as well - thanks! | 13:35 |
| *** ZZelle has quit IRC | 13:35 | |
| anteaya | sorry missed your message as I was reviewing | 13:35 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Print context when key fails to be formatted https://review.openstack.org/129492 | 13:35 |
| anteaya | I only have one screen | 13:36 |
| AJaeger | ;) np, anteaya | 13:36 |
| anteaya | erlon: well we will enable it depending on the request | 13:36 |
| anteaya | but we will look at the requests again today | 13:36 |
| *** amotoki has joined #openstack-infra | 13:37 | |
| anteaya | erlon: and you set what repos you watch | 13:37 |
| erlon | anteaya: what do you mean? | 13:37 |
| *** pblaho has joined #openstack-infra | 13:37 | |
| anteaya | I mean we read the email to create the account | 13:37 |
| *** mriedem has joined #openstack-infra | 13:37 | |
| * AJaeger does some shopping now, will join the IRC meeting tonight... | 13:37 | |
| anteaya | we review the data, if we have enough and it matches what we need, we can create the account | 13:37 |
| anteaya | if not, we can't, if it isn't the name of a stackforge project, we email you with more infor | 13:38 |
| anteaya | if it is a stackforge project, mordred will take those | 13:38 |
| erlon | anteaya: hmm, sorry, let me explain, me and marcusvrn_ are in charge of HDS account, we where disabled because our CI wasn't posting logs, despite it wasn't voting | 13:40 |
| *** dkliban_biab is now known as dkliban | 13:40 | |
| anteaya | oh yeah okay | 13:40 |
| anteaya | well I talked to thingee about this | 13:40 |
| openstackgerrit | Ricardo Carrillo Cruz proposed a change to openstack-infra/system-config: Use absolute path in nodepool_template default value https://review.openstack.org/129907 | 13:40 |
| anteaya | since i am tired of being point person for all this stuff | 13:40 |
| *** masayukig has quit IRC | 13:40 | |
| anteaya | and the projects could take up some of the responsibility here | 13:41 |
| anteaya | basically you use your personal gerrit account to read the stream and trigger a few tests, configure jenkins to *not* report back | 13:41 |
| *** radez is now known as radez_g0n3 | 13:41 | |
| *** BharatK has quit IRC | 13:41 | |
| anteaya | post the urls for the tests triggered by the stream you read from your personal account and have some cinder folks look at it | 13:42 |
| *** rkukura has joined #openstack-infra | 13:42 | |
| anteaya | if they tell me it looks stable, I will request you be reenabled | 13:42 |
| anteaya | I'm tired of reanableing systems to have to turn around and disable them again | 13:42 |
| anteaya | so we need some information that can be confirmed | 13:43 |
| anteaya | and I would rather deal with a person from each project that 80+ operators | 13:43 |
| anteaya | all insisting that their system works _fine_ when it doesn't | 13:43 |
| erlon | anteaya: hmmm ok, but I still don't the reason why the sandbox shouldn't allowed for disable CIs | 13:43 |
| anteaya | well I can explain | 13:44 |
| erlon | anteaya: but is ok, we will test this and contact thingee | 13:44 |
| anteaya | because third party gerrit accounts have no gui | 13:44 |
| anteaya | no, talk to the folks in the cinder channel | 13:44 |
| anteaya | since i have no doubt that thingee will want someone else to do this | 13:44 |
| *** obondarev_ has joined #openstack-infra | 13:45 | |
| anteaya | and because third party gerrit accounts have no gui, they can only communicate via ssh | 13:45 |
| anteaya | including connecting to read a stream of events | 13:45 |
| fungi | well, and also we can control whether those accounts can leave verify votes on specific projects, but we can't limit their ability to comment to specific projects (we can only disable the account if it's leaving unhelpful comments on changes) | 13:45 |
| anteaya | so when they are disabled, the ssh connection can't be made | 13:45 |
| openstackgerrit | A change was merged to openstack-infra/project-config: Unbreak translation proposals https://review.openstack.org/129827 | 13:46 |
| *** masayukig has joined #openstack-infra | 13:46 | |
| fungi | i wonder if we should request that each project which has instituted third-party testing as a requirement for specific in-tree drivers/backends/features designate a liaison who interacts with the infra team to get accounts switched to voting, reenabled, whatever needs coordinating | 13:46 |
| fungi | might reduce a lot of the random finger-pointing which goes on in those situations | 13:47 |
| *** katyafervent_awa is now known as katyafervent | 13:47 | |
| *** emagana has joined #openstack-infra | 13:48 | |
| fungi | so either the ptl or a designated delegate the third-party testers for that project should be in touch with, who then relays the wishes of the project to us | 13:48 |
| erlon | fungi:anteaya: ok | 13:48 |
| anteaya | well right now we have mestery and emagana for neutron | 13:49 |
| emagana | anteaya: Hi There! | 13:49 |
| fungi | right, i think in practice it's happening that way in some places | 13:49 |
| *** sabeen1 has joined #openstack-infra | 13:49 | |
| fungi | and it seems like a successful enough model we might ask other projects who have similar needs to follow | 13:49 |
| anteaya | I have talked to thingee and suspect that some combination of him DuncanT- and jungle_boy_j will do it | 13:49 |
| anteaya | emagana: hello | 13:50 |
| *** freemanbrandon has joined #openstack-infra | 13:50 | |
| *** talluri has quit IRC | 13:50 | |
| anteaya | was just saying that working with you to confirm neutron ci's is working well for me | 13:50 |
| emagana | anteaya: Just letting you know that I am around.. well just now! | 13:50 |
| anteaya | and that having the same for cinder and nova would be good | 13:50 |
| anteaya | emagana: thank you! | 13:50 |
| anteaya | so cinder is in play | 13:50 |
| anteaya | and frankly nova accounts are few and responsible | 13:51 |
| fungi | because as it stands right now, a lot of the third-party testing operators keep getting the impression they're supposed to get permission from the infra team for this sort of stuff when in actuality they should be talking to those projects | 13:51 |
| anteaya | I had asked in the past for a contact and had no offers, but really I haven't had any problems from nova accounts | 13:51 |
| *** nelsnelson has quit IRC | 13:51 | |
| fungi | just trying to figure out how to better pre-empt that and get them directed to the right contacts up front | 13:51 |
| *** marun has joined #openstack-infra | 13:51 | |
| anteaya | and salv and bobball and jhesketh are all awesome to talk to | 13:51 |
| anteaya | fungi: I'm for that | 13:52 |
| *** nelsnelson has joined #openstack-infra | 13:52 | |
| * jhesketh is happy to help out where he can | 13:52 | |
| anteaya | but first I was working on getting the structure in place before advertising | 13:52 |
| fungi | right, i'm not saying we lack de-facto liaisons, but rather than those liaisons are not obvious to many of the third-party testers for those corresponding projects | 13:52 |
| anteaya | since all the projects move at their individual pace | 13:52 |
| anteaya | and infra's attempts to get them to move at teh same pace usually don't go well | 13:53 |
| anteaya | jhesketh: and you do a great job of it | 13:53 |
| anteaya | fungi: right | 13:53 |
| anteaya | and part of it has been working with them in a defacto way | 13:53 |
| anteaya | so I didn't scare them away from doing the work | 13:53 |
| anteaya | since pointing the hoards at them prematurely tends to do that | 13:54 |
| fungi | heh, yep. can totally see that | 13:54 |
| anteaya | so yeah, it is an awkward dance | 13:55 |
| anteaya | and I fall down a lot | 13:55 |
| anteaya | but I am still trying to dance as fast as I can | 13:55 |
| anteaya | to feed the constantly hungry | 13:55 |
| anteaya | if I move too fast, I end up alone again | 13:55 |
| anteaya | I have to move at the pace of the structure | 13:56 |
| anteaya | which does not suit the hoards | 13:56 |
| *** chandankumar has joined #openstack-infra | 13:56 | |
| *** stevemar has joined #openstack-infra | 13:57 | |
| *** wenlock has joined #openstack-infra | 13:57 | |
| anteaya | I guess I just have to tolerate the screaming a while longer | 13:57 |
| *** freemanbrandon has quit IRC | 13:57 | |
| anteaya | since some of the bigger pieces, slowly and of their own volition, are feeling like they are moving into a mutually beneifical place | 13:58 |
| fungi | well, the infrastructure team can, as a whole, start to be more insistent about it is what i'm saying. to try and avoid burning ourselves out. the projects are the ones who have asked their specialty developers to set up external testing and reporting systems. we maintain the means to make that possible, but really shouldn't be responsible for hand-holding them | 13:58 |
| *** bcourt has quit IRC | 13:58 | |
| anteaya | sure | 13:58 |
| anteaya | let's use the word supportive rather than insistent | 13:58 |
| anteaya | and I'm onboard | 13:59 |
| *** ayoung has joined #openstack-infra | 13:59 | |
| fungi | and it can be up to those projects whether they decide dealing with third-party systems is in general a success or too much effort. but just heaping the necessary effort onto the infrastructure team is counter-productive | 13:59 |
| anteaya | agreed | 13:59 |
| *** freemanbrandon has joined #openstack-infra | 13:59 | |
| anteaya | but keep in mind that we have just now turned a corner when all programs with this issue have had a release dealing with it first hand | 14:00 |
| anteaya | rather than in theory | 14:00 |
| fungi | yep, so it's an ideal time to start iterating toward getting them to take on more of the responsibility | 14:00 |
| anteaya | so they now have some experience to use to bring to decison making | 14:00 |
| anteaya | great | 14:00 |
| anteaya | I agree | 14:00 |
| *** gokrokve has joined #openstack-infra | 14:01 | |
| fungi | and allowing us to do what we do best (developing and maintaining good project infrastructure) | 14:01 |
| anteaya | oh utopia | 14:01 |
| *** signed8bit has joined #openstack-infra | 14:01 | |
| anteaya | sunshine and wheat fields | 14:01 |
| *** markmcclain has joined #openstack-infra | 14:02 | |
| *** yfried_ has joined #openstack-infra | 14:02 | |
| *** cody-somerville has quit IRC | 14:03 | |
| *** freemanbrandon has quit IRC | 14:04 | |
| *** todd_dsm has joined #openstack-infra | 14:04 | |
| *** freemanbrandon has joined #openstack-infra | 14:04 | |
| *** esker has joined #openstack-infra | 14:04 | |
| *** hdd has joined #openstack-infra | 14:04 | |
| *** mfer has joined #openstack-infra | 14:05 | |
| *** esker has quit IRC | 14:06 | |
| *** esker has joined #openstack-infra | 14:06 | |
| *** zz_dimtruck is now known as dimtruck | 14:06 | |
| anteaya | fungi: so how is zuul doing? | 14:08 |
| anteaya | the test node graph looks great to my eyes | 14:08 |
| anteaya | the zuul job queue has decreased its rapid rate of descent | 14:08 |
| anteaya | the fact that the test graph doesn't show the deleting and building nodes slowly coming together like teeth is awesome | 14:09 |
| fungi | anteaya: it's running at full steam. i'm pruning some dead wood from the providers (cleaning up stale nodes which haven't hit the 8-hour mark yet, getting rid of disassociated floating ips, deleting orphaned "alien" nodes, et cetera) | 14:10 |
| *** jerryz has quit IRC | 14:11 | |
| fungi | just general de-crufting and seeing what i can do to get a little more throughput | 14:11 |
| anteaya | cool | 14:11 |
| anteaya | throughput looks fairly stable though for this long and full capacity | 14:11 |
| *** salv-orlando has joined #openstack-infra | 14:11 | |
| anteaya | so yay | 14:11 |
| fungi | so far no signs there's anything unusual going on, aside from the obvious that we heaped 0.5k rechecks onto it yesterday | 14:11 |
| anteaya | for this long at full capacity | 14:12 |
| anteaya | this is awesome | 14:12 |
| anteaya | for y'know stabilty stress testing | 14:12 |
| fungi | and just now i saw four changes merge at the head of the gate pipeline, so it's not being hindered by the pending check volume | 14:13 |
| anteaya | yay | 14:13 |
| anteaya | 88 in post | 14:13 |
| *** chandankumar has quit IRC | 14:13 | |
| anteaya | top job has been there 19.5 hours | 14:14 |
| fungi | right, post pipeline is at low priority so right now it's not going to get workers until existing demand in the check pipeline has been satisfied | 14:14 |
| anteaya | docs and specs nodes shortage | 14:14 |
| anteaya | okay | 14:14 |
| anteaya | well rather than add, I think I will take my walk now, get it out of the way | 14:15 |
| *** pcrews has joined #openstack-infra | 14:15 | |
| fungi | yesterday we talked about switching post to normal priority same as check | 14:15 |
| anteaya | I have a few things I could do but I see a lull in the rain so going to take it | 14:15 |
| anteaya | fungi: what were the pros and cons? | 14:16 |
| *** cody-somerville has joined #openstack-infra | 14:16 | |
| *** dangers_away is now known as dangers | 14:16 | |
| fungi | we only have three priority levels, as defined in the gearman protocol specification, so any adjustments we make to move a pipeline to a higher priority potentially starves work running at lower priorities | 14:16 |
| *** bradjones has quit IRC | 14:16 | |
| anteaya | I think right now priority for post below check is fine | 14:18 |
| anteaya | the stuff has merged | 14:18 |
| *** dkranz has joined #openstack-infra | 14:18 | |
| anteaya | just might be the weekend before we see the docs | 14:18 |
| anteaya | I wonder if the only post job is docs | 14:18 |
| *** bcourt has joined #openstack-infra | 14:18 | |
| anteaya | if only the last patch to that repo needs to be in post | 14:19 |
| anteaya | if we have 88 in post and more to come, the docs jobs just trigger a run of the docs on the code that is there | 14:19 |
| anteaya | we only need one trigger per repo | 14:19 |
| anteaya | calculate on patches there for more than say 3 hours | 14:20 |
| anteaya | so when the post queue is short or rapidly cleared that algorithim isn't triggered | 14:21 |
| anteaya | just evaluated in situations like this | 14:21 |
| *** wenlock has quit IRC | 14:21 | |
| *** pblaho has quit IRC | 14:21 | |
| fungi | i think right now we don't have enough information on what might run in post to say for sure that we can expire long-waiting post jobs across the board. might make sense to have a separate sort of pipeline though for jobs which can cancel their predecessors in the pipeline or something | 14:21 |
| anteaya | cool | 14:23 |
| fungi | perhaps they're jobs tied to the branch tip state rather than a specific sha | 14:23 |
| anteaya | something to chew on | 14:23 |
| *** pblaho has joined #openstack-infra | 14:23 | |
| anteaya | interesting | 14:23 |
| fungi | so rather than cancelling their predecessors in the pipeline, they just don't get new instances added if there's already one pending | 14:23 |
| anteaya | yes and as long as "latest" is in the pipeline we get what we need, once the pipeline clears | 14:23 |
| *** ZZelle has joined #openstack-infra | 14:24 | |
| *** marun has quit IRC | 14:24 | |
| anteaya | that could work, yes | 14:24 |
| fungi | for example, the branch-tarball jobs might benefit from that. no point in rapidly replacing that tarball over and over when the most recent version will do fine | 14:24 |
| *** msabramo has quit IRC | 14:24 | |
| *** yaguang has joined #openstack-infra | 14:24 | |
| anteaya | as long as it clears sometime and we don't end up in a perpetual loop state | 14:24 |
| anteaya | right | 14:24 |
| *** hdd has quit IRC | 14:24 | |
| anteaya | then once it hits a certain length, say 15 | 14:24 |
| anteaya | it can toggle a different priority, clear that queue and then go back to accumulateing more | 14:25 |
| anteaya | can we toggle priority based on queue length of a pipeline? | 14:25 |
| fungi | yeah, that's why tying teh changeish identity in the pipeline to a branch name instead of a sha could be ideal there... no lost resources from repeatedly cancelling running jobs or resetting the timer for it in the fifo | 14:25 |
| anteaya | is that possible? | 14:25 |
| anteaya | right | 14:25 |
| jhesketh | right, blog posted: http://josh.people.rcbops.com/2014/10/openstack-infrastructure-swift-logs-and-performance/ | 14:26 |
| *** reed has joined #openstack-infra | 14:26 | |
| jhesketh | now I'm going to disappear for sleep before the meeting | 14:26 |
| fungi | hmm... we could in theory have zuul dynamically prioritize new work requests based on demand in a given pipeline, but i don't think we'd actually want that | 14:26 |
| *** pblaho has quit IRC | 14:26 | |
| *** dangers is now known as dangers_away | 14:26 | |
| *** armax has joined #openstack-infra | 14:26 | |
| *** pblaho has joined #openstack-infra | 14:27 | |
| anteaya | jhesketh: enjoy sleep | 14:27 |
| anteaya | jhesketh: thanks for the blog post | 14:27 |
| anteaya | fungi: oh okay | 14:27 |
| fungi | anyway, i'll whip up a feature request for the branch pipeline idea so we don't forget the discussion | 14:27 |
| anteaya | fungi: cool thanks | 14:27 |
| anteaya | I will disappear for a walk | 14:27 |
| anteaya | back later | 14:27 |
| *** mancdaz_away is now known as mancdaz | 14:28 | |
| *** yamamoto has joined #openstack-infra | 14:28 | |
| clarkb | dtroyer: jogo Juno to Master is not happy http://logs.openstack.org/74/128974/1/check/check-grenade-dsvm/6b7d6f3/logs/grenade.sh.txt.gz#_2014-10-21_07_14_28_805 | 14:28 |
| *** mpaolino has joined #openstack-infra | 14:28 | |
| clarkb | jhesketh: thank you for blog I am reading now | 14:28 |
| *** gokrokve has quit IRC | 14:28 | |
| *** bradjones has joined #openstack-infra | 14:29 | |
| *** gokrokve has joined #openstack-infra | 14:29 | |
| *** chandankumar has joined #openstack-infra | 14:29 | |
| *** atiwari has joined #openstack-infra | 14:29 | |
| *** jp_at_hp has joined #openstack-infra | 14:30 | |
| *** yaguang has quit IRC | 14:32 | |
| *** yamamoto has quit IRC | 14:32 | |
| *** yaguang has joined #openstack-infra | 14:33 | |
| krotscheck | Question: Is releasing javascript libraries to NPM (js equiv to pipi) something I need to talk to the TC about? | 14:33 |
| clarkb | krotscheck: I don't think so | 14:33 |
| *** davidhadas_ has joined #openstack-infra | 14:33 | |
| *** andreaf has quit IRC | 14:33 | |
| clarkb | we haven't asked their permission to release java in the past | 14:33 |
| fungi | krotscheck: i would say just figure out what the release upload job would look like, and get one of the infra cores info on how to set up the necessary credentials/namespace/whatever is needed | 14:34 |
| krotscheck | Oh good. | 14:34 |
| krotscheck | So, a release involves tagging the repository in a specific way. | 14:34 |
| clarkb | is it different than the current specific way we tag releases? | 14:35 |
| krotscheck | I think so. | 14:35 |
| fungi | oh, specially-formatted tags? | 14:35 |
| clarkb | ok that will probably require a npm-release pipeline | 14:35 |
| clarkb | also npm-- for making that a thng | 14:35 |
| krotscheck | But ultimately I think it’s just a string. | 14:35 |
| fungi | right | 14:35 |
| *** msabramo has joined #openstack-infra | 14:36 | |
| clarkb | mtreinish: ^ see above grenade log | 14:36 |
| *** mmaglana has quit IRC | 14:36 | |
| fungi | clarkb: well, we *do* have a "tag" pipeline which triggers on arbitrary tags, only used by the openstack-infra/publications project to do its conference links | 14:36 |
| clarkb | mtreinish: we seem to have broken juno to master | 14:36 |
| krotscheck | I _think_ they want a v as a prefix. | 14:36 |
| krotscheck | I’m going to dig a little more. | 14:36 |
| clarkb | fungi: oh ya we can probably overload that | 14:36 |
| jeblair | fungi, anteaya: ++ third-party ci liasons | 14:37 |
| *** bradjones has quit IRC | 14:37 | |
| krotscheck | But once that tag’s there, all we have to do is register the repo with npm and they’ll lazy-scrape the tags. | 14:37 |
| jeblair | fungi: also, ++ branch-tip pipeline | 14:37 |
| fungi | so in theory if the projects publishing to npm only use tags for that purpose, they could leverage that pipeline (as opposed to release and pre-release where we need to be able to differentiate tags on a pattern) | 14:37 |
| *** lttrl has quit IRC | 14:37 | |
| *** eharney has joined #openstack-infra | 14:38 | |
| *** salv-orlando has quit IRC | 14:38 | |
| *** ChuckC has joined #openstack-infra | 14:38 | |
| jogo | clarkb: looking | 14:39 |
| mtreinish | clarkb: oh, fun | 14:39 |
| mtreinish | clarkb: http://logs.openstack.org/74/128974/1/check/check-grenade-dsvm/6b7d6f3/logs/new/screen-n-api.txt.gz#_2014-10-21_07_12_30_741 | 14:39 |
| mtreinish | jogo: ^^^ | 14:39 |
| jogo | clarkb: guuuu | 14:39 |
| *** derekh has joined #openstack-infra | 14:39 | |
| jogo | just saw that | 14:39 |
| *** teran_ has quit IRC | 14:40 | |
| mtreinish | did we land a past change already? | 14:40 |
| jogo | middleware.request_d | 14:40 |
| mtreinish | *paste | 14:40 |
| *** dangers_away has quit IRC | 14:40 | |
| *** teran has joined #openstack-infra | 14:40 | |
| jogo | I didn't think so | 14:40 |
| jogo | clarkb: why are you up so early? | 14:40 |
| krotscheck | fungi: Awesome, I’ll peek into that. | 14:41 |
| fungi | now that he works for the foundation, we're teaching him what morning looks like ;) | 14:41 |
| *** marun has joined #openstack-infra | 14:41 | |
| *** ZZelle has quit IRC | 14:41 | |
| * krotscheck will probably do that when he’s ready to peel storyboard into libraries. | 14:42 | |
| jogo | mtreinish: I approved the patch :( | 14:42 |
| jogo | not sure why grenade is using the new api.paste file though | 14:42 |
| ashp | I have a horrible feeling all our jenkins jobs at work are useless because zuul has changed so much :/ | 14:42 |
| *** ZZelle has joined #openstack-infra | 14:42 | |
| *** mmaglana has joined #openstack-infra | 14:43 | |
| *** reed has quit IRC | 14:43 | |
| jeblair | huh, why do we run bandersnatch as root? | 14:43 |
| mtreinish | jogo: I don't think it is, it looks like it was an incubator thing that got deleted from the nova tree | 14:43 |
| mtreinish | I think... | 14:43 |
| *** reed has joined #openstack-infra | 14:43 | |
| clarkb | jogo taking larissa to work | 14:44 |
| *** lttrl has joined #openstack-infra | 14:44 | |
| *** zz_jgrimm is now known as jgrimm | 14:44 | |
| fungi | jeblair: good question. though since it's the only thing of relevance on the servers where it's running i guess that's not an actual security risk | 14:44 |
| clarkb | ashp the jobs shouldnt have changed any | 14:44 |
| fungi | jeblair: however, now afs gives us a good reason to change that | 14:44 |
| ashp | clarkb: are they still meant to send json back to the zuul server on port 8001? | 14:45 |
| ashp | clarkb: because that's what all of ours are doing. I've been experimenting with our replacement stuff and I can make a review, I see zuul pick it up, try to do something with it then jenkins never seems to get it | 14:45 |
| jogo | mtreinish: you are right 641de5666f38532252e91735a1cd2611d96c489a | 14:45 |
| fungi | ashp: how old is your zuul? | 14:46 |
| * jogo blames me | 14:46 | |
| ashp | fungi: our existing setup seems to be a couple of years old, I can find out exactly (maybe) | 14:46 |
| ashp | lets just say our prod jenkins wasn't using any gearman plugins | 14:46 |
| jogo | clarkb: do you have a bug open for this? | 14:47 |
| clarkb | jogo not yet and I need to drive yet | 14:47 |
| fungi | ashp: yeah, that's what i was wondering. so you're pre-gearman | 14:47 |
| jogo | clarkb: kk filing a bug and a revert patch | 14:47 |
| ashp | feb 2013 for zuul-server binary, so I'm guessing then | 14:47 |
| ashp | I'm trying to figure out how to take our gate jobs and convert them to gearman | 14:47 |
| jogo | and sorry for approving the bad patch :( | 14:47 |
| ashp | which I assume is what I need to do | 14:47 |
| clarkb | jogo boo :P is it on master? | 14:48 |
| fungi | ashp: well, as stated, the jobs themselves should be fine. i'm trying to remember what the pre-gearman api communication between jenkins and zuul looked like. i think there was an event stream zuul listened on? | 14:48 |
| fungi | jeblair: clarkb: ^ ? | 14:48 |
| mtreinish | jogo: sigh, yeah we'll need to revert that | 14:48 |
| clarkb | if so how is icehouse master working? | 14:48 |
| jogo | clarkb: very good question | 14:48 |
| ashp | I'm brand new to all these bits, from what I see there's a callback from the job when done to zuul:8001 but nothing seems to listen there | 14:48 |
| *** oramraz has quit IRC | 14:49 | |
| clarkb | ashp the callback is no longer used | 14:49 |
| mtreinish | jogo: it also should have been -1'd for not cleaning up openstack-common.conf, but that's something else :) | 14:49 |
| clarkb | replaced by gearman | 14:49 |
| mtreinish | clarkb: it doesn't use the request_id middleware | 14:49 |
| mtreinish | so there is no issue trying to load something that doesn't exist | 14:49 |
| clarkb | oh but juno does | 14:49 |
| jeblair | fungi, ashp: yeah, you should be able to drop the callback from all the jobs | 14:49 |
| mtreinish | clarkb: yep | 14:49 |
| fungi | oh, right, job completion triggered a callback in jenkins to the zuul http api | 14:49 |
| ashp | is there a place on github you keep your current jenkins job builder yaml files so I can look at the diffs to see what it "should" look like now | 14:49 |
| jeblair | fungi, ashp: install the gearman-plugin from jenkins, and configure it to talk to zuul's gearman server | 14:50 |
| ashp | https://github.com/openstack-infra/project-config/tree/master/jenkins/jobs i guess | 14:50 |
| ashp | jeblair: i did that, and I can test the connection works fine | 14:50 |
| fungi | ashp: yep | 14:50 |
| ashp | but it still seems like somehow the jobs don't get triggered | 14:50 |
| fungi | ashp: check the gearman status in zuul (connect to the socket and issue the "status" command) to see what jobs your jenkins master has registered workers for | 14:51 |
| ashp | http://pastebin.com/s37fw9Kp | 14:52 |
| ashp | not sure I understand the columns | 14:52 |
| ashp | but it seems like those are the jobs in jenkins | 14:52 |
| fungi | ashp: you may also need to update your layout.yaml if you haven't already | 14:52 |
| *** hashar has quit IRC | 14:53 | |
| ashp | i am mostly using the original stuff, so I'm sure layout.yaml is wrong at this point | 14:53 |
| *** wenlock has joined #openstack-infra | 14:53 | |
| fungi | ashp: also current zuul has very detailed debug logging | 14:53 |
| ashp | I bet this would have been easier to do from scratch :) | 14:53 |
| fungi | ashp: possibly | 14:53 |
| *** mrmartin has quit IRC | 14:53 | |
| ashp | http://pastebin.com/96E1i2SZ the weird thing is it looks like zuul is doing the right stuff | 14:54 |
| fungi | ashp: at least re-setting up zuul from scratch would likely have been easier. rebuilding jenkins and redefining all your jobs wouldn't buy you much | 14:54 |
| ashp | Yeah, I kept the layout.yaml as I didn't want to rebuild their "workflow" at the same time as migrating to modern versions, I was hoping to get to modern versions and -then- go back and redesign things | 14:54 |
| *** david-lyle_ has joined #openstack-infra | 14:54 | |
| *** stevemar has quit IRC | 14:55 | |
| *** stevemar has joined #openstack-infra | 14:56 | |
| jogo | clarkb mtreinish: this got through for a funny reason | 14:56 |
| jogo | request-id was turned into a middleware in Icehouse | 14:57 |
| jogo | so icehouse -> master didn't have the middleware at all | 14:57 |
| *** chianingwang has quit IRC | 14:57 | |
| *** koolhead17 has quit IRC | 14:58 | |
| *** otherwiseguy has joined #openstack-infra | 14:58 | |
| fungi | ashp: hrm... looks like you may not have any mergers running | 14:58 |
| jogo | the good news is this was only for V3 | 14:58 |
| fungi | ashp: newer versions of zuul have a worker daemon to process merge calculations | 14:59 |
| fungi | ashp: it's a separate initscript | 14:59 |
| ashp | Ahhhh, interesting, I definitely don't have any merger stuff running | 14:59 |
| ashp | Would the lack of that stop the jenkins gate checks from running? | 14:59 |
| *** eharney has quit IRC | 14:59 | |
| fungi | ashp: for details of the gearman admin status fields see http://gearman.org/protocol/ (administrative protocol section) | 14:59 |
| *** david-lyle_ has quit IRC | 15:00 | |
| fungi | ashp: you have a merger:merge with 2 pending requests, no workers available or running | 15:00 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: [WIP] Left Join bug when filtering for StorySummary Status https://review.openstack.org/129929 | 15:00 |
| *** stevemar has quit IRC | 15:01 | |
| *** obondarev_ has quit IRC | 15:01 | |
| *** stevemar has joined #openstack-infra | 15:01 | |
| *** eharney has joined #openstack-infra | 15:02 | |
| ashp | fungi: I think what I need to do is go back to basics, read about how it's "meant to work" in modern zuul/jenkins/gerrit, and start from there | 15:02 |
| ashp | as I'm just stumbling around in the dark trying to "patch things up" and it's not working especially great | 15:03 |
| *** dangers_away has joined #openstack-infra | 15:03 | |
| fungi | ashp: not a bad idea | 15:03 |
| ashp | Is there a single great resource I shoudl be reading or just the ci.openstack.org linked docs? | 15:03 |
| fungi | ashp: those, yep | 15:03 |
| fungi | ashp: there's a "running your own" guide there which is probably mostly up to date, though likely covers more stuff than you need | 15:04 |
| jogo | clarkb: is juno -> master running everywhere? | 15:04 |
| *** Sukhdev has quit IRC | 15:04 | |
| ashp | i'll need all those things eventually :) | 15:04 |
| jogo | and if not can I get it to run on my fix patch | 15:04 |
| *** marun has quit IRC | 15:05 | |
| *** dangers_away is now known as dangers | 15:05 | |
| jeblair | ashp: you should also reed the zuul docs | 15:05 |
| jeblair | ashp: http://ci.openstack.org/zuul/ explains mergers, etc | 15:05 |
| *** marun has joined #openstack-infra | 15:05 | |
| jogo | fungi: ^ | 15:05 |
| reed | when did I become a verb? :) | 15:05 |
| *** david-lyle has joined #openstack-infra | 15:06 | |
| jeblair | reed: you've been verbed! ;) | 15:06 |
| *** Sukhdev has joined #openstack-infra | 15:08 | |
| jeblair | clarkb, fungi, mordred: i'm running bandersnatch on afstest to get some data in there | 15:09 |
| *** thedodd has joined #openstack-infra | 15:09 | |
| *** lttrl has quit IRC | 15:09 | |
| jeblair | i have to run now, but we can do some more with that later | 15:10 |
| ashp | __requires__ = 'zuul==1.1.0.23.gf231fa2' - this'll explain why so much has changed | 15:12 |
| *** msabramo1 has joined #openstack-infra | 15:12 | |
| *** jistr has quit IRC | 15:14 | |
| *** _buzz has quit IRC | 15:14 | |
| *** msabramo has quit IRC | 15:16 | |
| *** jcoufal has quit IRC | 15:16 | |
| *** msabramo has joined #openstack-infra | 15:16 | |
| mordred | clarkb: I disagree with your npm-- ... we made releasing via formatted tags a thing, and I honestly wish more people would too | 15:18 |
| *** jedimike has quit IRC | 15:18 | |
| *** jedimike has joined #openstack-infra | 15:20 | |
| *** msabramo1 has quit IRC | 15:20 | |
| bo_sh | fungi: i get connection refused when running log-gearman-worker.py | 15:22 |
| *** radez_g0n3 is now known as radez | 15:22 | |
| bo_sh | clarkb: i get connection refused when running log-gearman-worker.py | 15:23 |
| fungi | bo_sh: connection refused trying to retrieve logs from your log publication webserver? | 15:23 |
| fungi | bo_sh: or connection refused truing to connect to the gearman server/. | 15:23 |
| fungi | ? | 15:23 |
| *** yaguang has quit IRC | 15:24 | |
| bo_sh | socket.connect(self.host, self.port)) at worker.py _connect_socket method | 15:24 |
| *** chandankumar has quit IRC | 15:24 | |
| *** jlibosva has quit IRC | 15:25 | |
| bo_sh | i guess it is during the gearman server | 15:25 |
| mordred | nibalizer: why does puppet crap out if "Universität" is in the file I'm trying to change? | 15:25 |
| mordred | nibalizer: Error: /Stage[main]/Openafs::Client/File[/etc/openafs/CellServDB]/content: change from {md5}456847e74ecd2a58120535bc52b49023 to {md5}0e828e6f71da39bf3a10f1f813eed4ba failed: invalid byte sequence in UTF-8 | 15:25 |
| jeblair | that is _seriously_ uncool | 15:26 |
| jeblair | it's not like we can ask the rest of the world to change for puppet | 15:26 |
| *** cody-somerville has quit IRC | 15:26 | |
| mordred | yah | 15:26 |
| mordred | jeblair: I'm investigating options that don't involve just editing the file by hand first | 15:26 |
| *** dprince has quit IRC | 15:27 | |
| *** teran has quit IRC | 15:27 | |
| *** dprince has joined #openstack-infra | 15:27 | |
| *** denis_makogon has quit IRC | 15:28 | |
| bo_sh | fungi: socket.connect(self.host, self.port)) at worker.py _connect_socket method... i guess it is during the gearman server | 15:28 |
| *** yamamoto has joined #openstack-infra | 15:28 | |
| jeblair | mordred: that _file_ predates utf8. :) | 15:28 |
| mordred | jeblair: the best part is - it seems to be related to templates ... but there is not an invalid utf-8 byte _in_ the template - I removed it in the puppet source and replaced it with ae | 15:28 |
| mordred | jeblair: so, for some reason, puppet cares about the content of the file it's _replacing_ being valid utf-8 | 15:29 |
| *** teran has joined #openstack-infra | 15:29 | |
| fungi | bo_sh: and you have a gearman service listening on the port you've configured to connect into? | 15:29 |
| mordred | *mind boggles* | 15:29 |
| jeblair | mordred: !! | 15:29 |
| jeblair | ok, gotta run, biab | 15:29 |
| mordred | jeblair: have fun | 15:29 |
| bo_sh | i have gearman client. | 15:30 |
| bo_sh | fungi: ^^^ | 15:30 |
| bo_sh | * a gearman | 15:30 |
| fungi | bo_sh: have you checked the server to see if the service is listening on that port? | 15:31 |
| *** chianingwang has joined #openstack-infra | 15:31 | |
| bo_sh | fungi: wait, i am pretty confused. i thought that the worker waits for the client to submit jobs. | 15:32 |
| *** mancdaz is now known as mancdaz_away | 15:32 | |
| *** yamamoto has quit IRC | 15:33 | |
| *** _buzz has joined #openstack-infra | 15:33 | |
| bo_sh | fungi: and it listens on port 4730 | 15:33 |
| mordred | bo_sh: the worker and the client both connect to the server | 15:34 |
| bo_sh | fungi: is there additional serivce/daemon that i have to run in addition to log-gearman-worker/client? | 15:34 |
| bo_sh | mordred: yes, i know. | 15:34 |
| bo_sh | mordred: hoo wait, sorry. what server? | 15:35 |
| mordred | the gearman-server | 15:35 |
| mordred | the worker connects to the server and tells it what jobs it can run | 15:35 |
| *** tonytan4ever has joined #openstack-infra | 15:35 | |
| mordred | the client asks the server to runa job, and one of the workers connected to the server picks it up | 15:35 |
| *** berendt has quit IRC | 15:36 | |
| *** mancdaz_away is now known as mancdaz | 15:36 | |
| *** yamamoto has joined #openstack-infra | 15:36 | |
| bo_sh | mordred: according to http://ci.openstack.org/logstash.html (system arch part) there is nothing about a gearman-server | 15:36 |
| bo_sh | it looks like gearman-client sends jobs to gearman-worker. so i don't follow what you're saying. | 15:37 |
| mordred | bo_sh: right - that doc is skipping gearman impl details and is talking about logically what happens | 15:37 |
| mordred | I can see how that would be confusing - sorry about that | 15:37 |
| bo_sh | mordred: eh!? do you have any doc about the impl details of the gearman. | 15:38 |
| *** alexpilotti has quit IRC | 15:38 | |
| *** koolhead17 has joined #openstack-infra | 15:38 | |
| fungi | the client appears to embed the "gear" gearman server http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/log_processor/files/log-gearman-client.py#n126 | 15:38 |
| mordred | ah! ok | 15:39 |
| bo_sh | fungi: ho ok. no problem. | 15:39 |
| mordred | cool. bo_sh - don't listen to me :) | 15:39 |
| fungi | so basically the client daemon provides a gearman server. the workers connect to that server, register themselves, and wait for work requests to appear | 15:39 |
| *** gyee has joined #openstack-infra | 15:39 | |
| ashp | after dumb question but.. with the zuul merger, where does it get the git repos from in the first place? I don't have a key or anything configured for it to pull right from github | 15:40 |
| ashp | does it fetch it from gerrit itself? | 15:40 |
| *** hashar has joined #openstack-infra | 15:40 | |
| fungi | (clarkb will correct me if i'm wrong about the log worker stuff) | 15:40 |
| *** chianingwang has quit IRC | 15:40 | |
| *** yamamoto has quit IRC | 15:41 | |
| ashp | hmm, it is meant to be from gerrit, I wonder what I've done wrong, it's not fetching anything | 15:41 |
| fungi | ashp: yes it will fetch refs from gerrit | 15:41 |
| *** primeministerp has joined #openstack-infra | 15:41 | |
| *** primemin1sterp has joined #openstack-infra | 15:41 | |
| ashp | I wonder if it needs additional rights in gerrit it doesn't have other than stream-events to do that | 15:41 |
| ashp | as all zuul-merger has done is make an empty dir so far :) | 15:41 |
| *** primemin2sterp has joined #openstack-infra | 15:41 | |
| *** groknix has joined #openstack-infra | 15:41 | |
| *** primemin2sterp has quit IRC | 15:42 | |
| mtreinish | fungi: well it's actually called a gear client, it still needs a gearman server running | 15:42 |
| *** primemin1sterp has quit IRC | 15:42 | |
| *** primeministerp has quit IRC | 15:42 | |
| *** jistr has joined #openstack-infra | 15:43 | |
| *** 14WAADG9C has joined #openstack-infra | 15:43 | |
| *** 7GHAAEFQK has joined #openstack-infra | 15:43 | |
| *** primeministerp has joined #openstack-infra | 15:43 | |
| mtreinish | fungi: the clients push the jobs on to the work queue and the workers pull them off. (or at least that's how it worked when I was experiementing with it for subunit2sql) | 15:43 |
| *** changbl has joined #openstack-infra | 15:43 | |
| ashp | 2014-10-21 15:36:52,901 INFO zuul.IndependentPipelineManager: Unable to merge change <Change 0x7fa82c14c810 3853,1> | 15:46 |
| ashp | Damn. :/ | 15:46 |
| fungi | mtreinish: in this case the log-gearman-client.py script instantiates the gear server class and provides the server | 15:46 |
| fungi | mtreinish: and then stuffs the work requests into it | 15:46 |
| *** otherwiseguy has quit IRC | 15:47 | |
| *** pradk has joined #openstack-infra | 15:47 | |
| *** IvanBerezovskiy has left #openstack-infra | 15:47 | |
| fungi | mtreinish: hence lsof shows that script is listening on 4730/tcp | 15:48 |
| *** Sukhdev has quit IRC | 15:48 | |
| *** otherwiseguy has joined #openstack-infra | 15:48 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/system-config: Run puppet even if git servers fail https://review.openstack.org/129722 | 15:51 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/system-config: Put playbooks in a directory https://review.openstack.org/129956 | 15:51 |
| mtreinish | fungi: hmm, I don't remember that when I was reading the script, just that it initialized a client and connected to the server (which I think was hardcoded as localhost) | 15:52 |
| mtreinish | but whatever, I'm more inclined to believe you | 15:52 |
| *** gokrokve has quit IRC | 15:52 | |
| clarkb | fungi that sounds right to me | 15:52 |
| mtreinish | fungi: oh, nm me I see that now I just missed it before :) | 15:55 |
| bo_sh | clarkb: fungi: looking at the logs it seems it says "Waiting for change in available connections to poll" and then it exits. any suggestions? | 15:55 |
| *** pblaho has quit IRC | 15:55 | |
| *** emagana has quit IRC | 15:55 | |
| *** emagana has joined #openstack-infra | 15:56 | |
| *** ilyashakhat has quit IRC | 15:57 | |
| fungi | bo_sh: is it successfully connecting to the zeromq publisher plugin on your jenkins master? | 15:58 |
| marcusvrn_ | Hi all! how can I configure jenkins to not report back the results? | 15:58 |
| clarkb | I am not familiar with that error bo_sh | 15:59 |
| jesusaurus | clarkb: i finally got my test to pass on 74943, you mind adding that to your queue? | 16:00 |
| *** emagana has quit IRC | 16:00 | |
| bo_sh | fungi: the client logs doesn't mention anything about successfully connecting to the zmq publisher on my jenkins. | 16:01 |
| fungi | marcusvrn_: it depends on what you're using to connect to gerrit... gerrit-trigger plugin for jenkins? zuul server? | 16:01 |
| *** alexpilotti has joined #openstack-infra | 16:01 | |
| bo_sh | fungi i can try and run the job that sends zmq, though | 16:01 |
| marcusvrn_ | fungi: zuul server | 16:01 |
| *** ildikov has quit IRC | 16:02 | |
| *** zaphod42 has joined #openstack-infra | 16:02 | |
| fungi | marcusvrn_: look at how we configure our "silent" pipeline: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n110 | 16:02 |
| *** yamahata has quit IRC | 16:03 | |
| *** markmcclain has quit IRC | 16:03 | |
| bo_sh | fungi: i executed the job, but the logs don't say anything about receiving an event... | 16:03 |
| bo_sh | strange. | 16:03 |
| fungi | bo_sh: well, first try manually connecting from the machine where you're running the log-gearman-client daemon to the zeromq port on the jenkins master | 16:03 |
| *** emagana has joined #openstack-infra | 16:03 | |
| fungi | bo_sh: just to rule out connectivity issues between them, or the possibility that the zeromq event publisher isn't active or something | 16:04 |
| *** alexpilotti_ has joined #openstack-infra | 16:04 | |
| mordred | clarkb: also, 129956 and 129722 would be nice if you're reviewing things :) | 16:06 |
| *** alexpilotti has quit IRC | 16:07 | |
| *** alexpilotti_ is now known as alexpilotti | 16:07 | |
| *** MarkAtwood has joined #openstack-infra | 16:07 | |
| bo_sh | fungi: okay. it was my bad. i didn't give the correct publisher. now it works (the client gets the zmq) | 16:08 |
| bo_sh | fungi: however, the worker still fails "Errno 111 connection refused" | 16:08 |
| fungi | bo_sh: does it say in the log what it's trying to do when it raises that exception? trying to connect to the gearman server? | 16:09 |
| *** MarkAtwood has quit IRC | 16:10 | |
| bo_sh | yes, it happens when trying to connect to the gearman-host:gearman-port | 16:10 |
| fungi | bo_sh: if so, confirm the machine where the worker daemon is running can reach the gearman server port on the machine where the client is running | 16:10 |
| bo_sh | ho wait | 16:10 |
| *** cody-somerville has joined #openstack-infra | 16:10 | |
| *** MarkAtwood has joined #openstack-infra | 16:10 | |
| *** VijayTripathi has joined #openstack-infra | 16:10 | |
| ashp | Someone must know - does the zuul-merger require an initial checked out copy of the repo to operate on? | 16:10 |
| ashp | Because it seems like it only gets some json over ssh, and doesn't know how to check out the repo at all from gerrit | 16:11 |
| bo_sh | fungi: both the client and the worker are on the same host. in the worker.yaml i have configured gearman-host: localhost and gearman-port: 4730 | 16:11 |
| *** jlibosva has joined #openstack-infra | 16:11 | |
| marcusvrn_ | fungi: looking at the link that you shared, the difference between silent and check pipeline is that the first one doesn't have success and failure sections, does it? | 16:11 |
| fungi | ashp: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/merger/merger.py#n42 | 16:12 |
| ashp | I wonder why that fails then :/ | 16:12 |
| zaro | morning | 16:12 |
| ashp | I have no match for "Cloning from" either in the logs | 16:12 |
| ashp | so it's not even trying | 16:13 |
| fungi | bo_sh: and you see :::4730 or 127.0.0.1:4730 listening according to netstat? | 16:13 |
| *** afazekas has quit IRC | 16:13 | |
| fungi | marcusvrn_: correct, if there is no success or failure action then zuul will not report to the change for those states | 16:13 |
| zaro | clarkb: anytime you are ready to do the GP upgrade. | 16:14 |
| marcusvrn_ | fungi: ok! thanks for your help!! :) | 16:14 |
| *** vdo has quit IRC | 16:14 | |
| fungi | bo_sh: or 0.0.0.0:4730 maybe | 16:14 |
| bo_sh | fungi: netstat -ntlp gives 0.0.0.0:4730 | 16:15 |
| bo_sh | in the local address | 16:15 |
| fungi | bo_sh: try 'echo status|nc -q 3 localhost 4730' | 16:15 |
| bo_sh | push-log 43 0 0 | 16:16 |
| bo_sh | , | 16:16 |
| bo_sh | fungi: ^^^^ | 16:16 |
| *** msabramo has quit IRC | 16:17 | |
| clarkb | zaro I did 06 and 07 already | 16:17 |
| fungi | bo_sh: okay, so that confirms that the gearman service is active there, and has pending work requests, but no registered workers | 16:17 |
| clarkb | will do more when check queue clears | 16:17 |
| bo_sh | fungi: i will try to figure this out. i will let you know if i need any further help. thanks | 16:17 |
| *** bo_sh has quit IRC | 16:18 | |
| fungi | bo_sh: it's possible that for some reason the worker is having trouble resolving the name "localhost" | 16:18 |
| fungi | oh poo, he left | 16:18 |
| *** ihrachyshka has quit IRC | 16:18 | |
| ashp | oh for fu... it looks like zuul-merger fails as it sits around trying to authenticate an ssh key | 16:19 |
| *** isviridov is now known as isviridov_away | 16:19 | |
| fungi | ashp: you may need to manually install the gerrit service's public host key into the known hosts for your zuul merger | 16:20 |
| *** signed8bit is now known as signed8bit_ZZZzz | 16:20 | |
| fungi | ashp: or manually connect to gerrit's ssh api from that account and accept the server's public key | 16:20 |
| ashp | yeah, going to add that now | 16:20 |
| fungi | ashp: we use puppet to install an entry in ~zuul/.ssh/known_hosts | 16:20 |
| nibalizer | jeblair: mordred ohstupid puppet | 16:22 |
| *** msabramo has joined #openstack-infra | 16:22 | |
| ashp | that definitely seemed happier | 16:22 |
| ashp | nibalizer: "stupid puppet", words I live by ;) | 16:22 |
| mordred | nibalizer: YOU FIX NOW!!!! | 16:23 |
| *** mancdaz is now known as mancdaz_away | 16:23 | |
| mordred | nibalizer: but seriously - why is it even reading the target to-be-replaced file? is it because I was running puppet agent --test and it needed to show me diffs? | 16:24 |
| zaro | clarkb: ohh cool, did you install thru the jenkins plugin download page (from jenkins repo)? | 16:24 |
| *** rmcall has joined #openstack-infra | 16:24 | |
| yolanda | jedimike, i'm running the tests in my environment to check if works | 16:25 |
| *** mpaolino has quit IRC | 16:25 | |
| jedimike | yolanda, come join us in #storyboard :) | 16:25 |
| *** gokrokve has joined #openstack-infra | 16:25 | |
| *** teran has quit IRC | 16:28 | |
| *** jp_at_hp has quit IRC | 16:28 | |
| *** yamamoto has joined #openstack-infra | 16:28 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/jenkins-job-builder: add documentation for the delete command https://review.openstack.org/128105 | 16:29 |
| *** cdent has quit IRC | 16:29 | |
| *** AJaeger_ has joined #openstack-infra | 16:31 | |
| clarkb | zaro no I did it on disk | 16:31 |
| *** thedodd has quit IRC | 16:31 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/jenkins-job-builder: add documentation for globbed parameter feature https://review.openstack.org/128476 | 16:32 |
| *** thedodd has joined #openstack-infra | 16:32 | |
| *** yamamoto has quit IRC | 16:33 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/jenkins-job-builder: add a JJB command reference https://review.openstack.org/128106 | 16:33 |
| *** packet has joined #openstack-infra | 16:34 | |
| *** signed8bit_ZZZzz has quit IRC | 16:34 | |
| *** packet has quit IRC | 16:34 | |
| *** packet has joined #openstack-infra | 16:34 | |
| nibalizer | mordred: well so | 16:34 |
| nibalizer | i mean that ticket is ancient | 16:34 |
| nibalizer | but they claimed to fix it | 16:34 |
| nibalizer | and poking it on my local system it looks like its fixed | 16:34 |
| nibalizer | but im running 3.4 | 16:34 |
| nibalizer | ill go to 3.7 and see whats up | 16:34 |
| nibalizer | if they regressed that im gonna lol | 16:35 |
| mordred | nibalizer: I can send you the file that it doesn't want to overwrite if you like | 16:35 |
| *** mpaolino has joined #openstack-infra | 16:35 | |
| nibalizer | would you mind terribly finding the file thats being replaced so I don't have to install the openafs packages ? | 16:35 |
| nibalizer | mordred: thats exactly what i want | 16:35 |
| mordred | nibalizer: sent to your hp email | 16:36 |
| *** teran has joined #openstack-infra | 16:37 | |
| nibalizer | mordred: woot | 16:38 |
| nibalizer | also from the 'pointless and cool' desk comes this http://4.bp.blogspot.com/-GxaD0K8bvTg/VEX4t6MrgII/AAAAAAAAATQ/tjxuf83us10/s1600/diagram_grhapite.png | 16:38 |
| jogo | mordred: how do I tell pbr to not build man pages | 16:38 |
| mordred | jogo: uhm. | 16:38 |
| jogo | the governance repo doesn't need a man page | 16:38 |
| mordred | jogo: oh - you mean the build_sphinx? | 16:38 |
| jogo | yeah | 16:39 |
| jeblair | jogo: we TOTALLY need to be able to do "man openstack-tc" | 16:39 |
| zaro | clarkb: ohh opps, i forgot to create a tag. so it's not on the jenkins repo anyways. will do it now. | 16:39 |
| jeblair | frankly, i would like to read that man page ;) | 16:39 |
| mordred | jogo: looking - honestly not sure that's an option - is it causing a problem? | 16:40 |
| jogo | mordred: not a problem per se, but I want to enable error on warning | 16:40 |
| *** otherwiseguy has quit IRC | 16:41 | |
| mordred | jogo: if self.builder == 'man' and len(sphinx_config.man_pages) == 0: | 16:41 |
| mordred | return | 16:41 |
| nibalizer | im actually with jeblair, more man pages | 16:42 |
| jogo | mordred, ARNING: no "man_pages" config value found; no manual pages will be written | 16:42 |
| *** teran has quit IRC | 16:42 | |
| mordred | jogo: so ... pbr should not be actually running sphinx if there are no manpages to build | 16:43 |
| krotscheck | clarkb: So there’s this -> https://review.openstack.org/#/c/128736/ | 16:43 |
| mordred | jogo: the bug is in the repo | 16:43 |
| mordred | jogo: the doc/source/conf.py in governance defines a manpage | 16:43 |
| krotscheck | Which explicitly has a “Hey maybe clark should weigh back in on this” comment | 16:43 |
| fungi | nibalizer: where did you get the diagram tool? | 16:43 |
| *** arxcruz has quit IRC | 16:44 | |
| jogo | mojjjj:q | 16:44 |
| jogo | doh | 16:44 |
| clarkb | krotscheck Im good with it | 16:44 |
| * jogo blames the crumby HP trackpad | 16:44 | |
| mordred | jogo: it's a terrible object | 16:44 |
| krotscheck | clarkb: Cool, can you +3 it? | 16:44 |
| clarkb | krotscheck ya let me check the one tyop thing before I do | 16:45 |
| jogo | mordred: un wedge the governance repo: https://review.openstack.org/129973 | 16:45 |
| *** dimsum_ is now known as dims | 16:45 | |
| mordred | jogo: I can't do anything ... ttx ^^^ please land | 16:46 |
| nibalizer | fungi: so that led to much rage, pip install diagram is on the readme, but it wasn't working last night | 16:46 |
| zaro | pelix: +2 from you on these doc changes and i'm gonna approve. https://review.openstack.org/#/c/128106 and https://review.openstack.org/#/c/128476 | 16:47 |
| *** jgallard has quit IRC | 16:47 | |
| jogo | mordred: also governance doesn't have pbr as a dep | 16:47 |
| *** mpaolino has quit IRC | 16:47 | |
| * AJaeger_ ha sput the python 2.6 deprecation on today's infra meeting agenda - I'm confused on how to move forward with the followup patches.. | 16:47 | |
| *** ihrachyshka has joined #openstack-infra | 16:47 | |
| fungi | nibalizer: interesting | 16:47 |
| *** derekh has quit IRC | 16:47 | |
| nibalizer | fungi: okay you can pip install it now | 16:47 |
| nibalizer | maybe the mirror my machine was hitting hadn't synced | 16:47 |
| jeblair | jogo: fyi https://review.openstack.org/#/c/125782/1 | 16:47 |
| fungi | nibalizer: i can't wait. https://pypi.python.org/pypi/diagram looks neat | 16:47 |
| nibalizer | here is teh sauce https://github.com/tehmaze/diagram | 16:48 |
| jogo | jeblair: cool, just ran a recheck we will see | 16:48 |
| *** dteselkin_ has joined #openstack-infra | 16:49 | |
| *** sdake_ has quit IRC | 16:50 | |
| *** otherwiseguy has joined #openstack-infra | 16:50 | |
| ttx | jogo: unwedged | 16:50 |
| mordred | jogo: it doesn't need one | 16:53 |
| pelix | zaro: believe https://review.openstack.org/#/c/128106 needs to be rebased | 16:54 |
| mordred | jogo: pbr in requirements is only needed if there are runtime deps caused by importing pbr in the code for thigns like versions | 16:54 |
| jogo | mordred: and the oslosphinx stuff already has it in there I guess | 16:54 |
| jogo | ttx: now to wait until this get through the check queue | 16:54 |
| jogo | of 600 | 16:55 |
| *** talluri has joined #openstack-infra | 16:56 | |
| clarkb | jogo: where is that revert for the grenade thing? | 16:57 |
| *** davidhadas_ has quit IRC | 16:57 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/system-config: Add kerberos and afs servers to cacti https://review.openstack.org/129977 | 16:57 |
| clarkb | jogo: also how did you figure out it was a paste thing? | 16:57 |
| nibalizer | mordred: so.... http://paste.openstack.org/show/122779/ is my attempt to repro | 16:57 |
| *** sarob has joined #openstack-infra | 16:57 | |
| mtreinish | clarkb: https://review.openstack.org/129926 | 16:57 |
| *** mwagner_lap has quit IRC | 16:58 | |
| mordred | nibalizer: FASCINATING | 16:58 |
| *** tonytan4ever has quit IRC | 16:58 | |
| *** mrmartin has joined #openstack-infra | 16:59 | |
| *** ivar-lazzaro has joined #openstack-infra | 16:59 | |
| nibalizer | mordred: is that functionaly different than what you've been doing? it doesn't look different to me | 16:59 |
| clarkb | mtreinish: thank you | 16:59 |
| nibalizer | crinkle: ashp ^^ thoughts? | 16:59 |
| mordred | nibalizer: nope. it looks the same - the only difference I can see is apply vs. agent? | 16:59 |
| *** dizquierdo has quit IRC | 17:00 | |
| *** dizquierdo has joined #openstack-infra | 17:01 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Remove openstack-common.conf https://review.openstack.org/129585 | 17:01 |
| *** lttrl has joined #openstack-infra | 17:01 | |
| mordred | nibalizer: WOW. I cannot reproduce it now - I just tried putting the content back and now it's happily replacing it | 17:01 |
| *** dane_leblanc has quit IRC | 17:01 | |
| *** packet has quit IRC | 17:01 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/jenkins-job-builder: add a JJB command reference https://review.openstack.org/128106 | 17:02 |
| *** _nadya_ has quit IRC | 17:02 | |
| mordred | nibalizer: oh. ok. | 17:02 |
| mordred | nibalizer: so - if I directly copy the file from my box into place, it breaks | 17:02 |
| mordred | nibalizer: if I use vim to cut and paste the line, it does not | 17:02 |
| zaro | pelix: ok, it's rebased. | 17:02 |
| mordred | nibalizer: (like, if I scp the file) | 17:02 |
| nibalizer | wowwwwww | 17:03 |
| jeblair | mordred: so maybe you're not replacing exactly the same data with vim | 17:03 |
| jogo | clarkb: knowing the code base | 17:03 |
| fungi | mordred: yes, vim is detecting the encoding and then you're cutting to your buffer from your terminal encoding rather than the file encoding | 17:03 |
| jeblair | mordred: you might be encoding it in utf8 by doing that | 17:03 |
| nibalizer | maybe vim is helping with some cleanup or something | 17:03 |
| jogo | and the request_id thing | 17:03 |
| jeblair | fungi: what fungi said :) | 17:03 |
| mordred | jeblair: yah | 17:03 |
| jogo | clarkb: https://review.openstack.org/#/c/129926/ | 17:03 |
| jeblair | mordred: what fungi said | 17:03 |
| jogo | clarkb: stuck in the check queue of doom | 17:03 |
| nibalizer | ya fungi used words that sound smart | 17:03 |
| mordred | so I'm wondering if emailing the file to nibalizer had the same results | 17:03 |
| jeblair | mordred: possibly. may want to tar it. | 17:03 |
| * fungi is good at sounding smart while having no clue what he's actually saying | 17:03 | |
| anteaya | ah I keep trying to figure out what it means | 17:04 |
| nibalizer | mordred: i mean you introduced outlook into the equation when you emailed it to me | 17:04 |
| fungi | and yes, entirely possible that the file is getting converted if it's a text/plain attachment | 17:04 |
| nibalizer | so all bets are off | 17:04 |
| anteaya | like a philosophical puzzle | 17:04 |
| mordred | nibalizer: sent tarred | 17:05 |
| mordred | nibalizer: it's the diff | 17:06 |
| mordred | jeblair: | 17:06 |
| *** Ryan_Lane has joined #openstack-infra | 17:06 | |
| mordred | if I run puppet agent --onetime --no-daemonize --verbose --no-usecacheonfailure --ignorecache --no-splay | 17:06 |
| mordred | it does not complain | 17:06 |
| jogo | clarkb: so it will be a day before my patch can be approved | 17:07 |
| nibalizer | so the act of generating the diff casuses puppet to think harder about the situation and crap itself | 17:07 |
| mordred | SO - it will not be a problem for us in our normal runs because we don't use test | 17:07 |
| jogo | clarkb: both governance and nova | 17:07 |
| clarkb | jogo: well it just got approved | 17:07 |
| mordred | nibalizer: yes | 17:07 |
| nibalizer | HAHAHAHAHAHA | 17:07 |
| clarkb | jogo: I am tempted to promote it | 17:07 |
| jogo | clarkb: no objection from me | 17:07 |
| clarkb | mordred: fungi jeblair any thoughts on that? its not gate breaking in that the gate doesn't work | 17:07 |
| jeblair | clarkb: what patch? | 17:07 |
| clarkb | but it is gate breaking in that juno to master grenade does not work | 17:07 |
| *** zigo has quit IRC | 17:08 | |
| clarkb | jeblair: https://review.openstack.org/129926 | 17:08 |
| nibalizer | mordred: well here is the bug if you want to rage at them https://tickets.puppetlabs.com/browse/PUP-1038 | 17:08 |
| clarkb | I am fine if you want ot hold off on promoting that (I can go either way) | 17:08 |
| *** zigo has joined #openstack-infra | 17:08 | |
| *** teran has joined #openstack-infra | 17:08 | |
| *** dane_leblanc has joined #openstack-infra | 17:08 | |
| *** hashar has quit IRC | 17:08 | |
| jeblair | clarkb: if juno->master grenade does not work, why is the gate not broken? | 17:09 |
| fungi | clarkb: so that was blocking us switching master to upgrade from juno instead of icehouse? | 17:09 |
| *** teran_ has joined #openstack-infra | 17:09 | |
| clarkb | jeblair: because icehouse -> master is what we are testing now until my d-g changes get in | 17:09 |
| clarkb | and icehouse -> master does work | 17:09 |
| clarkb | fungi: yup | 17:09 |
| clarkb | jeblair: there is apparently a corner case where previous branch to master works but current stable to master cannot | 17:09 |
| *** koolhead_ has joined #openstack-infra | 17:09 | |
| jeblair | clarkb: seems sufficiently time-sensitive and gate-breaking-related to promote to me. | 17:09 |
| fungi | i can see an argument that delaying correcting the grenade test to be testing what we want leaves us exposed | 17:10 |
| *** luqas has quit IRC | 17:10 | |
| clarkb | fungi: ya | 17:10 |
| fungi | so wfm | 17:10 |
| clarkb | that was my angle to nova devs reviewing :) | 17:10 |
| jeblair | then let's be consistent :) | 17:10 |
| clarkb | ok I will enqueue to gate and promote when I get a chance | 17:10 |
| jogo | clarkb: yeah getting grenade to juno would be nice | 17:10 |
| jogo | clarkb: thanks | 17:10 |
| *** hdd has joined #openstack-infra | 17:10 | |
| clarkb | still catching up on things post breakfast | 17:10 |
| fungi | anybody have good suggestions for where we want to host the devstack.org -> docs.o.o/developer/devstack redirect? | 17:11 |
| annegentle | fungi: you can do it in the openstack-manuals/www/static/.htaccess | 17:11 |
| annegentle | I think I'd prefer it | 17:12 |
| *** teran_ has quit IRC | 17:12 | |
| fungi | annegentle: okay, great! | 17:12 |
| *** jpich has quit IRC | 17:12 | |
| fungi | annegentle: i'll offer a patch in moments | 17:12 |
| annegentle | fungi: cool | 17:12 |
| *** teran has quit IRC | 17:13 | |
| *** koolhead17 has quit IRC | 17:13 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/system-config: Add detailed-exitcodes to launch_node https://review.openstack.org/129983 | 17:13 |
| *** signed8bit has joined #openstack-infra | 17:13 | |
| fungi | where moments is defined as slightly longer than it takes me to clone openstack-manuals at ~100mb/sec | 17:13 |
| fungi | go hamsters, go! | 17:14 |
| mordred | jeblair: ^^ that should fix the thing you had a problem with when launching that node | 17:14 |
| jeblair | mordred: 'vos release' requires superuser access | 17:14 |
| jeblair | mordred: thank you | 17:14 |
| mordred | jeblair: oh. hrm | 17:14 |
| *** signed8bit has quit IRC | 17:14 | |
| mordred | jeblair: well that's disappointing | 17:14 |
| *** rmcall has quit IRC | 17:14 | |
| jeblair | mordred: so we may need some extra machinery around that, assuming we don't want to give bandersnatch superuser | 17:15 |
| mordred | yah | 17:15 |
| *** signed8bit has joined #openstack-infra | 17:15 | |
| fungi | some sort of signaling event | 17:15 |
| *** harlowja_away is now known as harlowja | 17:15 | |
| mordred | a releaser worker or something that takes vos release requests | 17:15 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: add documentation for the delete command https://review.openstack.org/128105 | 17:15 |
| *** mrmartin has quit IRC | 17:15 | |
| fungi | and then a listener which just does that as a superuser, yeah | 17:15 |
| harlowja | is the gate on fire? | 17:16 |
| harlowja | or zuul on fire :-P | 17:16 |
| fungi | harlowja: i think it's not ablaze so much as steadily smoldering | 17:16 |
| harlowja | :) | 17:16 |
| harlowja | its getting barbecued | 17:16 |
| harlowja | nice | 17:16 |
| jeblair | should be out by tomorrow | 17:16 |
| mordred | mmm. barbecue | 17:16 |
| *** mfink_ has joined #openstack-infra | 17:17 | |
| *** HeOS has quit IRC | 17:17 | |
| harlowja | ha | 17:17 |
| *** rmcall has joined #openstack-infra | 17:17 | |
| fungi | slow-roasted check jobs | 17:17 |
| fungi | you can taste the anticipation baked right in | 17:17 |
| *** markmcclain has joined #openstack-infra | 17:18 | |
| *** dteselkin__ has joined #openstack-infra | 17:18 | |
| jogo | docs look nice: http://docs-draft.openstack.org/73/129973/1/check/gate-governance-docs/5410d4c/doc/build/html/ | 17:18 |
| mordred | also, I'm in memphis today, so I think that's my cue to go find some bbq | 17:18 |
| fungi | mordred: seriously, what are you doing talking to us. go get bbq asap | 17:19 |
| jeblair | and bring some to the infra meeting | 17:19 |
| *** mfink has quit IRC | 17:19 | |
| fungi | in memphis, there's always enough bbq to share | 17:19 |
| clarkb | actually gate looks mostly healthy I think an enqueue is all that I need not a promote | 17:19 |
| clarkb | so I will start wit that | 17:19 |
| jeblair | clarkb: yeah, probably won't be too long | 17:20 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: add documentation for globbed parameter feature https://review.openstack.org/128476 | 17:20 |
| *** dteselkin_ has quit IRC | 17:21 | |
| *** thedodd has quit IRC | 17:21 | |
| *** katyafervent is now known as katyafervent_awa | 17:21 | |
| *** chandankumar has joined #openstack-infra | 17:21 | |
| fungi | annegentle: looking over the .htaccess idea... i'm not sure this is a use case we can address there. isn't cloudsites going to have a problem with the request being for a name-based virtual host it doesn't know about? also what ip address would i put in dns for the devstack.org address records in that event? | 17:22 |
| clarkb | jeblair: for the --trigger arg to zuul enqueue I want 'gerrit' ? | 17:23 |
| *** radez is now known as radez_g0n3 | 17:24 | |
| *** dizquierdo has quit IRC | 17:24 | |
| jogo | clarkb: what happened to multi-node gate jobs? | 17:24 |
| fungi | jogo: afazekas's change to implement them is still wip i think | 17:24 |
| clarkb | ya I think there was some waffling on whether or not we need to use private addrs | 17:24 |
| jeblair | clarkb: i think? check root's history for confirmation :) | 17:25 |
| clarkb | and things went quiet | 17:25 |
| fungi | clarkb: sudo zuul enqueue --trigger gerrit --project openstack/tempest --pipeline gate --change 123510,1 | 17:25 |
| *** patrickeast has joined #openstack-infra | 17:25 | |
| fungi | (or whatever parameters you need for your change) | 17:25 |
| jogo | fungi: too bad, this is something that would be nice to ahve | 17:26 |
| fungi | jogo: no argument there | 17:26 |
| clarkb | fungi: jeblair ty | 17:26 |
| clarkb | it is enqueued now | 17:26 |
| *** radez_g0n3 is now known as radez | 17:26 | |
| clarkb | jogo: if you want to pick it up go for it | 17:27 |
| clarkb | I honestly thinks its 99% there and we don't need the additionaly features | 17:27 |
| *** ryanpetrello has quit IRC | 17:27 | |
| anteaya | jogo: and we do need the capacity to do multi-node testing | 17:27 |
| *** amcrn has joined #openstack-infra | 17:27 | |
| *** cpowell has quit IRC | 17:27 | |
| jogo | anteaya: hrm yeah | 17:28 |
| *** yamamoto has joined #openstack-infra | 17:28 | |
| *** ZZelle_ has joined #openstack-infra | 17:28 | |
| bknudson | have you ever seen this oddity before? https://review.openstack.org/#/c/125923/ -- if you expand patch set 4 the author section is after the files? | 17:28 |
| jogo | clarkb: at this point we should pick this up at the summit I think | 17:29 |
| *** tonytan4ever has joined #openstack-infra | 17:29 | |
| anteaya | bknudson: when I expand patchset 4 the author section for patchset 4 is before the patchset 4 files | 17:30 |
| fungi | same here | 17:30 |
| *** whoops has joined #openstack-infra | 17:30 | |
| jeblair | jogo: why? afaik it needs doing, not talking :) | 17:30 |
| anteaya | bknudson: and the author sections for patchset 5 is below that | 17:30 |
| bknudson | anteaya: ok, weird... I'll try restarting my browser. | 17:30 |
| anteaya | followed by the files for patchset 5 | 17:30 |
| *** markmcclain1 has joined #openstack-infra | 17:31 | |
| bknudson | probably the poodle attack. | 17:31 |
| *** markmcclain has quit IRC | 17:31 | |
| anteaya | jeblair jogo more doing, less meetings | 17:31 |
| *** ryanpetrello has joined #openstack-infra | 17:31 | |
| *** chandankumar has quit IRC | 17:31 | |
| jogo | jeblair: so we could move forward even if we don't get more capacity? | 17:32 |
| *** yamamoto has quit IRC | 17:33 | |
| *** markmcclain1 has quit IRC | 17:33 | |
| *** markmcclain has joined #openstack-infra | 17:33 | |
| jeblair | jogo: i believe anteaya was suggesting that as a project we need the *ability* to do multi node testing and therefore continuing work is important. not that we require more test node quota before we can implement it. | 17:33 |
| jeblair | anteaya: correct me if i'm wrong :) | 17:33 |
| *** tonytan4ever has quit IRC | 17:34 | |
| anteaya | continuing work is important yes | 17:34 |
| *** markmcclain1 has joined #openstack-infra | 17:34 | |
| *** markmcclain has quit IRC | 17:34 | |
| *** thedodd has joined #openstack-infra | 17:35 | |
| anteaya | heck if we have the abilty to do mulit-node tests serially that would work, meaning one patch at a time, with each patch in the multi-node test pipeline waiting for the patch preceding to finish | 17:35 |
| fungi | i definitely don't see our worker capacity as a blocker to implementing a multi-node devstack job. it might be something we don't run a lot of initially due to node capacity concerns, but deciding that before we even try it seems premature | 17:35 |
| *** dane_leblanc has quit IRC | 17:35 | |
| anteaya | and by multinode we could start with two nodes | 17:35 |
| anteaya | that is all dvr needs to begin with | 17:35 |
| jeblair | yep. i'm pretty sure that was just a misunderstanding. if that's the case, then we are all in agreement and we do not need a summit session to determine that. :) | 17:36 |
| anteaya | jogo: point me to a patch to review | 17:37 |
| anteaya | jogo: and I'll round up some friends to join me | 17:37 |
| anteaya | :D | 17:37 |
| jogo | ahh works for me | 17:39 |
| fungi | so, assuming the limitations for .htaccess-based redirects will prevent us from implementing it on docs.openstack.org, is static.o.o an okay place for us to stick a hollow devstack.org vhost to do the redirecting to the docs site, or is there a better place (or do we need to make a new place for that sort of thing)? | 17:39 |
| jogo | anteaya: not sure where the multi node patch | 17:39 |
| *** MaxV has quit IRC | 17:39 | |
| jogo | is | 17:39 |
| anteaya | jogo: yay | 17:39 |
| anteaya | jogo: me either | 17:39 |
| anteaya | let's find it | 17:39 |
| jeblair | fungi: i like static for that | 17:39 |
| clarkb | fungi: I haven't followed everything. Why can't docs.o.o htaccess do it? | 17:40 |
| *** davidhadas_ has joined #openstack-infra | 17:40 | |
| fungi | clarkb: because the initial request is coming to a different http/1.1 name-based virtual host | 17:40 |
| clarkb | oh right | 17:40 |
| fungi | clarkb: i _believe_ .htaccess-based redirects don't handle that use case at all | 17:40 |
| jogo | anteaya: https://review.openstack.org/#/q/status:open++branch:master+topic:mnode-poc,n,z | 17:40 |
| clarkb | fungi: I am slightly worried it will get lost on static.o.o But it should be fine there | 17:41 |
| fungi | at least i can't find an explanation of how to do it | 17:41 |
| clarkb | fungi: ya I think you are correct | 17:41 |
| *** shashankhegde has joined #openstack-infra | 17:41 | |
| anteaya | jogo: nice | 17:41 |
| clarkb | so we seem to have jobs restarting in zuul | 17:41 |
| fungi | and also, i think cloudsites will just barf if a client connects to its http socket and passes a hostname header for a site it knows nothing about | 17:41 |
| clarkb | in theory because jenkins is unhappy with the number of concurrent jobs? | 17:41 |
| dhellmann | anteaya: as an election official, would you like to comment on the governance change to set the new TC membership? https://review.openstack.org/#/c/129274/ | 17:41 |
| *** zigo has quit IRC | 17:41 | |
| clarkb | I am going to try and track one down | 17:41 |
| *** amotoki has quit IRC | 17:41 | |
| *** zigo has joined #openstack-infra | 17:41 | |
| * anteaya as an election official, puts down her soup and reviews | 17:42 | |
| fungi | or else we end up setting up a devstack.org cloudsites site to host the redirect to docs.o.o | 17:42 |
| clarkb | (I just noticed the gate-nova-pep8 job for the nova revert change went back to queued) | 17:42 |
| *** mwagner_lap has joined #openstack-infra | 17:42 | |
| clarkb | trying to find where it ran now | 17:42 |
| fungi | at least i get the impression cloudsites hosts sites for multiple vhosts on a given server and therefore its default vhost is probably a bitbucket or error page | 17:43 |
| fungi | testing that now | 17:43 |
| *** jistr has quit IRC | 17:43 | |
| fungi | yep, it's a page which just says "Please come back later." | 17:44 |
| clarkb | hrm no record of it after a quick look across the jenkinses | 17:44 |
| clarkb | oh wait there it is | 17:44 |
| clarkb | zaro: jeblair https://jenkins06.openstack.org/job/gate-nova-pep8/7517/console ran and succeeded | 17:44 |
| clarkb | but zuul is rerunning that job | 17:44 |
| clarkb | now going to look at zuul logs | 17:45 |
| clarkb | but the gearman-plugin may be broken... | 17:45 |
| anteaya | dhellmann: thank you | 17:45 |
| jeblair | clarkb: do you suspect this is happening at a greater frequency on the hosts with the upgraded gearman plugin? | 17:45 |
| dhellmann | anteaya: next cycle we should remember to ask you to submit the patch :-) | 17:45 |
| clarkb | jeblair: not yet | 17:45 |
| anteaya | dhellmann: :D | 17:45 |
| anteaya | dhellmann: ttx is just so efficient | 17:46 |
| clarkb | jeblair: I only have one data point so trying to figure out why that happened | 17:46 |
| dhellmann | anteaya: indeed | 17:46 |
| jeblair | clarkb: i have noticed a lot of jobs that look like they have been restarted. | 17:46 |
| clarkb | jeblair: 2014-10-21 17:34:56,130 INFO zuul.Gearman: Build <gear.Job 0x7f8f3caa3450 handle: H:127.0.0.1:615646 name: build:gate-nova-pep8:bare-trusty unique: 891401e56d2d49a793d8412b1a37b3bd> complete, result None | 17:47 |
| clarkb | jeblair: I think result: None is what is breaking us | 17:47 |
| clarkb | but I don't think I can easily go from that back to which jenkins server ran the job | 17:47 |
| *** spzala has quit IRC | 17:48 | |
| jeblair | clarkb: grep 891401e56d2d49a793d8412b1a37b3bd | 17:48 |
| *** emagana has quit IRC | 17:48 | |
| clarkb | ~3k occurences of result none | 17:48 |
| *** jlibosva has quit IRC | 17:48 | |
| *** emagana has joined #openstack-infra | 17:49 | |
| *** david-lyle has quit IRC | 17:49 | |
| anteaya | that is a whole lot of none | 17:49 |
| jeblair | clarkb: let's pull out 20 of them at random and see what hosts they are on? | 17:49 |
| *** VijayTripathi has quit IRC | 17:49 | |
| *** mfer has quit IRC | 17:49 | |
| clarkb | jeblair: ok | 17:49 |
| *** david-lyle has joined #openstack-infra | 17:49 | |
| jeblair | clarkb: you want to do that or shall i? | 17:49 |
| *** dane_leblanc has joined #openstack-infra | 17:50 | |
| *** jedimike has quit IRC | 17:51 | |
| *** mrmartin has joined #openstack-infra | 17:51 | |
| clarkb | jeblair: I am working on it | 17:51 |
| clarkb | I just did grep 'result None' | tail | 17:52 |
| *** berendt has joined #openstack-infra | 17:52 | |
| clarkb | and am semi manually getting uuids from that and grepping | 17:52 |
| clarkb | done ~6 and they are all on 07 or 06 so far | 17:52 |
| *** mfer has joined #openstack-infra | 17:52 | |
| *** ivar-lazzaro has quit IRC | 17:52 | |
| fungi | that doesn't bode well for the plugin upgrade :/ | 17:53 |
| *** teran has joined #openstack-infra | 17:53 | |
| *** emagana has quit IRC | 17:53 | |
| *** ivar-lazzaro has joined #openstack-infra | 17:53 | |
| clarkb | yup did 10 | 17:53 |
| clarkb | I think we need to revert | 17:53 |
| clarkb | zaro: ^ | 17:53 |
| fungi | well, at least that should help our throughput a bit | 17:54 |
| marcusvrn_ | asselin: anteaya: Hi! I have changed the CI to use my SSH key and account...to run jobs in "silent" mode in order to generate a log to re-enable the CI, but it's not triggering the jobs....zuul-merger returns "Permission denied (publickey)." | 17:54 |
| *** emagana has joined #openstack-infra | 17:54 | |
| clarkb | fungi: jeblair: I can do that if you agree that is the next step we should take | 17:54 |
| fungi | clarkb: i think this is probably not a great time to try to debug it in production | 17:54 |
| jeblair | clarkb: yes | 17:55 |
| clarkb | ok starting that now | 17:55 |
| jeblair | (just offlining 06 and 07 will start to help our throughput | 17:55 |
| clarkb | yup | 17:55 |
| asselin | marcusvrn_, something probably didn't update correctly since you changed your key. | 17:55 |
| *** ivar-lazzaro has quit IRC | 17:56 | |
| *** jistr has joined #openstack-infra | 17:56 | |
| *** ivar-lazzaro has joined #openstack-infra | 17:56 | |
| anteaya | marcusvrn_: you can't change your ci account to use your ssh key | 17:57 |
| clarkb | zaro: jeblair: I think it is likely that gearman worker is shutdown before reporting the success back | 17:57 |
| clarkb | zaro: jeblair but after the job finishes | 17:57 |
| clarkb | also it doesn't seem to be related to job runtime so maybe a network packet race? | 17:58 |
| asselin | marcusvrn_, double-check /var/lib/zuul/ssh/id_rsa | 17:58 |
| *** bdpayne has joined #openstack-infra | 17:58 | |
| clarkb | it doesn't seem to occur on all jobs either best I can tell | 17:58 |
| marcusvrn_ | anteaya: I change my ci account to my personal account | 17:58 |
| anteaya | marcusvrn_: what happens when you open a terminal and run `ssh -p 29418 yourgerritusername@review.openstack.org gerrit stream-events` | 17:58 |
| clarkb | both 06 and 07 are in shutdown mode | 17:58 |
| *** spzala has joined #openstack-infra | 17:58 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Don't show ER patches under open reviews https://review.openstack.org/125117 | 17:59 |
| asselin | marcusvrn_, you may need to completly stop and restart zuul as well.... | 17:59 |
| fungi | marcusvrn_: you probably need to make sure you've reconfigured zuul to use the username of your normal user, as well as the ssh key of your normal user | 17:59 |
| zaro | clarkb: ok. i'm gonna test the response from the server. | 17:59 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 18:00 | |
| *** sarob has quit IRC | 18:00 | |
| *** sarob has joined #openstack-infra | 18:01 | |
| *** sarob has quit IRC | 18:01 | |
| asselin | marcusvrn_, yes, as fungi said: https://github.com/rasselin/os-ext-testing-data/blob/master/vars.sh.sample#L7 | 18:02 |
| *** dane_leblanc has quit IRC | 18:02 | |
| marcusvrn_ | asselin: fungi anteaya yes, I did it...maybe I forgot to change the user/ssh-key in somewhere else...I'll check again! | 18:04 |
| anteaya | I did it | 18:04 |
| *** VijayTripathi has joined #openstack-infra | 18:04 | |
| anteaya | does that mean you have achieved your goal? | 18:04 |
| *** _nadya_ has joined #openstack-infra | 18:04 | |
| anteaya | and your zuul can hear the stream? | 18:05 |
| fungi | the "i'll check again" makes me think it did not help | 18:05 |
| anteaya | yes | 18:06 |
| anteaya | two small pieces of information that feel contradictory | 18:06 |
| *** arnaud__ has joined #openstack-infra | 18:07 | |
| *** freemanb_ has joined #openstack-infra | 18:07 | |
| marcusvrn_ | anteaya: fungi lol, sry! let me explain..if I run `ssh -p 29418 yourgerritusername@review.openstack.org gerrit stream-events` in my personal machine I can hear the stream | 18:07 |
| *** e0ne has quit IRC | 18:07 | |
| *** _buzz has quit IRC | 18:08 | |
| marcusvrn_ | but when I configure my personal-gerrit-user and personal-ssh-key in the machine that zuul is running...It doesn't work | 18:08 |
| *** _buzz has joined #openstack-infra | 18:09 | |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack-infra/jenkins-job-builder: Support additional events for HipChat plugin https://review.openstack.org/129760 | 18:09 |
| fungi | marcusvrn_: in this case adjusting your gerrit user in /etc/zuul/zuul.conf and replacing the content of /var/lib/zuul/ssh/id_rsa (and then possibly restarting zuul) is all you should need to do | 18:09 |
| *** hashar has joined #openstack-infra | 18:09 | |
| *** mfink__ has joined #openstack-infra | 18:09 | |
| fungi | marcusvrn_: also make sure you don't change the ownership/permissions on /var/lib/zuul/ssh/id_rsa when replacing it | 18:10 |
| *** sdake_ has joined #openstack-infra | 18:10 | |
| *** teran has quit IRC | 18:10 | |
| *** talluri has quit IRC | 18:10 | |
| *** freemanbrandon has quit IRC | 18:11 | |
| marcusvrn_ | fungi: yes, let me do it and see what it returns :) | 18:11 |
| fungi | marcusvrn_: and backing up /var/lib/zuul/ssh/id_rsa so you can restore it to normal later when your account is reactivated is a good idea too | 18:13 |
| *** tonytan4ever has joined #openstack-infra | 18:13 | |
| asselin | marcusvrn_, btw, this is the name of the private key that get's put in /var/lib/zuul/ssh/id_rsa https://github.com/rasselin/os-ext-testing-data/blob/master/vars.sh.sample#L9 | 18:13 |
| *** mfink_ has quit IRC | 18:13 | |
| *** radez is now known as radez_g0n3 | 18:14 | |
| *** radez_g0n3 is now known as radez | 18:15 | |
| *** rmcall has quit IRC | 18:16 | |
| *** dane_leblanc has joined #openstack-infra | 18:16 | |
| zaro | clarkb: i'm getting valid responses from gearman server. | 18:16 |
| krotscheck | clarkb: Ever get around to checking that typo? https://review.openstack.org/#/c/128736/ | 18:16 |
| clarkb | zaro: the responses are valid which is why zuul doesn't say LOST | 18:17 |
| clarkb | zaro: but the responses have no result. the result is None which means go rerun the job | 18:17 |
| clarkb | zaro: also I do not think it happens 100% of the time. I believe this is a race | 18:17 |
| *** mfink_ has joined #openstack-infra | 18:18 | |
| zaro | clarkb: the response i get looks like this: ['{"manager":"10-0-0-8.openstacklocal","name":"guava","worker":"10-0-0-8.openstacklocal_exec-0","number":24,"url":"http://18.125.190.245:8080/job/guava/24/"}', '{"result":"SUCCESS","manager":"10-0-0-8.openstacklocal","name":"guava","worker":"10-0-0-8.openstacklocal_exec-0","number":24,"url":"http://18.125.190.245:8080/job/guava/24/"}'] | 18:18 |
| zaro | clarkb: isn't the result in that response valid? | 18:19 |
| clarkb | zaro: 2014-10-21 18:18:02,682 INFO zuul.Gearman: Build <gear.Job 0x7f8ef5d09910 handle: H:127.0.0.1:491581 name: build:check-tempest-dsvm-neutron-full:devstack-trusty unique: 6ac4fcf9a70f49eaa0981db15cce05bc> complete, result None | 18:19 |
| clarkb | that is what I get | 18:19 |
| *** mfer has quit IRC | 18:19 | |
| zaro | clarkb: i can't seem to repro that in my env. | 18:19 |
| *** mfink__ has quit IRC | 18:20 | |
| clarkb | zaro: there is no json blob like the above | 18:20 |
| hashar | good evening | 18:20 |
| clarkb | yes I think it is a race... | 18:20 |
| zaro | clarkb: ohh i didn't set node to offline though. let me give that a try | 18:20 |
| *** mfink has joined #openstack-infra | 18:20 | |
| *** rmcall has joined #openstack-infra | 18:20 | |
| anteaya | hashar: good evening | 18:21 |
| hashar | Seems you guys Zuul instance has some trouble triggering jobs .. Turns out I have more or less the same issue on my instance from time to time | 18:22 |
| hashar | but have yet to figure it out :/ | 18:22 |
| clarkb | hashar: the gearman-plugin is broken I think | 18:22 |
| clarkb | it doesn't handle offlined nodes properly | 18:23 |
| hashar | oh you upgraded it ? | 18:23 |
| hashar | :-(((( | 18:23 |
| clarkb | yes upgraded on two jenkins | 18:23 |
| *** thedodd has quit IRC | 18:23 | |
| *** mfink_ has quit IRC | 18:23 | |
| asselin | fungi, marcusvrn_ I just ran into this issue. I need to double-check still, but need to step out. So just FYI for now in case you have it too: http://paste.openstack.org/show/122799/ | 18:23 |
| clarkb | and am working on downgrading them now | 18:23 |
| fungi | asselin: i believe there is a fix proposed for that already | 18:23 |
| hashar | though I am pretty sure I manually tested that whenever a node is removed the associated functions are unregrestired | 18:23 |
| clarkb | hashar: that isn't the issue | 18:24 |
| clarkb | hashar: the problem is that jenkins doesn't report a result back to zuul | 18:24 |
| clarkb | so zuul reruns the job | 18:24 |
| zaro | clarkb: ok. i am able to repro. the result is missing when you pass in "OFFLINE_NODE_WHEN_COMPLETE":"false" | 18:25 |
| zaro | clarkb: sorry i meant "OFFLINE_NODE_WHEN_COMPLETE":"true" | 18:25 |
| hashar | on my setup the slaves are permanent and I don't use the magic var to set a node down | 18:25 |
| zaro | clarkb: event must be firing on that offline thing before job completes. | 18:26 |
| *** Sukhdev has joined #openstack-infra | 18:26 | |
| *** baoli has quit IRC | 18:26 | |
| clarkb | zaro: I thought we tested this... | 18:26 |
| zaro | clarkb: i must have only tested the false case. will fix up the code. | 18:28 |
| *** yamamoto has joined #openstack-infra | 18:28 | |
| *** pelix has quit IRC | 18:28 | |
| *** Hal has joined #openstack-infra | 18:30 | |
| mtreinish | clarkb: any chance that I can cajole you into reviewing https://review.openstack.org/108003 this week? | 18:30 |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack-infra/jenkins-job-builder: Support additional events for HipChat plugin https://review.openstack.org/129760 | 18:30 |
| *** Hal is now known as Guest53198 | 18:30 | |
| *** rmcall has quit IRC | 18:30 | |
| clarkb | maybe? currently fighting jenkins and grenade | 18:30 |
| *** jistr has quit IRC | 18:31 | |
| mtreinish | clarkb: cool, I can probably help with that (if there's something that needs help) | 18:31 |
| clarkb | mtreinish: mostly just wrangling software at this poitn thanks though | 18:31 |
| clarkb | I think I have it under control | 18:32 |
| *** shashankhegde has quit IRC | 18:32 | |
| *** rmcall has joined #openstack-infra | 18:32 | |
| clarkb | (jenkins should be happier soonish then grenade stuff and the check queue can clear) | 18:32 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/jenkins-job-builder: add a JJB command reference https://review.openstack.org/128106 | 18:32 |
| *** yamamoto has quit IRC | 18:33 | |
| dhellmann | fungi: I'm seeing those markupsafe unicode errors in some more doc builds. Before I spend a bunch of time debugging, is that something you're aware of anyone actively working on? | 18:35 |
| fungi | dhellmann: nope--i'd not seen it before you pointed it out, and haven't looked into it | 18:35 |
| dhellmann | fungi: ok, thanks | 18:35 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Plugins may now register their own user preferences https://review.openstack.org/129061 | 18:37 |
| *** jamielennox has quit IRC | 18:38 | |
| marcusvrn_ | asselin: fungi anteaya It's working now!! The problem was that sandbox repository has already been cloned...so the remote url of the repository was configured to the ci account...Then, when zuul was trying to update the repo already cloned, it returned "Permission Denied (public key)" because it was using ci account instead of my personal account ¬¬' | 18:38 |
| *** cpowell has joined #openstack-infra | 18:38 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Plugins may now register cron workers. https://review.openstack.org/129609 | 18:38 |
| fungi | marcusvrn_: oh, good point... i didn't think to suggest checking for usernames in existing git remotes | 18:39 |
| *** teran has joined #openstack-infra | 18:39 | |
| *** Guest53198 has quit IRC | 18:42 | |
| *** radez is now known as radez_g0n3 | 18:43 | |
| * JayF asks for some kind core on devstack-gate to help us get https://review.openstack.org/#/c/129653/ the agent_ssh nv job for Ironic working | 18:43 | |
| anteaya | marcusvrn_: good work | 18:44 |
| jeblair | clarkb or fungi if you have a sec? ^ | 18:44 |
| anteaya | fungi: I almost wonder if the process that marcusvrn_ has gone through would be useful to demonstrating that potential ci op is worthy of an account | 18:45 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/system-config: Host a devstack.org redirect on static https://review.openstack.org/130001 | 18:45 |
| fungi | JayF: looking | 18:45 |
| mtreinish | JayF: why don't you just do that in devstack? | 18:45 |
| anteaya | so they get the ci working first and the account comes last | 18:45 |
| clarkb | jeblair storyboard changes? | 18:46 |
| *** freemanbrandon has joined #openstack-infra | 18:46 | |
| jeblair | clarkb: devstack-gate ironic | 18:46 |
| clarkb | oh looking | 18:47 |
| JayF | mtreinish: jeblair initially asked the same question; I didn't know or have a good answer then. My only real comment is that the only thing in Devstack at all ATM that needs SWIFT_TEMPURL working is the Ironic agent driver | 18:47 |
| clarkb | we decided to not devstack that? | 18:48 |
| JayF | jroll: ^ | 18:48 |
| JayF | I just know we're running a check now, that fails everywhere because of that one variable, and I wanted to get it working | 18:48 |
| fungi | JayF: however you can switch around the same "$DEVSTACK_GATE_IRONIC_DRIVER" == "agent_ssh" test in devstack to set those things... but i agree it loks like much of this is already being set in d-g instead | 18:48 |
| JayF | that's my statement and my goal :( | 18:48 |
| jeblair | yeah, someday i'm going to get a bee in a bonnet about that whole section, but so far, the ironic folks have said that devstack-gate is doing something really weird that no one would ever want to do outside of the gate. | 18:48 |
| mtreinish | JayF: ok, it just feels like something that devstack should just default and allow to be overwritten if people need it. | 18:48 |
| jeblair | frankly, i don't believe them. | 18:48 |
| jeblair | and i agree with mtreinish and clarkb and fungi. :) | 18:48 |
| JayF | jeblair: I believe us | 18:49 |
| JayF | jeblair: that ssh driver we use in the gate is *awful* | 18:49 |
| jroll | JayF: jeblair and I decided that we should move the agent configs to devstack at some point | 18:49 |
| *** freemanb_ has quit IRC | 18:49 | |
| JayF | jeblair: and nobody should ever use it at all ever outside of the gate | 18:49 |
| jeblair | JayF: then we should not use it in the gate | 18:49 |
| JayF | jeblair: it's being worked on; someone is making an ipmi proxy so we can use "real" drivers in the gate | 18:49 |
| jroll | jeblair: if we don't use it in the gate, we need to put hardware in the gate | 18:49 |
| jroll | but yeah being worked on | 18:49 |
| fungi | anyway, for now it's not adding any additional burden to stick yet one more line in that section, so i'm +2 on it as a stop-gap | 18:50 |
| jeblair | however, we apparently missed the opportunity to reject that section a while ago, so i don't want to hold up this fix | 18:50 |
| fungi | unless anyone else wants to object, i'll approve | 18:50 |
| jeblair | jroll: i think developers can and should be able to work on and test this code without real hardware | 18:50 |
| clarkb | fungi no its fine I just didnt see an answer to the earlier concerns | 18:50 |
| jeblair | devstack should help them set up that kind of environment | 18:50 |
| clarkb | and jeblairs middle road compromise wfm | 18:51 |
| *** teran has quit IRC | 18:51 | |
| jroll | jeblair: right, so by "never use this outside of the gate" we actually mean "never use this outside of dev/test" | 18:51 |
| *** freemanbrandon has quit IRC | 18:51 | |
| jeblair | jroll: yep. and that's what the "dev" in "devstack" stand for :) | 18:51 |
| jeblair | jroll: i'm absolutely not arguing that it's production ready | 18:51 |
| fungi | it's for gettin devs all up in your stack | 18:51 |
| jeblair | jroll: i'm just saying that the development and test environment should be managed by devstack | 18:51 |
| mtreinish | fungi: I'm fine with it. I'm just not sure why the key needs to be set in d-g. Why doesn't devstack just default it to something? | 18:51 |
| JayF | jeblair: clarkb: fungi: So your concern is basically that as a dev, I should be able to flip a single switch and get a devstack that "works" for IPA? | 18:51 |
| dhellmann | fungi: it looks like the problem has to do with the encoding of the templates; I'm testing to make sure they are coming from the right package | 18:51 |
| jeblair | JayF: yep, like everything else in devstack | 18:52 |
| fungi | JayF: more or less, yes | 18:52 |
| JayF | jeblair: clarkb fungi: and with needing to set ~10 vars in ds-g, it means the devs have to set most of those vars too (in theory) | 18:52 |
| jeblair | JayF: exactly. way too much work | 18:52 |
| jeblair | too much magic in the wrong place | 18:52 |
| jroll | jeblair: to be clear, some of those things in that block are tempest settings, so we'll probably still need that block long-term. it should just be smaller. | 18:52 |
| fungi | dhellmann: encoding as in the text encoding, or as in the way the templates are coded? | 18:52 |
| *** oramraz has joined #openstack-infra | 18:52 | |
| *** jedimike has joined #openstack-infra | 18:53 | |
| jroll | jeblair: actually, exactly half of them are gate-related | 18:53 |
| jeblair | jroll: yep. we need to tell devstack how to set it self up, but we shouldn't need to tell the things that devstack sets up how to set themselves up :) | 18:53 |
| dhellmann | fungi: as in the text encoding of the template file -- the spot the error comes from looks like it is a code path used when the encoding can't be determined | 18:53 |
| jroll | (ish) | 18:53 |
| jroll | jeblair: yeah. I'd like to fix this. priorities are hard. | 18:53 |
| fungi | dhellmann: oh, eww. utf-8 everywhere! ;) | 18:54 |
| jhesketh | Morning | 18:54 |
| lifeless | fungi: seen fidoalliance.org ? | 18:54 |
| jeblair | jhesketh: good morning, how was your nap? :) | 18:54 |
| dhellmann | fungi: yeah, these files are ascii so they should be ok, but I wonder if something being injected into them isn't | 18:54 |
| anteaya | asselin: where is your jenkins module split patch? | 18:55 |
| fungi | lifeless: now i have. some interesting options there | 18:56 |
| jhesketh | Heh, short ;-) | 18:56 |
| *** thedodd has joined #openstack-infra | 18:57 | |
| jeblair | i assume that's how you log into fidonet | 18:57 |
| fungi | lifeless: i'll bring this to the attention of the devs who have been working on openstackid | 18:57 |
| fungi | lifeless: could provide some useful possibilities for moving away from passwords | 18:57 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: multi-node dg setup https://review.openstack.org/106043 | 18:58 |
| lifeless | fungi: thanks ;) | 18:58 |
| fungi | lifeless: thanks to you too | 18:58 |
| *** mfink_ has joined #openstack-infra | 18:59 | |
| fungi | meeting time in #openstack-meeting ! | 18:59 |
| *** mfink has quit IRC | 19:01 | |
| *** Guest53198 has joined #openstack-infra | 19:02 | |
| *** emagana has quit IRC | 19:02 | |
| *** emagana has joined #openstack-infra | 19:02 | |
| *** guibuild has joined #openstack-infra | 19:03 | |
| *** mfer has joined #openstack-infra | 19:03 | |
| *** Longgeek has quit IRC | 19:04 | |
| *** mfink_ has quit IRC | 19:04 | |
| *** sarob has joined #openstack-infra | 19:04 | |
| *** derekh has joined #openstack-infra | 19:06 | |
| *** emagana has quit IRC | 19:07 | |
| *** andreykurilin_ has joined #openstack-infra | 19:08 | |
| *** jistr has joined #openstack-infra | 19:10 | |
| *** tonytan4ever has quit IRC | 19:10 | |
| *** sarob has quit IRC | 19:10 | |
| *** sarob has joined #openstack-infra | 19:11 | |
| *** jamielennox has joined #openstack-infra | 19:12 | |
| *** cody-somerville has quit IRC | 19:13 | |
| *** radez_g0n3 is now known as radez | 19:14 | |
| *** ayoung has quit IRC | 19:15 | |
| *** otherwiseguy has quit IRC | 19:15 | |
| *** hdd has quit IRC | 19:16 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/project-config: Add a new qa-specs-core group and use it https://review.openstack.org/130012 | 19:17 |
| ashp | urgh, this is a really dumb apache question but it's that kind of day - when I add the AliasMatch stuff for /p/ for zuul so jenkins can see repos | 19:17 |
| ashp | i just get "request not supported" from apache, am I missing something obvious like.. do I need a <Directory> on /var/zuul/git as well? | 19:17 |
| *** emagana has joined #openstack-infra | 19:18 | |
| *** hashar has quit IRC | 19:18 | |
| *** emagana has quit IRC | 19:18 | |
| fungi | ashp: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/zuul/templates/zuul.vhost.erb | 19:19 |
| ashp | ahhh that's where it is, I must be blind | 19:19 |
| ashp | I was looking in the zuul module too but didn't see anything in manifests to drop it in place | 19:19 |
| ashp | shit, you don't have anything I don't, why does apache hate me. :/ | 19:19 |
| fungi | what distro/release? | 19:20 |
| fungi | we're using apache 2.2 on ubuntu precise there | 19:20 |
| ashp | 12.04, yeah | 19:20 |
| *** sdake_ has quit IRC | 19:20 | |
| ashp | Request not supported: '/var/zuul/git/metacloud/horizon refs/zuul/mc/2013.1.7/Zfb69f1305040430c95dc5c1db65f999e' | 19:20 |
| ashp | weirdly not a lot of useful stuff about this online either :/ | 19:20 |
| ashp | Maybe git fetch does something a regular http request wouldn't, so I can't debug it as easily, that kind of makes sense | 19:22 |
| ashp | because the actual job just gives me: 19:10:15 fatal: Couldn't find remote ref refs/zuul/mc/2013.1.7/Zd10acef0f2704b03a133c70e752b056d | 19:22 |
| ashp | I can confirm the actual ref is there, it's just the git fetch seems to fail :/ | 19:23 |
| *** freemanbrandon has joined #openstack-infra | 19:24 | |
| ashp | it's annoying, it gets the gerrit stuff, then just fails when it hits up zuul :/ | 19:25 |
| ashp | ok, so no apache failures when git fetch is ran, it just.. doesn't work | 19:26 |
| *** thedodd has quit IRC | 19:27 | |
| openstackgerrit | Ken Giusti proposed a change to openstack-infra/project-config: Add tests to verify oslo.messaging's AMQP 1.0 protocol support https://review.openstack.org/124433 | 19:28 |
| *** yamamoto has joined #openstack-infra | 19:28 | |
| *** MaxV has joined #openstack-infra | 19:28 | |
| *** dprince has quit IRC | 19:28 | |
| fungi | does that ref exist? | 19:28 |
| ashp | yeah | 19:29 |
| ashp | -rw-r--r-- 1 zuul zuul 40 Oct 21 19:29 Z49f440ba6f5b42f895a08bb82eda5b9b | 19:30 |
| ashp | (different ref, but I had to trigger it again) | 19:30 |
| *** thedodd has joined #openstack-infra | 19:30 | |
| *** yamamoto has quit IRC | 19:33 | |
| asselin | anteaya, nibalizer puppet module split. what I've done so far: https://review.openstack.org/#/c/129768/ https://github.com/rasselin/puppet-jenkins | 19:33 |
| *** packet has joined #openstack-infra | 19:33 | |
| anteaya | thanks | 19:34 |
| anteaya | asselin: I've added an agenda item to the infra meeting agenda | 19:34 |
| anteaya | do join us in -meeting | 19:34 |
| asselin | ok thanks | 19:34 |
| *** ildikov has joined #openstack-infra | 19:34 | |
| *** packet has quit IRC | 19:35 | |
| asselin | anteaya, fyi, what I was doing was just poc so I understand the process e2e | 19:35 |
| openstackgerrit | Mike Heald proposed a change to openstack-infra/storyboard: Paginate after filtering query https://review.openstack.org/130016 | 19:36 |
| anteaya | asselin: perfect | 19:38 |
| *** jamielennox has quit IRC | 19:38 | |
| anteaya | asselin: that was what I was hoping | 19:38 |
| *** tonytan4ever has joined #openstack-infra | 19:39 | |
| *** gyee has quit IRC | 19:42 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/devstack-gate: Ability to flag projects to use non-standard libraries https://review.openstack.org/130018 | 19:42 |
| *** shashankhegde has joined #openstack-infra | 19:42 | |
| *** otherwiseguy has joined #openstack-infra | 19:42 | |
| *** yolanda has quit IRC | 19:43 | |
| *** sarob has quit IRC | 19:47 | |
| *** hashar has joined #openstack-infra | 19:47 | |
| *** sarob has joined #openstack-infra | 19:47 | |
| *** e0ne has joined #openstack-infra | 19:51 | |
| *** terryw has joined #openstack-infra | 19:51 | |
| pleia2 | off the plane! (connection in CLT) | 19:52 |
| *** sarob has quit IRC | 19:52 | |
| derekh | Weird gierrit issue : I'm logged into gerrit and hit this link https://review.openstack.org/#/c/126513/ | 19:53 |
| derekh | I am getting told "The page you requested was not found, or you do not have permission to view this page." | 19:53 |
| *** dane_leblanc has quit IRC | 19:53 | |
| *** otherwiseguy has quit IRC | 19:53 | |
| derekh | if I log out, I can see the patch as expected | 19:53 |
| *** amcrn has quit IRC | 19:54 | |
| *** changbl has quit IRC | 19:55 | |
| *** kirscht has quit IRC | 19:56 | |
| *** changbl has joined #openstack-infra | 19:58 | |
| *** Guest53198 has quit IRC | 19:58 | |
| anteaya | pleia2: yay | 19:58 |
| *** boris-42 has quit IRC | 19:59 | |
| *** dane_leblanc has joined #openstack-infra | 19:59 | |
| *** markmcclain1 has quit IRC | 19:59 | |
| *** markmcclain has joined #openstack-infra | 19:59 | |
| jeblair | fungi, clarkb, mordred, SergeyLukjanov (if you happen to be around): anyone interested in an interactive afs walkthrough after the tc meeting? | 20:00 |
| fungi | jeblair: absolutely | 20:01 |
| * zaro is | 20:01 | |
| *** jistr has quit IRC | 20:01 | |
| fungi | jeblair: i played around a little on afstest | 20:01 |
| *** unicell has quit IRC | 20:01 | |
| fungi | jeblair: should i use that system as a testbed for scripting the uid/gid remapping? | 20:01 |
| *** derekh has quit IRC | 20:01 | |
| *** baoli has joined #openstack-infra | 20:02 | |
| jeblair | fungi: i think that would be fine -- though i have a bandersnatch running right now | 20:02 |
| ianw | fungi / jeblair / clarkb : didn't get to it in the meeting, but can we merge https://review.openstack.org/#/c/128498/ (Add HP Cloud Centos7 images). we agreed last week to use these images until we're in a position to have d-i-b working | 20:02 |
| fungi | oh, no worries. just trying to figure out somewhere good where we can demo that the uids/gids are showing up correctly after changing | 20:02 |
| fungi | (in relation to afs-created files) | 20:03 |
| nibalizer | jeblair: HI I AM TEACH ME | 20:03 |
| nibalizer | although if it requires root/shell access probably hard for me to actually participate | 20:03 |
| clarkb | jeblair maybe? will I need to be somewhere quiet? | 20:03 |
| fungi | yeah, i get the impression this is intended to be "how to manage the afs cell and kdcs" | 20:04 |
| *** dteselkin__ has quit IRC | 20:04 | |
| clarkb | I am lunching in not quiet location | 20:04 |
| jeblair | clarkb: i was thinking phone conf + screen session might be the best way? but won't be for another hour anyway | 20:04 |
| *** david-lyle has quit IRC | 20:04 | |
| jeblair | fungi: a little of afs 101 too | 20:04 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gearman-plugin: Fix job result not being sent back to gearman client https://review.openstack.org/130025 | 20:05 |
| zaro | clarkb: ^ | 20:05 |
| sdake | when I push a signed key for a release, where does gerrit get my gpg public key from? | 20:05 |
| *** ociuhandu has quit IRC | 20:05 | |
| fungi | sdake: it doesn't know anything about your key | 20:06 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Paginate after filtering query https://review.openstack.org/130016 | 20:06 |
| fungi | sdake: it's just authenticating you based on your ssh credentials | 20:06 |
| sdake | fungi so I can use any key I like and gerrit wont auth it? | 20:06 |
| sdake | cool but I still must use a signed key? | 20:06 |
| sdake | rather a signed tag? | 20:06 |
| fungi | sdake: yes, you need to sign a tag to be able to get gerrit to accept it, but can sign it with any key you want | 20:07 |
| * anteaya is interested in afs after meetings | 20:07 | |
| sdake | fungi++ thanks bro | 20:07 |
| fungi | ideally it would be a key you use for these sorts of things, and getting it into the web of trust where possible is encouraged | 20:07 |
| AJaeger_ | dhellmann: So, for https://review.openstack.org/#/c/129433/1/zuul/layout.yaml - we will keep python 2.6 on both master and branches for all oslo projects for now? | 20:08 |
| pleia2 | anteaya: I'm assuming as nibalizer is, hoping the 101 will help update docs for the rest of us though :) /me still has initial docs patch to review | 20:08 |
| fungi | sdake: down the road we might start setting up keyrings in the release automation and allow ptls to decide on a list of key fingerprints which are authorized to sign release tags for their projects or something, as an additional safety | 20:08 |
| sdake | yup makes sense | 20:08 |
| clarkb | jeblair I can mute my mic I guess | 20:08 |
| AJaeger_ | SergeyLukjanov: please review https://review.openstack.org/#/c/129433/1/zuul/layout.yaml - see fungi's comment on sahara-dashboard | 20:08 |
| clarkb | or move location | 20:08 |
| jeblair | clarkb: or we can wait a bit :) when would be a good time for you? | 20:09 |
| *** cody-somerville has joined #openstack-infra | 20:09 | |
| anteaya | pleia2: we certainly will try to ensure the broader interest is served | 20:09 |
| *** mrmartin has quit IRC | 20:10 | |
| hashar | zaro: poor gearman plugin :D | 20:10 |
| lifeless | man | 20:11 |
| lifeless | setuptools on python3.2 is bust :< | 20:11 |
| lifeless | it writes an entrypoint using execfile, which doth not exist | 20:11 |
| *** aysyd has quit IRC | 20:11 | |
| fungi | lifeless: i thought the only releases more broken than python 3.2 were 3.1 and 3.0 | 20:12 |
| dhellmann | AJaeger_: that's probably the safest thing for now, yes | 20:12 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gearman-plugin: Fix job result not being sent back to gearman client https://review.openstack.org/130025 | 20:12 |
| lifeless | fungi: yeah but 3.2 has security fixes through 2016 | 20:12 |
| lifeless | fungi: the other two are buried and dead dead dead | 20:12 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/project-config: Limit python26 jobs to icehouse/juno for server projects https://review.openstack.org/129433 | 20:13 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/project-config: Removed python26 from python-jobs https://review.openstack.org/128736 | 20:13 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/project-config: Remove python26 jobs from all stackforge projects https://review.openstack.org/129434 | 20:13 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/project-config: Remove python26 jobs from various projects https://review.openstack.org/129435 | 20:13 |
| clarkb | jeblair all day tomorrow should be good | 20:13 |
| AJaeger_ | ok, rebased the python 2.6 pachtes and updated commit messages, there was indeed a merge conflict. No other changes done to them. | 20:13 |
| clarkb | or thurs or friday | 20:13 |
| SergeyLukjanov | AJaeger_, okay, looking | 20:14 |
| jeblair | clarkb: i pretty much only have this afternoon available | 20:14 |
| *** david-lyle has joined #openstack-infra | 20:15 | |
| SergeyLukjanov | jeblair, I am (if I will not fall asleep) | 20:15 |
| *** ayoung has joined #openstack-infra | 20:16 | |
| * AJaeger_ waves good bye! | 20:16 | |
| anteaya | oh I was going to say something in the meeting, in the in case anyone cares category | 20:16 |
| anteaya | I'm away starting next Tuesday | 20:16 |
| anteaya | at the summit | 20:16 |
| *** AJaeger_ has quit IRC | 20:16 | |
| anteaya | and then off the week after | 20:16 |
| anteaya | I don't think i have anything big outstanding | 20:16 |
| pleia2 | I have much travel too, All Things Open this week, travel days next Monday & Friday | 20:17 |
| clarkb | jeblair hrm I can probably swing something later in afternoon. maybe 3ish? | 20:17 |
| pleia2 | then heading out Thursday after summit for Proper Vacation for a week | 20:17 |
| pleia2 | anteaya: maybe we need a team travel calendar :) | 20:18 |
| fungi | yeah, i expect a lot of people are travelling next week. i fly out wednesday | 20:18 |
| *** emagana has joined #openstack-infra | 20:18 | |
| jeblair | clarkb: ok, so 2200? | 20:18 |
| clarkb | ya lets do that | 20:18 |
| jeblair | fungi: work for you? | 20:18 |
| jeblair | SergeyLukjanov: ? | 20:18 |
| fungi | wfm | 20:19 |
| anteaya | 2200 is good for me | 20:19 |
| anteaya | pleia2: no | 20:19 |
| anteaya | pleia2: I don't want to be responsible to a calendar | 20:19 |
| SergeyLukjanov | jeblair, it's in 40 mins? | 20:20 |
| *** r1chardj0n3s_afk is now known as r1chardj0n3s | 20:20 | |
| jeblair | SergeyLukjanov: 1:40 | 20:20 |
| *** jamielennox has joined #openstack-infra | 20:20 | |
| SergeyLukjanov | jeblair, oh, not sure, it'll be 2 am for me | 20:20 |
| openstackgerrit | A change was merged to openstack-infra/project-config: Add a new qa-specs-core group and use it https://review.openstack.org/130012 | 20:21 |
| *** Guest53198 has joined #openstack-infra | 20:24 | |
| *** e0ne has quit IRC | 20:25 | |
| jeblair | clarkb: any chance 2pm would work for you? otherwise SergeyLukjanov, maybe we can schedule another (earlier) time some other day (maybe next week?) | 20:25 |
| *** Sukhdev has quit IRC | 20:26 | |
| *** sarob has joined #openstack-infra | 20:26 | |
| clarkb | 2 will be hard | 20:27 |
| anteaya | fungi: following the tc meeting? | 20:27 |
| anteaya | since you are often the add me to the gerrit group person | 20:28 |
| fungi | anteaya: trying | 20:28 |
| *** yamamoto has joined #openstack-infra | 20:28 | |
| clarkb | maybe you all do 2 and I catch up later? | 20:28 |
| openstackgerrit | A change was merged to openstack-infra/system-config: Add HP Cloud Centos7 images https://review.openstack.org/128498 | 20:28 |
| anteaya | fungi: so for tc repos ttx says add the original commiter to the first group for the repo | 20:29 |
| anteaya | fungi: unless the commit message says differently | 20:29 |
| fungi | sounds fine | 20:29 |
| *** miqui has quit IRC | 20:30 | |
| *** ayoung has quit IRC | 20:30 | |
| SergeyLukjanov | jeblair, IMO don't change time slot for me, I hope we could find some time later (before summit) to teach me a bit about afs | 20:30 |
| SergeyLukjanov | jeblair, and I will read logs (as always :) ) | 20:31 |
| *** boris-42 has joined #openstack-infra | 20:32 | |
| *** yamamoto has quit IRC | 20:32 | |
| *** changbl has quit IRC | 20:32 | |
| *** oramraz has quit IRC | 20:33 | |
| *** julim has quit IRC | 20:33 | |
| fungi | and stackforge/heat-translator finds its way into the rename pile | 20:33 |
| *** teran has joined #openstack-infra | 20:33 | |
| *** _nadya_ has quit IRC | 20:33 | |
| fungi | also also its lack of working pn python 3.4 ceases to be a low priority i guess | 20:34 |
| *** teran_ has joined #openstack-infra | 20:34 | |
| *** Guest53198 has quit IRC | 20:34 | |
| clarkb | also next week works too | 20:35 |
| anteaya | fungi: and dhellmann says tc group is also part of any tc repo gerrit group | 20:35 |
| fungi | yep, got that | 20:35 |
| anteaya | thanks | 20:37 |
| *** teran has quit IRC | 20:38 | |
| *** jedimike has quit IRC | 20:38 | |
| *** oramraz has joined #openstack-infra | 20:38 | |
| *** teran_ has quit IRC | 20:38 | |
| *** dmsimard is now known as dmsimard_away | 20:40 | |
| *** jamielennox has quit IRC | 20:41 | |
| openstackgerrit | A change was merged to openstack-infra/project-config: Add API Working Group project https://review.openstack.org/128466 | 20:41 |
| * fungi still never saw an explanation for why the content for that shouldn't have just gone into openstack/openstack-specs instead of creating yet another repo | 20:43 | |
| *** lttrl has quit IRC | 20:43 | |
| *** sabeen1 has quit IRC | 20:45 | |
| *** ayoung has joined #openstack-infra | 20:45 | |
| anteaya | fungi: I don't know that anyone ever asked that question | 20:46 |
| anteaya | I didn't think of it | 20:46 |
| *** radez is now known as radez_g0n3 | 20:46 | |
| openstackgerrit | Ken Giusti proposed a change to openstack-infra/project-config: Add tests to verify oslo.messaging's AMQP 1.0 protocol support https://review.openstack.org/124433 | 20:46 |
| fungi | oh, i broughht it up at least once in irc, but should have mentioned it in that change as well as the governance one, and prior to the tc meeting, so mea culpa | 20:47 |
| *** ayoung has quit IRC | 20:47 | |
| anteaya | guess I missed it in the backscroll | 20:47 |
| anteaya | but in hindsight I agree it should have been considered | 20:47 |
| *** ayoung has joined #openstack-infra | 20:47 | |
| fungi | i did add that question to the governance change, but not until after it was on the agenda | 20:47 |
| anteaya | because if things that should be specs just make their own repo, why have specs? | 20:47 |
| anteaya | ah | 20:48 |
| fungi | at which point the horse is already out of the barn | 20:48 |
| anteaya | yeah | 20:48 |
| *** whoops has quit IRC | 20:48 | |
| * anteaya watches the horse run around in the fields | 20:48 | |
| *** carl_baldwin has joined #openstack-infra | 20:49 | |
| *** whoops has joined #openstack-infra | 20:50 | |
| *** katyafervent_awa has quit IRC | 20:53 | |
| *** katyafervent_awa has joined #openstack-infra | 20:53 | |
| *** gyee has joined #openstack-infra | 20:55 | |
| *** freemanbrandon has quit IRC | 20:55 | |
| *** teran has joined #openstack-infra | 20:56 | |
| *** HeOS has joined #openstack-infra | 20:56 | |
| *** yjiang5 has left #openstack-infra | 20:58 | |
| ianw | jeblair / clarkb : so the nodepool logs are starting to get really quite large. turning on gzip turned out to be a whole afternoon adventure, after i realised that the current puppet-apache doesn't allow override in its template. so i've proposed https://review.openstack.org/#/c/129096/ - a new template for nodepool.o.o and also https://review.openstack.org/#/c/129496/ to fork puppet-apache, because we are so divergent from upstrea | 21:00 |
| ianw | m that upgrading in-place is, in my estimation, no feasible | 21:00 |
| *** cpowell has quit IRC | 21:01 | |
| *** wenlock has quit IRC | 21:02 | |
| *** hdd has joined #openstack-infra | 21:02 | |
| *** gary-smith has joined #openstack-infra | 21:02 | |
| *** mfer has quit IRC | 21:02 | |
| fungi | anteaya: there are lemmings around hudson bay... do they not range farther west? | 21:02 |
| anteaya | fungi: not sure | 21:03 |
| anteaya | not as far as vancouver | 21:03 |
| fungi | oh, too bad | 21:03 |
| anteaya | and as far as I know, no cities named after them | 21:03 |
| fungi | because i totally want an openstack lemming shirt | 21:03 |
| fungi | now that it's been suggested | 21:03 |
| anteaya | I shall not stand in your way | 21:03 |
| jeblair | SergeyLukjanov, clarkb: either of you around now? | 21:03 |
| anteaya | I'm sure mikal wants one too | 21:03 |
| SergeyLukjanov | jeblair, I'm on a project meeting, but mostly available ;) | 21:04 |
| anteaya | fungi: hudson bay to vancouver would be about 3000+ km | 21:04 |
| jeblair | SergeyLukjanov: was wondering what we decided about the afs show and tell | 21:04 |
| jogo | jeblair: was there headway on getting governance repo published in the TC meeting? | 21:04 |
| anteaya | a bit far to travel on such short legs | 21:04 |
| jeblair | jogo: i don't think it was on the agenda | 21:05 |
| fungi | jogo: it was discussed in the foundation staff meeting | 21:05 |
| jogo | jeblair: guess it will stay on the ML then, thanks | 21:05 |
| SergeyLukjanov | jeblair, if everyone else are ok | 21:05 |
| jogo | fungi: oh exciting | 21:05 |
| *** ChuckC has quit IRC | 21:05 | |
| fungi | i think it was decided that we can do either or both options proposed | 21:05 |
| anteaya | ttx flag would be considered federal | 21:05 |
| anteaya | federal is national level of government | 21:05 |
| anteaya | or national flag | 21:05 |
| anteaya | canada's next level of government is provincial | 21:06 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Set SWIFT_TEMPURL_KEY for Ironic agent_ssh checks https://review.openstack.org/129653 | 21:06 |
| anteaya | then municipal | 21:06 |
| fungi | jbryce suggested an scp/sftp account could be created to dump files to the www site in "a day or two" so as soon as someone's ready to do that we can show them to the jenkins ssh public key. in the meantime we can also publish to static.o.o and use whatever vhost name we like for that | 21:06 |
| fungi | jogo: ^ | 21:06 |
| jeblair | fungi: scp? | 21:07 |
| fungi | jeblair: so that we could use it with the jenkins scp publisher plugin? | 21:07 |
| *** emagana has quit IRC | 21:07 | |
| jeblair | fungi: yeah, that's my question; did not know cloud sites supported scp. | 21:07 |
| fungi | jeblair: it's no longer on cloud sites | 21:07 |
| fungi | evolution! | 21:08 |
| jeblair | fungi: where is it? | 21:08 |
| fungi | a rackspace-managed server | 21:08 |
| *** emagana has joined #openstack-infra | 21:08 | |
| fungi | tipit migrated it from cloudsites to a server plus trove | 21:08 |
| * anteaya wonders about inviting rick mercer to the vancouver openstack summit | 21:09 | |
| fungi | they also just gave me and fifieldt access to the private git repo of the source code and a sanitized dump of the database usable for recreating the bits under the silverstripe cms for testing purposes, and assuming we see no serious red flags they want to open it up public before paris | 21:09 |
| *** dizquierdo has joined #openstack-infra | 21:10 | |
| jeblair | fungi: so that we can start working on putting it on an openstack-managed server? | 21:10 |
| fungi | jeblair: that's what i see as the next step, yes | 21:10 |
| clarkb | jeblair: ya now is still bad. I can catch up later if you want ot go ahead | 21:11 |
| *** ChuckC has joined #openstack-infra | 21:11 | |
| clarkb | also I should probably fix teh jenkinses | 21:11 |
| fungi | the migration off cloud-sites was apparently more prompted by security concerns over cloudsites databases (being even worse than trove security in rackspace) and by deployment needs for openstackid | 21:11 |
| clarkb | or next week is fine with me | 21:11 |
| jeblair | clarkb: i'm trying to get you up to speed since you asked for a tldr :) | 21:12 |
| anteaya | fungi: can you add cyeoh to https://review.openstack.org/#/admin/groups/469,members | 21:12 |
| anteaya | and is release automatically a member of core? | 21:12 |
| clarkb | I know its just hard since I ended up outside of home to carpool with larissa | 21:12 |
| fungi | anteaya: nobody is automatically a member of anything, no | 21:12 |
| *** emagana has quit IRC | 21:13 | |
| anteaya | ah okay | 21:13 |
| anteaya | so perhaps release as a member of core then | 21:13 |
| *** oramraz has quit IRC | 21:13 | |
| clarkb | zaro: so the fix is to just stop triggering on those events? | 21:13 |
| anteaya | so he can add members to core | 21:13 |
| clarkb | zaro: why did we want to trigger on those events in the first plcae? I want to make sure that we don't break somethign else in the process | 21:13 |
| gary-smith | Horizon patchsets (e.g. https://review.openstack.org/#/c/112741/12) are being verified by Sahara Hadoop Cluster CI in a questonably short amount of time. Any idea why a Sahara CI is building horizon changes? | 21:14 |
| anteaya | SergeyLukjanov: ^^ | 21:14 |
| fungi | jeblair: i'm personally okay reviewing http://docs-draft.openstack.org/48/129748/1/check/gate-ci-docs/86e5548/doc/build/html/ and experimenting myself in non-impacting ways if we want to reschedule a tol;dr at a time when more root admins are around than just you and me | 21:14 |
| *** kgiusti has left #openstack-infra | 21:14 | |
| clarkb | zaro: looks like we expect the saveablelisteners onChange to catch these cases? | 21:15 |
| anteaya | SergeyLukjanov: plus the ci should not be commenting "starting check jobs" | 21:15 |
| *** terryw has quit IRC | 21:15 | |
| anteaya | gary-smith: how long as it been doing this? | 21:15 |
| gary-smith | at least since yesterday | 21:15 |
| *** markmcclain1 has joined #openstack-infra | 21:15 | |
| *** packet has joined #openstack-infra | 21:15 | |
| *** packet has quit IRC | 21:15 | |
| *** packet has joined #openstack-infra | 21:15 | |
| *** markmcclain1 has quit IRC | 21:15 | |
| *** markmcclain has quit IRC | 21:15 | |
| *** markmcclain1 has joined #openstack-infra | 21:15 | |
| *** emagana has joined #openstack-infra | 21:16 | |
| gary-smith | I just came back from a week of vacation, so it could have been longer | 21:16 |
| jeblair | clarkb, SergeyLukjanov, fungi: what about friday at 9am pst / 1600 utc? | 21:16 |
| *** packet has quit IRC | 21:16 | |
| anteaya | gary-smith: thanks | 21:16 |
| fungi | jeblair: that works for me | 21:16 |
| clarkb | jeblair: I can do that | 21:16 |
| *** packet has joined #openstack-infra | 21:16 | |
| anteaya | SergeyLukjanov: looks like I will have to disable your system unless you show up to tell me you have beaten me to it | 21:16 |
| SergeyLukjanov | gary-smith, hm, yeah, I see, we'll disable voting tomorrow to make it stable first | 21:16 |
| anteaya | SergeyLukjanov: can we disable teh system now please? | 21:16 |
| fungi | SergeyLukjanov: can we disable voting for you now? | 21:16 |
| SergeyLukjanov | gary-smith, the reason is that there are too many changes in horizon done that are breaks sahara part | 21:17 |
| fungi | heh. anteaya beats me to the question ;) | 21:17 |
| SergeyLukjanov | anteaya, fungi, okay, I'll disable the job in 10 mins | 21:17 |
| anteaya | SergeyLukjanov: thank you | 21:17 |
| gary-smith | SergeyLukjanov: thanks | 21:17 |
| fungi | thanks SergeyLukjanov! | 21:17 |
| jeblair | SergeyLukjanov: does 1600utc friday work for talking about afs? | 21:17 |
| SergeyLukjanov | jeblair, let me check | 21:18 |
| SergeyLukjanov | jeblair, yeah, it works for me | 21:18 |
| jeblair | yay! | 21:18 |
| *** otherwiseguy has joined #openstack-infra | 21:18 | |
| SergeyLukjanov | gary-smith, I think we'll raise this question on the next horizon meeting, because we need to do something with changes that are breaking sahara pages | 21:19 |
| gary-smith | SergeyLukjanov: sounds good, so long as it actually builds it :-) | 21:20 |
| clarkb | zaro: I have 06 and 07 in shutdown mode. If maybe we can get that fix merged I can upgrade to that new stuff | 21:21 |
| clarkb | otherwise I will just be reverting | 21:21 |
| SergeyLukjanov | gary-smith, the job you linked runs sahara integration tests using horizon | 21:21 |
| jeblair | clarkb: link? | 21:21 |
| clarkb | let me know what you think but hoping you can comment on my questions | 21:21 |
| clarkb | jeblair: https://review.openstack.org/#/c/130025/2 | 21:21 |
| gary-smith | SergeyLukjanov: In less than a minute? Really? | 21:21 |
| clarkb | jeblair: I am concerned that we are just removing the code that I asked to remove in the past | 21:21 |
| SergeyLukjanov | gary-smith, "it should" * | 21:21 |
| clarkb | jeblair: want to make sure we don't break something else since keeping that code was intentional | 21:22 |
| anteaya | SergeyLukjanov: also let's make sure your "starting check jobs" comment is removed | 21:23 |
| anteaya | SergeyLukjanov: that isn't allowed in ci accounts | 21:23 |
| SergeyLukjanov | anteaya, we decided to keep it for sahara jobs to make the vote cleared | 21:23 |
| jeblair | clarkb, zaro: whoah | 21:23 |
| jeblair | clarkb, zaro: why would de-registering a job cause a disconnect. that makes no sense. | 21:23 |
| *** markmcclain1 has quit IRC | 21:23 | |
| clarkb | jeblair: it was my initial concern | 21:24 |
| SergeyLukjanov | anteaya, if we'll build the jobs for other project it'll be removed | 21:24 |
| jeblair | s/job/function/ | 21:24 |
| clarkb | jeblair: because we offline when the job starts | 21:24 |
| jeblair | clarkb: not sure what you are saying. | 21:24 |
| clarkb | jeblair: so gearman sends h edisconnect before it sends the result or something | 21:24 |
| *** dkranz has quit IRC | 21:24 | |
| jeblair | clarkb: right, that's totally wrong. | 21:24 |
| clarkb | jeblair: that event fires in jenkins as soon as the job starts running | 21:24 |
| clarkb | so its ahead of the other gearman worker event stuff? (I don't actually understand the entire mechanism yet) | 21:24 |
| *** emagana has quit IRC | 21:25 | |
| *** terryw has joined #openstack-infra | 21:25 | |
| *** todd_dsm has quit IRC | 21:25 | |
| anteaya | SergeyLukjanov: who is we? | 21:25 |
| anteaya | of the we decided | 21:25 |
| anteaya | and where is the log of this decison | 21:26 |
| *** emagana has joined #openstack-infra | 21:26 | |
| anteaya | since making exceptions for third party tends not to go over well | 21:26 |
| anteaya | with the 90+ others | 21:26 |
| anteaya | in that they have no ability to comprehend | 21:26 |
| *** hashar has quit IRC | 21:27 | |
| anteaya | and will use it as an example to be not compliant themselves | 21:27 |
| SergeyLukjanov | anteaya, it was discussed by sahara team and the main reason is that the vote from sahara-ci is a mandatory vote for approving changes | 21:27 |
| *** yjiang5 has joined #openstack-infra | 21:27 | |
| anteaya | so you can get the vote | 21:27 |
| anteaya | I don't understand why you post starting check jobs | 21:28 |
| anteaya | and this sint' a sahara patch | 21:28 |
| clarkb | jeblair: sounds like maybe I should just revert 06 and 07 to old plugin? or would you like me to wait? | 21:28 |
| anteaya | this is horizon | 21:28 |
| SergeyLukjanov | anteaya, to reset the CI vote | 21:28 |
| *** yamamoto has joined #openstack-infra | 21:28 | |
| SergeyLukjanov | anteaya, it's not related to horizon | 21:28 |
| jeblair | clarkb: i think that letting offline nodes have functions registered is asking for it | 21:28 |
| anteaya | so when the new vote arrives, it will replace the old vote | 21:28 |
| *** otherwiseguy has quit IRC | 21:28 | |
| fungi | SergeyLukjanov: we don't even do that with openstack "jenkins" account votes in the check pipeline any longer | 21:28 |
| ianw | anteaya: if you could re-evaluate https://review.openstack.org/#/c/128793/, that would be cool | 21:29 |
| SergeyLukjanov | anteaya, as I said it was a test job for horizon that I was unaware about and I've just disable it | 21:29 |
| * anteaya reevaluates | 21:29 | |
| anteaya | SergeyLukjanov: and let's remove "starting check jobs" comments please | 21:29 |
| clarkb | jeblair: I think the problem is that we are stopping the worker | 21:30 |
| SergeyLukjanov | anteaya, okay, I'll assign someone to do update the layout | 21:30 |
| clarkb | jeblair: so maybe we need to decouple stop from unregister function? | 21:30 |
| anteaya | SergeyLukjanov: thank you | 21:30 |
| clarkb | jeblair: then run stop when the node is actually removed | 21:30 |
| *** berendt has quit IRC | 21:30 | |
| *** emagana has quit IRC | 21:30 | |
| jeblair | clarkb: i do not believe we previously stopped the worker when a node when temporarily offline | 21:31 |
| anteaya | jeblair mtreinish this changes a devstack and devstack-gate job from experiemental to check: https://review.openstack.org/#/c/128793/1 | 21:31 |
| clarkb | jeblair: correct | 21:31 |
| anteaya | jeblair mtreinish was about to approve unless you disagree | 21:32 |
| jeblair | clarkb: why was that change made? | 21:32 |
| *** sarob has quit IRC | 21:32 | |
| clarkb | jeblair: we discussed it in the previous change and zaro indicated he order of ops was correct. And the move was made because we are changing the listenable type we use | 21:32 |
| clarkb | jeblair: jenkins has made internal api changes so you can't rely on the old listener things that we relied on before | 21:32 |
| mtreinish | anteaya: I'll defer to dtroyer there, whether he considers keeping devstack on centos a first class support thing | 21:32 |
| anteaya | mtreinish: great, when is he next around? | 21:33 |
| *** yamamoto has quit IRC | 21:33 | |
| jeblair | clarkb: understood, but this seems like a behavior change. why would we start doing this action on temporarily on/offline when we did not before? | 21:33 |
| clarkb | jeblair: that was my question... My understanding was it should work (but doesn't) and its so that we dregister functions when the node is offlined | 21:34 |
| mtreinish | anteaya: dunno, he was online ealier today | 21:34 |
| *** esker has quit IRC | 21:34 | |
| *** emagana has joined #openstack-infra | 21:34 | |
| anteaya | mtreinish: kk thanks, I made a note in the comment | 21:35 |
| anteaya | I'll add them to the review | 21:35 |
| anteaya | him and jeblair | 21:35 |
| cyeoh | anteaya, btw the email address listed here: https://review.openstack.org/#/admin/groups/469,members is mine, but its not the email address I use for gerrit. I'm not sure if it came from launchpad where I used to have two accounts or something silly I did when I first started working on openstack | 21:35 |
| anteaya | oh they already are | 21:35 |
| dtroyer | Tuesdays are fun…I'm around mostly | 21:35 |
| *** alexpilotti has quit IRC | 21:35 | |
| anteaya | dtroyer: cool | 21:36 |
| anteaya | dtroyer: your thoughts: https://review.openstack.org/#/c/128793/ | 21:36 |
| jeblair | anteaya: i don't find reviews because i have been added to them | 21:36 |
| anteaya | jeblair: I didn't think so | 21:36 |
| *** dizquierdo has quit IRC | 21:36 | |
| jeblair | clarkb: those methods should probably call "registerJobs()" then :) | 21:36 |
| anteaya | just trying to do my best to leave ianw with something he can follow up on | 21:36 |
| *** sabeen1 has joined #openstack-infra | 21:37 | |
| clarkb | jeblair: that won't work because the node is still up | 21:37 |
| anteaya | fungi: hmmm, how do we straighten up things for cyeoh? | 21:37 |
| jeblair | clarkb: registerJobs knows to deregister them when the node is offline. that's why it worked before. | 21:37 |
| *** sabeen1 has quit IRC | 21:37 | |
| clarkb | oh ok so that I completely missed | 21:37 |
| clarkb | zaro: ^ | 21:37 |
| *** sabeen1 has joined #openstack-infra | 21:37 | |
| jeblair | if (!computer.isOffline()) { | 21:37 |
| jeblair | the whole job calculation is within that conditional | 21:38 |
| fungi | anteaya: cyeoh: sounds like another case of duplicate launchpad openids | 21:38 |
| jeblair | and if it is false, then we register the empty set | 21:38 |
| fungi | anteaya: cyeoh: i'll see what i can do to patch that up | 21:38 |
| anteaya | fungi: thanks | 21:38 |
| clarkb | jeblair: ok that was not communicated in the reviews | 21:38 |
| cyeoh | anteaya, fungi: thank! I also have submitted this bug report re: api-wg-core https://bugs.launchpad.net/openstack-ci/+bug/1381265 | 21:38 |
| uvirtbot | Launchpad bug 1381265 in openstack-ci "Request for Initial Gerrit Core Group Member for API WG" [Undecided,New] | 21:38 |
| *** alexpilotti has joined #openstack-infra | 21:38 | |
| clarkb | (which is how I got confused) | 21:39 |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/system-config: Test failure of apply test https://review.openstack.org/129837 | 21:39 |
| *** rlandy has quit IRC | 21:39 | |
| fungi | cyeoh: in the meantime i've added the other christopher yeoh instead | 21:39 |
| jeblair | clarkb: yep, i barely understood what was happening in that review. i just took zaro at his word that he tested it and it continued to work. | 21:39 |
| clarkb | jeblair: me too :) | 21:39 |
| cyeoh | fungi: heh, we both thank you :-) | 21:39 |
| ekarlso | would it be negative if I used "falcon" for a github trigger in zuul ? | 21:40 |
| fungi | cyeoh: i've added you to api-wg-core now as well | 21:40 |
| cyeoh | fungi: thanks! | 21:40 |
| *** alexpilotti has quit IRC | 21:41 | |
| jeblair | ekarlso: use what storyboard is using, or convince them to change. i believe that's pecan. | 21:41 |
| *** koolhead_ has quit IRC | 21:43 | |
| *** alexpilotti has joined #openstack-infra | 21:43 | |
| clarkb | jeblair: so I should probably put 06 and 07 back in service using the old plugin ya? | 21:43 |
| nibalizer | jeblair: so i re: the 'apply test is lies' i did make the apply test fail with a fail statement | 21:43 |
| clarkb | (prior to first patch to make it handle new jenkins better) | 21:44 |
| jeblair | clarkb: i think so | 21:44 |
| clarkb | will start that now | 21:44 |
| fungi | nibalizer: i've got puppet-apply-precise failures i'm debugging on my devstack.org redirect change too, so i think it is actually breaking for some things anyway | 21:44 |
| jeblair | nibalizer: okay, that's mildly reassuring | 21:45 |
| fungi | presumably breaking because i was bad, and not just because $random_stuff | 21:45 |
| jeblair | so it sounds like we should try to track down why mordred's spefific failure wasn't caught | 21:45 |
| nibalizer | fungi: jeblair im trying the opposite, which is a noop change that should pass | 21:46 |
| clarkb | killing 07 now | 21:46 |
| ekarlso | jeblair: needs to be the same ? :p | 21:46 |
| *** mjturek has quit IRC | 21:46 | |
| jeblair | ekarlso: if you want my +2, yes :) | 21:46 |
| ekarlso | :| | 21:46 |
| *** mjturek has joined #openstack-infra | 21:46 | |
| clarkb | 07 is starting | 21:47 |
| ekarlso | i think pecan is meh vs falcon | 21:47 |
| fungi | consistency means fewer things in whose caveats need to be well versed | 21:47 |
| fungi | er, we need to be well versed | 21:47 |
| clarkb | its up stuff is happening | 21:48 |
| clarkb | doing 06 now | 21:48 |
| fungi | ekarlso: sounds like you opt for #2... "convince them to change" | 21:48 |
| ekarlso | fungi: meh, that's too much work :P | 21:48 |
| *** signed8bit has quit IRC | 21:49 | |
| *** signed8bit has joined #openstack-infra | 21:49 | |
| clarkb | 06 is coming back up now | 21:50 |
| *** tonytan4ever has quit IRC | 21:51 | |
| *** andreykurilin_ has quit IRC | 21:51 | |
| *** pc_m has quit IRC | 21:51 | |
| jeblair | fungi: so back to the governance repo publishing... | 21:52 |
| fungi | oh, right, that | 21:53 |
| clarkb | 06 looks up | 21:53 |
| jeblair | fungi: someone creates an ssh account | 21:53 |
| clarkb | hopefully this helps chew through the backlog | 21:53 |
| jeblair | fungi: with the jenkins ssh public key | 21:53 |
| jeblair | fungi: which is here: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/init.pp#n5 | 21:53 |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/project-config: Define experimental check-heat-dsvm-functional-postgres https://review.openstack.org/129509 | 21:53 |
| jeblair | fungi: then we do an scp publisher for that? | 21:54 |
| *** amcrn has joined #openstack-infra | 21:54 | |
| fungi | jeblair: right. and spit the rendered file tree onto www.openstack.org in /some/path/to/foundation/tech-committee/ | 21:54 |
| jeblair | fungi: i'm moderately uncomfortable creating a publishing workflow with an endpoint that we don't control (and is generally unresponsive) using technologies we are going to migrate away from | 21:55 |
| fungi | jeblair: the people who will create the account will also be able to tell us what /some/path/to should really be | 21:55 |
| jeblair | fungi: because we could easily end up with this being the last thing on this particular tech stack | 21:55 |
| *** MaxV has quit IRC | 21:55 | |
| fungi | jeblair: i am personally in favor of just putting it on static.o.o but it sounded like maybe ttx had a preference for it being directly on the www site? (ttx can you comment?) | 21:56 |
| fungi | seemed related to public discoverability of governance-related policies | 21:57 |
| jeblair | fungi: i think this could be mitigated by universal understanding that we expect www.o.o to be running in infra by the time that becomes an issue, or the likewise understanding that we would simply drop governance publication if it is the last holdout (an action which would hopefully prompt some kind of swift positive resolution). | 21:57 |
| jeblair | oops, i said swift. sorry john. :) | 21:58 |
| *** terryw has quit IRC | 21:58 | |
| notmyname | jeblair: SWIFT!!! | 21:58 |
| *** mmedvede has quit IRC | 21:59 | |
| *** spzala has quit IRC | 21:59 | |
| jeblair | fungi: mind you, i do think this is the best way to proceed now. i just don't like backing us into a corner. :) | 21:59 |
| *** camunoz has joined #openstack-infra | 22:00 | |
| fungi | jeblair: understood. i think the "big hurdle" here is going to be convincing the people who currently control that system to entrust its management to not-a-24x7-staffed-company-with-an-sla-and-pagers | 22:00 |
| fungi | also, convincing the people who currently hack on the source code underlying it to go through a code review system they've never seen before | 22:01 |
| *** dustins has quit IRC | 22:01 | |
| ttx | fungi: I think there is value in pushign that info under www.o.o/foundation/tech-committee | 22:02 |
| jeblair | fungi: if you think that's a hurdle unlikely to be overcome, then i don't think we should do it. | 22:02 |
| ttx | and with the sftp creds that should be quite easy to do | 22:02 |
| jeblair | it == publish governance there. | 22:02 |
| zaro | clarkb, jeblair : relooking at the GP code again. jeblair, i believe you are saying that temporary online/offline should only register jobs not stop the executors. I think that's the real problem. | 22:02 |
| fungi | at some point we need to decide whether we're a free software project or a money-making corporate enterprise, because some of us are operating under one assumption and some possibly under the other | 22:02 |
| ttx | I'd keep governance.o.o as a plan B if for some reason we can't get the promised creds this week | 22:03 |
| jeblair | ttx: what about my concerns above? | 22:03 |
| clarkb | zaro: ya, can you update the code to just register rather than stop then test that? | 22:03 |
| ttx | jeblair: I may have missed them | 22:03 |
| zaro | clarkb, jeblair will try to put those events back in and retest. i think that's better than re-enabling the saveablelistener | 22:03 |
| clarkb | zaro: yes | 22:03 |
| jeblair | ttx: i'm moderately uncomfortable creating a publishing workflow with an endpoint that we don't control (and is generally unresponsive) using technologies we are going to migrate away from | 22:03 |
| jeblair | ttx: because we could easily end up with this being the last thing on this particular tech stack | 22:04 |
| ttx | jeblair: I think risk is limited. We could further mitigate risk by publishing in both places and internally linking to the governance.o.o page ? | 22:05 |
| fungi | stemming from questions around whether we'll be able to convince the foundation and tipit, once they have the code published, to work toward moving it into gerrit and onto infra-managed servers | 22:05 |
| *** gokrokve has quit IRC | 22:05 | |
| *** wenlock has joined #openstack-infra | 22:05 | |
| ttx | I kind of like the idea that the page with the TC roster would not get stale | 22:06 |
| ttx | so anything we can update directly from governance repo sounds like a good thing to do | 22:06 |
| fungi | i mean getting the source public has taken, what, 2 years at least? so wondering how fast we'll be able to get the rest done if ever | 22:07 |
| anteaya | ttx you said there is value to pushing to o.o/foundation/tech-committee | 22:07 |
| jeblair | fungi: the rest can happen in about 15 minutes. :) | 22:07 |
| anteaya | ttx what value do you feel tehre is? | 22:07 |
| fungi | jeblair: hah. depends on the ttl! | 22:07 |
| ttx | anteaya: discoverability for one | 22:07 |
| * anteaya is leaning goverance.o.o | 22:07 | |
| ttx | anteaya: and correctness. | 22:07 |
| anteaya | okay, fair | 22:08 |
| anteaya | I get discoverabiltiy | 22:08 |
| ttx | I don't want stale pages on www.o.o | 22:08 |
| anteaya | I'm missing the correctness point | 22:08 |
| ttx | since that's what 99% of people will read | 22:08 |
| anteaya | I was feeling that part of jeblair's concern publishing to www.o.o is staleness | 22:09 |
| ttx | anteaya: the page at https://www.openstack.org/foundation/tech-committee/ should be built from governance repo | 22:09 |
| anteaya | since we do't control the endpoint | 22:09 |
| ttx | if we publish elsewhere and not on www.o.o, then this page won't be geenrated from governance repo | 22:09 |
| anteaya | ttx oh, because there is already material at that endpoint | 22:09 |
| anteaya | I was missing that | 22:09 |
| anteaya | right | 22:10 |
| anteaya | I see | 22:10 |
| ttx | yes, you drill down from https://www.openstack.org/foundation | 22:10 |
| anteaya | but how fresh will it stay if we do't control the endpoint? | 22:10 |
| fungi | anteaya: i think the concern about lack of responsiveness was more about how to deal with it when it breaks | 22:10 |
| *** cody-somerville has quit IRC | 22:10 | |
| fungi | not as much about it being stale | 22:10 |
| ttx | anteaya: I don't get the "control the endpoint" | 22:10 |
| jeblair | and when i say unresponsive, i don't just mean about the www move, but also the fact that the plan was to set up the credentials we're talking about 10 months ago | 22:10 |
| ttx | we control what will end up there | 22:10 |
| *** pradk has quit IRC | 22:11 | |
| ttx | jeblair: I only recently took office :) | 22:11 |
| jeblair | ttx: we're locked into a tech stack that we can't change. we will soon get to a point where none of our publishing uses scp, but this will. | 22:11 |
| jeblair | ttx: you are empowered to change this? | 22:11 |
| anteaya | ttx I think jeblair is explaining the "control the endpoint" concerns | 22:12 |
| ttx | jeblair: I'm empowered to question it, for sure | 22:12 |
| fungi | so far my questioning it hasn't seemed to hasten anything | 22:12 |
| *** bknudson has quit IRC | 22:12 | |
| fungi | but i suppose i could have questioned it more... persistently? | 22:13 |
| ttx | sorry really need to jump in bed | 22:13 |
| fungi | night ttx! | 22:13 |
| jeblair | ttx: good night. thanks :) | 22:13 |
| anteaya | ttx night | 22:13 |
| ttx | fungi: hopefully will all be clearer after our first internal team meeting | 22:13 |
| ttx | and after some sleep | 22:13 |
| fungi | ttx: looking forward | 22:13 |
| *** david-lyle has quit IRC | 22:13 | |
| * jeblair would love it if the direction of the openstack project's web site didn't need an *internal* team meeting to be made clear | 22:14 | |
| *** cody-somerville has joined #openstack-infra | 22:15 | |
| fungi | my first action in the foundation engineering meeting will be to move for a change of venue to freenode | 22:15 |
| *** david-lyle has joined #openstack-infra | 22:15 | |
| anteaya | fungi: thank you | 22:15 |
| anteaya | good luck with that motion being adopted | 22:15 |
| jeblair | fungi: well, that's fine, but i won't be joining it | 22:15 |
| jeblair | fungi: because i don't think the foundation should be unilaterally engineering things | 22:16 |
| fungi | jeblair: nor i | 22:16 |
| *** atiwari has quit IRC | 22:16 | |
| jeblair | we have processes and venues in place within the project for that | 22:16 |
| *** bswartz has joined #openstack-infra | 22:17 | |
| fungi | i've already asserted to those involved that the infra teams priorities are to do what is best for the project, so if the priorities of the foundation aren't in sync with those then there's a miscommunication somewhere | 22:17 |
| anteaya | found out why recent third-party-requests are not giving us the requested information | 22:19 |
| anteaya | they are following http://www.joinfu.com/2014/02/setting-up-an-external-openstack-testing-system/#Getting_an_Upstream_Service_Account | 22:19 |
| anteaya | not ci.openstack.org | 22:19 |
| *** erlon has quit IRC | 22:19 | |
| *** emagana has quit IRC | 22:19 | |
| *** groknix has quit IRC | 22:20 | |
| anteaya | jaypipes: are you about? | 22:20 |
| fungi | of course they are! isn't www.joinfu.com where we maintain all our important documentation? | 22:20 |
| clarkb | ok the restarts of 06 and 07 have killed the green on the node graphs | 22:20 |
| *** emagana has joined #openstack-infra | 22:20 | |
| jeblair | jaypipes: your blog gets out of date very quickly | 22:20 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gearman-plugin: Fix job result not being sent back to gearman client https://review.openstack.org/130025 | 22:20 |
| anteaya | jaypipes: we have a problem, current requests for infra accounts are not starting off with required information: http://ci.openstack.org/third_party.html#requesting-a-service-account | 22:20 |
| jeblair | jaypipes: perhaps you could reconstruct it to do little more than point people to the right place | 22:20 |
| clarkb | and we seem to be killing the backlog so that is good | 22:20 |
| clarkb | and I missed the fun discussion | 22:21 |
| reed | the effects of google juice :/ | 22:21 |
| jeblair | jaypipes: and if our docs are too confusing, maybe you could improve them? :) | 22:21 |
| fungi | clarkb: there's always more fun discussion | 22:22 |
| fungi | clarkb: we can rehash it in said secret foundation cabal conspiracy meeting tomorrow | 22:22 |
| clarkb | now waiting on my d-g stack to get through the queue of doom so that I can recheck it with the nova revert in place | 22:22 |
| clarkb | but that should be the last of the grenade stuff I hope | 22:23 |
| zaro | clarkb, jeblair: ok. new PS for GP uses temp offline/online events again for node state changes. better than saveablelistner. sorry about that, i was rushing. | 22:23 |
| clarkb | zaro: is that tested too? | 22:23 |
| zaro | yes. | 22:23 |
| clarkb | and works under the offline node case? | 22:24 |
| *** mmedvede has joined #openstack-infra | 22:24 | |
| clarkb | awesome thanks | 22:24 |
| zaro | yes, ran the same tests documented in gerrit comments. | 22:24 |
| clarkb | cool will take a look | 22:24 |
| *** eharney has quit IRC | 22:24 | |
| *** emagana has quit IRC | 22:25 | |
| mtreinish | clarkb, fungi, jogo: do we have a bug open for a fedora heat image download going so slowly that it hits the jenkins job timeout? | 22:26 |
| clarkb | not that I am aware of | 22:26 |
| fungi | sounds new to me | 22:26 |
| mtreinish | clarkb, fungi: http://logs.openstack.org/45/129045/1/gate/gate-tempest-dsvm-neutron-heat-slow/7b888df/logs/devstacklog.txt.gz#_2014-10-21_21_20_56_436 | 22:26 |
| mtreinish | ok I'll open one then | 22:26 |
| clarkb | mtreinish: the fix is likely to make it so that devstack lists that image when we cache all the things | 22:28 |
| clarkb | then it iwll be local and don't have to wait? | 22:28 |
| *** yamamoto has joined #openstack-infra | 22:28 | |
| mtreinish | hmm, I thought it did that already | 22:28 |
| anteaya | there was one request and I replied to the email instructing the poster to follow the instructions for requesting an account | 22:28 |
| anteaya | so nothing this week, clarkb | 22:29 |
| mtreinish | clearly not based on that log though | 22:29 |
| clarkb | anteaya: ok good to know. let me know if anything else pops up though I have been trying to get better at tracking it myself | 22:29 |
| anteaya | clarkb: thank you for that | 22:29 |
| anteaya | next monday will be my last day before I am off | 22:30 |
| anteaya | so I will look again then | 22:30 |
| clarkb | anteaya: sounds good | 22:30 |
| clarkb | are you vacationing prior to summit? | 22:31 |
| anteaya | thanks | 22:31 |
| anteaya | yes | 22:31 |
| anteaya | some time on the Brittany coast | 22:31 |
| clarkb | cool have fun | 22:31 |
| anteaya | hoping the b and b doens't have a constant fan/generator/pump noise | 22:31 |
| anteaya | thanks | 22:31 |
| anteaya | then a week to myself after summit | 22:31 |
| * anteaya hopes for quiet | 22:32 | |
| anteaya | and I'm off for a walk | 22:32 |
| anteaya | back in a bit | 22:32 |
| *** yamamoto has quit IRC | 22:32 | |
| clarkb | krotscheck: into the stackforge projects... trying to do this review in chunks but I am still doing it :) | 22:34 |
| mtreinish | clarkb: oh, I think it might be because it was on stable/icehouse | 22:34 |
| krotscheck | clarkb: Thanks! | 22:34 |
| *** dims_ has joined #openstack-infra | 22:35 | |
| *** dims_ has quit IRC | 22:36 | |
| *** dims_ has joined #openstack-infra | 22:36 | |
| *** dims has quit IRC | 22:38 | |
| clarkb | krotscheck: and approved | 22:42 |
| krotscheck | clarkb: Thanks | 22:42 |
| openstackgerrit | Darragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Write cache to file on process exit https://review.openstack.org/117838 | 22:42 |
| *** nelsnelson has quit IRC | 22:44 | |
| jogo | mtreinish: hmm I *think* so | 22:45 |
| jogo | sounds familier | 22:45 |
| jogo | mtreinish: 1314786 | 22:45 |
| jogo | mtreinish: I64f371d2f2f30d5ea692b10c4d0b3c6cdaf9cc34 | 22:45 |
| mtreinish | clarkb, fungi: do you know off your top of the head where in (nodepool or config I guess) the images from devstack get cached | 22:45 |
| *** thedodd has quit IRC | 22:46 | |
| fungi | mtreinish: devstack has a list of files it needs and the nodepool prepare_devstack.sh script pre-caches them | 22:46 |
| fungi | mtreinish: in openstack-infra/project-config:nodepool/scripts | 22:46 |
| clarkb | it runs a devstack script that says this is the list of things you should care baout | 22:47 |
| mtreinish | fungi: ah, ok thanks | 22:47 |
| fungi | mtreinish: but ideally the change should be addition of a url in some file in devstack | 22:47 |
| * fungi doesn't recall which one just now | 22:47 | |
| *** dimtruck is now known as zz_dimtruck | 22:47 | |
| *** hdd has quit IRC | 22:47 | |
| mtreinish | fungi: I'm thinking it's a image version mismatch between icehouse and master in devstack | 22:48 |
| *** hdd has joined #openstack-infra | 22:48 | |
| *** gokrokve has joined #openstack-infra | 22:48 | |
| mtreinish | so I was going to look how hard it would be to cache the image from the stable branches too | 22:48 |
| *** jgrimm is now known as zz_jgrimm | 22:49 | |
| fungi | mtreinish: oh, good point. i bet right now we assume devstack master gives us a list of all the things we should cache | 22:49 |
| *** marun has quit IRC | 22:50 | |
| clarkb | we should do stable branches too already | 22:50 |
| fungi | mtreinish: is it just stable jobs which are timing out? | 22:50 |
| clarkb | it iterates over all the branches it can find | 22:50 |
| fungi | ahh, so it does | 22:50 |
| mtreinish | yeah just found that too | 22:51 |
| mtreinish | fungi: I only hit it on a stable job | 22:52 |
| openstackgerrit | Steve Martinelli proposed a change to openstack/requirements: Bump oslotest minimum requirements to 1.2.0 https://review.openstack.org/130058 | 22:52 |
| *** wenlock has quit IRC | 22:52 | |
| *** alexpilotti has quit IRC | 22:52 | |
| *** carl_baldwin has quit IRC | 22:53 | |
| *** mmedvede has quit IRC | 22:56 | |
| *** msabramo has quit IRC | 22:57 | |
| *** whoops has quit IRC | 22:58 | |
| mtreinish | but one sample really isn't useful for saying it's limited to stable | 22:59 |
| *** emagana has joined #openstack-infra | 22:59 | |
| *** baoli has quit IRC | 23:01 | |
| openstackgerrit | A change was merged to openstack-infra/project-config: Removed python26 from python-jobs https://review.openstack.org/128736 | 23:02 |
| *** dane_leblanc has quit IRC | 23:03 | |
| *** derekh has joined #openstack-infra | 23:05 | |
| *** MaxV has joined #openstack-infra | 23:06 | |
| clarkb | ok I have rechecked d-g grenade stack. it should work now | 23:06 |
| mtreinish | clarkb: awesome | 23:07 |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/project-config: Use sudo with apt-get in puppet jobs https://review.openstack.org/130059 | 23:07 |
| *** MarkAtwood has quit IRC | 23:08 | |
| *** MaxV has quit IRC | 23:10 | |
| *** msabramo has joined #openstack-infra | 23:10 | |
| *** wenlock has joined #openstack-infra | 23:11 | |
| *** frank_zhou has joined #openstack-infra | 23:12 | |
| openstackgerrit | Darragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Write cache to file on process exit https://review.openstack.org/117838 | 23:13 |
| *** doug-fish has left #openstack-infra | 23:13 | |
| *** dmsimard_away is now known as dmsimard | 23:13 | |
| clarkb | jeblair: fungi mordred any reason to not move centos7 test to check from experimental as a non voting test on devstack and d-g? | 23:15 |
| clarkb | dtroyer jhesketh and anteaya are all +X | 23:15 |
| clarkb | and we merged the change to add hpcloud centos7 nodes so are multiclouding that now | 23:15 |
| jeblair | clarkb: if it's multicloud, i'm okay. link? | 23:16 |
| clarkb | https://review.openstack.org/#/c/128793/1 | 23:16 |
| clarkb | it has a link to the multicloud nodepool change in the commit message | 23:16 |
| *** yamahata has joined #openstack-infra | 23:16 | |
| jeblair | clarkb: +2 | 23:17 |
| clarkb | cool I can approve | 23:17 |
| *** VijayTripathi has quit IRC | 23:17 | |
| *** _buzz has quit IRC | 23:18 | |
| ianw | ^ thanks ... i'll keep on it. glusterfs people are very interested, might be some of the first consumers | 23:18 |
| *** signed8bit has quit IRC | 23:19 | |
| *** marun has joined #openstack-infra | 23:19 | |
| *** frank_zhou has quit IRC | 23:19 | |
| *** frank_zhou has joined #openstack-infra | 23:19 | |
| *** erlon has joined #openstack-infra | 23:20 | |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/project-config: Define experimental check-heat-dsvm-functional-postgres https://review.openstack.org/129509 | 23:21 |
| clarkb | ianw: can't you gluster on ubuntu? | 23:22 |
| *** dangers is now known as dangers_away | 23:22 | |
| *** sarob has joined #openstack-infra | 23:23 | |
| clarkb | jeblair: https://review.openstack.org/#/c/130025/ appears to be what you indicated would be the proper way to solve this gearman thing | 23:24 |
| clarkb | it reregisters jobs whenever temporary* fires | 23:24 |
| clarkb | this should unregister on temp offlines and register when going back online (if say you manually do that) | 23:24 |
| ianw | clarkb: i'm no gluster expert, it is not entirely clear how straight-forward it is. i'm sure you can somehow. anyway i got a ping from some people about it, i am trying to encourage them to work upstream if possible | 23:24 |
| clarkb | ianw: ++ to upstream | 23:25 |
| jaypipes | jeblair, anteaya: ok, loud and clear. will do that. | 23:25 |
| *** yamamoto has joined #openstack-infra | 23:28 | |
| *** wenlock has quit IRC | 23:30 | |
| *** david-lyle has quit IRC | 23:30 | |
| *** bdpayne_ has joined #openstack-infra | 23:31 | |
| openstackgerrit | A change was merged to openstack-infra/project-config: Promote devstack centos7 to non-voting check job https://review.openstack.org/128793 | 23:32 |
| *** ihrachyshka has quit IRC | 23:32 | |
| *** bdpayne has quit IRC | 23:33 | |
| *** yamamoto has quit IRC | 23:33 | |
| *** dmsimard is now known as dmsimard_away | 23:35 | |
| *** mmedvede has joined #openstack-infra | 23:36 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/system-config: Rotate out nodepool logs every 8-hours https://review.openstack.org/130064 | 23:36 |
| clarkb | https://etherpad.openstack.org/p/py26-deprecation is my draft of the py26 depreaction warning email | 23:37 |
| clarkb | also a russellb twitter link is likely not best link | 23:39 |
| clarkb | but its still drafty | 23:39 |
| openstackgerrit | A change was merged to openstack-infra/gearman-plugin: Fix job result not being sent back to gearman client https://review.openstack.org/130025 | 23:39 |
| ianw | clarkb: do you just want suggestions made with direct edits? | 23:41 |
| clarkb | ianw: sure | 23:41 |
| ianw | clarkb: that would be all i would suggest. seems clear to me | 23:44 |
| clarkb | ianw: thanks those are good edits | 23:44 |
| *** zz_jgrimm is now known as jgrimm | 23:46 | |
| jeblair | clarkb: https://etherpad.openstack.org/p/juno-cross-project-future-of-python for link [0] ? | 23:47 |
| jeblair | clarkb: rest lgtm | 23:47 |
| *** frank_zhou has quit IRC | 23:48 | |
| clarkb | jeblair: much better link | 23:48 |
| clarkb | I will update | 23:49 |
| *** frank_zhou has joined #openstack-infra | 23:50 | |
| *** frank_zhou has quit IRC | 23:50 | |
| *** jamielenz has joined #openstack-infra | 23:52 | |
| *** jamielenz is now known as jamielennox | 23:52 | |
| *** sarob has quit IRC | 23:53 | |
| *** VijayTripathi has joined #openstack-infra | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
