| clarkb | yes I hope to review it as well, but I have done at least one pass on the code and overall it looks good | 00:00 |
|---|---|---|
| fungi | i've done several partial passes | 00:00 |
| openstackgerrit | A change was merged to openstack-infra/config: Add logging to the openstack-metering IRC channel https://review.openstack.org/41512 | 00:00 |
| fungi | and then taken breaks to reread python.org docs because my grasp of python is badly pedestrian | 00:00 |
| fungi | though at least now i have an improved grasp of how the @property, @setter et cetera decorators work underneath the covers | 00:02 |
| openstackgerrit | A change was merged to openstack-infra/config: Sort mailing lists in alpha order https://review.openstack.org/41756 | 00:03 |
| openstackgerrit | A change was merged to openstack-infra/config: Add new mysql_backup module. https://review.openstack.org/41790 | 00:03 |
| *** cyeoh has quit IRC | 00:04 | |
| *** cyeoh has joined #openstack-infra | 00:05 | |
| clarkb | mysqldump --all-databses grabs everything | 00:06 |
| clarkb | but the resulting file looks sane other than including a bunch of help tables and other mysql stuff | 00:06 |
| fungi | i am fine with the idea of restoring help tables | 00:07 |
| fungi | we can also filter those from the sql when restoring if we really care that much | 00:07 |
| clarkb | yup, and this way we don't miss stuff | 00:08 |
| fungi | i care far more about possibly missing stuff | 00:09 |
| fungi | mordred: i'm torn on my vote for 38177... a quick git grep shows stuff which got missed and other stuff which could be more consistent, but at the same time i'm inclined to just say go for it and then follow up with a smaller patch to catch the difference, because it's already pretty long and subject to major rebase hell if it sits | 00:11 |
| mordred | fungi: wow - you found stuff that got missed? can you give me your git grep command? | 00:11 |
| openstackgerrit | A change was merged to openstack-infra/gerritlib: Add option to query commit messages https://review.openstack.org/38865 | 00:11 |
| *** mkirk_ has quit IRC | 00:11 | |
| mordred | fungi: but yeah - I think there's actually existing inconsistencies | 00:12 |
| fungi | mordred: oh, wait, i may have misread my results now that i look | 00:12 |
| *** krtaylor has quit IRC | 00:12 | |
| mordred | fungi: yeah - there are github.com things that are legit | 00:12 |
| fungi | mordred: some of the things i thought were missed weren't our projects so that's okay | 00:12 |
| *** mkirk_ has joined #openstack-infra | 00:12 | |
| fungi | mordred: though git clone urls are a bit inconsistent | 00:12 |
| *** krtaylor has joined #openstack-infra | 00:12 | |
| fungi | so i think overall i'm +2 for now | 00:12 |
| jeblair | i'd really like to review 38177 | 00:12 |
| fungi | my +2 isn't final yet because i'm only about 50% through the actual patch | 00:13 |
| mordred | fungi: great call on looking for git clone - let me fix those up - | 00:13 |
| mordred | fungi: but I'll wait until you've got through before pushing again in case you find something else | 00:14 |
| fungi | mordred: particularly we could make protocol consistent as well as trailing .git | 00:14 |
| mordred | yes. I want to make all the protocol cases consistent | 00:14 |
| mordred | do you think we should do or not do trailing .git | 00:14 |
| mordred | wait - what port does git:// use? | 00:15 |
| fungi | i think the .git is probably more proper, but it's also longer so i could be convinced otherwise | 00:15 |
| mordred | (like, can it escape firewalls?) | 00:15 |
| fungi | 9418/tcp | 00:15 |
| mordred | I'm thining that perhaps in scripts like install_jenkins we might want to leave it https so that it's more likely to work for people? | 00:15 |
| fungi | so depends on how neckbearded and knuckle-dragging your firewall admins are | 00:16 |
| mordred | like, git clone https://git.openstack.org/openstack-infra/config | 00:16 |
| mordred | many of them are quite neckbearded and knuckle-dragging | 00:16 |
| *** ^d is now known as ^demon|away | 00:17 | |
| clarkb | I think we should still recommend https | 00:17 |
| fungi | as much as i hate to say it, https:// is probably more likely to work consistently for more people. we should also ask the hp and ibm people behind corporate proxies to test though | 00:17 |
| clarkb | because https will check certs and punch holes through firewalls more consistently | 00:17 |
| clarkb | git:// is good for people that know what they want | 00:17 |
| clarkb | mordred: while you are around and I remember, has the account comping happened for the new hpcloud openstack accoutn? | 00:19 |
| mordred | clarkb: yes | 00:19 |
| clarkb | mordred: so you heard back from them and we are all set to go? | 00:19 |
| mordred | clarkb: yes | 00:21 |
| clarkb | woot | 00:21 |
| mordred | fungi, clarkb so, you think we shoudl reference https:// everywhere in the docs and not git:// ? | 00:21 |
| *** openstack has joined #openstack-infra | 00:41 | |
| dtroyer | dstufft: hmmm…oh, exec without read only works on elf/a.out/binary-o-the-day format…so for python that sounds great. | 00:42 |
| dstufft | well these files could be anything | 00:42 |
| fungi | mordred: 38177 just made me realize that we don't currently serve our git repos there via apache | 00:42 |
| dstufft | The problem is that people include executable files inside of packages that they need to be able to execute | 00:43 |
| fungi | mordred: so more puppeting needed | 00:43 |
| mordred | fungi: oh. we should fix that | 00:43 |
| fungi | mordred: and do we do that via rewrites or a sub-path? | 00:43 |
| mordred | fungi: rewrites I believe? might be worth looking at the /p stuff in gerrit | 00:44 |
| fungi | for example we could do smart rewriting based on a search of /var/lib/git | 00:44 |
| mordred | butit's more complex there, because we're only service some parts of the urls | 00:44 |
| dtroyer | dstufft: yeah…I think we can still live with that though. | 00:45 |
| dstufft | ok | 00:45 |
| dtroyer | you've got to trust packages at some point | 00:45 |
| dstufft | I think it's about as reasonable as we're going to get with the current format | 00:45 |
| *** mberwanger has joined #openstack-infra | 00:45 | |
| dtroyer | yup. thanks | 00:46 |
| clarkb | lifeless: projects like zuul and the gearman plugin can use more eyes | 00:47 |
| lifeless | clarkb: url me up | 00:48 |
| clarkb | when I get home I will | 00:48 |
| *** jerryz has joined #openstack-infra | 00:49 | |
| lifeless | clarkb: thanks | 00:50 |
| *** dina_belova has joined #openstack-infra | 00:51 | |
| *** marun has quit IRC | 00:55 | |
| *** dina_belova has quit IRC | 00:56 | |
| fungi | mordred: pleia2: looking at git.o.o:/etc/httpd/conf.d/50-git.openstack.org.conf i think we can just set the docroot to /var/lib/git without any ill effects? | 00:56 |
| mordred | fungi: you can't serve git that way | 00:57 |
| *** mriedem has joined #openstack-infra | 00:57 | |
| *** kspear has joined #openstack-infra | 00:57 | |
| fungi | well, you can. but i gather it's suboptimal. need webdav? | 00:57 |
| lifeless | need to run a bidi channel to the git process | 00:58 |
| mordred | clarkb, lifeless: nodepool is probably a good thing to start looking at - especially since it's replacing a part of d-g and we'll be looking to plug the dib work in to it later | 00:58 |
| lifeless | mordred: oh, this is the thing my compute cache overlaps with ? :) | 00:58 |
| mordred | although I wouldn't go too mental on it right at the start, as it's starting from the existing d-g code | 00:58 |
| lifeless | mordred: anyhow, give me a url, I'll add it to my open-reviews-tabs thing | 00:58 |
| mordred | lifeless: potentially | 00:58 |
| mordred | lifeless: https://review.openstack.org/#/q/status:open+project:openstack-infra/nodepool,n,z | 00:59 |
| mordred | lifeless: tl;dr - we currently do all of this, but with a series of jenkins triggered scripts | 00:59 |
| mordred | lifeless: we're turning it in to a proper daemon, so that it can be smarter | 01:00 |
| *** mberwanger has quit IRC | 01:00 | |
| lifeless | yeah | 01:00 |
| lifeless | good thing | 01:00 |
| mordred | then, once that's up, we're going to talk about reworking the image creation part with dib/takeover/kexec | 01:00 |
| fungi | mordred: pleia2: looks like we can clone via https from the /cgit urls at least | 01:00 |
| mordred | fungi: really? | 01:00 |
| fungi | mordred: 'git clone https://git.openstack.org/cgit/openstack/nova' works | 01:01 |
| fungi | however git clone https://git.openstack.org/openstack/nova does not | 01:01 |
| mordred | yeah. I'd love it if the second thing worked so that it's closer to the git:: urls | 01:01 |
| lifeless | fungi: does that serve the smart server | 01:01 |
| lifeless | fungi: or just static files? | 01:01 |
| fungi | lifeless: dunno, it serves from the cgit cgi | 01:02 |
| fungi | mordred: perhaps we could build up an alias list based on the projects.yaml/cgitrepos content | 01:03 |
| mordred | fungi: ooh, in teresting | 01:03 |
| fungi | or use a rewrite rule with some -f lookups | 01:03 |
| mordred | fungi: I think we're alread running the git daemons for the git:// stuff, right? | 01:04 |
| fungi | mordred: yes | 01:04 |
| openstackgerrit | A change was merged to openstack-infra/git-review: Wrap long lines https://review.openstack.org/40800 | 01:04 |
| mordred | AliasMatch ^/p/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /var/lib/git/$1 | 01:04 |
| mordred | AliasMatch ^/p/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /var/lib/git/$1 | 01:05 |
| mordred | ScriptAlias /p/ /usr/lib/git-core/git-http-backend/ | 01:05 |
| fungi | how do we plug the git daemon into apache? webdav or something lighter? | 01:05 |
| mordred | there are the things we do for gerrit | 01:05 |
| mordred | we script alias /p/ to git-http-backend | 01:05 |
| fungi | ahh, yes that | 01:05 |
| mordred | but then we skip doing that and go back to plain file serving for the static things that can be done that way | 01:05 |
| fungi | so scriptaliasing / is going to be tough | 01:06 |
| mordred | I think as long as we pick up serving of /cgit/ first - and do the above with /p removed | 01:06 |
| mordred | what about script aliasing a different url but then rewrite ruling it? | 01:06 |
| mordred | like, do the /p | 01:06 |
| fungi | though i guess the ^/$ rewrite can also go before the scriptalias to git-http-backend | 01:07 |
| mordred | but then rewrite anything that's not /cgit to /p | 01:07 |
| clarkb | lifeless: https://review.openstack.org/#/q/status:open+project:openstack-infra/devstack-gate,n,z https://review.openstack.org/#/q/status:open+project:openstack-infra/gearman-plugin,n,z https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul,n,z | 01:07 |
| clarkb | lifeless: these are all fairly low traffic projects, but when we do need to make changes to them we tend to have very few eyes to look at them | 01:07 |
| *** blamar has joined #openstack-infra | 01:08 | |
| clarkb | and we often want to push things through fairly quickly to address one issue or another | 01:08 |
| *** markmcclain has quit IRC | 01:09 | |
| openstackgerrit | A change was merged to openstack-infra/config: Point to README.rst instead of .md for Devstack Gate. https://review.openstack.org/41495 | 01:14 |
| openstackgerrit | A change was merged to openstack-infra/gearman-plugin: moved python examples to jenkins wiki https://review.openstack.org/41307 | 01:14 |
| *** jerryz has quit IRC | 01:15 | |
| *** jerryz has joined #openstack-infra | 01:15 | |
| mordred | clarkb: I also pointed him at nodepool - since it wil eventually intersect with some stuff he's doing | 01:16 |
| *** hartsocks has left #openstack-infra | 01:16 | |
| openstackgerrit | A change was merged to openstack-infra/gerritbot: Make proposed and merged messages more terse https://review.openstack.org/40531 | 01:17 |
| openstackgerrit | A change was merged to openstack-infra/config: Properly load modules.conf https://review.openstack.org/41740 | 01:17 |
| mordred | clarkb: we shoudl get SpamapS to review it to - just so he can see what features heat needs to grow to do this for us :) | 01:17 |
| mordred | fungi, clarkb what's the git-review story? did we cut a release so it's safe to review the pbr changes? | 01:17 |
| fungi | mordred: i was seeing if anyone else wanted to merge that last cosmetic change for wrapping output | 01:17 |
| mordred | I just did | 01:17 |
| fungi | er, wanted to review | 01:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Clean up warning messages for asterisk https://review.openstack.org/41743 | 01:18 |
| fungi | ahh, cool. that's the one i wanted to tag then | 01:18 |
| mgagne | if I wish to install a shell script on cgit to filter commit message, where should I put it? | 01:18 |
| * fungi is simultaneously playing with git server apache config options and trying to get cacti upgraded, so somewhat tunnel-visiony | 01:19 | |
| mgagne | fungi: managed to add links for commit SHA1, bug (and blueprint) and gerrit id: http://centos01.calavera.ca/cgit/nova.git/commit/?h=stable/grizzly&id=5f12da37dd229f7afe596d4b0e9741834bf8ddaa | 01:21 |
| clarkb | mgagne: modules/cgit/files? | 01:21 |
| mgagne | clarkb: would it have any reusability for someone installing cgit on his own server with module from infra? | 01:22 |
| clarkb | I dunno. What sort of commit message filtering does it do? | 01:24 |
| mgagne | clarkb: the one mentioned about | 01:24 |
| fungi | mordred: i think i have a working httpd.conf adjustment for what we want, patch on the way | 01:24 |
| mgagne | clarkb: add links for commit SHA1, bug (and blueprint) and gerrit id | 01:24 |
| mgagne | clarkb: http://paste.openstack.org/show/44191/ | 01:25 |
| clarkb | mgagne: sounds like ti would be generally useful for anyone replicating to cgit from gerrit | 01:25 |
| openstackgerrit | benley proposed a change to openstack-infra/jenkins-job-builder: Add display-name job property. https://review.openstack.org/41828 | 01:30 |
| kspear | is horizon still breaking the gate? zuul status is a little confusing | 01:31 |
| clarkb | kspear: how was horizon breaking the gate? is this related to the tempest test that attempts to log in? | 01:33 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Serve repos via git+https outside of cgit https://review.openstack.org/42044 | 01:34 |
| fungi | mordred: pleia2: ^ | 01:34 |
| kspear | clarkb: yes. i'm presuming it was triggered by horizon? | 01:35 |
| mordred | oh my holy crap - I cleared my review queue | 01:36 |
| clarkb | mordred: wow | 01:36 |
| clarkb | kspear: so looking at the last 4 hours gate-tempest-devstack-vm-full passed 10 times and failed 9 times in the gate | 01:37 |
| clarkb | kspear: that is a very high fail rate so something is probably bothering it | 01:37 |
| clarkb | kspear: http://logs.openstack.org/12/41612/1/gate/gate-tempest-devstack-vm-full/53036d0/console.html.gz is the most recent fail in the gate that I can find | 01:38 |
| kspear | i commented on https://bugs.launchpad.net/tempest/+bug/1212405 earlier | 01:38 |
| clarkb | kspear: and it looks to be failing the dashboard tempest test. so I think horizon is still breaking things | 01:38 |
| uvirtbot | Launchpad bug 1212405 in tempest "test_basic_scenario fails in the gate with a 500 error" [Undecided,New] | 01:38 |
| fungi | clarkb: did you work on the last cacti server setup? debconf seems to have decided that when i told it not to mess with the database configuration that it needed to get rid of what was already there. even purging cacti (after backing up the rrd files) and reapplying the puppet manifest seems to have left it not knowing what db credentials to use | 01:45 |
| fungi | "FATAL: Cannot connect to MySQL server on ''. Please make sure you have specified a valid MySQL database name in 'include/config.php'" | 01:45 |
| *** fifieldt has joined #openstack-infra | 01:49 | |
| openstackgerrit | A change was merged to openstack-infra/config: Don't compress very, very recent log files https://review.openstack.org/41544 | 01:50 |
| openstackgerrit | A change was merged to openstack-infra/config: Ensure python pip is installed before zuul https://review.openstack.org/39611 | 01:51 |
| clarkb | fungi: I did not. jeblair did iirc | 01:51 |
| *** dina_belova has joined #openstack-infra | 01:51 | |
| openstackgerrit | A change was merged to openstack-infra/config: Start import process for clouddocs-maven-plugin. https://review.openstack.org/39337 | 01:52 |
| fungi | hrm. well i worry that we lack sufficient puppetage to set this up from scratch (data aside) | 01:52 |
| fungi | i'll see if i can run down the bits it's missing | 01:52 |
| *** dina_belova has quit IRC | 01:56 | |
| *** yaguang has joined #openstack-infra | 01:58 | |
| *** colinmcnamara has joined #openstack-infra | 02:00 | |
| *** yamahata has quit IRC | 02:01 | |
| mordred | don't people prefer icinga or something these days? | 02:01 |
| * mordred mentions because maybe the new hotness will actually get security updates | 02:01 | |
| *** yamahata has joined #openstack-infra | 02:02 | |
| clarkb | there are so many options | 02:02 |
| mordred | oh - wait - no, icinga is the new nagios | 02:02 |
| mgagne | mordred: from my experience, icinga UI is awful compared to Nagios | 02:02 |
| clarkb | mordred: a common one is collectd + graphite | 02:02 |
| clarkb | mordred: but then you have to know how to make graphite work | 02:02 |
| mordred | yeah - I mean, I know there are new fangled things, but I don't know what they are | 02:02 |
| clarkb | but my understanding is people have done that for collectd data | 02:02 |
| mordred | collectd. that's what I was thinking of | 02:02 |
| mordred | clarkb: srrsly. I only have 4 things in my reviewable pane | 02:03 |
| clarkb | mordred: I envy you | 02:03 |
| clarkb | tomorrow is my big review day once I get etherpad setup up with the new backup stuff which shouldn't take long | 02:03 |
| clarkb | mordred: did the star trick help? | 02:04 |
| fungi | icinga is the new netsaint ;) | 02:04 |
| clarkb | or is that coincidental? | 02:04 |
| mordred | clarkb: yes. immensely | 02:04 |
| mordred | it helped me set up a todo list and then check things off ot it | 02:04 |
| *** adalbas has quit IRC | 02:08 | |
| *** Ryan_Lane has joined #openstack-infra | 02:10 | |
| *** mriedem has quit IRC | 02:12 | |
| *** marun has joined #openstack-infra | 02:14 | |
| *** colinmcnamara has quit IRC | 02:18 | |
| *** marun has quit IRC | 02:20 | |
| *** marun has joined #openstack-infra | 02:21 | |
| *** wenlock has joined #openstack-infra | 02:23 | |
| *** erfanian has joined #openstack-infra | 02:24 | |
| *** wenlock has quit IRC | 02:25 | |
| annegentle | mordred: oh WAHOO for the import process for clouddocs-maven-plugin! | 02:25 |
| clarkb | annegentle: I started it, was hoping to get some feedback on it from you folks before we hit the go button | 02:27 |
| clarkb | oh mordred merged it. thats cool :) | 02:27 |
| clarkb | there are still a few minor things we will need to work through but nothing major | 02:27 |
| *** wenlock has joined #openstack-infra | 02:27 | |
| wenlock | hi, going to try some config install tonight... first time trying :D | 02:28 |
| *** marun has quit IRC | 02:29 | |
| *** marun has joined #openstack-infra | 02:31 | |
| clarkb | wenlock: good luck! | 02:31 |
| wenlock | thanks clark, i was thinking of using a bitnami image, is that recommended, or should i stick with vanilla ubuntu? | 02:32 |
| *** ryanpetrello has joined #openstack-infra | 02:33 | |
| clarkb | wenlock: for openstack VMs? I have no clue. We are more focused on running the infrastructure for the project and no openstack infrastructure | 02:33 |
| mordred | wenlock: I've never used a bitnami image myself - I always do stock ubuntu | 02:33 |
| mordred | wenlock: and welcome! | 02:34 |
| wenlock | thanks | 02:34 |
| *** dkehn has quit IRC | 02:36 | |
| *** rcleere has joined #openstack-infra | 02:36 | |
| *** dkehn has joined #openstack-infra | 02:46 | |
| *** nati_ueno has quit IRC | 02:51 | |
| *** dina_belova has joined #openstack-infra | 02:52 | |
| *** jfriedly has quit IRC | 02:54 | |
| *** melwitt has quit IRC | 02:57 | |
| *** dina_belova has quit IRC | 02:58 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Serve repos via git+https outside of cgit https://review.openstack.org/42044 | 03:05 |
| fungi | also adjusts the webui to match ^ | 03:05 |
| fungi | the cgit changelog says they included displaying git notes as of 0.9, but i don't actually see it doing so | 03:09 |
| *** zehicle has quit IRC | 03:11 | |
| *** dkliban has joined #openstack-infra | 03:21 | |
| openstackgerrit | Kui Shi proposed a change to openstack-dev/hacking: H233 regex can not match some print operator https://review.openstack.org/41355 | 03:22 |
| *** SergeyLukjanov has joined #openstack-infra | 03:22 | |
| *** tjones has quit IRC | 03:22 | |
| *** HenryG has quit IRC | 03:29 | |
| pleia2 | fungi: patches look good, I will need to set up a /static as well for theme images+css, I can just do an Alias for this after your change and it should be ok | 03:45 |
| *** dina_belova has joined #openstack-infra | 03:53 | |
| *** vipul-away is now known as vipul | 03:56 | |
| *** dina_belova has quit IRC | 03:57 | |
| *** dkliban has quit IRC | 04:00 | |
| *** SergeyLukjanov has quit IRC | 04:13 | |
| *** ^demon|away has quit IRC | 04:17 | |
| *** rcleere has quit IRC | 04:23 | |
| SpamapS | is the neutron gate like, totally borked? | 04:27 |
| SpamapS | been watching this 2 line change try to wend its way through for 6 hours now https://review.openstack.org/#/c/41860/ | 04:27 |
| *** erfanian has quit IRC | 04:28 | |
| *** jerryz has left #openstack-infra | 04:30 | |
| *** vogxn has joined #openstack-infra | 04:30 | |
| clarkb | SpamapS: apparently... also regular tempest has an almost 50% failure rate right now due to some horizon thing | 04:30 |
| SpamapS | osssuuummm | 04:31 |
| clarkb | or it did in the 4 hour time frame that I checked earlier | 04:31 |
| clarkb | but ya osum | 04:31 |
| *** marun has quit IRC | 04:34 | |
| *** Ryan_Lane has quit IRC | 04:35 | |
| *** ryanpetrello has quit IRC | 04:47 | |
| *** ryanpetrello has joined #openstack-infra | 04:48 | |
| *** vipul is now known as vipul-away | 04:50 | |
| *** dina_belova has joined #openstack-infra | 04:53 | |
| *** ryanpetr_ has joined #openstack-infra | 04:54 | |
| *** ryanpetrello has quit IRC | 04:55 | |
| *** dkliban has joined #openstack-infra | 04:56 | |
| *** afazekas has joined #openstack-infra | 04:56 | |
| *** wenlock has quit IRC | 04:58 | |
| *** dina_belova has quit IRC | 04:58 | |
| *** ryanpetrello has joined #openstack-infra | 04:58 | |
| *** ryanpetr_ has quit IRC | 05:00 | |
| *** sdake_ has quit IRC | 05:07 | |
| *** dina_belova has joined #openstack-infra | 05:13 | |
| *** dina_belova has quit IRC | 05:14 | |
| *** afazekas has quit IRC | 05:17 | |
| *** dina_belova has joined #openstack-infra | 05:19 | |
| *** nicedice_ has quit IRC | 05:21 | |
| *** ryanpetrello has quit IRC | 05:21 | |
| *** ^demon|away has joined #openstack-infra | 05:28 | |
| *** dina_belova has quit IRC | 05:28 | |
| *** fungi has quit IRC | 05:31 | |
| *** ^demon|away has quit IRC | 05:33 | |
| *** nayward has joined #openstack-infra | 05:33 | |
| *** ^demon|away has joined #openstack-infra | 05:40 | |
| *** ^demon|away has joined #openstack-infra | 05:40 | |
| *** thomasbiege has joined #openstack-infra | 05:43 | |
| *** ^demon|away has quit IRC | 05:45 | |
| *** fungi has joined #openstack-infra | 05:45 | |
| *** thomasbiege has quit IRC | 05:47 | |
| *** jjmb has joined #openstack-infra | 05:47 | |
| *** vogxn has quit IRC | 05:49 | |
| *** vogxn1 has joined #openstack-infra | 05:49 | |
| *** vogxn has joined #openstack-infra | 05:51 | |
| *** vogxn has quit IRC | 05:52 | |
| *** nati_ueno has joined #openstack-infra | 05:58 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Fix intermittent jenkins plugin build failure https://review.openstack.org/42062 | 05:59 |
| *** nati_ueno has quit IRC | 06:00 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Fix intermittent jenkins plugin build failure https://review.openstack.org/42062 | 06:05 |
| *** afazekas has joined #openstack-infra | 06:07 | |
| *** vogxn has joined #openstack-infra | 06:10 | |
| *** marun has joined #openstack-infra | 06:15 | |
| *** afazekas has quit IRC | 06:19 | |
| *** vogxn has quit IRC | 06:24 | |
| *** vogxn has joined #openstack-infra | 06:26 | |
| *** locke105 has quit IRC | 06:29 | |
| *** locke105 has joined #openstack-infra | 06:35 | |
| *** afazekas has joined #openstack-infra | 06:40 | |
| *** marun has quit IRC | 06:47 | |
| *** odyssey4me has joined #openstack-infra | 06:57 | |
| *** jerryz has joined #openstack-infra | 07:00 | |
| *** thomasbiege has joined #openstack-infra | 07:06 | |
| *** bnemec has quit IRC | 07:16 | |
| *** cybertron has joined #openstack-infra | 07:19 | |
| openstackgerrit | Petr Blaho proposed a change to openstack-infra/config: Adds Jenkins jobs for python-tuskarclient https://review.openstack.org/41887 | 07:20 |
| *** Daviey has quit IRC | 07:22 | |
| *** jerryz has quit IRC | 07:23 | |
| *** thomasbiege has quit IRC | 07:23 | |
| *** thomasbiege has joined #openstack-infra | 07:27 | |
| *** Daviey has joined #openstack-infra | 07:28 | |
| *** thomasbiege has quit IRC | 07:29 | |
| *** thomasbiege has joined #openstack-infra | 07:30 | |
| *** mjfork has quit IRC | 07:30 | |
| *** mjfork has joined #openstack-infra | 07:31 | |
| *** sungju_ has joined #openstack-infra | 07:35 | |
| *** sungju_ has quit IRC | 07:35 | |
| *** dina_belova has joined #openstack-infra | 07:40 | |
| *** dina_belova has quit IRC | 07:40 | |
| *** dina_belova has joined #openstack-infra | 07:41 | |
| openstackgerrit | Serg Melikyan proposed a change to openstack-infra/config: Added murano-common project https://review.openstack.org/41634 | 07:42 |
| *** thomasbiege has quit IRC | 07:47 | |
| *** yaguang has quit IRC | 07:49 | |
| openstackgerrit | A change was merged to openstack-dev/pbr: Sync requirements with global requirements https://review.openstack.org/40870 | 07:55 |
| *** yaguang has joined #openstack-infra | 07:56 | |
| *** jpich has joined #openstack-infra | 08:07 | |
| *** derekh has joined #openstack-infra | 08:09 | |
| *** mkerrin has quit IRC | 08:09 | |
| *** mkerrin has joined #openstack-infra | 08:17 | |
| *** vkuz has joined #openstack-infra | 08:23 | |
| *** vogxn1 has joined #openstack-infra | 08:26 | |
| *** mirrorbox has joined #openstack-infra | 08:26 | |
| *** Fengqian has joined #openstack-infra | 08:28 | |
| soren | jeblair: Sad indeed. :( | 08:28 |
| Fengqian | hello, everyone;) | 08:28 |
| vkuz | Hi. I signed USG CLA and System CLA by mistake. How can I cancel them? | 08:28 |
| *** vogxn has quit IRC | 08:29 | |
| Fengqian | I met a problem here. jenkins report a error toe me"StorageBadVersion: Need at least MongoDB 2.2" | 08:30 |
| Fengqian | did anybody met before? | 08:30 |
| Fengqian | and i can build succeed on my local machine | 08:31 |
| Fengqian | You may see the following link for details, http://logs.openstack.org/32/39432/6/check/gate-ceilometer-python27/f685f7f/console.html.gz | 08:33 |
| *** luisg has quit IRC | 08:34 | |
| *** luisg has joined #openstack-infra | 08:35 | |
| *** thomasbiege has joined #openstack-infra | 08:36 | |
| jpich | vkuz: This has to be done manually, someone from the infra team can help you when they come online (probably in a few hours) | 08:39 |
| Fengqian | oh, got it:), thx a lot | 08:40 |
| *** thomasbiege has quit IRC | 08:41 | |
| openstackgerrit | Kui Shi proposed a change to openstack-dev/hacking: H233 regex can not match some print operator https://review.openstack.org/41355 | 08:45 |
| BobBall_Away | mordred: I uploaded a new version of the patch - which now works for me (for some definition of 'works' - I still need to fix nova in my case) - but it seems I had a different base to you, so my changes were rebased. It might mean you have to rebase if you pull my update down | 08:50 |
| openstackgerrit | afazekas proposed a change to openstack-infra/devstack-gate: Skip devstack/exercises by default https://review.openstack.org/42082 | 08:51 |
| BobBall_Away | mordred: The only change was to move the definition of GLOBAL_VENV to before we source anything from lib/ (which was using get_python_exec_prefix or whatever it is) | 08:52 |
| *** dina_belova has quit IRC | 08:52 | |
| *** dina_belova has joined #openstack-infra | 08:53 | |
| vkuz | jpich: Who enters the infra team? and where should I write about my problem? | 08:55 |
| jpich | vkuz: Here is fine, they usually check the backlog when they come online so they'll get back to you then | 08:56 |
| *** BobBall_Away is now known as BobBall | 08:56 | |
| jpich | vkuz: They'll probably need your gerrit id actually, if you provide it now that'll speed things up | 08:57 |
| BobBall | Not sure who has the power to do that - but mordred / fungi / etc will be able to help and this will make their history flag so they know that vkuz is looking for them! ;) | 08:57 |
| *** vogxn1 has quit IRC | 08:58 | |
| *** vogxn has joined #openstack-infra | 09:02 | |
| *** dina_belova has quit IRC | 09:09 | |
| *** boris-42 has joined #openstack-infra | 09:14 | |
| *** SergeyLukjanov has joined #openstack-infra | 09:26 | |
| *** vogxn has quit IRC | 09:35 | |
| *** Fengqian has quit IRC | 09:49 | |
| *** sungju_ has joined #openstack-infra | 09:57 | |
| *** sungju_ has quit IRC | 09:58 | |
| *** psedlak has joined #openstack-infra | 10:02 | |
| *** nayward has quit IRC | 10:04 | |
| *** dina_belova has joined #openstack-infra | 10:09 | |
| openstackgerrit | afazekas proposed a change to openstack-infra/devstack-gate: Skip devstack/exercises by default https://review.openstack.org/42082 | 10:11 |
| *** dina_belova has quit IRC | 10:14 | |
| *** dina_belova has joined #openstack-infra | 10:19 | |
| *** yaguang has quit IRC | 10:20 | |
| *** cybertron has quit IRC | 10:21 | |
| *** bnemec has joined #openstack-infra | 10:22 | |
| *** Ryan_Lane has joined #openstack-infra | 10:23 | |
| *** dina_belova has quit IRC | 10:25 | |
| *** fbo_away is now known as fbo | 10:31 | |
| openstackgerrit | afazekas proposed a change to openstack-infra/devstack-gate: Preserve sudo permissions of the stack user https://review.openstack.org/42101 | 10:31 |
| *** blamar has quit IRC | 10:32 | |
| *** weshay has joined #openstack-infra | 10:58 | |
| *** dprince has joined #openstack-infra | 10:59 | |
| *** EntropyWorks has quit IRC | 11:05 | |
| *** EntropyWorks has joined #openstack-infra | 11:05 | |
| *** dina_belova has joined #openstack-infra | 11:05 | |
| *** vogxn has joined #openstack-infra | 11:10 | |
| *** dina_belova has quit IRC | 11:12 | |
| *** Guest71730 is now known as mdurnosvistov | 11:17 | |
| *** mdurnosvistov has left #openstack-infra | 11:18 | |
| *** fbo is now known as fbo_away | 11:23 | |
| *** nayward has joined #openstack-infra | 11:24 | |
| *** vogxn has quit IRC | 11:24 | |
| *** dina_belova has joined #openstack-infra | 11:24 | |
| *** pcm_ has joined #openstack-infra | 11:26 | |
| *** chuckieb has joined #openstack-infra | 11:32 | |
| *** chuckieb has quit IRC | 11:32 | |
| *** chuckieb has joined #openstack-infra | 11:39 | |
| *** dina_belova has quit IRC | 11:40 | |
| *** rfolco has joined #openstack-infra | 11:40 | |
| *** dprince has quit IRC | 11:40 | |
| *** SergeyLukjanov has quit IRC | 11:41 | |
| *** weshay has quit IRC | 11:42 | |
| *** nayward has quit IRC | 11:44 | |
| *** ArxCruz has joined #openstack-infra | 11:56 | |
| *** vogxn has joined #openstack-infra | 11:59 | |
| *** fifieldt has quit IRC | 12:03 | |
| *** vogxn has quit IRC | 12:03 | |
| vkuz | Is there anyone out from infra team? | 12:06 |
| *** woodspa has joined #openstack-infra | 12:07 | |
| *** vogxn has joined #openstack-infra | 12:09 | |
| fungi | vkuz: i'm in the process of waking up and looking at the scrollback | 12:13 |
| fungi | ahh, clearing wrong clas... doing that now | 12:14 |
| fungi | vkuz: done | 12:15 |
| fungi | we need to find a way to make those agreements look even more scary | 12:15 |
| vkuz | fungi: Thank you very much! | 12:17 |
| fungi | vkuz: you're very welcome | 12:18 |
| *** adalbas has joined #openstack-infra | 12:23 | |
| *** sandywalsh has quit IRC | 12:31 | |
| *** w_ has joined #openstack-infra | 12:43 | |
| *** olaph has quit IRC | 12:45 | |
| *** ruhe has joined #openstack-infra | 12:46 | |
| *** sandywalsh has joined #openstack-infra | 12:46 | |
| *** dprince has joined #openstack-infra | 12:47 | |
| dprince | fungi: who is the best person to ask about DNS? I'd like to update the record for SmokeStack if possible. | 12:50 |
| *** HenryG has joined #openstack-infra | 12:55 | |
| *** markmc has joined #openstack-infra | 12:59 | |
| openstackgerrit | afazekas proposed a change to openstack-infra/devstack-gate: Enable Load Balancer as a Service https://review.openstack.org/42119 | 13:00 |
| *** weshay has joined #openstack-infra | 13:02 | |
| fungi | dprince: smokestack.openstack.org? i can change it to whatever you need | 13:02 |
| *** anteaya has joined #openstack-infra | 13:04 | |
| dprince | fungi: my new IP is https://50.56.5.192/ (a load balancer) | 13:06 |
| dprince | fungi: the old server has been in place for like over 2 years now so its time to finally migrate over to RAX next-gen | 13:07 |
| dprince | fungi: it is my last server running on the old RAX infrastructure | 13:07 |
| fungi | dprince: updated, but the ttl on the old record was an hour so could take some time to expire out of caches | 13:08 |
| dprince | fungi: Cool. Thanks sir. While I've got you let me ask this... who is doing infra certs? Reason I ask is I usually just create my own... but it would be cool if it were signed, etc so people that were interested don't have to install a custom one. | 13:09 |
| dprince | fungi: this would just be for SSL | 13:09 |
| *** markmcclain has joined #openstack-infra | 13:13 | |
| fungi | dprince: jeblair has been handling those, and we'd probably need to discuss any potential implications of the foundation providing x.509 certs for unofficial systems (we don't do that currently, to my knowledge, and could be seen as a bit stronger endorsement than a simple dns record) | 13:13 |
| dprince | fungi: understood. I'll mention to jeblair. | 13:14 |
| *** burt has joined #openstack-infra | 13:17 | |
| *** nayward has joined #openstack-infra | 13:17 | |
| *** SergeyLukjanov has joined #openstack-infra | 13:21 | |
| *** dina_belova has joined #openstack-infra | 13:22 | |
| *** ruhe has quit IRC | 13:22 | |
| *** dina_belova has quit IRC | 13:22 | |
| *** SergeyLukjanov has quit IRC | 13:23 | |
| *** dina_belova has joined #openstack-infra | 13:23 | |
| openstackgerrit | A change was merged to openstack/requirements: add pycadf library https://review.openstack.org/40926 | 13:26 |
| *** SergeyLukjanov has joined #openstack-infra | 13:28 | |
| *** mriedem has joined #openstack-infra | 13:28 | |
| *** markmcclain has quit IRC | 13:29 | |
| openstackgerrit | A change was merged to openstack/requirements: Bump hacking to 0.7 globally https://review.openstack.org/41523 | 13:34 |
| *** changbl has quit IRC | 13:39 | |
| *** krtaylor has quit IRC | 13:48 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/reviewday: Switch smoker URL to ssl. Disable ssl validation. https://review.openstack.org/42136 | 13:49 |
| *** prad has joined #openstack-infra | 13:52 | |
| BobBall | how do you submit multiple, non-dependant changes to a BP? | 13:57 |
| openstackgerrit | A change was merged to openstack-infra/reviewday: Switch smoker URL to ssl. Disable ssl validation. https://review.openstack.org/42136 | 13:57 |
| *** pentameter has joined #openstack-infra | 13:59 | |
| *** blamar has joined #openstack-infra | 14:00 | |
| *** wenlock has joined #openstack-infra | 14:06 | |
| *** blamar has quit IRC | 14:06 | |
| *** blamar has joined #openstack-infra | 14:06 | |
| *** mrodden has joined #openstack-infra | 14:07 | |
| *** markmc has quit IRC | 14:07 | |
| wenlock | hi does anyone have a sample setup for hiera yml config file that can be used on a dev system? | 14:07 |
| *** mkerrin has quit IRC | 14:07 | |
| mordred | wenlock: you know - that would be an excellent idea | 14:07 |
| wenlock | i can try to hack at one, but im just a little lost atm with what it should look like | 14:08 |
| mordred | wenlock: one sec - lemme show you a small example | 14:08 |
| wenlock | cool | 14:09 |
| *** HenryG_ has joined #openstack-infra | 14:09 | |
| mordred | fungi: ^^ that's probably a thing we should capture in a sample doc at some point | 14:09 |
| *** rnirmal has joined #openstack-infra | 14:09 | |
| Alex_Gaynor | mordred: new tox with -e support is out | 14:09 |
| mordred | Alex_Gaynor: zomgzomgzom | 14:09 |
| mordred | zomgzomgzomg | 14:09 |
| * mordred now bouces | 14:09 | |
| * Alex_Gaynor thinks mordred may be a little^Wlot too excited | 14:09 | |
| Alex_Gaynor | WHat's the use case for this again? | 14:10 |
| mordred | I'm about to piss off EVERYBODY in openstack | 14:10 |
| *** vogxn has quit IRC | 14:10 | |
| openstackgerrit | Roman Podolyaka proposed a change to openstack-infra/config: WIP: Run Nova DB API tests on MySQL and PostgreSQL https://review.openstack.org/42142 | 14:10 |
| *** SergeyLukjanov has quit IRC | 14:10 | |
| Alex_Gaynor | mordred: for the greater good, I'm sure | 14:10 |
| mordred | Alex_Gaynor: the use case is that it doesn't run sdist every time | 14:10 |
| wenlock | ooh.... gota leave.... im late , be back on in an hr mordred, i'll ping you again later | 14:10 |
| Alex_Gaynor | ah, yays | 14:10 |
| BobBall | mordred: btw - I think based on the latest venv patch you uploaded that it WFM - although horizon doesn't run because I'm messing in xenserver-core (and I see that's the problem in the jenkins logs) | 14:10 |
| mordred | Alex_Gaynor: which is the thing that was keeping our run_Tests.sh from using tox for things | 14:11 |
| *** wenlock has quit IRC | 14:11 | |
| mordred | but if we can enforce tox >= 1.6 | 14:11 |
| mordred | then we can configure it to do develop mode | 14:11 |
| Alex_Gaynor | +1 on pissing everyone off then :) | 14:11 |
| mordred | yup! | 14:11 |
| mordred | it'll make things better | 14:11 |
| jeblair | mordred: to be fair, cacti released a security update; it's jus ubuntu didn't package it. if we want to run without packages, or use a different distro, we can do that with cacti. | 14:12 |
| *** HenryG has quit IRC | 14:12 | |
| mordred | jeblair: awesome. I'm fine with cacti for the record | 14:12 |
| mordred | just thought i'd bring up whatever the bearded hotness was when fungi was working on making packages | 14:12 |
| mordred | in case it was less work | 14:13 |
| jeblair | mordred: much of the new hotness isn't nearly as good as cacti, and what we have has about a person-month worth of work in configuring the templates (sadly, cacti isn't as good as our cacti, even), so it's not a thing to do lightly. | 14:13 |
| *** mkerrin has joined #openstack-infra | 14:13 | |
| mordred | jeblair: nod | 14:13 |
| mordred | jeblair: also, good morning! | 14:13 |
| jeblair | fungi: is there something i can do to help with cacti? | 14:14 |
| jeblair | mordred: good morning to you! | 14:14 |
| jeblair | afazekas: first -- yay! i love "don't run exercises!" | 14:14 |
| pleia2 | progress is going slowly along on the bug (thanks fungi!) https://bugs.launchpad.net/ubuntu/+source/cacti/+bug/1210822 | 14:14 |
| uvirtbot | Launchpad bug 1210822 in cacti "Please import 0.8.8b+dfsg-2 from Debian and backport security fixes to 12.04 LTS" [Undecided,Confirmed] | 14:14 |
| pleia2 | so hopefully not too long now | 14:15 |
| fungi | jeblair: i built the package yesterday on a throwaway precise vm and then upgraded us to it | 14:15 |
| jeblair | afazekas: but second -- the sudo permissions are removed to make sure that rootwrap is actually tested; otherwise, services can (and have in the past!) just run "sudo dosomethingwithoutrootwrap" and that would work. :( | 14:16 |
| fungi | however in the process of dpkg -i the deb i copied over, the maintscripts decided to overwrite the cacti database password, and we aren't currently puppeting that | 14:16 |
| afazekas | jeblair: :) | 14:16 |
| fungi | so we ended up with a ~7 hour hole in the graph | 14:16 |
| fungi | s | 14:16 |
| fungi | but should all be working now | 14:16 |
| pleia2 | great (except for the hole :)) | 14:17 |
| fungi | yeah, i felt bad | 14:17 |
| afazekas | jeblair: can we create a tempest user and give him sudo permission ? | 14:17 |
| fungi | now i'm over it | 14:17 |
| jeblair | fungi: oops. i bet we could puppet that. there's still a bunch of stuff that may be a little hard to puppet (like the initial import of the graph templates) | 14:17 |
| pleia2 | yeah, it's not a big deal | 14:17 |
| *** markmcclain has joined #openstack-infra | 14:17 | |
| jeblair | afazekas: the jenkins user retains sudo permission -- can you use that? | 14:17 |
| fungi | jeblair: yes, i put puppeting the /etc/cacti/debian.php on today's wishlist | 14:17 |
| afazekas | jeblair: I'll try it | 14:17 |
| jeblair | afazekas: also, you could add sudo back after running the tests | 14:18 |
| jeblair | afazekas: (i'm just throwing out ideas because i don't actually know where you need sudo) | 14:18 |
| afazekas | jeblair: I guess I just need to exclude the sudo from the tempest start | 14:19 |
| afazekas | https://review.openstack.org/#/c/40435/ I will need here | 14:19 |
| jeblair | afazekas: ok, yeah, I think that should be fine | 14:19 |
| afazekas | ip netns exec needs it and the iptables and several not implemented part .. | 14:20 |
| jeblair | afazekas: got it, thanks. i can't think of why the tempest run uses sudo. it's worth a try just removing that. | 14:21 |
| afazekas | ovs and volume related things will came | 14:21 |
| *** SergeyLukjanov has joined #openstack-infra | 14:22 | |
| *** ruhe has joined #openstack-infra | 14:22 | |
| *** ruhe has quit IRC | 14:25 | |
| *** _TheDodd_ has joined #openstack-infra | 14:28 | |
| *** adalbas has quit IRC | 14:30 | |
| afazekas | jeblair: is jenkins in the stack users's default group ? | 14:31 |
| *** rcleere has joined #openstack-infra | 14:32 | |
| jeblair | afazekas: i don't think so | 14:32 |
| afazekas | is it on issue if I change the tempest directory permission to 777 ? | 14:32 |
| jeblair | afazekas: works for me | 14:33 |
| jeblair | mordred: why not threading.Thread.__init__(self) ? | 14:34 |
| jeblair | mordred: i understand the difference between that and super, and chose the explicit version because i'm really not interested in using multiple inheritance with thread subclasses | 14:34 |
| mordred | jeblair: ok. | 14:34 |
| *** changbl has joined #openstack-infra | 14:36 | |
| openstackgerrit | afazekas proposed a change to openstack-infra/devstack-gate: Using the jenkis user for tempest run https://review.openstack.org/42101 | 14:39 |
| *** dina_belova has quit IRC | 14:39 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Initial commit https://review.openstack.org/41805 | 14:40 |
| BobBall | mordred: Could I ask for some advice on the venv patch... nova-dhcpbridge is called in /opt/stack/.venv/local/bin/nova-dhcpbridge - which has a "local" too many - probably coming from nova's paths.py where it says that bindir is sys.prefix+"/local/bin". Is the install location for nova-dhcpbridge wrong, or is nova wrong in assuming it's in local/bin? (of course, a quick fix is just to set the config option in /etc/nova/nova.conf...) | 14:40 |
| jeblair | mordred: ^ thanks! | 14:40 |
| mordred | BobBall: nova is wrong. dead wrong | 14:41 |
| mordred | BobBall: that is a bug in nova and should be fixed | 14:41 |
| *** ruhe has joined #openstack-infra | 14:41 | |
| mordred | it is not ok to assme bindir is sys.prefix+"/local/bin" | 14:41 |
| jeblair | mordred: i think we may want to abandon https://review.openstack.org/#/c/33204/ | 14:41 |
| mordred | BobBall: although I _suppose_ we could set bindir=/opt/stack/.venv/bin in nova.conf if GLOBAL_VENV is set | 14:42 |
| *** datsun180b has joined #openstack-infra | 14:42 | |
| mordred | but WOW what a broken design | 14:42 |
| mordred | BobBall: OMG- it's only ever used to find how to launch nova-dhcpbridge | 14:43 |
| mordred | BobBall: I'm going make a patch - that's assinine | 14:43 |
| mordred | etc/nova/nova.conf.sample:#bindir=/usr/local/bin | 14:43 |
| mordred | who was the crackhead who did that ^^ ? | 14:43 |
| BobBall | haha I hadn't even seen that... | 14:43 |
| mordred | gah | 14:43 |
| BobBall | It's more fun - the default is actually using sys.prefix - so it does it get ipartially right in the venv ;) | 14:43 |
| BobBall | so the default in nova.conf.sample isn't what it actually uses! | 14:44 |
| mordred | BobBall: if they're going to use sys.prefix - shouldn't it just be sys.prefix + 'bin' ? | 14:45 |
| *** wenlock has joined #openstack-infra | 14:45 | |
| mordred | hrm, I wonder how you'd find ... | 14:45 |
| mordred | why don't they just use a PATH? | 14:45 |
| BobBall | it was deliberate though https://review.openstack.org/#/c/31344/ | 14:45 |
| *** adalbas has joined #openstack-infra | 14:46 | |
| mordred | jog0: ^^ I may have just called you a crackhead | 14:46 |
| BobBall | used to be /bin | 14:46 |
| mordred | jog0: can you help me understand your patch and how I can fix it better to work in a venv? | 14:46 |
| mordred | BobBall: ok. for now, let's set the path in nova.conf | 14:48 |
| mordred | jeblair: abandon or rebase/fix? | 14:48 |
| mordred | jeblair: or just abandon and revisit after nodepool? | 14:49 |
| jeblair | mordred: i think all the python is going to be deleted from d-g; so unless there's other things that does, i think it won't be relevant soon | 14:49 |
| mordred | great. I'll abandon it then | 14:49 |
| fungi | i suspect the bits of shell which remain could end up in slave scripts or jjb config macros | 14:50 |
| jeblair | mordred: (it will linger for just a little bit until we make sure we have a nodepool equivalent of fungi's developer process) | 14:50 |
| mordred | fungi: I tihnk it's good to have them separate | 14:50 |
| mordred | it let's us gate on them | 14:50 |
| *** krtaylor has joined #openstack-infra | 14:50 | |
| jeblair | fungi: well, the d-g scripts themselves are quite complex, and are part of the gate, so i think they will remain | 14:50 |
| mordred | we have no gating currently on jjb macros or slave scripts | 14:51 |
| fungi | ah, fair point | 14:51 |
| mordred | honestly, I could see moving the slave scripts into d-g as a collection of scripts we run places - which _might_ give us the opportunity to try to test/gate on some of them too | 14:51 |
| mordred | but I'd have to think about that much onger | 14:51 |
| mordred | longer | 14:51 |
| fungi | not that we couldn't add opensack-infra/config to the gate, but it would be a substantial effort to get right | 14:51 |
| mordred | it would | 14:51 |
| *** jjmb has quit IRC | 14:52 | |
| mordred | and I'd honetly rather split opensack-infra/config into more things, than consolidate more things into it | 14:52 |
| fungi | agreed | 14:52 |
| *** dhellmann_ is now known as dhellmann | 14:52 | |
| *** ruhe has quit IRC | 14:53 | |
| * fungi could see the remnants of d-g getting renamed to job-scripts or something and inheriting the slave scripts dumping ground from openstack-infra/config in that case | 14:53 | |
| mordred | BobBall: initset $NOVA_CONF DEFAULT bindir get_python_exec_prefix | 14:55 |
| BobBall | thanks mordred - I'll add that. | 14:56 |
| mordred | BobBall: it looks like devstack is setting it in nova.conf | 14:56 |
| mordred | BobBall: no - it's already there | 14:56 |
| BobBall | oh | 14:56 |
| BobBall | uhhhh | 14:56 |
| BobBall | hang on | 14:56 |
| mordred | and I added: if [[ $GLOBAL_VENV = True ]]; then | 14:57 |
| mordred | echo $DEST/.venv/bin | 14:57 |
| mordred | to get_python_exec_prefix | 14:57 |
| BobBall | then I do not understand | 14:57 |
| mordred | is it in your nova.conf? | 14:57 |
| BobBall | because I didn't have that value in /etc/nova/nova.conf | 14:57 |
| BobBall | no - bindir wasn't there at all | 14:58 |
| BobBall | it was using the default | 14:58 |
| mordred | oh - wait | 14:58 |
| mordred | that's initset | 14:58 |
| * BobBall unstacks and restacks... | 14:58 | |
| BobBall | oh | 14:58 |
| BobBall | haha | 14:58 |
| mordred | what's initset? | 14:58 |
| mordred | it's nothing | 14:58 |
| mordred | initset is nothing | 14:58 |
| mordred | it doesn't exist | 14:58 |
| BobBall | so why doesn't it error | 14:58 |
| mordred | I have no clue | 14:59 |
| BobBall | nice | 14:59 |
| BobBall | ah well | 14:59 |
| BobBall | that's the fix | 14:59 |
| BobBall | *happy* | 14:59 |
| mordred | whee! | 14:59 |
| mordred | reuploading patch with that fix | 15:00 |
| *** HenryG_ has quit IRC | 15:00 | |
| *** marun has joined #openstack-infra | 15:00 | |
| *** boris-42 has quit IRC | 15:01 | |
| *** danger_fo_away is now known as danger_fo | 15:03 | |
| *** dina_belova has joined #openstack-infra | 15:10 | |
| *** cppcabrera has joined #openstack-infra | 15:11 | |
| *** nati_ueno has joined #openstack-infra | 15:14 | |
| *** afazekas has quit IRC | 15:17 | |
| *** dina_belova has quit IRC | 15:18 | |
| *** jjmb has joined #openstack-infra | 15:18 | |
| *** yolanda has joined #openstack-infra | 15:18 | |
| jeblair | mordred: i'm noticing a trend; before i leave a bunch of identical comments.... | 15:19 |
| jeblair | mordred: regarding 38177 | 15:19 |
| *** nayward has quit IRC | 15:20 | |
| jeblair | mordred: you have the vcsrepo sources in puppet using https://; shouldn't we use git:// for that? | 15:20 |
| *** bookmage has joined #openstack-infra | 15:20 | |
| jeblair | also, it looks like in earlier patchsets they _were_ git:// | 15:21 |
| jeblair | mordred: i don't see any indication in the comments why you changed them | 15:21 |
| *** ryanpetrello has joined #openstack-infra | 15:23 | |
| *** nati_ueno has quit IRC | 15:23 | |
| *** SergeyLukjanov has quit IRC | 15:24 | |
| *** mrodden has quit IRC | 15:25 | |
| openstackgerrit | Christoph Thiel proposed a change to openstack-infra/gitdm: add Deutsche Telekom colleagues and update domain-map https://review.openstack.org/42155 | 15:26 |
| *** dina_belova has joined #openstack-infra | 15:26 | |
| *** ruhe has joined #openstack-infra | 15:27 | |
| *** yolanda has quit IRC | 15:28 | |
| *** SergeyLukjanov has joined #openstack-infra | 15:29 | |
| mordred | jeblair: sorry- we had an IRC discussion about it last night | 15:31 |
| mordred | for scripts we expect might get re-used outside of our infra systems, cloning from https is more likely to work consistently (corporate firewalls and whatnot) | 15:32 |
| mordred | for scripts we do not expect to run outside of our systems or be reused directly (like our jenkins jobs) - cloning from git seemed like a good plan | 15:32 |
| jeblair | mordred: i get that, which is why the use in, say, install_jenkins_slave and of course the documentation seems reasonable | 15:32 |
| *** tjones has joined #openstack-infra | 15:33 | |
| jeblair | mordred: but there are something like 10 puppet modules that use vcsrepo... so puppet is going to hit those repos a lot. don't we want the performance improvement of git:// ? | 15:33 |
| mordred | same thing with puppet modules - we have a few sets of people re-using them verbatim | 15:33 |
| mordred | the world would be SO MUCH BETTER if people didn't run evil web proxy bullshits | 15:35 |
| jeblair | well, i guess i'm confused. we spent months talking about how great git:// is going to be, and now we're not going to use it in places where we would like to, in order to get a real performance improvement. | 15:36 |
| *** UtahDave has joined #openstack-infra | 15:36 | |
| jeblair | mordred: do we _know_ that any of our users (who are active contributors) have a problem with git:// urls? | 15:36 |
| mordred | we'rre going to use it in our jenkins jobs, which is where the performance improvement matter, no? | 15:36 |
| mordred | me | 15:36 |
| mordred | I have a problem with git urls inside of hp | 15:36 |
| jeblair | mordred: that is sad. can you get it fixed? | 15:36 |
| mordred | nope | 15:36 |
| jeblair | did you try? | 15:37 |
| mordred | I mean, _Eventually_ _maybe_ but the process takes between 6months to a year | 15:37 |
| mordred | and I'm fighting 29418 getting opened first | 15:37 |
| mordred | because I thinkn that's more important to open the door for more people to contribute code | 15:37 |
| fungi | mordred: heh, holger got your last name wrong in the tox release announcement ;) | 15:37 |
| mordred | fungi: hah | 15:37 |
| wenlock | mordred, me 2 | 15:37 |
| wenlock | all morning | 15:37 |
| mordred | jeblair: but yeah, it's on my list | 15:37 |
| wenlock | i have a workaround though | 15:37 |
| mordred | wenlock: on yeah? then maybe I can use it? | 15:38 |
| mordred | jeblair: meet wenlock - he's working on starting to stand up a copy of infra as well - clark and I were just telling him yesterday about the benefits of collaborating with us direcetly upstream | 15:38 |
| wenlock | so if your behind a dumb web-proxy... which i agree should be abolished.... setup an ssh tunel for the golden port thats allowed out, port 22 | 15:39 |
| jeblair | wenlock: hello | 15:39 |
| wenlock | hey jeblair | 15:40 |
| mordred | jeblair: but the jenkins jobs are where we care more about preformance, right? as opposed to the speed of pulling cloning jeepyb? | 15:40 |
| mordred | jeblair: (also, last night it seemed that updating those in a seperate follow-on patch seemed like a good idea, since they're pulling from review.o.o directly now) | 15:40 |
| wenlock | ssh -N -L 29418:127.0.0.1:29418 -i <your server ssh key> <username>@<gerrit server> | 15:40 |
| mordred | wenlock: nod. stupid proxies | 15:41 |
| jeblair | mordred: well, it's just that cumulatively, we have a whole host of servers that update repos every 10 minutes; i'm not sure what the total number of repos updated every 10 minutes is; maybe 15 or 20? | 15:42 |
| fungi | i personally think documentation should list at least https clone urls, but also listing git urls is fine. i think scripts we document as shortcuts for fairly inexperienced users who may be inside corporate networks should probably also default to https. i think our automation, which sysadmins may download and try to implement, should probably default to whatever performs best while still giving us | 15:42 |
| fungi | sufficient security | 15:42 |
| *** mrodden has joined #openstack-infra | 15:42 | |
| *** markmcclain has quit IRC | 15:42 | |
| fungi | anyone downloading our puppetry, for example... changing clone urls in it is likely to be the least of their hurdles on the way to deployment | 15:43 |
| *** cppcabrera has left #openstack-infra | 15:43 | |
| mordred | fungi: disagre. I have a fork of openstack-infra/config now because I had to make changes to the pip stuff because of proxy issues (beause puppet pip sucks) | 15:44 |
| mordred | as soon as you have to fork our puppet stuff, then the cost of doing your dev upstream goes up | 15:44 |
| *** tjones has quit IRC | 15:44 | |
| fungi | mmm... maybe urls are things we should parameterize? | 15:45 |
| mordred | and if you're not highly motivated (like I am) you'll just run locally and we won't get your help back upstream | 15:45 |
| mordred | oy | 15:45 |
| jeblair | mordred: i'm opposed to making design decisions solely to help people who do not contribute upstream use our work in a private and proprietary environment. however.... | 15:46 |
| mordred | jeblair: agree | 15:46 |
| jeblair | mordred: you have made a case that there is more than one (potential) upstream contributor who would benefit from using https, so i'm willing to go with that for now | 15:46 |
| mordred | jeblair: cool - thanks. and for the record, my goal there is to use work in that direction to try to incentivize more of the users to become contributors | 15:47 |
| mordred | I may or may not succeed | 15:47 |
| fungi | with https and git clone urls being identical for us save the protocol name, perhaps it's something we could easily add a switch for? | 15:47 |
| *** nicedice_ has joined #openstack-infra | 15:47 | |
| mordred | but I at least want to remove roadblocks when if/when I find them | 15:47 |
| jeblair | (i think that fixing the firewall would be a good hedge against that changing in the future) | 15:48 |
| jeblair | fungi: not a bad idea | 15:48 |
| *** vogxn has joined #openstack-infra | 15:48 | |
| mordred | fungi: interesting idea - we could set a global variable in site.pp perhaps | 15:48 |
| fungi | url => ${git_proto}://git.openstack.org/whatever | 15:48 |
| mordred | yeah | 15:48 |
| mordred | I have a complex idea... | 15:48 |
| fungi | needs the apache config patch i uploaded for review last night, of course | 15:48 |
| jeblair | mordred: i left comments on your change. not all of them are "git://?" so read carefully. :) | 15:49 |
| mordred | that might be simplier to use - no, let me ignore that | 15:49 |
| mordred | jeblair: k. will go address | 15:49 |
| *** vogxn has quit IRC | 15:49 | |
| mordred | jeblair: "Do we want this to go to /tree/ for easier navigation? That would match current behavior." - curent behavior is blob, not tree | 15:50 |
| mordred | jeblair: (figure I'll just ask that one here, while I'm fixing things) | 15:51 |
| jeblair | mordred: blob != raw | 15:51 |
| mordred | it's not? my bad then | 15:51 |
| jeblair | plain == raw | 15:51 |
| jeblair | mordred: no idea what they mean by blob, but at any rate, it shows the file contents in a navigatable context | 15:51 |
| openstackgerrit | A change was merged to openstack-infra/config: Serve repos via git+https outside of cgit https://review.openstack.org/42044 | 15:51 |
| *** burt has quit IRC | 15:54 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add high max-repo-count for index page in cgit https://review.openstack.org/41815 | 15:54 |
| mordred | jeblair, fungi: I'm torn on modules/openstack_project/files/jenkins-pypi-mirror.yaml | 15:54 |
| mordred | it's differnet than vcsrepo uses | 15:55 |
| mordred | I'm thinking git makes mre sense there | 15:55 |
| jeblair | mordred: (also, i think we have two versions of that file due to the previous (and possibly future) dedicated pypi host | 15:55 |
| fungi | mmm, yeah | 15:55 |
| openstackgerrit | A change was merged to openstack-infra/config: Cleanup cgit interface https://review.openstack.org/42016 | 15:56 |
| jeblair | mordred: it is a jenkins job that runs on a centos host | 15:56 |
| fungi | yeah, git:// should be much faster than https:// there | 15:56 |
| jeblair | mordred: (and is probably perceivable by developers (when it runs on requirements changes)) | 15:56 |
| *** afazekas has joined #openstack-infra | 15:56 | |
| fungi | for mirror26 at least | 15:57 |
| mordred | yup | 15:57 |
| mordred | git:: | 15:57 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Use cgit server instead of github for everything https://review.openstack.org/38177 | 15:59 |
| mordred | jeblair: awesome. thanks! | 15:59 |
| mgagne | fun, github is down atm | 16:01 |
| jeblair | oh if we only had mordred's change merged :) | 16:02 |
| mordred | jeblair: I think we're getting close | 16:02 |
| fungi | mgagne: funny... you should try git.openstack.org ;) | 16:03 |
| *** datsun180b_ has joined #openstack-infra | 16:04 | |
| mordred | jeblair: patch looks great - although you havea pep8 oops | 16:04 |
| *** datsun180b_ has quit IRC | 16:04 | |
| *** datsun180b has quit IRC | 16:05 | |
| fungi | mordred: also seems to need a rebase | 16:05 |
| *** datsun180b has joined #openstack-infra | 16:05 | |
| mordred | gha | 16:05 |
| mordred | oh 0- right | 16:05 |
| mordred | I landed annegentle's patch | 16:05 |
| *** changbl has quit IRC | 16:06 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Use cgit server instead of github for everything https://review.openstack.org/38177 | 16:10 |
| openstackgerrit | afazekas proposed a change to openstack-infra/devstack-gate: Using the jenkis user for tempest run https://review.openstack.org/42101 | 16:12 |
| jeblair | mordred: it's going to take me a while to fix that pep8 bug because pip and pbr are hosed on my workstation, meaning i can't run tox -e pep8 (the new patchset upgraded the pbr requirement) | 16:13 |
| jeblair | mordred: http://paste.openstack.org/show/44238/ | 16:13 |
| jeblair | mordred: i don't even know that that means | 16:13 |
| mordred | jeblair: try pip install -U pbr | 16:14 |
| jeblair | mordred: http://paste.openstack.org/ | 16:15 |
| mordred | jeblair: ? | 16:15 |
| jeblair | mordred: beats me. do i need to reinstall my workstation from scratch now? | 16:15 |
| mordred | jeblair: no - your last pastelink is missing half the url | 16:16 |
| jeblair | oh why does it do that? http://paste.openstack.org/show/44239/ | 16:16 |
| mordred | gah. I realy really hate easy install | 16:16 |
| mordred | jeblair: some how, /home/corvus/rs/github/gear/easy-install.pth has gotten in to your system dir | 16:17 |
| mordred | (I'm assuming you are not in an activated virtualenv) | 16:17 |
| mordred | can you look in /usr/local/lib/python2.7/dist-packages/easy-install.pth and see if there is a reference to it in there? | 16:18 |
| mordred | jeblair: you'll be happy to know that sometimes, after you setup.py develop something, easy_install cannot figure out how to undo what it did | 16:18 |
| jeblair | mordred: ok, i removed it from that file, but it still shows up in pip freeze | 16:19 |
| afazekas | https://review.openstack.org/#/c/39237/ "Gate fails because Ceilometer API requires MongoDB >= 2.2 and doesn't start because devstack gate has 2.0. Not really sure how to fix this." | 16:19 |
| mordred | wow. are there any .lnk files in /usr/local/lib/python2.7/dist-packages ? | 16:19 |
| jeblair | mordred: yes, i'll remove those too | 16:20 |
| mordred | jeblair: (I _just_ had to clean up a similar situation on my machine that had resulted from my setup.py develop-ing the tox tree into my global env) | 16:20 |
| mordred | essentially, as I'm learning, directly installing python anywhere other than a virtualenv apparently just flat is a bad idea | 16:21 |
| jeblair | mordred: ok, that did it. but i'm worried that this might be a problem if we use pbr with git-review | 16:21 |
| mordred | it's not releated to pbr | 16:21 |
| mordred | it's related to bad behavior by setup.py develop in a global context | 16:21 |
| jeblair | mordred: ok, so it shouldn't affect most people, but it will affect those of us who like to setup.py develop from the git-review tree. :( | 16:22 |
| mordred | yah | 16:22 |
| mordred | I have not yet tested to see if pip install -e . does a better job of cleaning up after itself | 16:22 |
| jeblair | mordred: how will i be able to test git-review patches? | 16:22 |
| *** krtaylor has quit IRC | 16:23 | |
| mordred | jeblair: let me do some investigate on pip install -e . | 16:23 |
| mordred | I _think_ it might be better about it | 16:23 |
| mordred | but I need to run through some scenarios | 16:23 |
| clarkb | morning | 16:24 |
| mordred | in any case, pip install -e . has the benefit over setup.py develop of installing the requirements via pip instead of easy_install - which is always good | 16:24 |
| mordred | and runs seutp.py develop behind the scenes | 16:24 |
| fungi | morning clarkb | 16:24 |
| mordred | jeblair: also, just imagine the fun I have with python setup.py develop-ing trunk of pbr and hacking at all times :) | 16:25 |
| mordred | actually, don't do that | 16:25 |
| *** dina_belova has quit IRC | 16:25 | |
| jeblair | afazekas: i believe mongodb is installed by puppet on our slaves; the version in ubuntu precise is 2.0 | 16:25 |
| afazekas | jeblair: can it be pulled from newer repo ? | 16:26 |
| clarkb | I thought we had this discussion with jd__ when they added mongodb to the unittest slaves | 16:27 |
| * clarkb looks in git to see what we did there | 16:27 | |
| mordred | is it in ubuntu cloud archive yet? | 16:27 |
| mordred | and epel? | 16:27 |
| jeblair | mordred: i think that is the best solution | 16:27 |
| jeblair | afazekas: ^ | 16:27 |
| *** SergeyLukjanov has quit IRC | 16:27 | |
| mordred | zul: ^^ ? | 16:27 |
| *** HenryG has joined #openstack-infra | 16:28 | |
| clarkb | looks like we just added mongodb as the package for the unittest slaves | 16:28 |
| jeblair | afazekas: because we want to help distros support openstack on LTS/RHEL, so our support policy is "develop on newest, try not to break long-term" | 16:28 |
| jeblair | afazekas: if using 2.0, or getting 2.2 into the cloud archive is not an option, then we would have to break ceilometer on the long-term distros | 16:29 |
| jd__ | some tests fail on ceilometer because of concurrency | 16:29 |
| zul | mordred: we are dealing with some issues that we have before it can go into the cloud archive https://bugs.launchpad.net/ubuntu/+source/mongodb/+bug/1187262 | 16:29 |
| uvirtbot | Launchpad bug 1187262 in snowball "[MIR] mongodb, libv8, snowball, gyp" [High,New] | 16:29 |
| jd__ | this has nothing to do with MongoDB version on Jenkins | 16:29 |
| mordred | jd__: yay! | 16:29 |
| mordred | it's almost like asking jd__ bring answers :) | 16:29 |
| * jeblair believes everything written in code reviews | 16:29 | |
| openstackgerrit | Christoph Thiel proposed a change to openstack-infra/gitdm: add Deutsche Telekom colleagues and update domain-map https://review.openstack.org/42155 | 16:30 |
| fungi | magic answer genie | 16:30 |
| jd__ | we are aware, I'm not sure a bug has been opened, sileht might know he's first one who found it and tried to tackle it | 16:30 |
| jd__ | :-) | 16:30 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Initial commit https://review.openstack.org/41805 | 16:30 |
| clarkb | jd__: this is for tests in tempest though | 16:31 |
| jd__ | clarkb: URL? | 16:31 |
| clarkb | which are not currently concurrent | 16:31 |
| clarkb | jd__: https://review.openstack.org/#/c/39237/ | 16:31 |
| *** vogxn has joined #openstack-infra | 16:31 | |
| clarkb | it is annoying that logrotate needs to run once before it actually rotates anything | 16:32 |
| clarkb | I am removing pypi.slave from jenkins.o.o then will delete the vm | 16:33 |
| *** ruhe has quit IRC | 16:34 | |
| jeblair | clarkb: thanks | 16:34 |
| jeblair | clarkb, fungi: would you please review https://review.openstack.org/#/c/39551/ despite lifeless's -1 ? | 16:34 |
| clarkb | er rather tx.slave | 16:34 |
| clarkb | woo don't want to mix those up when it counts :) | 16:34 |
| jeblair | i know that some times people ignore reviews with -1 code reviews, and i don't think that should happen to that review. | 16:35 |
| afazekas | mordred: it is not in epel | 16:35 |
| jeblair | also, anyone else reviewing the nodepool code? i'd like to merge it now and start writing the puppet vcsrepo code for it. | 16:35 |
| clarkb | afazekas: mordred I believe the version on centos 6 is new enough | 16:36 |
| clarkb | jeblair: I haven't had a chance to merge it after the review I did when it was in d-g | 16:36 |
| clarkb | s/merge/review | 16:36 |
| jd__ | clarkb: ah yeah in this case I can't help indeed; we skip the test on Precise because of that and run MongoDB tests on RHEL that has 2.2 | 16:36 |
| mordred | jeblair: +2 from me | 16:37 |
| jd__ | FWIW I've already asked zul over a month ago for MongoDB backports in Ubuntu, and things should be on their way (though you may want to ping him :) | 16:37 |
| *** krtaylor has joined #openstack-infra | 16:37 | |
| fungi | jd__: he commented above on the status | 16:38 |
| mordred | jd__: he responded just a bit ago: we are dealing with some issues that we have before it can go into the cloud archive | 16:38 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Fix intermittent jenkins plugin build failure https://review.openstack.org/42062 | 16:38 |
| *** pcrews has joined #openstack-infra | 16:38 | |
| mordred | ooh. I like the sound of that patch | 16:38 |
| *** jpich has quit IRC | 16:38 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add static web directory for cgit & initial files https://review.openstack.org/42168 | 16:39 |
| mordred | zaro: left comments | 16:40 |
| clarkb | did I derp when creating proposal and make it a stand in for tx instead of a stand in for pypi? | 16:40 |
| clarkb | or do we need to replace both slaves? | 16:41 |
| jeblair | clarkb: proposal is supposed to be tx | 16:41 |
| *** derekh has quit IRC | 16:41 | |
| *** tjones has joined #openstack-infra | 16:41 | |
| clarkb | oh good | 16:41 |
| jeblair | clarkb: it will propose to gerrit and transifex (not upload to pypi) | 16:41 |
| clarkb | I see. tx.slave is under the ci account and not the jenkins account | 16:41 |
| *** ruhe has joined #openstack-infra | 16:42 | |
| clarkb | but both tx.slave and pypi.slave appear to be first gen. That explains my confusion | 16:42 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Fix intermittent jenkins plugin build failure https://review.openstack.org/42062 | 16:42 |
| *** ^d has joined #openstack-infra | 16:42 | |
| *** thomasbiege has joined #openstack-infra | 16:42 | |
| clarkb | but proposal.slave is under the jenkins account where it belongs | 16:43 |
| jeblair | it's kind of a grey area, i guess. | 16:43 |
| clarkb | ya | 16:43 |
| jeblair | we didn't have 'really important long running permanent slaves' when we decided on the demarcation. | 16:43 |
| zaro | mordred: thanks. just uploaded new patch. | 16:44 |
| clarkb | I am ready to delete tx.slave now if everyone else is ready | 16:44 |
| *** BobBall is now known as BobBall_Away | 16:44 | |
| *** dkliban is now known as dkliban_afk | 16:46 | |
| * clarkb reviews nodepool and 39551 really quick | 16:46 | |
| *** thomasbiege has quit IRC | 16:46 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Initial commit https://review.openstack.org/41805 | 16:47 |
| jeblair | i decided i want the daemon just called 'nodepool' instead of 'nodepoold'. ^ | 16:47 |
| fungi | clarkb: seems safe at this point | 16:47 |
| mordred | jeblair: in zaro's change above - I've got to say - the new freestyle project looks cleaner | 16:47 |
| mordred | jeblair: but the other way, if you spoke with a dutch accent, it could rhyme with mold | 16:48 |
| jeblair | mordred: i think clarkb wanted to try to keep it as a maven project; so we'll want him to weigh in on it. i'm not sure if this would impact doc jobs. | 16:48 |
| jeblair | mordred: but yeah, if trying to dtrt with a maven job is crazy, we should consider cutting our losses. | 16:49 |
| *** fungi has quit IRC | 16:49 | |
| jeblair | mordred: also, must consider the docbook plugin; does it need to be a maven job for any reason, or can it follow this template. | 16:49 |
| mordred | jeblair: I'm think the only reason it's the rt is because if its ability to split out the various targets and show fancy reports, no? | 16:49 |
| mordred | mmm | 16:49 |
| jeblair | mordred: yeah, we certainly don't use it like that. | 16:50 |
| clarkb | jeblair: mordred: I wanted to keep it as a maven project because jenkins does some neat stuff with maven projects, but we have spent far too much time trying to make this work so I am happy to switch to freestyle which should just work | 16:50 |
| *** fungi has joined #openstack-infra | 16:50 | |
| mordred | clarkb: yeah - I think most of the time we try to use neat jenkins geatures, it bites us the ass | 16:50 |
| clarkb | mordred: that is one benefit. The other is custom maven installs per job | 16:50 |
| clarkb | mordred: switching to freestyle forces us to use the global maven version without extra work | 16:50 |
| clarkb | fungi: welcome back | 16:50 |
| fungi | clarkb: thanks. needed to restart for a weechat upgrade | 16:51 |
| clarkb | jeblair: 39551 lgtm. I will let you approve if you think lifeless' comments don't need to be addressed | 16:51 |
| clarkb | now to delete tx.slave | 16:52 |
| zaro | mordred, jeblair : i think only benefit to maven job is that it installs maven for the job (maven doesn't need to be pre-installed). | 16:52 |
| clarkb | tx.slave is deleting. I assume the dns record(s) are in the othe raccount though | 16:53 |
| zaro | mordred, jeblair: actually the other nice benefit is that it works better as a multi-config job. but we don't seem to use those. | 16:53 |
| *** sarob has joined #openstack-infra | 16:53 | |
| clarkb | zaro: and it installs an arbitrary maven version that can be assigned per job which can be useful | 16:54 |
| *** briancline has joined #openstack-infra | 16:54 | |
| *** vipul-away is now known as vipul | 16:54 | |
| *** vipul is now known as vipul-away | 16:55 | |
| *** ryanpetrello has quit IRC | 16:57 | |
| clarkb | all done cleaning up tx.slave | 16:57 |
| *** dims has joined #openstack-infra | 16:58 | |
| *** markmcclain has joined #openstack-infra | 16:59 | |
| russellb | is there any way to un-abandon a patch where the owner is no longer responsive (left his company, doing something else now) ? | 17:01 |
| clarkb | russellb: yes, I can dothat for you | 17:01 |
| clarkb | russellb: whcih change? | 17:01 |
| russellb | https://review.openstack.org/#/c/30028/ | 17:01 |
| russellb | clarkb: thanks! | 17:01 |
| clarkb | russellb: done | 17:01 |
| russellb | thank you sir | 17:02 |
| *** thomasbiege has joined #openstack-infra | 17:03 | |
| *** ruhe has quit IRC | 17:03 | |
| mordred | chmouel: when is the next openstack israel? the CFP doesn't say | 17:03 |
| mordred | ah. Dec 3 | 17:04 |
| *** nati_ueno has joined #openstack-infra | 17:05 | |
| *** sdake has joined #openstack-infra | 17:05 | |
| *** sdake has quit IRC | 17:05 | |
| *** sdake has joined #openstack-infra | 17:05 | |
| *** radix has joined #openstack-infra | 17:08 | |
| radix | hi guys, there's a patch that looks like it should be merged but it hasn't been yet, can someone take a look? https://review.openstack.org/#/c/41812/ | 17:08 |
| mordred | radix: I kicked it | 17:11 |
| radix | thanks mordred :D | 17:11 |
| mordred | radix: we were having an issue or two yesterday that are now cleared up | 17:11 |
| *** thomasbiege has quit IRC | 17:13 | |
| *** colinmcnamara has joined #openstack-infra | 17:14 | |
| mordred | jeblair: zomg. I just read through your re-org in zuul for the timer code | 17:15 |
| mordred | jeblair: beaufitul | 17:15 |
| *** colinmcnamara has quit IRC | 17:15 | |
| clarkb | jeblair: I am working my way through nodepool now. Will have a few not super important comments but holding off merging for a few more minutes would be great | 17:16 |
| *** colinmcnamara has joined #openstack-infra | 17:16 | |
| jeblair | mordred: thanks. :) there's the opportunity to have timer events have associated branches, etc -- in other words, create their own zuul refs. i have not taken advantage of that yet. but that's incidental to your work, of course. | 17:17 |
| jeblair | (though it's neat to think about how that could change our stable bitrot jobs) | 17:17 |
| jeblair | clarkb: ok; i'm not quite to needing it in the repo yet, can hold for a bit. | 17:18 |
| *** ^d has quit IRC | 17:21 | |
| *** nicedice_ has quit IRC | 17:21 | |
| *** sdake_ has joined #openstack-infra | 17:22 | |
| *** ^d has joined #openstack-infra | 17:22 | |
| *** ^d has joined #openstack-infra | 17:22 | |
| *** ^d has quit IRC | 17:22 | |
| clarkb | jeblair: comments left | 17:22 |
| *** ^d has joined #openstack-infra | 17:22 | |
| *** ^d has joined #openstack-infra | 17:22 | |
| *** nicedice_ has joined #openstack-infra | 17:27 | |
| *** HenryG has quit IRC | 17:27 | |
| jeblair | clarkb: thanks, responded | 17:29 |
| *** SnowDust has joined #openstack-infra | 17:31 | |
| jeblair | someone forgot to git commit after adding gerrit ssh replication key | 17:31 |
| *** pcm__ has joined #openstack-infra | 17:32 | |
| jeblair | it's getting lumped in with my addition of nodepool creds | 17:32 |
| mordred | jeblair: maybe we should have a private gerrit that manages that repo ... /me kidding | 17:33 |
| *** krtaylor has quit IRC | 17:33 | |
| jeblair | :) | 17:34 |
| *** pcm_ has quit IRC | 17:34 | |
| *** nicedice_ has quit IRC | 17:35 | |
| *** jfriedly has joined #openstack-infra | 17:38 | |
| *** nicedice has joined #openstack-infra | 17:40 | |
| *** chuckieb|2 has joined #openstack-infra | 17:40 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add static web directory for cgit & initial files https://review.openstack.org/42168 | 17:42 |
| pleia2 | ^^ added custom CSS too, so it ends up looking as seen here: http://15.185.127.146/cgit | 17:42 |
| uvirtbot | pleia2: Error: "^" is not a valid command. | 17:42 |
| pleia2 | (repos aren't populated, just for looking at front page) | 17:42 |
| *** chuckieb has quit IRC | 17:43 | |
| *** vogxn has quit IRC | 17:43 | |
| *** vipul-away is now known as vipul | 17:44 | |
| clarkb | pleia2: looks good | 17:44 |
| fungi | pleia2: me likey! | 17:45 |
| pleia2 | mgagne gets credit here too :) | 17:45 |
| fungi | mgagne: awesome! | 17:46 |
| *** changbl has joined #openstack-infra | 17:46 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Fix intermittent jenkins plugin build failure https://review.openstack.org/42062 | 17:48 |
| *** sdake_ has quit IRC | 17:49 | |
| pleia2 | so I'm thinking once we get this prettification patch + mordred's s/github/git.o.o one I can send a kick-the-tires email to the dev list, perhaps blog on planet too for feedback from devs | 17:50 |
| *** SergeyLukjanov has joined #openstack-infra | 17:50 | |
| mordred | ++ | 17:51 |
| clarkb | pleia2: it might be worth doing some quick performance type checks. Earlier I got the load to up around 3 fairly trivially | 17:51 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Initial commit https://review.openstack.org/41805 | 17:51 |
| pleia2 | clarkb: sure | 17:51 |
| clarkb | and it felt like cgit was unhappy about that :) | 17:51 |
| mordred | clarkb: we could always add like 5 of them and stick a load balancer in front | 17:52 |
| *** Ryan_Lane has quit IRC | 17:52 | |
| clarkb | mordred: yeah, or grow the node. I don't think it is a major issue but I was browsing the web ui and git cloning and that was all it took to get a little sluggish | 17:52 |
| mordred | clarkb, jeblair: https://review.openstack.org/#/c/42178/ | 17:52 |
| mordred | fwiw | 17:52 |
| openstackgerrit | A change was merged to openstack-infra/config: Configure Pecan to re-publish docs through RTFD on release. https://review.openstack.org/41969 | 17:52 |
| pleia2 | it does load everything dynamically every time, I think the only thing that saves it at all is that it's written in C | 17:53 |
| clarkb | mordred: that is super exciting. you should backport that change to all of the stable branches too if it works | 17:53 |
| clarkb | does it cache well? would varnish help? | 17:53 |
| jeblair | mordred: neat | 17:54 |
| mordred | jeblair, clarkb I believe that is the first patch we've landed and released to tox isn't it? | 17:54 |
| marun | clarkb: You mentioned yesterday that updating the nova unit tests to use testr was easy - did that include replacing the use of setupClass? | 17:55 |
| clarkb | mordred: I haven't gotten anything past them | 17:55 |
| mordred | marun: they did not use setupClass | 17:55 |
| clarkb | marun: but we did replace a lot of things with proper fixtures | 17:55 |
| clarkb | like the DB stuff | 17:55 |
| mordred | yes. there were SEVERAL evil things. the db was a big one | 17:55 |
| mordred | jink | 17:55 |
| marun | clarkb: I think the challenge in tempest is they use setupClass to handle ordinal teardown | 17:56 |
| mordred | marun: the fixtures part was easy - the grokking the intent of the code iwas the hard part | 17:56 |
| marun | clarkb: It's doable, but potentially a lot of work. | 17:56 |
| *** nicedice has quit IRC | 17:56 | |
| marun | mordred: fair enough | 17:56 |
| mordred | marun: yeah - so, the approach lifeless suggested at the last summit | 17:56 |
| clarkb | mordred: https://review.openstack.org/#/c/42178/1/tox.ini you s/testr/test/ do both work? | 17:57 |
| mordred | marun: was to extract the content of the setupClass in its current form, wrap it in a simple fixture unmodified, and then have setupClass call the fixture as a first pass | 17:57 |
| mordred | marun: then, as a second pass, have testresources (which nose doesn't know how to handle) also instanatiate the fixture | 17:57 |
| mordred | marun: so now there are two code paths to the fiture being started | 17:58 |
| mordred | and _then_ start refactoring the code in the fixtures so that it does sensible things | 17:58 |
| marun | mordred: Ah, that sounds doable actually. | 17:58 |
| marun | mordred: thanks for the info | 17:58 |
| mordred | sure thing! | 17:58 |
| *** nicedice has joined #openstack-infra | 17:59 | |
| *** sarob has quit IRC | 18:01 | |
| *** sarob has joined #openstack-infra | 18:01 | |
| jeblair | mordred, fungi, clarkb: fyi, nodepool is written so that if you send it a TERM (or hit C-c in foreground), it should finish what it's doing cleanly. this might take several minutes (or longer if it's saving a rackspace cloud image) | 18:04 |
| mordred | jeblair: cool. good to know | 18:05 |
| mordred | because I gotta tell you - there are other programs I run that block C-c | 18:05 |
| mordred | and they drive me batty | 18:05 |
| *** sarob has quit IRC | 18:06 | |
| fungi | jeblair: good to be aware of | 18:06 |
| openstackgerrit | A change was merged to openstack-infra/config: Document RTFD hook job template in StackForge doc https://review.openstack.org/41793 | 18:06 |
| clarkb | mordred: so s/testr/test/ in tox.ini. Is that related to moving to the pbr entrypoint? | 18:06 |
| mordred | clarkb: oh - pioop | 18:07 |
| mordred | I didnt' mean for that bit to land in there | 18:07 |
| fungi | jeblair: some response when trapping sigint or similar would be good too | 18:07 |
| jeblair | mordred: dramatic moment! nodepool is run in production for the first time! http://paste.openstack.org/show/44248/ | 18:07 |
| fungi | jeblair: assuming it doesn't yet, that it | 18:07 |
| fungi | is | 18:07 |
| mordred | jeblair: HAHAHAHA | 18:07 |
| clarkb | :) | 18:07 |
| *** tjones has left #openstack-infra | 18:07 | |
| mordred | I think pbr just has it out for jeblair | 18:08 |
| fungi | whee! | 18:08 |
| mordred | jeblair: can I ask, just for data collection, how you installed it? | 18:08 |
| jeblair | fungi: er, actually, i think the signal handling is wrong; that should be nodepool's first patch. | 18:08 |
| jeblair | mordred: i let puppet setup.py install it. | 18:09 |
| mordred | k. so - there is a bug, or a feature, or just a reality, that I've been talking with dstufft about | 18:09 |
| fungi | jeblair: but yeah, things like responding on stderr when trapping sigint or logging when trapping sigterm are always appreciated | 18:09 |
| mordred | that sometimes, when you python setup.py install something that has setup_requires that are also in install_requires | 18:09 |
| mordred | easy_install downloads the thing for setup_requires, and ten when it's trying to resolve install_requires, it sees the thing in the local dir and so it considers it installed | 18:10 |
| mordred | even though it's just in the local setup_requires location | 18:10 |
| mordred | it's quite amazing | 18:10 |
| jeblair | mordred: i have a vague memory of trying to understand this while walking down a street in portland with you. | 18:11 |
| dstufft | I have a sneaking suspicion it might have something to do with pbr, but I don't know that for a fact. I know i've never seen anyone else run into it before but it may just be openstack's uncanny ability to break all the things | 18:11 |
| jeblair | mordred: sadly, at this point i remember the strangers we passed on the street more than the details of the conversation. :( | 18:11 |
| fungi | we do, in fact, break lots and lots of things | 18:12 |
| clarkb | jeblair: don't be sad. Strangers in Portland are often quite entertaining | 18:12 |
| mordred | dstufft: I've scoured the code in setuptools to figure out why this happens sometimes and not others | 18:12 |
| mordred | dstufft: for the most part, I have not figure out a simple reproducible test case | 18:12 |
| *** burt has joined #openstack-infra | 18:12 | |
| jeblair | clarkb: they are! i'm thinking of the guy with horns growing out of his bald head right now! | 18:12 |
| mordred | although perhaps this is it - python setup.py install in a project that uses pbr that install_requires pbr and a project that also install_requires and setup_requires pbr | 18:13 |
| mordred | because that's what just happened | 18:13 |
| jeblair | mordred: yep | 18:13 |
| *** w_ is now known as olaph | 18:13 | |
| mordred | dstufft: your improvement to pip we talked about yesterday would make this problem go away 100% | 18:13 |
| Alex_Gaynor | Does the openstack-infra/zuul repo contain the web interface that lives at status.os.c/zuul ? | 18:14 |
| jeblair | mordred: or "an openstack project that uses an openstack project" :) | 18:14 |
| mordred | jeblair: yeah | 18:14 |
| dstufft | mordred: the setup_requires is made available to the setup.py (and thus the pip you shell out too) | 18:14 |
| dstufft | is my guess | 18:14 |
| mordred | dstufft: ah | 18:14 |
| jeblair | Alex_Gaynor: it's in infra/config; we run a 'customized' version instead of the 'upstream' version. | 18:14 |
| clarkb | fungi: where are we on git-review release? | 18:14 |
| jeblair | Alex_Gaynor: we occasionally suffer from multiple personality disorder, as a team. | 18:14 |
| mordred | dstufft: so - do ou know of any good - or hacky - way I can strip or modify that env? | 18:14 |
| dstufft | mordred: no, because you need it to run pbr at all | 18:15 |
| fungi | clarkb: released and announced (announced twice) | 18:15 |
| *** krtaylor has joined #openstack-infra | 18:15 | |
| Alex_Gaynor | jeblair: :) so, any objects to a patch which adds the number of jobs to the header for each queue. e.g. "gate (3)" | 18:15 |
| *** ruhe has joined #openstack-infra | 18:15 | |
| clarkb | clearly I am behind on email | 18:15 |
| fungi | clarkb: (i forgot to sign my first announcement :/) | 18:15 |
| mordred | dstufft: pbr should have been run by the time we're actually running setup() contents | 18:15 |
| clarkb | fungi: I will rereview the outstanding git-review changes | 18:15 |
| jeblair | Alex_Gaynor: I'd love that. | 18:15 |
| mordred | dstufft: I just need to prevent it from going to the pip subshells | 18:15 |
| Alex_Gaynor | jeblair: great, I'll go ahead and do that then | 18:15 |
| *** jerryz has joined #openstack-infra | 18:16 | |
| clarkb | is github still being derpy for anyone else? | 18:16 |
| fungi | clarkb: i'm uploading a new 35486 momentarily addressing mordred's suggestions | 18:16 |
| mordred | but the mechanism of how it's passing that path from setup.py context to the subshells is what I can't grok | 18:16 |
| mordred | jeblair: mind if I go look at your nodepool box? | 18:17 |
| jeblair | Alex_Gaynor: do you grok "pipelines", "shared change queues" and "severed heads" ? | 18:17 |
| jeblair | mordred: please | 18:17 |
| jeblair | mordred: nodepoo.o.o | 18:17 |
| dstufft | mordred: you're calling python -m pip aren't you? | 18:17 |
| Alex_Gaynor | jeblair: I grok the pipelines, not sure about the other two | 18:17 |
| mordred | dstufft: yes. or sys.executable -m pip | 18:17 |
| dstufft | mordred: that might add '' to sys.path | 18:17 |
| dstufft | and curdir is probably the builddir | 18:17 |
| mordred | yup | 18:17 |
| dstufft | which is where setup_requires isntalls stuff too | 18:17 |
| mordred | I'm almost certain it is | 18:17 |
| fungi | clarkb: i haven't noticed github derpage because i did a `cd work ; sed -i 's/hub.com/.openstack.org/' */*/.git/config` | 18:17 |
| clarkb | I should update all of my origin remotes to git.o.o | 18:17 |
| clarkb | jinx | 18:17 |
| mordred | dstufft: any idea _where_ it adds to sys.path? does it pass a PYTHONPATH in the env? | 18:18 |
| dstufft | mordred: I think invoke python itself just does that | 18:18 |
| * mordred can also go investigate | 18:18 | |
| dstufft | vis invoking it from a script | 18:18 |
| jeblair | Alex_Gaynor: the others only really matter for gate; but a pipeline has N queues; most of the openstack projects share one change queue in gate (because they all run the devstack jobs). but if someone approves, say, a stackforge project change, it goes into its own change queue in the gate pipeline | 18:18 |
| mordred | but I'm calling it from subprocess | 18:18 |
| dstufft | mordred: I mean I think ``python`` adds '' to sys.path but python being executed from a #! line doesn't | 18:19 |
| Alex_Gaynor | jeblair: ah right, I was going to just do sum(len(queue) for queue in pipeline) to start with, hopefully that's pretty intuitive? | 18:19 |
| mordred | dstufft: AHHHHHHHHHHHHHHHHHHHH | 18:19 |
| dstufft | mordred: I'm doing educated guessing here so I could be wrong | 18:19 |
| * mordred shoves an icepick in his eye | 18:19 | |
| mordred | dstufft: no, that would make sense | 18:19 |
| jeblair | Alex_Gaynor: so basically, a pipeline has N queues. A queue may have any number of heads. usually just one, but when something fails while at the head, we sever it, and it becomes it's own queue head while it finishes tests | 18:19 |
| Alex_Gaynor | jeblair: ah right, so I think you want to dedupe a thing, basiaclly you want to show the count of visible boxes :) | 18:20 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Enable Load Balancer as a Service https://review.openstack.org/42119 | 18:20 |
| jeblair | Alex_Gaynor: well, it's a good question... | 18:20 |
| *** HenryG has joined #openstack-infra | 18:20 | |
| Alex_Gaynor | jeblair: my intuition that showing anything besides "count of boxes" is going to result in "why is this count wrong" questions | 18:21 |
| mordred | dstufft: you are correct sir | 18:21 |
| *** melwitt has joined #openstack-infra | 18:22 | |
| jeblair | Alex_Gaynor: that's probably a good start then. :) it doesn't answer the question of "how many things are lined up to be merged?", because thet's the length of head[0] in each gate pipeline queue (disregarding head[1:]) | 18:22 |
| jeblair | Alex_Gaynor: but i agree, number of boxes is useful and easy to understand | 18:22 |
| jeblair | Alex_Gaynor: so that's len(head) for each head in each queue | 18:22 |
| mgagne | clarkb: cgit cache is disabled on git.o.o. Might be the cause of the load. | 18:23 |
| mordred | dstufft: python -c 'import sys; import pip.__init__; sys.path=sys.path[1:]; pip.__init__.main()' | 18:23 |
| mordred | dstufft: python -c 'import sys; import pip.__init__; sys.path=sys.path[1:]; pip.__init__.main()' install | 18:24 |
| mordred | dstufft: works | 18:24 |
| jeblair | Alex_Gaynor: thanks, and have fun! :) oh, you can see 3 change queues in the gate pipeline if you look right now | 18:24 |
| dstufft | mordred: yea I was about to say that | 18:24 |
| dstufft | or something like it | 18:24 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/git-review: Preserve merges when performing rebases. https://review.openstack.org/42187 | 18:24 |
| Alex_Gaynor | jeblair: err, looks like 5 changes? | 18:24 |
| dstufft | mordred: also to inject my typical if you don't do crazy things like make setup.py install call pip you don't get crazy problems ;P | 18:24 |
| jog0 | mordred: I did local for the path because thats what it was for me ... not the right reasoning of course | 18:24 |
| Alex_Gaynor | jeblair: there's 3 in the "main" one, but 2 other queues | 18:24 |
| clarkb | mordred: fungi jeblair ^ not sure how I feel about that. Slightly worried it trades one set of unexpectedness with another | 18:24 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Migrate to pbr. https://review.openstack.org/35486 | 18:25 |
| dstufft | mordred: you might want to ensure that the thing you're removing is actually '', I think eggs might shove themselve in front of the line | 18:25 |
| mordred | dstufft: yeah. I'll do that | 18:25 |
| jeblair | Alex_Gaynor: yep | 18:25 |
| dstufft | so sys.path = [x for x in sys.path if x != ''] or something | 18:25 |
| mordred | dstufft: also, if easy_install wasn't completely and totally useless, I wouldn't be mucking around like this | 18:25 |
| jeremyb | fakebeer! | 18:25 |
| clarkb | mgagne: pleia2: was disabling the cache a specific decision or did we simply miss adding a flag somewhere? | 18:25 |
| mordred | dstufft: and if upstream setuptools wasn't brain-numbingly insistent on keeping such a terrible aweful horrible pile of garbage, I wouldn't be working around it | 18:26 |
| mordred | dstufft: perfect | 18:26 |
| pleia2 | clarkb: intentional, it wasn't updating very quickly, could just turn down the cache | 18:26 |
| Alex_Gaynor | jeblair: is there a sane way to test this locally? | 18:28 |
| jog0 | is netaddr still failing ? | 18:28 |
| jog0 | for jenkins | 18:28 |
| clarkb | jog0: it didn't look like it | 18:29 |
| fungi | clarkb: yeah, i kinda wonder whether git-review shouldn't grow another --danger-will-robinson flag to make sure people are intending to submit merge commits | 18:29 |
| *** SnowDust has quit IRC | 18:29 | |
| clarkb | jog0: there was a change that had been rechecked that ran without running into the netaddr stuff | 18:29 |
| jog0 | clarkb: kk I will try again and hope for magic | 18:30 |
| *** nayward has joined #openstack-infra | 18:30 | |
| jeblair | Alex_Gaynor: yeah, with the html and js files local, you should be able to just it with a file:// url | 18:31 |
| Alex_Gaynor | jeblair: I appear to have written some javascript that worked the first time. Pinch me I'm dreaming! | 18:31 |
| jeblair | Alex_Gaynor: i don't think that's never happened to me :) | 18:32 |
| jeblair | mordred: nodepool pbr status? | 18:32 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Show a count of changes in each pipeline on the status page https://review.openstack.org/42189 | 18:33 |
| Alex_Gaynor | jeblair: ^ | 18:33 |
| clarkb | I have finally skimmed this mornings scrollback and I missed all the fun :( | 18:34 |
| jeblair | Alex_Gaynor: you even fixed my js :) | 18:35 |
| *** rnirmal has quit IRC | 18:35 | |
| Alex_Gaynor | jeblair: my editor whines (it also whines about using == 0 instead of === 0, but I didn't want to dirty up the diff too much) | 18:35 |
| dstufft | Alex_Gaynor: my editor locks up trying to whine about the pypi code base :[ | 18:36 |
| jerryz | hi guys, do you know how could i pass http_proxy to devstack-vm-gate-wrap.sh so that pip install and apt-get will use proxy ? | 18:37 |
| jerryz | i tried adding export http_proxy in the jenkins build step, but it didn't work | 18:37 |
| mordred | jeblair: almost done- I wanted to verify that a particular fix could be used in the first place to avoid the issue | 18:38 |
| *** mrmartin has joined #openstack-infra | 18:42 | |
| fungi | jerryz: for apt, you need to specify your proxy in apt.conf usually... http://itkia.com/using-squid-to-cache-apt-updates-for-debian-and-ubuntu/ | 18:45 |
| *** ryanpetrello has joined #openstack-infra | 18:46 | |
| fungi | at least that was the case last time i tried (these days i just run an apt-specific proxy daemon on a central server and point all my sources.list entries at it instead) | 18:46 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gear: fix project dependencies https://review.openstack.org/42194 | 18:47 |
| *** dkliban_afk is now known as dkliban | 18:48 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Reduce appearance of pipeline counts https://review.openstack.org/42195 | 18:49 |
| clarkb | jerryz: if using a jenkins shell build step you will need to do it in the same build step as the one that runs d-g | 18:49 |
| jeblair | Alex_Gaynor: I like your change, but I thought it could use a few adjustments ^ | 18:50 |
| clarkb | jerryz: env changes don't carry over across distinct shell builders. you can use the env inject plugin to make them carry over | 18:50 |
| Alex_Gaynor | jeblair: hah! I thought about doing the "ignore if 0" bit, but decided not to, I can't remember why | 18:50 |
| jerryz | env inject plugin is for jenkins , right | 18:50 |
| jeblair | Alex_Gaynor: probably a very important reason. :) | 18:50 |
| Alex_Gaynor | jeblair: yoru change looks good to me, how does this work, do we just merge them in order, do we approve yours and then I manaully roll it into mine? | 18:50 |
| clarkb | jerryz: correct | 18:50 |
| jeblair | Alex_Gaynor: i made mine depend on yours, so we can just merge both (zuul will order them) | 18:50 |
| Alex_Gaynor | jeblair: yay, software! | 18:51 |
| jeblair | Alex_Gaynor: though i just approved another change to the page -- we may want to make sure these rebase onto that, and we may also want to consider how they should interact... | 18:51 |
| jeblair | Alex_Gaynor: https://review.openstack.org/#/c/39773/ | 18:51 |
| jeblair | Alex_Gaynor: that one lets you filter by project name, so i wonder if the numbers should change with the filter or not. | 18:52 |
| Alex_Gaynor | jeblair: hmm, I could go either way on this | 18:52 |
| *** rnirmal has joined #openstack-infra | 18:52 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add projects filter to zuul dashboard https://review.openstack.org/39773 | 18:52 |
| mordred | jeblair: you should be set | 18:52 |
| * mordred afk for a bit | 18:52 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Show a count of changes in each pipeline on the status page https://review.openstack.org/42189 | 18:53 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Reduce appearance of pipeline counts https://review.openstack.org/42195 | 18:53 |
| jeblair | Alex_Gaynor: ^ i fixed the merge conflict | 18:54 |
| clarkb | woo elasticsearch cluster splitbrained | 18:54 |
| Alex_Gaynor | jeblair: doh, I was about to do that. Do you have an opinion on whether the count should update? | 18:54 |
| jeblair | Alex_Gaynor: let's go with no for now? | 18:55 |
| Alex_Gaynor | jeblair: sounds easier to me! | 18:55 |
| Alex_Gaynor | jeblair: I think this is shipit ready then | 19:00 |
| jeblair | yep! | 19:00 |
| fungi | clarkb: did you have details on the netaddr fails? i was going to try to triage bug 1212751 | 19:01 |
| uvirtbot | Launchpad bug 1212751 in openstack-ci "netaddr could not be downloaded / installed" [Undecided,New] https://launchpad.net/bugs/1212751 | 19:01 |
| clarkb | fungi: I don't | 19:02 |
| fungi | le sigh | 19:03 |
| clarkb | network trouble? that is what my elasticsearch cluster derp appears to be related to | 19:06 |
| *** moted has quit IRC | 19:06 | |
| clarkb | timeouts on requests | 19:06 |
| *** rfolco has quit IRC | 19:06 | |
| fungi | mebbe | 19:06 |
| clarkb | fungi: [2013-08-15 16:40:36,967][WARN ][transport ] [elasticsearch] Received response for a request that has timed out, sent [82898ms] ago, timed out [52897ms] ago | 19:07 |
| clarkb | is that when the netaddr failures were happening? | 19:07 |
| *** ryanpetrell has joined #openstack-infra | 19:08 | |
| jeblair | mordred: nodepool is unable to run as the nodepool user because of prettytable permissions. | 19:09 |
| jeblair | mordred: i will rewrite it in go. | 19:09 |
| clarkb | jeblair: I approve of that strategy | 19:10 |
| dtroyer | jeblair: ++ | 19:10 |
| clarkb | elasticsearch cluster health is yellow now. Everything should be happy again in a little bit | 19:11 |
| *** ryanpetrello has quit IRC | 19:11 | |
| clarkb | thats annoying. I will need to play with timeout values | 19:11 |
| jeblair | clarkb: how do we see the nifty page again? | 19:11 |
| *** sarob has joined #openstack-infra | 19:12 | |
| clarkb | jeblair: ssh elasticsearch.o.o -L9200:localhost:9200 then open https://github.com/lukas-vlcek/bigdesk against localhost:9200 | 19:13 |
| jeblair | oh | 19:13 |
| clarkb | you can do the same wtih https://github.com/mobz/elasticsearch-head to see different data | 19:13 |
| *** ryanpetrell has quit IRC | 19:13 | |
| clarkb | assuming those are the nifty pages you are talking about | 19:13 |
| jeblair | clarkb: i thought there was something a bit less involved | 19:14 |
| clarkb | I haven't made them publicly available because they let you do more dangerous things | 19:14 |
| clarkb | jeblair: its not too invovled. I have them cloned and open them locally, but it is more work than open url | 19:14 |
| *** vipul is now known as vipul-away | 19:14 | |
| jerryz | clarkb: Env injection still does not work. | 19:14 |
| clarkb | jerryz: :/ not sure then | 19:14 |
| jerryz | clarkb: it seems that export_proxy_variables is called in stack.sh and install_prereqs.sh | 19:15 |
| jerryz | 166function export_proxy_variables() { | 19:16 |
| jerryz | 167 if [[ -n "$http_proxy" ]]; then | 19:16 |
| jerryz | 168 export http_proxy=$http_proxy | 19:16 |
| jerryz | 169 fi | 19:16 |
| jerryz | 170 if [[ -n "$https_proxy" ]]; then | 19:16 |
| jerryz | 171 export https_proxy=$https_proxy | 19:16 |
| jerryz | 172 fi | 19:16 |
| jerryz | 173 if [[ -n "$no_proxy" ]]; then | 19:16 |
| jerryz | 174 export no_proxy=$no_proxy | 19:16 |
| jerryz | 175 fi | 19:16 |
| clarkb | jerryz: do they need to go in the localrc then? | 19:16 |
| clarkb | I would give that a shot | 19:16 |
| *** sarob has quit IRC | 19:17 | |
| clarkb | jeblair: fungi mordred for the record I restarted the elasticsearch.o.o's elasticsearch process which forced it to look for the master again and everything started recovering at that point | 19:17 |
| jeblair | mordred: that was an fyi, btw; i ran chmod to fix it. | 19:18 |
| *** colinmcnamara has quit IRC | 19:19 | |
| fungi | clarkb: wonder if intermittent packet loss or something got it confused? | 19:21 |
| clarkb | fungi: I think so. It reports that it eventually got a response for a request that had timed out almost a minute ago | 19:21 |
| jerryz | clarkb: how to put it in localrc, change the code ? | 19:22 |
| clarkb | jerryz: let me double check d-g | 19:22 |
| fungi | clarkb: the netaddr pip download failures look like they were around 1600 utc today | 19:22 |
| fungi | so probably | 19:22 |
| clarkb | jerryz: if you export KEEP_LOCALRC then devstack-gate won't delete it | 19:23 |
| clarkb | jerryz: that will allow you to just add it to the beginning of the file and all of the other options will be appended by devstack-gate | 19:23 |
| clarkb | jerryz: and assiging any value to KEEP_LOCALRC should work. The check in devstack-gate is just that that variable is not empty | 19:25 |
| *** ryanpetrello has joined #openstack-infra | 19:25 | |
| jerryz | how localrc is generated | 19:26 |
| jerryz | i see it is in devstack-vm-gate.sh | 19:26 |
| clarkb | jerryz: it is just a file that is sourced by devstack | 19:27 |
| clarkb | jerryz: so you can just edit it | 19:27 |
| *** radix has left #openstack-infra | 19:27 | |
| *** vipul-away is now known as vipul | 19:27 | |
| jerryz | but there is no localrc in the devstack code dir | 19:28 |
| jerryz | i created it? | 19:29 |
| clarkb | yes you can create it | 19:29 |
| jerryz | KEEP_LOCALRC can by injected? | 19:30 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Migrate to pbr. https://review.openstack.org/35486 | 19:31 |
| clarkb | yes KEEP_LOCALRC=1. Either inject it or do export KEEP_LOCALRC=1 prior to calling devstack gate in the same shell | 19:31 |
| jerryz | thank you. i will let you know if it works. | 19:32 |
| Alex_Gaynor | Did something change about the allocation of build servers recently, there seem to be way fewer non-devstack builders | 19:33 |
| clarkb | Alex_Gaynor: yes, and there aren't fewer works the graph is just wrong | 19:34 |
| Alex_Gaynor | clarkb: hmm, I wasn't looking at the graph, was just observing that those jobs seem to be spending more time in queued | 19:34 |
| clarkb | Alex_Gaynor: when we switched to mutliple jenkins masters each one got its own slave launcher worker and they all overwrite the same graph | 19:35 |
| Alex_Gaynor | hehe | 19:35 |
| clarkb | Alex_Gaynor: oh the problem there is we are just running a ton of tests | 19:35 |
| Alex_Gaynor | devstack seems to be doing a better job of keeping up is all, which seems backwards given those tests take longer | 19:36 |
| clarkb | Alex_Gaynor: I don't have numbers to support it but I think flakyness in the gate has caused a lot of gate restarts which consumes a lot of resources before the check queue can get them | 19:36 |
| jeblair | there is one idle precise builder: precise16 | 19:36 |
| jeblair | we should see if there's a bug there | 19:36 |
| Alex_Gaynor | I think that's true as well. ANyways just idle curiosity. | 19:36 |
| jeblair | but other than that, yeah, just busy. | 19:36 |
| Alex_Gaynor | good problems to have all said and done | 19:36 |
| clarkb | Alex_Gaynor: the nodepool stuff jeblair is working on will centralize the management of servers again which will make the graphs correct | 19:36 |
| *** rfolco has joined #openstack-infra | 19:41 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Add check for deprecated method assertEquals https://review.openstack.org/35962 | 19:44 |
| jeblair | it looks like jenkins decided to reconnect to precise16, and in the process, the gearman worker stopped, and did not restart. | 19:47 |
| clarkb | mordred: https://review.openstack.org/#/c/42178/1 passed testing \o/ | 19:47 |
| jeblair | i put it through a reconnection cycle and it's running jobs now | 19:47 |
| *** ruhe has quit IRC | 19:49 | |
| *** alexpilotti has joined #openstack-infra | 19:49 | |
| clarkb | mordred: are we not putting pbr in the requirements.txt anymore? | 19:52 |
| jerryz | clarkb: the localrc worked | 19:52 |
| clarkb | fungi: ^ I ask because the pbr for git-review change does not have it in requirements.txt | 19:52 |
| *** pcm__ has quit IRC | 19:52 | |
| clarkb | oh the runtime req is only for versioning. I see. | 19:53 |
| fungi | clarkb: i gather if it's not a runtime requirement it's not needed in requirements.txt, only in setup_requires | 19:53 |
| fungi | yeah, that | 19:53 |
| fungi | though i'm still struggling on replacing it with pkg_resources a little | 19:53 |
| fungi | clarkb: http://paste.openstack.org/show/44256 | 19:54 |
| fungi | distutils seems to dislike the version string... "ValueError: invalid version number '1.23.a3.g70655c3' | 19:55 |
| clarkb | StrictVersion doesn't seem to handle the dev version | 19:55 |
| clarkb | you might need to truncate the version? | 19:56 |
| clarkb | mordred: ^ you know all things related to versioning | 19:56 |
| fungi | is there an alternative to StrictVersion() which groks that? | 19:56 |
| clarkb | fungi: http://epydoc.sourceforge.net/stdlib/distutils.version.StrictVersion-class.html | 19:56 |
| clarkb | one of their examples clearly shows that our version is not valid | 19:56 |
| clarkb | fungi: yes, possibly the thing that the requirements checker uses | 19:57 |
| clarkb | jerryz: great! | 19:57 |
| clarkb | hmm we on;y check equality in that test | 19:57 |
| *** boris-42 has joined #openstack-infra | 19:58 | |
| fungi | clarkb: i wonder if it's worth dropping that misfeature from g-r anyway | 19:58 |
| fungi | clarkb: i hope we're nearing the stability level that people will be installing it from their distros | 19:59 |
| clarkb | fungi: I was going to suggest that :) | 19:59 |
| fungi | that's because ur smrt | 19:59 |
| fungi | whipping up a patch for that, and i'll slip it in ahead of the pbr patch | 20:00 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add static web directory for cgit & initial files https://review.openstack.org/42168 | 20:00 |
| clarkb | mordred: https://review.openstack.org/#/c/41927/ seems wrong | 20:01 |
| clarkb | if milestone proposed is off in never never land then the verison of master *is* different | 20:01 |
| clarkb | what purpose does milestone proposed branch servce if we epect all of its content to be in master already? | 20:02 |
| clarkb | fungi: having tests is so cool http://logs.openstack.org/87/42187/1/check/gate-git-review-python27/fe5a602/console.html | 20:05 |
| fungi | clarkb: i saw ;) | 20:06 |
| *** sandywalsh has quit IRC | 20:09 | |
| *** mriedem has quit IRC | 20:10 | |
| jeblair | clarkb: it's a one way relationship; everything in mp should be in master; everything in master should not be in mp. | 20:14 |
| *** sandywalsh has joined #openstack-infra | 20:20 | |
| *** mriedem has joined #openstack-infra | 20:25 | |
| *** mrodden has quit IRC | 20:26 | |
| *** ^d has quit IRC | 20:26 | |
| *** ^d has joined #openstack-infra | 20:27 | |
| *** ^d has joined #openstack-infra | 20:27 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add static web directory for cgit & initial files https://review.openstack.org/42168 | 20:28 |
| *** ^d has quit IRC | 20:31 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Handle paramiko and daemonization https://review.openstack.org/42211 | 20:32 |
| jeblair | fungi: i was wrong ^ | 20:32 |
| jeblair | i'm self-approving that ^ | 20:33 |
| *** gyee has joined #openstack-infra | 20:33 | |
| fungi | jeblair: ahh | 20:34 |
| fungi | yes, i recall that bug now | 20:35 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Handle paramiko and daemonization https://review.openstack.org/42211 | 20:36 |
| *** dprince has quit IRC | 20:36 | |
| clarkb | jaypipes: can I get you to look at https://review.openstack.org/#/c/42082/ ? | 20:38 |
| *** mrodden has joined #openstack-infra | 20:38 | |
| clarkb | jaypipes: last time removing the devstack exercises from the gate came up you had reasons that we shouldn't do that. Curious to know if things have changedsince then | 20:39 |
| zaro | jeblair: i noticed that the gearman plugin is only running jobs one executor on each slave. iirc this was on purpose but i don't remember the reason. could you remind me? | 20:39 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: No longer check for new git-review releases https://review.openstack.org/42214 | 20:39 |
| jeblair | zaro: we only have one executor on each slave | 20:39 |
| zaro | jeblair: do we want to restrict that if others have multiple executors? | 20:40 |
| jeblair | zaro: no | 20:40 |
| zaro | jeblair: are there an technical reason we want to restrict to one executor ? | 20:41 |
| zaro | jeblair: if not should we remove it? or maybe make it an option? | 20:42 |
| jeblair | zaro: i don't understand the question | 20:42 |
| jeblair | zaro: i thought you were talking about what we're currently running. are you saying there's a bug in the plugin? | 20:42 |
| jeblair | zaro: if there is, it should be fixed. | 20:42 |
| zaro | jeblair: i mean do you think it would break anything we are doing if we removed the restriction of runnning jobs on only one executor per slave? | 20:44 |
| jeblair | zaro: i don't believe there is intended to be a restriction | 20:45 |
| *** vipul is now known as vipul-away | 20:45 | |
| clarkb | ya I don't think we have that restriction. I do think the plugin will not handle >1 executor worker thread very gracefully in some cases though | 20:45 |
| jeblair | zaro: if that is happening, it is a bug. it should be fixed. | 20:45 |
| *** colinmcnamara has joined #openstack-infra | 20:46 | |
| jeblair | clarkb: the only thing it is not expected to handle is the number of executors changing | 20:46 |
| zaro | jeblair: ohh. ok. that's what's happening. it looks like the NodeAvailability locks when a buid is run then doesn't release until the job completes. | 20:46 |
| jeblair | zaro: ah, that would do it. | 20:47 |
| *** nayward has quit IRC | 20:47 | |
| jeblair | zaro: i introduced that bug when adding the disable on complete feature | 20:47 |
| zaro | i believe line 384 in MyGearmanWorkerImpl | 20:47 |
| openstackgerrit | A change was merged to openstack/requirements: Require lesscpy>=0.9j https://review.openstack.org/40910 | 20:48 |
| jeblair | zaro: perhaps we could call unlock as we did before if that flag is not set. | 20:49 |
| *** ^d has joined #openstack-infra | 20:50 | |
| zaro | jeblair: that seems like an option. but it seem also seems like you would never use that flag with multiple executors. | 20:50 |
| zaro | jeblair: iirc that flag doens't work correctly with multiple executors? | 20:51 |
| jeblair | zaro: depends on what you consider correct. i actually think the current behavior is the best behavior if that flag is set. | 20:52 |
| jeblair | zaro: it meas no more jobs will be run on the node, on any executor. in progress ones will finish. | 20:52 |
| jeblair | i think that's ideal | 20:52 |
| jeblair | so then just moving the unlock back to where it was in the case that the flag is not set should be fine. | 20:52 |
| jeblair | (and multiple unlocks should be fine, actually) | 20:53 |
| jeblair | (so it should be okay to leave the workerimpl alone and just add the unlock back to the old location in the startjobworker) | 20:53 |
| zaro | jeblair: i think that sounds good to me. | 20:54 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Migrate to pbr. https://review.openstack.org/35486 | 20:56 |
| *** vipul-away is now known as vipul | 20:56 | |
| *** ryanpetrello has quit IRC | 20:57 | |
| *** ryanpetrello has joined #openstack-infra | 20:58 | |
| clarkb | fungi: are you going to reattach https://review.openstack.org/#/c/35104/ to the queue or leave it as is? | 21:01 |
| *** nati_uen_ has joined #openstack-infra | 21:01 | |
| clarkb | fungi: I think there may be a merge conflict if left as is | 21:01 |
| fungi | clarkb: i am, working on it now | 21:01 |
| clarkb | great, I will hold off on reviewing it then | 21:02 |
| *** rfolco has quit IRC | 21:04 | |
| *** nati_ueno has quit IRC | 21:04 | |
| *** ArxCruz has quit IRC | 21:05 | |
| *** thomasbiege has joined #openstack-infra | 21:06 | |
| *** dkliban has quit IRC | 21:07 | |
| *** thomasbiege has quit IRC | 21:07 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Implement integration tests. https://review.openstack.org/35104 | 21:08 |
| fungi | clarkb: ^ | 21:08 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Show a count of changes in each pipeline on the status page https://review.openstack.org/42189 | 21:09 |
| fungi | i'm going to go grab dinner and then check back in to see if those are sane | 21:10 |
| *** melwitt has quit IRC | 21:10 | |
| fungi | my local tests are happy, but jenkins is slow at the moment and i want to see what it has to say | 21:10 |
| clarkb | fungi: ok. The first two lgtm | 21:10 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Show a count of changes in each pipeline on the status page https://review.openstack.org/42189 | 21:11 |
| clarkb | I intend on giving the change that adds tests a more rigorous review as it has been hit with a hammer a bunch | 21:11 |
| *** melwitt has joined #openstack-infra | 21:11 | |
| fungi | yes, it has been beaten into shape afaict, but more beatings can't hurt | 21:11 |
| clarkb | I miss the merge notifications in #openstack-dev. I actually have to check if the gate is going now | 21:11 |
| jeblair | clarkb: the everything is okay alarm? :) | 21:12 |
| *** moted has joined #openstack-infra | 21:12 | |
| fungi | hey! wake up! everything's fine and working normally! | 21:12 |
| jeblair | http://simpsonswiki.com/wiki/Everything%27s_OK_alarm | 21:13 |
| * fungi had forgotten about that episode | 21:13 | |
| clarkb | exactly | 21:14 |
| jeblair | it's really informed my thinking about ops. :) | 21:14 |
| fungi | ahh, season 10. i may not have seen that one as many times | 21:14 |
| * fungi dines | 21:14 | |
| jeblair | progress: http://paste.openstack.org/show/44274/ | 21:15 |
| clarkb | I should just run a lcoal instance of the bot and have it prv message me | 21:15 |
| clarkb | woot | 21:15 |
| *** mrmartin has quit IRC | 21:16 | |
| jeblair | unfortunately (fortunately?) it doesn't log output from the setup ssh session. so it's a bit hard to tell if, well, everything's OK. :) | 21:16 |
| *** sarob has joined #openstack-infra | 21:16 | |
| clarkb | jeblair: how did you work around the pbr issue? | 21:17 |
| jeblair | clarkb: mordred fixed it. no idea. i'm hoping he'll have it really fixed by the time another servers gets spun up. | 21:18 |
| *** sarob has quit IRC | 21:19 | |
| *** odyssey4me has quit IRC | 21:19 | |
| *** sarob has joined #openstack-infra | 21:19 | |
| *** dkliban has joined #openstack-infra | 21:20 | |
| *** sarob has quit IRC | 21:23 | |
| jeblair | i think the level of concurrency in nodepool is not particularly compatible with sqlite | 21:30 |
| jeblair | i'll see about having it talk to mysql instead | 21:30 |
| clarkb | jeblair: not mongodb? | 21:30 |
| clarkb | webscale | 21:30 |
| *** sarob has joined #openstack-infra | 21:31 | |
| jog0 | where is the source for the recheck page? | 21:32 |
| jeblair | clarkb: i'm copying your mysql stuff from etherpad; you think that's the best? | 21:32 |
| jeblair | jog0: infra/config | 21:32 |
| jog0 | thanks | 21:32 |
| *** sarob has quit IRC | 21:32 | |
| *** sarob has joined #openstack-infra | 21:33 | |
| clarkb | jeblair: no, I would look at puppetlabs mysql | 21:33 |
| jeblair | oh, do we use that anywhere? | 21:33 |
| clarkb | jeblair: ya, we use it for gerrit and the jenkins slaves iirc | 21:33 |
| *** sarob has quit IRC | 21:33 | |
| *** sarob has joined #openstack-infra | 21:34 | |
| jeblair | clarkb: why doesn't ep use it? | 21:34 |
| clarkb | jeblair: because mordred didn't port it when he was porting things. It should use it | 21:34 |
| clarkb | jeblair: I believe it was just overlooked | 21:34 |
| jeblair | k | 21:34 |
| *** mriedem has quit IRC | 21:38 | |
| *** colinmcnamara has quit IRC | 21:39 | |
| *** afazekas is now known as afazekas_zz | 21:42 | |
| *** HenryG has quit IRC | 21:42 | |
| *** colinmcnamara has joined #openstack-infra | 21:47 | |
| *** markmcclain has quit IRC | 21:47 | |
| *** nicedice has quit IRC | 21:47 | |
| *** weshay has quit IRC | 21:48 | |
| *** nicedice has joined #openstack-infra | 21:48 | |
| openstackgerrit | A change was merged to openstack/requirements: Bump the pip version and add wheel https://review.openstack.org/41217 | 21:50 |
| *** rnirmal has quit IRC | 21:50 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gearman-plugin: remove restriction on slave to run single job at a time https://review.openstack.org/42226 | 21:57 |
| *** zul has quit IRC | 21:57 | |
| *** prad has quit IRC | 22:01 | |
| *** burt has quit IRC | 22:01 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack/requirements: Allow use of oslo.messaging 1.2.0a5 https://review.openstack.org/42229 | 22:02 |
| *** prad has joined #openstack-infra | 22:02 | |
| *** zul has joined #openstack-infra | 22:02 | |
| *** pentameter has quit IRC | 22:04 | |
| *** nicedice has quit IRC | 22:04 | |
| *** datsun180b_ has joined #openstack-infra | 22:07 | |
| *** chmouel is now known as chmouel_away | 22:07 | |
| clarkb | jeblair: zaro: does adding that unlock to StartJobWorker prematurely unlock the monitor in the case of single use slaves? | 22:09 |
| zaro | clarkb: don't think so the lock happens pretty early. i've tested it with single use and doesn't seem to be a problem. | 22:10 |
| *** datsun180b has quit IRC | 22:11 | |
| *** datsun180b_ is now known as datsun180b | 22:11 | |
| *** prad has quit IRC | 22:12 | |
| jeblair | i like that i now actually have the nice mysql command line client instead of the sqlite one. | 22:12 |
| *** _TheDodd_ has quit IRC | 22:17 | |
| clarkb | jeblair: can you see my comment on https://review.openstack.org/#/c/42226/ | 22:18 |
| *** markmcclain has joined #openstack-infra | 22:18 | |
| jeblair | clarkb, zaro: ^ commented | 22:20 |
| *** vipul is now known as vipul-away | 22:21 | |
| clarkb | jeblair: even in that case how is it safe to have unlocked the monitor before starting a job | 22:22 |
| jeblair | clarkb: it's probably not. it should be added where it used to be. | 22:23 |
| jeblair | fungi, clarkb: i've disabled puppet on jenkins01 and jenkins02 to update the firewall there (to allow zmq connections from nodepool) | 22:25 |
| *** colinmcnamara has quit IRC | 22:25 | |
| *** changbl has quit IRC | 22:25 | |
| *** nicedice has joined #openstack-infra | 22:26 | |
| *** colinmcnamara has joined #openstack-infra | 22:26 | |
| clarkb | jeblair: ok | 22:27 |
| clarkb | zaro: jeblair: I updated my review on 42226 with the old location as determined by git log -p | 22:28 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add nodepool host https://review.openstack.org/42232 | 22:29 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Make the local script directory configurable https://review.openstack.org/42233 | 22:32 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Use MySQL https://review.openstack.org/42234 | 22:32 |
| *** nati_uen_ has quit IRC | 22:34 | |
| *** nati_ueno has joined #openstack-infra | 22:36 | |
| *** nicedice has quit IRC | 22:38 | |
| *** sarob has quit IRC | 22:38 | |
| *** zul has quit IRC | 22:39 | |
| jeblair | 2013-08-15 22:39:07,464 INFO nodepool.NodeLauncher: Node id: 25 added to jenkins | 22:39 |
| clarkb | jeblair: is that the first one? | 22:41 |
| *** alexpilotti has quit IRC | 22:42 | |
| jeblair | yeah; unfortunately they failed tests... | 22:42 |
| jeblair | we need to change the inprogress and update jobs to exit 0 if they can't find the node | 22:43 |
| *** vipul-away is now known as vipul | 22:43 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gearman-plugin: remove restriction on slave to run single job at a time https://review.openstack.org/42226 | 22:43 |
| *** nicedice has joined #openstack-infra | 22:46 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: Ignore node not found error https://review.openstack.org/42235 | 22:47 |
| jeblair | clarkb, fungi, mordred: ^ can you give that a speedy review? | 22:47 |
| clarkb | jeblair: yes, it is next up | 22:48 |
| *** zul has joined #openstack-infra | 22:48 | |
| *** danger_fo is now known as danger_fo_away | 22:55 | |
| *** boris-42 has quit IRC | 23:01 | |
| *** wenlock has quit IRC | 23:06 | |
| *** markmcclain has quit IRC | 23:08 | |
| *** vipul is now known as vipul-away | 23:09 | |
| *** zul has quit IRC | 23:09 | |
| *** datsun180b has quit IRC | 23:10 | |
| *** vipul-away is now known as vipul | 23:17 | |
| *** sarob has joined #openstack-infra | 23:17 | |
| *** woodspa has quit IRC | 23:21 | |
| *** blamar has quit IRC | 23:21 | |
| *** sarob has quit IRC | 23:22 | |
| *** sarob has joined #openstack-infra | 23:25 | |
| *** rcleere has quit IRC | 23:26 | |
| clarkb | I'm about to walk home. I do plan on putting the new mysql backup stuff on etherpad.o.o after 0000UTC if logrotate does the correct thing when run by hand on etherpad-dev | 23:27 |
| clarkb | jeblair: then we can start adding that everywhere and harness the power of bup | 23:28 |
| jeblair | clarkb: yaay! | 23:28 |
| jeblair | i'm a few minutes away from trying nodepool again | 23:28 |
| clarkb | jeblair: I intend on reviewing the nodepool changes once that is done | 23:28 |
| clarkb | jeblair: though it looks like puppet lint barfed on at least one of the changes | 23:29 |
| jeblair | cool. i think i have style checks to fix -- those changes are straight from production | 23:29 |
| jeblair | clarkb, fungi, mordred: nodepool nodes can be distinguished from devstack-gate nodes by their having < 5 digit id numbers. | 23:30 |
| clarkb | jeblair: because they started at 0? | 23:31 |
| jeblair | clarkb: well, 1. and actually by the time they actually started working it was up to 25. but yeah. :) | 23:31 |
| clarkb | jeblair your d-g change failed tests :/ | 23:33 |
| jeblair | clarkb: the gate tests are still running | 23:33 |
| clarkb | oh was that just check? | 23:34 |
| jeblair | clarkb: yeah. heh, and it failed because it ran on a nodepool node. :) | 23:34 |
| *** changbl has joined #openstack-infra | 23:35 | |
| *** prad has joined #openstack-infra | 23:37 | |
| *** ryanpetrello has quit IRC | 23:39 | |
| *** prad has left #openstack-infra | 23:42 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Ignore node not found error https://review.openstack.org/42235 | 23:43 |
| *** dkliban has quit IRC | 23:45 | |
| *** vipul is now known as vipul-away | 23:45 | |
| jeblair | okay, that seems to be working | 23:46 |
| *** pcrews has quit IRC | 23:48 | |
| *** SergeyLukjanov has quit IRC | 23:56 | |
| *** vipul-away is now known as vipul | 23:58 | |
| jeblair | and a job is running for real on a nodepool node: https://jenkins01.openstack.org/job/gate-tempest-devstack-vm-full/2303/console | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!