| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Notify in IRC when puppet run fails https://review.openstack.org/96519 | 00:01 |
|---|---|---|
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Use ansible instead of direct ssh calls https://review.openstack.org/87821 | 00:01 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add mirror management playbooks https://review.openstack.org/100066 | 00:01 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Move ansible things into a module https://review.openstack.org/97032 | 00:01 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove salt https://review.openstack.org/96361 | 00:01 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove old salt references from the puppet https://review.openstack.org/96362 | 00:01 |
| *** jamielennox|away is now known as jamielennox | 00:01 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add playbook for cleaning workspaces https://review.openstack.org/96646 | 00:01 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Rearrange into using roles https://review.openstack.org/96636 | 00:01 |
| *** SumitNaiksatam has joined #openstack-infra | 00:01 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gitdm: Change affiliation for jeblair https://review.openstack.org/102675 | 00:03 |
| *** jswarren has quit IRC | 00:03 | |
| *** mestery has quit IRC | 00:04 | |
| clarkb | mordred: jeblair: what do you think? do nova now or flip the order? | 00:05 |
| jeblair | mordred: the end date you put in stackalytics for my previous tenure at hp is curious... | 00:06 |
| *** sileht has quit IRC | 00:06 | |
| jeblair | mordred: it's not my actual hp end date, however, it is 2 years to the day before my foundation end-date. you've clearly been planning something for a while. | 00:07 |
| jeblair | clarkb: i'm not actually here to deal with consequences atm | 00:08 |
| clarkb | ok | 00:08 |
| *** shayneburgess has quit IRC | 00:10 | |
| mordred | jeblair: haha. well, I have no idea where I got the date then | 00:11 |
| *** asettle has joined #openstack-infra | 00:11 | |
| jeblair | mordred: it's too bad -- the error meant that hp missed out on about 6 months of my stats | 00:12 |
| mordred | oh noes! | 00:12 |
| jeblair | you will have to revise all your powerpoints! | 00:12 |
| mordred | jeblair: bah. it's in the past now | 00:13 |
| ianw | clarkb: re the assertListEqual ... testtools does just extend unittest, and it doesn't have any special patching for assertListEqual and a few other python2.7 added unittest things, i guess that leave that for unittest2 | 00:13 |
| mordred | jeblair: I know you're not really here ... but I fixed all of the above ^^ based on your feedback | 00:13 |
| mordred | jeblair: just so you know I care :) | 00:13 |
| jeblair | mordred: cool, thanks! i'll get to review them all again tomorrow! | 00:13 |
| *** unicell has joined #openstack-infra | 00:13 | |
| clarkb | mordred: do you have an opinion on nova? or pick it up tomorrow during US Germany | 00:14 |
| *** yamahata has joined #openstack-infra | 00:14 | |
| mordred | clarkb: well, it really annoys me that there isn't a working libvirt | 00:14 |
| *** zzelle has joined #openstack-infra | 00:15 | |
| mordred | clarkb: my first choice would be "can someone give us a copy of libvirt that works" | 00:15 |
| mordred | but that leads to the road of infra maintaining a linux distro | 00:15 |
| mordred | so in the meantime, I say we turn off the nova feature, file a bug, and lean on canonical to provide a fixed libvirt in trusty backports | 00:15 |
| mordred | (not UCA) | 00:15 |
| *** zzelle has quit IRC | 00:16 | |
| jeblair | mordred: ++ | 00:16 |
| mordred | if canonical can not do that, then I think we should revisit the default linux that the gate uses | 00:16 |
| mordred | and I know that's a very hardline stance | 00:16 |
| clarkb | that sounds reasonable and doesn't affect unittesting except for that last thing you said | 00:16 |
| mordred | but the assumption on ubuntu has been that it's both modern and has a responsive host org - if it does not ship with working pieces of key functionality | 00:17 |
| clarkb | well I tink libvirt as an upstream is partially to blame | 00:17 |
| mordred | then I think that we're not getting what we need out of it and the subject shoudl at least be broached | 00:17 |
| jeblair | i mordred had a nickel for ever time he heard "that sounds reasonable except for that last thing you said" .... ;)- | 00:17 |
| clarkb | that and how nova consumes libvirt | 00:17 |
| mordred | I'm sure that they are | 00:17 |
| mordred | jeblair: :) | 00:17 |
| clarkb | mordred: fwiw I agree | 00:17 |
| mordred | clarkb: :) ... I don't WANT to agree with me | 00:18 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move cinder default test node to trusty. https://review.openstack.org/102678 | 00:20 |
| *** achanda has joined #openstack-infra | 00:22 | |
| clarkb | debian wheezy has 0.9.12 libvirt which I think is new enough for pip installing python libvirt | 00:22 |
| clarkb | but not new enough to exercise the broken code in nova | 00:22 |
| clarkb | sid has 1.2.4 | 00:23 |
| clarkb | wheezy backports has 1.2.4 as well | 00:23 |
| *** dstanek is now known as dstanek_zzz | 00:24 | |
| *** matsuhashi has joined #openstack-infra | 00:24 | |
| mordred | and 1.2.4 is the bad stuff? | 00:25 |
| *** salv-orlando has quit IRC | 00:25 | |
| clarkb | well 1.2.5 is the supposed good version | 00:25 |
| clarkb | but this hasn't actually been confirmed | 00:25 |
| *** achanda_ has quit IRC | 00:25 | |
| *** _nadya_ has quit IRC | 00:26 | |
| openstackgerrit | K Jonathan Harker proposed a change to openstack-infra/infra-specs: Put the puppet modules in their own projects/repos https://review.openstack.org/99990 | 00:31 |
| morganfainberg | sdague, clarkb, mordred, https://bugs.launchpad.net/keystone/+bug/1334466 | 00:32 |
| uvirtbot | Launchpad bug 1334466 in keystone "Eventlet Log Spamming on Client Disconnect (Broken Pipe)" [Critical,Triaged] | 00:32 |
| morganfainberg | jeblair, ^ cc as well | 00:32 |
| morganfainberg | if this really isn't critical please let me know and i can knock it down a notch. | 00:33 |
| clarkb | morganfainberg: I think its at least worht attention | 00:33 |
| clarkb | morganfainberg: if you were running keystone in prod and that happened you would probably not be happy | 00:33 |
| morganfainberg | right. it looks like heat's stack delete is disconnecting before it can get data from keystone | 00:34 |
| morganfainberg | i can't tell why that is occurring, but that appears to be the culprit | 00:34 |
| * morganfainberg has nightmares about a splunk license being consumed by keystone eventlet logs. | 00:34 | |
| clarkb | hehe | 00:35 |
| clarkb | http://logs.openstack.org/64/102664/1/check/gate-glance-python27/36aa4a5/console.html | 00:37 |
| clarkb | I swear no one ever looks at these logs | 00:37 |
| clarkb | otherwise how would ^ be acceptable | 00:37 |
| clarkb | glance spits out about a million errors then successfully complete unittesting | 00:37 |
| *** ArxCruz has joined #openstack-infra | 00:38 | |
| *** e0ne has joined #openstack-infra | 00:39 | |
| clarkb | https://bugs.launchpad.net/glance/+bug/1334467 | 00:40 |
| uvirtbot | Launchpad bug 1334467 in glance "Glance unittests are very noisy and spit out many ERRORs despite the tests succeeding" [Undecided,New] | 00:40 |
| clarkb | discovered this testing on trusty... | 00:40 |
| *** otter768 has joined #openstack-infra | 00:43 | |
| *** e0ne has quit IRC | 00:44 | |
| *** dstanek_zzz is now known as dstanek | 00:45 | |
| SlickNik | Running into a weird issue with trying to mimic devstack-gate | 00:46 |
| SlickNik | Was following the instructions at https://github.com/openstack-infra/devstack-gate/blob/master/README.rst | 00:46 |
| SlickNik | And everything works fine; up till running devstack as part of the gate hook. | 00:46 |
| SlickNik | Devstack tries to install package pre-requisites, and the "sudo apt-get" command it runs just hangs. | 00:47 |
| clarkb | SlickNik: try a manual apt-get update? | 00:48 |
| SlickNik | devstack output logs from the run at: http://paste.openstack.org/show/84935/ | 00:48 |
| SlickNik | manual apt-get update succeeds. | 00:48 |
| SlickNik | And if I try to install the same package list manually it succeeds as well. | 00:48 |
| clarkb | looks like it managed to install some things/ | 00:49 |
| SlickNik | yeah, I'm not sure what it's stuck on. | 00:50 |
| SlickNik | http://paste.openstack.org/show/84936/ shows the process list | 00:50 |
| clarkb | SlickNik: wat is your disk situation? | 00:50 |
| clarkb | df -h | 00:50 |
| *** devkulkarni has quit IRC | 00:51 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move glance default test node to trusty. https://review.openstack.org/102681 | 00:51 |
| SlickNik | clarkb: http://paste.openstack.org/show/84937/ | 00:52 |
| clarkb | weird / looks fine | 00:52 |
| clarkb | I don't know what is happening | 00:52 |
| *** _nadya_ has joined #openstack-infra | 00:52 | |
| SlickNik | Very strange — me neither :( | 00:54 |
| SlickNik | Let me try with a different image / instance, in case it's something specific to that. | 00:54 |
| SlickNik | Thanks for looking. | 00:54 |
| clarkb | SlickNik: are you on trusty or precise? | 00:54 |
| SlickNik | precise | 00:54 |
| clarkb | huh I mean I expect both to work was just curious | 00:54 |
| *** dstanek is now known as dstanek_zzz | 00:55 | |
| ianw | mattoliverau: hey, so i was thinking in parallel with my trying out the dib discussed with clarkb yesterday, if we had one of the RHEL RC images available in nodepool we could iron out any early issues with the scripts, etc so we're in good shape when centos hits | 00:57 |
| ianw | mattoliverau: but i'm just wondering what that RHEL image should be and where it would come from | 00:58 |
| ianw | mattoliverau: so was wondering if rackspace had already done any testing, etc, before i waste any time duplicating efforts | 00:59 |
| clarkb | last we tried with rhel it was a dead end | 01:00 |
| clarkb | couldn't figure out the licensing/money part of it | 01:00 |
| ianw | clarkb: yeah, this would just be the rhel rc releases for a little while, so when centos comes it *should* just switch out and "just work" | 01:01 |
| ianw | it can't be worse than starting from scratch :) | 01:01 |
| clarkb | well why not centos7 beta? | 01:02 |
| ianw | clarkb: not out yet? | 01:02 |
| clarkb | someone told me there were betas | 01:02 |
| clarkb | oh nightly images may be what they meanth | 01:02 |
| *** timrc is now known as timrc-afk | 01:02 | |
| ianw | clarkb: nothing i think we could use easily. i think the latest RHEL RC image would be the easiest place to start | 01:03 |
| clarkb | my concern with that is the pupept definitely won't work in some places | 01:03 |
| *** markmcclain has quit IRC | 01:03 | |
| clarkb | and we would be fixing it for a platform we will likely never use | 01:03 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move swift default test node to trusty. https://review.openstack.org/102686 | 01:05 |
| ianw | clarkb: i feel like the only way to know is to try. i would like to be running on the cloud provider from the start, especially cause of it being in a Xen environment which doesn't get much testing | 01:06 |
| ianw | f20's libvirt was hanging because of it, etc when we first did bring-up | 01:06 |
| *** timrc-afk is now known as timrc | 01:10 | |
| morganfainberg | clarkb, yay more trusty! | 01:12 |
| clarkb | morganfainberg: well nothing has merged today due to libvirt | 01:12 |
| morganfainberg | clarkb, :( | 01:12 |
| morganfainberg | clarkb, boooooooooo | 01:12 |
| clarkb | but I will probably reorder before calling it a day and we can do everything but nova first thing | 01:12 |
| clarkb | also you remind me to check keystone too | 01:13 |
| morganfainberg | clarkb, ^_^ | 01:13 |
| mattoliverau | ianw: Iet me see what I can find out from the public cloud rackers :) | 01:13 |
| *** gokrokve has quit IRC | 01:14 | |
| *** gokrokve has joined #openstack-infra | 01:14 | |
| *** Sukhdev has quit IRC | 01:14 | |
| ianw | mattoliverau: thanks. i was kind of hoping there was a secret menu for other images. i tried up,up,down,down,left,right,left,right,b,a but nothing :) | 01:14 |
| morganfainberg | clarkb, do you think it's worth logging (debug) on a EPIPE failure (it would be a 14x reduction in logging, and i have it setup for debug only atm) | 01:15 |
| morganfainberg | clarkb, i could see it useful. | 01:15 |
| mattoliverau | ianw: there could be.. but cause I work from home, I'm one of the forgotten rackers.. apparently I should have secret powers, but if I do I haven't been told :P | 01:15 |
| clarkb | morganfainberg: if the failure isn't actually meaningful other than eventlet getting mad then that makes sense | 01:16 |
| clarkb | morganfainberg: but I Think that depends on whether or not this is an issue that a human will have to interact with in production | 01:16 |
| morganfainberg | clarkb, yeah it might be nice to know a client disconnected prematurely | 01:16 |
| clarkb | eg if it happens do I need t orestart keystone? | 01:16 |
| morganfainberg | clarkb, well i downgraded it to debug. | 01:16 |
| morganfainberg | clarkb, for gate purposes we can correlate it. but it def. isn't INFO worth | 01:17 |
| morganfainberg | y | 01:17 |
| morganfainberg | just not sure if we even care about debug. | 01:17 |
| *** asettle has quit IRC | 01:17 | |
| morganfainberg | and no, operators shouldn't care really | 01:17 |
| clarkb | right | 01:17 |
| *** SumitNaiksatam has quit IRC | 01:17 | |
| clarkb | so we have to decide if this info is worth caring about | 01:17 |
| clarkb | but it sounds like it isn't | 01:17 |
| morganfainberg | clarkb, https://review.openstack.org/#/c/102680/ | 01:18 |
| morganfainberg | i'll raise it with keystone folks as well. i am leaning towards yanking the debug as well | 01:18 |
| *** gokrokve has quit IRC | 01:19 | |
| *** penguinRaider has joined #openstack-infra | 01:19 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move heat default test node to trusty. https://review.openstack.org/102690 | 01:25 |
| *** markmcclain has joined #openstack-infra | 01:26 | |
| *** camunoz_gone is now known as camunoz | 01:26 | |
| clarkb | morganfainberg: is keystonemiddleware in proper development yet or can we just ninja in switch to trusty? | 01:27 |
| clarkb | morganfainberg: I can test it if it is actually being used now | 01:27 |
| morganfainberg | it isn't actually being used anywhere | 01:27 |
| morganfainberg | we're still waiting to get it into devstack-gate, devstack etc | 01:27 |
| morganfainberg | i have reviews open for it | 01:27 |
| morganfainberg | but, unit tests wouldn't hurt to convert to trusty (if thats what you meant) | 01:27 |
| clarkb | cool I may just ninja you then | 01:27 |
| clarkb | ya | 01:27 |
| morganfainberg | go for it :) | 01:28 |
| *** markmcclain has quit IRC | 01:28 | |
| clarkb | morganfainberg: I am testing keystone now and will lump them together in one go | 01:29 |
| morganfainberg | cool | 01:29 |
| *** jyu has joined #openstack-infra | 01:30 | |
| clarkb | morganfainberg: I am happy to see that keystone unittests still clone keystoneclient :P | 01:32 |
| morganfainberg | clarkb, right?! so awesome. | 01:32 |
| morganfainberg | clarkb, i think all the projects should clone keystoneclient too! :P | 01:33 |
| morganfainberg | in unit tests | 01:33 |
| *** marcoemorais has quit IRC | 01:35 | |
| *** e0ne has joined #openstack-infra | 01:39 | |
| *** e0ne has quit IRC | 01:42 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move keystone default test node to trusty. https://review.openstack.org/102694 | 01:46 |
| *** dstanek_zzz is now known as dstanek | 01:46 | |
| *** nosnos has joined #openstack-infra | 01:47 | |
| *** _nadya_ has quit IRC | 01:51 | |
| *** melwitt has quit IRC | 01:51 | |
| *** timrc is now known as timrc-afk | 01:56 | |
| *** achanda has quit IRC | 01:59 | |
| *** timrc-afk is now known as timrc | 02:03 | |
| *** asettle has joined #openstack-infra | 02:05 | |
| *** packet has quit IRC | 02:05 | |
| *** prad_ has quit IRC | 02:06 | |
| *** hogepodge has quit IRC | 02:08 | |
| *** Sam-I-Am has quit IRC | 02:10 | |
| *** achanda has joined #openstack-infra | 02:11 | |
| *** hogepodge has joined #openstack-infra | 02:12 | |
| *** otherwiseguy has joined #openstack-infra | 02:12 | |
| *** sweston_ has quit IRC | 02:15 | |
| *** _nadya_ has joined #openstack-infra | 02:17 | |
| *** SumitNaiksatam has joined #openstack-infra | 02:18 | |
| *** gokrokve has joined #openstack-infra | 02:20 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move glance default test node to trusty. https://review.openstack.org/102681 | 02:23 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move swift default test node to trusty. https://review.openstack.org/102686 | 02:23 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move nova default test node to trusty. https://review.openstack.org/102621 | 02:23 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move heat default test node to trusty. https://review.openstack.org/102690 | 02:23 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move ceilometer default test node to trusty. https://review.openstack.org/102672 | 02:23 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move cinder default test node to trusty. https://review.openstack.org/102678 | 02:23 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move neutron default test node to trusty. https://review.openstack.org/102663 | 02:23 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move keystone default test node to trusty. https://review.openstack.org/102694 | 02:23 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Move ironic default test node to trusty. https://review.openstack.org/102696 | 02:23 |
| clarkb | I changed the order, nova is last | 02:23 |
| *** gokrokve has quit IRC | 02:24 | |
| *** homeless has quit IRC | 02:26 | |
| nibalizer | jeblair: so how is the best way to go on the disucssion of beaker + openstack testing? | 02:26 |
| *** wenlock has joined #openstack-infra | 02:26 | |
| *** r1chardj0n3s is now known as r1chard_afk | 02:26 | |
| clarkb | nibalizer: you need to catch him before he is AFK for the night :) | 02:27 |
| nibalizer | maybe pick a time tomorrow or friday? I would feel bad chewing up meeting time with this but thats also an option | 02:27 |
| nibalizer | ya this is mostly for him to read when he returns | 02:27 |
| nibalizer | besides, openstack never sleeps :P | 02:27 |
| *** adalbas has quit IRC | 02:28 | |
| *** mestery has joined #openstack-infra | 02:30 | |
| *** timrc is now known as timrc-afk | 02:35 | |
| *** HenryG has joined #openstack-infra | 02:36 | |
| *** praneshp has quit IRC | 02:37 | |
| *** devkulkarni has joined #openstack-infra | 02:39 | |
| *** e0ne has joined #openstack-infra | 02:39 | |
| openstackgerrit | Aaron Rosen proposed a change to openstack-infra/config: Add python-congressclient as new project to stackforge https://review.openstack.org/102697 | 02:40 |
| *** timrc-afk is now known as timrc | 02:43 | |
| *** e0ne has quit IRC | 02:44 | |
| *** gokrokve has joined #openstack-infra | 02:44 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Modernize ATCs script to Gerrit 2.8, programs.yaml https://review.openstack.org/102698 | 02:45 |
| *** hogepodge has quit IRC | 02:46 | |
| openstackgerrit | Aaron Rosen proposed a change to openstack-infra/config: WIP: Add jenkins job for python-congressclient https://review.openstack.org/102699 | 02:50 |
| *** CaptTofu_ has quit IRC | 02:52 | |
| *** yamahata has quit IRC | 02:52 | |
| *** CaptTofu_ has joined #openstack-infra | 02:53 | |
| *** yamahata has joined #openstack-infra | 02:53 | |
| *** CaptTofu_ has quit IRC | 02:57 | |
| *** sweston_ has joined #openstack-infra | 03:00 | |
| openstackgerrit | Ryan Petrello proposed a change to openstack-infra/config: Gate pecan on barbican again. https://review.openstack.org/102706 | 03:03 |
| mordred | nibalizer: what's beaker? | 03:03 |
| jesusaurus | mordred: its mostly a wrapper around server-spec and either vagrant or docker | 03:07 |
| jesusaurus | mordred: https://github.com/puppetlabs/beaker/wiki/How-to-Write-a-Beaker-Test-for-a-Module | 03:07 |
| mordred | jesusaurus: neat | 03:07 |
| nibalizer | mordred: its for testing puppet code | 03:07 |
| nibalizer | really testing, its pretty sweet | 03:08 |
| Alex_Gaynor | mordred: (soft ping on the livegrep stuff) | 03:08 |
| mordred | nibalizer: so - it soudns great - just for sake of argument though ... we have a cloud, why would use vagrant or docker? | 03:11 |
| *** r1chard_afk is now known as r1chardj0n3s | 03:11 | |
| clarkb | thats what jeblair said | 03:11 |
| mordred | I mean - I get why it's exciting if you don't have zuul, nodepool or a cloud | 03:12 |
| mordred | and I'm not saying "horrible idea" | 03:12 |
| mordred | just really, I don't know what it gets us over spinning up a single-use slave that has root access? | 03:12 |
| jesusaurus | it automates the testing of various modules | 03:13 |
| jesusaurus | similar to what I'm trying to do with test.sh, but it doesnt noop, it checks to see what changed | 03:13 |
| jesusaurus | but yeah, we could just use server-spec directly to do that since we already have this whole provisioning thing mostly figured out | 03:14 |
| mordred | couldn't we also not no-op? | 03:14 |
| jesusaurus | yeah we could | 03:14 |
| mordred | https://github.com/puppetlabs/beaker/wiki/Creating-A-Test-Environment <-- looks like it duplicates nodepool | 03:14 |
| mordred | it seems like perhaps it has a multi-node thing figured out? | 03:15 |
| jesusaurus | yeah, my first thought was to teach its provisioner how to talk to nodepool | 03:15 |
| jesusaurus | but things sit in different places and that wouldnt really work out right | 03:16 |
| mordred | yah. honestly, this is the problem we have with lifeless' plans for multi-node testr - and also the problems we have with other similar things | 03:16 |
| mordred | is taht they want to be "helpful" by handling the provisiioning | 03:16 |
| jesusaurus | because that would require the beaker test to run before there was a node provisioned | 03:16 |
| mordred | except that we handle that | 03:16 |
| openstackgerrit | Isaku Yamahata proposed a change to openstack-infra/config: Add tacker project on StackForge https://review.openstack.org/97435 | 03:17 |
| *** praneshp has joined #openstack-infra | 03:17 | |
| mordred | like, I like this: | 03:17 |
| mordred | on master, "echo hello" | 03:17 |
| mordred | on database, "echo hello" | 03:17 |
| mordred | that's sexy | 03:17 |
| mordred | jesusaurus: the vSphere support section seems like MAYBE it's a thing that might be able to know something about nodepool? (although it's still the wrong sequencing) | 03:18 |
| mordred | jesusaurus: but it does at least reference a resource pool | 03:18 |
| * mordred rains on parades | 03:19 | |
| jesusaurus | i think nibalizer found a way to tell beaker 'you dont need to provision anything, heres an IP' which means we can just multi-node in nodepool and give beaker the subnodes | 03:19 |
| mordred | oh - that's neat | 03:20 |
| *** ArxCruz has quit IRC | 03:20 | |
| *** devkulkarni has quit IRC | 03:20 | |
| mordred | (also, we have this other problem where some of our systems want to do things like replicate to github) | 03:20 |
| * jesusaurus doesnt see the connection there | 03:21 | |
| mordred | if we actually run the real stuff not no-op, it could cause something to try to make an external connection somewhere? | 03:21 |
| jesusaurus | oh, hmm | 03:21 |
| mordred | BUT | 03:21 |
| mordred | that's just a thign to solve | 03:21 |
| mordred | I'd love to have proper live integration testing of our puppets | 03:22 |
| jesusaurus | yeah, especially if we start breaking modules out into their own projects | 03:23 |
| jesusaurus | we'll need some orchestration in our testing to pull it all back together sanely | 03:23 |
| mordred | infrastack | 03:24 |
| nibalizer | mordred: jesusaurus so if we do the multinode nodepool thing, we cna have the primary nodepool node run the beaker goodness | 03:29 |
| nibalizer | then the beaker node can hit the subnodes and configure them | 03:29 |
| nibalizer | and its cool because it gets it on like 'you are webserver' 'you are database server' 'now kisssss' | 03:29 |
| nibalizer | so its super slick | 03:30 |
| nibalizer | but yea beaker wants to run the show, which duplicates some of the effort elsewhere in our pipeline | 03:30 |
| nibalizer | mordred: as for proper sandboxing, beaker is happy to run commands before and after its meaty part of what it does | 03:31 |
| nibalizer | so we could have the first thing be install_puppet.sh, and the second thing be iptables -NOINTERNETFORYOU | 03:31 |
| nibalizer | now if we need to be able to pull from github, and that module wants to push to github..... yikes | 03:31 |
| nibalizer | but even that is easy to solve by giving it bad credentials maybe | 03:32 |
| mordred | yah - but that would (and should_ cause the thing to fail | 03:33 |
| mordred | let's call these two different problems | 03:33 |
| *** devkulkarni has joined #openstack-infra | 03:33 | |
| nibalizer | that works for me | 03:33 |
| nibalizer | there are a cpl WIP reviews i put up and you can see sorta what i'm going for | 03:34 |
| jesusaurus | so, the tests should definitely have not-production credentials | 03:34 |
| jesusaurus | they simply wont have any of the hiera | 03:34 |
| nibalizer | https://review.openstack.org/#/c/102020/ and https://review.openstack.org/#/c/102071/ | 03:35 |
| nibalizer | its actually one of the stackforge puppet modules to start because the infra puppet modules aren't really modules yet | 03:37 |
| nibalizer | okay /me -> rescue efi partition on laptop | 03:38 |
| *** pcrews has quit IRC | 03:39 | |
| *** e0ne has joined #openstack-infra | 03:39 | |
| *** crc32 has quit IRC | 03:40 | |
| *** yaguang has joined #openstack-infra | 03:40 | |
| *** e0ne has quit IRC | 03:43 | |
| *** andreaf_ has joined #openstack-infra | 03:44 | |
| *** gokrokve_ has joined #openstack-infra | 03:44 | |
| *** gokrokve has quit IRC | 03:48 | |
| *** adrian_otto has joined #openstack-infra | 03:54 | |
| *** matsuhashi has quit IRC | 03:54 | |
| *** zhiyan_ is now known as zhiyan | 03:55 | |
| *** matsuhashi has joined #openstack-infra | 03:55 | |
| *** achanda has quit IRC | 03:59 | |
| *** matsuhashi has quit IRC | 03:59 | |
| *** nati_ueno has quit IRC | 04:03 | |
| *** CaptTofu_ has joined #openstack-infra | 04:07 | |
| *** dstanek is now known as dstanek_zzz | 04:07 | |
| *** yfried has quit IRC | 04:08 | |
| *** nosnos has quit IRC | 04:09 | |
| *** gokrokve_ has quit IRC | 04:11 | |
| *** praneshp has quit IRC | 04:11 | |
| *** gokrokve has joined #openstack-infra | 04:11 | |
| *** CaptTofu_ has quit IRC | 04:11 | |
| *** achanda has joined #openstack-infra | 04:13 | |
| *** gokrokve has quit IRC | 04:16 | |
| *** otter768 has quit IRC | 04:25 | |
| kashyap | Hi, to ask Jenkins to rerun its tests, I just have to type "recheck" in a comment, is it? | 04:25 |
| ianw | kashyap : did you hit a known bug? you put the bug number in | 04:26 |
| clarkb | kashyap: not quite, you either have to assert that rerunning the checks is not due to a bug and do 'recheck no bug' or provide the bug number | 04:26 |
| clarkb | eg 'recheck bug XXXXXXX' | 04:26 |
| kashyap | ianw, Yeah, I submitted a trivial change - https://review.openstack.org/#/c/102490/ | 04:27 |
| *** Alexandra_ has joined #openstack-infra | 04:27 | |
| ianw | kashyap: oh, doh, i had a comment on that and didn't press review | 04:27 |
| *** asettle has quit IRC | 04:27 | |
| ianw | i don't think it's right, that should be coming from requirements.txt, can you post your log | 04:27 |
| kashyap | ianw, The bug is created last night (Matt Riederm), we had a long discussion on #openstack-nova | 04:27 |
| ianw | kashyap: ^ | 04:27 |
| kashyap | 1 sec | 04:28 |
| kashyap | clarkb, Thanks for the note | 04:28 |
| kashyap | ianw, http://logs.openstack.org/90/102490/1/check/check-tempest-dsvm-full/b241a13/console.html | 04:28 |
| *** AaronGr has quit IRC | 04:31 | |
| kashyap | ianw, Side question -- I see that you're running the "Red Hat CI". Nova guests are Level-2 (i.e. nested)guests, aren't they? In that case, do you use expose nested virt extensions on Intel hardware? | 04:31 |
| kashyap | ianw, The bug that we're hitting is this -- https://bugs.launchpad.net/nova/+bug/1320617 | 04:31 |
| uvirtbot | Launchpad bug 1320617 in nova "[Image] failed to reach ACTIVE status within the required time (196 s). Current status: SAVING" [Undecided,New] | 04:31 |
| *** praneshp has joined #openstack-infra | 04:32 | |
| ianw | kashyap: no, qemu binary translation. the rhel kernel we run on doesn't support nesting | 04:32 |
| *** Alexandra_ is now known as asettle | 04:33 | |
| kashyap | ianw, Are you using EL6 as host? | 04:33 |
| clarkb | ianw: kashyap: we tested nesting and found it to be quite unstable | 04:33 |
| clarkb | hypervisors and VMs would panic | 04:33 |
| kashyap | clarkb, Hm, I wish people make time to report. Upstream KVM is lightening fast to respond to such issues | 04:34 |
| clarkb | kashyap: when you are running an old kernel I don't think we expected anyone to care | 04:34 |
| clarkb | kashyap: and by old I mean not ancient but not latest either | 04:35 |
| kashyap | Hmm, that's true | 04:35 |
| kashyap | FWIW, I use it in test environments (but I run very new Kernels), and run all the Nova guests (at-least 20 to 30) as nested guests. | 04:35 |
| *** matsuhashi has joined #openstack-infra | 04:37 | |
| *** nosnos has joined #openstack-infra | 04:37 | |
| *** otherwiseguy has quit IRC | 04:37 | |
| ianw | kashyap: so why isn't pyyaml installed by pip? | 04:39 |
| *** e0ne has joined #openstack-infra | 04:39 | |
| kashyap | ianw, Hmm, not quite sure. Here is the DevStack invocation (devstacklog.txt) -- http://kashyapc.fedorapeople.org/temp/devstacklog-25JUN2014.txt | 04:39 |
| *** achanda has quit IRC | 04:39 | |
| *** AaronGr has joined #openstack-infra | 04:40 | |
| kashyap | (Grep for "fatal error") | 04:41 |
| dstufft | what'd pip do wrong now | 04:41 |
| ianw | kashyap: oh, yaml.h ... what's supposed to provide that? some -dev pkg | 04:42 |
| kashyap | ianw, PyYAML is the package | 04:42 |
| *** e0ne has quit IRC | 04:42 | |
| kashyap | Installing the above resolved it, let me get the 2nd devstack run after I installed the RPM | 04:42 |
| kashyap | http://kashyapc.fedorapeople.org/temp/devstacklog2.txt | 04:43 |
| clarkb | its not libyaml-devel that you need? | 04:43 |
| kashyap | Yes, the header file must be, some devel pkg | 04:43 |
| clarkb | PyYAML is probably just installing the python lib for you | 04:43 |
| *** gokrokve has joined #openstack-infra | 04:43 | |
| clarkb | and avoiding the compile step | 04:43 |
| ianw | kashyap: so it's libyaml-devel i think we want here | 04:44 |
| kashyap | Yeah, sorry, right - libyaml | 04:44 |
| * kashyap checks on his instance | 04:44 | |
| ianw | kashyap: i guess the limited install you're doing doesn't bring that in | 04:44 |
| ianw | kashyap: while my runs, doing a full devstack, probably something else has it as a dep before it gets to that | 04:44 |
| kashyap | ianw, Right. That's the local.conf I used -- http://kashyapc.fedorapeople.org/virt/openstack/local.conf | 04:45 |
| kashyap | (Just Nova, Neutron, Keystone, Glance) | 04:45 |
| kashyap | ianw, Yeah, still, we should allow granular DevStack installs, isn't it? | 04:45 |
| clarkb | yes if that is needed devstack should add it explicitly | 04:46 |
| clarkb | we ran into this with kerberos on trusty too | 04:46 |
| clarkb | it was being isntalled transitively on precise but not on trusty so we had to add it | 04:46 |
| clarkb | being explicit is good | 04:46 |
| kashyap | clarkb, Right. I often care about those above areas, so would like to keep systems from getting bloated | 04:46 |
| kashyap | Also, reduces debugging surface | 04:47 |
| ianw | kashyap: actually, http://paste.openstack.org/show/84941/ <-- we are missing that, but keep going anyway | 04:47 |
| *** praneshp_ has joined #openstack-infra | 04:47 | |
| *** otter768 has joined #openstack-infra | 04:48 | |
| kashyap | Hmm I do see libyaml-0.1.6-1.fc20.x86_64 installed on the DevStack instance | 04:48 |
| ianw | kashyap: yeah, need -devel for .h | 04:48 |
| ianw | kashyap: so next question, why does yours not fall back to a python model | 04:49 |
| ianw | kashyap: oh ,hang on, it does (too many log windows open :) | 04:49 |
| ianw | so it's not actually failing | 04:50 |
| *** praneshp has quit IRC | 04:50 | |
| *** praneshp_ is now known as praneshp | 04:50 | |
| ianw | kashyap: in fact, http://kashyapc.fedorapeople.org/temp/devstacklog-25JUN2014.txt looks fine | 04:50 |
| kashyap | ianw, Hmm, so would you recommend to drop that change? I'm not used to staring at these logs/debugging, still finding my way :-) | 04:51 |
| clarkb | I think it is worth adding as the C lib is apparently preferred | 04:51 |
| *** devkulkarni has quit IRC | 04:52 | |
| kashyap | clarkb, So, I'd revise & re-submit it with libyaml-devel? | 04:52 |
| *** maxbit has quit IRC | 04:52 | |
| ianw | kashyap: yeah, change to include libyaml-devel, can't hurt | 04:52 |
| kashyap | ianw, Alright. Making some breakfast, will do so in a few. | 04:53 |
| kashyap | Thanks both for the help. | 04:53 |
| *** amitpp has joined #openstack-infra | 04:57 | |
| openstackgerrit | Adrian Otto proposed a change to openstack-infra/config: List projects not restricted to pypi.openstack.org https://review.openstack.org/102716 | 04:57 |
| openstackgerrit | Adrian Otto proposed a change to openstack/requirements: List projects not restricted to pypi.openstack.org https://review.openstack.org/102719 | 04:58 |
| clarkb | adrian_otto: ^ isn't that a noop? | 04:58 |
| clarkb | we already fallback | 04:58 |
| clarkb | I don't think this solves anything | 04:58 |
| *** dstanek_zzz is now known as dstanek | 04:58 | |
| openstackgerrit | Adrian Otto proposed a change to openstack-infra/config: List projects not restricted to pypi.openstack.org https://review.openstack.org/102716 | 04:58 |
| *** otter768 has quit IRC | 04:58 | |
| *** achanda has joined #openstack-infra | 05:01 | |
| openstackgerrit | enikanorov proposed a change to openstack-infra/elastic-recheck: Add query for bug #1332500 https://review.openstack.org/102551 | 05:01 |
| uvirtbot | Launchpad bug 1332500 in neutron "Lock wait timeout inserting row into routers table" [High,Confirmed] https://launchpad.net/bugs/1332500 | 05:01 |
| clarkb | adrian_otto: any chance you have a sec to discuss what you are trying to accomplish that isn't accomplished by the current setup? | 05:01 |
| adrian_otto | clarkb: hi | 05:01 |
| clarkb | hello | 05:01 |
| clarkb | so that addition is basically a noop. If you remove yourself from the projects.txt it will just work that way | 05:02 |
| adrian_otto | yes, we want to continue to get the automatic requirements update patches from the Jenkins bot | 05:02 |
| clarkb | but that won't work because your requirements won't be the same | 05:02 |
| clarkb | otherwise you wouldn't need the exception | 05:02 |
| clarkb | I think adding the exception and still trying to enforce the reqs will lead to sadness | 05:02 |
| clarkb | as they are competing with each other | 05:02 |
| adrian_otto | and we want to be pinned to the same requirements as other openstack projects, with the exception to allow us to get the mistralclient requirement without listing it as a global requirement | 05:03 |
| clarkb | right I don't think the proposed changes will do that | 05:03 |
| adrian_otto | why? | 05:03 |
| clarkb | because you will break the enforcement side | 05:03 |
| clarkb | the requirements tools will say no to mistalclient | 05:03 |
| clarkb | so while it may work from the mirror side it won't work with the syncing and checking | 05:04 |
| adrian_otto | I really wanted to only allow the fallback for a specifically listed requirement | 05:04 |
| adrian_otto | I'm suggesting this as another alternative to consider | 05:04 |
| clarkb | right and I am saying it won't solve either problem | 05:04 |
| clarkb | because the syncing and checking will break | 05:04 |
| kashyap | ianw, Commit message is 60 characters, hope that's fine (I think Gate rejects about 72 chars, right?) - https://review.openstack.org/#/c/102490/ | 05:04 |
| clarkb | adrian_otto: I think we may have to reconsider requirements | 05:05 |
| clarkb | we have always told stackforge projects they are at the mercy of it | 05:05 |
| adrian_otto | clarkb: I want the same set of requirements that all openstack distros use | 05:05 |
| clarkb | or they can opt out | 05:05 |
| adrian_otto | plus the ability to add one more | 05:05 |
| clarkb | this is a third use case that we never intended on supporting (not that we can't support it) | 05:05 |
| adrian_otto | I don't want to drift to a whole new set of requirements | 05:05 |
| clarkb | but I think it needs a but more thinking through as the proposed changes likely won't work | 05:06 |
| ianw | kashyap: pretty sure pep8 will reject it if it's "wrong" | 05:06 |
| clarkb | adrian_otto: right and that is directly at odds with having a special req | 05:06 |
| adrian_otto | clarkb: help me understand what you think won't work | 05:06 |
| clarkb | adrian_otto: because now you have drifted and that special req may pull in things that conflict with the openstack requirements | 05:06 |
| clarkb | adrian_otto: ^ is one way it won't work | 05:06 |
| adrian_otto | the review I just posted will do the equivalent of removing stackforge/solum from projects.txt with respect to pip | 05:06 |
| clarkb | right which is why I am saying it is a noop from that perspective | 05:07 |
| adrian_otto | so it's functionally equivalent to your suggested fix | 05:07 |
| kashyap | ianw, Okay. Will check later what the tests say. Thank you. | 05:07 |
| clarkb | if you want requirements that aren't in the mirror you just remove yourself from projects.txt | 05:07 |
| clarkb | the problem with your proposed changes is that they won't actually do what you want | 05:07 |
| adrian_otto | but then I lose auto requirment updates | 05:07 |
| clarkb | because syncing and checking requires things be in sync | 05:08 |
| clarkb | because your out of sync reqs can do whatever they want and completetly hose the "vetted" requirements | 05:08 |
| adrian_otto | and I reduce the change that I'm aligned on the basic requirements | 05:08 |
| clarkb | I think doing what you want to do is a lot more involved | 05:08 |
| clarkb | and will require additional checks to at least know when the exceptions differ with requirements | 05:09 |
| adrian_otto | what other options can I consider | 05:09 |
| adrian_otto | ok, I'm willing to add code that checks that | 05:09 |
| clarkb | well lets back up | 05:09 |
| *** chuckC has quit IRC | 05:09 | |
| clarkb | the problem is this. we have a list of packages, you want to consume that list of packages except in some cases | 05:09 |
| adrian_otto | yes | 05:09 |
| clarkb | the whole reason that list of packages exists is to ensure no one does that | 05:09 |
| adrian_otto | in one case | 05:09 |
| clarkb | so all of the tooling is built to defeat your goal | 05:10 |
| adrian_otto | yes, that's the problem | 05:10 |
| clarkb | yup | 05:10 |
| *** Alexandra_ has joined #openstack-infra | 05:10 | |
| clarkb | given that I think the best way to solve the problem is to either add mistralclient to the requirements or remove solum from projects.txt | 05:10 |
| *** chuckC has joined #openstack-infra | 05:10 | |
| clarkb | which is essentially the stance infra has had on these issues in the past | 05:10 |
| adrian_otto | yes | 05:11 |
| clarkb | now if we want to add a third thing, we probably want to consider a spec to something (TC?) that re envisions the requirements repo as something else | 05:11 |
| *** Alexandra_ has quit IRC | 05:11 | |
| clarkb | because what you want is directly at odds with the current purpose of that project | 05:12 |
| adrian_otto | and we disqualified the adding of mistral as an option because we believe there is a lot of change coming still, and the upstreams would probably not like that, and the need to support potentially buggy versions of mistral as it evolves | 05:12 |
| clarkb | and there is a much larger discussion that should happen before we turn the ship | 05:12 |
| adrian_otto | that's why I started the ML thread | 05:13 |
| clarkb | yup and people responded :) | 05:13 |
| clarkb | my two major concerns with option 3 are it will be hard to keep you actually in sync if you allow exceptions | 05:13 |
| *** asettle has quit IRC | 05:14 | |
| StevenK | I think adding mistralclient to requirements is the best option. | 05:14 |
| clarkb | becase exceptions can install dependencies themselves and those may be at odds with the requirements repo | 05:14 |
| adrian_otto | Yes, and I'm continuing to hunt for an option that suits the various interests in play | 05:14 |
| StevenK | There was nary a whisper when I added os-cloud-config to requirements | 05:14 |
| clarkb | if that happens you have essentially defeated the functionality desired | 05:14 |
| adrian_otto | StevenK: in all honesty I'm fine with that solution | 05:14 |
| clarkb | the other concern is allowing things into openstack/ that arguably shouldn't be allowed | 05:14 |
| adrian_otto | I'm just representing the view of my team who expected friction with that | 05:14 |
| clarkb | if somehow we allowed things through that hadn't been "vetted" | 05:15 |
| adrian_otto | clarkb: exactly | 05:15 |
| adrian_otto | and we have an expectation at this point that we are including software that has stable APIs | 05:15 |
| clarkb | not necessarily | 05:15 |
| adrian_otto | oh? | 05:16 |
| clarkb | I believe the expectation is that libs are cared for, support python 3 or have a path to that, are appropriately licensed, and make sense in the context of existing requirements | 05:16 |
| clarkb | eg is there something that would work that we already depend on? | 05:16 |
| adrian_otto | no | 05:16 |
| adrian_otto | I ruled that out | 05:16 |
| adrian_otto | we did look for that first | 05:16 |
| clarkb | anyways I think those changes jump the gun and aren't going to do what you expect them to due to requirements being smart with its checks and syncs | 05:17 |
| adrian_otto | I suppose we could potentially substitute zuul for mistral, but then we don't get user customizable CI pipelines | 05:17 |
| clarkb | zuul isn't a requirement either | 05:17 |
| adrian_otto | it does not rely on a workflow API | 05:18 |
| adrian_otto | that's the difference | 05:18 |
| adrian_otto | it uses a gearman queue, I think | 05:18 |
| adrian_otto | or perhaps a series of them | 05:18 |
| clarkb | zuul uses gearman | 05:18 |
| nibalizer | there is only zuuuuuuuuuuuul | 05:18 |
| clarkb | with its own pipelines on top | 05:18 |
| clarkb | but thats a side issue | 05:18 |
| *** yfried has joined #openstack-infra | 05:18 | |
| clarkb | regardless of the actual dependency this is something that needs to be addressed and party line so far has been you get it in requirements or you remove yourself from enforcement | 05:19 |
| clarkb | and I think we need a good idea of what a third option looks like before we make changes | 05:19 |
| *** craigbr has quit IRC | 05:19 | |
| clarkb | because this stuff is complex and will break if we rush into it | 05:19 |
| adrian_otto | fair enough | 05:19 |
| adrian_otto | thanks for thinking through this with us | 05:19 |
| adrian_otto | I want a middle ground, where I get the benefits of enforcement, with the ability to bird dog a particular requirement | 05:20 |
| clarkb | ya and the issue to work around is making sure that doesn't affect openstack/ and ensuring that you don't completely hose yourself due to transitive deps | 05:21 |
| adrian_otto | and maybe StevenK is right. | 05:21 |
| clarkb | I think this is doable | 05:21 |
| clarkb | it just requires some thought | 05:21 |
| *** chuckC has quit IRC | 05:21 | |
| clarkb | when the integrated gate ran into the second problem we came up with this as the fix | 05:21 |
| adrian_otto | ok, so think for a sec about my two reviews I just posted up | 05:21 |
| adrian_otto | I understand the concern that mistralclient may call for other requirmeents | 05:22 |
| *** gyee has quit IRC | 05:22 | |
| adrian_otto | is there a way I can tell pip not to recurse for other requirements? | 05:22 |
| clarkb | dstufft: ^ | 05:22 |
| clarkb | adrian_otto: I think the answer is no because pip dep resolution is pretty simple | 05:22 |
| clarkb | so no knobs | 05:22 |
| clarkb | pip blindly goes where the current package takes it | 05:23 |
| adrian_otto | is there another way I can cause the mistralclient to be loaded by the gate jobs before it satisfies other requirements? | 05:23 |
| clarkb | you can do ugly hacks like install from source | 05:24 |
| * clarkb looks at morganfainberg and keystone | 05:24 | |
| *** chuckC has joined #openstack-infra | 05:24 | |
| adrian_otto | what did Morgan do? | 05:24 |
| * morganfainberg slinks away | 05:24 | |
| adrian_otto | I'm willing to entertain all options at this point | 05:24 |
| clarkb | adrian_otto: keystone unittests install python keystoneclient from source | 05:24 |
| clarkb | adrian_otto: it breaks a lot | 05:24 |
| clarkb | or did in the past | 05:24 |
| morganfainberg | clarkb, it's ok, i'll submit the same dirty hack to all the projects *evil laugh* | 05:24 |
| morganfainberg | clarkb, we made it waaaay better | 05:24 |
| jamielennox | morganfainberg: that's what oslo is for right | 05:25 |
| morganfainberg | clarkb, actually, i wanted to ask, would you be opposed to that check being a separate gate/check job? | 05:25 |
| morganfainberg | clarkb, if we need to keep it. | 05:25 |
| morganfainberg | jamielennox, ++ | 05:25 |
| clarkb | quick remove jamielennox's ssh keys from gerrit before he does too much damage | 05:25 |
| openstackgerrit | enikanorov proposed a change to openstack-infra/elastic-recheck: Add query for neutron bug 1332917 https://review.openstack.org/102724 | 05:25 |
| uvirtbot | Launchpad bug 1332917 in neutron "Deadlock when deleting from ipavailabilityranges " [High,Confirmed] https://launchpad.net/bugs/1332917 | 05:25 |
| clarkb | morganfainberg: nope, I think it fits into the idea of functional testing | 05:25 |
| clarkb | morganfainberg: where we have a little but more environment | 05:26 |
| morganfainberg | clarkb, cool. | 05:26 |
| adrian_otto | but my problem is not just about gate testing | 05:26 |
| morganfainberg | clarkb, *nod* | 05:26 |
| jamielennox | now that you can reverse test things we shouldn't need it right? | 05:26 |
| adrian_otto | it's about making my software work when it gets distributed too | 05:26 |
| jamielennox | as in we can make keystoneclient gate on the keystone tests | 05:26 |
| clarkb | adrian_otto: right and that is why my second concern is important | 05:26 |
| jamielennox | it's just for old versions we would keep it around | 05:26 |
| morganfainberg | jamielennox, yeah that was what i was implying. | 05:27 |
| clarkb | adrian_otto: because circumvention of the checking infrastructure essentially invalidates that | 05:27 |
| clarkb | adrian_otto: you can pretend that you are protected but in the general case you are not | 05:27 |
| morganfainberg | jamielennox, but we should have it run a matrix of versions | 05:27 |
| morganfainberg | jamielennox, something to think about. | 05:27 |
| adrian_otto | clarkb: point taken | 05:27 |
| morganfainberg | clarkb, pip install all of pypi | 05:27 |
| jamielennox | morganfainberg: it's come up before, we would just need to do a lot of test refactor so that we could do that job with just the client specific tests | 05:28 |
| morganfainberg | clarkb, is there a pip package that says "INSTALL ALL THE THNIGS"? cause... i totally want to make that a requirement... or maybe one that just depends on X11 being installed and running | 05:28 |
| clarkb | morganfainberg: I don't think so | 05:28 |
| morganfainberg | clarkb, awwww :( | 05:28 |
| clarkb | you could install bandersnatch then use find to pip install . all the things | 05:28 |
| morganfainberg | clarkb ooh, good idea totally needs to be part of keystoneclient's tests | 05:29 |
| adrian_otto | ok, ;et's come back to point here for a sec | 05:29 |
| adrian_otto | that leaves us with the option of adding mistralclient to global-requirements.txt, right? | 05:29 |
| clarkb | adrian_otto: and fwiw I am sure that if you make mordred and dstufft drink lots of caffeine they can probably come up with something that makes all the things happy | 05:30 |
| clarkb | adrian_otto: yes I think that is the most correct way to handle the problem | 05:30 |
| morganfainberg | adrian_otto, sorry i'll go back to my channel over there *walks off to the "sad walking away hulk song"</family guy>* | 05:30 |
| adrian_otto | because the truth is that I really do want to be pinned to the same requirements as all the integrated projects. | 05:30 |
| clarkb | well in fairness the integrated projects don't do a ton of pinning | 05:31 |
| adrian_otto | well, yes, I suppose that's true | 05:31 |
| clarkb | which I know was part of the r1chardj0n3s confusion last night | 05:31 |
| clarkb | they are intentionally uncapped so that we know when they break us as soon as they break us | 05:31 |
| adrian_otto | we only put in version ranges when problems creep in | 05:31 |
| clarkb | correct and we have successfully started pushing people more towards exclusings instead of caps | 05:32 |
| clarkb | *exclusions | 05:32 |
| clarkb | if there is reason to believe the next release will be corrected | 05:32 |
| adrian_otto | yes, I noticed a few of those in theere | 05:32 |
| adrian_otto | ok, I think the best next step is to follow up on the ML thread proposing addition of mistralcleint to global-requirements | 05:34 |
| adrian_otto | and I will abandon the two reviews I just made | 05:34 |
| clarkb | ya proposing that can't hurt | 05:34 |
| adrian_otto | and just reference them as an option that we would rather not use | 05:34 |
| clarkb | and may help bring out more corner cases in the whole thing | 05:34 |
| adrian_otto | clarkb: can you anticipate what any of those might be? | 05:35 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: WIP Add support for negative requirements https://review.openstack.org/102726 | 05:35 |
| clarkb | adrian_otto: not right now | 05:35 |
| clarkb | adrian_otto: I personally wouldn't have a huge issue with it | 05:35 |
| clarkb | as things like tooz were added | 05:35 |
| jhesketh | jogo: ^ that's for you man :-) | 05:35 |
| adrian_otto | so it's the same core team here that decides what get's placed there, or do you seek TC input on each one? | 05:36 |
| clarkb | neither, there is a dedicated review team for requirements | 05:36 |
| jogo | jhesketh: woot | 05:36 |
| adrian_otto | is there an ml topic for that team? | 05:36 |
| jogo | I like the commit message | 05:36 |
| clarkb | https://review.openstack.org/#/admin/groups/131,members | 05:36 |
| jhesketh | :-) | 05:36 |
| clarkb | adrian_otto: [requirements] ? | 05:37 |
| adrian_otto | or is the current topic appropriate? | 05:37 |
| adrian_otto | ok, I'll add that just to be safe | 05:37 |
| clarkb | adrian_otto: I dunno. I find topics are actually not so good and try to skim everything... | 05:37 |
| clarkb | not sure if crazy or actually helpful | 05:37 |
| adrian_otto | I suppose if there was at least one other project that could make sue of mistralclient, that argument would be much easier to make. | 05:38 |
| adrian_otto | (disadvantage of being a first mover) | 05:39 |
| adrian_otto | s/sue/use/ | 05:39 |
| *** e0ne has joined #openstack-infra | 05:39 | |
| *** dims__ has quit IRC | 05:40 | |
| clarkb | jhesketh: do you have a link to your removal of envinject handy? | 05:43 |
| clarkb | jhesketh: I should just review that | 05:43 |
| *** e0ne has quit IRC | 05:44 | |
| adrian_otto | clarkb: do you think I should post a review to the requirements repo to add mistralclient to the global requirements in order to have the discussion, or use the ML first? | 05:45 |
| adrian_otto | or both at the same time? | 05:45 |
| adrian_otto | I can cross link them | 05:45 |
| clarkb | you can probably just push a cahnge and see what requirements core says | 05:45 |
| clarkb | and if we run into more "this is hard" we can punt to ML | 05:45 |
| adrian_otto | ok, I'm going to try that and see what happens | 05:46 |
| *** harlowja is now known as harlowja_away | 05:49 | |
| jhesketh | clarkb: this is the tail https://review.openstack.org/#/c/102090/ | 05:49 |
| jhesketh | clarkb: I think there are still a couple of Maven jobs I don't know what to do for | 05:49 |
| jhesketh | *s/for/with | 05:49 |
| *** zzelle has joined #openstack-infra | 05:50 | |
| *** zzelle has quit IRC | 05:50 | |
| *** _nadya_ has quit IRC | 05:52 | |
| *** unicell has quit IRC | 05:52 | |
| clarkb | jhesketh: that stack mostly looks good to me | 05:53 |
| clarkb | I approved the first one | 05:53 |
| clarkb | did you want to address zaro's comments? | 05:53 |
| clarkb | I do think it helps make it more readable to colocate the generation of the files and sourcing of them | 05:53 |
| jhesketh | clarkb: yep, give me a sec | 05:54 |
| *** CaptTofu_ has joined #openstack-infra | 05:55 | |
| *** timrc is now known as timrc-afk | 05:58 | |
| openstackgerrit | Adrian Otto proposed a change to openstack/requirements: Add python-mistralclient to requirements https://review.openstack.org/102738 | 05:59 |
| *** achanda has quit IRC | 05:59 | |
| *** CaptTofu_ has quit IRC | 05:59 | |
| *** moted has quit IRC | 05:59 | |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/config: Bump the apt module version from 1.1 to 1.4 https://review.openstack.org/102739 | 06:01 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Remove envinject from gerrit https://review.openstack.org/102088 | 06:02 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Remove envinject from javascript https://review.openstack.org/102089 | 06:02 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Remove envinject from groups https://review.openstack.org/102090 | 06:02 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Remove envinject from openstackid https://review.openstack.org/102086 | 06:02 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Remove envinject from gerrit-plugin-jobs https://review.openstack.org/102087 | 06:02 |
| jhesketh | clarkb: ^ | 06:02 |
| dstufft | wait | 06:03 |
| dstufft | adrian_otto: there's --no-deps | 06:04 |
| dstufft | it won't install anything that wasn't explicitly given | 06:04 |
| clarkb | dstufft: how does it avoid setup.py doing stuff? | 06:04 |
| clarkb | dstufft: you monkey patch all the things? | 06:04 |
| *** achanda has joined #openstack-infra | 06:05 | |
| *** timrc-afk is now known as timrc | 06:06 | |
| dstufft | well it doesn't | 06:06 |
| dstufft | setup.py doesn't generally do stuf via pip | 06:07 |
| dstufft | except for setup_requires | 06:07 |
| dstufft | wel always tell setup.py --single-version-externally-managed which turns off isntall_requires and friends | 06:07 |
| clarkb | woo pbr | 06:08 |
| clarkb | (pbr breaks ^ right?) | 06:08 |
| *** Longgeek has joined #openstack-infra | 06:08 | |
| clarkb | actually it may do the single version externally managed | 06:08 |
| * adrian_otto perks up | 06:11 | |
| adrian_otto | it's past 1:00 AM here | 06:11 |
| clarkb | yes time for sleep soon | 06:12 |
| clarkb | tomorrow is an early morning | 06:12 |
| adrian_otto | dstufft: if you would be kind enough to add your ideas to this ML thread, I would deeply appreciate it: http://lists.openstack.org/pipermail/openstack-dev/2014-June/038609.html | 06:12 |
| clarkb | US germany should be fun | 06:12 |
| * adrian_otto turning in now | 06:12 | |
| *** matsuhashi has quit IRC | 06:13 | |
| *** zhiyan is now known as zhiyan_ | 06:13 | |
| *** matsuhashi has joined #openstack-infra | 06:13 | |
| *** zhiyan_ is now known as zhiyan | 06:14 | |
| dstufft | clarkb: yea pbr doesn't break becuase it respects --svem | 06:16 |
| dstufft | adrian_otto: I can tomorrow | 06:16 |
| *** achanda has quit IRC | 06:16 | |
| *** matsuhashi has quit IRC | 06:18 | |
| *** matsuhashi has joined #openstack-infra | 06:18 | |
| *** dstanek is now known as dstanek_zzz | 06:18 | |
| *** _nadya_ has joined #openstack-infra | 06:19 | |
| *** zehicle_at_dell has joined #openstack-infra | 06:21 | |
| *** nati_ueno has joined #openstack-infra | 06:23 | |
| *** matsuhashi has quit IRC | 06:24 | |
| *** matsuhashi has joined #openstack-infra | 06:24 | |
| *** rwsu has quit IRC | 06:26 | |
| *** e0ne has joined #openstack-infra | 06:26 | |
| *** _nadya__ has joined #openstack-infra | 06:31 | |
| *** ildikov has quit IRC | 06:31 | |
| *** matsuhashi has quit IRC | 06:31 | |
| *** matsuhashi has joined #openstack-infra | 06:32 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/devstack-gate: Capture apache configuration files https://review.openstack.org/102745 | 06:32 |
| *** matsuhas_ has joined #openstack-infra | 06:34 | |
| *** _nadya_ has quit IRC | 06:35 | |
| *** dims__ has joined #openstack-infra | 06:36 | |
| *** matsuhashi has quit IRC | 06:37 | |
| *** dims__ has quit IRC | 06:41 | |
| *** sileht has joined #openstack-infra | 06:44 | |
| *** srenatus has quit IRC | 06:45 | |
| *** srenatus has joined #openstack-infra | 06:46 | |
| *** rdopieralski has joined #openstack-infra | 06:47 | |
| rdopieralski | hi, does anybody know why this patch https://review.openstack.org/#/c/95716/8 is not being merged even though it has 2 +2s? | 06:47 |
| *** chandan_kumar has joined #openstack-infra | 06:47 | |
| rdopieralski | and what can I do to make it? | 06:48 |
| *** doude has joined #openstack-infra | 06:49 | |
| nibalizer | rdopieralski: i think its because +2 is not +a | 06:50 |
| nibalizer | basically cores can +2 all day but if no one says approved it wont go | 06:50 |
| *** e0ne has quit IRC | 06:50 | |
| *** ominakov has joined #openstack-infra | 06:50 | |
| nibalizer | generally in infra the cores want to make sure they're 'at work' and ready to respond if things go south before they approve stuff | 06:50 |
| *** e0ne has joined #openstack-infra | 06:51 | |
| *** rcarrillocruz has quit IRC | 06:51 | |
| rdopieralski | nibalizer: I see, so I just wait? | 06:51 |
| rdopieralski | nibalizer: I just want to make sure the ball is not in my court | 06:51 |
| *** ominakov has quit IRC | 06:52 | |
| *** achanda has joined #openstack-infra | 06:53 | |
| *** rcarrillocruz has joined #openstack-infra | 06:54 | |
| *** e0ne_ has joined #openstack-infra | 06:55 | |
| *** e0ne has quit IRC | 06:56 | |
| *** e0ne_ has quit IRC | 06:58 | |
| *** e0ne has joined #openstack-infra | 06:59 | |
| *** achanda has quit IRC | 07:00 | |
| *** e0ne has quit IRC | 07:04 | |
| *** pblaho has joined #openstack-infra | 07:06 | |
| *** jlibosva has joined #openstack-infra | 07:07 | |
| *** openstackgerrit has quit IRC | 07:07 | |
| *** salv-orlando has joined #openstack-infra | 07:10 | |
| *** _nadya__ has quit IRC | 07:10 | |
| *** ildikov has joined #openstack-infra | 07:11 | |
| nibalizer | rdopieralski: ya i think so :) | 07:11 |
| *** _nadya_ has joined #openstack-infra | 07:12 | |
| rdopieralski | nibalizer: thanks | 07:12 |
| *** _nadya_ has quit IRC | 07:12 | |
| *** rgerganov has joined #openstack-infra | 07:14 | |
| *** jcoufal has joined #openstack-infra | 07:16 | |
| *** andreaf_ has quit IRC | 07:18 | |
| *** mrda is now known as manslaughter | 07:20 | |
| *** andreykurilin_ has joined #openstack-infra | 07:21 | |
| *** timrc is now known as timrc-afk | 07:22 | |
| *** manslaughter is now known as mrda | 07:22 | |
| *** tkelsey has joined #openstack-infra | 07:22 | |
| *** oomichi has quit IRC | 07:23 | |
| *** basha has joined #openstack-infra | 07:25 | |
| *** timrc-afk is now known as timrc | 07:29 | |
| jhesketh | zaro: if you have any ideas how to remove envinject from jenkins-plugin-jobs.yaml I'd love some help | 07:29 |
| *** dims__ has joined #openstack-infra | 07:30 | |
| *** r1chardj0n3s has quit IRC | 07:30 | |
| *** rcarrill` has joined #openstack-infra | 07:31 | |
| lifeless | mordred: multi-node testr doesn't handle provisioning | 07:32 |
| lifeless | mordred: it has a contract where you tell it how to get a test process running, but it doesn't give a hoot about how you arrange for that to happen | 07:32 |
| *** rcarrillocruz has quit IRC | 07:33 | |
| *** dims__ has quit IRC | 07:35 | |
| *** skolekonov has joined #openstack-infra | 07:36 | |
| *** BobBall-Away is now known as BobBall | 07:37 | |
| *** praneshp has quit IRC | 07:38 | |
| *** basha has quit IRC | 07:39 | |
| *** CaptTofu_ has joined #openstack-infra | 07:43 | |
| *** hashar has joined #openstack-infra | 07:46 | |
| *** CaptTofu_ has quit IRC | 07:48 | |
| *** jgallard has joined #openstack-infra | 07:52 | |
| *** Hal_ has joined #openstack-infra | 07:53 | |
| *** lcheng has joined #openstack-infra | 07:54 | |
| *** ihrachyshka has joined #openstack-infra | 07:55 | |
| *** dizquierdo has joined #openstack-infra | 07:58 | |
| hashar | good morning | 08:00 |
| *** zz_johnthetubagu is now known as johnthetubaguy | 08:00 | |
| *** ominakov has joined #openstack-infra | 08:01 | |
| hashar | I like how a simple 'recheck no bug' triggers the gating of four changes | 08:01 |
| hashar | spam incoming | 08:01 |
| therve | You shouldn't use recheck no bug though | 08:03 |
| hashar | oh | 08:04 |
| SergeyLukjanov | clarkb, I've added switch to trusty CR for sahara too | 08:04 |
| *** nati_ueno has quit IRC | 08:04 | |
| hashar | therve: what is the recommendation? | 08:04 |
| therve | hashar, There is always a bug. So the recommendation is to find the bug number, or create a new one. | 08:04 |
| hashar | therve: ah true sorry. The context was change https://review.openstack.org/#/c/102030/ which had pep8 fail because of a pypi error: No distributions at all found for apscheduler>=2.1.1,<3.0 | 08:05 |
| hashar | console is http://logs.openstack.org/30/102030/3/gate/gate-zuul-pep8/f9209fb/console.html | 08:05 |
| *** mrda is now known as mrda-away | 08:08 | |
| therve | hashar, You should probably have created a bug for it. | 08:08 |
| hashar | therve: and looking in logstash that error seems to happen from time to time. | 08:09 |
| hashar | will fill a bug | 08:09 |
| *** e0ne has joined #openstack-infra | 08:12 | |
| *** Hal_ has quit IRC | 08:13 | |
| *** jpich has joined #openstack-infra | 08:14 | |
| hashar | https://bugs.launchpad.net/openstack-ci/+bug/1334550 ! | 08:14 |
| uvirtbot | Launchpad bug 1334550 in openstack-ci "Could not find any downloads that satisfy the requirement X" [Undecided,New] | 08:14 |
| hashar | therve: thank you | 08:14 |
| *** Hal_ has joined #openstack-infra | 08:14 | |
| *** roeyc has joined #openstack-infra | 08:17 | |
| *** andreykurilin_ has quit IRC | 08:27 | |
| *** Hal_ has quit IRC | 08:30 | |
| *** dims__ has joined #openstack-infra | 08:31 | |
| *** jedimike has joined #openstack-infra | 08:32 | |
| *** _nadya_ has joined #openstack-infra | 08:33 | |
| *** Longgeek has quit IRC | 08:34 | |
| *** ociuhandu has quit IRC | 08:34 | |
| *** srenatus has quit IRC | 08:34 | |
| *** Longgeek has joined #openstack-infra | 08:34 | |
| *** srenatus has joined #openstack-infra | 08:35 | |
| *** amitpp has quit IRC | 08:35 | |
| *** dims__ has quit IRC | 08:36 | |
| *** andreaf_ has joined #openstack-infra | 08:37 | |
| *** jamielennox is now known as jamielennox|away | 08:39 | |
| *** adrian_otto has quit IRC | 08:43 | |
| *** Hal_ has joined #openstack-infra | 08:44 | |
| *** rcarrillocruz has joined #openstack-infra | 08:46 | |
| *** rcarrill` has quit IRC | 08:46 | |
| *** _nadya_ has quit IRC | 08:48 | |
| *** Hal_ has quit IRC | 08:51 | |
| *** matsuhas_ has quit IRC | 08:56 | |
| *** matsuhashi has joined #openstack-infra | 08:57 | |
| *** jp_at_hp has joined #openstack-infra | 08:59 | |
| *** pelix has joined #openstack-infra | 08:59 | |
| *** matsuhashi has quit IRC | 09:01 | |
| *** dtantsur|afk is now known as dtantsur | 09:01 | |
| *** matsuhashi has joined #openstack-infra | 09:01 | |
| *** moted has joined #openstack-infra | 09:03 | |
| *** unicell has joined #openstack-infra | 09:05 | |
| *** Hal_ has joined #openstack-infra | 09:06 | |
| *** matsuhashi has quit IRC | 09:07 | |
| *** matsuhashi has joined #openstack-infra | 09:08 | |
| *** matsuhashi has quit IRC | 09:08 | |
| *** matsuhashi has joined #openstack-infra | 09:09 | |
| *** IvanBerezovskiy has joined #openstack-infra | 09:10 | |
| *** rcarrill` has joined #openstack-infra | 09:11 | |
| *** rcarrillocruz has quit IRC | 09:13 | |
| yfried | is this the place for help with jenkins-job-builder? | 09:15 |
| *** mkerrin has quit IRC | 09:18 | |
| *** jedimike has quit IRC | 09:19 | |
| *** Hal_ has quit IRC | 09:21 | |
| *** Hal_ has joined #openstack-infra | 09:21 | |
| *** pblaho is now known as pblaho|afk | 09:22 | |
| *** _nadya_ has joined #openstack-infra | 09:22 | |
| *** mkerrin has joined #openstack-infra | 09:25 | |
| *** dmakogon_ is now known as denis_makogon | 09:25 | |
| *** pblaho|afk is now known as pblaho | 09:27 | |
| *** roeyc has quit IRC | 09:28 | |
| *** CaptTofu_ has joined #openstack-infra | 09:31 | |
| *** GheRivero has quit IRC | 09:31 | |
| *** dims__ has joined #openstack-infra | 09:32 | |
| Longgeek | yfried: yes | 09:32 |
| *** GheRivero has joined #openstack-infra | 09:33 | |
| *** CaptTofu_ has quit IRC | 09:36 | |
| *** dims__ has quit IRC | 09:37 | |
| *** mrmartin has joined #openstack-infra | 09:37 | |
| *** markmc has joined #openstack-infra | 09:41 | |
| *** amitpp has joined #openstack-infra | 09:43 | |
| *** _nadya_ has quit IRC | 09:43 | |
| *** Hal_ has quit IRC | 09:48 | |
| *** yaguang has quit IRC | 09:51 | |
| *** nosnos has quit IRC | 09:52 | |
| *** mrmartin has quit IRC | 09:52 | |
| hashar | yfried: I might assist :-) | 09:55 |
| *** e0ne has quit IRC | 09:57 | |
| *** e0ne has joined #openstack-infra | 09:58 | |
| *** afazekas is now known as __afazekas | 09:58 | |
| *** yfried is now known as yfried|lunch | 09:59 | |
| *** _nadya_ has joined #openstack-infra | 09:59 | |
| *** matsuhashi has quit IRC | 10:09 | |
| *** matsuhashi has joined #openstack-infra | 10:09 | |
| *** matsuhashi has quit IRC | 10:10 | |
| *** matsuhashi has joined #openstack-infra | 10:11 | |
| *** isviridov|away is now known as isviridov | 10:15 | |
| *** wenlock has quit IRC | 10:15 | |
| *** jyu has quit IRC | 10:16 | |
| *** matsuhashi has quit IRC | 10:16 | |
| *** matsuhashi has joined #openstack-infra | 10:17 | |
| *** zehicle_at_dell has quit IRC | 10:18 | |
| *** zehicle_at_dell has joined #openstack-infra | 10:18 | |
| *** jgallard has quit IRC | 10:19 | |
| *** pblaho is now known as pblaho|afk | 10:20 | |
| *** matsuhas_ has joined #openstack-infra | 10:20 | |
| *** matsuhashi has quit IRC | 10:21 | |
| *** matsuhas_ has quit IRC | 10:26 | |
| *** matsuhashi has joined #openstack-infra | 10:26 | |
| srenatus | is there any way to change the email address of my gerrit account? | 10:27 |
| *** matsuhashi has quit IRC | 10:35 | |
| *** matsuhashi has joined #openstack-infra | 10:35 | |
| *** matsuhashi has quit IRC | 10:35 | |
| *** matsuhashi has joined #openstack-infra | 10:36 | |
| *** rcarrillocruz has joined #openstack-infra | 10:36 | |
| *** rcarrill` has quit IRC | 10:38 | |
| *** dims__ has joined #openstack-infra | 10:40 | |
| *** trono has joined #openstack-infra | 10:43 | |
| srenatus | so, I changed my email in "Contact Information", but it still shows the old one in "Profile" | 10:44 |
| *** dizquierdo has quit IRC | 10:49 | |
| *** _nadya_ has quit IRC | 10:50 | |
| *** chandan_kumar is now known as chkumar246 | 10:51 | |
| *** chkumar246 is now known as chandan_kumar | 10:53 | |
| *** chandan_kumar has quit IRC | 10:54 | |
| *** chandan_kumar has joined #openstack-infra | 10:54 | |
| Longgeek | srenatus: On the launchpad to modify it? | 10:55 |
| srenatus | Longgeek: did that | 10:55 |
| *** ociuhandu has joined #openstack-infra | 10:56 | |
| *** CaptTofu_ has joined #openstack-infra | 10:56 | |
| Longgeek | srenatus: Launchpad and Gerrit email stays the same | 10:56 |
| srenatus | Longgeek: hmm... let me refresh this | 10:57 |
| srenatus | Oh... | 10:57 |
| srenatus | thanks a lot. done. | 10:57 |
| *** salv-orlando has quit IRC | 10:59 | |
| chmouel | dstufft: hey there, i was wondering if that's behavior is expected http://pastie.org/pastes/9326097/text?key=knorjbzsjiral8ejmogetq | 11:03 |
| chmouel | dstufft: get_pip install as pip2 and devtack fail | 11:04 |
| chmouel | dstufft: that's on fedora20 | 11:04 |
| *** ildikov_ has joined #openstack-infra | 11:04 | |
| *** ildikov has quit IRC | 11:04 | |
| *** matsuhashi has quit IRC | 11:05 | |
| *** matsuhashi has joined #openstack-infra | 11:05 | |
| *** rcarrill` has joined #openstack-infra | 11:08 | |
| *** rcarrillocruz has quit IRC | 11:09 | |
| fungi | chmouel: it looks like your fedora may have pip installed from an rpm, with a different executable name than get-pip would create (but get-pip skips installing there because it looks for and find the python module rather than the executable) | 11:09 |
| *** trono has left #openstack-infra | 11:10 | |
| *** matsuhashi has quit IRC | 11:10 | |
| fungi | chmouel: in our systems we remove any distro-provided pip packages before running get-pip | 11:10 |
| fungi | perhaps devstack itself should do the same | 11:10 |
| * fungi looks to see if devstack tries to and is checking by executable name or something similarly unhelpful | 11:11 | |
| fungi | chmouel: maybe https://git.openstack.org/cgit/openstack-dev/devstack/tree/functions-common#n1245 should be expanded to also look for pip2 and pip3? | 11:12 |
| *** amitpp has quit IRC | 11:13 | |
| fungi | chmouel: probably same here too https://git.openstack.org/cgit/openstack-dev/devstack/tree/tools/install_pip.sh#n34 | 11:14 |
| *** ociuhandu has quit IRC | 11:19 | |
| *** flaper87|afk is now known as flaper87 | 11:19 | |
| Longgeek | fungi: Why does my patch did not trigger Jenkins? https://review.openstack.org/#/c/102796/ | 11:20 |
| *** yamahata has quit IRC | 11:21 | |
| *** rcarrillocruz has joined #openstack-infra | 11:23 | |
| *** amalagon has quit IRC | 11:23 | |
| *** rcarrill` has quit IRC | 11:25 | |
| fungi | Longgeek: according to zuul's log it tried to inspect the change yours depends on (23447,2 from the tip of the master branch) and failed to find it, causing the call to throw an exception and not enqueue your change. i'm digging into gerrit now to see if i can discover why | 11:26 |
| *** gcha has quit IRC | 11:27 | |
| *** jedimike has joined #openstack-infra | 11:27 | |
| Longgeek | fungi: What should I do? There are two patch is such a situation, https://review.openstack.org/#/c/102796/ https://review.openstack.org/#/c/102794/ | 11:28 |
| fungi | Longgeek: just wait, i think it's either a bug in zuul or something wrong in the gerrit backend | 11:29 |
| fungi | Longgeek: http://paste.openstack.org/show/84970/ | 11:29 |
| fungi | clearly the parent change it's complaining about not finding does exist, and was merged last year | 11:29 |
| Longgeek | fungi: Understand, thank you for your hard work. | 11:29 |
| fungi | after i get some coffee i'll see if i can find why this is happening | 11:30 |
| fungi | Longgeek: thanks for bringing it to my attention! | 11:30 |
| Longgeek | fungi: :) | 11:30 |
| *** hashar has quit IRC | 11:31 | |
| *** pblaho|afk is now known as pblaho | 11:32 | |
| Longgeek | fungi: Can I continue to other projects submitted to the Puppet code style patch? I do not know if my approach is correct? | 11:33 |
| *** mugsie has joined #openstack-infra | 11:33 | |
| *** _nadya_ has joined #openstack-infra | 11:35 | |
| sdague | fungi: I think we need to light anyone on fire that's setting pip == pipN | 11:36 |
| sdague | I mean, that's freaking crazy pants | 11:36 |
| *** pblaho is now known as pblaho|afk | 11:42 | |
| *** lcheng has quit IRC | 11:44 | |
| dhellmann | I have another trusty-related (I think) issue: http://logs.openstack.org/84/102584/3/check/check-requirements-integration-dsvm/8299e51/console.html | 11:45 |
| dhellmann | I'm unable to land patches in several (all?) oslo repositories now, so I would really appreciate help tracking these down and fixing them. | 11:45 |
| *** pblaho|afk is now known as pblaho | 11:46 | |
| *** amitpp has joined #openstack-infra | 11:46 | |
| chmouel | fungi: sorry just reading this again, i think that pipN can get very annoying | 11:50 |
| chmouel | fungi: i think get_pip.py may be fixed to create a default symlink or a default pip | 11:52 |
| dhellmann | today's error: "x86_64-linux-gnu-gcc: internal compiler error: Terminated (program cc1)" while installing numpy | 11:55 |
| *** rfolco has joined #openstack-infra | 11:56 | |
| *** ociuhandu has joined #openstack-infra | 11:56 | |
| *** lcheng has joined #openstack-infra | 11:57 | |
| *** afazekas_ has joined #openstack-infra | 11:59 | |
| sdague | dhellmann: so realistically, we should install numpy from packages. We really need that test job outside of pbr, because it's insane to be trying to fix it over there. | 12:00 |
| dhellmann | sdague: That's fine. I'd like to start with a plan that gets me back into a state where I can do the work I'm trying to do. Can we stop using trusty for this job until we've figured out how to make it work? | 12:00 |
| sdague | dhellmann: I don't know, ask clarkb on that | 12:01 |
| dhellmann | sdague: fwiw, numpy installed fine on my dev box | 12:01 |
| sdague | actually, numpy is installed, why pip is installing it again is beyond me | 12:02 |
| dhellmann | clarkb, jeblair : can we stop using trusty for the check-requirements-integration-dsvm jobs until the job works there? Oslo is blocked on these failures. | 12:02 |
| *** hashar has joined #openstack-infra | 12:03 | |
| *** adalbas has joined #openstack-infra | 12:04 | |
| dhellmann | at least the pbr release is, and I felt sure I'd had reports of failures in other repos, too, but maybe not | 12:04 |
| fungi | dhellmann: i'll see if i can spot the cause real quick/find a solution... switching those jobs back will make it harder to be certain we've fixed the issue and just delay the transition tha much more | 12:06 |
| dhellmann | fungi: ok, thanks | 12:07 |
| dhellmann | ah, the requirements repo gates on this, and so we won't be able to add any oslo libs to requirements, so that's why I was thinking oslo was blocked | 12:08 |
| sdague | honestly, I think we actually need mordred or someone that understands that reqs job test, because I'm not sure what it's doing, or why it's compiling numpy 3 times | 12:09 |
| sdague | and only on the 3rd time it seems to fail | 12:09 |
| sdague | jhesketh: https://review.openstack.org/#/c/102643/ turbo hipster seems to be a little sad | 12:10 |
| *** masayukig has left #openstack-infra | 12:10 | |
| mordred | sdague: waking up ... will look soon | 12:12 |
| dhellmann | fungi, mordred , sdague : I'm going to eat and then write my ptl webinar thingy. I'll check back in later this morning. Thanks for looking into the problem in the mean time. | 12:13 |
| fungi | sdague: that job tries to build a pypi mirror from the reqs list and makes sure everything can be co-installed | 12:15 |
| fungi | sdague: it's deep inside the run-mirror script where we're hitting it (execution starts at http://logs.openstack.org/84/102584/3/check/check-requirements-integration-dsvm/8299e51/console.html#_2014-06-25_21_59_35_443 ) | 12:17 |
| *** e0ne has quit IRC | 12:18 | |
| fungi | apparently you can git that opaque internal compiler error in gcc if you try to link a nonexistent lib | 12:18 |
| fungi | s/git/hit/ | 12:18 |
| fungi | sillyfingers | 12:18 |
| sdague | yeh, that's what I'm expecting is going on | 12:18 |
| sdague | especially when we're doing something like compiling numpy | 12:18 |
| fungi | i'm just trying to nail down what new package we're missing | 12:19 |
| *** ArxCruz has joined #openstack-infra | 12:20 | |
| *** e0ne has joined #openstack-infra | 12:23 | |
| fungi | probably something in the list here... http://packages.ubuntu.com/source/trusty/python-numpy | 12:23 |
| *** CaptTofu_ has quit IRC | 12:24 | |
| *** weshay has joined #openstack-infra | 12:24 | |
| sdague | so, how about just this - https://review.openstack.org/102809 | 12:24 |
| fungi | i guess we never added a dpkg -l output for the job logs. holding a node now and doing it the ugly way | 12:24 |
| sdague | then we don't need to go hunting | 12:24 |
| *** CaptTofu_ has joined #openstack-infra | 12:24 | |
| *** salv-orlando has joined #openstack-infra | 12:25 | |
| fungi | sdague: bwahahaha | 12:25 |
| fungi | oh, you just added that. openstackgerrit is mia | 12:25 |
| sdague | yep | 12:25 |
| *** e0ne_ has joined #openstack-infra | 12:26 | |
| *** openstackgerrit has joined #openstack-infra | 12:27 | |
| fungi | sdague: but yes, i expect that to solve the problem quite well | 12:27 |
| fungi | welcome back, openstackgerrit | 12:27 |
| fungi | watching the logs... looks like it's normally failing about 10 minutes in with this error | 12:28 |
| fungi | so we should have some confidence shortly | 12:28 |
| *** e0ne has quit IRC | 12:29 | |
| *** CaptTofu_ has quit IRC | 12:29 | |
| *** dprince has joined #openstack-infra | 12:29 | |
| fungi | sdague: i *suspect* that most of the line above those can also go away | 12:29 |
| fungi | sdague: we can try that as a follow-on cleanup patch once the dust settles | 12:30 |
| sdague | yeh, maybe, honestly I want minimal changes until that moves into requirements repo | 12:30 |
| sdague | the requirements integration job should not be in another repo | 12:30 |
| fungi | easy enough to forklift into the other repo and then repoint the job | 12:31 |
| fungi | though no simple way to preserve change history on it | 12:31 |
| fungi | other than leave a breadcrumb in the commit message/script comment | 12:31 |
| krtaylor | Longgeek, bugs https://bugs.launchpad.net/openstack-ci/+bug/1334609 and https://bugs.launchpad.net/openstack-ci/+bug/1334610 are the same | 12:31 |
| uvirtbot | Launchpad bug 1334609 in openstack-ci "top-scope variable being used without an explicit namespace" [Undecided,New] | 12:31 |
| *** dkranz has joined #openstack-infra | 12:32 | |
| sdague | fungi: also it would let us have versions per branch | 12:32 |
| sdague | because right now the job is branchless | 12:32 |
| sdague | but it's not tested against stable | 12:32 |
| krtaylor | Longgeek, did you hit report twice by chance? | 12:32 |
| sdague | so it's a break waiting to happy | 12:32 |
| sdague | happen | 12:32 |
| fungi | sdague: well, yes, but it will also lose us the current backward-compat testing in pbr | 12:32 |
| sdague | fungi: this isn't really a pbr test is it? | 12:33 |
| fungi | sdague: well, it started out being a pbr test | 12:33 |
| fungi | and to some extent still serves to confirm pbr changes don't break all our stuff | 12:33 |
| *** radez_g0n3 is now known as radez | 12:33 | |
| fungi | by trying to install-all-the-things | 12:33 |
| *** r1chardj0n3s has joined #openstack-infra | 12:34 | |
| sdague | ok, so there is a piece of that matrix that needs to be sorted | 12:34 |
| Longgeek | krtaylor: I'm sorry, probably because of network problems, played two | 12:34 |
| mordred | all of the locations we've thought of are also bad I think | 12:34 |
| sdague | the current state is weird though | 12:34 |
| mordred | but yah | 12:34 |
| mordred | totally agree | 12:34 |
| sdague | mordred: why is requirements repo bad? | 12:34 |
| mordred | can't remember ... brain stupid right now | 12:35 |
| sdague | :) | 12:35 |
| mordred | probably isn't | 12:35 |
| sdague | that's fair | 12:35 |
| fungi | i think maybe we initially discounted the reqs repo because it contained no code. then it grew some related scripts/tools | 12:35 |
| sdague | ok, before it gets insane hot here, I'm going to get out on the bike for a bit. Back later. | 12:35 |
| *** pdmars has joined #openstack-infra | 12:36 | |
| *** r1chardj0n3s has quit IRC | 12:36 | |
| sdague | mostly though I'm chasing the libvirt snapshot thing today though, which is our biggest issue since the trusty update | 12:36 |
| fungi | crap. now it's dying on the a2ensite issue | 12:36 |
| fungi | did we not land that patch yet? | 12:36 |
| * fungi checks | 12:36 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/config: Add proposed/* branches support to merge_tags.sh https://review.openstack.org/102812 | 12:37 |
| *** pdmars has quit IRC | 12:37 | |
| fungi | sdague: dhellmann: i suspect it 102809 needs to be squashed with 102584 | 12:37 |
| *** pdmars has joined #openstack-infra | 12:38 | |
| fungi | neither is going to work unless both are fixed | 12:38 |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/config: Add proposed/* branches support to merge_tags.sh https://review.openstack.org/102812 | 12:38 |
| *** ildikov_ has quit IRC | 12:39 | |
| *** dstanek_zzz is now known as dstanek | 12:39 | |
| fungi | or we cram one through past the gate | 12:39 |
| *** adalbas has quit IRC | 12:40 | |
| sdague | fungi: oh, seriously, that patch wasn't landed? | 12:40 |
| sdague | ok, let me do that fast. | 12:41 |
| dhellmann | https://review.openstack.org/#/c/102584/ was blocked on not having some library, and I added that to devstack yesterday -- took all day | 12:41 |
| dhellmann | libkrb5-dev | 12:41 |
| dhellmann | sdague: ^^ | 12:41 |
| openstackgerrit | Longgeek proposed a change to openstack-infra/puppet-apparmor: Update puppet coding style https://review.openstack.org/102794 | 12:44 |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/pbr: Update integration script for Apache 2.4 https://review.openstack.org/102584 | 12:44 |
| *** annegent_ has joined #openstack-infra | 12:44 | |
| sdague | ok, squashed together | 12:44 |
| *** annegent_ is now known as annegentle_ | 12:44 | |
| sdague | fungi: so we're definitely back into one of those failure rates where landing things in the normal way is hard | 12:45 |
| dhellmann | sdague: shouldn't devstack install the build-dep stuff? | 12:45 |
| sdague | dhellmann: I don't think so | 12:46 |
| sdague | it install numpy from packages | 12:46 |
| sdague | so no need to compile it | 12:46 |
| dhellmann | ok, that makes sense | 12:46 |
| dhellmann | and yaml, too? | 12:46 |
| fungi | dhellmann: not necessarily for the reqs integration job since it's not actually setting up devstack (it installs more stuff than devstack does) | 12:46 |
| sdague | yeh, I think so | 12:46 |
| sdague | fungi: a ninja merge on - https://review.openstack.org/#/c/102643/ would be appreciated | 12:47 |
| dhellmann | fungi: ok, I remembered that this was run after devstack, and I was told to add libkrb5-dev to devstack yesterday, so this was just a confusing difference in approach | 12:47 |
| sdague | it's managed to not fail on the bug it was trying to fix | 12:47 |
| sdague | but now it's bouncing off rax mirror failure | 12:47 |
| fungi | dhellmann: libkrb5-dev might actually be needed for devstack's sanity, i don't know | 12:47 |
| sdague | and basically everyone else is going to be blocked merging code until it's in | 12:47 |
| dhellmann | fungi: yeah, me neither, hence the question :-) | 12:47 |
| sdague | dhellmann: yeh, libkrb5-dev seems weird for devstack | 12:48 |
| *** yolanda has quit IRC | 12:48 | |
| sdague | anyway, I'm out for a bit, be back once I get some riding in | 12:48 |
| dhellmann | we can move it to this integration script, I guess | 12:48 |
| dhellmann | sdague: enjoy! | 12:48 |
| *** ildikov has joined #openstack-infra | 12:50 | |
| *** gokrokve_ has joined #openstack-infra | 12:50 | |
| *** gokrokve_ has quit IRC | 12:50 | |
| *** gokrokve_ has joined #openstack-infra | 12:51 | |
| *** gokrokv__ has joined #openstack-infra | 12:52 | |
| openstackgerrit | Longgeek proposed a change to openstack-infra/puppet-dashboard: Add variables and document https://review.openstack.org/102796 | 12:52 |
| ianw | sdague: when you get a chance, i notice the grenade logs are still a bit funny, https://review.openstack.org/#/c/99852/ is hopefully ok | 12:53 |
| *** gokrokve has quit IRC | 12:53 | |
| ianw | sdague: sorry https://review.openstack.org/#/c/100131/ is the grenade change, the other is a cleanup | 12:54 |
| *** gokrokve_ has quit IRC | 12:55 | |
| *** _nadya_ has quit IRC | 12:56 | |
| *** jcoufal has quit IRC | 12:58 | |
| *** adrian_otto has joined #openstack-infra | 13:02 | |
| *** dstanek is now known as dstanek_zzz | 13:02 | |
| *** dkranz has quit IRC | 13:03 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/config: Change ACLs to apply on proposed/* branches https://review.openstack.org/102822 | 13:03 |
| *** devkulkarni has joined #openstack-infra | 13:03 | |
| *** rlandy has joined #openstack-infra | 13:04 | |
| *** dstanek_zzz is now known as dstanek | 13:04 | |
| *** masayukig has joined #openstack-infra | 13:05 | |
| ttx | fungi: see ^ for ACL changes. looks like the top-level ACL file (project.config) is not driven from the config repo ? | 13:05 |
| *** annegentle_ has quit IRC | 13:06 | |
| fungi | ttx: the config for the all-projects metaproject is only documented in the config repo, but not puppeted due to chicken-and-egg bootstrapping complication | 13:07 |
| ttx | fungi: ok, so then my commit message shall be accurate | 13:08 |
| *** mriedem has joined #openstack-infra | 13:08 | |
| *** masayukig has left #openstack-infra | 13:08 | |
| fungi | ttx: though to avoid rebase hell and improve our ability to catch straggler acls being added in outstanding changes, we could instead implement it like i did with https://review.openstack.org/102271 | 13:09 |
| *** flaper87 is now known as flaper87|afk | 13:10 | |
| *** masayukig has joined #openstack-infra | 13:11 | |
| *** miqui has joined #openstack-infra | 13:11 | |
| *** pblaho is now known as pblaho|afk | 13:12 | |
| *** masayukig has left #openstack-infra | 13:12 | |
| *** masayukig has joined #openstack-infra | 13:12 | |
| *** rcarrill` has joined #openstack-infra | 13:12 | |
| *** masayukig has left #openstack-infra | 13:12 | |
| *** rcarrillocruz has quit IRC | 13:13 | |
| *** dstanek is now known as dstanek_zzz | 13:14 | |
| *** rcarrillocruz has joined #openstack-infra | 13:14 | |
| *** julim has joined #openstack-infra | 13:16 | |
| *** dkranz has joined #openstack-infra | 13:16 | |
| *** rcarrill` has quit IRC | 13:17 | |
| mordred | dhellmann, fungi: I'm here now, but it seems you've got it ... | 13:18 |
| mordred | although, in looking, I just discovered patch b07a50b | 13:18 |
| mordred | which is brilliant | 13:18 |
| mordred | and I think may solve the easy_install bug | 13:18 |
| mordred | where is kevin mccarthy so that I can hand him a bunny rabbit? | 13:19 |
| *** otherwiseguy has joined #openstack-infra | 13:20 | |
| *** ildikov has quit IRC | 13:22 | |
| *** pblaho|afk is now known as pblaho | 13:22 | |
| mordred | fungi: also, I _think_ we might be able to get rid of the apache in the script, because I think the bug with pip not being able to do file:// mirrors properly got fixed | 13:23 |
| fungi | mordred: i seem to recall dstufft saying so, yes | 13:25 |
| *** mrmartin has joined #openstack-infra | 13:25 | |
| dstufft | pip should be able to --find-links to a directory | 13:25 |
| dstufft | or --extra-index-url | 13:25 |
| dstufft | whatever | 13:25 |
| *** jcoufal has joined #openstack-infra | 13:25 | |
| fungi | dstufft: even when some packages have dependencies on other packages with capitalization/punctuation differences? | 13:26 |
| fungi | dstufft: i think that was the main blocker which had originally driven us to stick apache on localhost | 13:26 |
| dstufft | I dunno | 13:27 |
| dstufft | I think so | 13:27 |
| mordred | k. well, easy enough to test :) | 13:27 |
| dstufft | if apache works a file should | 13:27 |
| fungi | e.g. pip's indexing via http was different than via local file tree | 13:27 |
| *** pcrews has joined #openstack-infra | 13:28 | |
| fungi | and while we can correct the caps/punc we use to refer to direct depends, we have no control over the same in transitive depends | 13:28 |
| mordred | yah. but I think that got fixed | 13:28 |
| mordred | (I think StevenK or tchaypo got annoyed by it a few months ago ...) | 13:28 |
| fungi | also, not sure if that necessarily helps wit transitive depends using setup_requires in a similar fashion? | 13:29 |
| fungi | or does it? | 13:29 |
| mordred | nothing helps with those :) | 13:29 |
| *** oomichi has joined #openstack-infra | 13:29 | |
| *** oomichi has quit IRC | 13:29 | |
| * mordred hands dstufft a sacraficial goat and waits for pip to take over the world | 13:29 | |
| fungi | other than nuking setuptools from orbit | 13:29 |
| *** oomichi has joined #openstack-infra | 13:29 | |
| fungi | goatnukes | 13:30 |
| dstufft | I should probably work on my branch that steals control of setup_requires from setuptools | 13:30 |
| fungi | https://jenkins02.openstack.org/job/check-requirements-integration-dsvm/271/console is looking very, very promising | 13:30 |
| *** shayneburgess has joined #openstack-infra | 13:30 | |
| dstufft | so much to do :| | 13:30 |
| dstufft | wtb a minion or two to do my bidding | 13:30 |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Fix settings.php permission ownership https://review.openstack.org/102826 | 13:31 |
| *** nikhil___|afk is now known as nikhil___ | 13:31 | |
| *** mfer has joined #openstack-infra | 13:33 | |
| *** amitpp has quit IRC | 13:33 | |
| *** dstanek_zzz is now known as dstanek | 13:34 | |
| *** e0ne_ has quit IRC | 13:37 | |
| *** homeless has joined #openstack-infra | 13:38 | |
| mordred | fungi: also, I think we can move tools/integration.sh to the requirements repo and preserve the history | 13:40 |
| fungi | mordred: oh? | 13:41 |
| fungi | mordred: filter-branch into a separate branch and then add a merge commit? | 13:42 |
| mordred | fungi: yah. | 13:42 |
| mordred | *me waves hands* | 13:42 |
| fungi | i've done that to combine history on projects before (there is a trick to importing branch from a remote repo into an empty branch of a different local repo, but i forget the details at the moment) | 13:43 |
| * mordred dry-runs | 13:43 | |
| *** eharney has joined #openstack-infra | 13:43 | |
| fungi | just didn't know if it was worth the effort | 13:43 |
| hashar | hey folks. Gotta a gate error earlier today with pip complaining with "No distributions at all found for X" Someone here recommended me to fill a bug which is now https://bugs.launchpad.net/openstack-ci/+bug/1334550 | 13:43 |
| uvirtbot | Launchpad bug 1334550 in openstack-ci "Could not find any downloads that satisfy the requirement X" [Undecided,New] | 13:43 |
| openstackgerrit | Chuck Short proposed a change to openstack-dev/pbr: Add Python 3.4 checking https://review.openstack.org/102828 | 13:44 |
| hashar | (logstash has a bunch of such pip issue when querying for message:"Could not find any downloads" AND tags:console.html ) | 13:44 |
| *** shayneburgess has quit IRC | 13:47 | |
| openstackgerrit | Chuck Short proposed a change to openstack-dev/hacking: Add Python 3.4 checking https://review.openstack.org/102829 | 13:47 |
| *** CaptTofu_ has joined #openstack-infra | 13:47 | |
| mordred | fungi: yup. easy peasy. about 4 steps | 13:48 |
| openstackgerrit | enikanorov proposed a change to openstack-infra/elastic-recheck: Add query for neutron bug 1332500 https://review.openstack.org/102551 | 13:48 |
| uvirtbot | Launchpad bug 1332500 in neutron "Lock wait timeout inserting row into routers table" [High,Confirmed] https://launchpad.net/bugs/1332500 | 13:48 |
| *** dkranz has quit IRC | 13:49 | |
| *** fabo has quit IRC | 13:49 | |
| *** CaptTofu_ has quit IRC | 13:49 | |
| *** CaptTof__ has joined #openstack-infra | 13:49 | |
| fungi | hashar: yeah, appscheduler isn't something we have in our mirror, so the job fell back to looking on pypi.python.org and for whatever reason failed to find an appropriate package version there (though when i look, there are a couple) | 13:50 |
| hashar | fungi: should the package get added to the pypi mirror so ? | 13:50 |
| hashar | (just wondering, I have no clue how you handle the deps) | 13:50 |
| fungi | hashar: it's not a dependency of openstack itself, so we don't currently add those sorts of things to the openstack restricted pypi mirror | 13:51 |
| fungi | hashar: though in the near future we'll probably be switching to full mirrors now that we have more dynamic requirements enforcement implemented | 13:51 |
| hashar | ok ok :-) | 13:52 |
| *** Hal_ has joined #openstack-infra | 13:52 | |
| hashar | for small projects like zuul, it is just a tiny annoyance | 13:52 |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/release-tools: Support proposed/* branches in release scripts https://review.openstack.org/102831 | 13:52 |
| hashar | another thing I noticed and that might be consuming nodes fro nothing, is that sometime we have both the check and gate pipeline jobs being triggered | 13:52 |
| hashar | typically, commenting on some old patches and saying "this is good to land" then voting workflow +1 | 13:53 |
| *** signed8bit has joined #openstack-infra | 13:53 | |
| hashar | that will enqueue the patch in test which we really don't care about | 13:53 |
| hashar | so maybe a change in gate should cancel the change in test (if that make any sense). Can fill a bug for it | 13:53 |
| *** jgallard has joined #openstack-infra | 13:54 | |
| *** annegent_ has joined #openstack-infra | 13:55 | |
| fungi | sdague: the snapshot change for nova merged a couple minutes ago, so should be good on that front | 13:55 |
| *** shayneburgess has joined #openstack-infra | 13:56 | |
| fungi | sdague: dhellmann: mordred: https://review.openstack.org/102584 is well on its way to a jenkins check +1, so if we want to quick-approve it we can unblock reqs changes soon | 13:56 |
| dhellmann | fungi: thanks! | 13:58 |
| *** shayneburgess has quit IRC | 13:59 | |
| openstackgerrit | Sergey Skripnick proposed a change to openstack-infra/config: Add missing export -f in rally-cli job https://review.openstack.org/102834 | 14:00 |
| *** chuckC has quit IRC | 14:01 | |
| dstufft | fungi: could not find appropiate package if there are indeed packages probably means the connection died | 14:01 |
| *** annegent_ has quit IRC | 14:01 | |
| dstufft | pip hides connection errors if they occur while fetching the /simple/<whatever>/ pages | 14:01 |
| dstufft | becasue external links | 14:01 |
| dstufft | more or less | 14:01 |
| *** sballe has joined #openstack-infra | 14:01 | |
| *** markmcclain has joined #openstack-infra | 14:02 | |
| dstufft | you jsut get a totally useful message about not being able to find anything that satisifies | 14:02 |
| *** chuckC has joined #openstack-infra | 14:02 | |
| openstackgerrit | Sergey Skripnick proposed a change to openstack-infra/config: Add missing export -f in rally-cli job https://review.openstack.org/102834 | 14:03 |
| *** jcoufal has quit IRC | 14:03 | |
| fungi | dstufft: makes sense, unfortunately. all that spidering would be hella noisy from random broken links otherwise | 14:04 |
| dstufft | fungi: yea basically | 14:05 |
| *** cnesa has joined #openstack-infra | 14:05 | |
| dstufft | fungi: if my PEP goes through it won't be that way anymore | 14:05 |
| fungi | presumably pip doesn't (currently) consider pypi a first-class citizen there and report on errors for it differently from the rest | 14:05 |
| *** annegent_ has joined #openstack-infra | 14:06 | |
| *** jcoufal has joined #openstack-infra | 14:07 | |
| *** dkranz has joined #openstack-infra | 14:07 | |
| *** rcarrillocruz has quit IRC | 14:08 | |
| dstufft | fungi: no, we probably could with some special casing | 14:08 |
| dstufft | but then i'd have one less reason for my PEP to get accepted >:] | 14:09 |
| *** hdd_ has joined #openstack-infra | 14:09 | |
| *** markmc has quit IRC | 14:11 | |
| *** signed8bit has joined #openstack-infra | 14:11 | |
| *** nati_ueno has joined #openstack-infra | 14:13 | |
| *** rcarrillocruz has joined #openstack-infra | 14:13 | |
| *** prad_ has joined #openstack-infra | 14:14 | |
| fungi | dstufft: very good point ;) | 14:14 |
| *** zz_gondoi is now known as gondoi | 14:14 | |
| zaro | morning | 14:16 |
| tchaypo | mordred: fungi last I checked I think my patch had merged but I'm not sure ignites released | 14:16 |
| zaro | jhesketh: going thru envinject changes again | 14:16 |
| *** lttrl has joined #openstack-infra | 14:16 | |
| zaro | jhesketh: looks a little waco. some stuff got reverted? | 14:17 |
| tchaypo | My understanding is that pypi itself acts differently | 14:17 |
| *** devkulkarni has quit IRC | 14:17 | |
| tchaypo | I've, it has vase insensitivity logic you won't get from a bare apache | 14:17 |
| tchaypo | *case | 14:17 |
| *** ildikov has joined #openstack-infra | 14:17 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: cloner to easily clone dependent repositories https://review.openstack.org/70373 | 14:20 |
| *** wenlock has joined #openstack-infra | 14:26 | |
| *** srenatus has left #openstack-infra | 14:26 | |
| *** fabo has joined #openstack-infra | 14:26 | |
| *** yfried|lunch is now known as yfried | 14:27 | |
| mordred | tchaypo: well, we use apache mod-autoindex just fine for our mirror ... so as long as file:// can do the same thing as crawling mod_autoindex, we're fine | 14:27 |
| *** bknudson has quit IRC | 14:27 | |
| *** david-lyle has joined #openstack-infra | 14:27 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Adding api events notifier https://review.openstack.org/102842 | 14:28 |
| *** dkliban_afk is now known as dkliban | 14:28 | |
| tchaypo | File:// will lowercase the file and s/-/_/ but that's about all | 14:31 |
| tchaypo | I think mod_autoindex can do more | 14:31 |
| *** esker has joined #openstack-infra | 14:32 | |
| *** YorikSar has joined #openstack-infra | 14:32 | |
| openstackgerrit | Chuck Short proposed a change to openstack-infra/config: Add python34 build and job template https://review.openstack.org/102847 | 14:36 |
| *** yfried has quit IRC | 14:38 | |
| *** jgallard has quit IRC | 14:38 | |
| YorikSar | Hello, people! | 14:38 |
| *** jgallard has joined #openstack-infra | 14:38 | |
| YorikSar | Did anyone try to use Gerrit REST API to post comments to changes? | 14:38 |
| *** yfried has joined #openstack-infra | 14:38 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/devstack-gate: Capture QEMU logs https://review.openstack.org/102849 | 14:38 |
| *** oomichi has quit IRC | 14:39 | |
| anteaya | reed krtaylor I'm fine but something happened yesterday in a personal realm and I need to stay sort of quiet for a bit to let things sync up so I won't be at full power for a while (not sure how long) | 14:42 |
| anteaya | reed krtaylor thank you for the support and I agree third party ci maintainers should be contributing to openstack and aware of how their artifacts contribute to the developer experience | 14:42 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Notify in IRC when puppet run fails https://review.openstack.org/96519 | 14:42 |
| *** yfried has quit IRC | 14:42 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Use ansible instead of direct ssh calls https://review.openstack.org/87821 | 14:42 |
| mordred | YorikSar: yup. works great | 14:42 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add mirror management playbooks https://review.openstack.org/100066 | 14:43 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Move ansible things into a module https://review.openstack.org/97032 | 14:43 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove salt https://review.openstack.org/96361 | 14:43 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove old salt references from the puppet https://review.openstack.org/96362 | 14:43 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add playbook for cleaning workspaces https://review.openstack.org/96646 | 14:43 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Rearrange into using roles https://review.openstack.org/96636 | 14:43 |
| *** yfried has joined #openstack-infra | 14:43 | |
| YorikSar | mordred: Not for me :( | 14:43 |
| YorikSar | mordred: My requests are just being ignored. | 14:43 |
| anteaya | reed krtaylor unfortunately most are not, hence the volume of issues in the third party space | 14:43 |
| mordred | YorikSar: you might want to look at the gertty source code - it uses the rest api and posts comments back | 14:43 |
| *** yfried has quit IRC | 14:43 | |
| *** yfried_ has joined #openstack-infra | 14:43 | |
| *** sweston_ has quit IRC | 14:43 | |
| mordred | YorikSar: one sec - lemme point you somewhere | 14:44 |
| anteaya | reed krtaylor I welcome suggestions and support in how to address this situation, since it is at the heart of all other conversations are third party discussions | 14:44 |
| YorikSar | mordred: Do you remember if it creates drafts for inline comments first or post everything at once? | 14:44 |
| anteaya | reed krtaylor please do start any mailing list thread or discussion that would draw attention to this issue, don't wait for me | 14:45 |
| zul | fungi: ping what was the precise image problem you where having last night? | 14:45 |
| *** oomichi has joined #openstack-infra | 14:46 | |
| mordred | YorikSar: http://git.openstack.org/cgit/stackforge/gertty/tree/gertty/sync.py#n480 | 14:46 |
| YorikSar | mordred: I'm on vacation and finally got around to earning that ham of yours ;) | 14:46 |
| mordred | YorikSar: yay ham! | 14:46 |
| *** yfried has joined #openstack-infra | 14:46 | |
| mordred | YorikSar: you might have to fish around a little bit, but that's the bit in the code where it's starting to push comments and reviews back | 14:47 |
| YorikSar | mordred: And everything finally worked and final POST is emitted, but... Nothing happens. | 14:47 |
| mordred | :) | 14:47 |
| *** oomichi has quit IRC | 14:48 | |
| *** zehicle_at_dell has quit IRC | 14:48 | |
| YorikSar | mordred: Ok, looks like it does the same thing I do... | 14:48 |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/infra-specs: Added specification for storyboard story tags https://review.openstack.org/97211 | 14:48 |
| YorikSar | mordred: I've stripped request down to this: | 14:49 |
| mordred | YorikSar: do you have the API token/password thing from the gerrit web ui? | 14:49 |
| YorikSar | curl -v -d '{"message": "Will it ever work?..", "labels": {"Code-Review": -1}}' 'https://review.openstack.org/a/changes/102214/revisions/1/review' --digest -u yorik-sar:my_pass | 14:49 |
| mordred | ok. you do | 14:49 |
| YorikSar | mordred: Just some digest auth. | 14:49 |
| *** yfried_ has quit IRC | 14:49 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/infra-specs: Added specification for storyboard story tags https://review.openstack.org/97211 | 14:49 |
| fungi | zul: hpcloud has an ubuntu partner image which is supposedly provided by canonical but does not appear in canonical's signed json file listing image uuids in cloud providers | 14:50 |
| mordred | yah. | 14:50 |
| *** pblaho is now known as pblaho|afk | 14:50 | |
| mordred | YorikSar: it does seem like you're doing the right thing | 14:50 |
| fungi | zul: probably just an oversight/lag but wanted to make sure it's nothing more nefarious rather than merely assuming | 14:50 |
| mordred | YorikSar: might have to wait for jeblair to wake up - he knows way more about this than I | 14:50 |
| *** shayneburgess has joined #openstack-infra | 14:50 | |
| zul | fungi: ok lemme see if i can find out whats happening | 14:51 |
| *** rgerganov has quit IRC | 14:51 | |
| dhellmann | sdague, fungi : "Build timed out (after 125 minutes). Marking the build as failed" in http://logs.openstack.org/84/102584/4/check/check-requirements-integration-dsvm/6216219/console.html for https://review.openstack.org/#/c/102584/ | 14:51 |
| fungi | zul: clarkb provided the uuid and image name last night, but i can dig them back out of the irc log if needed | 14:51 |
| fungi | dhellmann: yeah, just saw that. apparently it was working, just taking a while | 14:51 |
| *** pblaho|afk is now known as pblaho | 14:51 | |
| YorikSar | mordred: Ok, I'll try. Thanks for your help! | 14:52 |
| zul | fungi: should be fixed now | 14:52 |
| fungi | dhellmann: seemed to have been cut down in the middle of compiling lxml for one of the branches | 14:52 |
| *** annegent_ has quit IRC | 14:52 | |
| fungi | zul: great--thanks! was there just a delay in publishing the updated list? | 14:53 |
| *** annegent_ has joined #openstack-infra | 14:53 | |
| zul | fungi: nah...i just need to find the right people at canonical | 14:53 |
| dhellmann | fungi: ok. Should I just try again, or do you think the job is likely to run slower now and needs a different timeout? | 14:53 |
| jeblair | YorikSar: the change id in that url shouldn't be a number, it should be the change id in the rest api | 14:53 |
| jeblair | YorikSar: example: https://review.openstack.org/a/changes/openstack-infra%2Fconfig~master~I428b1d2a2a0de93242af2f89f8bb487c931f304f/revisions/8c297b707a72e4d2e75d997c710c0edbabd5b024/review | 14:53 |
| jeblair | YorikSar: (also, the revision is a git sha, not a number) | 14:54 |
| YorikSar | jeblair: I've shortened it down so that it won't get too long in IRC. The same happens with full IDs. | 14:54 |
| YorikSar | jeblair: Good morning :) | 14:54 |
| fungi | dhellmann: not sure if it takes longer in hpcloud, or if it's a trusty-specific performance issue or something else | 14:55 |
| *** hdd_ has quit IRC | 14:55 | |
| *** dstanek is now known as dstanek_zzz | 14:55 | |
| jeblair | YorikSar: you understand that sort of thing makes it impossible to help you, right? | 14:55 |
| fungi | dhellmann: might have just lucked out and hit a slow node, or we might need to increase the timeout on that job to debug it better | 14:55 |
| YorikSar | jeblair: Here's my original request: | 14:57 |
| YorikSar | curl -v -d '{"message": "I noticed smth wrong with your PEP8 results.", "comments": {"stackalytics/dashboard/web.py": [{"line": "31", "message": "F821 undefined name 'breaking_pep8'"}]}}' 'https://review.openstack.org/a/changes/stackforge%2Fstackalytics~master~I633092a9541b50032543962c9551de8ff23989db/revisions/4468b0b4a049f55a2d738fc1d78ef3673995cf73/review' --digest -u yorik-sar:my_pass | 14:57 |
| YorikSar | jeblair: Although both simple and complex ID formats should work the same according to manual. | 14:58 |
| *** yolanda has joined #openstack-infra | 14:58 | |
| sdague | dhellmann: I wonder if something changed about the way the job runs so that we're compiling a lot more now | 14:58 |
| sdague | in which case, we probably need to figure that out | 14:58 |
| fungi | sdague: dhellmann: mordred: maybe we're not caching some build results we cached previously? | 14:58 |
| sdague | how long did devstack setup take? That's usually a pretty good indicator of inherently slow node | 14:58 |
| dhellmann | 9 minutes? | 14:59 |
| mordred | fungi: I believe trusty _does_ have a new version of gcc | 14:59 |
| dhellmann | http://logs.openstack.org/84/102584/4/check/check-requirements-integration-dsvm/6216219/console.html#_2014-06-26_12_47_17_325 | 14:59 |
| mordred | so it's entirely possible that something will make compiles slower | 14:59 |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Fix groups-dev portal update sync issue https://review.openstack.org/102858 | 14:59 |
| *** sballe has quit IRC | 15:00 | |
| fungi | sdague: 12 minutes from "Started by user anonymous" to "Removing sudo privileges for devstack user" | 15:00 |
| *** gondoi is now known as zz_gondoi | 15:00 | |
| *** mrmartin has quit IRC | 15:00 | |
| fungi | mordred: would ccache help for this, or should it be enabled by default? | 15:00 |
| dolphm | are the console logs from docs jobs not captured by logstash? | 15:00 |
| mordred | fungi: it might help - but also yeah, I think it shoudl just be enabled by default | 15:01 |
| *** jcoufal has quit IRC | 15:01 | |
| mordred | fungi: unless there is something in trusty that is changing how ccache works | 15:01 |
| *** achanda has joined #openstack-infra | 15:01 | |
| *** jlibosva has quit IRC | 15:01 | |
| yolanda | jeblair, i'm having some issues running tox for nodepool. It hangs just simply calling time.sleep(3), for example | 15:02 |
| sdague | fungi: so that's slower | 15:02 |
| yolanda | i could run some tests, but now it looks as it's hanging on every run | 15:02 |
| sdague | but not crazy so | 15:02 |
| sdague | well, also the 7 year old cpus that hp cloud tells us it has might have an impact | 15:03 |
| jeblair | YorikSar: only thing i see right now is i send the line numbers as integers, not strings | 15:03 |
| *** jlibosva has joined #openstack-infra | 15:03 | |
| sdague | be worth looking at an old version of these logs to figure out if we really compiled numpy this often | 15:03 |
| *** dizquierdo has joined #openstack-infra | 15:04 | |
| *** sabeen has joined #openstack-infra | 15:04 | |
| jeblair | YorikSar: i also send 'strict_labels': False, but that shouldn't affect anything as long as you have access (which you should for a code-review -1) | 15:04 |
| dhellmann | sdague: wow, yeah, I see 7 times | 15:05 |
| jeblair | yolanda: git master, or a local change? | 15:05 |
| yolanda | jeblair, local change | 15:05 |
| yolanda | tests seem to be doing nothing, and not writing nothing to the .testrepository logs | 15:05 |
| yolanda | i have to kill them by command line | 15:06 |
| *** _nadya_ has joined #openstack-infra | 15:06 | |
| YorikSar | jeblair: Tried int there instead. Nothing's changed. If I ommit inline comments altogether, nothing happens as well. | 15:06 |
| yolanda | any idea about what can be causing that? | 15:06 |
| jeblair | yolanda: that probably means that there's still a thread running; the tests wait for all of the launch and delete threads to finish before exiting | 15:07 |
| jeblair | yolanda: it may help to run the tests one at a time | 15:07 |
| jeblair | in the foreground | 15:07 |
| yolanda | jeblair, how can i do it? i tried directly with testr but i have the same problem | 15:08 |
| *** achanda has quit IRC | 15:08 | |
| *** kashyap is now known as kashyap_bbiab | 15:08 | |
| fungi | YorikSar: were you testing that *before* https://review.openstack.org/102214 was abandoned? you can't put code review votes on a closed change | 15:08 |
| jeblair | .tox/py27/bin/python -m testtools.run nodepool.tests.test_nodepool | 15:08 |
| *** annegent_ has quit IRC | 15:08 | |
| yolanda | ok, let me try that | 15:08 |
| mordred | fungi: nice catch | 15:09 |
| fungi | YorikSar: ahh, yes i see you commented on it through the webui before you abandoned it | 15:09 |
| fungi | YorikSar: so presumably so | 15:09 |
| YorikSar | fungi: Yes | 15:09 |
| YorikSar | fungi: Right now I'm trying to just put some comment there. | 15:10 |
| fungi | and it only has one patchset, so can't be an issue of trying to vote on a closed patchset either | 15:10 |
| mestery | I'm trying to push a tag for a new python-neutronclient release, and I'm hitting a problem, hoping someone can help: http://paste.openstack.org/show/85001/ | 15:11 |
| *** gokrokv__ has quit IRC | 15:11 | |
| *** james_li has joined #openstack-infra | 15:12 | |
| *** gokrokve has joined #openstack-infra | 15:12 | |
| *** yfried has quit IRC | 15:12 | |
| jeblair | mestery: is the tag signed? | 15:12 |
| mestery | jeblair: Yes, I signed it locally with my key. | 15:12 |
| fungi | mestery: please don't use --tags https://wiki.openstack.org/wiki/GerritJenkinsGit#Tagging_a_Release | 15:12 |
| mestery | fungi: I was reading that too, thanks for the pointer though! | 15:13 |
| mestery | fungi: I'll use the specific tag the next time. | 15:13 |
| jeblair | mestery: you need to be added to the neutron-ptl group | 15:13 |
| mordred | maybe we didn't add mestery to mestery ljjk | 15:13 |
| mestery | jeblair: That explains it :) | 15:13 |
| jeblair | mestery: done | 15:13 |
| mestery | mordred: What you said. :P | 15:13 |
| yolanda | jeblair, that works much better, thx | 15:13 |
| mestery | jeblair: thanks! | 15:13 |
| mordred | yah. | 15:13 |
| fungi | aha, yup. i took care of that for lp group administrator purposes but never thought to see if the same had been overlooked in gerrit | 15:14 |
| jeblair | yolanda: np; i use that all the time. once you find the test causing the problem, you can add its name to the end of that command to just run it | 15:14 |
| mestery | jeblair fungi: Thanks, much better now. :) | 15:14 |
| fungi | mestery: part of the benefit of those groups being self-administered (we assumed) is that outgoing ptls would add their successors to them | 15:15 |
| *** andreaf_ has quit IRC | 15:15 | |
| fungi | in practice that seems to not actually happen particularly often | 15:15 |
| * mestery makes a note to ensure he does this in the future. :) | 15:15 | |
| fungi | appreciated! | 15:16 |
| *** rdopieralski has quit IRC | 15:16 | |
| YorikSar | jeblair, fungi: Does gertty use digest auth or whatever Gerrit UI uses? | 15:18 |
| sdague | mordred: are we supposed to be compiling numpy 7 times in that script? | 15:20 |
| sdague | http://paste.openstack.org/show/85002/ | 15:20 |
| mordred | sdague: yes | 15:20 |
| sdague | 106 Running setup.py install for pytz | 15:20 |
| sdague | i.e. we install pytz 106 times | 15:20 |
| fungi | mordred: sdague: dhellmann: i think we're not actually using ccache | 15:21 |
| mordred | sdague: we ensure that we can install each of our projects clean each of the ways it can install | 15:21 |
| *** UtahDave has joined #openstack-infra | 15:21 | |
| mordred | fungi: we should fix that :) | 15:21 |
| *** dims__ has quit IRC | 15:21 | |
| mordred | sdague: yah. expected | 15:21 |
| fungi | mordred: sdague: dhellmann: it's installed and there's a symlink from /usr/local/bin/cc to /usr/bin/ccache, but compilation logs seem to imply we're running x86_64-linux-gnu-gcc which is a symlink in /usr/bin to /usr/bin/gcc-4.8 | 15:22 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Adds Third Party CI account name format directions https://review.openstack.org/101013 | 15:22 |
| sdague | can we do it in parallel? | 15:22 |
| *** dims__ has joined #openstack-infra | 15:22 | |
| mordred | we've actually hit problems with each form we test | 15:22 |
| *** skolekonov has quit IRC | 15:22 | |
| mordred | sdague: maybe? | 15:22 |
| mordred | but let's fix ccache first before we get cleverer | 15:23 |
| dhellmann | fungi: maybe that's how the python compiler was configured? | 15:23 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Updated AngularJS to 1.2.18 https://review.openstack.org/102651 | 15:24 |
| fungi | dhellmann: that's what i suspect | 15:24 |
| fungi | jeblair: one potential zuul bug from this morning i haven't gotten looking further into yet is http://paste.openstack.org/show/84970/ (hitting it on changes 102794 and 102796 apparently). refusing to enqueue because it can't find the parent (which is actually a merged change from last year) | 15:24 |
| fungi | jeblair: i've noticed gerrit seems to be adding the change corresponding to the tip of master as the parent dependency lately, which i think is new behavior in 2.8 | 15:25 |
| fungi | though that may only be if the tip of the branch is not a merge commit | 15:26 |
| * fungi needs to run some lunchtime errands, but will return in an hour or so | 15:26 | |
| *** dstanek_zzz is now known as dstanek | 15:28 | |
| mordred | sdague, fungi: I have a meeting but will try to figure out why ccache doesn't like the symlinks on trusty in a bit | 15:28 |
| fungi | jeblair: one other commonality between those two failures is that the commit used cherry-pick instead of merge, so the commit which was merged is not the same commit zuul is asking gerrit to cough up? | 15:29 |
| *** rcarrill` has joined #openstack-infra | 15:29 | |
| fungi | at least that's my current working theory | 15:29 |
| * fungi vanishes for a little while | 15:29 | |
| sdague | mordred: I'm not actually convinced it was working before either | 15:29 |
| sdague | in staring at an old log | 15:29 |
| mordred | sdague: well, if we can fix it, it should shave noticeable time | 15:30 |
| *** annegent_ has joined #openstack-infra | 15:30 | |
| *** dizquierdo has quit IRC | 15:30 | |
| *** dizquierdo has joined #openstack-infra | 15:30 | |
| jeblair | fungi: i can't see the error you pasted; paste.o.o is 500ing for me | 15:30 |
| *** rcarrillocruz has quit IRC | 15:31 | |
| sdague | maybe, the numpy compile is actually only 2 minutes from what I can see | 15:31 |
| sdague | or less than that | 15:31 |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack-infra/config: Disable Designate DevStack gate on stable/havana+icehouse https://review.openstack.org/102864 | 15:31 |
| mordred | jeblair: I'm so glad new paste is better than old drizzle paste | 15:31 |
| mordred | sdague: but with the iterations and lxml that adds up | 15:31 |
| *** zzelle has joined #openstack-infra | 15:32 | |
| *** mrmartin has joined #openstack-infra | 15:32 | |
| *** julim has quit IRC | 15:32 | |
| jeblair | mordred: does lodgeit log anything anywhere? | 15:32 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/devstack-gate: Capture apache configuration files https://review.openstack.org/102745 | 15:33 |
| mordred | jeblair: don't remember ... :( | 15:33 |
| joa | hi :) Anyone familiar with jaypipes' set of puppet scripts and his article about setting up a 3rd-party CI system ? Tried following it, envountered a few issues (can't tell if it's me or his scripts that might lack a few things) ? | 15:33 |
| *** nati_ueno has quit IRC | 15:35 | |
| *** jgallard has quit IRC | 15:35 | |
| *** devkulkarni has joined #openstack-infra | 15:35 | |
| jeblair | mordred, fungi: i restarted lodgeit and apache and it works better now :( | 15:36 |
| mordred | great :( | 15:36 |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/pbr: Update integration script for Apache 2.4 https://review.openstack.org/102584 | 15:37 |
| sdague | ok, that includes explitic ccache use | 15:37 |
| mordred | jeblair: the stackaltics guys swear by uwsgi as a way to deal with memory hungry things ... it's on my tdl to figure out if that helps any of our things | 15:38 |
| *** markmcclain has quit IRC | 15:38 | |
| openstackgerrit | yolanda.robla proposed a change to openstack-infra/nodepool: Build images using diskimage-builder https://review.openstack.org/46482 | 15:38 |
| *** jgriffith is now known as jfiffith | 15:38 | |
| *** jfiffith is now known as jgriffith | 15:39 | |
| *** annegent_ has quit IRC | 15:40 | |
| *** Longgeek has quit IRC | 15:40 | |
| *** Longgeek has joined #openstack-infra | 15:41 | |
| YorikSar | mordred: Depends on a thing. I've tried something with PyPy and CPython behind uWSGI. In short: sharind data between processes doesn't work. | 15:42 |
| YorikSar | mordred: I found it out after writing up a very nice article about how it should work... | 15:43 |
| mordred | well, I don't want to share data, so that's fine | 15:43 |
| YorikSar | mordred: But uWSGI is very smart and have gazilion of options that can make your processes work as you expected :) | 15:44 |
| mordred | cool. I need to read more | 15:45 |
| *** Longgeek has quit IRC | 15:45 | |
| YorikSar | mordred: Hm... Memory sharing is the only option I found to let bigger number of stackalytics web workers to fit to limited memory set. | 15:45 |
| *** dangers_away is now known as dangers | 15:46 | |
| mordred | well, my concern with lodgeit is unfounded hunch that it leaks | 15:46 |
| mordred | it shouldn't need tons of memory | 15:47 |
| *** dizquierdo has quit IRC | 15:47 | |
| *** mmaglana has joined #openstack-infra | 15:48 | |
| *** craigbr has joined #openstack-infra | 15:49 | |
| YorikSar | mordred: You might want to set some limit of either number of requests or lifetime for each worker | 15:50 |
| YorikSar | mordred: Then workers will get reloaded and memory leaks will heal | 15:50 |
| *** _nadya_ has quit IRC | 15:52 | |
| *** dstanek is now known as dstanek_zzz | 15:52 | |
| *** Longgeek has joined #openstack-infra | 15:52 | |
| Longgeek | Who is sahara-ci's managers? | 15:53 |
| Longgeek | Sorry, savanna-ci | 15:54 |
| SergeyLukjanov | Longgeek, hey | 15:54 |
| SergeyLukjanov | Longgeek, it's folk in my team, he's now offline | 15:54 |
| SergeyLukjanov | Longgeek, you can ask me | 15:55 |
| SergeyLukjanov | about it | 15:55 |
| Longgeek | SergeyLukjanov: The savanna-ci system is not working properly? https://review.openstack.org/#/c/102335/ | 15:55 |
| SergeyLukjanov | Longgeek, let me take a look | 15:56 |
| Longgeek | SergeyLukjanov: ok | 15:57 |
| SergeyLukjanov | Longgeek, it's unfortunately not always stable enough - we're running very heavy integration tests on the trunk devstack lab with a bunch of vms and it's not stable sometimes | 15:58 |
| SergeyLukjanov | Longgeek, you could "recheck sahara-ci" | 15:58 |
| SergeyLukjanov | Longgeek, and thanks for the patch :) | 15:58 |
| Longgeek | SergeyLukjanov: that's ok | 15:59 |
| *** rwsu has joined #openstack-infra | 15:59 | |
| *** annegent_ has joined #openstack-infra | 15:59 | |
| *** _nadya_ has joined #openstack-infra | 16:00 | |
| *** sballe has joined #openstack-infra | 16:01 | |
| *** chandan_kumar has quit IRC | 16:02 | |
| *** shayneburgess_ has joined #openstack-infra | 16:03 | |
| *** YorikSar has quit IRC | 16:03 | |
| *** IvanBerezovskiy has left #openstack-infra | 16:03 | |
| *** shayneburgess has quit IRC | 16:03 | |
| *** shayneburgess_ is now known as shayneburgess | 16:03 | |
| *** __afazekas is now known as afazekas | 16:04 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Clarifies documentation about wildcards in queries https://review.openstack.org/102573 | 16:04 |
| *** bknudson has joined #openstack-infra | 16:05 | |
| *** gyee has joined #openstack-infra | 16:05 | |
| *** chuckC has quit IRC | 16:07 | |
| *** shayneburgess has quit IRC | 16:08 | |
| *** reed has joined #openstack-infra | 16:08 | |
| *** shayneburgess has joined #openstack-infra | 16:08 | |
| *** ominakov has quit IRC | 16:09 | |
| *** ominakov has joined #openstack-infra | 16:10 | |
| krtaylor | reed, I'd like to increase awareness of the 3rd party CI weekly meetings, can we put a paragraph with links in the weekly newsletter? | 16:10 |
| *** gyee has quit IRC | 16:11 | |
| reed | krtaylor, good idea to increase visibility of meetings, besides links, do you think a regular update of the results would be better? | 16:11 |
| reed | krtaylor, like the TC is now doing, with regular blog posts, I think a summary of the relevant discussions and decisions would drive a lot more attention | 16:12 |
| krtaylor | reed, excellent, yes, I agree | 16:12 |
| reed | doesn't have to be a long blog post, just one sentence sent to me via email would be enough | 16:13 |
| krtaylor | reed, I like it - I'll draft a intro paragraph today | 16:13 |
| reed | krtaylor, let me pull the logs of the last meeting, I can work with you on that | 16:13 |
| *** ominakov has quit IRC | 16:14 | |
| krtaylor | reed, great thanks, I'll talk to anteaya about a weekly wrapup summary too | 16:14 |
| reed | krtaylor, only one meeting? | 16:14 |
| reed | is this a new effort? http://eavesdrop.openstack.org/meetings/3rd_party_ci/2014/ | 16:14 |
| reed | gosh, so many things happen I loose track myself sometimes | 16:15 |
| krtaylor | reed, hehheh, yeah, that is part of the problem, we have multiple names we call this effort | 16:15 |
| jeblair | reed: http://eavesdrop.openstack.org/meetings/third_party/2014/ | 16:15 |
| jeblair | reed: anteaya renamed it | 16:15 |
| *** annegent_ has quit IRC | 16:15 | |
| reed | got it, thanks | 16:16 |
| *** annegent_ has joined #openstack-infra | 16:16 | |
| *** craigbr_ has joined #openstack-infra | 16:17 | |
| *** craigbr has quit IRC | 16:17 | |
| *** craigbr_ is now known as craigbr | 16:17 | |
| *** amitpp has joined #openstack-infra | 16:19 | |
| *** gyee has joined #openstack-infra | 16:20 | |
| reed | krtaylor, there seem to be 2 interesting discussions: storage of logs and their availability over time | 16:20 |
| *** unicell1 has joined #openstack-infra | 16:20 | |
| reed | and two etherpads... | 16:21 |
| *** shayneburgess has quit IRC | 16:21 | |
| krtaylor | reed, and ease of reading those logs | 16:21 |
| reed | krtaylor, was there a resolution or is that just the beginning of a discussion? | 16:22 |
| *** shayneburgess has joined #openstack-infra | 16:22 | |
| krtaylor | reed, we have discussed other requirements, but those are the ones being addressed at the moment, but we need more discussion, more participation | 16:22 |
| *** unicell has quit IRC | 16:22 | |
| krtaylor | reed, the newsletter should help that | 16:23 |
| reed | krtaylor, here is how I would approach a summary: the TL;DR: "the Third Party CI team needs your help to define how logs will be consumed and stored." | 16:24 |
| *** signed8bit is now known as signed8bit_ZZZzz | 16:24 | |
| openstackgerrit | Longgeek proposed a change to openstack-infra/puppet-yum: Add .gitreview file and update puppet coding style https://review.openstack.org/102876 | 16:24 |
| reed | and below a longer version with links to eavesdrop and etherpads | 16:24 |
| *** Ryan_Lane has joined #openstack-infra | 16:24 | |
| *** mmaglana has quit IRC | 16:24 | |
| reed | krtaylor, and (even better) an email discussion | 16:25 |
| krtaylor | reed, agreed, specific questions will bring more interest, good ideas | 16:25 |
| *** pblaho is now known as pblaho|afk | 16:25 | |
| *** mmaglana has joined #openstack-infra | 16:25 | |
| reed | krtaylor, remember the first axiom of email communication: nobody reads beyond the first 3 words :) | 16:26 |
| krtaylor | reed, how true, except for the newsletters that is :) | 16:26 |
| reed | krtaylor, I don't have specific analytics but I'm ready to bet that 99% of people just skim through the words in *bold* :) | 16:27 |
| dstufft | reed: which is why you should bold random words to mess with people | 16:27 |
| *** adalbas has joined #openstack-infra | 16:27 | |
| joa | Hi there :) does anyone have any link/tip/guide or anything relevant to help a CI-newbie setting up a third-party CI ? I already got the ci.openstack.org article about third-party; as well as jaypipes'guide (which does not seem enough to me; or I missed something). | 16:28 |
| openstackgerrit | Longgeek proposed a change to openstack-infra/puppet-apparmor: Update puppet coding style https://review.openstack.org/102794 | 16:28 |
| *** pblaho|afk is now known as pblaho | 16:29 | |
| *** mmaglana has quit IRC | 16:29 | |
| *** _nadya_ has quit IRC | 16:31 | |
| *** hashar has quit IRC | 16:31 | |
| *** krtaylor is now known as krtaylor_brb | 16:32 | |
| *** zhiyan is now known as zhiyan_ | 16:32 | |
| *** jpich has quit IRC | 16:35 | |
| *** gokrokve has quit IRC | 16:38 | |
| *** gokrokve has joined #openstack-infra | 16:38 | |
| reed | joa, what exactly you think you're missing? | 16:38 |
| openstackgerrit | Longgeek proposed a change to openstack-infra/puppet-yum: Add .gitreview Rakefile files and update puppet coding style https://review.openstack.org/102876 | 16:38 |
| reed | joa, in general, questions about specific problems are more likely to find an answer while open ended questions require a huge investment from people who may potentially help | 16:39 |
| *** denis_makogon has quit IRC | 16:40 | |
| joa | reed: Well, my jenkins didnt seem to have the Gearman plugin; when I tried restarting it, as indicated (possible bug explained in guide); the jenkins didnt restart. | 16:40 |
| clarkb | o/ | 16:40 |
| reed | a request for review please: https://review.openstack.org/#/c/101227/ | 16:40 |
| joa | Tweaking the conf a bit I managed to get it running back up (seemed like a few conf values were missing) | 16:40 |
| joa | And finally, there still seems to be a configuration issue: I get a 404 on / from the jenkins | 16:41 |
| openstackgerrit | Longgeek proposed a change to openstack-infra/puppet-apparmor: Add Rakefile file and update puppet coding style https://review.openstack.org/102794 | 16:41 |
| clarkb | dhellmann: the only other known failures are libvirt (we just have to live with this imo otherwise we get nowhere, and solum rabbitmq which seems sketchy because nothing else dealing with rabbitmq has trouble) | 16:41 |
| reed | joa, outdated documentation maybe? | 16:41 |
| joa | so I wondered if there wasn't something missing in the guide/configuration samples; yeah | 16:41 |
| joa | thus asking if there was a (maybe?) more up-to-date guide. | 16:41 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: 403 status code added https://review.openstack.org/101146 | 16:42 |
| reed | joa, I am not aware of anything beyond the CI docs you already read | 16:42 |
| clarkb | we can stop running that job on trusty but we intentionally switched all of the integration testing because its integration testing and doesn't make sense to aprtition it | 16:42 |
| *** gokrokve has quit IRC | 16:42 | |
| joa | I currently almost feel like removing it all; and retry from scratch; but I dont know which openstack-infra/config.git scripts and files might be useful to ease my job at setting up trigger/polling and reporting. | 16:42 |
| reed | joa, but if you find issues with them, maybe adding comments to Jay's post or propose a patch to fix the CI docs would be a good exercise for you | 16:43 |
| openstackgerrit | Longgeek proposed a change to openstack-infra/puppet-dashboard: Add Rakefile file and dd variables and document https://review.openstack.org/102796 | 16:43 |
| joa | I'm thinking of writing some kind of article/wiki page once I'm done with it. (or yeah,commenting/proposing fixes) | 16:43 |
| reed | joa, that would be awesome. I know many developers who need help setting up their local CI systems and it's not trivial at all... enriching current documentation would be great | 16:44 |
| joa | yeah; I guess so | 16:44 |
| clarkb | joa: what version of jenkins are you using? we use the lts | 16:44 |
| joa | clarkb: I guess it's the one on ubuntu 12.04 lts | 16:45 |
| clarkb | oh we install from jenkins upstream | 16:45 |
| clarkb | and use their lts version | 16:45 |
| joa | 1.424.6 from the package description. | 16:45 |
| joa | Oh okay | 16:45 |
| clarkb | ya 1.424 is ancient | 16:45 |
| joa | so that might be a part of why ? | 16:45 |
| clarkb | yes | 16:45 |
| joa | mhhh. alright. got to try that first. | 16:46 |
| joa | thanks for the pointer :) Getting late here, I might came back to assult you guys with questions tomorrow :p | 16:46 |
| joa | reed: thanks also | 16:46 |
| clarkb | sdague: initial check of logstash has workers being happy at least judging by graphite numbers | 16:47 |
| sdague | clarkb: ok, so just slow volumes again? | 16:47 |
| *** _nadya_ has joined #openstack-infra | 16:47 | |
| clarkb | looking at cacti numbers the worker throughput is really bursty and not consistent | 16:48 |
| clarkb | so ya probably slow volumes. I can run sar in a bit to try and confirm | 16:48 |
| clarkb | and work backwards from there if now | 16:48 |
| clarkb | *if not | 16:48 |
| sdague | yeh, I haven't yet gotten to os-loganalyze changes, was poking at both the libvirt thing and the requirements integration job this morning | 16:50 |
| *** chuckC has joined #openstack-infra | 16:50 | |
| sdague | on the plus side, the libvirt work around was merged, and our failure rates seem to have gone back under control | 16:51 |
| clarkb | woo | 16:51 |
| sdague | at least by http://jogo.github.io/gate/ | 16:51 |
| clarkb | how is requirements integration doing? I am still trying to catch up on that | 16:51 |
| clarkb | looks like numpy tried to link against something that doesn't exist | 16:51 |
| sdague | here is the current aggregate set of bug fixes - https://review.openstack.org/#/c/102584/ | 16:53 |
| clarkb | libkrb5-dev is needed in devstack because we build python pacakges that link against it | 16:53 |
| sdague | once we got numpy to compile, then we ran out of time | 16:53 |
| clarkb | and one of devstack's responsibilities is making python libs compile | 16:53 |
| sdague | so being really explicit about ccache | 16:53 |
| sdague | clarkb: well for devstack we just install numpy from packages | 16:54 |
| clarkb | maybe it shouldn't | 16:54 |
| clarkb | the only reason that integration test uses devstack is so that devstack handles this | 16:54 |
| clarkb | if we are not going to have devstack handle this we should just stop using devstack | 16:54 |
| clarkb | libmysqlclient-dev, libpq-dev, et al | 16:54 |
| *** nati_ueno has joined #openstack-infra | 16:55 | |
| sdague | maybe. I also don't want to add 5 minutes to every gate run to prove we can compile numpy 25k times a week | 16:55 |
| sdague | and the rest of the stuff we can install from packages | 16:56 |
| sdague | because if we're saying, don't install anything from packages, we should just run arch :) | 16:56 |
| *** aaronorosen has joined #openstack-infra | 16:56 | |
| clarkb | ran out of time? looks like the change passed testing | 16:56 |
| clarkb | sdague: well I would be in favor of that >_> | 16:56 |
| clarkb | sdague: it would have the benefit of discovering how every libvirt release is broken as they happen | 16:56 |
| sdague | clarkb: the last run, before I forced ccache, ran out of time | 16:57 |
| clarkb | and ovs and kernel and so on | 16:57 |
| sdague | timed out at 2hrs | 16:57 |
| mtreinish | sdague: why do I feel like that was targetted at me... | 16:57 |
| clarkb | sdague: huh we symlink ccache into places on the image builds or at least try to | 16:57 |
| aaronorosen | Hi, I think there is an issue setting the contact information in gerrit which is required for a new contributor. Right now if you click save changes it returns an error saying Cannot store contact information. | 16:57 |
| *** dstanek_zzz is now known as dstanek | 16:57 | |
| sdague | clarkb: well, it was apparently wrong | 16:57 |
| *** sarob has joined #openstack-infra | 16:57 | |
| sdague | anyway, it was easy to do in the test script | 16:57 |
| clarkb | sdague: :/ ok so that should be debugged in the general case but being explicit here can't hurt | 16:57 |
| clarkb | ya | 16:57 |
| clarkb | aaronorosen: are you a member of the openstack foundation? | 16:58 |
| sdague | clarkb: I'm sure dhellmann would love +2s on that patch | 16:58 |
| *** gangil has joined #openstack-infra | 16:58 | |
| clarkb | aaronorosen: and does your primary membership email match your primary email in gerrit? | 16:58 |
| clarkb | sdague: ya reviewing now | 16:58 |
| *** aaronorosen is now known as arosen | 16:58 | |
| clarkb | sdague: mtreinish fwiw a rolling distro for testing isn't that crazy if we want to find braekages asap | 16:58 |
| gangil | clarkb: I was facing that problem, and yes my emails at both the places are same | 16:58 |
| sdague | clarkb: only if you have people to work on debugging and fixing them | 16:58 |
| * clarkb will test | 16:58 | |
| gangil | clarkb: do you want me to try again? | 16:58 |
| arosen | clarkb: ^ gangil is having the issue | 16:59 |
| arosen | thanks | 16:59 |
| clarkb | gangil: hold on I will give it a try | 16:59 |
| sdague | we already have *way* more issues to fix than we have fixers, I don't really want to own fixing the kernel as well :P | 16:59 |
| clarkb | gangil: works fine for me | 16:59 |
| clarkb | sdague: yup I agree | 16:59 |
| clarkb | sdague: just pointing out Arch would have its benefits | 17:00 |
| gangil | clarkb: I will give it another shot then | 17:00 |
| *** kashyap_bbiab is now known as kashyap | 17:00 | |
| sdague | clarkb: also, in looking at an old log on precise, I'm not convinced that ccache was actually working | 17:00 |
| clarkb | gangil: https://www.openstack.org/profile/ double check that your profile there matches gerrit | 17:00 |
| *** pblaho has quit IRC | 17:00 | |
| clarkb | sdague: its possible it wasn't | 17:00 |
| mtreinish | clarkb: heh, I was just thinking that too | 17:00 |
| *** dkehn_ has joined #openstack-infra | 17:00 | |
| *** packet has joined #openstack-infra | 17:01 | |
| *** kashyap is now known as kashyap_bbiab | 17:01 | |
| sdague | dhellmann: your chariot awaits - https://review.openstack.org/#/c/102584/ | 17:01 |
| *** mrmartin has quit IRC | 17:01 | |
| *** david-lyle has quit IRC | 17:01 | |
| gangil | I dont think I have an openstack.org account yet. I have launchpad and review/openstack accounts. If its linked to that then registering on openstack.org might fix the error. Trying that now | 17:02 |
| *** dkehn has quit IRC | 17:02 | |
| dhellmann | sdague: +2a | 17:02 |
| dhellmann | sdague: and *thank you* | 17:02 |
| *** david-lyle has joined #openstack-infra | 17:02 | |
| clarkb | gangil: the openstack.org profile is your foundation membership. it is required | 17:02 |
| *** dkehnx has quit IRC | 17:02 | |
| *** kashyap_bbiab is now known as kashyap | 17:02 | |
| *** dkehn_ has quit IRC | 17:02 | |
| *** dkehn_ has joined #openstack-infra | 17:03 | |
| *** david-lyle has quit IRC | 17:03 | |
| *** dkehn has joined #openstack-infra | 17:04 | |
| *** david-lyle has joined #openstack-infra | 17:04 | |
| *** mmaglana has joined #openstack-infra | 17:04 | |
| *** praneshp has joined #openstack-infra | 17:04 | |
| *** dkehn_ is now known as dkehnx | 17:05 | |
| *** krtaylor_brb is now known as krtaylor | 17:06 | |
| gangil | clarkb: Its asking me for a affiliation to a organization. I dont have any affiliation thats fits the criteria. And it doesnt even allow me to skip it. | 17:06 |
| clarkb | reed: ^ | 17:06 |
| clarkb | gangil: I would do something like NA, but reed was around recently and is the expert on these things | 17:07 |
| gangil | I will try NA | 17:07 |
| reed | gangil, "unaffiliated" is a good answer | 17:07 |
| *** Hal_ has quit IRC | 17:07 | |
| *** hogepodge has joined #openstack-infra | 17:08 | |
| *** dkliban is now known as dkliban_brb | 17:09 | |
| clarkb | dhellmann: that should get the trusty gears going again. Any opposition to the continued migration at this point? | 17:09 |
| reed | gangil, which documentation are you following ? in other words, how did you know that you needed an account on gerrit to submit code to openstack? | 17:09 |
| clarkb | dhellmann: I have a stack up to move a bunch of the integrated projects, but feedback very welcome | 17:09 |
| * reed on a quest to find broken documentation | 17:10 | |
| gangil | reed: arosen helped me out :) | 17:10 |
| *** harlowja_away is now known as harlowja | 17:10 | |
| dhellmann | clarkb: I don't object, but can you test some of these more obscure jobs? | 17:10 |
| arosen | reed: I don't think we followed any docs just went through review.openstack.org and signed in. Probably our fault then | 17:11 |
| clarkb | dhellmann: at this point the only changes being made have been tested | 17:11 |
| reed | arosen, remember to point people to the instructions to the wiki :) there is no shortcut, things are complicated and instructions need to be followed to the letter :) | 17:11 |
| clarkb | dhellmann: the issue with integration tests is we basically had to do them all at once | 17:11 |
| clarkb | dhellmann: because we could wedge the gate if we didn't | 17:11 |
| arosen | reed: :) | 17:11 |
| clarkb | dhellmann: granted we wedged some projects anyways but not all of them | 17:11 |
| *** penguinRaider__ has joined #openstack-infra | 17:12 | |
| dhellmann | clarkb: well, yeah, a thing that should have taken me 5 minutes yesterday has derailed me for 36+ hrs now so, you know. *shrug* | 17:12 |
| *** annegent_ has quit IRC | 17:12 | |
| krotscheck | GOOOOOOOOOOL | 17:13 |
| clarkb | krotscheck: boo | 17:13 |
| clarkb | :P | 17:13 |
| *** annegent_ has joined #openstack-infra | 17:13 | |
| reed | krotscheck, US or DE? | 17:13 |
| jeblair | de | 17:13 |
| clarkb | dhellmann: right so the alternative was to continue putting it off | 17:13 |
| krotscheck | Oh, no worries. The US is advancing anyway | 17:13 |
| *** rcarrillocruz has joined #openstack-infra | 17:13 | |
| reed | bah | 17:13 |
| clarkb | krotscheck: not if portugal gets another goal and the US is shutout iirc | 17:13 |
| reed | wake me up when this cup is interesting (never) | 17:14 |
| clarkb | reed: huh, this cup has been great | 17:14 |
| dhellmann | clarkb: that was the only alternative? | 17:14 |
| greghaynes | clarkb: I thought ghana is closer to us than portugal | 17:14 |
| reed | clarkb, ah! | 17:14 |
| krotscheck | .... | 17:14 |
| greghaynes | clarkb: and por needs like 5 | 17:14 |
| *** duran has joined #openstack-infra | 17:14 | |
| clarkb | dhellmann: it really wasn't feasible to test every corner case beacuse we have too many | 17:14 |
| *** james_li has quit IRC | 17:14 | |
| krotscheck | clarkb: Doesn’t portugal need another 3 point goal differential? | 17:14 |
| clarkb | dhellmann: it takes JJB about 35 minutes to update all of the devstack jobs now | 17:14 |
| clarkb | dhellmann: its insane | 17:15 |
| dhellmann | clarkb: you don't have some way to know which test jobs are going to run on a given node type so you can run one of each against master? | 17:15 |
| *** devkulkarni has quit IRC | 17:15 | |
| clarkb | greghaynes: krotscheck oh I thought it didn't matter | 17:15 |
| *** dkehn_ has joined #openstack-infra | 17:15 | |
| reed | clarkb, interesting how a spectacle of such high grade of corruption can be appreciated by US citizens... it makes US Congress seem like a paradise | 17:15 |
| gangil | submiteed my first patch! \m/ | 17:15 |
| gangil | *submitted | 17:15 |
| clarkb | it was basicaly lose by 1 and 2 goals up in the other game | 17:15 |
| reed | gangil, that was fast :) | 17:15 |
| clarkb | dhellmann: I do | 17:15 |
| *** rcarrill` has quit IRC | 17:15 | |
| greghaynes | clarkb: nah, ger spanked por so badly they need a lot of goals ;) | 17:15 |
| clarkb | dhellmann: there are hundreds | 17:15 |
| clarkb | dhellmann: at 60 minutes per tests that becomes really expensive | 17:15 |
| krotscheck | clarkb: Yes, but it’s goal-diff-portugal + goal-diff-germany >= 5 | 17:15 |
| *** ildikov has quit IRC | 17:16 | |
| *** markmcclain has joined #openstack-infra | 17:16 | |
| *** rcarrill` has joined #openstack-infra | 17:16 | |
| dhellmann | clarkb: well, I consider the approach selected to have been expensive, too. The rest of us don't get to do things like this, right? We have to make the gate jobs pass before we can land changes. | 17:16 |
| clarkb | dhellmann: I would have to script up building hundreds of nodes and trigger them all outside of zuul | 17:16 |
| *** adrian_otto has quit IRC | 17:17 | |
| clarkb | because unfortunately nodepool and zuul don't grok this use case | 17:17 |
| dhellmann | if the tooling isn't there, maybe we should work on that for the next upgrade | 17:17 |
| clarkb | dhellmann: I agree. mordred and I were talking about it a bit | 17:17 |
| clarkb | dhellmann: but honestly the fix is to have like 5 jobs | 17:17 |
| *** dkehn has quit IRC | 17:17 | |
| jeblair | dhellmann: tbh, i'd just as soon stay on precise | 17:17 |
| *** sarob has quit IRC | 17:17 | |
| jeblair | i think upgrading the distro we test on is everyones responsibility | 17:17 |
| jeblair | but if people don't want to help out with that | 17:18 |
| jeblair | no problem | 17:18 |
| *** rcarrillocruz has quit IRC | 17:18 | |
| *** dkehnx has quit IRC | 17:18 | |
| jeblair | we'll just stay where we are | 17:18 |
| *** markmcclain1 has joined #openstack-infra | 17:18 | |
| *** annegent_ has quit IRC | 17:18 | |
| clarkb | we test too much without getting any additional coverage and can really trim it down. Its on my list of things to do to look at the current matrix and put it up so we can trim it down | 17:18 |
| dhellmann | jeblair: I'm not objecting to helping. I'm trying to find a way we can do that without stopping forward progress on other efforts. I did, in fact, spend a considerable amount of time doing this instead of several other things I was supposed to have been doing yesterday and today because this is also important. | 17:19 |
| jeblair | dhellmann: i may have been a little quick on that reply, but the whole "us vs them" thing you just started kinda set me off | 17:19 |
| *** dkehn has joined #openstack-infra | 17:19 | |
| dhellmann | jeblair: yeah, I could have phrased that better | 17:19 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: remove envinject usage from job https://review.openstack.org/102893 | 17:20 |
| dhellmann | my point was, though, that in most other parts of this project we strive for extensive testing, and clarkb was saying this change was untestable | 17:20 |
| zaro | jhesketh: ^ | 17:20 |
| dhellmann | or roughly that | 17:20 |
| *** yfried has joined #openstack-infra | 17:20 | |
| clarkb | dhellmann: untestable to the rigorous degree you were asking for | 17:20 |
| *** markmcclain has quit IRC | 17:20 | |
| clarkb | it was tested in a way that covered 95% of our use of the d-g jobs | 17:20 |
| dhellmann | ok, so you missed one job? that's not bad. | 17:21 |
| fungi | sdague: check-requirements-integration-dsvm SUCCESS in 1h 05m 31s | 17:21 |
| clarkb | no we missed a lot of jobs | 17:21 |
| sdague | fungi: yep | 17:21 |
| clarkb | but they run a lot less frequently with a lot less impact on everyon | 17:21 |
| sdague | explicit ccache ftw | 17:21 |
| clarkb | we tested the core jobs. mysql, postgres, grenade | 17:21 |
| clarkb | and neutron | 17:22 |
| *** jlibosva has quit IRC | 17:22 | |
| dhellmann | clarkb: ok, maybe you're underestimating the impact of this job on oslo. Our #1 priority this cycle is making new libraries. We cannot add those to the requirements list while this job is failing. | 17:22 |
| *** melwitt has joined #openstack-infra | 17:22 | |
| dhellmann | I can accept some blame for not making that clear initially when the job started failing. | 17:22 |
| clarkb | so far solum has failed and the pbr/requirements integration failed | 17:23 |
| jeblair | so why is it failing? | 17:23 |
| jeblair | solum doesn't block us | 17:23 |
| clarkb | jeblair: the kerberos thing and numpy as in a similar position | 17:24 |
| dhellmann | jeblair: there were at least 3 different issues, which I think sdague and fungi have helped work out solutions to | 17:24 |
| clarkb | oh and apache 2.4 | 17:24 |
| dhellmann | yes | 17:24 |
| jeblair | which of those are unsolved? | 17:24 |
| sdague | solum | 17:25 |
| sdague | we just nailed the requirements job | 17:25 |
| *** dstanek is now known as dstanek_zzz | 17:25 | |
| jeblair | sdague: solum is a stackforge project and is outside of the scope of this conversation | 17:25 |
| sdague | jeblair: sure | 17:25 |
| jeblair | sdague: so there is no current problem with the requirements repo? | 17:25 |
| *** ihrachyshka has quit IRC | 17:25 | |
| dhellmann | That's right, I think we're clear now. | 17:25 |
| sdague | well, we don't know that it actually works for stable/ | 17:25 |
| sdague | because we don't test there | 17:26 |
| clarkb | does it ever run aginst stable? | 17:26 |
| sdague | we should send up a trial baloon that we didn't break stable with these fixes once the pbr patch merges | 17:26 |
| clarkb | I thought mordred said it was a non issue | 17:26 |
| sdague | clarkb: we run this job in stable | 17:26 |
| fungi | clarkb: i looked into the ccache thing before lunch. we do symlink cc to it, but python package compiles don't call cc they call a more arch/version-specific name which take cross-build toolchain possibilities into account | 17:26 |
| clarkb | fungi: that would explain it | 17:26 |
| sdague | out of pbr master (as far as I can tell) | 17:26 |
| clarkb | fungi: should /etc/profile just update PATH instead? | 17:27 |
| clarkb | and fwiw we would test that the trusty fixes didn't break precise because we run master pbr against stable/* other | 17:27 |
| sdague | clarkb: right, that was my point :) | 17:27 |
| sdague | once pbr change lands, push some random stable/icehouse requirements change and see what happens | 17:28 |
| *** jaypipes has joined #openstack-infra | 17:30 | |
| *** dkehn__ has joined #openstack-infra | 17:30 | |
| fungi | clarkb: i think setting some global path or other relevant envvar to mass-enable ccache might not be a bad idea. it does slightly change the platform we're testing in to be more nonstandard that what a random dev might have on hand, but i think the time/resource savings outweighs any slight inconsistency there | 17:32 |
| clarkb | right and on single use nodes the delta would be small because we essentially start from scratch every time anyways | 17:32 |
| *** dkehn__ has quit IRC | 17:32 | |
| clarkb | so no hot cache | 17:32 |
| *** otherwiseguy has quit IRC | 17:33 | |
| *** dkehn has quit IRC | 17:33 | |
| clarkb | time improvements only come from installign the same thing over and over | 17:33 |
| fungi | clarkb: also, as we were already pre-linking it on the job workers, this seems to have been the intent all along, however incompletely executed it turned out to be | 17:33 |
| sdague | well, honestly, what about putting ccache config into devstack ? | 17:33 |
| *** dkehn__ has joined #openstack-infra | 17:33 | |
| *** dkehn_ has quit IRC | 17:33 | |
| clarkb | sdague: that won't help the unittest jobs | 17:33 |
| sdague | because we're compiling at least a few things there | 17:33 |
| sdague | true | 17:33 |
| fungi | sdague: yeah, only helps devstack then | 17:33 |
| sdague | fungi: well it also helps the "random dev" problem | 17:34 |
| fungi | sdague: true for random devs who run things within devstack | 17:34 |
| sdague | which... is most of them | 17:34 |
| *** dkehn has joined #openstack-infra | 17:35 | |
| sdague | it would actually be really interesting to both do that, and build a warm ccache in nodepool | 17:35 |
| *** msabramo has joined #openstack-infra | 17:36 | |
| clarkb | dhellmann: on the oslo release front would it help to get requirements updates prior to libs being ready? I believe the mirror building script will handle that case and definitely would if we switched to bandersnatch | 17:36 |
| clarkb | projects wouldn't be able to consume them until the libs are actually available | 17:36 |
| dhellmann | clarkb: I'm not sure what you mean? | 17:36 |
| dhellmann | "get requirements updates prior to libs being ready"? | 17:36 |
| clarkb | dhellmann: just add oslo.foo>1.0 to globale requirements now | 17:37 |
| dhellmann | do you mean just add, e.g., "oslo.i18n" to the globa-requirements.txt even though there's not one there now? | 17:37 |
| clarkb | before oslo.foo has been released | 17:37 |
| dhellmann | oh, that works? | 17:37 |
| clarkb | I think so | 17:37 |
| dhellmann | I'm not sure how much that buys us, but it's nice to know doing that in parallel with the release might not be broken. :-) | 17:37 |
| clarkb | well it decouples your gantt chart bars | 17:38 |
| clarkb | if you can do both at the same time then you don't have to worry about either asm uch | 17:38 |
| dhellmann | true, landing the change to the requirements list doesn't usually take very long but decoupling is good | 17:38 |
| *** eharney has quit IRC | 17:38 | |
| *** ryanpetrello has quit IRC | 17:39 | |
| Longgeek | SergeyLukjanov: recheck sahara-ci, the same problem | 17:40 |
| *** devkulkarni has joined #openstack-infra | 17:41 | |
| *** chandan_kumar has joined #openstack-infra | 17:41 | |
| *** _nadya_ has quit IRC | 17:41 | |
| *** MarkAtwood has joined #openstack-infra | 17:42 | |
| *** dkliban_brb is now known as dkliban | 17:42 | |
| clarkb | dhellmann: but ya the mirror builder script should handle that and pbr integration dones't actually install from global requirements iirc | 17:42 |
| clarkb | dhellmann: the integration test installs real projects so as long as you don't add oslo.foo to a project until after it is a avialable it should work | 17:43 |
| clarkb | dhellmann: the bit that would need testing is the mirror bit | 17:43 |
| dhellmann | clarkb: that integration script installs the projects listed in the requirements project, but it doesn't use the global requirements list | 17:43 |
| clarkb | dhellmann: right so I think it could be gamed in this way | 17:43 |
| dhellmann | right, the mirror was the part I expected to see break | 17:43 |
| clarkb | if it helps | 17:43 |
| clarkb | sdague: also sorry I haven't dug into ES yet, but this game is hard to look away from | 17:44 |
| sdague | clarkb: no worries | 17:44 |
| *** ryanpetrello has joined #openstack-infra | 17:44 | |
| dhellmann | after this pbr change lands so the job passes otherwise, I'll try it with oslo.i18n. I'd be worried that a project might see adding a lib to the list as an indication that it is ready, so I'm not sure I would want us doing it in that order, but if it works it can save us some time if we need it. | 17:44 |
| clarkb | dhellmann: ya that is worth considering. "It is in requirements, it must work" | 17:45 |
| dhellmann | clarkb: right, part of our release process is giving a clear message about the new library when it's available and this might muddy things. OTOH, if we're in a pinch, it'll be good to know if it's an option. | 17:46 |
| *** cindyo has joined #openstack-infra | 17:51 | |
| *** amitpp has quit IRC | 17:52 | |
| *** _nadya_ has joined #openstack-infra | 17:53 | |
| *** annegent_ has joined #openstack-infra | 17:55 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 17:55 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 17:56 | |
| *** markwash has quit IRC | 17:57 | |
| *** markmcclain1 has quit IRC | 17:57 | |
| *** dkehn__ is now known as dkehnx | 17:57 | |
| *** james_li has joined #openstack-infra | 17:58 | |
| *** rkukura has joined #openstack-infra | 17:59 | |
| sdague | clarkb: unrelated to requirements jobs - http://www.nytimes.com/interactive/2014/06/25/upshot/984-ways-the-united-states-can-advance-to-the-next-round-of-the-world-cup.html?_r=0 is pretty cool visualization | 17:59 |
| clarkb | ok the stack at https://review.openstack.org/#/c/102663/ should be ready. I manually tested the -python27, -docs, -pep8 jobs for all of the projects and did unittests and docs for specs as well if they exist | 18:00 |
| *** annegent_ has quit IRC | 18:00 | |
| *** david-lyle has quit IRC | 18:00 | |
| jeblair | clarkb: what's the current state of nova/libvirt? | 18:01 |
| *** david-lyle has joined #openstack-infra | 18:01 | |
| clarkb | jeblair: sdague's nova libvirt hack around merged | 18:01 |
| clarkb | jeblair: so it appaers that nova will power through and work on fixing it. sdague also sent mail to the list about it | 18:01 |
| jeblair | clarkb, sdague: cool | 18:01 |
| *** ryanpetrello has quit IRC | 18:02 | |
| *** ildikov has joined #openstack-infra | 18:03 | |
| *** david-lyle has quit IRC | 18:04 | |
| *** david-ly_ has joined #openstack-infra | 18:04 | |
| *** rfolco has quit IRC | 18:05 | |
| *** MarkAtwood has quit IRC | 18:06 | |
| *** ryanpetrello has joined #openstack-infra | 18:07 | |
| *** eharney has joined #openstack-infra | 18:08 | |
| notmyname | where are the -specs repos' generated docs published? | 18:08 |
| *** esker has quit IRC | 18:08 | |
| *** cindyo has quit IRC | 18:10 | |
| openstackgerrit | A change was merged to openstack-infra/config: Added new python-magnetodbclient project to Stackforge https://review.openstack.org/91050 | 18:11 |
| *** cindyo_ has joined #openstack-infra | 18:11 | |
| jeblair | notmyname: nowhere atm | 18:11 |
| *** cindyo_ has quit IRC | 18:11 | |
| notmyname | jeblair: what's the plan there? | 18:12 |
| *** cindyo_ has joined #openstack-infra | 18:12 | |
| *** doude has quit IRC | 18:12 | |
| jeblair | notmyname: they should be published to specs.o.o eventually. i'm not sure that anyone is driving that atm. | 18:12 |
| notmyname | jeblair: ok, thanks | 18:12 |
| *** cindyo_ has quit IRC | 18:13 | |
| notmyname | jeblair: as I type that domain into my browser, I'm reminded (thanks autocomplete!) that "specs" is the name of a huge liquor shop chain in south texas. similar to bevmo | 18:13 |
| *** cindyo_ has joined #openstack-infra | 18:13 | |
| Alex_Gaynor | mordred: ping | 18:13 |
| jeblair | notmyname: you're right, we should totally make bevmo.o.o a cname for specs. | 18:14 |
| *** cindyo__ has joined #openstack-infra | 18:14 | |
| notmyname | jeblair: or just cut out all the dithering and start drinking any time we look at plans for openstack code ;-) | 18:14 |
| *** zehicle_at_dell has joined #openstack-infra | 18:14 | |
| *** dstanek_zzz is now known as dstanek | 18:14 | |
| notmyname | sdague: suppose one wanted a different dashboard in gerrit for a set of projects. are there any docs on how to create a new custom dashboard? | 18:15 |
| *** shayneburgess has quit IRC | 18:15 | |
| annegentle | notmyname: I did it by copy pasta for the docs program, let me find the patch | 18:15 |
| *** cindyo__ has quit IRC | 18:15 | |
| *** shayneburgess has joined #openstack-infra | 18:15 | |
| annegentle | notmyname: somethin' like this https://review.openstack.org/#/c/99960/ | 18:15 |
| *** cindyo has joined #openstack-infra | 18:15 | |
| sdague | notmyname: the easiest way to do it is client side with the gerrit-dash-creator that I built | 18:16 |
| *** cindyo has quit IRC | 18:16 | |
| sdague | notmyname: https://github.com/stackforge/gerrit-dash-creator | 18:16 |
| *** cindyo has joined #openstack-infra | 18:16 | |
| *** chianingwang has joined #openstack-infra | 18:16 | |
| sdague | lots of examples in there | 18:17 |
| notmyname | sdague: got a link? | 18:17 |
| notmyname | annegentle: thanks@ | 18:17 |
| openstackgerrit | yolanda.robla proposed a change to openstack-infra/nodepool: Build images using diskimage-builder https://review.openstack.org/46482 | 18:17 |
| *** cindyo has quit IRC | 18:17 | |
| *** pballand has joined #openstack-infra | 18:18 | |
| notmyname | sdague: ok. I'll take a look after lunch | 18:18 |
| *** cindyo has joined #openstack-infra | 18:18 | |
| sdague | notmyname: cool | 18:18 |
| *** cindyo_ has quit IRC | 18:18 | |
| *** cindyo has quit IRC | 18:18 | |
| *** cindyo has joined #openstack-infra | 18:18 | |
| *** cindyo_ has joined #openstack-infra | 18:20 | |
| *** openstackgerrit has quit IRC | 18:20 | |
| *** cindyo_ has quit IRC | 18:20 | |
| *** cindyo_ has joined #openstack-infra | 18:21 | |
| *** cindyo_ has quit IRC | 18:22 | |
| *** pballand has quit IRC | 18:22 | |
| *** cindyo_ has joined #openstack-infra | 18:22 | |
| *** cindyo_ has quit IRC | 18:22 | |
| *** _nadya_ has quit IRC | 18:23 | |
| *** cindyo_ has joined #openstack-infra | 18:23 | |
| *** ominakov has joined #openstack-infra | 18:23 | |
| fungi | wherefor goest thou, openstackgerrit? | 18:23 |
| *** cindyo has quit IRC | 18:23 | |
| fungi | i guess it was restarting for the 91050 merge | 18:24 |
| *** cindyo has joined #openstack-infra | 18:24 | |
| fungi | but hath not return'ed | 18:24 |
| *** cindyo has quit IRC | 18:25 | |
| *** cindyo has joined #openstack-infra | 18:25 | |
| *** tkelsey has quit IRC | 18:25 | |
| *** zehicle_at_dell has quit IRC | 18:26 | |
| *** cindyo has quit IRC | 18:26 | |
| *** cindyo has joined #openstack-infra | 18:27 | |
| *** zehicle_at_dell has joined #openstack-infra | 18:27 | |
| *** cindyo has quit IRC | 18:27 | |
| *** ominakov has quit IRC | 18:27 | |
| *** esker has joined #openstack-infra | 18:27 | |
| *** cindyo has joined #openstack-infra | 18:27 | |
| *** cindyo_ has quit IRC | 18:28 | |
| *** Ryan_Lane has quit IRC | 18:28 | |
| *** cindyo has quit IRC | 18:28 | |
| *** cindyo has joined #openstack-infra | 18:29 | |
| *** cindyo has quit IRC | 18:29 | |
| *** cindyo has joined #openstack-infra | 18:30 | |
| *** achanda has joined #openstack-infra | 18:30 | |
| *** pelix has quit IRC | 18:30 | |
| *** marcoemorais has joined #openstack-infra | 18:30 | |
| *** cindyo has quit IRC | 18:31 | |
| *** cindyo has joined #openstack-infra | 18:31 | |
| *** cindyo has quit IRC | 18:31 | |
| *** cindyo has joined #openstack-infra | 18:32 | |
| *** sarob has joined #openstack-infra | 18:32 | |
| *** cindyo has quit IRC | 18:33 | |
| *** cindyo has joined #openstack-infra | 18:33 | |
| *** cindyo has quit IRC | 18:34 | |
| *** cnesa2 has joined #openstack-infra | 18:34 | |
| *** cindyo has joined #openstack-infra | 18:34 | |
| *** cindyo has quit IRC | 18:35 | |
| *** cindyo has joined #openstack-infra | 18:35 | |
| *** cindyo_ has joined #openstack-infra | 18:36 | |
| *** cindyo__ has joined #openstack-infra | 18:37 | |
| *** praneshp_ has joined #openstack-infra | 18:37 | |
| *** cindyo___ has joined #openstack-infra | 18:38 | |
| *** cindyo__ has quit IRC | 18:38 | |
| clarkb | https://review.openstack.org/#/c/102663/ any takers? and nova is fine with moving to trusty as well | 18:39 |
| *** cindyo___ has quit IRC | 18:39 | |
| clarkb | they are at the end of the stack | 18:39 |
| *** praneshp has quit IRC | 18:39 | |
| *** praneshp_ is now known as praneshp | 18:39 | |
| clarkb | I am going to try and lunch early today so that I don't miss first half of next world cup games | 18:39 |
| *** cindyo__ has joined #openstack-infra | 18:39 | |
| *** rcarrill` has quit IRC | 18:39 | |
| *** openstackgerrit has joined #openstack-infra | 18:40 | |
| fungi | i ended up restarting gerritbot | 18:40 |
| fungi | so not it's back | 18:40 |
| fungi | er, now | 18:40 |
| fungi | clarkb: lgtm. approved, we can watch | 18:40 |
| clarkb | poor bots | 18:40 |
| *** rcarrillocruz has joined #openstack-infra | 18:40 | |
| *** cindyo has quit IRC | 18:40 | |
| *** cindyo__ has quit IRC | 18:40 | |
| fungi | openstackstatus is on the missing list as well. fixing | 18:40 |
| *** cindyo has joined #openstack-infra | 18:41 | |
| *** cindyo_ has quit IRC | 18:41 | |
| *** cindyo has quit IRC | 18:41 | |
| *** cindyo has joined #openstack-infra | 18:41 | |
| *** tkelsey has joined #openstack-infra | 18:42 | |
| *** openstackstatus has joined #openstack-infra | 18:42 | |
| *** ChanServ sets mode: +v openstackstatus | 18:42 | |
| *** signed8bit has quit IRC | 18:43 | |
| *** james_li has quit IRC | 18:46 | |
| *** dkehnx has quit IRC | 18:46 | |
| *** jaypipes has quit IRC | 18:46 | |
| *** UtahDave has quit IRC | 18:46 | |
| *** dkranz has quit IRC | 18:46 | |
| *** rlandy has quit IRC | 18:46 | |
| *** pdmars has quit IRC | 18:46 | |
| *** camunoz has quit IRC | 18:46 | |
| *** jpeeler has quit IRC | 18:46 | |
| *** davidlenwell has quit IRC | 18:46 | |
| *** grantbow has quit IRC | 18:46 | |
| *** annegentle has quit IRC | 18:46 | |
| *** jamielennox|away has quit IRC | 18:46 | |
| *** ianw has quit IRC | 18:46 | |
| *** ekarlso has quit IRC | 18:46 | |
| *** bcrochet has quit IRC | 18:46 | |
| *** rakhmerov has quit IRC | 18:46 | |
| *** slamont has quit IRC | 18:46 | |
| *** bradm has quit IRC | 18:46 | |
| *** cindyo has quit IRC | 18:46 | |
| *** praneshp has quit IRC | 18:46 | |
| *** chianingwang has quit IRC | 18:46 | |
| *** david-ly_ has quit IRC | 18:46 | |
| *** ildikov has quit IRC | 18:46 | |
| *** chandan_kumar has quit IRC | 18:46 | |
| *** craigbr has quit IRC | 18:46 | |
| *** bknudson has quit IRC | 18:46 | |
| *** zzelle has quit IRC | 18:46 | |
| *** sabeen has quit IRC | 18:46 | |
| *** prad_ has quit IRC | 18:46 | |
| *** mfer has quit IRC | 18:46 | |
| *** miqui has quit IRC | 18:46 | |
| *** mriedem has quit IRC | 18:46 | |
| *** ArxCruz has quit IRC | 18:46 | |
| *** afazekas_ has quit IRC | 18:46 | |
| *** lcheng has quit IRC | 18:46 | |
| *** ociuhandu has quit IRC | 18:46 | |
| *** SumitNaiksatam has quit IRC | 18:46 | |
| *** jhesketh has quit IRC | 18:46 | |
| *** ilyashakhat has quit IRC | 18:46 | |
| *** SlickNik has quit IRC | 18:46 | |
| *** jamespage has quit IRC | 18:46 | |
| *** vponomaryov has quit IRC | 18:46 | |
| *** gothicmindfood has quit IRC | 18:46 | |
| *** Kiall has quit IRC | 18:46 | |
| *** zhiyan_ has quit IRC | 18:46 | |
| *** jcooley has quit IRC | 18:46 | |
| *** saper has quit IRC | 18:46 | |
| *** skraynev has quit IRC | 18:46 | |
| *** jd__ has quit IRC | 18:46 | |
| *** portante has quit IRC | 18:46 | |
| *** briancline has quit IRC | 18:46 | |
| *** tnurlygayanov has quit IRC | 18:46 | |
| *** openstackgerrit has quit IRC | 18:46 | |
| *** sarob has quit IRC | 18:46 | |
| *** marcoemorais has quit IRC | 18:46 | |
| *** achanda has quit IRC | 18:46 | |
| *** melwitt has quit IRC | 18:46 | |
| *** wenlock has quit IRC | 18:46 | |
| *** comstud has quit IRC | 18:46 | |
| *** sweston has quit IRC | 18:46 | |
| *** mrda-away has quit IRC | 18:46 | |
| *** simonmcc has quit IRC | 18:46 | |
| *** wendar has quit IRC | 18:46 | |
| *** mgagne has quit IRC | 18:46 | |
| *** ityaptin has quit IRC | 18:46 | |
| *** roaet_ has quit IRC | 18:46 | |
| *** adam_g has quit IRC | 18:46 | |
| *** rainya has quit IRC | 18:46 | |
| *** dolphm has quit IRC | 18:46 | |
| *** russellb has quit IRC | 18:46 | |
| *** zz_gondoi has quit IRC | 18:46 | |
| *** vishy has quit IRC | 18:46 | |
| *** stevebaker has quit IRC | 18:46 | |
| *** Mithrandir has quit IRC | 18:46 | |
| *** alaski has quit IRC | 18:46 | |
| *** jbryce has quit IRC | 18:46 | |
| *** Adri2000 has quit IRC | 18:46 | |
| *** zigo has quit IRC | 18:46 | |
| *** jgriffith has quit IRC | 18:46 | |
| *** morgabra has quit IRC | 18:46 | |
| *** notmyname has quit IRC | 18:46 | |
| *** hdd has quit IRC | 18:46 | |
| *** antonym has quit IRC | 18:46 | |
| *** thingee has quit IRC | 18:46 | |
| *** clarkb has quit IRC | 18:46 | |
| *** ttx has quit IRC | 18:46 | |
| *** uvirtbot has quit IRC | 18:46 | |
| *** cindyo has joined #openstack-infra | 18:46 | |
| *** openstackgerrit has joined #openstack-infra | 18:46 | |
| *** praneshp has joined #openstack-infra | 18:46 | |
| *** sarob has joined #openstack-infra | 18:46 | |
| *** marcoemorais has joined #openstack-infra | 18:46 | |
| *** achanda has joined #openstack-infra | 18:46 | |
| *** chianingwang has joined #openstack-infra | 18:46 | |
| *** david-ly_ has joined #openstack-infra | 18:46 | |
| *** ildikov has joined #openstack-infra | 18:46 | |
| *** james_li has joined #openstack-infra | 18:46 | |
| *** chandan_kumar has joined #openstack-infra | 18:46 | |
| *** dkehnx has joined #openstack-infra | 18:46 | |
| *** jaypipes has joined #openstack-infra | 18:46 | |
| *** melwitt has joined #openstack-infra | 18:46 | |
| *** craigbr has joined #openstack-infra | 18:46 | |
| *** bknudson has joined #openstack-infra | 18:46 | |
| *** zzelle has joined #openstack-infra | 18:46 | |
| *** UtahDave has joined #openstack-infra | 18:46 | |
| *** sabeen has joined #openstack-infra | 18:46 | |
| *** wenlock has joined #openstack-infra | 18:46 | |
| *** prad_ has joined #openstack-infra | 18:46 | |
| *** dkranz has joined #openstack-infra | 18:46 | |
| *** mfer has joined #openstack-infra | 18:46 | |
| *** miqui has joined #openstack-infra | 18:46 | |
| *** mriedem has joined #openstack-infra | 18:46 | |
| *** rlandy has joined #openstack-infra | 18:46 | |
| *** pdmars has joined #openstack-infra | 18:46 | |
| *** ArxCruz has joined #openstack-infra | 18:46 | |
| *** afazekas_ has joined #openstack-infra | 18:46 | |
| *** lcheng has joined #openstack-infra | 18:46 | |
| *** ociuhandu has joined #openstack-infra | 18:46 | |
| *** SumitNaiksatam has joined #openstack-infra | 18:46 | |
| *** jhesketh has joined #openstack-infra | 18:46 | |
| *** camunoz has joined #openstack-infra | 18:46 | |
| *** jpeeler has joined #openstack-infra | 18:46 | |
| *** ilyashakhat has joined #openstack-infra | 18:46 | |
| *** tnurlygayanov has joined #openstack-infra | 18:46 | |
| *** SlickNik has joined #openstack-infra | 18:46 | |
| *** davidlenwell has joined #openstack-infra | 18:46 | |
| *** jamespage has joined #openstack-infra | 18:46 | |
| *** grantbow has joined #openstack-infra | 18:46 | |
| *** rakhmerov has joined #openstack-infra | 18:46 | |
| *** vponomaryov has joined #openstack-infra | 18:46 | |
| *** annegentle has joined #openstack-infra | 18:46 | |
| *** jamielennox|away has joined #openstack-infra | 18:46 | |
| *** ianw has joined #openstack-infra | 18:46 | |
| *** comstud has joined #openstack-infra | 18:46 | |
| *** gothicmindfood has joined #openstack-infra | 18:46 | |
| *** Kiall has joined #openstack-infra | 18:46 | |
| *** zhiyan_ has joined #openstack-infra | 18:46 | |
| *** sweston has joined #openstack-infra | 18:46 | |
| *** mrda-away has joined #openstack-infra | 18:46 | |
| *** ekarlso has joined #openstack-infra | 18:46 | |
| *** bcrochet has joined #openstack-infra | 18:46 | |
| *** simonmcc has joined #openstack-infra | 18:46 | |
| *** wendar has joined #openstack-infra | 18:46 | |
| *** bradm has joined #openstack-infra | 18:46 | |
| *** slamont has joined #openstack-infra | 18:46 | |
| *** ityaptin has joined #openstack-infra | 18:46 | |
| *** jcooley has joined #openstack-infra | 18:46 | |
| *** saper has joined #openstack-infra | 18:46 | |
| *** mgagne has joined #openstack-infra | 18:46 | |
| *** skraynev has joined #openstack-infra | 18:46 | |
| *** jd__ has joined #openstack-infra | 18:46 | |
| *** portante has joined #openstack-infra | 18:46 | |
| *** roaet_ has joined #openstack-infra | 18:46 | |
| *** adam_g has joined #openstack-infra | 18:46 | |
| *** rainya has joined #openstack-infra | 18:46 | |
| *** dolphm has joined #openstack-infra | 18:46 | |
| *** briancline has joined #openstack-infra | 18:46 | |
| *** russellb has joined #openstack-infra | 18:46 | |
| *** uvirtbot has joined #openstack-infra | 18:46 | |
| *** ttx has joined #openstack-infra | 18:46 | |
| *** clarkb has joined #openstack-infra | 18:46 | |
| *** thingee has joined #openstack-infra | 18:46 | |
| *** antonym has joined #openstack-infra | 18:46 | |
| *** hdd has joined #openstack-infra | 18:46 | |
| *** notmyname has joined #openstack-infra | 18:46 | |
| *** morgabra has joined #openstack-infra | 18:46 | |
| *** jgriffith has joined #openstack-infra | 18:46 | |
| *** zigo has joined #openstack-infra | 18:46 | |
| *** Adri2000 has joined #openstack-infra | 18:46 | |
| *** jbryce has joined #openstack-infra | 18:46 | |
| *** alaski has joined #openstack-infra | 18:46 | |
| *** Mithrandir has joined #openstack-infra | 18:46 | |
| *** stevebaker has joined #openstack-infra | 18:46 | |
| *** vishy has joined #openstack-infra | 18:46 | |
| *** zz_gondoi has joined #openstack-infra | 18:46 | |
| *** markmcclain has joined #openstack-infra | 18:46 | |
| *** david-ly_ has quit IRC | 18:47 | |
| *** david-lyle has joined #openstack-infra | 18:47 | |
| *** cyli has quit IRC | 18:48 | |
| *** NobodyCam has quit IRC | 18:48 | |
| *** jeblair has quit IRC | 18:48 | |
| *** dansmith has quit IRC | 18:48 | |
| *** dangers has quit IRC | 18:48 | |
| *** NithyaG_afk has quit IRC | 18:48 | |
| *** flaper87|afk has quit IRC | 18:48 | |
| *** soren has quit IRC | 18:48 | |
| *** lifeless has quit IRC | 18:48 | |
| *** pbelanyi has quit IRC | 18:48 | |
| *** mtreinish has quit IRC | 18:48 | |
| *** slagle has quit IRC | 18:48 | |
| *** cyli has joined #openstack-infra | 18:48 | |
| *** slagle has joined #openstack-infra | 18:48 | |
| *** dansmith has joined #openstack-infra | 18:48 | |
| *** dangers has joined #openstack-infra | 18:48 | |
| *** NobodyCam has joined #openstack-infra | 18:48 | |
| *** soren has joined #openstack-infra | 18:48 | |
| *** jeblair has joined #openstack-infra | 18:48 | |
| *** flaper87|afk has joined #openstack-infra | 18:48 | |
| *** lifeless has joined #openstack-infra | 18:48 | |
| openstackgerrit | A change was merged to openstack-infra/config: Move neutron default test node to trusty. https://review.openstack.org/102663 | 18:48 |
| *** NithyaG_afk has joined #openstack-infra | 18:48 | |
| *** flaper87|afk is now known as flaper87 | 18:48 | |
| *** pbelanyi has joined #openstack-infra | 18:49 | |
| *** david-lyle has quit IRC | 18:49 | |
| *** dangers has quit IRC | 18:49 | |
| *** dangers has joined #openstack-infra | 18:49 | |
| *** flaper87 has quit IRC | 18:49 | |
| *** flaper87 has joined #openstack-infra | 18:49 | |
| *** NithyaG_afk has quit IRC | 18:49 | |
| *** NithyaG_afk has joined #openstack-infra | 18:49 | |
| *** pbelanyi has quit IRC | 18:49 | |
| *** pbelanyi has joined #openstack-infra | 18:49 | |
| *** signed8bit has joined #openstack-infra | 18:49 | |
| *** nati_ueno has quit IRC | 18:49 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add missing 'solum-specs' section in zuul layout https://review.openstack.org/102246 | 18:49 |
| *** crc32 has joined #openstack-infra | 18:50 | |
| *** mtreinish has joined #openstack-infra | 18:50 | |
| *** david-lyle has joined #openstack-infra | 18:50 | |
| *** openstackgerrit has quit IRC | 18:51 | |
| *** nelsnelson has joined #openstack-infra | 18:52 | |
| *** openstackgerrit has joined #openstack-infra | 18:52 | |
| *** nati_ueno has joined #openstack-infra | 18:53 | |
| *** sarob has quit IRC | 18:53 | |
| *** _nadya_ has joined #openstack-infra | 18:53 | |
| notmyname | the rules about what patches are reported in IRC by the gerritbot are in openstack-infra-config right? | 18:53 |
| *** devkulkarni has quit IRC | 18:55 | |
| *** jaypipes_ has joined #openstack-infra | 18:55 | |
| notmyname | ie I don't think swift-specs merge commits are reported in #openstack-swift and I want to fix that | 18:55 |
| fungi | notmyname: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/gerritbot/files/gerritbot_channel_config.yaml | 18:56 |
| jeblair | notmyname: and http://ci.openstack.org/irc.html is the doc that has that link :) | 18:56 |
| *** cnesa has quit IRC | 18:56 | |
| *** jaypipes_ has quit IRC | 18:56 | |
| notmyname | fungi: jeblair: both links are great :-) | 18:56 |
| notmyname | thanks | 18:56 |
| *** james_li has quit IRC | 18:56 | |
| *** dkehnx has quit IRC | 18:56 | |
| *** jaypipes has quit IRC | 18:56 | |
| *** UtahDave has quit IRC | 18:56 | |
| *** dkranz has quit IRC | 18:56 | |
| *** rlandy has quit IRC | 18:56 | |
| *** pdmars has quit IRC | 18:56 | |
| *** camunoz has quit IRC | 18:56 | |
| *** jpeeler has quit IRC | 18:56 | |
| *** davidlenwell has quit IRC | 18:56 | |
| *** grantbow has quit IRC | 18:56 | |
| *** annegentle has quit IRC | 18:56 | |
| *** jamielennox|away has quit IRC | 18:56 | |
| *** ianw has quit IRC | 18:56 | |
| *** ekarlso has quit IRC | 18:56 | |
| *** bcrochet has quit IRC | 18:56 | |
| *** rakhmerov has quit IRC | 18:56 | |
| *** slamont has quit IRC | 18:56 | |
| *** bradm has quit IRC | 18:56 | |
| fungi | also, line 372 says openstack/swift-specs is reported in | 18:57 |
| clarkb | neutron should be trustied real soon now | 18:57 |
| notmyname | jeblair: still with the months-old bug that -2s aren't reported? | 18:57 |
| notmyname | fungi: yes, I see that. hmm | 18:57 |
| fungi | notmyname: gerritbot has been in and out of connectivity with freenode, so might have just merged while it was out to lunch | 18:58 |
| *** _nadya_ has quit IRC | 18:58 | |
| notmyname | ah ok. yes, this was a patch that just landed moments ago | 18:58 |
| *** sweston_ has joined #openstack-infra | 18:58 | |
| * fungi just restarted gerritbot moments ago, so entirely possible | 18:58 | |
| notmyname | well, 20 moments ago https://review.openstack.org/#/c/102417/ | 18:59 |
| *** dkranz has joined #openstack-infra | 18:59 | |
| fungi | it left at 18:20 utc, i got it to come back at 18:46, then it bounced again for a config update from 18:51 to 18:52 | 18:59 |
| *** camunoz has joined #openstack-infra | 18:59 | |
| *** dkehnx has joined #openstack-infra | 18:59 | |
| *** ianw has joined #openstack-infra | 18:59 | |
| clarkb | sdague: ok poking at es now | 18:59 |
| jeblair | notmyname: fixing that bug is not on my personal todo list | 19:00 |
| fungi | notmyname: sounds like that one did indeed merge during the dead time | 19:00 |
| clarkb | sdague: I notice that there is a relatively large number of workers that have been working on the same file for up to half an hour or more | 19:00 |
| clarkb | sdague: so your os loganalyze fix may help quite a bit | 19:00 |
| clarkb | sdague: running sar now | 19:00 |
| *** davidlenwell has joined #openstack-infra | 19:00 | |
| notmyname | jeblair: no worries. obviously not on my either, but it's something I notice every time I go much with that file | 19:00 |
| notmyname | jeblair: IMO it's a nice to have but not blocking anything today | 19:00 |
| jeblair | notmyname: same here | 19:00 |
| sdague | clarkb: ok, weill it's going to be a minute, as I realized the complexity of everything in one file was making my brain hurt building up the context every time | 19:00 |
| openstackgerrit | A change was merged to openstack-infra/config: Gate pecan on barbican again. https://review.openstack.org/102706 | 19:01 |
| clarkb | sdague: ya its fine | 19:01 |
| sdague | so I'm refactoring it out a bit to hopefully make the filtering piece easier | 19:01 |
| clarkb | sounds good | 19:01 |
| *** doude has joined #openstack-infra | 19:02 | |
| *** doude has quit IRC | 19:03 | |
| openstackgerrit | A change was merged to openstack-infra/config: Fix settings.php permission ownership https://review.openstack.org/102826 | 19:03 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix groups-dev portal update sync issue https://review.openstack.org/102858 | 19:04 |
| openstackgerrit | A change was merged to openstack-infra/config: Adding Tempest tests to Blazar Gate Job https://review.openstack.org/102303 | 19:04 |
| clarkb | sdague: elasticsearch04 has some non trivial await according to `sar -dp 5 5` and 07 has higher than everything but 04 | 19:04 |
| clarkb | sdague: but it doesn't look catastrophic like before | 19:04 |
| clarkb | sdague: this may partially explain why we are mostly keeping up but not eating through backlog | 19:04 |
| sdague | gotcha | 19:04 |
| *** lcheng has quit IRC | 19:05 | |
| sdague | is it possible to collect the files that are getting chunked on for long periods of time? | 19:05 |
| sdague | just to understand how off the rails they've gone | 19:05 |
| clarkb | sdague: ya I think I can probably script that up | 19:05 |
| *** cindyo has quit IRC | 19:05 | |
| *** lcheng has joined #openstack-infra | 19:06 | |
| *** rwsu has quit IRC | 19:06 | |
| *** ociuhandu has quit IRC | 19:06 | |
| *** cindyo has joined #openstack-infra | 19:07 | |
| openstackgerrit | A change was merged to openstack-infra/config: Remove dep on wget package for apt::key https://review.openstack.org/100333 | 19:07 |
| *** adrian_otto has joined #openstack-infra | 19:08 | |
| *** cindyo_ has joined #openstack-infra | 19:08 | |
| *** cindyo_ has quit IRC | 19:08 | |
| *** jaypipes has joined #openstack-infra | 19:08 | |
| *** cindyo_ has joined #openstack-infra | 19:09 | |
| *** cindyo_ has quit IRC | 19:10 | |
| *** cindyo_ has joined #openstack-infra | 19:10 | |
| *** hashar has joined #openstack-infra | 19:10 | |
| *** dkliban is now known as dkliban_bbiab | 19:11 | |
| *** cindyo__ has joined #openstack-infra | 19:11 | |
| *** cindyo__ has quit IRC | 19:11 | |
| *** cindyo has quit IRC | 19:11 | |
| *** cindyo has joined #openstack-infra | 19:12 | |
| *** rfolco has joined #openstack-infra | 19:13 | |
| *** yolanda has quit IRC | 19:13 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Handle task manager shutdown more correctly https://review.openstack.org/100589 | 19:14 |
| openstackgerrit | A change was merged to openstack-infra/infra-manual: Initial commit https://review.openstack.org/92475 | 19:14 |
| openstackgerrit | pritesh proposed a change to openstack-dev/hacking: Add hacking check for vim configuration in source files https://review.openstack.org/101969 | 19:14 |
| anteaya | jeblair: thanks | 19:15 |
| anteaya | whew | 19:15 |
| *** cindyo_ has quit IRC | 19:15 | |
| openstackgerrit | A change was merged to openstack-infra/config: Disable Designate DevStack gate on stable/havana+icehouse https://review.openstack.org/102864 | 19:16 |
| jeblair | jogo: ping | 19:16 |
| clarkb | sdague: looking at top the issue appears to be between the logstash java process and elasticsearch | 19:16 |
| clarkb | sdague: either we cannot parse logs quickly enough because they are many and numerous or elasticsearch can't index them quickly enough for tha reason or disk | 19:17 |
| *** rwsu has joined #openstack-infra | 19:17 | |
| sdague | clarkb: ok | 19:17 |
| clarkb | sdague: when things plug up it appears that the gearman workers and crm114 are idle | 19:17 |
| clarkb | and have handed the work off | 19:17 |
| sdague | ok, that's at least something | 19:17 |
| fungi | so is our log volume maybe becoming larger, or harder to parse in some way? | 19:18 |
| *** marcoemorais has quit IRC | 19:18 | |
| jeblair | jogo: so jhesketh started this really complicated zuul patch to do something you asked him to do... | 19:18 |
| clarkb | fungi: could be both | 19:18 |
| *** signed8bit has quit IRC | 19:18 | |
| sdague | fungi: it's definitely on a growth path, especially as we add more services | 19:18 |
| jeblair | jogo: saying you had talked to me and i said i agreed... | 19:18 |
| *** marcoemorais has joined #openstack-infra | 19:18 | |
| *** marcoemorais has quit IRC | 19:18 | |
| clarkb | sdague: fungi: that and we seem to have a bad habit of douling our log totals every couple of months | 19:19 |
| *** james_li has joined #openstack-infra | 19:19 | |
| *** UtahDave has joined #openstack-infra | 19:19 | |
| *** rlandy has joined #openstack-infra | 19:19 | |
| *** pdmars has joined #openstack-infra | 19:19 | |
| *** jpeeler has joined #openstack-infra | 19:19 | |
| *** rakhmerov has joined #openstack-infra | 19:19 | |
| *** annegentle has joined #openstack-infra | 19:19 | |
| *** jamielennox|away has joined #openstack-infra | 19:19 | |
| *** ekarlso has joined #openstack-infra | 19:19 | |
| *** bcrochet has joined #openstack-infra | 19:19 | |
| *** slamont has joined #openstack-infra | 19:19 | |
| *** bradm has joined #openstack-infra | 19:19 | |
| *** grantbow has joined #openstack-infra | 19:19 | |
| *** grantbow has quit IRC | 19:19 | |
| *** grantbow has joined #openstack-infra | 19:19 | |
| clarkb | which isn't sustainable | 19:19 |
| *** signed8bit has joined #openstack-infra | 19:19 | |
| *** marcoemorais has joined #openstack-infra | 19:19 | |
| jeblair | jogo: you're going to have to remind me when i agreed to not rechecking based on -1 reviews from a human was okay | 19:19 |
| sdague | clarkb: well we do this pesky thing of adding all these new services all the time :) | 19:19 |
| jeblair | jogo: because the only thing i can find about that is that you asked, and i said i had to leave | 19:19 |
| *** pballand has joined #openstack-infra | 19:19 | |
| sdague | and tests for them | 19:19 |
| clarkb | yeah addition of tests exercising new code paths may be the biggest factor | 19:19 |
| sdague | someone made a good point that perhaps we don't need to save good logs for as long as bad logs | 19:20 |
| *** dtantsur is now known as dtantsur|afk | 19:20 | |
| jeblair | jogo: but i'm trying to figure out why -- the idea of the freshness check isn't actually tied to how good we think the change is | 19:20 |
| *** sabeen1 has joined #openstack-infra | 19:20 | |
| *** sabeen has quit IRC | 19:20 | |
| sdague | which doesn't help the ES case | 19:20 |
| sdague | but might on the large case | 19:20 |
| *** crc32 has quit IRC | 19:20 | |
| jeblair | jogo: the fact that it triggers off of comments is just an implementation detail -- the point is to make sure that the current verified value is up to date | 19:20 |
| jeblair | jogo: i'm pretty sure that if we stopped running it if there is a -1 comment, then we will lose a lot of freshness check runs | 19:21 |
| *** lcheng has quit IRC | 19:21 | |
| fungi | sdague: granted, but it may be hard to identify those when stored at rest... though maybe we can use elasticsearch metadata to determine when to delete success logs from the logserver if we go that route | 19:21 |
| sdague | jeblair: there is a whacky edge case though | 19:21 |
| jeblair | jogo: and i think sdague may have something to say about that | 19:21 |
| *** maxbit has joined #openstack-infra | 19:21 | |
| sdague | where a patch is +A | 19:21 |
| sdague | fails jenkins | 19:21 |
| sdague | in gate | 19:21 |
| clarkb | sdague: the issue with that is it doesnt' change the upfront cost of indexing and parsing it all | 19:21 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/infra-manual: Create a .gitignore file https://review.openstack.org/102919 | 19:21 |
| clarkb | sdague: which is why I still think we should go to json | 19:21 |
| clarkb | which I hope to dig in at some point before the end of juno | 19:21 |
| sdague | you -1 it to provide feedback about why it failed | 19:21 |
| sdague | and it goes back to the gate | 19:22 |
| sdague | which happened yesterday | 19:22 |
| *** lcheng has joined #openstack-infra | 19:22 | |
| jeblair | sdague: that's a pretty wacky edge case. we can look at trying to solve that, but that was not jogo's suggestion. | 19:22 |
| clarkb | looking at bigdesk for the logstash workers java processes they all seem to be using all the memories | 19:22 |
| clarkb | I wonder if memory pressure is contributing to the problem | 19:23 |
| clarkb | any opposition to me rsetarting those java processes to see if that temproarily makes things happy? | 19:23 |
| sdague | jeblair: so I'd actually be ok with not rerunning on -1/-2 ... if zuul did a merge conflict check on a patch land | 19:23 |
| jeblair | clarkb: nope | 19:23 |
| sdague | which I think was discussed at some point | 19:23 |
| fungi | clarkb: i think it's worth a try at least as an experiment | 19:23 |
| mriedem | i'm seeing some consistent unit test failures in glance but i can only find them in the 'post' queue in logstash, not check or gate - not sure why that is? | 19:23 |
| *** harlowja is now known as harlowja_away | 19:24 | |
| sdague | basically if a nova patch lands, merge conflict test all the inflight nova changes | 19:24 |
| jeblair | sdague: i feel like the goals of the freshness check are changing. perhaps you should restate them. | 19:24 |
| jeblair | sdague: because it used to be "we always want current information, and we want _lots_ of elasticsearch data" | 19:24 |
| sdague | jeblair: well because we don't have the +A bit in the ES, we actually don't have lots of ES data that's useful | 19:25 |
| jeblair | sdague: merge conflicts don't slow the gate queue at all | 19:25 |
| jeblair | sdague: i feel like we've done a lot of work to support the freshness check, and we're now carving out exceptions to 95% of that, and i'm starting to wonder if it wouldn't just be easier to turn it off? | 19:26 |
| jeblair | sdague: or at least focus on what we want it to actually do, rather than layering side-effect on top of side-effect | 19:26 |
| sdague | jeblair: nope, but a dev not knowing that they have a merge conflict in the patch massively impacts the dev cycle | 19:27 |
| clarkb | gerrit automatically tells you now | 19:27 |
| clarkb | or should | 19:27 |
| clarkb | zaro: ^ | 19:27 |
| clarkb | is that only with change screen 2? | 19:27 |
| jogo | jeblair: pong | 19:27 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: remove envinject usage from jenkins-plugin-jobs.yaml https://review.openstack.org/102893 | 19:27 |
| sdague | jeblair: I think there are less carve outs then you think, but honestly, it's probably better left to the midcycle. | 19:28 |
| clarkb | ok initial indications are that restarting those services made a massive impact in indexes per second | 19:28 |
| clarkb | lets see if it keeps up for a reasonable amount of time | 19:28 |
| fungi | clarkb: sounds like we have leaks in the hull | 19:28 |
| jogo | jeblair: so I thought you signed off on it but, looks like I was wrong | 19:29 |
| *** achanda has quit IRC | 19:29 | |
| clarkb | we are doing about 50k request per second | 19:29 |
| anteaya | woooo http://docs.openstack.org/infra/manual/ | 19:29 |
| *** sabeen1 has quit IRC | 19:29 | |
| jogo | jeblair: so the idea was: we have two freshness checks. a 72 hour one on any comment. and a 24 one on a +W. | 19:29 |
| clarkb | which is well above the ~15 prior to restart | 19:29 |
| clarkb | fungi: quick plug them with your fingers | 19:29 |
| jogo | the idea was leave the +W/24 hour one as is. and that addresses the freshness check for the gate case | 19:30 |
| clarkb | fungi: if you have bigdesk open the spike is kind of cool | 19:30 |
| *** cnesa has joined #openstack-infra | 19:30 | |
| *** achanda has joined #openstack-infra | 19:30 | |
| *** sabeen1 has joined #openstack-infra | 19:30 | |
| *** crc32 has joined #openstack-infra | 19:31 | |
| clarkb | let us see how long this lasts | 19:31 |
| *** sabeen1 has quit IRC | 19:31 | |
| *** sabeen3 has joined #openstack-infra | 19:31 | |
| jeblair | jogo, sdague: yeah, but if the point of this is now to tell people about merge conflicts (?) then why wouldn't you want to know about a merge conflict even if someone -1s your change for some minor style thing that no one else agrees with | 19:31 |
| clarkb | but if this helps it would lend massive props to my goal of not spending all these resources on this step | 19:31 |
| jeblair | jogo, sdague: so i think we need to start over and decide what our goals are | 19:32 |
| *** sabeen3 has quit IRC | 19:32 | |
| *** sabeen1 has joined #openstack-infra | 19:32 | |
| jogo | jeblair: starting over sounds good | 19:32 |
| jeblair | jogo, sdague: because the initial goals of the freshness check had no ties to votes at all, it was strictly time-based, and the implementation that used comment triggers was merely convenience. | 19:32 |
| jogo | jeblair: and what was the goal of the 72 hour freshness check? | 19:33 |
| jogo | the 24 before gate, was to make sure we don't get wrecking balls in the gate queue | 19:33 |
| jeblair | jogo: same thing -- the time split was just because we realized we couldn't sustain 24h everywhere | 19:33 |
| clarkb | https://jenkins07.openstack.org/job/gate-neutron-python27/1919/ https://jenkins05.openstack.org/job/gate-neutron-docs/2137/ and https://jenkins05.openstack.org/job/gate-neutron-pep8/2192/ are all neutron on trusty | 19:34 |
| jeblair | jogo: it was basically, try to keep the verified value as current as possible under the constraints that we don't actually have unlimited test resources | 19:34 |
| jogo | so I don't see how the freshenss check on comments prevents wrecking balls | 19:34 |
| jeblair | jogo: it's not about the comments :) | 19:34 |
| jogo | that one seems to be much more for humans to see. | 19:35 |
| fungi | also, not just notifying devs and reviewers that a patch had slumped into a merge conflict, but also logical/testing conflicts (like older proposed change now would break newly merged feature/test) | 19:35 |
| jogo | jeblair: right, just calling it that to differentiate, what should i call it? | 19:35 |
| jeblair | jogo: i think you're talking about the 72 hour check as opposed to the 24h check? | 19:35 |
| jogo | yes | 19:35 |
| *** adrian_otto has quit IRC | 19:35 | |
| jeblair | jogo: so 72 hours gets us somewhat up to date information that may keep someone from aprving a patch that fails | 19:36 |
| jogo | but the 24 hour check means that cannot get into the gate queue if there is an issue | 19:36 |
| *** mrmartin has joined #openstack-infra | 19:36 | |
| jogo | even without the 72 hour check at all | 19:36 |
| jeblair | jogo: right, it narrows the window considerably | 19:36 |
| *** jp_at_hp has quit IRC | 19:37 | |
| fungi | and also prevents reviewers, our most valuable/limited resource, from reviewing a change which needs fixing | 19:37 |
| jogo | fungi: right | 19:37 |
| jeblair | jogo: the ideal is that the verified value is always correct in real time. we don't have enough clouds for that. so we re-test as often as we can. | 19:37 |
| jogo | thats what I see as the primary value in the 72 hour check | 19:37 |
| anteaya | when signing into gerrit for the first time, does gerrit currently prepopulate username from launchpad or leave gerrit username blank? | 19:37 |
| *** _nadya_ has joined #openstack-infra | 19:37 | |
| jeblair | anteaya: i do not know, that's a good question | 19:38 |
| anteaya | okay | 19:38 |
| *** david-lyle has quit IRC | 19:38 | |
| clarkb | so indexing didn't remain peaked for long but I think that may be beacuse the python side queues up | 19:38 |
| anteaya | reading the infra-manual I need to clarify | 19:38 |
| clarkb | and had a bunch of data to drain | 19:38 |
| jogo | jeblair: so the motivation for not running a freshness check on a -1 vote. is that in the vast majority of cases a -1 vote means the patch has to go through another revision and that reviewers won't look at it while there is a -1 anyway | 19:38 |
| clarkb | looking at how old files in process are none are more than a minute old | 19:38 |
| clarkb | which is way better than before | 19:39 |
| anteaya | since having different usernames between launchpad and gerrit confuses stackalytics | 19:39 |
| anteaya | as a for instance | 19:39 |
| jogo | and there are exceptions to this of course | 19:39 |
| *** annegent_ has joined #openstack-infra | 19:39 | |
| anteaya | if I create a fake launchpad and gerrit account to check can I delete? | 19:39 |
| jogo | fungi jeblair: and given we hit quota limits fairly often, this seemed like an easy way to drop some load with minimal side effects | 19:39 |
| anteaya | or I could check on review-dev I guess | 19:39 |
| anteaya | thoughts on how I should test this? | 19:40 |
| clarkb | restarting the daemons has also shifted cpu time up the stack which is a good thing | 19:40 |
| clarkb | so ya this may be a big part of the problem :/ | 19:40 |
| jeblair | jogo: it's far from easy :). i would have said that the side effects are that it invalidates part of why we're doing the freshness checks in the first place. however, apparently, i don't know those reasons anymore, so i can't actually say. | 19:40 |
| fungi | jogo: it seems from my observations that volume in the check pipeline has less impact that a high rate of resets in the gate pipeline | 19:40 |
| *** achanda has quit IRC | 19:41 | |
| fungi | s/less impact that/less impact than/ | 19:41 |
| clarkb | fungi: I would agree with that. Mostly because they aer on and done | 19:41 |
| jeblair | jogo: i do agree that if we consider what you say to be the primary purpose of the 72h check, the proposal is in line with that. | 19:41 |
| clarkb | whereas gate resets can happen quickly and often cycling through hundreds of nodes in a short period | 19:41 |
| anteaya | zaro: if I create a fake launchpad account can I use it to sign into review-dev? | 19:41 |
| fungi | anteaya: yep | 19:42 |
| jogo | fungi: so I don't think this would change the gate pipeline atall | 19:42 |
| anteaya | great, fake account alert | 19:42 |
| jogo | just drop overall usage of our compute resources | 19:42 |
| fungi | anteaya: review-dev is all about fake.test/throwaway things. no need to give anyone a heads up | 19:42 |
| jogo | jeblair: so I can live with that. now we just need to figure out what the 72 hour check is there for | 19:42 |
| jeblair | i _think_ the new logging i just added to zuul would let us count how many times the 72h check is triggered, and on which changes | 19:42 |
| jeblair | so we might be able to quantify the impact | 19:43 |
| jeblair | (we need a zuul restart to pick that up) | 19:43 |
| jogo | jeblair: woot | 19:43 |
| jeblair | jogo: yeah.. what sdague just said suggested that the real value is actually in merge conflict detection, not that test results may change | 19:43 |
| *** tkelsey has quit IRC | 19:44 | |
| jeblair | jogo: merge conflict detection is computationally extremely cheap; adding global conflict detection for all open project changes to zuul is probably about as much work as any other change to this system | 19:44 |
| jeblair | so it does sound like identifying our current goals is in order | 19:45 |
| clarkb | so gerrit will identify merge conflicts | 19:46 |
| openstackgerrit | Dolph Mathews proposed a change to openstack-infra/elastic-recheck: add query for bug 1334812 https://review.openstack.org/102927 | 19:46 |
| uvirtbot | Launchpad bug 1334812 in openstack-ci "intersphinx inventory u'http://docs.python.org/objects.inv' not fetchable due to <class 'urllib2.HTTPError'>: HTTP Error 503: Service Unavailable" [Undecided,New] https://launchpad.net/bugs/1334812 | 19:46 |
| clarkb | I do not think it should be added to zuul unless we expect zuul to be reasonable accurate as compared to gerrit | 19:46 |
| *** david-lyle has joined #openstack-infra | 19:47 | |
| *** anode has joined #openstack-infra | 19:47 | |
| anteaya | fungi: okay | 19:47 |
| jeblair | clarkb: i expect it to identify a subset of the conflicts which would stop gerrit (same as it does now) | 19:47 |
| sdague | clarkb: we need -1 V feedback on merge conflicts, or some other substantial vote construct so they single whose court the change is in | 19:47 |
| clarkb | jeblair: right which is better than nothing except gerrit should find all of them | 19:47 |
| clarkb | sdague: iirc it is a state | 19:47 |
| *** rcarrill` has joined #openstack-infra | 19:48 | |
| sdague | is it a queriable state? | 19:48 |
| clarkb | I think so, but would haev to go read up on it | 19:48 |
| *** james_li has quit IRC | 19:48 | |
| jeblair | i'll try to find one | 19:48 |
| *** rcarrillocruz has quit IRC | 19:49 | |
| clarkb | https://gerrit-review.googlesource.com/#/c/58004/ | 19:49 |
| jogo | doing just a merge conflict check as an option could be nice. and a really useful one | 19:49 |
| *** signed8bit is now known as signed8bit_ZZZzz | 19:49 | |
| clarkb | so that is an example of a chane that cannot merge that gerrit has detected. now to figure out if it is queryable | 19:50 |
| clarkb | zaro: ^ you don't happen to just know do you? | 19:50 |
| jeblair | https://review.openstack.org/#/c/89694/ | 19:50 |
| sdague | clarkb: ok, is:mergeable is a thing | 19:50 |
| jeblair | there's one in our gerrit | 19:50 |
| anteaya | on the first page in gerrit you are prompted to "Select a unique username:" | 19:50 |
| anteaya | so yeah you can create a name that doesn't match your launchpad username | 19:50 |
| sdague | is the developer sent a notification when their change goes NOT is:mergeable ? | 19:50 |
| clarkb | https://gerrit-review.googlesource.com/#/q/-is:mergeable | 19:51 |
| anteaya | so stackaltics will need a patch to track that person, if they choose to have different launchpad and gerrit usernames | 19:51 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Added a puppet module for livegrep. https://review.openstack.org/96877 | 19:51 |
| jeblair | i don't see a 'cannot merge' flag on the new change screen in our gerrit | 19:52 |
| jogo | jeblair: I guess we can idtenfy our current goals in Germany | 19:53 |
| fungi | seems like something which could be overlaid with a rest api query and some javascript, though that would be a bit ugly | 19:53 |
| *** yolanda has joined #openstack-infra | 19:53 | |
| clarkb | searching for is:mergeable on our gerrit is an error | 19:54 |
| jeblair | jogo, sdague: we have a lot going on in germany. i'm pretty sure we can handle this over irc or email, so i'm not sure i want to put it on the schedule explicitly, but we can always beer it. | 19:54 |
| clarkb | so maybe we need 2.9? | 19:54 |
| *** david-lyle has quit IRC | 19:54 | |
| sdague | jeblair: yeh, well the competing challenge is there is a lot going on before germany. So I'm not sure that I can focus on this discussion in a substantive way prior to that. | 19:57 |
| jogo | jeblair sdague: beer or irc works for me. | 19:57 |
| *** markwash has joined #openstack-infra | 19:57 | |
| *** david-lyle has joined #openstack-infra | 19:58 | |
| jeblair | sdague: sure, i just don't want "let's talk about it in germany" to be the answer to everything, especially a problem that's pretty tractable online. plus, if we knew _goals_ before then, we might be able to hack on it, or assist there. | 19:58 |
| jeblair | sdague: if you and jogo aren't in a rush, neither am i. :) | 19:59 |
| jogo | jeblair sdague: sure. so if we just needto sort out what the goal is, that shouldn't e too hard | 19:59 |
| jeblair | sdague: so maybe whenever you find time, you can elucidate what you think the goals are, and we'll pick it up then. | 19:59 |
| jogo | jeblair: well we seem to hit quota at least once a week as is. and with your new logging we can get a sense of what % of the load this is creating | 20:00 |
| jogo | jeblair sdague: sounds good to me | 20:00 |
| sdague | jeblair: sure | 20:00 |
| clarkb | I am still waiting on a full python27 neutron run on trusty (the last one was cancelled due to gate restart) | 20:01 |
| clarkb | I think we can probably move forward with the whole stack, I will be around to babysit | 20:01 |
| jeblair | i'm going to feed now, biab. | 20:01 |
| *** vhoward has left #openstack-infra | 20:02 | |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Added a puppet module for livegrep. https://review.openstack.org/96877 | 20:02 |
| hogepodge | ping anode nibalizer and anyone else talking about beaker | 20:03 |
| nibalizer | ohai | 20:03 |
| fungi | ugh... gerrit ssh api queries for 20558 and 23447 are returning no results, but i can find them in the webui | 20:03 |
| nibalizer | jeblair: so idunno if you know anode and hogepodge but they're both puppetlabs people, anode wrote beaker and hoge is the point person for the stackforge/puppet* modules | 20:03 |
| nibalizer | if/when you're ready to continue that conversation | 20:04 |
| clarkb | fungi: assuming not drafts because you can get them in webui | 20:04 |
| fungi | clarkb: right | 20:04 |
| *** arnaud has joined #openstack-infra | 20:04 | |
| fungi | clarkb: one of them is a very trivial change of mine, in fact, which i would never have originally set to draft anyway | 20:04 |
| hogepodge | nibalizer I think you’re thinking of a different project. (ohai) ;-) | 20:04 |
| nibalizer | hogepodge: oh? | 20:05 |
| * fungi is pretty sure we know hogepodge at the very least ;) | 20:05 | |
| nibalizer | hogepodge: are you not point on the puppet openstack modules? | 20:05 |
| * nibalizer confused about your job then lol | 20:05 | |
| *** sweston_ has quit IRC | 20:06 | |
| hogepodge | I am. I kid about ohai (a chef project) | 20:06 |
| nibalizer | oh right | 20:06 |
| nibalizer | ya i know nothing about chef :( | 20:06 |
| fungi | it has knives | 20:06 |
| fungi | also forks and spoons i assume | 20:06 |
| fungi | anyway, you caught jeblair's lunch break, but if you stick around he'll be back, i'm sure | 20:08 |
| hogepodge | cool. I hit a half day meeting block in about an hour | 20:09 |
| openstackgerrit | Dolph Mathews proposed a change to openstack-infra/elastic-recheck: add query for bug 1316926 https://review.openstack.org/102932 | 20:09 |
| uvirtbot | Launchpad bug 1316926 in tempest "failed to reach ACTIVE status and task state "None" within the required time (196 s). Current status: BUILD. Current task state: spawning." [Undecided,New] https://launchpad.net/bugs/1316926 | 20:09 |
| *** anode has quit IRC | 20:10 | |
| fungi | the tl;dr if memory serves is that beaker does a lot of things we already manage infrastructure to handle (resource pool management, worker selection, et cetera) | 20:10 |
| *** achanda has joined #openstack-infra | 20:10 | |
| clarkb | right | 20:10 |
| fungi | so there is some hesitance to managing a second and entirely different implementation of the same sorts of things in parallel with that | 20:10 |
| clarkb | jeblair and mordred are wary of adding more management on top of that to do beaker spec testing | 20:10 |
| *** zehicle_at_dell has quit IRC | 20:11 | |
| fungi | if it can be stripped down to just a puppet test specification language and runner, and hand off resource management to our existing solutions, that might be easier to deal with | 20:12 |
| *** Longgeek has quit IRC | 20:12 | |
| clarkb | I personally don't see much trouble with throwing docker at it. THe added benefit there is someone running it at home can do the exact same thing as the gate | 20:12 |
| clarkb | much harder to do that if yo urely on nodepool for everything | 20:12 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/infra-manual: Updates Gerrit Username and Elastic-Recheck information https://review.openstack.org/102933 | 20:13 |
| fungi | so how would that work in theory? have jenkins fire up beaker on a nodepool node and then beaker would spin up docker containers and run stuff and report results back to the controlling parent process? | 20:14 |
| clarkb | fungi: yes | 20:14 |
| clarkb | no different than say tempest | 20:14 |
| clarkb | fungi: that is my understanding of how beaker works in the default case | 20:15 |
| clarkb | it just talks to containers or VMs it spins up locally and reports the aggregate result back | 20:15 |
| clarkb | (or fails fast if you want) | 20:15 |
| fungi | got it. so not bolting onto our pools of separate virtual machines, just runnign entirely within a single vm | 20:15 |
| clarkb | correct | 20:15 |
| *** maxbit_ has joined #openstack-infra | 20:15 | |
| *** yfried has quit IRC | 20:15 | |
| clarkb | which would in theory make it simple for me to do on my laptop too | 20:16 |
| clarkb | or in a VM of my own | 20:16 |
| openstackgerrit | Dolph Mathews proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326708 https://review.openstack.org/102934 | 20:16 |
| uvirtbot | Launchpad bug 1326708 in tempest "failed to reach ACTIVE status in test_resize_server_revert; Current status: VERIFY_RESIZE" [Undecided,New] https://launchpad.net/bugs/1326708 | 20:16 |
| fungi | and yeah, i was pondering it more from the perspective of testing openstack-infra/config repo changes, but taken in the context of openstack puppet modules this becomes very much like devstack setting up openstack services and then tempest exercising them | 20:17 |
| clarkb | python27 neutron unittests succeeded in the gate | 20:18 |
| clarkb | I am ready to open the floodgates on bare-trusty testing now | 20:18 |
| fungi | yeah, i've been watching. looking promising | 20:18 |
| *** maxbit has quit IRC | 20:19 | |
| openstackgerrit | Dolph Mathews proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326708 https://review.openstack.org/102934 | 20:19 |
| uvirtbot | Launchpad bug 1326708 in tempest "failed to reach ACTIVE status in test_resize_server_revert; Current status: VERIFY_RESIZE" [Undecided,Incomplete] https://launchpad.net/bugs/1326708 | 20:19 |
| *** _nadya_ has quit IRC | 20:20 | |
| *** david-lyle has quit IRC | 20:20 | |
| *** anode has joined #openstack-infra | 20:20 | |
| openstackgerrit | Dolph Mathews proposed a change to openstack-infra/elastic-recheck: add query for bug 1316926 https://review.openstack.org/102932 | 20:20 |
| uvirtbot | Launchpad bug 1316926 in tempest "failed to reach ACTIVE status and task state "None" within the required time (196 s). Current status: BUILD. Current task state: spawning." [Undecided,New] https://launchpad.net/bugs/1316926 | 20:20 |
| *** david-lyle has joined #openstack-infra | 20:21 | |
| *** harlowja_away is now known as harlowja | 20:23 | |
| *** andreaf_ has joined #openstack-infra | 20:23 | |
| *** andreaf_ has quit IRC | 20:23 | |
| ryanpetrello | mordred, clarkb: I’m having *much* better luck w/ pecan gate jobs in the last day | 20:24 |
| ryanpetrello | must’ve just been had some particularly bad luck the other night | 20:25 |
| *** david-lyle has quit IRC | 20:25 | |
| clarkb | pypi was mad at you | 20:25 |
| ryanpetrello | clearly | 20:25 |
| fungi | ryanpetrello: apparently pypi was suffering at the hands of angry glusterfs gods that night | 20:26 |
| fungi | so sayeth its admins | 20:26 |
| jgriffith | jeblair: thanks for the response | 20:26 |
| ryanpetrello | fungi: I see | 20:26 |
| jgriffith | jeblair: ie the battle of names | 20:26 |
| jgriffith | so there may be some miscommunication but anywho... my main thing was not having 12 seperate CI systems for HP or others etc | 20:27 |
| *** sarob has joined #openstack-infra | 20:27 | |
| *** denis_makogon has joined #openstack-infra | 20:28 | |
| openstackgerrit | Dolph Mathews proposed a change to openstack-infra/elastic-recheck: add query for bug 1276778 https://review.openstack.org/71357 | 20:29 |
| uvirtbot | Launchpad bug 1276778 in tempest "test_lock_unlock_server: failed to reach ACTIVE status and task state "None" within the required time" [Undecided,New] https://launchpad.net/bugs/1276778 | 20:29 |
| *** mrmartin has quit IRC | 20:29 | |
| zaro | clarkb: zuul triggers build even when only commit message has changed? don't we want to stop that? | 20:29 |
| *** vhoward has joined #openstack-infra | 20:30 | |
| *** andreykurilin has joined #openstack-infra | 20:30 | |
| *** dprince has quit IRC | 20:30 | |
| clarkb | it does and no | 20:30 |
| fungi | zaro: a hacking check causes jenkins to -1 my change because it dislikes the cut of my commit message's jib, so i fix the commit message... and then leave a recheck comment? | 20:31 |
| clarkb | we cant stop that because our tests care about commit messages | 20:31 |
| zaro | ohh i didn't realize we have tests in place for commit messages. | 20:32 |
| sdague | clarkb: I think the question there is we have a 1s duration check for commit messages | 20:32 |
| zaro | i don't believe i've ever had a failed commit msg check. | 20:32 |
| sdague | and instead we run 6 1hr tests on it | 20:32 |
| *** rkukura has quit IRC | 20:33 | |
| sdague | it does seem like we might be able to make that better | 20:33 |
| hogepodge | fungi we can run in no-provider mode. Just give fqdn to connect to. | 20:33 |
| clarkb | sdague well would need new zuul features to do better | 20:33 |
| sdague | clarkb: sure | 20:33 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/infra-manual: Updates Gerrit Username and Elastic-Recheck information https://review.openstack.org/102933 | 20:33 |
| fungi | hogepodge: can the fqdn be localhost? | 20:33 |
| clarkb | first zuul would have to check every event by diffing commit messages | 20:33 |
| hogepodge | fungi I would think so | 20:33 |
| *** devkulkarni has joined #openstack-infra | 20:34 | |
| clarkb | then you would need a way to tag jobs that only run on subevents | 20:34 |
| fungi | hogepodge: as in install beaker in a single-use nodepool vm, then have the jenkins agent start it and tell it to do whatever it wants within that vm? | 20:34 |
| hogepodge | I think it would be better to isolate beaker, though, because of resource pollution. Say with docker | 20:34 |
| *** marcoemorais has quit IRC | 20:34 | |
| clarkb | we sort of have the second thing for files affected | 20:34 |
| hogepodge | fungi yeah, that would be the idea | 20:34 |
| sdague | oh, do we? | 20:34 |
| clarkb | so mostly the first thing is missing | 20:34 |
| *** marcoemorais has joined #openstack-infra | 20:34 | |
| *** Ryan_Lane has joined #openstack-infra | 20:34 | |
| zaro | clarkb: i believe gerrit alrady diffs commit messages and tells you that only the commit msg has changed. | 20:34 |
| zaro | example: https://review.openstack.org/#/c/102893/ | 20:34 |
| clarkb | sdague: yup requirements tests use it to only run those jobs on reqs changes | 20:35 |
| *** marcoemorais has quit IRC | 20:35 | |
| clarkb | zaro what does the event look like? | 20:35 |
| *** marcoemorais has joined #openstack-infra | 20:35 | |
| sdague | clarkb: oh... hey, can we start doing that for docs changes? | 20:35 |
| *** marcoemorais has quit IRC | 20:35 | |
| zaro | clarkb: haven't looked, can dig one up if you like. | 20:35 |
| clarkb | sdague: possibly | 20:35 |
| *** marcoemorais has joined #openstack-infra | 20:35 | |
| clarkb | zaro the event is what matters not the gui | 20:36 |
| sdague | I would love to only run docs job if only *.rst doc/ were changed | 20:36 |
| clarkb | fungi any chance I can bribe you on the rest of the trusty stack? | 20:36 |
| sdague | that would actually incentivize doc updates more often in tempest for sure | 20:36 |
| fungi | clarkb: looking at them now | 20:36 |
| * zaro digs | 20:37 | |
| clarkb | sdague I think that may require inverted logic | 20:37 |
| clarkb | which would be a small change to zuul | 20:37 |
| sdague | clarkb: oh, as today it only does extra jobs? | 20:37 |
| clarkb | ya | 20:38 |
| fungi | sdague: right, iff you changed a requirements file, then it adds a requirements job | 20:38 |
| sdague | well if we could get that, I'd be super happy :) | 20:38 |
| fungi | so it supports adding more jobs on a pattern match now, but subtracting jobs would still need some work (and designing how we would expect to configure the inverse match) | 20:39 |
| fungi | only running docs jobs when *.rst doc/ are changed is doable today. running only docs jobs when nothing besides *.rst doc/ are changed is not, afaik | 20:41 |
| clarkb | and we dont want the first thing because code affects docs in most projects | 20:42 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/infra-manual: Updates Gerrit Username and Elastic-Recheck information https://review.openstack.org/102933 | 20:43 |
| zaro | clarkb: ahh! 'NO_CODE_CHANGE' in the event.. http://paste.openstack.org/show/85018/ | 20:43 |
| *** dkranz has quit IRC | 20:43 | |
| zaro | clarkb: that's from master build of gerrit, double checking with review-dev to make sure we have it. | 20:44 |
| *** banix has joined #openstack-infra | 20:44 | |
| fungi | so while it may be possible to special-case NO_CODE_CHANGE in a patchset-created event, we probably want to consider mashing it together with whatever positive/negative file pattern solution we decide on | 20:44 |
| *** david-lyle has joined #openstack-infra | 20:44 | |
| *** achanda has quit IRC | 20:45 | |
| fungi | e.g. configuration could describe it by marking the job to run only when the files list matches the null set | 20:45 |
| *** signed8bit_ZZZzz is now known as signed8bit | 20:47 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: add py27debug target https://review.openstack.org/102939 | 20:47 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: move generator creation to separate file https://review.openstack.org/102940 | 20:47 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: move filtering mechanism to separate class https://review.openstack.org/102941 | 20:47 |
| *** radez is now known as radez_g0n3 | 20:47 | |
| *** dstanek is now known as dstanek_zzz | 20:47 | |
| zaro | hmm, looks like review-dev is running older version of gerrit than review | 20:47 |
| sdague | clarkb: so that doesn't yet give the limit feature | 20:47 |
| sdague | however it does slice it up so that limit can be the next patch in that stack | 20:48 |
| clarkb | sdague: cool, I think the limit feature may actually be useful for really large files | 20:49 |
| clarkb | eg >200k lines | 20:49 |
| sdague | clarkb: agreed | 20:49 |
| openstackgerrit | Graham Hayes proposed a change to openstack-infra/config: Move Designate from stackforge to openstack https://review.openstack.org/102328 | 20:49 |
| clarkb | sdague: because we seem to be cruising now | 20:49 |
| sdague | clarkb: nice | 20:49 |
| clarkb | and the problems are related to memory derps | 20:49 |
| *** sarob has quit IRC | 20:49 | |
| *** james_li has joined #openstack-infra | 20:49 | |
| clarkb | which may be caused by really large files | 20:49 |
| *** dims_ has joined #openstack-infra | 20:49 | |
| sdague | also, it will let me do this other thing, which I've really wanted recently, which is search= | 20:49 |
| sdague | with a regex | 20:49 |
| clarkb | ooh | 20:49 |
| clarkb | zaro: thats not good | 20:50 |
| clarkb | zaro: do we justneed to change the war in the review_dev.pp? | 20:50 |
| clarkb | zaro: we should do that | 20:50 |
| *** achanda has joined #openstack-infra | 20:50 | |
| sdague | look at the refactor, because my brain is a little jelly, to see if I borked things much. I'm going to work on the limit patch now, then the presentation layer pull out | 20:50 |
| *** Alexei_987 has joined #openstack-infra | 20:50 | |
| clarkb | ok | 20:50 |
| sdague | and the bottom of the queue is just a tox.ini change, which should be quick as long as no one is offended by it :) | 20:51 |
| *** maxbit_ has quit IRC | 20:51 | |
| clarkb | yup +2 already | 20:51 |
| *** dims__ has quit IRC | 20:52 | |
| zaro | clarkb: gerrit versions are same in review.pp and review_dev.pp, maybe need to run puppet on review-dev? | 20:53 |
| zaro | fungi, clarkb: no code change event from review-dev http://paste.openstack.org/show/85019/ | 20:54 |
| zaro | looks like insertions=0 and deletions=0 is same as 'NO_CODE_CHANGE' | 20:55 |
| clarkb | zaro: and those values are the delta between patchsets or the delta between branch and proposed patch? | 20:55 |
| clarkb | zaro: because i would expect every change event to say file foo changes +X,-Y | 20:55 |
| openstackgerrit | Matt Riedemann proposed a change to openstack/requirements: Update python-neutronclient>=2.3.5,<3 https://review.openstack.org/102944 | 20:55 |
| *** andreaf_ has joined #openstack-infra | 20:57 | |
| clarkb | zaro: I guess I am saying NO_CODE_CHANGE represents teh delta between patchsets and not the delta between change and destination branch | 20:57 |
| zaro | clarkb: i'm pretty sure it's between patchsets. the change is against master. | 20:57 |
| *** david-lyle has quit IRC | 20:57 | |
| clarkb | ya ok | 20:57 |
| clarkb | so that is someting we could teach zuul about | 20:57 |
| *** david-lyle has joined #openstack-infra | 20:57 | |
| zaro | ya, think it would be worthwhile. | 20:57 |
| *** david-lyle has quit IRC | 20:58 | |
| *** david-lyle has joined #openstack-infra | 20:58 | |
| openstackgerrit | Longgeek proposed a change to openstack-infra/config: Modify jenkins home directory https://review.openstack.org/97240 | 20:58 |
| *** pdmars has quit IRC | 20:59 | |
| zaro | clarkb: so how should we update review-dev to match review? | 21:00 |
| clarkb | zaro: you are probably correct we probably just need to run puppet there | 21:01 |
| clarkb | I am trying to review sdague's changes though | 21:01 |
| zaro | clarkb: and i don't know the sequence commands | 21:01 |
| clarkb | zaro: for puppet? | 21:02 |
| clarkb | zaro: I would do puppet agent --test --noop first | 21:02 |
| clarkb | read through the log for that and if it looks good do a puppet agent --test | 21:02 |
| *** otherwiseguy has joined #openstack-infra | 21:02 | |
| zaro | ok | 21:02 |
| clarkb | do this as root beacuse sometimes sudo doesn't play nice | 21:02 |
| fungi | especially puppet mysql module | 21:02 |
| *** cnesa2 has quit IRC | 21:03 | |
| clarkb | sdague: your third change fails according to jenkins | 21:04 |
| *** rfolco has quit IRC | 21:04 | |
| sdague | clarkb: ok, it was passing locally. I'll look into it after the limit patch is done | 21:05 |
| *** duran has quit IRC | 21:05 | |
| openstackgerrit | A change was merged to openstack-dev/pbr: Update integration script for Apache 2.4 https://review.openstack.org/102584 | 21:05 |
| *** slamont has quit IRC | 21:06 | |
| fungi | clarkb: before we shift too many more of these jobs to bare-trusty nodes, should we re-tweak the min-ready values for them some more? | 21:06 |
| fungi | for example, right now nodepool claims to have 15 bare-precise nodes ready in reserve, but no bare-trusty nodes ready | 21:08 |
| clarkb | sdague: fyi we are not indexing very quickly anymore. sar reports elasticsearch07 is currently in await hell | 21:08 |
| clarkb | 1291.62 is not a healthy await number | 21:08 |
| *** dstanek_zzz is now known as dstanek | 21:09 | |
| clarkb | fungi: we can | 21:09 |
| clarkb | let me whip that change up | 21:09 |
| *** Ryan_Lane1 has joined #openstack-infra | 21:10 | |
| openstackgerrit | Pablo Andres Fuente proposed a change to openstack-infra/config: Removing the Blazar API health check https://review.openstack.org/102945 | 21:10 |
| *** atiwari has joined #openstack-infra | 21:10 | |
| *** Ryan_Lane has quit IRC | 21:10 | |
| clarkb | fungi: what do you think of 21 each for bare-trusty and bare-precise? | 21:10 |
| fungi | hmm, though it does appear we're carrying a constant band of ready cruft looking at the graph... i'll see if i can confirm staleness | 21:10 |
| zaro | clarkb: err: /Stage[main]/Gerrit/File[/home/gerrit2/review_site/bin/gerrit.war]: Could not evaluate: Could not retrieve information from environment production source(s) file:///home/gerrit2/gerrit-wars/gerrit-v2.8.4.15.6dc8444.war at /opt/config/production/modules/gerrit/manifests/init.pp:535 | 21:10 |
| clarkb | zaro: that doesn't ring any bells | 21:11 |
| zaro | clarkb: will puppet add the file when i run test? | 21:11 |
| clarkb | zaro: if you do --noop it wont | 21:11 |
| clarkb | but i fyou remove --noop it should | 21:11 |
| openstackgerrit | A change was merged to openstack-infra/config: Move ceilometer default test node to trusty. https://review.openstack.org/102672 | 21:12 |
| clarkb | so if that was a noop run and the only real issue then you are probably ok to proceed | 21:12 |
| zaro | clarkb: ohh ok, then i'll do test without the noop now. | 21:12 |
| sdague | clarkb: it failed jenkins because of pip fail | 21:12 |
| *** markwash has quit IRC | 21:13 | |
| fungi | clarkb: those counts may have been a little off since there were ~40 nodes in a building state for up to 80 hours... i'm setting those to delete and rechecking | 21:13 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Start shifting balance of bare nodes to trusty https://review.openstack.org/102946 | 21:14 |
| clarkb | fungi: ^ | 21:14 |
| clarkb | fungi: that sounds like the rax issue. were they all in dfw? | 21:14 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: introduce limit parameter https://review.openstack.org/102947 | 21:15 |
| ianw | clarkb: i added testcases to https://review.openstack.org/#/c/101110/ (Track last allocations to ensure forward-progress) for "pinned" nodes, which is I think what you were talking about. be great if you could take a look when you get a chance | 21:15 |
| fungi | clarkb: after cleaning up the stale building nodes, it looks more like we're using as many of both as we're building at the moment, so hard to say which we need more of | 21:15 |
| clarkb | ianw: ya that was what I was hoping for, will try to take a look | 21:15 |
| clarkb | fungi: well nodepool should figure it out for us as well | 21:16 |
| clarkb | fungi: as it is demand based. the min ready is just there for ratio calculations and reducing lag time | 21:16 |
| fungi | we do have 37 bare-precise currently running jobs and only 5 bare-trusty doing so, but yeah reshifting the minimums in the wake of the first half dozen of your config changes will probably be a good idea | 21:16 |
| clarkb | 2293.53 so much await | 21:16 |
| clarkb | fungi: that and when we shift nova a ton of load will shift | 21:17 |
| clarkb | nova is the vast majority of our testing iirc | 21:17 |
| *** Alexei_987 has quit IRC | 21:17 | |
| clarkb | oh i didn't check horizon yet | 21:17 |
| *** dkliban_bbiab is now known as dkliban | 21:17 | |
| * clarkb does that next then will do oslo | 21:17 | |
| clarkb | or maybe clients | 21:17 |
| *** otherwiseguy has quit IRC | 21:18 | |
| zaro | fungi, clarkb : review-dev and review are sync now. | 21:19 |
| *** dstanek is now known as dstanek_zzz | 21:19 | |
| clarkb | zaro: awesome | 21:19 |
| clarkb | zaro: I take it puppet worked without the --noop? | 21:19 |
| fungi | zaro: thanks! | 21:19 |
| zaro | yup, all good | 21:19 |
| *** otherwiseguy has joined #openstack-infra | 21:19 | |
| *** andreaf_ has quit IRC | 21:20 | |
| fungi | clarkb: should we pause approving at nova (before? after?) or just press on through and deal with the possible deluge of reverts as they come up? | 21:20 |
| clarkb | fungi: I don't expect any reverts (I did test these things...) I think we can press on | 21:21 |
| clarkb | and hopefully those don't become my famous last wods | 21:21 |
| clarkb | *words | 21:21 |
| fungi | heh | 21:22 |
| ryanpetrello | mordred, clarkb: scratch that, it’s pypi fail-time for me currently :) | 21:22 |
| ryanpetrello | I’ll try again later :D | 21:23 |
| fungi | clarkb: okay, 8 move to trusty changes in the gate now | 21:25 |
| *** marcoemorais has quit IRC | 21:25 | |
| *** marcoemorais has joined #openstack-infra | 21:25 | |
| zaro | clarkb, fungi : sorry about that i read stream incorrectly. sizeInsertions and sizeDeletions is actually the just the number diff in changes from base, not from the previous patch. | 21:25 |
| clarkb | fungi: woot thanks | 21:26 |
| clarkb | zaro: ok but we can rely on that type flag | 21:26 |
| clarkb | zaro: so that works | 21:26 |
| fungi | zaro: clarkb: oh, right, we actually probably do want to special case NO_CODE_CHANGE then because that indicates a change between patchsets rather than a diff from the parent change | 21:26 |
| *** david-lyle has quit IRC | 21:26 | |
| clarkb | right | 21:27 |
| fungi | which we would potentially want to treat differently from file filters for a change | 21:27 |
| *** david-lyle has joined #openstack-infra | 21:27 | |
| clarkb | exactly | 21:27 |
| zaro | 'NO_CODE_CHANGE' is not in ver 2.8.x | 21:27 |
| *** lcheng has quit IRC | 21:27 | |
| clarkb | zaro: ok so we have to wait for 2.9 then | 21:27 |
| clarkb | that was easy :) | 21:27 |
| fungi | but it's on the table as a potentially useful thing for zuul to support once we can test reasonably against 2.9 | 21:28 |
| *** david-lyle has quit IRC | 21:28 | |
| *** jp_at_hp has joined #openstack-infra | 21:28 | |
| *** david-lyle has joined #openstack-infra | 21:28 | |
| clarkb | yup | 21:29 |
| zaro | probably not until 2.10 https://gerrit-review.googlesource.com/#/c/57231/ | 21:29 |
| fungi | oh well | 21:30 |
| *** lcheng has joined #openstack-infra | 21:31 | |
| clarkb | the other way of doing it is to diff every patchset created event | 21:31 |
| clarkb | which is a bit :( | 21:31 |
| fungi | hmmm... so right now nodepool is building 110 bare-precise nodes and 2 bare-trusty nodes. hopefully the balance shifts a bit once the config change hits it | 21:31 |
| fungi | otherwise we could end up with quite the surplus of precise nodes soonish | 21:32 |
| Alex_Gaynor | Will they be brought down automatically? | 21:32 |
| clarkb | fungi: that is probably because there is no demand for trusty | 21:33 |
| fungi | yep, for the moment | 21:34 |
| clarkb | Alex_Gaynor: yes 8 hours is the max lifetime for a node | 21:34 |
| *** cnesa has quit IRC | 21:34 | |
| openstackgerrit | A change was merged to openstack-infra/config: Move cinder default test node to trusty. https://review.openstack.org/102678 | 21:34 |
| fungi | except for hold state and for some extraordinary error conditions | 21:34 |
| clarkb | woot I haev my first trusty fail | 21:34 |
| clarkb | horizon | 21:34 |
| clarkb | ImportError: No module named httplib2 | 21:35 |
| *** markmcclain has quit IRC | 21:35 | |
| fungi | right now we have 3 held nodes older than 8 hours. and roughly a dozen hung in delete state in rax | 21:35 |
| clarkb | missing dependency? | 21:35 |
| *** annegent_ has quit IRC | 21:35 | |
| fungi | clarkb: tristanC saw that today when testing a patch for something | 21:35 |
| clarkb | indeed no httplib | 21:35 |
| fungi | clarkb: i think maybe it was present by default on precise and we don't install it | 21:35 |
| clarkb | I am guessing it was a transitive dep and now isn't? | 21:36 |
| fungi | something like that | 21:36 |
| clarkb | fungi: except we don't allow site packages usually | 21:36 |
| sdague | clarkb: or no longer part of the base | 21:36 |
| sdague | python-httplib2 | 21:36 |
| clarkb | sdague: but we don't allow site packages | 21:36 |
| sdague | oh | 21:36 |
| clarkb | ya there is no site packages = true in horizon's tox.ini | 21:36 |
| clarkb | so I don't know how that got in | 21:37 |
| sdague | clarkb: which tox target? | 21:37 |
| * clarkb tests explicitly adding it as a dep | 21:37 | |
| clarkb | sdague: py27 | 21:37 |
| fungi | tristanC: (if you haven't gone home for the night yet) when you saw the "ImportError: No module named httplib2" on a horizon patch testing by hand and jpich couldn't reproduce, was that on ubuntu trusty by any chance? | 21:37 |
| clarkb | unless run_tests is doing stuff | 21:37 |
| *** markwash has joined #openstack-infra | 21:38 | |
| clarkb | which probably does terrible things | 21:38 |
| *** mrda-away is now known as mrda | 21:39 | |
| *** cnesa has joined #openstack-infra | 21:39 | |
| *** markwash_ has joined #openstack-infra | 21:40 | |
| mtreinish | clarkb: at least it's not doing a venv in a venv... | 21:41 |
| sdague | I wonder if it's new nose | 21:41 |
| *** Sukhdev has joined #openstack-infra | 21:41 | |
| clarkb | mtreinish: well it might be | 21:42 |
| clarkb | sdague: or that | 21:42 |
| *** markwash has quit IRC | 21:42 | |
| *** markwash_ is now known as markwash | 21:42 | |
| clarkb | mtreinish: I am trying to sort out if with_venv is used even in tox | 21:42 |
| clarkb | which would create a second venv. But it doesn't look like it is because I have no .venv | 21:42 |
| sdague | clarkb: I'm running those unit tests on my workstation now | 21:42 |
| fungi | lolwut? https://review.openstack.org/102681 | 21:42 |
| sdague | with httplib2 installed in the venv, it's good | 21:42 |
| fungi | this time it bit us on the second change in the stack | 21:43 |
| sdague | I'm going to blame nose | 21:43 |
| sdague | because that's coming from site-packages | 21:43 |
| clarkb | sdague: ok | 21:43 |
| clarkb | sdague: if you source your .tox/py27/bin/activate can you import that lib? | 21:43 |
| clarkb | sdague: eg is the virtualenv correct? | 21:43 |
| sdague | if I source that, I can install that lib | 21:43 |
| sdague | let me try the other thing | 21:44 |
| mtreinish | clarkb: it looks like the tox.ini has -N passed to run_tests which should skip using with_venv | 21:45 |
| fungi | ick | 21:46 |
| sdague | mtreinish: yeh, I just noticed that | 21:46 |
| fungi | so basically, worse than system_site_packages=True | 21:46 |
| sdague | which is kind of hillarious | 21:46 |
| sdague | it builds the venv | 21:46 |
| clarkb | mtreinish: ya I think it should work | 21:46 |
| sdague | then says | 21:46 |
| sdague | meh | 21:46 |
| clarkb | sdague: oh does it? | 21:46 |
| clarkb | my current reading is it skips all of the env check stuff with -N | 21:47 |
| clarkb | but I haven't -x'd it | 21:47 |
| sdague | http://paste.openstack.org/show/85022/ | 21:47 |
| Sukhdev | anteaya: Ping | 21:47 |
| clarkb | sdague: right so it shouldn't create a venv | 21:47 |
| sdague | clarkb: no, I mean tox builds the venv | 21:47 |
| clarkb | and rely on the tox venv | 21:47 |
| clarkb | right | 21:47 |
| sdague | but doesn't the -N on run_tests.sh skip using the venv? | 21:48 |
| clarkb | it skips using the run tests venv | 21:48 |
| clarkb | it should run properly in the tox venv | 21:48 |
| clarkb | I think | 21:48 |
| sdague | oh, wait, I did a silly thing | 21:48 |
| *** denis_makogon has quit IRC | 21:48 | |
| sdague | I should be able to get my fail again | 21:48 |
| openstackgerrit | A change was merged to openstack-infra/config: Start shifting balance of bare nodes to trusty https://review.openstack.org/102946 | 21:50 |
| *** r1chardj0n3s has joined #openstack-infra | 21:50 | |
| clarkb | httplib2 has been around since 2011 | 21:50 |
| clarkb | so this is really weird | 21:50 |
| anteaya | Sukhdev: how can I help? | 21:52 |
| sdague | clarkb: it's definitely not in there | 21:53 |
| clarkb | sdague: weird | 21:53 |
| clarkb | even when you import directly? | 21:53 |
| clarkb | and yet somehow it works? | 21:53 |
| *** mriedem has quit IRC | 21:54 | |
| sdague | when I import directly on trusty, nope, not there | 21:54 |
| sdague | and doesn't work | 21:54 |
| sdague | and you can see it's not there in the fs | 21:54 |
| clarkb | sdague: right | 21:55 |
| clarkb | I guess I need to grab one of my precise VMs | 21:55 |
| * clarkb does this | 21:55 | |
| sdague | I'm going to propose the requirements add anyway | 21:55 |
| clarkb | ++ | 21:55 |
| clarkb | we should be explicit about it | 21:55 |
| jeblair | back | 21:55 |
| *** masayukig has joined #openstack-infra | 21:57 | |
| *** sarob has joined #openstack-infra | 21:58 | |
| *** hogepodge has quit IRC | 22:00 | |
| *** devkulkarni has quit IRC | 22:00 | |
| *** dkranz has joined #openstack-infra | 22:01 | |
| *** hogepodge has joined #openstack-infra | 22:02 | |
| clarkb | sdague: so it faield on my precise box too | 22:04 |
| *** sarob has quit IRC | 22:04 | |
| clarkb | sdague: I am going to install python-httplib2 globally on that machien and see what happens | 22:04 |
| clarkb | python-httplib2 is already the newest version. | 22:04 |
| clarkb | now Ireally don't understand | 22:04 |
| sdague | clarkb: gremlines? | 22:04 |
| clarkb | must be | 22:05 |
| *** esker has quit IRC | 22:05 | |
| *** clayg has left #openstack-infra | 22:05 | |
| fungi | clarkb: i'm rebasing the remaining changes in your move-to-trusty stack since jgit seems to want it | 22:06 |
| *** Sukhdev has quit IRC | 22:06 | |
| *** Sukhdev_ has joined #openstack-infra | 22:06 | |
| clarkb | gah jgit | 22:06 |
| fungi | unless you object | 22:06 |
| clarkb | no go for it | 22:07 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move ironic default test node to trusty. https://review.openstack.org/102696 | 22:07 |
| fungi | bam | 22:07 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move glance default test node to trusty. https://review.openstack.org/102681 | 22:07 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move swift default test node to trusty. https://review.openstack.org/102686 | 22:07 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move nova default test node to trusty. https://review.openstack.org/102621 | 22:07 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move heat default test node to trusty. https://review.openstack.org/102690 | 22:07 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move sahara default test node to trusty. https://review.openstack.org/102792 | 22:07 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Move keystone default test node to trusty. https://review.openstack.org/102694 | 22:07 |
| notmyname | fungi: tell me more? what does ^^ mean? | 22:08 |
| morganfainberg | notmyname, using 14.04 instead of 12.04 for the base system (unit tests) | 22:08 |
| jeblair | if we merge all of those at once we won't tickle the octopus merge | 22:08 |
| *** devkulkarni has joined #openstack-infra | 22:08 | |
| clarkb | jeblair: is it better to just use one big change? | 22:08 |
| clarkb | I could've squashed them too but like the revert cleaness this approach gave | 22:08 |
| jeblair | clarkb: this makes reverting easy | 22:08 |
| clarkb | yup | 22:08 |
| notmyname | morganfainberg: for all tests in the gate? what does "base system" mean? | 22:08 |
| Sukhdev_ | anteaya: Hi sorry - got pulled into something | 22:09 |
| *** signed8bit has quit IRC | 22:09 | |
| jeblair | yeah, just as long as we merge them all without merging anything intervening, they will be fast-forwards and not merges, so jgit will be happy | 22:09 |
| morganfainberg | notmyname, unit tests. so like py26/27 iirc | 22:09 |
| morganfainberg | notmyname, pep8, run those on ubuntu 14.04 is what those changes are for | 22:09 |
| notmyname | morganfainberg: ok. (to be clear I'm not griping here). was there a ML message or somthign that I missed about this happening? seems like changing the base OS for tests is a kinda big deal | 22:10 |
| Sukhdev_ | anteaya: had a quick question for you | 22:10 |
| *** UtahDave has left #openstack-infra | 22:10 | |
| morganfainberg | notmyname, i'd need to defer to the infra folks for that, but i think there was a topic. | 22:10 |
| clarkb | there was a ML mail sent | 22:11 |
| notmyname | clarkb: I see one now from you: "Tempest testing for master is now running on Trusty" | 22:12 |
| notmyname | clarkb: that one | 22:12 |
| notmyname | ? | 22:12 |
| hashar | clarkb: jeblair: hi :) Zuul has some tiny patches I added to the Merger. Would be nice to have some kind of review to them. Joshua already passed through them. https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul+branch:master+topic:merger,n,z | 22:12 |
| clarkb | notmyname: yes that one | 22:12 |
| clarkb | notmyname: it mentioned will be moving the python27 stuff as well | 22:13 |
| anteaya | Sukhdev_: please ask | 22:13 |
| Sukhdev_ | anteaya: I know there was a discussion about the duration for which CI systems are suppose to preserve the logs | 22:14 |
| notmyname | clarkb: that py27 and stable are still on precise. got it | 22:14 |
| notmyname | clarkb: err..I mean all stable stuff. py27 is moving for new code | 22:14 |
| Sukhdev_ | anteaya: what is the final conclusion on that - i.e. how long are we suppose to preserve the logs? | 22:14 |
| clarkb | notmyname: yup | 22:14 |
| clarkb | notmyname: ya thats important, we don't move the stable code because we would have to backport changes and :( | 22:14 |
| anteaya | Sukhdev_: as of right now one month | 22:14 |
| anteaya | there was an email up on the ml | 22:15 |
| anteaya | noone expressed a reason to have a different time frame | 22:15 |
| *** signed8bit has joined #openstack-infra | 22:15 | |
| notmyname | clarkb: and is there anything running on RHEL/cent? | 22:15 |
| anteaya | might that time frame change in future, it might | 22:15 |
| clarkb | notmyname: yes, python26 is all on centos6 | 22:15 |
| anteaya | but today the time frame is one month | 22:15 |
| *** andreykurilin has quit IRC | 22:15 | |
| clarkb | notmyname: becaues it is the only platform with supported python26 that is sanely available to us | 22:15 |
| Sukhdev_ | anteaya: perfect - the original documentation required 6 months. | 22:15 |
| clarkb | notmyname: there have been rumblings of doing tempest on centos7 eventually | 22:16 |
| anteaya | Sukhdev_: can you point to that document? | 22:16 |
| clarkb | but centos7 isn't a thing yet so who knows | 22:16 |
| anteaya | because I don't remember that it was every stipulated for third party ci | 22:16 |
| anteaya | OpenStack infra saves logs for 6 months | 22:16 |
| clarkb | notmyname: we do not do centos6 tempest today beacuse the kernel and ovs and things are too old or so I am told | 22:16 |
| clarkb | anteaya: 4 now | 22:16 |
| anteaya | okay | 22:16 |
| anteaya | 4 months | 22:16 |
| anteaya | due to space considerations | 22:16 |
| Sukhdev_ | anteaya: the original wiki and email which mark had sent late last year - | 22:16 |
| anteaya | but we don't expect third party ci to do that | 22:17 |
| anteaya | Sukhdev_: if you can dig up the email and wikipage I will update them | 22:17 |
| *** rlandy has quit IRC | 22:17 | |
| anteaya | but yes, right now, today 1 month log retention requirement for third party ci | 22:17 |
| Sukhdev_ | anteaya: but, it is not worth for me spend time to look for it. You word is good enough for me :-) | 22:18 |
| anteaya | okay | 22:18 |
| anteaya | if you happen across them, please post the url with a ping to me and I will ensure they are updated | 22:18 |
| Sukhdev_ | anteaya: I will try to dig the email - and send it to you if I am able to find | 22:18 |
| anteaya | thanks | 22:19 |
| anteaya | I don't know if that is up on the docs yet | 22:19 |
| anteaya | I will offer a patch | 22:19 |
| anteaya | which will probably have a merge conflict but I don't like to create dependencies | 22:19 |
| anteaya | since i never know which item we will reach agreement on first | 22:20 |
| *** cnesa has quit IRC | 22:21 | |
| openstackgerrit | A change was merged to openstack-infra/config: Move glance default test node to trusty. https://review.openstack.org/102681 | 22:23 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Adds test artifact retention requirement https://review.openstack.org/102960 | 22:24 |
| notmyname | clarkb: so to confirn, for new testing there is nothing running on precise anymore (ie once those patches land) | 22:24 |
| *** chandan_kumar has quit IRC | 22:24 | |
| *** Alexei_987 has joined #openstack-infra | 22:24 | |
| clarkb | notmyname: correct | 22:24 |
| clarkb | notmyname: you will have centos6 py26 testing, tempest and function tests on trusty devstack nodes and everything else on trusty | 22:24 |
| *** chianingwang has quit IRC | 22:24 | |
| *** hashar has quit IRC | 22:24 | |
| *** oomichi has joined #openstack-infra | 22:24 | |
| Sukhdev_ | anteaya: Just as an FYI - I looked for the email from Mark. Was able to find it, it has a link to the wiki, which has been modified now. So, there is no mention of preserving logs for 6 months. | 22:26 |
| Sukhdev_ | anteaya: hence, it seems we are good. | 22:26 |
| notmyname | clarkb: this is not an argument I want to start here or now or over any text-based medium actually. for the record, I don't think it's sufficient test coverage | 22:26 |
| anteaya | Sukhdev_: thanks for following taht up, I appreciate it | 22:26 |
| Sukhdev_ | anteaya: this was the email - From: Mark McClain <mark.mcclain@dreamhost.com> | 22:26 |
| clarkb | notmyname: ok, you think we need to test precise as well? | 22:26 |
| Sukhdev_ | Date: Thu, Nov 14, 2013 at 12:20 PM | 22:26 |
| anteaya | Sukhdev_: ah let's not paste it | 22:27 |
| anteaya | I'll see if I can find it in the archives | 22:27 |
| clarkb | notmyname: we are doing this because openstack TC has said we will target latest red hat and ubuntu and not break latest LTS | 22:27 |
| *** david-lyle has quit IRC | 22:27 | |
| Sukhdev_ | anteaya: Date: Thu, Nov 14, 2013 at 12:20 PM | 22:27 |
| clarkb | notmyname: so that is what drives the biggest determination of how the matrix is set up | 22:27 |
| anteaya | thanks | 22:27 |
| openstackgerrit | A change was merged to openstack-infra/config: Move swift default test node to trusty. https://review.openstack.org/102686 | 22:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Move heat default test node to trusty. https://review.openstack.org/102690 | 22:28 |
| *** david-lyle has joined #openstack-infra | 22:28 | |
| openstackgerrit | A change was merged to openstack-infra/config: Move keystone default test node to trusty. https://review.openstack.org/102694 | 22:28 |
| notmyname | clarkb: correct. I think precise is still a supported and widely used platform by users (and RHEL/cent for that matter) and should be tested as part of the overall gate process | 22:28 |
| Sukhdev_ | anteaya: NP - BTW, appreciate your help | 22:28 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Search use case 1: Quicknav https://review.openstack.org/102652 | 22:28 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added search interface https://review.openstack.org/99975 | 22:28 |
| anteaya | Sukhdev_: here it is: http://lists.openstack.org/pipermail/openstack-dev/2013-November/019219.html | 22:28 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added custom CSS aliases for common storyboard icons. https://review.openstack.org/102963 | 22:28 |
| anteaya | Sukhdev_: do you know how to access the email archives? | 22:28 |
| anteaya | Sukhdev_: go to lists.openstack.org | 22:28 |
| anteaya | Sukhdev_: all the email lists are collected there | 22:29 |
| *** chuckC has quit IRC | 22:29 | |
| clarkb | notmyname: roger its probably worth bringing up with TC again? there are also practical considerations | 22:29 |
| anteaya | find your list of choice and then select the archives link on the resultant page | 22:29 |
| Sukhdev_ | anteaya: Ah ha - no, I did not know - I learned something new today | 22:29 |
| clarkb | like we don't actually test on latest ubuntu because support for those releases is only 9 months | 22:29 |
| notmyname | clarkb: yes, I'm aware of the practical considerations | 22:29 |
| anteaya | Sukhdev_: great | 22:29 |
| anteaya | Sukhdev_: go forth and share | 22:29 |
| Sukhdev_ | anteaya: I should keep talking to you. Every time I learn something new from you…. | 22:29 |
| Sukhdev_ | anteaya :-) | 22:30 |
| openstackgerrit | A change was merged to openstack-infra/config: Move ironic default test node to trusty. https://review.openstack.org/102696 | 22:30 |
| anteaya | Sukhdev_: :D | 22:30 |
| anteaya | great I have another one for you | 22:30 |
| mattoliverau | Morning all | 22:31 |
| Sukhdev_ | anteaya: OK | 22:31 |
| anteaya | Sukhdev_: go to this page: https://wiki.openstack.org/wiki/Category:ThirdPartySystems | 22:31 |
| anteaya | Sukhdev_: see if you can figure out what to do next | 22:31 |
| *** hogepodge has quit IRC | 22:31 | |
| anteaya | Sukhdev_: let me know what you decide | 22:31 |
| Sukhdev_ | anteaya: let me look | 22:31 |
| anteaya | Sukhdev_: take your time | 22:31 |
| * anteaya goes to make tea | 22:32 | |
| *** david-lyle has quit IRC | 22:32 | |
| *** unicell1 has quit IRC | 22:33 | |
| *** james_li has quit IRC | 22:33 | |
| openstackgerrit | A change was merged to openstack-infra/config: Move nova default test node to trusty. https://review.openstack.org/102621 | 22:34 |
| openstackgerrit | A change was merged to openstack-infra/config: Move sahara default test node to trusty. https://review.openstack.org/102792 | 22:34 |
| *** eharney has quit IRC | 22:34 | |
| *** annegent_ has joined #openstack-infra | 22:36 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 22:36 | |
| Sukhdev_ | anteaya: this is a useful link | 22:38 |
| *** terryw has joined #openstack-infra | 22:39 | |
| *** otherwiseguy has quit IRC | 22:39 | |
| *** terryw is now known as otherwiseguy | 22:39 | |
| *** dangers is now known as dangers_away | 22:40 | |
| *** dstanek_zzz is now known as dstanek | 22:40 | |
| *** hogepodge has joined #openstack-infra | 22:40 | |
| *** asselin has quit IRC | 22:41 | |
| *** kmartin has quit IRC | 22:41 | |
| *** asselin has joined #openstack-infra | 22:41 | |
| *** kmartin has joined #openstack-infra | 22:42 | |
| *** annegent_ has quit IRC | 22:42 | |
| anteaya | Sukhdev_: great | 22:42 |
| anteaya | Sukhdev_: I'm glad you are finding it so | 22:42 |
| anteaya | Sukhdev_: think you might have anything to offer to expand its usefulness? | 22:43 |
| clarkb | sdague: fyi await is high again | 22:43 |
| clarkb | sdague: :/ silly volumes | 22:43 |
| sdague | clarkb: well the limit patches are up | 22:43 |
| clarkb | maybe I take 7 out of rotation, give it an ssd volume and see how that does as 07 appears to be the worst | 22:44 |
| Sukhdev_ | anteaya:I know you are still working on it - you could add http://stackalytics.com/report/driverlog link to this wiki as well | 22:44 |
| sdague | yeh, that will at least give an idea of what an ssd acts like for us | 22:44 |
| *** annegent_ has joined #openstack-infra | 22:46 | |
| *** lcheng has quit IRC | 22:46 | |
| sdague | clarkb: on the upside, we didn't break stable requirements with our pbr change - https://review.openstack.org/#/c/102950/ | 22:46 |
| clarkb | sdague: woot | 22:47 |
| sdague | I picked a randomly useful backport just to see | 22:49 |
| anteaya | registered | 22:49 |
| anteaya | thanks reed | 22:49 |
| anteaya | Sukhdev_: ah that really wasn't what had in mind | 22:49 |
| anteaya | since that log is populated by scraping wikipages | 22:49 |
| *** zzelle has quit IRC | 22:49 | |
| anteaya | do you think that your ci might have a place on this page? | 22:49 |
| anteaya | Sukhdev_: perhaps like this? https://wiki.openstack.org/wiki/ThirdPartySystems/IBMPowerKVMCI | 22:50 |
| *** signed8bit_ZZZzz is now known as signed8bit | 22:50 | |
| Sukhdev_ | anteaya: I saw this and immediate thought crossed my mind to add my CI here - but, then I thought may be this is for an example sake only | 22:51 |
| anteaya | go with your first though | 22:51 |
| anteaya | t | 22:51 |
| Sukhdev_ | anteaya: Are you OK if I add my wiki pointer here? | 22:51 |
| *** signed8bit is now known as signed8bit_ZZZzz | 22:51 | |
| anteaya | Sukhdev_: can you create a page like https://wiki.openstack.org/wiki/ThirdPartySystems/IBMPowerKVMCI | 22:52 |
| *** markmcclain has joined #openstack-infra | 22:52 | |
| anteaya | Sukhdev_: you can add your pointer in that page if you wish | 22:52 |
| *** atiwari has quit IRC | 22:52 | |
| anteaya | Sukhdev_: go here: https://wiki.openstack.org/wiki/ThirdPartySystems | 22:52 |
| anteaya | follow the instructions | 22:53 |
| *** sarob has joined #openstack-infra | 22:53 | |
| *** devkulkarni has quit IRC | 22:53 | |
| Sukhdev_ | anteaya: Ah cool - I will create a wiki page and add to this table - thanks | 22:53 |
| anteaya | the other way around | 22:53 |
| anteaya | add it to the table | 22:53 |
| anteaya | that creats the wikipage | 22:53 |
| anteaya | follow the instructions on that page | 22:53 |
| *** markmcclain1 has joined #openstack-infra | 22:53 | |
| Alex_Gaynor | jogo: Do you have an opinion about changing out flake8 requirement to be >=2.1.0,<=2.2.0 or similar? 2.2.0 was released, it doesn't appear to include new checks, and includes parallelization by default, which makes stuff way faster for big code bases | 22:54 |
| Sukhdev_ | anteaya: cool - I will follow the instructions and add it there | 22:55 |
| anteaya | Sukhdev_: great | 22:55 |
| anteaya | let me know how you do | 22:55 |
| *** markmcclain has quit IRC | 22:56 | |
| *** r1chardj0n3s is now known as r1chard_afk | 22:56 | |
| *** zzelle has joined #openstack-infra | 22:57 | |
| *** msabramo has quit IRC | 22:57 | |
| *** sarob has quit IRC | 22:57 | |
| *** Sukhdev has joined #openstack-infra | 22:59 | |
| *** sarob has joined #openstack-infra | 22:59 | |
| *** otherwiseguy has quit IRC | 22:59 | |
| *** Sukhdev_ has quit IRC | 23:00 | |
| *** dims_ has quit IRC | 23:01 | |
| bodepd | nibalizer: how goes the splitting stuff out efforts? | 23:01 |
| fungi | clarkb: did you confirm we have ssd available (i.e. it doesn't throw an error when you try to select it for a volume)? | 23:01 |
| *** david-lyle has joined #openstack-infra | 23:02 | |
| clarkb | fungi: well we used ssd with gerrit | 23:02 |
| clarkb | fungi: so I think so | 23:02 |
| *** bknudson has quit IRC | 23:02 | |
| arosen | Hi, I had a quick question, once this is approved I guess I'll be able to push the cookiecutter outline to review.openstack.org for this new project? https://review.openstack.org/#/c/102697/ | 23:03 |
| fungi | oh! i didn't realize we'd done that | 23:03 |
| *** asettle has joined #openstack-infra | 23:03 | |
| *** chuckC has joined #openstack-infra | 23:03 | |
| *** sweston_ has joined #openstack-infra | 23:03 | |
| *** sarob has quit IRC | 23:04 | |
| *** SumitNaiksatam has quit IRC | 23:04 | |
| clarkb | arosen: you will be able to propose it as a change | 23:04 |
| clarkb | arosen: you will need noop check and gate jobs at the very least | 23:04 |
| clarkb | (should be added ot that change) | 23:05 |
| anteaya | arosen: so you want all of core to be tagging that repo? we encourage only one person (the ptl) to have that permission | 23:05 |
| anteaya | s/so/do | 23:05 |
| clarkb | arosen: as an alternative you can put up a seed repo that already has the cookiecutterstuff ready to go | 23:06 |
| clarkb | gerrit will import it as is | 23:06 |
| mnaser | atc emails just went out? | 23:06 |
| mnaser | :D | 23:06 |
| mgagne | might be come issues with that email | 23:07 |
| mgagne | openstack-infra ML got a discount | 23:07 |
| mtreinish | cool, I can't wait to meet OpenStack Proposal bot in Paris | 23:07 |
| mnaser | mine is sent personally so i tihnk i'm all good :> | 23:08 |
| mgagne | they probably should filter out @lists.openstack.org | 23:08 |
| morganfainberg | mtreinish, OMG The Proposal bot will be there?! I totally need an autograph! | 23:08 |
| morganfainberg | :) | 23:08 |
| mtreinish | morganfainberg: http://lists.openstack.org/pipermail/openstack-infra/2014-June/001447.html | 23:08 |
| arosen | clarkb: I rather just push the cookiecutter though gerrit just so we have everything formally reviewed. | 23:09 |
| morganfainberg | mtreinish *snicker* | 23:09 |
| *** annegent_ has quit IRC | 23:09 | |
| morganfainberg | wonder if someone has used it yet | 23:09 |
| arosen | anteaya: for now i'm not worried about the cores tagging i'll just leave it as that for now since we don't have a ptl setup yet. | 23:09 |
| mtreinish | morganfainberg: the code looks the same as mine :) | 23:10 |
| clarkb | arosen: that will work just fine | 23:10 |
| jeblair | reed: ^ you may want to disable the discount code ATC_A07SV | 23:11 |
| *** r1chard_afk has quit IRC | 23:11 | |
| reed | ? | 23:11 |
| *** msabramo has joined #openstack-infra | 23:11 | |
| morganfainberg | reed, it went to a mailing list | 23:11 |
| reed | ho | 23:11 |
| morganfainberg | reed, http://lists.openstack.org/pipermail/openstack-infra/2014-June/001447.html | 23:11 |
| reed | sweet | 23:12 |
| *** sballe has quit IRC | 23:12 | |
| jeblair | also, apparently we need someone to dress up as a robot | 23:13 |
| *** pdmars has joined #openstack-infra | 23:13 | |
| mnaser | i say | 23:13 |
| mnaser | keep the discount code if it can be used once only | 23:13 |
| mnaser | and whoever claims it has that obligation | 23:13 |
| *** pdmars has quit IRC | 23:13 | |
| reed | problem solved | 23:13 |
| anteaya | arosen: okay | 23:13 |
| openstackgerrit | Aaron Rosen proposed a change to openstack-infra/config: Add python-congressclient as new project to stackforge https://review.openstack.org/102697 | 23:13 |
| jeblair | ++ | 23:13 |
| mnaser | also their name badge has to say openstack proposal bot | 23:13 |
| *** david-lyle has quit IRC | 23:13 | |
| *** pdmars has joined #openstack-infra | 23:14 | |
| arosen | clarkb: hopefully: https://review.openstack.org/#/c/102697/2/modules/openstack_project/files/zuul/layout.yaml is good now? | 23:14 |
| jeblair | this is like the first time i haven't waited till the last minute to register and now i am sad | 23:14 |
| sdague | jeblair: why sad? | 23:14 |
| jeblair | because i have missed the opportunity to register as the proposal bot | 23:14 |
| arosen | clarkb: I just added a check for pep8/python2.7 and will add gate for those once i get stuff going | 23:14 |
| anteaya | the mailing list summit pass lottery | 23:15 |
| *** banix has quit IRC | 23:15 | |
| *** r1chardj0n3s has joined #openstack-infra | 23:15 | |
| sdague | :) | 23:16 |
| *** sweston_ has quit IRC | 23:16 | |
| clarkb | arosen: ya ITHink that will work. will give you proper review in a bit | 23:16 |
| arosen | clarkb: sounds good thanks! | 23:17 |
| *** pballand has quit IRC | 23:18 | |
| *** pballand has joined #openstack-infra | 23:19 | |
| *** pballand has quit IRC | 23:20 | |
| *** _nadya_ has joined #openstack-infra | 23:20 | |
| jhesketh | Morning | 23:21 |
| jhesketh | so read some of the backlog but there's too much to track | 23:21 |
| jhesketh | looks like we still don't know if we want to not-recheck on -1 votes | 23:21 |
| anteaya | morning jhesketh | 23:21 |
| anteaya | oh and register for summit | 23:21 |
| anteaya | unless you want to show up as the proposal bot | 23:22 |
| anteaya | if that code is still open | 23:22 |
| reed | anteaya, that code is gone :) | 23:22 |
| jhesketh | zaro: thanks for you envinject reviews, very helpeful :-) | 23:22 |
| jhesketh | I'll roll some new ones shortly | 23:22 |
| *** CaptTof__ has quit IRC | 23:22 | |
| jeblair | jhesketh: yes. i'd avoid investing too much time in the not-recheck thing, unless you really just feel like hacking on zuul regardless of whether we'll use it. :) | 23:23 |
| anteaya | reed: kk, did someone use it to register? | 23:23 |
| jeblair | jhesketh: we've decided we need to take a step back and re-establish what the actual goals are for the freshness checks | 23:24 |
| jhesketh | jeblair: I like the idea of approval-any or approval-all filters, but I'll wait to see where this goes | 23:24 |
| reed | anteaya, yes, someone who failed at registering the first time using his own :) you know how eventbrite gets tricky :) | 23:24 |
| jhesketh | so my other thought for an improvement to zuul is to make it much more intelligent about which jobs might need a freshness check | 23:24 |
| *** _nadya_ has quit IRC | 23:24 | |
| anteaya | reed: that png with the red arrows, maybe we need a few more arrows in that | 23:25 |
| jhesketh | for example, pep8 or py27 jobs don't share dependencies with other projects, so if the sha hasn't changed then the vote should be okay | 23:25 |
| *** jamielennox|away is now known as jamielennox | 23:25 | |
| jhesketh | (the exception being for external dependencies breaking things but that can similarly be caught on a full check in the gate or something) | 23:25 |
| jeblair | jhesketh: sdague indicated that the answer to that might be "none" and we may only really care about merge conflicts | 23:25 |
| reed | anteaya, :) | 23:26 |
| anteaya | :D | 23:26 |
| jeblair | anteaya: it's my 7th summit and the first time i remembered where to put the reg code | 23:26 |
| *** dims_ has joined #openstack-infra | 23:26 | |
| anteaya | ha ha ha | 23:26 |
| anteaya | practice makes perfect | 23:26 |
| *** homeless has quit IRC | 23:29 | |
| clarkb | https://jenkins03.openstack.org/job/gate-nova-python27/4897/ | 23:29 |
| clarkb | that ran on trusty and was happy | 23:29 |
| arosen | clarkb: seems like it failed gate-config-layout with: Job gate-python-congressclient-pep8 not defined | 23:30 |
| arosen | is there another repo i need to add that? | 23:30 |
| clarkb | arosen: oh right we have to instantiate the template | 23:30 |
| openstackgerrit | A change was merged to openstack-infra/zuul: dequeue abandoned changes https://review.openstack.org/65467 | 23:30 |
| arosen | I grepped around to see what other projects did | 23:30 |
| clarkb | arosen: in modules/openstack_project/files/jenkins_job_builder/config/projects.yaml you want to add a section that adds python-jobs | 23:30 |
| clarkb | arosen: there should be other projects in there that do the same | 23:30 |
| clarkb | if you grep for python-jobs you should see what I mean | 23:31 |
| arosen | ah okay thanks. | 23:32 |
| clarkb | that is a collection of python related jobs including the py27 and pep8 job | 23:32 |
| openstackgerrit | Aaron Rosen proposed a change to openstack-infra/config: Add python-congressclient as new project to stackforge https://review.openstack.org/102697 | 23:33 |
| anteaya | arosen: I've missed why you don't want gate jobs | 23:34 |
| arosen | anteaya: i will add that later but just wanted to get things going for now. | 23:35 |
| anteaya | okay | 23:35 |
| anteaya | I've +1 and added your decisions in a comment | 23:36 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for bug 1334898 https://review.openstack.org/102979 | 23:37 |
| uvirtbot | Launchpad bug 1334898 in openstack-ci "reqs integration job fails with /home/jenkins/mirror_package_list.txt: No such file or directory" [Undecided,New] https://launchpad.net/bugs/1334898 | 23:37 |
| *** mriedem has joined #openstack-infra | 23:37 | |
| *** jp_at_hp has quit IRC | 23:37 | |
| jeblair | anteaya: fyi when i was reviewing the config repo, i skipped all new-project reviews that you have not +1d | 23:38 |
| *** devkulkarni has joined #openstack-infra | 23:40 | |
| clarkb | sdague: is horizon httplib2 change up? | 23:41 |
| fungi | jeblair: oh, i've partially solved the mystery of that zuul exception/traceback | 23:41 |
| *** jp_at_hp has joined #openstack-infra | 23:42 | |
| fungi | jeblair: the changes it throws those errors on definitely don't get returned by a gerrit ssh api query | 23:42 |
| *** shayneburgess has quit IRC | 23:42 | |
| *** devkulkarni has quit IRC | 23:43 | |
| fungi | jeblair: so it's something broken with those changes in gerrit i think. poked around in the db some and they look sane enough there, but i'll dig in the gerrit logs next and see if it's freaking out over them somehow | 23:43 |
| anteaya | jeblair: k, I have been not keeping up, I will address that now | 23:43 |
| anteaya | jeblair: can you take a look at: https://review.openstack.org/#/c/101361/ it brings some uniformity to olso acl files | 23:45 |
| *** zzelle has quit IRC | 23:45 | |
| anteaya | jeblair: if you like it, I can review dims 2 new-project patches to oslo | 23:46 |
| *** doug-fish1 has left #openstack-infra | 23:48 | |
| jeblair | fungi: oops, i got so distracted by the paste server not serving your paste that once i fixed the server, i forgot to actually look at the paste | 23:50 |
| jeblair | anteaya: hopefully tomorrow :) | 23:51 |
| *** dkehn_ has joined #openstack-infra | 23:53 | |
| *** sarob has joined #openstack-infra | 23:53 | |
| *** craigbr has quit IRC | 23:54 | |
| *** dkehn_ has quit IRC | 23:55 | |
| davidlenwell | For a new contributer is there a delay between when a user sets up a foundation account.. then signs the icla to when gerrit will accept a contribution from them? | 23:55 |
| *** dkehnx has quit IRC | 23:55 | |
| clarkb | davidlenwell: there shouldn't be | 23:55 |
| *** dkehn_ has joined #openstack-infra | 23:55 | |
| clarkb | davidlenwell: it was one of the benefits to the new system as compared to the old | 23:55 |
| davidlenwell | yes .. It hought so .. | 23:55 |
| davidlenwell | So the issue must be something else | 23:56 |
| *** dkehn has quit IRC | 23:56 | |
| clarkb | davidlenwell: do you have a specific failure? | 23:56 |
| davidlenwell | she's trying to commit a small change to openstack-planet and is getting "cannot read from remote repository" | 23:56 |
| davidlenwell | however git review -s doesn't get any errors | 23:56 |
| clarkb | davidlenwell: what does git branch -a say? | 23:56 |
| clarkb | er sorry | 23:56 |
| clarkb | git remote -v | 23:57 |
| *** dkehn has joined #openstack-infra | 23:57 | |
| davidlenwell | it shows the gerrit remotes | 23:57 |
| *** Ryan_Lane1 has quit IRC | 23:57 | |
| clarkb | right can you paste that output? it sounds like either you can't hit one of the remotes because it is down or wrong or being filtered | 23:58 |
| clarkb | we can at least check the first two pretty easily with some data | 23:58 |
| *** sarob has quit IRC | 23:58 | |
| davidlenwell | hang on .. its on another computer | 23:58 |
| *** yamahata has joined #openstack-infra | 23:59 | |
| davidlenwell | http://paste.openstack.org/show/85030/ | 23:59 |
| *** sarob has joined #openstack-infra | 23:59 | |
| davidlenwell | yes she made her username on gerrit Piston | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!