| *** atan8 has joined #openstack-infra | 00:00 | |
| *** dannywil_ has quit IRC | 00:00 | |
| sdague | um.... so interesting discovery | 00:02 |
|---|---|---|
| *** dannywilson has joined #openstack-infra | 00:02 | |
| krotscheck | …. Interesting? | 00:02 |
| sdague | pyeclib wgets files during it's installation | 00:02 |
| clarkb | sdague: its not the only one | 00:02 |
| sdague | really? we allow libs in that do that? | 00:02 |
| clarkb | sdague: but that was a particularly problematic package iirc. like they didn't know how make worked | 00:02 |
| clarkb | sdague: a lot of things that link against C will do the "if I don't find headers locally then get C lib from internet" | 00:03 |
| fungi | the authors think they are "helping" | 00:04 |
| sdague | helping wget an http site... hm | 00:04 |
| *** armax_ has joined #openstack-infra | 00:04 | |
| sdague | well, anyway, need to go do dinnery things | 00:04 |
| fungi | i can understand that going unnoticed during a requirements addition review | 00:04 |
| sdague | but we're pulling files from http://www.kaymgee.com/Kevin_Greenan/Software_files/ randomly during install | 00:04 |
| *** nijaba_ has joined #openstack-infra | 00:05 | |
| *** chlong has joined #openstack-infra | 00:05 | |
| *** Sukhdev has quit IRC | 00:05 | |
| clarkb | yes | 00:05 |
| clarkb | I remember giving them a hard time about it back when swift was working on it | 00:06 |
| fungi | i suppose we could set strict egress filters on the requirements integration job to only allow the sorts of network connections we expect that job to reasonably make | 00:06 |
| clarkb | particularly when they couldn't get make to work | 00:06 |
| clarkb | but the answer was 'well its not in the distros yet" | 00:06 |
| *** imcsk8_ has joined #openstack-infra | 00:06 | |
| *** Clint_ has joined #openstack-infra | 00:07 | |
| *** yjiang5_ has joined #openstack-infra | 00:07 | |
| sdague | fungi: that would be good | 00:07 |
| *** gyee has joined #openstack-infra | 00:07 | |
| *** sabeen2 has quit IRC | 00:07 | |
| *** yjiang5_ has left #openstack-infra | 00:07 | |
| clarkb | restrict to the mirror and jenkins would do it | 00:07 |
| *** changbl has quit IRC | 00:08 | |
| *** ivar-laz_ has quit IRC | 00:08 | |
| sdague | swift master doesn't reference pyeclib | 00:08 |
| sdague | what uses it? | 00:08 |
| clarkb | sdague: it may still be in a feature branch for them? | 00:08 |
| *** ivar-lazzaro has joined #openstack-infra | 00:08 | |
| clarkb | but I am 99% sure its for swift erasure coding | 00:08 |
| sdague | oh, still a feature branch apparently | 00:08 |
| *** rediskin has joined #openstack-infra | 00:09 | |
| *** yjiang5 has quit IRC | 00:09 | |
| *** imcsk8 has quit IRC | 00:09 | |
| *** shardy has quit IRC | 00:09 | |
| *** armax has quit IRC | 00:09 | |
| *** redixin has quit IRC | 00:09 | |
| *** nijaba has quit IRC | 00:09 | |
| *** Clint has quit IRC | 00:09 | |
| *** psedlak has quit IRC | 00:09 | |
| *** dstanek has quit IRC | 00:09 | |
| *** dstanek has joined #openstack-infra | 00:09 | |
| *** armax_ is now known as armax | 00:09 | |
| *** Clint_ is now known as Clint | 00:09 | |
| *** ivar-lazzaro has quit IRC | 00:09 | |
| clarkb | fungi: hpcloud stillseems to be going but slowly | 00:09 |
| *** nijaba_ has quit IRC | 00:09 | |
| *** nijaba_ has joined #openstack-infra | 00:09 | |
| clarkb | fungi: but notice how the uuids are all there in a list until a specific point. and if you relist that moves | 00:09 |
| sdague | ok, dinner time. | 00:10 |
| *** Clint has quit IRC | 00:10 | |
| *** Clint has joined #openstack-infra | 00:10 | |
| *** ivar-lazzaro has joined #openstack-infra | 00:10 | |
| *** psedlak has joined #openstack-infra | 00:10 | |
| *** Ryan_Lane has joined #openstack-infra | 00:10 | |
| clarkb | fungi: I think right now we might be limited by our serialization of requests there. So once it hits the bottom of the list it will loop around and start marking nodes ready | 00:10 |
| *** jraim has joined #openstack-infra | 00:10 | |
| clarkb | (this is my theory at least) | 00:10 |
| *** Longgeek has quit IRC | 00:10 | |
| *** shardy has joined #openstack-infra | 00:11 | |
| *** ZZelle_ has quit IRC | 00:14 | |
| *** stevemar has joined #openstack-infra | 00:17 | |
| *** shashankhegde has quit IRC | 00:17 | |
| *** simonmcc has joined #openstack-infra | 00:20 | |
| *** ctracey has joined #openstack-infra | 00:23 | |
| notmyname | clarkb: sdague: not 100% sure of the context, but yes pyeclib is for the EC feature branch (erasure codes) | 00:23 |
| clarkb | notmyname: I think the context is sdague is cleaning up the global requirements and noticed that pyeclib straight wgets stuff from http://www.kaymgee.com/Kevin_Greenan/Software_files/ | 00:24 |
| clarkb | notmyname: which makes the mirror less useful there and isn't super packaging friendly for distros | 00:25 |
| notmyname | ya. IIRC it's been added to debian. or at least there was some discussion like that | 00:25 |
| notmyname | zigo: ^ you were part of that, right? | 00:26 |
| *** arielb has quit IRC | 00:26 | |
| fungi | clarkb: sdague: i'm running a semi-automated analysis right now to see what might be able to get yanked out of global-requirements.txt (e.g. things not listed in any project in projects.txt's requirements.txt nor test-requirements.txt) | 00:26 |
| *** bswartz has joined #openstack-infra | 00:26 | |
| *** markmcclain has joined #openstack-infra | 00:26 | |
| clarkb | fungi: fwiw hpcloud seems more happy than before but not really happy | 00:26 |
| fungi | clarkb: not terribly surprised | 00:27 |
| clarkb | ya | 00:27 |
| *** markmcclain has quit IRC | 00:27 | |
| *** markmcclain1 has joined #openstack-infra | 00:27 | |
| *** stevemar has quit IRC | 00:27 | |
| *** david-lyle has joined #openstack-infra | 00:28 | |
| *** markmcclain1 has quit IRC | 00:29 | |
| mtreinish | clarkb: you mean like share it with the regular tempest-log? | 00:29 |
| *** markmcclain has joined #openstack-infra | 00:29 | |
| clarkb | mtreinish: ya they are both tempest.log aiui | 00:29 |
| clarkb | mtreinish: they just happen to be written to two different places | 00:29 |
| clarkb | fwiw I approved so meh | 00:30 |
| *** nfedotov has joined #openstack-infra | 00:30 | |
| mtreinish | clarkb: I just needed it for stack traces, because it's a side effect of the oslo.log defaults | 00:30 |
| fungi | and it merged | 00:30 |
| mtreinish | heh, ok | 00:30 |
| *** sambetts has joined #openstack-infra | 00:30 | |
| fungi | about 45 minutes ago | 00:30 |
| fungi | so you should be getting them on any new runs since that time | 00:31 |
| mtreinish | I was eating dumplings :) | 00:31 |
| fungi | i wish | 00:31 |
| fungi | dumplings sound gooood | 00:31 |
| fungi | soup dumplings? | 00:31 |
| mtreinish | no just regular pan fried | 00:31 |
| clarkb | I have fresh steelhead for dinner I am excited | 00:32 |
| clarkb | adam_g: ^ how did yours turn out? | 00:32 |
| * fungi suddenly has a craving for xiaolongbao | 00:32 | |
| mtreinish | fungi: https://www.zomato.com/auckland/nice-dumplings-wellesley-street-east/info#tabtop | 00:32 |
| jeblair | <drool> | 00:32 |
| jeblair | mtreinish: oh! li passed by that place twice and there's always a line | 00:33 |
| jeblair | which she took as a good thing | 00:33 |
| sambetts | Hi, I was wondering who I need to speak to, to get some more core attention on my patches for adding a couple of neutron plugin stackforge repos | 00:33 |
| fungi | oh that does look smart tasty | 00:33 |
| mtreinish | yeah it was really good | 00:33 |
| *** dizquierdo has quit IRC | 00:34 | |
| *** erikwilson has joined #openstack-infra | 00:34 | |
| *** erikwilson is now known as erikmwilson_ | 00:34 | |
| *** erikmwilson has joined #openstack-infra | 00:35 | |
| *** erikmwilson_ has quit IRC | 00:35 | |
| fungi | sambetts: we're a small team and split our time with keeping everything running, so get limited time to review but we do try to review more or less in the order that changes are proposed (modulo prioritizing emergency fix changes) | 00:35 |
| *** otter768 has joined #openstack-infra | 00:36 | |
| *** jvrbanac has quit IRC | 00:36 | |
| *** dkranz has quit IRC | 00:36 | |
| fungi | sambetts: we're still coming out of the holiday lull and also have a lot of people away at linuxconf au all this week, so it's slower going than usual | 00:36 |
| clarkb | and things are on fire because clouds | 00:37 |
| sambetts | fungi: sorry, I didn't mean to sound rude if I did, its just a blocker for me, and I already got a +2 on both | 00:38 |
| fungi | yes, we've had no shortage of broken things to triage unfortunately | 00:38 |
| fungi | sambetts: i understand. hopefully soon | 00:38 |
| *** tomoe has joined #openstack-infra | 00:38 | |
| sambetts | fungi: ok, thank you :-) | 00:39 |
| *** achanda has quit IRC | 00:39 | |
| *** EmilienM is now known as EmilienM|afk | 00:39 | |
| *** otter768 has quit IRC | 00:40 | |
| *** psedlak has quit IRC | 00:41 | |
| *** mattoliverau has quit IRC | 00:41 | |
| mtreinish | lifeless: ok, I updated the patch from this morning should have the correct indent from this morning. (sigh...) | 00:41 |
| *** mattoliverau has joined #openstack-infra | 00:42 | |
| clarkb | looking at a current nodepool stack dump we appear to be waiting on creating floating ips | 00:43 |
| clarkb | double checking that we haven't leaked those all over the place | 00:44 |
| clarkb | (note I don't see quota errors just waiting on those ips to show up | 00:44 |
| *** ameade has joined #openstack-infra | 00:44 | |
| clarkb | and the waiting is probabl because the manager queue is large and we haven't gotten to that point in the queue | 00:45 |
| fungi | i cleaned up about 40 disassociated floating ips in hpcloud earlier today | 00:45 |
| *** fandi has joined #openstack-infra | 00:45 | |
| *** jvrbanac has joined #openstack-infra | 00:45 | |
| *** Masahiro has joined #openstack-infra | 00:46 | |
| *** gyee has quit IRC | 00:46 | |
| *** psedlak has joined #openstack-infra | 00:46 | |
| *** achanda has joined #openstack-infra | 00:46 | |
| clarkb | we only have ~8 floating IPs so not going to be a quota issue there | 00:47 |
| clarkb | hpcloud managers last did things ~20 minute ago | 00:47 |
| clarkb | so now we see if keepalive unsticks them I guess | 00:47 |
| clarkb | queue was in the 90s | 00:48 |
| *** zhiyan has joined #openstack-infra | 00:48 | |
| *** markmcclain has quit IRC | 00:48 | |
| *** yjiang5 has joined #openstack-infra | 00:49 | |
| clarkb | I wonder if it is worth emailing support@hpcloud.(com|net) | 00:49 |
| fungi | i have 34 candidates for removal from global-requirements.txt on the master branch http://paste.openstack.org/show/157440 (note i haven't git blamed them yet to see if they're for open feature branches or pending changes) | 00:49 |
| fungi | sdague: ^ | 00:49 |
| clarkb | "You keep killing our connections make the novaclient sad" | 00:49 |
| *** ashp has quit IRC | 00:49 | |
| *** banix has joined #openstack-infra | 00:50 | |
| fungi | that's based on cloning all the projects listed in the requirements projects.txt file and then combining the requirements listed in their *requirements*.txt files | 00:51 |
| mordred | clarkb, sdague: where does pyeclib wget things? | 00:51 |
| fungi | mordred: during install | 00:51 |
| clarkb | mordred: in its setup.py aiui. it needs ot pull down the C lib it links against | 00:51 |
| * mordred is reading its setup.py ... | 00:51 | |
| clarkb | mordred: also any cahnce you want to talk to hpcloud? | 00:51 |
| clarkb | oh wait you are in auckland | 00:51 |
| clarkb | nevermind enjoy mordor | 00:51 |
| mordred | just doesn't see it - wants to see what I'm missing | 00:51 |
| mordred | clarkb: I can also talk to hpcloud if you need | 00:52 |
| jeblair | clarkb: you turned on keepalive? | 00:52 |
| *** rossella_s has quit IRC | 00:52 | |
| *** roaet_ has quit IRC | 00:52 | |
| *** EmilienM|afk has quit IRC | 00:52 | |
| clarkb | jeblair: ya, running nodepool with libkeepalive LD_PRELOAD'd | 00:52 |
| jeblair | oh, ha | 00:52 |
| clarkb | which in theory forces all the sockets to keepalive | 00:52 |
| *** sandywalsh_ has joined #openstack-infra | 00:54 | |
| jeblair | clarkb: and 20 mins ago, the network broke again? | 00:54 |
| *** jraim has quit IRC | 00:54 | |
| *** dougwig has quit IRC | 00:54 | |
| *** mmedvede has quit IRC | 00:54 | |
| *** rossella_s has joined #openstack-infra | 00:54 | |
| jeblair | clarkb: as for where the network problem is, i think it would be better if we knew it wasn't rax or intermediate transit.... | 00:55 |
| fungi | jeblair: a few hours ago when i restarted nodepoold it broke again within minutes | 00:55 |
| jeblair | clarkb: what's the keepalive timeout? (when sholud we expect an answer?) | 00:55 |
| *** mase_home has quit IRC | 00:55 | |
| jeblair | clarkb, fungi: at lunch, mark mcclain mentioned that if the server side is not configured to answer keepalives, we may still have a problem | 00:56 |
| *** nfedotov has quit IRC | 00:56 | |
| jeblair | this should help us get to that answer soon | 00:56 |
| clarkb | jeblair: 7200 seconds is kernel default | 00:56 |
| clarkb | so in about an hour or so we should know | 00:56 |
| clarkb | I think | 00:56 |
| clarkb | I should've reduced that value for libkeepalive | 00:56 |
| *** ashp has joined #openstack-infra | 00:56 | |
| jeblair | clarkb: would resetting the sysctl manually now work? | 00:56 |
| *** sandywalsh has quit IRC | 00:56 | |
| *** EmilienM has joined #openstack-infra | 00:56 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for nova quota KeyError bug 1410310 https://review.openstack.org/147041 | 00:57 |
| clarkb | jeblair: possibly? I am not sure if those values are dynamic with open sockets | 00:57 |
| uvirtbot | Launchpad bug 1410310 in nova "'headroom' KeyError while creating instance" [High,In progress] https://launchpad.net/bugs/1410310 | 00:57 |
| jeblair | clarkb: still might be faster to restart with new values, if the breaking is that reliable... otoh, disruption, so... whatever you think :) | 00:57 |
| *** markmcclain has joined #openstack-infra | 00:57 | |
| jeblair | swift is cool | 00:57 |
| clarkb | mordred: https://bitbucket.org/kmgreen2/pyeclib/src/d19cf85cdb40f06ab25b68304c189e6d71a7066e/c_eclib-0.9/install-c_eclib.sh?at=v0.9.8 wget is thee | 00:58 |
| *** markmcclain has quit IRC | 00:58 | |
| *** david-lyle has quit IRC | 00:58 | |
| *** dannywilson has quit IRC | 00:58 | |
| fungi | mordred: https://bitbucket.org/kmgreen2/pyeclib/src/d19cf85cdb40f06ab25b68304c189e6d71a7066e/setup.py?at=v0.9.8#cl-89 | 00:58 |
| *** markmcclain has joined #openstack-infra | 00:58 | |
| fungi | er, what clarkb just linked | 00:58 |
| *** david-lyle has joined #openstack-infra | 00:58 | |
| *** dougwig has joined #openstack-infra | 00:58 | |
| fungi | my link is where it's calling what he linked | 00:58 |
| *** dannywilson has joined #openstack-infra | 00:59 | |
| fungi | looks like that may be significantly different in the master branch, but this is the most recent tag | 00:59 |
| *** Ng has joined #openstack-infra | 00:59 | |
| clarkb | fungi: yup | 00:59 |
| clarkb | jeblair: mabe? its getting late enough in the day where low clarkb overhead is good | 00:59 |
| clarkb | I have steelhead to cook in a bit | 00:59 |
| jeblair | clarkb: rgr. i am definitely not in a position to help | 00:59 |
| *** sputnik13 has quit IRC | 01:00 | |
| *** markmcclain has quit IRC | 01:00 | |
| *** jraim has joined #openstack-infra | 01:00 | |
| *** bhuvan has joined #openstack-infra | 01:01 | |
| *** mohitsharma has joined #openstack-infra | 01:01 | |
| *** roaet_ has joined #openstack-infra | 01:01 | |
| lifeless | mtreinish: :) | 01:01 |
| lifeless | thanks | 01:01 |
| *** pc_m has quit IRC | 01:02 | |
| fungi | i am in a position to sort of check out for the evening and hope that it maybe fixes itself | 01:02 |
| *** dannywilson has quit IRC | 01:03 | |
| *** gyee has joined #openstack-infra | 01:03 | |
| *** maxbit has joined #openstack-infra | 01:04 | |
| *** david-lyle has quit IRC | 01:05 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 01:05 | |
| *** TravT_ has joined #openstack-infra | 01:06 | |
| *** mmedvede has joined #openstack-infra | 01:06 | |
| *** rossella_s has quit IRC | 01:07 | |
| *** ihrachyshka has quit IRC | 01:07 | |
| *** EmilienM has quit IRC | 01:07 | |
| *** gyee has quit IRC | 01:08 | |
| *** EmilienM has joined #openstack-infra | 01:08 | |
| mordred | clarkb: that doesn't seem to be in trunk, fwiw | 01:08 |
| *** TravT has quit IRC | 01:08 | |
| *** gyee has joined #openstack-infra | 01:08 | |
| clarkb | mordred: but trunk doesn't matter since we install from pypi | 01:08 |
| *** rossella_s has joined #openstack-infra | 01:08 | |
| mordred | clarkb: agree - just making sure I grok the world | 01:08 |
| *** david-lyle has joined #openstack-infra | 01:09 | |
| mordred | clarkb: the "fix" seems to be to tell you to get the lib from git if you don't have it | 01:09 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 01:11 | |
| *** jraim has quit IRC | 01:13 | |
| *** shakamunyi has quit IRC | 01:13 | |
| *** banix has quit IRC | 01:14 | |
| timrc | clarkb, Hi, so you are also seeing hash mismatches with pypi installs? | 01:15 |
| clarkb | timrc: we were in the past I don't think that is a major current current issue | 01:16 |
| clarkb | timrc: http://status.openstack.org/elastic-recheck/#1343313 | 01:16 |
| notmyname | jeblair: swift is cool! | 01:16 |
| *** mbacchi has quit IRC | 01:17 | |
| timrc | clarkb, Thanks, sir. | 01:18 |
| timrc | clarkb, We're seeing it in a check-requirements-integration-dsvm with Django :( | 01:18 |
| *** shayneburgess has quit IRC | 01:18 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for nova quota KeyError bug 1410310 https://review.openstack.org/147041 | 01:18 |
| uvirtbot | Launchpad bug 1410310 in nova "'headroom' KeyError while creating instance" [High,In progress] https://launchpad.net/bugs/1410310 | 01:18 |
| *** camunoz has quit IRC | 01:19 | |
| *** jraim has joined #openstack-infra | 01:19 | |
| clarkb | timrc: working theory at the time is that it was related to neutron router problems | 01:19 |
| clarkb | timrc: noisy neighbors or similar | 01:19 |
| *** rmcall has quit IRC | 01:20 | |
| timrc | clarkb, Aye. Okay. Thanks for the info | 01:21 |
| *** pc_m has joined #openstack-infra | 01:21 | |
| timrc | clarkb, DId you have to change routers or did it magically go away? | 01:21 |
| clarkb | timrc: it magically went away | 01:21 |
| *** sabari is now known as zz_sabari | 01:21 | |
| *** rwsu has quit IRC | 01:21 | |
| clarkb | timrc: we were also able to confirm that the apache on the mirror side claimed to write X bytes less than Y bytes for the package and client side saw 0 bytes | 01:21 |
| *** achanda has quit IRC | 01:23 | |
| *** TravT_ has left #openstack-infra | 01:23 | |
| fungi | also worth pointing out the pip error message there is misleading, and almost certainly masking a connection reset or other premature disconnect | 01:24 |
| *** david-lyle has quit IRC | 01:24 | |
| fungi | hopefully easier to diagnose in the next release | 01:25 |
| *** afazekas has quit IRC | 01:25 | |
| clarkb | ya I pushed a patch to pip that merged | 01:26 |
| *** bhuvan has quit IRC | 01:27 | |
| timrc | clarkb, What if YOU are our noisy neighbor... :P HM | 01:27 |
| timrc | fungi, That is pretty crazy if a connection reset is getting masked by a connection reset. | 01:27 |
| clarkb | timrc none of our nodes build right now so unlikely | 01:28 |
| timrc | er | 01:28 |
| *** shashankhegde has joined #openstack-infra | 01:28 | |
| timrc | er getting masked by a hashsum mismatch | 01:28 |
| *** ivar-lazzaro has quit IRC | 01:28 | |
| jogo | hmm nova dash crashes chrome now | 01:28 |
| *** ivar-lazzaro has joined #openstack-infra | 01:29 | |
| cody-somerville | clarkb: What do you mean by "none of our nodes build right now"? | 01:29 |
| clarkb | timrc connection reset meant hash was wrong. that exception was raised instead of short circuiting on connection fail | 01:29 |
| *** david-lyle has joined #openstack-infra | 01:29 | |
| clarkb | I mean all ~500 nodes we are trying to build in hpcloud fail to finish because the provider manager quues back up on tcp connections that appear to die | 01:30 |
| clarkb | we added keepalives and should know if that helps in a little while | 01:30 |
| *** jyuso has quit IRC | 01:32 | |
| grue_pm | clarkb: so those keepalives may help our issue to? | 01:33 |
| *** plaurin_ has joined #openstack-infra | 01:33 | |
| clarkb | maybe? its not entirely clear this will fix it | 01:33 |
| *** Sukhdev has joined #openstack-infra | 01:34 | |
| jogo | sdague: you have seen something like this in the past right? a dashboard causing chrome to become unresponsive | 01:34 |
| *** ddieterly has joined #openstack-infra | 01:35 | |
| *** david-lyle has quit IRC | 01:35 | |
| Sukhdev | Folks, how do I force a re-run of jenkins check. "recheck no bug" does not kick of the run. | 01:36 |
| clarkb | that should work | 01:36 |
| clarkb | did you check zuul status? | 01:37 |
| *** markmcclain has joined #openstack-infra | 01:37 | |
| *** jerryz has joined #openstack-infra | 01:37 | |
| *** dtroyer has quit IRC | 01:37 | |
| *** markmcclain has quit IRC | 01:38 | |
| *** markmcclain has joined #openstack-infra | 01:38 | |
| *** banix has joined #openstack-infra | 01:38 | |
| *** salv-orlando has quit IRC | 01:41 | |
| *** camunoz has joined #openstack-infra | 01:42 | |
| *** jyuso has joined #openstack-infra | 01:46 | |
| Sukhdev | Folks, I want to re-trigger the jenkins run on this - https://review.openstack.org/#/c/146244/ - "recheck no bug" did not do it. Any idea how to force the run? | 01:46 |
| *** maxbit has quit IRC | 01:46 | |
| *** maxbit has joined #openstack-infra | 01:47 | |
| clarkb | tests ran for that change after the latest patxh was pushed | 01:48 |
| *** TravT has joined #openstack-infra | 01:49 | |
| Sukhdev | clarkb: were you responding to my question? | 01:50 |
| clarkb | ye | 01:50 |
| Sukhdev | clarkb: I purposely made a change to commit message to force the run because "recheck no bug" did not force the run | 01:51 |
| grue_pm | clarkb: some of us are waiting with baited breath for the keepalive success ;-p | 01:51 |
| clarkb | Sukhdev: they ran | 01:51 |
| *** david-lyle has joined #openstack-infra | 01:52 | |
| clarkb | grue_pm: we are at 1.99 hours and 2 hours is when the kernel defaults kick in | 01:52 |
| clarkb | grue_pm: so we should know in about 10 minutes at 9*75seconds ~= 10minutes | 01:52 |
| grue_pm | clarkb: woot! | 01:52 |
| *** EmilienM has quit IRC | 01:52 | |
| clarkb | Sukhdev: if you hit the toggle ci button at the bottom of the change page you can see that they ran | 01:52 |
| clarkb | Sukhdev: and `recheck` should make them run too | 01:53 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 01:53 | |
| *** rossella_s has quit IRC | 01:53 | |
| clarkb | but they will be cancelled if you push a new patchset so likely what happened there | 01:53 |
| *** TravT has quit IRC | 01:53 | |
| Sukhdev | clarkb: I just did a recheck - let me go check | 01:53 |
| *** dtroyer has joined #openstack-infra | 01:55 | |
| *** Longgeek has joined #openstack-infra | 01:55 | |
| Sukhdev | clarkb: You are correct - my bad. I was looking at the wrong place. Thanks | 01:55 |
| *** EmilienM has joined #openstack-infra | 01:56 | |
| *** rossella_s has joined #openstack-infra | 01:56 | |
| *** maxbit has quit IRC | 01:57 | |
| *** banix has quit IRC | 01:57 | |
| *** Guest85585 is now known as mfisch | 01:59 | |
| *** david-lyle has quit IRC | 01:59 | |
| *** markmcclain has quit IRC | 01:59 | |
| *** mfisch has quit IRC | 01:59 | |
| *** mfisch has joined #openstack-infra | 01:59 | |
| *** pc_m has quit IRC | 02:00 | |
| *** salv-orlando has joined #openstack-infra | 02:00 | |
| *** ayoung has joined #openstack-infra | 02:01 | |
| *** amrith is now known as _amrith_ | 02:02 | |
| *** dims__ has quit IRC | 02:02 | |
| *** dims__ has joined #openstack-infra | 02:03 | |
| *** baoli has quit IRC | 02:04 | |
| *** yaguang has joined #openstack-infra | 02:05 | |
| *** zhiyan has quit IRC | 02:05 | |
| *** mmedvede has quit IRC | 02:06 | |
| *** masayukig has quit IRC | 02:06 | |
| *** zhiyan has joined #openstack-infra | 02:06 | |
| clarkb | not seeing new task getting executed by the provider manages | 02:06 |
| clarkb | so keepalive may not solve all the problems | 02:07 |
| *** roaet_ has quit IRC | 02:07 | |
| *** rainya has quit IRC | 02:07 | |
| *** Masahiro has quit IRC | 02:07 | |
| *** masayukig has joined #openstack-infra | 02:07 | |
| *** dims__ has quit IRC | 02:07 | |
| clarkb | mordred: ^ so talking to hpcloud may be useful | 02:08 |
| *** dtroyer has quit IRC | 02:08 | |
| adam_g | clarkb, wife is cooking some miso marinated steelhead thing now. i baked and ate the collar after she filleted them that night, it was good | 02:08 |
| *** mmedvede has joined #openstack-infra | 02:08 | |
| clarkb | nice. | 02:08 |
| *** gyee has quit IRC | 02:08 | |
| clarkb | adam_g: rumor has it sunday was good too? was it a natve? | 02:09 |
| *** dtroyer has joined #openstack-infra | 02:09 | |
| *** samueldmq has quit IRC | 02:09 | |
| adam_g | clarkb yeah, it was all beat up and old looking. and not huge. probably similar to the downriver one you got | 02:10 |
| adam_g | http://no-carrier.net/~adam/steelhead.jpg | 02:10 |
| clarkb | I think that one is bigger. the downriver one I got was really tiny | 02:11 |
| clarkb | barely not trout | 02:11 |
| *** Longgeek has quit IRC | 02:11 | |
| greghaynes | I still like that clarkbro outfished everyone | 02:11 |
| *** vishy has quit IRC | 02:11 | |
| clarkb | greghaynes: I had to grow up with it so it was somewhat expected | 02:11 |
| *** rainya has joined #openstack-infra | 02:12 | |
| *** baoli has joined #openstack-infra | 02:12 | |
| *** roaet_ has joined #openstack-infra | 02:13 | |
| *** cyeoh has quit IRC | 02:14 | |
| *** sarob has quit IRC | 02:14 | |
| grue_pm | clarkb could it be some sort of possibly interemittent issue pypi end? | 02:14 |
| *** banix has joined #openstack-infra | 02:14 | |
| *** vishy has joined #openstack-infra | 02:14 | |
| *** thingee has quit IRC | 02:15 | |
| *** cyeoh has joined #openstack-infra | 02:15 | |
| *** ChuckC_ has joined #openstack-infra | 02:16 | |
| clarkb | no our issue is unrelated to pypi | 02:16 |
| adam_g | clarkb, greghaynes should be getting footage/pics from guide soon | 02:16 |
| grue_pm | clarkb: ah okay I'm probably a bit confused - I thought it was realted to what tim was saying earlier | 02:16 |
| greghaynes | adam_g: \O/ | 02:17 |
| *** Rockyg has quit IRC | 02:18 | |
| *** salv-orlando has quit IRC | 02:19 | |
| *** mriedem1 has joined #openstack-infra | 02:20 | |
| *** samueldmq has joined #openstack-infra | 02:20 | |
| *** ChuckC has quit IRC | 02:20 | |
| *** erikmwilson has quit IRC | 02:21 | |
| *** mriedem has quit IRC | 02:22 | |
| *** EmilienM has quit IRC | 02:22 | |
| *** zz_sabari is now known as sabari | 02:23 | |
| *** dtroyer has quit IRC | 02:23 | |
| *** masayukig has quit IRC | 02:23 | |
| *** EmilienM has joined #openstack-infra | 02:23 | |
| *** julim has quit IRC | 02:24 | |
| *** masayukig has joined #openstack-infra | 02:24 | |
| *** dtroyer has joined #openstack-infra | 02:25 | |
| *** mtanino has quit IRC | 02:25 | |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config: Add networking-cisco-ml2-nexus to stackforge https://review.openstack.org/145793 | 02:25 |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config: Add networking-cisco-ml2-apic to stackforge https://review.openstack.org/145928 | 02:25 |
| *** hdd has quit IRC | 02:26 | |
| *** Masahiro has joined #openstack-infra | 02:28 | |
| *** mwagner_lap has quit IRC | 02:29 | |
| timrc | adam_g, Do you flyfish? | 02:30 |
| *** Masahiro has quit IRC | 02:30 | |
| timrc | clarkb too^ | 02:30 |
| *** Masahiro has joined #openstack-infra | 02:30 | |
| clarkb | nope thats work | 02:31 |
| greghaynes | haha | 02:31 |
| *** yamahata has quit IRC | 02:31 | |
| timrc | clarkb, So rewarding :) | 02:31 |
| grue_pm | timrc: are you pushing for 96 hours without sleep now ;-p | 02:31 |
| *** Masahiro has quit IRC | 02:34 | |
| timrc | grue_pm, Nah. | 02:34 |
| *** Masahiro has joined #openstack-infra | 02:34 | |
| * grue_pm hands timrc a fish to sleep on. | 02:35 | |
| *** wuhg has joined #openstack-infra | 02:35 | |
| *** baoli has quit IRC | 02:36 | |
| *** otter768 has joined #openstack-infra | 02:37 | |
| *** Masahiro has quit IRC | 02:38 | |
| *** mmedvede has quit IRC | 02:38 | |
| *** ivar-laz_ has joined #openstack-infra | 02:39 | |
| *** Masahiro has joined #openstack-infra | 02:40 | |
| *** david-lyle has joined #openstack-infra | 02:40 | |
| *** mmedvede has joined #openstack-infra | 02:41 | |
| *** otter768 has quit IRC | 02:41 | |
| *** mwagner_lap has joined #openstack-infra | 02:41 | |
| *** shakamunyi has joined #openstack-infra | 02:41 | |
| *** markmcclain has joined #openstack-infra | 02:42 | |
| *** ivar-lazzaro has quit IRC | 02:42 | |
| *** markmcclain has quit IRC | 02:42 | |
| *** markmcclain has joined #openstack-infra | 02:42 | |
| *** ivar-laz_ has quit IRC | 02:43 | |
| *** stevemar has joined #openstack-infra | 02:46 | |
| *** harlowja is now known as harlowja_away | 02:46 | |
| *** sc68cal has quit IRC | 02:46 | |
| *** zz_dimtruck has quit IRC | 02:46 | |
| *** zz_dimtruck has joined #openstack-infra | 02:46 | |
| *** sambetts has quit IRC | 02:47 | |
| *** jbryce has quit IRC | 02:47 | |
| *** zz_dimtruck is now known as dimtruck | 02:47 | |
| *** sc68cal has joined #openstack-infra | 02:47 | |
| *** Ryan_Lane has quit IRC | 02:47 | |
| *** cyeoh has quit IRC | 02:47 | |
| *** shakamunyi has quit IRC | 02:47 | |
| *** lbragstad has quit IRC | 02:47 | |
| *** mgagne has quit IRC | 02:47 | |
| *** russellb has quit IRC | 02:47 | |
| *** cyeoh has joined #openstack-infra | 02:48 | |
| *** shakamunyi has joined #openstack-infra | 02:48 | |
| *** rlandy has quit IRC | 02:48 | |
| *** otherwiseguy has quit IRC | 02:48 | |
| *** pabelanger has quit IRC | 02:48 | |
| *** hughsaunders has quit IRC | 02:48 | |
| *** StevenK has quit IRC | 02:48 | |
| *** krtaylor has quit IRC | 02:48 | |
| *** rm_work has quit IRC | 02:48 | |
| *** stevebaker has quit IRC | 02:48 | |
| *** koolhead17 has joined #openstack-infra | 02:48 | |
| *** camunoz has quit IRC | 02:49 | |
| *** markmcclain has quit IRC | 02:49 | |
| *** koolhead17 has quit IRC | 02:49 | |
| *** markmcclain has joined #openstack-infra | 02:49 | |
| *** koolhead17 has joined #openstack-infra | 02:50 | |
| *** koolhead17 has joined #openstack-infra | 02:50 | |
| *** masayukig has quit IRC | 02:50 | |
| *** ctracey has quit IRC | 02:52 | |
| *** rm_work has joined #openstack-infra | 02:53 | |
| *** stevebaker has joined #openstack-infra | 02:53 | |
| *** jbryce has joined #openstack-infra | 02:53 | |
| *** mmedvede has quit IRC | 02:53 | |
| *** shayneburgess has joined #openstack-infra | 02:55 | |
| *** zhiyan has quit IRC | 02:55 | |
| *** lbragstad has joined #openstack-infra | 02:55 | |
| *** hughsaunders has joined #openstack-infra | 02:55 | |
| *** StevenK has joined #openstack-infra | 02:57 | |
| *** zhiyan has joined #openstack-infra | 02:57 | |
| *** ctracey has joined #openstack-infra | 02:58 | |
| *** pabelanger has joined #openstack-infra | 02:59 | |
| *** jroll has quit IRC | 03:00 | |
| *** ChuckC has joined #openstack-infra | 03:00 | |
| *** patrickeast has quit IRC | 03:01 | |
| *** krtaylor has joined #openstack-infra | 03:01 | |
| *** camunoz has joined #openstack-infra | 03:01 | |
| *** masayukig has joined #openstack-infra | 03:02 | |
| *** russellb has joined #openstack-infra | 03:02 | |
| *** rlandy has joined #openstack-infra | 03:03 | |
| *** mgagne has joined #openstack-infra | 03:04 | |
| *** ChuckC_ has quit IRC | 03:04 | |
| *** mgagne is now known as Guest33025 | 03:05 | |
| *** shakamunyi has quit IRC | 03:05 | |
| *** shashankhegde has quit IRC | 03:06 | |
| *** rainya has quit IRC | 03:07 | |
| *** romcheg has quit IRC | 03:08 | |
| *** lbragstad has quit IRC | 03:08 | |
| *** lbragstad has joined #openstack-infra | 03:10 | |
| *** jroll has joined #openstack-infra | 03:10 | |
| *** rainya has joined #openstack-infra | 03:11 | |
| *** mmedvede has joined #openstack-infra | 03:12 | |
| *** mohitsharma has quit IRC | 03:13 | |
| *** shashankhegde has joined #openstack-infra | 03:13 | |
| *** fandi has quit IRC | 03:14 | |
| *** russellb has quit IRC | 03:15 | |
| *** pabelanger has quit IRC | 03:17 | |
| *** dimtruck has quit IRC | 03:17 | |
| *** ashp has quit IRC | 03:17 | |
| *** otherwiseguy has joined #openstack-infra | 03:17 | |
| *** Sukhdev has quit IRC | 03:18 | |
| *** jraim has quit IRC | 03:18 | |
| *** plaurin_ has quit IRC | 03:18 | |
| *** ddieterly has quit IRC | 03:19 | |
| *** roaet_ has quit IRC | 03:19 | |
| *** roaet_ has joined #openstack-infra | 03:19 | |
| *** jraim has joined #openstack-infra | 03:19 | |
| *** pabelanger has joined #openstack-infra | 03:20 | |
| *** mohitsharma has joined #openstack-infra | 03:20 | |
| *** yamamoto_ has joined #openstack-infra | 03:21 | |
| *** zz_dimtruck has joined #openstack-infra | 03:23 | |
| *** zz_dimtruck is now known as dimtruck | 03:24 | |
| *** yamahata has joined #openstack-infra | 03:24 | |
| *** david-lyle has quit IRC | 03:25 | |
| *** russellb has joined #openstack-infra | 03:25 | |
| *** jroll has quit IRC | 03:26 | |
| *** ashp has joined #openstack-infra | 03:26 | |
| *** markmcclain has quit IRC | 03:28 | |
| *** miqui has joined #openstack-infra | 03:29 | |
| *** masayukig has quit IRC | 03:30 | |
| *** ayoung has quit IRC | 03:31 | |
| *** jroll has joined #openstack-infra | 03:32 | |
| *** roaet_ has quit IRC | 03:32 | |
| * fungi can't wait to decide on a house so he can have a dock (and a bunch of crab traps) | 03:33 | |
| fungi | much prefer the saltwater fare here to the freshwater catches i grew up with in the mountains | 03:33 |
| openstackgerrit | Tushar Gohad proposed openstack/requirements: Bump eventlet version to 0.16.1 https://review.openstack.org/145816 | 03:35 |
| *** dimtruck has quit IRC | 03:36 | |
| *** dtroyer has quit IRC | 03:36 | |
| *** shayneburgess has quit IRC | 03:37 | |
| *** russellb has quit IRC | 03:38 | |
| *** lbragstad has quit IRC | 03:38 | |
| *** atan8 has quit IRC | 03:39 | |
| *** dtroyer has joined #openstack-infra | 03:39 | |
| *** russellb has joined #openstack-infra | 03:39 | |
| *** masayukig has joined #openstack-infra | 03:40 | |
| *** roaet_ has joined #openstack-infra | 03:40 | |
| *** koolhead17 has quit IRC | 03:40 | |
| *** EmilienM has quit IRC | 03:40 | |
| *** rossella_s has quit IRC | 03:40 | |
| *** comstud has quit IRC | 03:40 | |
| *** comstud has joined #openstack-infra | 03:41 | |
| *** ayoung has joined #openstack-infra | 03:43 | |
| *** zz_dimtruck has joined #openstack-infra | 03:43 | |
| *** zz_dimtruck is now known as dimtruck | 03:43 | |
| *** salv-orlando has joined #openstack-infra | 03:43 | |
| *** rossella_s has joined #openstack-infra | 03:43 | |
| *** EmilienM has joined #openstack-infra | 03:44 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for nova bug 1410626 https://review.openstack.org/147060 | 03:44 |
| uvirtbot | Launchpad bug 1410626 in nova "AttributeError: 'module' object has no attribute 'LibvirtDriver'" [Undecided,New] https://launchpad.net/bugs/1410626 | 03:44 |
| *** jraim has quit IRC | 03:45 | |
| *** Masahiro has quit IRC | 03:45 | |
| *** simonmcc has quit IRC | 03:46 | |
| *** jraim has joined #openstack-infra | 03:47 | |
| *** lbragstad has joined #openstack-infra | 03:47 | |
| *** simonmcc has joined #openstack-infra | 03:48 | |
| *** weshay has quit IRC | 03:48 | |
| *** dhritishikhar has joined #openstack-infra | 03:49 | |
| *** ddieterly has joined #openstack-infra | 03:49 | |
| *** Longgeek has joined #openstack-infra | 03:51 | |
| *** dtroyer has quit IRC | 03:52 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for cinder unit test bug 1410485 https://review.openstack.org/147061 | 03:53 |
| uvirtbot | Launchpad bug 1410485 in cinder "cinder unit tests currently failing with due to missing required option lock_path" [Critical,Fix committed] https://launchpad.net/bugs/1410485 | 03:53 |
| *** amotoki has joined #openstack-infra | 03:53 | |
| *** sabari is now known as zz_sabari | 03:54 | |
| *** ddieterly has quit IRC | 03:54 | |
| *** pabelanger has quit IRC | 03:54 | |
| *** patrickeast has joined #openstack-infra | 03:56 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for nova bug 1410626 https://review.openstack.org/147060 | 03:56 |
| uvirtbot | Launchpad bug 1410626 in nova "AttributeError: 'module' object has no attribute 'LibvirtDriver'" [Undecided,New] https://launchpad.net/bugs/1410626 | 03:56 |
| *** yaguang has quit IRC | 03:56 | |
| *** asselin_ has joined #openstack-infra | 03:58 | |
| *** dtroyer has joined #openstack-infra | 03:58 | |
| *** dannywilson has joined #openstack-infra | 03:59 | |
| *** dhritishikhar has quit IRC | 03:59 | |
| *** pabelanger has joined #openstack-infra | 04:00 | |
| *** spzala has joined #openstack-infra | 04:00 | |
| *** lbragstad has quit IRC | 04:00 | |
| *** rossella_s has quit IRC | 04:00 | |
| *** ashp has quit IRC | 04:00 | |
| *** rlandy has quit IRC | 04:00 | |
| mriedem1 | fungi: you can move to iowa and fish for bullheads all summer with my 93 year old gramps, he loves him some bullheads | 04:00 |
| *** bhunter71 has joined #openstack-infra | 04:01 | |
| *** avozza is now known as zz_avozza | 04:01 | |
| *** EmilienM has quit IRC | 04:01 | |
| mriedem1 | there are at least 3 things in that sentence that can sell you on the idea | 04:01 |
| *** nijaba_ has quit IRC | 04:01 | |
| *** Masahiro has joined #openstack-infra | 04:02 | |
| *** Ryan_Lane has joined #openstack-infra | 04:02 | |
| *** lbragstad has joined #openstack-infra | 04:02 | |
| *** salv-orlando has quit IRC | 04:02 | |
| *** ashp has joined #openstack-infra | 04:03 | |
| *** dannywilson has quit IRC | 04:03 | |
| *** mriedem1 has quit IRC | 04:04 | |
| *** roaet_ has quit IRC | 04:06 | |
| *** dimtruck has quit IRC | 04:07 | |
| *** nijaba has joined #openstack-infra | 04:07 | |
| *** nijaba has joined #openstack-infra | 04:07 | |
| *** rossella_s has joined #openstack-infra | 04:07 | |
| *** mohitsharma has quit IRC | 04:08 | |
| *** EmilienM has joined #openstack-infra | 04:08 | |
| *** markmcclain has joined #openstack-infra | 04:09 | |
| *** markmcclain has quit IRC | 04:10 | |
| *** markmcclain1 has joined #openstack-infra | 04:10 | |
| *** mohitsharma has joined #openstack-infra | 04:10 | |
| *** mohitsharma has quit IRC | 04:11 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 04:11 | |
| *** mohitsharma has joined #openstack-infra | 04:11 | |
| *** dtroyer has quit IRC | 04:13 | |
| jogo | so stable/juno us broken | 04:14 |
| jogo | making grenade broken and the gate | 04:14 |
| jogo | https://review.openstack.org/#/c/147062/1 should fix it | 04:14 |
| jogo | clarkb fungi: ^ | 04:14 |
| *** zz_dimtruck has joined #openstack-infra | 04:16 | |
| *** zz_dimtruck is now known as dimtruck | 04:16 | |
| *** koolhead17 has joined #openstack-infra | 04:16 | |
| *** roaet_ has joined #openstack-infra | 04:17 | |
| adam_g | icehouse, too: https://review.openstack.org/#/q/I4bbbeb5bf9c22ed36f5c9a74fec6b487d2c15697,n,z | 04:18 |
| jogo | who has stable global reqs core anyway? | 04:19 |
| *** masayukig has quit IRC | 04:19 | |
| *** mmedvede has quit IRC | 04:20 | |
| *** dtroyer has joined #openstack-infra | 04:21 | |
| *** masayukig has joined #openstack-infra | 04:24 | |
| *** mmedvede has joined #openstack-infra | 04:24 | |
| *** shashankhegde has quit IRC | 04:25 | |
| *** mohitsharma has quit IRC | 04:25 | |
| *** mohitsharma has joined #openstack-infra | 04:25 | |
| *** david-lyle has joined #openstack-infra | 04:26 | |
| *** mohitsharma has quit IRC | 04:26 | |
| *** david-lyle_ has joined #openstack-infra | 04:27 | |
| *** kedkul has joined #openstack-infra | 04:29 | |
| *** doug-fish has left #openstack-infra | 04:29 | |
| kedkul | Hi, on my CI setup, all builds are failing because /etc/mysql/my.cnf is not present. | 04:30 |
| kedkul | it started happening since yesterday | 04:30 |
| *** david-lyle has quit IRC | 04:31 | |
| kedkul | /home/jenkins/workspace/openstack/functions-common: line 178: /etc/mysql/my.cnf: No such file or directory | 04:31 |
| *** david-lyle_ has quit IRC | 04:31 | |
| *** david-lyle_ has joined #openstack-infra | 04:32 | |
| *** Guest33025 has quit IRC | 04:32 | |
| kedkul | can someone help me? | 04:33 |
| *** rossella_s has quit IRC | 04:34 | |
| *** mgagne has joined #openstack-infra | 04:34 | |
| *** EmilienM has quit IRC | 04:35 | |
| *** mgagne is now known as Guest57605 | 04:35 | |
| *** salv-orlando has joined #openstack-infra | 04:35 | |
| *** dimtruck has quit IRC | 04:35 | |
| *** zz_dimtruck has joined #openstack-infra | 04:37 | |
| *** zz_dimtruck is now known as dimtruck | 04:37 | |
| *** otter768 has joined #openstack-infra | 04:37 | |
| *** hichtakk has quit IRC | 04:38 | |
| *** rossella_s has joined #openstack-infra | 04:38 | |
| *** yamahata has quit IRC | 04:38 | |
| *** EmilienM has joined #openstack-infra | 04:38 | |
| *** hichtakk has joined #openstack-infra | 04:39 | |
| *** hichtakk has quit IRC | 04:39 | |
| *** TravT has joined #openstack-infra | 04:39 | |
| *** salv-orlando has quit IRC | 04:39 | |
| *** yamahata has joined #openstack-infra | 04:39 | |
| *** hichtakk has joined #openstack-infra | 04:39 | |
| *** hichtakk has quit IRC | 04:40 | |
| *** hichtakk has joined #openstack-infra | 04:41 | |
| *** hichtakk has quit IRC | 04:41 | |
| *** hichtakk has joined #openstack-infra | 04:42 | |
| *** hichtakk has quit IRC | 04:42 | |
| *** Longgeek has quit IRC | 04:42 | |
| *** otter768 has quit IRC | 04:42 | |
| *** hichtakk has joined #openstack-infra | 04:42 | |
| *** hichtakk has quit IRC | 04:42 | |
| *** hichtakk has joined #openstack-infra | 04:43 | |
| *** koolhead17 has quit IRC | 04:44 | |
| *** Longgeek has joined #openstack-infra | 04:44 | |
| *** markmcclain1 has quit IRC | 04:45 | |
| *** changbl has joined #openstack-infra | 04:46 | |
| *** david-lyle_ has quit IRC | 04:51 | |
| *** jvrbanac has quit IRC | 04:51 | |
| *** dims__ has joined #openstack-infra | 04:52 | |
| *** yaguang has joined #openstack-infra | 04:53 | |
| *** jvrbanac has joined #openstack-infra | 04:54 | |
| *** spzala has quit IRC | 04:55 | |
| *** masayukig has quit IRC | 04:56 | |
| *** patrickeast has quit IRC | 04:57 | |
| *** dims__ has quit IRC | 04:57 | |
| *** dimtruck has quit IRC | 04:57 | |
| *** mmedvede has quit IRC | 04:57 | |
| *** vishy has quit IRC | 04:57 | |
| *** banix has quit IRC | 04:58 | |
| *** masayukig has joined #openstack-infra | 04:59 | |
| *** koolhead17 has joined #openstack-infra | 04:59 | |
| *** mohitsharma has joined #openstack-infra | 04:59 | |
| *** jamespage has quit IRC | 05:00 | |
| *** gnuoy has quit IRC | 05:00 | |
| *** gnuoy has joined #openstack-infra | 05:00 | |
| *** mase_home has joined #openstack-infra | 05:02 | |
| *** vishy has joined #openstack-infra | 05:03 | |
| *** mmedvede has joined #openstack-infra | 05:03 | |
| *** zz_dimtruck has joined #openstack-infra | 05:04 | |
| *** zz_dimtruck is now known as dimtruck | 05:04 | |
| *** teran__ has quit IRC | 05:04 | |
| *** asselin_ has quit IRC | 05:10 | |
| *** Sukhdev_ has joined #openstack-infra | 05:11 | |
| *** masayukig has quit IRC | 05:11 | |
| *** EmilienM has quit IRC | 05:12 | |
| *** rossella_s has quit IRC | 05:12 | |
| *** otherwiseguy has quit IRC | 05:12 | |
| *** rainya has quit IRC | 05:12 | |
| *** masayukig has joined #openstack-infra | 05:14 | |
| *** sarob has joined #openstack-infra | 05:15 | |
| *** otherwiseguy has joined #openstack-infra | 05:16 | |
| *** rossella_s has joined #openstack-infra | 05:16 | |
| *** EmilienM has joined #openstack-infra | 05:16 | |
| *** dimtruck has quit IRC | 05:17 | |
| *** mmedvede has quit IRC | 05:17 | |
| *** zz_dimtruck has joined #openstack-infra | 05:18 | |
| *** zz_dimtruck is now known as dimtruck | 05:18 | |
| *** praveenkumar_ is now known as praveenkumar | 05:19 | |
| *** sarob has quit IRC | 05:19 | |
| *** ddieterly has joined #openstack-infra | 05:19 | |
| *** rainya has joined #openstack-infra | 05:19 | |
| *** zigo has quit IRC | 05:20 | |
| *** roaet_ has quit IRC | 05:21 | |
| *** zigo has joined #openstack-infra | 05:21 | |
| *** dannywilson has joined #openstack-infra | 05:21 | |
| *** hichtakk has quit IRC | 05:23 | |
| *** hichtakk has joined #openstack-infra | 05:23 | |
| *** hichtakk has quit IRC | 05:23 | |
| *** mmedvede has joined #openstack-infra | 05:23 | |
| *** ddieterly has quit IRC | 05:23 | |
| *** hichtakk has joined #openstack-infra | 05:24 | |
| *** pabelanger has quit IRC | 05:24 | |
| *** hichtakk has quit IRC | 05:24 | |
| *** hichtakk_ has joined #openstack-infra | 05:25 | |
| *** hichtak__ has joined #openstack-infra | 05:26 | |
| *** hichtakk_ has quit IRC | 05:26 | |
| *** pabelanger has joined #openstack-infra | 05:27 | |
| *** roaet_ has joined #openstack-infra | 05:32 | |
| *** rainya has quit IRC | 05:32 | |
| *** bhunter71 has quit IRC | 05:32 | |
| *** koolhead17 has quit IRC | 05:33 | |
| *** rainya has joined #openstack-infra | 05:33 | |
| *** dimtruck has quit IRC | 05:36 | |
| *** rossella_s has quit IRC | 05:36 | |
| *** masayukig has quit IRC | 05:36 | |
| *** rkukura has quit IRC | 05:36 | |
| *** rossella_s has joined #openstack-infra | 05:36 | |
| *** comstud has quit IRC | 05:38 | |
| *** hdd has joined #openstack-infra | 05:39 | |
| *** sdake_ has joined #openstack-infra | 05:40 | |
| *** comstud has joined #openstack-infra | 05:40 | |
| *** hichtak__ has quit IRC | 05:41 | |
| *** zz_dimtruck has joined #openstack-infra | 05:43 | |
| *** Ryan_Lane has quit IRC | 05:43 | |
| *** zz_dimtruck is now known as dimtruck | 05:43 | |
| *** masayukig has joined #openstack-infra | 05:44 | |
| *** pcrews has quit IRC | 05:45 | |
| *** pabelanger has quit IRC | 05:49 | |
| *** ashp has quit IRC | 05:49 | |
| *** russellb has quit IRC | 05:49 | |
| *** achanda has joined #openstack-infra | 05:49 | |
| *** russellb has joined #openstack-infra | 05:49 | |
| *** ddieterly has joined #openstack-infra | 05:49 | |
| *** pabelanger has joined #openstack-infra | 05:50 | |
| *** vigneshvar has joined #openstack-infra | 05:50 | |
| *** lbragstad has quit IRC | 05:52 | |
| *** subscope_ has joined #openstack-infra | 05:52 | |
| *** otherwiseguy has quit IRC | 05:53 | |
| *** ddieterly has quit IRC | 05:54 | |
| *** otherwiseguy has joined #openstack-infra | 05:54 | |
| *** lbragstad has joined #openstack-infra | 05:55 | |
| *** punith has joined #openstack-infra | 05:55 | |
| *** koolhead17 has joined #openstack-infra | 05:56 | |
| *** koolhead17 has joined #openstack-infra | 05:56 | |
| *** dimtruck has quit IRC | 05:58 | |
| *** rainya has quit IRC | 05:58 | |
| *** rainya has joined #openstack-infra | 05:58 | |
| *** zz_dimtruck has joined #openstack-infra | 05:58 | |
| *** zz_dimtruck is now known as dimtruck | 05:58 | |
| *** ashp has joined #openstack-infra | 05:59 | |
| *** yfried__ has joined #openstack-infra | 06:00 | |
| *** yfried has joined #openstack-infra | 06:03 | |
| *** yfried__ has quit IRC | 06:05 | |
| *** teran has joined #openstack-infra | 06:05 | |
| *** tobiash has quit IRC | 06:08 | |
| *** tobiash has joined #openstack-infra | 06:09 | |
| *** mohitsharma has quit IRC | 06:09 | |
| *** teran has quit IRC | 06:11 | |
| *** achanda has quit IRC | 06:11 | |
| *** ashp has quit IRC | 06:11 | |
| *** koolhead17 has quit IRC | 06:14 | |
| *** sdake_ has quit IRC | 06:14 | |
| *** teran has joined #openstack-infra | 06:14 | |
| *** ildikov has quit IRC | 06:14 | |
| *** koolhead17 has joined #openstack-infra | 06:15 | |
| *** koolhead17 has joined #openstack-infra | 06:15 | |
| *** ashp has joined #openstack-infra | 06:17 | |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Interpolate params inside of params. https://review.openstack.org/143138 | 06:18 |
| *** mohitsharma has joined #openstack-infra | 06:18 | |
| openstackgerrit | Maru Newby proposed openstack-infra/zuul: Add support for a skip-if filter on jobs https://review.openstack.org/146762 | 06:18 |
| *** Longgeek has quit IRC | 06:19 | |
| *** koolhead17 has quit IRC | 06:19 | |
| marun | jeblair, fungi, clarkb, sdague, jaypipes: feedback welcome ^^ | 06:20 |
| *** teran_ has joined #openstack-infra | 06:21 | |
| *** sdake_ has joined #openstack-infra | 06:23 | |
| *** teran has quit IRC | 06:24 | |
| *** salv-orlando has joined #openstack-infra | 06:26 | |
| *** yfried is now known as yfried|afk | 06:27 | |
| *** yfried|afk is now known as yfried | 06:29 | |
| *** baoli has joined #openstack-infra | 06:32 | |
| *** Longgeek has joined #openstack-infra | 06:34 | |
| *** baoli has quit IRC | 06:35 | |
| *** baoli has joined #openstack-infra | 06:36 | |
| *** otter768 has joined #openstack-infra | 06:38 | |
| *** sdake_ has quit IRC | 06:39 | |
| *** sdake_ has joined #openstack-infra | 06:42 | |
| *** otter768 has quit IRC | 06:43 | |
| *** sdake_ has quit IRC | 06:45 | |
| *** Masahiro has quit IRC | 06:45 | |
| *** subscope_ has quit IRC | 06:45 | |
| *** salv-orlando has quit IRC | 06:46 | |
| *** stevemar has quit IRC | 06:47 | |
| *** Masahiro has joined #openstack-infra | 06:47 | |
| *** ddieterly has joined #openstack-infra | 06:49 | |
| *** salv-orlando has joined #openstack-infra | 06:51 | |
| *** salv-orlando has quit IRC | 06:52 | |
| *** Longgeek has quit IRC | 06:53 | |
| *** mrunge has joined #openstack-infra | 06:53 | |
| *** Masahiro has quit IRC | 06:54 | |
| *** Masahiro has joined #openstack-infra | 06:54 | |
| *** ddieterly has quit IRC | 06:54 | |
| *** camunoz is now known as camunoz_away | 06:55 | |
| *** camunoz_away is now known as camunoz | 06:55 | |
| *** mmedvede has quit IRC | 06:55 | |
| *** belmoreira has joined #openstack-infra | 06:57 | |
| *** booly-yam has quit IRC | 06:58 | |
| *** Longgeek has joined #openstack-infra | 06:59 | |
| *** dannywilson has quit IRC | 07:00 | |
| *** mmedvede has joined #openstack-infra | 07:01 | |
| *** sdake_ has joined #openstack-infra | 07:02 | |
| *** sdake_ has quit IRC | 07:02 | |
| *** sdake_ has joined #openstack-infra | 07:02 | |
| *** jyuso has quit IRC | 07:03 | |
| *** teran has joined #openstack-infra | 07:06 | |
| *** camunoz is now known as camunoz_away | 07:06 | |
| *** pabelanger has quit IRC | 07:07 | |
| *** enikanorov__ has quit IRC | 07:08 | |
| *** zhiyan has quit IRC | 07:08 | |
| *** teran_ has quit IRC | 07:08 | |
| *** ashp has quit IRC | 07:10 | |
| *** ashp has joined #openstack-infra | 07:10 | |
| *** Sukhdev_ has quit IRC | 07:11 | |
| *** zhiyan has joined #openstack-infra | 07:11 | |
| *** hdd has quit IRC | 07:13 | |
| *** mmedvede has quit IRC | 07:14 | |
| *** otherwiseguy has quit IRC | 07:14 | |
| *** roaet_ has quit IRC | 07:14 | |
| *** ildikov has joined #openstack-infra | 07:14 | |
| *** andreaf has joined #openstack-infra | 07:15 | |
| *** pabelanger has joined #openstack-infra | 07:15 | |
| *** teran_ has joined #openstack-infra | 07:15 | |
| *** roaet_ has joined #openstack-infra | 07:15 | |
| *** otherwiseguy has joined #openstack-infra | 07:17 | |
| *** rainya has quit IRC | 07:17 | |
| *** masayukig has quit IRC | 07:17 | |
| *** baoli has quit IRC | 07:18 | |
| *** dtroyer has quit IRC | 07:18 | |
| *** teran has quit IRC | 07:19 | |
| *** ddieterly has joined #openstack-infra | 07:19 | |
| *** masayukig has joined #openstack-infra | 07:20 | |
| *** rainya has joined #openstack-infra | 07:20 | |
| *** teran_ has quit IRC | 07:21 | |
| *** armax has quit IRC | 07:22 | |
| *** sdake_ has quit IRC | 07:22 | |
| *** sdake_ has joined #openstack-infra | 07:23 | |
| *** ddieterly has quit IRC | 07:24 | |
| *** dhritishikhar has joined #openstack-infra | 07:25 | |
| *** sdake_ has quit IRC | 07:25 | |
| *** dtroyer has joined #openstack-infra | 07:26 | |
| *** mmedvede has joined #openstack-infra | 07:26 | |
| *** doude has joined #openstack-infra | 07:28 | |
| *** yamahata has quit IRC | 07:29 | |
| nuritv | Hi there,is there anyone here that can refer me to the CI requirments of Nova? can't seem to find it in the wiki | 07:30 |
| *** ssam2 has joined #openstack-infra | 07:34 | |
| *** otherwiseguy has quit IRC | 07:34 | |
| *** masayukig has quit IRC | 07:34 | |
| *** masayukig has joined #openstack-infra | 07:37 | |
| *** oomichi_ has quit IRC | 07:37 | |
| *** pabelanger has quit IRC | 07:37 | |
| *** ashp has quit IRC | 07:37 | |
| *** russellb has quit IRC | 07:37 | |
| *** pabelanger has joined #openstack-infra | 07:38 | |
| *** salv-orlando has joined #openstack-infra | 07:38 | |
| *** salv-orlando has quit IRC | 07:39 | |
| *** apmelton has quit IRC | 07:39 | |
| *** russellb has joined #openstack-infra | 07:40 | |
| *** kedkul has quit IRC | 07:41 | |
| *** teran has joined #openstack-infra | 07:41 | |
| *** apmelton has joined #openstack-infra | 07:42 | |
| *** Sukhdev has joined #openstack-infra | 07:44 | |
| *** otherwiseguy has joined #openstack-infra | 07:45 | |
| *** dimtruck has quit IRC | 07:45 | |
| *** ashp has joined #openstack-infra | 07:45 | |
| *** teran has quit IRC | 07:46 | |
| *** zz_dimtruck has joined #openstack-infra | 07:46 | |
| *** zz_dimtruck is now known as dimtruck | 07:46 | |
| *** sdake has quit IRC | 07:47 | |
| *** chlong has quit IRC | 07:47 | |
| *** Ala has joined #openstack-infra | 07:48 | |
| *** mpaolino has joined #openstack-infra | 07:56 | |
| *** jyuso has joined #openstack-infra | 07:57 | |
| *** ociuhandu has quit IRC | 07:58 | |
| *** imeerovi has joined #openstack-infra | 07:59 | |
| *** fandi has joined #openstack-infra | 08:00 | |
| *** jistr has joined #openstack-infra | 08:00 | |
| *** jerryz has quit IRC | 08:03 | |
| *** jerryz has joined #openstack-infra | 08:04 | |
| *** mmedvede has quit IRC | 08:09 | |
| *** dtroyer has quit IRC | 08:10 | |
| *** dtroyer has joined #openstack-infra | 08:12 | |
| *** Sukhdev has quit IRC | 08:13 | |
| *** markus_z has joined #openstack-infra | 08:14 | |
| *** booly-yam has joined #openstack-infra | 08:15 | |
| *** mmedvede has joined #openstack-infra | 08:15 | |
| *** mohitsharma has quit IRC | 08:16 | |
| *** jcoufal has joined #openstack-infra | 08:18 | |
| *** mohitsharma has joined #openstack-infra | 08:19 | |
| *** pabelanger has quit IRC | 08:24 | |
| *** lbragstad has quit IRC | 08:24 | |
| *** marun has quit IRC | 08:24 | |
| *** hichtakk has joined #openstack-infra | 08:25 | |
| *** hichtakk has quit IRC | 08:25 | |
| *** ronis has joined #openstack-infra | 08:25 | |
| *** belmoreira has quit IRC | 08:26 | |
| *** pabelanger has joined #openstack-infra | 08:26 | |
| *** pabelanger has joined #openstack-infra | 08:26 | |
| *** masayukig has quit IRC | 08:27 | |
| *** booly-yam has quit IRC | 08:29 | |
| *** dtantsur|afk is now known as dtantsur | 08:30 | |
| *** zz_avozza is now known as avozza | 08:30 | |
| *** koolhead17 has joined #openstack-infra | 08:30 | |
| *** koolhead17 has joined #openstack-infra | 08:30 | |
| *** russellb has quit IRC | 08:30 | |
| *** rainya has quit IRC | 08:30 | |
| *** roaet_ has quit IRC | 08:30 | |
| *** cyeoh has quit IRC | 08:30 | |
| *** rainya has joined #openstack-infra | 08:31 | |
| *** cyeoh has joined #openstack-infra | 08:31 | |
| *** roaet_ has joined #openstack-infra | 08:31 | |
| *** Guest57605 has quit IRC | 08:31 | |
| *** arxcruz has joined #openstack-infra | 08:31 | |
| *** teran has joined #openstack-infra | 08:31 | |
| *** mgagne has joined #openstack-infra | 08:33 | |
| *** mgagne is now known as Guest5538 | 08:33 | |
| *** lbragstad has joined #openstack-infra | 08:34 | |
| *** masayukig has joined #openstack-infra | 08:34 | |
| *** otherwiseguy has quit IRC | 08:36 | |
| *** avozza is now known as zz_avozza | 08:37 | |
| *** dtroyer has quit IRC | 08:37 | |
| *** zz_avozza is now known as avozza | 08:37 | |
| *** links has joined #openstack-infra | 08:38 | |
| *** marun has joined #openstack-infra | 08:38 | |
| *** Masahiro has quit IRC | 08:38 | |
| *** russellb has joined #openstack-infra | 08:39 | |
| *** otherwiseguy has joined #openstack-infra | 08:39 | |
| *** Masahiro has joined #openstack-infra | 08:39 | |
| *** dtroyer has joined #openstack-infra | 08:39 | |
| *** pabelanger has quit IRC | 08:39 | |
| *** pabelanger has joined #openstack-infra | 08:39 | |
| *** pabelanger has quit IRC | 08:39 | |
| *** pabelanger has joined #openstack-infra | 08:39 | |
| *** otter768 has joined #openstack-infra | 08:39 | |
| *** jamespage has joined #openstack-infra | 08:40 | |
| *** cyeoh has quit IRC | 08:43 | |
| *** rainya has quit IRC | 08:44 | |
| *** otter768 has quit IRC | 08:44 | |
| *** Hal_ has joined #openstack-infra | 08:44 | |
| *** rainya has joined #openstack-infra | 08:45 | |
| *** cyeoh has joined #openstack-infra | 08:45 | |
| *** avozza is now known as zz_avozza | 08:47 | |
| *** jcoufal has quit IRC | 08:54 | |
| *** jcoufal has joined #openstack-infra | 08:54 | |
| *** salv-orlando has joined #openstack-infra | 08:57 | |
| *** dtroyer has quit IRC | 08:59 | |
| *** dtroyer has joined #openstack-infra | 09:00 | |
| *** zz_avozza is now known as avozza | 09:01 | |
| *** ronis has quit IRC | 09:04 | |
| *** ronis has joined #openstack-infra | 09:04 | |
| *** jpich has joined #openstack-infra | 09:06 | |
| *** salv-orlando has quit IRC | 09:11 | |
| *** mrunge has quit IRC | 09:16 | |
| *** luqas has quit IRC | 09:17 | |
| *** mohitsharma has quit IRC | 09:22 | |
| *** mrunge has joined #openstack-infra | 09:22 | |
| *** Hunner has quit IRC | 09:22 | |
| *** Hunner has joined #openstack-infra | 09:23 | |
| *** jlibosva has joined #openstack-infra | 09:23 | |
| *** MaxV has joined #openstack-infra | 09:24 | |
| *** MaxV has quit IRC | 09:27 | |
| *** MaxV has joined #openstack-infra | 09:27 | |
| *** mfink has quit IRC | 09:27 | |
| *** jamielennox is now known as jamielennox|away | 09:31 | |
| *** dims__ has joined #openstack-infra | 09:31 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 09:34 | |
| *** mohitsharma has joined #openstack-infra | 09:34 | |
| *** ZZelle has quit IRC | 09:35 | |
| *** mmedvede has quit IRC | 09:35 | |
| *** ZZelle has joined #openstack-infra | 09:36 | |
| *** dims__ has quit IRC | 09:36 | |
| *** hashar has joined #openstack-infra | 09:38 | |
| *** mmedvede has joined #openstack-infra | 09:38 | |
| *** ociuhandu has joined #openstack-infra | 09:40 | |
| *** derekh has joined #openstack-infra | 09:42 | |
| *** yaguang has quit IRC | 09:42 | |
| *** dtroyer has quit IRC | 09:44 | |
| *** dimtruck has quit IRC | 09:44 | |
| *** ociuhandu has quit IRC | 09:44 | |
| *** ociuhandu has joined #openstack-infra | 09:46 | |
| *** Guest5538 has quit IRC | 09:51 | |
| *** dizquierdo has joined #openstack-infra | 09:51 | |
| *** salv-orlando has joined #openstack-infra | 09:51 | |
| *** zz_dimtruck has joined #openstack-infra | 09:52 | |
| *** zz_dimtruck is now known as dimtruck | 09:52 | |
| *** andreaf has quit IRC | 09:53 | |
| *** mgagne has joined #openstack-infra | 09:55 | |
| *** mgagne is now known as Guest80979 | 09:55 | |
| *** dtroyer has joined #openstack-infra | 09:55 | |
| *** mmedvede has quit IRC | 09:59 | |
| *** vishy has quit IRC | 09:59 | |
| *** coolsvap|afk is now known as coolsvap | 09:59 | |
| *** gibi has joined #openstack-infra | 10:00 | |
| *** e0ne has joined #openstack-infra | 10:00 | |
| *** dannywilson has joined #openstack-infra | 10:01 | |
| *** ihrachyshka has joined #openstack-infra | 10:01 | |
| *** jedimike has joined #openstack-infra | 10:02 | |
| *** pabelanger has quit IRC | 10:03 | |
| *** otherwiseguy has quit IRC | 10:03 | |
| *** zaro has quit IRC | 10:04 | |
| *** devananda has quit IRC | 10:04 | |
| *** mohitsharma has quit IRC | 10:04 | |
| *** pabelanger has joined #openstack-infra | 10:05 | |
| *** otherwiseguy has joined #openstack-infra | 10:05 | |
| *** dannywilson has quit IRC | 10:05 | |
| *** zaro has joined #openstack-infra | 10:07 | |
| *** mmedvede has joined #openstack-infra | 10:09 | |
| *** che-arne has joined #openstack-infra | 10:09 | |
| *** andreaf has joined #openstack-infra | 10:10 | |
| *** vishy has joined #openstack-infra | 10:10 | |
| *** devananda has joined #openstack-infra | 10:10 | |
| *** pblaho has joined #openstack-infra | 10:10 | |
| *** andrearosa has joined #openstack-infra | 10:16 | |
| *** naggappan has quit IRC | 10:17 | |
| *** dhritishikhar has quit IRC | 10:21 | |
| *** vishy has quit IRC | 10:21 | |
| *** russellb has quit IRC | 10:22 | |
| *** masayukig has quit IRC | 10:22 | |
| *** dizquierdo has quit IRC | 10:24 | |
| *** rainya has quit IRC | 10:25 | |
| *** Hunner has quit IRC | 10:25 | |
| andreaf | andreaf nm, it seems to be the eventlet issue | 10:26 |
| *** Hunner has joined #openstack-infra | 10:26 | |
| *** russellb has joined #openstack-infra | 10:26 | |
| openstackgerrit | Ori Livneh proposed openstack-infra/zuul: Ensure the repository configuration lock is released https://review.openstack.org/147101 | 10:27 |
| *** adalbas has quit IRC | 10:27 | |
| *** yamamoto_ has quit IRC | 10:30 | |
| *** yfried is now known as yfried|afk | 10:31 | |
| *** jistr has quit IRC | 10:31 | |
| *** e0ne is now known as e0ne_ | 10:31 | |
| openstackgerrit | Nikita Konovalov proposed openstack-infra/python-storyboardclient: Project Groups and Projects support https://review.openstack.org/147104 | 10:33 |
| *** masayukig has joined #openstack-infra | 10:33 | |
| *** vishy has joined #openstack-infra | 10:34 | |
| *** rainya has joined #openstack-infra | 10:35 | |
| *** ldnunes has joined #openstack-infra | 10:36 | |
| *** coolsvap is now known as coolsvap|afk | 10:39 | |
| *** otter768 has joined #openstack-infra | 10:40 | |
| *** dims__ has joined #openstack-infra | 10:41 | |
| *** e0ne_ has quit IRC | 10:42 | |
| *** Masahiro has quit IRC | 10:42 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/storyboard-webclient: Add button to remove all recent events in dashboard https://review.openstack.org/147105 | 10:43 |
| *** otter768 has quit IRC | 10:45 | |
| *** yolanda has quit IRC | 10:47 | |
| *** yfried|afk is now known as yfried | 10:47 | |
| *** pelix has joined #openstack-infra | 10:48 | |
| *** jistr has joined #openstack-infra | 10:48 | |
| *** mpaolino has quit IRC | 10:49 | |
| *** yolanda has joined #openstack-infra | 10:51 | |
| *** coolsvap|afk is now known as coolsvap | 10:51 | |
| jedimike | anyone around that's familiar with pypi_mirror? i'm getting some weird behaviour from it | 10:53 |
| *** e0ne has joined #openstack-infra | 10:57 | |
| *** avozza is now known as zz_avozza | 10:59 | |
| *** dims__ has quit IRC | 11:00 | |
| *** russellb has quit IRC | 11:00 | |
| *** pabelanger has quit IRC | 11:00 | |
| *** dims__ has joined #openstack-infra | 11:00 | |
| *** pabelanger has joined #openstack-infra | 11:02 | |
| *** coolsvap is now known as coolsvap|afk | 11:02 | |
| *** alexpilotti has joined #openstack-infra | 11:03 | |
| *** mpaolino has joined #openstack-infra | 11:04 | |
| *** dims__ has quit IRC | 11:05 | |
| *** russellb has joined #openstack-infra | 11:06 | |
| *** coolsvap|afk is now known as coolsvap | 11:09 | |
| *** dizquierdo has joined #openstack-infra | 11:12 | |
| *** hashar has quit IRC | 11:13 | |
| *** _nadya_ has joined #openstack-infra | 11:14 | |
| *** adalbas has joined #openstack-infra | 11:19 | |
| *** mrunge has quit IRC | 11:21 | |
| *** aysyd has joined #openstack-infra | 11:22 | |
| *** e0ne_ has joined #openstack-infra | 11:23 | |
| *** mrunge has joined #openstack-infra | 11:24 | |
| *** zz_avozza is now known as avozza | 11:25 | |
| *** dhritishikhar has joined #openstack-infra | 11:26 | |
| *** e0ne has quit IRC | 11:26 | |
| *** rfolco has joined #openstack-infra | 11:26 | |
| *** asselin has quit IRC | 11:28 | |
| *** mpaolino has quit IRC | 11:39 | |
| *** sputnik13 has joined #openstack-infra | 11:41 | |
| *** Masahiro has joined #openstack-infra | 11:43 | |
| *** dims__ has joined #openstack-infra | 11:44 | |
| *** yamamoto has joined #openstack-infra | 11:44 | |
| *** yamamoto has quit IRC | 11:44 | |
| *** Masahiro has quit IRC | 11:47 | |
| *** erlon has joined #openstack-infra | 11:49 | |
| *** e0ne_ has quit IRC | 11:55 | |
| *** e0ne has joined #openstack-infra | 11:58 | |
| *** mwagner_lap has quit IRC | 12:00 | |
| *** yfried is now known as yfried|afk | 12:01 | |
| *** jerryz has quit IRC | 12:02 | |
| *** chlong has joined #openstack-infra | 12:06 | |
| *** _amrith_ is now known as amrith | 12:07 | |
| *** otter768 has joined #openstack-infra | 12:09 | |
| *** jp_at_hp has joined #openstack-infra | 12:13 | |
| *** marcusvrn1 has quit IRC | 12:15 | |
| *** miarmak has joined #openstack-infra | 12:16 | |
| *** otter768 has quit IRC | 12:20 | |
| sdague | ihrachyshka: for stable, I think the eventlet cap is fine | 12:21 |
| sdague | https://review.openstack.org/#/c/147062/ | 12:22 |
| *** yfried|afk is now known as yfried | 12:22 | |
| ihrachyshka | sdague, that is fine, but since we already have fixes in queue that would avoid that, I think it's better to fix it without narrowing version range. what do you think? | 12:23 |
| sdague | honestly, I think the cap is more appropriate for stable | 12:23 |
| sdague | because people aren't going to randomly upgrade eventlet | 12:23 |
| sdague | in real environments | 12:23 |
| jedimike | hey all, pypi_mirror seems to be broken with the new pip | 12:23 |
| *** HeOS has quit IRC | 12:24 | |
| jedimike | it relies on tarballs being in the nominated cache directory to create a tarball mirror, but now pip does its own thing with cache and tarballs don't end up in the cache dir you configure in mirror.yaml | 12:24 |
| sdague | ihrachyshka: which fixes do you have to avoid this? | 12:24 |
| ihrachyshka | https://review.openstack.org/#/q/Idbb9d2b53829dae0e807cd1260dee3dce155d5f3,n,z | 12:25 |
| sdague | ihrachyshka: because a lot of stable code is currently getting approved that can't pass | 12:25 |
| ihrachyshka | sdague, though I think they should be rebased to include boto cap | 12:25 |
| ihrachyshka | not the case for Juno at least | 12:25 |
| sdague | has a jenkins -1 https://review.openstack.org/#/c/145955/ | 12:25 |
| ihrachyshka | sdague, yes, due to boto. rebase is needed (I was going to do it right now) | 12:26 |
| sdague | so that's potentionally hours away from useful, plus it means you actually have to fix icehouse first, because of the upgrade | 12:27 |
| sdague | and in the mean time, no body can merge code | 12:27 |
| ihrachyshka | sdague, okay, got it, let's move with requirements cap then | 12:28 |
| ihrachyshka | also, nova requirements update is not merged still :| | 12:28 |
| sdague | it is on master | 12:28 |
| sdague | is dave walker on IRC somewhere? | 12:30 |
| sdague | need him to un WIP - https://review.openstack.org/#/c/147063/ | 12:30 |
| sdague | otherwise we have to restart the clock on it | 12:30 |
| ihrachyshka | sdague, we can rebase, then Workflow-1 will be dropped | 12:31 |
| ihrachyshka | sdague, seems he is offline | 12:31 |
| sdague | we can, then we need to wait over an hour for new test results | 12:31 |
| ihrachyshka | sdague, but icehouse fix is for Juno only, right? | 12:31 |
| ihrachyshka | sdague, at least master is not blocked | 12:31 |
| sdague | true | 12:31 |
| *** isviridov_away is now known as isviridov | 12:32 | |
| ihrachyshka | let's restart it (we will need to modify commit message I guess to make gerrit create a new patchset) | 12:32 |
| sdague | ok, done | 12:32 |
| sdague | I just added a commit comment | 12:32 |
| *** marcusvrn has joined #openstack-infra | 12:34 | |
| ihrachyshka | sdague, ninja approve? | 12:34 |
| *** cdent has joined #openstack-infra | 12:35 | |
| *** _nadya_ has quit IRC | 12:37 | |
| openstackgerrit | Mike Heald proposed openstack-infra/pypi-mirror: Pin pip so download cache location works https://review.openstack.org/147128 | 12:39 |
| *** tomoe has quit IRC | 12:40 | |
| jedimike | ^^ fixes pypi-mirror tarball mirroring, currently it's broken due to the new pip version | 12:40 |
| openstackgerrit | Aleksey proposed openstack-infra/storyboard: Team and project groups delete methods https://review.openstack.org/143675 | 12:40 |
| *** punith has quit IRC | 12:41 | |
| *** enikanorov_ has joined #openstack-infra | 12:43 | |
| *** tomoe has joined #openstack-infra | 12:44 | |
| openstackgerrit | Mike Heald proposed openstack-infra/pypi-mirror: Pin pip so download cache location works https://review.openstack.org/147128 | 12:44 |
| *** MaxV has quit IRC | 12:44 | |
| *** _nadya_ has joined #openstack-infra | 12:45 | |
| *** Masahiro has joined #openstack-infra | 12:46 | |
| *** ildikov has quit IRC | 12:48 | |
| *** sandywalsh_ has quit IRC | 12:49 | |
| *** koolhead17 has quit IRC | 12:49 | |
| *** doug-fish has joined #openstack-infra | 12:50 | |
| *** Masahiro has quit IRC | 12:50 | |
| *** ryanpetrello has joined #openstack-infra | 12:53 | |
| *** denis_makogon has joined #openstack-infra | 12:55 | |
| *** avozza is now known as zz_avozza | 12:55 | |
| *** boris-42 has joined #openstack-infra | 12:56 | |
| *** e0ne has quit IRC | 12:56 | |
| *** dkranz has joined #openstack-infra | 12:57 | |
| *** e0ne has joined #openstack-infra | 12:57 | |
| *** ssam2 has quit IRC | 13:01 | |
| *** sdake has joined #openstack-infra | 13:02 | |
| *** chlong has quit IRC | 13:02 | |
| *** sdake has quit IRC | 13:03 | |
| *** sdake has joined #openstack-infra | 13:03 | |
| openstackgerrit | Kyle Rockman proposed openstack-infra/jenkins-job-builder: adding support for the xcode builder plugin https://review.openstack.org/146918 | 13:04 |
| *** zz_avozza is now known as avozza | 13:04 | |
| *** ildikov has joined #openstack-infra | 13:06 | |
| *** amotoki_ has joined #openstack-infra | 13:07 | |
| *** _nadya_ has quit IRC | 13:08 | |
| *** hashar has joined #openstack-infra | 13:08 | |
| sdague | ihrachyshka: so is stable/juno unit tests passing now, or is something else needed there? | 13:08 |
| ihrachyshka | sdague, we'll need to merge boto cap after we receive it with eventlet included: https://review.openstack.org/#/c/146344/ | 13:09 |
| sdague | ihrachyshka: ok | 13:09 |
| ihrachyshka | for now it's (probably) unmergeable | 13:09 |
| *** dprince has joined #openstack-infra | 13:09 | |
| ihrachyshka | sdague, btw I started collecting current state at: https://etherpad.openstack.org/p/stable-tracker | 13:10 |
| ihrachyshka | sdague, too many gate breakages these days. | 13:12 |
| *** MaxV has joined #openstack-infra | 13:12 | |
| sdague | yep, agreed | 13:12 |
| *** e0ne is now known as e0ne_ | 13:14 | |
| *** mpaolino has joined #openstack-infra | 13:18 | |
| *** yfried is now known as yfried|afk | 13:18 | |
| *** dhritishikhar has quit IRC | 13:19 | |
| *** avozza is now known as zz_avozza | 13:20 | |
| *** yfried|afk is now known as yfried | 13:20 | |
| *** mpaolino has quit IRC | 13:22 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: split up pbr tests from requirements tests https://review.openstack.org/146981 | 13:23 |
| *** e0ne_ is now known as e0ne | 13:23 | |
| *** koolhead17 has joined #openstack-infra | 13:24 | |
| *** baoli has joined #openstack-infra | 13:24 | |
| *** koolhead17 has quit IRC | 13:24 | |
| *** koolhead17 has joined #openstack-infra | 13:24 | |
| *** mpaolino has joined #openstack-infra | 13:25 | |
| *** mbacchi has joined #openstack-infra | 13:25 | |
| *** koolhead17 has quit IRC | 13:26 | |
| *** jerryz has joined #openstack-infra | 13:27 | |
| *** baoli has quit IRC | 13:27 | |
| *** baoli has joined #openstack-infra | 13:28 | |
| *** radez_g0n3 is now known as radez | 13:29 | |
| *** amrith is now known as _amrith_ | 13:30 | |
| *** mwagner_lap has joined #openstack-infra | 13:33 | |
| *** mestery has quit IRC | 13:33 | |
| *** hdd has joined #openstack-infra | 13:35 | |
| ihrachyshka | sdague, can you help me with enabling periodic checks for requirements and oslo-incubator for stable branches? | 13:37 |
| *** bcourt has joined #openstack-infra | 13:37 | |
| ihrachyshka | sdague, I hardly know how project-config works, so help would be appreciated | 13:37 |
| *** _nadya_ has joined #openstack-infra | 13:39 | |
| openstackgerrit | Rodrigo Duarte proposed openstack-infra/project-config: Graduate oslo.policy https://review.openstack.org/146645 | 13:40 |
| *** zz_avozza is now known as avozza | 13:40 | |
| sdague | ihrachyshka: sure | 13:41 |
| sdague | so https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/python-bitrot-jobs.yaml is a good example of some periodic jobs that are defined | 13:41 |
| *** jerryz has quit IRC | 13:42 | |
| *** dhritishikhar has joined #openstack-infra | 13:43 | |
| sdague | actually... never mind that | 13:43 |
| sdague | look how tempest defines things | 13:43 |
| sdague | there is a periodic-qa queue | 13:43 |
| sdague | on the openstack/tempest project definition | 13:43 |
| ihrachyshka | sdague, so that's what I have now: http://paste.fedoraproject.org/169511/21242890 | 13:44 |
| *** rlandy has joined #openstack-infra | 13:45 | |
| sdague | that seems about right | 13:46 |
| sdague | more importantly though, is anyone actually reading these? | 13:46 |
| sdague | that's the issue we have with the tempest ones, no one seems to really care that much | 13:46 |
| sdague | until everything is buste | 13:46 |
| ihrachyshka | sdague, hm, we track stable updates | 13:47 |
| ihrachyshka | sdague, it's just that sometimes we lag on a day or two | 13:47 |
| *** weshay has joined #openstack-infra | 13:47 | |
| *** _nadya_ has quit IRC | 13:47 | |
| ihrachyshka | sdague, hm, though I wonder whether we really need requirements check. won't it be the same as for tempest? | 13:48 |
| ihrachyshka | I guess we also sync requirements there, so probably any issues would be caught anyway. | 13:48 |
| ihrachyshka | for oslo-incubator, it's different, we need unit tests there. | 13:48 |
| ihrachyshka | sdague, btw bot updated nova patch: https://review.openstack.org/#/c/146344/ | 13:50 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:50 | |
| *** _nadya_ has joined #openstack-infra | 13:50 | |
| *** kgiusti has joined #openstack-infra | 13:51 | |
| sdague | ihrachyshka: great, +A | 13:51 |
| sdague | yeh, I think realistically tempest will catch the requirements issues | 13:51 |
| sdague | well... requirements actually tests *all* of projects.txt, which is more than tempest tests | 13:52 |
| ihrachyshka | ok, I'll go with incubator only then | 13:52 |
| ihrachyshka | sdague, but how would those projects get a conflicting update? | 13:52 |
| *** yfried has quit IRC | 13:54 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Run periodic stable checks for oslo-incubator https://review.openstack.org/147157 | 13:55 |
| *** jcoufal_ has joined #openstack-infra | 14:00 | |
| *** julim has joined #openstack-infra | 14:02 | |
| *** mestery has joined #openstack-infra | 14:02 | |
| *** jcoufal has quit IRC | 14:02 | |
| *** sandywalsh has joined #openstack-infra | 14:04 | |
| sdague | ihrachyshka: yeh, honestly, I thik the requirements job on stable is probably overkill | 14:05 |
| ihrachyshka | ack, I've removed it from the patch (link above) | 14:07 |
| fungi | sdague: dave walker is Daviey | 14:07 |
| fungi | (for future reference) | 14:07 |
| *** fandi has quit IRC | 14:09 | |
| *** mkerrin has joined #openstack-infra | 14:11 | |
| *** mattfarina has joined #openstack-infra | 14:11 | |
| *** fandi has joined #openstack-infra | 14:11 | |
| *** dustins has joined #openstack-infra | 14:12 | |
| *** dustins has quit IRC | 14:12 | |
| *** sputnik13 has quit IRC | 14:12 | |
| *** dhritishikhar has quit IRC | 14:15 | |
| *** vigneshvar has quit IRC | 14:16 | |
| *** dalgaaf has joined #openstack-infra | 14:17 | |
| *** koolhead17 has joined #openstack-infra | 14:17 | |
| *** banix has joined #openstack-infra | 14:18 | |
| sdague | fungi: coolio | 14:19 |
| *** imeerovi has quit IRC | 14:20 | |
| BobBall | Anyone seen an issue with pip install extras (getting "module version not found") or pip install version? (getting "module object has no attribute generate_egg_info_metadata") | 14:20 |
| BobBall | https://dpaste.de/AmKA/raw / https://dpaste.de/aK6H/raw for the raw logs of the failures... | 14:20 |
| *** banix has quit IRC | 14:21 | |
| *** hashar_ has joined #openstack-infra | 14:21 | |
| sdague | fungi: per the requirements discussion yesterday - https://review.openstack.org/#/c/146981/ is I think what we want to do | 14:21 |
| fungi | BobBall: seems to work fine for me using latest pip and virtualenv | 14:21 |
| *** otter768 has joined #openstack-infra | 14:21 | |
| BobBall | Poo. | 14:21 |
| BobBall | It's very consistent for me - with a scripted environment that used to work | 14:22 |
| BobBall | thanks for looking. | 14:22 |
| dstufft | BobBall: what version of pip and setuptools? | 14:23 |
| *** hashar has quit IRC | 14:23 | |
| fungi | sdague: so... one issue i can think of since i went halfway down this road once already | 14:23 |
| *** pc_m has joined #openstack-infra | 14:24 | |
| fungi | sdague: you won't be able to install all of global-requirements.txt on a devstack node without additional effort to satisfy system requirements | 14:24 |
| sdague | fungi: sure, that's fixable though | 14:24 |
| *** bhunter71 has joined #openstack-infra | 14:24 | |
| sdague | and honestly, we already install system packages at the beginning of that script | 14:25 |
| openstackgerrit | Nikolay Markov proposed openstack-infra/project-config: Add Fuel Plugins to Stackforge https://review.openstack.org/147169 | 14:25 |
| fungi | yeah, just setting expectations for some disappointment on that front | 14:25 |
| sdague | yep | 14:25 |
| *** pc_m_ has joined #openstack-infra | 14:25 | |
| sdague | it was really useful to try to install the whole stack though, and find the gear thing in the process | 14:25 |
| * fungi digs up his wip change which was trying to test installation of test-requirements.txt lists | 14:25 | |
| sdague | https://review.openstack.org/#/c/147021/ | 14:25 |
| *** otter768 has quit IRC | 14:26 | |
| *** plaurin_ has joined #openstack-infra | 14:26 | |
| fungi | sdague: not sure if you saw, but i generated a list last night of packages listed in global-requirements.txt which are not in *requirements*.txt files in master branches of any projects listed in projects.txt http://paste.openstack.org/show/157440/ | 14:26 |
| sdague | nope | 14:26 |
| BobBall | dstufft: I'm using the versions from Ubuntu; python-setuptools 3.3-1ubuntu1 and pip 1.5.4 | 14:27 |
| *** erikmwilson has joined #openstack-infra | 14:27 | |
| sdague | fungi: can you propose that tool to the requirements repo? | 14:27 |
| *** erikmwilson has quit IRC | 14:27 | |
| fungi | sdague: obviously some false positives in there, like pyeclib (i didn't check feature branches) | 14:27 |
| dstufft | BobBall: can you post the entire output? | 14:27 |
| dstufft | or is that it? | 14:27 |
| fungi | sdague: it wasn't really a tool, but i can turn it into one | 14:27 |
| sdague | fungi: yeh, even if it's kind of janky, having a basis to work from would be good | 14:28 |
| fungi | was mostly to get a baseline and see how much cruft we might have | 14:28 |
| *** _amrith_ is now known as amrith | 14:28 | |
| BobBall | the raw files above are the pip.log files - they seemed slightly more verbose than the actual output (https://dpaste.de/AmKA/raw / https://dpaste.de/aK6H/raw) | 14:28 |
| sdague | the cffi one is surprising though | 14:28 |
| *** pc_m has quit IRC | 14:28 | |
| fungi | and, for example, pip_missing_reqs is in there because the review to add it to nova's test-requirements.txt is stalled | 14:28 |
| sdague | fungi: yep | 14:29 |
| *** dkliban_afk is now known as dkliban | 14:29 | |
| dstufft | BobBall: hmm, interesting | 14:29 |
| fungi | but yeah i'll modify the process to check feature branches too and then propose it as a script | 14:29 |
| dstufft | I've not seen that error before, and It's not clear to me what would be causing it | 14:30 |
| dstufft | if this was working before and it's just stopped working, my guess would be that something's changed with the versions of those projects that are on PyPI | 14:30 |
| BobBall | Yay. I like being unique. Any suggestions for digging? | 14:30 |
| BobBall | Unfortunately no releases for 6 months apparently | 14:30 |
| sdague | fungi: thanks | 14:31 |
| dstufft | BobBall: you might make sure that setuptools didn't get upgraded at some point | 14:32 |
| sdague | fungi: where is the pip_missing_reqs ? | 14:32 |
| dstufft | BobBall: you might also try to rm -rf /opt/nodepool/env/build | 14:32 |
| *** yamahata has joined #openstack-infra | 14:32 | |
| sdague | BobBall: realize that pbr depends on pip, so can uplift it to current if you pip install -U pbr | 14:32 |
| fungi | BobBall: is this in the system context and did you maybe end up with distro-packaged versions of the same installed? maybe they were installed long ago, then you pip installed over top of them, then more recently your package manager brought in updates for them? | 14:33 |
| sdague | that was causing some hp internal ci failures the other day | 14:33 |
| BobBall | omg dstufft... That fixed it... Installing extras fails because it needs version, but then the build dir is broken so you can't install version. I'll update my script to install version first then extras (and look at why extras doesn't depend on version) | 14:34 |
| openstackgerrit | Doug Hellmann proposed openstack/requirements: Update to oslo.db 1.4.1 https://review.openstack.org/146613 | 14:34 |
| dstufft | BobBall: pip 6.0 includes a randomizes build directory fwiw, so you might want to consider upgrading to that | 14:34 |
| dstufft | prior to that it uses a static build directory, which meant that if a problem was caused from a broken build dir it might persistent until you remove it | 14:35 |
| *** Masahiro has joined #openstack-infra | 14:35 | |
| *** erikmwilson has joined #openstack-infra | 14:35 | |
| BobBall | ah nice. | 14:36 |
| dstufft | fungi: sdague clarkb I don't know if it's useful at all to you guys, but I'm working with the Fedora python maintainer to make a copr repo (essentially Fedora/RHEL PPAs) that'll install the latest PyPA tooling always | 14:36 |
| dstufft | it supports RHEL6+ and Fedora 19+ | 14:37 |
| dstufft | https://copr.fedoraproject.org/coprs/pypa/pypa/ it's got setuptools, pip, and wheel so far | 14:38 |
| dstufft | virtualenv will be soon hopefully | 14:38 |
| sdague | ok, though we don't use fedora much | 14:38 |
| sdague | if that existed for ubuntu, that would be handy | 14:38 |
| dstufft | sdague: Ubuntu and Debian is on my list too :) | 14:39 |
| dstufft | I just didin't do them yet | 14:39 |
| dstufft | Fedora/RHEL is mostly done because Slavek is awesome and did most of the work for me | 14:39 |
| *** mestery has quit IRC | 14:39 | |
| *** armax has joined #openstack-infra | 14:39 | |
| *** Masahiro has quit IRC | 14:39 | |
| *** garyh has joined #openstack-infra | 14:39 | |
| fungi | dstufft: neat idea. and yeah, for our current tooling we'd want it for centos and ubuntu | 14:40 |
| fungi | centos 6 for now, but soon 7 as well | 14:40 |
| fungi | though you said 6+ so that works | 14:40 |
| dstufft | yea the copr repo should work for CentOS too I assume, since it works for RHEL | 14:41 |
| *** erikmwilson has quit IRC | 14:41 | |
| dstufft | we didn't do RHEL 5 becuase lol python 2.4 | 14:41 |
| *** dkranz has quit IRC | 14:41 | |
| fungi | indeed. we don't do rhel^H^H^H^Hcentos 5 because lol why | 14:41 |
| *** radez is now known as radez_g0n3 | 14:42 | |
| openstackgerrit | Maru Newby proposed openstack-infra/zuul: Add support for a skip-if filter on jobs https://review.openstack.org/146762 | 14:42 |
| dstufft | python-setuptools is the really important one, since it's more or less impossible not to get that installed via yum/apt since a ton of things depend on it | 14:42 |
| dstufft | and if you install it via pip you end up with pip and yum fighting | 14:42 |
| fungi | yep | 14:44 |
| fungi | totally agree there | 14:44 |
| *** yamahata has quit IRC | 14:44 | |
| fungi | we basically go through some nasty gyrations to blow it away and replace it with a pip-installed version | 14:44 |
| *** dustins_ has joined #openstack-infra | 14:45 | |
| *** yamahata has joined #openstack-infra | 14:45 | |
| fungi | and that works okay for short-lived machines, but the moment centos 6 gets a python-setuptools package update all hell is going to break loose on our (thankfully) very few long-lived centos servers | 14:45 |
| *** amrith has left #openstack-infra | 14:46 | |
| *** jgrimm is now known as zz_jgrimm | 14:46 | |
| *** mohitsharma has joined #openstack-infra | 14:48 | |
| *** mtanino has joined #openstack-infra | 14:49 | |
| *** armax has quit IRC | 14:50 | |
| *** miarmak has quit IRC | 14:51 | |
| *** erikmwilson has joined #openstack-infra | 14:52 | |
| *** hdd has quit IRC | 14:52 | |
| *** erikmwilson has quit IRC | 14:55 | |
| *** mpaolino has quit IRC | 14:57 | |
| marun | sdague: I have a patch up to add 'skip-if' configuration for jobs keyed on project and included files: https://review.openstack.org/#/c/146762/ | 14:57 |
| *** erikmwilson has joined #openstack-infra | 14:58 | |
| sdague | marun: did you discuss with jeblair, he's going to be the right person to evaluate | 14:58 |
| marun | sdague: I'm curious if what I've proposed is at all useful for what you were investigating | 14:58 |
| *** baoli has quit IRC | 14:59 | |
| marun | sdague: Not yet, waiting for the tests to finally pass before I talk to him. | 14:59 |
| *** sabeen1 has joined #openstack-infra | 14:59 | |
| openstackgerrit | Salvatore Orlando proposed openstack-infra/project-config: Allow VMware NSX CI to vote on stackforge/vmware-nsx https://review.openstack.org/147192 | 14:59 |
| sdague | the config syntax is pretty verbose there, it would make the current layout really hard to review | 14:59 |
| marun | sdague: I was aiming for self-documenting. How would you suggest making it more terse without affecting readability? | 15:00 |
| sdague | I don't know, I just know that if all the projects added that to their various incantations it would be pretty unreviewable | 15:01 |
| marun | sdague: I think some guidelines are all that is required. | 15:02 |
| *** dustins_ has quit IRC | 15:02 | |
| marun | sdague: It would be easy enough to have a global skip rule for doc/.* and .*\.rst and [project_name]/tests/.* | 15:02 |
| *** baoli has joined #openstack-infra | 15:03 | |
| marun | sdague: hopefully projects that deviate from this convention would be few and far between | 15:03 |
| marun | sdague: but I'm not sure I follow the logic of 'unreviewable'. | 15:03 |
| *** pc_m_ has quit IRC | 15:03 | |
| marun | sdague: I think this kind of feature has real value, and if reviewability is a concern, then maybe the structure of the configuration needs to be changed. | 15:04 |
| sdague | so having to mentally handle a ton of project specific regex and get that right... I expect we'd make a bunch of mistakes | 15:04 |
| marun | sdague: I don't think it matters, frankly. | 15:04 |
| marun | sdague: If a project gets a skip rule terribly wrong, they'll notice. | 15:05 |
| *** pkarikh has joined #openstack-infra | 15:05 | |
| sdague | marun: ... you'd be surprised | 15:05 |
| *** pblaho has quit IRC | 15:05 | |
| marun | sdague: so, you'd rather this capability simply didn't exist? | 15:05 |
| *** mohitsharma has quit IRC | 15:06 | |
| sdague | marun: I didn't say that | 15:06 |
| marun | sdague: because I'm not hearing you suggest a workable alternative, just that what I've proposed is problematic. | 15:06 |
| sdague | marun: correct, just because something doesn't look right to me doesn't mean I'm always going to have the right answer | 15:06 |
| marun | sdague: the functionality to skip is pretty well separated from the syntax, at least. | 15:06 |
| marun | sdague: so reworking to support something that doens't impact reviewability as much should be straightforward. | 15:07 |
| sdague | and I also have a ton of other things in flight, so I don't have the mental capacity to drop everything and focus on this one right now | 15:07 |
| marun | sdague: I do hear you on the potential configuration complexity this introduces. | 15:07 |
| marun | sdague: no worries, I don't mean to distract you. | 15:08 |
| *** AJaeger has joined #openstack-infra | 15:08 | |
| *** jp_at_hp has quit IRC | 15:09 | |
| *** mpaolino has joined #openstack-infra | 15:09 | |
| *** ssam2 has joined #openstack-infra | 15:09 | |
| pkarikh | Hi folks! Can anybody tell me, is there any case, when the release tag on an openstack library wouldn't be shown on PyPi? Looks like django_openstack_auth 1.1.8 has been released, but is not shown in PyPi index. https://pypi.python.org/pypi/django_openstack_auth | 15:10 |
| pkarikh | here the tag: https://github.com/openstack/django_openstack_auth/tree/1.1.8 | 15:10 |
| fungi | pkarikh: it's possible whoever tagged that didn't do it correctly, or that the upload tooling isn't configured correctly for that project, or that something broke during the release process | 15:11 |
| fungi | pkarikh: i'm looking now to see which is the case | 15:12 |
| *** amotoki_ has quit IRC | 15:12 | |
| marun | jeblair: Hi! I've submitted a proposal for adding a 'skip-if' job configuration option to zuul: https://review.openstack.org/#/c/146762/ | 15:13 |
| marun | jeblair: I think this capability is useful, and I'm curious to see what you think. | 15:13 |
| *** koolhead17 has quit IRC | 15:13 | |
| pkarikh | fungi: ok, thank you! | 15:13 |
| fungi | marun: he may not be awake right now... not sure what time of day it is at lca | 15:13 |
| *** amitgandhinz has joined #openstack-infra | 15:13 | |
| marun | fungi: 7:13am :) | 15:14 |
| fungi | marun: in new zealand? | 15:14 |
| marun | fungi: your feedback is also welcome. | 15:14 |
| marun | fungi: oh, for some reason I thought he was in CA | 15:14 |
| fungi | lca | 15:14 |
| *** mase_home has quit IRC | 15:14 | |
| fungi | (linuxconf au) | 15:14 |
| marun | ah, ok | 15:14 |
| marun | I'm guessing no, then. | 15:14 |
| *** mriedem has joined #openstack-infra | 15:16 | |
| fungi | pkarikh: it looks like it was rejected by pypi for permissions reasons http://logs.openstack.org/50/50e001401763070861ca41634d992aaa7682b963/release/django_openstack_auth-pypi-both-upload/29abd81/console.html.gz | 15:16 |
| *** avozza is now known as zz_avozza | 15:16 | |
| fungi | pkarikh: i'll try retriggering it since it looks like our openstackci account is listed as a package index owner for that project on pypi | 15:17 |
| *** ddieterly has joined #openstack-infra | 15:18 | |
| pkarikh | fungi: ok, thank you once again! :) | 15:18 |
| *** amotoki_ has joined #openstack-infra | 15:18 | |
| *** erikmwilson has quit IRC | 15:18 | |
| *** _nadya_ has quit IRC | 15:19 | |
| *** dhritishikhar has joined #openstack-infra | 15:19 | |
| *** bhuvan has joined #openstack-infra | 15:19 | |
| fungi | pkarikh: same error when rerun just now http://logs.openstack.org/50/50e001401763070861ca41634d992aaa7682b963/release/django_openstack_auth-pypi-both-upload/29abd81,1/console.html | 15:19 |
| fungi | seeing if i can figure out what's set differently for this on pypi that could be causing it | 15:20 |
| *** mestery has joined #openstack-infra | 15:21 | |
| *** tonytan4ever has joined #openstack-infra | 15:21 | |
| *** sputnik13 has joined #openstack-infra | 15:24 | |
| *** stevemar has joined #openstack-infra | 15:24 | |
| *** koolhead17 has joined #openstack-infra | 15:25 | |
| *** koolhead17 has joined #openstack-infra | 15:25 | |
| openstackgerrit | Julien Danjou proposed openstack-dev/hacking: H105: also check for Authors and authors https://review.openstack.org/147201 | 15:27 |
| openstackgerrit | Dmitry Teselkin proposed openstack-infra/system-config: Make project_config branch variable https://review.openstack.org/147202 | 15:28 |
| *** jroll has quit IRC | 15:28 | |
| *** jroll has joined #openstack-infra | 15:28 | |
| *** hdd has joined #openstack-infra | 15:28 | |
| fungi | dstufft: when trying to upload django_openstack_auth-1.1.8-py2-none-any.whl with twine, getting "HTTPError: 403 Client Error: You are not allowed to edit 'django-openstack-auth' package information" even though the account with which we're authenticating is a package index owner according to https://pypi.python.org/pypi/django_openstack_auth | 15:30 |
| *** dustins has joined #openstack-infra | 15:30 | |
| fungi | dstufft: any ideas? is that maybe a misleading error message? | 15:30 |
| *** mfink has joined #openstack-infra | 15:31 | |
| openstackgerrit | Mauro Rodrigues proposed openstack-infra/subunit2sql: Convert tags metadata to string before persist it https://review.openstack.org/147205 | 15:32 |
| *** erikmwilson has joined #openstack-infra | 15:34 | |
| openstackgerrit | Aleksey proposed openstack-infra/storyboard: Convert db exceptions to api exceptions https://review.openstack.org/144442 | 15:35 |
| *** andreaf has quit IRC | 15:35 | |
| *** wenlock has joined #openstack-infra | 15:37 | |
| dhellmann | fungi, clarkb : is there a way to delete a tag from one of our git repos? I just tagged oslo.concurrency with a vastly wrong version # | 15:38 |
| *** e0ne is now known as e0ne_ | 15:38 | |
| *** baoli has quit IRC | 15:38 | |
| dhellmann | jeblair: ^^ | 15:38 |
| *** e0ne_ is now known as e0ne | 15:38 | |
| *** dkranz has joined #openstack-infra | 15:38 | |
| AJaeger | dhellmann, either of them can do it. | 15:39 |
| dhellmann | AJaeger: thanks | 15:39 |
| *** amotoki_ has quit IRC | 15:39 | |
| dhellmann | fungi, clarkb, jeblair : oslo.concurrency 1.4.1 should have been 0.4.1. It's not the end of the world, but if I can get a do-over I'd like to. | 15:39 |
| fungi | dhellmann: sort of | 15:40 |
| *** arielb has joined #openstack-infra | 15:40 | |
| *** arielb has quit IRC | 15:40 | |
| *** arielb has joined #openstack-infra | 15:40 | |
| fungi | dhellmann: we can delete the tag, and that deletion will propagate, but anyone who pulled in that repo (including our test and release automation infrastructure) will have indefinitely cached that tag | 15:41 |
| *** ddieterly has quit IRC | 15:41 | |
| dhellmann | fungi: yeah. I think I'm OK with that. | 15:41 |
| fungi | dhellmann: so, for example, any zuul mergers which handled the release jobs for that tag will henceforth think that 1.4.1 is still tagged and will generate dev versions accordingly | 15:41 |
| *** ddieterly has joined #openstack-infra | 15:42 | |
| dhellmann | when you say "cached", how long will zuul keep it? forever? | 15:42 |
| dhellmann | or does it re-clone repositories periodically | 15:42 |
| fungi | dhellmann: and if you ever accidentally reuse that tag, it's not going to get refreshed on any systems which previously saw it, so unexpected behavior (like release jobs using the old sha with that tag number) are expected | 15:42 |
| fungi | dhellmann: basically forever, or until we replace the merger workers with new machines | 15:43 |
| dhellmann | ugh, ok, so problems propogate | 15:43 |
| dhellmann | I guess I'll have to live with it | 15:43 |
| * dhellmann makes a note to add a sanity check to the release script | 15:43 | |
| fungi | yeah, and anyone who did a remote update will have that tag in their local clones forever (until they manually delete the tag themselves or reclone from scratch) | 15:43 |
| *** erikmwilson has quit IRC | 15:43 | |
| *** tomoe has quit IRC | 15:43 | |
| fungi | also we'd presumably need to go delete artifacts from other places (tarballs, pypi, et cetera) | 15:44 |
| *** erikmwilson has joined #openstack-infra | 15:44 | |
| fungi | it's complex and nuanced enough that we basically don't do it | 15:44 |
| fungi | because the last few times we've tried, we inevitably missed some piece which then caused problems later | 15:45 |
| openstackgerrit | Aleksey proposed openstack-infra/storyboard: Team and project groups delete methods https://review.openstack.org/143675 | 15:47 |
| *** r-daneel has joined #openstack-infra | 15:48 | |
| *** _nadya_ has joined #openstack-infra | 15:51 | |
| fungi | dhellmann: awesome release notes btw ;) | 15:54 |
| *** Sukhdev has joined #openstack-infra | 15:54 | |
| *** asselin has joined #openstack-infra | 15:55 | |
| *** Guest80979 is now known as mgagne | 15:57 | |
| *** mgagne has joined #openstack-infra | 15:57 | |
| clarkb | looks like hpcloud is still giving nodepool a hard time | 15:57 |
| openstackgerrit | Valeriy Ponomaryov proposed openstack-infra/project-config: Make manilclient functional tests job votable https://review.openstack.org/147212 | 15:58 |
| clarkb | fungi: maybe we should try another restart? we can either remove libkeepalive or tweak the numbers | 15:58 |
| dstufft | fungi: are you using twine>=1.4 | 15:58 |
| dstufft | dhellmann: fungi in the future (basically waiting on me to implement it), PyPI won't allow you to re-use a version you've already uploaded | 15:59 |
| *** Ala has quit IRC | 16:00 | |
| fungi | dstufft: twine version 1.4.0 | 16:01 |
| dstufft | fungi: hmm, the error with wheels and projects that have _ in their name was supposed to be fixed in 1.4 | 16:01 |
| dstufft | https://github.com/pypa/twine/pull/70 | 16:02 |
| dstufft | oh | 16:02 |
| dstufft | HM | 16:02 |
| *** ddieterly has quit IRC | 16:02 | |
| dstufft | I bet | 16:02 |
| dstufft | lol | 16:02 |
| dstufft | I think this is going to have to be fixed in PyPI itself | 16:02 |
| * fungi waits for the lolzy bits | 16:02 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack-infra/project-config: Do not use job 'gate-manila-tempest-dsvm-neutron' with manilaclient https://review.openstack.org/147216 | 16:03 |
| fungi | pkarikh: ^ so sounds like might actually be a bug with pypi | 16:03 |
| *** zz_sabari is now known as sabari | 16:04 | |
| dstufft | So PyPI treats - and _ as equivilant, except on upload for I don't know why. Previously twine would fail if a project was named foo-bar when trying to upload a wheel, because the wheel would change the name to foo_bar and then PyPI wouldn't treat it as the same thing and reject it with a message you saw | 16:04 |
| dstufft | so the fix was to normalize the names using pkg_resources, which changes it from foo_bar back to foo-bar | 16:04 |
| dstufft | however you have the opposite problem, your thing is called foo_bar on PyPI, and now we're normalizing it to foo-bar | 16:04 |
| *** ryanpetrello_ has joined #openstack-infra | 16:04 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack-infra/project-config: Do not use job 'gate-manila-tempest-dsvm-neutron' with manilaclient https://review.openstack.org/147216 | 16:05 |
| dkranz | fungi: Is the gate broken today with icehouse? | 16:05 |
| *** imcsk8_ has quit IRC | 16:05 | |
| *** imcsk8 has joined #openstack-infra | 16:05 | |
| clarkb | dkranz: yes eventlet according to mail list threads | 16:05 |
| fungi | yeah, probably needs eventlet capped | 16:05 |
| * fungi checks for an icehouse equivalent of the juno patch | 16:06 | |
| BobBall | fungi/dstufft: It seems that simply trying to install the requirements from nodepool gives the "no module named version" error - Can either of you reproduce that? http://paste.openstack.org/show/157936/ | 16:07 |
| clarkb | thinking about hpcloud a bit more, maybe we should just convince requests/novaclient to use a new connection for each operation? | 16:07 |
| fungi | dkranz: https://review.openstack.org/136717 | 16:07 |
| *** ryanpetrello has quit IRC | 16:08 | |
| *** ryanpetrello_ is now known as ryanpetrello | 16:08 | |
| *** zz_avozza is now known as avozza | 16:08 | |
| dkranz | fungi: thanks | 16:08 |
| *** shayneburgess has joined #openstack-infra | 16:08 | |
| fungi | the requirements repo caps for that already merged to stable branches a couple hours ago | 16:09 |
| *** jp_at_hp has joined #openstack-infra | 16:09 | |
| *** pc_m has joined #openstack-infra | 16:09 | |
| clarkb | or use http://docs.python-requests.org/en/latest/user/advanced/#timeouts | 16:10 |
| fungi | clarkb: definitely inefficient, but might provide a useful debugging datapoint | 16:10 |
| pkarikh | fungi, dstufft: ok, looks like I understand. | 16:10 |
| *** alexpilotti_ has joined #openstack-infra | 16:10 | |
| *** banix has joined #openstack-infra | 16:10 | |
| pkarikh | fungi: so, now I just need to wait? | 16:11 |
| fungi | pkarikh: yeah, it may take a little time to get fixed | 16:12 |
| fungi | sounds nontrivial | 16:12 |
| clarkb | fungi: I can confirm no tasks completed in hpcloud since they stopped at 0029ish UTC | 16:12 |
| clarkb | or use a different pypi package name right? | 16:12 |
| *** alexpilotti has quit IRC | 16:12 | |
| *** alexpilotti_ is now known as alexpilotti | 16:12 | |
| clarkb | or don't use wheels? | 16:12 |
| dstufft | fungi: if you want to work around it you can upload using the web UI I think, or upload using an older twine | 16:13 |
| fungi | dstufft: yeah, but i'd like to be able to use this to confirm once it's suspected fixed | 16:13 |
| pkarikh | fungi: ok, thank you! | 16:13 |
| fungi | this package has already been not updated on pypi for a month, so a little longer isn't likely to hurt | 16:14 |
| dstufft | fungi: no worries, I'll poke at pypi in a little bit | 16:14 |
| dstufft | I have stand up in 15 minutes or so | 16:14 |
| fungi | dstufft: good idea--helps prevent blood clots | 16:15 |
| dstufft | lol | 16:15 |
| sigmavirus24 | twine problems? | 16:15 |
| *** markvoelker has joined #openstack-infra | 16:15 | |
| dstufft | I'm not actually standing very long right now :( stupid sprained ankle | 16:15 |
| openstackgerrit | Nikolay Markov proposed openstack-infra/project-config: Add Fuel Plugins to Stackforge https://review.openstack.org/147169 | 16:15 |
| dstufft | sigmavirus24: sort of, though I don't think it's exactly related to twine, the fix for things with a _ in the name doesn't seem to be working correctly, I think it needs fixed in PyPI though | 16:16 |
| * fungi imagines stereotypical japanese company morning mandatory group calisthenics | 16:16 | |
| *** ronis has quit IRC | 16:16 | |
| *** dmsimard_away is now known as dmsimard | 16:17 | |
| clarkb | we don't overridethe default connection_pool value of false when creating our novaclient Client object in nodepool | 16:17 |
| *** rkukura has joined #openstack-infra | 16:17 | |
| sigmavirus24 | dstufft: we could also add this to the list of things to retry | 16:17 |
| sigmavirus24 | (when we add retries) | 16:17 |
| fungi | clarkb: so in theory it's already not pooling there? | 16:18 |
| *** pcrews has joined #openstack-infra | 16:18 | |
| fungi | clarkb: it seemed like it was though | 16:18 |
| dstufft | sigmavirus24: eh, I'd argue that it's just wrong for PyPI to only sometimes consider certain things equivilant | 16:18 |
| sigmavirus24 | fair | 16:18 |
| clarkb | fungi: ya it definitely looks like it is, but I Think that may be because the connection isn't dying | 16:18 |
| clarkb | fungi: so the since connection we opened is still sitting there all happy | 16:18 |
| clarkb | s/since/single/ | 16:19 |
| fungi | causing us to block rather than start a new connection | 16:19 |
| clarkb | yes, that is my working theory at least | 16:19 |
| *** arielb has quit IRC | 16:19 | |
| fungi | tteggel: gilliard: are either of you around at the moment? | 16:20 |
| clarkb | fungi: since we serialize tasks on each manager to make rate limiting sane | 16:20 |
| clarkb | though we do have 5 managers that all roughly derp at the same time | 16:20 |
| *** arxcruz has quit IRC | 16:21 | |
| tteggel | fungi: yes | 16:22 |
| *** otter768 has joined #openstack-infra | 16:22 | |
| tteggel | how can I help? | 16:22 |
| *** bhuvan has quit IRC | 16:22 | |
| fungi | tteggel: we're getting odd behavior with api connections | 16:22 |
| *** armax has joined #openstack-infra | 16:23 | |
| clarkb | fungi: looks likenodepool can set a requests request timeout though so I am going to work on a nodepool change to support that | 16:23 |
| fungi | tteggel: after a few minutes they seem to go dead. still established at our end but all 5 of the connections we've opened go dead at about the same time | 16:23 |
| *** krtaylor has quit IRC | 16:23 | |
| fungi | tteggel: to region b/1.1 east/whatever the cool kids are calling it these days | 16:24 |
| *** Masahiro has joined #openstack-infra | 16:24 | |
| clarkb | and it happens consistently. Restart the service to open new sockets and within a short window of time same thing | 16:24 |
| fungi | tteggel: seems like it's started in the past week, possibly over the weekend | 16:24 |
| tteggel | fungi: clarkb: i'll see if there have been any changes to our endpoints | 16:25 |
| *** uvirtbot has quit IRC | 16:25 | |
| fungi | no obvious changes on our end that should account for it, and not seeing the same issue with rackspace region endpoints just hpcloud | 16:25 |
| *** spzala has joined #openstack-infra | 16:25 | |
| *** baoli has joined #openstack-infra | 16:26 | |
| openstackgerrit | gordon chung proposed openstack/requirements: add elasticsearch python client https://review.openstack.org/147229 | 16:26 |
| clarkb | we can go back to logs/graphite to figure out approximately when it started but from memory it was early morning PST yesterday | 16:26 |
| *** avozza is now known as zz_avozza | 16:26 | |
| clarkb | about 1300 UTC maybe | 16:27 |
| *** otter768 has quit IRC | 16:27 | |
| fungi | oh, perhaps. i thought monday was bad too but maybe not? checking graphs now for a rough idea | 16:27 |
| *** Masahiro has quit IRC | 16:28 | |
| *** reed has joined #openstack-infra | 16:29 | |
| tteggel | fungi: clarkb: checking our change tickets for all the bits in the API path | 16:29 |
| *** ildikov has quit IRC | 16:29 | |
| *** Longgeek has quit IRC | 16:29 | |
| *** rakhmerov has quit IRC | 16:30 | |
| *** pkarikh has quit IRC | 16:30 | |
| *** dteselkin has quit IRC | 16:30 | |
| *** tsufiev has quit IRC | 16:30 | |
| *** HeOS_ has quit IRC | 16:30 | |
| *** teran has quit IRC | 16:30 | |
| *** HeOS_ has joined #openstack-infra | 16:31 | |
| *** pkarikh has joined #openstack-infra | 16:31 | |
| *** devlaps has joined #openstack-infra | 16:31 | |
| *** yfried has joined #openstack-infra | 16:31 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Add support for an API timeout to nodepool https://review.openstack.org/147231 | 16:32 |
| *** teran has joined #openstack-infra | 16:32 | |
| *** dteselkin has joined #openstack-infra | 16:32 | |
| fungi | hard to tell from graphs, might have started ~20:00 utc on monday | 16:32 |
| fungi | or might have been around 1300-1400 yesterday | 16:33 |
| *** tsufiev has joined #openstack-infra | 16:33 | |
| fungi | the overnight lull in activity makes it nonobvious | 16:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Set api timeouts on hpcloud to one minute https://review.openstack.org/147233 | 16:34 |
| clarkb | I think that pair of changes will set up timeouts. I think 60 seconds is generous too | 16:34 |
| sdague | clarkb: ok, monkeypatch question time | 16:35 |
| sdague | https://review.openstack.org/#/c/146899/1/nova/tests/fixtures.py,cm | 16:35 |
| jesusaurus | I agree that 60 seconds is probably generous | 16:35 |
| sdague | I thought we came to the realization that I need that line | 16:35 |
| sdague | otherwise the patch might not apply | 16:35 |
| *** achanda has joined #openstack-infra | 16:36 | |
| fungi | clarkb: is that second change going to cause problems if it merges before restarting nodepoold with the first applied? | 16:36 |
| clarkb | sdague: it should only be necessary if the path you are monkey patching is not a module iirc | 16:36 |
| clarkb | sdague: fixtures will properly import modules for monkey patching but not a class orfunctions | 16:36 |
| clarkb | fungi: I don't think so. nodepool should just ignore that piece of config | 16:37 |
| fungi | cool beans | 16:37 |
| sdague | clarkb: so it's not needed there because it's a module? | 16:37 |
| clarkb | fungi: also we need to reenable puppet on nodepool.o.o to get those in. We will lose the libkeepalive configs but I am fine with that at this point | 16:37 |
| sdague | ok | 16:37 |
| *** rakhmerov has joined #openstack-infra | 16:37 | |
| fungi | clarkb: i'm +2 on both changes if we want to press forward taking them for a spin | 16:37 |
| *** hashar_ has quit IRC | 16:38 | |
| jesusaurus | clarkb: nodepool will ignore keys in the config it doesn't understand, right? so the second change doesn't strictly depend on the first? | 16:38 |
| clarkb | jesusaurus: correct | 16:38 |
| *** dizquierdo has quit IRC | 16:38 | |
| fungi | unlike zuul which does a thorough validation on its yaml input | 16:38 |
| clarkb | fungi: I think my only concern is tuning the timeout value so we don't timeout improperly on some expected long requests | 16:38 |
| clarkb | but we should be able to do that on the fly with my change (it checks if the value changes) | 16:39 |
| fungi | clarkb: do we do a block/wait on nova boot calls, waiting for an affirmative response? | 16:39 |
| fungi | i've sometimes seen that take a long time | 16:39 |
| *** markvoelker has quit IRC | 16:40 | |
| clarkb | fungi: I think nova is async there once you are scheduled | 16:40 |
| clarkb | its the node listings that can be slow iirc | 16:40 |
| *** dkranz has quit IRC | 16:40 | |
| fungi | okay, so just waiting for the scheduler to acknowledge on boot | 16:40 |
| jesusaurus | yeah, you only wait for the 'ok i'll get around to that' from nova | 16:40 |
| *** teran has quit IRC | 16:41 | |
| *** pkarikh has quit IRC | 16:41 | |
| *** rakhmerov has quit IRC | 16:41 | |
| *** HeOS_ has quit IRC | 16:41 | |
| *** tsufiev has quit IRC | 16:41 | |
| *** dteselkin has quit IRC | 16:41 | |
| *** HeOS_ has joined #openstack-infra | 16:41 | |
| fungi | and yeah, since it uses the nova list call to determine when a node has switched to ready, we will definitely pile up building nodes if we tune that too short | 16:41 |
| *** dkranz has joined #openstack-infra | 16:41 | |
| *** krtaylor has joined #openstack-infra | 16:41 | |
| *** pkarikh has joined #openstack-infra | 16:41 | |
| *** jlibosva has quit IRC | 16:42 | |
| fungi | i should go ahead and get lunch and run errands now so i'll be around for tuning later | 16:42 |
| clarkb | ya I can do morning things too then come back to this | 16:42 |
| *** teran has joined #openstack-infra | 16:42 | |
| clarkb | otherwise I risk splitbraining | 16:42 |
| *** dteselkin has joined #openstack-infra | 16:43 | |
| *** jlibosva has joined #openstack-infra | 16:43 | |
| *** tsufiev has joined #openstack-infra | 16:44 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Fix python3 unittests https://review.openstack.org/145651 | 16:45 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Explain obj_to_dict https://review.openstack.org/146724 | 16:45 |
| *** hdd has quit IRC | 16:45 | |
| Shrews | mordred: that dependency order needed fixed ^^^^ | 16:45 |
| fungi | k, heading out now. back asap | 16:45 |
| *** jlibosva has quit IRC | 16:46 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Remove py26 support https://review.openstack.org/146717 | 16:46 |
| *** rakhmerov has joined #openstack-infra | 16:46 | |
| *** ChuckC_ has joined #openstack-infra | 16:47 | |
| *** gyee has joined #openstack-infra | 16:48 | |
| openstackgerrit | Aleksey proposed openstack-infra/storyboard: Modified users put method https://review.openstack.org/147246 | 16:48 |
| clarkb | tteggel: I can get you source ip and port info if you need it too | 16:49 |
| *** shayneburgess has quit IRC | 16:49 | |
| *** belmoreira has joined #openstack-infra | 16:50 | |
| openstackgerrit | Aleksey proposed openstack-infra/storyboard: Modified users put method https://review.openstack.org/147246 | 16:50 |
| *** ChuckC has quit IRC | 16:51 | |
| *** ssam2 has quit IRC | 16:51 | |
| *** radez_g0n3 is now known as radez | 16:54 | |
| *** Guest8210 is now known as redrobot | 16:54 | |
| *** hashar has joined #openstack-infra | 16:55 | |
| *** markus_z has quit IRC | 16:55 | |
| *** melwitt has joined #openstack-infra | 16:58 | |
| *** sarob has joined #openstack-infra | 16:58 | |
| *** hdd has joined #openstack-infra | 16:58 | |
| *** _nadya_ has quit IRC | 16:59 | |
| *** dims__ has quit IRC | 17:00 | |
| *** dims__ has joined #openstack-infra | 17:01 | |
| *** skolekonov has quit IRC | 17:03 | |
| *** achanda has quit IRC | 17:03 | |
| *** dtantsur is now known as dtantsur|afk | 17:03 | |
| *** coolsvap is now known as coolsvap|afk | 17:03 | |
| *** Ryan_Lane has joined #openstack-infra | 17:04 | |
| *** dims__ has quit IRC | 17:05 | |
| *** ddieterly has joined #openstack-infra | 17:05 | |
| tteggel | clarkb: yes please. no documented changes that could impact this | 17:05 |
| tteggel | clarkb: and our graphite is down so can't get connection stats right now | 17:06 |
| *** ihrachyshka has quit IRC | 17:06 | |
| *** achanda has joined #openstack-infra | 17:07 | |
| *** MaxV has quit IRC | 17:07 | |
| *** ildikov has joined #openstack-infra | 17:08 | |
| *** sarob has quit IRC | 17:09 | |
| openstackgerrit | Mark Vanderwiel proposed openstack-infra/project-config: stackforge cookbook-ceph is being removed https://review.openstack.org/147250 | 17:10 |
| *** Longgeek has joined #openstack-infra | 17:10 | |
| clarkb | tteggel: I think they are sources 104.130.155.213:42046 104.130.155.213:42034 104.130.155.213:42047 104.130.155.213:42006 104.130.155.213:42033 all to 206.164.176.21:443 | 17:10 |
| *** dimtruck is now known as zz_dimtruck | 17:10 | |
| tteggel | clarkb: ack, thanks | 17:10 |
| *** dhritishikhar has quit IRC | 17:10 | |
| *** bdpayne has joined #openstack-infra | 17:10 | |
| clarkb | tteggel: they all show as ESTABLISHED on our side | 17:11 |
| clarkb | also tcp | 17:11 |
| *** garyk has joined #openstack-infra | 17:11 | |
| *** asselin has quit IRC | 17:11 | |
| garyk | dtroyer: are you around? | 17:11 |
| dtroyer | garyk: am now | 17:12 |
| garyk | sdague: dtroyer:is there any chance that you guys can please look at https://review.openstack.org/144312. we need to get this in so that we can start to get our CI posting | 17:12 |
| *** zz_jgrimm is now known as jgrimm | 17:12 | |
| zaro | morning | 17:13 |
| annegentle_ | dhellmann: I have some sphinx theme questions when you have a minute or two. I can set up a meeting too, just let me know your preference. | 17:13 |
| sdague | garyk: so why not as a plugin? | 17:14 |
| *** banix has left #openstack-infra | 17:14 | |
| garyk | sdague: part of it is as plaugin and part is third party | 17:15 |
| tteggel | clarkb: can I find tennant ids from anywhere? | 17:15 |
| clarkb | tteggel: ya I can get those to you too | 17:15 |
| garyk | sdague: the reason for this is due to the neutron decomposition. that is we will have a shim in neutron and then that will use a third party library | 17:16 |
| sdague | garyk: right, but more importantly, why not done as an external plugin entirely | 17:16 |
| *** zz_dimtruck is now known as dimtruck | 17:16 | |
| garyk | sdague: in devstack or neutron? | 17:16 |
| sdague | in devstack, put all the logic in stackforge/vmware-nsx per the external plugin specification | 17:17 |
| *** jpich has quit IRC | 17:17 | |
| garyk | sdague: ok, let me take a look and see what we can do | 17:18 |
| sdague | garyk: thanks | 17:19 |
| dtroyer | sdague: it's a neutron plugin now… I haven't though through the path to make those external, there are some assumptions about file locations IIRC. I'll be interested to see what garyk finds | 17:19 |
| openstackgerrit | yolanda.robla proposed openstack-infra/infra-specs: Add spec for duplicate stories https://review.openstack.org/147253 | 17:19 |
| *** hockeynut has joined #openstack-infra | 17:20 | |
| dtroyer | garyk: another thing to keep in mind is sdague's '-o nounset' work and the effect it'll have on your config vars when it merges | 17:20 |
| *** links has quit IRC | 17:21 | |
| garyk | dtroyer: ok, thanks! | 17:21 |
| *** e0ne is now known as e0ne_ | 17:21 | |
| *** ronis has joined #openstack-infra | 17:24 | |
| *** Longgeek has quit IRC | 17:24 | |
| *** roeyc has joined #openstack-infra | 17:25 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: Add sanity check script for new version tags https://review.openstack.org/147256 | 17:25 |
| BobBall | Has anyone tried to pip install nodepool lately? :) It fails for me with "ImportError: No module named version" http://paste.openstack.org/show/157976/ Now using python-pip-1.3.1-4.el6.noarch python-setuptools-0.6.10-3.el6.noarch | 17:27 |
| BobBall | I realised that I didn't need anything complicated so hopefully this is an issue that's easy to understand... | 17:27 |
| *** shashankhegde has joined #openstack-infra | 17:28 | |
| *** rwsu has joined #openstack-infra | 17:29 | |
| BobBall | I've also had the same issue with python-pip 1.5.4-1 and python-setuptools 3.3-1ubuntu1 which is a much newer setuptools | 17:29 |
| *** derekh has quit IRC | 17:30 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: Add sanity check script for new version tags https://review.openstack.org/147256 | 17:30 |
| clarkb | BobBall: let me test really quickly | 17:30 |
| *** Ryan_Lane has quit IRC | 17:31 | |
| *** asselin has joined #openstack-infra | 17:32 | |
| *** e0ne_ has quit IRC | 17:32 | |
| *** yamahata has quit IRC | 17:32 | |
| *** _nadya_ has joined #openstack-infra | 17:32 | |
| *** EmilienM is now known as EmilienM|afk | 17:32 | |
| yolanda | fungi, clarkb, i'm having an issue with excess of logging in jenkins, i was looking at the logger.conf upstream but i cannot find that anymore? | 17:32 |
| *** zz_avozza is now known as avozza | 17:33 | |
| yolanda | i see that you point at that on jenkins.default, but where is that file generated? | 17:33 |
| *** hashar has quit IRC | 17:33 | |
| *** bknudson has quit IRC | 17:33 | |
| clarkb | yolanda: https://git.openstack.org/cgit/openstack-infra/puppet-jenkins/tree/files/logger.conf | 17:34 |
| clarkb | BobBall: it seems to work here | 17:35 |
| BobBall | gah! :( | 17:35 |
| clarkb | BobBall: I made a virtualenv then `pip install -U pip setuptools && pip install $PATH_TO_NODEPOOL` | 17:35 |
| BobBall | which version of python? 3? | 17:35 |
| clarkb | BobBall: 2, I am pretty sure we don't support 3 | 17:36 |
| BobBall | Damn. Then I'm running out of ideas of what's going wrong... | 17:36 |
| *** jcoufal_ has quit IRC | 17:36 | |
| *** shayneburgess has joined #openstack-infra | 17:36 | |
| yolanda | clarkb, thx | 17:36 |
| yolanda | i was looking in system-config and project-config, i really need to take a look on how that works now | 17:37 |
| clarkb | BobBall: how are you installing it? | 17:38 |
| BobBall | Exactly as you say. Let me create a new VM in RAX to try and reproduce there. | 17:38 |
| *** jistr has quit IRC | 17:39 | |
| *** mrmartin has joined #openstack-infra | 17:39 | |
| *** dhritishikhar has joined #openstack-infra | 17:39 | |
| jesusaurus | yolanda: that means we should switch over to the new split-out module \o/ | 17:41 |
| *** sabari is now known as zz_sabari | 17:42 | |
| yolanda | yes, we need to properly be following that | 17:42 |
| *** zz_sabari is now known as sabari | 17:42 | |
| SlickNik | AJaeger: around? | 17:43 |
| *** avozza is now known as zz_avozza | 17:43 | |
| SlickNik | AJaeger: Seeing the seeing the trove-tox-doc-publish-checkbuild fail pretty consistently with an ArchiverException | 17:44 |
| SlickNik | AJaeger: Something about "Negative seek offset" | 17:44 |
| SlickNik | example http://logs.openstack.org/22/136722/3/check/gate-trove-tox-doc-publish-checkbuild/ed29d0c/console.html | 17:44 |
| *** _nadya_ has quit IRC | 17:44 | |
| SlickNik | Any ideas what's going on? | 17:45 |
| *** e0ne has joined #openstack-infra | 17:45 | |
| *** sarob has joined #openstack-infra | 17:45 | |
| clarkb | SlickNik: bad package archive? | 17:45 |
| *** KanagarajM has joined #openstack-infra | 17:46 | |
| clarkb | the checksum for that package is wrong according to the log too | 17:47 |
| *** sabari is now known as zz_sabari | 17:47 | |
| clarkb | surprised it didn't fail there | 17:47 |
| *** achanda has quit IRC | 17:47 | |
| clarkb | they are not new files though so odd that it would start failing now | 17:47 |
| SlickNik | clarkb: yeah — not sure why it started happening all of a sudden though | 17:48 |
| clarkb | tteggel: find anything? I am going to step away for a bit again but do let us know if we can provide more information | 17:48 |
| tteggel | clarkb: fungi: the issue must be upstream of the bits that I'm allowed to touch | 17:49 |
| *** Longgeek has joined #openstack-infra | 17:49 | |
| clarkb | tteggel: ya I would not be surprised if its a firewall or load balancer being special | 17:49 |
| tteggel | clarkb: yup, we have api gateway boxen too :( | 17:49 |
| *** dannywilson has joined #openstack-infra | 17:50 | |
| clarkb | next step for us at least is to try the timeouts | 17:50 |
| *** annegentle_ has quit IRC | 17:51 | |
| tteggel | clarkb: good idea. not sure how loaded these other teams are right now. I've engaged with them and raised tickets | 17:51 |
| clarkb | tteggel: great thank you for the help | 17:52 |
| tteggel | clarkb: welcome, sorry I can't get to the bottom of it | 17:52 |
| SlickNik | clarkb: Looks like the download failed — the file is ~293K, the log seems to indicate that only 184b were downloaded. | 17:52 |
| SlickNik | Looking into it. | 17:54 |
| BobBall | clarkb: Just reproduced it in a RAX VM I'm using for devstack testing... | 17:54 |
| *** KanagarajM has quit IRC | 17:54 | |
| *** shashankhegde has quit IRC | 17:54 | |
| *** Sukhdev has quit IRC | 17:54 | |
| *** sarob has quit IRC | 17:54 | |
| clarkb | SlickNik: would be intereting if you get the same incomplete download to our laptop | 17:54 |
| clarkb | BobBall: can you paste the entire output of installing it? | 17:54 |
| *** ChuckC has joined #openstack-infra | 17:55 | |
| zaro | Gerrit old change screen is eminent! https://gerrit-review.googlesource.com/#/c/49601/ | 17:55 |
| zaro | ^ opps, meant removal of Gerrit old change screen | 17:55 |
| clarkb | I guess I better prep to use gerrty for everything | 17:56 |
| jroll | ^ | 17:56 |
| zaro | old or new screen, i'm better with the browser | 17:57 |
| *** ChuckC_ has quit IRC | 17:57 | |
| BobBall | clarkb: http://paste.openstack.org/show/157987/ | 17:58 |
| *** signed8bit has joined #openstack-infra | 17:58 | |
| clarkb | BobBall: you are pip installing from pypi? it may be that old nodepool on pypi doesn't work with some new dependency | 17:59 |
| *** garyk has quit IRC | 17:59 | |
| clarkb | specifically extras? | 18:00 |
| clarkb | let me try that locally | 18:00 |
| BobBall | I'll double-test the latest checkout, but I did test that earlier | 18:00 |
| BobBall | extras is always coming from pypi though | 18:00 |
| *** thingee has joined #openstack-infra | 18:00 | |
| BobBall | and there hasn't been a newer version for over a year I think | 18:00 |
| *** dannywilson has quit IRC | 18:00 | |
| BobBall | version for 6 months | 18:00 |
| *** trey has joined #openstack-infra | 18:00 | |
| BobBall | Just tried the cloned version (with pip install -U .) and got the same error | 18:01 |
| clarkb | it could be that version of extras doesn't like some transitive thing | 18:01 |
| clarkb | so your case doesn't work for me but installing current nodepool from scratch does work | 18:01 |
| BobBall | I'm always installing from scratch | 18:02 |
| *** bhuvan has joined #openstack-infra | 18:02 | |
| clarkb | installing extras in a clean venv works | 18:03 |
| BobBall | yes | 18:03 |
| clarkb | so I think it must be an interaction against something in with old nodepool from pypi | 18:03 |
| BobBall | there is some dependency - I think an earlier install changes the behaviour which means that extras fails | 18:03 |
| BobBall | No, I mean earlier in that same install | 18:03 |
| BobBall | from nodepools' requirements | 18:03 |
| dhellmann | anteaya: I'm seeing what I think are spurious errors from the MineSweeper third-party CI system. Have you had similar reports from anyone else? | 18:03 |
| clarkb | BobBall: right | 18:04 |
| clarkb | BobBall: but it doesn't seem to effect nodepool from source at least not for me | 18:04 |
| *** e0ne has quit IRC | 18:04 | |
| * fungi returns | 18:04 | |
| clarkb | I wonder if this is pbr installing the egg info writer breaking things | 18:04 |
| BobBall | that makes a _lot_ of sense | 18:04 |
| clarkb | actually ya that would explain it \o/ | 18:04 |
| BobBall | I've seen lots of failures with the egg info stuff | 18:04 |
| clarkb | because it would work in a git repo for nodepool | 18:04 |
| clarkb | but may not work if things are silly in other cases | 18:04 |
| BobBall | trying to install docutils for example also gave an error there... | 18:05 |
| clarkb | BobBall: thats differnt | 18:05 |
| *** mpaolino has quit IRC | 18:05 | |
| BobBall | Ah ok shame :) | 18:05 |
| clarkb | docutils just completely failed to depend on a thing they dependend on | 18:05 |
| BobBall | Just confirmed again - fresh install in another VM from the github clone reproduced it for me... | 18:05 |
| clarkb | in this case I think the pbr egg info writer is running against extras and failing | 18:05 |
| BobBall | ok | 18:06 |
| clarkb | but I have no idea how to fix this | 18:07 |
| clarkb | mordred: ^ | 18:07 |
| BobBall | So why would it break for me though and not you though? | 18:07 |
| *** sputnik13 has quit IRC | 18:07 | |
| clarkb | BobBall: either because I am running in a git repo so the egg info writer cheats or due to a difference in entry point loaders? | 18:08 |
| clarkb | I still don't know why it works for me | 18:08 |
| BobBall | ok | 18:08 |
| *** hdd has quit IRC | 18:08 | |
| clarkb | hrm `pip install pbr && pip install extras` works | 18:09 |
| *** spzala has quit IRC | 18:09 | |
| clarkb | so does pip install -U pbr extras so this may be more complicated than I thuought | 18:09 |
| BobBall | Both of those work for me too | 18:10 |
| clarkb | its definitely dying trying to load an egg info entry point based on that traceback | 18:10 |
| *** chlong has joined #openstack-infra | 18:10 | |
| *** prad has joined #openstack-infra | 18:10 | |
| *** harlowja_away is now known as harlowja | 18:10 | |
| *** signed8bit has quit IRC | 18:11 | |
| BobBall | I get the error from pip install -U gear | 18:12 |
| BobBall | without worrying about pbr or extras | 18:12 |
| *** achanda has joined #openstack-infra | 18:12 | |
| *** Masahiro has joined #openstack-infra | 18:12 | |
| clarkb | fungi: tl;dr from before is tteggel looked into it but didn't have access to all the things so subitted some tickets | 18:14 |
| *** dhritishikhar has quit IRC | 18:14 | |
| clarkb | fungi: you ready to move forward on the timeout changes? I can reenable puppet on nodepool.o.o | 18:14 |
| *** zz_avozza is now known as avozza | 18:14 | |
| fungi | clarkb: yeah, just caught up on scrollback. ready when you are | 18:15 |
| clarkb | fungi: ok enabling puppet now | 18:15 |
| *** signed8bit has joined #openstack-infra | 18:15 | |
| fungi | also bug 1410887 is definitely odd | 18:15 |
| *** Guest97290 has joined #openstack-infra | 18:16 | |
| *** BobBall is now known as BobBall_AWOL | 18:16 | |
| fungi | and uvirtbot is mia | 18:16 |
| clarkb | fungi: I think for now I will apply my system config via my puppet environment and fetch my nodepool change similarly | 18:16 |
| clarkb | BobBall_AWOL: I would look at the pbr egg info writer entrypoint fwiw | 18:16 |
| clarkb | BobBall_AWOL: I am pretty sure it is the cause | 18:16 |
| fungi | clarkb: fair enough, but if it seems to work then we should approve asap | 18:17 |
| clarkb | yup | 18:17 |
| *** sputnik13 has joined #openstack-infra | 18:17 | |
| clarkb | fungi: or are you comfortable just approving both? we can do that too, the changes are relatively simple to backout by removing the api timeout values from the config | 18:17 |
| *** Masahiro has quit IRC | 18:18 | |
| *** roeyc has quit IRC | 18:18 | |
| clarkb | oh apparently BobBall_AWOL's thing is hitting nodepool CI | 18:19 |
| fungi | clarkb: i was about to say, yes | 18:19 |
| fungi | http://logs.openstack.org/31/147231/1/check/gate-nodepool-pep8/0f40364/console.html | 18:19 |
| fungi | ImportError: No module named version | 18:19 |
| * fungi performs wider sweep for culprits | 18:20 | |
| clarkb | ya its basically the same as BobBall_AWOL's thing but hits pyyaml there instead of extras | 18:20 |
| *** ddieterly has quit IRC | 18:21 | |
| dhellmann | is there a list of contacts for third-party ci systems? I'm not finding anything in the wiki | 18:21 |
| *** banix has joined #openstack-infra | 18:22 | |
| fungi | dhellmann: third-party-announce@lists.openstack.org is where we send e-mail to let them know we've shut them off | 18:22 |
| *** sarob has joined #openstack-infra | 18:23 | |
| dhellmann | fungi: ok. I'm trying to figure out if MineSweeper is broken (at least w.r.t. oslo.vmware) or if there's something else going on that's causing it to report the same error with glance for a bunch of test runs. | 18:23 |
| *** jp_at_hp has quit IRC | 18:23 | |
| *** otter768 has joined #openstack-infra | 18:23 | |
| dhellmann | I'll email some of the folks at vmware and see if I can get in touch with someone that way | 18:23 |
| *** wuhg has quit IRC | 18:24 | |
| *** mrmartin has quit IRC | 18:24 | |
| clarkb | https://bitbucket.org/pypa/setuptools/src/1116e568f534ad8f4f41328a0f5fa183eb739c90/pkg_resources/__init__.py?at=11.3.1#cl-2320 is where the error comes from | 18:25 |
| clarkb | I think that means self.module_name is 'version' and there is no version module | 18:25 |
| fungi | most recent setuptools release was 6 days ago though | 18:26 |
| clarkb | fungi: happens with setuptools 10.0 too | 18:27 |
| clarkb | so I don't think setuptools is the issue | 18:27 |
| *** shashankhegde has joined #openstack-infra | 18:27 | |
| *** Ryan_Lane has joined #openstack-infra | 18:27 | |
| fungi | yeah, i'm having very little luck narrowing down the trigger here | 18:28 |
| *** otter768 has quit IRC | 18:28 | |
| fungi | worth noting that in this case pyyaml is the _next_ thing installed after extras | 18:28 |
| *** garyk has joined #openstack-infra | 18:28 | |
| fungi | though i don't immediately see how that might break anything | 18:29 |
| *** banix has quit IRC | 18:29 | |
| *** EmilienM|afk is now known as EmilienM | 18:29 | |
| *** patrickeast has joined #openstack-infra | 18:30 | |
| *** bhuvan_ has joined #openstack-infra | 18:31 | |
| *** funzo_ has quit IRC | 18:32 | |
| *** vigneshvar has joined #openstack-infra | 18:32 | |
| fungi | clarkb: strangely, pip freeze says pbr is not installed either | 18:33 |
| *** bhuvan has quit IRC | 18:34 | |
| clarkb | ya also note the PEP440Warnings for pbr | 18:34 |
| *** ociuhandu has quit IRC | 18:34 | |
| *** mwagner_lap has quit IRC | 18:35 | |
| fungi | uhuh | 18:36 |
| clarkb | pep440 warning goes away if I manually isntall latest pbr first but the error doesn't go away | 18:36 |
| *** bknudson has joined #openstack-infra | 18:36 | |
| *** ZZelle_ has joined #openstack-infra | 18:36 | |
| * mordred loves waking up to this conversation | 18:36 | |
| fungi | mordred: pbr has broken everything | 18:37 |
| fungi | no idea how yet, just placing blame | 18:37 |
| mordred | sweet | 18:37 |
| *** mrmartin has joined #openstack-infra | 18:37 | |
| * fungi refuels the blamethrower | 18:37 | |
| *** funzo has joined #openstack-infra | 18:37 | |
| *** arielb has joined #openstack-infra | 18:37 | |
| *** arielb has quit IRC | 18:37 | |
| *** arielb has joined #openstack-infra | 18:37 | |
| mordred | maybe we should replace it with a small kitten | 18:37 |
| clarkb | the leap being made here is the installs break when loading the egg info writer | 18:38 |
| clarkb | at least that is my reading of the traceback | 18:38 |
| fungi | so interesting note, the log mentions installing a wheel for pbr 0.10.7 pretty much first thing | 18:38 |
| fungi | and then immediately complains that the gear 0.5.5 tarball it's trying to install has a non-pep440-compliant version of 0.5.2.5.g5b3e942 | 18:39 |
| clarkb | probably because its already imported pbr with the old version? | 18:40 |
| *** signed8bit has quit IRC | 18:40 | |
| clarkb | but fixing that doesn't make the importerror go away | 18:40 |
| *** yamahata has joined #openstack-infra | 18:40 | |
| mordred | why does 0.5.5 have a version of 0.5.2.5.g5b3e942 | 18:40 |
| *** signed8bit has joined #openstack-infra | 18:40 | |
| *** signed8bit has quit IRC | 18:40 | |
| morganfainberg | mordred, more cats huh? | 18:40 |
| mordred | is this reproducible locally? | 18:40 |
| clarkb | mordred: yes just pip install gear to a virtualenv | 18:40 |
| fungi | oh, it's complaining about pbr 0.5.2.5.g5b3e942 | 18:40 |
| morganfainberg | mordred, that is a git-ism with pbr isn't it? the version? | 18:40 |
| clarkb | fungi: yes | 18:40 |
| fungi | not actually gear | 18:40 |
| fungi | i misread the log | 18:40 |
| fungi | so... we have a preinstalled pbr? | 18:41 |
| clarkb | morganfainberg: ya but in this case I think python is running the equivalent of `import version` and there is no python version module | 18:41 |
| rm_work | shameless plug for reviewers for https://review.openstack.org/#/c/146231/ (simple global-reqs change) | 18:41 |
| clarkb | fungi: I think maybe the setup requires finds that? | 18:41 |
| clarkb | but I think its unrelated. | 18:41 |
| fungi | but... from where?!? | 18:41 |
| clarkb | if you preinstall latest pbr warning goes away and importerror still happens | 18:41 |
| sdague | rm_work: isn't there an in flight set of changes to add barbican to nova using the old api? | 18:42 |
| *** e0ne has joined #openstack-infra | 18:42 | |
| rm_work | sdague: not sure, if so that needs to change because the old API literally *will not work* with the version of barbican-server that is in git-master currently | 18:42 |
| *** sarob_ has joined #openstack-infra | 18:43 | |
| rm_work | so IMO we should get this in sooner than later :P | 18:43 |
| sdague | ummm... so barbican doesn't provide any form of compatibility? that seems kind of ... wrong. | 18:43 |
| rm_work | it's not technically promoted yet | 18:44 |
| sdague | it's a project with releases out there | 18:44 |
| rm_work | they've cleaned up the last of the major issues in the 3.x branch of the client | 18:44 |
| rm_work | which was actually released on the Icehouse schedule | 18:44 |
| rm_work | anyone using 2.x now is... way behind | 18:44 |
| rm_work | Icehouse was their last major breaking release IIRC | 18:44 |
| *** rfolco has quit IRC | 18:44 | |
| rm_work | which was also their first real official release? | 18:45 |
| rm_work | not that it's actually packaged yet anyway | 18:45 |
| morganfainberg | clarkb, ahhhhh | 18:45 |
| *** sarob has quit IRC | 18:45 | |
| fungi | so, er, still no clue why extras and pyyaml are suddenly not able to import version | 18:46 |
| *** arosen has joined #openstack-infra | 18:46 | |
| rm_work | sdague: I think compatibility will only be guaranteed from 3.x+ | 18:46 |
| *** yfried is now known as yfried|afk | 18:46 | |
| rm_work | but redrobot would have a more official answer possibly | 18:46 |
| fungi | and also no idea why pip freeze doesn't mention pbr _at_all_ | 18:47 |
| mordred | Requirement already satisfied (use --upgrade to upgrade): setuptools in ./testgear/lib/python2.7/site-packages (from python-daemon->gear) | 18:47 |
| mordred | so that's a little evil | 18:47 |
| *** sputnik13 has quit IRC | 18:48 | |
| sdague | rm_work: so, basically, how a project acts in incubation more or less sets the tone for how they get evaluated later. A giant break like this is pretty much a no no. Granted, the incubation process is going away, but this looks pretty clown car to me. | 18:48 |
| *** e0ne is now known as e0ne_ | 18:48 | |
| rm_work | sec | 18:48 |
| rm_work | getting someone who might be able to explain better | 18:48 |
| rm_work | I am only peripherally related to the project so I don't want to get them in trouble for no reason :P | 18:48 |
| fungi | this also might explain why the log shows pbr-0.10.7-py2.7.egg getting installed over and over and over and... | 18:48 |
| *** woodster_ has joined #openstack-infra | 18:49 | |
| fungi | unless setup_requires is really just that dumb | 18:49 |
| mordred | fungi: it's python-daemon | 18:49 |
| rm_work | woodster_: hey | 18:49 |
| mordred | that is breaking things | 18:49 |
| clarkb | mordred: is it? | 18:49 |
| *** weshay has quit IRC | 18:49 | |
| sdague | mordred: so merge my gear drop :) | 18:49 |
| woodster_ | rm_work, hello | 18:49 |
| rm_work | sdague: woodster_ might have a better explanation for you | 18:49 |
| rm_work | woodster_: we're discussing https://review.openstack.org/#/c/146231/ | 18:49 |
| rm_work | see sdague's comment | 18:49 |
| clarkb | how does that affect extras? | 18:50 |
| mordred | something about installing python-daemon breaks subsequence egg-info calls | 18:50 |
| sdague | mordred: last night, in trying to get global requirements fully installable - https://review.openstack.org/#/c/147021/ | 18:50 |
| fungi | ouch | 18:50 |
| sdague | mordred: well it's installation blows up terribly | 18:50 |
| rm_work | woodster_: I was explaining that the compatibility break from 2.x to 3.x was because of the last set of major changes, which happened in Icehouse -- in preparation for being more stable henceforth | 18:50 |
| rm_work | woodster_: maybe I explained it badly | 18:51 |
| *** dhritishikhar has joined #openstack-infra | 18:51 | |
| clarkb | ya can confirm `pip install python-daemon && pip install extras` is sufficent | 18:51 |
| clarkb | so not pbr at all | 18:51 |
| fungi | sdague: 147021 doesn't really help _thos_ problem, which is pip/setuptools brokenness for things depending on gear | 18:51 |
| clarkb | now I feel bad for blaming the pbr egg info writer | 18:51 |
| openstackgerrit | Kiall Mac Innes proposed openstack-infra/project-config: Add designate-coverage to designate check pipeline https://review.openstack.org/147272 | 18:51 |
| sdague | fungi: oh, sorry, didn't realize there was another issue | 18:51 |
| fungi | so, er, latest python-daemon needs blacklisting in the gear reqs list (and elsewhere?) | 18:52 |
| clarkb | now to test pinning python-daemon | 18:52 |
| *** zz_sabari is now known as sabari | 18:52 | |
| clarkb | fungi: probably nodepool since it depends directly on it iirc | 18:52 |
| fungi | yeah | 18:52 |
| clarkb | fungi: and zuul and gerritbot and anything else that daemons | 18:52 |
| fungi | well, probably _anything_ depending on python-daemon directly | 18:52 |
| fungi | yeah | 18:52 |
| fungi | i'm checking their bug tracker now | 18:52 |
| arosen | if you set: export DEVSTACK_GATE_TEMPEST_ALLOW_TENANT_ISOLATION=0 do the tempest tests run serially? | 18:52 |
| woodster_ | rm_work, that's my recollection, but rebrobot would be good to weigh in here | 18:53 |
| rm_work | yeah he doesn't seem to be active right now (lunch?) | 18:53 |
| *** e0ne_ has quit IRC | 18:53 | |
| clarkb | python-daemon<2.0 works | 18:53 |
| fungi | fwiw python-daemon 2.0.3 is new as of today | 18:53 |
| *** Guest97290 is now known as annegentle | 18:53 | |
| *** weshay has joined #openstack-infra | 18:53 | |
| mordred | yup | 18:53 |
| woodster_ | rm_work, I think part of the change was to use more keystone client components and remove barbican-only stovepiping | 18:53 |
| fungi | fun, their bugs link requires an alioth account | 18:54 |
| mordred | they have ztag_svn_revision = 0 | 18:54 |
| mordred | in their setup.cfg - fwiw | 18:54 |
| *** sarob_ is now known as sarob | 18:54 | |
| clarkb | mordred: is that a thing that does a thing? | 18:54 |
| mordred | unknown - but it was what was breaking things the other day for sqlalchemy-migrate | 18:55 |
| mordred | so we _may_ have a bug in pbr that we're not accounting for | 18:55 |
| *** isq has joined #openstack-infra | 18:55 | |
| mordred | or it may be unrelated | 18:55 |
| rm_work | woodster_: so basically, fixing stuff that prior to incubation wasn't a priority, but would need to be fixed before gradution | 18:55 |
| morganfainberg | mordred, is that equivalent to using epoch in an rpm (which i also hear is a fantastic idea and should always be done /s) | 18:55 |
| morganfainberg | ? | 18:55 |
| woodster_ | sdaque, rm_work did a broken gate indicate the 2.x to 3.x incompatibility of barbican client? | 18:56 |
| clarkb | mordred: I think its unrelated as I did an install with no pbr and it fails | 18:56 |
| mordred | OH MY DEAR GOD | 18:56 |
| *** hdd has joined #openstack-infra | 18:56 | |
| *** isq has quit IRC | 18:56 | |
| clarkb | mordred: make a new venv and install python-daemon then install extras | 18:56 |
| mordred | their setup.py is evil | 18:56 |
| clarkb | it will fail and that shouldn't touch pbr at all | 18:56 |
| rm_work | woodster_: the gate issue I'm having is that I need to specifically disallow the 2.x client for our changes in Neutron-LBaaS, but global requirements doesn't allow this | 18:56 |
| *** isq has joined #openstack-infra | 18:57 | |
| clarkb | rm_work: and nova has work inflight to support 2.x | 18:57 |
| dstufft | python-daemon | 18:57 |
| dstufft | lol | 18:57 |
| rm_work | clarkb: though nova's code should hopefully be able to be replaced pre-Kilo release with the new Castellan interface, I'd hope | 18:57 |
| dstufft | I wonder if the latest version is still known broken | 18:57 |
| mordred | they are actually installing a module called version | 18:57 |
| mordred | top level | 18:57 |
| fungi | after some effort to register for an alioth account, i can confirm one of the two open bugs for python-daemon is this problem | 18:57 |
| mordred | py_modules=[ | 18:57 |
| mordred | "version", "test_version", | 18:57 |
| mordred | ], | 18:57 |
| clarkb | dstufft: yes we can confirm :) | 18:57 |
| fungi | bug title "Syntax error in ‘version’ module" | 18:57 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for cinder unit test bug 1410485 https://review.openstack.org/147061 | 18:58 |
| dstufft | I tabbed over to openstack-infra to see mordred saying "their setup.py is evil" which meant I had to read more :D | 18:58 |
| rm_work | woodster_: though I don't believe the 2.x branch of the client would actually be able to properly communicate with barbican-server as it is right now in git-master | 18:58 |
| *** TravT has quit IRC | 18:58 | |
| clarkb | I will put a python-daemon pin under my nodepool change | 19:00 |
| fungi | sounds good | 19:00 |
| *** pelix has quit IRC | 19:00 | |
| *** MaxV has joined #openstack-infra | 19:01 | |
| mordred | hahaha. their version module depends on docutils.core | 19:01 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Add support for an API timeout to nodepool https://review.openstack.org/147231 | 19:01 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Pin python-daemon to <2.0 https://review.openstack.org/147273 | 19:01 |
| clarkb | fungi: mordred ^ | 19:02 |
| *** rfolco has joined #openstack-infra | 19:02 | |
| *** pc_m has quit IRC | 19:03 | |
| *** pc_m has joined #openstack-infra | 19:03 | |
| fungi | mordred: the previous release used it without depending on it | 19:04 |
| dstufft | mordred: docutils.core, not distutils.core? | 19:04 |
| mordred | dstufft: yah | 19:04 |
| mordred | ImportError: No module named docutils.core | 19:04 |
| dstufft | w t f | 19:04 |
| *** sputnik13 has joined #openstack-infra | 19:04 | |
| mordred | class VersionInfoWriter(docutils.writers.Writer, object): | 19:05 |
| *** bdpayne has quit IRC | 19:05 | |
| mordred | """ Docutils writer to produce a version info JSON data stream. | 19:05 |
| woodster_ | clarkb, sdaque, rm_work, I don't recall API incompatibilities with 2.0, other than not having containers/consumers support perhaps? | 19:05 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 19:05 | |
| *** cdent has quit IRC | 19:06 | |
| garyk | dhellmann: you around? | 19:06 |
| *** e0ne has joined #openstack-infra | 19:06 | |
| *** sarob has quit IRC | 19:08 | |
| *** dannywilson has joined #openstack-infra | 19:09 | |
| clarkb | fungi: I had to recheck my puppet change it failed on the precise test and the log was not found | 19:09 |
| mordred | clarkb: +3 | 19:09 |
| clarkb | fungi: havne't dug up the log in jenkins yet but looking for it now | 19:09 |
| clarkb | mordred: danke | 19:09 |
| *** dhritishikhar has quit IRC | 19:10 | |
| *** alexpilotti has quit IRC | 19:10 | |
| *** ihrachyshka has joined #openstack-infra | 19:10 | |
| clarkb | https://jenkins04.openstack.org/job/gate-infra-puppet-apply-precise/797/consoleFull proxy error | 19:11 |
| clarkb | mordred: https://review.openstack.org/#/c/145645/ | 19:11 |
| clarkb | that should fix the proxy errors | 19:12 |
| clarkb | I hope | 19:12 |
| fungi | or will at least replace them with real errors | 19:13 |
| fungi | and give us something more specific to go on | 19:13 |
| clarkb | fungi: so looks like it may take a while to get these two changes merged. You ok with my slapping them on nodepool.o.o premerge? | 19:13 |
| fungi | yeah, i expected you would anyway | 19:15 |
| fungi | go for it | 19:15 |
| mordred | clarkb, fungi: do we still care about pypi-mirror? jedimike has a patch for it that solves a usage that they have | 19:15 |
| fungi | mordred: i say we make jedimike a pypi-mirror core reviewer | 19:16 |
| mordred | I'd be tempted to make a new core team for it since we don't really use it anymore | 19:16 |
| mordred | fungi: yeah - that | 19:16 |
| Kiall | Heya infra folks.. designate doing a testing sprint tomorrow, would be excellent to get https://review.openstack.org/#/c/147272/ merged beforehand - adds coverage report to the check pipeline for us :) | 19:16 |
| clarkb | no opposition from me | 19:16 |
| *** dizquierdo has joined #openstack-infra | 19:16 | |
| fungi | i mean, we were going to just retire it and set it read-only with big warnings about how it's unmaintained, but some tripleo and hpcloud people commented they were still using it | 19:17 |
| openstackgerrit | Merged openstack-infra/nodepool: Pin python-daemon to <2.0 https://review.openstack.org/147273 | 19:17 |
| mordred | jedimike: do you want to own pypi-mirror? | 19:17 |
| openstackgerrit | Merged openstack-infra/nodepool: Add support for an API timeout to nodepool https://review.openstack.org/147231 | 19:17 |
| jedimike | fungi, mordred, i'm organising meetings next week to discuss getting bandersnatch set up for gozer too, now I have a proof-of-concept snapshot script written | 19:17 |
| fungi | i'm pretty sure having _someone_ who uses it reviewing changes for it would be better than us currently ignoring it while there are people still relying on it | 19:17 |
| jedimike | mordred, yeah, i'll take it over! | 19:18 |
| *** achanda has quit IRC | 19:18 | |
| clarkb | ok nodepool is upgraded and has timeouts in the config | 19:18 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Make pypi-mirror owned by its own core team https://review.openstack.org/147277 | 19:19 |
| clarkb | fungi: ready for me to restart it? | 19:19 |
| fungi | yep | 19:19 |
| *** Rockyg has joined #openstack-infra | 19:19 | |
| mordred | jedimike: anybody else other than you we should put in that core team? | 19:19 |
| clarkb | we are off to the races | 19:20 |
| clarkb | mordred: at one time greghaynes was using it | 19:20 |
| mordred | yay races! | 19:20 |
| greghaynes | hrm? | 19:20 |
| greghaynes | oh, yes, I still run it actually | 19:20 |
| jedimike | mordred, don't think so, i don't think we're going to modify it that much | 19:21 |
| fungi | enovance has officially extended the proverbial olive branch http://lists.openstack.org/pipermail/openstack-infra/2015-January/002329.html | 19:21 |
| greghaynes | it hasnt broken yet so I havent had a reason to switch to anything else | 19:21 |
| * mordred will put jedimike and greghaynes in the core team just for fun | 19:21 | |
| jedimike | cool :) | 19:21 |
| greghaynes | sure, why not ;) | 19:21 |
| EmilienM | fungi: :) | 19:22 |
| mordred | fungi: woot! | 19:22 |
| mordred | does this mean we get olive oil | 19:22 |
| mordred | ? | 19:22 |
| EmilienM | lol | 19:22 |
| fungi | mordred: i hope it means we get bug fixes! ;) | 19:22 |
| mordred | ooh - or just a bowl of olives | 19:22 |
| fungi | olive-flavored bug fixes | 19:22 |
| sdague | mordred: https://review.openstack.org/#/c/146981/ - you're pov on this is important, I want to split appart the tests optimized for requirements and those for pbr, as I think we can drop the requirements test time if we do | 19:23 |
| mordred | meh. bug fixes don't taste as good as olives | 19:23 |
| sdague | but there might be other fall out that I don't understand | 19:23 |
| *** cschwede has joined #openstack-infra | 19:23 | |
| Kiall | thanks fungi / AJaeger for the +2's on that change :) | 19:23 |
| *** achanda has joined #openstack-infra | 19:23 | |
| fungi | Kiall: never let anyone say that i don't care about dns | 19:23 |
| Kiall | ;) | 19:23 |
| EmilienM | fungi: I called cschwede who is the main developer | 19:24 |
| clarkb | ok we are doing demand calculations now I think the restart initialization is done | 19:24 |
| fungi | EmilienM: cschwede: tristanC: thanks! | 19:25 |
| *** spzala has joined #openstack-infra | 19:25 | |
| clarkb | I did have a question about multiple nodes job assignment. Do you mean running a single job on more than one node? if so does the solution we currently have not work well? | 19:26 |
| *** yfried|afk is now known as yfried | 19:27 | |
| redrobot | rm_work back from lunch. | 19:27 |
| fungi | yeah, an official reply to that message on the ml is definitely in order, but we could probably hash through several of the points it makes in irc more effectively | 19:28 |
| tristanC | clarkb: I'm not entirely sure what is available right now, but the need is to have a job that requires more than one node to succeed (e.g., an openstack 3 node deployment functional tests) | 19:28 |
| fungi | tristanC: ooh--we already have that! | 19:28 |
| *** esker has joined #openstack-infra | 19:28 | |
| fungi | tristanC: nodepool subnodes | 19:28 |
| *** radez is now known as radez_g0n3 | 19:28 | |
| redrobot | sdague the Barbican team would prefer that projects use the 3.x release of python-barbicanclient, as it handles authentication in a much better way using Sessions from python-keystoneclient | 19:29 |
| clarkb | tristanC: we have that today and are using it to run 2 node devstack installs. The way it works is we have a primary node + X subnodes. The primary node connects to jenkins and is the brains of the operation | 19:29 |
| clarkb | tristanC: each node is given lists of the other nodes with an ssh key they they can use to talk to each other | 19:29 |
| clarkb | tristanC: its simple but seems to work well so far. | 19:29 |
| tristanC | oh well that's it, we haven't yet use nodepool so far... | 19:30 |
| fungi | tristanC: the current multi-nova example uses layer 2 over gre to establish a flat nova network between the nodes. effective | 19:30 |
| redrobot | sdague I understand that you're concerned about changes in incubation, but we feel that the 3.x line is a much better client. Basically we made a bunch of bad choices in 2.x and decided to fix them with a 3.x release. | 19:30 |
| redrobot | sdague to maintain backwards compatibility with a crappy api just for the sake of compatibility seems silly to me. | 19:30 |
| fungi | redrobot: probably means the current changes implementing barbican in nova need to switch to the "good" api then | 19:31 |
| clarkb | tristanC: the only real major downside from a testing perspective is the tests themselves have the responsibility to set up more of the environment | 19:31 |
| dhellmann | garyk: I'm back | 19:31 |
| clarkb | tristanC: which can be a small amount of overhead, but in practice isn't terrible | 19:32 |
| *** Hal_ has quit IRC | 19:32 | |
| fungi | clarkb: tristanC: i see that as a feature. leaves more of the design of the test environment up to the job using it | 19:32 |
| *** ddieterly has joined #openstack-infra | 19:32 | |
| redrobot | fungi possibly... but there's an effort on a generic interface called Castellan to abstract barbican away from projects like Nova and Cinder. The Johns Hopkins crew is working on patches for those. | 19:32 |
| clarkb | wait isn't barbican the abstraction? | 19:33 |
| *** adalbas has quit IRC | 19:33 | |
| fungi | clarkb: it's turtles all the way down | 19:33 |
| fungi | redrobot: perhaps the current nova changes should be wip/abandoned then? | 19:33 |
| tristanC | clarkb: thanks for the pointer. IIRC the problem was also about knowing when required nodes are available (e.g. using gearman) | 19:34 |
| *** belmoreira has quit IRC | 19:34 | |
| redrobot | clarkb yes, I thought so too, but projects who were integrating started abstracting Barbican, so to avoid code duplication we started Castellan to capture that abstraction | 19:34 |
| redrobot | clarkb maybe this explains it better https://review.openstack.org/#/c/138875/ | 19:34 |
| clarkb | redrobot: that tells me tbe existing abstraction is broken not that we need more abstractions | 19:35 |
| *** JamesK has joined #openstack-infra | 19:35 | |
| clarkb | fungi: still no ready nodes but haven't seen the connections all stop yet | 19:35 |
| redrobot | clarkb I wouldn't go so far as to call it broken. we're mainly trying to address the fact that projects don't want to integrate with Barbican directly. | 19:36 |
| clarkb | fungi: it does appear that we list floating IPs a lot and that may not be the quickest of operations. Something worht looking at later | 19:36 |
| mordred | sdague: seems like a great idea to me (requirements split) | 19:36 |
| fungi | clarkb: probably should cache that the same way as the instance list, if we don't already | 19:37 |
| fungi | tristanC: yeah, nodepool solves that by bringing all the subnodes in a group ready before adding the main node to jenkins | 19:38 |
| mordred | redrobot: if python-barbicanclient isn't using keystoneclient Sessions - what if I'm writing something that is using Sessions already and I want to get a barbicanclient instance? | 19:38 |
| clarkb | fungi: tristanC they are also forced to build in the same region/az | 19:38 |
| mordred | also - didn't we _just_ do all the work to get everyone using Sessions? | 19:38 |
| morganfainberg | redrobot, mordred, Session should be (at least) able to be used with barbicanclient | 19:38 |
| jedimike | greghaynes, i want to make pypi-mirror not fail if a single package doesn't install. It should just skip that package and log the error. Then we don't end up with mirrors not updating just because one package has a broken install. sound sensible? | 19:39 |
| redrobot | mordred python-barbicanclient 3.x does use sessions. We're trying to bump global requirements from 2.x to 3.x so that only the version that has Sessions is used by other projects. | 19:39 |
| morganfainberg | redrobot, mordred, if barbicanclient wants alternative ways to work without keystone sessions etc, thats fine. | 19:39 |
| morganfainberg | redrobot, ++ cool. | 19:39 |
| mordred | cool | 19:39 |
| mordred | just making sure | 19:39 |
| greghaynes | jedimike: sure, seems reasonable to me | 19:39 |
| mordred | redrobot: (I'm working on a bunch of client library things, so anytime I hear someone say they're doing somethign different I freak out because it sounds like more work for me :) ) | 19:40 |
| *** doude has quit IRC | 19:41 | |
| redrobot | mordred hehe, makes sense. | 19:41 |
| garyk | dhellmann: we are looking into the oslo.vmware failures - it was a devstack config issue. hope that can get the thumbs up soon. i havelooked at the code and it looks good. i am calling it a day. hopefully tomorrow it will be sorted. | 19:41 |
| mordred | redrobot: given the explanation Castellan seems better tahn not-Castellan - but honest to god I thin kthe idea of openstack projects making an interface to allow for a non-barbican implemenation is CRAZY | 19:41 |
| mordred | and I'd really like to argue with those projects taht they need to stop making life hard for everyone | 19:41 |
| greghaynes | So, from what I read, I think the real motivation is that projects dont want to hard depend on barbicn - they want to also support just reading certs from disk | 19:42 |
| mordred | I hear those words | 19:42 |
| greghaynes | I kind of agree that abstracting this to be a general plugin system might be a bit much ;) | 19:43 |
| mordred | but this hits my "people are being territorial and stupid" trigger | 19:43 |
| mordred | and I think clearly somewhere inside of this beast we call opensatck someone has lost their mind | 19:43 |
| greghaynes | heh | 19:43 |
| greghaynes | clearly we all have | 19:43 |
| mordred | there is a reason we think barbican is a good idea | 19:43 |
| fungi | ultimately, using barbican _is_ just reading certs from disk | 19:44 |
| mordred | maybe, now that it exists, we shoudl use it | 19:44 |
| mordred | fungi: ++ | 19:44 |
| fungi | it's merely a question of what disk and the service you happen to use to store/retrieve them there | 19:44 |
| mordred | yup | 19:45 |
| fungi | well, keys | 19:45 |
| fungi | e.g. "projects are doing some of the same things with secrets, let's turn that into a library so they will stop duplicating code" | 19:45 |
| mordred | I mean, clearly nobody will likely listen to me because we're obsessed with factory factories for creating plugins of plugins around here - but JUST USE BARBICAN | 19:45 |
| *** Sukhdev has joined #openstack-infra | 19:46 | |
| *** ronis has quit IRC | 19:46 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Run periodic stable checks for oslo-incubator https://review.openstack.org/147157 | 19:46 |
| * mordred waits for someone to make a plugin interface of which castellan is the default implementation ... | 19:46 | |
| tristanC | fungi: clarkb: alright, I guess we should really more look like at the other project you guys are using. The idea behind that multiple node assignment thing is to be able to set a list of node (or number) in jjb configuration and somehow let zuul manage node queueing | 19:46 |
| *** sarob has joined #openstack-infra | 19:46 | |
| openstackgerrit | Merged openstack-infra/system-config: Set api timeouts on hpcloud to one minute https://review.openstack.org/147233 | 19:47 |
| fungi | tristanC: right, we moved the abstraction point so that it's at the gearman worker end (one gearman worker mapped to multiple machines) | 19:47 |
| mordred | AJaeger: I think we don't need a separeate release team for pypi-mirror - I don't expect very many people to be involved anyway | 19:47 |
| fungi | tristanC: rather than zuul having to care about that | 19:47 |
| tristanC | though as it might requires quite a change this may be better tackle by that subnodes feature from nodepool | 19:48 |
| fungi | tristanC: though that feature aside, the sales pitch for nodepool is "it's like the jenkins jclouds plugin but tuned to your needs and with predictive ai" | 19:49 |
| openstackgerrit | Merged openstack-infra/puppet-jenkins: Fix Jenkins reverse proxy according to docs https://review.openstack.org/145645 | 19:50 |
| *** dims__ has joined #openstack-infra | 19:51 | |
| *** david-lyle has joined #openstack-infra | 19:52 | |
| *** shashankhegde has quit IRC | 19:52 | |
| *** sarob has quit IRC | 19:53 | |
| *** david-lyle_ has joined #openstack-infra | 19:54 | |
| clarkb | fungi: hpcloud clients seems to still be running and their queues are falling | 19:55 |
| clarkb | still no ready nodes though | 19:55 |
| *** mwagner_lap has joined #openstack-infra | 19:56 | |
| *** dims__ has quit IRC | 19:56 | |
| *** ihrachyshka has quit IRC | 19:56 | |
| *** david-lyle has quit IRC | 19:57 | |
| fungi | yeah, my hopes are waning but present | 19:57 |
| *** fbo- has joined #openstack-infra | 19:57 | |
| *** TravT has joined #openstack-infra | 19:58 | |
| *** sputnik13 has quit IRC | 19:59 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: Add sanity check script for new version tags https://review.openstack.org/147256 | 19:59 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: Enable pep8 checks https://review.openstack.org/147283 | 19:59 |
| *** avozza is now known as zz_avozza | 19:59 | |
| *** jp_at_hp has joined #openstack-infra | 20:00 | |
| openstackgerrit | Merged openstack-infra/project-config: Add designate-coverage to designate check pipeline https://review.openstack.org/147272 | 20:00 |
| *** prad has quit IRC | 20:00 | |
| clarkb | fungi: I just saw some add floating ip tasks scroll by | 20:01 |
| clarkb | fungi: maybe we will manage to ssh into them and makr them ready. fingers are crossed | 20:01 |
| *** Masahiro has joined #openstack-infra | 20:01 | |
| *** e0ne is now known as e0ne_ | 20:01 | |
| *** vigneshvar has quit IRC | 20:02 | |
| *** koolhead17 has quit IRC | 20:02 | |
| *** TravT has quit IRC | 20:02 | |
| *** vigneshvar has joined #openstack-infra | 20:02 | |
| *** TravT has joined #openstack-infra | 20:03 | |
| *** krtaylor has quit IRC | 20:05 | |
| *** wenlock has quit IRC | 20:05 | |
| *** Masahiro has quit IRC | 20:06 | |
| *** e0ne_ has quit IRC | 20:07 | |
| *** sabari is now known as zz_sabari | 20:08 | |
| *** aysyd has quit IRC | 20:09 | |
| *** wenlock has joined #openstack-infra | 20:09 | |
| *** david-lyle has joined #openstack-infra | 20:10 | |
| clarkb | fungi: we have used nodes \o/ | 20:11 |
| clarkb | if this keeps up I am going to go do a happy dance | 20:11 |
| *** dims__ has joined #openstack-infra | 20:11 | |
| *** e0ne has joined #openstack-infra | 20:12 | |
| fungi | clarkb: pretty awesome. also puppet can be reenabled now that those changes merged | 20:12 |
| fungi | clarkb: though presumably we want to keep an eye on it for signs that the timeouts aren't quite right | 20:12 |
| jeblair | clarkb, fungi: morning | 20:13 |
| fungi | (and also hopefully hpcloud comes back to us saying it's fixed) | 20:13 |
| clarkb | fungi: ya I think we should watch it a bit longer | 20:13 |
| fungi | jeblair: morning! | 20:13 |
| clarkb | but I will do the cleanup after lunch | 20:13 |
| jeblair | did you find a potential cause for the hp dead connections? | 20:13 |
| clarkb | jeblair: no, keepalives didn't seem to help either. But I added nova api timeouts to nodepool | 20:13 |
| fungi | jeblair: just a seemingly effective workaround unfortunately | 20:13 |
| clarkb | and this seems to have us moving | 20:13 |
| *** pc_m has quit IRC | 20:14 | |
| clarkb | still slow going though | 20:14 |
| rcarrillocruz | pleia2: hi, i replied to your review on https://review.openstack.org/#/c/146889/ , when you get a change pls have a look | 20:14 |
| jeblair | bummer about the keepalives -- do we think the servers are configured not to answer them? | 20:14 |
| *** ldnunes has quit IRC | 20:15 | |
| jeblair | clarkb: change for nova api timeouts? | 20:16 |
| pleia2 | rcarrillocruz: not /sbin/nologin look for /usr/sbin/nologin | 20:16 |
| clarkb | jeblair: https://review.openstack.org/#/c/147231/ | 20:16 |
| fungi | jeblair: https://review.openstack.org/147231 | 20:16 |
| pleia2 | rcarrillocruz: even 12.04 should have /usr/sbin/nologin | 20:16 |
| fungi | jeblair: also apparently latest python-daemon is very, very broken | 20:16 |
| *** shashankhegde has joined #openstack-infra | 20:17 | |
| clarkb | jeblair: either that or libkeepalive wasn't set to work proprely | 20:17 |
| clarkb | jeblair: tteggel looked into it and when tteggel hit a wall submitted some tickets for us | 20:17 |
| *** vigneshvar has quit IRC | 20:17 | |
| clarkb | so hopefully we get answers | 20:17 |
| jeblair | cool, thanks. what did you set the timeout to? | 20:18 |
| clarkb | jeblair: 60 seconds | 20:18 |
| fungi | jeblair: via https://review.openstack.org/147233 | 20:18 |
| rm_work | mordred: i am tempted to agree with you too, but there is a use-case for being able to not hard-depend on Barbican for testing/dev purposes at least, and depending on the deployment, there too -- so an extra interface is what we get | 20:18 |
| *** krtaylor has joined #openstack-infra | 20:18 | |
| jeblair | only set it on hpcloud? | 20:19 |
| clarkb | jeblair: correct | 20:19 |
| openstackgerrit | Merged openstack-infra/pypi-mirror: Pin pip so download cache location works https://review.openstack.org/147128 | 20:19 |
| *** yfried is now known as yfried|afk | 20:20 | |
| sdague | mordred: so, I would argue that barbican not having a history of sanely deprecating library interfaces (and not doing giant breaks) makes it tough for other projects to just "depend on it". Which is part of my objection to the fact that the client seems to have been rewritten with no compatibility in mind :{ | 20:21 |
| mordred | sdague: sure - but the solution to that is not to add another library that depends on the library we're worried about | 20:22 |
| mordred | sdague: I mean, I hear that - but either barbican is the thing we want to use to do this function or it's not | 20:22 |
| mordred | and either we hould learn to say no in a hard and fast way | 20:22 |
| mordred | or we should fix whatever bothers us | 20:22 |
| rm_work | sdague: but it was rewritten this extensively with a clean break now, SPECIFICALLY with graduation in mind -- so we could have clean compatibility from this point | 20:22 |
| mordred | and we shoudl stop doing all of this "no real architecture but a deployer can mix and match 100 different things in any combination because bunnies" | 20:23 |
| rcarrillocruz | pleia2: ok, i'm cool pushing a patchset to address for correct path. Too funny you never hit this :-) | 20:23 |
| rm_work | since pre-graduation is the last point where a major change like that is feasible | 20:23 |
| sdague | rm_work: actually, it was pre-incubation | 20:23 |
| rm_work | hmm | 20:23 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: Remove pypy and py26 jobs from release-tools https://review.openstack.org/147290 | 20:24 |
| sdague | https://github.com/openstack/governance/blob/master/reference/incubation-integration-requirements.rst - Project APIs should be reasonably stable | 20:24 |
| *** otter768 has joined #openstack-infra | 20:24 | |
| sdague | for incubation | 20:24 |
| rm_work | the issue being, pre-incubation there was much less interest in the project, so post-incubation we picked up a TON of use-cases which called for a more extensive feature set and some major changes | 20:24 |
| pleia2 | rcarrillocruz: I suspect having a non-existant shell for the etherpad user doesn't matter much since it's meant to be a nologin anyway, so it just throws errors in the dark :) | 20:25 |
| fungi | mordred: rabbitmq provides bunnies | 20:25 |
| mordred | fungi: we should stop using that oo | 20:25 |
| mordred | too | 20:25 |
| clarkb | hrm only hpcloud-b5 has done anything recently and a bunch of things transitioned to delete | 20:26 |
| sdague | rm_work: yes, and incorporating new use cases and extending the API gracefully is kind of important. All openstack projects have that same preasure over time. | 20:26 |
| clarkb | this may not be the magical fix, but need to see if periodic cleanup removes those nodes in delete state | 20:26 |
| sdague | you will not stop getting new use cases :) | 20:26 |
| fungi | clarkb: there are an awful lot of them | 20:26 |
| *** shashankhegde has quit IRC | 20:27 | |
| rm_work | yes, but with the gravity of the changes it really only made sense to do a major version release | 20:27 |
| clarkb | fungi: looks like jenkins may be refusing connections (possibly more proxy errors?) | 20:28 |
| sdague | rm_work: so it seems like there was a disconnect here between the barbican team expectations of what was kosher, and at least my reading of the pretty clear expectations of openstack projects | 20:29 |
| fungi | clarkb: i think it's all of them, probably the proxy config "fix" | 20:29 |
| *** otter768 has quit IRC | 20:29 | |
| *** ddieterly has quit IRC | 20:29 | |
| fungi | clarkb: we can revert but i'll check logs for apache real quick to see what it dislikes there | 20:29 |
| sc68cal | Is there a third party CI system set up for Microsoft Hyper-V? | 20:29 |
| sdague | sc68cal: yeh | 20:29 |
| sc68cal | *(sorry wrong channel) | 20:29 |
| clarkb | fungi: thanks | 20:30 |
| *** harlowja is now known as harlowja_away | 20:30 | |
| clarkb | fungi: I did test it on jenkins-dev... | 20:30 |
| *** pc_m has joined #openstack-infra | 20:30 | |
| *** ddieterly has joined #openstack-infra | 20:31 | |
| fungi | clarkb: hrm, no error, it just stopped running apache on all of them i think | 20:31 |
| *** signed8bit has joined #openstack-infra | 20:31 | |
| clarkb | ya it doesn't like RequestHeader | 20:32 |
| fungi | Syntax error on line 40 of /etc/apache2/sites-enabled/50-jenkins01.openstack.org.conf: Invalid command 'RequestHeader', perhaps misspelled or defined by a module not included in the server configuration | 20:32 |
| fungi | yep! | 20:32 |
| *** ivar-lazzaro has joined #openstack-infra | 20:32 | |
| fungi | missing a module maybe? | 20:32 |
| clarkb | I bet mod_headers isn't enabled there for some reason but is on jenkins-dev | 20:32 |
| fungi | that's what i'm checkiong | 20:32 |
| clarkb | ya that is what it looks like | 20:33 |
| clarkb | my bad | 20:33 |
| fungi | yeah, an a2enmod headers fixes | 20:33 |
| clarkb | I am going to manually do that and then whip up a patch | 20:33 |
| fungi | k | 20:33 |
| fungi | i've already done it on jenkins01 | 20:33 |
| *** vigneshvar has joined #openstack-infra | 20:33 | |
| *** sarob has joined #openstack-infra | 20:34 | |
| clarkb | all should be done now | 20:35 |
| *** hashar has joined #openstack-infra | 20:36 | |
| clarkb | writing patch now then lunch | 20:36 |
| fungi | i can do the patch if you want | 20:36 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-jenkins: Add Apache mod_headers for RequestHeader support https://review.openstack.org/147291 | 20:37 |
| * fungi was already writing it | 20:37 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-jenkins: Enable mod_headers for new vhost config https://review.openstack.org/147292 | 20:38 |
| clarkb | ha | 20:38 |
| clarkb | I will abandon mine :) | 20:38 |
| *** harlowja_away is now known as harlowja | 20:38 | |
| fungi | they do look identical | 20:38 |
| fungi | glad for some confirmation i had the right spot ;) | 20:38 |
| clarkb | wow that really made nodepool unhappy | 20:39 |
| fungi | it's like an omelette kitchen in here | 20:40 |
| clarkb | oh sweet apparently I can has openstack.org db dump now /me tries to get it | 20:44 |
| *** achanda has quit IRC | 20:45 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Add support to suppress bot notifications https://review.openstack.org/147293 | 20:45 |
| openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Add query for bug 1410969 https://review.openstack.org/147294 | 20:45 |
| *** Rockyg has quit IRC | 20:46 | |
| rm_work | sdague: so, that whole debate aside (you are likely correct, but there's not a whole lot that can be done now), would the situation legitimately prevent you from giving the requirements update a +1? | 20:46 |
| *** Rockyg has joined #openstack-infra | 20:47 | |
| clarkb | hrm github doesn't show the readme update yet. maybe its cached? | 20:47 |
| sdague | rm_work: honestly, yes. Because I feel like before a change like this gets approved we need to understand why this went off the rails, and how this is never going to happen again. | 20:48 |
| rm_work | sdague: well, again, I can tell you that the REASON this decision was made was in great part specifically *so this doesn't need to happen again* :P | 20:48 |
| rm_work | also, because until now, no one actually had begun integration with Barbican (to our knowledge) and the update was released alongside Juno, which was the first release cycle Barbican had been part of since incubation | 20:49 |
| sdague | anyway, I think barbican should probably cycle in front of the tc again. Giant new API is something I don't think anyone in the TC knew about | 20:49 |
| rm_work | (at least, AFAIK there were no previous releases as part of incubation) | 20:50 |
| sdague | and that kind of is on the barbican team to communicate there | 20:50 |
| *** ivar-lazzaro has quit IRC | 20:50 | |
| *** ivar-lazzaro has joined #openstack-infra | 20:50 | |
| rm_work | sdague: well, that is also an interesting point -- the API itself had only minor changes (we could make the 2.x branch work with some small effort) but the expanded featureset (which makes everything actually usable) wouldn't be accessible without client 3.x | 20:51 |
| rm_work | so really what we're talking about is *client library* incompatibility | 20:51 |
| clarkb | hpcloud managers still logging that they are running tasks. at this point likely just a matter of slowly catching up again | 20:51 |
| *** TravT has left #openstack-infra | 20:51 | |
| *** shashankhegde has joined #openstack-infra | 20:51 | |
| rm_work | the Barbican server's REST API is not significantly changed | 20:51 |
| *** signed8bit is now known as signed8bit_ZZZzz | 20:51 | |
| rm_work | this is a purely client lib issue | 20:51 |
| rm_work | the *client lib* was redesigned from the ground up | 20:52 |
| rm_work | to be easier to use and facilitate new required features like Keystone Session support | 20:52 |
| rm_work | if anyone was using the Barbican server's API directly, their code would likely not have changed significantly | 20:52 |
| rm_work | I don't know the intricacies of how the openstack guidelines apply to python-*client libs | 20:53 |
| rm_work | sdague: ^^ | 20:53 |
| rm_work | so we may have started with a bit of a misunderstanding here | 20:53 |
| sdague | rm_work: were the ML threads about these changes coming down the pipe? | 20:54 |
| *** markmcclain has joined #openstack-infra | 20:54 | |
| *** ivar-lazzaro has quit IRC | 20:54 | |
| rm_work | sdague: I am not sure... we did however work closely with some teams that we were aware were integrating, but i am not sure about the ML (as I said, I am peripherally related to the team, not actually *on it*, though many of the client changes were mine) | 20:55 |
| *** ivar-lazzaro has joined #openstack-infra | 20:55 | |
| *** zz_sabari is now known as sabari | 20:56 | |
| *** signed8bit_ZZZzz has quit IRC | 20:56 | |
| *** achanda has joined #openstack-infra | 20:56 | |
| rm_work | sdague: so you are saying that the python-barbicanclient lib would be subject to the same stability guidelines as the project itself? and that these apply even before the first release is made? (not even official release -- I mean the first release as an incubated project) | 20:56 |
| sdague | rm_work: anyway, my recommendation is anything like this needs public discussion. Because if it hadn't been for gate breaks I would have approved a barbican 2.x client integration into nova *today* | 20:56 |
| *** signed8bit has joined #openstack-infra | 20:56 | |
| sdague | rm_work: python-barbicanclient has been released | 20:56 |
| sdague | it's on pypi | 20:56 |
| rm_work | 3.x was the first release as part of the openstack cycle AFAIK | 20:57 |
| mordred | my pov on the importance of client lib api stuff changes if there is another openstack project actually consuming the api | 20:57 |
| mordred | and by api I mean python api | 20:57 |
| rm_work | 2.x was on pypi, but so are other things that couldn't possibly be considered stable | 20:58 |
| sdague | like this - https://review.openstack.org/#/c/104001/ | 20:58 |
| rm_work | My view would be that until a release is made as part of the openstack cycle (first release as an incubated project, for Juno, as our example) there should be no implied warranty | 20:58 |
| rm_work | but I may be alone in that view :/ | 20:58 |
| sdague | rm_work: so, on that I think you are flat wrong | 20:58 |
| mordred | no, I disagree with that | 20:58 |
| mordred | the version number is 2.x | 20:58 |
| openstackgerrit | Merged openstack-infra/project-config: Make pypi-mirror owned by its own core team https://review.openstack.org/147277 | 20:59 |
| sdague | if you've released to pypi, you have a contract | 20:59 |
| mordred | that indicates a stable thing | 20:59 |
| mordred | if it was 0.x, maybe | 20:59 |
| rm_work | 2.x was released pre-openstack incubation | 20:59 |
| mordred | but also, this: https://review.openstack.org/#/c/104001/ | 20:59 |
| rm_work | reading it | 20:59 |
| mordred | is EXACTLY what I want to happen - which is direct integration | 20:59 |
| mordred | and it already has 1 +2 from a nova core | 20:59 |
| rm_work | yeah, i am in active contact with JHU including Briana | 20:59 |
| rm_work | I am not sure why she is on 2.x | 20:59 |
| sdague | and seriously, I would have approved that this morning, except the requirements break | 20:59 |
| sdague | rm_work: because it's the published API | 21:00 |
| *** kmartin has quit IRC | 21:00 | |
| *** signed8bit has quit IRC | 21:00 | |
| rm_work | 3.x has also been published for several months | 21:00 |
| rm_work | well, at least 2 months | 21:00 |
| *** bdpayne has joined #openstack-infra | 21:00 | |
| *** signed8bit has joined #openstack-infra | 21:00 | |
| rm_work | hold on | 21:00 |
| rm_work | that change is already using 3.x code | 21:01 |
| rm_work | i just reviewed it | 21:01 |
| rm_work | generated_data = secret.payload | 21:01 |
| sdague | ok | 21:01 |
| rm_work | ^^ this only works in 3.x | 21:01 |
| rm_work | as I said, we were in communications with her, so I wasn't sure why she'd be on 2.x | 21:01 |
| *** david-lyle has quit IRC | 21:01 | |
| rm_work | the answer is that she is not :) | 21:01 |
| *** teran has quit IRC | 21:02 | |
| rm_work | especially since given the current requirements rule, tox/jenkins will pull in 3.0.2 | 21:02 |
| mordred | so how did that pass tests since the requirement is !=3.0.0 ... OH - it's just excluding 3.0.0 - is there a later one | 21:02 |
| mordred | yah | 21:02 |
| sdague | anyway, like I said, I think we need to make people more aware of all of this | 21:02 |
| rm_work | so if she were not using 3.x it wouldn't pass gates anyway | 21:02 |
| rm_work | right | 21:02 |
| mordred | so everything is already on 3.x | 21:02 |
| rm_work | yes | 21:02 |
| timrc | jeblair, Hey hey. We're seeing an issue downstream in Gozer-land where parameter expansion is not happening for a variable that is injected via a parameter file causing this sort of thing: [SCP] uploading file: '/srv/static/tarballs/foo/${BUILD_NAME}/logs/undercloud.d/dib-manifests/dib_environment' - I mentioned this to cody-somerville and he suggested I come talk to you :) | 21:02 |
| rm_work | currently it would not be possible for code to pass gates using 2.x | 21:02 |
| sdague | because seriously, a requirements bump of "hey we're dropping all compat of a major version" is not the first time the community should see it | 21:02 |
| mordred | sdague: I think the milk is already spilled | 21:03 |
| mordred | it's already happened | 21:03 |
| clarkb | timrc: check the parameters page in jenkins on that job as a quick sanity check | 21:03 |
| timrc | jeblair, From what I can tell nothing about the job, jenkins, or the scp plugin has changed. Maybe it's the system the job is running on? | 21:03 |
| rm_work | 3.0.0 is a broken release but yes 3.0.1/2 are already out | 21:03 |
| sdague | mordred: so that's fine, but how do we not do this again? | 21:03 |
| mordred | but yes - there should be much more trumpteing about this - and we should be clearer in the future | 21:03 |
| mordred | sdague: well, we'll yell some for sure | 21:03 |
| rm_work | we were working directly with any project we were aware of doing integration | 21:03 |
| sdague | rm_work: this is open source | 21:04 |
| rm_work | it's possible a ML notice went out? I am not sure | 21:04 |
| sdague | that's not good enough | 21:04 |
| clarkb | timrc: but BUILD_NAME should come from jenkins itself | 21:04 |
| rm_work | i am not saying there WASNT a ML announce | 21:04 |
| clarkb | timrc: so odd that it doesn't substitute it | 21:04 |
| sdague | because you don't know who is using stuff | 21:04 |
| rm_work | I actually don't know | 21:04 |
| rm_work | it's possible there was and this discussion is void | 21:04 |
| timrc | clarkb, It appears that it's being set by the job it self in a build step and written to a parameter file. | 21:04 |
| sdague | rm_work: sure, which is why I asked the question :) | 21:04 |
| rm_work | and i said I am not sure :P | 21:04 |
| *** Rockyg has quit IRC | 21:04 | |
| clarkb | timrc: oh hrm | 21:04 |
| sdague | I just haven't noticed a single barbican mailing list thread in 6 months | 21:04 |
| clarkb | timrc: if thats the case export FOO isn't enough | 21:05 |
| *** Shrews has quit IRC | 21:05 | |
| clarkb | you have to load the parameter file with one of the jenkins things that modifies the environment | 21:05 |
| mordred | rm_work: while we're on the subject - the readme docs for python-barbicanclient should probabl include https urls in the example just for good form ... :) | 21:05 |
| mordred | >>> print(secret.secret_ref) | 21:05 |
| mordred | http://localhost:9311/v1/secrets/091adb32-4050-4980-8558-90833c531413 | 21:05 |
| mordred | just looks evil... | 21:05 |
| rm_work | sdague: i think the answer is, people are very aware of the issue at this point, and your concerns -- the PTL and primary architect were in this discussion and have heard your concerns | 21:05 |
| timrc | clarkb, What's odd is that this worked yesterday and started failing today. The only relevant change that I can think of is the system this job runs on. | 21:05 |
| clarkb | timrc: https://jenkins07.openstack.org/job/check-tempest-dsvm-postgres-full-icehouse/1675/injectedEnvVars/ should show you what was preserved in the envs | 21:06 |
| rm_work | mordred: well, the development install uses http by default I think <_< since the deployment uses a very basic config for gunicorn | 21:06 |
| rm_work | or actually might still be on uswgi | 21:06 |
| *** david-lyle_ has quit IRC | 21:07 | |
| rm_work | actual deployments would use HTTPS via nginx or something, but the docs are intended for developers :/ | 21:07 |
| rm_work | though I agree it's gross :P | 21:07 |
| mordred | rm_work: right... but if I'm a person consuming a cloud | 21:07 |
| timrc | clarkb, For the job that failed BUILD_NAME is absent. | 21:07 |
| clarkb | timrc: ok so that at least confirms it likely wasn't in the env and not a derp in scp plugin | 21:07 |
| mordred | I'm likely not going to be pointing my python-barbicanclient at a local gunicorn | 21:07 |
| *** devlaps has quit IRC | 21:07 | |
| mordred | either way though - it's not functional - just windowdressing, so whichever | 21:07 |
| rm_work | mordred: true, but you're probably also getting your URI from a specific source, not manually generated :P | 21:08 |
| clarkb | timrc: note JOB_NAME is probably synonymous if you need a work around | 21:08 |
| rm_work | which will already have https in it | 21:08 |
| *** JamesK has quit IRC | 21:08 | |
| rm_work | (if the server uses https) | 21:08 |
| timrc | clarkb, I'm going to go check the parameter file on that build node since it's static and this job has not built again on it. | 21:08 |
| *** achanda has quit IRC | 21:08 | |
| rm_work | alright. so, I think people understand the concern, and I am also certain it's not something we plan to repeat | 21:09 |
| mordred | rm_work: indeed | 21:09 |
| rm_work | Raising a stink is fine -- possibly deserved -- but will that hold up the CR to fix the requirements to be actually readable? | 21:09 |
| rm_work | Or is holding up this CR for a while the actual punishment? :P | 21:10 |
| *** kmartin has joined #openstack-infra | 21:10 | |
| *** AJaeger has quit IRC | 21:11 | |
| timrc | Oh poo, it cleaned itself. No debugging materials. Yay. | 21:11 |
| *** achanda has joined #openstack-infra | 21:11 | |
| *** jcoufal has joined #openstack-infra | 21:12 | |
| sdague | rm_work: I'd like to see something on the mailing list explaining the intent to retire the 2.x client interface entirely, so that people you don't directly talk with might also know about it, including the technical details of the code restructuring that makes this not need to happen again. | 21:12 |
| rm_work | alright | 21:13 |
| clarkb | we have used nodes on hpcloud again | 21:14 |
| *** Shrews has joined #openstack-infra | 21:15 | |
| mordred | clarkb: woot! | 21:15 |
| *** jp_at_hp has quit IRC | 21:16 | |
| openstackgerrit | Maru Newby proposed openstack-infra/zuul: Add support for a skip-if filter on jobs https://review.openstack.org/146762 | 21:16 |
| openstackgerrit | Merged openstack-infra/puppet-jenkins: Add Apache mod_headers for RequestHeader support https://review.openstack.org/147291 | 21:16 |
| *** devlaps has joined #openstack-infra | 21:17 | |
| *** carl_baldwin has joined #openstack-infra | 21:17 | |
| carl_baldwin | Hey infra! Can anyone help me figure out what is going on here: http://logs.openstack.org/97/146597/5/check/gate-neutron-python27/54b9a29/console.html | 21:18 |
| *** e0ne is now known as e0ne_ | 21:18 | |
| clarkb | carl_baldwin: http://logs.openstack.org/97/146597/5/check/gate-neutron-python27/54b9a29/console.html#_2015-01-14_18_32_39_147 that test ran sudo so it failed | 21:18 |
| *** mrunge has quit IRC | 21:20 | |
| carl_baldwin | clarkb: I saw that but I’m not sure what it means. This error seems to come and go this week. It seems to be outside the context of the unit test runs. | 21:20 |
| clarkb | carl_baldwin: our unittest runner checks if sudo was used during the test. If it was used it fails the test (because your unittesst should not sudo) | 21:21 |
| *** achanda has quit IRC | 21:21 | |
| clarkb | carl_baldwin: its likely not a consistent thing if whatever uses sudo is only mocked some of the time | 21:22 |
| *** erikmwilson has quit IRC | 21:23 | |
| *** AJaeger has joined #openstack-infra | 21:24 | |
| *** sarob has quit IRC | 21:24 | |
| *** sarob has joined #openstack-infra | 21:25 | |
| carl_baldwin | clarkb: Thanks, I’ll keep looking. The error seems to come and go without much rhyme or reason so far. | 21:25 |
| *** signed8bit has quit IRC | 21:25 | |
| *** signed8bit has joined #openstack-infra | 21:26 | |
| timrc | clarkb, FWIW the BUILD_NAME injected correctly on a build I kicked off so I'm thinking this is more than likely something to do with the system it ran on, though what, I do not know. | 21:26 |
| timrc | Other than cosmic radiation, I think it's the only "variable" that's changing here. | 21:27 |
| mrmartin | free beer, chocolate bar and / or palinka ( http://en.wikipedia.org/wiki/Pálinka ) depending on your bravery for review of this patch: https://review.openstack.org/#/c/140043/ | 21:27 |
| *** julim has quit IRC | 21:27 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for neutron ipv6 slaac bug 1410982 https://review.openstack.org/147309 | 21:28 |
| *** e0ne_ is now known as e0ne | 21:28 | |
| *** ihrachyshka has joined #openstack-infra | 21:28 | |
| *** jedimike has quit IRC | 21:29 | |
| *** sarob has quit IRC | 21:29 | |
| carl_baldwin | clarkb: It would help if it reported this error in the context of the test that tried to run sudo. I can’t correlate the sudo command with any of the tests this patch touches. Any way to get more information? | 21:29 |
| clarkb | carl_baldwin: its not really possible to do that | 21:29 |
| clarkb | carl_baldwin: the only effective way to check this is to look at sudo logs after the job runs | 21:29 |
| clarkb | carl_baldwin: also its kinda scary that sudo can run and fail during a job and not make whatevr test ran it fail | 21:30 |
| clarkb | carl_baldwin: you do however get timestamps, so you can likely map back to some tests via the subunit log | 21:30 |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck: remove bug 1370496 signature https://review.openstack.org/147310 | 21:31 |
| carl_baldwin | clarkb: thanks. | 21:33 |
| *** signed8bit has quit IRC | 21:33 | |
| *** erikmwilson has joined #openstack-infra | 21:33 | |
| fungi | clarkb: carl_baldwin: it does at least indicate that something running over the course of the test tried to "ip netns exec ..." with in one case a repr() of a MagicMock object | 21:33 |
| *** signed8bit has joined #openstack-infra | 21:34 | |
| *** JamesK has joined #openstack-infra | 21:34 | |
| *** sputnik13 has joined #openstack-infra | 21:34 | |
| fungi | (passed to sudo) | 21:34 |
| carl_baldwin | fungi: Thanks. I was trying to use that information to correlate it to a test. I will also try timestamps. | 21:34 |
| fungi | unfortunately the timestamp granularity it got out of the auth log is not subsecond | 21:36 |
| clarkb | also interseting that publicly routable IPs are used | 21:37 |
| clarkb | owned by HP too | 21:37 |
| clarkb | carl_baldwin: if you grep for that IP it seems to narrow it down a bunch | 21:38 |
| fungi | yeah | 21:38 |
| fungi | neutron/tests/unit/cisco/cfg_agent/test_csr1kv_routing_driver.py or neutron/tests/unit/test_l3_agent.py | 21:38 |
| *** mrmartin has quit IRC | 21:38 | |
| *** spzala has quit IRC | 21:38 | |
| fungi | only about a dozen hits for that ip address | 21:38 |
| fungi | cross-check that against the tests which ran around the time those errors were logged and it's probably a slam dunk | 21:39 |
| *** sarob has joined #openstack-infra | 21:40 | |
| carl_baldwin | clarkb: fungi: Thanks. I’m off on a gate issue the just came to my attention. | 21:40 |
| *** e0ne is now known as e0ne_ | 21:40 | |
| clarkb | my money is on neutron.tests.unit.test_l3_agent.TestBasicRouterOperations.test_process_router_dist_floating_ip_add for the first one | 21:41 |
| clarkb | but could be others they all run close together there | 21:41 |
| clarkb | one thing to try is each test in isolation | 21:42 |
| *** e0ne_ is now known as e0ne | 21:42 | |
| *** david-lyle has joined #openstack-infra | 21:42 | |
| *** achanda has joined #openstack-infra | 21:43 | |
| *** e0ne has quit IRC | 21:43 | |
| *** zz_avozza is now known as avozza | 21:43 | |
| *** markmcclain has quit IRC | 21:44 | |
| *** erikmwilson has quit IRC | 21:45 | |
| *** Sukhdev has quit IRC | 21:45 | |
| ekarlso- | can I get https://review.openstack.org/#/c/145561/ merged please ? add a python-cueclient before I need to rebase again :( | 21:47 |
| *** chlong has quit IRC | 21:47 | |
| mtreinish | fungi: do you have any thoughts on: https://storyboard.openstack.org/#!/story/2000096 I'm wondering if we can update the mysql version on the trove node | 21:48 |
| *** stevemar has quit IRC | 21:48 | |
| clarkb | mtreinish: we don't really have that ability | 21:49 |
| clarkb | mtreinish: we get what trove gives us | 21:49 |
| mtreinish | clarkb: ok, that's what I figured | 21:49 |
| clarkb | you could run a not trove DB | 21:49 |
| *** david-lyle has quit IRC | 21:49 | |
| clarkb | but then you would have to figure out how to get new mysql on the node | 21:49 |
| clarkb | or use pgsql | 21:49 |
| openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Add query for bug 1410969 https://review.openstack.org/147294 | 21:50 |
| openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Add support to suppress bot notifications https://review.openstack.org/147293 | 21:50 |
| clarkb | not sure there is a good approach here :) | 21:50 |
| *** dprince has quit IRC | 21:50 | |
| mtreinish | yeah that's always an option. I'm thinking it would probably be easier to hack around in the db api layer in subunit2sql | 21:50 |
| *** Masahiro has joined #openstack-infra | 21:50 | |
| mtreinish | just start storing subsecond time separately | 21:50 |
| fungi | there may be a troveclient option to select the mysql release you want, but yes at least by default we get whatever they deem latest | 21:51 |
| *** erikmwilson has joined #openstack-infra | 21:51 | |
| fungi | also, it's possible that if we create a new trove instance now, it will have that newer version | 21:52 |
| *** erikmwilson has quit IRC | 21:53 | |
| *** r-daneel has quit IRC | 21:53 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for neutron ipv6 slaac bug 1410982 https://review.openstack.org/147309 | 21:53 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: remove bug 1370496 signature https://review.openstack.org/147310 | 21:54 |
| *** Masahiro has quit IRC | 21:54 | |
| mtreinish | fungi: hmm, yeah it might, according to maurosr in the bug we need > 5.6.4 to get the sub second time stored | 21:55 |
| *** erikmwilson has joined #openstack-infra | 21:55 | |
| mtreinish | of course doing that (or manually running one) we have the fun game of migrating everything to a new server | 21:55 |
| fungi | the trove details for this instance say "Type: MySQL 5.6" | 21:56 |
| fungi | but i guess it could be some 5.6 earlier than 5.6.4 | 21:57 |
| mtreinish | fungi: nah, it looks like the version on the db is: 5.6.17 | 21:57 |
| *** bhunter71 has quit IRC | 21:57 | |
| mtreinish | at least according to SHOW VARIABLES LIKE "%version%"; | 21:58 |
| fungi | yep | 21:58 |
| mtreinish | so maybe something else is needed to get that, I'll work with maurosr on that | 21:58 |
| mtreinish | because clearly even having the newer version doesn't store subsecond timestamps by default | 21:59 |
| *** ivar-lazzaro has quit IRC | 22:00 | |
| *** ivar-lazzaro has joined #openstack-infra | 22:00 | |
| *** radez_g0n3 is now known as radez | 22:01 | |
| *** hashar has quit IRC | 22:03 | |
| *** sputnik13 has quit IRC | 22:04 | |
| fungi | maybe sqlalchemy is filtering them for compatibility? | 22:05 |
| *** TravT has joined #openstack-infra | 22:05 | |
| *** Rockyg has joined #openstack-infra | 22:06 | |
| *** sputnik13 has joined #openstack-infra | 22:06 | |
| mtreinish | fungi: yeah that might be, my first assumption was that I was doing something wrong :) | 22:06 |
| *** teran has joined #openstack-infra | 22:07 | |
| *** amitgandhinz has quit IRC | 22:08 | |
| *** achanda has quit IRC | 22:08 | |
| *** MaxV has quit IRC | 22:08 | |
| *** JamesK has quit IRC | 22:09 | |
| *** TravT has quit IRC | 22:10 | |
| *** vinkman has joined #openstack-infra | 22:10 | |
| *** amitgandhinz has joined #openstack-infra | 22:10 | |
| *** bcourt has quit IRC | 22:11 | |
| *** achanda has joined #openstack-infra | 22:12 | |
| *** ashp has quit IRC | 22:13 | |
| *** ashp has joined #openstack-infra | 22:13 | |
| sdague | oh, this is a new fun gerrit thing - https://review.openstack.org/#/c/146999/ | 22:14 |
| *** dkranz has quit IRC | 22:15 | |
| *** dmsimard is now known as dmsimard_away | 22:15 | |
| sdague | so... question | 22:15 |
| sdague | do I actually have +A on glance, or is gerrit showing me something weird? | 22:15 |
| mtreinish | sdague: did you try it? :) | 22:17 |
| sdague | I haven't, gerrit is showing me W +1 on a bunch of stuff though | 22:17 |
| *** vinkman has left #openstack-infra | 22:17 | |
| sdague | nope, it's a ui bug it seems | 22:18 |
| *** mattfarina has quit IRC | 22:18 | |
| sdague | https://review.openstack.org/#/c/134238/ is something it showed it to me on | 22:18 |
| openstackgerrit | Maru Newby proposed openstack-infra/zuul: Add support for a skip-if filter on jobs https://review.openstack.org/146762 | 22:18 |
| *** esker has quit IRC | 22:19 | |
| *** dims__ has quit IRC | 22:20 | |
| *** bhunter71 has joined #openstack-infra | 22:21 | |
| *** dims__ has joined #openstack-infra | 22:21 | |
| *** nelsnels_ has joined #openstack-infra | 22:21 | |
| *** mhanif has joined #openstack-infra | 22:24 | |
| mhanif | Hi there! | 22:24 |
| *** nelsnelson has quit IRC | 22:25 | |
| mhanif | Is there anyone from core infra team on this channel? | 22:25 |
| clarkb | yes | 22:25 |
| *** otter768 has joined #openstack-infra | 22:25 | |
| *** dims__ has quit IRC | 22:25 | |
| mhanif | clarkb: Thanks! Checking to see the next steps on #link https://review.openstack.org/#/c/145160/ | 22:26 |
| clarkb | patience. We are still catching upfrom holidays and have been fire fighting recently | 22:26 |
| *** nelsnels_ has quit IRC | 22:27 | |
| clarkb | hopefully I will be able to do a good pass at reviews soon | 22:27 |
| *** nelsnelson has joined #openstack-infra | 22:28 | |
| mhanif | clarkb: Got it. Sorry, didn't mean to rush you. It is that I didn't know when to expect this one to be looked up by the team. Thanks! | 22:28 |
| *** otter768 has quit IRC | 22:29 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 22:31 | |
| *** EmilienM is now known as EmilienM|afk | 22:32 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 22:32 | |
| clarkb | apparently I manged to skip lunch. Doing that now then reenabling puppet and cleaning up nodepool.o.o now that it seems to be working somewhat | 22:32 |
| *** david-lyle has joined #openstack-infra | 22:33 | |
| *** Sukhdev has joined #openstack-infra | 22:33 | |
| fungi | clarkb: i'll take care of the nodepool puppet/cleanup. go have your lunch-ish | 22:33 |
| clarkb | thanks | 22:34 |
| clarkb | fungi: note /opt/nodepool just has FETCH_HEAD checked out, you likely want to checkout master and update master prior to running puppet | 22:34 |
| clarkb | otherwise I think puppet will get confused | 22:34 |
| fungi | yep, thanks for the pointer | 22:35 |
| pleia2 | woo, complete manual zanata install directions that work, I'm going to puppetize this as is like clarkb has done with www.o.o and we can hash out "no that's insane" in reviews | 22:35 |
| clarkb | :) | 22:36 |
| *** bknudson has quit IRC | 22:37 | |
| *** dizquierdo has quit IRC | 22:38 | |
| *** jgrimm is now known as zz_jgrimm | 22:39 | |
| *** dustins has quit IRC | 22:40 | |
| *** ChuckC_ has joined #openstack-infra | 22:41 | |
| *** ChuckC_ has quit IRC | 22:42 | |
| mfisch | in the integration queue do the lines show dependencies? | 22:43 |
| mfisch | thats a terrible way to ask | 22:43 |
| *** jcoufal has quit IRC | 22:43 | |
| mfisch | trying to figure out why 144796 and 146003 are connected | 22:43 |
| *** bhunter71 has quit IRC | 22:43 | |
| jeblair | mfisch: they show the order that zuul is testing them, which may be because of dependencies, but also may just be order of approval | 22:44 |
| mfisch | okay order of approval makes sense here | 22:44 |
| *** radez is now known as radez_g0n3 | 22:44 | |
| mfisch | its checking them as a batch? | 22:44 |
| mfisch | yep cool jenkins shows the batch | 22:44 |
| mfisch | I know my glance doc change didn't break dvsm | 22:45 |
| *** dims__ has joined #openstack-infra | 22:45 | |
| *** ChuckC has quit IRC | 22:45 | |
| mfisch | looks like I got unlucky with apt | 22:45 |
| mfisch | W: Failed to fetch http://az2.clouds.archive.ubuntu.com/ubuntu/dists/trusty-updates/main/i18n/Translation-en Hash Sum mismatch | 22:45 |
| fungi | mfisch: it's like they say, unlucky with apt, lucky with yum | 22:47 |
| mfisch | will that job retrigger or get unbatched and retried? | 22:48 |
| *** markmcclain has joined #openstack-infra | 22:48 | |
| jeblair | mfisch: if nothing else ahead of it fails, it'll report a failure when it gets to the top | 22:49 |
| jeblair | mfisch: if something ahead fails before it gets there, it'll roll the apt dice again | 22:49 |
| mfisch | c'mon lucky 7! | 22:49 |
| fungi | mfisch: this is the part where you get to hope openstack has bugs | 22:50 |
| mfisch | this will only be the 4th or so recheck on this doc change, pretty sure i'm due for a pass | 22:50 |
| *** markmcclain has quit IRC | 22:52 | |
| *** markmcclain has joined #openstack-infra | 22:52 | |
| *** teran_ has joined #openstack-infra | 22:53 | |
| *** Ryan_Lane has quit IRC | 22:54 | |
| *** markmcclain has quit IRC | 22:54 | |
| *** markmcclain1 has joined #openstack-infra | 22:54 | |
| *** teran has quit IRC | 22:56 | |
| *** jamielennox|away is now known as jamielennox | 22:56 | |
| *** mhanif has quit IRC | 22:58 | |
| *** markmcclain1 has quit IRC | 22:58 | |
| *** Rockyg has quit IRC | 22:58 | |
| fungi | wow, plenty of alien nodes in hpcloud again. cleaning those up now | 22:59 |
| *** dimtruck is now known as zz_dimtruck | 23:01 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Retrieve task statuses from the backend https://review.openstack.org/138389 | 23:02 |
| *** ryanpetrello has quit IRC | 23:02 | |
| *** marcusvrn has quit IRC | 23:02 | |
| *** Ryan_Lane has joined #openstack-infra | 23:02 | |
| *** bknudson has joined #openstack-infra | 23:03 | |
| *** ryanpetrello has joined #openstack-infra | 23:03 | |
| *** marcusvrn has joined #openstack-infra | 23:03 | |
| *** dims__ has quit IRC | 23:07 | |
| *** ihrachyshka has quit IRC | 23:08 | |
| *** mfink has quit IRC | 23:08 | |
| *** sputnik13 has quit IRC | 23:11 | |
| *** carl_baldwin has quit IRC | 23:11 | |
| *** shayneburgess has quit IRC | 23:12 | |
| *** Rockyg has joined #openstack-infra | 23:13 | |
| *** mfink has joined #openstack-infra | 23:15 | |
| *** dims__ has joined #openstack-infra | 23:15 | |
| *** sputnik13 has joined #openstack-infra | 23:16 | |
| *** david-lyle has quit IRC | 23:16 | |
| *** wenlock has quit IRC | 23:19 | |
| *** erlon has quit IRC | 23:20 | |
| *** fbo- has quit IRC | 23:20 | |
| *** fbo has joined #openstack-infra | 23:23 | |
| *** mase_home has joined #openstack-infra | 23:23 | |
| *** ChuckC_ has joined #openstack-infra | 23:25 | |
| *** achanda has quit IRC | 23:25 | |
| reed | can someone have a look at this patch and tell me if I need to provide more fixes ? https://review.openstack.org/#/c/142544/ | 23:26 |
| clarkb | fungi: so are we all set on nodepool now? | 23:26 |
| *** achanda has joined #openstack-infra | 23:26 | |
| fungi | clarkb: alien node deletion is still going. hpcloud is somewhat slow to respond to delete calls | 23:27 |
| fungi | nearing completion i think | 23:27 |
| clarkb | reed: that looks fine I have rechecked it to get jenkins to +1 it | 23:27 |
| reed | thanks clarkb | 23:28 |
| pleia2 | reed: just did a grep through our configs, I think that's it | 23:28 |
| fungi | the test nodes graph is looking much better | 23:28 |
| *** signed8bit is now known as signed8bit_ZZZzz | 23:28 | |
| jesusaurus | fungi: are you also seeing deletes taking upwards of 15 minutes? | 23:28 |
| fungi | jesusaurus: at random, but yes. most complete (fairly) quickly | 23:28 |
| *** david-lyle has joined #openstack-infra | 23:30 | |
| fungi | okay, alien nodes are cleaned up, now moving on to disassociated floating ips | 23:30 |
| *** teran has joined #openstack-infra | 23:30 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 23:31 | |
| *** dims__ has quit IRC | 23:32 | |
| *** david-lyle_ has joined #openstack-infra | 23:33 | |
| *** dannywil_ has joined #openstack-infra | 23:34 | |
| *** teran_ has quit IRC | 23:34 | |
| *** dannywil_ has quit IRC | 23:34 | |
| *** david-lyle has quit IRC | 23:35 | |
| *** dannywil_ has joined #openstack-infra | 23:35 | |
| *** patrickeast_ has joined #openstack-infra | 23:36 | |
| *** patrickeast has quit IRC | 23:36 | |
| *** patrickeast_ is now known as patrickeast | 23:36 | |
| *** dannywilson has quit IRC | 23:37 | |
| *** david-lyle_ has quit IRC | 23:37 | |
| asselin | Cinder team needs a 3rd party ci system "CloudByte CI" disabled. | 23:39 |
| *** amitgandhinz has quit IRC | 23:39 | |
| *** Masahiro has joined #openstack-infra | 23:39 | |
| *** reed has quit IRC | 23:39 | |
| openstackgerrit | Merged stackforge/python-jenkins: testtools requires unittest2 https://review.openstack.org/146618 | 23:39 |
| *** sdake_ has joined #openstack-infra | 23:39 | |
| *** sdake_ has quit IRC | 23:39 | |
| *** sdake_ has joined #openstack-infra | 23:39 | |
| adam_g | mriedem, fyi https://review.openstack.org/147143 | 23:40 |
| *** weshay has quit IRC | 23:40 | |
| clarkb | asselin: have an example of why? | 23:40 |
| asselin | I'm on the "Third-Party Coordiantors" group, but I don't know what special powers I have to help | 23:40 |
| mriedem | adam_g: thanks | 23:40 |
| clarkb | asselin: that just controls email for the most part. Cinder release can either toggle them out of the voting group or I can hard disable them depending on whether or not voting is the issue | 23:41 |
| *** Rockyg has quit IRC | 23:42 | |
| asselin | clarkb, looking for refs. I had another e-mail about their merge errors. but actually don't see anything here (yet?) https://review.openstack.org/#/c/141475/ | 23:42 |
| *** Rockyg has joined #openstack-infra | 23:42 | |
| clarkb | maybe they corrected it? | 23:43 |
| clarkb | I do see the commen for ps8 on that change | 23:43 |
| *** Masahiro has quit IRC | 23:43 | |
| *** bhuvan_ has quit IRC | 23:44 | |
| *** wuhg has joined #openstack-infra | 23:44 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Task status endpoint now matches api https://review.openstack.org/147335 | 23:44 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Task status endpoint now matches api https://review.openstack.org/147335 | 23:45 |
| asselin | clarkb, ok that's why, it's just in the comments for https://review.openstack.org/#/c/141475/ | 23:46 |
| *** ddieterly has quit IRC | 23:46 | |
| *** ryanpetrello has quit IRC | 23:46 | |
| asselin | clarkb, duncan sent out a request in the mailing list. it came up in the cinder meeting this morning. I couldn't find it onthe wiki page or gerrit group. will look again | 23:46 |
| clarkb | Oh I hadn't checked email recently I see it now | 23:47 |
| asselin | clarkb, shouldn't they be here? https://review.openstack.org/#/admin/groups/270,members | 23:47 |
| clarkb | only if you add them | 23:47 |
| clarkb | that group is mostly there so that when jogo complains about third party email I can tell him to add the user to that group | 23:48 |
| *** sdake_ has quit IRC | 23:48 | |
| *** TravT has joined #openstack-infra | 23:49 | |
| asselin | clarkb, so if I add CloudByte there, I won't get e-mails from them again? | 23:49 |
| clarkb | asselin: correct | 23:49 |
| clarkb | asselin: so at this point I am trying to figre out if the issue was dealt with in a way that means I don't have to hard disable that account | 23:50 |
| asselin | clarkb, no, I think it still needs to be disable. | 23:51 |
| asselin | but now I can send them an e-mail too. | 23:51 |
| *** teran has quit IRC | 23:51 | |
| openstackgerrit | Merged openstack-infra/system-config: Remove community.openstack.org from the list of server https://review.openstack.org/142544 | 23:51 |
| clarkb | asselin: https://review.openstack.org/#/c/147338/ that shows its still happening so I will disable | 23:52 |
| *** sabeen1 has quit IRC | 23:52 | |
| clarkb | asselin: can you respond to that thread that was started? and I will send a thing to announce | 23:52 |
| *** mriedem has quit IRC | 23:52 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 23:52 | |
| clarkb | or you can send to announce as well or is it moderated? | 23:52 |
| clarkb | pleia2: ^ do you know? | 23:52 |
| asselin | I can reply to the thread | 23:52 |
| asselin | clarkb, thanks | 23:52 |
| pleia2 | I can let anything through | 23:53 |
| clarkb | asselin: its done | 23:53 |
| clarkb | I can respond to announce | 23:54 |
| clarkb | er rather make an announcement there | 23:54 |
| pleia2 | clarkb: I'll add you to accepts if you're not already | 23:54 |
| clarkb | pleia2: thanks | 23:54 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Task status endpoint now matches api https://review.openstack.org/147335 | 23:54 |
| clarkb | oh wait duncan sent to announce not infra | 23:54 |
| clarkb | so asselin can just respond /me unconfuses himself | 23:55 |
| asselin | clarkb, ok will send now. | 23:55 |
| *** Rockyg has quit IRC | 23:55 | |
| *** Rockyg has joined #openstack-infra | 23:56 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Task status endpoint now matches api https://review.openstack.org/147335 | 23:57 |
| *** dalgaaf has quit IRC | 23:57 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Retrieve task statuses from the backend https://review.openstack.org/138389 | 23:57 |
| *** che-arne has quit IRC | 23:57 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Retrieve task statuses from the backend https://review.openstack.org/138389 | 23:58 |
| asselin | sent, but awaiting moderator approval | 23:58 |
| *** avozza is now known as zz_avozza | 23:58 | |
| asselin | pleia2, is that you? ^^ | 23:58 |
| pleia2 | asselin: I let it through | 23:58 |
| asselin | thanks | 23:58 |
| pleia2 | sure thing | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
