| *** jepoy has quit IRC | 00:09 | |
| *** thuc_ has quit IRC | 00:09 | |
| *** jepoy has joined #openstack-infra | 00:09 | |
| *** thuc has joined #openstack-infra | 00:10 | |
| *** matsuhashi has joined #openstack-infra | 00:12 | |
| *** resker has quit IRC | 00:13 | |
| *** VijayTripathi has joined #openstack-infra | 00:13 | |
| *** jepoy has quit IRC | 00:14 | |
| *** thuc has quit IRC | 00:14 | |
| *** cyeoh is now known as Guest20271 | 00:18 | |
| *** sarob_ has joined #openstack-infra | 00:18 | |
| *** mwagner_lap has quit IRC | 00:19 | |
| *** sarob_ has quit IRC | 00:23 | |
| *** yamahata has quit IRC | 00:24 | |
| *** matsuhashi has quit IRC | 00:31 | |
| *** esker has joined #openstack-infra | 00:33 | |
| *** esker has quit IRC | 00:38 | |
| *** rdwrer is now known as marktraceur | 00:43 | |
| *** marktraceur is now known as rdwrer | 00:43 | |
| mordred | lifeless, dstufft: I'd use build tags if there were available - I like the concept of those - but what fungi says is right on | 00:44 |
|---|---|---|
| mordred | the problem we're trying to solve is to ensure we're expressing the uniqueness of a commit | 00:45 |
| mordred | and although there are circumstances, such as what fungi says, where it can fall down and is non-perfect | 00:45 |
| *** Sukhdev has quit IRC | 00:45 | |
| mordred | it should DTRT if you are following openstack trunk yourself by making packages automatically from every commit and instaling those | 00:46 |
| mordred | while at the same time it shoudl be abundantly clear to anyone who gets a tarball that's autogenerated from this that it's autogenerated and not a release | 00:46 |
| mordred | I'm happy to tweak areas where we've gotten a pep440 and semver inspired view of that wrong | 00:47 |
| lifeless | mordred: so, semver says 1.0.0.a1, pep440 says 1.0.0a1 | 00:47 |
| lifeless | mordred: I don't care which we do, but I'd like to make what I do consistent within pbr's own docs :) | 00:48 |
| mordred | lifeless: is 1.0.0.a1 actually a violation of pep440? I raed it to be able to go either way | 00:48 |
| *** thuc has joined #openstack-infra | 00:49 | |
| lifeless | mordred: the reason my tests failed tempest etc was that oslo.config has a version X.Y.Za0 | 00:50 |
| lifeless | mordred: dstufft said that 1.0.0a1 is what PEP440 would expect | 00:50 |
| *** bearhands is now known as comstud | 00:51 | |
| mordred | hrm. you're right. that's an ugly problem | 00:52 |
| lifeless | mordred: dstufft: I don't want to be caught in the middle here :). I just need to know if oslo's latest tag is invalid (its not consistent with pbr semver) or if semver.rst is wrong. | 00:52 |
| *** matsuhashi has joined #openstack-infra | 00:52 | |
| mordred | what is oslo's latest tag? | 00:52 |
| lifeless | mordred: *secondly* I need to figure out what to issue as a version number for 'first commit after a tag that has an alpha/beta/tc marker' | 00:52 |
| *** thuc has quit IRC | 00:53 | |
| lifeless | 1.3.0a0 | 00:53 |
| lifeless | oslo.config specifically | 00:53 |
| *** thuc has joined #openstack-infra | 00:53 | |
| lifeless | mordred: *thirdly* I need to knwo if I should file a bug about setup.py sdist ignoring local content and only including git committed content | 00:54 |
| mordred | lifeless: do you mean files that have not been git added? | 00:54 |
| mordred | that are also not things that would normally be in sdist/ | 00:54 |
| mordred | ? | 00:54 |
| mordred | (can you give me an example?) | 00:55 |
| lifeless | mordred: I mean if I edit pbr/core.py; run setup.py sdist; pip install the resulting tarball; the edits will not be present in whats installed. | 00:56 |
| mordred | I would consider that a bug, yes. although I'm not fully sure I grok how that would be accomplished, pbr does not do any thing other than auto-edit manifests | 00:56 |
| dstufft | mordred: I filed an issue for making build tags be a thing in PEP440 | 00:57 |
| mordred | dstufft: cool | 00:57 |
| lifeless | mordred: does it suck in setuptools-git or something braindamaged like that? | 00:57 |
| mordred | lifeless: no | 00:57 |
| dstufft | I think that's where the git sha information belongs | 00:57 |
| mordred | but | 00:57 |
| dstufft | in a PEP440 view of the world | 00:57 |
| mordred | it does its own thing | 00:57 |
| mordred | dstufft: does that wind up in the tarball name/ | 00:57 |
| mordred | ? | 00:57 |
| dstufft | mordred: well it doesn't exist at all yet, but it would yes | 00:57 |
| mordred | ok. then I'm on board with it. :) | 00:57 |
| dstufft | Probably ideally pip and such would ignore it, unless you specify it | 00:58 |
| mordred | ++ | 00:58 |
| dstufft | so you could do pip install oslo.config==1.0a1+thinger | 00:58 |
| mordred | do you think there is any chance to get pep440 to accept .a1? | 00:58 |
| mordred | or should I consider that in-stone and to be dealt with/ | 00:59 |
| lifeless | https://bugs.launchpad.net/pbr needs triage love | 01:00 |
| *** esker has joined #openstack-infra | 01:00 | |
| mordred | I'm sure it does. | 01:00 |
| mordred | oh good. I get to go answer not-a-bug to things | 01:01 |
| *** rdwrer is now known as marktraceur | 01:02 | |
| *** marktraceur is now known as rdwrer | 01:02 | |
| dstufft | Maybe? Feels like it's just making an already complicated spec more complicated than it needs to be, is there a reasoning for it? | 01:02 |
| mordred | well, it makes the argument for integrating with semver simpler | 01:02 |
| dstufft | FWIW the reason that a/b/c/rc don't have a period and .dev and .post do is so you can do dev releases of an a/b/c/rc or post releases | 01:03 |
| dstufft | because there are projects that do that :/ | 01:03 |
| mordred | semer says X.Y.Z-stuff and really only cares about the content of X.Y.Z | 01:03 |
| dstufft | e.g. 1.0a0.dev0 is valid | 01:03 |
| mordred | but - is kinda evil in a version because debian | 01:03 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Allow examining parsing exceptions. https://review.openstack.org/80856 | 01:04 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Permit pre-release versions with git metadata https://review.openstack.org/80857 | 01:04 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Teach pbr about post versioned dev versions. https://review.openstack.org/80449 | 01:04 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Add a converter to version_tuples. https://review.openstack.org/80457 | 01:04 |
| mordred | so my compromise was to replace the - as a version/other demarcation with a . | 01:04 |
| mordred | and then things become simple from a semver perspective - is it beyond the 3rd chunk? it's additional info | 01:04 |
| dstufft | mordred: I would raise an issue here https://bitbucket.org/pypa/pypi-metadata-formats personally I'd be -0 on it because I think accepting both is just complication and the 1.0a1 is already there and tooling using it | 01:05 |
| mordred | but if it's "replace '-' with (a|b|c|rc)" | 01:05 |
| mordred | hrm. ok. I honestly didn't realize that the tooling only accepted 0a1 | 01:05 |
| *** dcramer_ has joined #openstack-infra | 01:05 | |
| dstufft | well | 01:05 |
| dstufft | So there's two things | 01:06 |
| lifeless | dstufft: so the goal we have with semver is deterministic, automatic version generation that can be mechanically transformed into debian and rpm versions | 01:06 |
| lifeless | dstufft: e.g. dev1 -> ~1 | 01:06 |
| dstufft | 0.a1 is accepted by pip as close enough to PEP440 | 01:06 |
| * mordred ACTUALLY doesn't care about semver per-se | 01:06 | |
| dstufft | because it uses a fuzzy version parser | 01:06 |
| lifeless | dstufft: as well as doing the right thing for version dependencies across the board for python | 01:06 |
| mordred | dstufft: then I'm fine with 0.a1 because pip is all I care about anyway | 01:07 |
| mordred | personally | 01:07 |
| dstufft | mordred: That's fine for your use cases :) It just means if we make a PEP440 + semver pep your semver doc will differ in that way:) | 01:07 |
| mordred | we should probably just comply | 01:08 |
| mordred | I had a magical fairy world in my head where we coudl actually come up with a PE440 + semver that might be adopted outside of python | 01:09 |
| dstufft | PyPI may or may not accept .a1 down the road, I don't know, it needs a PEP before I can do that but i'd like it to at least enforce that we can fuzzy parse the version into a PEP440 style thin | 01:09 |
| mordred | I think 0a1 is more complex to reason about than 0.a1 - but I'm late to the party, so I'd rather play ball than be a bitch | 01:10 |
| dstufft | FWIW that predates me too | 01:10 |
| lifeless | so 0a1 is plain incompatible with debian | 01:11 |
| lifeless | but | 01:11 |
| mordred | right | 01:11 |
| mordred | I think that's the other thing :) | 01:12 |
| lifeless | if there's a pep440 parser that can highlight 'and sort before' we can do something using ~ | 01:12 |
| dstufft | we took PEP386 and adjusted it to be more compatible with how setuptools did ordering, and also Nick added some additional stuff to make it easier | 01:12 |
| dstufft | pip uses distlib.version atm | 01:12 |
| dstufft | it doesn't have the greatest API in the world | 01:12 |
| dstufft | but it does a thing | 01:12 |
| lifeless | anyhow like I say | 01:12 |
| *** CaptTofu has joined #openstack-infra | 01:12 | |
| lifeless | the formatting thing really doesn't affect me, serialising a tuple of version components to 0.0.0.a0 or 0.0.0a0 is easy | 01:13 |
| dstufft | distlib.version._normalized_key("1.0.1.0a1") | 01:14 |
| dstufft | ((1, 0, 1), ('a', 1), ('_',), ('final',), ()) | 01:14 |
| lifeless | >>> distlib.version._normalized_key("1.0.1.a1") | 01:15 |
| lifeless | distlib.version.UnsupportedVersionError: Not a valid version: 1.0.1.a1 | 01:15 |
| dstufft | distlib.version._suggest_normalized_version("1.0.1.0.a1") | 01:15 |
| dstufft | '1.0.1.0a1' | 01:15 |
| lifeless | >>> distlib.version._normalized_key("1.0.1.0.a1") | 01:15 |
| lifeless | distlib.version.UnsupportedVersionError: Not a valid version: 1.0.1.0.a1 | 01:16 |
| mordred | lifeless: you're still preceeding a with . there | 01:16 |
| mordred | lifeless: what if we compromized between the two by just inserting a 0 in our semver doc | 01:16 |
| dstufft | yea _normalized_key is strict PEP440, you have to use the _suggest_normalized_version to turn something that is almost PEP440 compliant into a PEP440 compliant version | 01:16 |
| mordred | 1.0.1.0a1 | 01:16 |
| mordred | it's PEP440 compliant | 01:16 |
| dstufft | yes | 01:16 |
| dstufft | that would be equivilant | 01:17 |
| dstufft | to 1.0.1a1 | 01:17 |
| mordred | and it still makes the "just do semver but replace the - with a ." make sense | 01:17 |
| dstufft | when comparing values PEP440 has you strip all the trailing zeros | 01:17 |
| *** CaptTofu has quit IRC | 01:17 | |
| mordred | (leading) | 01:17 |
| mordred | ? | 01:17 |
| dstufft | No, 1 == 1.0 == 1.0.0 == 1.0.0.0 | 01:17 |
| mordred | ah | 01:17 |
| mordred | gotcha | 01:18 |
| dstufft | because people are inconsistent with the number of segments | 01:18 |
| *** Guest20271 is now known as cyeoh | 01:18 | |
| dstufft | they'll do 1.5 and 1.5.1 | 01:18 |
| *** sarob_ has joined #openstack-infra | 01:18 | |
| mordred | lifeless: does that pass your sniff test? | 01:18 |
| dstufft | So you need to either make 1.5 < 1.5.0 or make them equiv | 01:19 |
| lifeless | mordred: lost track, does what? | 01:19 |
| mordred | 1.0.0.0a1 instead of 1.0.0.a1 | 01:19 |
| mordred | lifeless: pre-pend a 0 to the a1 which makes pep440 happy | 01:19 |
| mordred | and is just as deterministic and simple to parse for | 01:19 |
| lifeless | mordred: I'm fine doing that. The olso.config tag is wrong then | 01:20 |
| lifeless | mordred: 1.3.0a1 - should be 1.3.0.0a1 ? | 01:20 |
| mordred | in theory - although it's water under the bridge at this point | 01:20 |
| lifeless | mordred: well, the question is whether I write compat code, or we issue new tags in a few projects | 01:21 |
| mordred | lifeless: where is the code? | 01:21 |
| lifeless | mordred: which code? | 01:21 |
| mordred | lifeless: I don't know where you're writing compat code | 01:21 |
| dstufft | mordred: lifeless fwiw I hope i'm not coming off as being like YOU MUST DO IT OUR WAY or anything, If you want to stick with how you're doing it that's OK with me, I just wanted y'al to know :) | 01:21 |
| mordred | dstufft: no - I appreciate the feedback! | 01:22 |
| lifeless | mordred: it would be in pbr; I haven't written it yet. | 01:22 |
| mordred | the goal is to tend towards compliance, not to continue to be snowflakes | 01:22 |
| lifeless | mordred: my semver code breaks because 0a1 isn't an int. | 01:22 |
| mordred | lifeless: gotcha. well, eithe way you're going to need to handle 0a1 to be pep440 | 01:23 |
| mordred | apparently | 01:23 |
| *** sarob_ has quit IRC | 01:23 | |
| dstufft | you could do what pip does and bundle distlib ! | 01:23 |
| lifeless | dstufft: DIAF :)O | 01:23 |
| mordred | heh. probably not | 01:23 |
| dstufft | :D | 01:23 |
| dstufft | I find it hilarous pip bundles all the things | 01:24 |
| dstufft | but it's less terrible :/ | 01:24 |
| lifeless | mordred: so, do we need to handle any pep440 version in pbr ? | 01:24 |
| mordred | lifeless: no- we should handle our versions - but our current spec is apparently non-compliant | 01:25 |
| mordred | lifeless: so if we align our spec to be pep440 compliant moving forward | 01:25 |
| lifeless | mordred: then yes, we need to handle N{a/b/rc/dev}Y | 01:25 |
| *** thuc_ has joined #openstack-infra | 01:26 | |
| *** CaptTofu has joined #openstack-infra | 01:26 | |
| mordred | dstufft: do you think the idea of 1.0.0.0a1 as a basis for a pep440 + semver is a decent start? or before proposing that would you think something should change? | 01:26 |
| lifeless | mordred: https://bugs.launchpad.net/pbr/+bug/1293285 | 01:27 |
| mordred | lifeless: great. I have no idea how that bug is possible, btw | 01:27 |
| dstufft | mordred: well my personal preference would be for 1.0.0a1, but at that point it's just bikeshedding, it's fully compliant and it's not one of the horrendous versions that PEP440 allows | 01:27 |
| dstufft | (PEP440 allows some pretty dumb combinations in order to be flexible enough to capture more use cases) | 01:28 |
| lifeless | mordred: so semver can be PEP440 compliant with out us having to accept 0a1 as part of major/minor/patch | 01:28 |
| mordred | dstufft: yeah - our subset of pep440 + modified semver is intended to get rid of most of those variations | 01:28 |
| mordred | lifeless: what? | 01:28 |
| mordred | how | 01:29 |
| lifeless | mordred: PEP440 accepts 1.3.0.0a1 right | 01:29 |
| mordred | yes | 01:29 |
| lifeless | mordred: major 1, minor 3, patch 0. | 01:29 |
| *** thuc has quit IRC | 01:29 | |
| mordred | lifeless: you are saying exactly what I was just saying I think | 01:29 |
| lifeless | sure | 01:30 |
| lifeless | so | 01:30 |
| lifeless | 13:52 < lifeless> mordred: *secondly* I need to figure out what to issue as a version number for 'first commit after a tag that has an alpha/beta/tc marker' | 01:30 |
| *** nosnos has joined #openstack-infra | 01:30 | |
| *** yaguang has joined #openstack-infra | 01:30 | |
| lifeless | mordred: lets (for arguments sake) say oslo.config has a tag 1.3.0.0a1 | 01:30 |
| lifeless | mordred: and I do another commit locally. | 01:30 |
| *** thuc_ has quit IRC | 01:30 | |
| mordred | yah | 01:30 |
| lifeless | mordred: what version should pbr output | 01:30 |
| mordred | hrm. excellent question | 01:32 |
| lifeless | if we issue 1.3.0.0a2 | 01:32 |
| lifeless | then the use can't tag that sanely | 01:32 |
| *** thuc has joined #openstack-infra | 01:32 | |
| lifeless | because we'll already generate 0a3 and 0a4 etc | 01:32 |
| dstufft | PEP440 allows 1.3.0.0a1.dev0 :V | 01:33 |
| dstufft | although it sorts before | 01:33 |
| dstufft | not after | 01:33 |
| mordred | right | 01:33 |
| dstufft | you could do 0a2.dev0 | 01:33 |
| lifeless | we could exmit 1.3.0.0a2dev1 etc | 01:33 |
| lifeless | or 0a2.0dev1 | 01:33 |
| dstufft | dev has a leading period because ~reasons~ | 01:33 |
| dstufft | mostly because it can be used after a a1 | 01:34 |
| lifeless | ok sure | 01:34 |
| lifeless | 0a2.dev1 | 01:34 |
| dstufft | yea | 01:34 |
| lifeless | but semver doesn't capture this today | 01:34 |
| mordred | lifeless: so - we do not currently handle this, btw | 01:34 |
| lifeless | mordred: btw where do pypi-mirror bugs go? | 01:35 |
| mordred | openstack-ci | 01:35 |
| mordred | lifeless: 1.3.0a0.12.g0c3a643 is what pbr is currently doing | 01:36 |
| mordred | 1.3.0.0a0.12.g0c3a643 does seem slightly excessive - but it is deterministic | 01:37 |
| *** shakamunyi has joined #openstack-infra | 01:37 | |
| *** bknudson has quit IRC | 01:38 | |
| dstufft | oh hm | 01:38 |
| dstufft | 0a0.12 isn't PEP440 btw (nor is the git tag, but I think that should jsut move to the build tag and PEP440 should get a build tag) | 01:39 |
| mordred | one could argue that it should be 1.3.0.0a0.post12.g0c3a643 | 01:39 |
| mordred | I thnk g0c3a643 should clearly be in the build tag | 01:39 |
| dstufft | so post releases are supposed to be things tha don't affect the software | 01:40 |
| dstufft | "supposed" | 01:40 |
| mordred | gotcha | 01:40 |
| mordred | then this would clearly be not post | 01:40 |
| dstufft | like fixing a typo in the docs | 01:40 |
| dstufft | kind of things | 01:40 |
| dstufft | obviously you're allowed to do it, but it's "strongly discouraged" | 01:41 |
| lifeless | mordred: 0a2.dev12 would make sense | 01:41 |
| dstufft | version numbers are hard :/ | 01:41 |
| dstufft | yea what lifeless said | 01:41 |
| mordred | lifeless: no it wouldn't | 01:41 |
| *** nibalizer has quit IRC | 01:41 | |
| mordred | isn't dev supposed to be pre-prelease? | 01:41 |
| dstufft | 0a1 is the last tag, 0a2.devN | 01:42 |
| dstufft | is the subsequent | 01:42 |
| lifeless | mordred: yes, this is pre-release | 01:42 |
| mordred | no. | 01:42 |
| mordred | hang on | 01:42 |
| lifeless | mordred: so 5th commit after 1.3.0.0a1 | 01:42 |
| dstufft | I think mordred missed the fact you incremented the aN | 01:42 |
| lifeless | mordred: would be 1.3.0.0a2.dev5 | 01:42 |
| mordred | right. please don't do that | 01:42 |
| lifeless | mordred: why not ? | 01:42 |
| mordred | because the next release might not be 1.3.0.0a2 and now we're mixing pre adn post versioning semantics | 01:43 |
| lifeless | mordred: we're not mixing anything | 01:43 |
| lifeless | mordred: forget about the pre and post heuristics. Just focus on semver for now. | 01:43 |
| *** yamahata has joined #openstack-infra | 01:44 | |
| dstufft | it doesn't really matter if the next release isn't 1.3.0.0a2, I don't think you can reasonably automatically version the next commit without making some assumption there | 01:44 |
| lifeless | mordred: what is the version for the Nth commit after a tag, assuming no API breaks are involved. | 01:44 |
| mordred | right | 01:44 |
| mordred | so, we can't forget about pre post heuristics | 01:44 |
| mordred | because I will tell you that we will have open revolt | 01:45 |
| dstufft | mordred: do you regret starting a packaging tool yet? ;) | 01:45 |
| mordred | if you have code in pbr that auto-versions the next commit after 1.3.0 as 1.3.1.dev1 | 01:45 |
| mordred | dstufft: it's not the tool that's the problem - this discussion has to be had in person with every person with a project in openstack if the tool doesn't do it right | 01:46 |
| lifeless | mordred: there is a bug *asking for that behaviour* right now | 01:46 |
| *** thuc has quit IRC | 01:46 | |
| mordred | lifeless: from who? | 01:46 |
| dstufft | mordred: I've more or less come to the conclusion that no matter what you do you're going to do it wrong for somebody | 01:46 |
| lifeless | Jay Buffington | 01:47 |
| *** thuc has joined #openstack-infra | 01:47 | |
| dstufft | I think what lifeless is saying is probably correct for the most cases | 01:47 |
| mordred | not from ttx? | 01:47 |
| lifeless | mordred: does ttx do client releases? | 01:47 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Allow merge failures to have unique reporters. https://review.openstack.org/70636 | 01:47 |
| lifeless | https://bugs.launchpad.net/pbr/+bug/1206730 | 01:48 |
| mordred | lifeless: he's talking about a COMPLETELY different thing | 01:49 |
| mordred | which is the servers | 01:49 |
| mordred | which are not PE440 | 01:49 |
| mordred | PEP440 | 01:49 |
| dstufft | why not? | 01:49 |
| dstufft | aren't they just YYYY.NN | 01:49 |
| mordred | yah | 01:50 |
| lifeless | mordred: i'm thoroughly confused. | 01:50 |
| mordred | lifeless: server and client have different release numbering strategies | 01:50 |
| lifeless | mordred: I'm not confused about that. | 01:50 |
| dstufft | mordred: that's PEP440 now :) | 01:50 |
| mordred | dstufft: oh neat! | 01:50 |
| dstufft | we added that | 01:50 |
| lifeless | mordred: but this discussion feels like boxing a cloud | 01:51 |
| *** Daisy_ has joined #openstack-infra | 01:51 | |
| *** thuc has quit IRC | 01:51 | |
| lifeless | mordred: Its very simple. Right now pbr versions post versions *backwards*. | 01:51 |
| lifeless | mordred: its broken. In every regard. | 01:51 |
| *** nibalizer has joined #openstack-infra | 01:51 | |
| mordred | lifeless: can we start from scratch then please - because I think we may be talkign about a few different things | 01:53 |
| lifeless | sorry, backwards is the wrong description for post-versioning | 01:53 |
| mordred | lifeless: let's just focus on post-versioing for now | 01:53 |
| mordred | as pre-versioning is its own special can of worms | 01:53 |
| *** CaptTofu has quit IRC | 01:53 | |
| mordred | (I do concede that this is not correct, so I think we're close to getting somewhere) | 01:54 |
| lifeless | mordred: so post versioning currently emits full releases on untagged commits. | 01:54 |
| lifeless | mordred: this is wrong. agreed? | 01:54 |
| *** CaptTofu has joined #openstack-infra | 01:54 | |
| *** thuc has joined #openstack-infra | 01:54 | |
| mordred | no. I'm not sure that is agreed | 01:55 |
| lifeless | the 5th commit after 1.3.0 in oslo.config will ae an autogeenrated version of 1.3.0.5 | 01:56 |
| mordred | yes | 01:56 |
| dstufft | that seems wrong to me FWIW | 01:56 |
| lifeless | this isn't a release (its not tagged). | 01:56 |
| mordred | I hear that neither of you like that | 01:56 |
| mordred | I don't care | 01:56 |
| lifeless | I don't care isn't a helpfu answer | 01:56 |
| mordred | fair | 01:56 |
| mordred | let me try again | 01:57 |
| lifeless | it makes me want to go fuckit and just do my own thing rather than trying to collaborate | 01:57 |
| clarkb | I'm with mordred. It isnt wrong but may not be desired | 01:57 |
| dstufft | it's not a release so calling it a release doesn't seem sane | 01:57 |
| mordred | lifeless: right. I'm just saying that you're stating an opinion is "wrong" which is also frustrating | 01:57 |
| mordred | but let's figure it out | 01:57 |
| clarkb | I think lifeless should have the ability to do his thing | 01:58 |
| mordred | I agree | 01:58 |
| clarkb | but it isnt an xor | 01:58 |
| mordred | what I'm trying to commuicate is the piece of inforamtion I know at this point | 01:58 |
| lifeless | clarkb: why isn't it wrong? | 01:58 |
| dstufft | or at least pip is going to see that as a final release and any other PEP440 tool will (also any setuptools based tool and probably any other tool) | 01:58 |
| mordred | which is that the fifth commit after 1.3.0 is related to 1.3.0 | 01:58 |
| lifeless | clarkb: is the 5th commit (untagged) after a tag a release, in the openstack world ? | 01:58 |
| *** CaptTofu has quit IRC | 01:58 | |
| clarkb | lifeless because each commit can be a release | 01:59 |
| clarkb | (CD) | 01:59 |
| lifeless | clarkb: it may become one, but until its tagged, is it one? | 01:59 |
| mordred | it's only a release if we upload it to pypi - and it's not a release because it has a g2342345 after it | 01:59 |
| clarkb | lifeless sure | 01:59 |
| lifeless | mordred: PEP440 doesn't understand that .gxxxx means 'not a release' | 01:59 |
| mordred | it doesn't make 1.3.0.5 it makes 1.3.0.5.g$sha | 01:59 |
| lifeless | clarkb: it is a release? | 01:59 |
| clarkb | its a commir | 01:59 |
| mordred | lifeless: we were just discussing that we shoudl add build tags to pep440 | 02:00 |
| dstufft | lifeless: well that's invalid in PEP440 all together, that syntax, in a PEP440 + build tags world I'd assume it'd become 1.3.0.5+g$sha, which is kind of worse in this regards because then pip will see it as a final (whereas the current form it won't because it's invalid) | 02:00 |
| mordred | right. we're exploiting pip's behavior to get a build tag semantic | 02:01 |
| mordred | lifeless: I DO agree that it's non-optimal | 02:01 |
| lifeless | so I say its wrong because our intent is *not* tht it be a release, but pip perceives it as a release. | 02:02 |
| *** alexandra_ is now known as alex-luncheon | 02:02 | |
| lifeless | I will say 'incorrectly perceived as a release' for clearer comms | 02:02 |
| lifeless | IPAAR | 02:02 |
| jhesketh__ | Does anybody here know where Jenkins sets the environment variables for workers to access? eg I'm looking for the part that takes zuul's parameters from gearman and sets them as OS envs | 02:03 |
| mordred | lifeless: pip doesn't perceive it as a release though | 02:04 |
| lifeless | mordred: no? | 02:04 |
| mordred | no | 02:04 |
| mordred | see above | 02:04 |
| dstufft | with the current version string it won't | 02:04 |
| dstufft | 1.3.0.5.gwhatever | 02:04 |
| lifeless | because its an invalid version | 02:05 |
| mordred | right | 02:05 |
| lifeless | but if it gets taught about build tags | 02:05 |
| lifeless | it will become a valid release | 02:05 |
| dstufft | correct | 02:05 |
| mordred | gotcha | 02:05 |
| mordred | ok. we're on the same page now | 02:05 |
| lifeless | ok, now what I want | 02:06 |
| dstufft | automatically versioning after a tagged release is probably not possible to do without either pretending to be a release (1.3.0.5) or by making an assumption about what the _next_ release is going to be | 02:06 |
| lifeless | dstufft: so my proposal is to assume the minimum next version and do dev releases for that. | 02:07 |
| lifeless | dstufft: the minimum next version may never be released, but that doesn't matter | 02:07 |
| dstufft | that's how I would do it | 02:07 |
| dstufft | it seems the most sane | 02:07 |
| mordred | k. now, I completely agree | 02:07 |
| mordred | but | 02:07 |
| mordred | you will get pushback from ttx and notmyname | 02:07 |
| mordred | which to me means they need to be convinced | 02:08 |
| mordred | because they think that a version that indicates it's leading up to a version that never gets released is undesirable | 02:08 |
| dstufft | it's the least bad option afaict | 02:08 |
| mordred | s/I completely agree/you've done a good job convincing me/ | 02:08 |
| lifeless | notmyname: ttx: ok ^ why? Not that pre-releases are not automatically installed by pop... | 02:08 |
| lifeless | bah, Note that... | 02:09 |
| *** sarob_ has joined #openstack-infra | 02:09 | |
| *** thuc has quit IRC | 02:09 | |
| dstufft | what if someone releases an actual 1.3.0.1 | 02:09 |
| *** thuc has joined #openstack-infra | 02:09 | |
| lifeless | dstufft: thats larger than 1.3.0.0a2, so thats fine. | 02:10 |
| dstufft | sorry I mean if you're doing 1.3.0.5 as the 5th release since 1.3.0 | 02:10 |
| lifeless | dstufft: if the last tag was 1.3.0, the next higher version is either 1.3.1 (if we do semver), or 1.3.0.1 (if we do generalised PEP440) | 02:10 |
| lifeless | dstufft: oh right, yes, issues with the current implementation | 02:11 |
| mordred | right. we assume that there will be a bit of sanity there | 02:11 |
| mordred | lifeless: however, let's assume we can make the case | 02:11 |
| lifeless | if we're doing semver, then 1.3.1.0a5 would be the 5th commit after 1.3.0, which leaves room for 1.3.1 as the next release. | 02:12 |
| milki | o.O | 02:12 |
| dstufft | in either case you have to make some assumptions about how many digits you're going to use in the version number | 02:13 |
| dstufft | lifeless: it might want to be 1.3.1.0a0.dev5 | 02:13 |
| dstufft | so that you can release a 1.3.1 alpha 1 | 02:14 |
| lifeless | dstufft: good point, that is what I should have typed | 02:14 |
| clarkb | jhesketh it happens in the gearman jenkins plugin | 02:14 |
| lifeless | dstufft: given the prose I wrote above | 02:14 |
| *** sarob_ has quit IRC | 02:14 | |
| *** thuc has quit IRC | 02:14 | |
| clarkb | it was an explicit thing before gearman but the vars are implicit now via the plugin | 02:14 |
| jhesketh__ | clarkb: right, thanks. I took a look there where it decodes the JSON and places it into an object, but I can't see where the system sets it on the OS? | 02:14 |
| mordred | lifeless: so we'd need to account for .dev being able to come after the patch version or the pre-release version | 02:15 |
| mordred | I think in our case we should only allow those two | 02:15 |
| clarkb | jenkins has an envar java object that ir splats into the env | 02:15 |
| mordred | so 1.3.1.dev1 and 1.3.1.0a1.dev1 - but no more, yah? | 02:15 |
| *** Daisy_ has quit IRC | 02:16 | |
| lifeless | mordred: major/minor/patch surely ? | 02:18 |
| lifeless | dstufft: actually | 02:18 |
| *** sarob_ has joined #openstack-infra | 02:18 | |
| lifeless | 1.3.1 -> 1.3.2.devN (because dev0 < a0) | 02:19 |
| lifeless | 1.3 -> 1.3.1.devN (becase 1.3 == 1.3.0) | 02:19 |
| lifeless | 1 -> 1.0.1.devN | 02:19 |
| lifeless | 1.3.1.0a1 -> 1.3.1.0a2.devN | 02:19 |
| lifeless | 1.3.1.0b1 -> 1.3.1.0b2.devN | 02:20 |
| mordred | lifeless: I don't think we shoudl be that flexible - I think we shoudl enforce semver if we're going here | 02:20 |
| lifeless | 1.3.1.0rc1 -> 1.3.1.0rc2.devN | 02:20 |
| lifeless | mordred: I believe I'm enforcing semver above | 02:20 |
| lifeless | mordred: using the precedence rules in the semver doc, with the serialisation tweak we discussed to get more PEP440 friendly versions | 02:20 |
| mordred | ++ | 02:20 |
| mordred | k. I follow | 02:20 |
| lifeless | dev tags are illegal | 02:21 |
| dstufft | lifeless: yes that makes more sense I think | 02:21 |
| jhesketh__ | thanks clarkb, I'll take a look | 02:21 |
| lifeless | I presume | 02:21 |
| mordred | I think I was saying that we shoudl not really support 1.3.1.0a1.dev1.0a2.dev4 | 02:21 |
| dstufft | that's illegal in PEP440 too FWIW | 02:21 |
| mordred | good | 02:21 |
| mordred | it shold be | 02:21 |
| dstufft | you can add a .devN to anything | 02:21 |
| dstufft | but you can't do multiple layers of stuff | 02:21 |
| dstufft | so you can do like, 1.0.0a1.post3.dev5 | 02:22 |
| dstufft | but that's like the craziest | 02:22 |
| dstufft | (5th development version of the 3rd post release of the first alpha of 1.0.0) | 02:22 |
| lifeless | mordred: so I think the rules for tags become | 02:23 |
| *** asalkeld has joined #openstack-infra | 02:23 | |
| *** sarob_ has quit IRC | 02:23 | |
| asalkeld | hi, are there instruction on releasing a python client for the first time? | 02:23 |
| *** jhesketh__ has quit IRC | 02:24 | |
| asalkeld | from what I understand it's just "git tag -a v1" and "git push gerrit" | 02:25 |
| *** jhesketh__ has joined #openstack-infra | 02:25 | |
| lifeless | you can tag any X, X.Y, X.Y.Z release, and you can use 0aN or 0bN or 0rcN on any of those | 02:25 |
| lifeless | but you can't tag a dev version | 02:25 |
| lifeless | and you can't tag arbitrary PEP440 stuff | 02:25 |
| mordred | asalkeld: two thigns | 02:26 |
| dstufft | lifeless: that seems completely reasonable to me | 02:26 |
| mordred | asalkeld: you need to make sure it's a signed tag | 02:26 |
| lifeless | mordred: ^ ? | 02:26 |
| mordred | asalkeld: and please don't prefix with the v | 02:26 |
| asalkeld | ok | 02:26 |
| mordred | asalkeld: and then it's git push gerrit $tagname | 02:27 |
| mordred | lifeless: ++ | 02:27 |
| lifeless | ok | 02:27 |
| asalkeld | so 1.0.0 | 02:27 |
| lifeless | so I'll send an email to the list I think | 02:27 |
| asalkeld | is that better? | 02:27 |
| *** thuc has joined #openstack-infra | 02:27 | |
| lifeless | or hmm | 02:27 |
| lifeless | I'll make an etherpad | 02:27 |
| mordred | asalkeld: yes | 02:27 |
| lifeless | then send mail | 02:27 |
| *** yamahata has quit IRC | 02:27 | |
| dstufft | lifeless: will this go to openstack-dev? | 02:27 |
| asalkeld | sure | 02:27 |
| lifeless | dstufft: yes | 02:28 |
| dstufft | Ok | 02:28 |
| mordred | asalkeld: git tag -s 1.0.0 ; git push gerrit 1.0.0 will be perfect | 02:28 |
| *** thuc_ has joined #openstack-infra | 02:28 | |
| lifeless | https://etherpad.openstack.org/p/pbr-postversion-semver | 02:28 |
| dstufft | i was convinced to subscribe to that | 02:28 |
| dstufft | so I'll watch for it | 02:28 |
| dstufft | and comment :D | 02:28 |
| asalkeld | thanks mordred | 02:28 |
| mordred | asalkeld: sure nuff! | 02:28 |
| mordred | asalkeld: also, you'll need to have permissions to push tags ... | 02:28 |
| asalkeld | yeah, I know - another issue tho' | 02:29 |
| *** thuc_ has quit IRC | 02:30 | |
| *** thuc_ has joined #openstack-infra | 02:30 | |
| *** zhiyan_ is now known as zhiyan | 02:30 | |
| *** thuc has quit IRC | 02:32 | |
| *** weshay has quit IRC | 02:35 | |
| *** zhiyan is now known as zhiyan_ | 02:35 | |
| jhesketh__ | clarkb: so the problem I'm faced with is that I was asked to make zuul's swift instructions able to handle multiple result destinations. I implemented this as a list in the gearman data but it occurs to me this won't be very accessible to the worker | 02:39 |
| *** thuc_ has quit IRC | 02:40 | |
| jhesketh__ | as best as I can tell Jenkins expects there to only be a key+value pair and casts the value to a string | 02:40 |
| jhesketh__ | I'm actually not sure how the list would be interpreted at this point | 02:40 |
| jhesketh__ | clarkb: I don't suppose you have any thoughts there? | 02:40 |
| *** masayukig has joined #openstack-infra | 02:41 | |
| *** CaptTofu has joined #openstack-infra | 02:42 | |
| *** Daisy_ has joined #openstack-infra | 02:43 | |
| *** asalkeld has left #openstack-infra | 02:46 | |
| *** matsuhashi has quit IRC | 02:48 | |
| *** matsuhas_ has joined #openstack-infra | 02:52 | |
| *** Daisy_ has quit IRC | 02:53 | |
| *** Daisy_ has joined #openstack-infra | 02:56 | |
| jeblair | jhesketh__: oh hi! i think we should probably explode the parameters... | 02:59 |
| jhesketh__ | jeblair: yeah, that's my feeling | 02:59 |
| *** dcramer_ has quit IRC | 02:59 | |
| jeblair | jhesketh__: so if you have instructions for the foo and bar containers, we end up with FOO_SOMETHING and BAR_SOMETHING parameters | 02:59 |
| jhesketh__ | I poked at Jenkins and I don't want to patch that | 02:59 |
| jeblair | jhesketh__: (or equivalent) | 02:59 |
| jeblair | jhesketh__: yeah, i think it's better not to have to encode anything about this into the intermediary gearman stuff | 03:00 |
| nibalizer | /quit | 03:00 |
| *** nibalizer has quit IRC | 03:00 | |
| jeblair | jhesketh__: and instead just expect that the things on both sides of the communication channel (the zuul swift module and the jobs) expect those flattened values | 03:01 |
| jhesketh__ | jeblair: yep, that was what I had in mind too | 03:02 |
| jhesketh__ | although it's not really an intermediary gearman thing, it's more of a zuul worker thing | 03:02 |
| jhesketh__ | ie if I fixed it on the Jenkins level it'd allow zuul to send more complicated data structures (via gearman) | 03:03 |
| jeblair | jhesketh__: true; i guess that goes to whether the zuul-gearman protocol should become complex; i lean toward simple at the moment though -- i think it'll be easier to keep it stable and therefore support more kinds of workers | 03:04 |
| *** dkehn has joined #openstack-infra | 03:04 | |
| jhesketh__ | jeblair: I agree. Although theoretically we're making the protocol more complicated by sending these instructions. For example, is it expected/required that a worker utilise the swift instructions | 03:05 |
| jhesketh__ | (clearly not in our case, but as an example) | 03:05 |
| *** eharney has quit IRC | 03:05 | |
| *** SumitNaiksatam has quit IRC | 03:06 | |
| *** dkehn_ has quit IRC | 03:06 | |
| jeblair | jhesketh__: *nod* | 03:07 |
| jhesketh__ | jeblair: okay, I'm rerolling now.. do you think we should use name.upper() in the swift destination name, or just leave it to the operator | 03:09 |
| jeblair | jhesketh__: operator i think | 03:13 |
| jhesketh__ | jeblair: should we allow a set of instructions named "MyLogs" and "mylogs"? | 03:13 |
| jhesketh__ | jeblair: Also, I think it perhaps should be "SWIFT_name_PARAM" | 03:18 |
| jeblair | jhesketh__: do you know if swift container names are case sensitive? i'd follow the lead there, but i don't know off the top of my head. | 03:18 |
| *** sarob_ has joined #openstack-infra | 03:18 | |
| lifeless | dstufft: mordred: mail sent | 03:18 |
| *** Daisy__ has joined #openstack-infra | 03:18 | |
| jhesketh__ | jeblair: a quick test shows they are case sensitive (ie I can have Logs and LOGS) | 03:19 |
| jhesketh__ | but the name does not define the container | 03:20 |
| *** Daisy_ has quit IRC | 03:21 | |
| *** peddamat has quit IRC | 03:21 | |
| jeblair | jhesketh__: agreed, re SWIFT_name_PARAM | 03:21 |
| *** matsuhas_ has quit IRC | 03:22 | |
| openstackgerrit | Angus Salkeld proposed a change to openstack-infra/config: Add new services to solum devstack setup https://review.openstack.org/80883 | 03:22 |
| jeblair | jhesketh__: true; my inclination would be to follow swift and just pass things through as much as possible; but i don't feel too strongly about that if, as you dig into this, you think we should take another approach. | 03:22 |
| *** sarob_ has quit IRC | 03:23 | |
| jhesketh__ | yeah I'm also not opinionated here | 03:23 |
| jhesketh__ | I think we'll leave it to the operator | 03:23 |
| *** dkehn is now known as dkehn_ | 03:23 | |
| jeblair | cool | 03:23 |
| *** dkehn__ is now known as dkehn | 03:23 | |
| *** yjiang has joined #openstack-infra | 03:26 | |
| *** medieval1 has quit IRC | 03:33 | |
| *** medieval1 has joined #openstack-infra | 03:33 | |
| *** medieval1 has quit IRC | 03:38 | |
| *** CaptTofu has quit IRC | 03:48 | |
| *** thuc has joined #openstack-infra | 03:51 | |
| *** thuc has quit IRC | 03:55 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Send swift upload instructions to workers https://review.openstack.org/68297 | 03:56 |
| *** emagana has joined #openstack-infra | 04:05 | |
| *** alex-luncheon is now known as alexandra_ | 04:08 | |
| *** sarob_ has joined #openstack-infra | 04:18 | |
| *** wenlock has joined #openstack-infra | 04:20 | |
| *** Daisy__ has quit IRC | 04:22 | |
| mordred | lifeless: thanks. and thanks for hammering away on this with me - I think we've refined things nicely | 04:22 |
| *** sarob_ has quit IRC | 04:23 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/nodepool: Add bash completion to nodepool https://review.openstack.org/80889 | 04:24 |
| *** yamahata has joined #openstack-infra | 04:25 | |
| mordred | jeblair: you still aroudn? | 04:25 |
| * mordred had a possibly terrible idea related to sequencing of puppet runs taht's possibly quicker/simpler/less moving parts than the salt plan | 04:26 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Add a slave-script to Jenkins for pushing to swift https://review.openstack.org/71189 | 04:26 |
| mordred | which is - what if we allow the puppetmaster to ssh to each of the node and run puppet? as in - stop running puppet agent anywhere, and write a script on the master which does the git pull then the puppet run on each of the hosts | 04:27 |
| *** wenlock has quit IRC | 04:28 | |
| *** wenlock has joined #openstack-infra | 04:28 | |
| mordred | it would allow us to avoid stuck puppet runs, and also allow us to encode sequencing very easily | 04:29 |
| mordred | such as "run puppet on cgit slaves, then on gerrit" - purely by order | 04:29 |
| mordred | the downside as I can see it is a) we'd serialize puppet runs (although maybe that's not terrible) b) security - we'd be granting puppetmaster some power | 04:29 |
| mordred | however, given that rooting the puppetmaster box is essentially full root everywhere, maybe it's not bad | 04:30 |
| mordred | but also - we could turn off the 8140 port in puppetmaster's iptables rules - and we could have the ssh commands to the hosts forward 8140 so that puppetmaster port is only accessible during the puppet run | 04:31 |
| mordred | and with that- we could even put ina jump box if we wanted to and completely turn off public ip access to the puppetmaster | 04:31 |
| mordred | HOWEVER - I don't care about any of those things | 04:32 |
| mordred | the main one was that turning off puppet agent daemon and doing sequencing would be a pretty easy thing to do | 04:33 |
| mordred | fungi, jeblair: ^^ when you wake up | 04:35 |
| *** alff has joined #openstack-infra | 04:38 | |
| *** matsuhashi has joined #openstack-infra | 04:43 | |
| *** dstufft|laptop has joined #openstack-infra | 04:46 | |
| mattoliverau | mordred: lol, I see you thinking out loud. Interesting idea. Would you then have to cron the script, or will humans control when it is run? If the former, could stuck puppet runs still exist? | 04:48 |
| *** alff has quit IRC | 04:48 | |
| *** alff has joined #openstack-infra | 04:50 | |
| *** Daisy_ has joined #openstack-infra | 04:54 | |
| mordred | mattoliverau: it would be cron'd - and I think they could stick - but my gut tells me that the stuck runs is realted to concurrency on the master | 05:02 |
| *** zhiyan_ is now known as zhiyan | 05:02 | |
| mordred | also, however, we'd notice that puppet in general is stuck more easily - and could put a timeout in the driver script | 05:03 |
| mattoliverau | That's true, a sensible timeout would be a useful plan even if it is stable.. cause you never know what future change will break a run. | 05:04 |
| mordred | yah | 05:05 |
| mordred | but if we put the timeout into the cron (with an error message that we timed it out) it SHOULDN'T timeout frequently - but it would also allow us to log it and move on rather than just stick for all time | 05:06 |
| mattoliverau | As long as a senible timeout is decided upon, even if it's rather large like an hour to 4 :) Depending on the frequency of the cron job of course. | 05:07 |
| mattoliverau | The thing is to stop stuck runs not penalise slower runs I aseume. | 05:08 |
| mordred | yah | 05:08 |
| mordred | most of our runs are only a minute or two really | 05:08 |
| mordred | so long-runs are either somethign that needs to be designed or a problem | 05:09 |
| mattoliverau | True | 05:09 |
| mattoliverau | And in the script you could background any run that can run in parallel and serialise the ones that can't.. that's pretty cool. | 05:11 |
| *** VijayTripathi has quit IRC | 05:14 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/nodepool: Reuse floating ips https://review.openstack.org/80893 | 05:17 |
| mordred | yah | 05:17 |
| mordred | I mean, it feels hacky - but the more I think about it the more I like it | 05:17 |
| mattoliverau | Or even better, grabs the list of nodes from site.pp and by default runs them in parellel (back ground) and you specify the ones you want to serialise. That way the maintenance of the script in minimalised. I.e. everytime a new infra node needed to be maintained is created the script doesn't need to me modified. | 05:17 |
| mattoliverau | Yeah, I know what you mean. Puppet and salt exist for a reason.. and a script seems hacky. | 05:17 |
| mordred | well... we don't have a list in site.pp because of wildcards | 05:17 |
| mattoliverau | or hosts file or dns or even better 'puppet cert list -a' :) | 05:18 |
| *** sarob_ has joined #openstack-infra | 05:18 | |
| mordred | puppet cert list -a seems perfect | 05:18 |
| mordred | might incite us to actually revoke certs for things where are not real things anymroe | 05:20 |
| *** sarob__ has joined #openstack-infra | 05:21 | |
| mordred | puppet cert list -a | grep '^\+' | 05:22 |
| mattoliverau | yeah, and you'll find them pretty quickly too, cron should log them for you :) | 05:22 |
| mattoliverau | Yeah, the * are the ones that are signed, so even better :) | 05:22 |
| *** sarob_ has quit IRC | 05:23 | |
| mattoliverau | *and the ones that are start with a * i mean. So you might want to grep them out, unless everything is autosigned then I guess it doesn't matter. | 05:23 |
| mordred | biggest question would be how we detect if someone is working and wants puppet to not run | 05:24 |
| mattoliverau | Check to see if someone is logged in? who or w? | 05:24 |
| mattoliverau | Your sshing anyway. | 05:24 |
| mattoliverau | Or see if a bash binary and/or screen is running in case someone is working but not logged in. | 05:26 |
| *** sarob__ has quit IRC | 05:26 | |
| mordred | I guess "puppet agent --disable" actually prevents command line runs too | 05:26 |
| mordred | so as long as someone has done that, it should be fine | 05:27 |
| mattoliverau | Yeah, I dunno, but that can be tested :) | 05:27 |
| mordred | yah | 05:27 |
| mattoliverau | If it doesn't you could make a business decision like, if you want to work on a server and don't want it to run then create a file /root/NO-PUPPET and the existence can be checked in the script. | 05:29 |
| mattoliverau | Yup, puppet agent --disable works: | 05:31 |
| mattoliverau | notice: Skipping run of Puppet configuration client; administratively disabled; use 'puppet agent --enable' to re-enable. | 05:32 |
| mordred | yah. I Think that's perfect | 05:32 |
| *** nicedice has quit IRC | 05:32 | |
| *** Daisy_ has quit IRC | 05:33 | |
| *** wenlock has quit IRC | 05:33 | |
| *** morganfainberg_Z is now known as morganfainberg | 05:34 | |
| *** alff has quit IRC | 05:41 | |
| *** vogxn has joined #openstack-infra | 05:43 | |
| *** Daisy_ has joined #openstack-infra | 05:48 | |
| *** CaptTofu has joined #openstack-infra | 05:49 | |
| *** vogxn has quit IRC | 05:51 | |
| *** nosnos has quit IRC | 05:54 | |
| *** nosnos has joined #openstack-infra | 05:54 | |
| *** CaptTofu has quit IRC | 05:54 | |
| *** matsuhashi has quit IRC | 05:56 | |
| *** matsuhashi has joined #openstack-infra | 05:57 | |
| *** matsuhashi has quit IRC | 05:57 | |
| *** vogxn has joined #openstack-infra | 05:59 | |
| *** matsuhashi has joined #openstack-infra | 06:00 | |
| *** rpodolyaka has joined #openstack-infra | 06:02 | |
| *** matsuhashi has quit IRC | 06:09 | |
| *** ildikov_ has quit IRC | 06:09 | |
| *** matsuhashi has joined #openstack-infra | 06:10 | |
| *** matsuhas_ has joined #openstack-infra | 06:12 | |
| *** matsuhashi has quit IRC | 06:14 | |
| *** rpodolyaka has quit IRC | 06:16 | |
| *** rpodolyaka has joined #openstack-infra | 06:16 | |
| jamielennox | can anyone tell me what we left off of this new_project: https://review.openstack.org/#/c/73074/ to give not_registered in reviews: https://review.openstack.org/#/c/80897/ | 06:17 |
| *** sarob_ has joined #openstack-infra | 06:18 | |
| *** rpodolyaka has quit IRC | 06:19 | |
| *** sarob_ has quit IRC | 06:23 | |
| *** nosnos has quit IRC | 06:24 | |
| mattoliverau | Well I'm off to cook dinner, see you all tomorrow, for all you American's who will read this in scroll back while I sleep, have a good Monday :) | 06:24 |
| *** nosnos has joined #openstack-infra | 06:25 | |
| *** matsuhas_ has quit IRC | 06:25 | |
| *** matsuhashi has joined #openstack-infra | 06:25 | |
| *** rpodolyaka has joined #openstack-infra | 06:38 | |
| *** matsuhashi has quit IRC | 06:39 | |
| *** matsuhashi has joined #openstack-infra | 06:40 | |
| *** ildikov_ has joined #openstack-infra | 06:40 | |
| *** thomasbiege has joined #openstack-infra | 06:47 | |
| *** saju_m has joined #openstack-infra | 06:47 | |
| *** thomasbiege has quit IRC | 06:49 | |
| *** mrda is now known as mrda_away | 06:53 | |
| *** rpodolyaka has quit IRC | 06:56 | |
| *** rpodolyaka has joined #openstack-infra | 06:58 | |
| *** rpodolyaka has quit IRC | 07:00 | |
| *** skraynev_afk is now known as skraynev | 07:01 | |
| *** rpodolyaka has joined #openstack-infra | 07:03 | |
| *** rpodolyaka has quit IRC | 07:07 | |
| *** yolanda has joined #openstack-infra | 07:09 | |
| *** sarob_ has joined #openstack-infra | 07:18 | |
| *** sarob_ has quit IRC | 07:23 | |
| *** alexandra_ is now known as alex-awayu | 07:28 | |
| *** alex-awayu is now known as alex-away | 07:28 | |
| *** flaper87|afk is now known as flaper87 | 07:30 | |
| *** vogxn has quit IRC | 07:30 | |
| *** peddamat has joined #openstack-infra | 07:32 | |
| *** mkoderer has joined #openstack-infra | 07:33 | |
| *** dstufft|laptop has quit IRC | 07:36 | |
| mordred | SergeyLukjanov: ping | 07:38 |
| *** jlibosva has joined #openstack-infra | 07:43 | |
| *** alff has joined #openstack-infra | 07:46 | |
| *** e0ne has joined #openstack-infra | 07:46 | |
| *** bookwar has joined #openstack-infra | 07:48 | |
| *** jamielennox is now known as jamielennox|away | 07:50 | |
| *** CaptTofu has joined #openstack-infra | 07:50 | |
| *** CaptTofu has quit IRC | 07:55 | |
| *** e0ne has quit IRC | 07:56 | |
| *** rpodolyaka has joined #openstack-infra | 07:57 | |
| *** nijaba has joined #openstack-infra | 08:02 | |
| *** rcarrillocruz has joined #openstack-infra | 08:02 | |
| *** rpodolyaka1 has joined #openstack-infra | 08:03 | |
| * ttx yawns | 08:05 | |
| *** rpodolyaka1 has quit IRC | 08:07 | |
| *** amotoki has joined #openstack-infra | 08:12 | |
| *** rcarrillocruz1 has joined #openstack-infra | 08:13 | |
| *** nosnos has quit IRC | 08:13 | |
| *** rcarrillocruz has quit IRC | 08:13 | |
| *** nosnos has joined #openstack-infra | 08:14 | |
| *** matsuhashi has quit IRC | 08:17 | |
| *** matsuhashi has joined #openstack-infra | 08:17 | |
| *** rcarrillocruz1 has quit IRC | 08:21 | |
| *** openstack has quit IRC | 08:21 | |
| *** openstack has joined #openstack-infra | 08:30 | |
| *** dizquierdo has joined #openstack-infra | 08:30 | |
| *** andreaf has joined #openstack-infra | 08:31 | |
| *** vponomaryov has joined #openstack-infra | 08:32 | |
| *** e0ne has joined #openstack-infra | 08:32 | |
| *** vogxn has joined #openstack-infra | 08:34 | |
| *** nosnos has quit IRC | 08:34 | |
| *** jcoufal has joined #openstack-infra | 08:36 | |
| *** bada_ has quit IRC | 08:40 | |
| SergeyLukjanov | mordred, pong | 08:43 |
| SergeyLukjanov | ttx, morning :) | 08:44 |
| *** e0ne_ has joined #openstack-infra | 08:44 | |
| ttx | SergeyLukjanov: o/ | 08:44 |
| *** morganfainberg is now known as morganfainberg_Z | 08:45 | |
| ttx | SergeyLukjanov: how does the renaming go so far ? | 08:45 |
| * ttx is still a bit behind in emails | 08:45 | |
| SergeyLukjanov | ttx, it's not bad | 08:45 |
| ttx | (nothing serious just a couple thousand emails to read) | 08:45 |
| SergeyLukjanov | ttx, :) | 08:45 |
| SergeyLukjanov | ttx, I hope will complete renaming this week | 08:46 |
| *** e0ne has quit IRC | 08:48 | |
| *** jgallard has joined #openstack-infra | 08:55 | |
| *** vkozhukalov_ has joined #openstack-infra | 08:55 | |
| *** matsuhas_ has quit IRC | 08:56 | |
| kashyap | ttx, [Offtopic] I picked up the book by Bruce Schneir, by (your) implicit recommendation from your blog. I'm liking it, thank you! | 09:02 |
| ttx | kashyap: great! | 09:03 |
| *** rpodolyaka1 has joined #openstack-infra | 09:03 | |
| * ttx grabs coffee before reading -dev | 09:03 | |
| *** johnthetubaguy has joined #openstack-infra | 09:04 | |
| *** andreaf2 has joined #openstack-infra | 09:04 | |
| *** johnthetubaguy1 has joined #openstack-infra | 09:06 | |
| *** johnthetubaguy has quit IRC | 09:06 | |
| *** matsuhashi has joined #openstack-infra | 09:06 | |
| *** andreaf has quit IRC | 09:07 | |
| *** rpodolyaka1 has quit IRC | 09:08 | |
| *** emagana has quit IRC | 09:09 | |
| *** rcarrillocruz1 has joined #openstack-infra | 09:11 | |
| *** rcarrillocruz has quit IRC | 09:12 | |
| *** yassine has joined #openstack-infra | 09:13 | |
| *** nosnos has joined #openstack-infra | 09:14 | |
| *** dizquierdo has quit IRC | 09:17 | |
| *** rcarrillocruz has joined #openstack-infra | 09:18 | |
| openstackgerrit | A change was merged to openstack-infra/config: Test only projects listed in projects.txt. https://review.openstack.org/79170 | 09:18 |
| *** sarob_ has joined #openstack-infra | 09:18 | |
| *** rcarrillocruz1 has quit IRC | 09:20 | |
| *** jcoufal has quit IRC | 09:20 | |
| *** tnurlygayanov has joined #openstack-infra | 09:21 | |
| *** sarob_ has quit IRC | 09:23 | |
| *** rossella_s has joined #openstack-infra | 09:26 | |
| openstackgerrit | Ivan Berezovskiy proposed a change to openstack-infra/config: Enable Gearman as default on Jenkins slaves https://review.openstack.org/73214 | 09:27 |
| *** noorul has joined #openstack-infra | 09:27 | |
| *** johnthetubaguy1 has quit IRC | 09:27 | |
| *** johnthetubaguy has joined #openstack-infra | 09:31 | |
| *** persia has joined #openstack-infra | 09:32 | |
| *** persia has quit IRC | 09:32 | |
| *** persia has joined #openstack-infra | 09:32 | |
| *** Daisy_ has quit IRC | 09:34 | |
| *** yjiang has quit IRC | 09:37 | |
| *** krtaylor has quit IRC | 09:38 | |
| *** emagana has joined #openstack-infra | 09:39 | |
| *** saschpe has quit IRC | 09:42 | |
| *** chandan_kumar has joined #openstack-infra | 09:48 | |
| *** CaptTofu has joined #openstack-infra | 09:51 | |
| *** vkozhukalov_ has quit IRC | 09:51 | |
| *** saschpe has joined #openstack-infra | 09:52 | |
| *** IvanBerezovskiy has joined #openstack-infra | 09:55 | |
| *** nosnos_ has joined #openstack-infra | 09:55 | |
| *** emagana has quit IRC | 09:56 | |
| *** CaptTofu has quit IRC | 09:56 | |
| *** masayukig has quit IRC | 09:58 | |
| *** yolanda has quit IRC | 09:58 | |
| *** nosnos has quit IRC | 09:58 | |
| *** saschpe has quit IRC | 10:02 | |
| *** derekh has joined #openstack-infra | 10:02 | |
| *** derekh has quit IRC | 10:04 | |
| *** saschpe has joined #openstack-infra | 10:08 | |
| *** nosnos_ has quit IRC | 10:10 | |
| *** e0ne_ has quit IRC | 10:12 | |
| *** nosnos has joined #openstack-infra | 10:12 | |
| SergeyLukjanov | ttx, am I right that we're not adding requirements of non-official projects to global requirements | 10:13 |
| SergeyLukjanov | ? | 10:13 |
| *** saschpe has quit IRC | 10:13 | |
| *** e0ne has joined #openstack-infra | 10:14 | |
| ttx | SergeyLukjanov: not sure | 10:14 |
| *** saschpe has joined #openstack-infra | 10:15 | |
| SergeyLukjanov | ttx, it was when sahara wasn't incubated yet, but probably something changed | 10:16 |
| *** yolanda has joined #openstack-infra | 10:16 | |
| SergeyLukjanov | https://review.openstack.org/#/c/80756/ | 10:16 |
| *** saschpe has quit IRC | 10:16 | |
| *** saju_m has quit IRC | 10:17 | |
| *** tteggel has quit IRC | 10:18 | |
| *** sarob_ has joined #openstack-infra | 10:18 | |
| *** tteggel has joined #openstack-infra | 10:18 | |
| *** saschpe has joined #openstack-infra | 10:21 | |
| *** sarob_ has quit IRC | 10:23 | |
| *** saschpe has quit IRC | 10:26 | |
| *** pbelanyi has joined #openstack-infra | 10:27 | |
| *** saschpe has joined #openstack-infra | 10:28 | |
| *** saschpe has quit IRC | 10:28 | |
| *** saju_m has joined #openstack-infra | 10:32 | |
| *** matsuhashi has quit IRC | 10:38 | |
| *** yolanda has quit IRC | 10:39 | |
| *** rcarrillocruz1 has joined #openstack-infra | 10:42 | |
| *** afazekas has joined #openstack-infra | 10:43 | |
| SergeyLukjanov | fungi, probably, you'd like to take a look on v | 10:45 |
| SergeyLukjanov | https://review.openstack.org/#/c/56823/ | 10:45 |
| *** rcarrillocruz has quit IRC | 10:45 | |
| openstackgerrit | A change was merged to openstack-infra/config: Gate python-glanceclient on PyPy https://review.openstack.org/76072 | 10:46 |
| *** nosnos has quit IRC | 10:46 | |
| *** nosnos has joined #openstack-infra | 10:47 | |
| *** yamahata has quit IRC | 10:52 | |
| *** matsuhashi has joined #openstack-infra | 10:54 | |
| *** vkozhukalov_ has joined #openstack-infra | 10:57 | |
| *** saschpe has joined #openstack-infra | 10:57 | |
| openstackgerrit | A change was merged to openstack-infra/config: Implements: blueprint add-manila-to-gerritbot https://review.openstack.org/72872 | 10:58 |
| *** rpodolyaka1 has joined #openstack-infra | 11:01 | |
| *** lcostantino has joined #openstack-infra | 11:03 | |
| *** yolanda has joined #openstack-infra | 11:04 | |
| *** rpodolyaka1 has quit IRC | 11:05 | |
| *** matsuhashi has quit IRC | 11:05 | |
| *** jgallard has quit IRC | 11:07 | |
| *** jcoufal has joined #openstack-infra | 11:09 | |
| *** openstackgerrit has quit IRC | 11:10 | |
| *** openstackgerrit has joined #openstack-infra | 11:11 | |
| *** matsuhashi has joined #openstack-infra | 11:13 | |
| *** jhesketh__ has quit IRC | 11:13 | |
| *** jhesketh has quit IRC | 11:13 | |
| *** saju_m has quit IRC | 11:15 | |
| rcarrillocruz1 | Ng: hi chris, are you around? If you are not too busy, I'd appreciate if you could give a core review to https://review.openstack.org/#/c/79205/ please... | 11:16 |
| *** vogxn has quit IRC | 11:17 | |
| Ng | rcarrillocruz1: hey, I'm afraid I'm not a core on openstack-infra. I can look and give it a +1, but I have no +2 powers there, sorry! | 11:17 |
| *** sarob_ has joined #openstack-infra | 11:18 | |
| rcarrillocruz1 | ah, no biggie then, thanks :-) | 11:18 |
| *** andreaf2 has quit IRC | 11:21 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: make highlighting work on the active document https://review.openstack.org/80951 | 11:22 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: provide a -e run target to run server locally https://review.openstack.org/80952 | 11:22 |
| *** sarob_ has quit IRC | 11:23 | |
| *** dkranz has quit IRC | 11:24 | |
| *** matsuhashi has quit IRC | 11:31 | |
| *** chuck_ has joined #openstack-infra | 11:31 | |
| openstackgerrit | Cyril Roelandt proposed a change to openstack-infra/nose-html-output: Port to Python 3. https://review.openstack.org/80956 | 11:33 |
| openstackgerrit | Cyril Roelandt proposed a change to openstack-infra/nose-html-output: Port to Python 3. https://review.openstack.org/80956 | 11:34 |
| *** chuck_ is now known as zul | 11:34 | |
| *** zul has joined #openstack-infra | 11:34 | |
| *** rcarrillocruz1 has quit IRC | 11:35 | |
| *** saju_m has joined #openstack-infra | 11:36 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: set the group on nova-specs to nova https://review.openstack.org/80957 | 11:38 |
| *** yfried has joined #openstack-infra | 11:39 | |
| *** yfried has left #openstack-infra | 11:41 | |
| *** dizquierdo has joined #openstack-infra | 11:42 | |
| *** e0ne has quit IRC | 11:45 | |
| *** weshay has joined #openstack-infra | 11:46 | |
| openstackgerrit | Alexander Jones proposed a change to openstack-infra/git-review: Fix parsing of SCP-style URLs, as these are valid in Git itself https://review.openstack.org/72751 | 11:46 |
| *** katyafervent_awa is now known as katyafervent | 11:48 | |
| *** e0ne has joined #openstack-infra | 11:49 | |
| *** sdake has quit IRC | 11:49 | |
| openstackgerrit | Noorul Islam K M proposed a change to openstack-infra/config: Use python-jobs template and add release jobs for solum project https://review.openstack.org/80780 | 11:52 |
| *** CaptTofu has joined #openstack-infra | 11:52 | |
| *** jhesketh__ has joined #openstack-infra | 11:54 | |
| *** jhesketh__ has quit IRC | 11:54 | |
| *** jhesketh has joined #openstack-infra | 11:55 | |
| openstackgerrit | A change was merged to openstack-infra/config: Note requirement to use 'new-project' https://review.openstack.org/80693 | 11:56 |
| *** CaptTofu has quit IRC | 11:56 | |
| *** CaptTofu has joined #openstack-infra | 12:02 | |
| *** dprince has joined #openstack-infra | 12:03 | |
| *** aysyd has joined #openstack-infra | 12:04 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add qa-specs to tempest group https://review.openstack.org/80959 | 12:06 |
| pleia2 | good morning | 12:07 |
| *** yamahata has joined #openstack-infra | 12:07 | |
| *** e0ne_ has joined #openstack-infra | 12:08 | |
| *** sandywalsh_ has quit IRC | 12:10 | |
| *** alexpilotti has joined #openstack-infra | 12:11 | |
| *** pdmars has joined #openstack-infra | 12:12 | |
| *** e0ne has quit IRC | 12:12 | |
| sdague | pleia2: morning | 12:12 |
| sdague | you up in maine this week? | 12:13 |
| *** pdmars has quit IRC | 12:14 | |
| SergeyLukjanov | sdague, morning, how are you? | 12:16 |
| *** alexpilotti has quit IRC | 12:16 | |
| pleia2 | sdague: yep, snow and all | 12:16 |
| sdague | SergeyLukjanov: good, up to 2nd cup of coffee | 12:17 |
| sdague | pleia2: heh. yeh, I'm very happy that we missed the storm that hit DC last night. I like winter, but I'm ready for spring | 12:17 |
| sdague | pleia2: what part of maine? | 12:17 |
| *** dims has quit IRC | 12:18 | |
| pleia2 | sdague: central, inland; the cold and boring part :) | 12:18 |
| sdague | heh | 12:18 |
| *** sarob_ has joined #openstack-infra | 12:18 | |
| *** dims has joined #openstack-infra | 12:18 | |
| pleia2 | storm hit here right before I arrived, so I just get to enjoy the "already fallen" type of snow, which is nice | 12:18 |
| *** yaguang has quit IRC | 12:19 | |
| *** pdmars has joined #openstack-infra | 12:20 | |
| *** esker has quit IRC | 12:21 | |
| *** chandan_kumar has quit IRC | 12:21 | |
| *** esker has joined #openstack-infra | 12:21 | |
| *** sarob_ has quit IRC | 12:23 | |
| *** miarmak has joined #openstack-infra | 12:25 | |
| *** esker has quit IRC | 12:26 | |
| *** lcostantino has quit IRC | 12:26 | |
| *** matsuhashi has joined #openstack-infra | 12:27 | |
| *** CaptTofu has quit IRC | 12:31 | |
| *** rfolco has joined #openstack-infra | 12:32 | |
| *** e0ne has joined #openstack-infra | 12:32 | |
| *** e0ne_ has quit IRC | 12:36 | |
| *** matsuhashi has quit IRC | 12:37 | |
| *** nosnos_ has joined #openstack-infra | 12:37 | |
| *** mbacchi has joined #openstack-infra | 12:37 | |
| *** nosnos has quit IRC | 12:37 | |
| *** matsuhashi has joined #openstack-infra | 12:37 | |
| *** AJaeger has joined #openstack-infra | 12:38 | |
| *** yassine has quit IRC | 12:42 | |
| *** matsuhashi has quit IRC | 12:42 | |
| *** yassine has joined #openstack-infra | 12:42 | |
| *** eharney has joined #openstack-infra | 12:43 | |
| *** e0ne has quit IRC | 12:45 | |
| *** e0ne has joined #openstack-infra | 12:45 | |
| *** lcostantino has joined #openstack-infra | 12:47 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: highlighting the way you expect https://review.openstack.org/80966 | 12:48 |
| *** pblaho has quit IRC | 12:48 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: provide a -e run target to run server locally https://review.openstack.org/80952 | 12:50 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: highlighting the way you expect https://review.openstack.org/80966 | 12:50 |
| *** CaptTofu has joined #openstack-infra | 12:50 | |
| *** jeckersb_gone is now known as jeckersb | 12:52 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: highlighting the way you expect https://review.openstack.org/80966 | 12:52 |
| *** weshay has quit IRC | 12:54 | |
| *** saschpe has quit IRC | 12:54 | |
| *** CaptTofu has quit IRC | 12:54 | |
| *** adalbas has joined #openstack-infra | 12:56 | |
| *** smarcet has joined #openstack-infra | 12:56 | |
| *** jlibosva has quit IRC | 12:57 | |
| *** mriedem has joined #openstack-infra | 12:57 | |
| *** ArxCruz_ has joined #openstack-infra | 12:57 | |
| *** jlibosva has joined #openstack-infra | 12:58 | |
| *** kiall_ is now known as Kiall | 12:59 | |
| openstackgerrit | Alexander Jones proposed a change to openstack-infra/git-review: Fix parsing of SCP-style URLs, as these are valid in Git itself https://review.openstack.org/72751 | 12:59 |
| *** jgallard has joined #openstack-infra | 13:02 | |
| *** matsuhashi has joined #openstack-infra | 13:03 | |
| *** saschpe has joined #openstack-infra | 13:05 | |
| *** matsuhashi has quit IRC | 13:05 | |
| *** sandywalsh has joined #openstack-infra | 13:05 | |
| *** matsuhashi has joined #openstack-infra | 13:05 | |
| maurosr | guys where can I find the piece of code that filters the logs into warnings, traces, infos.. ? | 13:08 |
| *** matsuhashi has quit IRC | 13:10 | |
| sdague | maurosr: what do you mean? | 13:11 |
| *** alexpilotti has joined #openstack-infra | 13:11 | |
| *** krtaylor has joined #openstack-infra | 13:11 | |
| *** nosnos_ has quit IRC | 13:12 | |
| maurosr | sdague: like here http://logs.openstack.org/39/80939/1/check/check-tempest-dsvm-neutron/f09386c/logs/screen-c-api.txt.gz in the top of page there is a display level | 13:12 |
| *** thuc has joined #openstack-infra | 13:12 | |
| *** saschpe has quit IRC | 13:12 | |
| sdague | that's os-loganalyze, which I was hacking on this morning again | 13:13 |
| *** thuc_ has joined #openstack-infra | 13:13 | |
| *** jlibosva1 has joined #openstack-infra | 13:14 | |
| *** ryanpetrello has joined #openstack-infra | 13:14 | |
| maurosr | sdague: cool, guys here will need that to start hunt our ci skips ;), thanks for it | 13:14 |
| *** dkehn__ is now known as dkehn | 13:14 | |
| sdague | the config repo shows how we install it on our log server | 13:15 |
| sdague | I'm not sure how you'd do it in front of swift | 13:15 |
| maurosr | sdague: yup, we cant, but I thought to run that in a local machine, and incrementally/mannualy refresh it as soon as we get rid of a new bug | 13:16 |
| openstackgerrit | Alexander Jones proposed a change to openstack-infra/git-review: Fix parsing of SCP-style URLs, as these are valid in Git itself https://review.openstack.org/72751 | 13:16 |
| *** emagana has joined #openstack-infra | 13:17 | |
| *** thuc has quit IRC | 13:17 | |
| *** jlibosva has quit IRC | 13:17 | |
| *** dstanek has joined #openstack-infra | 13:17 | |
| *** sarob_ has joined #openstack-infra | 13:18 | |
| *** hashar has joined #openstack-infra | 13:19 | |
| *** mfer has joined #openstack-infra | 13:19 | |
| *** fifieldt has quit IRC | 13:20 | |
| *** esker has joined #openstack-infra | 13:21 | |
| *** emagana has quit IRC | 13:21 | |
| *** julim has joined #openstack-infra | 13:22 | |
| *** sarob_ has quit IRC | 13:24 | |
| sdague | maurosr: yeh, or build another handler in front of it. At some point we're going to have to figure out how to handle swift back ends as well | 13:27 |
| *** alff has quit IRC | 13:29 | |
| *** noorul has left #openstack-infra | 13:32 | |
| *** wchrisj has joined #openstack-infra | 13:34 | |
| *** dkliban has joined #openstack-infra | 13:35 | |
| *** mwagner_lap has joined #openstack-infra | 13:36 | |
| sdague | fungi: you up yet? | 13:36 |
| sdague | I was curious about new libvirt in the nodepool builds | 13:37 |
| fungi | sdague: yeah | 13:37 |
| fungi | sdague: curious about it in what way? | 13:37 |
| *** VINOD__ has joined #openstack-infra | 13:37 | |
| sdague | did it land | 13:37 |
| sdague | http://status.openstack.org/elastic-recheck/index.html libvirt is still our top bug right now | 13:37 |
| fungi | oh. um, did libvirt land in what way? | 13:37 |
| fungi | like did the new version wind up on the ubuntu mirrors? | 13:38 |
| fungi | or what? | 13:38 |
| sdague | yeh, I though they had published it out to precise updates | 13:38 |
| fungi | i think jamespage was planning to get the precise packages updated in universe... checking to see if it's new | 13:39 |
| *** blamar has joined #openstack-infra | 13:39 | |
| *** wchrisj has quit IRC | 13:39 | |
| sdague | I guess not - http://packages.ubuntu.com/precise-updates/libvirt-bin | 13:39 |
| *** yaguang has joined #openstack-infra | 13:40 | |
| fungi | libvirt_0.9.8-2ubuntu17.17.debian.tar.gz30-Jan-2014 20:35 142K | 13:42 |
| fungi | http://archive.ubuntu.com/ubuntu/pool/main/libv/libvirt/ | 13:42 |
| fungi | nothing newer in precise or precise-updates since that source package | 13:42 |
| *** VINOD__ has quit IRC | 13:43 | |
| *** weshay has joined #openstack-infra | 13:43 | |
| *** jgrimm has quit IRC | 13:45 | |
| pleia2 | fungi: this review needs a nudge https://review.openstack.org/#/c/74196/ (approved, but got stuck) | 13:45 |
| fungi | pleia2: i'll make sure it's got snow chains on the tires this time | 13:46 |
| pleia2 | thanks :) | 13:46 |
| *** jpeeler has joined #openstack-infra | 13:47 | |
| *** jpeeler has quit IRC | 13:47 | |
| *** jpeeler has joined #openstack-infra | 13:47 | |
| AJaeger | fungi, good morning. the new jobs like openstack-manuals-manuals-propose-translation-update do not work. Do you have any idea why those are not executed at all ? | 13:49 |
| openstackgerrit | Lukas Bednar proposed a change to openstack-infra/jenkins-job-builder: scms: MultiSCM is generated for scms hidden in macro https://review.openstack.org/69827 | 13:50 |
| *** jnoller has joined #openstack-infra | 13:50 | |
| fungi | AJaeger: i started to look at them over the weekend... the jobs are definitely in jenkins and listed as being in the periodic pipeline in zuul's debug log from the last time it reloaded, but it doesn't seem to have tried to trigger them. i was going to pick jeblair's brain on that once he wakes up, in case he has suggestions on where else i should look | 13:52 |
| *** jeckersb is now known as jeckersb_gone | 13:52 | |
| AJaeger | fungi, thanks for seeing my note on the weekend! | 13:52 |
| fungi | i saw it running other periodic jobs, i think, but i'll reconfirm that | 13:52 |
| openstackgerrit | Ivan Melnikov proposed a change to openstack-infra/config: Adding pypy jobs for taskflow https://review.openstack.org/80973 | 13:53 |
| AJaeger | fungi, ok, let's wait for jeblair to show up. Thanks for having this on your radar! | 13:53 |
| *** amotoki has quit IRC | 13:53 | |
| *** dcramer_ has joined #openstack-infra | 13:54 | |
| *** dkranz has joined #openstack-infra | 13:54 | |
| AJaeger | fungi, check https://jenkins.openstack.org/job/propose-requirements-updates/ - isn't that periodic as well? | 13:54 |
| AJaeger | Last run: 4 days ago | 13:54 |
| *** sc68cal_ is now known as sc68cal | 13:54 | |
| fungi | AJaeger: no, that runs in the post pipeline when a change merges to the openstack/requirements repository | 13:55 |
| AJaeger | fungi, ok | 13:55 |
| *** saschpe has joined #openstack-infra | 13:56 | |
| AJaeger | This one run 7 hours ago as it should - and is periodic https://jenkins.openstack.org/job/ceilometer-propose-translation-update/ | 13:56 |
| *** jeckersb_gone is now known as jeckersb | 13:57 | |
| *** thuc_ has quit IRC | 13:58 | |
| *** bknudson has joined #openstack-infra | 13:58 | |
| *** thuc has joined #openstack-infra | 13:58 | |
| fungi | hashar: this seems to be caused by the pbr patch to jjb... http://paste.openstack.org/show/73660/ any idea what the solution is? | 13:59 |
| *** saschpe has quit IRC | 14:00 | |
| hashar | fungi: hello ! | 14:00 |
| *** wchrisj has joined #openstack-infra | 14:01 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Drive puppet from the master over ssh https://review.openstack.org/80976 | 14:01 |
| hashar | fungi: yeah I think I noticed a follow up patch, basically setup.cfg is missing a [file] section | 14:01 |
| mordred | fungi: ^^ I had a bad idea | 14:01 |
| fungi | mordred: i saw your idea, still trying to decice how i feel about it | 14:01 |
| fungi | decide | 14:01 |
| hashar | fungi: https://review.openstack.org/#/c/80724/ Install the jenkins_jobs package on setup | 14:02 |
| *** thuc has quit IRC | 14:03 | |
| *** jcoufal has quit IRC | 14:03 | |
| fungi | hashar: yep, thanks! just found it myself | 14:03 |
| mordred | fungi: I figured code would be a good way to discuss it | 14:03 |
| fungi | i'll see if that fixes it for us | 14:03 |
| fungi | mordred: agreed | 14:03 |
| hashar | fungi: I am not sure why one need the [files]Â section though | 14:04 |
| sdague | mordred: have you considered running puppet agent on cron? | 14:04 |
| hashar | fungi: nor whether it is used by pbr or setuptools :] | 14:04 |
| *** jcoufal has joined #openstack-infra | 14:04 | |
| fungi | hashar: it does indeed allow me to run jenkins-jobs update on our jenkins masters again | 14:05 |
| openstackgerrit | A change was merged to openstack-infra/gitdm: Add myself to the email and launchpad maps https://review.openstack.org/74196 | 14:05 |
| hashar | fungi: so I guess we can vote that change :° | 14:05 |
| hashar | fungi: would you mind voting/commenting on https://review.openstack.org/#/c/80724/ please? | 14:06 |
| fungi | hashar: yes, i'm +2'ing but finding you a good answer on the purpose of the packages option in the files section of setup.cfg | 14:06 |
| mordred | sdague: yes. it doesn't help the problem we're talking about here | 14:06 |
| *** rwsu has joined #openstack-infra | 14:07 | |
| mordred | sdague: I'm looking at this as a way to potentially run some of the puppet runs in a sequence | 14:07 |
| sdague | mordred: well, it helps the hang issue that you comment on | 14:07 |
| sdague | sure | 14:07 |
| fungi | sdague: the problem statement is order-dependent puppet updates between separate hosts | 14:07 |
| sdague | cool, is there a good example of one of those? | 14:08 |
| fungi | (run when foo changes, first update host bar, then when that completes update host baz) | 14:08 |
| sdague | sure, but is there something specific? | 14:08 |
| mordred | sdague: yes | 14:09 |
| *** markmcclain has joined #openstack-infra | 14:09 | |
| sdague | because the theory on puppet is idempotency, so I wonder if there is a good test you con run on host baz to know if bar is correct | 14:09 |
| fungi | sdague: yes, when review.projects.yaml is modified, first update puppet on the cgit servers so the create-cgitrepos exec creates empty repositories for any new projects, then once that's done update puppet on the gerrit server so that new projects are created and mirror replication is initiated to the git servers | 14:09 |
| *** esker has quit IRC | 14:09 | |
| mordred | sdague: what fungi said | 14:09 |
| mordred | this is actually where puppet and chef are both shit | 14:10 |
| fungi | sdague: basically, we need to create empty git repositories on multiple hosts, then once those exist (and only once they exist) trigger git replication to them from another host | 14:10 |
| mordred | because attempts to do cross-machine deps, while possible, quickly lead you into the land of insanity | 14:10 |
| sdague | yeh, they definitely don't touch the orchestration side | 14:10 |
| *** jasondotstar has joined #openstack-infra | 14:10 | |
| mordred | I was looking at doing this with salt | 14:10 |
| *** jraim has quit IRC | 14:11 | |
| *** thuc has joined #openstack-infra | 14:11 | |
| *** zul has quit IRC | 14:11 | |
| *** unicell has quit IRC | 14:11 | |
| *** simonmcc_ has quit IRC | 14:11 | |
| fungi | mordred: did you find a way to go about it without reactors (or did you find a good reactor example which touches depending on one thing completing successfully on multiple systems before doing another thing?) | 14:11 |
| *** jraim has joined #openstack-infra | 14:11 | |
| openstackgerrit | Fei Long Wang proposed a change to openstack-infra/config: Add docs gate for Marconi https://review.openstack.org/80479 | 14:12 |
| sdague | fungi: what if the replication process tries a sample clone on all the targets first to make sure they exist? | 14:12 |
| sdague | which I get is punting on the orchestration problem | 14:12 |
| *** simonmcc_ has joined #openstack-infra | 14:12 | |
| fungi | sdague: that's an option, or so is having teh project creation script ssh to each git server and run create-cgitrepos there before proceeding on the gerrit project creation piece | 14:12 |
| fungi | sdague: though the latter would need to update puppet on them | 14:13 |
| mordred | fungi: I didn't - and all the salt people I talk to never use reactors, so they don't know how to help me :) | 14:13 |
| *** zul has joined #openstack-infra | 14:13 | |
| fungi | sdague: also if we go with the wait option, it's basically rechecking for 10-15 minutes before the git servers possibly are all caught up | 14:13 |
| fungi | sdague: which significantly increases the runtime of the project creation exec | 14:14 |
| sdague | fungi: sure, but it's a computer, doing things repetatively is easy for it | 14:14 |
| fungi | all things to take into consideration | 14:14 |
| mordred | just - I mean, look at the patch - it's so simple - the other things seem much more complex to deal with | 14:15 |
| mordred | we could also do a variation on it that instead of ssh puppet apply did salt $f puppet_apply | 14:15 |
| *** Ryan_Lane has joined #openstack-infra | 14:16 | |
| miarmak | hi all | 14:18 |
| *** sarob_ has joined #openstack-infra | 14:18 | |
| sdague | mordred: sure, though I wonder what happens when it grows past this one problem and you end up with cycles in the dependencies | 14:19 |
| pleia2 | fungi, mordred - when you have a chance, I could use some help moving this one forward (clarkb and jeblair reviewed): https://review.openstack.org/#/c/69510/ | 14:19 |
| mordred | sdague: sure. well, maybe by that point we'll have heat up and going :) | 14:19 |
| sdague | heh | 14:19 |
| mordred | pleia2: looking | 14:19 |
| miarmak | I need a piece of advice. Can anybody help please? | 14:20 |
| sdague | speaking of, is there a heat for dummies guide anywhere. Because I'd actually like to try to build the HA case as a tempest test, and I'm finding heat documentation very lacking | 14:20 |
| pleia2 | miarmak: as away :) | 14:20 |
| pleia2 | ask too | 14:20 |
| openstackgerrit | Noorul Islam K M proposed a change to openstack-infra/config: Make gate-solum-requirements non-voting https://review.openstack.org/80980 | 14:20 |
| *** pblaho has joined #openstack-infra | 14:21 | |
| sdague | fungi: something probably easy to take a look at - https://review.openstack.org/#/q/status:open+project:openstack-infra/os-loganalyze+branch:master+topic:onclick_inline,n,z I was trying to get highlighting workign this weekend, self landed a couple of patches which almost do it, and here are the remaining fixes | 14:22 |
| miarmak | pleia2: okey) the question is about the virtualenv version) will it be updated to the newer version in the in the near future? Because it is 1.10 using now and we have some problems with it... | 14:22 |
| *** emagana has joined #openstack-infra | 14:23 | |
| *** sarob_ has quit IRC | 14:23 | |
| *** thuc has quit IRC | 14:24 | |
| *** thuc has joined #openstack-infra | 14:25 | |
| *** flaper87 has quit IRC | 14:26 | |
| fungi | miarmak: the virtualenv version where? | 14:26 |
| fungi | miarmak: i assume you mean on our testing systems? | 14:27 |
| miarmak | fungi: here: https://github.com/openstack-infra/config/blob/master/modules/openstack_project/manifests/base.pp#L54 | 14:27 |
| *** jlibosva1 is now known as jlibosva | 14:28 | |
| *** rcleere has quit IRC | 14:28 | |
| *** thuc has quit IRC | 14:29 | |
| jamespage | fungi, sdague: see hallyns last comment on https://bugs.launchpad.net/nova/+bug/1254872 | 14:30 |
| jamespage | he's uploaded it but in order for it to be accepted into proposed and start the SRU process the bug needs a basic test case | 14:30 |
| *** flaper87|afk has joined #openstack-infra | 14:31 | |
| *** flaper87|afk is now known as flaper87 | 14:32 | |
| *** flaper87 has quit IRC | 14:32 | |
| *** flaper87 has joined #openstack-infra | 14:32 | |
| *** julim_ has joined #openstack-infra | 14:33 | |
| fungi | miarmak: yes, we pinned that because of https://github.com/pypa/virtualenv/issues/521 which should now be fixed in 1.11.1 but at this point we're waiting for https://bitbucket.org/hpk42/tox/issue/150/posargs-configerror to be solved first because in most places we're actually running virtualenv under tox so we need a usable (for us) tox which bundles the newer virtualenv version anyway | 14:33 |
| *** caleb_ has joined #openstack-infra | 14:34 | |
| *** julim has quit IRC | 14:36 | |
| fungi | mordred: distutils documentation is teg failz. i know what files.packages is supposed to be used for in setup.cfg, but i can't for the life of me find a mention in any official documentation backing up my understanding | 14:36 |
| yolanda | hi jeblair, i was thinking in taking that bug | 14:37 |
| yolanda | https://bugs.launchpad.net/openstack-ci/+bug/1274731 | 14:37 |
| fungi | is there some secret place i should be looking? i can find it documented in distutils2 but that's not exactly canonical | 14:37 |
| yolanda | i saw you commented that a bit, can you explain me more about it? | 14:37 |
| miarmak | fungi: so you are planning to update virtualenv to 1.11.1 (or later) after tox posargs bugfix? | 14:40 |
| *** maxbit has joined #openstack-infra | 14:43 | |
| fungi | miarmak: we're planning to revert https://review.openstack.org/64709 at that point, along with https://review.openstack.org/69872 | 14:43 |
| sdague | jamespage: who are you expecting to write the test case? | 14:44 |
| *** alff has joined #openstack-infra | 14:44 | |
| *** zns has joined #openstack-infra | 14:46 | |
| miarmak | fungi: okey, clear. Thanks! And do you have some time expectations? When it can be reverted? (approximately) | 14:48 |
| fungi | miarmak: it's mostly in the hands of the tox developers now to either decide they're okay with clarkb's proposed fix or come up with an alternate solution (which may involve changes to all of our projects before we can make use of it) | 14:48 |
| fungi | miarmak: so really, not sure what the timeline is. i try not to place bets on such things | 14:49 |
| sdague | fungi: https://review.openstack.org/#/c/80957 and https://review.openstack.org/#/c/80959/ would be nice to +A if they do what I think they do | 14:50 |
| *** alff has quit IRC | 14:50 | |
| *** emagana has quit IRC | 14:50 | |
| fungi | sdague: i'm having trouble parsing the commit message on 80957 | 14:51 |
| *** rcleere has joined #openstack-infra | 14:51 | |
| openstackgerrit | Brad P. Crochet proposed a change to openstack-infra/jenkins-job-builder: Added support for Exclusion plugin https://review.openstack.org/77940 | 14:52 |
| fungi | sdague: the groups parameter for a project in review.projects.yaml is (as far as i'm aware) only used to decide which bug to update if someone adds a bug header in the commit message of a review | 14:52 |
| fungi | er, rather, limits the scope of bugs updated to match the specified lp project name, rather than a project named the same as teh gerrit project the review changes | 14:53 |
| sdague | fungi: so how do we get nova-specs changes to update nova blueprints? | 14:53 |
| sdague | I thought when I traced the code, that would allow that | 14:54 |
| miarmak | fungi: ok, thanks) actually, we have a problems with setuptools. One of our dependencies needs setuptools 1.1.6 or later. But in virtualenv 1.10.1 its version is 0.9.8... And our solution is updating virtualenv version. Have you faced such problem? | 14:54 |
| *** rcarrillocruz has joined #openstack-infra | 14:54 | |
| *** che-arne has joined #openstack-infra | 14:54 | |
| fungi | sdague: oh, actually it does look like update_blueprint.py uses that too http://git.openstack.org/cgit/openstack-infra/jeepyb/tree/jeepyb/cmd/update_blueprint.py | 14:54 |
| miarmak | fungi: maybe it has another solution? | 14:54 |
| sdague | fungi: yeh, if it's not intended, that would be good to know | 14:55 |
| *** skraynev is now known as skraynev_afk | 14:55 | |
| fungi | sdague: since we don't particularly use blueprints for infra work i keep forgetting what that hook does | 14:55 |
| *** thuc has joined #openstack-infra | 14:55 | |
| sdague | heh | 14:56 |
| fungi | miarmak: i don't think we've dealt with that particular problem, though i'm also led to believe that explicit dependencies on setuptools break a lot of things and are generally not recommended | 14:56 |
| *** atiwari has joined #openstack-infra | 14:56 | |
| *** thuc has quit IRC | 14:58 | |
| *** thuc has joined #openstack-infra | 14:58 | |
| *** thuc_ has joined #openstack-infra | 14:59 | |
| *** saschpe_ has joined #openstack-infra | 15:00 | |
| *** pcrews has joined #openstack-infra | 15:00 | |
| openstackgerrit | Martin Konrad proposed a change to openstack-infra/jenkins-job-builder: SCM module: Add support for multiple Git remotes. https://review.openstack.org/80084 | 15:00 |
| *** thedodd has joined #openstack-infra | 15:01 | |
| miarmak | fungi: but we don't have a choice( we have one dependency and in its turn it have another dependency) and only that dependency depends on setuptools =) | 15:02 |
| openstackgerrit | A change was merged to openstack-infra/config: set the group on nova-specs to nova https://review.openstack.org/80957 | 15:02 |
| openstackgerrit | A change was merged to openstack-infra/config: add qa-specs to tempest group https://review.openstack.org/80959 | 15:03 |
| *** thuc has quit IRC | 15:03 | |
| *** saju_m has quit IRC | 15:03 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Install the jenkins_jobs package on setup https://review.openstack.org/80724 | 15:05 |
| fungi | miarmak: well, there's always a choice. we often work with the developers of our dependencies to find/implement alternative solutions to problems we face, help them better understand python packaging convention and so on | 15:05 |
| fungi | miarmak: is it possible there's an earlier version of that package which doesn't depend on newer setuptools? if so, you could just limit the maximum version of it in your requirements | 15:07 |
| fungi | as a short-term workaround | 15:07 |
| *** saschpe_ has quit IRC | 15:11 | |
| jeblair | fungi, mordred: so one of the advantages of salt is "salt mumble git*.openstack.org" catches all the git hosts. i suppose we can template the run_remote_puppet script so that whenever we add a replication target we update that script | 15:12 |
| *** ominakov has joined #openstack-infra | 15:13 | |
| *** vkozhukalov_ has quit IRC | 15:14 | |
| *** andreaf has joined #openstack-infra | 15:14 | |
| *** yassine has quit IRC | 15:14 | |
| miarmak | fungi: yeah) we have already done it) but we are looking for better solution | 15:16 |
| *** unicell has joined #openstack-infra | 15:17 | |
| *** sarob_ has joined #openstack-infra | 15:18 | |
| miarmak | fungi: and about work with the developers of our dependencies - it is great idea, but we are young project and do not have such practice | 15:19 |
| miarmak | fungi: maybe will discuss about it, thanks for advice | 15:19 |
| *** saju_m has joined #openstack-infra | 15:20 | |
| *** markmcclain has quit IRC | 15:21 | |
| devananda | g'morning, all | 15:22 |
| zul | heylo | 15:24 |
| *** sarob_ has quit IRC | 15:24 | |
| mordred | jeblair: yah. I actually think we could easily move run_remote_puppet to run salt instead of ssh | 15:25 |
| openstackgerrit | Chad Lung proposed a change to openstack-infra/config: Removing non-voting for gate-barbican-devstack-dsvm https://review.openstack.org/80686 | 15:25 |
| mordred | fungi: you will be quite sad if you try to look for docs in distutils for stuff we do in pbr | 15:25 |
| fungi | mordred: oh, is setup.cfg parsing of files.packages done directly in pbr? | 15:26 |
| mordred | yes | 15:26 |
| mordred | fungi: what's the problem you're hitting? | 15:26 |
| fungi | ahh! that 'splains why i couldn't find it mentioned in any part of stdlib packaging docs | 15:26 |
| fungi | so borrowed the syntax from distutils2 then? | 15:27 |
| *** jgrimm has joined #openstack-infra | 15:28 | |
| devananda | anyone have a minute to review two small patches for ironic testing? https://review.openstack.org/#/c/80652/ and https://review.openstack.org/#/c/80653/ | 15:28 |
| miarmak | fungi: so thanks for answering my question. I am grateful for your help! | 15:28 |
| *** david-lyle has joined #openstack-infra | 15:28 | |
| fungi | miarmak: you' | 15:28 |
| fungi | re welcome | 15:28 |
| *** wenlock has joined #openstack-infra | 15:28 | |
| *** yassine has joined #openstack-infra | 15:29 | |
| *** shakamunyi has quit IRC | 15:31 | |
| *** zehicle_at_dell has joined #openstack-infra | 15:31 | |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Openstackid track site version https://review.openstack.org/80996 | 15:32 |
| *** flaper87 is now known as flaper87|afk | 15:34 | |
| *** unicell has quit IRC | 15:37 | |
| *** emagana has joined #openstack-infra | 15:38 | |
| dkranz | fungi: This morning, Firefox started crashing when I click on a link from email or xchat but only links to review.openstack.org. Has any one else seen anything like this? Did something change? | 15:39 |
| fungi | dkranz: nothing i'm aware of. maybe firefox or one of your plugins updated? | 15:39 |
| *** mrodden has quit IRC | 15:40 | |
| dkranz | fungi: ok. I don't know. I'll try re-installing. Really weird cause any other sites seem to have no problem. | 15:40 |
| fungi | dkranz: well, gerrit's webui is pretty javascript-heavy, so it doesn't surprise me that it might interact negatively with some plugins | 15:41 |
| dkranz | fungi: Makes sense. | 15:41 |
| *** CaptTofu has joined #openstack-infra | 15:41 | |
| *** andreaf has quit IRC | 15:42 | |
| *** alff has joined #openstack-infra | 15:44 | |
| *** annegentle has joined #openstack-infra | 15:46 | |
| *** unicell has joined #openstack-infra | 15:47 | |
| mordred | fungi: yeah - and then distutils2 got killeded | 15:47 |
| *** miqui has joined #openstack-infra | 15:47 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Use unbound https://review.openstack.org/80092 | 15:49 |
| *** saschpe_ has joined #openstack-infra | 15:49 | |
| pleia2 | oh, bugdaystats may not be on anyone's radar yet, could use some review: https://review.openstack.org/#/c/79800/ | 15:50 |
| openstackgerrit | Ruslan Kamaldinov proposed a change to openstack-infra/storyboard: Fix DB migrations in unit tests https://review.openstack.org/80836 | 15:50 |
| pleia2 | once this is done I'll add -infra to this so we'll have pretty graphs for our bug days :) | 15:50 |
| jeblair | pleia2: yep, it wasn't in my list; added, thx. | 15:52 |
| *** marun has joined #openstack-infra | 15:53 | |
| *** mrodden has joined #openstack-infra | 15:53 | |
| mordred | jeblair: I makred this WIP because it's untested - but I'd love feedback on the idea/approach if you have a sec | 15:54 |
| mordred | https://review.openstack.org/#/c/80893/ | 15:54 |
| *** andreaf has joined #openstack-infra | 15:54 | |
| *** markmcclain has joined #openstack-infra | 15:57 | |
| jeblair | mordred: is the current thing bad or suboptimal in some way? | 15:57 |
| fungi | jamielennox|away: your ci jobs are running correctly now. we had a packaging-related error introduced in jenkins-job-builder shortly before it would have created your jobs in our jenkins masters, so it was failing to do that until just a little while ago when i approved the fix | 15:57 |
| *** nprivalova has joined #openstack-infra | 15:58 | |
| jeblair | mordred: commented in review | 15:59 |
| *** pbelanyi has quit IRC | 15:59 | |
| *** reed has joined #openstack-infra | 15:59 | |
| *** rossella_s has quit IRC | 16:01 | |
| *** rossella_s has joined #openstack-infra | 16:01 | |
| zaro | morning | 16:02 |
| *** IvanBerezovskiy has left #openstack-infra | 16:02 | |
| *** saschpe_ has quit IRC | 16:02 | |
| *** yassine has quit IRC | 16:03 | |
| *** saschpe has joined #openstack-infra | 16:04 | |
| *** saschpe has quit IRC | 16:06 | |
| *** primeministerp has quit IRC | 16:07 | |
| openstackgerrit | Martin Konrad proposed a change to openstack-infra/jenkins-job-builder: SCM module: Add support for multiple Git remotes. https://review.openstack.org/80084 | 16:07 |
| gondoi | I had a comment on https://review.openstack.org/79058 mentioning the use of #openstack-satori instead of #satori | 16:10 |
| gondoi | is there a guideline on irc room names I could review? | 16:10 |
| gondoi | also, we are currently only openstack related | 16:11 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Fix DB migrations in unit tests https://review.openstack.org/80836 | 16:12 |
| *** gyee has joined #openstack-infra | 16:13 | |
| *** wenlock has quit IRC | 16:15 | |
| *** yassine has joined #openstack-infra | 16:15 | |
| *** saju_m has quit IRC | 16:15 | |
| *** CaptTofu has quit IRC | 16:16 | |
| *** yaguang has quit IRC | 16:16 | |
| *** wenlock has joined #openstack-infra | 16:17 | |
| *** CaptTofu has joined #openstack-infra | 16:17 | |
| *** ihrachys has joined #openstack-infra | 16:17 | |
| *** jnoller has quit IRC | 16:18 | |
| *** Hefeweizen has quit IRC | 16:18 | |
| ihrachys | hi all. doesn't it look like a circular dependency in gate that blocks the patch to pass checks: https://review.openstack.org/#/c/77982/ ? It fails when touching trove/stable branch in gate even though it's designed to disable this exact check | 16:18 |
| *** sarob_ has joined #openstack-infra | 16:18 | |
| *** jasondotstar has quit IRC | 16:19 | |
| fungi | ihrachys: jogo was trying to solve it similarly with 80687 | 16:20 |
| fungi | ihrachys: but we discovered the script which runs that job needs fixing in pbr first | 16:21 |
| *** CaptTofu has quit IRC | 16:22 | |
| fungi | ihrachys: https://review.openstack.org/80723 points out the underlying problem | 16:22 |
| *** sarob_ has quit IRC | 16:23 | |
| *** afazekas has quit IRC | 16:23 | |
| jogo | fungi: those issues go even deeper | 16:23 |
| ihrachys | fungi: tnx for info! | 16:24 |
| jogo | fungi: https://review.openstack.org/#/c/79756/1 trove isn't part of integrated reqs | 16:24 |
| *** jcoufal has quit IRC | 16:24 | |
| *** caleb_ has quit IRC | 16:24 | |
| clarkb | morning | 16:24 |
| *** SumitNaiksatam has joined #openstack-infra | 16:25 | |
| *** e0ne has quit IRC | 16:25 | |
| jogo | clarkb: morning | 16:25 |
| *** e0ne has joined #openstack-infra | 16:25 | |
| *** SumitNaiksatam has quit IRC | 16:26 | |
| fungi | jogo: argh! and it looks like yesterday hpcloud started failing pypy jobs the same way rax was wrt setuptools being upgraded by pip at the same time as installing other packages | 16:29 |
| clarkb | fungi: oh no! | 16:29 |
| fungi | dstufft: ^ did you have any luck figuring out what's going on in pip there | 16:29 |
| fungi | ? | 16:29 |
| jogo | fungi: so many bugs so little time | 16:29 |
| openstackgerrit | A change was merged to openstack-infra/gerritlib: Expose the gerrit watcher as a thread with defined transitions https://review.openstack.org/70564 | 16:29 |
| jogo | fungi: is there an e-r query for it? | 16:30 |
| *** jasondotstar has joined #openstack-infra | 16:30 | |
| *** SumitNaiksatam has joined #openstack-infra | 16:30 | |
| fungi | jogo: not sure. infra-facing bug is https://launchpad.net/bugs/1290562 | 16:33 |
| *** ildikov_ has quit IRC | 16:33 | |
| clarkb | no response on DNS ticket yet | 16:34 |
| fungi | however, since i still have an hpcloud py3k-precise node held from before it started failing there, maybe i have a better chance of spotting how it differs from a current hpcloud node | 16:34 |
| * fungi starts digging | 16:34 | |
| *** hashar has quit IRC | 16:36 | |
| *** eharney has quit IRC | 16:36 | |
| jogo | fungi: how does message:" UserWarning\: Unknown distribution option: 'zip_safe'" AND tags:"console" sound | 16:37 |
| *** jswarren has joined #openstack-infra | 16:37 | |
| *** alff has quit IRC | 16:38 | |
| fungi | jogo: " | 16:38 |
| fungi | er | 16:38 |
| fungi | jogo: "error: option --single-version-externally-managed not recognized" may be more accurate for this | 16:39 |
| fungi | (matching on the error, not the warning) | 16:39 |
| *** sabari2 has joined #openstack-infra | 16:40 | |
| jogo | fungi: sounds good | 16:40 |
| clarkb | is this failing 100% of the jobs? | 16:40 |
| clarkb | for pypy | 16:40 |
| openstackgerrit | Ricardo Carrillo Cruz proposed a change to openstack-infra/gerritbot: Add support for notification of releases on IRC https://review.openstack.org/79205 | 16:43 |
| fungi | clarkb: at this point, i believe so | 16:43 |
| fungi | we stopped running them on rax because it was failing there and not on hpcloud | 16:43 |
| clarkb | ya | 16:43 |
| clarkb | and now hpcloud fails too | 16:44 |
| clarkb | fungi: is that setuptools complaining the option doesn't exist? | 16:45 |
| *** yassine has quit IRC | 16:45 | |
| fungi | clarkb: i believe setuptools is not installed at that stage | 16:45 |
| clarkb | it appearts to be running mccabe's setup.py | 16:45 |
| clarkb | and is passing --single-version-externall-managed to that setup.py | 16:46 |
| fungi | clarkb: if you upgrade setuptools and mccabe independently (in either order) it's fine | 16:46 |
| fungi | clarkb: but if you upgrade them in the same pip command line, boom | 16:46 |
| clarkb | huh | 16:46 |
| fungi | mccabe just being one example. you can recreate it with other packages using the same option | 16:46 |
| *** sarob_ has joined #openstack-infra | 16:47 | |
| clarkb | is this happening because distutils is being used as the fallback because setuptools is removed? | 16:47 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1290562 https://review.openstack.org/81015 | 16:47 |
| clarkb | and it is removed as part of the upgrade? | 16:47 |
| *** rcarrillocruz1 has joined #openstack-infra | 16:48 | |
| dstufft | ugh | 16:48 |
| dstufft | sorry | 16:48 |
| dstufft | I forgot :( | 16:48 |
| dstufft | basically yes setuptools is getting removed as part of the process | 16:49 |
| clarkb | and not being replaced immeditely | 16:49 |
| dstufft | but it should be getting immediately reinstalled | 16:49 |
| dstufft | something is making it so it's not | 16:49 |
| *** amcrn has joined #openstack-infra | 16:49 | |
| dstufft | the actual error you're seeing comes from passing a setup tools flag to a distutils setup.py | 16:49 |
| dstufft | becuse of the fallback | 16:49 |
| fungi | clarkb: i've confirmed that my older held hpcloud instance is still working and the one i just held is failing, so ought to be easy to play spot the difference there | 16:50 |
| clarkb | woot I groked the problem | 16:50 |
| *** rcarrillocruz has quit IRC | 16:50 | |
| dstufft | you'll also see it show up as not being able to import setuptools, if the setup.py doesn't have a distutils fallback | 16:50 |
| dstufft | what version of pip is involved here again? | 16:51 |
| jogo | sdague: https://review.openstack.org/#/c/80027/ is ready | 16:52 |
| *** sarob_ has quit IRC | 16:52 | |
| jogo | (make partial-ncpu work) | 16:52 |
| clarkb | dstufft: I think pip 1.4.1 but fungi can confirm as he has nodes at hand | 16:52 |
| jogo | dansmith: ^^ | 16:52 |
| clarkb | jogo: oh cool it passes on that change \o/ | 16:53 |
| rcarrillocruz1 | hey guys, any core reviewer willing to review https://review.openstack.org/#/c/79205/ ? I rebased my branch to master, as last gate check run failed .... | 16:53 |
| jogo | clarkb: yup, keystone folks were kind enough to revert the breaking stable patches over the weekend | 16:53 |
| fungi | clarkb: dstufft: inside the virtualenv it's pip 1.5.4 http://paste.openstack.org/show/73060/ | 16:54 |
| dstufft | let me see if Marcus is around | 16:54 |
| *** jlibosva has quit IRC | 16:54 | |
| dstufft | he's delt with these errors before and put the fixes we currently have into place | 16:54 |
| jogo | dtroyer: https://review.openstack.org/#/c/80027/ -- is now working for partial-ncpu | 16:54 |
| clarkb | fungi: isn't that a newer version of virtualenv than we expect? | 16:55 |
| clarkb | rcarrillocruz1: looking | 16:55 |
| *** dstanek has quit IRC | 16:55 | |
| *** hogepodge_ has joined #openstack-infra | 16:55 | |
| fungi | clarkb: yep, but that doesn't seem to explain away the issue | 16:56 |
| jeblair | clarkb, fungi: i'm thinking of approving a change that touches db configuration on slaves; do you anticipate any other need to build new images soon? | 16:56 |
| openstackgerrit | A change was merged to openstack-infra/config: Trim down gantt check/gate jobs https://review.openstack.org/71648 | 16:56 |
| fungi | jeblair: not unless i end up needing to rebuild py3k images to solve this pypy job issue | 16:56 |
| clarkb | jeblair: I don't have anything | 16:57 |
| *** hogepodge has quit IRC | 16:57 | |
| clarkb | jeblair: what db change? | 16:57 |
| jeblair | clarkb: the baremetal postgres one | 16:57 |
| fungi | clarkb: so, pip freeze on both nodes is identical. dpkg -l reports four differences... sudo, tzdata, tzdata-java and udisks packages got autoupgraded | 16:57 |
| *** yassine has joined #openstack-infra | 16:58 | |
| clarkb | fungi: pip freeze won't show you setuptools and pip differences | 16:58 |
| clarkb | pip list may work better | 16:58 |
| *** sarob_ has joined #openstack-infra | 16:59 | |
| fungi | oh, hey, setuptools 3.3 released the same day too... every time we see one provider or the other start breaking this way, it coincides with a new setuptools release the same day... i wonder whether rax nodes have spontaneously started working now that hpcloud's have begun failing ;) | 16:59 |
| dstufft | maybe I'll just make pip 1.6 default to installing "setuptools damnit" | 17:00 |
| dstufft | as part of executing setup.py :V | 17:00 |
| dstufft | ~isolation~ | 17:00 |
| rcarrillocruz1 | hmm, I miss the code reviews that people put previously, what happened? is it maybe because of the rebase the changeset has a dependency , making previous code review comments disappear? | 17:00 |
| clarkb | rcarrillocruz1: they should still be there on the old patchsets | 17:01 |
| *** JoshNang_ is now known as JoshNang | 17:01 | |
| *** saschpe has joined #openstack-infra | 17:01 | |
| rcarrillocruz1 | hmm, but they don't appear the +1s from the reviewer mini-spreadsheet | 17:02 |
| rcarrillocruz1 | :( | 17:02 |
| clarkb | rcarrillocruz1: you lose the old votes when non trivial new patcsets are pushed | 17:02 |
| *** hogepodge has joined #openstack-infra | 17:03 | |
| fungi | where non-trivial means that git patch-id on both patchsets is identical | 17:03 |
| fungi | er, is not identical | 17:04 |
| rcarrillocruz1 | ah gotcha, ok, thx for the review, will look into it | 17:04 |
| *** markmcclain has quit IRC | 17:04 | |
| clarkb | fungi: does the old hpcloud node have setuptools 3.2? | 17:05 |
| clarkb | fungi: and new node 3.3? | 17:05 |
| *** ildikov_ has joined #openstack-infra | 17:06 | |
| fungi | clarkb: both report 3.3 inside the virtualenv | 17:06 |
| clarkb | fungi: in the log of a fiaeld run it says found setuptools 2.2 | 17:07 |
| clarkb | is 3.3 being installed after the mccabe failure? (/me wonders where 3.3 comes from if the job finds 2.2) | 17:08 |
| fungi | clarkb: yeah, i believe that's what's getting initially installed by virtualenv | 17:08 |
| *** saschpe has quit IRC | 17:08 | |
| jeblair | fungi: what's the status of the install modules without dependencies part of the puppetboard change? is that tested and working now? | 17:08 |
| fungi | jeblair: no idea. let me see if i said i tested/was testing it | 17:09 |
| fungi | clarkb: virtualenv bundles setuptools and installs it into the venv it creates | 17:09 |
| *** harlowja_away is now known as harlowja | 17:09 | |
| fungi | clarkb: then pip install comes along hitting some package declaring a dependency on setuptools and upgrades it to the most recent version | 17:09 |
| jeblair | fungi: ok; i thought i vaguely remember conversation about that; nbd if you don't know off the top of your head. | 17:10 |
| clarkb | fungi: right, but it seems to bail after uninstalling 2.2 and before updating to a newer version | 17:10 |
| fungi | jeblair: i think i tested the previous bits but only reviewed that one | 17:10 |
| *** sarob__ has joined #openstack-infra | 17:10 | |
| fungi | clarkb: well, it seems to bail on using the new setuptools in evaluating the other package being installed, but looks like it does at least think setuptools is upgraded after that | 17:11 |
| clarkb | ah gotcha | 17:11 |
| *** jgrimm has quit IRC | 17:11 | |
| fungi | at least based on inspection of the venv contents afterward | 17:11 |
| *** rhsu has joined #openstack-infra | 17:11 | |
| clarkb | fungi: /me has a theory for a workaround | 17:12 |
| fungi | clarkb: i'm all ears | 17:12 |
| clarkb | one sec I will push a patch | 17:12 |
| clarkb | It is a bit of a hack though :/ | 17:13 |
| *** e0ne has quit IRC | 17:13 | |
| *** sarob_ has quit IRC | 17:14 | |
| *** morganfainberg_Z is now known as morganfainberg | 17:14 | |
| *** afazekas has joined #openstack-infra | 17:14 | |
| *** jgrimm has joined #openstack-infra | 17:14 | |
| * fungi expects "upgrade setuptools in the venv prior to installing any other packages | 17:14 | |
| fungi | since that's the ugly hack workaround i was thinking of proposing | 17:15 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Keep py3.X compatibility for urllib https://review.openstack.org/76122 | 17:15 |
| openstackgerrit | A change was merged to openstack-infra/config: Set up opportunistic bare-metal postgres db https://review.openstack.org/73461 | 17:15 |
| *** mwagner_lap has quit IRC | 17:16 | |
| clarkb | fungi: https://review.openstack.org/#/c/81023/ | 17:16 |
| fungi | clarkb: that's definitely an alternative approach, until we hit a dependency declaring it needs a newer setuptools than virtualenv is providing | 17:17 |
| *** markmcclain has joined #openstack-infra | 17:17 | |
| clarkb | fungi: ya | 17:17 |
| clarkb | fungi: we can restrict that with an explicit pypy testenv that sets a different install_command too | 17:18 |
| sdague | jogo: +A | 17:18 |
| clarkb | if we want to keep the existing behavior on py26 and py27 | 17:18 |
| clarkb | oh and py33 | 17:18 |
| fungi | clarkb: i am less and less sure that this issue is unrelated to pypy... so far we've *only* seen it crop up with pypy virtualenvs. the bug may still be in pip, but something about pypy seems to be tickling it | 17:19 |
| fungi | but it's *possible* we could see it with other interpreters | 17:20 |
| fungi | we just haven't so far | 17:20 |
| openstackgerrit | A change was merged to openstack-infra/config: Add oslo.test integration test https://review.openstack.org/76381 | 17:20 |
| *** hogepodge has quit IRC | 17:21 | |
| *** CaptTofu has joined #openstack-infra | 17:21 | |
| clarkb | fungi: https://jenkins07.openstack.org/job/gate-python-novaclient-pypy/15/ it does seem to be a viable workaround | 17:22 |
| morganfainberg | clarkb, since you seem to be a person involved with tox (and therefore probably more familiar with mercurial than i am), is there a way to update the commit in a pull request on bitbucket? or do i just abandon the PR and make a new one? | 17:23 |
| *** hogepodge has joined #openstack-infra | 17:23 | |
| clarkb | morganfainberg: you can add a second commit to the pr | 17:23 |
| morganfainberg | clarkb, i know, totally random | 17:23 |
| morganfainberg | clarkb, hm. ok so it works like github's PRs then | 17:23 |
| clarkb | morganfainberg: but aiui hg doesn't let you do a proper rebase and change history on stuff other things know about | 17:24 |
| clarkb | so if you want clean history I think you may need a new PR | 17:24 |
| clarkb | morganfainberg: I pushed a second commit to my PR and they seemed happy with that | 17:24 |
| morganfainberg | clarkb, i'll just do that. i don't like messy history (have i mentioned i _really_ like the gerrit workflow over PRs) | 17:24 |
| dstufft | I think mercurial lets you do that, you just need extensions :V | 17:24 |
| dstufft | also git supremacy | 17:24 |
| morganfainberg | dstufft, and probably not with hosted. | 17:24 |
| morganfainberg | dstufft, aka bitbucket | 17:25 |
| clarkb | dstufft: I tried tem once (the rebase extension) it didn't do what I expected | 17:25 |
| clarkb | it was weirdness | 17:25 |
| *** sweston has joined #openstack-infra | 17:25 | |
| morganfainberg | clarkb, we should take over the world (of opensource) and convince everyone to use gerrit workflows. | 17:25 |
| dstufft | you can force push on Hg | 17:25 |
| dstufft | afaik | 17:25 |
| dstufft | I doubt bitbucket prevents you | 17:25 |
| morganfainberg | dstufft, it actually rejected the force push. | 17:26 |
| dstufft | lol wat | 17:26 |
| dstufft | bitbucket u cray | 17:26 |
| morganfainberg | dstufft, but i am a hg noob | 17:26 |
| morganfainberg | dstufft, i'll just add another commit that fixes all the stuff to my previous one | 17:26 |
| morganfainberg | dstufft, clarkb, thanks | 17:26 |
| dstufft | clarkb: fungi I talked to Marcus, he's going to take a look later at the setuptools issue, he knows this code better than I do | 17:26 |
| *** rcarrillocruz has joined #openstack-infra | 17:26 | |
| *** rcarrillocruz1 has quit IRC | 17:27 | |
| fungi | dstufft: thanks! | 17:27 |
| dstufft | [13:26:14] <qwcode> dstufft, can they actually confirm after that setuptools is missing? | 17:28 |
| dstufft | [13:26:33] <qwcode> dstufft, i.e. that it's not a path/env issue | 17:28 |
| dstufft | fungi: ^ :] | 17:28 |
| *** sarob__ has quit IRC | 17:29 | |
| fungi | dstufft: inspecting the virtualenv shows that the new setuptools installed after it throws that error | 17:29 |
| sdague | fungi / clarkb: https://review.openstack.org/#/c/80691/ - adding horizon error log will let us narrow a much of the horizon failures in ER | 17:29 |
| *** sarob_ has joined #openstack-infra | 17:29 | |
| sdague | s/much/bunch/ | 17:29 |
| dstufft | fungi: if you do foo/bin/pip list it shows the new setuptools? | 17:29 |
| *** yassine has quit IRC | 17:30 | |
| fungi | dstufft: foo/bin/pypy -c 'import setuptools;print(setuptools.__version__)' says 3.3 but list says setuptools (3.1) | 17:30 |
| fungi | very weird | 17:31 |
| fungi | this was after "Found existing installation: setuptools 3.1" | 17:31 |
| *** annegentle has quit IRC | 17:31 | |
| fungi | or on the even older system, s/3.1/2.2/ | 17:31 |
| clarkb | sdague: https://review.openstack.org/#/c/80691/2/modules/openstack_project/templates/logstash/indexer.conf.erb does the rfc822 timestamp match one of the formats on line 104 of that file? | 17:32 |
| *** zhiyan is now known as zhiyan_ | 17:32 | |
| *** annegentle has joined #openstack-infra | 17:32 | |
| * clarkb is looking | 17:32 | |
| *** sarob_ has quit IRC | 17:33 | |
| fungi | dstufft: so i guess the situation after that failure is that latest setuptools is available on import in the venv, but pip list doesn't register that the upgrade of it completed | 17:33 |
| dstufft | what does import setuptools; print(setuptools.__file__) say? | 17:34 |
| fungi | dstufft: /home/fungi/foo/site-packages/setuptools/__init__.py | 17:35 |
| dstufft | ok | 17:35 |
| clarkb | sdague: you will need to update line 104. I am writing a comment with the new format | 17:35 |
| sdague | clarkb: cool | 17:36 |
| clarkb | sdague: done | 17:39 |
| sdague | clarkb: thanks | 17:40 |
| *** CaptTofu has quit IRC | 17:42 | |
| sdague | clarkb: what is the language of this file anyway? | 17:43 |
| sdague | if I wrap the array is it going to be an issue? | 17:44 |
| clarkb | sdague: logstash config http://logstash.net/docs/1.3.3/filters/date is the date filter used on line 104 | 17:44 |
| clarkb | sdague: yeah that I am not sure | 17:44 |
| *** eharney has joined #openstack-infra | 17:44 | |
| sdague | because the date formats definitely make more sense when you can see them on top of each other | 17:44 |
| clarkb | it isn't terrible to test it if you grab logstash and fire it up locally | 17:44 |
| sdague | is there a cheat sheet for that? | 17:45 |
| *** vkozhukalov_ has joined #openstack-infra | 17:45 | |
| clarkb | not really. I always refer to those docs and often read the ruby source | 17:45 |
| clarkb | oh you mean spinnin gup logastsh | 17:45 |
| clarkb | yup | 17:45 |
| devananda | clarkb: hi! have a minute to look at two small patches for ironic? would like to see if this fixes our CI job | 17:46 |
| *** zns has quit IRC | 17:46 | |
| clarkb | sdague: http://logstash.net/docs/1.3.3/tutorials/getting-started-simple then basically erplace our input and output with stdin input and stdout output | 17:46 |
| clarkb | sdague: since you don't need the fancy es and tcp stuff when doing simple local tests | 17:46 |
| *** khyati has joined #openstack-infra | 17:47 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add horizon_error to the indexed logs https://review.openstack.org/80691 | 17:47 |
| openstackgerrit | A change was merged to openstack-infra/config: Repack git repositories daily https://review.openstack.org/77919 | 17:47 |
| sdague | clarkb: so this isn't tested that the wrapped lines work, but that should be the changes | 17:48 |
| clarkb | devananda: sure, do you have links? | 17:48 |
| clarkb | sdague: ok, I can give it a whirl maybe | 17:48 |
| sdague | clarkb: does that go in order? | 17:49 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add horizon_error to the indexed logs https://review.openstack.org/80691 | 17:49 |
| sdague | it just occurred to me that I needed to change the order on optional Z | 17:49 |
| devananda | clarkb: https://review.openstack.org/#/c/80652/2 and https://review.openstack.org/#/c/80653/ | 17:49 |
| devananda | clarkb: thanks! | 17:49 |
| clarkb | sdague: their docs don't say but typically yes they iterate over arrays in order | 17:49 |
| *** pblaho has quit IRC | 17:50 | |
| *** Ajaeger1 has joined #openstack-infra | 17:50 | |
| fungi | dstufft: another point of interest, 'foo/bin/pip install -U setuptools==3.1 mccabe' succeeds but ==3.2 or 3.3 breaks as described | 17:51 |
| *** atiwari has quit IRC | 17:51 | |
| dstufft | ugh | 17:51 |
| dstufft | this migt be a seupools bug | 17:51 |
| fungi | and the rabbit hole goes deeper still... | 17:52 |
| *** ominakov has quit IRC | 17:52 | |
| clarkb | be careful you don't get stuck down there | 17:52 |
| *** wenlock has quit IRC | 17:52 | |
| *** ominakov has joined #openstack-infra | 17:52 | |
| *** wenlock has joined #openstack-infra | 17:53 | |
| *** dizquierdo has quit IRC | 17:54 | |
| clarkb | devananda: any idea why all of those havana tests failed on the d-g change? | 17:54 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Document the Zuul triggers https://review.openstack.org/77843 | 17:55 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Add configurable footer-message reports https://review.openstack.org/77740 | 17:55 |
| openstackgerrit | A change was merged to openstack-infra/config: offset nodepool update by 12 hrs https://review.openstack.org/78939 | 17:56 |
| *** ominakov has quit IRC | 17:57 | |
| fungi | dstufft: also, downgrading pip to 1.3.1 in the virtualenv before installing/upgrading setuptools and other packages seems to cause this problem not to happen | 17:57 |
| *** markmcclain has quit IRC | 17:57 | |
| sdague | clarkb: havana was bonkered on requirements and keystoneclient friday | 17:58 |
| *** isviridov has quit IRC | 17:58 | |
| clarkb | sdague: thanks | 17:59 |
| *** markmcclain has joined #openstack-infra | 17:59 | |
| *** tstevenson has joined #openstack-infra | 17:59 | |
| *** fbo is now known as fbo_away | 17:59 | |
| clarkb | sdague: can you look at https://review.openstack.org/#/c/80653/ and see if those failures look familiar? | 17:59 |
| sdague | will do | 17:59 |
| openstackgerrit | A change was merged to openstack-infra/config: Make storyboard run over ssl https://review.openstack.org/78747 | 18:00 |
| * clarkb installs java | 18:00 | |
| clarkb | sdague: first step looks good. logstash didn't throw any errors when I started it up with your config changes | 18:01 |
| sdague | cool | 18:02 |
| *** VijayTripathi has joined #openstack-infra | 18:02 | |
| sdague | I think it's a ton more readable with the fold, so easier to be right in the reviews | 18:02 |
| clarkb | now I need to get my geard running and throw jobs at it, but first breakfast | 18:03 |
| *** salv-orlando has joined #openstack-infra | 18:03 | |
| Ajaeger1 | fungi, did you have a chance to talk with jeblair ? | 18:03 |
| sdague | devananda: why is ironic getting added to enabled services? | 18:04 |
| *** zns has joined #openstack-infra | 18:04 | |
| devananda | sdague: lemme check | 18:04 |
| fungi | Ajaeger1: oh, i hadn't done so yet... jeblair: not sure if you saw in scrollback, but there were several new periodic translation jobs for openstack manuals which zuul showed (in its debug log) were in the periodic pipeline and which the jenkins masters had as available but which were not getting triggered when the other periodic jobs ran... wondering if you had suggestions for where else i should | 18:05 |
| fungi | look? | 18:05 |
| Ajaeger1 | jeblair: this is one of the jobs: https://jenkins.openstack.org/view/Openstack-manuals/job/operations-guide-manuals-propose-translation-update/ | 18:05 |
| Ajaeger1 | jeblair: jobs adding them: https://review.openstack.org/#/c/80515/ | 18:05 |
| Ajaeger1 | fungi, thanks | 18:05 |
| *** rcarrillocruz1 has joined #openstack-infra | 18:05 | |
| jeblair | Ajaeger1, fungi: my guess is "branch: master" may have something to do with it | 18:06 |
| fungi | jeblair: ah! good eye | 18:07 |
| fungi | i keep forgetting to check for that on periodic jobs | 18:07 |
| jeblair | Ajaeger1: i think that would cause the branch to only run on an event triggered on the "master" branch, but timer triggers don't have branches (at least, not yet anyway; they could, and perhaps should) | 18:07 |
| *** rcarrillocruz has quit IRC | 18:07 | |
| devananda | sdague: so i'm not sure -- there are checks for is_service_enabled ir-api and ir-cond, but I dont see any explicitly for "ironic" | 18:07 |
| jeblair | "cause the job to run" i mean | 18:07 |
| devananda | sdague: is there a reason /not/ to add "ironic" to enabled services? | 18:08 |
| Ajaeger1 | jeblair: ok, will sent a patch. Thanks! | 18:08 |
| fungi | Ajaeger1: so, yes, need to remove the jobs matches for those which specify branch master | 18:08 |
| *** mrodden has quit IRC | 18:08 | |
| *** ildikov_ has quit IRC | 18:08 | |
| sdague | devananda: because it's not supposed to be there | 18:09 |
| *** ildikov_ has joined #openstack-infra | 18:09 | |
| sdague | ENABLED_SERVICES should be daemon names | 18:10 |
| fungi | Ajaeger1: so just need to remove the one for ^.*manuals-propose-translation-update$ (which is in periodic) but not ^.*manuals-upstream-translation-update$ (which is in post) i think | 18:10 |
| sdague | or screen session names, basically | 18:10 |
| JayF | jeblair: Hey, I'm the guy working on ironic-python-agent merge req. When I tried to do 'git review -t new-project', my changes are being rejected because there are no changes. Any advice? | 18:10 |
| sdague | the "meta" names, like nova are deprecated, and we're purging them all out | 18:10 |
| devananda | sdague: ah, gotcha | 18:10 |
| sdague | so we shouldn't be adding more in | 18:10 |
| sdague | if you remove that add, I'm +2 | 18:10 |
| devananda | sdague: I saw several still there so didn't realize that. will remove it now | 18:11 |
| jeblair | JayF: if you don't have anything else to change, make a trivial change to the commit message (add a space or period or something) | 18:11 |
| Ajaeger1 | fungi: yes, I agree | 18:11 |
| sdague | yeh, it's still in flux, I was trying to handle the heat case the other day | 18:11 |
| JayF | jeblair: okay, thanks. Was going to do that but wanted to ensure there wasn't a better trick | 18:11 |
| openstackgerrit | Jason-oldos proposed a change to openstack-infra/config: Add openstack/ironic-python-agent https://review.openstack.org/79088 | 18:11 |
| *** rcarrillocruz has joined #openstack-infra | 18:12 | |
| fungi | JayF: the better trick is, i believe, that once we upgrade to latest gerrit you | 18:12 |
| fungi | 'll be able to edit the topic without a new patchset | 18:12 |
| JayF | Cool, thanks | 18:12 |
| *** rcarrillocruz1 has quit IRC | 18:12 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/devstack-gate: Configure localrc for virtual-ironic tests https://review.openstack.org/80653 | 18:12 |
| fungi | at least i believe i heard that was in the list of new features | 18:12 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Fix periodic manuals-propose-translation-update jobs https://review.openstack.org/81037 | 18:13 |
| Ajaeger1 | fungi, jeblair: Here's the patch^ | 18:13 |
| *** afazekas has quit IRC | 18:13 | |
| jroll | is there a way to install *only* openstack.common.log to a project? I don't see an oslo.log or anything :/ | 18:13 |
| devananda | sdague: thanks for pointing that out. after some more grep'ing, it looks like nothing in devstack is keyed off that anyway | 18:13 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Make IRC bot list which failures were seen in which job. https://review.openstack.org/78790 | 18:14 |
| dhellmann | jroll: we can help with that in #openstack-oslo | 18:15 |
| fungi | jroll: looks like it's still incubating... http://git.openstack.org/cgit/openstack/oslo-incubator/tree/openstack/common/log.py ? | 18:15 |
| zaro | sdague: you wanted the build timeout plugin to report timeout, right? | 18:15 |
| fungi | jroll: but yeah, what dhellmann said | 18:15 |
| jroll | thanks | 18:15 |
| sdague | zaro: to pass it down, yes | 18:15 |
| *** rcarrillocruz1 has joined #openstack-infra | 18:15 | |
| sdague | devananda: yeh, I remember the ironic devstack patch actually did is_ironic_enabled correctly (new style), so I didn't think it would be need | 18:16 |
| *** mrodden has joined #openstack-infra | 18:16 | |
| *** rcarrillocruz has quit IRC | 18:16 | |
| zaro | sdague: PR https://github.com/jenkinsci/build-timeout-plugin/pull/22 has merged. new release at end of month. | 18:17 |
| sdague | zaro: awesome! | 18:17 |
| sdague | nice work | 18:17 |
| zaro | thx. | 18:17 |
| sdague | once that gets into the infrastructure, we should be able to simplify a bunch of stuff | 18:17 |
| sdague | thanks for tackling it | 18:18 |
| zaro | np. | 18:18 |
| adalbas | sdague, hi! is is somewhere documented how to install os-loganalyze (deps, configuration)? I'm looking to install it in a fresh server, with no other ci stuff. | 18:21 |
| *** mkoderer has quit IRC | 18:21 | |
| sdague | adalbas: mostly in the config repository | 18:23 |
| fungi | adalbas: looks like http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/manifests/static.pp#n111 says we just do 'python setup.py install' | 18:23 |
| sdague | there is a sample apache.conf in the os-loganalyze project as well | 18:23 |
| dhellmann | mordred: have a sec to discuss a pbr release? | 18:23 |
| sdague | it's honestly a pretty simple wsgi filter | 18:23 |
| fungi | adalbas: and the apache vhost config we use with it is http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/logs.vhost.erb | 18:23 |
| adalbas | fungi, sdague tks. i was looking if there are any specific requirements, other then the ones in requirements.tx | 18:24 |
| fungi | adalbas: on line #64 there, "WSGIScriptAlias /htmlify /usr/local/lib/python2.7/dist-packages/os_loganalyze/wsgi.py" | 18:24 |
| sdague | adalbas: no, I think we basically are only using python built ins | 18:24 |
| fungi | and the rewrites immediately above it | 18:24 |
| adalbas | tks!! | 18:24 |
| sdague | fungi: the docs link seems to be broken on os-loganalyze at the moment | 18:25 |
| sdague | * Documentation: http://docs.openstack.org/developer/os_loganalyze | 18:25 |
| sdague | any idea what's not connected there | 18:25 |
| fungi | sdague: was it ever? was that some boilerplate link which got copied, but maybe we have no documentation job publishing to it? | 18:26 |
| sdague | could be | 18:26 |
| sdague | honestly, I don't know | 18:26 |
| * fungi looks | 18:26 | |
| adalbas | i could not find it here http://ci.openstack.org/ | 18:27 |
| sdague | adalbas: yeh, this used to be a funny little script inside config, and we broke it out mostly to add better testing | 18:27 |
| fungi | sdague: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n2802 no jobs besides pep8 and python27 | 18:27 |
| *** _nadya_ has joined #openstack-infra | 18:27 | |
| sdague | fungi: you have a preference about whether it's worth adding that, or just purging the reference? | 18:28 |
| fungi | sdague: no preference... looks like the content in http://git.openstack.org/cgit/openstack-infra/os-loganalyze/tree/doc/source is still just the template | 18:29 |
| *** jgallard has quit IRC | 18:29 | |
| sdague | yep | 18:29 |
| *** Hefeweizen has joined #openstack-infra | 18:30 | |
| sdague | I'm leaning on delete the reference | 18:30 |
| *** blamar has quit IRC | 18:30 | |
| Ajaeger1 | fungi: thanks for the +2 on 81037. I would appreciate if somebody could approve this today. | 18:30 |
| clarkb | Ajaeger1: looking now | 18:31 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: update README for accuracy https://review.openstack.org/81043 | 18:32 |
| clarkb | Ajaeger1: fungi actually wait | 18:32 |
| clarkb | Ajaeger1: fungi: we only want that job to run on the master branc | 18:32 |
| clarkb | so we should double check that the script does that for us | 18:32 |
| fungi | clarkb: good point | 18:32 |
| *** vhoward has left #openstack-infra | 18:33 | |
| _nadya_ | fungi, clarkb, hi! I need to test Ceilometer against Mongo in tempest. That's why I need an ability to upgrade mongo on devstack+ubuntu. Could you please advise? I've created a change request to devstack https://review.openstack.org/#/c/81001/ . Is it appropriate solution in general? | 18:33 |
| *** vhoward has joined #openstack-infra | 18:33 | |
| fungi | clarkb: the script will only be run once, but if it has a hard dependency on getting a branch passed to it from zuul or a zuul ref or something, it will not work as expected | 18:33 |
| *** vhoward has left #openstack-infra | 18:33 | |
| clarkb | _nadya_: dims is working to get UCA into the gate | 18:33 |
| clarkb | _nadya_: UCA should come with newer mongodb | 18:34 |
| clarkb | fungi: ya | 18:34 |
| Ajaeger1 | clarkb: script is modules/jenkins/files/slave_scripts/propose_translation_update_manuals.sh | 18:34 |
| fungi | _nadya_: also, the devstack maintainers mostly hang out in #openstack-qa | 18:34 |
| *** sdake_ is now known as sdake | 18:34 | |
| * fungi is not a devstack core reviewer | 18:34 | |
| Ajaeger1 | it only expects as parameter the project, no ZUUL ref | 18:35 |
| clarkb | fungi: Ajaeger1 I think we need to add gerrit git prep to the job definition in jenkins jobs | 18:35 |
| *** afazekas has joined #openstack-infra | 18:35 | |
| clarkb | which should fall back on master nicely | 18:35 |
| Ajaeger1 | clarkb: will look into it, brb. Thanks! | 18:36 |
| *** jerryz has joined #openstack-infra | 18:36 | |
| Ajaeger1 | you're right, git-prep was missing ;( | 18:37 |
| fungi | clarkb: Ajaeger1: good point, as it's written now, it's going to just use the state of the workspace on proposal.slave, which likely will contain no repository at all, right? | 18:37 |
| clarkb | fungi: riht | 18:37 |
| fungi | and then fail mightily | 18:37 |
| jogo | whats the benifit from rerunning the check queue on a -1 patch? | 18:38 |
| fungi | jogo: it might depend, for example, on finctioanlity in another project which hadn't previously merged? | 18:38 |
| fungi | functionality | 18:38 |
| Ajaeger1 | looking at the change that I did (https://review.openstack.org/#/c/80515/), I also removed the scm setting - I guess that was wrong, wasn't it? Let me fix it... | 18:38 |
| jogo | fungi: then you can just say 'recheck no bug' | 18:39 |
| Ajaeger1 | https://review.openstack.org/#/c/80515/8/modules/openstack_project/files/jenkins_job_builder/config/translation-jobs.yaml - line 72 | 18:39 |
| clarkb | jogo: -1 patch? do you mean -1 comment? | 18:39 |
| jogo | clarkb: yeah | 18:39 |
| *** changbl has quit IRC | 18:39 | |
| fungi | jogo: as a common example, change to a project's requirements file is failing the requirements enforcement job, and then suddenly starts succeeding once the global requirements change merges | 18:39 |
| jogo | example: https://review.openstack.org/#/c/77341/ | 18:39 |
| clarkb | jogo: because it will show us bitrot | 18:39 |
| clarkb | remember you can merge with a -1 | 18:39 |
| _nadya_ | fungi: ok, thanks. I just not sure in solution. Is it ok to create a job with param like mongo_deb=http://repos/mongo-latest.deb to install it on devstack. I will go to qa-channel anyway | 18:40 |
| *** nicedice has joined #openstack-infra | 18:40 | |
| *** blamar has joined #openstack-infra | 18:41 | |
| fungi | _nadya_: we don't reuse the machines where that gets run, so it's not necessarily a security risk if it only happens for the jobs which need it... but it's more up to the devstack maintainers as to whether they're okay with that approach | 18:41 |
| jogo | clarkb: sure thats true, but it doesn't seem very useful ... especially when we regularly use up all our resources | 18:41 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Fix periodic manuals-propose-translation-update jobs https://review.openstack.org/81037 | 18:41 |
| Ajaeger1 | fungi, clarkb: How does this look like? | 18:41 |
| jogo | it just doesn't seem like a lot of bang for our buck | 18:41 |
| jogo | currently jbos are waiting about 16 minutes in check queue before being processed | 18:42 |
| jogo | and we have a light load | 18:42 |
| clarkb | jogo: but it is useful if we correctly mark a change as unmergable | 18:43 |
| rcarrillocruz1 | jeblair , clarkb: re: gerritbot irc notifications, here's my rationale for treating ref-updated event as a special case | 18:43 |
| rcarrillocruz1 | http://paste.openstack.org/show/73683/ | 18:43 |
| rcarrillocruz1 | just posted the comment in the review as well | 18:43 |
| jogo | clarkb: how? and why not just run the mergable job then? | 18:43 |
| _nadya_ | clarkb: and what is the estimate? Will it be available soon? The problem is that all ceilometer tests works only on Mongo (I know, it's mostly Ceilometer's problem, but anyway). It would be great to test on gating+Mongo. Do you think it is possible within icehouse? | 18:43 |
| *** melwitt has joined #openstack-infra | 18:43 | |
| clarkb | jogo: because it will catch the bitrot if there is bitrot | 18:44 |
| jogo | clarkb: if a patch gets a -1 review, 9 out of 10 times that means there will another revision or at least a +1 or +2 review | 18:44 |
| clarkb | _nadya_: you will need to ask dims | 18:44 |
| jogo | clarkb: sure but we don't have enough resources to check everything for bitrot all the time | 18:44 |
| _nadya_ | clarkb: ok, I will :) Thanks for your time! | 18:44 |
| *** afazekas has quit IRC | 18:44 | |
| _nadya_ | fungi: Thank you for help, will consult with devstack-guys | 18:45 |
| jogo | clarkb: also bitrot is commonly used to refer to media failure (disk failures etc) | 18:45 |
| jogo | clarkb: it just seems like that rerunning check queue won't do anything useful the vast majority of the time. and we are limited on the number of nodes we have | 18:46 |
| *** e0ne has joined #openstack-infra | 18:48 | |
| fungi | jogo: we're regularly running out of available resources, or just during feature freeze run-up and exception? | 18:48 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Fix periodic manuals-propose-translation-update jobs https://review.openstack.org/81037 | 18:48 |
| Ajaeger1 | fungi, now without scm builder. thanks | 18:49 |
| * jogo checks graphite | 18:50 | |
| *** johnthetubaguy has quit IRC | 18:51 | |
| *** rcarrillocruz has joined #openstack-infra | 18:52 | |
| *** Sukhdev has joined #openstack-infra | 18:52 | |
| jogo | sdague: well for one thing we don' | 18:53 |
| jogo | t have enough data since this change was so recent | 18:53 |
| *** ominakov has joined #openstack-infra | 18:53 | |
| *** rcarrillocruz1 has quit IRC | 18:54 | |
| jogo | http://graphite.openstack.org/render/?from=-2weeks&height=180&until=now&width=334&bgcolor=ffffff&fgcolor=000000&areaMode=stacked&target=color(alias(sumSeries(stats.gauges.nodepool.target.*.*.*.building),%20%27Building%27),%20%27ffbf52%27)&target=color(alias(sumSeries(stats.gauges.nodepool.target.*.*.*.ready),%20%27Available%27),%20%2700c868%27)&target=color(alias(sumSeries(stats.gauges.nodepool.target.*.*.*.used),%20%27In%20Use%27),%20%276464ff%27)&targe | 18:54 |
| fungi | jogo: yeah, i'd rather see what things look like through more of a dev cycle before we go fine-tuning for a base activity level which is unrealistic most of the time | 18:54 |
| jogo | fungi: do we collect how long something stays in check queue for? | 18:55 |
| jogo | fungi: if there is one thing we have learned is 'base activity level' just goes up over time | 18:56 |
| fungi | jogo: i think we collect the number of jobs waiting for workers at any point in time. you could probably extrapolate | 18:56 |
| jogo | also we are in feature freeze things should be pretty silent now | 18:56 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: add persistence of anchors https://review.openstack.org/81047 | 18:56 |
| jogo | fungi: do you know what that is called | 18:56 |
| zaro | fungi: looks like git.o.o picked up gerrit-2.8.3 release. you have time to make a branch today? | 18:56 |
| *** mrda_away is now known as mrda | 18:57 | |
| fungi | jogo: | 18:57 |
| fungi | http://graphite.openstack.org/render/?from=-24hours&height=180&until=now&width=334&bgcolor=ffffff&fgcolor=000000&target=color(alias(stats.gauges.zuul.geard.queue.running,%20%27Running%27),%20%27blue%27)&target=color(alias(stats.gauges.zuul.geard.queue.waiting,%20%27Waiting%27),%20%27red%27)&target=color(alias(stats.gauges.zuul.geard.queue.total,%20%27Total%20Jobs%27),%20%27888888%27)&target=color(ali | 18:57 |
| fungi | as(stats.gauges.zuul.geard.workers,%20%27Workers%27),%20%27green%27)&title=Zuul%20Job%20Queue&_t=0.13254144179941774 | 18:57 |
| *** gordc has joined #openstack-infra | 18:57 | |
| fungi | jogo: or the job queue graph at the bottom of the zuul status page | 18:57 |
| jeblair | jogo: resident | 18:58 |
| fungi | zaro: yes, meant to do so friday night and then got sidetracked--sorry. doing it now | 18:58 |
| jeblair | jogo: resident_time | 18:58 |
| clarkb | sdague: I am getting grokparse failures with your regex | 18:58 |
| *** ominakov has quit IRC | 18:58 | |
| clarkb | sdague: currently debugging that, but I think we are close to having this in | 18:58 |
| sdague | ok | 18:58 |
| fungi | zaro: you want to branch openstack/2.8.3 from the v2.8.3 (2014-03-14 07:44:49) tag? | 19:00 |
| *** dguerri_ has joined #openstack-infra | 19:00 | |
| sdague | jogo: what's the thing you want to change? | 19:00 |
| fungi | zaro: done. the head of that openstack/2.8.3 branch is 315ef42 | 19:01 |
| *** rpodolyaka1 has joined #openstack-infra | 19:02 | |
| sdague | jogo: I'd be +1 on not rerunning changes on a -2, but on a -1, you can still get overridden, so we should have fresh results on it. | 19:02 |
| *** sabari2 is now known as sabari | 19:02 | |
| fungi | zaro: http://git.openstack.org/cgit/openstack-infra/gerrit/log/?h=openstack/2.8.3 | 19:02 |
| sdague | the other idea would be take the "idle refresh" completely out of band | 19:03 |
| jogo | jeblair: found stats.timers.zuul.pipeline.check.resident_time.mean but not sure hwo to parse it | 19:03 |
| jogo | sdague: don't trigger the auto recheck on a -1 comment | 19:03 |
| sdague | that woul require other zuul patches, but that would probably be a good option | 19:03 |
| jogo | if someone else comes along and gives a +1 then you trigger then anyway | 19:03 |
| clarkb | sdague: it appears to be an issue with the brakcet escapes | 19:03 |
| sdague | jogo: yeh, I'm -1 on removing on -1 | 19:03 |
| clarkb | sdague: the first [] don't need them but the second does | 19:03 |
| jogo | sdague: becuase? | 19:03 |
| sdague | clarkb: interesting... | 19:03 |
| clarkb | sdague: not sure why that is | 19:03 |
| clarkb | jesusaurus: ^ do you know what is going on there? | 19:04 |
| sdague | <sdague> jogo: I'd be +1 on not rerunning changes on a -2, but on a -1, you can still get overridden, so we should have fresh results on it. | 19:04 |
| sdague | jogo: because that patch you linked right now I could +A it in | 19:04 |
| jogo | right, when someone comments on it they get new data anyway | 19:04 |
| Ajaeger1 | fungi, clarkb: I hope that https://review.openstack.org/81037 is fine now. | 19:04 |
| jogo | so yes there are edge cases for this | 19:04 |
| sdague | jogo: the edge cases where what was killing us | 19:05 |
| * jesusaurus reads a bit of scrollback | 19:05 | |
| jogo | but we still rerun everything in check before gate anyway | 19:05 |
| jogo | so I don't see how this can hurt us | 19:05 |
| clarkb | jesusaurus: [%{DATESTAMP_RFC822:logdate}]%{SPACE}\[%{LOGLEVEL:loglevel}\]%{SPACE}%{GREEDYDATA:logmessage} as a filter works according to https://grokdebug.herokuapp.com/ | 19:05 |
| jogo | us==gate | 19:05 |
| clarkb | jesusaurus: that is one pair of [] escaped and one pair not escaped | 19:05 |
| sdague | jogo: honestly, I'm not going into another one of these roundabouts today. :) I'm -1, on changing that fact, we can dive through it all again at summit. | 19:06 |
| jesusaurus | clarkb: i think you want to escape both sets | 19:06 |
| jogo | sdague: and the check queue can take hours to run sometimes. right now things are not bad at all with a 10 minute delay | 19:06 |
| jogo | so I bring this up now because this is hitting everybody in a real way | 19:06 |
| clarkb | jesusaurus: it doesn't work that way | 19:06 |
| clarkb | jesusaurus: whcih is really odd | 19:06 |
| fungi | Ajaeger1: clarkb: it made me start looking closely at the other translations proposal job, and i don't see where it's dealing with the initial git checkout either, i wonder if that one is also broken but working because there happens to have been a git clone in the workspace previously? | 19:07 |
| *** jnoller has joined #openstack-infra | 19:07 | |
| jesusaurus | otherwise they will be parsed as a set of chars | 19:07 |
| clarkb | fungi: the others have g-g-p | 19:07 |
| jesusaurus | clarkb: thats odd | 19:07 |
| jogo | today its not very bad but it can be | 19:07 |
| jogo | and has been | 19:07 |
| *** afazekas has joined #openstack-infra | 19:07 | |
| clarkb | jesusaurus: oh huh | 19:07 |
| clarkb | jesusaurus: so it "works" but doesn't actually aprse the date correctly | 19:07 |
| jesusaurus | clarkb: [abc] is parsed as (a|b|c) | 19:08 |
| sdague | jogo: there are plenty of other places to look for purging things, or get more quota. I think this is the wrong place | 19:08 |
| fungi | clarkb: '{name}-propose-translation-update' does not... http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/translation-jobs.yaml#n15 | 19:08 |
| jogo | sdague: if you have had this discussion, before is there a place you can point me to to catch up | 19:08 |
| clarkb | jesusaurus: yeah gotcha so it makes the other matches work, I tink the rfc822 thing isn't quite right | 19:08 |
| jesusaurus | so it's valid, just not what you want | 19:08 |
| fungi | clarkb: and seems to use the scm builder instead? | 19:09 |
| * jesusaurus -> lunch | 19:09 | |
| *** mgagne has joined #openstack-infra | 19:09 | |
| fungi | clarkb: is that provided by the scm jenkins plugin? | 19:09 |
| clarkb | fungi: oh huh, yup we need scm | 19:09 |
| clarkb | fungi: ya | 19:09 |
| fungi | clarkb: does ggp need tweaks to support this use case then, or should ir already be able to do what we want (and so we can switch off scm on the old job too)? | 19:10 |
| jogo | sdague: in the long run perhaps, but this delay can get pretty long on a regular basis. | 19:10 |
| clarkb | sdague: the problem with that grok is rfc822 is year before time | 19:10 |
| jogo | we broke 120 jobs in check queue last week -- during FF | 19:11 |
| clarkb | but horizon error log is time then year... | 19:11 |
| *** thuc_ has quit IRC | 19:11 | |
| *** mbacchi has quit IRC | 19:11 | |
| sdague | clarkb: oh, man, so I messed that part up | 19:11 |
| zaro | fungi: thanks. | 19:11 |
| sdague | jogo: if by "we" you mean races in openstack that aren't fixed, sure | 19:11 |
| *** thuc has joined #openstack-infra | 19:11 | |
| *** mgagne1 has quit IRC | 19:11 | |
| jogo | this is check queue not gate, what do races have to do with it? | 19:12 |
| sdague | I'm really not positive about optimizing around bugs and running tests less often because we might block code | 19:12 |
| *** afazekas has quit IRC | 19:12 | |
| sdague | ok, you mean we had > 120 jobs | 19:12 |
| sdague | I thought you mean breaking test results :) | 19:12 |
| jogo | ohh haha. | 19:12 |
| jogo | btw check this out: http://status.openstack.org/elastic-recheck/data/uncategorized.html and click 2 days | 19:13 |
| sdague | jogo: well, it was a really slow weekend :) | 19:13 |
| sdague | so that doesn't really count | 19:13 |
| *** afazekas has joined #openstack-infra | 19:14 | |
| sdague | jogo: you want more nodes back, #1 - patch zuul to be able to gate-noop without throwing away a nodepool node | 19:14 |
| jogo | so the problem as I see it is: the check queue can get backed up. The underlying issue is not enough resources for the load we are generating. | 19:14 |
| sdague | that gets us back a few hundred nodes a day | 19:14 |
| jogo | sdague: can you point me in the right direction for that? | 19:14 |
| *** caleb_ has joined #openstack-infra | 19:14 | |
| sdague | jeblair might be able to | 19:14 |
| sdague | that was a discussion last week | 19:14 |
| jogo | sdague: that sounds like a good use of my time | 19:14 |
| *** _nadya_ has quit IRC | 19:15 | |
| jogo | or anyones | 19:15 |
| jogo | jeblair: ^ | 19:15 |
| jeblair | jogo: we have not hit our quota today | 19:15 |
| jeblair | jogo: look at the graph | 19:15 |
| *** sarob_ has joined #openstack-infra | 19:15 | |
| jogo | jeblair: then why are check queue jobs queued | 19:15 |
| jogo | also I don't know what the quota | 19:15 |
| jogo | is | 19:15 |
| jeblair | jogo: i have to get lounch, back later | 19:16 |
| *** thuc has quit IRC | 19:16 | |
| jogo | jeblair: kk | 19:16 |
| jeblair | jogo: (it's the flat line at the top of the test nodes graph) | 19:16 |
| jeblair | jogo: (it's not visible currently) | 19:16 |
| jogo | jeblair: ohh | 19:16 |
| clarkb | sdague: I am still hitting it with a hammer, hope to have a proper regex shortly | 19:16 |
| jogo | sdague: in that case my mistake, there are other factors in delay right now | 19:16 |
| sdague | clarkb: cool, thank you | 19:16 |
| jogo | I only see building, ready, inuse and deleting in the graphite source for the test nodes graph | 19:18 |
| sdague | jogo: basically if you can see green in that graph, and there isn't a flat line on top, we're fine | 19:19 |
| *** weshay has quit IRC | 19:19 | |
| sdague | jogo: or figure out that with the new nova-network bit if we can go back to higher parallism | 19:20 |
| jogo | so given it lools like we do hava extra resources. I am still perplexed why jobs are in queued in check queue | 19:23 |
| jogo | and there are no zuul events to process | 19:24 |
| *** thomasbiege has joined #openstack-infra | 19:25 | |
| *** vkozhukalov_ has quit IRC | 19:25 | |
| *** mrodden has quit IRC | 19:25 | |
| *** mrodden has joined #openstack-infra | 19:26 | |
| sdague | jogo: well that would be the one to get to the bottom of | 19:26 |
| fungi | clarkb: the reason i suspect ggp may not work for this as-is... compare http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/gerrit-git-prep.sh#n31 and https://jenkins.openstack.org/job/nova-propose-translation-update/564/parameters/ | 19:27 |
| devananda | sdague, clarkb: looks like https://review.openstack.org/#/c/80653/ passed tests after my last fix | 19:27 |
| *** gokrokve has joined #openstack-infra | 19:27 | |
| sdague | devananda: +2 | 19:27 |
| jamespage | fungi, dims: libvirt 1.2.2 + selected patches is now in the icehouse cloud archive | 19:27 |
| jamespage | sdague, fungi: I think hallyn was looking for someone who could reproduce the problem with 0.9.8 to write a short test case | 19:28 |
| fungi | clarkb: how do you feel about a patch to ggp which will fall back on master when there's no zuul ref passed? we'll presumably need it in a post-jenkins world anyway, and this would let us drop the scm plugin from the other template too | 19:28 |
| fungi | jamespage: as i understand it, the bug we're encountering is known upstream, but reproducing it requires high-volume testing to get it to sometimes occur | 19:29 |
| *** changbl has joined #openstack-infra | 19:29 | |
| jamespage | fungi, OK - we have some precedent for this type of issue | 19:29 |
| jamespage | fungi, I'll catch hallyn tomorrow to discuss | 19:29 |
| devananda | sdague: thanks! | 19:29 |
| clarkb | jamespage: run tempest against libvirt | 19:30 |
| sdague | jamespage: right, there was a very specific scenario layed out in the upstream commit | 19:30 |
| fungi | (a bunch) | 19:30 |
| sdague | yeh, a bunch | 19:30 |
| *** jasondotstar has quit IRC | 19:30 | |
| sdague | jamespage: the reason I asked, was mostly to make sure it was clear that I don't think anyone here is going to build an ubuntu specific test case that's outside of the openstack flow | 19:30 |
| sdague | and I wanted to make sure there wasn't a deadlock | 19:31 |
| *** yassine has joined #openstack-infra | 19:31 | |
| *** alff has joined #openstack-infra | 19:32 | |
| *** thuc has joined #openstack-infra | 19:33 | |
| *** ominakov has joined #openstack-infra | 19:34 | |
| gordc | hi folks, i recently added openstack-server-publish-jobs to pyCADF library (https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/layout.yaml#L985) | 19:35 |
| gordc | i merged in the first batch of doc files yesterday (https://review.openstack.org/#/c/65139/) but they didn't seem to get pushed to docs.openstack.org/developer/pycadf... (i did see a NOT REGISTERED comment in Zuul when i glanced at the job)... was there a step i missed to enable doc generation? | 19:36 |
| *** _nadya_ has joined #openstack-infra | 19:36 | |
| *** zns has quit IRC | 19:37 | |
| fungi | gordc: jenkins job builder was broken for a few days in master, and since we cd from master on our jenkins servers no jobs added in the past few days worked until earlier today when the fix merged | 19:37 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: add horizon_error to the indexed logs https://review.openstack.org/80691 | 19:37 |
| clarkb | sdague: ^ | 19:37 |
| fungi | gordc: any which run now should in theory work | 19:37 |
| clarkb | fungi: that was the pbr installation thing? | 19:38 |
| *** comstud has quit IRC | 19:38 | |
| fungi | clarkb: yes | 19:38 |
| clarkb | and with that time for lunch | 19:38 |
| gordc | fungi: ah cool. let me give it another try again. thanks for the info. | 19:38 |
| sdague | clarkb: is %{SPACE} optional? | 19:38 |
| *** comstud has joined #openstack-infra | 19:38 | |
| clarkb | sdague: dependson | 19:38 |
| * fungi is in %{SPACE} | 19:38 | |
| clarkb | sdague: %{SPACE} will optionally match any number of whitespace | 19:39 |
| clarkb | so the areas without it must have a single space | 19:39 |
| sdague | but not 0? | 19:39 |
| clarkb | it will match 0 | 19:39 |
| sdague | ok | 19:39 |
| clarkb | which is why i put it before TZ because the TZ may not be present | 19:39 |
| sdague | yep | 19:39 |
| sdague | ok, good | 19:39 |
| sdague | +1 | 19:39 |
| sdague | thanks for fully debugging that | 19:39 |
| clarkb | sdague: I tested against http://logs.openstack.org/77/80177/4/check/check-tempest-dsvm-full/e840a87/logs/horizon_error.txt.gz fwiw | 19:40 |
| sdague | great | 19:40 |
| dguerri_ | hello everyone | 19:40 |
| *** thuc_ has joined #openstack-infra | 19:40 | |
| *** thuc has quit IRC | 19:40 | |
| clarkb | dguerri_: hello | 19:40 |
| dguerri_ | I would like to propose a tool we are currently using at HP to track upstream openstack project that now has been released as an opensource sw (apache lic) | 19:41 |
| dguerri_ | it's called git-upstream (so it can be used as a git subcommand) | 19:41 |
| clarkb | dguerri_: and by track you mean it keeps a local repository in sync with an upstream? | 19:42 |
| clarkb | jeepyb can do this for you today | 19:42 |
| dguerri_ | clarkb: it allows to simplify some steps needed to rebase local carried patches onto upstream commits (or tags) | 19:42 |
| *** ominakov has quit IRC | 19:43 | |
| dguerri_ | if you cherry-picked some commits from upstream, it will detect them (comparing gerrit Change-Id, ATM) | 19:43 |
| jogo | sdague: any thoughts on how to deal with troves depedencies? https://review.openstack.org/#/c/80690/4 | 19:44 |
| fungi | dguerri_: while the tool sounds neat and useful, i worry that it may encourage workflows openstack is trying hard to undo (companies carrying their own patches on top of openstack rather than getting features integrated upstream) | 19:45 |
| dguerri_ | basically we are using it with 2 jenkins jobs: one periodically mirrors an upstream project and the other one rebase local carried patches on top of the mirrored copy of the repo | 19:45 |
| *** jasondotstar has joined #openstack-infra | 19:45 | |
| jogo | sdague: full list of missing deps https://review.openstack.org/#/c/79756/1 | 19:45 |
| clarkb | fungi: not only that but itmay encourage forks | 19:46 |
| *** ominakov has joined #openstack-infra | 19:46 | |
| clarkb | which as an upstream doesn't seem like an appropriate thing to do | 19:46 |
| dguerri_ | fungi: yep, I understand. But it can be used to maintain patches which have been proposed upstream but that have not been yet merged/included | 19:46 |
| sdague | jogo: that was already proposed as individual reviews from the trove team | 19:47 |
| jogo | sdague: ahh | 19:47 |
| *** mkoderer has joined #openstack-infra | 19:47 | |
| jogo | and they all hit the pypy failure | 19:47 |
| sdague | yeh, everything is hitting that | 19:47 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: refresh devstack log https://review.openstack.org/81057 | 19:47 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: update README for accuracy https://review.openstack.org/81043 | 19:47 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: provide a -e run target to run server locally https://review.openstack.org/80952 | 19:47 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: add persistence of anchors https://review.openstack.org/81047 | 19:47 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/os-loganalyze: highlighting the way you expect https://review.openstack.org/80966 | 19:47 |
| *** hashar has joined #openstack-infra | 19:48 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/devstack-gate: Don't enable savanna service for sahara https://review.openstack.org/81058 | 19:49 |
| *** _nadya_ has quit IRC | 19:49 | |
| jogo | sdague: do we need to get that sorted out by icehouse? | 19:50 |
| jogo | or not | 19:50 |
| fungi | i'm a little worried that it may be necessary to either pin all our infra to setuptools<3.2 (if that's even doable), stop upgrading packages in virtualenvs when pip installs, or cease testing pypy until it's solved | 19:50 |
| dguerri_ | fungi: clarkb: let me know if you are interested | 19:51 |
| *** dstanek has joined #openstack-infra | 19:52 | |
| clarkb | dguerri_: I don't think it is something upstream should carry. I agree it would have its uses, but openstack shouldn't have an official fork openstack project | 19:52 |
| *** caleb_ has quit IRC | 19:52 | |
| fungi | or really anything which makes it easier for providers not to bother getting their patches incorporated upstream, further driving incompatibilities between different openstack services throughout the internet (something as a project we've been trying very hard to reign in recently) | 19:53 |
| *** alff has quit IRC | 19:54 | |
| lifeless | clarkb: so uhm | 19:54 |
| sdague | fungi: why does this only nuke pypy? | 19:54 |
| lifeless | clarkb: why not? We have an explicit expectation that CD deployers- HP, Rackspace, etc - are going to carry minor divergence regularly. | 19:55 |
| clarkb | lifeless: do we? | 19:55 |
| lifeless | clarkb: why shouldn't supporting that be something the community maintains? | 19:55 |
| clarkb | lifeless: because of fungi's statement | 19:55 |
| clarkb | both of them | 19:55 |
| sdague | fungi: and, honestly, do we care? because I don't think pypy being busted in a funny way should block the rest of openstack | 19:55 |
| clarkb | sdague: beacuse peopel put effort into making pypy work. maybe they can help fix this | 19:56 |
| lifeless | clarkb: you can't both not gate on providers exact configurations and expect that they never need to do their own QA polish | 19:56 |
| dims | jamespage, excellent, will take a look | 19:56 |
| sdague | clarkb: so who are those pypy people? because I think blocking all of requirements changes for icehouse forever is probably not a good trade off | 19:57 |
| lifeless | clarkb: we've chosen to gate on only what is open source and reproducable by individual devs; the corollary is that CD deployers can't presume trunk works, and thus they have a floating set of 'current fixes' on top of trunk. | 19:57 |
| lifeless | clarkb: making it harder to do that well increases the cost of CD, which means they have less bandwidth for working upstream. | 19:57 |
| fungi | lifeless: it seems like something that interested members of the community could collectively maintain if they want to use it. i just don't think it sounds suitable as an openstack-infra project | 19:57 |
| lifeless | clarkb: its basically the exact opposite of what we want to achieve. | 19:58 |
| lifeless | fungi: I don't think its openstack-infra either | 19:58 |
| lifeless | fungi: but clarkb was saying 'upstream doesn't want this' | 19:58 |
| lifeless | fungi: which is a different statement | 19:58 |
| fungi | lifeless: "upstream" in this channel implying openstack-infra was completely the basis for my evaluation | 19:59 |
| sdague | lifeless: so you are making an explicit assumption that if CD is easier people will contribute more to upstream, and if CD is harder, they will contribute less | 19:59 |
| sdague | which I'm not sure I buy | 19:59 |
| *** weshay has joined #openstack-infra | 19:59 | |
| lifeless | sdague: I'm relaying what folk at both HP and rackspace have told me about where they get to spend time | 19:59 |
| clarkb | sdague: Alex_gaynor | 19:59 |
| *** changbl has quit IRC | 20:00 | |
| lifeless | sdague: I agree that it is certainly not as simple as as a linear see-saw :) | 20:00 |
| sdague | lifeless: and those people said if they weren't doing that they'd be contributing to upstream QA more (for instance?) | 20:00 |
| dguerri_ | I think using that tool is a way better than maintaining patches "manually". Its focus is on maintaining patches on top of upstream commits which means you are willing to be updated (which is good) and as you will run into some rebase problem if you are just patching existing code | 20:00 |
| sdague | because I think one of the major reasons for carrying these out of tree patches is requiring undefined behavior in upstream | 20:01 |
| clarkb | I agree using a tool is a great idea and there is utility there. | 20:01 |
| dguerri_ | probably if you are maintaining some new cool feature you don't want to release, you will not release it even if rebasing it is difficult... | 20:01 |
| lifeless | sdague: so for HP, upstream QA and upstream tracking+feature work are different teams, so I'm quite certain lower friction around CD wouldn't get more tempest work in and of itself | 20:01 |
| *** dprince has quit IRC | 20:01 | |
| lifeless | sdague: actually, we can probably get some stats on exactly what and why for HP cloud | 20:02 |
| sdague | lifeless: that would be valuable regardless | 20:02 |
| lifeless | sdague: so if we're up for a little latency, we can have facts rather than speculation for this discussion :) | 20:02 |
| clarkb | but I also agree with fungi that it sets a bad stage for upstream cooperation | 20:02 |
| sdague | lifeless: yeh, I don't think there is an urgency on a decision here, so facts are good | 20:02 |
| *** mbacchi has joined #openstack-infra | 20:03 | |
| clarkb | supporting not releasing a cool new feature isnt something I want to encourage | 20:03 |
| lifeless | dguerri_: reckon we could get a pareto chart (reason for carrying a local patch vs frequency?) for the patches we've carried using this tool ? | 20:03 |
| lifeless | clarkb: *that* case is almost totally unreleated | 20:04 |
| lifeless | clarkb: HPCloud uses plugins for that | 20:04 |
| sdague | I also share concerns with making it an openstack project that basically tells everyone to expect to run with out of tree patches. Because I think there are already way too many folks doing that, and honestly we should be trying to figure out how to help people avoid that | 20:04 |
| clarkb | it was just used as an example | 20:04 |
| lifeless | clarkb: if the set of examples is 0, the argument is void | 20:04 |
| clarkb | huh | 20:04 |
| lifeless | clarkb: so, while I realise it was an example, we need real ones. | 20:04 |
| clarkb | see sb | 20:04 |
| Alex_Gaynor | clarkb: what's up? | 20:05 |
| clarkb | pypy is broken | 20:05 |
| sdague | Alex_Gaynor: pypy is 100% failing requirements changes atm | 20:05 |
| mordred | dhellmann: hey | 20:05 |
| Alex_Gaynor | link? | 20:05 |
| sdague | Alex_Gaynor: https://review.openstack.org/#/c/80850/ | 20:05 |
| dhellmann | mordred: hi, I'd like to do a pbr release to fix an issue with "python setup.py --name" that breaks the oslo config generator in some situations | 20:05 |
| sdague | one example | 20:05 |
| dhellmann | mordred: wanted to check with you as the primary maintainer, first | 20:06 |
| sdague | but if yuo look at the requirements repo in general, the wall of -1s are all from the pypy job | 20:06 |
| dguerri_ | sdague: I disagree (sorry) you are not encouraging ppl to run their own patches. You are encouraging them to stay updated... | 20:06 |
| mordred | dhellmann: I'm probably pretty fine with that - while I've got you - did you happen to read scrollback about versioning or read lifeless' email? | 20:06 |
| Alex_Gaynor | sdague: this looks like the bug someone was looking into the other day where doing things in a certain order made pip barf if you tried to pip install setuptools <something else> ? And it can be solved with pip install setuptools && pip install something else | 20:06 |
| dguerri_ | they will run their patches if they want to.. with or without a tool | 20:06 |
| dhellmann | mordred: I saw the email, but not the scrollback. I haven't absorbed the email yet. | 20:07 |
| mordred | Alex_Gaynor, sdague: pip installing setuptools <something else> is almost always fraught with peril in our experience aroud here | 20:07 |
| dguerri_ | it is all about how you are presenting it | 20:07 |
| dguerri_ | :) | 20:07 |
| sdague | dguerri_: sure, I don't expect people not to have local patches | 20:07 |
| *** atiwari has joined #openstack-infra | 20:07 | |
| sdague | I'm a realist | 20:08 |
| *** mwagner_lap has joined #openstack-infra | 20:08 | |
| mordred | dhellmann: the scrollback _MAY_ give better or worse framing - it's a long conversation between lifeless dstufft and I | 20:08 |
| sdague | however I also do get concerned with making it dumb simple, because people won't realize that git rebase can get things terribly wrong | 20:08 |
| mordred | dhellmann: it's possible that seeing the arguments for and/or against would be helpful - but I'd specifically like you and ttx to provide feedback | 20:09 |
| sdague | and at a certain point, it's like building your own lightsaber | 20:09 |
| * mordred wants a lightsaber | 20:09 | |
| dguerri_ | they will realise that earlier if you are fostering upstream tracking (staying up to date) | 20:09 |
| dguerri_ | hehe | 20:09 |
| dhellmann | mordred: I'll take a look, but it would be really nice if one of you took the time to summarize on the ML. Is that what lifeless' etherpad is? | 20:09 |
| sdague | dguerri_: maybe... I'm not sure you've seen the same crazy that gets submitted to our projects :) | 20:10 |
| lifeless | dhellmann: my mail the ML was the summary :) | 20:10 |
| mordred | dguerri_: I actually have some patches on git-upstraem I need to give you | 20:10 |
| dguerri_ | mordred: cool | 20:10 |
| *** esker has joined #openstack-infra | 20:11 | |
| mordred | sdague, clarkb: my general thought was, as we refactored manage-projects into multiple instead of one tool - making our current upstream tracking code and the git-upstream tools for faciliting merging of such upstream branches into a thing | 20:12 |
| dhellmann | lifeless: that may be too summarized | 20:12 |
| dhellmann | lifeless, mordred : let me read a bit | 20:12 |
| *** jeckersb is now known as jeckersb_gone | 20:12 | |
| sdague | mordred: is this the gerrit of gerrits idea? | 20:13 |
| mordred | we ourselves have been tracking an upstream with patches for a couple of years - and we have forks of a couple of others. it happens, even with the best of intentions | 20:13 |
| mordred | sdague: yah | 20:13 |
| mordred | or one of the steps towards it | 20:13 |
| *** jasondotstar has quit IRC | 20:14 | |
| mordred | if we have the smarts to tell a gerrit that it's tracking another gerrit's thing - then I think overall that helps facilitate people submitting their patches upstream | 20:14 |
| *** adalbas has quit IRC | 20:14 | |
| mordred | but I may just be an optimist - it's certainly my hope that that's what the result would be | 20:14 |
| sdague | yeh, I just get concerned with handing a tool to "bob's local cloud" which tries to autorebase his hot fixes and push to production, we're probably not doing him a service | 20:15 |
| sdague | because there is a lot you need to be sure of in that process | 20:15 |
| sdague | if it was more this gerrit chain, which would include a robust revalidation, that's a bit different | 20:16 |
| fungi | Alex_Gaynor: right, that was me looking into it, and then dstufft... bug on the openstack side is https://launchpad.net/bugs/1290562 and it seems that with the setuptools 3.3 release yesterday this is now impacting hpcloud as well, not just rackspace | 20:16 |
| Alex_Gaynor | fungi: at least it's more consistently reproducable? ~~computer~~ | 20:16 |
| lifeless | sdague: so thats where tripleo comes in, because we're all about CD - testing | 20:16 |
| mordred | sdague: that would certainly be the intent -which is why I think it shoudl associate with manage-projects | 20:16 |
| lifeless | sdague: and only deploying things we know work | 20:16 |
| fungi | Alex_Gaynor: right | 20:16 |
| *** rcarrillocruz1 has joined #openstack-infra | 20:17 | |
| mordred | dhellmann: tl;dr - 1.0.0b1 is pep440 but not semver. 1.0.0.b1 is current pbr semver but compat with nothing. 1.0.0.0b1 happens to fill both intents - and there are a few other tweaks | 20:17 |
| pleia2 | apparently when a 2 year old needs to be fed, dinner comes early :) bbiab | 20:18 |
| sdague | Alex_Gaynor: so any idea why it's pypy only that's failing this? | 20:18 |
| Alex_Gaynor | sdague: Not really, I can imagine vaguely some gc thing, or dict ordering, but I don't know; let me poke around the pip source code | 20:18 |
| *** rcarrillocruz has quit IRC | 20:18 | |
| sdague | cool | 20:18 |
| sdague | because this is blocking some things we need for icehouse, so if we don't have an answer soon, I'd like to drop the pypy job on requirements from voting | 20:19 |
| *** rcarrillocruz has joined #openstack-infra | 20:19 | |
| *** alff has joined #openstack-infra | 20:19 | |
| dhellmann | mordred: so what's the potential downside? just that we're changing? | 20:20 |
| lifeless | dhellmann: the change I'm really interested in is the version calculation bit, not the serialisation of 1.0.0 beta 1 :) | 20:20 |
| dhellmann | lifeless: the "next version" calculation stuff? is that different from what we do now, or do we punt and use "number of commits + hash"? | 20:21 |
| lifeless | dhellmann: e.g. from 1.0.0.5 as fifth commit after 1.0.0 to 1.0.1.0a0.dev5 | 20:21 |
| lifeless | dhellmann: right now we punt and use number of commits + hash (the hash would be present in both cases) | 20:21 |
| fungi | Alex_Gaynor: if you're trying to reproduce, use virtualenv 1.11.4 to make a pypy venv (it will get an initial setuptools 2.2 bundled in). then try to 'pip install -U setuptools mccabe' in that venv, and when pip tries to upgrade setuptools from 2.2 to 3.3 it seems to end up not being available when setup.py gets run for mccabe (which is just an example where we've seen it fail in the wild, but pretty | 20:21 |
| dhellmann | lifeless: yeah, that seems like a reasonable enhancement to me | 20:21 |
| fungi | much any package which relies on setuptools functionality could be substituted to a similar result) | 20:21 |
| dhellmann | lifeless: perhaps this should wait until the openstack feature freeze lifts, thought | 20:22 |
| mordred | dhellmann: ++ OR ... | 20:22 |
| *** rcarrillocruz1 has quit IRC | 20:22 | |
| mordred | dhellmann: we release your pbr relase you want | 20:22 |
| lifeless | dhellmann: perhaps. OTOH I'm blocked moving some stuff to git on this, so I will whinge ;) | 20:22 |
| * dhellmann is becoming better at ignoring whinging | 20:22 | |
| mordred | then we put lifeless change into a major version bump or something that this version of openstack won't consume and we don't bump the pins until post feature freeze | 20:22 |
| dhellmann | mordred: yeah, that release is a bug fix for the doc folks | 20:23 |
| dhellmann | mordred: do we ping our version of pbr? | 20:23 |
| * dhellmann can't type | 20:23 | |
| mordred | lifeless: I think we shoudl add a simple utility that will take a pbr version and do the filter to produce a debian version | 20:23 |
| dhellmann | do we *pin* our version of pbr? | 20:23 |
| lifeless | mordred: I'm not sure this counts as an incompatible change :) | 20:23 |
| mordred | dhellmann: yes. to my knowledge | 20:23 |
| fungi | Alex_Gaynor: as for systems where we've seen it happen consistently, ubuntu 12.04 lts with the pypy backport ppa | 20:23 |
| mordred | lifeless: it does | 20:23 |
| mordred | lifeless: there are people currently doing version transforms on our intermediary versions to my knowledge | 20:24 |
| lifeless | oh | 20:24 |
| jeblair | jogo: to finish the thought from earlier; the 0-16 minute delay before starting check jobs is likely the time needed to spin up nodes, which means that nodepool doesn't have nodes immediately ready | 20:24 |
| lifeless | ok, so yes it does then. | 20:24 |
| jeblair | jogo: that's certainly related to volume of patches | 20:24 |
| jeblair | jogo: but could possibly be improved by having nodepool set its threshold to "demand+min_ready" instead of "max(demand, min_ready)" which i think is what it does now | 20:24 |
| mordred | lifeless: which is why I'm thinking - perhaps we also make a little filter program as a present for those folks, so we can say "just use this now" | 20:25 |
| jeblair | jogo: that would probably need to be followed by re-tuning the min_ready values in our config | 20:25 |
| mordred | dguerri_: https://github.com/emonty/git-upstream | 20:25 |
| jeblair | jogo: but sdague's point about noop is good -- it's very silly that we are running 200+ jobs a day that do literally nothing; that should be internalized into zuul either as a special case, or a zuul-internal noop worker | 20:25 |
| mordred | dguerri_: I did some work to get it up to general build tooling standards | 20:26 |
| jeblair | jogo: and finally, we need to move from hpcloud 1.0 to hpcloud 1.1 where we will recover some quota that we have lost (at hp's request) | 20:26 |
| jeblair | jogo: and would actually be in a position to ask for more | 20:26 |
| sdague | jeblair: https://review.openstack.org/#/c/76796 this is the swift backend to os-loganalyze patch | 20:26 |
| *** adalbas has joined #openstack-infra | 20:26 | |
| mordred | dguerri_: I'd like to chat about how we can move the upstream-tracking functionalities from jeepyb's manage-projects into this - but it's bedtime for me now - so perhaps we can chat about that tomorrow | 20:27 |
| lifeless | mordred: that would be nice. setup.py --debian-version ? | 20:27 |
| jeblair | sdague: yes! | 20:27 |
| jeblair | sdague: the zuul part of that is just about to go in, and jhesketh has a tool to upload logs staged | 20:28 |
| sdague | I'm reading through it now, it completely breaks the progressive nature of it, so for nova-cpu we're going to pull a 20MB file from swift to apache, explode it into an array (another 20+ MB) then spoon feed it back to the client | 20:28 |
| jeblair | sdague: so we're very close to being able to test that | 20:28 |
| mordred | lifeless: or "setup.py --version | pbr debian-version-filter" - or something | 20:28 |
| mordred | lifeless: I'm meh on the exact vehicle if you have strong feelings | 20:28 |
| sdague | any idea if there is a away to get the content from swift in a more progressive way? | 20:28 |
| jeblair | sdague: very good point, we definitely don't want to do that. obj[1].split('\n') on line 316 does look :( | 20:29 |
| sdague | yep | 20:30 |
| *** SumitNaiksatam has quit IRC | 20:30 | |
| *** SumitNaiksatam has joined #openstack-infra | 20:31 | |
| *** lcostantino has quit IRC | 20:31 | |
| jeblair | notmyname: do you know if there's a way to read an object in a streaming fashion with swiftclient? get_object looks like it grabs the entirety... | 20:32 |
| *** afazekas has quit IRC | 20:32 | |
| notmyname | jeblair: ya, give me a sec. | 20:33 |
| jeblair | notmyname: oh, i'm looking at the source, it looks like in some cases, the body get_object returns might be a generator | 20:34 |
| notmyname | jeblair: ya, it should be. the client isn't buffering the whole object anywhere (that would be a bug) | 20:34 |
| *** jnoller has quit IRC | 20:34 | |
| *** zns has joined #openstack-infra | 20:35 | |
| jeblair | http://git.openstack.org/cgit/openstack/python-swiftclient/tree/swiftclient/client.py#n802 | 20:35 |
| dhellmann | mordred, lifeless : I tried to summarize our conversation for the ML, please correct me if I got it wrong. | 20:35 |
| jeblair | notmyname: it looks like it uses the generator if you specify resp_chunk_size, though that defaults to None | 20:36 |
| notmyname | jeblair: ah, yes. do that | 20:36 |
| sdague | notmyname: any chance we could specify a separator instead of a chunk size? | 20:37 |
| *** dizquierdo has joined #openstack-infra | 20:37 | |
| notmyname | sdague: how do you mean? what are you trying to do? | 20:37 |
| *** julim_ has quit IRC | 20:37 | |
| sdague | we're trying to make the wsgi filter which does the log formatting on logs.openstack.org be able to back to swift | 20:37 |
| *** yassine has quit IRC | 20:38 | |
| sdague | it's currently backing to files, and we're using fileinput to make it a generator based on reading lines from files and processing them one at a time | 20:38 |
| notmyname | sdague: are you talking about a delimiter in the stream? or multipart mime chunks? or... | 20:38 |
| sdague | so I was mostly trying to avoid having to write a line oriented generator on top of a chunk oriented generator | 20:39 |
| sdague | notmyname: the first | 20:39 |
| notmyname | ah, so I'm guessing you want to ask the cluster for "give me the next line of the object" | 20:39 |
| sdague | notmyname: right, exactly | 20:39 |
| sdague | which I get is a weird thing | 20:39 |
| notmyname | heh | 20:39 |
| sdague | depending on implementation | 20:39 |
| sdague | but figured it was worth asking before doing these generator stacks | 20:39 |
| *** sandywalsh has quit IRC | 20:41 | |
| *** caleb_ has joined #openstack-infra | 20:41 | |
| *** lcostantino has joined #openstack-infra | 20:42 | |
| *** caleb_ has quit IRC | 20:42 | |
| openstackgerrit | A change was merged to openstack-dev/pbr: Factor run_cmd out of the base class. https://review.openstack.org/80447 | 20:43 |
| rcarrillocruz | bleh | 20:43 |
| rcarrillocruz | tox 1.7 was giving me quite a bit of grief | 20:43 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Teach pbr VersionInfo about debian versions. https://review.openstack.org/81074 | 20:44 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Allow examining parsing exceptions. https://review.openstack.org/80856 | 20:44 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Permit pre-release versions with git metadata https://review.openstack.org/80857 | 20:44 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Teach pbr about post versioned dev versions. https://review.openstack.org/80449 | 20:44 |
| openstackgerrit | lifeless proposed a change to openstack-dev/pbr: Add a converter to version_tuples. https://review.openstack.org/80457 | 20:44 |
| *** harlowja is now known as harlowja_away | 20:44 | |
| lifeless | dhellmann: looking | 20:45 |
| lifeless | sdague: there are lines-from-chunk generators around already; not sure if they are accessible for reuse but - pretty trivial. | 20:45 |
| lifeless | sdague: in fact - the io module one is definitely reusable. | 20:45 |
| sdague | lifeless: yeh, it's not terrible. Just trying to avoid if possible. | 20:46 |
| sdague | I did run into an issue around compressed files the last time | 20:46 |
| sdague | which is why the current approach is written like it is | 20:46 |
| *** ominakov has quit IRC | 20:46 | |
| sdague | jeblair: which, we should confirm, we're going to store uncompressed in swift, right? | 20:47 |
| sdague | because otherwise this gets way more complicated | 20:47 |
| notmyname | sdague: if that's to be done, it would need to be done client-side, not server side. therefore it would either be in python-swiftclient or in your app that is using swiftclient | 20:47 |
| notmyname | sdague: in which case, it seems to make sense to have it in your app | 20:47 |
| sdague | notmyname: sure, just thought I'd ask | 20:47 |
| *** jeckersb_gone is now known as jeckersb | 20:48 | |
| *** rossella_s has quit IRC | 20:49 | |
| jeblair | notmyname: what's your thinking on compression? should we compress before sending to swift, or does swift compress internally and we shouldn't worry about it? | 20:49 |
| jeblair | sdague: if we do compressioun ourselves, surely we could stick an uncompress generator between the linebreak and http read ones? | 20:50 |
| sdague | jeblair: so my experience is it's a lot trickier than you think | 20:50 |
| sdague | I actually tried that the last time | 20:50 |
| jeblair | sdague: oh, hrm, what was the problem? | 20:51 |
| notmyname | jeblair: hang on, I gotta go check that. but I think we might have a really good story there | 20:51 |
| sdague | I don't remember right now | 20:51 |
| jeblair | notmyname: cool, i love good stories! | 20:51 |
| sdague | the general problem with generators on top of generators though is none of your chunks align | 20:51 |
| *** signed8bit has joined #openstack-infra | 20:52 | |
| *** rpodolyaka1 has quit IRC | 20:52 | |
| jeblair | sdague: yeah, but having, say, 8k of memory used because you're reading 4k chunks doesn't sound terrible | 20:52 |
| *** Sukhdev has quit IRC | 20:52 | |
| sdague | sure | 20:52 |
| sdague | that's not really the issue | 20:52 |
| *** rpodolyaka1 has joined #openstack-infra | 20:53 | |
| sdague | the issue is you need natural alignment boundaries | 20:53 |
| sdague | so you have to figure out if you have enough to yield to the next layer sanely, and then only pass up the boundary correctly | 20:53 |
| sdague | keep the extra, to build the next go around | 20:53 |
| dhellmann | mordred: so you think it's safe to release pbr 0.7? | 20:54 |
| *** sandywalsh has joined #openstack-infra | 20:54 | |
| jeblair | sdague: hrm, i don't look at it that way; i look at it as "the linebreak generator reads from the uncompress generator until it hits a newline and retains the extra for the next iteration" | 20:54 |
| *** dizquierdo has quit IRC | 20:55 | |
| jeblair | sdague: and so forth down the stack | 20:55 |
| sdague | jeblair: sure | 20:55 |
| sdague | except uncompress generators are tricky | 20:55 |
| *** dizquierdo has joined #openstack-infra | 20:55 | |
| *** sarob_ has quit IRC | 20:55 | |
| sdague | maybe someone else will have better luck this time around | 20:56 |
| sdague | I hit some real issues on that last time, and the basic answer was "don't do that in python" | 20:56 |
| jeblair | sdague: true, in that they output, in our case, 10x more than they are given in input, so it's much more than the 2x waste of ram i cited earlier, but it still seems to be sane overall | 20:56 |
| clarkb | ya thats why gearman log worker loads into memory | 20:56 |
| *** thuc_ has quit IRC | 20:56 | |
| mattoliverau | Morning all | 20:57 |
| clarkb | incremental inflate is tricky with existing python zip tools best I could tell | 20:57 |
| *** thuc has joined #openstack-infra | 20:57 | |
| sdague | I'm almost 100% sure the fileinput compressed filter is implemented in native code | 20:57 |
| *** rpodolyaka1 has quit IRC | 20:57 | |
| mordred | dhellmann: do you think it's a major ersion bump? | 20:57 |
| *** ominakov has joined #openstack-infra | 20:57 | |
| jeblair | sdague: okay, let's see what notmyname says is the best approach, and ask jhesketh to look into that; i'm sure he'll dig into the problem | 20:57 |
| sdague | clarkb: yeh, that was my findings. Basically all the other approaches assumed you just loaded into memory | 20:57 |
| mordred | dhellmann: "major" | 20:58 |
| notmyname | jeblair: still looking | 20:58 |
| dhellmann | mordred: python version support changes; reporting output of some commands change; requirements are updated | 20:58 |
| jeblair | sdague: main thing is, i think we probably want this stuff compressed somehow, because "hey rax, can we have a petabyte to store logfiles uncompressed" isn't a question i'm looking forward to asking. :) | 20:58 |
| mordred | dhellmann: lemme look | 20:58 |
| *** mfer has quit IRC | 20:59 | |
| *** jlibosva has joined #openstack-infra | 20:59 | |
| dhellmann | mordred: http://paste.openstack.org/show/73693/ | 20:59 |
| sdague | jeblair: heh | 20:59 |
| dhellmann | jeblair: do they need to be on rax? I don't have compute to offer yet, but could come up with some storage | 21:00 |
| sdague | dhellmann: a petabyte? | 21:00 |
| lifeless | mordred: btw - debian version stuff in gerrit now | 21:00 |
| mordred | lifeless: ossum | 21:00 |
| sdague | well, in fairness, it would probably only be about 30 TB / 6 months | 21:00 |
| *** afazekas has joined #openstack-infra | 21:00 | |
| *** jlibosva has quit IRC | 21:00 | |
| lifeless | since dhellmann is on board | 21:00 |
| lifeless | I'm going to polish up the branch, make sure it supports a/b/rc versions etc | 21:01 |
| dhellmann | lifeless: is there a blueprint for that work? | 21:01 |
| * dhellmann puts on PTL hat | 21:01 | |
| jeblair | sdague: we're currently using 9.4TB -- so assuming 90% compression, that's close to 100TB / 6 months | 21:01 |
| lifeless | dhellmann: FIIK | 21:01 |
| * dhellmann isn't sure if he wants to google that acronym | 21:01 | |
| *** rossella_s has joined #openstack-infra | 21:01 | |
| lifeless | dhellmann: there's an etherpad. I can create tracking paperwork if you want | 21:01 |
| *** thuc has quit IRC | 21:01 | |
| lifeless | dhellmann: swearword If I Know | 21:02 |
| jeblair | dhellmann: that's a really good point. we've mostly been focused on getting this working on one provider, but we really should see if we can distribute it out across multiple swift clusters | 21:02 |
| dhellmann | lifeless: it's ok for this cycle, but I think next cycle we're going to start keeping better records | 21:02 |
| *** pdmars has quit IRC | 21:02 | |
| lifeless | dhellmann: I still totally fail to see the value in blueprints here, they are terrible for discoverability, accessability, and usability. | 21:02 |
| dhellmann | lifeless: by "here" do you mean pbr or at all? | 21:03 |
| mordred | dhellmann: I'm still not sure, looking at the diff, that the major needs bumped. BUT - I'm also not strongly opinioned | 21:03 |
| mordred | dhellmann: so - I defer to your judgement | 21:03 |
| lifeless | dhellmann: I think they have value in the at-scale parts of openstack | 21:03 |
| dhellmann | mordred: sdague scolded me the last time I didn't bump the major version with a requirements change | 21:03 |
| lifeless | dhellmann: but things with 5 contributors.... | 21:03 |
| mordred | dhellmann: although if you do want to 0.7 - could you semver it at 0.7.0 ish/ | 21:04 |
| dhellmann | lifeless: pbr is part of oslo, and I'd like to just have one way I track stuff for release planning | 21:04 |
| dhellmann | mordred: ok | 21:04 |
| lifeless | dhellmann: if you want one, I'll give you one. | 21:04 |
| lifeless | dhellmann: wearing my PTL hat, I just find them annoying :) | 21:04 |
| dguerri_ | mordred: ack | 21:05 |
| dhellmann | sdague: DreamObjects has just over 2 petabytes available, but you can't have all of that for free :-) | 21:05 |
| *** ominakov has quit IRC | 21:05 | |
| dhellmann | lifeless: yeah, I get it, the tool isn't great, but it beats keeping little slips of paper on my desk to remind me which changes actually need to be reviewed :-) | 21:05 |
| *** yolanda has quit IRC | 21:06 | |
| lifeless | dhellmann: so, you'd like one? | 21:06 |
| *** e0ne has quit IRC | 21:06 | |
| dhellmann | lifeless: not this time around, but like I said, during juno I think I'm going to start requiring them more | 21:06 |
| dhellmann | assuming it's up to me, of course | 21:06 |
| *** e0ne has joined #openstack-infra | 21:07 | |
| lifeless | dhellmann: FWIW I use gerrit to tell me what I need to review :) | 21:07 |
| *** ominakov has joined #openstack-infra | 21:07 | |
| *** afazekas has quit IRC | 21:07 | |
| dhellmann | lifeless: I have not had as much luck with that lately; scale of number of repos changed this cycle | 21:07 |
| dguerri_ | mordred: I also have a couple of new commits, I will push them to your repo tomorrow | 21:08 |
| dhellmann | and that's only going to get worse in juno | 21:08 |
| dhellmann | jeblair, sdague : email me some details about volume for these logs and I'll see if I can hook you up with an account | 21:08 |
| *** dguerri_ is now known as dguerri | 21:08 | |
| *** rcarrillocruz1 has joined #openstack-infra | 21:09 | |
| *** caleb_ has joined #openstack-infra | 21:09 | |
| notmyname | jeblair: ah, found what I was looking for | 21:10 |
| *** caleb_ has quit IRC | 21:10 | |
| dhellmann | lifeless: reviewing open pull requests, I saw https://review.openstack.org/#/c/73737/ -- is that going to interfere with what you're doing? | 21:10 |
| dhellmann | lifeless: nevermind, that's parsing requirements input files not dealing with the version number generation | 21:10 |
| *** e0ne has quit IRC | 21:10 | |
| lifeless | yeah | 21:11 |
| notmyname | jeblair: so 3 years ago (almost to the day!) I was involved in getting a new feature out for cloud files. but unfortunately it was just for the CDN, not for swift itself. basically, it allows stuff to honor the "Accept-Encoding: gzip" request header | 21:11 |
| jogo | jeblair: ahh, I am a litte shocked at how long it takes to spin up a node | 21:11 |
| *** hashar is now known as hasharMeeting | 21:11 | |
| notmyname | jeblair: http://www.rackspace.com/blog/cloud-files-cdn-compresses-at-the-edge/ is the CDN thing, and no, it isn't supported in requests directly to swift (but it would be a cool feature!!) | 21:11 |
| dhellmann | mordred: I'm going to wait until tomorrow morning to push the 0.7.0 tag, so I don't break things and then sign off for the night | 21:11 |
| *** rcarrillocruz has quit IRC | 21:12 | |
| jogo | jeblair: how much of that is openstack | 21:12 |
| lifeless | dhellmann: https://blueprints.launchpad.net/pbr | 21:12 |
| lifeless | dhellmann: is not configured | 21:12 |
| dhellmann | lifeless: :-| | 21:12 |
| dhellmann | lifeless: fixed | 21:12 |
| sdague | dhellmann: instead of using a million lps, why not use one? | 21:13 |
| sdague | like infra does | 21:13 |
| dhellmann | sdague: yeah, I'm not creating any new ones | 21:13 |
| jogo | jeblair: and lastly what can I do to help. | 21:13 |
| dhellmann | I guess we could put the pbr blueprints in the oslo project | 21:13 |
| dhellmann | lifeless: yeah, I'm going to do that, deconfigure it and add a note about using the oslo tracker instead | 21:14 |
| clarkb | gah what did we do to logstash | 21:14 |
| lifeless | dhellmann: doh | 21:14 |
| lifeless | dhellmann: I was just about to create first-post-better-untagged-versions as the blueprint | 21:14 |
| dhellmann | lifeless: have you done it yet? | 21:14 |
| dhellmann | I can just move it | 21:14 |
| *** ominakov has quit IRC | 21:15 | |
| lifeless | dhellmann: I haven't | 21:15 |
| *** e0ne has joined #openstack-infra | 21:15 | |
| lifeless | dhellmann: and it wouldn't be as funny in a group :) | 21:15 |
| dhellmann | lifeless: heh | 21:15 |
| *** rwsu has quit IRC | 21:16 | |
| lifeless | dhellmann: sdague: BTW LP has a very strong one-codebase-one-project model - e.g. you cannot have multiple tasks within a single project | 21:16 |
| lifeless | what infra does isn't something to copy (for bugs) - blueprints I don't care about ;> | 21:16 |
| lifeless | tripleo does just one place for blueprints | 21:16 |
| *** dkranz has quit IRC | 21:17 | |
| dhellmann | we already have separate projects for oslo.messaging and pbr, but I was going to use the oslo project for all the new libs we make next cycle | 21:17 |
| dhellmann | and then use tags on bugs and blueprints to make it easier to filter stuff | 21:17 |
| openstackgerrit | A change was merged to openstack-infra/config: Save md5 and sha1 checksums with artifacts https://review.openstack.org/76933 | 21:17 |
| Ajaeger1 | clarkb, fungi - I would really appreciate if tomorrow's periodic job includes the fixed propose-translation-update jobs - could you please put https://review.openstack.org/#/c/81037/ on your list? | 21:17 |
| clarkb | Ajaeger1: did you catch where fungi thought we would need a g-g-p patch to make that work? | 21:18 |
| *** ominakov has joined #openstack-infra | 21:18 | |
| clarkb | Ajaeger1: the other jobs use the scm plugin | 21:18 |
| Ajaeger1 | clarkb: I've updated the patch to use gerrit-git-prep | 21:19 |
| clarkb | Ajaeger1: right I think fungi discovered g-g-p won't work today | 21:20 |
| Ajaeger1 | do I just need to wait another day - or change something? | 21:20 |
| jeblair | jogo: all of it is openstack (we spin up from custom images; i gather that could make things a little slower); hpcloud 1.0 is faster than rax, i don't have as good of an idea about hpcloud 1.1. | 21:20 |
| *** ominakov has quit IRC | 21:20 | |
| fungi | clarkb: did you have any input on whether or not i should just make ggp support that so we can ditch the scm plugin? | 21:20 |
| jogo | jeblair: once we are on hpcloud 1.1 we should revisit what is slow in openstack. because at that point we will be running trunk | 21:21 |
| clarkb | fungi: I think we will need that eventually, but that takes at least a day for new images (typically) | 21:21 |
| clarkb | fungi: probably easier to add scm plugin to oen job now then g-g-p all other jobs later | 21:21 |
| jogo | and can fix things upstream and get it in CI in a few weeks | 21:21 |
| jogo | jeblair: silly question -- where are nodepool unit tests | 21:21 |
| fungi | clarkb: basically make the slave script for it allow for a missing zuul ref and assume head of the default branch? | 21:22 |
| Ajaeger1 | Argh, missed fungi's latest comment on the other translation jobs. Let me double check what else I missed ;( | 21:22 |
| jogo | jeblair: min_demand = start_demand + total_image_min_ready[image_name] | 21:22 |
| *** esker has quit IRC | 21:22 | |
| clarkb | fungi: yeah thats what I am thinking. if -z ZUUL_REF (or whatever the appopriate ZUUL_* is) then git checkout remotes/origin/HEAD | 21:22 |
| fungi | Ajaeger1: basically you added the scm plugin because you copied it from the other translation proposal job, which was working because as it turns out gerrit-git-prep assumes it will have a zuul ref and periodic jobs don't | 21:22 |
| *** rpodolyaka1 has joined #openstack-infra | 21:23 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack/requirements: Add saharaclient to global requirements https://review.openstack.org/81083 | 21:23 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack/requirements: Remove python-savannaclient from globalr req https://review.openstack.org/81084 | 21:23 |
| fungi | Ajaeger1: so just repropose your patch like you had in the previous patchset and i'll stack a wip change on top of that which makes ggp do what we want and removes/replaces all instances of the scm plugin | 21:23 |
| Ajaeger1 | fungi: so, I should send the previous patch again that included both ggp and scm plugin? | 21:24 |
| jeblair | jogo: probably. testing nodepool involves running the steps in README; it doesn't have a test suite. I would love it if it had a functional test suite (i'm generally more excited about that than unit tests per se) | 21:24 |
| clarkb | Ajaeger1: no remove ggp | 21:24 |
| Ajaeger1 | Ok, will do. Thanks | 21:24 |
| dstufft | mordred: soooo | 21:24 |
| dstufft | I had an idea | 21:24 |
| jeblair | jogo: (the faked out stuff described in the readme might make a good start for a functional test suite) | 21:25 |
| dstufft | to prevent setuptools from every touching the network even if someone isn't using pip (in a pip 1.6+ world at least :V) | 21:25 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Rename Savanna to Sahara https://review.openstack.org/81086 | 21:25 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Add alazarev to sahara-core https://review.openstack.org/81087 | 21:25 |
| lifeless | dhellmann: https://blueprints.launchpad.net/oslo/+spec/better-untagged-versions | 21:26 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Fix periodic manuals-propose-translation-update jobs https://review.openstack.org/81037 | 21:26 |
| dhellmann | lifeless: thanks | 21:26 |
| Ajaeger1 | fungi, clarkb: like this? | 21:26 |
| fungi | Ajaeger1: yep. lgtm | 21:27 |
| Ajaeger1 | Uff, great | 21:27 |
| *** thomasbiege has quit IRC | 21:27 | |
| *** alff has quit IRC | 21:27 | |
| jeblair | fungi, clarkb: deleting leaked tripleo keypairs (we're at quota) | 21:27 |
| clarkb | Ajaeger1: do the docs have a {gitbub-org} var? | 21:27 |
| *** rpodolyaka1 has quit IRC | 21:27 | |
| clarkb | might be good to use that for more flexibility but I don't think it is strictly required here | 21:27 |
| Ajaeger1 | clarkb: All use openstack | 21:27 |
| *** thuc has joined #openstack-infra | 21:27 | |
| SergeyLukjanov | fungi, clarkb, jeblair, mordred, sdague, could you please take a look at https://review.openstack.org/81083 (sahara client addition to the global requirements) | 21:28 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/nodepool: Raise min_demand due to slow node boot times https://review.openstack.org/81088 | 21:28 |
| fungi | Ajaeger1: in jenkins_jobs/config/projects.yaml | 21:28 |
| Ajaeger1 | clarkb: then I would need to double check that it's set correctly; ) let me check projects.yaml... | 21:28 |
| sdague | SergeyLukjanov: so until the pypy thing is resolved, it's sort of pointless | 21:28 |
| dhellmann | lifeless: I marked that as a juno item, but I see in your ML thread you want it for icehouse. How much disruption do you expect that to cause? | 21:28 |
| Ajaeger1 | yeah, is set correctly | 21:29 |
| * Ajaeger1 changes | 21:29 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Fix periodic manuals-propose-translation-update jobs https://review.openstack.org/81037 | 21:29 |
| Ajaeger1 | here it comes again... | 21:29 |
| SergeyLukjanov | sdague, oh... | 21:29 |
| SergeyLukjanov | sdague, reading scrollback | 21:29 |
| *** rfolco has quit IRC | 21:29 | |
| clarkb | something changed around 20:15 to make all of our logstash worker output graphs look like http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=1365&rra_id=all | 21:30 |
| clarkb | and that is about when the gearman queue graph takes off | 21:31 |
| *** rwsu has joined #openstack-infra | 21:31 | |
| clarkb | but I don't see anything that is a blatant culprit. any ideas? | 21:32 |
| Ajaeger1 | fungi, clarkb: Thanks for all your help! I'm signing off now. If I should do more changes, just comment on the patch and I'll fix tomorrow. | 21:32 |
| lifeless | dhellmann: it only affects untagged versions, not tagged versions | 21:32 |
| clarkb | sdague: we didn't do anything like the libvirt crazy logging again did we? | 21:32 |
| * clarkb checks worker node | 21:32 | |
| lifeless | dhellmann: and the versions it will produce are all strictly greater than the current behaviour | 21:33 |
| sdague | clarkb: not that I know of | 21:33 |
| lifeless | dhellmann: so I expect the fallout to concentrate on folk that are consuming untagged versions and casting them into e.g. debian package versions | 21:33 |
| dhellmann | lifeless: so this won't change alpha versions, for example? | 21:33 |
| dhellmann | lifeless: hmm, ok, so me then :-) | 21:33 |
| *** Ajaeger1 has quit IRC | 21:33 | |
| dhellmann | oh, no, we don't use the version info from the package, we make our own version numbers, so not me | 21:34 |
| dstufft | lifeless: doesn't it make the version number 0aN instead of aN | 21:34 |
| dstufft | fir alphas | 21:34 |
| dstufft | to be PEP440 compliant | 21:34 |
| lifeless | dhellmann: if you tag a version, the tag will always be obeyed | 21:35 |
| dhellmann | lifeless: ok | 21:35 |
| *** beekneemech is now known as bnemec | 21:36 | |
| clarkb | sdague: is nova cond logs usually >6MB compressed? | 21:36 |
| *** thuc has quit IRC | 21:36 | |
| sdague | I think it got more gorpy after oslo.messaging sync | 21:36 |
| lifeless | dhellmann: dstufft: yes, part of the proposal is to change how we label alphas | 21:37 |
| lifeless | dhellmann: so 1.3.0a1 isn't valid PEP440 | 21:37 |
| clarkb | sdague: or is this os-loganalyze being slow? /me isn't finding the bottleneck in the pipelines yet | 21:37 |
| clarkb | ES is happy | 21:37 |
| lifeless | dhellmann: but that is a tagged release. The proposal is to use 1.3.0.0a1 to tag such a release in future. | 21:38 |
| sdague | clarkb: it shouldn't be unless the log server is overloaded | 21:38 |
| sdague | is there a cacti graph on that? | 21:38 |
| lifeless | dhellmann: (and to issue new tags for all our current pre-releases to make them PEP440 compliant, but that should happen *before* the changes go into pbr | 21:38 |
| sdague | also, conductor is huge, however it's all debug | 21:38 |
| jeblair | sdague: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1 | 21:38 |
| dhellmann | lifeless: but we'd have to do that by hand? you're not going to add, for example, the ability to say "python setup.py tag-alpha" and have it compute that tag automatically? | 21:38 |
| sdague | I think there are 40 non debug lines in it | 21:38 |
| vishy | dhellmann: ping | 21:38 |
| clarkb | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=311&rra_id=all seems to be happy | 21:38 |
| jeblair | sdague: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=13 | 21:38 |
| *** changbl has joined #openstack-infra | 21:38 | |
| lifeless | dhellmann: that would be a logical further extension | 21:39 |
| jogo | jeblair: put up a nodepool patch. as for the functional testing thing. as for the functional test idea I can look into it later today, but have a few questions first when you have a few minutes (no rush) | 21:39 |
| lifeless | dhellmann: and I have some prototype code for deciding when to do major/minor/point changes as well | 21:39 |
| *** gordc has left #openstack-infra | 21:39 | |
| jeblair | jogo: did we lose gerritbot again? | 21:39 |
| lifeless | but thats future | 21:39 |
| vishy | dhellmann: trying to figure out if there is an ubuntu package for the oslo.sphinx rename | 21:40 |
| fungi | openstackgerrit is here | 21:40 |
| vishy | it has http://mirrors.kernel.org/ubuntu/pool/main/o/oslo-sphinx/python-oslo.sphinx_1.1-0ubuntu2_all.deb | 21:40 |
| dhellmann | lifeless: ok -- I think I'm ok with making this part of 0.8 if we're careful with the pins in the stable branches | 21:40 |
| vishy | do you know if that will be updated to the new stuff? or does it need a new package to work? | 21:40 |
| dhellmann | vishy: yeah, sorry about that -- it should be "oslosphinx" everywhere now | 21:40 |
| jogo | jeblair: I saw the notifcation pop up but this channel is pretty active today | 21:40 |
| kevinbenton | who runs stackalytics.com? | 21:41 |
| dhellmann | vishy: I don't know if they have created a package with the new name | 21:41 |
| fungi | dhellmann: the debian/ubuntu package maintainer should hopefully switch to the new package name and leave a dummy transitional package at the old name to accommodate that | 21:41 |
| vishy | adam_g: ping ^^ | 21:41 |
| fungi | kevinbenton: i believe that's a mirantis internal project | 21:41 |
| vishy | zul: ^^ | 21:41 |
| jeblair | jogo: ok, thx | 21:42 |
| kevinbenton | fungi: ok, thx | 21:42 |
| SergeyLukjanov | fungi, kevinbenton, yup, sources located at stackforge | 21:42 |
| jeblair | kevinbenton: http://git.openstack.org/cgit/stackforge/stackalytics/ | 21:42 |
| fungi | kevinbenton: hopefully they put some contact info on the site? | 21:42 |
| dhellmann | vishy, zul, adam_g : that package definitely has the old version of the lib | 21:42 |
| fungi | oh, totally forgot that ended up in stackforge | 21:42 |
| *** smarcet has quit IRC | 21:43 | |
| kevinbenton | fungi: yeah, that was the part that threw me off | 21:43 |
| sdague | clarkb: nothing I can see | 21:43 |
| dhellmann | fungi: thanks | 21:43 |
| jogo | jeblair: so two questions about testing nodepool | 21:43 |
| jogo | 1) would it work with sqllite? | 21:43 |
| sdague | clarkb: we didn't accidentally lose filtering by debug from gearman did we? | 21:44 |
| clarkb | sdague: I don't think so | 21:44 |
| jogo | 2) where does the load for nodepool come from (the data to test it out) | 21:44 |
| clarkb | reading the logs it is still pulling with ?level=INFO | 21:44 |
| sdague | the console logs are bigger again, because we're dumping errors there, but they are no bigger than they used to be (still smaller from when devstack output was in them) | 21:44 |
| clarkb | sdague: but we are indexing devstack too? is there a significant different between the combined size? | 21:45 |
| sdague | actually not | 21:45 |
| sdague | as I put the devstack logs on diet | 21:45 |
| devananda | one more review for ironic's CI support - https://review.openstack.org/#/c/80652/2 | 21:45 |
| jeblair | jogo: 1) unlikely -- sqlite deals very poorly with the massive threading nodepool does; we basically had to move production from sqlite to mysql because of that, so tests are probably likely to run into the same problem | 21:46 |
| *** ominakov has joined #openstack-infra | 21:46 | |
| jeblair | 2) the fake-servers.py script in tools has some basic stuff to generate load; it lets you tell nodepool that a job is finished, or that gearman has a huge demand for a certain job | 21:46 |
| sdague | clarkb: any way to figure out what files things die on? | 21:46 |
| lifeless | mordred: sound ok to you? | 21:46 |
| jeblair | jogo: ^ it's _very_ rudimentary, but it does exercise all the bits | 21:47 |
| jogo | jeblair: too bad because sqllite would make things very self contained. | 21:47 |
| *** ominakov has quit IRC | 21:47 | |
| jogo | I was thinking of setting something up like this http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/tox.ini#n27 | 21:47 |
| jogo | where you just run a tox env | 21:47 |
| sdague | jogo: you can mostly do that in mysql, with only minor setup | 21:48 |
| clarkb | sdague: I don't think it is dying on stuff | 21:48 |
| sdague | we do that for the opportunistic tests in nova | 21:48 |
| jeblair | jogo: yeah; though we have the openstack_citest mysql db on the slaves. i wouldn't mind having that locally | 21:48 |
| clarkb | sdague: everything seems to be moving, just more slowly | 21:48 |
| sdague | clarkb: that rise is pretty crazy | 21:48 |
| *** dcramer_ has quit IRC | 21:48 | |
| sdague | jogo: or the way the migrate project does real db testing | 21:48 |
| clarkb | which is why I was curious about upstream stuff (like file sizes and os-loganalyze) | 21:48 |
| jogo | sdague: right, just thought sqllite would make it very self contained | 21:48 |
| jeblair | jogo, sdague, clarkb: this always reminds me of the "use drizzle in unit tests idea" because drizzle is mysql compatible and completely self-contained (eg, can run in a throwaway tmpdir) | 21:48 |
| clarkb | jeblair: except for the places where it isn't quite compatible :) | 21:49 |
| jogo | jeblair: yeah just waiting for mordred to do it ;) | 21:49 |
| sdague | jeblair: sure, but the setup for a development mysql is all of 30 seconds, once | 21:49 |
| clarkb | but it speaks the protocol | 21:49 |
| sdague | as long as you are on a reasonable os :) | 21:49 |
| jogo | anyway I will play around with this a little later tonight ... overall it looks like getting a rudamentry *mostly* automated test is fairly straight forward | 21:50 |
| jeblair | sdague: in general, i agree, and lacking something like this, expecting a dev to have a mysql they can use is not a high bar. but re drizzle, i believe the thinking is that the unit tests could set up drizzle to run in a tmpdir, so no config required, plus is more secure on a workstation. | 21:50 |
| *** MarkAtwood has joined #openstack-infra | 21:51 | |
| *** mattymo has quit IRC | 21:51 | |
| clarkb | jeblair: sdague: and we could do stuff like not have crazy perms on test slaves for mysql | 21:51 |
| *** mattymo has joined #openstack-infra | 21:51 | |
| jeblair | jogo: cool, thanks! | 21:52 |
| sdague | clarkb: so are you trying to get to the bottom of the indexing issue? | 21:52 |
| clarkb | sdague: I am definitely poking at it, but need to afk shortly to go back to seattle | 21:53 |
| clarkb | the airlock on the fermentor is bubbling my work here is done >_> | 21:53 |
| sdague | that giant leap, and then flatline, is really suspicious | 21:54 |
| *** rcarrillocruz has joined #openstack-infra | 21:54 | |
| clarkb | the watchdog for logstash processes didn't trip at 20:15 so I don't think ES/logstash themselves are to blame | 21:54 |
| *** sarob has joined #openstack-infra | 21:56 | |
| sdague | did they suffer a network issue - http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=111 | 21:56 |
| sdague | it looks like their throughput dropped to almost nothing | 21:56 |
| clarkb | I am digging into the indexer logs now | 21:56 |
| *** afazekas has joined #openstack-infra | 21:56 | |
| clarkb | sdague: right in which case the watchdog should'ev tripped | 21:56 |
| *** rcarrillocruz1 has quit IRC | 21:56 | |
| clarkb | I think throughput dropped to nothing because they weren't doing work | 21:56 |
| sdague | clarkb: that looks like of like a classic slowstart curve on the upside of eth0 | 21:57 |
| sdague | I wonder if rax network just went to crap | 21:57 |
| clarkb | sdague: you may be right, the are massive gaps in log timestamps indicating they weren't pulling jobs off the queue | 21:58 |
| *** CaptTofu has joined #openstack-infra | 21:58 | |
| clarkb | sdague: http://paste.openstack.org/show/73700/ | 22:00 |
| clarkb | sdague: whcih seems to fit your theory | 22:00 |
| clarkb | it took 25 minutes to deal with that one file | 22:00 |
| clarkb | and the trend is much better now | 22:01 |
| *** e0ne has quit IRC | 22:02 | |
| zul | vishy: not yet | 22:03 |
| *** jamielennox|away is now known as jamielennox | 22:03 | |
| vishy | zul: when you do it will the package be renamed to python-oslosphinx? | 22:03 |
| *** david-lyle has quit IRC | 22:04 | |
| zul | vishy: this week probably | 22:04 |
| *** rossella_s has quit IRC | 22:05 | |
| zul | vishy: i was on vacation last week :P | 22:05 |
| *** wchrisj has quit IRC | 22:05 | |
| *** jhesketh_ has joined #openstack-infra | 22:06 | |
| vishy | zul: also we need a new version python-pbr as well it appears | 22:06 |
| jhesketh_ | Morning | 22:07 |
| zul | vishy: yep will do it this week | 22:07 |
| *** lcostantino has quit IRC | 22:08 | |
| *** thedodd has quit IRC | 22:08 | |
| *** thuc has joined #openstack-infra | 22:10 | |
| *** adalbas has quit IRC | 22:10 | |
| *** thuc_ has joined #openstack-infra | 22:12 | |
| *** nibalizer has joined #openstack-infra | 22:12 | |
| *** thuc has quit IRC | 22:15 | |
| clarkb | the logstash queue is falling, must've been a network thing | 22:16 |
| clarkb | I think I am going to take this as an opportunity to begin the drive back to seattle. Back in about 3-4 hours to hopefully review more code | 22:16 |
| jeblair | jhesketh_: good morning; there was some talk about os-loganalyze and swift. most of it is in the review... | 22:17 |
| *** harlowja_away is now known as harlowja | 22:17 | |
| *** ryanpetrello has quit IRC | 22:17 | |
| jhesketh_ | oh cheers jeblair, I'll take a look | 22:17 |
| fungi | clarkb: drive safe. btw i came up with another distasteful workaround to the pypy testing issue... http://paste.openstack.org/show/73701/ | 22:17 |
| jeblair | jhesketh_, sdague: we did have a mostly unfinished conversation about compression; and my take on what we heard from notmyname is that we should compress before sending to swift and decompress in os-loganalyze. | 22:17 |
| clarkb | fungi: nice! | 22:18 |
| jeblair | sdague and clarkb say that's very difficult/impossible in python (in a chunky manner, say, using a generator) | 22:18 |
| clarkb | fungi: we should hit approve on those changes then chase that with some whiskey >_> | 22:18 |
| notmyname | jeblair: I've got to make a phone call, but I'll be back in a bit | 22:18 |
| jeblair | jhesketh_: though i think finding a way to do it is worthwhile | 22:18 |
| *** afazekas has quit IRC | 22:19 | |
| clarkb | also if anyone makes it to portland in about 3 weeks I should have 5 gallons of homebrew in a keg >_> | 22:19 |
| jeblair | notmyname: cool, thanks for your help. i think we want to avoid requiring the rax cdn, so it sounds like we need to compress at the app level (or submit a change to add that to swift itself) | 22:19 |
| openstackgerrit | A change was merged to openstack-infra/config: Add direct-release for meetbot and nose-html-output https://review.openstack.org/79330 | 22:19 |
| *** aysyd has quit IRC | 22:19 | |
| sdague | jhesketh_: so that will be the trick of it | 22:20 |
| *** rcleere has quit IRC | 22:20 | |
| sdague | jhesketh_: https://github.com/openstack-infra/os-loganalyze/blob/master/os_loganalyze/wsgi.py#L212-L225 basically means that if we are file backed, we only have 1 line of the file in memory at a time | 22:20 |
| sdague | so something equiv has to be done if we are swift backed | 22:21 |
| jhesketh_ | sure, okay | 22:21 |
| jhesketh_ | I had admittedly thought about that when I wrote it, but figured as a starting point it was less important and wanted to get a proof of concept up | 22:22 |
| *** yamahata has quit IRC | 22:22 | |
| jhesketh_ | that said, I'm happy to reroll that today | 22:22 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix periodic manuals-propose-translation-update jobs https://review.openstack.org/81037 | 22:22 |
| sdague | it's probably worth making a test case for this as well which we could stand up on a devstack node easily enough, to actually test it working with swift | 22:23 |
| *** thuc_ has quit IRC | 22:24 | |
| *** adalbas has joined #openstack-infra | 22:24 | |
| *** rpodolyaka1 has joined #openstack-infra | 22:24 | |
| *** thuc has joined #openstack-infra | 22:24 | |
| sdague | and if anyone is interested in reviewing some incremental improvements in UI - https://review.openstack.org/#/q/status:open+project:openstack-infra/os-loganalyze+branch:master+topic:onclick_inline,n,z | 22:24 |
| sdague | I was hacking some this weekend | 22:24 |
| *** yamahata has joined #openstack-infra | 22:25 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Add more logging to zuul merger process https://review.openstack.org/79385 | 22:26 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove logging of sahara projects to #savanna https://review.openstack.org/79545 | 22:27 |
| *** rcarrillocruz1 has joined #openstack-infra | 22:28 | |
| *** rpodolyaka1 has quit IRC | 22:28 | |
| *** fifieldt has joined #openstack-infra | 22:29 | |
| jeblair | nibalizer, mattoliverau: is it possible to disable the ec2 facter plugin in puppet? | 22:29 |
| *** rcarrillocruz has quit IRC | 22:29 | |
| jeblair | nibalizer, mattoliverau: it's timing out a lot on one of our providers (who is currently having problems with their metadata server) | 22:29 |
| vishy | any idea what this is? /usr/share/openstack-pkg-tools/pkgos.make | 22:29 |
| jeblair | nibalizer, mattoliverau: and i'm pretty sure we don't actually use any facts from it | 22:30 |
| *** e0ne has joined #openstack-infra | 22:30 | |
| jeblair | clarkb: btw, any word on that ticket? | 22:30 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added user preference handling https://review.openstack.org/81100 | 22:30 |
| *** mriedem has quit IRC | 22:32 | |
| clarkb | jeblair: nope nothing yet | 22:32 |
| nibalizer | jeblair: i don't know how to disable facts | 22:32 |
| nibalizer | im not sure if you can | 22:32 |
| *** asalkeld has joined #openstack-infra | 22:36 | |
| asalkeld | hi, should we be temporarily disabling pypy until https://code.launchpad.net/bugs/1290562 is fixed? | 22:37 |
| *** adrian_otto has joined #openstack-infra | 22:37 | |
| fungi | asalkeld: quite possibly. Alex_Gaynor: did you get a chance to reproduce that yet? | 22:38 |
| asalkeld | yeah, I can't get anythink though | 22:38 |
| asalkeld | anything | 22:38 |
| asalkeld | they all fail | 22:38 |
| fungi | asalkeld: for projects which want to test pypy compliance, one fairly trivial way is to patch your tox.ini for now to do http://paste.openstack.org/show/73701/ | 22:38 |
| *** e0ne has quit IRC | 22:38 | |
| fifieldt | morning all | 22:39 |
| jeblair | fifieldt: good morning! | 22:39 |
| jeblair | fifieldt: have you registered your nick with nickserv? if not, you should do that | 22:39 |
| fungi | asalkeld: for reasons as of yet unidentified, pip is able to upgrade setuptools to 3.1 safely but if it tries to go any higher, that's when we hit this bug | 22:39 |
| *** openstackgerrit has quit IRC | 22:39 | |
| asalkeld | ok | 22:39 |
| fifieldt | jeblair, indeed I should - thanks for the reminder | 22:40 |
| *** openstackgerrit has joined #openstack-infra | 22:40 | |
| jeblair | fifieldt: let me know once you're done, and i'll add you to the list of irc ops for all of our channels (54+) | 22:41 |
| fifieldt | ooh | 22:41 |
| fifieldt | now there's a carrot :) | 22:41 |
| jeblair | fifieldt: heh, i'm not sure if that particular carrot is delicious or poisoned. ;) | 22:42 |
| notmyname | jeblair: correct | 22:43 |
| fifieldt | mmmm poison carrot | 22:43 |
| fungi | jeblair: fifieldt: it's a carrot which is only poisonous during apac daytime hours | 22:43 |
| fifieldt | mmm variably poisonous carrot | 22:44 |
| fifieldt | I think I'm identified onw | 22:44 |
| fungi | and temporally variable at that | 22:44 |
| fifieldt | now* | 22:44 |
| *** hasharMeeting has quit IRC | 22:44 | |
| fifieldt | https://review.openstack.org/#/q/reviewer:welcome-message,n,z <-- does my gerrit query syntax look OK there ? I feel like I got it right, because it didn't pop up an error message (though there are no reviews) :) | 22:45 |
| *** asalkeld has left #openstack-infra | 22:45 | |
| jeblair | fifieldt: cool, i'll add you to the list in a bit (may be a couple days for the changes to make it through review) | 22:45 |
| fungi | fifieldt: i'll try to do a correlation from the log | 22:46 |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: workaround pypy in requirements https://review.openstack.org/81103 | 22:46 |
| fifieldt | cheers jeblair | 22:46 |
| sdague | fungi: given that saraha is blocking on that one, it's probably worth taking in | 22:47 |
| sdague | it didn't seem like anyone else had yet proposed it | 22:47 |
| jeblair | clarkb: i've got the dns stuff in a good enough order that we are now failing on the unbound+ubuntu+dnssec+puppet bug in hpcloud. i'll try your suggested workaround. | 22:48 |
| fungi | sdague: thanks--let's see if it passes | 22:48 |
| *** _nadya_ has joined #openstack-infra | 22:49 | |
| *** andreaf has quit IRC | 22:50 | |
| *** michchap has joined #openstack-infra | 22:50 | |
| nibalizer | jeblair: is the fact comming from our code, can we just comment it out? | 22:50 |
| *** markmcclain has quit IRC | 22:51 | |
| mattoliverau | jeblair: looks like thre are feature requests in to add the funtionality to disable facts but you can't in code. Looking at the ruby code, there is a Facter.add so you can add custom facts but no delete/remove function. That being said if you want to do a dodgy, it looks like ec2 facter plugin is: /usr/lib/ruby/vendor_ruby/factor/ec2.rb which also includes vendor_ruby/factor/util/ec2.rb. | 22:51 |
| mattoliverau | nibalizer jeblair looks like it's built in to factor (a factor plugin included in the puppet package). | 22:51 |
| *** ominakov has joined #openstack-infra | 22:52 | |
| nibalizer | oh ew | 22:52 |
| openstackgerrit | A change was merged to openstack-infra/config: Add docs gate for Marconi https://review.openstack.org/80479 | 22:52 |
| mattoliverau | So I'm guessing we *could* use Factor.add in puppet to potentically override it in the factor collection, or do the dodgy and remove/move it from the source and hope nothing breaks :P | 22:53 |
| *** michchap_ has quit IRC | 22:53 | |
| *** _nadya_ has quit IRC | 22:53 | |
| nibalizer | whats the facter version? | 22:54 |
| mattoliverau | But I haven't done it before, so don't know if it breaks things.. it'll need to be tested on a test environment. Here is some examples of add custom http://is.gd/ybr6gh I wonder if you can use it to override as well, as it seems Factor is just treated as a collection (map/dict) in ruby. | 22:57 |
| *** mbacchi has quit IRC | 22:57 | |
| nibalizer | so the problem is what? facter just hangs waiting for a response from the amazon api? | 22:58 |
| mattoliverau | I can't get onto the infra servers, but I do have a puppet master in my test environment build with the infra build scripts, and there version of facter I have is 1.7.4 | 22:59 |
| jeblair | so looking at ec2.rb, i'm wondering why it's even trying... | 23:03 |
| jeblair | i don't see the arp table entry or the mac addr it's looking for | 23:03 |
| jeblair | though i have no idea why the arp table entry _isn't_ there, since there actually is a host at 169.254.169.254... | 23:03 |
| *** caleb_ has joined #openstack-infra | 23:04 | |
| jeblair | (even after i telnet to it's port 80) | 23:04 |
| jeblair | perhaps it sends gratuitous arps at boot or something and they are gone now... | 23:04 |
| *** blamar has quit IRC | 23:07 | |
| mattoliverau | yeah maybe | 23:09 |
| *** jhesketh_ has quit IRC | 23:09 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/config: Fix the virtual-ironic job's export list https://review.openstack.org/80652 | 23:13 |
| *** blamar has joined #openstack-infra | 23:13 | |
| *** ryanpetrello has joined #openstack-infra | 23:13 | |
| arosen | Hi, I saw this bug report: https://github.com/openstack/tripleo-incubator and was wondering if anyone could tell me where this is exactly happening. I don't see any tripleo support in devstack | 23:14 |
| arosen | nor do i see this configuration being done in the tripleo-incubator repo | 23:14 |
| arosen | https://bugs.launchpad.net/tripleo/+bug/1293782 whoops this bug report<---- | 23:14 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Use unbound https://review.openstack.org/80092 | 23:15 |
| arosen | nvm i think i found it. | 23:17 |
| *** dhellmann has quit IRC | 23:18 | |
| *** yamahata has quit IRC | 23:18 | |
| openstackgerrit | A change was merged to openstack-infra/config: Removing non-voting for gate-barbican-devstack-dsvm https://review.openstack.org/80686 | 23:20 |
| *** 23LAAV2X6 has joined #openstack-infra | 23:21 | |
| *** jhesketh_ has joined #openstack-infra | 23:21 | |
| *** dhellmann has joined #openstack-infra | 23:23 | |
| *** dims has quit IRC | 23:24 | |
| zaro | fungi: can we fast track these? https://review.openstack.org/#/q/status:open+project:openstack-infra/gerrit+branch:openstack/2.8.3,n,z | 23:24 |
| *** rpodolyaka1 has joined #openstack-infra | 23:24 | |
| *** HenryG has quit IRC | 23:25 | |
| zaro | fungi: those are the same changes that merged into openstack/2.8 branch | 23:25 |
| fungi | zaro: those are the reproposals of the patches which were on the 2.8 branch? | 23:25 |
| fungi | right. approving if tests pass | 23:25 |
| fungi | and seems they do | 23:25 |
| jeblair | oh i'm just approving | 23:25 |
| *** MarkAtwood has quit IRC | 23:26 | |
| zaro | thanks. | 23:26 |
| fungi | yeah, jeblair got all of them except 81061, which i just approved | 23:27 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Use unbound https://review.openstack.org/80092 | 23:27 |
| *** ominakov has quit IRC | 23:28 | |
| *** rpodolyaka1 has quit IRC | 23:29 | |
| *** thuc has quit IRC | 23:29 | |
| *** thuc has joined #openstack-infra | 23:30 | |
| *** ryanpetrello has quit IRC | 23:30 | |
| *** thuc_ has joined #openstack-infra | 23:31 | |
| arosen | Hi, I was wondering if anyone here could give me any insight on how: check-tripleo-overcloud-precise setup works. I'm trying to fix that gate which was broken by the event stuff in neutron. | 23:33 |
| arosen | I need to add a few things to the neutron.conf it's using. | 23:33 |
| arosen | Seems like it's using devstack-gate but I don't see any tripleo stuff in there. | 23:34 |
| *** thuc has quit IRC | 23:34 | |
| jeblair | sdague: when you have a sec: https://review.openstack.org/#/c/80727/ | 23:35 |
| *** reed has quit IRC | 23:37 | |
| jeblair | arosen: you can find the job definition here: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/tripleo.yaml#n80 | 23:37 |
| *** rhsu has quit IRC | 23:37 | |
| arosen | jeblair: thanks for the pointer. I'll check it out. | 23:38 |
| jeblair | arosen: in turn, that should lead you here: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_gate_test.sh | 23:38 |
| arosen | yup | 23:38 |
| *** bknudson has quit IRC | 23:39 | |
| jeblair | arosen: and if you have questions about things much deeper in the stack than that, you may want to ask in #tripleo | 23:39 |
| *** dims has joined #openstack-infra | 23:40 | |
| *** CaptTofu has quit IRC | 23:41 | |
| *** CaptTofu has joined #openstack-infra | 23:41 | |
| *** zns has quit IRC | 23:42 | |
| *** miqui_ has joined #openstack-infra | 23:42 | |
| *** ryanpetrello has joined #openstack-infra | 23:42 | |
| *** miqui has quit IRC | 23:43 | |
| *** sarob has quit IRC | 23:44 | |
| *** thuc has joined #openstack-infra | 23:45 | |
| *** sabari has quit IRC | 23:45 | |
| *** CaptTofu has quit IRC | 23:46 | |
| *** thuc_ has quit IRC | 23:46 | |
| *** thuc_ has joined #openstack-infra | 23:47 | |
| *** ryanpetrello has quit IRC | 23:47 | |
| openstackgerrit | A change was merged to openstack-infra/config: Use python-jobs template and add release jobs for solum project https://review.openstack.org/80780 | 23:47 |
| *** maxbit has quit IRC | 23:48 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add new services to solum devstack setup https://review.openstack.org/80883 | 23:48 |
| *** Ryan_Lane has quit IRC | 23:49 | |
| *** thuc has quit IRC | 23:49 | |
| *** adrian_otto has quit IRC | 23:50 | |
| *** maxbit has joined #openstack-infra | 23:53 | |
| *** blamar has quit IRC | 23:53 | |
| openstackgerrit | A change was merged to openstack-infra/config: Simplify gate jobs for python-openstacksdk and add docs https://review.openstack.org/80713 | 23:54 |
| openstackgerrit | SlickNik proposed a change to openstack/requirements: Add mockito to global_requirements https://review.openstack.org/80850 | 23:54 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack/requirements: workaround pypy in requirements https://review.openstack.org/81103 | 23:57 |
| fungi | sdague: minor correction ^ | 23:57 |
| openstackgerrit | SlickNik proposed a change to openstack/requirements: Add pexpect to global_requirements https://review.openstack.org/80849 | 23:57 |
| *** rdwrer has quit IRC | 23:58 | |
| openstackgerrit | SlickNik proposed a change to openstack/requirements: Add wsgi_intercept to global_requirements https://review.openstack.org/80851 | 23:59 |
| *** sarob has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!