20:29:59 #startmeeting requirements 20:30:00 Meeting started Wed Apr 11 20:29:59 2018 UTC and is due to finish in 60 minutes. The chair is prometheanfire. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:30:01 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:30:04 The meeting name has been set to 'requirements' 20:30:07 so close 20:30:13 o/ 20:30:15 #topic Roll-call 20:30:20 \o 20:30:20 tonyb, prometheanfire, number80, dirk, coolsvap, toabctl, smcginnis, dhellmann 20:30:26 \o 20:30:34 \o/ 20:30:40 o/ 20:30:47 huzzah! 20:30:54 dhellmann is in australia? 20:30:59 eastern USA 20:31:01 it's 0630 and today is a success already :) 20:31:07 lol 20:31:17 * dhellmann will return with a beverage in 30 secs 20:31:20 k 20:31:28 o/ 20:31:35 #topic Any controversies in the Queue? 20:31:41 other than dougs item 20:31:47 we should probably discuss that separately 20:32:21 Is Doug's item the OVO thing you just brought up? 20:32:26 no 20:32:34 that's not a controversy :P 20:32:55 https://review.openstack.org/558604 20:33:01 let's talk about that one 20:33:32 prometheanfire: sure 20:33:42 * dirk doesn't know the background of that one 20:33:49 is 3.26.0 a queens version of oslo.concurrency? 20:33:50 prometheanfire: it's a policy violation 20:34:11 it doesn't look like it 20:34:13 https://releases.openstack.org/queens/index.html#queens-oslo-concurrency 20:34:36 dhellmann: no it's the first rocky release 20:34:58 although that patch linked in the commit message is on queens 20:34:59 that's the only problem with it 20:35:16 this is a security-related thing? 20:35:18 the problem the change aims to fix is a security related one 20:35:19 yes 20:35:22 I get that we missed the fact that cinder *needs* 3.26.0 but we've missed it that baot has salied 20:35:39 did we release a new oslo.concurrency from queens? 20:35:50 dhellmann: a 3.25.1? 20:36:03 yeah, thats my question as well - why aren't security fixes backported anymore to queens series? 20:36:03 yeah, I guess that's what it would be 20:36:10 the fix is backported 20:36:11 tonyb: so how do we exclude security fails from stable branches? 20:36:15 they're just going about this wrong 20:36:26 they need to release from queens and then update the constraint 20:36:40 then they need to update their code to work if the parameter is not present 20:36:40 yes, that's the most optimal solution 20:36:40 do we want to allow them to require the version with the security fix on the queens branch? 20:36:43 what dhellmann said 20:36:49 3 steps 20:37:01 tonyb: can we have !=3.25.0 while updating uc to 3.25.1 then? 20:37:05 that 3rd step means we don't have to update the min version and distros don't *have* to take the new release 20:37:14 why would we block the older release? 20:37:25 dhellmann: security is the only reason I have 20:37:42 prometheanfire: No we can't havde >=$x,!=$x as that's a policy violation as we're bu,ping the minimum on a stabe; branch 20:37:54 Yeah, looks like prashkre was confused about how stable requirements are handles with that patch. 20:37:57 yeah, we don't usually update mins for "bug fixes" regardless of the nature of the bug 20:38:13 its an interesting test case for the requirements validator though 20:38:15 right, I think this is just a matter of needing to educate folks about the right process to follow 20:38:17 tonyb: for some reason I thought we did that as a way of not chaning the lower bounds but still effectivly changing the lower bounds 20:38:19 it clearly misses that point 20:38:23 prometheanfire: we can look at, given this security related we make an exception but I'm not totally convinced this qualifies 20:38:34 prometheanfire: Nope we don't 20:38:42 this is a logging security thing, right? not every deployment is going to need to worry about it. 20:38:47 * prometheanfire could have sworn that we did, but ok 20:38:59 dhellmann: true, the severity isn't the highest 20:39:04 we allow the constraint to change but we don't try to force the use of the new version 20:39:28 I'm ok with that 20:39:30 prometheanfire: If we have done it then it's a human error 20:39:48 tonyb: k 20:39:50 who's going to work with the cinder folks on this? 20:39:59 dhellmann: I'm making a comment now 20:40:13 and bnemec to get that oslo release 20:40:35 dhellmann: Yup, the only complication is for consumers with backports do they just assume they kwarg is there (normally they do) or add additional (not in master) code to handle both cases 20:40:38 I guess I'm the oslo release liaison, so I could just do that 20:40:51 tonyb : they need to not assume it is there 20:41:00 which probably means not a clean backport 20:41:13 I'm happy to propose the backport and release if bnemec and dhellmann are okay with it 20:41:33 commented 20:41:35 sure, I can +1/+2 the release 20:41:50 next 20:41:51 https://review.openstack.org/555269 20:41:53 dhellmann: cool. I'll post them today 20:41:58 can we just abandon that? 20:42:32 prometheanfire: Yup 20:42:50 done 20:42:55 prometheanfire: we don't add setuptools to u-c as manytimes it's a bootstrap problem 20:43:08 ya, I commented as such 20:43:17 now if only gertty would catch up 20:43:25 also newton is EOL so we're closed for business there 20:43:48 tonyb: this one's yours 20:43:49 https://review.openstack.org/551111 20:44:28 prometheanfire: Oh yeah I need to rebase it now the master version has mereged 20:44:38 I'll do that today 20:44:40 k 20:45:04 #topic remove lower bounds from global requirements 20:45:10 #link https://review.openstack.org/560108 20:45:14 dhellmann: you're up 20:45:21 also 55 11 11 is a cool change number ;P 20:45:32 tonyb: indeed 20:45:39 I was envious when I saw it 20:45:49 ahh small things :) 20:45:49 right, so we're almost done with the changes to allow lower bound divergence and add lower-constraints tests 20:46:06 the one thing that's left is that we have a few projects, swift notably among them, that still have lower lower bounds than g-r has 20:46:11 and different exclusions 20:46:22 I'm trying to sync those up so we can get the lower-constraints job in place 20:46:36 and as I point out in my comment on the bug, I see 4 options 20:46:45 1. Force projects to update their minimums. I include this for completeness, but think it's a terrible idea. 20:46:50 2. Force projects to drop those exclusions. I also think this is a terrible idea and unlikely to result in wide-spread adoption. 20:46:55 3. Add exclusions to the global-requirements list that are lower than the >= value. I don't know if anything else will complain about this, but it will certainly look odd if it does work. 20:47:00 4. Remove the minimum values here. These values are demonstrably wrong, since I have had to update the lower-constraints patches in many cases to use different values. We also know that some projects support lower versions than are specified here, and we want to allow that. 20:47:12 I went with the last option, because it felt like it made the most sense 20:47:38 prometheanfire indicated that option 3 might be preferred, and I could live with that, but it's going to look weird 20:48:04 dhellmann: my prefrence is for 4, but fall back to 3 if we still require tracking the minimums for some reason I don't know of 20:48:07 dhellmann: what do you mean by demonstratably wrong? 20:48:08 I would like to resolve this so I can move off of this work, because I do have other things I need to do this cycle 20:48:33 dirk : I mean that given the number of times I've had to change the lower-constraints.txt file in projects based on the original list, those lower bounds are higher than they need to be in a lot of cases. 20:49:04 dhellmann: a patch doing option 3 would be intreresting to test though (just to see what breaks) 20:49:08 and also that as we go forward in time, projects are going to drift apart from each other and from that list 20:49:10 dhellmann: ah, ok. so its the case where a project supports an older version than the g-r min version is. ok 20:49:16 dirk : yes, that's right 20:49:21 dhellmann: but those older versions wouldn't be coinstallable 20:49:37 dirk : maybe. I do not have a goal of making the lower bounds co-installable. 20:49:48 and I think that is the source of a lot of the conflict and confusion over these changes 20:50:06 I want the values in each project repo to be accurate. I don't care if they're the same. 20:50:31 We are providing a list of co-installable versions. Other people may want to produce other lists. I don't think the community needs to maintain multiple lists. 20:50:50 If we *do* maintain multiple lists, I think we need to do it in a way that does not add friction to the project teams. 20:50:55 it might be all obvious but I havent' been able to find the time to closely look at what has been done 20:50:59 and I think trying to maintain a single central list does add too much friction 20:51:12 are the projects running sensible tests against their in-project lower constraints? ike a functional test suite? or unit tests? 20:51:26 for now most of them are just running unit tests 20:51:38 are they all at least running unit tests? 20:51:48 Matthew Thode proposed openstack/requirements master: Add swift exclusions https://review.openstack.org/560636 20:51:58 dirk: unit tests but it's totally doable to add funcational tests 20:52:19 some are. not all have landed the job. not all of the jobs are working 20:52:24 https://review.openstack.org/#/q/status:open++topic:requirements-stop-syncing 20:52:43 most of the major projects have added the unit test job 20:53:22 distros don't care that openstack is co-installable for the lower version if they have multiple versions available to install 20:53:36 distros should install what is co-installable if they can (dep solving is there for a reason) 20:53:43 at least that's my perspective from gentoo 20:53:49 prometheanfire: I'm not sure that's dirk's POV with hsi distro hat on 20:53:52 right, I suspect we could have a different set of co-installable versions from each vendor. 20:53:57 ya, that's why I said gentoo :p 20:54:13 well, I respect the right of distros to use different versions. I do not think it is our job to test those versions for them. 20:54:27 prometheanfire: in almost all cases the uc version isn't available yet due to $reasons 20:54:34 because uc updates within hours of the upstream release 20:54:37 this is an area where we could completely consume all community testing resources (human and bot) to satisfy this need 20:54:56 and I don't think the *community* benefit warrants doing that 20:55:07 dhellmann: agreed (Openstack isn't responsible for disto testing) 20:55:20 I also don't think these changes make that work any harder 20:55:28 I agree to the latter part 20:55:40 removing the lower bounds from g-r.txt doesn't change anything in that area 20:55:51 dirk: really? 20:56:06 it would be relatively straightforward to build a tool to read lower-constraints.txt files from a set of repos and build a "highest min" set of projects 20:56:28 dirk: I thought you wanted to maintain a global-maximum list of minimums 20:56:31 and *that* set would be (a) accurate and (b) targeted to the projects being packaged 20:56:38 tonyb: what am I missing? remember it is very late here and I had a very long and stressful day :) 20:56:43 and removeing the minimums from g-r makes that *much* harder 20:56:54 tonyb : no, it doesn't, because those numbers are *already wrong* 20:56:59 dhellmann: that's what I've been wanting to track in a separate file 20:57:04 pull vs push the min 20:57:13 tonyb: but those numbers are tracked in the lower-constraints.txt in the requirements repo already? 20:57:14 but that's a separate project 20:57:17 I'm not sure why we need to check the results of that file in, but ok 20:57:18 right now they're just duplicated 20:57:22 dirk : exactly 20:57:51 dhellmann: we could generate it from the sum of projects indeed 20:57:58 dhellmann: Yes they're wrong I don't deny that 20:58:09 dhellmann: just as an aid to deployers, like you said it's easy to generate we could even not track the file ourselves and just give deployers the tool + instructions on how to use it 20:58:16 dirk : right. that would let other people track the values for you, and you could consume them as a "point in time" set of values when you need them 20:58:31 dhellmann: but that's ok while we fix them Stien+ if we set expectations 20:58:32 prometheanfire : sure 20:58:50 tonyb : the "fix" is going to be to lower a bunch of values. Are you ok with that? 20:58:53 dirk: If you're okay with that outcome then I'm fine with it 20:58:58 so, about your patch, I voted 20:59:12 dirk: but we ruled that out in Denver ... I don't recall why :/ 20:59:23 I think we are all ok with option 4 20:59:31 which is what his patch represents 20:59:34 dhellmann: I'm not sure that's the case 20:59:44 tonyb: I'm certainly not able to recall that at this time of the day anymore either 21:00:25 dhellmann: I've been quiet as I'm working through a question "is it wrong for $project to have a lower minimum that g-r" 21:00:44 yeah, exactly 21:00:45 dirk : maybe tomorrow (or another day) during more normal hours you could write up what you'd like to be able to do with a "highest min" set of values and send that to the ML, and then I can try to respond there 21:00:55 many of those subtleties in all the changes that we did are not clear to me 21:01:01 dhellmann: I *think* that's okay as the minimums in g-r were s'posed to represent the global minimums 21:01:02 tonyb : we already have projects that are like that and it causes no issues 21:01:03 tonyb: individually no, I don't think it's a problem 21:01:21 I'm trying to enable 2 new cases 21:01:25 tonyb: that's my understanding as well 21:01:35 1. Separate installations of a service without a bunch of newer dependencies 21:01:47 dhellmann: I think thats in the README.rst alread of the repo. basically integration test rather than unit test of the lower constraints of the core projects 21:01:48 2. Stable libraries appearing stable because their requirements don't change automatically 21:02:20 dirk : ok. I think to do that you do need to have a way to produce a "highest min" set of values, but you want it for the projects being tested not the global list 21:02:31 dhellmann: in many cases from what I've seen complicated external deps are mocked in unit tests so running unit tests against lower constraints does not actually tell that the lower requirements would work outside a unit test env 21:02:36 because projects.txt includes *tons* of things in it that are probably irrelevant for your distribution 21:02:48 I agree, that's likely the case 21:03:20 whether we track a set of values centrally or not won't change that 21:03:40 dhellmann: right, aiming a full global lower-constraints is too much. starting with a $set of projects and combining theirs lower-constrains might be better 21:03:54 but I wonder how we would want to resolve conflict in a tool. so thats why you still end up maintaining a file 21:03:54 dhellmann: No, but the central lower-constraints.txt was supposed to make an integrated run "easy" 21:04:00 if we can stabilize what we have now, and have those unit tests running, that should give project teams the confidence to set up functional test jobs 21:04:08 and we could make that a goal for stein or T 21:04:15 we should be figuring out what the highest global min is (if we care to) not prescribing what it is 21:04:22 so removal makes sense to me 21:04:33 tonyb : it makes it slightly easier to run the job at the expense of making it much harder to fix the list when the job fails 21:04:41 well, slightly harder 21:05:26 dhellmann: Yeah for me it was an acceptable level of pain ... but my setpint is probably off 21:06:07 dhellmann: I guess I'll have to start a test job that runs against the lc to see how bad it is to get it to work 21:06:10 I'm trying to reduce the number of patches we need to make to the requirements repo, to cut load on this team and to reduce friction for project teams who want to make changes in their own repo 21:06:10 anyway I *think* we've all agreed that er can remove the minimums in g-r and them's that care can maintain the lower-constratints.txt in openstack/requirements 21:06:26 yep, +1 21:06:26 is that the simple take away? 21:06:30 yes 21:06:33 exactly 21:06:42 one step at a time, we'll all be more wise tomorrow 21:06:51 yeah, I don't mind keeping the lower-constraints.txt file (although I reserve the right to change my mind if it causes problems later) 21:07:00 lol 21:07:43 dhellmann: ;P that's pretty much the std. disclaimer on IRC ;P 21:07:47 dhellmann: I'd really like to understand the problems you'e been hitting more concretely btw, so if you could tell me obvious take aways that you learned while doing that work in a more detailed fashion, I'd be happy to listen 21:08:12 once we have a tool to build that file from a bunch of repos, then the question is just when it makes sense to run it -- on a merge that changes lower-constraints in a repo, or inside the lower bounds integration job when it runs 21:08:15 (doesn't have to be now) 21:08:17 okay so we did reach consensus. 21:08:31 open floor next? 21:08:49 dirk : I had to use https://review.openstack.org/#/c/558610/ to fix a bunch of the original lower-constraints.txt files in various repos 21:09:33 so I think the global lower-constraints file is a "highest min" but within each project I needed the *actual* min 21:09:59 #topic open floor 21:10:02 and if those are already different, then we've already drifted apart (yay!) but that means the global values are already not right 21:10:12 thank you all for hashing this out one more time 21:10:26 dhellmann: right, I agree the highest-min currently are also actually likely a bit higher then the actually lowest highest min 21:11:00 * dhellmann is getting confused with the conflicting highest-lowest-highest qualifiers 21:11:02 :-) 21:11:15 dhellmann: Yeah it's always a problem ;P 21:11:15 yeah 21:11:18 this is tricky 21:11:29 lowest-coinstallable-set? 21:11:41 yes 21:11:44 dhellmann: took me a while to get used to 21:11:52 maybe that's a better name then 21:12:00 the global highest min may not be co-installable 21:12:18 project-b may have a != on it or something 21:12:24 lowest-coinstallable-set-that-may-work-but-probably-not.txt ;P 21:12:32 or is that a little passive aggressive ? 21:12:34 tonyb: yes, lets use that :D 21:12:38 prometheanfire : no, because we enforce that projects cannot exclude things that are not also excluded in the global list 21:12:39 no, I like it :D 21:12:50 dhellmann: true, so.. should be good 21:12:54 tonyb : just-dont-even-try-it.txt? 21:13:06 maybe that's too aggressive :-) 21:13:10 dhellmann: #winner ;P 21:13:19 anyone have any other topics before I close? 21:13:42 can you remind us about other prios other than no req sync? 21:13:58 I've just seen that py36 patch that I neglected to followup 21:14:12 dirk: what do you mean? other things to do this cycle? 21:14:19 yes 21:14:27 or upcoming things to worry about 21:14:33 uncapping and py36 are the main things I guess 21:14:41 dirk: Yeah we need to sort out and test py36 this cycle 21:14:42 we have https://review.openstack.org/#/q/topic:uncap-eventlet 21:14:45 if you are working on p36 I can focus on uncapping eventlet 21:14:48 dhellmann: ++ 21:15:44 anyone have anything else? 21:15:45 it looks liekt he cross job worked but neutron and some others failed on py36 21:16:04 well, I guess finding-sensible-goal-for-dont-try-it-txt :-) 21:16:05 I'd like to work out if we can switch *constratints.txt to use <= and > in python_version markers 21:16:13 what are we doing with 3.6? that seems like a community-wide thing 21:16:37 providing+generating a py36 compatible uc 21:16:40 tonyb : is that related to my hacky evaluation logic? 21:16:54 tonyb: that's a nice to have thing imo 21:16:58 dhellmann: There are some aspects we need to nail down before the community can make meaningful progress 21:16:59 dirk : so we don't just assume what we have now works and let project teams tell us otherwise when their jobs fail? 21:17:42 dhellmann: partially but I really don't like that we basically have ==3.4 ; ==3.5 and ==3.6 21:17:43 the next thing I was going to get to work on was updating some secondary jobs like docs and release notes to run on python 3, but I was just going to say "3" not "3.5" or "3.6" 21:17:49 dhellmann: the principle of g-r so far was "we know it better than you" ;-) 21:18:01 now with the g-r sync disabled I guess we have to find a new mission 21:18:06 dhellmann: it's just wrong and we're basically making up thr 3.4 and 3.6 data anyway 21:18:08 dhellmann: the review I was talking about is https://review.openstack.org/#/c/554824/ 21:18:11 tonyb : sure. maybe the thing to do is get the packaging library to expose the resolver in a better way so we can reuse it 21:18:23 dirk: we record the wisdom of the masses :P 21:18:24 right now it assumes no one is going to pass environment values in, but we need to be able to do that 21:18:45 dirk : I think this team is too small and overworked to stick with that "we know better" policy :-/ 21:18:59 * dirk isn't that small 21:19:05 dhellmann: I'm not sure what we can get out of packaging to make this better 21:19:17 dhellmann: but if you have ideas I'm open 21:19:19 * prometheanfire fits neatly into overhead bins 21:19:28 I think the most valuable thing to do is maintain the g-r list in terms of licenses and "goodness" and to start encouraging teams to reduce the number of redundant dependencies we have 21:19:45 dhellmann: that's a constant work in progress 21:19:58 tonyb : they have a thing to evaluate the environment markers against the current environment. we just need a version of that where we can pass environment values into it instead of having it compute them for us 21:20:06 dhellmann: yeah, dropping redundant/bad dependencies (e.g. not py3 comptible) would be a good goal to work on 21:20:16 if you're working for a distro then openstack is still a major pain 21:20:21 so we can say "we're going to pretend to be python 2.7 even though we're 3.5, does this rule evaluate to true?" 21:20:47 dhellmann: Hmmm I'm not sure that'd help as we don't have a system that has py3.4, py3.5 and py3.6 on it to get real data 21:20:48 I think *many* people would appreciate reducing redundant dependencies as a goal. I'm not sure many developers would. :-) 21:21:03 dhellmann++ 21:21:29 dhellmann: usually noone minds if somebody else does the work.. 21:21:40 tonyb : I think I worked out that we don't need all of those values. We only need to be able to say "do these two sets of environment markers evaluate as compatible" 21:21:48 it was so far never a problem to cleanup stuff if you spent the time to figure out how to port it from one tech to another tech 21:21:50 that's what I'm finding with the cryptography migration... 21:22:10 dirk: that's partially true by $project now needs to grok as some level $new_lib instead of $old_lib and that's a pain point 21:22:33 that was one of the benefits of having thin wrappers for some things in oslo; we could change the underlying implementation without having to change every app 21:22:53 tonyb: are we over complexifying the problem? 21:23:07 dirk: I don't think so 21:23:15 tonyb: wouldn't it be enough to let the proposal bot to suggest something and then we test it on some other nodepool slave with the $right python version whether that is satisfactory? 21:23:43 I m not sure many libs have conflicting dependencies for individual py3.x versions 21:23:47 dirk: I don't think that works. 21:23:57 ok, tell me why? 21:24:25 I know botocore has problems with py3.3, but that's it (and couldn't understand the PDT timezone in 3.6 for some reason) 21:24:37 dirk: It's not really about conflicting dependencies, it's about generating a constraints file that's valid and actually constrains things 21:25:06 dirk: And do you really want to start setting up meta reviews? 21:25:06 right, validity checks can be done in our existing check-uc job 21:25:14 why are we using == for several versions now instead of >='3.5'? 21:25:33 dhellmann: That's the root of the question 21:26:12 maybe we should just change the list and see what we end up with 21:26:28 dhellmann: it could be that the answer is "because lifeless made a mistake, we approved it and ran with it", but it could equally be "lifeless considered things from $y angle and we've forgotten it" 21:26:38 yeah 21:27:08 I suspect that when we switch to 3.6 we're not going to need to worry about supporting 3.5 any more 21:27:15 we can continue discussing this after the meeting, we are coming up on time 21:27:18 until we are off of 2.7 I don't see us supporting more than one version of 3.x 21:27:23 ok 21:27:30 splitting on <3.$x where $x would vary based on out CI environment (4 for xenial 6 for biaonic) is right 21:27:41 it's certainly simplier 21:27:45 tonyb++ 21:27:48 I guess we need to ask lifeless 21:27:57 we don't need to support 3.4 though 21:27:59 that's my understanding as why we need multiple py3 versions as well 21:28:07 at least not on master 21:28:09 #endmeeting