20:30:09 #startmeeting requirements 20:30:10 Meeting started Wed May 15 20:30:09 2019 UTC and is due to finish in 60 minutes. The chair is prometheanfire. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:30:11 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:30:13 The meeting name has been set to 'requirements' 20:30:19 #topic rollcall 20:30:23 tonyb, prometheanfire, number80, dirk, coolsvap, toabctl, smcginnis, dhellmann ping 20:30:26 o/ 20:30:35 o/ 20:34:04 #topic Any controversies in the Queue? 20:35:06 not too bad, just need to email horizon for django (email list) 20:35:09 added to todo 20:37:10 Does that django one fall under the same category as the requests ones? 20:37:33 ya, it really does 20:37:35 dirk: ping 20:37:45 less impact though 20:38:02 Yeah same idea 20:38:23 We need to decide how proactive we're going to be there 20:38:45 https://review.opendev.org/658636 20:38:54 ignore that 20:39:27 ya, I feel like it's all or nothing. as soon as we start accepting some out of tree stuff we become 'secure' and all that bagage 20:40:54 Yeah there is a little of that 20:41:09 Still there is scope to 'play' with it and then write up a policy 20:42:22 I agree with fungi though that it's a vendor thing for the most part *but* I'm pretty sure vendors aren't doign it as well as they could *and* that us doing it is a weaker signal than we'd like :( 20:43:36 i really worry that us attempting to do it poorly ends up being a strong but dangerously misleading signal 20:44:25 honestly, if you want some semblance of a "secure" deployment from source and pypi packages, continuously deploy master and drink from the firehose 20:45:19 what if we accept updates but do not do them ourselves? 20:45:23 I wouldn't say continuous from master would be secure, but updating to latest release would be better than expecting a secure stable older branch. 20:45:30 by bot 20:45:46 heh, everyone should be on master at all times 20:46:00 prometheanfire: that's a pretty weak line to draw 20:46:13 "it wasn't us" ... just a bot we wrote and ran 20:46:17 proposed updates allowed? 20:46:27 no, I mean not us as in not bot driven 20:46:33 i'll be the first to admit that security isn't an all-or-nothing game, and selectively applying fixes for high-risk vulnerabilities can be useful, but the proposal isn't about doing that it's about upgrading the things which are easy to upgrade rather than the things which are critical to upgrade 20:46:42 if someone wants to come by and propose something.... 20:47:50 as dirk is currently doing, we'd allow that 20:47:53 If we did allow it, it has potential for vendors to rally around to watch for security issues. But that's also the argument for extended maintenance, and stable in general hasn't attracted much vendor support. 20:47:55 fungi: to be fair I don 20:48:04 and when the thing they want to propose requires patching projects in non-backward-compatible ways on stable branches to be able to use newer version whatever of some dependency... we tell them (and users who are relying on this as their only means of securnig python dependencies for openstack source deployments) "tough luck"? 20:48:04 't think we;ve formalised the proposal yet 20:48:14 Even though many of the vendors are shipping products based on those older releases. 20:48:32 I *think* it's use publically available data to inform and upgrade pypi packages with known CVEs 20:48:34 fungi: that's the only option we have right now 20:48:38 I don't know that we have moer than that 20:48:58 tonyb: I think that's the direction dirk wants to head in 20:49:00 we have the option of telling users *clearly* not to rely on this as a secure deployment model, right? 20:49:03 I *suspect* that the larger chnages or more core chnages will get resistance 20:49:16 fungi: true, that is always an option 20:49:17 fungi: Yes we can do that 20:49:23 Well, maybe for now we should just hold on any of these, then wait for someone to come up with a proposal that's at least marginally sane and maintainable. 20:49:36 okay 20:49:53 I don't want to add to dirk's workload but that sounds sane 20:50:29 dirk: Can you hash out a policy we'll discuss the *policy* on the list with wider input and then implement whatever that outcome is 20:50:33 Stable security pop up team? :) 20:50:40 i think whether or not we laggingly apply and test a handful of arbitrary dependency bumps for some external python deps, we have to tell users this isn't a safe solution 20:50:52 what sounds sane? can you add your proposal to https://etherpad.openstack.org/p/non-openstack-stable-security-constraints-updates ? 20:51:57 fungi: That 20:51:57 because 1. we won't even know about them as far in advance as most of the distros, 2. we can't guarantee we'll be able to apply them since we don't carry forks of our deps where we apply selective backports of fixes 20:52:01 s fair 20:52:06 Is there somewhere we should add a doc warning saying "These were the tested requirements at the time of release but there may be newer security fixes available that have not been thoroughly tested upstream"? Something like that to set the right expectation? 20:52:27 Or rather, have somewhere to point to when someone comes along with an issue. 20:52:34 fungi: that's why I had two lists 20:52:46 smcginnis: at min, yes 20:53:50 +1 20:54:19 yep, i think both not conflating this solution with the (intentionally frozen) upper constraints list and making it clear to users that this isn't really a significant improvement security-wise are both necessary features of any proposal, i just still question the utility of it at all 20:55:14 I'd rather not do it to be honest 20:55:17 we're basically saying "hey, we sometimes update this other list over here with some more secure dependencies, but you're still going to want to solve the harder problem of updating the ones we can't work out a safe way to update ourselves" 20:55:21 so... why bother? 20:56:10 exactly 20:56:19 the thing we'd be solvnig is the easier and less useful thing to be solved, and users still have to deal with the harder thing they needed solved that we couldn't 20:56:29 I guess I'll point dirk to the meeting notes, but he's really the one pushing it I think 20:56:44 and solving the harder problem solves the easy one anyway 20:57:22 The only ones I am concerned about are the LOCI and others like that. But that's kind of a general issue with container images. 20:58:32 at some point they're either going to need to carry patches of dependencies to secure them while keeping them working with stable branches of our software, or carry patches to stable branches of our software to work with newer dependencies 20:59:01 ya, given the way we branch it's not going to support a source based install any other way 20:59:04 Containers might be an easier argument to just upgrade to latest anyway. 20:59:19 I know that's what's told in OSA 21:01:52 anything else? I'd recommend adding stuff to https://etherpad.openstack.org/p/non-openstack-stable-security-constraints-updates for this 21:04:37 I 21:04:51 ll read it later when I have more coffee onboard 21:05:06 ok 21:05:08 ... actually looking at my todo list for today it wont happen today 21:05:10 #topic open floor 21:06:14 I'd like to understand why we can't do the python_version capping just in upper-constraints. I don't think I understand that yet, if there actually is a reason. 21:06:35 And seeing a lot of grambling in other projects about requirements breaking them now with needing to update their sphinx entries. 21:07:39 So with my understanding of how pip works, it's just fundamentally needed 21:08:00 do we have a good test case for it?] 21:08:02 what I don't get is how generate-constraints seemed to do the right thing 21:08:35 we all saw how the integration job broke without those python_version markers 21:08:40 If pip is passed the upper constraints though, it shouldn't have an issue figuring out the max version it can use. 21:08:58 I think the job broke because we didn't have the python_version flags in upper-constraints. 21:09:05 smcginnis: right *if* it's passed one we totally don 21:09:07 We just happened to fix that by putting them in g-r instead. 21:09:09 it shouldn't be even trying to figure versions out 21:09:15 t need the markers in g-r 21:09:50 tonyb: But shouldn't projects always be using upper constraints? I would consider that an issue for that project, not u-c vs g-r. 21:10:07 but *we* should need them to make our tools do the right thing 21:10:27 smcginnis: in the gate yes 21:10:51 smcginnis: but nothing actually makes them install with them 21:11:10 smcginnis: some vendors do a better job than others of staying on top of versions 21:11:26 and for most vendors they only have one version of python to deal with 21:11:48 Kind of makes our debate about raising upper constraints on stable branches moot if we're now saying they aren't used anyway. 21:11:55 smcginnis: I agree that it's be nice to drop them but without a bunch of understanding of pip I don't see we can (yet) 21:12:08 smcginnis: A little yes 21:12:17 pip still doesn't dep solve :D 21:12:23 OK, at least I understand a little better now. Thanks for explaining it. 21:12:48 Hopefully we don't hit too many of these before we can drop py2. 21:13:26 agreed 21:13:31 anything else? 21:13:32 prometheanfire: that's an over simlification 21:13:57 I have it on my todo list to look into this but it'll be at least a week 21:14:07 tonyb: I know, was just looking at https://github.com/pypa/pip/issues/988 is all 21:14:18 was looking at it earlier today actually 21:14:31 Yeah 21:14:43 was mentioned in https://github.com/kennethreitz/requests/issues/5067 21:14:55 looks like they are targeting 2.22.0 for urllib3/requests 21:15:16 ok, going to close the meeing unless someone speaks up 21:15:47 I'm good 21:15:55 I think it's probably worth writing up something for the list 21:16:03 to describe what happened last week 21:16:18 prometheanfire: do you have time to draft something on an etherpad? 21:17:01 tonyb: maybe, not tonight though 21:17:09 prometheanfire: cool 21:17:22 prometheanfire: that's still sooner than I could do it ;P 21:17:45 tomorrow at the soonest 21:19:11 cool beans 21:20:09 #endmeeting