20:43:52 <prometheanfire> #startmeeting requirements
20:43:53 <openstack> Meeting started Wed Aug 29 20:43:52 2018 UTC and is due to finish in 60 minutes.  The chair is prometheanfire. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:43:54 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:43:56 <openstack> The meeting name has been set to 'requirements'
20:44:01 <prometheanfire> #topic rollcall
20:44:07 <prometheanfire> tonyb, prometheanfire, number80, dirk, coolsvap, toabctl, smcginnis, dhellmann ping
20:44:10 <prometheanfire> o/
20:44:25 <tonyb> \o
20:44:44 <dhellmann> o/
20:45:51 <prometheanfire> #topic Any controversies in the Queue?
20:46:02 <prometheanfire> eventlet timing
20:46:15 <prometheanfire> release is soon, so I'd say early next week?
20:46:54 <dhellmann> we'll tag the final release tomorrow
20:47:23 * tonyb would suggest waiting until the cycle-trailing projects have released or at the very least validating that they're pulling in rocky barnches of the $projects and not using eventlet themselves
20:47:47 <tonyb> s/would suggest/suggests/
20:47:50 <prometheanfire> iirc tripleo was/is always the holdout
20:48:46 <tonyb> prometheanfire: They're often late to branch
20:48:49 <prometheanfire> and they've branched
20:48:58 <tonyb> great
20:49:12 <tonyb> they're not the only cycle-trailing team
20:49:26 <prometheanfire> I'll send the email tomorrow after the release stating monday is the day
20:49:35 <tonyb> we had a sctipt in gerrit to work this out
20:49:40 <prometheanfire> they aren't
20:50:10 <tonyb> prometheanfire: As long as you do the research as part of that email ok
20:50:15 <prometheanfire> yep
20:50:16 * smcginnis walks in late
20:50:36 <prometheanfire> smcginnis: any other yet to branch projects?
20:50:51 <prometheanfire> kolla is one
20:51:17 <smcginnis> There are some ansible repos, heat, one tripleo.
20:51:26 <prometheanfire> heat still hasn't branched?
20:51:29 <smcginnis> The others looks like tempest-plugins or QA non-branching stuff.
20:51:41 <smcginnis> heat-agents and heat-dashboard have not.
20:51:45 <prometheanfire> oh, ya
20:51:54 <prometheanfire> I thought you meant the main project
20:52:18 <smcginnis> Nope, all main service projects are good.
20:52:21 <prometheanfire> tonyb: I've used the release repo tooling to check branching
20:52:40 <prometheanfire> ok, anything else on the eventlet plan?
20:52:59 <tonyb> prometheanfire: Great.  As long as it's communicated it sounds ok to me
20:53:27 <prometheanfire> ya, there will be a list of yet-to-brach repos
20:54:28 <tonyb> and an idea of the impact to each
20:54:43 <prometheanfire> #topic moving projects.txt under governance
20:54:47 <prometheanfire> my reasoning is here https://gist.github.com/prometheanfire/d2b9302b2e6574b91fa3d0399b20d5b2
20:55:12 <tonyb> prometheanfire: gimme a few to read that
20:55:40 <dhellmann> while tonyb reads, I will point out that we don't do similar enforcement for any of the other PTI jobs
20:56:07 <dhellmann> so this would be a new thing, and we talked about ways to implement it but we should also talk about whether we need to do that or if we should just add it to the PTI
20:56:37 <prometheanfire> PTI?
20:56:48 <dhellmann> "project testing interface"
20:56:57 <tonyb> ZOMG that hurts my brain
20:56:59 <prometheanfire> smcginnis: https://releases.openstack.org/reference/ has no index
20:57:00 <dhellmann> we could, for example, simply expand https://governance.openstack.org/tc/reference/pti/python.html#requirements-listing to talk about the job template that projects should adopt
20:57:21 <smcginnis> prometheanfire: Does it need one?
20:57:25 <smcginnis> PTI - https://governance.openstack.org/tc/reference/project-testing-interface.html
20:58:04 <smcginnis> This is the reference section navigation point - https://releases.openstack.org/#references
20:58:05 <prometheanfire> dhellmann: I'm of the opinion that the test should be enforced, I've seen projects try to break co-installability in the past
20:58:16 <prometheanfire> smcginnis: up to you
20:58:25 <dhellmann> "try to break" or "accidentally break"?
20:59:13 <prometheanfire> second one, but when told about it probably don't care as much as they should
20:59:16 <tonyb> Actually I think a) we move projects.txt to project-config; b) we remove tox-validate; c) rely of projects using requierments-0check from in-repo and d) add a check at release-time that a projects *requirements.txt are still good
20:59:56 <dhellmann> what would we do with projects.txt in project-config?
20:59:57 <tonyb> this moves the trust circle onto the project, it does add small amounts of work for the release team but really it's just +/1- from zuul and an cmment doe a core member
21:00:30 <tonyb> well it's still used by $jobs there so it's pulling it closer to the use and we can slowly phase it out
21:00:57 <dhellmann> and what pressure do we use to keep it up to date?
21:01:11 <prometheanfire> tonyb: what happens when a project adds a lib (or changed version or whatever) that fails the release check
21:01:29 <prometheanfire> time crunch + potentially massive code change doesn't sound nice
21:01:35 <tonyb> prometheanfire: they get a -1 from zull ate release time and need to fix it
21:02:05 <dhellmann> if we're going to check it at release time, I think the check would be that the requirements-check job is applied to the repository
21:02:12 <tonyb> dhellmann: I'm not sure we need to keep it in sync anymore
21:02:12 <dhellmann> that way we don't have to reproduce that job
21:02:29 <dhellmann> tonyb : ok, me either. I feel like this is another big opportunity to stop doing something.
21:02:33 <prometheanfire> dhellmann: applied at the commit in question, that still leaves a hole though
21:02:44 <tonyb> dhellmann: Sure
21:02:44 <dhellmann> prometheanfire : nothing is going to be perfect, right?
21:02:47 <prometheanfire> just adding the job doesn't trigger it
21:02:54 <prometheanfire> true
21:03:10 <prometheanfire> perfect is the enemy of good and all that
21:03:15 <dhellmann> we don't run any other test jobs against the repo at the time of release, and I don't think we want to start trying to do that
21:03:34 <tonyb> dhellmann: Yup.  There are a couple of places that use it "for real" but the main consumer requirements updates is gone
21:03:42 <dhellmann> those validation checks right now are "are you set up right" not "does everything work as promised"
21:04:20 <tonyb> dhellmann: Yup I'm not really convinced there is a ton of value in doing it at release time but I feel like we want a poin tin ttime check
21:05:05 <dhellmann> yeah, I don't mind checking for the job to be configured
21:05:06 <smcginnis> We just don't want the release time to be the only time where they find out something is wrong.
21:05:08 <prometheanfire> have it somewhere
21:05:13 <smcginnis> That could be months after the problem was introduced.
21:05:26 <dhellmann> true
21:05:26 <prometheanfire> smcginnis: that's my worry
21:05:33 <smcginnis> So checking for the job is good. Checking that things work would be problematic.
21:05:53 <tonyb> smcginnis: Ture but it's a problem that $project team would have taken explict steps to introduce
21:07:11 <prometheanfire> remove job, add bad lib, ..., readd job, release
21:07:21 <prometheanfire> that's how you work around it
21:07:39 <tonyb> prometheanfire: right
21:07:42 <dhellmann> the way to ensure that everyone runs the check job is to explain why that's important well enough and often enough to convince them to do it. We don't want to just hammer a bunch of rules onto teams without their understanding -- that just leads to them subverting the inconvenient rules
21:08:04 <prometheanfire> true
21:08:27 <prometheanfire> and name/shame the teams that do it :P
21:08:55 <dhellmann> that's not really a way to convince them either
21:09:03 <prometheanfire> one thing to do would be for the reqs team to submit a noop change to all projects reqs files to check their gate
21:09:14 <prometheanfire> dhellmann: it's not, true
21:09:14 <tonyb> at the moment what we have works only because you need to talk to project-config-cores to do that step (removing the job)
21:09:33 <dhellmann> prometheanfire : it's possible to run that test without submitting a patch, using a tox environment in the requirements repo
21:09:51 <dhellmann> step 1 should be to figure out which repos actually need the job, and then to look at that list and see which ones don't
21:10:08 <dhellmann> and then we can go talk to those teams about why they should and get them to add it
21:10:10 <prometheanfire> all repos that wish to be co-installable need the job
21:10:13 <tonyb> moving the file will be a big job as we need to deal with brched and branchless repos etc
21:10:34 <dhellmann> prometheanfire : yes, but we need to be more specific about what that list is. for example, earlier you seemed surprised when I said the specs repos did not need it
21:10:51 <dhellmann> tonyb : why do we need to move the file then?
21:10:53 <prometheanfire> I think my surprise was about something else, not specs
21:11:04 <dhellmann> oh, ok, I misunderstood then
21:11:23 <tonyb> dhellmann: we don't *need* to move it.  I think we probably just need to stop updating it
21:11:50 <dhellmann> tonyb : ok, that sounds better. and maybe turn off that validation job that's failing because the file is out of date. :-)
21:12:01 <tonyb> I think we're going around in circles a little
21:12:06 <prometheanfire> so do we not care about co-installability?
21:12:16 <prometheanfire> that's the reason we validate that the jobs exist right?
21:12:18 <dhellmann> ?
21:12:18 <tonyb> the file is used in "intersting" wasy by virtue of what it once meant
21:13:39 <dhellmann> prometheanfire : we care about the code working, too, but we don't have anything in place that forces teams to run specific test jobs because we don't need that. they understand why those jobs are useful.
21:13:42 <tonyb> dhellmann: I'm not even sure anymore :/
21:14:04 <dhellmann> tonyb : so does that mean we *should* update the file?
21:14:21 <prometheanfire> projects are less concerned about co-installability anymore (venvs, containers, etc)
21:14:36 <dhellmann> I'm not suggesting that convincing them is going to be easy, just that it needs to be done.
21:14:54 <tonyb> dhellmann: I don't think so.  I guess we need to look at all the non-trivial uses of thew file and decide if it still has value or not
21:14:54 <prometheanfire> I agree, but it only takes one
21:15:15 <dhellmann> we're not going to achieve perfection
21:15:36 <tonyb> prometheanfire: I'm not sure we really need to worry too much about projects teams doing the clearly bad thing
21:15:47 <prometheanfire> clear to us, not to them
21:15:48 * tonyb wants to err on thre side of trust here
21:16:28 <prometheanfire> I'm not as trusting
21:16:40 <prometheanfire> it does come down to that though
21:16:46 <tonyb> prometheanfire: If $project does the disble/update/enable dance the *next* time they want to update that file for reall they'll also need to do that dance and *every* subsequent time
21:16:59 <prometheanfire> yep
21:17:02 <tonyb> that's pretty clearly a bad thing
21:17:04 <dhellmann> yeah, the point of this team is to help the project teams achieve a good release, not build a bunch of rules without explaining them.
21:17:14 <prometheanfire> not saying it'd happen like that
21:17:15 <tonyb> dhellmann: ++
21:17:34 <prometheanfire> never said they wouldn't be explained :|
21:17:41 <dhellmann> I believe we can have all of the necessary repos running the test job without forcing things
21:17:49 <tonyb> Yup
21:18:23 <prometheanfire> if ia good release means X, and X requires Y and Z then ensuring that Y and Z are actually Y and Z is a good idea
21:18:27 <tonyb> like I said I want to err on the side of trust here
21:18:28 <dhellmann> prometheanfire : ok. I'm reacting to what seems like your impression that teams want to *not* have co-installability.  I don't think that's true. I think some don't understand what it is, or why they should care.
21:19:00 <prometheanfire> dhellmann: I think what you think leads to what I think
21:19:18 <dhellmann> right, so it's *our* job to explain things to them and convince them
21:19:58 <dhellmann> it is not our job to force anyone to do anything
21:20:17 <prometheanfire> tonyb: which release types would get the check-reqs-check job?
21:20:17 * tonyb will write up something for the mailing list
21:20:35 <tonyb> prometheanfire: what is check-reqs-check ?
21:20:44 <prometheanfire> check requiremets-check
21:21:09 <prometheanfire> because the name of the job is check-requirements (I think) and we are checking for it's existance
21:21:15 <tonyb> check-requirements ?
21:21:19 <prometheanfire> just to add more confusion
21:21:51 <tonyb> check-requirements: is the job that any project that wants to subscribe to our 'vetted' list of libraries would run
21:21:56 <dhellmann> what is it we want to achieve?
21:22:41 <prometheanfire> tonyb: it also helps to ensure co-installability by denying the addition of 'bad libs'
21:22:43 <tonyb> check-requirements: would be managed in repo as per PTI and is the job that project teams would remove/disbale if they wanted to opt out of that list or libraries
21:23:05 <tonyb> prometheanfire: where does it do that?
21:23:29 <tonyb> prometheanfire: it ensures consistency between *requiremenst and lower-constraints.txt
21:23:29 <dhellmann> tonyb : it requires that all of the things in the dependency list appear in the global list
21:23:38 <dhellmann> no, that's a different job
21:23:51 <dhellmann> oh, wait it does do that
21:23:54 <prometheanfire> tonyb: part of what the job does is check for playbooks/files/project-requirements-change.py
21:23:56 <tonyb> dhellmann: right but that's less about co-installability and more about library vetting at this point
21:24:18 <dhellmann> it doesn't test with those constraints, that's a different job
21:24:32 <dhellmann> yeah, check-requirements matches the local dependency lists with the global dependency list, and with the global upper constraints and the local lower constraints
21:24:41 <prometheanfire> yep
21:24:55 <dhellmann> if there is a local lower constraints list
21:25:44 <tonyb> wheer does it do 'and with the global upper constraints'
21:26:34 <prometheanfire> I thought it just checked the requirements files
21:26:58 <tonyb> I'm under the impression that our check-uc, openstack-tox* and *dsvm* jobs do that for us
21:27:05 <dhellmann> http://git.openstack.org/cgit/openstack/requirements/tree/openstack_requirements/check.py#n151
21:28:07 <dhellmann> oh, sorry, that's just the dependency list not the constraints
21:28:13 <dhellmann> hmm
21:28:13 <tonyb> prometheanfire: IIUC it checks a projects *requiremenst.txt only lists libraries in g-r *and* if the project has a lower-constratins.txt it checks that the value in that file matches the one in the projects *requirements.tx
21:29:07 <dhellmann> it does also check that the local exclusions are a subset of the global exclusions
21:29:11 <tonyb> so with that understanding that job is ensuring self-consistency and that projects are only using 'vetted' code
21:29:19 <prometheanfire> ah
21:29:23 <tonyb> dhellmann: True I forgot that part
21:29:38 <dhellmann> but you're right that it does not explicitly check against the u-c list
21:29:40 <tonyb> so again that consistency and vetting not co-installability
21:29:55 <dhellmann> well, we get it commutatively
21:30:01 <prometheanfire> isn't that a prereq for co-installability?
21:30:19 <dhellmann> if a projects dependencies are compatible with the global list, and the global list is compatible with the constraint, then the projects' dependencies are compatible with the constraint
21:30:20 <tonyb> prometheanfire: not really
21:30:49 <prometheanfire> dhellmann: right
21:31:10 <tonyb> dhellmann: okay that's fair I stand corrected
21:31:16 <prometheanfire> dhellmann: that's basically my reasoning
21:31:58 <dhellmann> I think that's actually transitive, not commutative
21:32:04 <dhellmann> it has been a long time since I studied math :-)
21:32:08 <tonyb> dhellmann: it is but meh
21:32:10 <prometheanfire> same
21:33:04 <tonyb> So I think after 45mins we've decided this is complex and we need to dicuss it in the community
21:33:17 <dhellmann> heh
21:33:20 <prometheanfire> lol
21:33:31 <prometheanfire> true, that was always the case
21:33:43 <dhellmann> I don't feel like anything has really changed about this except that we should make sure the requirements repo is not blocking changes because projects.txt is out of date
21:33:46 <tonyb> I feel like projects.txt isn't really part of that discussion as we no longer consult it from a requiremenst POV
21:33:50 <prometheanfire> letting them know that they should use the job for x y z reasons
21:34:02 <dhellmann> right
21:34:22 <prometheanfire> ya, the real 'source of truth' now is project-config
21:34:34 <prometheanfire> and relying on jobs not being removed
21:34:39 <tonyb> prometheanfire: no
21:34:47 <dhellmann> project-config?
21:34:49 <tonyb> prometheanfire: the source of truth now is zuul
21:35:04 <tonyb> (and the as yet un merged API we were promised)
21:35:20 <prometheanfire> ?
21:35:21 <smcginnis> I thought there was some progress on that.
21:35:26 <prometheanfire> now I'm confused
21:35:42 <dhellmann> there's an API to fetch the settings for a job, but they do not include which repos it's attached to
21:35:44 <tonyb> smcginnis: you can query a job defn. but not get a list of jobs for $project
21:35:49 <tonyb> ... at least alst time I checked
21:35:51 <prometheanfire> the check-requirements job defined in project config for various projects is considered valuable still right?
21:35:53 <dhellmann> prometheanfire : we are working on moving most of the zuul settings out of the project-config repository
21:36:13 <dhellmann> the job is valuable. we are moving that setting into the repos, though.
21:36:15 <tonyb> prometheanfire: No thre PTI changed and $projecst can move it in-repo
21:36:18 <smcginnis> Maybe getting this all written in a post to the ML would help.
21:36:29 <prometheanfire> dhellmann: I'm aware, iirc we kept the job in project-config because it's harder to remove
21:36:36 * tonyb notes hes' said he'd do that 3 times ;p
21:36:59 <tonyb> prometheanfire: nope we agreed to move it
21:36:59 <dhellmann> that is not one of the jobs the scripts are leaving behind
21:37:25 <prometheanfire> ok, guess we just assume it's going to work in the end
21:37:30 <smcginnis> tonyb: I was trying to get it back around to that. ;)
21:37:30 <dhellmann> http://git.openstack.org/cgit/openstack/goal-tools/tree/goal_tools/python3_first/jobs.py#n22
21:39:15 <tonyb> Okay we're 7mins over now
21:39:56 <prometheanfire> we don't have anything for verifying our upper-constraints is used either, so I guess veriifcation is overblown
21:40:04 <prometheanfire> trust but verify is just trust now
21:40:15 <prometheanfire> tonyb: ya, I don't have anything else for the meeting
21:40:19 <dhellmann> teams understand the value of the uc list because it helps protect their gate jobs
21:40:24 <prometheanfire> yep
21:41:06 <smcginnis> We could maybe add a check that their tox.ini has something like http://git.openstack.org/cgit/openstack/cinder/tree/tox.ini#n15
21:41:55 <tonyb> smcginnis: Perhaps but there are so many other places it's used/needed and some projects have deliberatly opt-out of u-c
21:41:56 <prometheanfire> ya, but at this point we are beyond the meeting
21:42:02 <prometheanfire> can carry this on after/later
21:42:19 * tonyb need to breakfast and get kids to school
21:42:21 <prometheanfire> good with that?
21:42:27 <prometheanfire> we don't have to decide this today
21:42:36 <smcginnis> Yep
21:42:46 <prometheanfire> #endmeeting