16:00:00 #startmeeting api-wg 16:00:01 Meeting started Thu Jan 5 16:00:00 2017 UTC and is due to finish in 60 minutes. The chair is cdent. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:02 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:04 The meeting name has been set to 'api_wg' 16:00:09 hola 16:00:14 hello! who has shown up from the api-wg meeting? 16:00:14 o/ 16:00:23 hello, e'rybody! 16:00:27 #chair elmiko etoews edleafe 16:00:28 Current chairs: cdent edleafe elmiko etoews 16:00:53 #link https://wiki.openstack.org/wiki/Meetings/API-WG#Agenda 16:01:07 There's, like, actual stuff on the agenda this week! 16:01:08 o/ 16:01:16 #topic previous action items 16:01:29 #link last meeting http://eavesdrop.openstack.org/meetings/api_wg/2016/api_wg.2016-12-15-16.01.html 16:01:37 ooh neat 16:01:43 agitating about capabilities (done) 16:02:42 ed making some new guidelines on booleans and state status (done) 16:03:11 #topic Glance Community Images and the API stability guidelines (rosmaita) 16:03:27 #link https://etherpad.openstack.org/p/glance-ocata-community-images-api-stability 16:03:29 take it away rosmaita 16:03:33 thanks 16:03:38 #link https://etherpad.openstack.org/p/glance-ocata-community-images-api-stability 16:03:51 you may have seen the item on the ML before the holidays 16:04:07 the etherpad linked above has the basic info about my question 16:04:28 probably best to read through the etherpad? 16:04:51 reading now 16:05:11 * rosmaita sits on his hands 16:05:15 lol 16:06:04 I'l start by saying that I don't agree with the evaluating api changes document, but it is the "standard" that we have. 16:06:52 cdent: what don't you agree with? 16:07:08 It's also not as if all standards are written in stone 16:07:09 nor do I agree that tempest maintainers shoud be the arbiters of what is correct behavior in an api 16:07:17 rosmaita: question about the visibility stuff, are these values in the body response? 16:07:35 elmiko: they're both request parameters and response body parameters iirc 16:07:42 but I make both comments as an independent observer, not as API-WG coreā„¢ 16:07:48 * sigmavirus is here with Glance and also not with Glance ;) 16:07:48 sigmavirus: ack, thanks 16:07:55 nice 16:07:58 elmiko: what sigmavirus said 16:08:23 * stevelle is like sigmavirus 16:08:27 * sigmavirus is also here to discuss pagination 16:08:35 as to #1, i don't see why that is not possible 16:08:50 the way i read it is that the tempest maintainers are being the arbiters of backwards incompatible changes but i may not have all of the context 16:09:01 if you are rev'ing the version anyways, seems ok to expand those definitions 16:09:05 elmiko: I agree. It's an API expansion which I always understood to be acceptable 16:09:11 etoews: agreed with you 16:09:33 elmiko: i bring it up, becase a literal reading of the guideline seems to rule it out 16:10:06 correct me if i'm wrong, but i thought with the new versioning changes we are bumping the major release for each openstack version. so, wouldn't that dictate that backward incompat changes are acceptable? 16:10:33 elmiko: which versioning changes? 16:10:35 rosmaita: ok, let me re-review the guidance 16:10:40 as in Glance 13.0.0 to 14.0.0? 16:10:55 sigmavirus: yeah, not exactly new. bad phrasing on my part. 16:11:15 elmiko: I don't think that applies to exposed HTTP APIs 16:11:24 cdent: ah 16:11:35 elmiko: right, so I've disagreed with the way openstack versions and does "semantic versioning" for a long time but that does not really follow convention or defacto rule son backwards compat 16:11:59 yeah, certainly seems that way 16:12:06 * edleafe is back 16:12:41 rosmaita: can I present a summary to you so you can tell me if my brain is wrong? 16:12:42 elmiko: yeah, the APIs are versioned independently of the software 16:12:51 cdent: fire away! 16:12:54 cdent: that'd probably benefit everyone :) 16:13:38 Before 2017 there was a long an arduous debate about how to deal with visibility. That reached a conclusion that didn't make everyone happy but at least made everyone not want to burn it all to the ground. Then 16:14:10 implementation of that solution has resulted in a conflict with the norms enforced in tempest 16:14:25 because behavior has changed 16:14:58 a proposal to work around that changed behvior by changing the test has been rejected as that is, in fact, a work around, and as such violates proper backwards compatibility 16:15:13 cdent: correct thus far 16:15:29 thus we are now trying to decide which norms to follow, which to break, or just some way to move foward 16:15:41 rem acu tetigisti 16:15:53 it isn't terribly important, but I'll point it out: the tempest test in question was added during Ocata 16:16:19 If that's all correct, my only source of confusion is really around: what are we trying to decide? What are our options? Do we want to/think we can tell tempest to do things differently? 16:16:39 here's the situation 16:16:57 stevelle: I think that's a pretty important consideration 16:17:02 the specific test breaks if you include an optional 'visibility' when creating an image 16:17:04 (the age of the test) 16:17:15 if you don't include the optional 'visibility', it would pass 16:18:00 so our opinion is that the change is "mostly" backward compatible 16:18:16 (or *my* opinion, anyway) 16:18:16 cdent: specifically if you say you want your image to have "visibility = private" and then try to share it, that's what breaks 16:18:17 rosmaita: was 'visibility' a valid param in the past? 16:18:22 edleafe: yes 16:18:36 sigmavirus: then it breaks the API 16:19:06 edleafe: the break doesn't happen in the creation of the image 16:19:07 is there an issue with simply expanding the tempest tests for visbility, or does that run against the backward compat issue? 16:19:07 edleafe: it was default to private, and the other choice (public) was limited by to admin by RBAC by default 16:19:09 ...or at least the tempest test 16:19:17 it happens when you try to add a member to the image 16:19:18 i don't suppose you have any real data on who is depending on the current behaviour in real life (non-tempest-wise) 16:19:21 edleafe: I absolutely sympathize with that, but I think along those lines that this will break the API regardless based on what tempest suggests we do 16:19:24 random drive by comment, these changes are particularly bad for users when there is no way to discover whether or not such a param is valid from the outside 16:19:47 clarkb: the param is valid 16:19:51 if I have been using it for years then all of a sudden it stops working on some portion of my clouds then I have a bad day 16:20:10 what happens is that what you can do with your image depends on what visibility you give it 16:20:12 edleafe: a similar test could create the image with visibility "private", add an image and then assert the visibility is still private afterwards. The "workaround" the tempest team suggested will break that 16:21:20 That's not meant to justify anything, just to give perspective that really the whole thing can be considered breaking, especially provided a user isn't already familiar with glance or changes introduced in ocata 16:21:38 my understanding from the pre-2107 debate was that there was awareness that this chang was going to have user impact but that it was an important change to make a better future, the past was a bug, and this is pain that just needs to be eaten, now. 16:21:40 Well, unless the existing test is invalid or incorrect in what it tests (which happens a lot), then the change isn't backwards compatible 16:21:48 If a user creates the image, adds a member to it, and then sees that it's suddenly marked as "shared" instead of "private" I have to wonder if they'll be surprised by that or not 16:22:14 If my statement is correct, then I would think that changing tempest is the right thing to do. 16:22:19 edleafe: see above, where it is not claimed to be backwards compatible 16:22:26 rosmaita: right my specific thing is don't change behavior without any way of making it discoverable between clouds that otherwise appear the same to the user (which I think is the case here if I am reading about why tempest fails. Yesterday it was fine, today not, clouds look the same) 16:22:26 claimed to be mostly* 16:22:31 cdent: +1 16:23:05 clarkb: so you're right. Since Glance doesn't have microversions, there's no way for a user to get yesterday's behaviour back 16:23:17 cdent: i agree with your "my understanding" comment above 16:23:21 But Glance will report that the version has changed which is a pittance of a way to discover the new version 16:23:24 (or changes) 16:23:44 sigmavirus: if you add a member, how could it be considered anything but shared? it's literally being shared. 16:23:47 sigmavirus: this would be a non-issue with microversions 16:23:54 etoews: because right now it remains private 16:23:59 edleafe: agreed 16:24:03 sigmavirus: right and that to me is the actual major issue with these changes 16:24:10 sigmavirus: it makes using openstack as a user incredibly frustrating 16:24:26 clarkb: I'm 100% with you 16:24:36 We have a vocal minority in glance that's opposed to microversions 16:24:49 So it's a trade-off of one frustrating thing for another potentially frustrating thing 16:24:52 clarkb: the tradeoff is that the visibility semantics will now make sense 16:25:02 just different users being frustrated 16:25:10 edleafe: +1 16:25:14 edleafe: agreed 16:25:17 edleafe: correct. And don't get me wrong. I absolutely think this behaviour is a huge improvement 16:25:18 the theory is: fewer users being frustrated 16:25:50 cdent: fewer is better, unless it's in a serious way 16:25:51 rosmaita: edleafe yes but the impact to someone who is now broken in production and has no clue why vs someone learning funky semantics vs just telling the user whats up properly are not the same 16:26:11 clarkb: makes sense 16:26:12 we need to timebox this in some fashion. 16:26:22 Additional consideration: This change also auto-migrates existing images 16:26:35 So private images with members will automagically become shared after this change 16:26:46 clarkb: the change is designed so that if you use defaults, everything works the same 16:26:57 so don't anticipate problems in production systems 16:27:00 ^ is very true 16:27:19 that's significant 16:27:20 From what I recall the discussion of the details of the change were _well_ trod when making the spec and discussing the email. We don't want to revisit that now. 16:27:23 sigmavirus: how serious is that? 16:27:34 sigmavirus: IOW, is it a potential security hole? 16:27:43 edleafe: is what a potential security hole? 16:27:47 auto-translating image visibility? 16:27:48 edleafe: it is backwards compatible 16:27:49 I think it is 16:27:54 sigmavirus: yeah 16:28:00 glance is clearly making a real effort to mitigate the risk 16:28:13 E.g.: I made this private, but suddenly it's shared 16:28:14 (I don't intend to retrod anything, just want to point out that to me as a user that is the actual concern, I think improving APIs is great we just need to be better and communicating to users how things have changed across clouds) 16:28:16 copy-paste wrong image-id and now you've shared an image, but that's the same thing as the current behaviour 16:28:19 so it's not a new risk 16:28:26 clarkb: agreed 16:28:40 edleafe: right now "private" can mean "private" or "private with members == shared" 16:28:54 that's the key problem, i think 16:29:06 'private' != "private" 16:29:16 sigmavirus: I only ask because unless it would do something horribly nasty, it sure sounds like this change would be a big improvement 16:29:16 yeah, that seems imprecise 16:29:44 So let me ask again: How can the api-wg help in this matter, now? 16:30:03 the issue is whether this change would violate the guidelines 16:30:18 edleafe: really only nastyness it does is that it breaks one very precise tempest test 16:30:19 the -1 on the tempest patch says it does 16:30:20 cdent: I think Glance is just stuck on technicalities with guidelines and dealing with Tempest's/QA Team's feedback 16:31:11 jokke_: which means it doesn't do anything nasty to actual users 16:31:16 a) they are just guidelines, they are violated all the time b) the existing semantics are (to me) a bug, at least mentally a security bug, so gravy 16:31:43 where "gravy" means "the guidance approves" 16:31:59 i tend to agree with cdent 16:32:03 So could a case be made that the existing Tempest test suffers from the same imprecision? 16:32:25 edleafe: and anyone who has bee creating images with --visibility='private' will need to first change the image to shared before running the member actions 16:32:29 (as the private has more than one meaning) 16:32:32 edleafe: the current test represents a potential real usage of Glance 16:32:39 edleafe: what i'd say is this: the existing test explicitly says to create a private image 16:32:48 then, when you share it, it fails, as it should 16:32:57 (though, currently, it would succeed) 16:33:08 cdent: we've actually had multiple bugs opened complaining that private is really not private 16:33:26 the test would succeed if you (a) created an image with no viz specified, or (b) created it with viz == shared 16:33:27 So it currently passes on something that should really be a failure? 16:33:29 we also don't recommend that users specify visibility necessarily but allow it 16:33:41 edleafe: ugh that's half-true, I guess? 16:33:45 edleafe: correct 16:33:51 Yeah sounds like the test is pretty much wrong 16:33:55 I mean presently with only two visibilities, a failure is hard to quantify 16:33:59 so as an action, either replies in support of changing the tempest test to support the spec as written, or responses to the tempest change may be helpful 16:34:01 Because public visibility's members don't matter 16:34:08 And I don't think you can add members to "public" images now 16:34:13 So you have to have a private image to add a member 16:34:15 edleafe: not by design should fail but by any basic logic should fail 16:34:18 can someone # link the review of the tempest in here please? 16:34:20 So it's a failure and it's a not failure too 16:34:21 Then I think it's clear that the test should be changed if it passes on a disallowed action 16:34:23 https://review.openstack.org/#/c/414261/ 16:34:26 thanks 16:34:51 Glance's visibility logic is just really bad presently 16:34:55 #action (everyone) go comment on tempest test change https://review.openstack.org/#/c/414261/ 16:34:56 i abhor backwards incompatible changes but the current behaviour really is a bug and flat out needs to be fixed. i lean towards it being acceptable. 16:34:57 * edleafe doesn't hold tests as sacred 16:34:59 we need to move on 16:35:13 ok, thanks everyone 16:35:18 sigmavirus: seems like it 16:35:28 i think this has been helpful, let's see what the QA team says 16:35:34 that was fun 16:35:37 (That's not the only thing that has that description in glance though =P) 16:35:37 thanks all ... we really appreciate your time on this! 16:35:50 #topic open mic 16:36:08 resource actions came up again shortly before christmas 16:36:19 sigmavirus, rosmaita, jokke_, thanks for bringing it up =) 16:36:20 #link resource actions email: http://lists.openstack.org/pipermail/openstack-dev/2016-December/109136.html 16:36:42 the usual three perspectives on that were represented, with the usual lack of consensus 16:36:59 I mention it here because it is something we probably need to continue to think about as it is going to keep coming back up 16:38:30 thoughts for the record? or if you want to add to that thread, that would be cool. Or if you want to resurrect the guideline process that would be cool too. 16:38:40 If not, I'll move on. 16:38:42 I added my thoughts to the thread 16:39:06 Who added the boston cfp link? 16:39:11 moi 16:39:18 #link boston cfp: https://www.openstack.org/summit/boston-2017/call-for-presentations/ 16:39:37 just as a reminder that we need to do the usual song and dance for boston 16:40:04 Not confirmed but I predict I will not be there (I won't be at ptg either). 16:40:30 I will be at ptg, but boston is still a question mark 16:40:55 both are questionable for me as well 16:41:08 elmiko? 16:41:28 i highly doubt i'll be able to attend, unless i pay my own way there 16:42:11 maybe, if i got a paper accepted about the stuff i'm currently working on, but it's kinda removed from the openstack world. sadly 16:42:30 next open mic topic: Do we want to have a process/discussion about setting some 2017 goals, or shall we go with the flow? 16:43:10 You mean like New Years resolutions? :) 16:43:14 hehe 16:43:14 The world at large thinks we are doing a pretty good job and not overreaching, so the status quo may be just fine 16:43:17 pretty much yeah 16:43:23 things we plan to do but don't 16:43:36 hmm, given that feedback cdent, go with the flow seems appropriate 16:43:49 i like the idea of setting some goals 16:43:50 our new year's resolution is to maintain the status quo 16:43:54 We don't have any huge gaping voids that we're aware of, so the flow seems right 16:43:58 etoews: LOL +1 16:43:59 but that's no fun 16:44:09 my feelings exactly sigmavirus 16:44:12 always be agitating 16:44:16 ok, maintain the status quo, but lose 10 pounds 16:44:16 that too 16:44:36 our plan for 2017 is to increase overreach? 16:44:43 * sigmavirus wonders if there is weight gain associated with working on OpenStack like there is with college 16:44:48 I personally think we can do more on the aspirational side of thing, but some people think that would be overreach 16:44:56 sigmavirus: don't even get me started... 16:45:00 * rosmaita has experienced the OpenStack weight gain 16:45:10 metoo@aol.com 16:45:15 cdent: aspirational *ideas* wouldn't be bad 16:45:15 haha 16:45:33 (Says the person including them in his present review) 16:45:39 i think one the real issues around this is that we don't necessarily have the bandwidth to take action on the aspirations. 16:45:42 (totally not a shameless plug right there) 16:45:53 etoews: +1 16:46:00 etoews: true that 16:46:04 etoews: but I thought we were here to lead other's thoughts 16:46:12 heh 16:46:13 thought leaders don't take action. They come up with actions for others 16:46:16 one thing we could do is propose work/issues for the arch-wg around api stuff 16:46:22 jinx! 16:46:24 to the twitters! 16:46:45 * sigmavirus lies face in palm 16:46:48 anyway, don't need to do anything immediately, another one for the "think about" queue 16:46:57 #topic guidelines 16:47:16 #link guidelines https://review.openstack.org/#/q/status:open+project:openstack/api-wg,n,z 16:47:17 we have some 16:47:37 but I don't reckon any are ready for freeze (due to xmas) 16:47:50 yeah, I'd like further review 16:47:56 the capability one had a lot of discussion in the nova-api meeting recently: 16:47:58 I was wondering if people had comments on edleafe's comment on the pagination review 16:48:05 #link capability in nova-api: http://eavesdrop.openstack.org/meetings/nova_api/2017/nova_api.2017-01-04-13.00.log.html#l-59 16:48:05 I just kind of threw some of those out there for others to pound on 16:48:22 sigmavirus: comments on comments? 16:48:33 I don't doubt that having "last" is expensive for some projects, but I wonder if there needs to be mention of optional components on something that's purely aspirational 16:49:06 people like clarity, ed's comment corresponds to the one I made earlier about optionality 16:49:28 some of those are MUST and some are MAY 16:49:40 sigmavirus: I think that someone looking at these as guidelines will try to copy them as closely as possible. It's good to let them know where the pitfalls may lie 16:49:41 Sure, I thought I had expanded on that well enough 16:50:11 sigmavirus: i haven't looked at version 4 yet 16:50:19 edleafe: sure, I just think that last tends to be more useful than folks reckon and making it optional will introduce inconsistency where Cinder always includes it and NOva doesn't because it's optional 16:50:38 i think we can just workflow+1 https://review.openstack.org/#/c/411391/ 16:50:41 I don't know that we want to allow that kind of problem 16:50:57 We could just not recommend "last" at all 16:51:02 sigmavirus: sure, but the reason for paginating in the first place is not having to pull large amounts of data 16:51:19 etoews: go for it 16:51:37 I think it is helpful to offer explicit stretch goals, like last, but it has to be clear there is no expectation. For those that opt in, it lets them standardize which is helpful. 16:51:59 stevelle: that's pretty much what I feel 16:51:59 edleafe: agreed. I just feel like there are absolutely ways around pulling large amounts of data to provide "last" 16:52:47 using the right data structures, ^ 16:52:47 sigmavirus: sure, but with a million records, it can be tough to find the one that's 30 from the end :) 16:53:16 cdent: ermmmm...apparently i can't workflow+1 anymore... 16:53:26 ...and do that with every hit on the link 16:53:43 etoews: need +2s first? 16:53:51 etoews: +1 on wf for that 16:54:05 i can't even +2 :P 16:54:06 edleafe: yeah I think if you're only considering pulling it from the database dynamically every time, you'll always have terrible API performance around your pagination 16:54:35 etoews: are you logged in as your usual person? 16:54:40 sigmavirus: and if you're not pulling it every time, it will be stale and pretty much useless 16:55:05 edleafe: that's categorically not true 16:55:17 I disagree strongly, but we have 3 conversations happening right now 16:55:18 sigmavirus: if you want, I can post an update that adds language around last 16:55:18 but openstack has little experience with effective caching and caching namespaces 16:55:26 cdent: oops. guess not. 16:55:32 I guess large non-OpenStack APIs that do Last consistently and correctly and usefully are just wrong 16:55:55 (or they don't exist and I'm imagining having worked on them) 16:56:11 so I would _love_ to have a discussion about how openstack is out of date in the api world, but let's leave that for another time please as we only have a few more minutes left 16:56:12 sigmavirus: like I said, it depends on how stale they are 16:56:23 cdent: LOL 16:56:39 are there other things to contend about on the other reviews? 16:56:57 things we know need to happen: more review on ed's strawmen for booleans and state 16:57:17 more eyes on the capabiities review because it is being driven by nova and cinder right now and they are not the whole world 16:57:32 and further discussion on the pagination thing so we can reach some consensus 16:57:35 anything else? 16:57:51 #topic bug review 16:58:04 There is a new bug that came along today as a result of a keystone bug. 16:58:24 #link 400 on bad query params: https://bugs.launchpad.net/openstack-api-wg/+bug/1654084 16:58:24 Launchpad bug 1654084 in openstack-api-wg "Listing resources with invalid filters should result in a 400" [Medium,Confirmed] 16:58:46 we have guidance on that issue for bodies, but not that easy to find on query params. that bug says we shoudl 16:58:57 somebody want to volunteer for that? 16:59:08 if so, assign yourself to the bug 16:59:16 * edleafe takes it 16:59:34 any one eager on the newletter or shall I do it? 17:00:07 Thanks to everyone for coming today, I think we've actually set some good foundation for stuff that matters for the coming year, based on the things we've disagreed about today. 17:00:09 that's good 17:00:10 so 17:00:12 awesome 17:00:16 #endmeeting