14:00:03 <abhishekk> #startmeeting glance 14:00:03 <opendevmeet> Meeting started Thu Jun 24 14:00:03 2021 UTC and is due to finish in 60 minutes. The chair is abhishekk. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:03 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:03 <opendevmeet> The meeting name has been set to 'glance' 14:00:06 <abhishekk> #topic roll call 14:00:12 <jokke_> o/ 14:00:12 <abhishekk> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda 14:00:16 <abhishekk> o/ 14:00:30 <abhishekk> lets wait couple of minutes more for others 14:00:35 <Luzi> o/ 14:00:38 <rosmaita> o/ 14:00:44 <dansmith> o/ 14:00:53 <abhishekk> cool lets start 14:00:57 <abhishekk> #topic release/periodic jobs update 14:01:07 <abhishekk> M2 3 weeks from now 14:01:19 <Steap_> o/ 14:01:24 <abhishekk> Which also means our spec freeze date is also approaching 14:01:51 <abhishekk> I would like all cores and members to go through policy refctoring spec as soon as possible 14:02:14 <abhishekk> I would like to get it approved before next meeting 14:02:31 <abhishekk> Periodic jobs, all green again 14:02:51 <abhishekk> Moving ahead, 14:02:52 <rosmaita> \o/ 14:03:04 <abhishekk> #topic M2 Target progress check 14:03:25 <abhishekk> For quotas implementation, documentation, tempest work is complete for long now 14:03:38 <abhishekk> We need REVIEWS on top priority 14:03:57 <abhishekk> To achieve its due date and get it done before M2 14:04:31 <abhishekk> So once again requesting voluntary core members and all members to spend some time on reviews as well 14:04:43 <abhishekk> Cache-API 14:05:05 <abhishekk> For cache API there is one suggestion for Delete all call 14:05:30 <abhishekk> at the moment we proposed to delete entire cache + queued images for delete all call 14:06:08 <abhishekk> So we would like to provide one flag which will tell to delete cached images only and keep queued list as it is 14:06:43 <abhishekk> In legacy tool we had separate calls for these two operations which we are merging into one in this proposal 14:06:56 <abhishekk> So any objection over this suggestion? 14:07:05 <dansmith> sounds better to be able to evict or expire one thing and not the whole cache, especially given the size 14:07:20 <jokke_> Or more like a scope where the requestor would be able to clear the current cache or the queue separately instead of just hitting them both 14:07:44 <jokke_> dansmith: so individual control exists in the current proposal already 14:07:50 <dansmith> oh I just re-read, 14:08:01 <dansmith> the separation is between the queue and currently-cached, I got it 14:08:03 <jokke_> on image basis, but the clear all whacks both 14:08:08 <dansmith> yep, sorry 14:08:37 <jokke_> so if oyur automation went crazy and queued 100 images, you could clear that queue without affecting current cache 14:08:54 <dansmith> for sure 14:09:17 <abhishekk> cool, as our proposal is already merged, how we can go for this 14:09:21 <jokke_> or wise versa, you could just whack the current cache if you know they are not in use anymore to speed up the queue processing 14:09:33 <abhishekk> 1. submit different spec (lite) for this change 14:09:39 <abhishekk> 2. amend the original spec 14:09:46 <abhishekk> 3. Cover it in documentation 14:10:15 <jokke_> As this is pretty simple adding a header to the delete call, do you guys feel we need to amend the spec or is this something we can consider implementation detail and handle on the code review? 14:11:19 <abhishekk> Ok, I will decide on approach later, lets move ahead 14:11:21 <dansmith> just amend the spec after you get the implementation done, IMHO 14:11:37 <abhishekk> ack 14:11:47 <jokke_> I'm fine with that 14:12:04 <abhishekk> Hopefully by next week we will have patches up for reviews 14:12:23 <abhishekk> #topic Image Encryption spec lite 14:12:45 <abhishekk> #link https://review.opendev.org/c/openstack/glance-specs/+/792134 14:12:55 <abhishekk> Is Luzi around? 14:13:16 <Luzi> yeah 14:14:02 <abhishekk> do you have any specific questions? 14:14:05 <Luzi> i had a question specific to dansmith - in the spec-lite you mentioned this might be worth a full spec 14:14:51 <Luzi> i want to understand why? the WIP-patch builds up on a full spec with secret Consumers 14:14:54 <dansmith> yeah, IMHO this has a lot of API impact and user behavior stuff in it, which I think is worth defining in more detail 14:15:06 <Luzi> https://review.opendev.org/c/openstack/glance-specs/+/609667/13/specs/victoria/approved/glance/image-encryption.rst 14:15:16 <dansmith> yeah, the original spec just says "meh, we'll figure this out later" for this part right? 14:15:57 <dansmith> specifically around the consumer bits 14:16:21 <dansmith> the original spec is for image encryption in general, and has lots of detail, but it just assumes the consumer stuff will be done 14:16:53 <Luzi> exactly - because the secret consumer API is still under work. 14:17:03 <dansmith> right, but the impact to glance's API is pretty substantial, 14:17:11 <jokke_> dansmith: how? 14:17:23 <rosmaita> yeah, i don't see that either 14:17:25 <dansmith> because you're talking about promising something to the users which you can't provide today, but may in the future.. that'll be a behavioral change that you can't insulate them from 14:17:51 <rosmaita> no, all we're promising is that if they have keys available, we will use them 14:17:57 <rosmaita> but key management is up to them 14:18:03 <dansmith> because the implication, according to the original spec, is that when you've created an image based on the key, that you've grabbed a refcount on that key 14:18:13 <Luzi> so glance will only deal with two things: registering a secret consumer when creating an image and unregistering when it is deleted. 14:18:24 <dansmith> "Glance register as a consumer of that 14:18:24 <dansmith> key (secret in Barbican [1]) when the corresponding encrypted image is 14:18:24 <dansmith> uploaded and unregister as a consumer when the image is deleted in Glance." 14:18:26 <dansmith> from the spec ^ 14:18:49 <rosmaita> i don't know that the consumer API is keeping an actual refcount 14:18:49 <dansmith> but what the lite spec is describing is just making that silently fail 14:19:18 <dansmith> rosmaita: it's keeping a list of consumers, right? I'm just shortening that to refcount because that's the goal of it, IMHO 14:19:20 <jokke_> dansmith: so what comes to the Glance side, is that we store the information and have dedicated property keys for them. Yes we do have in the full spec stating that we expect the consumer ap being present for this so we can register as consumer, and drop our registration, but glance is not doing the encryption nor the key management for those encrypted images, so I fail to see how this affects 14:19:26 <jokke_> our API 14:20:15 <rosmaita> dansmith: yeah, whether its an actual refcount is not relevant, i was just wondering 14:20:34 <dansmith> jokke_: because the spec says that we will have registered as a consumer of the key, which is the right behavior.. if we don't do that, and just return the image anyway, the user thinks they've got a hold of the key when they don't 14:20:45 <Luzi> dansmith, it is all up to the user. The secret consumers are just a help for users so a "secret delete" will not succeed as long as there are consumers (you should be able to force it though) 14:21:18 <dansmith> jokke_: so my suggestion was to provide an intent flag during creation of an image with a key, saying "I really care that you secured a reference as a consumer on the key" or "I don't really care" 14:21:19 <abhishekk> dansmith, are you worrying about accidental deletion of key by user? 14:21:23 <Luzi> wait - lets start with the workflow here 14:21:27 <dansmith> today we can honor the latter, but can't honor the former 14:21:41 <dansmith> but after the consumer api is done, we can enable the former, and keep the same api semantics 14:21:51 <jokke_> That said, like with lot of other our new features, I think would make lot of sense clearly mark this feature as EXPERIMENTAL until the secret consumer side is worked out and not claim it stable without 14:21:56 <dansmith> otherwise people have to know what version of glance is on the backend in order to know if it should work ornot 14:22:47 <dansmith> Luzi: yes, as I've said, I understand that it's just informational to the user, but it's a thing they are expecting to have to avoid losing data by deleting a key that is still in use 14:23:12 <dansmith> jokke_: well, leaving it experimental is certainly better than nothing, I'd just rather make it very explicit 14:24:20 <jokke_> dansmith: while I see your point and somewhat agree with it, As I understood the point with this lite-spec was to get the implementation parts moving while waiting for barbican (that we have done for over year now) 14:24:38 <abhishekk> I guess our intention behind allowing this place holding to get things moving 14:24:41 <Luzi> dansmith, i would really like to have the secret consumers ready now. but the barbican team is drowning in other work as it seems and they are at a point where i cannot help them. 14:24:47 <dansmith> jokke_: yep, and all I'm saying is that if we make the API intent more explicit, we're totally good to do that :) 14:24:53 <jokke_> To have the bits in place to actually iron the rest out meanwhile. And experimental would be the approach for that 14:25:56 <dansmith> POST /v2/images?consumer=required and POST /v2/images?consumer=optional 14:26:18 <dansmith> the former would always be 409 today, the latter would be 200, and we can go straight to supported for this API 14:26:37 <jokke_> dansmith: with the intent flag on the API call, the problem is that we're stuck with it in the API ... ok, if it's experimental we in theory could drop it before making the API stable, but I'd rather avoid doing backwards incompatible changes to the API 14:26:40 <dansmith> the users today have to declare "I understand that I may not have a refcount on my key" 14:27:07 <dansmith> jokke_: well, it always makes sense to say "I don't care about the accounting" so it's not really something we're stuck with, it's just something we get 14:27:31 <dansmith> jokke_: in fact, you could s/optional/ignore/ and then we could avoid the trip to barbican anyway, even once it's supported, if the user doesn't care 14:28:00 <dansmith> but see, all I said was "I think we might want to discuss this in a real spec" ... for this reason :D 14:28:39 <jokke_> IMO that ^^ is the worst option around and will confuse shit out of everyone :D 14:29:09 <Luzi> i don't think we should support a mix between being able to add a secret consumer and just not do it... 14:29:11 <dansmith> which? the spec or ignore? 14:29:18 <jokke_> dansmith: ignore 14:29:23 <dansmith> ack 14:29:35 <jokke_> sorry, I was fixing my typos while you posted the spec comment 14:30:05 <Luzi> still this is confusing to me - as i am the one writing things since years now... 14:30:24 <Luzi> we have an original spec with the secret consumers 14:30:45 <rosmaita> the other thing is, isn't it possible for a user to upload the image data, and then add the metadata later? or are we disallowing that somehow? 14:31:05 <abhishekk> I don't think we allow it 14:31:11 <dansmith> they have to be write-once 14:31:21 <dansmith> otherwise people will try to re-encrypt their images with a new key right? :) 14:31:21 <Luzi> if you want to upload an encrypted image some sort of metadata is needed 14:31:40 <Luzi> re-encrypting would be a whole new issue 14:32:12 <Luzi> well but whats different to the current situation in cinder for example 14:32:26 <jokke_> I agree with the write once and the key metadata should be present when the container type is encrypted before the image goes active 14:33:00 <Luzi> you can go to barbican and wildly delete random keys... or is there any way to prevent this? 14:33:21 <dansmith> the whole point of the consumer thing, is to at least *inform* the user that deleting a key will orphan real data 14:33:39 <dansmith> they should still be able to do it, of course, but the accounting is seriously important 14:34:06 <dansmith> also for auditing.. knowing which images or volumes or whatever are all encrypted with the same key is what keeps auditors up at night 14:34:17 <jokke_> Luzi: the point was that we should protect the glance metadata. If you go and delete those keys from barbican, the image is useless after, just delete it and re-create with up-to-date metadata 14:34:23 <dansmith> and making it easy to create a thousand images where all the accounting silently failed is problematic 14:35:12 <jokke_> Luzi: but we do have mechanism for that already, so it's one of those implementation details that are pretty easy to enforce 14:35:38 <jokke_> and imo not relevant for the intent of this light spec to just get the work moving while we wait for barbican 14:35:44 <Luzi> jokke_, with "you" I mean a user or project administrator 14:36:07 <Luzi> not glance 14:36:22 <jokke_> Luzi: correct, glance will never delete those keys 14:36:35 <abhishekk> So coming back to square 1, how much time it will take barbican to complete consumer work? 14:36:41 <rosmaita> ok, i guess i agree with dan (assuming this is his point) ... we need to amend the original spec assuming that there is no secret consumer api and say precisely what the workflows are 14:36:45 <jokke_> only once the consumer api is done letting brbican know that we refer to them 14:37:20 <dansmith> rosmaita: that's not my point, but that would be better than nothing :) 14:37:33 <Luzi> abhishekk, i really have no idea - could be weeks or months... :/ 14:37:44 <abhishekk> Luzi ack 14:38:22 <jokke_> Based on the fact that we have been waiting this for over a year and the last update I got was "maybe in this cycle" I'd not hold my breath with it 14:38:35 <Luzi> so instead of this lite spec you want me to edit the original spec - once for now without the secret consumers and once when we add them? 14:39:07 <abhishekk> Luzi I think you can convert this lite spec to spec 14:39:12 <dansmith> is everyone else opposed to an intent flag? because that lets us out of this now and later when we get the consumer api 14:39:21 <dansmith> the default can be "optional" and nothing really has to change today 14:39:39 <abhishekk> and mention the parent/former spec will be the feature work on top of this 14:40:32 <jokke_> I'd prefer not. If we change the spec to state we are not using the consumer api that gives the expectation that we won't ... I'd rather really just start merging the bits Luzi need to keep the momentum going without expecting the secret consumer part is the first thing we depend on and not claim the work done before that's in place 14:41:03 <jokke_> I thought that was the point of this lite-spec 14:41:04 <dansmith> okay, and when the consumer api comes, we do what? convert to always using it *and* failing hard if we can't register? 14:41:19 <jokke_> dansmith: correct 14:41:23 <jokke_> as intended 14:41:25 <Luzi> dansmith, that would be the way 14:41:40 <dansmith> okay, as long as we don't squelch the failures (which is what the lite spec proposes) then I'm fine with that 14:42:15 <abhishekk> ack 14:43:20 <abhishekk> Luzy hopefully your concern is addressed now 14:43:48 <Luzi> it's a bit more clear to me, what the problem is 14:44:04 <abhishekk> Limited time, lets move ahead and circle back to it in open discussion if we have time left 14:44:19 <abhishekk> #topic Policy refactoring 14:44:26 <abhishekk> Master spec: https://review.opendev.org/c/openstack/glance-specs/+/796753 14:44:27 <abhishekk> Tests refactoring lite spec: https://review.opendev.org/c/openstack/glance-specs/+/797593/1 14:44:50 <abhishekk> As I said we are approaching spec freeze and this time I want to honor it 14:44:59 <abhishekk> Kindly provide some time to review the spec 14:45:07 <abhishekk> I have one +2 on the spec 14:45:22 <abhishekk> rosmaita, jokke_ Steap_ smcginnis ^^^ 14:45:41 <rosmaita> ack 14:45:47 <abhishekk> Also I would like to have one session about this work sometime in next two weeks 14:46:05 <abhishekk> so if anyone has any concerns then we can discuss it there only 14:46:31 <dansmith> abhishekk: one session? meaning a voice call? 14:46:44 <abhishekk> yeah 14:46:48 <dansmith> roger 14:47:35 <abhishekk> I will shoot one mail for the time slot and we could finalize the one 14:48:04 <jokke_> ++ 14:48:20 <abhishekk> If we fail to get it done this time then I guess it will never going to happen in future 14:48:50 <abhishekk> that's it from me for today 14:49:07 <abhishekk> Kindly put some time in reviews as well 14:49:16 <abhishekk> #topic Open discussion 14:49:51 <abhishekk> Nothing from me 14:50:15 <dansmith> please review quota patches! 14:50:32 <abhishekk> yes 14:50:57 * Steap_ is on his way to the reviewing room 14:51:00 <abhishekk> I guess if that's it then we can wrap up and utilize remaining time in review 14:51:22 <abhishekk> Thank you all 14:51:52 <abhishekk> Have a nice weekend ahead 14:52:01 <abhishekk> #endmeeting