Thursday, 2022-07-21

*** haleyb_ is now known as haleyb12:52
*** dasm|off is now known as dasm|ruck13:05
pdeore#startmeeting glance14:00
opendevmeetMeeting started Thu Jul 21 14:00:09 2022 UTC and is due to finish in 60 minutes.  The chair is pdeore. Information about MeetBot at
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
opendevmeetThe meeting name has been set to 'glance'14:00
pdeore#topic roll call14:00
* abhishekk will be here for short time only14:00
abhishekkjokke_, is on leave14:01
pdeorewe have short agenda for today14:01
pdeorelet's start14:01
abhishekkLets start 14:01
pdeore#topic release/periodic jobs14:01
pdeoreWe have tagged M2 last week14:02
pdeoreand M3 is 6 weeks from now...14:02
pdeorePeriodic Jobs, all green except the 'translation update' jobs failing with RETRY_LIMIT14:02
abhishekkFor M3 we have 2 major patches to be merged14:02
abhishekk1 Glance download import method14:03
abhishekk2 Extend store details information for all stores14:03
abhishekkboth patches are up for reviews14:03
abhishekkUnfortunately we did not here anything from tobias regarding injecting metadata during upload process14:04
abhishekkso probably we are going to skip it in this cycle14:05
* dansmith nods14:05
abhishekklets move ahead14:06
pdeore#topic Bugs14:06
pdeore#link 14:06
croelandtthat was me :)14:06
croelandtI think we've stated multiple times that we only want to use image IDs in commands14:06
croelandtand not image names14:06
croelandtam I mistaken or can we"close" this bug?14:06
abhishekkWe can mark it as won't fix14:07
croelandtgreat, thanks for confirming that :)14:07
dansmithI really hate that behavior, fwiw :)14:07
dansmithand glance is the outlier here, AFAICT14:07
croelandtit's kind of more user-friendly I guess14:07
dansmithbut basically, I have to use my mouse to interact with glanceclient, which is infuriating14:08
croelandtin bug reports sometimes I wish we had names rather than UUIDs14:08
dansmithit also makes writing docs more confusing because we'll show a list of images, and then a bunch of commands with UUIDs and the reader has to do their own correlation to know what we're doing14:08
croelandtdansmith: I would not be against it, but I'm not willing to do it myself, and it's not a hill I want to die on14:09
dansmithyeah, I would do it, but I also feel like it's going to be some giant fight for basic usability, which is hard to get excited about14:09
dansmithbut literally I can interact with the whole rest of the stack with names, except for glance14:09
croelandtyeah and it's the kind of patch that's gonna need multiple rounds14:10
croelandtbecause someone is not gonna be happy about the tests etc.14:10
croelandtand it's going to drag on for 3 years14:10
dansmithwell, I don't think it's about the tests or implementation, but the principle I suspect :)14:10
croelandtyeah also that14:10
croelandtI mean the implementation is interesting as well14:11
croelandtthere's some potential for errors if people name multple images the same14:11
dansmithit's the same for all other resources though14:11
croelandtmaybe something to revisit in the future once we're tired of copy/pasting uuids with the damn mouse14:11
abhishekkmostly principle, I remember there is lots of negative opinions from glance team at that time14:11
dansmithnames are not unique constrants in many places14:11
croelandtjsut wait for our wrists to be broken14:11
pdeore:D :D14:12
croelandtabhishekk: we **are** the Glance team :D14:12
abhishekkcroelandt, yeah, but it was discussed during Hawana cycle as well14:12
croelandtlet'sd discuss it again at the next H cycle14:12
abhishekkI remember I had a PoC ready for this at that time :D14:13
dansmithmaybe we could get an operator survey question about it14:13
abhishekkSounds good14:13
croelandtdansmith: +114:13
pdeoremove to next ? :)14:14
abhishekkLets take help of rosmaita for drafting operators question14:14
abhishekkyes, lets move ahead14:15
pdeore#topic Glance download import work14:15
abhishekkalistarle, is back now14:15
pdeoreyeah, and there are some updates on the patch ,
abhishekkalistarle, do you have any questions related to review which you want to discuss?14:16
alistarlehmmm yes, I see dan comment14:17
alistarleFor the URL check, do you think a cast to UUID check is enough ?14:17
dansmithalistarle: probably, or a quick check that they're just hex characters14:18
dansmithfor the image_id you mean14:18
dansmithI still think we should re-use that scheme checker on the url we construct14:18
dansmithwhich should be easy I think14:18
dansmithif glance image ids are always uuids then cast to uuid seems okay14:19
dansmithI can do that work if you want, but I think it's easy.. a couple lines in each location at most14:20
alistarleI agree with the use of scheme checker, but we need to move it to the task execution because we don't have the URL yet, so it not fail fast as the 400 we return during a web_download14:20
dansmithright for sure14:20
dansmithapologize that my "add glance-download here" meant physically "here" but I meant "add this check for glance-download too"14:21
alistarleok so no problem, I will do and update the doc accordingly14:21
dansmithwe should have some tests for these things too, so let me know if I need to write those14:22
dansmithbut hopefully my test module got you started enough to copy for other scenarios14:22
alistarleyep sure14:22
alistarleand regarding the content-length stuff, so the header check isn't enough like for the web-download ?14:23
alistarle(I will check if glance return it btw)14:23
dansmithI don't think glance sets it AFAIK14:24
dansmithif it does, then perhaps that's okay, but I think wrapping glance with apache that does gzip offloading will eliminate it, since you can't pre-calculate the compressed size for many gigs of data14:24
alistarleHmmm ok, but checking the image size glance side will require an additional call to remote glance API14:25
alistarleAnd are we sure glance will provide us the right size ?14:25
dansmithyeah, which is what we're trying to do here.. provide a glance-aware download mechanism :)14:26
dansmithabhishekk: I don't see where that is set in the code, but if apache is doing gzip for us, I think it has to eliminate that14:27
abhishekkwill look at it14:27
dansmithalso this, if we do our own gzip in middleware:
dansmithalistarle: maybe you can implement this in the metadata copy part, where you compare what our glance and the remote glance think is the size,14:30
dansmithbut that means you'll lose that check if you don't have the metadata copy module enabled, which is probably not what people would expect14:30
alistarleissue is we copy metadata before we download the image, so we can't do the check14:32
dansmithoh right, heh :)14:32
dansmithwell, then we probably need to just do the extra call then14:33
dansmithit would be so easy for long-running downloads to terminate mid-stream, we claim "okay your image is good to go, metadata and all" and have it be corrupted14:33
abhishekkcan we pass the size to following task as an input?14:33
alistarleor we can manually update the image size during the metadata copy, but maybe it is a bad idea to deal with that14:33
dansmithabhishekk: we could set it as os_glance_remote_size and then remove that when we've checked14:34
dansmithbut honestly, I don't think the extra call is that bad14:34
dansmithalistarle: yeah bad idea I think14:34
alistarleabhishekk: I don't know if we can easily pass param between task, but if yes maybe giving the size as an input is good14:35
abhishekkalistarle, afaik, we can pass param between tasks14:35
dansmithbut if metadata is disabled then we won't have a size to check that way14:36
dansmithif we put an explicit call to remote in dowload, then that continues to work if you've disabled the metadata plugin14:36
abhishekkdisabled means there is no item in that config option, right?14:36
alistarlenope, it is forcefully added to the flow14:37
dansmithno I think disabled means you don't list the metadata plugin in the plugins.. wasn't that one of the things desired? to be able to disable that separately?14:37
dansmithoh okay, then nevermind14:37
alistarlewe don't wanted the make it configurable for inconsistency reason14:37
alistarleyou can choose to copy nothing, but actually the plugin will run (at least in the current patch set)14:38
alistarleto update at least the container_format and disk_format14:38
dansmithit's so weird to me that we have some of the flow steps in api_image_import, but the body of the implementation in the plugin file14:38
dansmithbut yeah I see now14:38
alistarleand should we rely more on the size or virtual_size attribute ? I remember there is two field in the API14:40
abhishekkalistarle, if it is not possible to pass the param as an input then as dan suggested set reserved property to image os_glance_remote_size and then remove it after checking14:40
abhishekkI think size is what we should verify14:41
dansmithI really don't like setting it on the image, so if there's a better way we should do that, but if not, that should be an option14:41
dansmithyes, size14:41
dansmithvirtual_size could be correct if we read the first few kilobytes of a qcow2 file, but not have any more of it14:42
alistarleok let's do that, I will rework the patch accordingly14:43
dansmithalistarle: please speak up if you get stuck so we can help.. we don't want this to get delayed any longer than necessary14:43
alistarleso to sum up: Will handle the content-length based on the size attribute of the remote glance, I need to revert the metadata to the original value, and I need to check the target url against the scheme validator14:44
alistarleseems ok to me14:44
dansmithalistarle: and sanitize the image_id14:44
alistarleabhishekk: I will check with you if I struggle to forward param between tasks14:44
abhishekkalistarle, ack14:44
dansmithalistarle: to be clear, you need to compare size==size, not size==content_length, because the latter will be different with gzip14:45
dansmithall good to move on?14:47
alistarlesorry for the long time we spend on that topic ;) 14:47
dansmithnot a problem IMHO, glad we're progressing14:48
pdeoremoving to Open discussion...14:49
pdeore#topic Open Discussion14:49
abhishekkalistarle, hopefully this will help14:49
abhishekknothing from me for Open discussion14:50
dansmithabhishekk: ah nice14:50
pdeoreso should we wrap up ? 14:51
abhishekkthank you all14:52
pdeoregr8!! Thanks evryone for joining !!14:52
opendevmeetMeeting ended Thu Jul 21 14:52:29 2022 UTC.  Information about MeetBot at . (v 0.1.4)14:52
opendevmeetMinutes (text):
*** dasm|ruck is now known as dasm|off21:04

Generated by 2.17.3 by Marius Gedminas - find it at!