14:01:35 <markwash> #startmeeting glance
14:01:36 <openstack> Meeting started Thu Apr 24 14:01:35 2014 UTC and is due to finish in 60 minutes.  The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:37 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:39 <markwash> anybody here? :-)
14:01:41 <openstack> The meeting name has been set to 'glance'
14:01:46 <arnaud__> o/
14:01:59 <ativelkov> o/
14:02:21 <zhiyan> o/
14:02:46 <jokke_> o/
14:03:24 <markwash> hi folks. thanks for coming!
14:03:31 <markwash> sorry about my surprise absence last week
14:03:58 <markwash> agenda link
14:04:07 <markwash> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda
14:04:39 <markwash> any objections to jumping right into the artifacts stuff?
14:05:28 <markwash> #topic artifacts design document
14:05:45 <markwash> #link https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit
14:05:58 <markwash> ativelkov has been so kind as to produce and share ^^
14:06:00 <ativelkov> It is quite a long one, sorry :)
14:06:13 <ativelkov> But I've tried to cover most of the aspects
14:06:15 <markwash> ativelkov: care to talk about it some today? or would you prefer if folks just read it and get back to you?
14:06:50 <ativelkov> Well, I can highlite some of the topics
14:06:59 <ativelkov> And will wait for the detailed feedback
14:07:43 <markwash> ativelkov: okay cool, please highlight for us
14:08:03 <ativelkov> The general idea was to cover as many aspects as possible. I don't expect all of them to be implemented at once.
14:08:29 <ativelkov> The document just gives a definition of "what is the artifact" and defines its basic properties
14:08:55 <ativelkov> like immutability etc
14:09:38 <ativelkov> Then, it says that the metadata fields will be devided into two groups: some fileds are common for all the types of artifacts, while some others are type-specific
14:10:46 <ativelkov> Also, it says that the binary data is not a single blob anymore, but a a collection of blobs, and the structure of this collection is also defined by the type of artifact
14:11:47 <ativelkov> Then it introduces a concept of "artifact dependency links": being able to reference one artifact from another, and highlights different kinds of these links (static vs dynamic)
14:12:17 <ativelkov> And then it covers the artifact lifecycle and different actions which can be executed towards artifacts
14:12:37 <ativelkov> That was a brief contents :)
14:12:59 <markwash> ativelkov: thanks!
14:13:05 <flwang_> ativelkov: sorry for the hijack, is there any link about the artifcats' detailed design? thanks
14:13:06 <ativelkov> So, any feedback is welcome, and sorry if creating this doc took too long
14:13:19 <markwash> it will be helpful to keep these notes in mind while reading through your docs
14:13:31 <markwash> flwang_: yes, the link is https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit
14:13:38 <flwang_> markwash: cool, thanks
14:13:55 <ativelkov> Also, I've started composing some kind of FAQ document
14:14:00 <markwash> ativelkov: is there any particular way you prefer feedback? irc? mailing list? etc?
14:14:23 <arnaud__> ativelkov: thank you! will read it carefully. are there aspects where you expect debate?
14:14:24 <jokke_> ativelkov: Quick question as I haven't read the doc yet, Is the idea to extend the images to cover these, create a totally separate repo/store/api or replace the images with the artifacts in future?
14:15:25 <ativelkov> markwash: for small things I'd prefer comments in the document itself. The conceptual topic are better discussable in ML and IRC
14:15:45 * markwash nods
14:15:48 <ativelkov> arnaud__: I don't have any expectations, I am ready for any feedback )
14:15:53 <arnaud__> ok :)
14:16:24 <flwang_> ativelkov: make sense for me
14:16:46 <ativelkov> jokke_: we wanted to start developing artifacts as independent set of APIs in parallel with the current images. But evebtually - when artifacts are mature enough - I'd like to see images as artifacts as well
14:17:08 <ativelkov> #link https://etherpad.openstack.org/p/artifact-faq small FAQ I've started composing
14:17:09 <jokke_> ativelkov: cool, makes sense to me. Thanks!
14:17:22 <markwash> jokke_: for my take on your question. . .
14:17:40 <markwash> one thing that holds us back today is the weak definition of images
14:18:10 <markwash> for example, an image can be a single block device image, or it can contain multiple, or honestly it can be anything any given nova hypervisor knows how to deal with
14:18:29 <jokke_> markwash: I see
14:18:41 <markwash> the breadth of the glance image category is too wide to allow common features/use cases to be expressed sensibly
14:19:06 <markwash> now, artifacts when they come into play may have more strongly defined types
14:19:21 <markwash> (as I understand it, still need to square my expectations with ativelkov's doc)
14:19:41 <flwang_> markwash: so I assume we can resolve the 1xn mapping problem, right?
14:19:47 <ativelkov> I understand it the same way
14:19:48 <jokke_> yeah, that definitely needs few runs of careful reading
14:19:59 <arnaud__> flwang_: yep
14:20:02 <markwash> this will allow us to translate certain container/disk format categories of images into certain types of artifacts
14:20:20 <flwang_> arnaud__: ok, good
14:20:47 <jokke_> is the ultimate goal that the artifact types will be defined by glance or by consumer/user
14:20:51 <jokke_> ?
14:21:13 <ativelkov> artifact type will be defined by plugins
14:21:31 <ativelkov> the plugins are supposed to be designed by the project which consumes the artifact
14:21:44 <ativelkov> The most important ones - like images - will be part of the core
14:22:10 <flwang_> ativelkov: sounds reasonable
14:22:26 <ativelkov> I.e. Heat will design their own plugin which will enable Glance store the catalog of Heat templates, etc
14:22:28 <flwang_> ativelkov: I can't wait to review it :)
14:23:09 <flwang_> ativelkov: so the plugin will define the metadata of the targeted 'object', is it?
14:23:13 <ativelkov> Yes
14:23:23 <ativelkov> The type-specific metadata fields
14:23:25 <jokke_> ativelkov: will that plugin be a artifact that can be pushed in or does that need maintenance from operator to be enabled? :P
14:23:30 <ativelkov> And the binary data structure
14:23:42 <ativelkov> Plugin is written in Python
14:23:50 <flwang_> ativelkov: cool, it can resolve a requirement from heat team
14:23:53 <ativelkov> So it is not safe to push it at will
14:24:17 <zhiyan> ativelkov: do we need a length field for binary data of artifact ? (as a common field?)
14:24:42 <ativelkov> zhiyan: different types of artifact may have different set of binary objects
14:25:22 <ativelkov> Say, there may be an artifact composed of the VM image and a .png logo (the latter to be displayed in catalog UI)
14:25:37 <ativelkov> so, this means that the artifact contain 2 binary blobs
14:25:59 <ativelkov> and their length is computed and returned independently
14:26:33 <ativelkov> The amount and structure of these blobs is once again up to the particular type, i.e. up to plugin
14:26:52 <zhiyan> ativelkov: and iiuc, glanceclient will need such plugins structure as well?
14:27:38 <ativelkov> I think it may use some dynamic schema-fetching logic - similar to what it has now for v2 dynamic schemas
14:27:49 <flwang_> zhiyan: maybe not, if it still follows the schema
14:28:06 <flwang_> ativelkov: yep, that what I want to say
14:28:16 <ativelkov> there will be an API to discover the structure of particular type - and then the client will understand that structure
14:29:17 <flwang_> ativelkov: API?
14:29:31 <ativelkov> API endpoint, I mean
14:29:33 <flwang_> ativelkov: for now, we just load the schema and cache it
14:29:53 <flwang_> ativelkov: ah, you mean the API on glance server side, is it?
14:30:03 <ativelkov> yes
14:30:14 <ativelkov> same as /v2/schemas/images
14:30:26 <arnaud__> ativelkov: how do we define that an artifact is ready to be consumed? for an image, it is more or less when the file is uploaded, in the artifact case, how do you know what needs to be there to become ready?
14:30:47 <ativelkov> arnaud__: good question. There are two ways of doing that
14:30:57 <ativelkov> 1) explicitly call "Publish" method
14:31:01 <flwang_> ativelkov: got it
14:31:18 <ativelkov> This runs some checks to validate (including cusom plugin-defined logic)
14:31:27 <ativelkov> and then changes the state to "Ready"
14:31:32 <ativelkov> 2) Use import
14:31:48 <jokke_> wouldn't that be the plugins job to implement?
14:32:01 <ativelkov> jokke_: yes
14:32:36 <ativelkov> Importing is uploading a single binary file which contains everything needed for the artifact to be registered. The plugin knows the format of this file (this is usually a zip, but any container wil work)
14:33:22 <ativelkov> Then a plugin takes this file, parses it and does all the job - sets the metadata field values, save the binary data - and then the same set of checks is run, and the state changes to "ready"
14:33:37 <ativelkov> There is a reverse of this: export
14:33:49 <arnaud__> so import cannot be you in your VM/png case?
14:34:10 <arnaud__> s/you/your
14:34:13 <flwang_> so the import/export is based on current/coming async workers, right?
14:34:44 <arnaud__> flwang_: I hope so :)
14:35:04 <ativelkov> arnaud__: why? There is a zip file containing png and VM image. It gets uploaded to some temp location and is unzipped, then the plugin saves png and VM image into underlying storage and fill in the meta values
14:35:04 <markwash> I'd like to pause this very good discussion for just a moment
14:35:09 <flwang_> arnaud__: i see
14:35:18 <arnaud__> oh ok ativelkov that makes sense
14:35:44 <ativelkov> All the long-runners are supposed to be done on backing workers
14:35:54 <markwash> so I think this is going very well, and the main take away from the meeting is to review the document
14:36:05 <markwash> and it will help prime our review to have these various items highlighted here
14:36:15 <markwash> but are there other items folks want to discuss today that are not on the agenda?
14:37:24 <markwash> that's probably a no :-)
14:37:30 <flwang_> markwash: when will the design session be reviewed?
14:37:56 <flwang_> markwash: and have we scheduled a team lunch/dinner in the summit like we did in HK?
14:38:17 <markwash> sounds like we might want to take some time out to discuss the summit a bit
14:38:23 <markwash> we can come back to artifacts stuff if we have more time
14:38:28 <markwash> #topic design summit
14:38:29 <ativelkov> sure
14:38:45 <markwash> the design summit sessions need to be reviewd and approved/rejected by the end of this week
14:39:11 <flwang_> markwash: may i know how many slots we got?
14:39:16 <flwang_> 4?
14:39:27 <markwash> I believe the timeslot for glance is 1 or 2 slots Wednesday afternoon, and then the remainder Thursday morning
14:39:31 <markwash> I believe we have 5 slots total
14:39:56 <flwang_> markwash: it's a 'ok' number :D
14:40:14 <markwash> ativelkov: will you be at the summit to discuss the artifacts work?
14:40:16 <flwang_> zhiyan: will you be there?
14:40:26 <zhiyan> flwang_: i will
14:40:26 <ativelkov> markwash: yes, I am coming
14:40:32 <arnaud__> zhiyan: yeah :)
14:40:37 <markwash> zhiyan: yay!
14:40:39 <zhiyan> arnaud__: ;)
14:40:40 <flwang_> arnaud__: you?
14:40:44 <arnaud__> sure :)
14:40:49 <zhiyan> markwash:  ;)
14:40:51 <flwang_> arnaud__: cool
14:41:20 <markwash> flwang_: jokke_: will you be present at the summit?
14:41:27 <zhiyan> arnaud__: hope we can talk about glance-v2-api, image-handler plan as well
14:41:34 <jokke_> I will not
14:41:37 <flwang_> markwash: I think so if everything goes well
14:41:49 <arnaud__> zhiyan: oh yes...!  definitely
14:41:58 <zhiyan> arnaud__: coolcool
14:42:10 <markwash> flwang_: your second question was about a team lunch or dinner
14:42:21 <flwang_> markwash: yeah yeah
14:42:24 <markwash> I think its a good idea, but I don't believe anything has been arranged yet
14:42:25 <zhiyan> arnaud__: (probably with jay and others who like it)
14:42:50 <arnaud__> agreed
14:42:51 <flwang_> markwash: we can just leverage the lunch time if dinner is not ok
14:42:58 <rosmaita> we should at least plan on grabbing a table for lunch one day
14:43:14 <markwash> rosmaita: +1
14:43:15 <flwang_> rosmaita: hey
14:43:26 <flwang_> just like we did in HK
14:43:32 * rosmaita apologizes for being late
14:43:36 <zhiyan> rosmaita: hi, and rax pay it :-D
14:43:46 <markwash> I will check to see if we have a team pod / table in the conference area
14:44:10 <markwash> if so, this will also be very good for discussion and meeting
14:44:22 <flwang_> markwash: awesome awesome awesome
14:44:55 <markwash> in some ways its a bit early, but I think it is useful now to consider our options about a mini summit later in the Juno cycle
14:45:32 <flwang_> markwash: +100, it would be a good time to review the status of artifacts
14:45:39 <markwash> can we just get a brief show of hands for folks who would be interested in a mini summit sometimes towards the middle of Juno?
14:45:51 <jokke_> o/
14:45:55 <arnaud__> o/
14:45:56 <flwang_> \o/
14:45:59 <ativelkov> o/
14:46:23 <zhiyan> o/
14:46:47 <markwash> okay. . I'll talk to the project folks to see if they prefer for these mini things to remain unaffilliated with openstack in any official way
14:47:09 <markwash> if so, I'll send out an email to ask for possible support / sponsorship for the event
14:47:23 <zhiyan> markwash: btw, do you think it will cool if we make a "scheduled" BP list for J cycle? and i think folks will have some review focus (and todo priority)
14:47:38 <zhiyan> after summit
14:48:27 <markwash> zhiyan: this sounds like a good idea to me
14:48:56 <markwash> anything that can help focus and clarify the job of reviewing probably is a big help
14:49:19 <zhiyan> and todo follow-up, IMHO
14:49:45 <markwash> zhiyan: can you talk about such a list a bit more? just to give an impression
14:49:57 <zhiyan> like async-task
14:50:16 <zhiyan> what arnaud__ did currently
14:50:52 <flwang_> async workers +1, I'm working on the export part
14:50:57 <markwash> ah okay
14:51:23 <markwash> any other thoughts about the design summit? again if all goes well there should be a schedule up by this weekend
14:51:37 <arnaud__> flwang_: we need to sync up :)
14:51:57 <rosmaita> markwash: was distracted, me & hemanth are up for mini-summit too
14:52:01 <flwang_> arnaud__: sure, beer on the summit :D
14:52:03 <rosmaita> and probably rest of our teem
14:52:17 <arnaud__> flwang_:  :) :) :)
14:52:22 <rosmaita> flwang_: did you see d berrange's note about os_type ?
14:52:38 <flwang_> rosmaita: yep
14:53:19 <markwash> so let's open up more generally then for the rest of today
14:53:21 <flwang_> rosmaita: but that didn't explain why we don't have it now
14:53:23 <markwash> #topic open discussion
14:53:52 <markwash> ativelkov: thanks for putting together the document
14:53:58 <rosmaita> flwang_: just think we didnt' need it
14:54:07 <flwang_> rosmaita: haha, ok
14:54:19 <markwash> ativelkov: its very exciting!
14:54:35 <markwash> heh well if you like programming python server projects I guess :-)
14:54:41 <ativelkov> :) Sorry it took longer then I expected
14:54:43 <flwang_> ativelkov: markwash : yep, it's impacting the scope of Glance
14:55:14 <jokke_> just small addition to the artifact discussion. I hope we are really careful about the plugin specs. I'm just afraid that it will become support nightmare if we give even closely free hands what those plugins are allowed to do. After all if they do not work it will be the glance getting the blame on the first hit
14:55:26 <markwash> ativelkov: in the end I think this is good timing to make our summit discussions of it a bit more tactical
14:55:37 <rosmaita> ativelkov: is this the artifact url? https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit
14:55:48 <ativelkov> rosmaita: yes
14:55:57 <flwang_> jokke_: +1
14:55:59 <rosmaita> thanks
14:56:05 <ativelkov> jokke_: right. This needs to be carefully designed.
14:56:11 <markwash> jokke_: I think that is a fair point. there are definitely some tradeoffs to be managed
14:56:13 <flwang_> jokke_: so we need to review it at the mini summit
14:56:37 <ativelkov> I was thinking about timeout limits for plugin operations and other stuff to gurantee their stability
14:56:59 <zhiyan> spec + review
14:57:43 <rosmaita> i'm kind of worried about overspecifying, couldn't some plugin operations be really long running?
14:58:50 <ativelkov> import and export are usually long, because this usually involves creating a container (zip) etc
14:59:14 <markwash> all right folks, that is about all we have time for, thank you for your participation
14:59:33 <rosmaita> bye!
14:59:38 <ativelkov> thanks
14:59:42 <markwash> next week's meeting is at 20:00 UTC
14:59:44 <arnaud__> thanks
14:59:57 <markwash> #endmeeting