14:01:35 #startmeeting glance 14:01:36 Meeting started Thu Apr 24 14:01:35 2014 UTC and is due to finish in 60 minutes. The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:37 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:01:39 anybody here? :-) 14:01:41 The meeting name has been set to 'glance' 14:01:46 o/ 14:01:59 o/ 14:02:21 o/ 14:02:46 o/ 14:03:24 hi folks. thanks for coming! 14:03:31 sorry about my surprise absence last week 14:03:58 agenda link 14:04:07 #link https://etherpad.openstack.org/p/glance-team-meeting-agenda 14:04:39 any objections to jumping right into the artifacts stuff? 14:05:28 #topic artifacts design document 14:05:45 #link https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit 14:05:58 ativelkov has been so kind as to produce and share ^^ 14:06:00 It is quite a long one, sorry :) 14:06:13 But I've tried to cover most of the aspects 14:06:15 ativelkov: care to talk about it some today? or would you prefer if folks just read it and get back to you? 14:06:50 Well, I can highlite some of the topics 14:06:59 And will wait for the detailed feedback 14:07:43 ativelkov: okay cool, please highlight for us 14:08:03 The general idea was to cover as many aspects as possible. I don't expect all of them to be implemented at once. 14:08:29 The document just gives a definition of "what is the artifact" and defines its basic properties 14:08:55 like immutability etc 14:09:38 Then, it says that the metadata fields will be devided into two groups: some fileds are common for all the types of artifacts, while some others are type-specific 14:10:46 Also, it says that the binary data is not a single blob anymore, but a a collection of blobs, and the structure of this collection is also defined by the type of artifact 14:11:47 Then it introduces a concept of "artifact dependency links": being able to reference one artifact from another, and highlights different kinds of these links (static vs dynamic) 14:12:17 And then it covers the artifact lifecycle and different actions which can be executed towards artifacts 14:12:37 That was a brief contents :) 14:12:59 ativelkov: thanks! 14:13:05 ativelkov: sorry for the hijack, is there any link about the artifcats' detailed design? thanks 14:13:06 So, any feedback is welcome, and sorry if creating this doc took too long 14:13:19 it will be helpful to keep these notes in mind while reading through your docs 14:13:31 flwang_: yes, the link is https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit 14:13:38 markwash: cool, thanks 14:13:55 Also, I've started composing some kind of FAQ document 14:14:00 ativelkov: is there any particular way you prefer feedback? irc? mailing list? etc? 14:14:23 ativelkov: thank you! will read it carefully. are there aspects where you expect debate? 14:14:24 ativelkov: Quick question as I haven't read the doc yet, Is the idea to extend the images to cover these, create a totally separate repo/store/api or replace the images with the artifacts in future? 14:15:25 markwash: for small things I'd prefer comments in the document itself. The conceptual topic are better discussable in ML and IRC 14:15:45 * markwash nods 14:15:48 arnaud__: I don't have any expectations, I am ready for any feedback ) 14:15:53 ok :) 14:16:24 ativelkov: make sense for me 14:16:46 jokke_: we wanted to start developing artifacts as independent set of APIs in parallel with the current images. But evebtually - when artifacts are mature enough - I'd like to see images as artifacts as well 14:17:08 #link https://etherpad.openstack.org/p/artifact-faq small FAQ I've started composing 14:17:09 ativelkov: cool, makes sense to me. Thanks! 14:17:22 jokke_: for my take on your question. . . 14:17:40 one thing that holds us back today is the weak definition of images 14:18:10 for example, an image can be a single block device image, or it can contain multiple, or honestly it can be anything any given nova hypervisor knows how to deal with 14:18:29 markwash: I see 14:18:41 the breadth of the glance image category is too wide to allow common features/use cases to be expressed sensibly 14:19:06 now, artifacts when they come into play may have more strongly defined types 14:19:21 (as I understand it, still need to square my expectations with ativelkov's doc) 14:19:41 markwash: so I assume we can resolve the 1xn mapping problem, right? 14:19:47 I understand it the same way 14:19:48 yeah, that definitely needs few runs of careful reading 14:19:59 flwang_: yep 14:20:02 this will allow us to translate certain container/disk format categories of images into certain types of artifacts 14:20:20 arnaud__: ok, good 14:20:47 is the ultimate goal that the artifact types will be defined by glance or by consumer/user 14:20:51 ? 14:21:13 artifact type will be defined by plugins 14:21:31 the plugins are supposed to be designed by the project which consumes the artifact 14:21:44 The most important ones - like images - will be part of the core 14:22:10 ativelkov: sounds reasonable 14:22:26 I.e. Heat will design their own plugin which will enable Glance store the catalog of Heat templates, etc 14:22:28 ativelkov: I can't wait to review it :) 14:23:09 ativelkov: so the plugin will define the metadata of the targeted 'object', is it? 14:23:13 Yes 14:23:23 The type-specific metadata fields 14:23:25 ativelkov: will that plugin be a artifact that can be pushed in or does that need maintenance from operator to be enabled? :P 14:23:30 And the binary data structure 14:23:42 Plugin is written in Python 14:23:50 ativelkov: cool, it can resolve a requirement from heat team 14:23:53 So it is not safe to push it at will 14:24:17 ativelkov: do we need a length field for binary data of artifact ? (as a common field?) 14:24:42 zhiyan: different types of artifact may have different set of binary objects 14:25:22 Say, there may be an artifact composed of the VM image and a .png logo (the latter to be displayed in catalog UI) 14:25:37 so, this means that the artifact contain 2 binary blobs 14:25:59 and their length is computed and returned independently 14:26:33 The amount and structure of these blobs is once again up to the particular type, i.e. up to plugin 14:26:52 ativelkov: and iiuc, glanceclient will need such plugins structure as well? 14:27:38 I think it may use some dynamic schema-fetching logic - similar to what it has now for v2 dynamic schemas 14:27:49 zhiyan: maybe not, if it still follows the schema 14:28:06 ativelkov: yep, that what I want to say 14:28:16 there will be an API to discover the structure of particular type - and then the client will understand that structure 14:29:17 ativelkov: API? 14:29:31 API endpoint, I mean 14:29:33 ativelkov: for now, we just load the schema and cache it 14:29:53 ativelkov: ah, you mean the API on glance server side, is it? 14:30:03 yes 14:30:14 same as /v2/schemas/images 14:30:26 ativelkov: how do we define that an artifact is ready to be consumed? for an image, it is more or less when the file is uploaded, in the artifact case, how do you know what needs to be there to become ready? 14:30:47 arnaud__: good question. There are two ways of doing that 14:30:57 1) explicitly call "Publish" method 14:31:01 ativelkov: got it 14:31:18 This runs some checks to validate (including cusom plugin-defined logic) 14:31:27 and then changes the state to "Ready" 14:31:32 2) Use import 14:31:48 wouldn't that be the plugins job to implement? 14:32:01 jokke_: yes 14:32:36 Importing is uploading a single binary file which contains everything needed for the artifact to be registered. The plugin knows the format of this file (this is usually a zip, but any container wil work) 14:33:22 Then a plugin takes this file, parses it and does all the job - sets the metadata field values, save the binary data - and then the same set of checks is run, and the state changes to "ready" 14:33:37 There is a reverse of this: export 14:33:49 so import cannot be you in your VM/png case? 14:34:10 s/you/your 14:34:13 so the import/export is based on current/coming async workers, right? 14:34:44 flwang_: I hope so :) 14:35:04 arnaud__: why? There is a zip file containing png and VM image. It gets uploaded to some temp location and is unzipped, then the plugin saves png and VM image into underlying storage and fill in the meta values 14:35:04 I'd like to pause this very good discussion for just a moment 14:35:09 arnaud__: i see 14:35:18 oh ok ativelkov that makes sense 14:35:44 All the long-runners are supposed to be done on backing workers 14:35:54 so I think this is going very well, and the main take away from the meeting is to review the document 14:36:05 and it will help prime our review to have these various items highlighted here 14:36:15 but are there other items folks want to discuss today that are not on the agenda? 14:37:24 that's probably a no :-) 14:37:30 markwash: when will the design session be reviewed? 14:37:56 markwash: and have we scheduled a team lunch/dinner in the summit like we did in HK? 14:38:17 sounds like we might want to take some time out to discuss the summit a bit 14:38:23 we can come back to artifacts stuff if we have more time 14:38:28 #topic design summit 14:38:29 sure 14:38:45 the design summit sessions need to be reviewd and approved/rejected by the end of this week 14:39:11 markwash: may i know how many slots we got? 14:39:16 4? 14:39:27 I believe the timeslot for glance is 1 or 2 slots Wednesday afternoon, and then the remainder Thursday morning 14:39:31 I believe we have 5 slots total 14:39:56 markwash: it's a 'ok' number :D 14:40:14 ativelkov: will you be at the summit to discuss the artifacts work? 14:40:16 zhiyan: will you be there? 14:40:26 flwang_: i will 14:40:26 markwash: yes, I am coming 14:40:32 zhiyan: yeah :) 14:40:37 zhiyan: yay! 14:40:39 arnaud__: ;) 14:40:40 arnaud__: you? 14:40:44 sure :) 14:40:49 markwash: ;) 14:40:51 arnaud__: cool 14:41:20 flwang_: jokke_: will you be present at the summit? 14:41:27 arnaud__: hope we can talk about glance-v2-api, image-handler plan as well 14:41:34 I will not 14:41:37 markwash: I think so if everything goes well 14:41:49 zhiyan: oh yes...! definitely 14:41:58 arnaud__: coolcool 14:42:10 flwang_: your second question was about a team lunch or dinner 14:42:21 markwash: yeah yeah 14:42:24 I think its a good idea, but I don't believe anything has been arranged yet 14:42:25 arnaud__: (probably with jay and others who like it) 14:42:50 agreed 14:42:51 markwash: we can just leverage the lunch time if dinner is not ok 14:42:58 we should at least plan on grabbing a table for lunch one day 14:43:14 rosmaita: +1 14:43:15 rosmaita: hey 14:43:26 just like we did in HK 14:43:32 * rosmaita apologizes for being late 14:43:36 rosmaita: hi, and rax pay it :-D 14:43:46 I will check to see if we have a team pod / table in the conference area 14:44:10 if so, this will also be very good for discussion and meeting 14:44:22 markwash: awesome awesome awesome 14:44:55 in some ways its a bit early, but I think it is useful now to consider our options about a mini summit later in the Juno cycle 14:45:32 markwash: +100, it would be a good time to review the status of artifacts 14:45:39 can we just get a brief show of hands for folks who would be interested in a mini summit sometimes towards the middle of Juno? 14:45:51 o/ 14:45:55 o/ 14:45:56 \o/ 14:45:59 o/ 14:46:23 o/ 14:46:47 okay. . I'll talk to the project folks to see if they prefer for these mini things to remain unaffilliated with openstack in any official way 14:47:09 if so, I'll send out an email to ask for possible support / sponsorship for the event 14:47:23 markwash: btw, do you think it will cool if we make a "scheduled" BP list for J cycle? and i think folks will have some review focus (and todo priority) 14:47:38 after summit 14:48:27 zhiyan: this sounds like a good idea to me 14:48:56 anything that can help focus and clarify the job of reviewing probably is a big help 14:49:19 and todo follow-up, IMHO 14:49:45 zhiyan: can you talk about such a list a bit more? just to give an impression 14:49:57 like async-task 14:50:16 what arnaud__ did currently 14:50:52 async workers +1, I'm working on the export part 14:50:57 ah okay 14:51:23 any other thoughts about the design summit? again if all goes well there should be a schedule up by this weekend 14:51:37 flwang_: we need to sync up :) 14:51:57 markwash: was distracted, me & hemanth are up for mini-summit too 14:52:01 arnaud__: sure, beer on the summit :D 14:52:03 and probably rest of our teem 14:52:17 flwang_: :) :) :) 14:52:22 flwang_: did you see d berrange's note about os_type ? 14:52:38 rosmaita: yep 14:53:19 so let's open up more generally then for the rest of today 14:53:21 rosmaita: but that didn't explain why we don't have it now 14:53:23 #topic open discussion 14:53:52 ativelkov: thanks for putting together the document 14:53:58 flwang_: just think we didnt' need it 14:54:07 rosmaita: haha, ok 14:54:19 ativelkov: its very exciting! 14:54:35 heh well if you like programming python server projects I guess :-) 14:54:41 :) Sorry it took longer then I expected 14:54:43 ativelkov: markwash : yep, it's impacting the scope of Glance 14:55:14 just small addition to the artifact discussion. I hope we are really careful about the plugin specs. I'm just afraid that it will become support nightmare if we give even closely free hands what those plugins are allowed to do. After all if they do not work it will be the glance getting the blame on the first hit 14:55:26 ativelkov: in the end I think this is good timing to make our summit discussions of it a bit more tactical 14:55:37 ativelkov: is this the artifact url? https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit 14:55:48 rosmaita: yes 14:55:57 jokke_: +1 14:55:59 thanks 14:56:05 jokke_: right. This needs to be carefully designed. 14:56:11 jokke_: I think that is a fair point. there are definitely some tradeoffs to be managed 14:56:13 jokke_: so we need to review it at the mini summit 14:56:37 I was thinking about timeout limits for plugin operations and other stuff to gurantee their stability 14:56:59 spec + review 14:57:43 i'm kind of worried about overspecifying, couldn't some plugin operations be really long running? 14:58:50 import and export are usually long, because this usually involves creating a container (zip) etc 14:59:14 all right folks, that is about all we have time for, thank you for your participation 14:59:33 bye! 14:59:38 thanks 14:59:42 next week's meeting is at 20:00 UTC 14:59:44 thanks 14:59:57 #endmeeting