14:00:21 <nikhil> #startmeeting glance
14:00:21 <openstack> Meeting started Thu Sep 24 14:00:21 2015 UTC and is due to finish in 60 minutes.  The chair is nikhil. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:24 <ativelkov> o/
14:00:25 <openstack> The meeting name has been set to 'glance'
14:00:27 <harshs> o/
14:00:29 <bunting> o/
14:00:30 <nikhil> #topic agenda
14:00:30 <abhishekk> o/
14:00:31 <dhellmann> o/
14:00:32 <mfedosin> o/
14:00:34 <rosmaita> o/
14:00:35 <wxy_> o/
14:00:37 <nikhil> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda
14:01:24 <hemanthm> o/
14:01:27 <kragniz> o/
14:01:30 <bpoulos> o/
14:01:33 <jokke_> o/
14:01:54 <flaper87> o/
14:01:57 <nikhil> Welcome everyone
14:02:23 <nikhil> We have a short agenda today but this meeting is important at the advent of Mitaka planning
14:02:37 <nikhil> So, let's discuss some of those items towards the end of the meeting
14:02:47 <nikhil> and thanks for flaper87 for already putting Nova items on the list
14:02:54 <nikhil> Let's get startec
14:03:02 <nikhil> #topic updates
14:03:13 <nikhil> #info artifacts updates
14:03:13 <jcook> o/
14:03:28 <ativelkov> so, we've got all the blockers merged as fast-track patches
14:03:39 <ativelkov> thanks to all who helped reviewing them
14:03:40 <mfedosin> that's cool
14:04:01 <abhishekk> #link https://etherpad.openstack.org/p/fastTrackPatches
14:04:09 <ativelkov> Murano project, as our first (hopefully not the last) adopter has confirmed artifacts are working properly for them
14:04:49 <nikhil> :)
14:04:51 <ativelkov> The feature is enabled as experimental (for obvious reasons) in Murano-Liberty, all are loking forward for our API to be stable in M
14:04:59 <nikhil> ++
14:05:20 <nikhil> Mitaka is going to be API centric!
14:05:32 <jokke_> ativelkov: \o ltns ... good to hear that it actually works ofr them :)
14:05:36 <mfedosin> we're still waiting for you article Alex ;)
14:05:41 <mfedosin> your
14:05:41 <ativelkov> Right now I am documenting artifact-related changes for Murano. When finished, will start working on the big "About-v3" article for openstack Wiki
14:06:15 <flaper87> ativelkov: ++ on documentation
14:06:22 <nikhil> So, I think we need to discuss some of the use cases that API v2 does not solve and put a Artifacts perspective into the domain
14:06:35 <flaper87> ativelkov: it'd be great to go slow about the V3 article
14:06:49 <flaper87> as in, I'd like us to be very careful on not creating more confusion around glance
14:06:59 <nikhil> There are some compelling use cases for the same however, the tradeoffs exist and having those out there would help
14:07:33 <ativelkov> flaper87: the idea behind the article is to get some community awareness before the talk at the summit
14:07:43 <flaper87> what talk ?
14:07:46 <flaper87> :P
14:08:03 <nikhil> one thing that everyone seems to be disjoint on is the reason for existence of certain features, so idea behind Artifacts would be great to have documented and get inputs from interested developers, operators, etc.
14:08:07 <flaper87> Let me put my concern in other words: We're trying to solve a huge confusion and mess with v2 right now
14:08:29 <nikhil> That's mostly true
14:08:30 <flaper87> I'd like us to be careful on how we sell (or if we should at all do that in this phase) the artifacts work
14:09:02 <flaper87> I'm not saying to ignore it and to not write things down. I'm just saying lets be careful with the wording. I'm happy to provide feedback on the article
14:09:03 <ativelkov> we have an approved talk on the main summit conference on the evolution of Glance APIs
14:09:03 <nikhil> We are trying to solve the problem os interoperability around a ecosystem development around OpenStack "Images"
14:09:32 <ativelkov> "we" stands for nikhil and me :)
14:09:54 <dhellmann> yes, I think the rest of the community is going to be somewhat upset if the emphasis for this summit is about a new API when the existing one is not seen as finished
14:09:58 <nikhil> And we are targeting our v2 Images API for that standard given it works out for all the critical use cases
14:10:04 <flaper87> nikhil: sure, and the confusion on whether you need tasks to create images or not. Interoperability is just the technical side of it
14:10:25 <ativelkov> flaper87: sure, the wording should be very careful. The idea is to explain things, not to make them more confusing :)
14:10:34 <nikhil> ativelkov ++
14:10:48 <rosmaita> ativelkov ++
14:10:54 <flaper87> I'm just asking for us to be more careful on that side, especially because we're talking about an experimenting API
14:11:03 <flaper87> I'm not saying it shouldn't be written down.
14:11:06 <flaper87> that's all
14:11:08 <rosmaita> yes, but it's been in the works for 3 cycles
14:11:12 <dhellmann> ativelkov: I would resist the urge to make schedule predictions, for example
14:11:33 <ativelkov> dhellmann: noted
14:11:52 <dhellmann> rosmaita: yes, many folks outside of the project think it is a distraction from finishing the image API work. I don't know if that's true, I'm reporting the perception.
14:11:56 <nikhil> Schedule predictions seem tentative anyways. It about use cases and addressing them right for a long term support, dhellmann
14:12:02 <flaper87> rosmaita: again, "in the works" != stable or productoin ready
14:12:29 <nikhil> and that being the reason for this talk
14:12:40 <dhellmann> nikhil: sure, but if someone leaves that talk with the impression that there is yet another API coming "soon" that will further damage the image of the project as being unable to complete its commitments
14:12:56 <ativelkov> If we do not explain things, we may be asked the questions of different kinds, i.e. why the artifacts are not part of v2 if it isgoing to be included in DerCore. That's why we need all that explanations in some centralized place
14:13:09 <dhellmann> so, congratulations on getting the talk, and I hope it goes well, but consider the broader impact as you write it
14:13:34 <dhellmann> ativelkov: yep, that's a good point to cover
14:13:34 <flaper87> dhellmann: ++ that's what I was trying to pass as a message. Thanks!
14:13:40 <nikhil> yeah, and the reason for the talk is to reduce that confusion
14:13:59 <nikhil> get the use cases correct and everyone on the same page
14:14:05 <ativelkov> Also, the talk is not just aoput the v3: it is exactly about the evolution of Glance API: from V1  to v@ and then eventually to v3
14:14:19 <ativelkov> v2* :)
14:14:23 <nikhil> not distract people into thinking incorrectly
14:14:51 <nikhil> ativelkov yeah, thanks for that explicit clarification. seemed important and missed
14:15:06 <flaper87> ativelkov: I'd love to provide feedback on the article, if you don't mind.
14:15:14 <ativelkov> flaper87: sure!
14:15:30 <flaper87> ativelkov: thanks!
14:16:33 <nikhil> #info drivers meetings updates
14:17:15 <nikhil> The ova lite feature was not able to get merge in Liberty and the dev team has been notified to target it for M
14:18:01 <nikhil> We had a discussion around the concept of "Image", v2 APIs and the confusion
14:18:27 <nikhil> If you have feedback, this etherpad is where the collab is happening
14:18:27 <nikhil> https://etherpad.openstack.org/p/glance-upload-mechanism-reloaded
14:18:41 <jokke_> hopefully they don't give up and wake again at the end of the M cycle ... I think it's really close, we just ran out of time
14:19:05 <flaper87> I'd also like to mention that we're planning to have a quick video call on Tuesday at 15 UTC. Everyone interested is invited!
14:19:16 <flaper87> please, ping me and I'll provide details
14:19:20 <nikhil> malini wants it in early M and we should give them early feedback as a courtesy
14:19:30 <dhellmann> flaper87: I'd like to be on that call
14:19:36 <nikhil> I had pushed for it (think) in L2
14:19:38 <flaper87> dhellmann: you got it!
14:20:35 <nikhil> and the feedback had started coming in early Sept so just as a courtesy -- I would like one of the first feedback on this spec (besides other things of course)
14:20:54 <jokke_> flaper87: was really nice of you to pick time outside of your options that suites really badly :P
14:21:19 <nikhil> yeah, thanks for considering the round globe
14:21:42 <flaper87> Anything that helps making Glance better and finding a solution for this issue :)
14:21:53 <nikhil> ++
14:22:17 <nikhil> #topic Was there a reason checksum and size were left out of v2 server? (bunting)
14:22:41 <nikhil> bunting can you please elaborate on the issue?
14:22:41 <bunting> Yeah, i noticed this whilst working on osc and was wondering why this changed?
14:23:01 <bunting> For example checksum was in my opinion a useful feature
14:23:07 <bunting> For data integrity
14:23:17 <nikhil> wait, checksum on the image exists
14:23:20 <nikhil> it's a base prop
14:23:32 <rosmaita> but not exposed in the compute api
14:23:32 <jokke_> yes the checksum gets calculated
14:23:34 <nikhil> (immutable one for that matter)
14:23:42 <rosmaita> i think bunting is talking about nova api
14:23:47 <nikhil> oh
14:23:52 <rosmaita> i think?
14:24:05 <nikhil> bunting ?
14:24:12 <bunting> I mean when you create a image you can't supply a checksum to have the image deleted if its diffrent
14:24:24 <dhellmann> bunting: are you using the nova proxy to glance, or talking to glance directly?
14:24:27 <bunting> It seems this functallity is gone
14:24:41 <bunting> glance directly
14:24:45 <nikhil> user level validation
14:24:55 <jokke_> afaik nova does not support v2
14:25:10 <flaper87> jokke_: see the next topic for this meeting
14:25:12 <flaper87> ;P
14:25:16 <flaper87> jokes apart
14:25:18 <rosmaita> bunting: could you please explain your use case?
14:25:18 <bunting> nikhil: Oh so its moved so the user manually checks it now?
14:25:29 <dhellmann> bunting: so you're saying you used to be able to upload an image with a given name and if there was already an image with that name but a different checksum then the old image would be removed and replaced?
14:26:05 <bunting> So if you do glance image-create --checsum hash - if it does not meat hash the image would automatically be deleted
14:26:09 <jokke_> dhellmann: no, if thew checksum provided with image create didn't match the checksum calculated in the server the image used to be killed
14:26:13 <flaper87> I think bunting is saying that it was possible to upload the image + checksum in v1 and it'd check whether the uploaded checksum was the same as the uploaded image's
14:26:18 <dhellmann> jokke_: ah
14:26:32 <bunting> flaper87: Yeah
14:26:47 <dhellmann> ok, thanks for clarifying -- I'm still learning about the glance features :-)
14:27:03 <jokke_> on v2 it changed that we don't take checksum as input but we just calculate the checksum in the server, if the returned checksum does not match what the user has it's user's discreet to decide what happens next
14:27:04 <nikhil> bunting what is the response while uploading w/ checksum?
14:27:19 <nikhil> jokke_ correct
14:27:21 <rosmaita> that sounds like a python-glanceclient feature
14:27:41 <nikhil> I'm a bit worried on the return response
14:27:51 <nikhil> as it's not disallowed afaict
14:27:53 <flaper87> I think that's useful too and, tbh.
14:28:04 <jokke_> nikhil: iirc it's forbidden
14:28:15 <bunting> flaper87: Yeah to me it sounds like a useful feature
14:28:24 <bunting> to have it automatically killed
14:28:27 <flaper87> Leaving that to glanceclient means that you have to create the image, upload the whole thing, check if the checksum is the same that you had, and then delete the image if it doesn't match
14:28:32 <jokke_> valueated from same list as you would try to specify owner manually
14:28:45 <bpoulos> bunting: that sounds a little bit like what is currently available with signature verification
14:28:50 <nikhil> https://github.com/openstack/glance/blob/master/glance/api/v2/images.py#L375
14:28:51 <flaper87> it seems a bit of a waste considering that we already calculate the checksum on the server.
14:28:53 <nikhil> jokke_ ^
14:29:00 <bpoulos> you can provide a signature of your checksum, and it will be killed if the signature doesn't validate
14:29:18 <bunting> Ah so its now moved to signature?
14:29:29 <flaper87> bpoulos: right!
14:29:30 <bpoulos> i think you could use the signature for what you're wanting
14:29:40 <nikhil> it's not actually moved
14:30:03 <flaper87> it just accidentally happens to be there too
14:30:04 <nikhil> rather a more advanced way of image validation exists
14:30:05 <flaper87> :P
14:30:24 <bunting> nikhil: Okay
14:30:45 <jokke_> For jury reading the logs, please forget anything said on this topic to avoid confusion around the API :P
14:30:54 <jcook> clarifying question: the current behavior is to pass in signature ahead of upload, correct?
14:31:01 <bpoulos> correct
14:31:03 <nikhil> yep
14:31:08 <bpoulos> if you pass it after, it doesn't validate
14:31:08 <jcook> therefore, we can use this for immutability and retries
14:31:19 <nikhil> jcook correct
14:31:22 <flaper87> yup
14:31:32 <jcook> if you upload, it fails, you upload again and sig doesn't match original sig, fail
14:31:39 <nikhil> I think we are trying to narrow down on the immutability clause on just "active"
14:31:43 <flaper87> but it requires to have signature configured and everythin
14:31:45 <jcook> perfect, I'll update the upload retry spec
14:31:48 <jokke_> jcook: nope ... you cannot reupload on killed image ;P
14:32:00 <nikhil> however, we first need to contain the race conditions to avoid major bugs
14:32:03 <jcook> jokke_: https://review.openstack.org/#/c/206250/
14:32:10 <nikhil> jokke_ ^
14:32:11 <jcook> jokke_: not yet :)
14:32:36 <nikhil> jcook I think we sort of need a set_and_test on that retry thing
14:33:11 <jcook> as in upload and verify it is right thing?
14:33:20 <nikhil> better journaling (in other words)
14:33:46 <nikhil> keep the upload, deletes (cleanup), verification non-racy
14:33:49 <jcook> I was thinking send signature first attempt, retry as much as you want (to inifinity and beyond), it won't succeed until original sig matches and you cannot change sig
14:34:18 <jcook> so your upload could succeed, but it would fail because the original sig you sent in very first attempt did not match
14:34:28 <jcook> the sig is the immutable bit, and it won't go active until sig matches
14:34:33 <jcook> at that point a retry is a NOP not a fail
14:34:44 <jcook> sig already matches, don't bother doing anything
14:35:29 <nikhil> sounds good, we can discuss impl details on this later on the spec.
14:36:03 <nikhil> I guess the validation point that jcook  and bunting want (for different reasons) is covered here?
14:36:20 <jcook> +1
14:36:24 <bunting> nikhil: covered for me
14:36:33 <nikhil> thanks!
14:36:54 <nikhil> moving on. (For now)
14:37:01 <nikhil> #topic Nova migration from Glance v1 -> v2 (flaper87)
14:37:12 <flaper87> o/
14:37:14 <mfedosin> oh yeah
14:37:27 <flaper87> I don't have much to say other than give some updates on the current status and plan:
14:37:37 <jokke_> this sounds like familiar topic ... have I heard about this before?
14:37:46 <flaper87> I've been talking with some folks trying to identify who can work on this and help moving this effort forward
14:38:15 <flaper87> mfedosin: has volunteered to help with code from the Glance team, whereas sudipto from nova, has raised his hand to help with code as well
14:38:21 <flaper87> This is great!
14:38:24 <flaper87> Thanks to both
14:38:24 <mfedosin> yup
14:38:28 <flaper87> On the other hand
14:38:35 <dhellmann> great!
14:38:41 <flaper87> I'll be helping with the spec and I'll be working with jaypipes on it
14:38:50 <mfedosin> so, my goal for next month is to test Flavio's patch, split it in several small patches and help in reviewing the spec
14:39:03 <jokke_> flaper87: I'd like to get pause here
14:39:32 <jokke_> I think it's extremely important to get the API mess with defcore ironed out before we move forward with Nova
14:39:53 <dhellmann> the nova work is blocking defcore, too
14:40:03 <dhellmann> I think the image create and nova work can happen in parallel
14:40:05 <hemanthm> jcook ^
14:40:14 <jokke_> Last thing I want to see is that we get Nova migrated to current state and with defcore decide to bring up some new standard upload process
14:40:22 * jcook reading back
14:40:41 <jokke_> dhellmann: work itself yes, but I think the main principles needs to be agreed first
14:41:09 <jcook> ah the upgrade to v2 or glance seam in nova spec stuff
14:41:11 <dhellmann> jokke_: what parts of glance's v2 api that nova is consuming do you expect to change?
14:42:34 <nikhil> So, one very important thing
14:42:42 <dhellmann> I'm worried that, given the amount of time it tends to take to get work like this done in  nova, if it is not started soon it will not be finished this cycle
14:42:50 <jokke_> dhellmann: iiuc the biggest questionmark currently is how the images will be brought into glance in future. Is it the images API upload as it stands now, do we need to change that or bring something new at the side to provide the needed or will it go via some task interface
14:42:53 <nikhil> we should get a new spec in Nova approved possibly prior to summit
14:42:56 <flaper87> sorry, got distracted. Here now.
14:42:59 <flaper87> so
14:43:01 <jcook> https://review.openstack.org/#/c/194303/
14:43:01 <flaper87> Back to the spec
14:43:26 <flaper87> As I mentioned before, I'm working on the spec now
14:43:27 <jcook> that's my spec on the matter, and I think I spent all my wind on it there ^ I'll try to update the spec today
14:43:29 <dhellmann> jokke_: so start the nova work with the parts of the API not related to image upload
14:43:33 <flaper87> and I'll have something ready for review soon
14:43:48 <flaper87> I'll work with jaypipes and obviously, it'll need feedback from everyone
14:44:08 <nikhil> dhellmann I am not worried about that as I have been discussing with Nova how to adjust that in one cycle. there are many efforts around images in nova and it would be good to get this one bit done asap
14:44:53 <nikhil> flaper87 thank you. One thing Nova folks wanted is to get first and early feedback from GLance community
14:45:27 <nikhil> we should avoid what happened in Liberty
14:45:30 <flaper87> Yes! I think we should do that asap, which is why I've been working on the spec today
14:45:48 <flaper87> I hope to have something ready soon for review by y'all
14:46:08 <nikhil> Also, one thing I have as input is that people interested in images stuff in Nova, please attend their meeting and help with this inter-project communication work
14:46:31 <nikhil> the meetings can be at odd hours due to TZ adjustments
14:46:46 <flaper87> that'd be useful too. I can't commit to that as the nova meetings seem to happen in weird TZs for me but we can figure that out
14:47:01 <flaper87> I'll be syncing back from/to nova
14:47:08 <flaper87> so, expect updates from me on this side
14:47:13 <nikhil> help from around the world will be appreciated, but then again it would be best to have one or two dedicated images liaisons for M.
14:47:24 <nikhil> flaper87 sounds good!
14:48:00 <nikhil> with a little over 12 mins, do we have anything more on this topic?
14:48:02 <flaper87> I think we have enough people to get this going. More are totally welcome but with this small sub-group, we should be ok to move this forward
14:48:06 * flaper87 stfu
14:48:17 <nikhil> ++
14:48:37 <nikhil> #topic winding up Liberty RCs
14:48:39 <flaper87> nikhil: I bet that's re "Stfu"
14:48:41 <flaper87> :P
14:48:50 <nikhil> #info     RC1 coming, https://review.openstack.org/#/c/223633/ example config files ( jokke_ )
14:48:57 <nikhil> flaper87 lol, no :P
14:49:29 <flaper87> Is that the only thing blocking RC1 ?
14:49:37 <flaper87> I'll get to it right after the meeting
14:49:42 <jokke_> bit hand to sabari who took time yesterday to review the change and spotted some inconsistencies ... should be all good now
14:49:50 * flaper87 hopes it is
14:49:53 <flaper87> jokke_: ++
14:50:01 <flaper87> sabari ++
14:50:35 <dhellmann> jokke_: does glance use the config sample generator from oslo.config?
14:50:47 <nikhil> I was hoping this one gets a pass of testing https://review.openstack.org/#/c/221307/
14:50:52 <jokke_> dhellmann: it's not automated
14:50:53 <flaper87> dhellmann: yes
14:50:58 <flaper87> but not automated
14:51:18 <jokke_> dhellmann: those are pulled from it and some adjustments done so we got all the sections in we need
14:51:43 <dhellmann> jokke_: ok, maybe we can discuss what adjustments were needed offline so we can make this easier for you in the future
14:52:21 <jokke_> dhellmann: sounds good ... this is actually the major move to the autogenerated configs ... next round should be lots easier
14:52:28 <dhellmann> k
14:52:46 <nikhil> I think we need more awareness around adding configs in general
14:53:04 <nikhil> (plus review)
14:53:42 <jokke_> nikhil: do we have anything else we're waiting for RC1?
14:53:48 <nikhil> #info if RC1 goes out this week, I have asked for a possible RC2 next week
14:54:25 <nikhil> jokke_ no blockers, but good to get this in RC1 for some thorough testing  https://review.openstack.org/#/c/221307/
14:54:48 <nikhil> RC1 most likely will be cut right after the config refresh
14:55:12 <ativelkov> Are we going to make proposed/liberty branch with the release of RC1?
14:55:13 <jokke_> ok, so please folks lets get the reviews going ... it's beast
14:55:54 <nikhil> dhellmann is our rel-mgmt expert :) please educate us on the timeline for proposed/liberty branch
14:55:57 <jokke_> ativelkov: it's gonna be stable/liberty treated like proposed/ until final release iiuc
14:56:11 <jokke_> dhellmann: can correct me if I'm wrong
14:56:59 <dhellmann> yes, that's right, we'll create the stable/liberty branch with special acls that lock it down tightly until the release is finished, and then change the acls so it acts like a normal stable branch
14:57:03 <nikhil> we can continue this in open discussion, just to give some time for last minute awareness topics
14:57:05 <nikhil> oops
14:57:24 <nikhil> #topic open discussion
14:57:57 <nikhil> I wanted to put this out there for more awareness -- https://review.openstack.org/#/c/206431/
14:58:07 <bunting> I am happy to work on osc, bty following our disscussion about it last week
14:58:16 <flaper87> I don't recall if we have an etherpad to collect topics for Mitaka already
14:58:32 <nikhil> just to help educate Nova team if that's the best way of going about it
14:58:40 <flaper87> If we don't, it'd be great to create one. If we do, please, remember to add things there!
14:58:42 <flaper87> :D
14:58:49 <flaper87> I seem to recall there being one
14:59:00 <jokke_> nikhil: I really liked what they are planning there and it indeed makes way more sense to do in Nova than in Glance
14:59:36 <nikhil> jokke_ agreed
15:00:42 <nikhil> I am trying to find that etherpad, will link if offline if exists
15:00:48 <nikhil> But we are out of time now
15:00:54 <nikhil> THanks all for joining!
15:00:57 <nikhil> #endmeeting