14:01:08 <flaper87> #startmeeting Glance Drivers
14:01:09 <openstack> Meeting started Tue Nov 17 14:01:08 2015 UTC and is due to finish in 60 minutes.  The chair is flaper87. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:11 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:14 <openstack> The meeting name has been set to 'glance_drivers'
14:01:23 <mclaren> o/
14:01:28 <rosmaita_> o/
14:01:36 <flaper87> w000h0000
14:02:16 <flaper87> I just added 1 topic to the agenda as there are some things to discuss. mclaren has raised some concerns that would be great to have a short discussion on
14:02:26 <nikhil_k> o/
14:02:30 <flaper87> #topic Glance image import process: https://review.openstack.org/#/c/232371/ (flaper87 / mclaren)
14:02:34 <flaper87> nikhil_k: helloooooooo
14:02:42 <rosmaita> i also want to discuss some of doug's comments
14:02:43 <nikhil_k> yehlooo!
14:02:53 <flaper87> nikhil_k: :P
14:03:00 <flaper87> rosmaita: +1
14:03:04 <flaper87> so, mclaren you've 15 mins
14:03:09 <flaper87> :D
14:03:12 <flaper87> go crazy
14:03:20 <mclaren> heh, ok
14:03:35 <mclaren> I guess I have two things on my mind
14:04:29 <mclaren> 1) What we're proposing seems like a like of code/maintenance -- we just need to ensure it's all absolutely required
14:05:21 <mclaren> 2) It's unclear to me what will happen to boring old image upload -- which is actually the closest we currently have to a standard way to get your images into clouds
14:06:12 <mclaren> following on from (1) I guess in part I don't want us to write something that doesn't end up being used, is there a risk that folks would just keep using direct upload?
14:06:51 <flaper87> I think 1 and 2 are somewhat related and it'll depend on us to provide a way to move from the old way of doing things to the new one
14:07:04 <mclaren> yes 1 and 2 are related, I agree
14:07:19 <flaper87> In the future, we can even switch the old one to admin only
14:07:22 <flaper87> by default
14:07:24 <rosmaita> mclaren: we are not going to use direct upload in the public cloud, for sure
14:07:46 <mclaren> rosmaita: you're also not going to use the import from glance
14:08:01 <rosmaita> no, we are going to align with upstream
14:08:08 <rosmaita> otherwise i wouldn't be working on this at all!
14:08:11 <nikhil_k> I would like to think of it as a popular way rather than standard way as there are technical limitations/questions around it (direct upload).
14:08:21 <flaper87> nikhil_k: ++
14:08:25 <flaper87> I like that
14:08:30 <rosmaita> nikhil_k: +1
14:08:39 <mclaren> the question is more around folks who currently allow direct upload -- how happy are they? Are they strongly motivated to switch? Do they want that to be effectively de-emphasissed
14:09:50 <flaper87> I'm not willing to assume things about how they feel and instead, I think we should focus on the current technical issues that have been identified
14:10:10 <rosmaita> i have sent requests for comments to product WG, didn't get anything
14:10:13 <flaper87> As far as moving from one to the other goes, I still think providing a smooth way to transition will be the key
14:10:30 <rosmaita> i was hoping they could tell us how they see the cloud being used for image import
14:10:58 <rosmaita> also, API WG is silent ... i am beginning to get a complex
14:11:11 <rosmaita> guess i need to attend a meeting and ask directly
14:11:12 <mclaren> We can't ignore it's what 99% of sites are doing right now, and that means user's have certain expectations. It would be ironic to break users' stuff in the name of interoperability. I'm not saying we can't go there, but I'd feel better if we had clear feedback from operators.
14:11:17 <flaper87> rosmaita: that'd be great
14:11:25 <rosmaita> ok, i will do that
14:11:46 <nikhil_k> I think the # of users/operators who are preferring direct uploads are more as they are smaller deployments, mostly private and are not necessarily exposed to security, scalability and reliability threats.
14:11:52 <flaper87> one of the main points of every discussion has been not breaking existing users, how are we doing that?
14:12:11 <mclaren> If sites switch off direct upload
14:12:19 <mclaren> in favor of a new mechanism
14:12:23 <flaper87> nikhil_k: the numbers we have are all from public clouds
14:12:25 <rosmaita> but they will do what their users want
14:12:47 <flaper87> mclaren: but again, that doesn't mean we'd break them
14:13:01 <rosmaita> i don't see the problem, stuart ... if the old way works, they can use it, if there's a new/better/safer way, they can work with their end users to adopt it
14:13:04 <flaper87> we've talked about mechanisms to help this transition (Even from a client perspective)
14:13:13 <nikhil_k> OTOH, we don't really have many larger public clouds coming and giving feedback as to what their issues are. or have they at the summit? (like cisco, walmart, bloomberg, verizon (not public but large) ) ?
14:13:49 <flaper87> nikhil_k: ah, I meant the numbers w.r.t how many folks are using upload as an endpoint
14:13:59 <flaper87> nikhil_k: but yeah, what you said is true as well
14:14:27 <flaper87> rosmaita: Stuart's concern is that adding `/bikeshed` is too much work and that we may end up in a place where no one migrates to it
14:14:31 <flaper87> which means we'd waste time
14:14:42 <rosmaita> ok, i misunderstood
14:14:51 <rosmaita> sorry, stuart
14:14:57 <mclaren> pretty much, and you have three ways to upload images!
14:14:58 <rosmaita> so i kind of agree with him there
14:15:01 <flaper87> My current point is, I don't think there's a way we can use the old `/file` endpoint without breaking the interoperability
14:15:26 <flaper87> We talked about re-using it at the summit but we ended up agreeing it's not a good idea
14:15:49 <rosmaita> well, maybe we just say phase 1: import with task processing is via swift only
14:15:51 <mclaren> So I think the potential 'killer app' of the 'new way' is allowing asynchronous processing of the image (to either validate or convert it)
14:16:00 <rosmaita> mclaren: ++
14:16:16 <flaper87> right
14:16:29 <flaper87> one of the things we discussed yday on -glance was to break this into 2 cycles
14:16:34 <mclaren> but it's not clear if that would be sufficient for operators/users to get over the complacency of just using the existing way
14:16:40 <flaper87> because it's getting bigger and we're close to M-1 already
14:16:53 <rosmaita> flaper87: ++
14:17:27 <flaper87> I don't mind holding off the `/bikeshed` work until N as long as we disable this "new way" by default until it's complete
14:17:29 <mclaren> I guess it comes down to "what do operators want"? If there's a real appetite for the 'new way' then we go for it
14:17:33 <nikhil_k> That's true, can we break this down into baby steps?
14:17:47 <flaper87> mclaren: fwiw, we've asked that already
14:17:56 <flaper87> the spec is part of their feedback
14:18:05 <flaper87> I don't think we're ignoring what operators want
14:18:06 <rosmaita> is there any appetite for just using the current tasks framework, maybe with a more specified schema?
14:18:56 <flaper87> wait, wait, wait. I still think we should do this work, we can start with things we've agreed on already
14:18:57 <mclaren> flaper87: hmm, really? Can you be more specific? Do you mean the vancouver summit feedback?
14:19:03 <rosmaita> because basically, that's what we're doing anyway, just with a fancier front end
14:19:55 <flaper87> mclaren: not just vancouver. IIRC, the etherpad and the spec had some OPs chiming in. The summit session had OPs as well. Some of the complains/requests are being considered (automatic task trigger)
14:20:46 <mclaren> flaper87: some indication that operators in the real world would actulally switch to using the 'new way' once it's been implemented would be great
14:21:02 <rosmaita> flaper87: here's my frustration ... when i talk to people individually at the product wg, people either (A) don't want to expose end-user upload at all, or (B) want the ability to validate end-user data before using it
14:21:11 <flaper87> If I can be completely honest, I'm kind of sad we're back to discussing this because I feel like we're going backwards. I think it's healthy to ask ourselves answered questions from time to time but we've been talking about this since ~2 months before the summit
14:21:13 <rosmaita> but there is silence on the spec
14:21:32 <flaper87> we're close to M-1 and now we're back to "why ware we doing this?" "Do we have all the info?"
14:22:01 <flaper87> rosmaita: yeah, silence on the spec is sad
14:22:23 <flaper87> but at this point, I believe this has been communicated quite openly
14:22:31 <nikhil_k> has anyone sent the spec link and review request to the operator and general openstack MLs?
14:22:33 <flaper87> I'll join one of the OPs meetings
14:22:36 <flaper87> nikhil_k: yeah
14:22:43 <rosmaita> nikhil_k: i did last week
14:22:43 <nikhil_k> cool
14:22:46 <mclaren> So let me explain my queries at this point ... I thought we were going to be forced to have a single API that looked the same everywhere. After the summit having different paths for swift and glance seemed to be considered ok
14:23:26 <mclaren> So at this point I'm hitting on 'what does the different glance path look like'
14:23:35 <rosmaita> right, because it would still "look the same" with the upload method clearly specified
14:24:08 <mclaren> and tbh, I'd find it hard to spend 6 months writing the code while still wondering that :-)
14:24:19 <flaper87> But the current way is not discoverable
14:24:31 <rosmaita> flaper87: that can be fixed easily
14:24:54 <rosmaita> add "use import task" to POST /v2/images response header
14:25:37 <nikhil_k> yes, when I had this conversation with month 8 months back, I proposed a simple discovery mechanism to simplify users experiece
14:25:38 <rosmaita> and define the task schema more clearly
14:25:40 <flaper87> I guess but we also wanted to unify things in a single process. For example, the old `/file` endpoint doesn't trigger the task engine
14:25:44 <nikhil_k> oops,
14:25:47 <mclaren> the discoverability will look the same no matter what the actual calls are. The discoverability just tells us which fork of the path we go down ... glance or swift
14:25:49 <nikhil_k> with Monty* :)
14:26:35 <nikhil_k> I see 3 main concerns broken down from mclaren's raise
14:26:46 <mclaren> flaper87: yes, the 'old way' doesn't give asynchronous processing. If there's a clear signal from operators that they want *and will deploy* that, then the debating is done I think
14:27:01 <flaper87> mclaren: there have been several signals
14:27:13 <flaper87> you can read that in my automatic-task-triggering spec
14:27:25 <flaper87> there were at the vancouver summit
14:27:41 <mclaren> ok, more specifically, are they happy to switch off direct upload in order to get async processing
14:28:09 <flaper87> Yes, that's my understanding from their feedback and the feedback from other folks in the community
14:28:46 <flaper87> if we want to go and ask again on the OPs mailing list, then fine
14:28:57 <nikhil_k> 1) we have to maintain 2 old APIs as we have API contract (right?) 2) we need to add a bunch of code to support interop that needs to converge into defcore standards and we don't have a BETA with testing done, so that will involve code writing, testing and improvements 3) the support value and documentation for all 3 APIs would be massive even to the extent that a new operator and user would be a bit taken a
14:28:57 <nikhil_k> back by the number of ways to upload image and would need bit of handholding
14:29:53 <nikhil_k> umm, I am not too sure about it
14:30:14 <mclaren> I'd personally feel happier with some direct feedback from ops on this. I'd be happy to send the mail myself.
14:30:16 <nikhil_k> at least the operators who have openstack deployed in small labs, offices want the easier way
14:30:46 <nikhil_k> easier == direct upload (or even just ability to set location)
14:30:52 <flaper87> mclaren: please do. As I said, there has been direct feedback from ops some more won't hurt
14:31:29 <nikhil_k> flaper87: I realized that there is one potential look hole into using location setting even with the new way, I will add that comment on the spec and continue the convo there
14:31:35 <nikhil_k> loop*
14:31:43 * nikhil_k is quite sleepy
14:31:45 <flaper87> nikhil_k: awesome, thanks
14:31:50 <flaper87> we ran out of time
14:32:00 <flaper87> thanks folks, I guess we need to discuss this more
14:32:07 <mclaren> will disabling direct upload will require two glance clusters?
14:32:09 <rosmaita> so action items? rosmaita - product WG mtg, stuart - ops list
14:32:17 <flaper87> rosmaita: yes
14:32:25 <flaper87> rosmaita: also, API wg
14:32:27 <flaper87> :)
14:32:32 <rosmaita> ty!
14:32:34 <rosmaita> will do
14:32:40 <flaper87> nope, thank YOU
14:32:41 <flaper87> :D
14:32:44 <mclaren> will do
14:32:47 <flaper87> #endmeeting