14:03:20 #startmeeting glance 14:03:20 getting the agenda 14:03:20 Meeting started Thu Aug 6 14:03:20 2015 UTC and is due to finish in 60 minutes. The chair is sigmavirus24. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:03:21 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:03:24 The meeting name has been set to 'glance' 14:03:25 #chair nikhil_k 14:03:25 Current chairs: nikhil_k sigmavirus24 14:03:34 #link https://etherpad.openstack.org/p/glance-team-meeting-agenda 14:03:38 That's our agenda ^ 14:03:43 o/ 14:03:49 o/ 14:03:52 o/ 14:03:54 Courtesy meeting reminder: ativelkov, cpallares, esheffield, flaper87, flwang1, hemanthm, ivasilevskaya, jokke_, kragniz, lakshmiS, mclaren, mfedosin, nikhil_k, Nikolay_St, Olena, pennerc, rosmaita, sigmavirus24, sabari, TravT, zhiyan, pkoniszewski, krykowski, ajayaa, GB21, bpoulos, harshs, abhishekk, bunting 14:04:06 o/ 14:04:14 o/ 14:04:28 sigmavirus24: thanks for starting 14:04:35 nikhil_k: all yours ;) 14:04:44 o/ 14:05:08 nikhil_k: take it away :D 14:05:40 SO, we had the midcycle last week and there were a couple of discussion items that I wanted to bring up here as the write couldn't be completed in time. Other items can come on the ML 14:05:51 #topic glance_store and glance 14:06:12 The relationship, as you all know, is bitter sweet 14:06:37 and a proposal was made to put back the store modules in glance tree (Same repo) 14:06:53 ++ :P 14:06:57 Why? 14:07:02 I'm interested in knowing the reasoning behind that 14:07:05 no objections were raised at that time and the use cases known intially weren't supporting the need of a separate lib 14:07:22 I see absolutely no benefits to doing so and there was no reasoning provided for it on the ML or elsewhere as part of a summary 14:07:37 for example, glance_store being consumed by other projects isnt considered favorable 14:07:57 my personal reasoning would be that having it separate causes more issues than solves them as there is no use case to have them separate 14:08:06 sigmavirus24: ? 14:08:07 In addition to that, I'd also like to mention that we've put lots of efforts in splitting out and merging it back gives us nothing 14:08:14 jokke_: what issues? 14:08:23 nikhil_k: why isn't it considered favorable? 14:08:26 the summary can be as long as needed 14:08:57 glance_store being separate lib has one extra surface to break our dependencies and gating due to the fact that we gate glance against released libs, not against the head 14:08:59 nikhil_k: sure, but this discussion wasn't forwarded to the mailing list. 14:09:19 jokke_: that's true for every library we have 14:09:22 jokke_: so we add a reverse gate at glance_store that tests with glance against the latest glance_store 14:09:25 there are ways around that 14:09:33 just check on #openstack-glance discussion today between myself and jamielennox 14:10:03 flaper87: true, so why have one extra there that has no benefots being separate 14:10:29 jokke_: there are benefits, it's just that people don't agree with those 14:10:34 flaper87: for the reason that the projects do not want to know the backend details, deal with authN/Z issues, creation of cyclic dependency between projects consuming glance which in turn consuming store 14:10:42 sigmavirus24: sure, but as said that's a benefit for you to merge it back in. I'd like to hear benefits keeping it out 14:10:55 jokke_: administrators may only have a bug in glance_store and only want to update glance_store to include that fix on a stable branch. They might not want to analyze all of the fixes between what they have installed and what they need for glance before upgrading 14:10:57 the discussion was complementary, so the summarization won't be accurate or complete 14:11:02 nikhil_k: if that's the case, then we should prevent users to get the locations and remove direct_url 14:11:09 guess, someone needs to start it 14:11:09 because that exposes details about the backend 14:11:12 I posit there will be fewer fixes in stable glance_store than stable glance 14:11:50 it's sad that for such an important topic there's not a good summary of what happened, TBH 14:12:06 sigmavirus24: so we should separate every module to their own repo's to give 200 packages for admins to keep track of 14:12:14 ? 14:12:27 jokke_: I expect better from you than strawman arguments 14:12:36 sigmavirus24: ++ 14:12:40 The store is not a service 14:12:40 um, is this glance_store? 14:12:49 mclaren: yup 14:12:53 well expression of sadness might be good but more important is to raise rational arguments for any outstanding use cases 14:12:53 That's my fault then. 14:13:20 I just threw out the idea of re-integrating it to see what folks thought. 14:13:33 jokke_: we could easily split out glance into micro-services though, I'm sure that would go great 14:13:37 the glance_Store topic was a bit unexpected and the summarization was even tricker with no one taking notes at that time for the same reason 14:13:39 nikhil_k: which I'm doing and I'd expect those arguments to be raised for us to understand why this is happening 14:13:47 sigmavirus24: I honestly would be really surprised any admin being interested to have projects split for them to keep track of 14:13:54 mclaren: I'm currently asking what's the reasoning behind that idea 14:14:02 I don't want to misinterpret people so trying to get some thoughts before creating another community deadlock 14:14:19 well it wasn't a firm proposal first of all 14:14:33 but my only reasoning bringing it back is that there is no known use cases for it and it causes overhead to maintain alone 14:14:37 jokke_: I guess following your logical progression, we should be merging requests back into Glance? 14:14:42 nothing more religious than that ;) 14:14:46 Because it's one less library for operators to be concerned with 14:15:01 I guess the motivation was that decoupling the store and the rest of the code can create some additional overhead/complexity. 14:15:10 sigmavirus24: if glance is only use case for requests, absolutely 14:15:15 jokke_: there are use cases, they just haven't been implemented yet because, oh well, nova still uses v1 and has some hacks to talk to glance 14:15:22 but I think there might be few others as well 14:15:55 mclaren: but it's been around for what, 2, 3, cycles? 14:16:02 flaper87: in that case the issue is that whether nova wants to use glance_store and that was proposed against 14:16:24 I'm pretty sure OPs/packages are way passed delivering it 14:16:38 with the proposal for a seam for glance as a long term solution the needs for store is less 14:16:40 nikhil_k: I keep missing the when and why, mostly the why 14:16:49 ^ 14:16:59 because we talked about this in Paris and everything was good 14:17:01 less == minimal 14:17:16 * flaper87 really doens't like the proposal of seam 14:17:20 :P 14:17:27 (as stated in the review) 14:17:35 my use case then was store was needed for some task operations but then we don't want users to use glance_store for that 14:17:46 so that use case has gone down the drain 14:17:57 nikhil_k: agreed 14:18:16 We've always been reluctant to let users consume glance_store 14:18:25 I guess in terms of the store the questions are: 1) is it/will it be consumed by anything other than glance 14:18:26 The real use case would be for the under-cloud 14:18:38 so, is the only outstanding use case now is deciding whther nova needs to use glance_store or not? 14:18:41 mclaren: there seem to be 2 answeres to that 14:18:56 also, in addition to nova's use case, we also have cinder 14:19:03 or more generic question like mclaren proposed 14:19:05 which interacts with images too 14:19:34 if you ask me, I'd say yes, there are still use-cases (the ones I've mentioned already) 14:19:39 I haven't heard anyone from cinder side being interested of consuming glance_store 14:19:58 jokke_: when the store was proposed, there was interest 14:20:08 we'd have to check again to make sure they didn't change their minds 14:20:30 alright, so this is exactly what we needed some action items! 14:21:13 #action flaper87, mclaren : check with cinder if they want to /should use glance_store 14:21:13 fwiw I don't feel terribly stongly about this. Basically if there's consensus it's unlikely to be used, and people think its a good idea we can put it back. If not, no problemo! 14:21:56 #action flwang (images liaison for nova): check with nova if they need to /should use glance_store 14:21:59 tbh, I'd rather have folks interested in more important tasks than merging glance_store back. Even if the original use-cases are gone 14:22:13 We should also carry this discussion to the ML 14:22:16 I don't see, as of now, glance_store as a big burden for us 14:22:21 sigmavirus24: ++ 14:22:27 For example flwang isn't in here and probably hasn't seen our mention and will not know they have an action item 14:22:27 that'd be easier than pinging each team 14:22:32 flaper87: agreed 14:22:36 * flaper87 will do that 14:22:41 * flaper87 ME! 14:22:48 sigmavirus24: not everyone affected by this are likely to respond on ML and I want a consensun and not a deadlock at the end of the cycle 14:23:09 nikhil_k: seriously, ML is the medium for cross-project discussions 14:23:14 lets ask those folks to respond 14:23:21 nikhil_k: ok, will check with cinder 14:23:25 or the cp meeting 14:23:32 which is very hard for me to attend because TZs 14:23:46 whoever sends to the ML, make sure it reaches cores/drivers/active contributors inbox without tags/filters 14:24:14 which I can do 14:24:22 nikhil_k: to be fair, I think cinder/nova cores monitor the ML much better than we do =/ 14:24:35 sigmavirus24: ++ 14:24:44 that's a preference problem that I can't solve, unfortunately 14:25:03 nikhil_k: whatever works for me, I'm happy to send it unless you prefer sending it. 14:25:12 irc is expected but governance doesn't enforce/soft enforce ML 14:25:25 nikhil_k: but that's also not playing nice with the community, which has chosen the ML as a medium for these dscussions 14:25:27 flaper87: I will send it tonight 14:25:33 nikhil_k: thanks 14:25:44 nikhil_k: you can cc folks there 14:25:47 flaper87: might be a good discussion topic for this cycle 14:26:15 nikhil_k: I've started that discussion N times but I digress (and agree) 14:26:32 :) 14:26:33 ok, moving on in the interest of time 14:26:43 #topic glance v2 reupload and image status change 14:27:09 so, glance v1 allows status changes as a part of the metadata update 14:27:41 glance v2 can result into unrecoverable 409s (conflicts during uploads) 14:28:20 there's a proposal for enabling that so that when a upload fails from say nova, a reupload to glance be possible on the killed state 14:28:24 status* 14:28:37 nikhil_k: remind me, which status changes does v1 allow users to do? thanks 14:28:58 #link https://review.openstack.org/#/c/206250/6/specs/liberty/upload-retry.rst 14:29:32 mclaren: PUT on active can put image to queued status. hemanthm can confirm as he raised this yesterday 14:30:11 this is big change in terms of the way uploads would work 14:30:45 so the existing v1 api allows the user to take the image out of active state? I hadn't realised... 14:30:52 :) 14:31:13 bug or feature? :-) 14:31:17 perhaps v3 is good place to consider such, specially thinking how much there has been cry to stabilize the v2 api 14:31:32 similar to this #link https://github.com/openstack/nova/blob/master/plugins/xenserver/xenapi/etc/xapi.d/plugins/glance#L184 14:33:02 wow, I had no idea! 14:33:11 mclaren: I'd say that's very much bug as it breaks the guarantee to immutability 14:33:17 tbh, I had no idea either 14:33:37 neither did I. I found out yday working on the upload retry spec 14:33:42 hmm, have we verified this? 14:33:47 I did 14:33:57 I haven't read the spec but I will 14:34:11 hemanthm: thanks 14:34:12 hemanthm: ok, so this will move and image from active back to queued? 14:34:17 and->an 14:34:42 https://github.com/openstack/nova/blob/master/plugins/xenserver/xenapi/etc/xapi.d/plugins/glance#L184 14:35:00 mclaren: yes. I created a snapshot and with a PUT operation, I could change the status to queued and back to active again 14:36:12 jeez 14:36:23 do we have a bug for this already? 14:36:36 Mindblown. 14:36:53 **2 14:36:56 flaper87: I think we should check if this is not a breaking change. It may be API backward incompatible 14:36:58 not, yet. Will create one after reproducing in devstack. I reproduced it in our staging env 14:37:42 it may come in the realm of API backward incompatible 14:37:49 thanks hemanthm 14:37:52 mmhh, true 14:38:15 hemanthm: so between state changes, does it allow to upload a new data into the image as well? 14:38:15 * flaper87 needs to think about this 14:38:16 anyways, let's give people time to digest this lightning strike 14:38:26 lol 14:38:31 i don't think it's a breaking change if it was never supposed to happen 14:38:40 jokke_: you can reupload if image is put again in queued state 14:38:42 rosmaita: that's my thinking too 14:38:47 jokke_: I have't tried that 14:38:59 rosmaita: that's a bug in the API design so, not really a breaking change 14:39:06 flaper87: +1 14:39:10 sorry, in the API implementation 14:39:17 which was designed to be immutable 14:39:18 also seems like a security problem 14:39:19 the original image bytes won't be handled properly ... lots of problems 14:39:23 rosmaita: agreed 14:39:25 we guarantee that the image is immutable ... this is security vulnerability and if it breaks someone abusing it, we seriously can't care 14:39:37 jokke_: +1 14:39:46 omg, did we just talked for 20mins on public, logged ,IRC about a security issue? 14:39:53 * flaper87 head -> desk 14:39:54 yes 14:39:55 :P 14:40:08 I think this is the issue with glance v1 exposed publicly 14:40:10 guess i shouldn't have used the "S" word 14:40:12 in our defense, no one knew anything about this 14:40:32 as most glance v1 are internal only and aren't subjected to this risk 14:40:49 moving on for now 14:40:50 sigmavirus24: it's a private cloud problem! 14:41:00 I mean, are supposed to be internal but they aren;t 14:41:14 #topic Reuse the deleted image-member 14:41:20 hemanthm: mind filing a bug for this? :D 14:41:29 (erm, this -> the above) 14:41:33 rosmaita: hm? 14:41:35 #link https://review.openstack.org/#/c/207042/ 14:41:41 flaper87: yes. I'm just holding off until I reproduce this on devstack, which I'm doing currently 14:41:43 that's the spec 14:41:48 hemanthm: awesome 14:42:01 sigmavirus24: i'm thinking only private clouds would expose v1 to end-users 14:42:01 #link https://review.openstack.org/#/c/190895/ 14:42:06 rosmaita: yeah 14:42:21 shalq: was that you? 14:42:21 hemanthm: thanks for that, please ping the -glance channel when you're done 14:42:29 yes 14:42:36 jokke_: sure 14:43:09 shalq: can you please elaborate on what you wanted to discuss 14:43:13 ? 14:43:36 the spec is ready for review, and we have a customer issue for that case. 14:44:14 I reviewed the code as well and it looked mostly good last seen 14:44:55 I'll get back to this spec tomorrow morning 14:45:02 can we wait till then? 14:45:07 I'd love to take a look at it 14:45:23 thanks, flaper87 14:45:37 shalq: thank you :) 14:45:45 shalq: all it needs are some functional and unit tests 14:46:26 okay, I will design some unit test for that later. 14:46:47 thanks 14:46:55 #topic Returning when illiegal body ( bunting ) 14:47:06 #link https://review.openstack.org/#/c/207150/ 14:47:22 bunting: around? 14:47:31 bunting: type faster! 14:47:32 As start points out on the review, using chunked encoding means that weob seems to remove the body 14:48:20 unless someone knows how to detect if a body exists in a chuncked encoding, it may mean that if it is not chuncked it will return 400 and if it is it will ignore the body 14:48:36 by quick look I'd rather say we do not deserialize it 14:48:59 basically we can't figure out a way to always detect if the initial request had a body or not 14:49:15 we're either being dumb, or webob just won't let us do it 14:49:23 webob.Request seems to be quite happy with all bodies on all requests 14:49:47 sigmavirus24: any idea if there's a canonical way to figure out if the request had a body? 14:50:15 we've tried a few things but keep hitting gotchas, eg is_body_readable doesn't seem to work for some HTTP verbs 14:51:28 hmm, yeahhh 14:52:21 there is also problem that some of the ways to identify body are invasive so we really don't want to check them on all requests 14:52:31 mclaren: in webob, I'm not sure 14:53:33 my ideas stock is empty momentarily 14:53:40 sigmavirus24: nikhil_k ok 14:53:45 guess, we can discuss more on the review/-glance 14:53:52 thanks mclaren 14:54:02 #topic reviews 14:54:03 so some mechanisms to access the body in the request-object actually changes content lenght etc. so if we check only requests we don't want to have body, we don't need to care and that's doable 14:54:03 sure, just thought someone might have seen/done this before 14:54:37 hi, https://review.openstack.org/#/c/207847/ 14:54:46 mclaren: I think there's some environ var you can set on wsgi.__ but it's been a while 14:54:46 ^^^ This patch fixes below three bugs, 14:54:57 https://bugs.launchpad.net/glance/+bug/1478690 - Request ID has a double req- at the start 14:54:57 Launchpad bug 1478690 in Glance "Request ID has a double req- at the start" [Undecided,In progress] - Assigned to Abhijeet Malawade (abhijeet-malawade) 14:54:57 https://bugs.launchpad.net/glance/+bug/1480191 - User can send 'request-id' from headers to glance api 14:54:59 https://bugs.launchpad.net/glance/+bug/1480196 - Request-id is not getting returned if glance throws 500 error 14:54:59 Launchpad bug 1480191 in Glance "User can send 'request-id' from headers to glance api" [Undecided,In progress] - Assigned to Abhijeet Malawade (abhijeet-malawade) 14:55:00 Launchpad bug 1480196 in Glance "Request-id is not getting returned if glance throws 500 error" [Undecided,In progress] - Assigned to Abhijeet Malawade (abhijeet-malawade) 14:55:14 I have removed the code which is generating the request-id and instead using oslo_middleware's RequestId in the default paste deploy pipeline. 14:55:41 abhishekk: the second one is by design, not a bug 14:55:45 Please review and give your feedback for the same. 14:56:22 jokke_: ok, but this can be used to flood the log files 14:56:43 I can send large value in a header right? 14:57:49 abhishekk: you can but the proxy servers can stripe such things and is sort of recommended deployment strategy 14:57:53 but we can restrict user to send x-openstack-request-id in headers by using request-id middleware 14:58:21 and no other services are accepting this from header 14:58:26 abhishekk: what's the max length of the req-id that can be sent? 14:58:32 sabari: around for quick peak in https://review.openstack.org/#/c/184351/ ? 14:58:51 nikhil_k: half-awake :) 14:59:12 sabari: any pre-prepared notes? 14:59:14 :) 14:59:17 can I ask you to review these patches and the spec for it? https://review.openstack.org/#/c/197899/ 14:59:24 nikhil_k: just want to make sure the image is completely deleted is not stuck in upload 14:59:25 jokke_: header length is configurable 14:59:27 these -> that one and the one it depends on 14:59:36 wko: I approved your blocked review :) 14:59:37 I'd love to get feedback on the spec 14:59:41 #action all: review https://review.openstack.org/#/c/195820/ 14:59:54 thanks nikhil_k 15:00:04 jokke_: https://github.com/openstack/glance/blob/master/glance/common/wsgi.py#L88 15:00:04 abhishekk: so we have no limitation on the req-id leght apart from max header size? 15:00:17 sabari: gotcha, it should ideally be put in killed state when it's data is deleted and then deleted 15:00:19 jokke_: yes 15:00:40 abhishekk: ah cool 15:00:58 we are sort out of time, thanks for adjusting in this busy meeting 15:01:11 o/ 15:01:12 have a good one all! I can be available for any review requests on #openstack-glance 15:01:15 thank you all, please review the patches 15:01:27 Thanks all! 15:01:29 #endmeeting