14:00:22 <rosmaita> #startmeeting glance
14:00:22 <openstack> Meeting started Thu Sep 28 14:00:22 2017 UTC and is due to finish in 60 minutes.  The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:26 <openstack> The meeting name has been set to 'glance'
14:00:32 <rosmaita> #topic roll call
14:00:34 <jokke_> o/
14:01:54 <rosmaita> looks like not a big turnout
14:02:05 <gb21> Hi @rosmaita
14:02:23 <arcolife> rosmaita, Hello
14:02:39 <jokke_> moar peoples! :D
14:02:42 <smcginnis> o/
14:03:10 <rosmaita> ok, hello everyone
14:03:17 <rosmaita> #topic updates
14:03:19 <nikhil> o/
14:03:25 <rosmaita> nikhil!
14:03:47 <rosmaita> first update: spec proposal freeze is now in effect
14:03:51 <nikhil> Ha, yes that's me :)
14:04:02 <gb21> Hi @nikhil
14:04:15 <rosmaita> i'll put up a patch later opening a 'rocky' directory for new proposals
14:04:32 <nikhil> Ohai gb21
14:04:37 <rosmaita> also, you may have noticed that i'm changing the way we do lite specs
14:04:54 <rosmaita> i had to put up like 5 of them yesterday, and don't want to deal with merge conflicts
14:04:57 <jokke_> rosmaita: ref that, if the "copy-from" needs its own, I can get that written over the weekend
14:05:16 <jokke_> if not we can just point to the IIR spec
14:05:20 <rosmaita> so i'll be changing the spec-lite template
14:05:30 <rosmaita> jokke_ : i was thinking just use the original IIR spec
14:06:09 <rosmaita> anyway, the key point is that we all need to review specs so that people can meet the deadline for the spec freeze
14:06:19 <belmoreira> Hi everyone
14:06:29 <jokke_> rosmaita: I noticed ... that will still get messy on the spec page so we might want to put them to their own dir and have the page generated from those for specs.openstack.org
14:06:31 <rosmaita> which is next Friday, 5 oct
14:06:46 <rosmaita> jokke_ : let's wait until the spec page actually gets messy
14:06:58 <rosmaita> i figure "messy" == more than 10 titles
14:07:10 <rosmaita> or say, 12, an even dozen
14:07:16 <rosmaita> i think we will be under that
14:07:20 <jokke_> rosmaita: indeed ^^, thus I +2'd most of yours earlier ;)
14:07:48 <rosmaita> but i do see your point, it's an evolving process!
14:08:31 <rosmaita> ok, that's it for updates ... if people want to discuss specific aspects of some of the specs, we can do that in general discussion
14:08:44 <rosmaita> which we should have plenty of time for, light agenda today
14:09:15 <rosmaita> nikhil : someone has posted a quotas spec, you might want to take a look
14:09:33 <nikhil> Sure
14:09:36 <rosmaita> #topic image import testing
14:09:56 <belmoreira> rosmaita what's the priority for quotas?
14:10:21 <rosmaita> belmoreira: don't know, it really depends on whether the proposers are serious about working on them or not
14:10:43 <rosmaita> we are going to be low on reviewer bandwidth, so they'd need to happen early in the cycle
14:10:54 <rosmaita> but also, i haven't looked at the proposal yet
14:11:14 <rosmaita> it needs to be consistent with the plans to eventually store limits in keystone
14:11:17 <jokke_> so I did poke our QE folks for help. By the looks of it they are quite busy trying to get their queues cleaned before Israel shuts down, so I assume no help from there short term. Abhishek promised to start working on those next week os will sync up with him
14:11:24 <rosmaita> because we don't want to do this twice
14:11:55 <rosmaita> yeah, back to the image import testing
14:12:00 <arcolife> I've seen nikhil's video on youtube regarding an interoperable sprint, about 1 year ago.. ?  read the docs and etherpad, have an idea.  is
14:12:02 <arcolife> do we have a time for a quick video chat to summarize things for me to get started ?
14:12:22 <arcolife> https://etherpad.openstack.org/p/glance-image-import-tests
14:12:28 <belmoreira> rosmaita my understanding is that the work in keystone limits is stale.
14:12:32 <arcolife> these are what we'd like in there? and there's 0 work regarding that ?
14:12:38 <arcolife> ^^
14:12:53 <rosmaita> arcolife: yes, that;'s the initial list of stuff
14:13:11 <rosmaita> belmoreira: let's hold off on quotas until open discussion
14:13:29 <rosmaita> arcolife: you are interested in working on the import tests, i take it?
14:13:42 <arcolife> rosmaita, yes, that's what seems available atm ?
14:13:56 <arcolife> is there anything else higher priority and starting point ?
14:14:08 <jokke_> arcolife: what timezone you're on? Abhishek is utc +5.5 IIRC so it would likely be super helpful to catch up with him on the testing work
14:14:09 <gb21> @rosmaita, I am willing to work along arcolife on the tests
14:14:20 <arcolife> jokke_, utc+5:30
14:14:23 <rosmaita> gb21 tha's great
14:14:25 <arcolife> yep same
14:14:40 <jokke_> I'm sure he will be more than happy to have helping hands there and I'll do what ever I can to help out
14:14:54 <jokke_> arcolife: brilliant. Do you know Abhishek?
14:15:04 <arcolife> jokke_, we started an email thread :)
14:15:25 <rosmaita> arcolife: sorry, did not recognize your nick
14:15:27 <jokke_> :)
14:15:32 <arcolife> rosmaita, ah. :D
14:15:59 <rosmaita> arcolife why don't you ping abhishek and set up a time to talk with him?
14:16:10 <rosmaita> that way you can all talk at a reasonable hour
14:16:26 <rosmaita> also, we should all wish Abhishek's daughter a happy birthday
14:16:30 <rosmaita> (that's why he's not here)
14:16:32 <arcolife> rosmaita, thought we'd be doing a thurs knowledge sharing session. realized it's an irc scrum later.
14:16:48 <arcolife> goodies. will setup a mtg now
14:17:29 <jokke_> arcolife: if you schedule something around this with Abhishek and you manage to get it to your afternoon, feel free to count me in, I'll try to make time for it
14:18:07 <arcolife> jokke_, 3.5 hrs behind 5.5  are we?
14:18:24 <jokke_> arcolife: I'm in Ireland so 5.5 behind you guys
14:18:30 <arcolife> gotcha
14:18:57 <jokke_> and no, I won't be waking up silly-o-clock for that ... but voluteered as I'm sure rosmaita won't either to join us :P
14:19:17 <jokke_> and I don't want to hold you guys working until super late just to get us all there
14:19:17 <rosmaita> well, depends on what time you all agree on :)
14:19:23 <arcolife> rosmaita, iteroperable image import, there was a discussion an year back on YT in a sprint. we were concerned about not letting too much leeway for people to add non-standardized import formats ?
14:20:05 <rosmaita> arcolife right, you can look over the original spec to see all the issues and how they're being addressed for interoperability
14:20:25 <rosmaita> #link https://specs.openstack.org/openstack/glance-specs/specs/mitaka/approved/image-import/image-import-refactor.html
14:20:41 <jokke_> arcolife: prepare full pot of coffee ... that will take a while to digest ... good bedtime reading 'though
14:20:42 <rosmaita> also, you can look in the current api-ref to see what's been implemented so far
14:20:46 <nikhil> Look at the review of the spec too
14:21:06 <nikhil> That will help much more
14:21:51 <arcolife> roger that 0_0
14:21:53 <rosmaita> #link https://developer.openstack.org/api-ref/image/v2/#interoperable-image-import
14:23:07 <rosmaita> here's the review nikhil mentioned:
14:23:11 <rosmaita> #link https://review.openstack.org/#/c/232371/
14:23:28 <rosmaita> and also
14:23:33 <rosmaita> #link https://review.openstack.org/#/c/278086/
14:23:44 <rosmaita> and also
14:23:48 <rosmaita> #link https://review.openstack.org/#/c/318313/
14:24:24 <rosmaita> that should keep you busy for a while!
14:24:35 <nikhil> ;)
14:24:48 <rosmaita> also, there's a section in the glance admin docs about how to turn on image import
14:25:21 <rosmaita> #link https://docs.openstack.org/glance/latest/admin/interoperable-image-import.html
14:25:28 <rosmaita> and then of course there's the code
14:26:18 <rosmaita> anyway, please organize a chat with abhishekk and you can get started ... glad to have you helping us out with testing!
14:26:49 <arcolife_> rosmaita, referring to this link (lost you after #link) http://eavesdrop.openstack.org/meetings/glance/2017/glance.2017-09-28-14.00.log.txt )
14:27:16 <arcolife_> rosmaita, glad to be a part. cheers. will read up and get back
14:27:29 <rosmaita> ok, cool ... ty~
14:27:52 <rosmaita> ok, time for open discussion
14:27:59 <rosmaita> #topic open discussion
14:29:24 <rosmaita> ok, i've got one
14:29:31 <rosmaita> i need some feedback on https://review.openstack.org/#/c/468179/
14:29:33 * arcolife_ too
14:29:38 <rosmaita> jokke_ doesn't like it
14:29:49 <rosmaita> but i think he is being unreasonable
14:29:54 <rosmaita> but that's just me
14:30:04 <rosmaita> so it would be good to get some wider opinions
14:30:16 * smcginnis opens tab
14:30:26 <rosmaita> and maybe i should take tim bell's advice and do an operator survey
14:30:48 <smcginnis> rosmaita: Maybe even just an email to openstack-operators to get some feedback?
14:31:15 <rosmaita> smcginnis yeah, maybe that would be quicker
14:31:49 <belmoreira> openstack-operators will be good. I need to revive my memory about it
14:32:14 <belmoreira> just uploaded https://review.openstack.org/#/c/508133/ few hours ago. Can it be considered for this cycle?
14:32:17 <rosmaita> ok, i'll do that and we can get some operator feedback
14:32:58 <rosmaita> belmoreira yes, you got it in before the proposal freeze
14:33:03 <rosmaita> i haven't looked it over yet
14:33:15 <rosmaita> so not sure what resources it requires
14:33:31 <rosmaita> but i'll read and comment later today
14:33:44 <rosmaita> and we can continue the discussion on the patch
14:34:02 <belmoreira> I think it needs some iterations but it will be great to have feedback and new ideas on it
14:34:24 <rosmaita> sounds good
14:34:40 <rosmaita> belmoreira honestly, i doubt we will have dev resources to do it in queens
14:34:47 <rosmaita> unless you have a dev team available?
14:34:58 <jokke_> belmoreira: that touches very very closely to the Image Lifecycle discussion we had in PTG
14:35:36 <rosmaita> but if we can get it refined before the sydney summit, and get feedback at the summit/forum, maybe it can be ready to do early in rocky
14:35:38 <belmoreira> this was based in the discussion in the mailing list and the other spec that is as reference
14:35:54 <rosmaita> ok, well i should shut up and read it
14:36:14 <rosmaita> no sense speculating about how much effort it will take without knowing what it's about :)
14:37:11 <belmoreira> I think it will be more to trigger discussing than having an implementation immediately. let's see
14:37:17 <belmoreira> and about quotas?
14:37:39 <rosmaita> sure, what are your thoughts about quotas?
14:38:15 <jokke_> belmoreira: so I'd be pretty confident saying that your proposal will not be done, but that said you might end up with something like "stale":true that is not listed by default
14:38:42 <jokke_> we will most likely not be filtering by default based on any given property needing to be set
14:39:18 <jokke_> but having property, set of properties that is basis for something fitered out of the list is way more likely to go through
14:39:22 <belmoreira> jokke_ it will be great to have feedback and find a good solution for this problem
14:39:58 <belmoreira> about quotas... I feel that a solution in glance is being postponed for a long time
14:40:08 <rosmaita> you are right about that!
14:40:52 <jokke_> belmoreira: there is light capture of what we were brainstorming in https://etherpad.openstack.org/p/glance-queens-ptg-lifecycle
14:41:09 <belmoreira> I agree about the dependencies (limits, hierarchical projects...) but it will be great if we find a compromise
14:42:27 <jokke_> ok so what comes to quotas, we had some people working on that cycles ago
14:42:39 <nikhil> Maybe the dependency on hierarchical is gone? rosmaita ?
14:43:10 <rosmaita> so this is the spec that was proposed: https://review.openstack.org/#/c/487777/
14:43:20 <rosmaita> i thought it had been updated, but apparently not
14:43:21 <belmoreira> as private cloud that don't charge the users I rely on quotas. Deployments like mine can't control the image utilization. Any user can basically use all our storage for images.
14:43:43 <jokke_> the work got halted as there was supposed to be first cross project spec how this is done across projects, then it moved to Keystone coordinating it and providing the tools (read storing the actual quotas) and like some of you mentioned earlier it has been pretty quiet lately
14:44:33 <nikhil> Ack ,ty
14:45:24 <rosmaita> belmoreira there is a user_storage_quota, but i guess that is insufficient for your use case?
14:46:03 <jokke_> so I'm frankly not interested to talk about Glance specific solution again unless 1) we get OS community agreement that this is not going to be done "ever" as coordinated effort or 2) that coordinated effort moves to the point where we can use it and be aligned
14:46:48 <belmoreira> in my case needs to be very large because some projects actually need it, but then all projects have that large quota available
14:46:57 <rosmaita> belmoreira i understand
14:47:16 <rosmaita> belmoreira are you at CERN?
14:47:41 <nikhil> belmoreira: what storage backends do you use? Fs, swift, cinder?
14:48:15 <belmoreira> it would be great have a basic quota system (allowed used_space per project). That would solve my problem. And then we can iterate for the common solution with the other projects
14:48:32 <belmoreira> rosmaita yes, I'm at CERN
14:48:51 <belmoreira> nikhil we use ceph
14:49:12 <nikhil> Ack
14:49:40 <rosmaita> belmoreira is storage your only concern?
14:49:48 <belmoreira> at the moment our ceph poll only for images has 200TB
14:49:48 <jokke_> belmoreira: that actually makes the thing bit more difficult :P
14:50:26 <jokke_> belmoreira: IIRC your snapshots actually shows up as 0 size images in glance :(
14:51:12 <belmoreira> rosmaita that is my bigger concern. If I can limit the amount of space that a project can use the number of images/snapshots is not a very big issue
14:51:40 <belmoreira> jokke_ wow. I didn't noticed
14:51:58 <rosmaita> yeah, but if jokke_ is correct, we won't be able to enforce it accurately
14:53:04 <jokke_> belmoreira: not 100% sure about that, but I remember discussion not so long ago. Apparently when Nova uses the ceph backend together with Glance it does not populate the size as the snapshot image is never streamed through glance API
14:53:49 <belmoreira> but is that should be the case for nova ephemeral storage with ceph
14:54:05 <belmoreira> we use the local storage in the compute nodes
14:54:22 <jokke_> so implementing quota that can't handle again that special case with Ceph is one of those things that is just security bug waiting to be filed
14:54:40 <jokke_> belmoreira: oh, so you're not affected by that very specific thing then
14:55:57 <belmoreira> jokke_ I believe not. I never noticed that snapshots are marked with 0 size. But what you mentioned adds complexity in finding a proper solution
14:56:10 <rosmaita> well, i've been thinking as quotas as a broad topic, but if we limit it to storage only, might be a good stopgap (assuming htere's a good way around the 0-size snapshot problem)
14:56:10 <jokke_> belmoreira: indeed
14:56:35 <rosmaita> i imagine the 0 size is because nova is using user creds, and not allowed to set size on the image record?
14:57:06 <jokke_> rosmaita: I think that would need poking the locations code (and I'm not sure how ceph reports the size of snapshots)
14:57:20 <jokke_> rosmaita: I think that might be the case
14:57:36 <jokke_> rosmaita: yet another thing that could be solved with the service tokens
14:57:53 <rosmaita> i was just about to say that
14:58:04 <rosmaita> we need to get a good sense of how stable the service token stuff is in keystone
14:58:11 <jokke_> ++
14:58:27 <rosmaita> because there are several improvements we could make if we use them
14:58:41 <jokke_> but my main point on that was, that as long as we have these kind of "show stoppers" there is really no good way to make that happen
14:58:51 <rosmaita> we're running low on time
14:59:09 <rosmaita> i think the first step would be figuring out what's necessary to get accurate image size reporting in glance
14:59:26 <rosmaita> and after that, we can tackle a simple quota for space consumed per tenant
14:59:40 <jokke_> Also if that quotas need is big enough, it might be good opportunity to get first quota type into keystone and file down how that is going to look like in larger scale
14:59:43 <rosmaita> no sense going for a quota if we can't accurately measure what's limited
14:59:48 <arcolife_> rosmaita, the service token thing blew up for >100 tenants w/ CFME https://bugzilla.redhat.com/show_bug.cgi?id=1476371
14:59:49 <openstack> bugzilla.redhat.com bug 1476371 in -- Unknown "CFME has multiple issues adding OSP 8 and OSP 10 as providers with >100 projects/tenants via AD CF User" [Low,New] - Assigned to jhardy
15:00:03 <rosmaita> arrrrgh
15:00:14 <rosmaita> arcolife_ thanks for that link, will take a look
15:00:18 <arcolife_> we fixed that partially with new patches of cfme.
15:00:22 <jokke_> thanks all!
15:00:25 <rosmaita> ok, we are out of time, thanks everyone
15:00:36 <nikhil> Thanks
15:00:36 <rosmaita> continue discussion in glance channel if you want
15:00:39 <arcolife_> but the fix relies on generating 100k tokens or so, per 24 hrs. it was going much more than that/
15:00:42 <arcolife_> cool. thanks all
15:00:46 <rosmaita> #endmeeting