16:00:57 <jgriffith> #startmeeting cinder
16:00:58 <openstack> Meeting started Wed Dec 18 16:00:57 2013 UTC and is due to finish in 60 minutes.  The chair is jgriffith. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:59 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:02 <openstack> The meeting name has been set to 'cinder'
16:01:04 <thingee> o/
16:01:06 <eharney> hi
16:01:09 <bswartz> hello
16:01:10 <zhaoqin> hello
16:01:24 <dosaboy> aloha
16:01:25 <jgriffith> DuncanT: avishay eharney thingee bswartz zhaoqin gooooooooodddddd moooooornnnnning
16:01:34 <jgriffith> dosaboy: you in Hawaii.. or just dreaming?
16:01:35 <xyang1> hi
16:01:36 <avishay> hello!
16:01:37 <jgriffith> :)
16:01:45 <zhaoqin> happy new year
16:01:47 <DuncanT> Hey
16:01:52 <aguzikova> howdy
16:01:54 <caitlin56> morning
16:01:58 <jgriffith> aguzikova: hey there
16:01:59 <avishay> evening
16:02:00 <jgriffith> hi caitlin56
16:02:02 <dosaboy> jgriffith: hallucinating
16:02:07 <jgriffith> alright.. let's get this rollin
16:02:12 <jgriffith> dosaboy: I want what you had
16:02:15 <rushiagr> dosaboy: lol
16:02:18 <avishay> haha
16:02:28 <jgriffith> avishay: you wanna kick it off with your bp discussion?
16:02:39 <jgriffith> #topic cinder backup recovery api
16:02:46 <avishay> https://wiki.openstack.org/wiki/CinderMeetings
16:02:54 <jgriffith> #link https://blueprints.launchpad.net/cinder/+spec/cinder-backup-recover-api
16:02:55 <jungleboyj> Hello all.
16:03:02 <avishay> So this isn't mine, but I'm representing because Ronen couldn't make it
16:03:29 <jgriffith> avishay: fair enough
16:03:36 <avishay> The idea is to create an import_backup function so that if the DB goes away for some reason, you can import the backup's metadata into the DB
16:04:16 <DuncanT> Question 1.... Why is the host needed? Why not just let the scheduler pick one?
16:04:17 <jungleboyj> avishay:  Wasn't this discussed a month or so ago?
16:04:24 <avishay> export_backup will print out a string that can be used later on for import_backup when things go bad
16:04:39 <avishay> jungleboyj: possibly
16:04:57 <jungleboyj> avishay:  There was discussion of backing up metadata.
16:04:58 <DuncanT> We discussed the idea I think, not the implementation...
16:05:03 <dosaboy> jungleboyj: briefly but I have not had a chance to do anything about it
16:05:11 <jungleboyj> DuncanT: Ok.  That makes sense.
16:05:20 <avishay> DuncanT: what if only one host has access to the backup?  honestly i'm a bit rusty on the details of the backup service
16:05:21 <caitlin56> avishay: why isn't the approach to just make the backups more self-sufficient, less dependent on db entries?
16:05:21 <dosaboy> so ronen has decided to take it on
16:05:41 <winston-d> o/ hi~
16:05:44 <avishay> caitlin56: that is the approach, but once you have some self-sufficient backup, you need to import it
16:05:46 <jgriffith> avishay: that's sort of a problem, but I think Ronen accounted for that
16:05:53 <jungleboyj> dosaboy: Cool.
16:06:06 <jgriffith> avishay: IIUC the idea is you have an object store / dump and you can just restore *everythin* from it
16:06:07 <DuncanT> avishay: I'm not aware of how that could happen... but I might be missing something
16:06:16 <dosaboy> backups are ot tied to hosts but the volume that is backed up may be
16:06:26 <jgriffith> dosaboy: correct
16:06:36 <avishay> DuncanT: could i back up to two different swift installations?
16:06:36 <jgriffith> dosaboy: but the db has the record of the backup
16:06:41 <jgriffith> dosaboy: ie the mapping to find it etc
16:06:45 <caitlin56> Will you be able to restore "everything" or *just* the volume?
16:06:48 <jgriffith> so here's my opinion
16:06:52 <DuncanT> avishay: Taht would be deviant, but ok
16:06:56 <avishay> caitlin56: everything
16:07:02 <jgriffith> I think having mechanisms to do things like backup the cinder nodes is a good idea
16:07:05 <jgriffith> BUT
16:07:06 <avishay> DuncanT: yep
16:07:16 <jgriffith> I don't think throwin git into the existing backup model necessarily works
16:07:19 <caitlin56> avishay is their an option to restore just the volume content?
16:07:33 <DuncanT> avishay: Optional parameter maybe? Currently backup services are pretty much active/active HA
16:07:51 <avishay> DuncanT: i guess we can do that
16:08:07 <jgriffith> DuncanT: avishay ?
16:08:09 <avishay> caitlin56: without anything in the DB?
16:08:13 <jgriffith> sorry.. not sure what you're getting at
16:08:18 <DuncanT> avishay: If you don't specify it you should also get load ballancing for free...
16:08:26 <avishay> jgriffith: sorry i'm trying to answer 3 people at once :)
16:08:31 <jgriffith> avishay: :)
16:08:36 <caitlin56> avishay: allowing changes to DB, say if I'm restoring on a site with different users.
16:08:54 <jgriffith> Ok... let's stop here for a second
16:09:03 <jgriffith> we're getting wayyyy down in the dirt here
16:09:13 <jgriffith> and there are still a number of things that I think need answered in general
16:09:24 <avishay> jgriffith: shoot
16:09:41 <jgriffith> So let's make sure we're on the same page
16:09:54 <jgriffith> The concept of the BP is to be able to perform a backup of a cinder node
16:10:07 <jgriffith> control node more specifically
16:10:13 <jgriffith> but also volume-service node
16:10:26 <jgriffith> correct?
16:10:39 <DuncanT> That isn't what I understood by the proposal, no
16:10:41 <avishay> don't follow
16:10:49 <jgriffith> The only thing this BP really talks about is backing up the db
16:10:58 <jgriffith> quite honestly IMO *who cares*
16:11:02 <DuncanT> jgriffith: I don't read the blueprint as that at all
16:11:06 <jgriffith> it hardly seems worth making an API for
16:11:11 <jgriffith> DuncanT:
16:11:16 <jgriffith> DuncanT: ok, please explain
16:11:19 <dosaboy> the bp is about being able to take a backup from one cloud and restore to another
16:11:33 <avishay> jgriffith: if i backed up a volume to swift, and then my DB is lost, i want to import the backup back from swift so that i could restore the volume if i want
16:11:37 <DuncanT> jgriffith: I read it as 'I've got some backups, but I've lost my DB.... create the DB entries for these backups please'
16:11:44 <dosaboy> i.e. cloud A dies so we build cloud B and want to restore our volumes from A
16:11:58 <avishay> yea, what they said :)
16:12:03 <DuncanT> Where 'backup' very specifically means 'backups created by the cinder backup service'
16:12:10 <dosaboy> correct
16:12:11 <jgriffith> dosaboy: DuncanT avishay ok, thanks :)
16:12:20 <thingee> I read it as backup/restore metadata
16:12:21 <jungleboyj> jgriffith: +2
16:12:26 <jgriffith> thingee: ha!
16:12:34 <thingee> seriously, that's the last sentence
16:12:42 <jgriffith> thingee: yeah
16:12:48 <jungleboyj> thingee: I did too, but I think I am following better now.
16:13:00 <jgriffith> I know... I'm laughing because anybody that just says "metadata" is asking for trouble
16:13:01 <caitlin56> So you backup the metadata and data separately?
16:13:19 <DuncanT> caitlin56: No no no
16:13:31 <thingee> caitlin56: right, I don't understand why it has to be separate. I read this as it should be included with the backups.
16:13:40 <DuncanT> Metadata is overloaded here
16:13:50 <winston-d> thingee: +1
16:13:57 <DuncanT> Volume metadata *will be* (once we merge the code) included in the volume
16:14:04 <dosaboy> thingee: could probably do with rewording/refining a bit, it's been sat there unloved for a while
16:14:05 <DuncanT> Backups have their own metadata
16:14:24 <DuncanT> i.e. what goes in the backup table... the URL for swift, for example
16:14:33 <jgriffith> Ok... well I'm thumbs down
16:14:43 <caitlin56> So you backup the volume, and the object you create includes the db entries (i.e. the "metadata"), correct?
16:14:44 <DuncanT> This is an import/export API for the backup metadata, *not* the volume metadata
16:14:48 <jgriffith> The goal is vague IMO
16:14:55 <thingee> DuncanT: I'm talking about metadata that goes with a volume.
16:15:08 <jgriffith> and I don't know what problem you're trying to solve other than the recommended "restore volumes" to a lost db
16:15:08 <thingee> if I restore a backup, I get a volume with the metadata it previous had
16:15:20 <DuncanT> thingee: We've a separate plan to include that directly in the backup... see dosaboy's BP & patches
16:15:23 <jgriffith> which I'd rather see something like a manage API to import/reset info
16:15:36 <thingee> DuncanT: heh, that's why I was confused by this bp.
16:15:38 <jgriffith> thingee: but that's "volume-metadata" not "db-metadata"
16:15:45 <thingee> got it
16:16:00 <avishay> can i try again?
16:16:08 <dosaboy> so the pre-requisite for this bp is the volume metadata backup stuff i'm working on now
16:16:10 <jgriffith> avishay: sure :)
16:16:24 <dosaboy> avishay: shoot
16:16:26 <DuncanT> This (the way I read it) is just a way to add backups done externally (be it a different cloud or an old incarnation of a cloud) to the list of backups you can restore from
16:16:46 <avishay> so imagine you're backing up volume to geo-replicated Swift, and all your backups are nice and safe.  then your site goes down.
16:17:01 <avishay> you have all those nice backups safe in swift, but your backups DB is empty
16:17:35 <avishay> so this command will go to the swift backup driver, and have it query those backups, and fill in the DB so that you can restore your precious volumes
16:17:48 <jgriffith> avishay: so it's an import feature kinda
16:17:48 <dosaboy> agreed
16:17:54 <DuncanT> avishay: +1, that's how I read it too
16:18:03 <jungleboyj> avishay:  That makes sense.
16:18:03 <avishay> jgriffith: yes, import_backup
16:18:20 <jgriffith> avishay: but why not use a general import volume
16:18:24 <jgriffith> regardless of backup or not
16:18:31 <jgriffith> that way it can be used for "other" things
16:18:36 <DuncanT> Because the backup format is defined
16:18:49 <jgriffith> DuncanT: so?
16:18:50 <DuncanT> The disk format, the metadata, etc
16:18:56 <dosaboy> i.e. import a backup (db) into cinder so that it can then be used to restore volumes
16:19:00 <caitlin56> So "populate my list of backup images from what is stored on this external backup target"?
16:19:11 <DuncanT> caitlin56: Pretty much, yes
16:19:15 <avishay> jgriffith: here the backup driver needs to go to the actual backup and look stuff up.  in import_volume the volume driver needs to query the storage.
16:19:32 <jgriffith> right, but what I'm saying is why not take it another step
16:19:41 <jgriffith> I have one of two cases:
16:19:47 <caitlin56> What is the name of the object that has this info for the external backup target? Is it a constant or do you have to remember it somewhere else?
16:19:50 <jgriffith> 1. the case you're describing, backups and lost db
16:19:57 <jgriffith> 2. volumes on the backend but no db
16:20:10 <jgriffith> It would be cool to import either one
16:20:13 <DuncanT> caitlin56: Currently you have to remember it
16:20:20 <thingee> jgriffith: +1
16:20:29 <jungleboyj> jgriffith: +1.
16:20:30 <jgriffith> IMHO the whole db in the volume backup is kinda silly anyway
16:20:39 <avishay> jgriffith: i don't know offhand how to combine import volume and import backup
16:20:49 <jungleboyj> I have my volumes but no Cinder.  Have Cinder rebuild itself.
16:20:51 <DuncanT> jgriffith: To my mind those are totally different cases
16:20:55 <jgriffith> The mode of loosing your backend volume AND your controller nodes DB is a bit of a corner case IMO
16:21:13 <jgriffith> DuncanT: well yes
16:21:14 <thingee> avishay: I think where jgriffith is getting it at, is something to consider on this command. Can we make abstract enough to support both ideas in the future.
16:21:29 <jgriffith> DuncanT: but why not use the same mechanism to achieve both of them (or at least the same API call)
16:21:36 <DuncanT> jgriffith: The import volume, for the swift case, can conceivably also be used to import volumes backup up on another provider...
16:21:36 <jgriffith> Why explode API calls?
16:21:56 <jgriffith> thingee: +1
16:22:09 <jgriffith> DuncanT: I don't think I'm getting my point across correctly here
16:22:15 <jgriffith> thingee: seems to get it maybe
16:22:22 <DuncanT> jgriffith: I'd avoid using the same API because the semantics are very different... a backup (should) know if it is bootable, glace details etc.... an 'external' volume is just a pile of bits
16:22:30 <jgriffith> DuncanT: regardless by itself I don't see this as being overly useful anyway
16:22:34 <avishay> i guess we could have a flag to say if it's a volume or backup that you want to import, but i think the code path inside will be totally different
16:22:47 <jgriffith> DuncanT: no, it's exactly the same
16:22:47 <rushiagr> thingee: +1
16:22:55 <jgriffith> DuncanT: So... say it's something like this:
16:22:57 <jungleboyj> jgriffith: I get it.
16:23:18 <jgriffith> DuncanT: "cinder import-volume --source <backup|path>
16:23:26 <thingee> jgriffith, DuncanT: can we discuss offline about the differences. maybe we can think of ways of having an "import" feature that can import backend volumes or backups..but the implementations are hidden from the import command.
16:23:48 <jgriffith> DuncanT: if it's a backup the API does the right thing and repops the volume based on the backup
16:23:59 <jgriffith> thingee: fair enough
16:24:04 <thingee> import is going to import "an object"...we don't care what. and that object might have metadata...and it might not
16:24:09 <avishay> ok, we can take it offline
16:24:12 <jgriffith> thingee: +111111
16:24:18 <DuncanT> jgriffith: The semantics are, I think, so different that there's no gain to multiplexing the call
16:24:26 <avishay> i agree with DuncanT
16:24:33 <jgriffith> DuncanT: avishay you're both wrong :)
16:24:36 <jgriffith> DuncanT: avishay just kidding
16:24:39 <avishay> haha
16:24:46 <jgriffith> ok, we can talk later
16:24:49 <DuncanT> jgriffith: e.g. import backup does *not* creat a volume.... it creates a backup entry
16:24:50 <thingee> DuncanT, avishay: I just want to humor the idea...and see the differences.
16:25:05 <caitlin56> When you point at your Swift object storage, are you restoring "db info about backup x" or "db info about a lot of backups stored here"?
16:25:07 <avishay> thingee: no problem, we can try
16:25:11 <jgriffith> Ok
16:25:11 <rushiagr> I am on thingee's side. Makes more sense
16:25:14 <jgriffith> Let's change topics
16:25:22 <avishay> caitlin56: backup x
16:25:24 <DuncanT> I might have 2000 backups... when I import them, I don't wany 2000 new volumes....
16:25:25 <jgriffith> I don't see us making progress on that one
16:25:44 <rushiagr> I am ready to move forward :)
16:25:45 <thingee> DuncanT: I agree. That wasn't my intentions.
16:25:49 <jgriffith> DuncanT: what?
16:25:51 <avishay> let's leave it for now, we'll think how we can make them into one API call, and post some code
16:26:08 <jgriffith> #topic taskflow
16:26:09 <caitlin56> avishay: so you are dependent on Swift to provide your catalog of all available backups, presumably by having a policy on how you named them in the first place.
16:26:16 <jgriffith> https://blueprints.launchpad.net/cinder/+spec/copy-volume-to-image-task-flow
16:26:38 <jgriffith> Wondering if anybody else has looked at this?
16:26:49 <avishay> caitlin56: we'll have to figure that out, yes :)
16:26:49 <jgriffith> And if anybody knows Abhishek?
16:27:18 <caitlin56> avishay: using the directory capabilities of the object store strikes me as a good idea.
16:27:33 <rushiagr> jgriffith: you know which company he is from? if it is NTT I can track him down
16:27:50 <jgriffith> rushiagr: ntt
16:27:52 <avishay> rushiagr: abhishek.kekane@nttdata.com
16:27:55 <thingee> jgriffith: I haven't...but I just wanted to say I think the taskflow reviews should get some good priority. Good things in core are dependent on these getting through.
16:27:57 <jgriffith> I can send him an email
16:28:04 <jgriffith> thingee: agree
16:28:15 <jgriffith> but I wanted to make sure where we were in terms of taskflow in general
16:28:23 <avishay> caitlin56: we'll have to see how to get all the backups, and only the backups, but yes - we'll figure something out
16:28:28 <thingee> I think everyone else has been way better than me on getting those through, and I'll try better to get those reviewed.
16:28:29 <rushiagr> cool
16:28:41 <Abhishek> jgriffith: I'm Abhishek, but that's not my email
16:28:47 <jgriffith> Everybody went off with ideas about "states, consistency etc" but I haven't really seen/heard anything about that
16:28:51 <jgriffith> Abhishek: ha!
16:28:59 <jgriffith> Abhishek: well, welcome at any rate
16:29:10 <jgriffith> Abhishek: sounds like you need to update launchpad :)
16:29:19 <Abhishek> yeah, I don't think you guys were looking for me :)
16:29:19 <jgriffith> Abhishek: are you still planning on working on this?
16:29:21 <thingee> I asked DuncanT about it yesterday, but I don't think it's going to happen in I-2...which means not likely in I.
16:29:32 <jgriffith> Ohh... it's not your blueprint?
16:29:38 <avishay> this is not the Abhishek you're looking for?
16:29:40 <DuncanT> I'm hopeful about I-2 thingee
16:29:42 <jgriffith> avishay: hehe
16:30:01 <jgriffith> DuncanT: TBH at this point I'm doubtful
16:30:12 <Abhishek> yep, I don't have anything assigned to me in taskflow
16:30:13 <jgriffith> The next two weeks are basicly dead weeks
16:30:25 <jgriffith> Abhishek: alright then
16:30:34 <jgriffith> I'll try and track the submitter down offline
16:30:36 <avishay> jgriffith: for me these are the 2 most productive weeks of the year - no meetings! :)
16:30:37 <thingee> DuncanT: if harlowja_away and I can help in anyway, please let us know. We've been tossing around ideas...but we're available if you want to discuss ideas.
16:30:48 <jgriffith> avishay: actually that's a really good point :)
16:31:11 <jgriffith> Ok, that was a noop topic
16:31:13 <avishay> jgriffith: speaking of which , are we canceling the next one/two?
16:31:13 <winston-1> jgriffith: not for me. :) we don't do xmas much here
16:31:19 <jungleboyj> jgriffith: I think avishay is volunteering to keep Cinder going for the next two weeks.  ;-)
16:31:24 <jgriffith> winston-1: indeed
16:31:30 <jgriffith> winston-1: I actually thought of that while typing that
16:31:39 <rushiagr> winston-1: avishay same goes with me too, here in India :)
16:31:40 <DuncanT> thingee: I'm trying to go through the code and put in all the transition points, with no enforcement... I think the enforcement code can better be written by anybody who isn't me
16:31:50 <avishay> winston-1: rushiagr :)
16:31:53 <DuncanT> thingee: I should have that done before I finish for xmas
16:31:56 <jgriffith> ok ok...
16:32:00 * jgriffith is an idiot
16:32:01 <DuncanT> thingee: Does that make sense?
16:32:23 <jgriffith> ollie1: you about?
16:32:26 <ollie1> yep
16:32:35 <avishay> jgriffith: nah, don't be so hard on yourself :)
16:32:38 <jgriffith> #topic admin-capabilities
16:32:41 <jgriffith> avishay: :)
16:32:43 <jgriffith> https://blueprints.launchpad.net/cinder/+spec/admin-defined-capabilities
16:33:17 <jgriffith> ollie1: so I'm afraid I need some help understanding the benefit here
16:33:22 <ollie1> so the controversy here is whether to put these into the db or config?
16:33:29 <jgriffith> ollie1: I'm again the only one that doesn't seem to get it... sorry :(
16:33:41 <ollie1> ok so
16:34:27 <bswartz> jgriffith: you're not alone
16:34:29 <ollie1> is there a way for an admin running a  cinder installation to identify different backends acording to some arbitarty quality
16:34:50 <guitarzan> first step should be to nuke the word capability and call it backend management
16:34:57 <jgriffith> ollie1: not sure what you mean?
16:35:03 <jgriffith> ollie1: form what context?
16:35:03 <thingee> DuncanT: I have read quite a few state design patterns now. Some involving state charts too..I'm curious on your approaches.
16:35:05 <ollie1> so there is a set of backends that is high quality and a set that is lower  quality
16:35:17 <thingee> DuncanT: but yes it makes sense.
16:35:19 <DuncanT> thingee: I'll talk to you after the meeting?
16:35:31 <jgriffith> ollie1: I'm with ya
16:35:31 <thingee> maybe after the import stuff too ;)
16:35:36 <bswartz> I think we already proposed a facility to assign arbitrary "capabilities" to backends, as text values in cinder.conf didn't we?
16:35:47 <ollie1> maybe some local to vm's and some remote
16:35:50 <avishay> bswartz: that's what this BP is :)
16:35:55 <bswartz> doh!
16:36:11 <bswartz> okay I'm no longer lost (thx avishay)
16:36:20 <jgriffith> I thought that's what types were :)
16:36:27 <jgriffith> I'm so confused
16:36:34 <ollie1> can the admin assign types to a backend?
16:36:35 <jgriffith> we keep piling layer upon layer
16:36:39 <jgriffith> ollie1: yes
16:36:43 <jgriffith> ollie1: absolutely
16:37:02 <bswartz> jgriffith: what if we allowed more than one key/value? types are good, but limitted to just one key and value
16:37:07 <ollie1> ok, so I don't know how to do that :)
16:37:08 <jgriffith> type uses extra-spec entry "volume-backend-name" key
16:37:17 <thingee> jgriffith: so in types you can set extra specs. When the capability filters run, how does it know a host has those additional capabilities that are set in extra_spec table?
16:37:19 <jgriffith> ollie1: LOL
16:37:27 <DuncanT> jgriffith: This simply allows an admin to put their own labels on the backend, rather than being limited to the names
16:37:40 <jgriffith> DuncanT: umm... why?
16:37:59 <bswartz> it would allow you to form arbitrary groupings of your backends
16:38:06 <jgriffith> DuncanT: if that's the goal it just seems like a lot of churn and yet another table/api for little gain
16:38:13 <jgriffith> bswartz: now that's different
16:38:23 <jgriffith> bswartz: but I don't see that at all here
16:38:24 <DuncanT> jgriffith: So if I've twenty identical backends but with different names, I can list the labeel in the type spec and just put the label in the backends, rather than continuously updating the central type spec
16:38:25 <bswartz> you could label 3 different backends "red", 2 other ones "blue", and a bunch more of them "green"
16:38:29 <winston-1> jgriffith: because you guys don't agree with each other at all
16:38:32 <jgriffith> bswartz: if that's the driving intent I'm all for it
16:38:43 <DuncanT> jgriffith: I don't think this should have a table or an API
16:38:43 <jgriffith> aggregate volume-types
16:38:44 <dosaboy> so who/what would be the intended 'consuemr' for this new labelling?
16:38:51 <dosaboy> consumer
16:38:55 <winston-1> so, with that, admin can stick any label to any back-end the way they likes
16:39:00 <DuncanT> dosaboy: volume type definitions
16:39:03 <ollie1> dosaboy the cinder admin
16:39:07 <avishay> winston-1: can i change extra_specs on a volume type that's in use?
16:39:14 <DuncanT> winston: Yes
16:39:16 <jgriffith> DuncanT: yeah... sorry that was poor to say
16:39:21 <dosaboy> ok so they would tie in
16:39:36 <jgriffith> DuncanT: ollie1 but then what?
16:39:44 <caitlin56> winston: +1 I have no problem with admins labeling. But anyone else labeling my backends would be a problem.
16:39:51 <jgriffith> So I now have this "label", so.... what's next?
16:40:11 <DuncanT> jgriffith: So the type for 'silver service' just says 'any backend with the label silver'
16:40:11 <ollie1> so users creating volumes can specify the label they want
16:40:33 <jgriffith> DuncanT: but there's nothing in the API here?
16:40:42 <avishay> ollie1: not the label they want, the volume type they want - the user doesn't know about these labels
16:40:42 <jgriffith> ollie1: ^^
16:40:45 <rushiagr> umm, isn't that same as volume type?
16:40:47 <jgriffith> so how would a user specify that?
16:40:51 <jgriffith> rushiagr: EXACTLY :)
16:40:53 <DuncanT> jgriffith: The labels get added to the capabilties list of a backend
16:40:54 <thingee> _spec table?
16:41:00 <jgriffith> rushiagr: except it takes it one step further
16:41:14 <thingee> whoops...so in types you can set extra specs. When the capability filters run, how does it know a host has those additional capabilities that are set in extra_spec table?
16:41:15 <jgriffith> rushiagr: and says something like 'n' backends can all serve a single volume type
16:41:20 <ollie1> cinder create --volume-type=silver
16:41:21 <jgriffith> ollie1: DuncanT correct? ^^
16:41:37 <DuncanT> jgriffith: It can be used for that, yes
16:41:45 <jgriffith> DuncanT: but that's not your goal?
16:41:45 <DuncanT> jgriffith: That is the use we envisage
16:41:53 <avishay> instead of my extra specs for "gold" being "volume-backend-name=backend1 or volume-backend-name=backend5 or volume-backend-name=backend6" i can say "gold:label=gold"
16:41:55 <rushiagr> jgriffith: true, but that all can be done already by Cinder, isnt it?
16:41:59 <dosaboy> rushiagr: it's an extra level of granularity on top of standard types fwict
16:42:15 <jgriffith> rushiagr: yes, just can be *hard* to setup/explain
16:42:16 <DuncanT> It is nothing on top of types. It still is types
16:42:16 <caitlin56> Who is creating these attributes? Don't we have to distinguish between attributes specified by a user and those that the backend vendor can change when it updates its code?
16:42:19 <avishay> and then i never have to update my extra specs if i change backends, which i don't think works for volume types already being used
16:42:23 <winston-1> so i have 3 back-ends from 3 vendors, they all claimed to support X feature (but differently). i can evaluate the feature for those back-ends i have, and then based on the result, i can Backend A 'label, feature X-good', Backend B 'X-medium', Backend C 'X-low'.
16:42:42 <DuncanT> winston: +1
16:42:46 <dosaboy> s/on top of/within
16:42:47 <DuncanT> Taht's anotehr use
16:42:52 <avishay> winston-1: +1
16:42:53 <jgriffith> avishay: I see almost no benefit
16:43:01 <guitarzan> because indirection!
16:43:09 <jgriffith> avishay: DuncanT winston-1 We can't even agree on reporting capacity
16:43:18 <jgriffith> avishay: DuncanT you think we're going to agree on "features" LOL
16:43:20 <thingee> jgriffith: so in types you can set extra specs. When the capability filters run, how does it know a host has those additional capabilities that are set in extra_spec table?
16:43:21 <avishay> jgriffith: if i add another gold backend, i need to update extra_specs?
16:43:27 <DuncanT> jgriffith: This system means we don't have to agree on anything!
16:43:42 <ollie1> jgriffith so this is a way around agreeing on anything
16:43:43 <avishay> jgriffith: no - that's the beauty of this - the admin can decide everything
16:43:43 <caitlin56> jgriffith: +1 a label totally controlled by the admin is going to see limited use.
16:43:45 <jgriffith> avishay: But that's completely arbitrary
16:43:53 <jgriffith> avishay: what is "Gold" or "Purple"
16:43:57 <guitarzan> avishay: you have to do "something" anyway
16:44:03 <ollie1> up to the admin
16:44:03 <winston-1> then i can create 3 vol types, with extra specs: type 1 'X-good', type 2 'X-medium', type 3 'X-low', instead of type 1 'slowfire', type 2 'appnet', type 3 'cme'
16:44:11 <avishay> winston-1: can i change extra_specs on a volume type that's in use?
16:44:19 <winston-1> avishay: yes, you can
16:44:21 <guitarzan> sure
16:44:23 <jgriffith> wouldn't it be easier to build something on top of volume-types
16:44:31 <jgriffith> so have a type-group
16:44:43 <guitarzan> this is all just a big piece of indirection to map types to backends
16:44:44 <ollie1> easier in what way?
16:44:44 <jgriffith> or I guess, sub-types would be the right way to go
16:44:58 <jgriffith> don't try and auto-set things because it won't work
16:45:00 <DuncanT> jgriffith: This is totally for admin convenience, and if you don't like it you never even see it except as an empty config option
16:45:10 <jgriffith> DuncanT: I'll see it
16:45:18 <jgriffith> I'll see it every time I look/work on the code
16:45:22 <jgriffith> or a bug gets submitted
16:45:31 <jgriffith> or I have to try and explain it at a training session
16:45:38 <avishay> i don't see this as indirection, i see it as "we'll never agree on capabilities, so instead of limiting the admin to what the driver reports, allow the admin to define things that he/she cares about"
16:45:44 <jgriffith> I'm not saying I don't like the idea
16:45:47 <caitlin56> jgriffith: right, if we can't agree on metrics then we won't agree on meta-labels. Labeling can only come from a) the vendor and b) the site admin -- and they have to be clearly separated.
16:45:49 <thingee> I still need clarification on this.
16:45:49 <DuncanT> jgriffith: sorry, I meant 'you' as in 'an admin', bad word choice
16:45:54 <jgriffith> I'm just wondering aobut the usefulness of the current implementation
16:45:55 <bswartz> I really don't see any downside to allowing this -- it gives the admin a little more power a no cost to anybody
16:46:00 <winston-1> avishay: +1
16:46:03 <jgriffith> and the assumptions being made here
16:46:16 <jgriffith> avishay: I agree with that
16:46:30 <jgriffith> I'm just saying a more logical grouping might be better
16:46:33 <jgriffith> so for example:
16:46:45 <jgriffith> I create a volume-type of "foo"
16:46:45 <DuncanT> jgriffith: How about we work up some detailed usecases and send you those... I think you're looking at this totally differently to us
16:47:01 <DuncanT> (please carry on with the example)
16:47:01 <jgriffith> in the extra-specs I have the ability to set multiple backend-names
16:47:06 <jgriffith> that does the same thing no?
16:47:19 <dosaboy> jgriffith: i think an abstraction from types is the opposite of what ollie1 is trying to do
16:47:27 <kenhui> It's essentially a poor man's qos//tiering. class of services decided by a human the admin) wihout input from the vendor supplying the storage.
16:47:31 <dosaboy> if i understand correctly
16:47:32 <DuncanT> jgriffith: Except that can lead to really complex extra-specs that need updating every time I add/remove a backend
16:47:34 <jgriffith> sighhh
16:47:49 <jgriffith> cinder extra-specs add blah
16:47:54 <jgriffith> that doesn't seem so hard
16:48:06 <jgriffith> or "type-key set"
16:48:07 <jgriffith> whatever
16:48:19 <winston-1> we usually would run a set of tests against the storage system we have and then we give them scores, aka labels.
16:48:28 <DuncanT> jgriffith: Using labels, I can change the label on the backend, rather than having to think of a central config
16:49:00 <bswartz> jgriffith: you're right that you can associate a volume type with more than one backend already, but that approach requires you to update the volume type each time you add a new backend
16:49:06 <DuncanT> jgriffith: So you can also use them to temporarily disable a backend for new volumes, as an example, by changing the backend config rather than the central config
16:49:10 <rushiagr> ollie1: is the discussion still on lines with 'ordering' backends wrt some capability?
16:49:25 <bswartz> jgriffith: this approach would let you set your volume type once, and add new backends and they would automatically be able to serve that volume type
16:49:32 <avishay> jgriffith: in my opinion the difference is that listing backends in the volume type would be harder to manage and be less flexible.  with admin-defined capabilities, i can give quantities, booleans, etc and have more logical queries in extra specs rather than just a list of backends.
16:49:43 <jgriffith> bswartz: it's just updating the extra-spec key
16:49:45 <rushiagr> ollie1: if yes, are you proposing grouping of volume types, or separate labels _inside_ volume types?
16:49:47 <jgriffith> DuncanT: bswartz avishay ok
16:49:50 <ollie1> rushiagr: not really more grouping
16:49:51 <jgriffith> ollie1: go for it then
16:49:53 <jgriffith> whatevs
16:49:58 <avishay> haha
16:49:58 <ollie1> than necessarily ordering
16:50:04 <DuncanT> jgriffith: If my extra-specs key has 25 backends listed, that's a bit error prone
16:50:10 <jgriffith> DuncanT: ok
16:50:20 <avishay> DuncanT: see the white flag waving?
16:50:21 <jgriffith> if you have 25 backends in your cloud you've got bigger problems IMO
16:50:44 <jgriffith> DuncanT: and if you expose that many types you're going to have another set of issues as well
16:50:46 <bswartz> hyperscale here we come!
16:50:48 <kenhui> doesn't see 2 be scalable since the underlying capabilities of the storage relative to each other change as u add devices.
16:50:51 <jgriffith> bswartz: ha!
16:50:55 <DuncanT> jgriffith: 25 of the same type of backend... if you want to disable one for a while for new volume for example, same problem
16:51:09 <jgriffith> DuncanT: ok
16:51:11 <avishay> 9 minutes left
16:51:14 <ollie1> jgriffith: do you want this re implemented  using the db instead of config?
16:51:15 <jgriffith> whatever crazy notion you have go for it
16:51:22 <jgriffith> ollie1: no please
16:51:25 <winston-1> avishay: haha, we now have a chance to make a peace world with all vendors in our DC
16:51:28 <jgriffith> ollie1: you'll give thingee  a stroke :)
16:51:29 <ollie1> :)
16:51:38 <avishay> winston-1: :)
16:51:40 <ollie1> thingee suggested it!
16:51:52 <DuncanT> jgriffith: That was the only upside to putting it in the DB I could see ;-)
16:52:24 <kenhui> every vendor will be competing 2 be labeled gold. many gifts 2 the openstack admins will be had. :)
16:52:45 <avishay> i guess the upside to DB would be that you can change labels without taking down the service?  not sure it's worth the effort though?
16:52:54 <winston-1> kenhui: not really, it's not for marketing, it's for operation.
16:52:57 <caitlin56> I'd like to see the following use-case explained: admin has set up their categorization rules for three different vendor backends. Then Vendor Z updates their volume driver. What of the prior work is kept, and what is at risk?
16:53:09 <ollie1> avishay: yup
16:53:14 <avishay> kenhui: nobody except for the admin knows about extra specs or labels
16:53:39 <avishay> caitlin56: no driver updates necessary
16:53:52 <DuncanT> caitlin56: The vendor can't touch these label... they are entirely defined by cinder.conf on the volume service node(s)
16:54:03 <ollie1> caitlin56: no work is necessary, the admin could decide to change the label on that backend
16:54:24 <avishay> 6 minutes remaining, still have 1 topic in the queue
16:54:46 <caitlin56> ollie: so the heavy lifting is on the admin to determine if an update changed any of their rules. correct?
16:54:59 <jgriffith> #topic why is type-manage an extension
16:55:02 <jgriffith> thingee: I don't knwo :)
16:55:04 <jgriffith> know
16:55:07 <ollie1> yes, they have to decide whether performance changed in any way
16:55:11 <thingee> Why is type manage an extension? Is it because we think some admins don't want that expose in the api?
16:55:28 <thingee> and to set things manually in the db
16:55:31 <thingee> that's all I can come up with
16:55:32 <guitarzan> you can policy it out even if it's in the api
16:55:40 <thingee> guitarzan: exactly
16:55:59 <guitarzan> does anyone have a better answr than "cruft?"
16:56:01 <avishay> git blame ... who put it there? :)
16:56:13 <winston-1> avishay: i guess it's jgriffith
16:56:13 <kenhui> this is completely admin defined. in theory, he could change labels at anytime even if nothing changes with the storage just cuz he feels like it.
16:56:33 <thingee> my proposal was to bring it in since types is already core
16:56:37 <winston-1> kenhui: why not, that's the beauty of it
16:56:45 <winston-1> thingee: +1000
16:56:51 <avishay> thingee: +1001
16:57:01 <DuncanT> What does  the type manage API provide? I can't say I've looked
16:57:02 <thingee> ok, that's all
16:57:23 <winston-1> DuncanT: create/delete/update type, extra specs
16:57:30 <thingee> crud
16:57:56 <avishay> thingee: any backward compatibility issues with moving it?  i'm sure DuncanT wants to know :P
16:57:57 <DuncanT> Ah, that one... sounds core to me, as long as the policy rules don't suddenly need an update
16:58:05 <thingee> where core just provides the "R" of crud :)
16:58:19 <DuncanT> (Avishay knows me so well...)
16:58:25 <kenhui> winston-1: no issues. just clarifying this a admin defined process.
16:58:28 <thingee> avishay, DuncanT: I think you guys can trust me on backewards compatibility.
16:58:30 <avishay> DuncanT and his moving bus...
16:58:32 <thingee> ;)
16:58:36 <avishay> thingee: yessir
16:58:42 <thingee> even spelling it right
16:58:43 <guitarzan> buses that don't move are called buildings
16:59:13 <avishay> :)
16:59:18 <avishay> 1 minute left...
16:59:20 <DuncanT> guitarzan: You've not seen the traffic round here....
16:59:22 <thingee> I'm done
16:59:27 <thingee> jgriffith: ^
16:59:45 <avishay> one more topic - is next week canceled?
16:59:51 <avishay> and the week after?
17:00:16 <thingee> xmas for some folks next meeting
17:00:32 <avishay> and then new years
17:00:55 <thingee> jgriffith: end meeting?
17:01:17 <avishay> thingee: i think he's on mute
17:01:24 <bswartz> time's up
17:01:41 * hartsocks waves
17:02:11 <garyk> \o
17:02:13 <jgriffith> #endmeeting