16:00:57 #startmeeting cinder 16:00:58 Meeting started Wed May 30 16:00:57 2018 UTC and is due to finish in 60 minutes. The chair is jungleboyj. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:59 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:01:01 The meeting name has been set to 'cinder' 16:01:17 courtesy ping: jungleboyj DuncanT diablo_rojo, diablo_rojo_phon, rajinir tbarron xyang xyang1 e0ne gouthamr thingee erlon tpsilva patrickeast tommylikehu eharney geguileo smcginnis lhx_ lhx__ aspiers jgriffith moshele hwalsh felipemonteiro lpetrut lseki _alastor_ 16:01:18 hey 16:01:23 <_alastor_> o/ 16:01:30 hi! o/ 16:01:30 hey 16:01:35 hi 16:01:47 hello 16:01:48 o/ 16:01:56 hi 16:02:06 hi 16:02:09 hi 16:02:33 hi 16:03:08 Hey, looks like a good turnout. 16:03:40 Have a lot to talk about so lets get started. 16:03:45 bah 16:04:04 #topic Announcements 16:04:19 hello 16:04:27 I have gotten the summary of our forum topics put together: https://wiki.openstack.org/wiki/VancouverSummit2018Summary 16:04:44 There are links to the associated etherpads. 16:04:48 o/ 16:04:55 Lots of good discussion at the Forum. 16:05:14 Please take a look when you have a chance. I have one recording up. Working on getting the rest up. 16:05:18 jungleboyj: thanks for writing the summary! 16:05:29 Thanks to smcginnis for getting me YouTube access. 16:05:56 geguileo: You are welcome. You were missed there. 16:06:11 jungleboyj, thanks 16:06:18 good stuff 16:06:32 I missed you all too (pictures didn't help) 16:06:38 Swanson: o_O 16:06:51 geguileo: Sorry. ;-) 16:07:16 Oops. #link https://wiki.openstack.org/wiki/VancouverSummit2018Summary 16:07:38 A reminder that Rocky Milestone 2 is next week. Will be tagging that in about a week. 16:07:49 #link https://releases.openstack.org/rocky/schedule.html 16:07:58 More on that a little later. 16:08:31 Also want to make everyone aware that I am planning to go through CIs are start marking drivers unsupported early next week. 16:08:49 * smcginnis knows there are a lot 16:08:59 Give them a month to address before the end of Rocky which seems fair. 16:09:05 smcginnis: Yeah, afraid of that. 16:09:30 I will also send out a note on that to the mailing list so that I can at least say I have given fair warning. 16:09:52 jungleboyj: sounds reasonable for me, a month should be enough to fix CI 16:10:02 e0ne: One would hope. :-) 16:10:11 :) 16:10:59 I think that is all I had for announcements. 16:11:16 #topic Rocky Priorities Review 16:11:28 Main thing here is looking at drivers with Milestone 2 approaching. 16:11:41 Hedvig is still failing CI. That may be a lost cause. 16:12:02 Nexenta looks like it is good to go. I will make time to look at/merge that in the next day or so. 16:12:23 Anyone have anything else to cover on that topic? 16:12:25 Speaking of milestone 2, I think there may be one or two specs that are good to go too. 16:12:47 smcginnis: Good point. I can take a look at those too. 16:12:51 smcginnis, do we have a spec priority list? 16:13:08 erlon: 16:13:11 erlon: Just jungleboyj's overall priority list 16:13:11 #link https://etherpad.openstack.org/p/cinder-spec-review-tracking 16:13:25 ok, thanks 16:13:46 #action jungleboyj to get Nexenta driver merged. 16:13:57 #action core team to look at specs and merge what we can. 16:14:14 Anything else? 16:14:36 #topic HA Development Progress 16:14:43 geguileo: Anything to report here? 16:14:53 jungleboyj: yup, writing the doc is a nightmare!!! 16:15:03 I just submitted a WIP patch: https://review.openstack.org/571242 16:15:05 geguileo: :) 16:15:09 geguileo: Hey, that is progress though. :-) 16:15:17 geguileo, I saw you are working in a nice documentation patch 16:15:24 that ^ 16:15:28 yup 16:15:40 I submitted the WIP patch so I can get feedback 16:15:53 geguileo: thanks! 16:16:01 Because there is a lot of stuff in there that we know because we've been doing Cinder for a while 16:16:05 geguileo, I started reading it, ill give some feedback 16:16:11 geguileo, thanks for that! 16:16:14 Awesome. Thank you. Will try to take a look soon. 16:16:22 erlon: jungleboyj thanks! 16:16:40 I tried to give an overview of everything that must be taken into consideration 16:16:52 but I didn't explain everything in detail, because that would take me forever 16:16:52 geguileo: That would be a great start. 16:17:12 jungleboyj: that's the idea, have something 16:17:26 Feedback from the Forum session is any documentation is better than none. 16:17:26 and then we can iterate over it, split it, extend it, etc 16:17:36 jungleboyj: rofl 16:18:34 :-) 16:19:23 Ok, so I think the todo there is to get everyone looking at your patch. 16:19:36 #action team to review geguileo 's documentation patch. 16:19:47 thanks 16:20:18 geguileo: Thank you for working on this! 16:20:32 We will talk a little more about this in my last topic. 16:20:42 Want to make sure that we get to your topic first though. 16:20:57 #topic Reusing Cinder Drivers 16:21:30 geguileo: The floor is yours. 16:21:37 thanks 16:21:51 Some of you heard me talk about cinderlib on the Dublin PTG 16:22:09 It's a library that allows any python application using the Cinder drivers 16:22:18 without changing the Cinder drivers code 16:22:39 So you can install something like the RDO cinder package and then from a python program manage your storage backend 16:22:51 The idea is to reuse Cinder drivers in other places 16:23:02 So I wrote an Ansible POC that uses this 16:23:09 And also a CSI plugin 16:23:13 geguileo: in general, I like this idea 16:23:28 geguileo: just need to read your blog posts first 16:23:32 So now a Cinder driver can be used in OpenStack, Ansible, Containers, and our own programs 16:23:38 geguileo: and try to use it 16:23:51 This is making our internal interface with drivers a public contract. 16:23:52 geguileo, do you have the link for the blog post? 16:23:52 e0ne: sure, and feel free to tell me what doesn't work ;-) 16:23:59 So sorry, but #vote helzno 16:24:07 My 2 cents. :] 16:24:12 geguileo: I like it but still trying to understand the main motivation versus using Cinder 16:24:35 why would we not want to have a well-defined driver contract? 16:24:39 jgriffith: the motivation is that many people don't want to deploy Cinder 16:24:51 eharney: who said we did not? 16:24:59 eharney: We do, but we also need to be able to change and evolve that as history has shown. 16:25:15 smcginnis: each cinderlib release would either be pinned to a specific Cinder release 16:25:26 smcginnis: or it would have to internally manage the different versions 16:25:32 geguileo: but the only thing this really removes from the Cinder deployment is rabbit and the API server no? 16:25:51 jgriffith: it removes all the services 16:26:05 jgriffith: API, volume, scheduler, rabbit, database 16:26:13 jgriffith: you can use your own data storage for the metadata 16:26:29 for example for Kubernetes it would be a plugin that stores the data in kubernetes as CRDs 16:26:30 eharney: my question simply was what's the motivation, I didn't know that we didn't have a defined interface ? 16:26:42 that way you don't need to deploy antyhing else 16:27:01 jgriffith: just trying to understand smcginnis's comment about not wanting to have a public contract 16:27:02 geguileo: but on the DB you add back in sqlite or as you said it's pluggable 16:27:06 jgriffith: the motivation is that many, many people kept asking me about using Cinder drivers outside of Cinder 16:27:08 so I'm confused by that 16:27:15 that's a good news for the vendors 16:27:16 jgriffith: it's pluggable 16:27:26 jgriffith: right now I only have memory and database plugin 16:27:34 jgriffith: but I'll soon be writing a CRD plugin 16:27:44 So... 16:28:01 geguileo: If that is true I can see motivation to get ahead of the users and provide a solution. 16:28:04 here's what I'm not following 16:28:24 if you use things like block-box I would love to know what the complaints are about deploying Cinder? 16:28:31 How do we not end up just re-implementing Cinder though> :-) 16:28:34 Now if you're talking about deploying openstack, sure I get that 16:28:42 So deploy a few containers, or setup up cinderlib and configure several plugins to match your environment? 16:29:16 smcginnis: you just configure 1 plugin 16:29:23 My other point of confusion is how adding a CSI entrypoint to Cinder doesn't accomplish the same thing in terms of driver reuse and well defined interface? 16:29:25 smcginnis: and the plugin is a python library 16:29:45 The final option is, dump Cinder altogether, create a CSI based storage project for OpenStack and move along 16:29:50 jgriffith: if you want to use the drivers in Ansible, it's not the same thing 16:29:59 becuase what cinder-lib proposes is zero cinder value and just driver code 16:30:03 jgriffith: If I want to write my own code, it's not the same thing 16:30:07 but CSI drivers are pretty dead simple to write 16:30:32 geguileo: wait.. use the drivers in Ansible? Sorry.. what's that look like? 16:30:44 * jgriffith is confused now.. sorry 16:31:03 jgriffith: that's another POC, it's a generic storage Ansible role 16:31:15 sigh 16:31:16 jgriffith: it will manage block, filesystems, and object storage 16:31:29 So there's a number of objectives it sounds like 16:31:35 jgriffith: it's trying to do the same as the network ansible role 16:31:47 an abstraction, and the base driver would be cinderlib 16:32:09 so replace the cinder abstraction with a cinder-lib abstraction to consume the Cinder drivers 16:32:14 jgriffith: on the topic of CSI being easy to write, I disagree 16:32:24 jgriffith: it's easy to write "something" 16:32:29 Are the cinder drivers by themeselves really that valuabe? 16:32:34 not so easy to cover all corner cases 16:32:40 jgriffith: yes, they are 16:32:41 geguileo: true 16:32:58 geguileo: my contention is that most vendors are going to be writing CSI Plugins anyway though 16:33:12 jgriffith: maybe once the CSI spec is finalized 16:33:21 jgriffith: but while they are making breaking changes... 16:33:24 probably not 16:33:36 Just as you say "most customer you talk to find Cinder to difficult to deploy" most customer and vendors I talk to would prefer independent CSI plugins 16:33:39 jgriffith: many implementations haven't even caught up with the current specs 16:34:13 True, but honestly part of that is the fault of K8's not being able to decide what to do here :) 16:34:27 TBF we have changed our position on drivers like 3 times in the last year :) 16:34:29 jgriffith: and they are going to keep working like that 16:34:43 :-) 16:34:50 Nah, I think it's pretty clear that CSI is the way forward, and that's being communicated fairly well now 16:34:59 We are evolving. 16:35:00 geguileo: but I do see your point 16:35:18 jgriffith: and I'm sure customers will prefer having a cinderlib driver than having nothing 16:35:19 So all of that aside, a CSI endpoint in a Standalone Cinder... 16:35:33 jgriffith: it could be 16:35:40 The only difference from that and cinderlib is the scheduler and the rpc service that it requires no? 16:35:58 standalone cinder driver 16:36:22 tommylikehu: I mean standalone cinder service, but you mean equate cinder-lib to standalone cinder-driver? 16:36:45 yeah 16:37:07 It's funny.. years ago we went round and round about things like external libs for storage device API's :) 16:37:10 jgriffith: and the volume service, and how cinder is tied to DB for storing the metadata 16:37:20 now we're basically at a point where that's really what we want :) 16:37:39 jgriffith: it's not what we want, it's just another use for Cinder drivers 16:37:45 geguileo: the volume service is mostly just the driver 16:37:54 jgriffith: not really... 16:38:01 geguileo: it is in my world :) 16:38:06 rofl 16:38:10 if you take out the replication and A/A stuff :) 16:38:26 I do think there's a lot of demand and value in things like scale-out and multi-backend 16:38:31 and the scheduler obviously 16:38:48 and the abstraction that it provides, particularly in a K8's env 16:39:10 but I have a feeling that no amount of discussion is going to budge your opinion on this :) 16:39:21 lol 16:39:33 it's not like that 16:39:37 and it would seem that maybe you and eharney have had some discussions and some insight on this 16:39:50 geguileo: nahh... I didn't mean that in a critical way 16:39:55 I just think there is value in the possibilities that using Cinder drivers outside of Cinder 16:40:16 jgriffith: I haven't really discussed this much with eharney XD 16:40:16 I just mean that you seem to have very specific data and experience that you're trying to solve a certain problem 16:40:44 geguileo: ok, sorry then 16:40:52 alright.. so let's talk about this a little more 16:40:58 so by using just the driver.. 16:41:02 originally, I thought it's about using api-scheduler-volume-driver but w/o rpc. I need to read blog and take a look in the code 16:41:16 e0ne: that was one idea I originally had 16:41:17 I do see some value of just sucking in the python and *cheating* so to speak 16:41:22 I have a number of concerns though 16:41:24 e0ne: But I decided to go on a different direction 16:41:46 We've pushed people a lot over the years to move more and move functionality up in to the manager 16:42:03 jgriffith: +1 16:42:08 and every driver is very much specifically written from a 'cinder' perspective 16:42:34 I'm a bit concerned that as CSI matures we won't really have the mechanism to do what is needed 16:42:56 The bigger problem with that is how do we deal with special "CSI only" features being submitted to Gerrit for a Cinder plugin? 16:43:18 jgriffith: that one is true 16:43:26 We don't test any of the CSI stuff, and we're most likely going to have code int he drivers over time that would *appear* to be just "dead code" 16:43:31 jgriffith: but I believe CSI is mostly going to redo what we did here :-( 16:43:34 but in reality it's consumed/used elsewhere 16:43:45 hehe.. I have no argument on that! 16:43:56 I've always said CSI should just be Cinder :) 16:43:58 BUT 16:44:19 That being said, I do think they'll likely move forward proposing the same features 16:44:37 BUT as we know defining those features and how they actually work is different for everyone 16:44:37 jgriffith: I am thinking that more and more. 16:44:50 that includes CSI vs Cinder 16:44:57 not just Vendor-a vs Vendor-b 16:45:03 jgriffith: yeah, but we are also evolving Cinder 16:45:23 geguileo: ++ 16:45:25 so we can help them on the CSI front like some Cinder folks have been doing 16:45:35 fair point 16:45:53 geguileo: I think anything we can do to be relevant to CSI is valuable. 16:46:00 If that makes sense ... 16:46:08 jungleboyj: to me it does 16:46:27 it's easier to explain to a company why they should have an engineer working on Cinder 16:46:31 Feedback from the Forum is that people are looking for Cinder and k8s integration with and without OpenStack. 16:46:33 jungleboyj: so to be clear, I'm not saying we should be relevant to CSI 16:46:40 if you tell them their driver will work on Ansible and on Containers as well 16:46:59 jgriffith: You mean you aren't saying we shouldn't be relevant? 16:47:13 I am saying however that instead of hacking a layer over the drivers it might be worth considering being an actual CSI plugin or providing a CSI interface 16:47:42 Or we follow neutrons lead a couple years ago, create the stadium and drivers are just things on their own 16:47:51 FWIW that didn't work out very well for them I don't think 16:47:56 but I could be wrong 16:48:12 jgriffith: Ok, I understand that. 16:48:30 jgriffith: we could have both options: cinderlib and a CSI service 16:48:36 jgriffith: I agree that free range drivers hasn't been the best for Neutron. 16:48:38 I feel like I'm monopolizing the conversation here so I'll shut up for a few minutes :) 16:48:59 jgriffith: so we could have a node with the Cinder-API and the Cinder-CSI services 16:49:13 geguileo: yeah, that's certainly possible; but TBH I'd rather have concensus and build somethign as a community 16:49:25 that would be usefull for deployments where they have containers and OpenStack 16:49:27 one of my gripes in the new world has been fragmentation 16:50:02 I also can't handle creating another *thing* that never gets used :( 16:50:16 cinderlib-csi should be easy to maintain if we test cinderlib with Cinder 16:50:18 * jgriffith sheds a tear 16:50:35 lol 16:50:38 jgriffith: that's another reason to have cinderlib-csi, you don't have to write a CSI driver 16:50:40 :) 16:50:40 rofl 16:50:44 jgriffith: What did you make that didn't get used? 16:50:45 LOL 16:50:47 touchet geguileo 16:50:54 but I already did :( 16:51:05 * jungleboyj sighs 16:51:14 jgriffith: You are just too on top of things. 16:51:30 jungleboyj: no, that's not the case... ask smcginnis I'm just ADD 16:51:38 :) 16:51:40 :-) 16:51:45 jgriffith: does't it have the new snapshot feature? };-) 16:51:51 jgriffith: Nice to have you back in the meetings. 16:52:06 s/doesn't/does 16:52:07 geguileo: one of them does :) 16:52:11 rofl 16:52:11 Regardless of the rest, I am in favor of adding a cinder-csi service as an official Cinder team deliverable as an alternative/addition to cinder-api. 16:52:15 jgriffith: nice! 16:52:16 but yeah... I'm with ya 16:52:23 So, geguileo What was your goal today? 16:52:41 my goal was to see if there was interest in cinderlib 16:52:44 to get /me to shut up :) 16:52:51 ROFL 16:52:54 as a gateway to make Cinder and its drivers more relevant 16:53:04 smcginnis: +1 16:53:16 and if we would be willing to run it's functional tests at the gate on the tempest jobs 16:53:20 smcginnis: I think that makes sense. 16:53:36 it takes under a minute to run them once you have an already deployed Cinder node 16:53:38 So, we are running out of time and I would like to roll up a plan here. 16:54:02 I think if we are going to do cinderlib we need to have it documented and tested in the gate. 16:54:16 +1 16:54:21 I don't feel like we have reached consensus as to whether we want it as a project. 16:54:28 jungleboyj: it is documented, I just need to keep improving the documentation 16:54:32 jungleboyj: geguileo I think it would be good for folks with drivers to clone the repo and play with things a bit and see what they think 16:54:33 I don't think we have agreed that we don't want it as a project. 16:54:38 jungleboyj: +1 16:54:54 geguileo: That is great . I remember seeing that. 16:54:55 then maybe we can chat again next week assuming everybody does their homework? 16:54:55 jgriffith, +1 16:54:58 jgriffith: ++ 16:55:12 jgriffith: +1 16:55:20 jgriffith: I am good with that. 16:55:22 and for the record, I'm certainly NOT saying I don't want it 16:55:27 jgriffith: +1 16:55:31 I just want to weigh things out thoroughly 16:55:33 :) 16:55:39 jgriffith: I know, and I appreciate the discussion 16:55:52 * geguileo prefers discussion to silence 16:56:01 I also think that we need to make time at the PTG or some time between now and then to have the larger Cinder/CSI/etc etc. discussion. 16:56:03 geguileo: good, thanks! I know I can sometimes come across the wrong way on discussions like this 16:56:13 geguileo: ++ Nice to have a lively meeting. 16:56:23 jungleboyj: +1 16:56:38 Ok. So ... 16:56:38 jungleboyj: ^ on the PTG discussion thing 16:56:59 #action Driver developers and team to pull down the cinder-lib code and try it out. 16:57:20 #action jungleboyj to put continued discussion on the agenda for next week. 16:57:47 there's a section in the docs 16:57:51 #action Ensure we have future discussion around cinder-lib / CSI / general future direction for Cinder. 16:57:53 on how to test a driver 16:57:56 https://cinderlib.readthedocs.io/en/latest/validating_backends.html 16:58:03 geguileo: cool. 16:58:16 #link https://cinderlib.readthedocs.io/en/latest/validating_backends.html 16:58:18 geguileo, cool 16:58:37 In the last minute I would like to move on. 16:58:48 #topic Follow-up topics from the Forum: 16:59:02 So, there are some things from the Forum that we need to discuss as a team. 16:59:09 I have included the info in the agenda. 16:59:21 #link https://etherpad.openstack.org/p/cinder-rocky-meeting-agendas 16:59:37 Please take a look at those. I will get the audio posted later today/tonmorrow. 17:00:11 I am going to push those items forward to next week's agenda after people have had chance to look at the discussion that happened there and we can figure out the path forward. 17:00:18 Hope that sounds good to everyone. 17:00:34 jungleboyj: sounds good 17:00:38 And on that note we need to wrap up. 17:00:38 sounds like we are running out of time :) 17:00:48 Thank you everyone for good discussion today. 17:00:55 #endmeeting