16:00:39 <jungleboyj> #startmeeting cinder
16:00:40 <openstack> Meeting started Wed Aug  7 16:00:39 2019 UTC and is due to finish in 60 minutes.  The chair is jungleboyj. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:42 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:43 <whoami-rajat> Hi
16:00:44 <openstack> The meeting name has been set to 'cinder'
16:00:57 <jungleboyj> courtesy ping:  jungleboyj whoami-rajat rajinir lseki carloss pots woojay erlon geguileo eharney rosmaita enriquetaso e0ne smcginnis davidsha walshh_ xyang hemna _hemna tosky
16:00:59 <smcginnis> o/
16:01:04 <pots> o/
16:01:08 <e0ne> hi
16:01:09 <rosmaita> o/
16:01:09 <eharney> hi
16:01:11 <tosky> o/
16:01:12 <geguileo> hi! o/
16:01:26 <kaisers> hi
16:01:30 <jungleboyj> @!
16:01:30 <_pewp_> jungleboyj (。･∀･)ﾉ
16:01:34 <woojay> o/
16:01:52 <jungleboyj> This week's agenda:
16:01:53 <xyang> hi
16:01:56 <jungleboyj> #link https://etherpad.openstack.org/p/cinder-train-meetings
16:03:24 <jungleboyj> Looks like we have all the usual suspects here.
16:03:29 <jungleboyj> So, we can get started.
16:03:36 <jungleboyj> #topic announcements
16:03:57 <jungleboyj> We are just two weeks from our mid-cycle:  https://etherpad.openstack.org/p/cinder-train-mid-cycle-planning
16:04:04 <jungleboyj> #link https://etherpad.openstack.org/p/cinder-train-mid-cycle-planning
16:04:30 <jungleboyj> Topics are still a little light.
16:04:44 <jungleboyj> Glad to see we are back up to 5 people in physical attendance.
16:04:47 <jungleboyj> Welcome woojay !
16:04:53 <woojay> thank you.
16:05:20 <whoami-rajat> jungleboyj: should we also add the feature progress discussion there.
16:05:21 <lseki> hi
16:05:34 <jungleboyj> So, please get your topics into the etherpad.
16:05:41 <davidsha> Hey!
16:05:53 <jungleboyj> whoami-rajat:  Yes.  I was going to work on going through our previous PTGs and pull todos and stuff out of there.
16:05:57 <pots> pwd
16:06:11 <jungleboyj> If you want to start a section on feature progress that would be great.
16:06:24 <jungleboyj> pots: #/root
16:06:41 <e0ne> :)
16:06:48 <whoami-rajat> jungleboyj:  okay, great.
16:06:58 <jungleboyj> whoami-rajat:  Thank you.
16:07:25 <jungleboyj> I will be getting in to Raleigh early Monday so I should be in town before everyone if you need anything.
16:07:31 <jungleboyj> I will add my cell phone to the etherpad.
16:09:09 <jungleboyj> Ok.  I think that is all on that for today.  More next week.
16:09:28 <jungleboyj> I have gone through all our CI systems and marked those that just aren't reporting as unsupported.
16:09:38 <jungleboyj> #link https://review.opendev.org/#/q/status:open+project:openstack/cinder+branch:master+topic:ci_unsupported
16:10:25 <jungleboyj> In a sign that the process works, Synology saw the patch and got things fixed.  :-)
16:10:50 <rosmaita> \o/
16:11:00 <jungleboyj> rosmaita: Indeed.
16:11:08 <jungleboyj> Or they are working on fixing things at least.
16:11:38 <jungleboyj> More on CIs later.
16:11:45 <jungleboyj> rosmaita: Had an announcement.
16:12:08 <rosmaita> just a reminder of some upcoming deadlines
16:12:23 <rosmaita> Train release of non-client libraries (os-brick, cinderlib) roughly 4 weeks away (week of 2 September)
16:12:34 <rosmaita> M-3 is the week after that (week of 9 Sept)
16:12:46 <rosmaita> M-3 marks the Train release of client libraries (python-cinderclient, python-brick-cinderclient-ext)
16:12:53 <rosmaita> it is also the feature freeze
16:13:18 <rosmaita> that's all
16:14:00 <jungleboyj> rosmaita:  How is patch throughput on those libraries?
16:14:11 <jungleboyj> Do we need more attention to get patches in?
16:14:18 <rosmaita> haven't really checked
16:14:39 <jungleboyj> rosmaita:  Ok, something we should look into before the mid-cycle so we can get reviews done if necessary.
16:14:54 <rosmaita> right
16:15:01 <jungleboyj> :-)
16:15:44 <jungleboyj> I added it to the etherpad.
16:15:56 <rosmaita> spot check of os-brick looks good
16:16:01 <jungleboyj> Ok.  Good.
16:16:14 <jungleboyj> I thought I had looked a little while back and things looked ok.
16:16:44 <jungleboyj> I think that is good for announcements.
16:17:03 <jungleboyj> #topic Python 3rd Party CI requirement
16:17:26 <jungleboyj> So, despite our best efforts to communicate this new requirement.  It appears that half our CIs haven't made a chance.
16:17:38 <jungleboyj> #link https://etherpad.openstack.org/p/cinder-py3-ci-review
16:17:40 <e0ne> :(
16:18:11 <e0ne> unfortunately, it will be a hard requirement in the next release anyway
16:18:13 <kaisers> just as a note: I'm one of the stragglers and now working on it
16:18:22 <kaisers> (quobyte_ci)
16:18:47 <jungleboyj> I sent out a note to all CI Maintainers and have not seen e-mail back.
16:19:00 <jungleboyj> I do see that two maintainers have updated the etherpad as requested.
16:19:08 <jungleboyj> kaisers: Thank you for the update.
16:19:24 <jungleboyj> Also Hedvig is working on support.
16:19:39 <jungleboyj> We have a couple CIs that need to be updated from py3.5 to 3.7
16:19:54 <jungleboyj> Not rushing to mark them unsupported as they are on 3.5
16:20:22 <jungleboyj> Thank you to rosmaita and pots for putting together some helpful information.
16:20:26 <jungleboyj> #link https://wiki.openstack.org/wiki/Cinder/3rdParty-drivers-py3-update
16:20:44 <jungleboyj> So, the question is what do we do now?
16:20:55 <jungleboyj> Do I mark half out drivers as unsupported?
16:21:01 <jungleboyj> *our
16:21:24 * jungleboyj doesn't really want to be the PTL that cut the project in half.  :-)
16:21:25 <e0ne> IMO we need to do it
16:21:31 <smcginnis> Better to do it now and give some warning than just having them removed in U because they can no longer work.
16:21:33 <e0ne> because they won't work in U
16:21:38 <smcginnis> e0ne: ++
16:21:50 <jungleboyj> e0ne: smcginnis ++
16:21:52 <eharney> just for context... why won't they work in U?
16:21:52 <jungleboyj> Ok.
16:22:07 <e0ne> eharney: U release will drop python2 support
16:22:17 <eharney> ah
16:22:56 <jungleboyj> The thing that scares me is that the vendors continue to be ignorant of this.
16:22:57 <rosmaita> i wonder whether we leave them in for T but have an additional flag or something for the driver page
16:23:14 <rosmaita> give just a bit more time to get them fully supported for U
16:23:36 <jungleboyj> We pull the drivers in V and then when customers update to that level in a couple years we have a crap storm on our hands.
16:23:48 <e0ne> rosmaita: that's why we introduced 'unsupported' flag
16:24:13 <smcginnis> They will be left in for T, they just will require an operator to explicitly declare in their config that they are aware they are running a driver that is not validated.
16:24:35 <jungleboyj> rosmaita:  I don't have a problem with adding a py3 support flag in the matrix so it is clear that is why it is being deprecated.
16:24:45 <rosmaita> well, the thing is, if they are running T on py2, they should actually be supported, right?
16:25:33 <smcginnis> But then if they don't in U, we haven't gone through a deprecation period to remove them.
16:25:38 <pots> I didn't make any changes to my driver when I switched from 2.7 to 3.7, so it's conceivable that these other drivers won't all stop working when python2 is gone.
16:26:08 <jungleboyj> :-)
16:26:09 <smcginnis> pots: Yes, but no way to know that for sure and that is why we have the third party CI requirement we have had since ~icehouse.
16:26:19 <eharney> have we heard anything about issues blocking CIs from moving to py3?  or is it mostly just about making sure someone goes and does it?
16:26:29 * jungleboyj loves pots' optimism.
16:26:49 <smcginnis> Hedvig did find they have a code issue with py3.7, but likely something easy(ish) to fix.
16:26:58 <jungleboyj> eharney:  Based on pots  it is about people actually paying attention and doing it.
16:28:02 <jungleboyj> eharney or rosmaita:  Do you have any strings as Red Hat to pull to try to get more attention to this?
16:28:12 <jungleboyj> This is going to impact you guys as much as anyone.
16:28:35 <eharney> jungleboyj: our string is that drivers will have to certify on python 3.6 for the Stein release of our product
16:29:11 <jungleboyj> eharney: So, they need to show support even before train.
16:30:11 <jungleboyj> So, some vendors will get hit by this for RHOSP16 ?
16:31:10 <eharney> osp15
16:31:17 <jungleboyj> Oh.  That is right.
16:31:36 <jungleboyj> That is in beta right now.  So, what testing has been done there?
16:32:45 <eharney> i'm not sure it's gotten too far down the path of driver certification yet
16:32:54 <jungleboyj> Ok.
16:33:08 <jungleboyj> eharney:  So, that isn't too helpful at this point.
16:33:16 <pots> just curious, does anyone offer Cinder CI as a service, as an alternative to every vendor rolling their own?
16:33:28 <smcginnis> :)
16:33:37 <jungleboyj> Do you think you can find out a bit more before the mid-cycle eharney.
16:33:42 <smcginnis> A few of us have joked about starting a business to do that for years.
16:33:45 <jungleboyj> pots: Someone could be rich.
16:33:57 <jungleboyj> pots:  Looking to start a business?
16:33:59 <eharney> jungleboyj: sure
16:34:01 <e0ne> pots: only for opensource storages :)
16:34:53 <jungleboyj> eharney:  Thanks.
16:35:16 <pots> well, this is so different from how we certify storage for Windows, VMware, etc.  It might be easier for some companies to just write a check than to stand up a CI like this in house.
16:35:44 <smcginnis> Yeah, I do think it's a legitimate business idea.
16:35:44 <jungleboyj> pots:  But then someone needs to have one of every storage backend.
16:35:48 <e0ne> pots: such CIs require to have a hardware
16:35:55 <jungleboyj> e0ne: ++
16:36:24 <pots> hardware is the easy part for the vendors
16:36:25 <smcginnis> Just need a colo facility and customers to ship their hardware to you.
16:36:49 <pots> I'm in NYC.  I DO have free electricity in my apartment but...
16:36:55 <smcginnis> Haha
16:37:00 <jungleboyj> pots:  Come join us at the mid-cycle and we can start a business plan.
16:37:04 <smcginnis> Anyway, probably should get back to business.
16:37:32 <jungleboyj> I am going Solar and to 1 GB fibre this fall.  Not sure I have enough outlets.
16:37:39 <jungleboyj> Yeah, back to business.
16:37:50 <jungleboyj> So, it sounds like I should start pushing up the unsupported patches.
16:38:03 <jungleboyj> Do we want to add a field to the support matrix?
16:38:06 <e0ne> jungleboyj: +1
16:38:24 <rosmaita> so getting back to the 'unsupported' issue, is it possible to keep them supported in T but 'deprecated' due to lack of py3 CI?
16:38:50 <jungleboyj> rosmaita: The previous answer was that that is what the unsupported flag is for.
16:39:08 <rosmaita> yeah, but it seems different semantically
16:39:17 <rosmaita> though i haven't been around here very long, admittedly
16:39:26 <jungleboyj> The point there is to get people's attention when they upgrade.  The driver is about to not work and the operator should know that.
16:39:44 <smcginnis> Well, if it was a requirement to run CI under py37, and they didn't update their CI, then that does meet the requirement that it is unsupported.
16:39:57 <smcginnis> But I get the nuance.
16:40:18 <rosmaita> i'm just worried about the sheer number of noncompliant drivers
16:40:35 <jungleboyj> So, I guess that is an optics issue.
16:40:58 <jungleboyj> Do we add a new flag/warning for T and hope that people do something about it.
16:41:00 <whoami-rajat> jungleboyj: the drivers running py35 or py36 won't be marked unsupported right?
16:41:30 <jungleboyj> whoami-rajat: No, but I will be following up with them to get them to udpate it.
16:41:51 <eharney> is there a doc/landing page somewhere for people to get all the info if they are suddenly finding a "your driver is unsupported." message from cinder?
16:42:40 <jungleboyj> Well, we have the release note that goes with it explaining what it means.
16:42:55 <jungleboyj> We don't have a page explaining all that that I know of.  Happy to add some documentation.
16:43:29 <smcginnis> I think the release notes we include with that are pretty good - https://review.opendev.org/#/c/674642/1/releasenotes/notes/storpool-unsupported-11b221fed109535f.yaml
16:43:31 <jungleboyj> That is probably a good thing to do.
16:44:24 <jungleboyj> Yeah, it tells them what to do.
16:44:42 <jungleboyj> I think it would be good to have an operator facing page that explains why this happens thouhg.
16:44:59 <jungleboyj> Just now that we are likely to have more people running into this.
16:45:17 <jungleboyj> Want to make it clear that this is something they should follow up with the vendor on, not be mad at OpenStack.
16:46:20 <jungleboyj> So, I will look into doing that.
16:46:31 <rosmaita> i guess my caveat to marking these unsupported would be the releasenote should be clear it's a py3 CI issue
16:46:47 <rosmaita> i think we want to distinguish people doing py2 CI vs. no CI at all
16:46:47 <jungleboyj> rosmaita:  Ok.  Agreed.
16:46:55 <jungleboyj> rosmaita: ++
16:47:04 <rosmaita> ok
16:47:35 <jungleboyj> Ok, so, make the reason clear in the release note.  Improve the documentation on what it means to be unsupported.
16:48:14 <rosmaita> i can take a crack at the doc improvement, because i clearly don't have a good grasp of this at the moment
16:48:17 <jungleboyj> Start submitting patches for this.  Try to hold out until late in the release to merge the patches to give other people time to start fixing this.
16:48:26 <jungleboyj> rosmaita: Ok.  Sounds good.
16:48:32 <jungleboyj> I can help you with that.
16:48:40 <rosmaita> great
16:48:51 <jungleboyj> Think we should do that in official documentation.  Not a wiki.
16:49:13 <rosmaita> right, i think on the support matrix page (though we can discuss offline)
16:49:21 <jungleboyj> Ok.  Sounds good.
16:49:30 <pots> Does OpenStack have a marketing person who should be aware of this?  It seems like we should be reaching out to the vendors' marketing folks in case the right people didn't see Jay's e-mail.
16:49:56 <rosmaita> pots: ++
16:49:57 <jungleboyj> pots: Ah, good question.
16:50:07 <jungleboyj> Let me follow up on that with the foundation.
16:51:19 <jungleboyj> Wes.  I will send an e-mail to him today.
16:51:41 <jungleboyj> And Erin too maybe.
16:52:12 <smcginnis> I'm a little concerned that we are flipping this where the community needs to be the ones watching and taking care of vendor drivers when it should be the other way around. If a vendor has customers that want to use their storage in OpenStack, it's on them to stay involved and make sure that is available.
16:53:02 <e0ne> smcginnis: +1
16:53:04 <jungleboyj> smcginnis: I agree, but this is also at a point where I feel like this should be escalated.
16:53:06 <rosmaita> smcginnis: ordinarily, i would agree with you, but when half the drivers are noncompliant, i think we need to look on our side too
16:53:27 <jungleboyj> I don't want Cinder to be cut in half and have the foundation come back and say WTF?
16:53:35 <jungleboyj> rosmaita: ++
16:53:52 <jungleboyj> I mean, this makes me stick to my stomach.
16:54:01 <rosmaita> sick, even
16:54:15 <jungleboyj> Yeah, that too.  Can't type while wretching
16:54:23 <rosmaita> :P
16:54:32 <jungleboyj> This speaks to the health of the community.
16:54:49 <jungleboyj> Anyway, I am going to reach out to the foundation and see what happens.
16:54:50 <smcginnis> These vendors are not participating and making sure OpenStack is healthy. I'm just fine seeing them go away (and lose markey opportunities) if they don't care about the project.
16:55:04 <smcginnis> *marketing
16:55:29 <jungleboyj> smcginnis:  I know.
16:55:51 <jungleboyj> Anyway, I think we have a general direction here.  Can talk more at the mid-cycle.
16:55:59 <rosmaita> ++
16:56:05 <jungleboyj> There is one more topic on the schedule.
16:56:23 <jungleboyj> #topic encryption effort update
16:56:26 <jungleboyj> rosmaita:
16:56:28 <rosmaita> proposed changes to os-brick will be along the lines of what was proposed as an oslo library
16:56:34 <rosmaita> #link https://review.opendev.org/#/c/618754/
16:56:41 <rosmaita> with the Train os-brick release a month away, i'm pessimistic that they'll make train
16:56:46 <rosmaita> i was thinking it might be a good idea to request a feature branch for this work?
16:56:59 <rosmaita> or is that too much trouble?
16:57:31 <rosmaita> that's all i have
16:57:36 <jungleboyj> Not sure how hard that is to do.
16:57:38 <eharney> i think we usually get pretty far with a well-organized patch series
16:57:44 <smcginnis> eharney: ++
16:57:49 <jungleboyj> eharney: ++
16:57:50 <eharney> not sure a feature branch would provide much benefit
16:58:00 <smcginnis> From what I've heard, feature branches end up being mroe trouble than they're worth.
16:58:16 <rosmaita> ok, i will not suggest it, then
16:58:19 <smcginnis> Their easy to make, just need to add it to the deliverable file in the releases repo.
16:58:24 <smcginnis> *they're
16:58:26 <smcginnis> Bleh
16:59:01 <jungleboyj> Ok.  But if they are too much trouble lets not go that direction.
16:59:17 <jungleboyj> Ok.  So, I think that is it for this meeting.
16:59:45 <jungleboyj> Thanks for the discussion everyone.
16:59:55 <jungleboyj> #endmeeting