16:00:24 <smcginnis> #startmeeting Cinder
16:00:29 <openstack> Meeting started Wed Jul 13 16:00:24 2016 UTC and is due to finish in 60 minutes.  The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:30 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:34 <openstack> The meeting name has been set to 'cinder'
16:00:41 <smcginnis> Hey everyone.
16:00:45 <xyang> hi
16:00:46 <jseiler> hi
16:00:47 <erlon> hey
16:00:48 <ntpttr> hi everyone
16:00:49 <dulek> o/
16:00:50 <eharney> hi
16:00:53 <adrianofr> hi
16:00:57 <jgregor> Hey
16:01:06 <jungleboyj> o/
16:01:07 <jacob-infinidat> hi
16:01:08 <baumann> Hi
16:01:10 <watanabe_isao> o/
16:01:10 <_alastor_> o/
16:01:16 <geguileo> Hi
16:01:21 <guyr-at-infinida> Hi
16:01:30 <smcginnis> ping dulek duncant eharney geguileo winston-d e0ne jungleboyj jgriffith thingee smcginnis hemna xyang tbarron scottda erlon rhedlind jbernard _alastor_ vincent_hou kmartin patrickeast sheel dongwenjuan JaniceLee cFouts Thelo vivekd adrianofr mtanino yuriy_n17 karlamrhein diablo_rojo jay.xu jgregor baumann rajinir wilson-l reduxio wanghao thrawn01 chris_morrell watanabe.isao
16:01:38 <smcginnis> #topic Announcements
16:01:41 <DuncanT> Hey
16:01:43 <winston-d> o/
16:01:44 <cFouts> hi
16:01:45 <patrickeast> o/
16:01:48 <smcginnis> We are at the N-2 milestone.
16:01:48 <scottda> hi
16:02:01 <e0ne> hi
16:02:05 <smcginnis> I will be submitting the request to tag the milestone later today.
16:02:08 <bswartz1> Yesterday actually
16:02:19 <e0ne> #link https://wiki.openstack.org/wiki/CinderMeetings#Next_meeting
16:02:32 <mtanino> o/
16:02:42 <smcginnis> bswartz1: Well, technically 12-14. :)
16:02:53 <e0ne> smcginnis: I'm very disappointed about os-brick release blocker:(
16:02:59 <bswartz1> Yeah sometime this week
16:03:06 <dulek> Do we have anything that needs to get in before N-2 gets tagged?
16:03:10 <smcginnis> e0ne: I agree.
16:03:17 <bswartz1> Don't worry we're late too
16:03:42 <smcginnis> dulek: For the most part it's just a mark in the sand, so I don't think there's anything critical we need to make sure gets in before we mark the release.
16:03:46 <e0ne> smcginnis: we're blocked for more than one month:(
16:04:02 <smcginnis> #link https://etherpad.openstack.org/p/cinder-spec-review-tracking Review focus
16:04:12 <smcginnis> SO we have a lot to get done yet for the release.
16:04:18 <tbarron> hi
16:04:19 <bswartz1> What about new drivers? There's a deadline for those to merge right?
16:04:35 <e0ne> bswartz1: deadline is mitaka-2 milestone
16:04:41 <smcginnis> A lot of focus lately has been on the new drivers, but we really need to start focusing on the release priorities we identified in Austin.
16:04:43 <e0ne> bswartz1: oops, newton-2
16:04:51 <hemna> mep
16:04:56 <smcginnis> bswartz1: Last night was the deadline at 00:00 PDT.
16:05:35 <smcginnis> #link https://etherpad.openstack.org/p/newton-cinder-midcycle Midcycle planning
16:05:44 <smcginnis> Next week is the midcycle.
16:05:51 <hemna> did we not have some drivers make it ?
16:05:55 <DuncanT> I'll be attending remotely only :-(
16:06:05 <timwilliams> Sean we submitted something for our driver just after the time last night any chance that can make it in?
16:06:13 <smcginnis> hemna: Yeah, a few missed the cut off.
16:06:14 <hemna> any reason to keep trying to land any ?
16:07:04 <smcginnis> timwilliams: Which driver? It's a hard deadline, so likely not, but there is one that was just hung up on a slow CI that we should discuss later if we should let that one through. It had positive votes and was otherwise ready.
16:07:29 <smcginnis> hemna: No, I think otherwise we need to stick to our deadlines and move on to the more important stuff.
16:07:38 <hemna> timwilliams, review url ?
16:08:00 <hemna> timwilliams, do you have CI in place, reporting and passing ?
16:08:06 <timwilliams> The falconstor driver you had made a review comment last night our developer submitted fixes just after your review
16:08:17 <timwilliams> Sorry for being late
16:08:19 <hemna> otherwise I'd say you don't have much of a chance of an extension
16:08:33 <timwilliams> Yes it past the publich CI this morning
16:08:55 <hemna> ok, well it's really up to the PTL to decide on extensions or not.
16:09:24 <timwilliams> Okay I understand
16:09:25 <jungleboyj> hemna: ++
16:10:18 <smcginnis> OK, I'd like input from the community. My take is if code is being pushed up after the deadline, the deadline is missed.
16:10:24 <smcginnis> That said, these were really close.
16:10:30 <smcginnis> And otherwise looking pretty good.
16:10:38 <smcginnis> We had three drivers in question:
16:10:47 <smcginnis> Falconstor had updates needed.
16:11:03 <smcginnis> Violin had a slow CI, but was otherwise in place in time.
16:11:15 <jgriffith> smcginnis: IMHO if they had things posted prior to deadline and made fixes based on comments, AND have a CI system running then I think they should be allowed to finish up
16:11:19 <smcginnis> And Infinidat we need to discuss in a bit because it is a bigger topic.
16:11:32 <jungleboyj> smcginnis: I would say, as with other releases.  If the code is pushed up and there is just a slow CI or something and you are ok with it getting merged, that is fine.
16:11:40 <jungleboyj> jgriffith: Yeah, that was what I was trying to say.
16:11:40 <jgriffith> smcginnis: as far as Violins CI, we all know the trials and tribulations of running CI's :(
16:11:47 <smcginnis> jgriffith: OK, I'm good with Violin.
16:11:50 <smcginnis> jgriffith: True!
16:11:59 <smcginnis> Falconstor I have not looked at yet.
16:12:08 <jungleboyj> smcginnis: Cool.  I +2'd this morning.  Will merge when CI reports.
16:12:09 <hemna> jgriffith, +1
16:12:12 <jacob-infinidat> re:Infinidat driver - will be happy to discuss. (thats us)
16:12:48 <smcginnis> Let's move on in the agenda and get to that.
16:12:55 <smcginnis> #topic 3rd party CI
16:13:02 <smcginnis> watanabe_isao: Hi
16:13:10 <watanabe_isao> smcginnis, hi
16:13:27 <watanabe_isao> smcginnis, am I up?
16:13:29 <smcginnis> #link https://github.com/openstack-infra/system-config/commit/4fb97f7ed44272fdde51cd373dd465314ed913ed Third party recheck recommendation
16:13:53 <smcginnis> watanabe_isao: Yep. If you can give an overview, otherwise I think I got it and can also discuss.
16:14:06 <patrickeast> ah splendid, another recheck command for ci's
16:14:08 <watanabe_isao> smcginnis, sure.
16:14:15 <jgriffith> patrickeast: hehe
16:14:43 <watanabe_isao> OK, it is more like a fake alarm now.
16:15:25 <watanabe_isao> Infra asked all CIs support recheck and system: recheck. However the fix has been reverted.
16:15:40 <smcginnis> watanabe_isao: So the recommendation is to standardize third party CI recheck triggers on the format "[name]: recheck"
16:15:42 <erlon> watanabe_isao: I think most of CIS on Cinder are using run XXXCI, isn't better to stick to this intead of having everyone to change?
16:15:48 <watanabe_isao> Now it is just recheck only, others is fine.
16:15:50 <smcginnis> watanabe_isao: Oh, it was reverted? How come?
16:16:13 <smcginnis> I really wish we could standardize on a recheck trigger just for CIs so we don't waste gate resources.
16:16:32 <hemna> smcginnis, +1
16:16:38 <smcginnis> Well, we kind of did. We did official state for Cinder at least all CIs should recheck on "run-[ci name]"
16:16:39 <e0ne> smcginnis: +1
16:16:42 <jungleboyj> smcginnis: +1
16:16:51 <watanabe_isao> smcginnis, no the recommendation now is just recheck. Others like for us, we decided "run XXX CI" is fine.
16:17:11 <smcginnis> watanabe_isao: I guess we just stick to our own guidance then? Nothing official from infra.
16:17:13 <jungleboyj> watanabe_isao: So, we do not need to change what our drivers are doing.
16:17:43 <xyang> watanabe_isao: recheck will trigger everything to run, that’s what we are trying to avoid
16:17:53 <watanabe_isao> What I want to say is, we need to ask CIs to support not only "run XX CI", but also recheck.
16:18:12 <smcginnis> "run-XX"
16:18:26 <watanabe_isao> smcginnis, yes "run-XX"
16:18:46 <smcginnis> #link https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#How_do_I_trigger_my_CI_to_rerun_on_gerrit_comments.3F Official Cinder CI recheck recommendation.
16:19:01 <erlon> watanabe_isao: you mean CIs should also respond to recheck not only run-XX??
16:19:13 <watanabe_isao> erlon, yes.
16:19:13 <patrickeast> erlon: yea
16:19:26 <erlon> watanabe_isao: ok, got it
16:19:46 <smcginnis> The problem there for me is my CI triggers on a Jenkins +1, so if I trigger on a "recheck" I will run, then run again once Jenkins passes.
16:19:49 <_alastor_> Regardless of what the format is, as long as it's documented accurately on the wiki I'm happy :)
16:19:52 <smcginnis> So that's a no from me.
16:20:01 <xyang> watanabe_isao: you were just saying system: recheck is reverted?
16:20:09 <bswartz> IMO respond to recheck after jenkins votes +1 ¬_¬
16:20:18 <smcginnis> bswartz: +1
16:20:34 <e0ne> bswartz: good point, I agree
16:20:46 <bswartz> presumably the user is rechecking because jenkins voted -1, and if you CI chose not to ran after than then wait again
16:20:49 <watanabe_isao> xyang, yes "system: recheck" is reverted. That also means you can have "system: recheck" but not forced by infra.
16:21:21 <smcginnis> OK, so kick off either on recheck if you don't trigger on Jenkins +1, or wait for Jenkins +1 is fine (recommended). And support "run-CIName" for individual CI runs.
16:21:31 <smcginnis> watanabe_isao: Anything else then?
16:21:55 <watanabe_isao> smcginnis, one more
16:22:23 <hemna> smcginnis, what is the official word on how long a 3rd party CI can fail before drivers are removed ?
16:22:23 <watanabe_isao> smcginnis, how about we ask CI mentioners to update this in their wiki?
16:22:56 <smcginnis> watanabe_isao: I did ask that a couple times in the past. Some have updated the wiki and some have not.
16:23:01 <_alastor_> I already sent out an email this week to all the CI maintainers
16:23:01 <smcginnis> But we should probably keep asking.
16:23:06 <smcginnis> _alastor_: Thanks!
16:23:15 <bswartz> hemna: 5 years?
16:23:16 <smcginnis> hemna: Not sure. One week?
16:23:20 <smcginnis> bswartz: Hah!
16:23:23 <hemna> lolz
16:23:53 <bswartz> 1 week isn't even long enough to cover someone going on vacation
16:23:58 <smcginnis> I probably will have to put up some removal patches soon. We have a few drivers that have not reported in quite a while.
16:23:59 <erlon> smcginnis: there are acually a lot of CIs that are showing up
16:24:07 <hemna> 1 week is too short IMHO
16:24:07 <smcginnis> bswartz: 2 weeks?
16:24:27 <jungleboyj> hemna: Yeah, way too short for removal.
16:24:40 <erlon> I counted 80 drivers, +-30 CIs in this patch: https://review.openstack.org/#/c/336092/
16:24:41 <smcginnis> A subjective amount of time between 1 week and "what the heck happened to those guys?"
16:24:47 <bswartz> there needs to be an escalation path and removal should only be considered at last resort
16:24:53 <patrickeast> imo something like 2 weeks without any response becomes a problem... if they are working on it, have the wiki updated showing its under maintenance or whatever thats ok
16:24:58 <_alastor_> Just so folks know, I built a simple ci-status tool that you can use to quickly check if a ci is reporting without relying on a third-party.  https://github.com/openstack/third-party-ci-tools/tree/master/monitoring/ci-status
16:25:04 <smcginnis> patrickeast: I agree with that.
16:25:07 <hemna> I'd just like to get it documented, so that everyone knows it's there.
16:25:15 <jungleboyj> bswartz: Agreed.
16:25:17 <hemna> I have a feeling there are some that need to be removed.
16:25:20 * DuncanT emails people occasionally about broken CIs, but not to any schedule
16:25:20 <hemna> ...
16:25:26 <smcginnis> hemna: Right, we should be very clear on our policy.
16:25:33 <hemna> smcginnis, +1
16:25:47 <smcginnis> But I do think we have been very clear in all our documentation that a consistently reporting CI is an absolute requirement.
16:25:53 <eharney> i think we also need to consider where we are in the release cycle for removal -- doing it near the end of the release is not good when we're past feature freeze etc
16:25:58 <jgriffith> _alastor_: nice
16:26:08 <smcginnis> eharney: Right
16:26:17 <jungleboyj> I think people should be notified after a week or so and then given up to a month depending on responsiveness and what the issue is.
16:26:30 <hemna> I think that's too long man
16:26:41 <hemna> if you can't get your CI working in 4 weeks?
16:26:45 <hemna> you aren't doing it right.
16:27:07 <hemna> notified after 1 week, 2 weeks to fix after that.
16:27:11 <smcginnis> Let's discuss this more at the midcycle.
16:27:12 <eharney> however, pulling a driver in less than a month is pretty extreme
16:27:12 <hemna> even that seems long to me.
16:27:13 <jungleboyj> hemna: Fair enough
16:27:21 <jungleboyj> eharney: That is my concern.
16:27:27 <e0ne> hemna: sounds good
16:27:27 <eharney> mine too
16:27:37 <hemna> the entire point of having CI up is that it reports and is functional
16:27:42 <smcginnis> watanabe_isao: I'm going to move on if that's OK.
16:27:49 <watanabe_isao> smcginnis, yes sir
16:27:50 <hemna> if it's broken, then there is no value in 3rd party CI at all.
16:27:51 <jungleboyj> I think 3 weeks is good.  No less than 2 though.
16:28:05 <smcginnis> Can someone make sure CIs are on the midcycle agenda?
16:28:06 <bswartz> I agree a month is plenty long to fix issues, but removal for first violation is a bit extreme -- maybe censure the vendor and only remove repeat offenders
16:28:08 <e0ne> hemna: +2
16:28:12 <eharney> but, given the failure rates we see in general on third party CI this far into the efforts  ...
16:28:14 <smcginnis> #topic Guidance for drivers that use external libs
16:28:25 <smcginnis> DuncanT: I have some opinions here as well, but please take it away.
16:28:43 <jungleboyj> Yeah, good topic for mid-cycle.
16:28:53 <DuncanT> I bought this one up, since I was very unhappy with one driver,  then they (quite reasonably) pointed out that the IBM driver is the same
16:29:25 <jungleboyj> DuncanT: :-)  Here we go again.
16:29:30 <hemna> (added it to the midcycle etherpad)
16:29:36 <e0ne> DuncanT: I'm agree with you. it's not opensourced driver
16:29:44 <DuncanT> I consider a cinder driver that contains nothing but one line calls into an external library to be effectively closed source
16:29:47 <smcginnis> hemna: Thanks
16:29:48 <_alastor_> Here's an example of a query to see which CIs have reported in the last month: http://pastebin.com/raw/nhHaiP3k
16:29:52 <hemna> DuncanT, +1
16:29:54 <bswartz> what's the license on the library?
16:30:00 <eharney> DuncanT: even if the lib is apache-licensed?
16:30:01 <smcginnis> DuncanT: +2
16:30:10 <bswartz> if the library is also apache then I don't agree
16:30:25 <DuncanT> License doesn't matter to me, I don't want to have to pull it to figure out basics
16:30:35 <smcginnis> Even with a public library, I feel if all that is in Cinder just wraps calls to this library, there is no need to have it in tree.
16:30:43 <jgriffith> _alastor_: the 30+ days entry indicates nothing in last 30 days right?
16:30:49 <smcginnis> That's just a marketing ploy, and we have more important things to use up our time.
16:30:51 <_alastor_> jgriffith: Yes
16:30:53 <e0ne> smcginnis: +2
16:30:58 <DuncanT> I think a good guideline is generic functions in external libs is ok, but cinder specific stuff should all be in-tree
16:31:01 <jgriffith> _alastor_: smcginnis DuncanT nuke them
16:31:26 <e0ne> DuncanT: we have to document it in our guidelines
16:31:26 <hemna> smcginnis, +1
16:31:27 <_alastor_> jgriffith: Well, I would probably verify that I'm not lying about it first :P
16:31:32 <guyr-at-infinida> smcginnis: there's one caveat; if a driver wants to be on the HCL, he needs to be included in the tree, isn't that right?
16:31:34 <DuncanT> i.e. an external lib should never import cinder.*
16:31:38 <smcginnis> There is the OpenStack marketplace for vendors that want their offering known.
16:31:44 <smcginnis> That is for marketing purposes.
16:31:51 <smcginnis> There is no HCL.
16:31:53 <bswartz> DuncanT: that's a very hard judgement to make -- who decides if what the driver does is generic or cinder specific?
16:31:53 <jgriffith> _alastor_: nah... you're a sharp guy, I trust ya.  And I'm not on the list so all is good :)
16:31:59 <hemna> guyr-at-infinida, then why have your entire driver code outside cinder to begin with.
16:32:02 <guyr-at-infinida> https://wiki.openstack.org/wiki/CinderSupportMatrix
16:32:04 <jungleboyj> DuncanT: I agree.  The driver in question is looking to resolve the issue.
16:32:22 <jungleboyj> The question of library licensing, however, could be complicated.
16:32:25 <smcginnis> We are not VMware. We do not publish and HCL and we are not running a certification program.
16:32:28 <DuncanT> bswartz: Start with if you need to import cinder.* then you're being cinder specific
16:32:37 <jgriffith> smcginnis: +1
16:32:49 <jungleboyj> What is the license on the HP and NetApp libraries that are pulled in?
16:33:02 <jgriffith> so not to be annoying (but I am)... we tend to rat hole on this topic an awful lot
16:33:08 <jungleboyj> There are a number of vendors that leverage external libraries.
16:33:10 <DuncanT> licensing for me is simple - it needs to be freely distributable and repackagable - I don't actually care about anything else, though I understand others might
16:33:12 <hemna> my client libraries are apache
16:33:21 <hemna> same as openstack
16:33:27 <jungleboyj> hemna: Ok, that is good.
16:33:29 <hemna> and the source is all available on github
16:33:40 <hemna> AND, my drivers aren't shims to the library.
16:33:41 <jgriffith> hemna: and available on github and Pypi so effectively open  IMO
16:33:47 <smcginnis> jungleboyj: Leveraging external libraries - OK. Having your entire driver implementation external to Cinder is the point in question.
16:33:50 <jungleboyj> hemna: Right, that is where we have an issue.
16:33:53 <hemna> the libraries are purely for REST communication.
16:33:57 <jungleboyj> smcginnis: ++
16:34:04 <hemna> all of the logic if the drivers is in our drivers.
16:34:07 <hemna> of
16:34:07 <DuncanT> hemna: I haven't had a problem with the HP libraries, I can read the driver code and get a good idea how it works, which is what I /actually/ care about
16:34:22 <smcginnis> DuncanT: +1
16:34:37 <dulek> In case someone has just calls to other lib - why not simply support it as pip installable driver? Then you can enable it with volume_driver = vendor.sth.CinderDriver, right?
16:34:37 <jungleboyj> So, personal opinion is having the shim isn't right.  I am working to resolve that to get it to the point where you can see all the Cinder stuff going on.
16:34:49 <DuncanT> dulek: Yup
16:34:56 <jungleboyj> Not sure that I would be able to win the battle to release the underlying library.  I am sure others would have the same problem.
16:35:10 <bswartz> DuncanT: imagine if some vendor implemented a REST API that exactly matched the cinder driver interface and every driver call was a 1 line rest call to their proprietary implementaion. Would that bother you?
16:35:13 <hemna> dulek, I wish all Cinder drivers were just pip packages.....
16:35:22 <smcginnis> jgriffith: I may starting to be swayed to your side on the remove drivers from tree proposal. ;)
16:35:24 <jgriffith> Ya know ya'll can use the same lib method but have it in tree
16:35:30 <dulek> And having all the code in external lib prevent Cinder team from fixing anything inside them…
16:35:31 <jgriffith> smcginnis: haha!
16:35:42 <hemna> smcginnis, w00t!
16:35:48 <patrickeast> smcginnis: jgriffith: do we have that as a topic at the mid-cycle :D
16:36:02 <jungleboyj> patrickeast: It is going to be.
16:36:04 <smcginnis> Maybe we should.
16:36:05 <hemna> patrickeast, we do now :)
16:36:10 <dulek> :)
16:36:12 <DuncanT> bswartz: I'm not sure TBH.
16:36:22 <patrickeast> maybe check in and see how its going for the neutron folks
16:36:53 <bswartz> DuncanT: I'm afraid there are too many gray areas and you can't draw a clear line around what's allows vs not allowed
16:36:55 * DuncanT is still against out-of-tree drivers, but we can debate that again at the appropriate time
16:37:13 <jungleboyj> DuncanT: Yeah ... Agreed.
16:37:19 <hemna> we can argue that again in Ft. Collins.
16:37:30 <hemna> it's worth talking about I thinks....even if we do it again.
16:37:31 <jungleboyj> I think the closed drivers are the more immediate concern.
16:37:35 <hemna> at least it's not taskflow.
16:37:38 <DuncanT> bswartz: We can and should give guidance though, and the two cases we've got (IBM's and the one proposed) are IMO clearly wrong
16:37:40 <jungleboyj> hemna: ++
16:37:53 <smcginnis> DuncanT: I think it would be bad for Cinder quality, but if we're moving to the point of effectively being out of tree while putting in breadcrumbs so users can see the name in the Cinder source, what's the point?
16:37:53 <jungleboyj> DuncanT: IBMer agrees.  :-)
16:38:29 <jungleboyj> smcginnis: So, XIV and DS8k are working to pull the code into tree.  No more shim.
16:38:33 <jacob-infinidat> one of the rationales for us being able to keep some logic out-of-tree is to enable adoption of new functionality in the storage array that becomes available after/before next release of Cinder.
16:38:37 <bswartz> yes and human judgement should matter too, but the decision is likely to be controversial no matter what
16:38:38 <smcginnis> jungleboyj: Excellent
16:38:51 <patrickeast> i tend to lean towards the side of its OK, as long as the lib being used is the right license... but i don't package and distribute openstack, so maybe my motivations aren't lining up with others
16:39:02 <guyr-at-infinida> If there's a proper test suite and CI reporting - moving the drivers out of the tree won't damage the quality
16:39:03 <patrickeast> or support other vendors drivers **
16:39:04 <jungleboyj> smcginnis: We are working on figuring out how we can do it.  I need to work with you guys to understand the expectations as far as libraries being used.
16:39:04 <e0ne> infinidat library: License: PSF
16:39:25 <hemna> jacob-infinidat, and also to fix bugs in drivers outside the release cycle of openstack....re fixing Kilo bugs in drivers that can never land upstream due to upstream policy
16:39:31 <smcginnis> jacob-infinidat: "some logic out-of-tree" < that's an understatement though, right? Everything is out of tree other than an empty wrapper.
16:39:32 <_alastor_> jacob-infinidat: We maintain a public github repository with bleeding edge changes that haven't made it into the Cinder tree yet
16:39:42 <DuncanT> patrickeast: I'm less worried about packaging (except cases like Netapp where we weren't allowed to distribute) but for just understanding the code to figure out if a change will work or not
16:39:46 <_alastor_> jacob-infinidat: Specifically for that reason
16:39:59 <hemna> most of our customers aren't running in life support openstack releases.
16:39:59 <eharney> fixing bugs in stable branches is a mess if all of the code is in an external lib, too
16:40:01 <hemna> :(
16:40:07 <patrickeast> DuncanT: yea, so i mean... its a trade off the vendors/driver maintainers have to make
16:40:21 <smcginnis> DuncanT: Good point.
16:40:31 <patrickeast> DuncanT: if they want full control to make changes whenever to their *real* driver, they can... but it means others might break it on accident
16:40:38 <jungleboyj> My 0.02 is that the Cinder logic should all be in-tree.  If there is a library required to actually interface with the backend, we shouldn't really care.
16:40:43 <smcginnis> If we get stuck being able to not move forward on something - that's a huge issue.
16:40:53 <DuncanT> jungleboyj: +1
16:40:55 <smcginnis> It's now on us to make sure we don't break everything external.
16:40:56 <jungleboyj> The difference is whether that code is running locally or out on the backend in question.  :-)
16:41:18 <hemna> the biggest issue for me is getting bug fixes in drivers that can't land upstream, due to being out of support lifecycle.
16:41:24 <hemna> pip packages solves that.
16:41:41 <patrickeast> hemna: definitely
16:41:48 <jacob-infinidat> hemna: +1
16:41:52 <e0ne> hemna: +1
16:41:55 <hemna> I face this very frequently
16:41:59 <patrickeast> trying to get changes into downstream distros, even like a month or two after a release is painful
16:42:11 <hemna> yup
16:42:29 <bswartz> IMO any code running inside the python interpreter should be apache licensed -- so anything the driver imports -- beyond that it gets really hard to draw a clear boundary
16:42:39 <hemna> the upstream policy on lifecycle support works for the core of the project, just not for 3rd party drivers.
16:43:08 <DuncanT> hemna: Core bugs are no less frequent than driver bugs in my experience.... it sounds like the openstack support period is broken and we're trying to hack around it rather than fix the actual issue?
16:43:11 <jacob-infinidat> IMHO the focus should be on strengthening the CI and making it as strict as possible, rather than relying on code reviews.
16:43:18 <smcginnis> So how about this - we maintain a docs page that vendors can propose patches to add their name and installation points for their out of tree drivers.
16:43:23 <eharney> jacob-infinidat: that will not work
16:43:35 <smcginnis> In order for us to approve those doc patches we need to see their CI reporting on patches.
16:43:43 <jacob-infinidat> eharney: please elaborate
16:43:45 <patrickeast> jacob-infinidat: thats been the plan for years, still a WIP
16:43:52 <DuncanT> jacob-infinidat: You can' just rely on CI, since there are a /lot/ of bad ways of doing things that break in subtle ways that don't show up on a small CI
16:43:53 <smcginnis> Then they are "official" but we don't need to care.
16:44:09 <eharney> jacob-infinidat: CI is best for ensuring common behaviors and preventing regressions -- it will never find many of the bugs that can be found by careful review
16:44:19 <smcginnis> eharney: +1
16:44:24 <bswartz> DuncanT: the *upstream* support period has always been broken, by design, so that downstream distros have to carry the burden of backporting fixes to releases that customers actually run
16:44:26 <DuncanT> eharney: ++
16:44:33 <eharney> this is a fallacy that i think a lot of us have been on the wrong side of here
16:44:39 <e0ne> smcginnis: it will works onluy with good working  3rd party CI, IMO
16:45:13 <DuncanT> I don't think our CI is even close to being good enough to move to that model yet
16:45:24 <smcginnis> e0ne: Right. And same policy - if CI stops reporting we can remove the driver from Cinder (remove its info from docs)
16:45:29 <jungleboyj> smcginnis: Interesting work around.
16:45:33 <e0ne> DuncanT, unfortunately, you're right
16:45:42 <smcginnis> It's a compromise for sure.
16:45:45 <hemna> and there are lots of skips in 3rd party CI as well
16:45:57 <DuncanT> We'd kill any forward progress on core changes (or just trash code quality even more)
16:46:00 <hemna> CI has always been unstable
16:46:01 <jacob-infinidat> judging by the list of growing cinder drivers, I wonder how scalable the on-going code-review process is.
16:46:05 <hemna> infra has always been unstable
16:46:07 <hemna> horribly so
16:46:10 <e0ne> there is one more problem with CIs
16:46:22 <hemna> they don't tag infra projects
16:46:28 <hemna> and don't stabilize them
16:46:33 <hemna> it's always a moving target.   it's a nightmare
16:46:39 <e0ne> we don't verify that it runs all cinder-related tests
16:47:12 <jungleboyj> smcginnis: It doesn't really address the question of licensing with drivers that need their own library to interact with the backend.
16:47:36 <hemna> jungleboyj, I think the licensing issue is completely separate
16:47:37 <smcginnis> jungleboyj: I actually don't care then. If they are not trying to be part of the Cinder source, so be it.
16:47:38 <jungleboyj> Do we really need to pull out drivers because of library licensing?>
16:47:54 <hemna> but if it's a shim to a non opensource library ?
16:47:54 <DuncanT> e0ne: Log scraping the CIs currently is an epic project... there are at least 7 different formats last time I tried
16:48:03 <hemna> I think that's bad mmmkay
16:48:08 <jungleboyj> hemna: Yeah, I guess that is my main concerns.  We can talk more about that at the mid-cycle though.
16:48:18 <jungleboyj> hemna: Yeah, that is bad.  No argument here.
16:48:20 <DuncanT> smcginnis: 3rd party packagers have to care about licensing
16:48:28 <e0ne> jungleboyj: we need an expert in licencing
16:48:31 <smcginnis> DuncanT: It wouldn't be packaged though.
16:48:37 <jungleboyj> e0ne: *Gag*
16:48:47 <jacob-infinidat> hemna: and what if the library is opensource - i.e. on github with Apache license?
16:48:50 <DuncanT> smcginnis: Not by you, but RDO or Helion or similar need to package it
16:49:11 <smcginnis> DuncanT: They wouldn't be able to either. But that would be a decision for the given vendor to make it they care.
16:49:11 <bswartz> I'm actually interested in solving the log-scraping problem for Manila too -- I think the best option is to mandate a common format
16:49:16 <patrickeast> from my experience they don't package them...
16:49:26 <DuncanT> smcginnis: I think that's a rode to madness
16:49:32 <patrickeast> its up to the vendor to go add code to tripleo, for fuel or whatever to install them
16:49:42 <smcginnis> I don't think we are going to get a resolution here today.
16:49:48 <hemna> jacob-infinidat, like....https://github.com/hpe-storage   then I think it's ok.  but the driver can't be a shim to the library IMHO.
16:49:52 <e0ne> jacob-infinidat: TBH, your driver could fit in 3 lines. it's not an open driver, IMO
16:50:00 <hemna> e0ne, +1
16:50:06 <smcginnis> Let's devote an afternoon in Ft Collins to this.
16:50:07 <jgriffith> patrickeast: indeed... for better or worse :)
16:50:29 <e0ne> class InfiniboxVolumeDriver(InfiniboxVolumeDriver): pass
16:50:30 <e0ne> done
16:50:39 <_alastor_> smcginnis: +1
16:50:41 <DuncanT> jacob-infinidat: My big problem with your driver is I can't read it and get an idea how it works. I don't even know the transport, the features, nothing
16:50:47 <dulek> e0ne: You're right. :D
16:50:54 <patrickeast> smcginnis: +1
16:50:56 <e0ne> class InfiniboxVolumeDriver(infinidat_openstack.cinder.volume.InfiniboxVolumeDriver): pass
16:51:00 <DuncanT> jacob-infinidat: That's not a good thing to have in-tree IMO
16:51:02 <jungleboyj> smcginnis: +1  I need to get more info on my side and would like to talk more.
16:51:12 <hemna> DuncanT, +1
16:51:15 <smcginnis> jungleboyj: Definitely
16:51:19 <jungleboyj> smcginnis: I got agreement to move away from the shim though.
16:51:32 <smcginnis> jungleboyj: Certainly
16:51:33 <jacob-infinidat> DuncanT: to be fair, it is here: https://github.com/infinidat/cinder/
16:51:42 <_alastor_> DuncanT: +1
16:51:42 <jungleboyj> hemna: and DuncanT can celebrate.
16:52:07 <guyr-infindiat> https://github.com/infinidat/infinidat_openstack/
16:52:12 <hemna> smh
16:52:20 <DuncanT> jacob-infinidat: And if I have to check out 80 git repos to grep for a function usage or similar (something I do frequently during reviews) I'm stuffed
16:52:24 <hemna> I really don't like seeing forks of cinder on github :(
16:52:25 * hemna has a sad
16:52:39 <e0ne> I would like -2 on it only because it doesn't follow openstack code guidelines
16:52:40 <jungleboyj> :-(
16:52:49 <guyr-infindiat> openstack users treat the CinderSupportMatrix as the HCL
16:52:59 <eharney> just to mirror what DuncanT was saying, if you want your driver to work in distros like RDO, having it (actually, fully) in-tree is a huge advantage
16:53:06 <guyr-infindiat> it has been listed so in some presentations in the last conference
16:53:24 * smcginnis updates slide deck to remove that slide
16:53:34 <DuncanT> jacob-infinidat: It also doesn't follwo code conventions, isn't reviewed by the community and is generally a huge additional pain to core reviews
16:53:48 <jgriffith> hemna: I love my forks
16:53:49 <guyr-infindiat> but that's the thing -- its not for core reviewers
16:53:55 <e0ne> guyr-infindiat: please, don't mix marketing and technical things
16:53:59 <hemna> if folks are forced to fork cinder in github just to maintain their driver, then Cinder has failed IMO
16:54:01 <smcginnis> It has been listed as what drivers are in tree. There are many vendors that supply a driver directly to their customers that are not in tree.
16:54:12 <guyr-infindiat> we implement the API you define and we're giving support for it
16:54:19 <DuncanT> guyr-infindiat: It /has/ to be for core reviewers when we're trying to change core things (active/active H/A for example)
16:54:28 <guyr-infindiat> we're unburdening you from maintaining it
16:54:38 <DuncanT> guyr-infindiat: I have spend literally /hours/ going through drivers for H/A work
16:54:48 <eharney> that's not unburdening... it ends up being a huge burden when it breaks and i want to figure out why
16:54:49 <jgriffith> guyr-infindiat: just FYI, I know you probably haven't heard this before BUT OpenStack and Cinder is not just an API
16:54:51 <guyr-infindiat> but at least put my name of the drivers list, because that's what my customers are looking for
16:54:58 <e0ne> hemna: it depends on why did they fork cinder
16:54:59 <jgriffith> guyr-infindiat: that's been an interesting conversation point for years
16:55:13 <hemna> e0ne, as I said, to support their driver...
16:55:29 <jgriffith> hemna: backport of features for customers
16:55:37 <smcginnis> OK, for the sake of moving this forward - do we just accept the Infinidat driver for now and in Ft Collins come up with an official policy that may result in it's removal (and others that are not compliant) in Ocata?
16:55:38 <hemna> this is one of the arguments for making drivers pip packages
16:55:38 <jgriffith> anyway... what's the topic currently?
16:55:48 <e0ne> hemna: IMO, it's good enough before it will be merged to upstream
16:55:48 <hemna> the forks of cinder are making my argument for me.
16:55:52 <DuncanT> guyr-infindiat: There's a difference between wrapping generic calls that are unique to your backend (which is a good thing, often) and hiding conder related logic out-of-tree
16:55:55 <bswartz> out of tree drivers are fine but they shouldn't get recognition from this community at all -- we should be able to act as if they don't exist
16:55:59 <jacob-infinidat> DuncanT: for the active/active H/A work - couldnt it have been covered through CI ?
16:56:03 <eharney> smcginnis: i'd rather we pick a policy and then merge it later rather than push it in and back out
16:56:16 <smcginnis> eharney: Problem being our deadline was yesterday.
16:56:17 <eharney> smcginnis: i.e. exception for later merge in newton if the criteria are met etc
16:56:26 <hemna> jgriffith, backporting features and fixing bugs to old releases, could also get solved w/ pip packages of drivers.
16:56:29 <hemna> bleh
16:56:33 <hemna> horse....dead.
16:56:35 <DuncanT> jacob-infinidat: CI is not H/A at all, and you need to do thousands of runs to have a hope of catching some of the races I'm looking at, so no, not at all
16:56:47 * jungleboyj pleads the 5th
16:56:50 <jgriffith> hemna: haha... sorry, my example is actually changes to cinder's core code base
16:56:52 <smcginnis> Are folks OK with that. We hold on this until after the midcycle, then allow it in if it's determined to be acceptable?
16:56:58 <jgriffith> anyway... it's irrelevant and doesn't matter
16:57:01 <sdake> yo
16:57:02 <sdake> sorry wrong channel
16:57:07 <smcginnis> sdake: ;)
16:57:12 <jungleboyj> smcginnis: I think that is fair.
16:57:21 <patrickeast> smcginnis: yea that sounds like a good plan
16:57:29 <DuncanT> smcginnis: I'd rather hold off and merge late if we come to a decision
16:57:33 <hemna> jgriffith, ah ok that's different and a real fork of cinder.
16:57:42 <smcginnis> eharney, hemna, jgriffith: OK with you?
16:57:47 <e0ne> smcginnis: for note: I will vote to not add such drivers to cinder
16:57:47 <hemna> I'm more concerned with forks of cinder just to mx a driver
16:57:50 <jungleboyj> We are agreed that those of us that are in violation but currently in will have until Ocata to work this out?
16:57:52 <DuncanT> hemna: Most distributions are forks of openstack currently
16:58:01 <DuncanT> jungleboyj: I'm fine with that
16:58:09 <jgriffith> smcginnis: I'm fine
16:58:12 <jungleboyj> DuncanT: Ok, cool.
16:58:13 <eharney> smcginnis: yes
16:58:15 <smcginnis> OK, no action at this point, we defer to the midcycle.
16:58:19 <jgriffith> smcginnis: honestly it's how we've done it in the past :)
16:58:24 <smcginnis> 2 minutes left
16:58:28 <smcginnis> #topic Logo
16:58:29 <jacob-infinidat> sorry to do finger-pointing here, but it does seem like there is a double standard here. Where there is at least one additional cinder driver that is similar 'external' to our submission
16:58:33 <smcginnis> Just want this brought up.
16:58:34 <jacob-infinidat> and its in.
16:58:39 <hemna> DuncanT, yah and that's usually to fix bugs in core, and some to fix driver bugs that can't make it upstream, because there is no mechanism to backport fixes to older releases
16:58:45 <smcginnis> Attempt to standardize logos across projects.
16:58:50 <jungleboyj> jacob-infinidat: Yes, and we are addressing that.
16:58:51 <dulek> I want to explain myself - I've added two mascot propositions to the etherpad, but if there's a possibility of keeping our brick logo, then I would prefer that. :)
16:58:56 <hemna> pip packages of drivers fixes the driver issues.
16:58:57 <smcginnis> We'll have to discuss later, but wanted to make sure everyone is aware of it.
16:59:05 <DuncanT> hemna: Yup. Would be much easier if the -stable trees lasted longer....
16:59:12 <jgriffith> dulek: I don't think there is :(
16:59:22 <e0ne> hemna: thats why I like 'out-of-tree-drivers' idea
16:59:22 <smcginnis> And we're out of time.
16:59:24 <eharney> i love the volcano ideas
16:59:25 <timwilliams> smcginnis: just want to lobby one more time if any chance to revist falconstor driver https://review.openstack.org/#/c/331579 thanks...
16:59:25 <smcginnis> Thanks everyone.
16:59:26 <xyang> dulek: they say it must be something from nature
16:59:32 <smcginnis> timwilliams: Will do.
16:59:33 <jungleboyj> dulek: ++
16:59:34 <dulek> jgriffith: "They would like to work with Cinder to either update the existing logo to look like the new style or come up with a new one to be consistent with the other projects
16:59:35 <dulek> "
16:59:46 <timwilliams> thnaks!
16:59:51 <jungleboyj> xyang: Bricks are based on natural resources.  ;-)
16:59:59 <patrickeast> haha
17:00:00 <smcginnis> They did contact me about keeping our logo but maybe tweaking it.
17:00:01 <patrickeast> just a rock
17:00:02 <hemna> e0ne, +1
17:00:02 <xyang> jungleboyj: :)
17:00:03 <smcginnis> I hope we can do that.
17:00:06 <smcginnis> That's all folks.
17:00:07 <smcginnis> Thanks
17:00:09 <dulek> :D
17:00:10 <xyang> can we pick a rock?
17:00:11 <jgriffith> dulek: oh, I though it had to be an animal, nature etc
17:00:13 <smcginnis> #endmeeting