16:00:31 <jungleboyj> #startmeeting Cinder
16:00:32 <openstack> Meeting started Wed Nov 28 16:00:31 2018 UTC and is due to finish in 60 minutes.  The chair is jungleboyj. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:35 <openstack> The meeting name has been set to 'cinder'
16:00:55 <jungleboyj> courtesy ping:  jungleboyj diablo_rojo, diablo_rojo_phon, rajinir tbarron xyang xyang1 e0ne gouthamr thingee erlon tpsilva ganso patrickeast tommylikehu eharney geguileo smcginnis lhx_ lhx__ aspiers jgriffith moshele hwalsh felipemonteiro lpetrut lseki _alastor_ whoami-rajat yikun rosmaita
16:01:03 <geguileo> hi! o/
16:01:05 <smcginnis> o/
16:01:06 <erlon> hey
16:01:07 <_alastor_> o/
16:01:07 <xyang> hi
16:01:08 <rosmaita> o/
16:01:11 <rajinir> o/
16:01:13 <whoami-rajat> Hi
16:01:16 <yikun> hello
16:01:29 <jungleboyj> Hey, look at all these people.  :-)
16:01:32 <walshh_> hi
16:01:33 <jungleboyj> @!
16:01:33 <_pewp_> jungleboyj ( ･ω･)ﾉ
16:01:34 <eharney> hi
16:02:28 <e0ne> hi
16:02:56 <jungleboyj> So, looks like we can get started.
16:03:10 <jungleboyj> I don't have any announcements this week.
16:03:18 <jungleboyj> smcginnis:  You have anything you need to share?
16:03:51 <smcginnis> Not that I can think of right now.
16:03:55 <jungleboyj> Ok.
16:04:03 <ganso> hello
16:04:15 <jungleboyj> I will just do our weekly PSA to take a look at the Spec Review list:  https://etherpad.openstack.org/p/cinder-spec-review-tracking
16:04:28 <jungleboyj> I haven't had a chance to really keep up with what is in there at the moment.
16:04:45 <jungleboyj> If you have patches associated with priority work items having them linked in there would be apprecaited.
16:05:37 <smcginnis> I think I saw a couple new drivers that need to be added to the list in that etherpad.
16:05:49 <smcginnis> Though I haven't seen any CI from them, so maybe fine to leave them off of there for now.
16:06:21 <jungleboyj> smcginnis:  Yeah, good point.
16:06:38 <jungleboyj> I have seen those patches too.  I should investigate that as well.
16:07:04 <jungleboyj> I see updates going into the etherpad so that is good.  Thank you.
16:08:13 <jungleboyj> So, on with the agenda.
16:08:23 <jungleboyj> #topic mid-cycle planning
16:08:35 <jungleboyj> #link https://etherpad.openstack.org/p/cinder-stein-mid-cycle-planning
16:08:59 <jungleboyj> I realized we hadn't talked about this for a bit so I wanted to bring it up again.
16:09:16 <jungleboyj> We have some names in the etherpad showing interest.
16:09:33 <jungleboyj> Are people still generally interested in trying to do a mid-cycle meetup?
16:09:44 <rosmaita> yes!
16:09:58 <smcginnis> I think they've always been useful in the past.
16:10:11 <smcginnis> More so than Summits even.
16:10:12 <jungleboyj> smcginnis: ++
16:10:21 <e0ne> jungleboyj: +1 but only virtual this time :(
16:10:22 <jungleboyj> eharney:  If we do it in RTP would you be able to attend?
16:10:27 <eharney> yes of course
16:10:40 <smcginnis> Midcycle at Eric's house.
16:10:45 <jungleboyj> Does the proposed date work for you?
16:10:55 <jungleboyj> smcginnis:  Careful, he will make us meet in the garage.
16:11:09 <jungleboyj> eharney:  Time to get the pool table so we can use it for a workspace.
16:11:23 <jungleboyj> :-)
16:11:24 <eharney> i think the date works fine
16:11:24 <eharney> :)
16:11:59 <jungleboyj> Ok.  So we would have at least 5 people physically there.
16:12:08 <jungleboyj> We could do that at Eric's house.  ;-)
16:12:20 <tbarron> I'll likely kibbutz, especially if you guys talk about the quota-rework and active-active stuff
16:12:28 <jungleboyj> Though, if we schedule it I bet more will come.
16:12:33 <smcginnis> Was just going to ping you tbarron ;)
16:12:34 <jungleboyj> tbarron:  Cool.
16:13:00 <jungleboyj> eharney:  Are there other people on your team there that might join?
16:13:30 <eharney> Rosmaita, and could probably talk jbernard into some of it
16:13:39 <jungleboyj> Ok.  Cool.
16:13:52 <eharney> no idea about that Gorka guy
16:14:09 * jungleboyj chants geguileo , geguileo
16:14:12 <smcginnis> :)
16:14:32 <geguileo> it is unlikely that I'll join the mid-cycle
16:14:36 <geguileo> as usual  :-(
16:14:40 <rosmaita> :(
16:14:43 <jungleboyj> :-(
16:15:02 <jungleboyj> geguileo:  Could you make it virtually for topics that impact you if we schedule them appropriately?
16:15:22 <smcginnis> We'll just assign a lot of the work to you and then jungleboyj can ask you about it every week in the meeting. :)
16:15:28 <geguileo> jungleboyj: that's a given  :-)
16:15:33 <jungleboyj> smcginnis:  ++
16:15:37 <geguileo> smcginnis: rofl rofl rofl
16:15:38 <jungleboyj> geguileo:  Ok.  Cool.
16:16:31 <jungleboyj> I have sent an e-mail to management asking if we can host.
16:16:49 <jungleboyj> It would be pretty barebones I am guessing but I am pretty sure that Wifi works.  :-)
16:16:53 <rosmaita> so you weren't kidding about eric's house
16:17:06 <jungleboyj> Anyone else in RTP interested in hosting?
16:17:07 <smcginnis> haha
16:18:03 <jungleboyj> That sounds like a no.
16:18:38 <jungleboyj> Ok.  I will keep pushing it forward on my side and keep working towards the proposed date.
16:18:56 <jungleboyj> Try to see if I can get our hosting confirmed in the next week or two.
16:20:09 <smcginnis> Type faster jungleboyj
16:20:16 <jungleboyj> :-p
16:20:39 <jungleboyj> #topic Nova not compatible with Cinder earlier than Queens?
16:20:39 <smcginnis> :)
16:20:50 <jungleboyj> #link http://lists.openstack.org/pipermail/openstack-discuss/2018-November/000156.html
16:21:09 <jungleboyj> So, there was this discussion on the ML from mriedem asking about this stance.
16:21:25 <jungleboyj> Is anyone aware of this stance being made?
16:21:29 <smcginnis> I could have sworn we had stated that before, but I also could not find anywhere it was written down.
16:21:35 <eharney> not aware of it
16:21:39 <smcginnis> And it was probably from before microversions.
16:21:56 <smcginnis> That said, removal of v1 is so far overdue, I don't think it's really an issue.
16:21:58 <jungleboyj> If anyone made the statement it would have been jgriffith ?
16:22:28 <smcginnis> v1 has been disabled for awhile and deprecated for a long while.
16:22:35 <smcginnis> v2 will be another story.
16:22:50 <smcginnis> s/story/saga/
16:23:14 <e0ne> I don't understand the use case
16:23:22 <jungleboyj> e0ne:
16:23:27 <jungleboyj> That too is a question.
16:23:32 <e0ne> why nova uses N-2 version?
16:23:48 <e0ne> I do understand N-1 use case during upgrades
16:24:01 <smcginnis> I think it's just that we never state anywhere that you can't use a newer nova with an older cinder.
16:24:22 <smcginnis> I don't think anyone would recommend that, but it's not officially declared as something you cannot do.
16:24:35 <e0ne> smcginnis: +1
16:24:41 <smcginnis> Anyone aware of any of your customers running this way?
16:24:55 <jungleboyj> smcginnis:  Right.  So, was the line drawn at the point where microversions started?
16:25:05 <smcginnis> Does Red Hat or other distros have a policy on supporting something like this?
16:25:44 <eharney> we only support deployments with all versions of the same release AFAIK
16:26:37 <smcginnis> I think given the lack of response on that ML thread, no one is concerned about running that way. Or no one really cares. Or no one actually reads the ML anymore. One of the aboce.
16:26:40 <eharney> i think there's always been a theory that mixing versions should work, but i don't know how much anyone has ever tried to make sure it works...
16:26:41 <smcginnis> abovee
16:26:43 <smcginnis> Gah
16:26:46 <jungleboyj> eharney:  Given my experience with RHOSP that makes esne.
16:26:50 <jungleboyj> *sense
16:27:41 <smcginnis> We don't really test anything other than N with N-1, so there are no guarantees it would work really.
16:27:41 <jungleboyj> smcginnis:  True enough.
16:28:06 <jungleboyj> So, is this just a documentation issue?
16:28:42 <smcginnis> Matt might want a release note in Nova.
16:28:55 <whoami-rajat> I think Kolla ansible allows deploying modules with different releases (or even services with different releases) in containers and it's used for PoC purposes.
16:28:56 <eharney> i think some people would ask, if we have compatibility guarantees around APIs with microversions etc, why wouldn't it work, exactly?
16:29:32 <jungleboyj> eharney:  Right, but that would only start at the point where we had Microversion Support.  Right?
16:29:59 <smcginnis> Since v3 is our only supported API version, I think that's the minimum. Once there, we should be able to support any mix theoretically if we are using microversions right.
16:30:04 <eharney> jungleboyj: maybe..
16:30:44 <jungleboyj> V3 started back in Mitaka.
16:31:20 <jungleboyj> So, in theory, Mitaka or high should work.
16:31:24 <tbarron> which version of brick do you use when you mix versions of nova and cinder?
16:31:37 <jungleboyj> tbarron: Ugh.
16:31:54 <eharney> :)
16:32:23 <jungleboyj> The one thing about queens is that is when we got all the attachment changes in place that jgriffith and ildikov had worked on.
16:32:36 <jungleboyj> I am wondering if that was why a line had been drawn in the sad.
16:32:38 <jungleboyj> *sand
16:32:57 * tbarron likes 'line in the sad' :)
16:33:05 <jungleboyj> :-)
16:33:17 * tbarron is drawing a face right now
16:34:21 <jungleboyj> Lets see if this works ... jgriffith
16:34:27 <jungleboyj> That was 3 times.
16:34:51 * jungleboyj now is sad
16:35:36 <jungleboyj> Ok.  So, the summary seems to be that we don't see anywhere that Queens is documented as the limit for compatability.
16:35:46 <smcginnis> Anyway, I think we're fine with the change.
16:35:53 <smcginnis> If nova wants to add a release note, they can.
16:36:04 <jungleboyj> In theory it could be back to Mitaka but that isn't tested.
16:36:27 <jungleboyj> We don't recommend running more than N and N-1 .
16:36:35 <smcginnis> I don't think we are going to ask multiple mixed version tests and I don't think we need to change anything in Cinder.
16:36:41 <smcginnis> s/ask/add/
16:36:41 <e0ne> +1
16:36:42 <jgriffith> kinda
16:36:47 <smcginnis> Typing hard
16:36:54 <jungleboyj> smcginnis:  Typing is ahrd today.
16:36:59 <jungleboyj> OMG
16:37:08 <smcginnis> jgriffith: I thought you were going to stay hidden. :)
16:37:13 <jungleboyj> smcginnis:  Ok.
16:37:31 <jungleboyj> He is getting slow with the say his name three times thing.
16:37:43 <jgriffith> smcginnis: nahh :)  Trying to distill the question
16:38:08 <smcginnis> Something with new attachments with mix of old and new os-brick. Not really sure myself.
16:38:24 <jungleboyj> jgriffith: Is there a reason someone would have said Cinder earlier than Queens is not compatible with Nova.
16:39:06 <ildikov> Not my fault! :)
16:39:15 <jungleboyj> He he.
16:39:21 <jgriffith> Not that I'm aware of
16:39:22 <jungleboyj> Now the gang's all here!
16:39:26 <jgriffith> I don't know where that came from
16:39:42 <jungleboyj> jgriffith: Ok.  Cool.  Thank you.
16:39:45 <smcginnis> ildikov must have an alert set up on "attachments". :)
16:39:59 <ildikov> On a conference so couldn't read back on full context though
16:40:04 <jungleboyj> smcginnis:  Nah, I mentioned her too.
16:40:10 <jgriffith> I believe the confusion (maybe?) was that johnthetubaguy did raise points about the other direction (obviously)
16:40:16 <jungleboyj> ildikov: So you just respond not my fault?  he he he.
16:40:22 <ildikov> smcginnis: don't we all by now? :)
16:40:27 <smcginnis> :)
16:40:37 <jgriffith> But honestly the versioning should make all of that work out
16:40:38 <jungleboyj> jgriffith:  Oh, that makes sense.
16:40:49 <ildikov> I read a few lines around your ping
16:40:56 <jgriffith> jungleboyj: well... kinda; but again versioning makes it kind of irrelavant I think
16:41:19 <smcginnis> So the answer is just to use v3. ;)
16:41:25 <jgriffith> LOL
16:41:26 <ildikov> And yeah, we put a lot of work into versions
16:41:30 <jgriffith> Which V3?
16:41:35 <smcginnis> Cinder API v3.
16:41:40 <jgriffith> You mean the V2 we poin to from V3?
16:41:44 <smcginnis> This came about because I removed some handling for v1 from Nova.
16:41:56 <jgriffith> or one of the V3."<insert-one-of-70-numbers-here>
16:42:04 <smcginnis> Once we're able to use microversions, then it doesn't really matter what versions the services are.
16:42:12 <ildikov> LOL
16:42:13 <smcginnis> But we need to get rid of v1 AND v2.
16:42:19 <jgriffith> smcginnis: +1
16:42:25 <jungleboyj> smcginnis: +1
16:42:31 <e0ne> smcginnis: +2
16:42:34 <ildikov> smcginnis: +1
16:42:36 <jgriffith> I thought we already did started that process?
16:42:37 <smcginnis> Hah
16:42:50 <smcginnis> v1 is gone, but there was still a check for it in Nova.
16:42:56 <smcginnis> That's gone now.
16:43:03 <jgriffith> Ok, gotcha
16:43:18 <smcginnis> And I've started the work on removing v2, but between tempest changes, client changes, etc., that's taking awhile.
16:43:38 <jungleboyj> smcginnis: Thanks for working on that.
16:43:49 <jgriffith> yeah, and the intertwining of V3 and V2 I'll be curious to see what that looks like in the end :)
16:44:20 <smcginnis> Should just be a few moves and getting rid of some proxy classes.
16:44:24 * smcginnis knocks on wood
16:44:33 <jungleboyj> So, I think the answer is that we don't have a reason for saying that anything earlier than Queens isn't supported but we don't test anything less than N with N-1 .
16:44:38 <jgriffith> `cp -r ..../V2/* .../V3/`
16:44:46 <smcginnis> :)
16:45:10 <jgriffith> jungleboyj: not that I'm aware of; but my brain forgets things
16:45:52 <e0ne> I remember how many things were broken while I tried to remov v1
16:46:00 <e0ne> s/remov/remove
16:46:09 <smcginnis> Yeah, that took some work.
16:46:24 <e0ne> tempest, devstack, nova, glance
16:46:28 <jungleboyj> :-)  Apparently every line we type needs to have a tpyo.
16:46:32 <e0ne> that what I have in the mind
16:46:43 <smcginnis> Idneed
16:46:48 <e0ne> jungleboyj: it's a Typo Day
16:46:56 <jungleboyj> :-)
16:47:10 <jungleboyj> So, sohuld we move on to the next topic?
16:47:44 <jungleboyj> #action jungleboyj to respond to the mailing list thread.
16:48:06 <jungleboyj> #topic Python 3.7 Support
16:48:19 <jungleboyj> smcginnis: All your's.
16:48:27 <jungleboyj> *yours
16:48:46 <smcginnis> Not sure if folks saw some of the discussion, but we've kind of gotten complacent that our python versions have been stable for awhile.
16:49:19 <smcginnis> That's changing though, and we need to be prepared for Python 3.7 support which will be the default version on our supported platforms in T or U.
16:49:40 <smcginnis> There's a patch out there to add a job, mostly as a pipecleaner to see what's needed to get there.
16:49:49 <smcginnis> #link https://review.openstack.org/617840
16:50:12 <smcginnis> Looks like the VMAX driver (if I remember right) has some issues, but I think it's actually through its use of taskflow.
16:50:23 <smcginnis> So there's a bug open against Cinder and taskflow for this.
16:50:23 <e0ne> can we have this job in non-voting until taskflow will be fixed?
16:50:47 <smcginnis> It might be the case that once taskflow is fixed we don't have anything to worry about in Cinder, but leaving the bug out there until we confirm that.
16:51:05 <smcginnis> e0ne: Yeah, maybe that would be good to make sure we don't introduce anything else breaking.
16:51:11 <e0ne> smcginnis: it's VNX driver: http://logs.openstack.org/40/617840/1/check/openstack-tox-py37/dcf0ed1/testr_results.html.gz
16:51:29 <smcginnis> V*X :)
16:51:44 <e0ne> ok, it was a typo in a driver name
16:51:57 <smcginnis> Haha, yeah.
16:52:23 <smcginnis> So I just wanted to raise this for awareness. We aren't going to be on 3.5 forever, so we need to be preparing for newer versions.
16:52:24 <jungleboyj> What is the bug for the taskflow issue?
16:52:46 <smcginnis> #link https://bugs.launchpad.net/taskflow/+bug/1780823
16:52:47 <openstack> Launchpad bug 1780823 in taskflow "Py3.7 RuntimeError: generator raised StopIteration" [High,Confirmed]
16:52:57 <e0ne> I'm OK to have this job in non-voting for now if we're going to switch to 3.7 in T or U release
16:53:32 <smcginnis> Might also be a good opportunity to remind CI operators that you should be moving to running under python 3.
16:53:49 <e0ne> smcginnis: +1
16:53:59 <smcginnis> Our unit tests pass, but there have been a few cases where actually runtime has uncovered issues under py3 that were not a problem under py2.
16:54:10 <smcginnis> We are dropping Python 2.7 in U.
16:54:42 <smcginnis> And if anyone wants to dig into that taskflow bug, I don't think they have a lot of attention right now.
16:56:08 <smcginnis> That's all I really had on that for now. Just raising awareness.
16:56:09 * jungleboyj waits for everyone to jump on our favorite taskflow .
16:56:16 <e0ne> :)
16:56:19 <jungleboyj> smcginnis:  Cool.  Thanks.
16:56:42 <jungleboyj> Last think I had was a bug that came up.
16:56:48 <jungleboyj> #topic bugs
16:57:01 <jungleboyj> #link https://bugs.launchpad.net/cinder/+bug/1805506
16:57:02 <openstack> Launchpad bug 1805506 in Cinder "Tempest tests failing with some errors related to cinder backups" [Undecided,New]
16:57:55 <jungleboyj> So, neutron's jobs are seeing some failures due to Cinder backup.
16:58:09 <jungleboyj> e0ne:  Any thoughts on this given your backup experience?
16:58:22 <e0ne> I need to read logs
16:58:32 <jungleboyj> e0ne:  Ok.
16:58:42 <jungleboyj> Mind taking a look at that?
16:59:44 <e0ne> I'll check the logs but can't promise that I'll have a time to fix it this week
17:00:01 <jungleboyj> e0ne:  No problem.  Just wanted to make sure to get it to the right person.
17:00:24 <jungleboyj> And, that is all we have time for this week.
17:00:39 <smcginnis> Thanks
17:00:40 <jungleboyj> Thank you for joining the meeting and working on Cinder!
17:00:45 <jungleboyj> Talk to you all next week!
17:00:49 <e0ne> see you next week
17:00:50 <jungleboyj> #endmeeting