16:00:29 <jungleboyj> #startmeeting Cinder
16:00:30 <openstack> Meeting started Wed Oct  4 16:00:29 2017 UTC and is due to finish in 60 minutes.  The chair is jungleboyj. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:35 <openstack> The meeting name has been set to 'cinder'
16:00:37 <Swanson> hello.
16:00:39 <bswartz> .o/
16:00:40 <gouthamr> o/
16:00:44 <smcginnis> o/
16:00:51 <ganso> hello
16:00:51 <jungleboyj> Courtesy ping:  jungleboyj DuncanT diablo_rojo, diablo_rojo_phon, rajinir tbarron xyang xyang1 e0ne gouthamr thingee erlon patrickeast tommylikehu eharney geguileo smcginnis lhx_ lhx__
16:01:00 <geguileo> hi! o/
16:01:03 <erlon> hey
16:01:14 <patrickeast> Hi
16:01:29 <eharney> hi
16:01:30 <jungleboyj> Give people a minute to gather.
16:01:32 * DuncanT wakes up
16:01:33 <e0ne> hi
16:01:33 <tbarron> hi
16:01:45 * jungleboyj hands a RedBull to DuncanT
16:01:51 <DuncanT> Cheers
16:01:59 <jungleboyj> :-)
16:02:32 * bswartz hands a Four Loko to jungleboyj
16:02:57 <smcginnis> DuncanT: Wake up? Isn't it a little late there?
16:03:12 <jungleboyj> bswartz:  A little early on a work day for that.  :-)
16:03:14 <e0ne> #link https://etherpad.openstack.org/p/cinder-queens-meeting-agendas
16:03:25 <jungleboyj> Malt Liquor!
16:03:46 <jungleboyj> Ok, I am guessing we can get started.
16:03:56 <jungleboyj> #topic Announcements
16:04:10 <jungleboyj> #link https://etherpad.openstack.org/p/cinder-spec-review-tracking
16:04:21 <jungleboyj> I spent time on Friday and Monday going through Specs.
16:04:37 <jungleboyj> I have gotten the ones I can get approved into the review tracking etherpad.
16:05:10 <jungleboyj> If you have comments outstanding, appreciate attention to those so we can continue to make progress getting the work for Queens defined.
16:05:28 <jungleboyj> If you have patches for specs that are in there appreciate help getting those documented.
16:05:50 <jungleboyj> I have also gotten our release schedule officially posted out on the Queens release schedule page.
16:06:05 <jungleboyj> So, if you need to know the important dates, etc, please see that page.
16:06:34 <jungleboyj> #topic Check/Gate Status
16:06:45 * jungleboyj start crying
16:06:49 <e0ne> #link https://releases.openstack.org/
16:07:11 <jungleboyj> e0ne:  Thanks for being my link master.  ;-)
16:07:19 <e0ne> jungleboyj: :)
16:07:32 <jungleboyj> So, we were seeing many jobs failing on a volume revert test.
16:07:42 <jungleboyj> #link https://review.openstack.org/#/c/508541
16:07:57 <jungleboyj> Keep hitting various issues trying to get it merged.
16:08:08 <jungleboyj> I am continuing to try to get that patch through.
16:08:23 <jungleboyj> Then a number of patches will need to be rechecked .
16:08:42 <jungleboyj> Spec jobs were broken.  Thanks to smcginnis for fixing them.
16:08:54 <jungleboyj> #link https://review.openstack.org/509427
16:09:04 <jungleboyj> That patch should have us back on track.
16:09:52 <jungleboyj> For those that haven't monitored the whole process we now have the new ZuulV3 running non-voting (reports as Zuul) and Jenkins is back in place as the voting job.
16:10:06 <jungleboyj> #link http://lists.openstack.org/pipermail/openstack-dev/2017-October/123049.html
16:10:15 <smcginnis> Basically zuul is like a third party CI for now until they fix it.
16:10:15 <jungleboyj> That is the mailing list thread on the state of things.
16:10:22 <jungleboyj> smcginnis: ++
16:11:04 <jungleboyj> I am keeping a list of things that I would like to +W and will do so once things stabilize.
16:11:54 <jungleboyj> So, I think that is all I have on that.
16:12:07 <jungleboyj> Thanks to everyone for their patience and support right now.
16:12:21 <smcginnis> This has been painful for sure.
16:12:29 <jungleboyj> smcginnis:  Indeed.
16:12:35 <Swanson> zuul never broke under the old PTL.
16:12:40 <smcginnis> Hah!
16:12:44 <jungleboyj> #topic LVM thin provisioning errors
16:12:46 <bswartz> lol
16:12:50 <jungleboyj> Swanson: :-p
16:12:59 <jungleboyj> I know, I have ruined it for everyone!
16:13:07 <jungleboyj> smcginnis:  Take it away.
16:13:09 <smcginnis> So I wanted to mention the LVM errors.
16:13:19 <erlon> haha
16:13:38 <smcginnis> I'm pretty sure this is due to switching devstack to go to "auto" which results in using thin provisioned volumes.
16:14:01 <smcginnis> We had done this once before and had some errors, so the devstack change ended up getting reverted to go back to thick.
16:14:08 <smcginnis> I don't think that's the right answer though.
16:14:21 <smcginnis> We need to figure out the thin provisioning errors and fix them.
16:14:37 <e0ne> smcginnis: do we have any steps to reproduce it?
16:14:37 <smcginnis> The errors look like this:
16:14:40 <smcginnis> #link http://logs.openstack.org/09/508209/5/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/44aa0eb/logs/screen-c-vol.txt.gz#_Oct_04_13_03_10_481121
16:14:59 <smcginnis> e0ne: That's the hard part. Locally I set up a test to loop trying to repro it and never did.
16:15:10 <smcginnis> So it's not all the time at least.
16:15:23 * diablo_rojo_phon sneaks in late
16:15:24 <smcginnis> Just need to find the specific conditions that trigger it.
16:15:30 <e0ne> smcginnis: the same for me. I tried to reproduce similar issue in the past
16:15:59 <eharney> it's probably race conditions related to the activation calls, may need to beat on things to trigger it
16:16:18 <smcginnis> eharney: That was kind of my hunch, but surprised retries didn't fix it.
16:16:34 <smcginnis> There was also an os-brick change I think. Let me see if I can find that.
16:16:42 <eharney> i can't remember when the cascade delete tests turned on, that may be "helping" it fail too
16:17:22 <smcginnis> I thought this might fix it: https://review.openstack.org/#/c/505601/
16:17:39 <jungleboyj> #link https://review.openstack.org/#/c/505601/
16:18:24 <smcginnis> Well, just raising it for awareness. I'll try to bang on it a little more.
16:18:32 <smcginnis> Good to hear you've tried as well e0ne.
16:18:47 <jungleboyj> #action smcginnis to continue to try to recreate and debug.
16:19:00 <smcginnis> I think if we can't fix it, if we go back to thick we should probably state thin is unsupported.
16:19:07 <smcginnis> *isn't
16:19:32 <Swanson> *is
16:19:49 <smcginnis> Hah, yep. Distracted I guess.
16:20:00 <eharney> we need to sort out the differences between how activation works on thick and thin to get a handle on what's happening there, i think
16:20:01 <smcginnis> I'd like it to be isn't unsupported.
16:20:08 <jungleboyj> So we would state is unsupported or wouldn't?
16:20:23 <smcginnis> Yeah, if anyone here is an "LVM expert", nows the time to show off. :)
16:20:38 <smcginnis> jungleboyj: Only if we are forced to go back to thick.
16:20:57 <smcginnis> jungleboyj: That's really all.
16:21:00 <eharney> doing the same tests on centos instead of xenial would be interesting, too
16:21:03 <Swanson> Turn on jgriffith signal!
16:21:24 <jungleboyj> :-)  Yeah, was kind of hoping he would say something there.
16:21:41 <diablo_rojo_phon> Haha yeah where is jgriffith?
16:21:53 <jungleboyj> I have started looking at some of that LVM code but am not an expert at all.
16:22:25 <jungleboyj> diablo_rojo_phon:  Not sure, he hasn't been able to make it to this meeting as often.
16:22:52 <jungleboyj> smcginnis: Anything else on that topic?
16:23:16 <diablo_rojo_phon> Probably dealing with customers?
16:23:16 <smcginnis> Nope, that's it for now.
16:23:25 <jungleboyj> smcginnis:  Cool.  Thanks.
16:23:33 <jungleboyj> #topic Open Discussion
16:23:41 <jungleboyj> Anyone else have topics for today?
16:24:09 <diablo_rojo_phon> Who all is going to Sydney? Are we collecting a list yet?
16:24:26 <jungleboyj> I will be there.  smcginnis will .
16:24:33 <jungleboyj> I haven't started a list yet.
16:24:39 <diablo_rojo_phon> Also, is there a list somewhere of all the Cinder related forum topics?
16:24:52 <diablo_rojo_phon> Might be another good list to create :)
16:25:18 <smcginnis> We had the brainstorming one, but I don't think we ended up with any Cinder-specific forum sessions.
16:25:24 <jungleboyj> diablo_rojo_phon:  I had an etherpad for that that I put together a few weeks ago and no one responded.
16:25:34 <jungleboyj> I have a cross project session with nova scheduled.
16:25:46 <smcginnis> Oh right.
16:25:47 <jungleboyj> I also proposed doing office hours for people to stop in if they had questions and concerns.
16:25:55 <jungleboyj> Not sure if those have both been approved.
16:26:06 <jungleboyj> smcginnis:  We had those two.
16:26:18 <Swanson> smcginnis, practice up on Knifey Spoony before you go to Sydney.
16:26:44 <jungleboyj> Then there is the on-boarding room.  Would like to get as many people as we can from the team there for that.
16:26:59 <jungleboyj> Will have more info on that as we get closer.
16:27:24 <jungleboyj> I can start an etherpad with Cinder Related Summit info and send the link out.
16:27:44 <smcginnis> jungleboyj: Cool, we can collect names there too of attendees.
16:27:50 <jungleboyj> smcginnis:  ++
16:28:00 <diablo_rojo_phon> Oh yeah that would be good. All in one place.
16:28:45 <jungleboyj> #action jungleboyj to put together a Cinder Sydney Summit etherpad for relevant information and attendance recording
16:29:31 <jungleboyj> Anything else we need to discuss?
16:29:49 <diablo_rojo_phon> jungleboyj: will be sending out on boarding info to the dev list Thursday so look for that
16:30:03 <jungleboyj> diablo_rojo_phon:  Great.  Thank you.
16:30:17 <jungleboyj> diablo_rojo_phon:  Are they going to be recorded this time?
16:31:26 <jungleboyj> I take it we have nothing else to discuss today then?
16:31:37 <smcginnis> Nothing from me.
16:32:00 <jungleboyj> Ok.  Please work on getting specs in if you had ToDo's coming out of the PTG.
16:32:31 <jungleboyj> Appreciate all the help we can get getting patches through with the current state of the world.
16:32:47 <e0ne> see you next week
16:32:52 <diablo_rojo_phon> jungleboyj: not sure it's in the budget but there will be projection
16:33:11 <jungleboyj> diablo_rojo_phon:  Ok.  May need to do our own thing there then.
16:33:28 <jungleboyj> Ok, thanks for your work team.  Talk to you all next week!
16:33:36 <jungleboyj> #endmeeting