16:00:22 <thingee> #startmeeting cinder
16:00:22 <openstack> Meeting started Wed Mar 25 16:00:22 2015 UTC and is due to finish in 60 minutes.  The chair is thingee. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:24 <e0ne> hi!
16:00:26 <openstack> The meeting name has been set to 'cinder'
16:00:31 <smcginnis> o/
16:00:33 <thingee> hi everyone!
16:00:34 <xyang2> hi
16:00:35 <tbarron> hi
16:00:37 <jgravel> hello
16:00:38 <abhishekk> o/
16:00:39 <akerr> o/
16:00:40 <ameade> good day folks
16:00:40 <cebruns> Hi
16:00:42 <patrickeast> hi
16:00:42 <jungleboyj> o/
16:00:46 <Swanson> hello
16:00:47 * jungleboyj waves from the beach
16:00:48 <Liu> hi
16:00:50 <scottda> hi
16:00:52 <dulek> Hi
16:01:00 <thingee> agenda items for today:
16:01:02 * smcginnis doesn't think jungleboyj does vacations well.
16:01:02 <thingee> #link https://wiki.openstack.org/wiki/CinderMeetings#Next_meeting
16:01:26 <thingee> #topic Future of tasflow in cinder
16:01:28 <thingee> abhishekk: hi
16:01:37 <asselin> o/
16:01:37 <abhishekk> thingee: hi
16:01:39 <kmartin> hello
16:01:41 <thangp> o/
16:01:48 <thingee> abhishekk: so there has been zero progress on discussion with Cinder and taskflow
16:01:53 <abhishekk> yes
16:01:56 <thingee> abhishekk: just like the last time this was brought up
16:02:17 <thingee> abhishekk: there have been a bit more pressing issues going on in Cinder, if you're keeping up with the mailing list
16:02:17 <abhishekk> thingee: yes, we are ready to put constructive efforts for L
16:02:22 <e0ne> thingee, abhishekk: we've got design session proposals for it
16:02:46 <abhishekk> thats great
16:03:02 <thingee> abhishekk: so yes there's a summit session. unless there is anything new to bring to the table on this topic
16:03:05 <dulek> yes, I've proposed it, but to prepare it would be great to know the direction
16:03:27 <abhishekk> I agree with dulek
16:03:33 <thingee> ok what's new?
16:04:13 <e0ne> dulek: i propose to update our patches before summit
16:04:27 <dulek> thingee: I guess not much, I would just love to know if the decision on stepping out from TaskFlow is taken or not
16:04:41 <thingee> I would like to ask that there is a decide approach and not X number of approaches
16:04:41 <abhishekk> thingee: yes, Are we going to step out from using TaskFlow?
16:04:54 <thingee> I just find the whole thing confusing otherwise...which is why I think no progress has been made.
16:05:02 <thingee> on getting things merged
16:05:25 <e0ne> thingee: TBH, we started work on it too late for K
16:05:29 <thingee> abhishekk: that's the point of the summit
16:05:40 <thingee> abhishekk: which is a topic I will say comes up every summit and midcycle meetup
16:05:46 <thingee> abhishekk: which is not a great sign for a project
16:05:54 <thingee> e0ne: ^
16:06:18 <e0ne> thingee: agree, it's reasonable
16:06:25 <dulek> thingee: that's ok for me, so we'll decide at the summit
16:06:36 <abhishekk> thingee: right, hopefully we can come to some conclusion in this summit
16:07:02 <jungleboyj> I agree that it is concerning that we keep discussing it and not making progress.  Taskflow is hard to wrap our heads around.
16:07:10 <thingee> if there is a split in the decision, I'll weigh in more after giving my initial thoughts at the summit based on the decided approach from you all on state transitions.
16:07:32 <thingee> who wants to make sure that there is a decided approach to give at the summit?
16:07:43 <thingee> and ideally decided early before the summit so we can familiarize ourselves
16:07:57 <e0ne> thingee: +1
16:08:05 <eikke> is there an overview of cons of using taskflow?
16:08:12 <jungleboyj> We need better examples and reasons for why taskflow is better if we are going to make a decision at the Summit.
16:08:24 <abhishekk> thingee: +1
16:08:41 <tbarron> I for one need guidance on how to debug issues when taskflow is involved.  I get lost.
16:08:56 <dulek> thingee: I can gather approach ideas and communicate to the ML before the summit.
16:08:57 <jungleboyj> tbarron: +2
16:09:00 <thingee> eikke: there is a lot of scattered reviews of us realizing some issues we hit along the way. If you take the recent allocation bug that jgriffith was trying to solve for sometime and then dulek had to write a hack for to get around...
16:09:04 <thingee> dulek: which by the way, thank you
16:09:22 <dulek> thingee: np, but I got to agree, it was harsh
16:09:34 <thingee> #action dulek to make sure there is a decided approach to give at the summit and sent to the mailing list beforehand
16:09:37 <eharney> it worries me that most of the taskflow discussions seem to revolve around general feeling and not analysis of real pros/cons... it would be good to get some organized information
16:09:51 <jungleboyj> eharney: +2
16:09:58 <winston-d_zzZ> eharney: +2
16:09:58 <thingee> eharney: +1 eikke mentioning that made me realize that as well
16:10:04 <e0ne> eharney: +1
16:10:05 <ameade> +10
16:10:10 <geguileo> eharney: +1
16:10:14 <jungleboyj> I ust don't understand what it improves yet.
16:10:15 <eikke> eharney: +1
16:10:20 <thingee> to help the taskflow folks, I think it would be good if someone gathered the issues from the cinder side
16:10:24 <jungleboyj> Need concrete pros and cons.
16:11:05 * thingee would vote for jgriffith to provide some information to whoever collects this info
16:11:06 <e0ne> jungleboyj: as i sad few month before, it's not very useful w/o persistance
16:11:31 <thingee> anyone able to volunteer to gather the issues with taskflow + cinder?
16:11:45 <e0ne> thingee: i can do it
16:11:57 <thingee> #action e0ne to gather the cons with taskflow in Cinder
16:12:03 <thingee> e0ne: please talk to jgriffith
16:12:07 <thingee> e0ne: and thank you!
16:12:14 <e0ne> thingee: sure, i'll do
16:12:16 <dulek> e0ne: Would you mind to use ML to communicate that? We may start a discussion here which is goo.d
16:12:28 <thingee> dulek: great idea
16:12:30 <dulek> s/here/there
16:12:31 <abhishekk> thank you e0ne
16:12:37 <thingee> #undo
16:12:38 <openstack> Removing item from minutes: <ircmeeting.items.Action object at 0x8d94a50>
16:12:43 <cebruns> Adding to etherpad for summit would be good too.
16:12:55 <thingee> #action e0ne to gather the cons with taskflow in Cinder and post to ML for discussion
16:13:01 <thingee> cebruns: it's there
16:13:03 <eharney> also remember to actually compare not using taskflow to an alternative (missing functionality we'd like, implementing some complex thing ourselves...)
16:13:06 <thingee> from what others told me
16:13:12 * thingee checks
16:13:17 <e0ne> dulek: i'll chat you too for it:)
16:13:24 <dulek> e0ne: cool :)
16:13:49 <thingee> #link https://etherpad.openstack.org/p/cinder-liberty-proposed-sessions
16:13:50 <cebruns> Yeah - meant adding the investigation/pros/cons results to etherpad as wella s ML
16:13:52 <thingee> cebruns: ^
16:13:57 <thingee> cebruns: good idea
16:14:19 <thingee> ok I think this was great to discuss to get ourselves more organized for the topic at the summit
16:14:23 <e0ne> eharney: yes, imo, the main problem, we don't use needed features from taskflow
16:14:23 <dulek> thingee: I guess we'll take care of that with e0ne
16:14:25 <thingee> thanks dulek e0ne and abhishekk
16:14:28 <winston-d_zzZ> it could be eaiser to understand the real benefit of taskflow if there were some carefully designed synthetic 'failure' cases, which one can actually reproduce if needed, that taskflow helps.
16:16:23 <e0ne> winston-d_zzZ: +1
16:16:43 <cebruns> winston-d: +1 and showing "when Cinder encounters this..." taskflow can do "this" to alleviate it...
16:16:45 <abhishekk> winston-d: +1
16:16:46 <thingee> winston-d: I agree. I think some of persistence patches have not been completely visible of what scenario it actually fixes. e.g. if this node fails, we have to wait for that node to come back to resume
16:16:49 <jungleboyj> winston-d: +1
16:17:17 * thingee might be wrong but remembers DuncanT being a bit involved with that and calling them out
16:17:42 <DuncanT> Erk, sorry, hadn't noticed the time
16:17:54 <thingee> also with some of the persistence stuff being limited to use sqlite on the node itself. did that get resolved?
16:17:56 <e0ne> thingee: we don't use taskflow for cinder recovering at all
16:18:14 <thingee> e0ne: correct. there are patches in discussion about this though
16:18:33 <e0ne> thingee: no. we can use and DB that is supported by TF(Mysql, postgresql, etc)
16:18:38 <dulek> thingee: This creates new issues, which I wanted to start discussion in ML thread mentioned by abhishekk
16:18:53 <e0ne> thingee: it was consern for it in dulek's spec
16:18:54 <dulek> thingee: But that's true, we should revisit that
16:19:22 <thingee> dulek: ok so nothing has changed there? I apologize but I'm not up to speed on the particular patch that this was brought up on
16:20:11 <e0ne> thingee: we moved our focus from persisnance to K-3, afair
16:20:12 <dulek> thingee: I've changed the priorities, will make sure we have clear vision before the summit.
16:20:23 <e0ne> so nobody was involved in it
16:20:43 <thingee> got it
16:20:58 <thingee> dulek, abhishekk, e0ne anything else for this topic? I think this was great to start preparing
16:21:13 <dulek> thingee: I'm fine, thanks a lot!
16:21:20 <abhishekk> thingee: thanks a lot!
16:21:34 <thingee> #topic CI Status Page
16:21:40 <abhishekk> dulek, e0ne: thank you
16:21:47 <thingee> #idea The status page either needs to be updated or we should link to Mike's spreadsheet.
16:21:47 <kmartin> the 3PAR driver is using taskflow for retype, as we have to make 2-3 calls to the array depending on what needs to be changed and rollback if one of those fails.
16:21:49 <thingee> jungleboyj: hi
16:21:58 <jungleboyj> thingee: Hey, this is just real quick.
16:22:23 <jungleboyj> Had a question about whether the status page that is currently linked here:  https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
16:22:36 <jungleboyj> Should be updated or if we should change that link to the spreadsheet you have for CI status.
16:22:54 <thingee> I don't think my non-editable spreadsheet is a good idea. I realize it's creating confusion with people as being what others should follow. It was purely for my tracking purposes to temporarily keep track of CIs
16:22:55 <jungleboyj> This is out of date:  https://wiki.openstack.org/wiki/Cinder/third-party-ci-status
16:23:00 <patrickeast> i thought the decision a while ago was that we were not updating the wiki page
16:23:08 <thingee> #link https://wiki.openstack.org/wiki/Cinder/third-party-ci-status
16:23:11 <patrickeast> people should be keeping their thirdpartysystems wiki pages up to date
16:23:26 <ameade> yeah do we even need that page?
16:23:31 <patrickeast> https://wiki.openstack.org/wiki/ThirdPartySystems
16:23:57 <thingee> ameade: it's a good point
16:23:59 <thingee> patrickeast: +1
16:24:02 <thingee> anyone disagree?
16:24:15 <akerr> +1, one less thing to keep track of
16:24:20 <eharney> agreeing to not update it is rather confusing if we leave it there for people to find and link to...
16:24:21 <DuncanT> The page was my attempt to do an open version of Thingee's spreadsheet... feel free to nuke it
16:24:35 <patrickeast> eharney: +1, we should probably remove it
16:24:56 <thingee> #agreed nuke https://wiki.openstack.org/wiki/Cinder/third-party-ci-status in favor of https://wiki.openstack.org/wiki/ThirdPartySystems
16:25:13 <jungleboyj> patrickeast: +1 if that page isn't used we should get rid of it and link to the Third Party systems page instead.
16:25:14 <thingee> I will also leave a link on my spreadsheet at the top
16:25:21 <thingee> in case people find that
16:25:24 <jungleboyj> Sounds good.
16:25:37 <thingee> jungleboyj: sounds good...I'll do a redirect
16:25:50 <thingee> #action thingee to redirect old CI page to decided CI page
16:26:10 <thingee> #action thingee to leave a link on his spreadsheet to the decided page as well
16:26:15 <thingee> jungleboyj: anything else?
16:26:34 <jungleboyj> thingee: No, that was it.  I will make sure we have our pages updated.
16:26:40 <jungleboyj> Thank you for the decision.
16:26:59 <thingee> #action thingee to communicate to driver maintainers about making sure pages are up-to-date
16:27:06 <kmartin> should the https://wiki.openstack.org/wiki/CinderSupportMatrix be update with the drivers removed?
16:27:35 <thingee> kmartin: good point. Nothing is final right now...after RC I'll do a clean up if needed
16:28:02 <thingee> #action thingee to clean up driver support matrix page after RC
16:28:09 <thingee> jungleboyj: thanks
16:28:15 <thingee> #topic Deadline for readding volume drivers and RC
16:28:27 <jungleboyj> Thank you!
16:28:57 <ameade> I have a couple questions on how to test unmerged drivers once we figure this out
16:29:08 <thingee> so RC, is projected to 4/9 on the launchpad page. I learned this morning from ttx that's not a real date.
16:29:41 <thingee> we will do a cut when the bug list is empty
16:29:47 <xyang2> ameade: submit a patch with your drivers added. then you should be able to test it
16:30:09 <akerr> xyang2: would that be sufficient to show stability?
16:30:22 <ameade> lets figure out the deadline stuff first
16:30:27 <jungleboyj> thingee: What does that mean?
16:30:51 * thingee looks at calendar
16:30:56 <asselin> ameade, https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers & specific CINDER_BRANCH as shown there
16:31:07 <jungleboyj> thingee: I assume you mean when the targeted bug list is empty?
16:31:20 <thingee> jungleboyj: yes
16:31:40 <thingee> so the bug list can change
16:31:43 <thingee> can grow
16:31:49 <thingee> if there are real showstoppers
16:31:57 <thingee> that's why setting deadlines in RC isn't easy
16:32:08 <ameade> can we set a minimum date?
16:32:09 <asselin> ameade, or manually cherry-pick your driver's patch on top of the latest
16:32:11 <thingee> #link https://launchpad.net/cinder/+milestone/kilo-rc1
16:32:16 <thingee> ameade: yea
16:32:22 <thingee> so I was think 4/6 ?
16:32:23 <ameade> asselin: that's what i'm doing now but i gotta make sure everyone agrees thats fine
16:32:29 <xyang2> thingee: only show stoppers can be targeted?
16:32:42 <thingee> xyang2: should be at this point
16:32:58 <thingee> xyang2: I'll definitely help drivers in focus on big bug fixes for them though of course
16:33:06 <thingee> xyang2: already did it for a couple of drivers so far
16:33:39 <thingee> so is 4/6 fair?
16:33:48 <ameade> thingee: that sounds ok to me
16:33:49 <xyang2> thingee: so you mean don't target driver bugs, but we can still review and approve them?
16:33:56 <xyang2> thingee: sounds good
16:33:56 <thingee> gives me enough time to not drive other projects insane on waiting on cinder to do a cut
16:33:57 <ameade> how long do we want to see new CIs run for?
16:34:02 <e0ne> thingee: could we target medium bugs for RC-1 and not for RC-2, etc?
16:34:19 <Swanson> Sure.  Is cinder the long pole?
16:34:42 <thingee> ameade: I'll defer to DuncanT and asselin on that.
16:34:52 <thingee> on how long things should be running stable
16:35:07 <asselin> ameade, here's what I would do to run your unmerged driver on all cinder patches: http://paste.openstack.org/show/196670/
16:35:18 <asselin> ameade, assuming you're using devstack-gate to setup the job
16:35:36 <thingee> DuncanT: any opinions?
16:35:38 <ameade> I'd like for us to lay out some of these requirements
16:36:04 <thingee> or asselin
16:36:18 <ameade> should i be posting on sandbox? is it ok if I just append to our other CIs? Should i post on every single patch or just the ones that pass jenkins?
16:36:44 <ameade> i know that running all 304 tests is now a requirement
16:36:57 <asselin> sandbox is good for setting up and testing. after that works, you should really be commenting on all cinder patches
16:37:06 <DuncanT> thingee: We don't have long to make the decision, so we can't make it very long. If the earliest you're setting the RC is 6th, then I'd say 7 days with a ping on channel for all breakages to say they've been noticed int hat period?
16:37:14 <zongliang> How long the driver stable?
16:37:46 <DuncanT> That gives nearly a week to sort out any issues
16:37:54 <thingee> DuncanT: 7 days? hmm...how far along are you ameade ?
16:38:16 <ameade> well i specifically pretty much have it done
16:38:31 <ameade> scaling it to all patches is my particular challenge
16:38:34 <ameade> with limited hardware
16:39:18 <zongliang> How to test the driver is stable?according to the reporting?
16:39:51 <thingee> I would like to propose five days of the CI being stable on posting results. I think anyone who is going to make this exception window should be pretty much there.
16:39:53 <thingee> by now
16:40:03 <DuncanT> ameade: There's been a general consensious that running after a jenkins +1 is reasonable
16:40:17 <thingee> DuncanT: I think asselin and you would conflict there.
16:40:30 <thingee> but yes there are CI's doing this today
16:40:34 <ameade> >.<
16:40:36 <DuncanT> asselin: Oh?
16:40:50 <asselin> I've no objection to running after jenkins +1
16:40:51 <thingee> so here are my thoughts on that...
16:41:04 <jungleboyj> thingee: +1 Seems reasonable.
16:41:04 <thingee> I think this is really great progress we have made on all vendors
16:41:14 <thingee> lets not ruin that
16:41:47 <thingee> if people having scaling issues, lets allow it and once we got things more figured out/stable lets revisit the idea of being proactive on all patches
16:41:49 <zongliang> Ok,
16:42:17 <ameade> thingee: that would be great, we have more hardware coming mid april
16:42:28 <ameade> and i think we get 99% of the value just running on most patches
16:43:02 <thingee> ameade: I was impressed netapp was already addressing scaling issues, but I can certainly say some people are running on some pretty "interesting" setups to meet the requirement
16:43:27 <ameade> thingee: we definitely went all in with CI
16:43:58 <ameade> so my plan is to have this thing run silently for a day or two more
16:44:06 <thingee> #agreed 4/6 cinder community will verify CI's for exception
16:44:23 <thingee> #agreed CI's for exception must be stable/reporting for 5 days before 4/6
16:44:35 <thingee> #action thingee to communicate to ML about this
16:45:15 <xyang2> thingee: including weekends so starting at 4/1?
16:45:15 <thingee> #action thingee to do reminder in next cinder meeting
16:45:27 <thingee> xyang2: yes =/
16:45:30 <jungleboyj> Who all is hoping for an exception?
16:45:44 <ameade> jungleboyj: <--this guy
16:45:46 <ameade> i mean
16:45:49 <ameade> <--this guy
16:45:52 <ameade> lol
16:45:55 <thingee> jungleboyj: huawei, netapp, oracle have a good chance from the sound of things
16:46:16 <thingee> the rest of the drivers have no been communicating anything on the ML
16:46:18 <jungleboyj> :-)
16:46:19 <thingee> not*
16:46:21 <zongliang> Thanks
16:46:24 <Liu> thingee: thanks:)
16:46:35 <hemna> jungleboyj, I think huawei is
16:46:46 <jungleboyj> Ok, that was what I expected.  Wnated to make sure there weren't others.
16:46:47 <zongliang> We will do our best
16:46:58 <ameade> I appreciate it
16:46:58 <xyang2> thingee: are there also a few windows drivers too?
16:47:03 <thingee> that said, the community and specifically asselin has been responding to people on the ML about open issues on their CI's
16:47:21 <jungleboyj> thingee: I would like to note that you are being more than considerate with this exception.
16:47:33 <thingee> xyang2: I heard they would like to get exceptions on those. haven't seen ci's reporting yet or communication on the ML
16:47:41 <zongliang> May be we met some network problem,we will fix
16:47:50 <thingee> jungleboyj: yes
16:49:13 <thingee> To be clear, I didn't have to grant anyone exceptions on Kilo RC. I know people are angry with me, but regardless I saw this was an opportunity to continue to help people that were removed.
16:49:48 <hemna> thingee, +1
16:49:54 <ameade> thingee: what seems to be the important thing is we know the drivers work lol
16:49:54 <thingee> regardless I'm very happy with progress we're all making
16:49:59 <zongliang> +1
16:50:01 <jungleboyj> Just to be clear, drivers have to be in on 4/6.  We will continue to accept bugs until we cut for RC.
16:50:21 <jungleboyj> Should we only merge targetted bug fixes?
16:50:22 <thingee> and it's all thanks to driver maintainers getting their ci's ready to better openstack
16:50:41 <thingee> and then people like jgriffith patrickeast and asselin helping others with their ci's
16:50:47 <jungleboyj> thingee: +2
16:50:58 <DuncanT> jungleboyj: Certainly you should -1 any bug fixes you think are high risk
16:51:13 <jungleboyj> DuncanT +1
16:51:15 <thingee> and i think we should all recognize asselin and anteaya for their hard work of the third party ci meetings and helping in infra channels with any issues people hit
16:51:35 <thingee> also thanks to DuncanT for help in the channel with CI questions and being our liason as well!
16:52:04 <jordanP> yep !
16:52:53 <thingee> ok anything else?
16:53:07 <thingee> thingee: nope
16:53:11 <xyang2> thingee: for liberty,
16:53:23 <xyang2> thingee: do we require driver to have CI before merge?
16:53:30 <thingee> xyang2: good question
16:53:31 <xyang2> thingee: by Liberty-1?
16:53:37 <thingee> my thoughts are yes
16:53:43 <jungleboyj> thingee: +1
16:53:44 <thingee> wanted to poll others what they thought
16:53:47 <ameade> +1
16:53:55 <patrickeast> we should... no point going through the stress of another deadline like this
16:53:56 <e0ne> +1
16:54:06 <xyang2> I think it is better this way. so we don't have to remove them later
16:54:06 <asselin> +1
16:54:14 <thingee> xyang2: agreed
16:54:19 <jungleboyj> Agreed.  Now that we have enforced the requirement, lets not do this again.
16:54:22 <Swanson> I'd do new drivers the same way as kilo.  Takes time to move big companies.
16:54:25 <jungleboyj> Havea  deadline.
16:54:34 <thingee> I'd like to avoid another ML thread like this later
16:54:59 <xyang2> Swanson: I think we should not even merge drivers without CI, so we don't have to remove them
16:55:01 <DuncanT> The thing is, we don't need to alter our plans for big companies... we just say no
16:55:12 <thingee> #agreed in liberty, volume drivers are required to have a CI *before* merge
16:55:27 <thingee> #action thingee to update wiki with this information
16:55:41 <thingee> #action thingee to make sure to send in beginning of L about volume driver plans
16:55:50 <thingee> what about target drivers?
16:55:52 <thingee> hemna: ^
16:55:54 <hemna> heh
16:56:20 <hemna> I think we need to talk about the plan to CI target drivers as well as the Fibre Channel Zone manager drivers (2 of them) in Vancouver
16:56:20 <thingee> Target drivers are already setting up CI's for K-3...didn't even have to ask people :)
16:56:31 <thingee> hemna: yes good ideas
16:56:38 <thingee> hemna: anyone from brocade showing up?
16:56:44 <hemna> I think brocade has said they are starting to look at their driver
16:56:49 <thingee> hemna: also is there a proposed session?
16:56:52 <hemna> but I don't know the status.  haven't heard from Cisco at all.
16:57:02 * asselin had a meeting with brocade on ci
16:57:11 <thingee> hemna: I don't think this needs to be in the fishbowl sessions. Doesn't need user input
16:57:21 <hemna> thingee, agreed
16:57:29 <thingee> asselin: care to share?
16:57:34 <hemna> I have also toyed with the idea of taking the FCZM out of cinder itself
16:57:37 <hemna> and make it a lib.
16:57:38 <hemna> dunno
16:57:47 <thingee> hemna: brocade has started to look at their driver.... OK that's a start!
16:57:55 <asselin> just gave them an overview.what it is, why, etc.
16:58:19 <e0ne> 2min reminder
16:58:22 <thingee> asselin: well received in your opinion?
16:58:29 <asselin> thingee, yes
16:58:32 <thingee> great
16:58:41 <jungleboyj> That is good news.
16:58:47 <thingee> progress!
16:58:55 <thingee> #topic open discussion
16:59:07 <Liu> Hi all
16:59:11 <Liu> For the drivers have been removed now, can I change the stack.sh script to add the removed driver before start c-volume in the CI?
16:59:33 <Liu> to test against the real storage backend
16:59:41 <asselin> Liu, yes
17:00:01 <DuncanT> Liu: http://paste.openstack.org/show/196670/ was recommended earlier
17:00:03 <thingee> thanks everyone
17:00:04 <asselin> Liu, I posted this earlier which is one way to do that: http://paste.openstack.org/show/196670/
17:00:06 <thingee> #endmeeting