14:59:53 <scottda> #startmeeting cinder_testing 14:59:53 <openstack> Meeting started Wed Aug 10 14:59:53 2016 UTC and is due to finish in 60 minutes. The chair is scottda. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:59:54 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:59:56 <cFouts> o/ 14:59:56 <openstack> The meeting name has been set to 'cinder_testing' 14:59:57 <ntpttr__> o/ 15:00:00 <smcginnis> scottda: Shoot, I still have a todo from the last meeting I haven't done. Sorry. 15:00:04 <xyang> hi 15:00:09 <patrickeast> Hi 15:00:23 <eharney> hi 15:00:26 <DuncanT> Hi 15:00:58 <_alastor_> hi 15:01:10 <akerr> o/ 15:01:12 <scottda> smcginnis: That's ok, since you haven't put up a devref patch to define the tests, we can Discuss the definitions some more today!! 15:01:22 <smcginnis> scottda: ;) 15:01:23 <scottda> I'm kidding, of course. 15:01:32 <scottda> #link http://gorka.eguileor.com/manual-validation-of-cinder-aa-patches/ 15:01:53 <scottda> I've been playing with geguileo 's manual tests for HA-ActiveActive ^^^ 15:02:24 <scottda> Good stuff there, if anyone else wants to play around with it... 15:02:58 <scottda> I intend to start scripting some of that, probably today. I'll post a link to github when I'm going... 15:02:59 <openstackgerrit> Tom Swanson proposed openstack/cinder: Dell SC: Requests async header is invalid https://review.openstack.org/353551 15:03:08 <e0ne> hi 15:03:36 <scottda> e0ne: I think this week's Thursday QA meeing is friendly to your TimeZone? Can you attend? 15:03:53 <tommylikehu> time to start 15:04:03 <e0ne> scottda: yes, I'm going to addend it tomorrow 15:04:17 <Swanson> Hi All. Requests updated to 2.11.0 and smote my driver. https://review.openstack.org/353551 adds two 's to fix it. Quick review? 15:04:21 <scottda> e0ne: Looks like 0900 UTC 15:05:56 <scottda> e0ne: It would be great if you could get some QA core eyes on this for adding multi-backend list to tempest.conf: https://review.openstack.org/#/c/325895 15:06:14 <scottda> That has one core +2, so hopefully it is close/ready 15:06:14 <e0ne> scottda: ok. added to my list 15:06:39 <jbernard> i would like to raise awareness of rbd replication patch https://review.openstack.org/333565 15:06:51 <jbernard> it's not quite done yet, but the big pieces are there 15:07:03 <scottda> We 15:07:07 <scottda> We 15:07:18 <jbernard> any feedback and/or testing is much appreciated 15:07:33 <scottda> We've no real agenda for today...does anyone have anything about cinder testing to discuss? 15:07:55 <geguileo> jbernard: Any instructions on how to easily test it? 15:08:43 <geguileo> scottda: I will be updating the HA A/A patches later today, as I've seen a problem with queueing of retype and migrate 15:08:45 <jbernard> geguileo: I will get those posted 15:09:04 <e0ne> jbernard: what ceph version is required for this feature? 15:09:18 <geguileo> scottda: I'll update the .conf file in my site to point to the latest patch chain 15:09:38 <jbernard> e0ne: >= jewel 15:09:52 <scottda> geguileo: Ok. I'll look for the updates and re-test 15:09:56 <patrickeast> Maybe something of interest, I added an experimental pipeline for my ci so if you comment "run Pure Storage experimental" it will run with multi-node active active using a cluster of c-vols 15:10:09 <smcginnis> patrickeast: Getting all fancy now, huh? :) 15:10:15 <eharney> patrickeast: nice 15:10:26 <patrickeast> Smcginnis: haha something like that 15:10:38 <scottda> #info patrickeast added an experimental pipeline for my ci so if you comment "run Pure Storage experimental" it will run with multi-node active active using a cluster of c-vols 15:10:52 <xyang> patrickeast: how many c-vol nodes are in your cluster 15:10:54 <e0ne> jbernard: noted, thanks 15:11:03 <patrickeast> xyang: just two 15:11:06 <jbernard> e0ne: thanks 15:11:38 <scottda> patrickeast: Do you know offhand the status of patches to get multi-node into the upstream gate? 15:12:37 <patrickeast> Scottda: not really, there are some changes I had to make (linked on the testing etherpad) 15:12:45 <scottda> patrickeast: Does you experimental job pull in AA/HA from master, or does it grap some later patchset from in-review queue? 15:13:08 <nikesm> hi, 15:13:29 <patrickeast> scottda: it uses whatever is up for review, so to get the full effect run it on the last patch of the ha series 15:13:45 <scottda> patrickeast: Cool. 15:14:14 <patrickeast> Although my ci is borked right now... So wait like a day to try it out :p 15:14:20 <scottda> haha 15:14:22 <nikesm> self.backend_name = self.configuration.safe_get('volume_backend_name') what will be the value of self.backend_name if volume_backend_name is not defined in cinder.conf 15:14:31 <ntpttr__> I have a quick question about testing rolling upgrades - if I had OpenStack running with say two of every cinder service, and upgraded them one at a time from mitaka to master running tempest in between each upgrade would that be good validation? Starting with the db upgrade as well 15:14:53 <ntpttr__> or is there any other step that would need to be done there 15:15:46 <DuncanT> ntpttr__: That sounds reasonable to me. You might need something more than tempest runs, since tempest cleans everything up between runs - something simple that creates some volumes, leaves some attached during the upgrade, etc 15:16:08 <ntpttr__> DuncanT: ah good idea, I can do that manually 15:16:20 <ntpttr__> or write some sort of script 15:16:47 <scottda> Does grenade do anything like that? create and attach volumes, then upgrade, and see if all is well? 15:16:54 <DuncanT> I always recommend a script, you rarely end up doing such tests once. It need not be complicated though 15:17:16 <DuncanT> No idea about grenade, not gotten into it, sorry 15:18:27 <scottda> ntpttr__: dulek may have more detailed test scenarios for rolling upgrade 15:18:46 <scottda> but I don't see him here. 15:18:47 <ntpttr__> scottda: do you know the status of the grenade tests right now? There's a patch up to get those in right 15:19:33 <ntpttr__> scottda: yeah, I'm going to be working with him some on this, I think I'm going to have a meeting with him on the topic sometime soon 15:19:41 <scottda> ntpttr__: grenade for rolling upgrades? I'm not sure 15:20:36 <scottda> I think the controversy is about how the test matrix can easily explode with something like rolling upgrades 15:22:19 <scottda> Other test things to discuss? 15:22:22 <DuncanT> I think what we need is a thorough test set we can run on demand, like the experimental queue 15:22:52 <DuncanT> It isn't reasonable to try to get good, detailed upgrade coverage in the normal gate 15:23:05 <ntpttr__> DuncanT: +1 15:23:08 <scottda> DuncanT: Agreed. And perhaps something that must be done in a local lab, if the tests take too long or take too many resources. 15:24:30 <DuncanT> Certainly it should be easier to start that way 15:25:55 <scottda> ntpttr__: Would you consider documenting your (and dulek 's ) test scenarios and samples? It'd be nice to have even the manual steps you are doing documented. 15:26:05 <openstackgerrit> Tom Swanson proposed openstack/cinder: Dell SC: delete_live_volume payload incorrect https://review.openstack.org/350262 15:26:19 <ntpttr__> scottda: yeah, I'd be happy to 15:26:39 <scottda> #action ntpttr__ will document manual tests and examples for rolling upgrades 15:26:41 <ntpttr__> I'm still thinking about what deployment solution would be best for this sort of testing 15:27:20 <ntpttr__> we're looking at kolla because it has upgrades playbooks, but we'd have to hack it a bit to get it to stop after upgrading services to run tempest and do other tests 15:27:25 <openstackgerrit> Tom Swanson proposed openstack/cinder: WIP: Dell SC: Live Volume Autofailover https://review.openstack.org/350782 15:27:59 <DuncanT> ntpttr__: I had some good luck with containers. The nice thing is, you only need to build the HEAD one, the rest can be (and even are better if they are) the same ones from previously 15:28:07 <Swanson> xyang, thanks! 15:28:23 <xyang> Swanson: np 15:28:32 <DuncanT> ntpttr__: I wasn't using kolla but it is what I'd look at now 15:28:58 <ntpttr__> DuncanT: cool, I'll give it a try and keep notes on how it goes 15:29:56 <scottda> What else, folks? 15:31:18 <scottda> We can be done, then. thanks everyone. 15:31:26 <scottda> #endmeeting