17:00:15 <ildikov> #startmeeting cinder-nova-api-changes
17:00:16 <openstack> Meeting started Mon Jun 20 17:00:15 2016 UTC and is due to finish in 60 minutes.  The chair is ildikov. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:20 <openstack> The meeting name has been set to 'cinder_nova_api_changes'
17:00:25 <ildikov> scottda ildikov DuncanT ameade cFouts johnthetubaguy jaypipes takashin alaski e0ne jgriffith tbarron andrearosa hemna erlon mriedem gouthamr ebalduf patrickeast smcginnis diablo_rojo gsilvis
17:00:31 <mriedem> o/
17:00:40 <hemna> yough
17:00:42 <DuncanT> Hi
17:01:32 <ildikov> hi all :)
17:02:13 <ildikov> for today I would like to check on the refactoring and testing work
17:02:52 <ildikov> does anyone have any further topics for today?
17:03:15 <mriedem> just a reminder that nova's non-priority bp freeze is 6/30
17:04:07 <mriedem> i just reviewed jgriffith's nova change
17:04:10 <ildikov> mriedem: I know, although as the above mentioned items are dependencies I'm less sure we're gonna make it
17:04:13 <mriedem> https://review.openstack.org/#/c/330285/
17:04:28 <mriedem> comments are inline. the dependent cinder change needs a rebase.
17:04:41 <mriedem> i didn't check previous patch sets in jenkins to see if they passed a tempest run
17:04:47 <ildikov> mriedem: what's up with those three items that ended up high prio items but they were expected to get finished before the deadline?
17:05:11 <mriedem> glance v2 is the only priority item that nova has completed already
17:05:23 <hemna> haven't had a chance to look at the nova patch yet
17:05:28 <mriedem> https://specs.openstack.org/openstack/nova-specs/priorities/newton-priorities.html
17:05:28 * smcginnis is here but not really here
17:05:58 <mriedem> so, the nova change didn't ever pass jenkins,
17:06:08 <mriedem> zuul failed to merge the series for the job run
17:06:18 <mriedem> which is something i was hitting last week with my large get-me-a-network series,
17:06:36 <mriedem> rechecking in order usually gets it going, but the cinder dependency for the new APIs is a legit merge conflict
17:06:51 <mriedem> which is this one https://review.openstack.org/#/c/327408/
17:07:02 <hemna> the cross project deps never seem to work from what I've seen
17:07:03 <ildikov> mriedem: can multiattach come up and considered as some extent high priority?
17:07:23 <mriedem> i'd rather not get into that right now
17:07:29 <hemna> ildikov, I think multi-attach is dead unless we get the new cinder API in place no?
17:07:34 <ildikov> I have an issue with my Devstack env, I wanted to try the patches in action
17:07:51 <ildikov> hemna: right, I meant it with its dependencies
17:08:10 <mriedem> so for jgriffith's series we obviously need the api change rebased
17:08:21 <mriedem> then we can recheck the nova change and see if it passes a jenkins/tempest run
17:08:33 <hemna> oops yah the cinder patch is merge conflict
17:08:39 <mriedem> if jgriffith isn't around, can someone do that?
17:08:55 <hemna> I can ping him today
17:08:58 <smcginnis> I reminded him.
17:08:59 <hemna> and see if he's around
17:09:01 <mriedem> looks like there were some questions in ps3 on the cinder patch that went unanswered also
17:09:11 <jgriffith> mriedem: I'm back :)
17:09:18 <jgriffith> mriedem: I'll get on it right now... thanks smcginnis
17:09:21 <smcginnis> jgriffith: Hey John!
17:09:27 <jgriffith> smcginnis: 0/
17:10:51 <ildikov> jgriffith: we were discussing the Cinder API change patches and got to the merge conflict just now
17:12:02 <mriedem> right, so next steps are just getting that rebased and have a ci run through it
17:12:09 <mriedem> jgriffith: i have comments in the nova change too
17:12:28 <jgriffith> mriedem: :)  Yeah, I just sent you a thank you for that on dev channel
17:12:37 <jgriffith> mriedem: I'll address those items and rebase later today
17:12:41 <jgriffith> mriedem: Thank you!!!!
17:12:45 <mriedem> sure :)
17:12:51 <jgriffith> I've been hoping for some feedback
17:13:17 <mriedem> ildikov: what's next?
17:13:31 * mriedem is hungry
17:13:45 <ildikov> I don't know whether we have scottda around to see where we are with the tempest tests
17:14:16 <mriedem> https://review.openstack.org/#/c/326681/
17:14:19 <smcginnis> I got the impression he made some progress with that, but don't know the current state.
17:14:40 <hemna> jenkins isn't happy w/ it
17:15:04 <mriedem> has 2 deps
17:15:08 <ildikov> yeah I saw both
17:15:10 <mriedem> the devstack one has some +1s https://review.openstack.org/#/c/325895/
17:16:14 <mriedem> the other is a project-config change for a new job...
17:16:51 <mriedem> i'm not totally sure why a new job is needed for multibackend
17:17:03 <mriedem> can't cinder run normally in the gate with 2 lvm backends as a default in devstack?
17:18:24 <hemna> looks like the job config is setting up 2 lvm backends
17:18:37 <hemna> which isn't the default afaik
17:19:24 <mriedem> can you get the list of backends via the cinder API?
17:19:30 <mriedem> so it doesn't need to be configured in tempest?
17:20:05 <hemna> service-list might give you the backends
17:20:09 <hemna> or pools
17:20:58 <mriedem> so he's using this backend_names variable in tempest which already exists
17:21:05 <mriedem> i wonder what jobs are testing that today
17:21:24 <hemna> http://paste.openstack.org/show/520656/
17:21:47 <hemna> doesn't give the backend_name though, just the host
17:22:15 <mriedem> regular gate doesn't test multi-backend http://logs.openstack.org/73/328673/1/check/gate-tempest-dsvm-full/d2e8a67/console.html.gz#_2016-06-12_13_28_29_392068
17:22:39 <hemna> yah that has to be enabled in tempest conf
17:22:46 <hemna> and specificaly configured afaik
17:23:06 <mriedem> i just wonder what existing job, if any, is testing this
17:23:15 <mriedem> so that we don't need https://review.openstack.org/#/c/330678/
17:24:03 <mriedem> so let's start getting some action items
17:24:18 <mriedem> #action jgriffith to cleanup and rebase nova/cinder create/remove attachment API series
17:24:37 <mriedem> #action scottda et al to figure out if there is an existing CI job that uses cinder multi-backend
17:25:51 <hemna> sounds good
17:26:09 <mriedem> -1 on the devstack change https://review.openstack.org/#/c/325895/
17:26:13 <mriedem> they are changing some existing behavior
17:26:19 <mriedem> which if there is a job that relies on those, it's going to break them
17:27:30 <hemna> I don't think there is a cinder api for fetching the backend name (cinder.conf section name)
17:28:03 <mriedem> unfortunate
17:28:19 <hemna> you'd have to extract it from the host name I think
17:28:37 <mriedem> so a rookie question,
17:28:43 <mriedem> for retype (volume migration),
17:29:13 <mriedem> can you have a single backend cinder on 2 hosts and migrate between them? or does it really have to be 2 different cinder backend drivers for the migration/retype?
17:29:30 <mriedem> and those can live on the same host
17:29:54 <hemna> a single cinder backend that has 2 different drivers?
17:30:10 <hemna> you can have volume_backend_name=foo
17:30:18 <hemna> with 2 different drivers
17:30:39 <hemna> so yes in that case
17:31:00 <hemna> 2 different solidfire arrays, 2 different driver instances, both with the same volume_backend_name defined
17:31:39 <hemna> you could migrate between them
17:31:46 <hemna> and retype for that matter
17:32:39 <mriedem> but if i have an lvm and an rbd,
17:32:42 <mriedem> then 2 different backends?
17:33:32 <hemna> sure
17:34:15 <hemna> the part that's confusing is there is a conf entry called "volume_backend_name"
17:34:24 <hemna> and some people confuse that with the section name in cinder.conf
17:34:28 <hemna> [3parfc]
17:34:32 <hemna> driver=....
17:34:41 <hemna> volume_backend_name=3parfc (or foo)
17:34:47 <hemna> the host is created from the section name
17:35:19 <hemna> the volume types use the volume_backend_name value
17:36:41 <hemna> and cinder.conf enabled_backends=[list of section names]
17:37:23 <mriedem> does gate-tempest-dsvm-full-drbd-devstack ever run on anything?
17:37:29 <hemna> not sure if that helps or answers your question
17:37:37 <mriedem> that sets CINDER_ENABLED_BACKENDS in devstack
17:38:54 <hemna> http://logs.openstack.org/85/327285/12/check/gate-tempest-dsvm-full-drbd-devstack-nv/6e0e031/
17:38:59 <hemna> I think that's a different test though
17:39:44 <mriedem> heh, and doesn't run the tests we care about http://logs.openstack.org/85/327285/12/check/gate-tempest-dsvm-full-drbd-devstack-nv/6e0e031/console.html#_2016-06-17_18_41_04_809894
17:40:10 <mriedem> even though CINDER_ENABLED_BACKENDS=drbd:drbdmanage is in http://logs.openstack.org/85/327285/12/check/gate-tempest-dsvm-full-drbd-devstack-nv/6e0e031/logs/localrc.txt.gz
17:40:42 <mriedem> anyway, this is a rabbit hole
17:40:49 <mriedem> i've posted comments in scott's changes
17:40:57 <mriedem> what he's really adding is an lvm multibackend job
17:41:01 <mriedem> to cinder's experimental queue
17:41:14 <hemna> sure
17:41:28 <hemna> we can circle back with scottda when he's around
17:42:31 <mriedem> there is a way to test this before the project-config change is merged
17:42:45 <mriedem> with a one off devstack change
17:42:53 <mriedem> that depends on the tempest change
17:43:37 <mriedem> i don't know if i have the time to tinker with that today though
17:44:46 <mriedem> anyway, anything else for today?
17:44:49 <mriedem> i'd like to end early
17:44:53 <mriedem> ildikov: ^?
17:44:53 <hemna> I don't have anything
17:45:02 <ildikov> mriedem: hemna thanks
17:45:05 <hemna> I haven't had time to look at the check_attach patchset in a week
17:45:09 <ildikov> I don't have anything more for today
17:45:12 <hemna> I'd like to get to that soon
17:45:18 <hemna> but I am training my replacements this week
17:45:20 <hemna> :(
17:45:24 <mriedem> boo
17:45:25 <hemna> and I still don't have a new job.  bleh
17:45:27 <ildikov> hemna: lemme know if you need any help with that
17:45:35 <ildikov> hemna: I have free time this week
17:45:35 <mriedem> hemna: tell those a-holes to rebase your change
17:45:43 <hemna> :)
17:45:49 <hemna> they don't even know what CI is
17:45:52 <hemna> *sigh*
17:45:59 <ildikov> :( :)
17:47:00 <mriedem> ildikov: have you started looking at what the nova changes would look like after jgriffith's changes?
17:47:02 <mriedem> with the new APIs?
17:47:05 <mriedem> i mean for multiattach
17:47:39 <ildikov> we need to clean up check_attach as well along with the API changes
17:47:44 <mriedem> because if the plan is to have multiattach build on that, you probably need to start POCing that code
17:47:49 <smcginnis> hemna: :(
17:48:05 <ildikov> besides that I think there are mostly the multiattach related changes, like the shareable flag and the version checks, etc
17:48:51 <ildikov> mriedem: I think we need to refactor the VM actions as well, like live migrate, shelve offload, etc
17:49:15 <mriedem> ildikov: as part of multiattach?
17:49:18 <mriedem> or just in general?
17:49:32 <ildikov> mriedem: johnthetubaguy walked us through those a short while ago and I think we might look into those as well
17:49:56 <ildikov> mriedem: if we don't want to disable those for multiattach volumes, then I think before
17:50:09 <ildikov> mriedem: but I might over worry this
17:51:26 <mriedem> i guess my point is, there are <10 working days left in the schedule before the bp freeze, so i think it's going to be important to figure out the work order of what needs to happen
17:51:32 <mriedem> and making priorities on what can get done
17:51:56 <ildikov> mriedem: also I don't think we have too many options besides building on this refactoring work
17:51:56 <mriedem> i don't want to start talking about exceptions when we don't have a clear vision on a plan
17:52:09 <ildikov> mriedem: can the refactoring go on after the deadline?
17:52:11 <mriedem> sure, but then does that mean that for newton, refactoring is the priority and goal
17:52:42 <mriedem> hemna's change for check_attach could be considered a bug fix
17:52:46 <mriedem> so i don't have a problem with that
17:53:04 <hemna> coolio
17:53:18 <mriedem> i'm pretty sure the nova core team isn't going to be comfortable with mass refactorings of the nova/cinder API interaction after the freeze
17:53:20 <ildikov> cool, I think we touched on this earlier as well
17:53:47 <mriedem> it can certainly be hacked on, but that doesn't mean it gets a pass
17:54:11 <ildikov> mriedem: is there any chance to discuss it on one of the upcoming meetings maybe?
17:54:20 <mriedem> which upcoming meeting?
17:54:27 <ildikov> I think this reafctoring work has much value in general not just for multiattach
17:54:50 <ildikov> I meant a Nova team meeting
17:54:53 <mriedem> i agree, but that doesn't mean we can just drop it in in n-3
17:55:01 <mriedem> feel free to bring it up in a nova meeting
17:55:05 <ildikov> or just bring it up on the channel when people around
17:55:40 <hemna> I think we'd have some momentum if the refactoring patches would pass jenkins and show the stuff works
17:55:43 <ildikov> ok, cool, I will sync up with jgriffith and then bring up the topic
17:56:04 <ildikov> hemna: +1, agreed
17:56:14 <hemna> do we have working live migration tests ?
17:56:30 <mriedem> hemna: for which thing? or just in general?
17:57:06 <mriedem> basic block migration and i think voume-backed live migration were passing in the experimental queue job last week once we disabled nfs and ceph from that job config
17:57:08 <hemna> just in general
17:57:22 <mriedem> live migration meeting is tomorrow, we were going to come back to see how that job was doing
17:57:54 <mriedem> as in, if the basic config is working, we were going to add the job as non-voting in the nova check queue
17:58:24 <hemna> ok
17:58:28 <mriedem> well we know it works, but at least with trusty versions of libvirt there were pretty sporadic failures
17:58:33 <hemna> I think that job needs to work to tests the refactoring patches
17:58:34 <mriedem> the job uses xenial now
17:58:47 <mriedem> hemna: yeah agreed
17:58:56 <hemna> I'd like to see 3rd party CI using that as well
17:59:06 <hemna> so we can confirm some segment of the cinder drivers work w/ live migration
17:59:20 <hemna> but ceph, lvm should work
17:59:28 <hemna> since those are the reference cinder backends
17:59:53 <ildikov> hemna: yeap, we will need automated tests as well, not just manual verification
17:59:55 <mriedem> yeah, we just have to fix the job, some setup broke once moving to xenial
18:00:10 <mriedem> tdurakov fixed the nfs setup but we kept it disabled for now
18:00:19 <mriedem> anyway, time is up
18:00:43 <ildikov> yeap, we have the actions points, let's move forward with those
18:00:50 <hemna> coolio
18:00:53 <hemna> thanks guys
18:00:56 <ildikov> anything urgent we haven't touched today?
18:01:21 <ildikov> thanks all!
18:01:31 <ildikov> talk in a week the latest
18:01:36 <ildikov> #endmeeting