16:00:41 <ildikov> #startmeeting cinder-nova-api-changes
16:00:41 <openstack> Meeting started Thu May 18 16:00:41 2017 UTC and is due to finish in 60 minutes.  The chair is ildikov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:42 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:45 <openstack> The meeting name has been set to 'cinder_nova_api_changes'
16:00:49 <ildikov> DuncanT ameade cFouts johnthetubaguy jaypipes takashin alaski e0ne jgriffith tbarron andrearosa hemna erlon mriedem gouthamr ebalduf patrickeast smcginnis diablo_rojo gsilvis  xyang1 raj_singh lyarwood breitz jungleboyj
16:00:50 <jungleboyj> o/
16:01:26 <ildikov> jungleboyj: hi :)
16:01:26 <stvnoyes> o/
16:01:31 <smcginnis> ./
16:01:44 <jungleboyj> ildikov: Hello.
16:01:51 <smcginnis> ildikov: Back home yet?
16:02:07 <ildikov> smcginnis: yep, landed yesterday afternoon
16:02:23 <jungleboyj> ildikov: Glad you made it safe.  Hope it was a good trip.
16:02:27 <smcginnis> ildikov: Did you remember how to get back to your apartment? :D
16:02:36 <ildikov> smcginnis: my brain is still useless though, after Summit syndrome...
16:02:38 <hemna> mep
16:02:53 <ildikov> smcginnis: luckily taxi drivers help you out if you know the address :)
16:03:00 <jungleboyj> ildikov: ++
16:03:01 <ildikov> jungleboyj: yep, it was alright
16:03:35 <ildikov> so let's get started
16:04:20 <ildikov> the main news from last week is that I've never had this many people approaching me about multi-attach ever before
16:05:02 <ildikov> a guy even told me they fixed up my two Nova patches and running it in production on Kilo
16:05:23 <jungleboyj> ildikov:  Wow.
16:05:29 <hemna> ildikov, w00t
16:05:44 <ildikov> the conclusion is that we better get ourselves together and finally finish this thing! :)
16:05:56 * johnthetubaguy wonders in a touch later than planned
16:06:02 <ildikov> at least that's my conclusion :)
16:06:31 <ildikov> johnthetubaguy: no worries, I'm just sharing my experiences from last week
16:06:40 <johnthetubaguy> yup, lets make it safe before they all do it very unsafe!
16:06:52 <johnthetubaguy> honestly its the feature most people ask me about at openstack events too, at least right now
16:07:27 <ildikov> johnthetubaguy: big +1
16:07:51 <ildikov> so that said we started to look into swap with jgriffith
16:08:00 <hemna> has anyone asked those folks what they plan on doing with it? :)
16:08:11 <hemna> and what filesystem they plan on using?
16:08:21 <stvnoyes> our customers want Oracle RAC
16:09:09 <ildikov> hemna: I don't remember what the guy said who's running this in production, but he didn't complain it didn't work
16:10:03 <ildikov> I don't think this is gonna be the first feature doing harm if you don't use it correctly
16:10:24 <hemna> yah, I'm just curious
16:10:43 <hemna> I think most folks are smart and know they can't run ext4fs on it, but some will try.
16:10:48 <ildikov> I tried to recruit everyone, so I asked them more about how much they can get involved
16:11:04 <ildikov> I will feel sorry for those ones for sure
16:11:11 <johnthetubaguy> hemna: I have this crazy clustered thingy is the usual answer I get
16:11:33 <hemna> clusterfs!
16:11:38 <hemna> *magic*
16:11:47 <johnthetubaguy> its really easy to use this feature badly, I am curious if we want a "read-only" multi-attach mode too
16:11:48 <ildikov> hemna: also there are cases where they would want to have one instance that writes on the volume and the rest just read
16:12:21 <johnthetubaguy> ildikov: not sure thats always the case, but that sounds like the most common case
16:12:32 * hemna wonders if that even works
16:12:42 <ildikov> johnthetubaguy: no, certainly not always
16:12:46 <johnthetubaguy> some stuff was a simple active/passive HA pair, that both needs storage
16:12:52 <hemna> I might try hacking the brick cinderclient extension and see if I can do a multiattach with cinder directly
16:12:59 <hemna> to see if I can test that out at all
16:13:11 <ildikov> hemna: sounds good
16:13:16 <johnthetubaguy> hemna: there is a libvirt property you want to set on the connection, to stop the caches doing funky things
16:13:27 <johnthetubaguy> hemna: I think thats in the attach WIP patch anyways
16:13:35 <hemna> ok cool.
16:13:40 <hemna> and for baremetal ?
16:13:42 <ildikov> hemna: I can rebase my old Nova patches and enable multi-attach in them again so you could even play with Nova
16:13:52 <hemna> just has to be mounted with -o ro ?
16:14:02 <smcginnis> hemna: I think so.
16:14:18 <stvnoyes> one of our guys has ma working, sorta basic. no migrate, swap, etc. keeping the connection from getting terminated was the biggest challenge, tho it wasn't much code
16:14:34 <hemna> I'm a little wary that even mounting ext4fs in ro mode will try induce a write somehow.
16:14:42 <johnthetubaguy> are we getting distracted?
16:14:42 <hemna> anyway, ignore me.
16:14:46 <stvnoyes> this was not using the new v3 api. just a proto for now. a learning experience.
16:15:18 <johnthetubaguy> we have a bunch of patches pending with my +2 on them, the others all looked WIP right now, did I just miss them?
16:16:43 <johnthetubaguy> oh wait there was a think about ildikov looking at swap?
16:16:49 <johnthetubaguy> how did that go?
16:16:51 <ildikov> johnthetubaguy: no, you didn't
16:16:52 <stvnoyes> i'd like someone to take a look at this rv -https://review.openstack.org/#/c/463987/ - just to let me know if i'm on the right direction.
16:17:03 <ildikov> johnthetubaguy: stvnoyes works on the live migrate snippets
16:17:27 <ildikov> johnthetubaguy: with jgriffith we got to the conclusion of re-writing swap
16:17:30 <johnthetubaguy> stvnoyes: I can take a peak, there was a live-migrate one I voted on recently
16:17:45 <stvnoyes> great thx
16:18:08 <johnthetubaguy> ildikov: thats probably for the best, happy to have a new Nova API if that makes it easier, or at least, I am open to the idea
16:18:33 <ildikov> I think the idea is to let Nova do the attach and detach
16:19:01 <ildikov> and have things there in a clean way as opposed to the current call-back
16:19:15 <johnthetubaguy> Nova does do the attach and detach today
16:19:33 <johnthetubaguy> oh... API wise
16:19:54 <ildikov> there are some attach/detach calls on the Cinder side today too for some corner cases I think
16:19:57 <johnthetubaguy> I think I see what you mean now
16:20:03 <ildikov> that code is a bit confusing...
16:20:10 <johnthetubaguy> nova does detach, then does the call back?
16:20:23 <jungleboyj> Sorry, was pulled off on other things.  The Read Only use case was asked for by several people at the Summit.
16:20:25 <ildikov> I think Nova does terminate
16:20:26 <johnthetubaguy> the call back does the crazy volume uuid rename nonsense
16:20:35 <ildikov> yeah
16:20:41 <johnthetubaguy> jungleboyj: I kept hearing that too
16:20:42 <ildikov> that's the main part
16:20:52 <johnthetubaguy> ildikov: I like the sound of your plan
16:21:08 <jungleboyj> johnthetubaguy:  Ok, so not just me.  :-)
16:21:12 <johnthetubaguy> stvnoyes: can I ask about attachment_ids in your live-migrate patch
16:21:40 <johnthetubaguy> jungleboyj: if I were an operator again, I would only enable the read-only version, basically attach a snapshot or somesuch
16:22:01 <stvnoyes> johnthetubaguy: sure
16:22:10 <ildikov> johnthetubaguy: we can move on from swap, I will sync up with jgriffith as soon as I can regarding where he is with the code changes he started
16:22:11 <johnthetubaguy> stvnoyes: we have a new attachment_id after the attach call in the setup I guess, in pre_live_migration
16:22:23 <johnthetubaguy> on the destination host, basically
16:22:33 <jungleboyj> johnthetubaguy: ++
16:22:47 <johnthetubaguy> but when in the tidy up we do the _detach call... we need the old attachment_id
16:22:56 <johnthetubaguy> it doesn't seem like your patch is doing that
16:23:05 <johnthetubaguy> but I might be missing some magic
16:23:10 <stvnoyes> ah good catch, i'll take a look at that
16:24:22 <stvnoyes> we don't delete the first attachment until the migrate is successful, so the orig bdm is still around. but i'll verify
16:24:22 <johnthetubaguy> do we want a destination_attachment_id in the BDM?
16:24:26 <johnthetubaguy> that would make it clear
16:24:46 <johnthetubaguy> not sure what mriedman would think about that mind
16:24:48 <ildikov> I wanted to ask whether we need to store two attachment_id's
16:25:19 <johnthetubaguy> I think we have to
16:25:31 <johnthetubaguy> I mean we can go look it up, but that seems stupid
16:25:53 <ildikov> you mean storing or not storing it?
16:26:11 <stvnoyes> i would like to ask if we could get attach with v3 cinder in the code, but disabled. then as we develop these bits we can emable it in a dev env to test it through.
16:26:34 <smcginnis> stvnoyes: Like a temporary flag to switch?
16:26:40 <johnthetubaguy> stvnoyes: I think we should do that, get ildikov to rebase her patch?
16:26:50 <stvnoyes> maybe or a config option. or even disabled in source.
16:27:06 <johnthetubaguy> why not just cherry-pick the attach patch on top for testing?
16:27:08 <ildikov> johnthetubaguy: I did the rebase, just couldn't get there yet to spin it up in Devstack and test it
16:27:23 <johnthetubaguy> ildikov: what did the gate say about the rebase?
16:27:43 <ildikov> johnthetubaguy: making the unit tests work requires more work with the microversions and I certainly didn't want to get deep in that just to be able to test some stuff
16:28:44 <ildikov> johnthetubaguy: I know we will have to, I just would like to go a bit step by step here and try to get something working for real and then play with the microversion discovery and unit tests
16:28:53 <johnthetubaguy> ildikov: I was thinking the functional stuff shoudl run OK
16:29:01 <johnthetubaguy> ah, gotcha
16:29:09 <ildikov> I know, I'm evil, but just for practicality...
16:29:39 <ildikov> johnthetubaguy: I will get myself back on track with that patch and will ping people who're involved, when it seems working
16:30:02 <johnthetubaguy> all good
16:30:04 <ildikov> a ton of things have changed, since jgriffith and me made that one work for the last time... :(
16:31:34 <johnthetubaguy> stvnoyes: so I added comments on your live-migrate patch
16:31:49 <johnthetubaguy> stvnoyes: do ping me for questions, I know that flow fairly well at the moment
16:31:52 <stvnoyes> thx
16:32:10 <johnthetubaguy> I think we should be creating a new attachment in the pre-live-migration stuff
16:32:16 <johnthetubaguy> before we do that first attach call
16:32:31 <ildikov> johnthetubaguy: did you add a note about whether or not store the old attachment_id in the BDM as well?
16:32:42 <johnthetubaguy> yeah, I said please store it in the BDM
16:33:12 <johnthetubaguy> I think you might be able to hide it without needing a db migration, but I would be tempted to make it a top level item and be done with it
16:33:30 <johnthetubaguy> anyways, you are totally in the correct bit of code there
16:35:29 <ildikov> cool, tnx, just would want to have mriedem's eyes on that part as well before ending up in rabbit holes
16:36:00 <johnthetubaguy> anyways, stvnoyes, I would think about being explcit about which of the two attachments you use in each case, and for each place we touch a volume in the flow, I think that will shake out more of the details
16:36:16 <johnthetubaguy> I suspect a lot of... bugs to be found in that code, FWIW
16:36:42 <smcginnis> Undocumented features.
16:37:04 <jungleboyj> smcginnis:   He he he.
16:37:15 <ildikov> johnthetubaguy: I also wonder whether we can get rid of the second use of initialize_connection there
16:37:37 <johnthetubaguy> we do that twice? yeah, probably
16:37:51 * johnthetubaguy nods at smcginnis
16:38:02 <ildikov> I mean we should, just cannot see that far as of yet on when we get there and how that will look like exactly... :)
16:38:15 <ildikov> johnthetubaguy: yeah, it's used in that flow a couple of times
16:38:47 <johnthetubaguy> for live-migration always thinking about plumbing really
16:39:16 <johnthetubaguy> you know a hosepipe with one of those shut off values, so when you pull ou the VM you don't get water everywhere, or something
16:39:21 <johnthetubaguy> my head is a strange place at times
16:40:06 <johnthetubaguy> sounds like everyone has their next steps for the comming week
16:40:11 <johnthetubaguy> or did we miss something?
16:40:41 <stvnoyes> johnthetubaguy: kk will do
16:40:45 <ildikov> I think we should be fine
16:41:17 <ildikov> I haven't done my homework so will not finger point at others either
16:41:26 <johnthetubaguy> thats all good
16:41:28 <ildikov> thanks stvnoyes for taking care of live migrate!
16:42:13 <johnthetubaguy> its a very scary part of the code, its good to see that starting
16:42:14 <ildikov> johnthetubaguy: I will try to ping lyarwood to take a look at those patches with +2's and get them merged sooner rather than later
16:42:36 <johnthetubaguy> cool, more eyes on those would be good
16:42:45 <ildikov> agreed
16:42:50 <stvnoyes> is this rv still needed. i noticed after i posted mine. i think it
16:43:02 <johnthetubaguy> sfinucan might be able to +2 them, its worth asking him
16:43:03 <stvnoyes> s redunandant now - https://review.openstack.org/#/c/456988/
16:43:24 <johnthetubaguy> ah, thats the one I head in my head from early in the week
16:43:27 <johnthetubaguy> yeah
16:43:32 <stvnoyes> (not typing well, hand in sling)
16:44:05 <ildikov> stvnoyes: if you cover everything in your WIP, I'll just abandon mine
16:44:12 <stvnoyes> kk
16:44:43 <johnthetubaguy> yeah, I think stvnoyes has got it covered
16:46:19 <ildikov> stvnoyes: abandoned now, sorry for the confusion
16:47:12 <johnthetubaguy> cool, are we all done for now?
16:47:18 <stvnoyes> ildikov: np, i should have noticed it before
16:47:44 <ildikov> stvnoyes: it's ok, it was easy to take care of :)
16:48:01 <ildikov> I don't have more for today
16:48:31 <ildikov> the main focus is live migrate with detach, re-write swap and make attach PoC work again with Cinder v3 to be able to test
16:48:54 <ildikov> and as the last bit of the summary get the patches with +2 merged
16:49:00 <ildikov> did I miss anything?
16:49:21 <johnthetubaguy> sounds good
16:49:47 <ildikov> johnthetubaguy: cool, tnx
16:50:11 <ildikov> does anyone have questions to any of the above items or else?
16:51:45 <ildikov> ok, I think we're good for today :)
16:52:04 <ildikov> thank you all!
16:52:29 <ildikov> #endmeeting