09:00:12 <anteaya> #startmeeting nova-net-to-neutron-migration 09:00:13 <openstack> Meeting started Tue Jan 20 09:00:12 2015 UTC and is due to finish in 60 minutes. The chair is anteaya. Information about MeetBot at http://wiki.debian.org/MeetBot. 09:00:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 09:00:17 <openstack> The meeting name has been set to 'nova_net_to_neutron_migration' 09:00:20 <anteaya> hello 09:00:24 <jlibosva> hi 09:00:26 <anteaya> who is here? 09:00:27 <belmoreira> hi 09:00:30 <obondarev> good evening/morning everybody :) 09:00:36 <anteaya> hello jlibosva belmoreira and obondarev 09:00:40 <anteaya> :D 09:00:48 <anteaya> gus: are you about? 09:00:49 <gus> hi 09:00:55 <anteaya> there you are 09:01:10 <gus> yep. hotel wifi appears to be working :) 09:01:14 <anteaya> markmcclain is/was flying so won't be able to join us 09:01:17 <anteaya> gus: yay 09:01:23 <anteaya> okay let's get started 09:02:05 <anteaya> here is our agenda 09:02:08 <anteaya> #link https://wiki.openstack.org/wiki/Meetings/Nova-nettoNeutronMigration 09:02:20 <anteaya> #topic the state of the Neutron spec (obondarev) 09:02:29 <anteaya> obondarev: you're up 09:02:35 <obondarev> yep 09:02:40 <anteaya> how are we doing on the specs 09:02:44 <obondarev> so this week we had a progress on the specs 09:02:48 <anteaya> yay! 09:02:53 <obondarev> thanks to gus who uploaded new revisions representing the result of discussions at lca 09:03:00 <anteaya> thanks gus 09:03:21 <obondarev> so we now have two specs in neutron 09:03:26 <obondarev> https://review.openstack.org/#/c/147723/ 09:03:29 <obondarev> https://review.openstack.org/#/c/142456/ 09:03:40 <anteaya> #link https://review.openstack.org/#/c/147723/ 09:03:47 <anteaya> #link https://review.openstack.org/#/c/142456/ 09:03:54 <obondarev> the first one basically sets expectations for the proposed way to migrate OpenStack deployments from nova-network to neutron 09:04:05 <obondarev> it describes the end-user/deployer impact, starting/end points of it 09:04:19 <anteaya> this was something jogo really wanted 09:04:20 <belmoreira> I add some comments to understand to migration when using cells 09:04:25 <obondarev> the second is a follow-up which adds the description/steps of the overall proposed process 09:04:27 <anteaya> belmoreira: great thank you 09:04:29 <obondarev> and major work items required 09:04:35 <anteaya> wonderful 09:04:42 <anteaya> thanks for that summary obondarev 09:04:42 <gus> they're two changes building on the same spec doc. so we'll have one spec file when they're both submitted. 09:04:43 <obondarev> yesterday I updated the specs, addressing more comments and adding some details 09:04:55 <anteaya> obondarev: great, thank you 09:04:57 <obondarev> currently we have +2 from markmcclain on both of them 09:05:04 <obondarev> however still need reviews from nova folks 09:05:04 <anteaya> that is a good sign 09:05:10 <anteaya> obondarev: yes 09:05:29 <anteaya> let's take a look at the first one, 147723 09:06:08 <anteaya> belmoreira: have you had a chance to take a look at 147723 yet? 09:06:40 <belmoreira> yes, I look into it 09:06:43 <anteaya> gus: and how are you feeling about obondarev's latest patchset to that chanage 09:06:57 <anteaya> belmoreira: any comments for 147723? 09:07:03 <belmoreira> and then I got confused where to put my comments... 09:07:10 <anteaya> ah okay 09:07:26 <anteaya> belmoreira: are you still confused? 09:07:36 <belmoreira> the comments that I have for 147723 are the same that I put in 142456 09:07:36 <gus> anteaya: the latest revision looks just fine to me - I was just now rolling it into my local git tree.. 09:07:45 <anteaya> gus: great 09:08:29 <anteaya> belmoreira: are you able to vote on 147723 separately from 142456? 09:08:49 <gus> belmoreira: this is re cells? 09:09:14 <gus> belmoreira: you're right that we should state something regarding cells in 147723. 09:09:27 <belmoreira> gus: yes, I think the specs should consider the cells use case 09:09:59 <obondarev> so as per markmcclain reply we're not going to support cells/multihost cases in the first iteration 09:10:00 <belmoreira> anteaya: yes 09:10:12 <gus> The discussion we had at LCA was (iirc): we believe this approach should work with cells (by rolling across all machines one-by-one, not necessarily cell-by-cell). 09:10:16 <obondarev> trying to keep sings simple initially 09:10:28 <gus> .. but the intention was to get non-cell case working first. 09:11:07 <belmoreira> gus: I imagine that the migration will affect mostly large deployments 09:11:15 <belmoreira> gus: and those are using cells 09:11:23 <obondarev> ultimately we should support cells migration for sure 09:11:25 <gus> if it is important to be able to completely upgrade a cell before moving on to the rest (ie: use neutron l3 nodes, and make neutron api read-write) then that will be difficult.. 09:12:09 <gus> belmoreira: agreed re importance of cell use-case. 09:12:24 <belmoreira> gus: also, I think that cells are great to isolate possible problems and do a step by step migration 09:13:05 <gus> my hesitation is how you would make neutron API available to users cell-by-cell. 09:13:28 <obondarev> neutron doesn't support cells at the moment.. 09:15:04 <gus> right. so given that neutron isn't sliced up into cells, I think host-by-host is the only gradual rollout we have available. 09:15:43 <obondarev> I think we can still continue to discuss cells in gerrit, with more people than we have at the meeting now 09:15:53 <anteaya> belmoreira: we are trying to come up with a solution you can support, I fear that we might go down into a rabbit hole if we get blocked on cells 09:16:09 <anteaya> belmoreira: what parts of the offered specs are you able to support? 09:17:04 <gus> belmoreira: you could certainly migrate all the hypervisors in one cell before moving on to the next, for example. 09:17:16 <belmoreira> during the migration I'm looking into the simple use case of creating/delete a VM 09:17:45 <belmoreira> I don't expect to have a full set neutron API available to users... 09:17:46 <gus> yep, that should continue to work just fine during the migration. 09:18:03 <gus> (except for the brief moment where the database is dumped/migrated at the start) 09:18:21 <obondarev> neutron API will be read-only during the migration 09:18:50 <belmoreira> It would be acceptable to not allow floating IPs/security groups, ... during the migration 09:19:23 <gus> urgh, async replies. Yes: create/delete VMs should continue to work (except for the brief read-only period at start). neutron API will not be available (read-write) until the entire thing is complete. 09:19:50 <belmoreira> that's way I think the cell use case can be archievable if we have some restrictions about what is offered to users during the migration 09:20:37 <obondarev> but floating ips and sec groups will still be accessible for crreate/get/update/delete through nova api 09:20:58 <obondarev> so should be transparent to the users 09:21:28 <belmoreira> if a deployment is using cells security groups are not supported anyway 09:21:53 <belmoreira> floating IPs we don't use them... not sure if they are available with cells. 09:23:15 <gus> so I'll add a statement to 147723 describing cells impact (nova-api should basically keep working, you can choose whatever order of hypervisors to upgrade, neutron won't be available until the last hypervisor is done) 09:23:31 <gus> the catch will be testing, and whether we want to deal with cells in this spec, or stick to the easy case here and punt to a followup. 09:24:14 <gus> .. punt cells upgrades to a followup spec. 09:25:28 <belmoreira> gus: sorry, also considering cells... 09:26:11 <belmoreira> gus: using cells we have a db per cell, where the nova-network info is stored. 09:26:50 <gus> yep. so these will need to be dumped/restored together. ie: nova-api will be read-only for every cell until that is complete. 09:27:23 <obondarev> gus: you mean neutron api, no? 09:28:18 <gus> so step 1 is getting a description of the nova-network setup into neutron equivalents, so neutron DB becomes the single source of truth. 09:29:52 <gus> the way we've described this so far involves making the nova-api read-only, using a DB migration tool to dump/restore the nova-network db into neutron db, enabling a nova-network-proxy that looks the same to nova (python) but reads from neutron DB, and then making the nova-api read-write again. 09:30:36 <gus> I think this will need to be performed for all cells simultaneously given that neutron is cells-ignorant. 09:31:45 <gus> I see this is turning into a complex discussion that we should move to elsewhere. 09:31:53 <obondarev> probably this can be done cell by cell in a way where nova uses proxy for cells whose data is transitioned to neutron 09:32:13 <belmoreira> gus: the proxy works for nova-network calls? (when requesting an IP for a VM) 09:32:14 <obondarev> gus: +1 09:32:56 <belmoreira> obondarev: that would be great 09:33:27 <gus> belmoreira: yes, that's basically its entire job. Be a drop-in python replacement for nova-network, but do it by talking to neutron. 09:34:01 <obondarev> belmoreira: just thinking aloud, I don't know nova cells well enough 09:34:53 <gus> belmoreira/obondarev: ok, so do we want to have a separate chat to work out cells issues, or do we feel we can do it via gerrit review, (or is it sufficiently complex that we should move it into its own spec). 09:34:57 <gus> ? 09:35:01 <belmoreira> gus: in that case as obondarev mentioned, if we move only a specific DB and enable the proxy per cell the migration should be possible 09:35:30 <obondarev> so still thinking we should do it for simple case first, test it and them move to more complex cases 09:35:51 <anteaya> belmoreira: are you able to support any part of the offered specs? 09:36:25 <obondarev> even if the first version will not be applicable for real-world deployments 09:36:43 <belmoreira> gus: I think it will be better to have this discussion on IRC at least to understand the implications. 09:37:16 <belmoreira> gus: then we can iterate on gerrit 09:38:38 <gus> belmoreira: ack. I need to do some further research on how cells/nova-network works too .. 09:39:32 <belmoreira> anteya: The spec is great for small deployments. Having a large deployment I would prefer to stage the migration. In this case using cells. 09:40:20 <anteaya> belmoreira: as of the latest discussions about the spec, cells were considered out of scope 09:40:54 <anteaya> belmoreira: part of the consideration for hinging your support for the process on cells maybe loss of support from the community 09:41:04 <anteaya> and a possible derailment of this effort 09:41:22 <anteaya> I would like to see if we can agree to any part of what has been offered as in scope right now 09:41:32 <anteaya> so we can at least move forward in some part 09:42:02 <belmoreira> anteaya: fair enough... 09:42:09 <anteaya> thanks for understanding 09:42:29 <anteaya> it is possible that if we can make some progress in this direction, however small 09:42:37 <anteaya> we may build community support 09:42:53 <anteaya> if the feel their efforts are valued and convey results 09:43:12 <obondarev> I'll look into nova cells as well to see if there is something in the proposed plan that will cause problems for supporting cells in next iterations 09:43:24 <anteaya> obondarev: thank you 09:43:35 <anteaya> yes, I think further research is warrented 09:43:38 <belmoreira> obondarev: thanks 09:43:54 <anteaya> but I would like to see if we can get some consencous at least on https://review.openstack.org/#/c/147723/3 09:44:39 <anteaya> so for 147723 09:45:02 <anteaya> do those present feel they can comment/vote/support this patch? 09:45:57 <belmoreira> anteaya: sure 09:46:06 <jlibosva> will do 09:46:08 <anteaya> belmoreira: thank you, I appreciate that 09:46:12 <anteaya> jlibosva: thank you 09:46:26 <anteaya> so I will communicate with nova and get some reviews from them 09:46:42 <anteaya> and hopefully we can get this patch moving forward 09:47:09 <anteaya> let's agree to discuss cells again next week after further research from/by obondarev and gus 09:47:17 <obondarev> agreed 09:47:19 <gus> ack 09:47:38 <anteaya> and continue to work on https://review.openstack.org/#/c/142456/ this week and see where we are next meeting 09:47:39 <belmoreira> thanks for the effort 09:47:41 <anteaya> fair? 09:47:46 <anteaya> belmoreira: great thank you 09:47:52 <obondarev> +1 09:48:06 <anteaya> wonderful 09:48:14 <anteaya> can we move onto the next agenda topic? 09:48:32 <anteaya> moving on 09:48:40 <anteaya> #topic the state of implementation (obondarev) 09:48:45 <obondarev> so 09:48:46 <anteaya> obondarev: you again :D 09:48:48 <obondarev> speaking of implementation 09:48:59 <obondarev> I haven't put my wip prototype on gerrit mostly because it is a neutron proxy plugin to nova 09:49:11 <obondarev> while new plan discussed and reflected in specs is to have a nova proxy to neutron 09:49:23 <obondarev> the reasoning for that is described in the second spec 09:49:33 <obondarev> I'm going to start working on it shortly 09:49:43 <obondarev> together with exploring nova cells 09:49:49 <anteaya> thank you 09:49:59 <obondarev> jlibosva has uploaded a WIP on data migration from nova to neutron db 09:50:02 <jlibosva> obondarev: does it mean your current code will be thrown away? 09:50:35 <obondarev> jlibosva: I guess so, I'm not sure how to apply it with the current plan 09:50:53 <anteaya> obondarev: you could offer up the patch to gerrit and let folks look at it 09:50:58 <anteaya> mark it wip 09:51:10 <anteaya> shame to waste an effort 09:51:24 <obondarev> well, I can but it may cause confusion 09:51:33 <anteaya> that is a good point 09:51:49 <anteaya> you are right, let's proceed in the new direction 09:51:50 <jlibosva> you can write it to commit message that it goes with proposal out of date 09:52:05 <anteaya> but I do want to acknowledge your work here obondarev even if noone ever sees it 09:52:19 <obondarev> anteaya: thanks you :) 09:52:24 <anteaya> :) 09:52:31 <gus> indeed :) 09:52:47 <anteaya> so jlibosva has a patch up 09:52:58 <obondarev> so returning to db migration by jlibosva 09:53:06 <obondarev> #link https://review.openstack.org/#/c/148260/ 09:53:14 <obondarev> which is now the first step in the proposed plan 09:53:20 <obondarev> thanks jlibosva 09:53:45 <jlibosva> that means I should hurry up, I was living in the fact I still have time as I thought it'll be actually the last one 09:53:54 <anteaya> jlibosva: ah 09:54:11 <anteaya> jlibosva: it looks like you have some reviews on that, thanks obondarev 09:54:16 <obondarev> ok, we still need to agree the whole design 09:54:28 <anteaya> obondarev: we meaning who? 09:54:28 <obondarev> and to merge the specs 09:54:43 <anteaya> meaning the design in the specs, yes? 09:54:44 <obondarev> we - the community 09:54:54 <obondarev> right 09:55:00 <anteaya> okay great 09:55:17 <anteaya> jlibosva: do you have what you need to continue working on your patch? 09:55:33 <obondarev> but db migration is something that is required anyway I guess 09:55:35 <jlibosva> anteaya: yes, I have some gaps in my knowledge but nothing crucial 09:55:58 <anteaya> jlibosva: great, let's see where you get to by next week and do speak up if you get blocked 09:56:07 <anteaya> obondarev: yes 09:56:08 <jlibosva> sure 09:56:11 <anteaya> thanks 09:56:29 <anteaya> and you will be offering a patch for the proxy sometime, obondarev? 09:56:42 <obondarev> yes 09:56:45 <anteaya> great 09:56:47 <anteaya> thank you 09:56:57 <anteaya> anything more under implementation? 09:57:03 <obondarev> not from my side 09:57:11 <anteaya> wonderful, moving on 09:57:15 <anteaya> #topic documentation (obondarev) 09:57:19 <obondarev> we have good news on the docs side as well 09:57:23 <anteaya> yay 09:57:24 <obondarev> I talked to emagana and he agreed to help with docs for the migration 09:57:31 <anteaya> wonderful, thank you 09:57:32 <obondarev> I've added him to the specs reviews so he's aware of the proposed plan 09:57:38 <anteaya> obondarev: perfect 09:57:49 <anteaya> obondarev: did you get a chance to chat with annegentle as well? 09:58:04 <obondarev> no, I didn't 09:58:07 <anteaya> would like to take her up on her offer 09:58:14 <anteaya> okay I will find her this week 09:58:20 <anteaya> we can't have too much help with docs 09:58:23 <obondarev> anteaya: thank you 09:58:29 <obondarev> anteaya: agree 09:58:46 <anteaya> #action anteaya to chat with annegentle about offer of docs help 09:58:59 <anteaya> #topic testing (belmoreira) 09:59:12 <anteaya> belmoreira: I assigned the testing topic to you 09:59:19 <anteaya> belmoreira: I hope that is okay with you 09:59:26 <anteaya> belmoreira: any thoughts here this week? 10:00:01 <belmoreira> anteaya: :) well, I give my comments how I would like to test the migration 10:00:14 <anteaya> belmoreira: that helps, thank you :D 10:00:22 <anteaya> and we are at time 10:00:31 <obondarev> thanks everybody 10:00:33 <jlibosva> thanks 10:00:34 <gus> I can have a go at grenade, etc work too. 10:00:35 <anteaya> thank you all for your attendance and support 10:00:41 <belmoreira> thanks 10:00:43 <anteaya> gus: awesome thank you 10:00:47 <gus> I feel we're mostly constrained by what our (current) testing frameworks are capable of. 10:00:55 <anteaya> see you all next week 10:01:02 <gus> thanks all. 10:01:04 <anteaya> gus: yes 10:01:09 <anteaya> #endmeeting