14:00:31 <PaulMurray> #startmeeting Nova Live Migration 14:00:32 <openstack> Meeting started Tue Feb 2 14:00:31 2016 UTC and is due to finish in 60 minutes. The chair is PaulMurray. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:36 <openstack> The meeting name has been set to 'nova_live_migration' 14:00:40 <andrearosa> hello 14:00:45 <eliqiao> hi 14:00:48 <tdurakov> o/ 14:00:48 <davidgiluk> o/ 14:00:53 <eliqiao> o/ 14:00:56 <mdbooth_> o/ 14:00:57 <pkoniszewski> o/ 14:00:57 <jlanoux_> hi 14:01:25 <PaulMurray> Hi all 14:01:54 <PaulMurray> Agenda here: https://wiki.openstack.org/wiki/Meetings/NovaLiveMigration 14:02:22 <PaulMurray> diving straight in.... 14:02:28 <PaulMurray> #topic Priority reviews 14:03:02 <PaulMurray> I noticed yesterday that we have not completed any of our specs yet 14:03:14 <PaulMurray> and I wanted to do an update message after the mid cycle 14:03:15 <pkoniszewski> that's true 14:03:18 <andrearosa> panic mode ON 14:03:20 <andrearosa> :) 14:03:47 <eliqiao> We don't have any cores in team. 14:03:49 <PaulMurray> so I would like to go through the ones on the review etherpad quickly and understand what has to be done to complete 14:04:09 <PaulMurray> and I will see what I can do to get core attention 14:04:31 <PaulMurray> #info we have 4 weeks to feature freeze 14:04:42 <tdurakov> +1, let's go through 14:05:02 <PaulMurray> #link reviews here https://etherpad.openstack.org/p/mitaka-nova-priorities-tracking 14:05:27 <PaulMurray> what is left for Enable block live migration with attached volumes 14:06:08 <PaulMurray> pkoniszewski, I think you are doing those? 14:06:43 <pkoniszewski> so I proposed new solution according to Daniel's review, can't really do anything more there 14:07:25 <PaulMurray> do you need danpb to review? 14:07:50 <pkoniszewski> question is whether my solution covers all scenarios with shared devices, at the midcycle Daniel said that it should work, we need cores eyes on it as they are most knowledgable ppl there 14:09:08 <PaulMurray> making a note on pad 14:09:55 <PaulMurray> I'll see if we can get him and others 14:10:05 <PaulMurray> I think a lot of this is going to be about gettin reviews 14:10:12 <PaulMurray> next one in Split network 14:10:30 <eliqiao> I reviewed that one, and have some comments on it. 14:10:34 <pkoniszewski> AFAIK split network got some good quality reviews past last 2 days 14:10:39 <pkoniszewski> it require some more work 14:10:46 <eliqiao> alex_xu -1 on it for some UT stuff. 14:11:42 <PaulMurray> I saw the object change, I should have seen that - I've done a lot of objects..... 14:11:53 <eliqiao> I have a concern on it. for the validation of live_migration_connect_addr which is configured by admin/operator, can we trust it and use it without any verification? 14:12:14 <pkoniszewski> eliqiao: we dont validate config options 14:12:35 <eliqiao> that is not friendly. 14:12:50 <tdurakov> eliqiao, if someone has access to conf... 14:13:06 <PaulMurray> eliqiao, what validation are you thinking of - check format? 14:13:37 <eliqiao> if ip address, check if it is valid, if host name, check if it is valid. and more over, check if that hostname/ip can be pinged. 14:13:39 <tdurakov> PaulMurray, i think it's about security? 14:13:57 <eliqiao> currently, this patch has nothing validation on it. 14:14:16 * alex_xu thinks splits one is pretty close, he will +2 to very soon after update 14:14:20 <eliqiao> so if operation configure it by mistake(typo) or something like that, LM will fail 14:14:45 <PaulMurray> I wouldn't worry about pinging it - that can fail anyway 14:14:51 <pkoniszewski> eliqiao: pinging is not an option imo as you might play with hostnames at any point 14:15:03 <pkoniszewski> s/point/time 14:15:08 <PaulMurray> other places with ip addresses may check format, but not if it actually works 14:15:22 <PaulMurray> you find that out when you try to use it 14:15:26 <andrearosa> +1 14:15:29 <tdurakov> no need to this kind of validation, imo 14:16:03 * davidgiluk can imagine checking the address is on an internal rather than an external network 14:16:16 <eliqiao> Okay, then we, please make high note on the new feature releast note, this option should be take care by operatior themself. 14:16:19 <pkoniszewski> eliqiao: i wouldn't worry about validation at all, let's say a bit about ip format, but not that much 14:16:52 <pkoniszewski> eliqiao: remember that it will fallback to old method if IP is invalid so it will still work 14:17:30 <PaulMurray> Its worth a log message if it doesn't work 14:17:36 <PaulMurray> comments on patch if anything else 14:17:52 <PaulMurray> next is Pause VM during live migration 14:17:55 <eliqiao> pkoniszewski: I don't find any logic say that will fallback to old method. 14:17:58 <PaulMurray> (skipping the bug for now) 14:18:12 <eliqiao> Okay, talk it offline. 14:18:31 <pkoniszewski> eliqiao: might be that Zhenyu only proposed that but haven't added it yet 14:19:14 <eliqiao> pkoniszewski: hmm.. don't think so, I propose the validation, he is kinds of agree, but no update about it. 14:19:20 <pkoniszewski> so, forcing LM is up for review, alex_xu made some good reviews from the API side, IBM folks are also ok with my change after I reorganized layers a bit 14:19:55 <pkoniszewski> change looks like it is pretty big, but it is because of docs and microversions 14:20:19 <PaulMurray> I saw you rebased yesterday - was there any other change with that patchset? 14:20:22 <eliqiao> yeah, but first we should merge https://review.openstack.org/#/c/257270 too, I can help on API layer too. 14:20:50 <pkoniszewski> no, only a merge conflict... because I need to bump both microversions, RPC and API, im getting a lot of merge conflicts there 14:21:00 <pkoniszewski> that's why im rebasing it very often 14:21:25 <eliqiao> yeah, cause dan's migration_data patch get merged, we need to bump RPC vesion. 14:21:27 <PaulMurray> yes - there are 47 patches in the same code at the moment 14:22:03 <pkoniszewski> im not changing the code itself 14:22:35 <eliqiao> PaulMurray: I think we can move up this patch to 'Nova core review' section, right ? 14:22:57 <andrearosa> as eliqiao said, we should have some core approving the change which introduces the new DB Api method 14:23:13 <pkoniszewski> +1 14:23:14 <PaulMurray> eliqiao, I think so, they don't want more than a couple of patches up there though 14:23:27 <pkoniszewski> we have three changes that require this new DB API... 14:23:48 <eliqiao> the biggest problem for LM team is 'we are lack of core' 14:24:07 <PaulMurray> eliqiao, noted 14:24:17 <andrearosa> eliqiao: yes but LM is high prioprirty so hopefully we will have full attention from cores 14:25:00 <PaulMurray> can we look at report live migration progress next 14:25:38 <PaulMurray> anyone know about these? 14:25:47 <PaulMurray> not so much movement in last couple of weeks 14:25:57 <eliqiao> yeah, there is a question about update inerval. 14:26:29 <eliqiao> PaulMurray made a comment that 2s is reasonable, need to ping shaohe_feng to update patch. 14:26:42 <PaulMurray> I had a chat with rackspace guys - HP and rax think 2 sseconds is ok 14:26:58 <PaulMurray> it could be made a config, but we weren't worried 14:27:04 <tdurakov> PaulMurray, got concerns about writing to frequent to db on this 14:27:04 <pkoniszewski> well one proposition from me 14:27:36 <eliqiao> oh, ignore me, it's alreay 2s.. :( 14:27:54 <pkoniszewski> I think that this patch to implement index/show on migrations is actually more important than writing migration progress to the DB 14:28:05 <pkoniszewski> I mean this one https://review.openstack.org/#/c/258771/ 14:28:10 <paul-carlton2> tdurakov, even if you are doing 100 migrations concurrently this is still a tiny query rate given overall system load on db 14:28:24 <pkoniszewski> it is required by cancelling and forcing LM 14:28:36 <tdurakov> paul-carlton2, there are also rpc for this 14:28:39 <pkoniszewski> there's imo no reason to write progress to the DB if don't have way to read it... 14:29:04 <paul-carlton2> true, but rabbitmq can handle much higher rates than db 14:29:54 <paul-carlton2> pkoniszewski, we do have a way to read it, the GET on server/<uuid>/migrations 14:30:05 <pkoniszewski> will it report migration progress? 14:30:13 <tdurakov> pkoniszewski, paul-carlton2 PaulMurray, as decided that compute will have state by itself 14:30:23 <pkoniszewski> thought that it is just a list of migrations 14:30:33 <pkoniszewski> and progress will be excluded from this list 14:30:49 <tdurakov> it will be easier to make things clear in that area 14:31:01 <paul-carlton2> there is a spec for getting this info, I'm sure 14:31:07 <pkoniszewski> paul-carlton2: https://review.openstack.org/#/c/258797/11/nova/api/openstack/compute/migrations.py 14:31:47 <paul-carlton2> also GET on specific migration will return details 14:31:47 <pkoniszewski> according to this patch list will not include progress details 14:31:55 <pkoniszewski> but its not implemented yet 14:32:03 <pkoniszewski> and it is in, let's be honest, in bad condition 14:32:30 <pkoniszewski> so we shouldn't focus that much on writing the progress to the DB because we can't read it yet - https://review.openstack.org/#/c/258771/ 14:33:19 <PaulMurray> It sounds like these patches need some work to get them straight 14:33:28 <paul-carlton2> this is crazy, why would we srupress this info if we are collecting for user info 14:33:37 <PaulMurray> How about a question on ML regarding update rate 14:33:43 <PaulMurray> to settle it 14:34:02 <PaulMurray> We can go back over the specs and leave discussion on the patch 14:34:04 <pkoniszewski> paul-carlton2: we can talk offline, this is reasonable because current call returns only a list of migrations, there is no GET yet 14:34:23 <eliqiao> can someone help to summarize it? 14:34:32 <paul-carlton2> What is the point of collecting info if you can't see it 14:34:40 <eliqiao> pkoniszewski: seems it related to at least 2 specs ? 14:34:43 <pkoniszewski> paul-carlton2: thats my point 14:35:14 <paul-carlton2> the GET server/<uuid>/migration/<id> call should display details, if not that is wrong 14:35:57 <pkoniszewski> let's agree that there is more work to be done in reporting and getting progress and let's move forward 14:36:06 <tdurakov> +1 14:36:33 <paul-carlton2> agreed, list should be summary info, GET ... <id> should display details 14:36:49 <PaulMurray> Those are the main priority specs that can make progress this cycle 14:36:54 <PaulMurray> and andrearosa is about 14:37:03 <PaulMurray> to add the concel based on the pause 14:37:09 <andrearosa> correct 14:37:14 <PaulMurray> It would be good to get those 5 through 14:37:34 <pkoniszewski> that's great, if you can add me to review i will take a look on this 14:37:45 <PaulMurray> pause, cancel, progress, block migration with volumes and split network 14:37:54 <andrearosa> pkoniszewski:sure thing 14:38:27 <PaulMurray> So I will see what I can do to get some reviews from cores on the ones that are near and we can all try to push these through 14:38:31 <pkoniszewski> I think that we should distinct progress and show/index on migrations 14:38:31 <eliqiao> andrearosa: please add me too. thx. 14:38:47 <andrearosa> ack 14:39:01 <pkoniszewski> show/index on migrations is more important than progress 14:39:24 <PaulMurray> There is also Making the live-migration API friendly 14:39:33 <PaulMurray> anyone know what is left on that? 14:39:35 <andrearosa> but is the original author working onshow/index? 14:39:40 <andrearosa> it seems pretty much stuck to me 14:39:45 <eliqiao> yeah, I am owner. 14:39:51 <andrearosa> cool 14:40:09 <eliqiao> not many reviews on it. 14:40:47 <eliqiao> I rebased today due to dan's migrate_data patch get merged(bump RPC version) 14:40:57 <PaulMurray> There is a request for release notes on https://review.openstack.org/#/c/259319/ 14:41:00 <tdurakov> eliqiao, this on in merge conflict https://review.openstack.org/#/c/259319/ 14:41:48 <eliqiao> PaulMurray: tdurakov yeah, I notied that, and it will be alway on merge confilt so I don't update it for log time. 14:42:14 <PaulMurray> eliqiao, does it need the release notes ? 14:42:16 <eliqiao> But can we do the review on the dependency patch first (virt layer and compute api layer) 14:42:27 <tdurakov> eliqiao, sure 14:42:46 <eliqiao> PaulMurray: it changes API behaviors, it's better to have a release notes. 14:42:46 <tdurakov> one that returns data from task.execute almost fine for me 14:43:07 <tdurakov> eliqiao, will leave comments 14:43:39 <tdurakov> also got question for libvirt side 14:43:51 <eliqiao> tdurakov: thanks, I will update tomorrow, hope it can be done before next week since I will be on vacation due to Chinese New year in next 1 or 2 weeks 14:44:21 <eliqiao> tdurakov: please? 14:45:00 <PaulMurray> eliqiao, can someone take over for your vacation? 14:45:24 <eliqiao> yeah, I can ask alex_xu for help. 14:45:28 <eliqiao> PaulMurray: ^^ 14:45:43 <PaulMurray> can you ask him to approave some as well ? 14:45:59 <PaulMurray> he is a core :) 14:46:09 <PaulMurray> it might be better if we do the work and he reviews 14:46:16 <eliqiao> PaulMurray: good idea :), he promised me that he will do the review next week :) 14:46:48 <eliqiao> PaulMurray: yeah. I will try to talk to him before I am on vacation. 14:47:06 <PaulMurray> I want to do an action now 14:47:21 <PaulMurray> someone needs to do the email for ML about report live migration progress 14:47:33 <PaulMurray> who will do that? 14:47:54 <pkoniszewski> PaulMurray: I can do that 14:48:01 <PaulMurray> thanks 14:48:23 <eliqiao> +1 for pkoniszewski , I will follow. Thanks! 14:48:24 <PaulMurray> #action pkoniszewski to write email for dev ML about tie interval for writing to DB in report live migration progress 14:48:56 <PaulMurray> Lastly there are a couple of patches hanging around for Series to deprecate migration flags config: 14:49:36 <pkoniszewski> PaulMurray: today sdague approved one and -1 another 14:50:10 <sdague> pkoniszewski: yeh, it's mostly for a missing test case 14:50:25 <sdague> it should be quick turn around the 2 things I -1ed today 14:50:45 <pkoniszewski> exactly, so this series is almost in 14:50:52 <PaulMurray> sdague, do you know if mark is around (can't remember nick) 14:51:06 <eliqiao> markmcclain: 14:51:10 <eliqiao> shoud be. 14:51:13 <sdague> PaulMurray: it's markmc, and I don't know that he's been hanging around much 14:51:18 <sdague> eliqiao: no, that's the other mark 14:51:30 <eliqiao> sdague: get it, thx. 14:51:33 <sdague> markmc is in #nova right now 14:51:42 <sdague> I think it would also be fine for someone else to update with the fixes 14:51:54 <PaulMurray> sdague, well, I'll give it a few days and see if I can find him if he doesn't see it 14:52:05 <sdague> well, I'd suggest turning it around while it's fresh 14:52:21 <sdague> especially given that they are pretty straight forward 14:52:29 <sdague> and mark is a pretty busy person 14:52:44 <PaulMurray> yes, can do - can someone do that? (I'm pretty busy right now too) 14:52:54 <PaulMurray> any offers to fix the patch up? 14:52:57 <pkoniszewski> i've got to go, thanks everyone, bye! 14:53:19 <eliqiao> markmc just replied sdague on the patch one hour ago. 14:53:24 <sdague> eliqiao: cool 14:53:42 <PaulMurray> which patch? 14:53:52 <eliqiao> #link https://review.openstack.org/#/c/263434/9/nova/tests/unit/virt/libvirt/test_driver.py 14:54:01 <PaulMurray> oh yes, I see it 14:54:05 <eliqiao> he got test case proposed on Ic480be79772e2721e0f57f17ccc9d893e92af4c9 14:54:13 <PaulMurray> lets leave it with him then 14:54:43 <PaulMurray> We are nearly on the hour now 14:54:50 <PaulMurray> #topic Bugs 14:55:13 <PaulMurray> we have not done much in the way of bug triage that I am aware of 14:55:20 <PaulMurray> has anyone been looking at them? 14:55:35 <sdague> I'm happy with markmc's comments as soon as the oslo.config test results come back 14:55:59 <PaulMurray> sdague, cool - thanks 14:56:59 <PaulMurray> mikal was going to do something for bugs but ended up too busy 14:57:08 <PaulMurray> I think its too late to discuss here now 14:57:09 <eliqiao> PaulMurray: I promised to help, but I don't get any response from mikal. 14:57:32 <sdague> PaulMurray: if you could come up with alternative wording for https://review.openstack.org/#/c/263436 - I'll +2 it, but I agree the deprecation warning is really hard to get your head around. 14:57:53 <PaulMurray> sdague, I'll take a look right after this 14:57:58 <sdague> cool 14:58:17 <PaulMurray> eliqiao, lets go offline with the bugs topic 14:58:29 <eliqiao> PaulMurray: Okay. 14:58:42 <PaulMurray> no time for other discussion, so sorry if you have any 14:58:57 <PaulMurray> remember that you can add things to the agenda if you want to make sure you get time 14:59:05 <PaulMurray> thanks all for coming 14:59:13 <mdbooth> PaulMurray: Any chance of a higher-bandwidth chat about storage pools some time soon with whoever's interested? 14:59:19 <mdbooth> Unfortunately I couldn't make the midcycle. 14:59:23 <paul-carlton2> mdbooth, you back in uk now? 14:59:25 <PaulMurray> #endmeeting