*** wolverineav has quit IRC | 00:09 | |
*** wolverineav has joined #openstack-nova | 00:10 | |
*** wolverineav has quit IRC | 00:12 | |
*** wolverineav has joined #openstack-nova | 00:13 | |
*** brinzhang has joined #openstack-nova | 00:18 | |
*** threestrands has joined #openstack-nova | 00:18 | |
*** frippe75_ has quit IRC | 00:23 | |
*** tbachman has quit IRC | 00:25 | |
*** rambo_li has joined #openstack-nova | 00:35 | |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: WIP: API microversion bump for handling-down-cell https://review.openstack.org/591657 | 00:38 |
---|---|---|
*** tssurya has quit IRC | 00:55 | |
*** hamzy_ has joined #openstack-nova | 01:07 | |
*** hongda has joined #openstack-nova | 01:22 | |
*** hshiina has joined #openstack-nova | 01:36 | |
*** erlon has quit IRC | 01:38 | |
*** wolverineav has quit IRC | 01:38 | |
*** dave-mccowan has joined #openstack-nova | 01:51 | |
*** gyee has quit IRC | 01:52 | |
*** dklyle has joined #openstack-nova | 01:59 | |
*** tbachman has joined #openstack-nova | 02:05 | |
*** dklyle has quit IRC | 02:05 | |
*** cfriesen has quit IRC | 02:06 | |
*** tbachman_ has joined #openstack-nova | 02:08 | |
*** tbachman has quit IRC | 02:10 | |
*** tbachman_ is now known as tbachman | 02:10 | |
*** tbachman has quit IRC | 02:18 | |
*** mriedem_afk has quit IRC | 02:19 | |
*** mrsoul has quit IRC | 02:22 | |
*** takashin has joined #openstack-nova | 02:27 | |
*** Dinesh_Bhor has joined #openstack-nova | 02:27 | |
*** _hemna has quit IRC | 02:29 | |
*** mlavalle has quit IRC | 02:34 | |
*** _hemna has joined #openstack-nova | 02:35 | |
*** liuyulong_ has joined #openstack-nova | 02:35 | |
openstackgerrit | Alex Xu proposed openstack/nova master: Parse the numa topology from the new flavor when resize instance https://review.openstack.org/620874 | 02:38 |
*** psachin has joined #openstack-nova | 02:39 | |
*** mhen has quit IRC | 02:40 | |
*** mhen has joined #openstack-nova | 02:43 | |
*** dave-mccowan has quit IRC | 02:46 | |
*** dave-mccowan has joined #openstack-nova | 02:56 | |
*** _hemna has quit IRC | 02:56 | |
*** _hemna has joined #openstack-nova | 02:57 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Drop pre-cinder 3.44 version compatibility https://review.openstack.org/621061 | 02:57 |
*** bhagyashris has joined #openstack-nova | 03:02 | |
*** hongbin has joined #openstack-nova | 03:02 | |
*** dklyle has joined #openstack-nova | 03:11 | |
*** whoami-rajat has joined #openstack-nova | 03:12 | |
*** _hemna has quit IRC | 03:13 | |
openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Support for changing deleted_on_termination after boot https://review.openstack.org/580336 | 03:15 |
*** dklyle has quit IRC | 03:17 | |
openstackgerrit | Merged openstack/nova master: Add I/O Semaphore to limit concurrent disk ops https://review.openstack.org/609180 | 03:25 |
openstackgerrit | Merged openstack/nova stable/pike: Fix NoneType error in _notify_volume_usage_detach https://review.openstack.org/614872 | 03:25 |
openstackgerrit | Merged openstack/nova stable/queens: Default embedded instance.flavor.is_public attribute https://review.openstack.org/619351 | 03:25 |
*** _hemna has joined #openstack-nova | 03:30 | |
*** _hemna has quit IRC | 03:30 | |
*** dave-mccowan has quit IRC | 03:33 | |
*** dave-mccowan has joined #openstack-nova | 03:34 | |
openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Support for changing deleted_on_termination after boot https://review.openstack.org/580336 | 03:39 |
*** psachin has quit IRC | 03:43 | |
*** jarodwl has quit IRC | 03:43 | |
*** jarodwl has joined #openstack-nova | 03:43 | |
*** _hemna has joined #openstack-nova | 03:46 | |
*** _hemna has quit IRC | 03:50 | |
*** _hemna has joined #openstack-nova | 03:56 | |
*** _hemna has quit IRC | 03:56 | |
*** hongbin has quit IRC | 03:56 | |
*** udesale has joined #openstack-nova | 04:01 | |
*** janki has joined #openstack-nova | 04:03 | |
*** _hemna has joined #openstack-nova | 04:11 | |
*** _hemna has quit IRC | 04:11 | |
*** _hemna has joined #openstack-nova | 04:21 | |
*** _hemna has quit IRC | 04:21 | |
*** ivve has joined #openstack-nova | 04:25 | |
*** rambo_li has quit IRC | 04:25 | |
*** dave-mccowan has quit IRC | 04:30 | |
*** markvoelker has quit IRC | 04:32 | |
*** _hemna has joined #openstack-nova | 04:40 | |
*** _hemna has quit IRC | 04:40 | |
*** _hemna has joined #openstack-nova | 04:42 | |
*** _hemna has quit IRC | 04:43 | |
*** _hemna has joined #openstack-nova | 04:45 | |
*** _hemna has quit IRC | 04:45 | |
*** ratailor has joined #openstack-nova | 04:50 | |
*** wolverineav has joined #openstack-nova | 04:52 | |
*** _hemna has joined #openstack-nova | 04:57 | |
*** _hemna has quit IRC | 04:57 | |
*** markvoelker has joined #openstack-nova | 05:02 | |
*** Nel1x has quit IRC | 05:14 | |
*** wolverineav has quit IRC | 05:15 | |
*** _hemna has joined #openstack-nova | 05:17 | |
*** _hemna has quit IRC | 05:17 | |
*** threestrands has quit IRC | 05:20 | |
openstackgerrit | Alex Xu proposed openstack/nova master: Parse the numa topology from the new flavor when resize instance https://review.openstack.org/620874 | 05:30 |
*** _hemna has joined #openstack-nova | 05:31 | |
*** _hemna has quit IRC | 05:32 | |
*** pcaruana has quit IRC | 05:35 | |
*** _hemna has joined #openstack-nova | 05:38 | |
*** _hemna has quit IRC | 05:39 | |
*** imacdonn has quit IRC | 05:39 | |
*** imacdonn has joined #openstack-nova | 05:39 | |
*** _hemna has joined #openstack-nova | 05:45 | |
*** _hemna has quit IRC | 05:45 | |
*** _hemna has joined #openstack-nova | 05:52 | |
*** _hemna has quit IRC | 05:52 | |
*** _hemna has joined #openstack-nova | 05:59 | |
*** _hemna has quit IRC | 05:59 | |
*** rambo_li has joined #openstack-nova | 06:03 | |
*** _hemna has joined #openstack-nova | 06:05 | |
*** _hemna has quit IRC | 06:06 | |
*** _hemna has joined #openstack-nova | 06:13 | |
*** _hemna has quit IRC | 06:16 | |
*** sridharg has joined #openstack-nova | 06:21 | |
*** _hemna has joined #openstack-nova | 06:21 | |
*** _hemna has quit IRC | 06:21 | |
*** _hemna has joined #openstack-nova | 06:29 | |
*** _hemna has quit IRC | 06:29 | |
*** hoangcx has quit IRC | 06:29 | |
*** _hemna has joined #openstack-nova | 06:39 | |
*** ondrejme has quit IRC | 06:39 | |
*** _hemna has quit IRC | 06:39 | |
*** _hemna has joined #openstack-nova | 06:47 | |
*** _hemna has quit IRC | 06:48 | |
*** noonedeadpunk[h] has joined #openstack-nova | 06:48 | |
*** noonedeadpunk has quit IRC | 06:49 | |
openstackgerrit | Alex Xu proposed openstack/nova master: Parse the numa topology from the new flavor when resize instance https://review.openstack.org/620874 | 06:53 |
openstackgerrit | Alex Xu proposed openstack/nova master: Parse the new PCI request from flavor extra specs when resize https://review.openstack.org/621077 | 06:53 |
*** _hemna has joined #openstack-nova | 06:54 | |
*** _hemna has quit IRC | 06:54 | |
*** _hemna has joined #openstack-nova | 07:01 | |
*** _hemna has quit IRC | 07:02 | |
openstackgerrit | Filippo Inzaghi proposed openstack/python-novaclient master: Add Python 3.6 classifier to setup.cfg https://review.openstack.org/621093 | 07:11 |
*** hamzy__ has joined #openstack-nova | 07:11 | |
*** hamzy_ has quit IRC | 07:12 | |
*** ohorecny2 has joined #openstack-nova | 07:12 | |
*** _hemna has joined #openstack-nova | 07:13 | |
*** _hemna has quit IRC | 07:15 | |
*** pooja_jadhav has joined #openstack-nova | 07:21 | |
*** _hemna has joined #openstack-nova | 07:21 | |
*** stakeda has joined #openstack-nova | 07:22 | |
*** pcaruana has joined #openstack-nova | 07:22 | |
ohorecny2 | Everyone: Do you know some closer information about this bug: https://bugs.launchpad.net/nova/+bug/1795920 , please? I mean, if anybody is working on it or if you have some proposals how to fix it. | 07:25 |
openstack | Launchpad bug 1795920 in OpenStack Compute (nova) "SR-IOV shared PCI numa not working " [Undecided,Confirmed] | 07:25 |
*** slaweq has joined #openstack-nova | 07:34 | |
*** Luzi has joined #openstack-nova | 07:38 | |
*** jangutter has quit IRC | 07:41 | |
*** jangutter has joined #openstack-nova | 07:41 | |
*** takashin has left #openstack-nova | 07:45 | |
belmoreira | mriedem: thanks for ping me. See: https://bugs.launchpad.net/nova/+bug/1805984 https://bugs.launchpad.net/nova/+bug/1805989 | 07:53 |
openstack | Launchpad bug 1805984 in OpenStack Compute (nova) "Placement is not aware of disable compute nodes" [Undecided,New] | 07:53 |
openstack | Launchpad bug 1805989 in OpenStack Compute (nova) "Weight policy to stack/spread instances and "max_placement_results"" [Undecided,New] | 07:54 |
*** sahid has joined #openstack-nova | 07:56 | |
*** hshiina has quit IRC | 08:04 | |
*** rcernin has quit IRC | 08:06 | |
*** dpawlik has joined #openstack-nova | 08:07 | |
*** awalende has joined #openstack-nova | 08:08 | |
*** awalende_ has joined #openstack-nova | 08:08 | |
*** awalende_ has quit IRC | 08:10 | |
*** moshele has joined #openstack-nova | 08:11 | |
*** awalende has quit IRC | 08:12 | |
*** ralonsoh has joined #openstack-nova | 08:17 | |
openstackgerrit | Yikun Jiang proposed openstack/nova master: Use new ``initial_xxx_allocation_ratio`` CONF https://review.openstack.org/602804 | 08:21 |
*** helenaAM has joined #openstack-nova | 08:23 | |
*** ondrejme has joined #openstack-nova | 08:35 | |
*** xek has joined #openstack-nova | 08:38 | |
*** maciejjozefczyk has quit IRC | 08:38 | |
*** maciejjozefczyk has joined #openstack-nova | 08:39 | |
*** dpawlik has quit IRC | 08:46 | |
*** ccamacho has joined #openstack-nova | 08:46 | |
*** pooja_jadhav has quit IRC | 08:56 | |
*** ileixe has quit IRC | 09:05 | |
*** noonedeadpunk[h] is now known as noonedeadpunk | 09:14 | |
*** k_mouza has joined #openstack-nova | 09:15 | |
*** dpawlik has joined #openstack-nova | 09:20 | |
kashyap | alex_xu: If you're still about: https://review.openstack.org/#/c/620327/ | 09:23 |
*** dpawlik has quit IRC | 09:24 | |
*** k_mouza has quit IRC | 09:28 | |
*** k_mouza has joined #openstack-nova | 09:28 | |
*** bhagyashris has quit IRC | 09:32 | |
*** rambo_li has quit IRC | 09:32 | |
*** stakeda has quit IRC | 09:33 | |
*** derekh has joined #openstack-nova | 09:33 | |
*** s10 has joined #openstack-nova | 09:40 | |
*** takamatsu has quit IRC | 09:41 | |
*** ttsiouts has joined #openstack-nova | 09:41 | |
*** ttsiouts has quit IRC | 09:44 | |
*** ttsiouts has joined #openstack-nova | 09:44 | |
*** ttsiouts has quit IRC | 09:49 | |
*** tssurya has joined #openstack-nova | 10:03 | |
*** pooja_jadhav has joined #openstack-nova | 10:04 | |
*** erlon has joined #openstack-nova | 10:04 | |
*** brinzhang has quit IRC | 10:05 | |
*** k_mouza has quit IRC | 10:07 | |
openstackgerrit | Yikun Jiang proposed openstack/nova master: Use new ``initial_xxx_allocation_ratio`` CONF https://review.openstack.org/602804 | 10:09 |
openstackgerrit | Yikun Jiang proposed openstack/nova master: Add ratio online data migration when load compute node https://review.openstack.org/613499 | 10:09 |
openstackgerrit | Yikun Jiang proposed openstack/nova master: Add compute_node ratio online data migration script https://review.openstack.org/609995 | 10:09 |
*** Dinesh_Bhor has quit IRC | 10:09 | |
openstackgerrit | Merged openstack/nova-specs master: Re-propose numa-aware-live-migration spec https://review.openstack.org/599587 | 10:09 |
*** k_mouza has joined #openstack-nova | 10:20 | |
gibi | jackding: hi! the last open patch for the io semaphor work has merged https://review.openstack.org/#/c/609180/ Does it mean that the bp io-semaphore-for-concurrent-disk-ops is done? | 10:24 |
*** ttsiouts has joined #openstack-nova | 10:27 | |
*** maciejjozefczyk has quit IRC | 10:29 | |
*** maciejjozefczyk has joined #openstack-nova | 10:31 | |
*** slaweq has quit IRC | 10:32 | |
*** liuyulong_ has quit IRC | 10:38 | |
*** pooja_jadhav has quit IRC | 10:38 | |
*** pcaruana has quit IRC | 10:44 | |
*** pcaruana has joined #openstack-nova | 10:50 | |
*** ttsiouts has quit IRC | 10:55 | |
*** ttsiouts has joined #openstack-nova | 10:55 | |
*** udesale has quit IRC | 10:59 | |
*** cdent has joined #openstack-nova | 10:59 | |
*** ShilpaSD has quit IRC | 11:00 | |
*** ttsiouts has quit IRC | 11:00 | |
*** dtantsur|mtg is now known as dtantsur|afk | 11:00 | |
*** cdent has quit IRC | 11:20 | |
*** cdent has joined #openstack-nova | 11:46 | |
*** slaweq has joined #openstack-nova | 11:51 | |
*** Luzi has quit IRC | 12:00 | |
*** pcaruana has quit IRC | 12:11 | |
*** slaweq has quit IRC | 12:24 | |
*** udesale has joined #openstack-nova | 12:26 | |
*** eharney has quit IRC | 12:34 | |
openstackgerrit | Elod Illes proposed openstack/nova master: Transform scheduler.select_destinations notification https://review.openstack.org/508506 | 12:37 |
artom | bauzas, thanks for the +W on the NUMA migration spec :) What wasn't clear about the table? Matt had come up with some improved wording, so I can reword in a amendment patch | 12:40 |
sean-k-mooney | artom you could have just said the new code would only work if both the source and dest node supported it | 12:41 |
sean-k-mooney | for the other 3 cases the old workflow will be used so you dont really need the table but i could follow it in its redered from much easier then in the raw rst doc | 12:42 |
sean-k-mooney | i only read the raw rst doc normally so its a little annoying to have to read the rendered version | 12:43 |
*** jaypipes is now known as leakypipes | 12:44 | |
sean-k-mooney | leakypipes: im going to send an email to the list regrading the serialisation of os-vif objects | 12:45 |
*** Nel1x has joined #openstack-nova | 12:46 | |
sean-k-mooney | leakypipes: so that kuryr-kubernetes are aware that this will break in the future https://github.com/openstack/kuryr-kubernetes/commit/7cc187806b42fee5ea660f86d33ad2f59b009754 | 12:46 |
*** xek has quit IRC | 12:46 | |
*** xek has joined #openstack-nova | 12:47 | |
sean-k-mooney | leakypipes: and that everything they are doing in this file is using os-vif out of contract https://github.com/openstack/kuryr-kubernetes/blob/master/kuryr_kubernetes/objects/vif.py | 12:47 |
sean-k-mooney | i have told them we do not allow external plugins to define there own vif object 3 releases ago but we definetly do support serialising and prestingign between potically differed versions of os-vif | 12:48 |
artom | sean-k-mooney, yeah, the rst doc is unreadable for the table | 12:49 |
sean-k-mooney | artom: anyway the spec is merged and it was more of a nit i think then anything else so i would not worry about it | 12:50 |
leakypipes | sean-k-mooney: ack, thanks for getting that conversation started. if you need a spellcheck before sending out, lemme know :P | 12:51 |
sean-k-mooney | people keep telling me that specs are appreently not documentation so as long as we understand what you ment its all good | 12:51 |
leakypipes | sean-k-mooney: I actually don't understand why kuryr-kubernetes is defining os-vif objects at all (versus just defining the things in os-vif itself) | 12:52 |
sean-k-mooney | leakypipes: they did not know that they were not ment too so they did it really early on | 12:53 |
leakypipes | sean-k-mooney: ok | 12:53 |
leakypipes | sean-k-mooney: can we undo that? :) | 12:53 |
sean-k-mooney | more recently the have added it to os-vif when i pointed out that it would break going forward if tehy didnt | 12:53 |
sean-k-mooney | get rid of there objects yes | 12:54 |
sean-k-mooney | i ment to do it in rocky but move from intel to redhat and didnt get around to it | 12:54 |
jangutter | sean-k-mooney, leakypipes: Re: ( https://review.openstack.org/#/c/607610/ ), I left the option open to do the "duelling options" right till the runway engages... mainly to keep progress unblocked, with the idea to update the spec with the decision made. | 12:55 |
sean-k-mooney | jangutter: i personally hate updateing spec retroactivly. so im happy to conceed just to avoid that | 12:56 |
*** erlon has quit IRC | 12:56 | |
sean-k-mooney | well almost | 12:56 |
jangutter | sean-k-mooney, leakypipes: I have been selling tickets, but I'm pretty sure I can track down the people and refund them with only a limited loss of kneecaps. | 12:57 |
sean-k-mooney | jangutter: if you want to continue the discussion over code review then we can | 12:58 |
sean-k-mooney | i would like to see if we can merge the spec soon however and move on to th code as we only have until midle of febuary to release os-vif and i would prefer to have this mered earlier then later. | 12:59 |
sean-k-mooney | e.g. the code | 12:59 |
jangutter | sean-k-mooney, leakypipes: on that note, do you think the spec is in a good enough shape for a wider review? | 12:59 |
jangutter | sean-k-mooney: agreed on that, want to fixup the code early next week to match the spec in any case. | 13:00 |
*** ttsiouts has joined #openstack-nova | 13:00 | |
sean-k-mooney | jangutter: i would like to see if we can get an os-vif release in the next 2 weeks and alost another one at the end of january to give some time to test it with nova, so sound good. | 13:01 |
sean-k-mooney | jangutter: i think the spec i fine for wider review | 13:02 |
leakypipes | jangutter: I do, which is what I said in the review last night :) | 13:05 |
*** tbachman has joined #openstack-nova | 13:07 | |
jangutter | leakypipes, sean-k-mooney: thanks! Any idea who would be sufficiently interested and ... I want to say "still haven't lost hope in all that is good"? | 13:09 |
*** ratailor has quit IRC | 13:11 | |
*** erlon has joined #openstack-nova | 13:11 | |
leakypipes | jangutter: give it a few hours. you'll have lost all faith in humanity by that point. | 13:15 |
jangutter | leakypipes: you're describing what I call "Monday". | 13:17 |
leakypipes | happy Friday. | 13:17 |
sean-k-mooney | jangutter: well i woudl recommend starting with the nova specs core team since they are the only people that can give you the remaining +2 and +w | 13:17 |
*** ShilpaSD has joined #openstack-nova | 13:18 | |
sean-k-mooney | jangutter: dansmith might have some input on ovo versionng and the inheritance/compostion element of the rest https://review.openstack.org/#/admin/groups/302,members it does not jump out at me as an area the rest are stongly invovled in | 13:20 |
jangutter | sean-k-mooney: (facepalm, didn't notice that nova-specs and nova have different ACLs) | 13:21 |
sean-k-mooney | ya nova specs is much smaller | 13:21 |
*** jackivanov has quit IRC | 13:26 | |
*** annp has quit IRC | 13:29 | |
*** cdent has quit IRC | 13:37 | |
*** mchlumsky has joined #openstack-nova | 13:44 | |
s10 | Hello. We've faced a bug, that is claimed to be fixed by in https://bugs.launchpad.net/nova/+bug/1404867 . I added a comment about conditions, in which it can happen during the bulk instances creation by Heat. | 13:48 |
openstack | Launchpad bug 1404867 in OpenStack Compute (nova) queens "Volume remains in-use status, if instance booted from volume is deleted in error state" [Medium,Fix committed] - Assigned to Mohammed Naser (mnaser) | 13:48 |
s10 | When instances are created in parallel, they can fail to build because of the quota.recheck_quota=True in nova.conf, and then it's impossible to remove volumes without admin intervention, because volumes are stuck in 'attaching' state. | 13:48 |
s10 | What will be the right approach to fix this bug? | 13:48 |
*** udesale has quit IRC | 13:48 | |
*** whoami-rajat has quit IRC | 13:49 | |
*** mlavalle has joined #openstack-nova | 13:49 | |
*** udesale has joined #openstack-nova | 13:49 | |
*** ttsiouts has quit IRC | 13:54 | |
*** takamatsu has joined #openstack-nova | 13:55 | |
*** ttsiouts has joined #openstack-nova | 13:55 | |
*** slaweq has joined #openstack-nova | 13:56 | |
*** ttsiouts has quit IRC | 14:00 | |
*** nicholas has joined #openstack-nova | 14:02 | |
s10 | I can reproduce this bug even without Heat, with small terraform config... | 14:06 |
mnaser | s10: what release? | 14:08 |
s10 | mnaser: I tried it with the Pike. I can try it with the Queens, if those fixes weren't backported to Pike. | 14:09 |
maciejjozefczyk | sean-k-mooney: Hey, I finally took time to check https://review.openstack.org/#/c/591607 | 14:10 |
maciejjozefczyk | sean-k-mooney: If you have a minute please look also at: https://review.openstack.org/#/c/614167 | 14:12 |
maciejjozefczyk | Its related | 14:12 |
sean-k-mooney | maciejjozefczyk: hi | 14:12 |
sean-k-mooney | yes i saw the follow up patch | 14:12 |
sean-k-mooney | i only tested the first one because it was a single node deployment and i did not have to worry about updgrades | 14:13 |
maciejjozefczyk | Ok | 14:14 |
sean-k-mooney | so just looking at https://review.openstack.org/#/c/591607/9/nova/network/neutronv2/api.py@2914 | 14:14 |
sean-k-mooney | the value of the port id will be a neutron port uuid not an int right | 14:15 |
*** ttsiouts has joined #openstack-nova | 14:15 | |
sean-k-mooney | or is it a nova port id form the nova virtural interface table | 14:16 |
maciejjozefczyk | this 'id' is a sequence from table | 14:17 |
maciejjozefczyk | so it will always be int | 14:17 |
sean-k-mooney | ok what i was concerned about really was trying to maintian the order of interface withing the guest if possible | 14:18 |
*** etp has quit IRC | 14:18 | |
sean-k-mooney | but to be honest i dont think there is a good way to do that so the code as written is proably fine | 14:18 |
maciejjozefczyk | https://github.com/openstack/nova/blob/master/nova/objects/virtual_interface.py#L36 | 14:18 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Deprecate the unversioned notifications https://review.openstack.org/603079 | 14:19 |
sean-k-mooney | ya i assume it was this object | 14:19 |
maciejjozefczyk | For me its not a ideal solution also, but its only one source of real order of interfaces, that I know in nova | 14:19 |
*** mriedem has joined #openstack-nova | 14:19 | |
maciejjozefczyk | It was like first idea how to make it after Vlad comment | 14:20 |
s10 | mnaser: Same thing with Queens. In Pike volumes are stuck in attaching state, in Queens - in reserved. | 14:20 |
mnaser | s10: indeed sounds like a bug | 14:21 |
sean-k-mooney | maciejjozefczyk: ya if you are sorting by that id and not the uuid i think the code makes sense | 14:21 |
mnaser | unfortunately i dont run this setting on and i'm a bit out of cycles to dig in deeper :X | 14:21 |
sean-k-mooney | i can leve that feedback on the reivew | 14:21 |
s10 | mnaser: and easy to reproduce. Just set CPU quota for the project to 1 and then create 2 instances with 1vcpu each in parallel. | 14:22 |
*** etp has joined #openstack-nova | 14:22 | |
mnaser | s10: oh i can see it but digging into solution and all, but out of time for that :) | 14:22 |
maciejjozefczyk | sean-k-mooney: thanks | 14:23 |
maciejjozefczyk | s10: If you have a minute PTAL https://review.openstack.org/#/c/591607/ | 14:23 |
sean-k-mooney | maciejjozefczyk: part of my confustion was we also have this object to model vifs in nova https://github.com/openstack/nova/blob/master/nova/network/model.py#L378 and the id field there is the neutron port uuid | 14:24 |
gibi | jackding: ping | 14:24 |
maciejjozefczyk | sean-k-mooney: hmm, idk whats that, but anyway this model is not used there | 14:26 |
sean-k-mooney | its how we model neutron port when talking to the virt dirivers. | 14:27 |
sean-k-mooney | maciejjozefczyk: it is also the object that is contained in the networkinfo object https://github.com/openstack/nova/blob/master/nova/network/model.py#L485 and i had assuemd in the netwrok info cache | 14:29 |
maciejjozefczyk | ok, so its clearly some duplication of data :) | 14:30 |
*** janki has quit IRC | 14:30 | |
mriedem | is mdbooth around today? | 14:30 |
sean-k-mooney | maciejjozefczyk: VirtualInterface i think is a legacy object we have for nova networks and it is what we persit in the db | 14:31 |
mriedem | sean-k-mooney: we also create virtualinterfaces in the db for neutron ports since newton | 14:31 |
sean-k-mooney | maciejjozefczyk: the other object are the ones we construct form neutron and use for port binding and creating the os-vif objects | 14:31 |
maciejjozefczyk | sean-k-mooney: mriedem said that its about storing 'tag' | 14:31 |
sean-k-mooney | mriedem: we do yes | 14:31 |
mriedem | it is | 14:31 |
*** cdent has joined #openstack-nova | 14:32 | |
sean-k-mooney | maciejjozefczyk: right for the device role tagging spec | 14:32 |
sean-k-mooney | mriedem: does the netwrok info cache contain the VIF form here https://github.com/openstack/nova/blob/master/nova/network/model.py#L378 or the virtualInterface objecst from here https://github.com/openstack/nova/blob/master/nova/objects/virtual_interface.py#L28 | 14:33 |
maciejjozefczyk | I used this field, btw, to store the last port I verified during online_data_migrations, same way as bauzas did for other migrations | 14:33 |
bauzas | mmm ? | 14:33 |
mriedem | the vif | 14:34 |
mriedem | https://github.com/openstack/nova/blob/master/nova/network/model.py#L488 | 14:34 |
sean-k-mooney | thats what i had understood too | 14:34 |
mriedem | https://github.com/openstack/nova/blob/master/nova/objects/instance_info_cache.py#L40 | 14:34 |
sean-k-mooney | so the id filed that is being sorted on here https://review.openstack.org/#/c/591607/9/nova/network/neutronv2/api.py@2914 is the neutron port uuid that is stored in the VIF objects id field | 14:35 |
bauzas | shit, anyone knows why my nova-specs dashboard doesn't work ? | 14:36 |
*** rambo_li has joined #openstack-nova | 14:36 | |
*** lbragstad is now known as elbragstad | 14:36 | |
bauzas | my own dashboard goo.gl/MgN7mp | 14:36 |
sean-k-mooney | maciejjozefczyk so that code is sorting on the uuid in the VIF.id field not the id of the VirtualInterface id field | 14:37 |
maciejjozefczyk | bauzas: I took your logic from https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py to have a marker, thanks btw | 14:37 |
bauzas | ah ok | 14:37 |
bauzas | http://goo.gl/MgN7mp is my own dashboard | 14:37 |
maciejjozefczyk | sean-k-mooney: so how to preserve the order after attaching new interface? if it sorts by VIF.id (that is uuid?)? | 14:38 |
*** whoami-rajat has joined #openstack-nova | 14:38 | |
sean-k-mooney | maciejjozefczyk: actully no you are ok you are build it from the db https://review.openstack.org/#/c/591607/9/nova/network/neutronv2/api.py@2356 | 14:39 |
maciejjozefczyk | sean-k-mooney: about get_vifs_by_instance: ok | 14:39 |
*** eharney has joined #openstack-nova | 14:42 | |
sean-k-mooney | maciejjozefczyk: so yes _get_ordered_port_list is constuting an orderd port list from the VirtualInterface objects retruned form get_vifs_by_instance and then in _build_network_info_model you are construting a NetworkInfo object containing VIF objects | 14:43 |
sean-k-mooney | maciejjozefczyk: so ya the logic is correct | 14:44 |
*** takamatsu has quit IRC | 14:48 | |
*** brault has quit IRC | 14:51 | |
maciejjozefczyk | thats right :) | 14:53 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Recalculate request group - RP mapping during re-schedule https://review.openstack.org/619529 | 14:54 |
*** ttsiouts has quit IRC | 14:54 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding https://review.openstack.org/569459 | 14:54 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317 | 14:54 |
*** ttsiouts has joined #openstack-nova | 14:55 | |
kashyap | If someone has time, a refactor that makes your brain less warpy: https://review.openstack.org/#/c/620327/ ("libvirt: Refactor handling of PCIe root ports") | 14:56 |
*** ttsiouts has quit IRC | 14:56 | |
*** ttsiouts has joined #openstack-nova | 14:57 | |
*** ttsiouts has quit IRC | 14:58 | |
*** awaugama has joined #openstack-nova | 15:00 | |
gibi | kashyap: +2, thanks! | 15:07 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Mention size limit on user data in docs https://review.openstack.org/620700 | 15:07 |
kashyap | gibi: Sweet, thank you, sir. | 15:07 |
kashyap | That means, then I _really_ need to workout the fix to address the more complex TODO item in the comment. | 15:08 |
s10 | mriedem: We've faced a bug, which is related to https://bugs.launchpad.net/nova/+bug/1404867 . I've written about it in last comment, but I'm not sure, if I should open a new bug about this issue. We can't set quota.recheck_quota=False in our deployments, and some users sometimes ends up with volumes in incorrect status. | 15:08 |
openstack | Launchpad bug 1404867 in OpenStack Compute (nova) queens "Volume remains in-use status, if instance booted from volume is deleted in error state" [Medium,Fix committed] - Assigned to Mohammed Naser (mnaser) | 15:08 |
mriedem | s10: which release? because i'm pretty sure mnaser already fixed that | 15:10 |
s10 | mriedem: Pike and Queens. | 15:10 |
mriedem | oh i'm thinking of I64074923fb741fbf5459f66b8ab1a23c16f3303f | 15:11 |
mriedem | https://review.openstack.org/#/c/544748/ | 15:11 |
s10 | mriedem: this issue is almost fixed, but still can happen, if this exception happens https://github.com/openstack/nova/blob/stable/rocky/nova/conductor/manager.py#L1308 | 15:12 |
mriedem | s10: open a new bug, you can refer to ^ as being related but a different failure | 15:12 |
s10 | mriedem: because after this exception instance is not being buried in cell0 | 15:12 |
mriedem | lyarwood: bauzas: this upgrade-impacting backport needs review https://review.openstack.org/#/c/610974/ | 15:13 |
mriedem | s10: yeah i get it | 15:13 |
mriedem | just report a new bug | 15:13 |
bauzas | mriedem: on a meeting but looking | 15:13 |
mriedem | bauzas: while you're on the call, you want to ask mdbooth if he'll be around the next two weeks to respond to review comments on https://blueprints.launchpad.net/nova/+spec/local-disk-serial-numbers if it's in a runway? | 15:14 |
s10 | mriedem: ok, I will report. | 15:14 |
mriedem | because it's next in the queue | 15:14 |
bauzas | mriedem: I can try to ask :) | 15:14 |
mriedem | i know you can do this! | 15:14 |
mriedem | be strong! | 15:14 |
bauzas | I'm a weak guy | 15:15 |
*** mdbooth has joined #openstack-nova | 15:15 | |
mriedem | you're a bear | 15:15 |
mriedem | big ol papa bear | 15:15 |
bauzas | maybe more a Winnie the Bear one | 15:15 |
bauzas | on another point, I finally found how to fix my specs dashboard :) | 15:16 |
openstackgerrit | Chris Dent proposed openstack/nova master: Remove placement perf check https://review.openstack.org/621180 | 15:16 |
mdbooth | mriedem bauzas: I'm not working Mondays for the next few weeks, but Tuesday->Friday I can prioritise responding to reviews. | 15:18 |
bauzas | so, #1 I fixed my dashboard, #2 I have less downstream tasks to do, #3 there are a huge number of specs left to review | 15:19 |
bauzas | good news, I'll sharpen my pen | 15:19 |
mriedem | mdbooth: ok. i had some comments at one point on one of the changes in that blueprint which was merged in rocky (i think) and i don't think i ever heard back. i'll try to dig that up. | 15:21 |
mriedem | i may or may not be grinding an axe because of it | 15:22 |
*** moshele has joined #openstack-nova | 15:22 | |
mriedem | anyway, your bp is in the runways queue now | 15:22 |
mriedem | good luck | 15:22 |
mdbooth | mriedem: I rebased and addressed all prior feedback a few weeks back, so hopefully all good. | 15:23 |
mriedem | i meant this i think https://review.openstack.org/#/c/528362/ | 15:23 |
coreycb | sean-k-mooney: do you recall if the stestr change to stable/pike was supposed get reverted? | 15:27 |
*** cfriesen has joined #openstack-nova | 15:28 | |
*** takamatsu has joined #openstack-nova | 15:29 | |
leakypipes | mdbooth: what is "a V2V workflow"? | 15:29 |
mdbooth | leakypipes: Oops, jargon. | 15:29 |
mdbooth | leakypipes: Moving workloads between 2 different hypervisors | 15:30 |
mdbooth | P2V == physical to virtual | 15:30 |
s10 | mriedem: I've written https://bugs.launchpad.net/nova/+bug/1806064 | 15:30 |
openstack | Launchpad bug 1806064 in OpenStack Compute (nova) "Volume remains in attaching/reserved status after TooManyInstances exception in nova-conductor" [Undecided,New] | 15:30 |
leakypipes | bauzas: there can be only one Winnie the Bear... https://twitter.com/jaypipes/status/1012713615097335809 | 15:30 |
mdbooth | leakypipes: It's pretty much the definition of not cloudy. | 15:30 |
mdbooth | But people want to do it, so I ask the question... | 15:30 |
bauzas | leakypipes: I just can't not heart this | 15:31 |
leakypipes | mdbooth: by "two different hypervisors", you don't mean migration, you mean like VMWare to KVM or something like that? | 15:31 |
bauzas | <3 | 15:31 |
leakypipes | bauzas: :) | 15:31 |
mdbooth | leakypipes: E.g. Move a legacy workload from a physical server to an openstack Panda. | 15:31 |
mdbooth | leakypipes: Yeah, or moving stuff from (non-openstack) VMware to Openstack. | 15:32 |
cdent | vast mental resources are being spent on this. it's so weird to me | 15:32 |
bauzas | someday, there will be Dogtivities at Summit | 15:32 |
cdent | monetary and stuff | 15:32 |
cdent | dogtivities++ | 15:32 |
mriedem | s10: thanks; should be pretty easy to write a functional regression test for that scenario. | 15:33 |
jackding | gibi: yes the bp io-semaphore-for-concurrent-disk-ops is done | 15:33 |
gibi | jackding: thank. meanwhile mriedem already marked the bp completed | 15:33 |
gibi | jackding: so the administration is also done :) | 15:34 |
leakypipes | mdbooth: I think what you're describing sounds a little like Ironic's node import functionality. | 15:34 |
jackding | gibi: mriedem: thanks :) | 15:34 |
*** bnemec is now known as beekneemech | 15:34 | |
leakypipes | mdbooth: but what mnaser mentioned about just adding the image to glance makes some sense to me. | 15:34 |
mnaser | also if you use bfv (this might be a bit more of a specific use case scenario) | 15:35 |
mnaser | you can totally do this too | 15:35 |
mdbooth | Glance isn't a great fit, because we never want to create 2 of these things. They're volume backed. | 15:35 |
mdbooth | Thing is when you're mass importing 1000 of these things. | 15:36 |
kashyap | sean-k-mooney: I'm reading this spec that you reviewed, and the "problem description" is not clear to me at all: https://review.openstack.org/#/c/620959/1/specs/stein/approved/cpu-model-selection.rst | 15:36 |
mdbooth | And 50 are powered off. | 15:36 |
*** moshele has quit IRC | 15:36 | |
mnaser | cinder manage volumes and boot the ones that are running only? | 15:36 |
leakypipes | mdbooth: we have this awful register/import script that does similar things for non-OpenStack baremetal workloads... basically making OpenStack aware of them, creating records in Nova/Ironic, etc and we have this awful hack called "--no-reimage" that prevents Ironic from cleaning/imaging the imported nodes (and prevents nova from trying to schedule to them) | 15:36 |
mdbooth | You don't want those 50 to be 'special'. You want them to be exactly the same as all the rest, just not running. | 15:36 |
leakypipes | mdbooth: so, in short, I'm familiar with the ask at least ;) | 15:36 |
*** eharney has quit IRC | 15:36 | |
kashyap | sean-k-mooney: I'll respond on that change once I read through it | 15:37 |
mdbooth | leakypipes: We're copying the data into a VM so the physical can be retired. | 15:37 |
leakypipes | mdbooth: understood. I'm saying we have a similar-ish use case that we've got nasty-hacks for :) so I feel your pain. | 15:37 |
mdbooth | leakypipes: Interesting that we're not the only one creating pandas from legacy workloads. | 15:38 |
* mnaser thinks it might be less-than-trivial to do the create-but-dont-start thing | 15:38 | |
leakypipes | mdbooth: having some sort of "register this thing as a VM but don't do anything with it (yet)" would be useful for getting data/systems into OpenStack I think. | 15:38 |
mriedem | pretty sure there are lots of products that have something that does something like this, | 15:38 |
leakypipes | mdbooth: oh, we have a whole friggin zoo of pandas. | 15:38 |
mriedem | there was a lot of chatter in wechat one night about how openstack didn't have this import feature | 15:38 |
mriedem | especially for DR scenarios | 15:38 |
cdent | DR is the context I keep hearing it too | 15:39 |
mriedem | e.g. operator lost the nova server data, but had the hypervisor/ovs/volume info and just wanted to recreate the records | 15:39 |
leakypipes | well, that's just... well, ewww. but yeah I can kinda see that. | 15:39 |
mnaser | i remember that discussion | 15:39 |
mnaser | it was in the scope of someone's database blowing up | 15:40 |
mriedem | yes | 15:40 |
mnaser | and now everything was running but no records for anything | 15:40 |
leakypipes | (ewww == "guh, you should have a backup of your data") | 15:40 |
mdbooth | The way our v2v works is we create an ephemeral conversion instance, create a volume, the instance connects out to tools on the legacy hypervisor/physical and copies data to the volume. Then it does unspeakable hacks to it. | 15:40 |
mriedem | and the majority of replies were either "openstack should totally be able to fix that for you!" to "why don't you backup your database dummy" | 15:40 |
leakypipes | mriedem: :) | 15:40 |
mdbooth | Then we throw away the conversion instance and create a new one for the migrated workload, using bfv. | 15:40 |
leakypipes | mriedem: I think you know where I stand on that. :P | 15:40 |
mnaser | i mean we can just say keep the volume and boot that when you need it | 15:41 |
leakypipes | mriedem: then again, there's a reason that the mysql command line client has a --i-am-a-dummy option. | 15:41 |
mnaser | but i also dont think that is a really trivial way for users | 15:41 |
mnaser | leakypipes: hah, except only people who are advanced users know about it... :p | 15:41 |
leakypipes | mnaser: I know right? :) | 15:41 |
*** maciejjozefczyk has quit IRC | 15:42 | |
mdbooth | mnaser: Right, but that means you need to keep that metadata around until you want to boot it at some unspecified time. I can understand why they don't like that solution. | 15:42 |
mriedem | beggars can't be choosers | 15:42 |
mnaser | mdbooth: yeah, that's the only reason that i feel like this might make value i guess | 15:42 |
mriedem | well, unless they are beggars with tons of cash | 15:42 |
mnaser | but i also dont think it's a trivial change given the whole logic of nova boot is to boot an instance | 15:43 |
leakypipes | (in case people thought I was joking... https://www.percona.com/blog/2017/03/06/mysql-i-am-a-dummy/) | 15:43 |
mdbooth | mriedem: They have *some* cash :) | 15:43 |
mriedem | mnaser: +1 to that | 15:43 |
*** dansmith is now known as SteelyDan | 15:43 | |
mriedem | this might be a shocker, but given our current review rate to bp ratio, i won't hold my breath that this is something we're going to seriously consider soon, especially given we're just now getting deployments upgrading to the big changes we've worked on the last 2 years and they are having problems with them that we're trying to solve | 15:44 |
mdbooth | mnaser: It looked technically simple enough if we pass a flag in to not boot. I think the change in the libvirt driver, at least, would be relatively small. | 15:44 |
mdbooth | mriedem: ack. That's exactly why I'm not jumping on this. | 15:45 |
mriedem | you're talking about the power_on flag right? we use that to keep a stopped vm shutoff during cold migration | 15:45 |
mnaser | would it? dont we have to shortcircuit the whole: plug port, start up paused, wait for notification, resume thing | 15:45 |
mriedem | we already have plumbing to not wait for the vif plug if we didn't power on the vm | 15:46 |
mnaser | ah okay | 15:46 |
mriedem | it was one of my first big bug fixes in nova | 15:46 |
mnaser | https://github.com/openstack/nova/commit/9e7c7706a76ad76612ba75314d436a8ba419a3eb | 15:47 |
mnaser | my amazing nova start | 15:47 |
leakypipes | :) | 15:48 |
*** mlavalle has quit IRC | 15:50 | |
*** mriedem has quit IRC | 15:50 | |
*** mriedem has joined #openstack-nova | 15:51 | |
*** munimeha1 has joined #openstack-nova | 15:51 | |
mnaser | btw | 15:51 |
mnaser | we have 2 more releases to go | 15:51 |
mnaser | https://review.openstack.org/#/c/619352/ | 15:51 |
*** eharney has joined #openstack-nova | 15:51 | |
mnaser | not that i run pike but it'd be nice to flush it out | 15:52 |
*** mlavalle has joined #openstack-nova | 15:53 | |
*** takamatsu has quit IRC | 15:53 | |
*** moshele has joined #openstack-nova | 15:57 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Default zero disk flavor to RULE_ADMIN_API in Stein https://review.openstack.org/603910 | 15:59 |
mriedem | mnaser: not sure about your thoughts on a nova-status upgrade check for ^ | 16:00 |
mriedem | to see if (1) you have root_gb=0 flavors and (2) your policy checks out for those | 16:01 |
mriedem | and dump a warning if not | 16:01 |
mriedem | i posted to the ML, but likely no one has read it, or grokked it, or cares because they aren't upgraded to that point of caring yet | 16:01 |
mriedem | checking the policy would be kind of tough though if there is a custom rule configured | 16:02 |
mnaser | mriedem: i saw it and mentally responded, i should write up a reply | 16:03 |
mnaser | i'd like to warn against it but also i feel like it might break upgrade tooling in a way | 16:03 |
mnaser | i guess it's about trying to see it as "behavioural change" vs "bug fix" | 16:03 |
mnaser | i wouldn't want to see a warning because my cloud is operating as expected.. maybe a warning in stable/rocky though? | 16:04 |
*** artom is now known as temka | 16:08 | |
openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: Add regression test for bug #1764883 https://review.openstack.org/621199 | 16:08 |
openstack | bug 1764883 in OpenStack Compute (nova) "Evacuation fails if the source host returns while the migration is still in progress" [Undecided,Fix released] https://launchpad.net/bugs/1764883 - Assigned to Lee Yarwood (lyarwood) | 16:08 |
openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: compute: Ensure pre-migrating instances are destroyed during init_host https://review.openstack.org/621200 | 16:08 |
mriedem | mnaser: by operating as expected, you mean you have 0 disk flavors but allow non-admins to use them intentionally even if they aren't booting from volume? | 16:12 |
mriedem | btw, this is another area where https://review.openstack.org/#/c/511965/ would help | 16:13 |
openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: Update docs for _destroy_evacuated_instances https://review.openstack.org/621203 | 16:13 |
openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: Add regression test for bug #1764883 https://review.openstack.org/621204 | 16:13 |
openstack | bug 1764883 in OpenStack Compute (nova) "Evacuation fails if the source host returns while the migration is still in progress" [Undecided,Fix released] https://launchpad.net/bugs/1764883 - Assigned to Lee Yarwood (lyarwood) | 16:13 |
openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: compute: Ensure pre-migrating instances are destroyed during init_host https://review.openstack.org/621205 | 16:13 |
lyarwood | temka: ^ there you go | 16:14 |
*** Sundar has joined #openstack-nova | 16:14 | |
*** erlon has quit IRC | 16:15 | |
Sundar | efried: Please ping me on cyborg channel when you can. Thanks. | 16:16 |
mnaser | mriedem: sorry, i meant i have 0 disk flavors and i am ok with non-admins no longer being able to boot | 16:16 |
*** k_mouza has quit IRC | 16:16 | |
temka | lyarwood, ack | 16:17 |
mriedem | mnaser: yeah in my mind the upgrade check wouldn't warn on that - if the policy is the default, you're ok even with 0-disk flavors. i'd think the check would warn if you have 0-disk flavors and your policy was not the default (admin_api) | 16:19 |
mriedem | but like i said, the policy rule could be custom so it's hard to say | 16:19 |
mnaser | ah i see what you mean, i could have renamed by admin_api to foo_bar and it would warn anyways | 16:19 |
mriedem | right | 16:19 |
mriedem | that's why i'm hesitant to add something | 16:19 |
mnaser | + if someone is actually running their own policy | 16:20 |
mnaser | its very likely that its custom anyways | 16:20 |
*** mriedem has quit IRC | 16:21 | |
*** s10 has quit IRC | 16:21 | |
*** mriedem has joined #openstack-nova | 16:21 | |
*** jangutter has quit IRC | 16:21 | |
*** mchlumsky has quit IRC | 16:21 | |
mriedem | SteelyDan: before i make this change that leakypipes suggested in my cross-cell resize series, do you agree with it or a meh? https://review.openstack.org/#/c/614012/3/nova/db/sqlalchemy/models.py@794 | 16:22 |
SteelyDan | mriedem: HMM | 16:24 |
SteelyDan | so, | 16:25 |
SteelyDan | on the one hand, expanding the schema for a boolean seems kinda silly when we could store something useful | 16:25 |
*** tssurya has quit IRC | 16:25 | |
SteelyDan | BUT, the migration lives in the cell db, and I don't like ever telling the cell what cell it is, which it can kinda infer from that data | 16:25 |
*** pcaruana has joined #openstack-nova | 16:26 | |
SteelyDan | like, imagine you do some migrations and then later collapse or split a cell | 16:26 |
mriedem | as noted in the change, i get the source cell mapping from the host mapping tied to the source_compute in the migration object https://review.openstack.org/#/c/603930/15/nova/conductor/tasks/cross_cell_migrate.py@871 | 16:26 |
mriedem | so i don't *need* the explicit uuid in the migrations table | 16:26 |
SteelyDan | right | 16:27 |
mriedem | the boolean flag is really for flow control | 16:27 |
SteelyDan | I know | 16:27 |
mriedem | i don't know if the source/dest cell uuids being in the migration record could be abused later | 16:27 |
mriedem | i mean, i'm sure they could, i'm just not sure in what case | 16:27 |
*** gyee has joined #openstack-nova | 16:27 | |
SteelyDan | well, it's not a matter of abuse so much as just separation of responsibilities | 16:28 |
mriedem | once the migration is done, we shouldn't really care about the record anymore, except reporting | 16:28 |
mriedem | and if an operator is going to split cells, they had better make sure they don't have migrations going on while doing so | 16:28 |
SteelyDan | yeah, not a matter of ongoing ones, but just those being wrong later | 16:29 |
SteelyDan | nothing else in the cell db knows about its position in the hierarchy and I think we should keep it that way | 16:29 |
SteelyDan | so while I was going to argue against just storing a boolean, | 16:29 |
mriedem | right. i mean, there could be a comment in there saying, "don't trust these after the migration is done"... but that's a bit hokey | 16:29 |
SteelyDan | I think I'd prefer that over the uuids just for that reason | 16:29 |
SteelyDan | heh | 16:29 |
mriedem | SteelyDan: ok care to reply so i can move on? | 16:29 |
*** ShilpaSD has quit IRC | 16:29 | |
SteelyDan | yep | 16:30 |
mriedem | thanks much | 16:30 |
mriedem | <3 | 16:30 |
*** helenaAM has quit IRC | 16:31 | |
*** Sundar has quit IRC | 16:33 | |
*** ccamacho has quit IRC | 16:35 | |
*** ccamacho has joined #openstack-nova | 16:35 | |
*** xek has quit IRC | 16:36 | |
*** k_mouza has joined #openstack-nova | 16:37 | |
*** ttsiouts has joined #openstack-nova | 16:38 | |
openstackgerrit | Corey Bryant proposed openstack/nova stable/pike: Revert "Backport tox.ini to switch to stestr" https://review.openstack.org/621227 | 16:38 |
*** ttsiouts has quit IRC | 16:39 | |
*** ttsiouts has joined #openstack-nova | 16:40 | |
*** ttsiouts has quit IRC | 16:44 | |
*** rambo_li has quit IRC | 16:46 | |
*** _hemna has quit IRC | 16:47 | |
*** _pewp_ has quit IRC | 16:47 | |
*** whoami-rajat has quit IRC | 16:48 | |
*** udesale has quit IRC | 16:49 | |
*** _pewp_ has joined #openstack-nova | 16:55 | |
*** whoami-rajat has joined #openstack-nova | 16:55 | |
*** rnoriega- has quit IRC | 16:56 | |
*** med_ has quit IRC | 16:56 | |
*** _hemna has joined #openstack-nova | 16:57 | |
*** rnoriega has joined #openstack-nova | 16:58 | |
*** eharney has quit IRC | 16:59 | |
*** med_ has joined #openstack-nova | 17:02 | |
*** pcaruana has quit IRC | 17:04 | |
*** hongda has quit IRC | 17:04 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid https://review.openstack.org/614012 | 17:09 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.openstack.org/614035 | 17:09 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036 | 17:09 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037 | 17:09 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353 | 17:09 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Cross-cell resize https://review.openstack.org/603930 | 17:09 |
*** mriedem is now known as mriedem_lunch | 17:11 | |
openstackgerrit | Merged openstack/python-novaclient master: Add Python 3.6 classifier to setup.cfg https://review.openstack.org/621093 | 17:13 |
*** eharney has joined #openstack-nova | 17:14 | |
*** moshele has quit IRC | 17:20 | |
*** mdbooth has quit IRC | 17:22 | |
*** eharney has quit IRC | 17:22 | |
*** erlon has joined #openstack-nova | 17:23 | |
openstackgerrit | Eric Fried proposed openstack/nova master: Reduce calls to placement from _ensure https://review.openstack.org/615677 | 17:26 |
openstackgerrit | Eric Fried proposed openstack/nova master: Consolidate inventory refresh https://review.openstack.org/615695 | 17:26 |
openstackgerrit | Eric Fried proposed openstack/nova master: Commonize _update code path https://review.openstack.org/615705 | 17:26 |
openstackgerrit | Eric Fried proposed openstack/nova master: Rip the report client out of SchedulerClient https://review.openstack.org/617042 | 17:26 |
openstackgerrit | Eric Fried proposed openstack/nova master: Rip out the SchedulerClient https://review.openstack.org/617049 | 17:26 |
openstackgerrit | Eric Fried proposed openstack/nova master: Use a static resource tracker in compute manager https://review.openstack.org/620711 | 17:26 |
efried | leakypipes: This all ought to be ready now ^ | 17:26 |
leakypipes | efried: k | 17:26 |
*** efried is now known as fried_rolls | 17:27 | |
openstackgerrit | Eric Fried proposed openstack/nova master: Turn off rp association refresh in nova-next https://review.openstack.org/616033 | 17:28 |
*** moshele has joined #openstack-nova | 17:30 | |
*** tbachman has quit IRC | 17:35 | |
*** _hemna has quit IRC | 17:36 | |
*** _hemna has joined #openstack-nova | 17:38 | |
*** igordc has joined #openstack-nova | 17:41 | |
*** k_mouza_ has joined #openstack-nova | 17:44 | |
*** k_mouza has quit IRC | 17:47 | |
*** moshele has quit IRC | 17:48 | |
*** k_mouza_ has quit IRC | 17:48 | |
*** openstackgerrit has quit IRC | 17:51 | |
*** derekh has quit IRC | 18:00 | |
*** munimeha1 has quit IRC | 18:02 | |
*** sridharg has quit IRC | 18:04 | |
*** Swami has joined #openstack-nova | 18:09 | |
*** k_mouza has joined #openstack-nova | 18:09 | |
*** k_mouza has quit IRC | 18:14 | |
*** wolverineav has joined #openstack-nova | 18:17 | |
*** sahid has quit IRC | 18:34 | |
*** tbachman has joined #openstack-nova | 18:35 | |
*** ralonsoh has quit IRC | 18:41 | |
*** wolverineav has quit IRC | 19:02 | |
*** erlon has quit IRC | 19:02 | |
*** munimeha1 has joined #openstack-nova | 19:02 | |
*** wolverineav has joined #openstack-nova | 19:06 | |
*** mriedem_lunch is now known as mriedem | 19:13 | |
*** pacharya has joined #openstack-nova | 19:25 | |
pacharya | Hi I ran into weird issue where nova compute received empty port list for an instance during heal instance info cache. This caused the instance info cache for that instance to be set to empty list and now the APIs are not returning any network information. In the subsequent run neutron returned correct port list but nova does not fix the instance info cache so it remains broken. Any ideas on how to fix this? | 19:35 |
SteelyDan | mriedem: ^ | 19:36 |
*** wolverineav has quit IRC | 19:37 | |
mriedem | pacharya: https://review.openstack.org/#/c/591607/ | 19:40 |
mriedem | without ^ the heal_instance_info_cache and network-changed events just refresh the cache from the cache...which is why it doesn't help you | 19:41 |
mriedem | unfortunately you need to rebuild the cache somehow, and off the top of my head i think that might mean attaching/detaching a port to/from the server, | 19:41 |
mriedem | if possible, | 19:41 |
*** wolverineav has joined #openstack-nova | 19:41 | |
mriedem | could you just attach a dummy port to the server and then immediately detach it? | 19:41 |
mriedem | otherwise you might have to try migrating the server, but i'm not sure if that will rebuild the network info cache for the instance | 19:42 |
*** wolverineav has quit IRC | 19:42 | |
*** wolverineav has joined #openstack-nova | 19:42 | |
*** wolverineav has quit IRC | 19:42 | |
*** wolverineav has joined #openstack-nova | 19:43 | |
*** wolverineav has quit IRC | 19:43 | |
*** wolverineav has joined #openstack-nova | 19:43 | |
*** wolverineav has quit IRC | 19:44 | |
mriedem | looking at the code, attaching a new port might just put that port into the cache... | 19:44 |
mriedem | so i'm not really sure what to tell you beside look at https://review.openstack.org/#/c/591607/ | 19:44 |
*** wolverineav has joined #openstack-nova | 19:45 | |
mriedem | i know we have https://docs.openstack.org/python-novaclient/latest/cli/nova.html#nova-reset-network but i don't think that will help | 19:46 |
*** noonedeadpunk has quit IRC | 19:47 | |
*** wolverineav has quit IRC | 19:52 | |
*** wolverineav has joined #openstack-nova | 19:53 | |
pacharya | Thanks ! @mriedem. I did try detaching and attaching the port but that did not seem to help. Will try it again just to be sure. | 19:58 |
*** wolverineav has quit IRC | 19:59 | |
mriedem | pacharya: as i said, i think that at most will just refresh the cache for that single port - but if that's the only port attached to the server then that should be enough | 20:01 |
*** ivve has quit IRC | 20:04 | |
pacharya | Yeah and you are also correct about reset-network. that does not work either. I will look into the fix that you pointed out | 20:06 |
*** eharney has joined #openstack-nova | 20:07 | |
*** whoami-rajat has quit IRC | 20:08 | |
*** jistr has quit IRC | 20:11 | |
*** jistr has joined #openstack-nova | 20:13 | |
*** xek has joined #openstack-nova | 20:19 | |
zzzeek | cdent: so with https://review.openstack.org/#/c/614350/18, you tried putting the oslo fixtures into placement/tests/fixtures.py and it blew up ? | 20:20 |
*** wolverineav has joined #openstack-nova | 20:33 | |
*** slaweq has quit IRC | 20:34 | |
pacharya | mriedem: will this get backported to queens https://review.openstack.org/#/c/591607/ ? | 20:34 |
pacharya | just curious | 20:34 |
*** openstackgerrit has joined #openstack-nova | 20:35 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Use new ``initial_xxx_allocation_ratio`` CONF https://review.openstack.org/602804 | 20:35 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add ratio online data migration when load compute node https://review.openstack.org/613499 | 20:35 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add compute_node ratio online data migration script https://review.openstack.org/609995 | 20:35 |
zzzeek | cdent: ummmm....it doesnt' seem to be provisioning anonymously named databases....odd | 20:36 |
mriedem | leakypipes: i have gone through the first 3 of 5 changes in that series, you might want to start warming up to it sooner than later since it's in a runway slot (for the 2nd time) and you are familiar given it was your spec | 20:36 |
zzzeek | wonder how it works | 20:36 |
mriedem | pacharya: idk, i haven't looked at it in awhile - maciej has taken it over | 20:36 |
mriedem | pacharya: i would think so though if we can | 20:36 |
mriedem | jackding: oh boy the io semaphore unit test changes are causing a race http://logs.openstack.org/22/606122/7/check/openstack-tox-py27/d70a4d5/testr_results.html.gz | 20:40 |
mriedem | looks like mdbooths new evacuate functional tests are also racy | 20:48 |
*** openstackgerrit has quit IRC | 20:50 | |
*** fried_rolls is now known as fried_rice | 20:51 | |
*** itlinux has joined #openstack-nova | 20:51 | |
*** takamatsu has joined #openstack-nova | 20:53 | |
cdent | zzzeek: what's there is the result of much experimentation and it does work: i can witness the databases being created and manipulated (turned on query logging in the database logs) | 20:56 |
cdent | the names of things are being overridden | 20:56 |
zzzeek | cdent: OK so it makes anonymously named databases in PG / MySQL for the migration tests ? | 20:56 |
cdent | zzzeek: no, it skips that part because it was getting in way of managing multiople databases in the same process. also, as far as I could tell the anonymous naming was only working for mysql, not for postgresql | 20:57 |
zzzeek | cdent: do you run multiple tests concurrently in a single process? | 20:58 |
sean-k-mooney | zzzeek: in the nova tests we have eventlet so maybe | 20:58 |
*** itlinux_ has joined #openstack-nova | 20:58 | |
cdent | on the postgres side is was always the same name, so given that worked for postgres, I made it work for mysql and sqlite too, and then locked the tests with an external lock so there aren't concurrency issues between processes. There aren't concurrent tests in the same process but there is concurrency inter-process | 20:58 |
*** itlinux has quit IRC | 20:59 | |
zzzeek | cdent: yes that is all normal | 20:59 |
cdent | sean-k-mooney: we're talking about the placement side right now | 20:59 |
zzzeek | cdent: OK so the oslo_db way here is good to go, so I will see if i can make it work | 20:59 |
zzzeek | cdent: also there's just one enginefacade in placement, it seems | 20:59 |
zzzeek | cdent: which also makes this straightforwad, I see normal create_all() for sqlite then dispose on the end, that's all good | 20:59 |
sean-k-mooney | cdent: ah ok well im caling it a day anyway. | 21:00 |
cdent | zzzeek: when we talked about this before (on that review) you said that the existing fixtures wouldn't be quite adequate, which is how what's there ended up the way it did. you said it was "probably good enough| | 21:00 |
zzzeek | cdent: i belive that's because early on I thougth i was looking at nova :) | 21:01 |
cdent | zzzeek: but if there are ways it can be made more tidy, that would be great | 21:01 |
zzzeek | cdent: the oslo_db fixtures should work here fully and if they dont thats a bug in oslo.db i have to fix | 21:01 |
cdent | btw: we've got #openstack-placement now for this sort of thing | 21:01 |
zzzeek | cdent: I had guessed that but was too lazy to try :) | 21:01 |
cdent | I know how that can be | 21:01 |
cdent | If you traverse the several rounds of me trying different things on that review, where it went wrong may be obvious to me | 21:02 |
cdent | lots of different things were tried so it is entirely possible that multiple factors are making it icky, but it really came down to one issue: trying to have all three of sqlite, postgres and mysql in the same process, in-series | 21:02 |
mriedem | https://bugs.launchpad.net/nova/+bug/1806126 | 21:02 |
openstack | Launchpad bug 1806126 in OpenStack Compute (nova) "LibvirtRbdEvacuateTest and LibvirtFlatEvacuateTest tests race fail" [High,Confirmed] | 21:02 |
mriedem | we probably want to revert on that one ^ | 21:03 |
mriedem | https://review.openstack.org/#/c/591733/ | 21:03 |
mriedem | any cores around for revert discussoin? | 21:03 |
mriedem | friday is the best time for us to fuck the gate over | 21:03 |
cdent | zzzeek: the in series is what caused the need for the complicated reset business. the situation has probably changed somewhat now that https://review.openstack.org/#/c/619121/ has merged, which changes the Database handling-in-tests | 21:03 |
zzzeek | cdent: the reset handling here doesn't look that complicated. nova had soemthig much more exotic | 21:04 |
cdent | i meant the stuff where _started is being manipulated | 21:04 |
mriedem | zzzeek: unrelated, do you know much about group by and distinct clauses in postgresql? | 21:04 |
zzzeek | mriedem: sure | 21:05 |
mriedem | zzzeek: any sort of knowledge on https://review.openstack.org/#/c/619061/ for that would be great | 21:05 |
zzzeek | mriedem: i would say, it's unusual to be mixing those two things | 21:05 |
mriedem | my fix is basically based on internet forums... | 21:05 |
zzzeek | mriedem: also....i dont follow that often, are we supporting postgresql again in openstack ? | 21:05 |
mriedem | i believe the official stance is "meh" | 21:05 |
zzzeek | mriedem: OK so no change | 21:06 |
mriedem | we don't turn away fixes | 21:06 |
mriedem | but we don't bend over backwards to test and support it either | 21:06 |
cdent | my understanding is that every now and again mriedem and I try to make it happen | 21:06 |
mriedem | some neutron and manila people apparently pay attention to pg also | 21:06 |
zzzeek | mriedem: um, distinct(col_expr) doesnt work on mysql or sqlite unless I'm deeply confused | 21:07 |
zzzeek | mriedem: how is that working? | 21:07 |
mriedem | i assumed the underlying sqla plugins were just nooping or something, idk | 21:07 |
mriedem | i threw up a fix and it made the pg job work again | 21:07 |
zzzeek | mriedem: also...i dont see why this query uses group_by | 21:07 |
cdent | zzzeek: it's late and friday for me, so I'm going to watch bad tv and fall asleep. If you've got thoughts on that placement alembic stuff, please let me know, either on that review or on some wip that you invite me to look at | 21:07 |
mriedem | as noted in my comments, it might not be the best query to start with | 21:07 |
cdent | everyone have a good weekend | 21:07 |
mriedem | and simply rewriting the query could fix it for both | 21:07 |
mriedem | o/ | 21:08 |
* cdent waves | 21:08 | |
*** wolverineav has quit IRC | 21:08 | |
zzzeek | mriedem: does InstanceMapping have any column_property() that points to like func.max() or func.count() or an aggregate of some kind ? | 21:10 |
zzzeek | cdent: my plan is to make this work and put up a review | 21:10 |
zzzeek | cdent: or if oslo_db is broke, do taht first | 21:10 |
cdent | zzzeek: great, whichever it is point it out to me, will be very curious | 21:11 |
mriedem | zzzeek: this is the model https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api_models.py#L131 | 21:12 |
mriedem | so i think the answer is "no" | 21:13 |
zzzeek | mriedem: sure, it would be extremely weird to map an aggregate | 21:14 |
*** openstackgerrit has joined #openstack-nova | 21:15 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert "Add regression test for bug 1550919" https://review.openstack.org/621289 | 21:15 |
openstack | bug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9) | 21:15 |
*** awaugama has quit IRC | 21:29 | |
*** pacharya has quit IRC | 21:31 | |
*** wolverineav has joined #openstack-nova | 21:35 | |
*** tbachman has quit IRC | 21:38 | |
*** munimeha1 has quit IRC | 21:47 | |
mriedem | SteelyDan: leakypipes: there is some sage advice on here that i wanted to share with you https://www.siouxfallsshoppingnews.com/publishers-message.html | 21:50 |
mriedem | "Life is simpler when you plow around the stump." | 21:51 |
mriedem | i can tell you privately what i think that means | 21:51 |
SteelyDan | um. | 21:56 |
openstackgerrit | Merged openstack/nova stable/rocky: Don't use private interface in oslo.service https://review.openstack.org/619360 | 21:57 |
*** wolverineav has quit IRC | 21:59 | |
*** wolverineav has joined #openstack-nova | 22:04 | |
*** wolverineav has quit IRC | 22:12 | |
*** itlinux_ has quit IRC | 22:13 | |
*** wolverineav has joined #openstack-nova | 22:13 | |
*** wolverineav has quit IRC | 22:18 | |
*** wolverineav has joined #openstack-nova | 22:19 | |
openstackgerrit | Eric Fried proposed openstack/nova master: Reduce calls to placement from _ensure https://review.openstack.org/615677 | 22:20 |
openstackgerrit | Eric Fried proposed openstack/nova master: Consolidate inventory refresh https://review.openstack.org/615695 | 22:20 |
openstackgerrit | Eric Fried proposed openstack/nova master: Commonize _update code path https://review.openstack.org/615705 | 22:20 |
openstackgerrit | Eric Fried proposed openstack/nova master: Rip the report client out of SchedulerClient https://review.openstack.org/617042 | 22:20 |
openstackgerrit | Eric Fried proposed openstack/nova master: Rip out the SchedulerClient https://review.openstack.org/617049 | 22:20 |
openstackgerrit | Eric Fried proposed openstack/nova master: Use a static resource tracker in compute manager https://review.openstack.org/620711 | 22:20 |
openstackgerrit | Merged openstack/nova stable/queens: Handle missing marker during online data migration https://review.openstack.org/610974 | 22:27 |
*** wolverineav has quit IRC | 22:28 | |
*** wolverineav has joined #openstack-nova | 22:29 | |
*** slaweq has joined #openstack-nova | 22:36 | |
*** xek has quit IRC | 22:37 | |
*** slaweq has quit IRC | 22:43 | |
*** slaweq has joined #openstack-nova | 22:50 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid https://review.openstack.org/614012 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.openstack.org/614035 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Cross-cell resize https://review.openstack.org/603930 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Isolate cell-targeting code in MigrationTask https://review.openstack.org/621310 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Extract compute API _create_image to compute.utils https://review.openstack.org/621311 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Extract shelve API logic to compute.utils https://review.openstack.org/621312 | 22:51 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method https://review.openstack.org/621313 | 22:51 |
mriedem | oooo under 2K LOC now | 22:52 |
mriedem | winning | 22:52 |
*** slaweq has quit IRC | 22:52 | |
*** mriedem has quit IRC | 22:54 | |
*** wolverineav has quit IRC | 22:54 | |
*** slaweq has joined #openstack-nova | 22:54 | |
*** wolverineav has joined #openstack-nova | 22:55 | |
*** wolverineav has quit IRC | 23:00 | |
*** wolverineav has joined #openstack-nova | 23:00 | |
*** Nel1x has quit IRC | 23:01 | |
*** burt has quit IRC | 23:03 | |
*** Miouge- has quit IRC | 23:05 | |
*** cdent has quit IRC | 23:07 | |
*** itlinux has joined #openstack-nova | 23:07 | |
*** Miouge has joined #openstack-nova | 23:08 | |
*** slaweq has quit IRC | 23:22 | |
*** Swami has quit IRC | 23:27 | |
*** threestrands has joined #openstack-nova | 23:28 | |
*** wolverineav has quit IRC | 23:42 | |
*** wolverineav has joined #openstack-nova | 23:45 | |
*** igordc has quit IRC | 23:48 | |
*** wolverineav has quit IRC | 23:48 | |
*** wolverineav has joined #openstack-nova | 23:49 | |
*** slaweq has joined #openstack-nova | 23:54 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!