Monday, 2019-10-14

*** jawad_axd has quit IRC00:01
*** bbowen has quit IRC00:29
*** tbachman has joined #openstack-nova00:46
*** jawad_axd has joined #openstack-nova00:59
*** jawad_axd has quit IRC01:04
*** tbachman has quit IRC01:17
*** jawad_axd has joined #openstack-nova01:20
*** yaawang has quit IRC01:24
*** jawad_axd has quit IRC01:24
*** gbarros has quit IRC01:25
*** yaawang has joined #openstack-nova01:25
*** markvoelker has joined #openstack-nova01:32
*** brinzhang has joined #openstack-nova01:32
*** jawad_axd has joined #openstack-nova01:41
*** markvoelker has quit IRC01:42
*** markvoelker has joined #openstack-nova01:43
*** ircuser-1 has quit IRC01:43
*** jawad_axd has quit IRC01:45
*** markvoelker has quit IRC01:47
*** brinzhang_ has joined #openstack-nova02:00
*** gbarros has joined #openstack-nova02:02
*** brinzhang has quit IRC02:03
*** SonPham has joined #openstack-nova02:09
*** BjoernT has joined #openstack-nova02:11
SonPhamHi. i wrote some script on horizon to show instance info. But dev environment is devstack.02:11
SonPhami want to write code on nova02:12
SonPhamhave any docs about nova api?02:12
*** brinzhang has joined #openstack-nova02:19
*** brinzhang_ has quit IRC02:23
*** yaawang has quit IRC02:24
*** yaawang has joined #openstack-nova02:27
SonPhamHi. i wrote some script on horizon to show instance info. But dev environment is devstack.02:28
*** brinzhang_ has joined #openstack-nova02:40
*** jawad_axd has joined #openstack-nova02:43
*** brinzhang has quit IRC02:44
*** jawad_axd has quit IRC02:48
mordredSonPham: https://docs.openstack.org/api-ref/compute/02:56
*** gbarros has quit IRC02:58
*** jawad_axd has joined #openstack-nova03:04
*** jawad_axd has quit IRC03:08
*** gbarros has joined #openstack-nova03:26
*** psachin has joined #openstack-nova03:35
*** brinzhang_ has quit IRC03:37
*** brinzhang_ has joined #openstack-nova03:37
*** gbarros has quit IRC03:39
*** gbarros has joined #openstack-nova03:42
*** brinzhang has joined #openstack-nova03:44
*** mkrai has joined #openstack-nova03:45
*** SonPham has quit IRC03:45
*** brinzhang_ has quit IRC03:47
*** gbarros has quit IRC03:47
*** gbarros has joined #openstack-nova04:01
*** gbarros has quit IRC04:05
*** udesale has joined #openstack-nova04:07
*** brinzhang_ has joined #openstack-nova04:07
*** brinzhang has quit IRC04:10
*** ricolin has joined #openstack-nova04:12
*** gbarros has joined #openstack-nova04:14
*** ricolin_ has joined #openstack-nova04:15
*** dave-mccowan has quit IRC04:18
*** ricolin has quit IRC04:19
*** BjoernT has quit IRC04:24
*** brinzhang has joined #openstack-nova04:28
openstackgerritmelanie witt proposed openstack/nova master: Add regression test for bug 1824435  https://review.opendev.org/68820504:29
openstackbug 1824435 in OpenStack Compute (nova) "fill_virtual_interface_list migration fails on second attempt" [Medium,In progress] https://launchpad.net/bugs/1824435 - Assigned to melanie witt (melwitt)04:29
openstackgerritmelanie witt proposed openstack/nova master: Use a separate transaction to read default security group  https://review.opendev.org/68820604:29
*** brinzhang_ has quit IRC04:31
*** lpetrut has joined #openstack-nova04:42
*** yaawang has quit IRC04:52
*** lpetrut has quit IRC04:52
*** yaawang has joined #openstack-nova04:52
*** brinzhang_ has joined #openstack-nova05:44
*** brinzhang has quit IRC05:47
*** larainema has joined #openstack-nova05:51
openstackgerritmelanie witt proposed openstack/nova master: Add regression test for bug 1824435  https://review.opendev.org/68820506:05
openstackbug 1824435 in OpenStack Compute (nova) "fill_virtual_interface_list migration fails on second attempt" [Medium,In progress] https://launchpad.net/bugs/1824435 - Assigned to melanie witt (melwitt)06:05
openstackgerritmelanie witt proposed openstack/nova master: Use a separate transaction to read default security group  https://review.opendev.org/68820606:05
*** ccamacho has joined #openstack-nova06:10
*** jawad_axd has joined #openstack-nova06:13
*** dpawlik has joined #openstack-nova06:23
openstackgerritSundar Nadathur proposed openstack/nova master: ksa auth conf and client for Cyborg access  https://review.opendev.org/63124206:23
openstackgerritSundar Nadathur proposed openstack/nova master: Add Cyborg device profile groups to request spec.  https://review.opendev.org/63124306:23
openstackgerritSundar Nadathur proposed openstack/nova master: Create and bind Cyborg ARQs.  https://review.opendev.org/63124406:23
openstackgerritSundar Nadathur proposed openstack/nova master: Get resolved Cyborg ARQs and add PCI BDFs to VM's domain XML.  https://review.opendev.org/63124506:23
openstackgerritSundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted.  https://review.opendev.org/67373506:23
openstackgerritSundar Nadathur proposed openstack/nova master: [WIP] add cyborg tempest job  https://review.opendev.org/67099906:23
*** pcaruana has joined #openstack-nova06:43
*** maciejjozefczyk has joined #openstack-nova06:46
*** slaweq has joined #openstack-nova07:04
*** tesseract has joined #openstack-nova07:04
*** rcernin has quit IRC07:11
*** damien_r has joined #openstack-nova07:13
*** rpittau|afk is now known as rpittau07:40
*** jangutter has joined #openstack-nova07:42
*** markvoelker has joined #openstack-nova07:45
*** ivve has joined #openstack-nova07:46
*** markvoelker has quit IRC07:51
*** ralonsoh has joined #openstack-nova07:54
*** brinzhang has joined #openstack-nova07:57
gibigood morning nova07:58
*** awalende has joined #openstack-nova08:00
*** brinzhang_ has quit IRC08:00
bauzasgibi: good morning :)08:02
gibi:)08:02
*** dpawlik has quit IRC08:03
*** brinzhang_ has joined #openstack-nova08:05
*** brinzhang has quit IRC08:08
openstackgerritYongli He proposed openstack/nova master: Clean up orphan instances virt driver  https://review.opendev.org/64891208:11
openstackgerritYongli He proposed openstack/nova master: clean up orphan instances  https://review.opendev.org/62776508:11
*** lpetrut has joined #openstack-nova08:13
*** brinzhang has joined #openstack-nova08:24
*** brinzhang_ has quit IRC08:27
*** CeeMac has joined #openstack-nova08:33
*** dtantsur|afk is now known as dtantsur08:34
*** dpawlik has joined #openstack-nova08:38
*** derekh has joined #openstack-nova08:38
*** gbarros has quit IRC08:38
stephenfinbauzas: Gooood morning. I have a another real easy patch for you here, if you want a stats boost for the morning https://review.opendev.org/#/c/677969/08:39
bauzasstephenfin: heh, I took my vitamins this morning but sure08:39
* bauzas refrains the idea to comment on "we don't distrubute eggs" about explaining how collective farms work08:41
*** ricolin_ is now known as ricolin08:41
*** brinzhang_ has joined #openstack-nova08:41
*** dpawlik has quit IRC08:43
*** brinzhang has quit IRC08:44
*** xek_ has joined #openstack-nova08:47
*** brinzhang has joined #openstack-nova08:57
*** brinzhang_ has quit IRC09:00
*** dpawlik has joined #openstack-nova09:13
*** dpawlik has quit IRC09:18
*** trident has quit IRC09:23
*** trident has joined #openstack-nova09:28
*** dpawlik has joined #openstack-nova09:30
*** brtknr_ has quit IRC09:31
*** brtknr has joined #openstack-nova09:32
*** ricolin has quit IRC09:41
*** brinzhang_ has joined #openstack-nova09:51
*** tbachman has joined #openstack-nova09:52
*** ociuhandu has joined #openstack-nova09:53
*** brinzhang has quit IRC09:54
*** tbachman has quit IRC10:13
*** cdent has joined #openstack-nova10:24
*** ociuhandu has quit IRC10:24
*** SonPham has joined #openstack-nova10:24
SonPhamhi10:24
SonPhamhow this code work?10:24
SonPham    def reboot_instance(self, ctxt, instance, block_device_info,10:24
SonPhamblock_device_info=block_device_info,10:24
*** SonPham has quit IRC10:25
*** mdbooth has quit IRC10:26
*** SonPham has joined #openstack-nova10:26
*** mdbooth has joined #openstack-nova10:28
SonPhamhi . i just traced nova-compute code10:28
SonPhamfrom nova-compute (api.py) call to rpcapi.py10:29
SonPhambut i can;t trace how it call to manager.py10:29
*** ociuhandu has joined #openstack-nova10:29
gibiSonPham: ComputeManager.reboot_instance() is called via RPC from ComputeAPI.reboot_instance()10:32
SonPhamgibi can i add more RPCapi?10:33
SonPhami'm finding docs about this10:33
gibiSonPham: there is some documentation here https://docs.openstack.org/nova/latest/reference/rpc.html10:34
jkulikHi, what's the expected outcome if I create a server with imageRef and a boot-from-volume block_device_mapping_v2 entry in the same request? I know some code checks for instance.image_ref for bfv. But it looks like in this case image_ref is still set even though the bfv disk is kept.10:34
gibiSonPham: but this is even better from dansmith http://www.danplanet.com/blog/2015/10/05/upgrades-in-nova-rpc-apis/10:35
kashyapgibi: I wouldn't normally randomly spam, but mind taking this no-risk (can explain) backport for Queens?  -- https://review.opendev.org/#/c/668111/10:36
gibikashyap: I can read it but note that I don't have +2 on stable branches10:37
kashyapgibi: It already has +2, FWIW.  I'd have asked Eric, as he ACKed the change before, but he's on PTO10:37
kashyapAargh.  Who is awak that can ACK it now?10:37
kashyaps/awak/awake/10:37
gibikashyap: https://review.opendev.org/#/admin/groups/540,members maybe bauzas can help10:39
* kashyap clicks10:39
kashyapThank you, yeah10:40
kashyapbauzas: Mind having a look?  It is largely a procedural ACK.  It was discussed here a week or so ago to be safe: https://review.opendev.org/#/c/668111/10:40
*** maciejjozefczyk has quit IRC10:40
*** maciejjozefczyk has joined #openstack-nova10:41
SonPhami have push a nova-specs rts file and tag nova core-reviewer but no want review it. what must i do10:46
gibiSonPham: be patient :)10:48
SonPhammy boss not patient with me ;(10:51
*** elod has quit IRC10:51
*** elod has joined #openstack-nova10:52
openstackgerritHuachang Wang proposed openstack/nova-specs master: Use PCPU and VCPU in one instance  https://review.opendev.org/66865610:52
*** ociuhandu has quit IRC10:53
*** mkrai_ has joined #openstack-nova10:55
gibiSonPham: then you have to explain to your boss how the openstack upstream community works.10:55
SonPham(y)  nice. good ideal10:56
gibiSonPham: you got questions from Eric in your spec https://review.opendev.org/#/c/686948/10:57
gibiSonPham: so the next step would be to answer them10:58
*** mkrai has quit IRC10:58
*** mkrai__ has joined #openstack-nova10:58
*** brinzhang has joined #openstack-nova10:59
SonPhamgibi yep . i uploaded new patch set11:00
*** mkrai_ has quit IRC11:00
gibiSonPham: cool Eric (efried) is on PTO (vacation) when he returns I suggest that you ping him telling him that you answered his questions in the spec reviw11:01
gibireview11:01
*** brinzhang_ has quit IRC11:03
SonPhamgibi how to ping him?11:04
*** mkrai__ has quit IRC11:04
cdentblargh: launchpad is timing out on write for me11:04
gibiSonPham: his nick is efried (now efried_pto as he is on vacation) if you mention his nick his IRC client will notify him11:05
openstackgerritMerged openstack/nova master: setup.cfg: Cleanup  https://review.opendev.org/67796911:11
*** udesale has quit IRC11:13
SonPhamgibi what is his nick's name in IRC chat?11:15
gibiSonPham: 'efried_pto' now, 'efried' when  he is not on vacation11:15
*** brinzhang has quit IRC11:15
*** brinzhang has joined #openstack-nova11:16
SonPhamok thanks11:16
openstackgerritBalazs Gibizer proposed openstack/nova master: Allow evacuating server with port resource request  https://review.opendev.org/68838711:25
*** ociuhandu has joined #openstack-nova11:28
*** SonPham has quit IRC11:31
*** pcaruana has quit IRC11:34
openstackgerritBalazs Gibizer proposed openstack/nova master: Allow evacuating server with port resource request  https://review.opendev.org/68838711:34
openstackgerritStephen Finucane proposed openstack/nova master: Stop testing Python 2  https://review.opendev.org/68795411:35
openstackgerritStephen Finucane proposed openstack/nova master: Remove unused 'nova-dsvm-base' job  https://review.opendev.org/68838911:35
sean-k-mooneystephenfin: i think your topic is incorrect on https://review.opendev.org/#/c/687954/511:37
stephenfinYes. Yes it is :D11:38
stephenfinfixed11:38
sean-k-mooneyyou need to fix the follow up patch too11:38
*** elod has quit IRC11:38
sean-k-mooneybut cool11:38
*** dave-mccowan has joined #openstack-nova11:39
*** elod has joined #openstack-nova11:39
bauzaskashyap: /me clicks on the stable change11:40
openstackgerritStephen Finucane proposed openstack/nova master: Remove unused 'nova-dsvm-base' job  https://review.opendev.org/68838911:41
bauzashah, queens backport11:41
*** pcaruana has joined #openstack-nova11:41
bauzaskashyap: FWIW, the queens branch will become on Extended Maintenance in around 3 weeks :)11:44
bauzashttps://releases.openstack.org/11:44
bauzasQueens Maintained 2018-02-28 Extended Maintenance estimated 2019-10-2511:44
openstackgerritArthur Dayne proposed openstack/nova master: libvirt:volume:Disallow AIO=native when no 'O_DIRECT' is available  https://review.opendev.org/68277211:47
*** tbachman has joined #openstack-nova11:54
kashyapbauzas: Yeah, I'm aware of that.  And thank you! :-)11:55
*** brinzhang_ has joined #openstack-nova12:01
*** brinzhang has quit IRC12:04
*** FlorianFa has joined #openstack-nova12:05
*** jroll has quit IRC12:14
*** jroll has joined #openstack-nova12:15
*** brinzhang has joined #openstack-nova12:20
*** brinzhang_ has quit IRC12:24
*** cdent has left #openstack-nova12:39
*** larainema has quit IRC12:44
*** bbowen has joined #openstack-nova12:48
alex_xustephenfin: gibi efried_pto, just try to kick off the work https://etherpad.openstack.org/p/nova-train-release-project-update12:49
gibialex_xu: good initiative! I will check the etherpad out12:50
alex_xugibi: thanks12:50
*** ociuhandu has quit IRC12:53
*** ttsiouts has joined #openstack-nova12:55
*** Guest31368 has joined #openstack-nova12:58
*** Guest31368 is now known as redrobot_13:00
*** redrobot_ is now known as redrobot13:01
*** liuyulong has joined #openstack-nova13:04
*** Guest69423 has quit IRC13:06
*** eharney has joined #openstack-nova13:10
*** CeeMac has quit IRC13:12
*** mriedem has joined #openstack-nova13:14
*** openstackstatus has quit IRC13:15
*** liuyulong has quit IRC13:16
*** slaweq_ has joined #openstack-nova13:20
*** slaweq has quit IRC13:23
*** nweinber has joined #openstack-nova13:24
*** BjoernT has joined #openstack-nova13:26
*** ociuhandu has joined #openstack-nova13:27
*** liuyulong has joined #openstack-nova13:28
*** tbachman has quit IRC13:30
*** bnemec has joined #openstack-nova13:31
*** ociuhandu has quit IRC13:32
openstackgerritSundar Nadathur proposed openstack/nova-specs master: Updated Nova-Cyborg interaction spec.  https://review.opendev.org/68415113:32
*** jawad_axd has quit IRC13:34
*** slaweq_ is now known as slaweq13:35
*** awalende has quit IRC13:36
*** awalende has joined #openstack-nova13:37
openstackgerritBalazs Gibizer proposed openstack/nova master: Allow evacuating server with port resource request  https://review.opendev.org/68838713:38
*** awalende has quit IRC13:41
*** lbragstad_ is now known as lbragstad13:44
*** brinzhang_ has joined #openstack-nova13:47
*** BjoernT_ has joined #openstack-nova13:47
*** BjoernT has quit IRC13:49
*** brinzhang has quit IRC13:50
*** gbarros has joined #openstack-nova13:57
*** brinzhang has joined #openstack-nova14:00
*** brinzhang has quit IRC14:01
*** brinzhang has joined #openstack-nova14:02
*** brinzhang_ has quit IRC14:03
*** beekneemech has joined #openstack-nova14:06
*** bnemec has quit IRC14:06
openstackgerritMerged openstack/nova stable/queens: libvirt: Rework 'EBUSY' (SIGKILL) error handling code path  https://review.opendev.org/66811114:06
*** munimeha1 has joined #openstack-nova14:06
*** dpawlik has quit IRC14:07
dansmithmriedem: https://review.opendev.org/#/c/633293 needs a rebase and compute rpc bump because of me... sorry :(14:09
mriedemjust replied on that one and noticed14:09
mriedemdon't worry, not my first rodeo chasing compute rpc version bumps on that series14:10
mriedemyou also have to rebase your api change and address some stuff in it :)14:10
mriedemi misunderstood your question about the long rpc call14:11
mriedemthought you were asking why it was a call and not a cast14:11
openstackgerritStephen Finucane proposed openstack/nova master: Remove compute compat checks for aborting queued live migrations  https://review.opendev.org/68840914:12
dansmithmriedem: no, I was asking why a long call and not a regular call14:12
mriedemyeah i realized14:13
*** ricolin has joined #openstack-nova14:13
mriedemi don't have a good answer except it's what i did in all the new compute methods in this series14:13
dansmithheh okay, like I said, I'm not sure there's a reason *not* to do it really, other than just signaling that some of them are expected to run *really* long14:13
dansmithwell, and I guess there's a teensy bit of messaging overhead14:14
bauzasmriedem: just saw this morning that Queens will be on Extended Maintenance in 3 weeks, so maybe we should look at https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:stable/queens ?14:14
dansmithmriedem: I guess I thought the point of the schema was so we didn't have to test for things that the schema would reject for us14:15
dansmithwell, one of the points of the schema14:15
mriedemwe still have to test that we defined the schema correctly14:15
dansmithisn't that like kinda like silly tests that are actually only testing that mock is working?14:15
dansmithokay14:15
mriedemand i'd say to prevent regressions in later microversions on the same api14:16
dansmithokay, that's fine it just seems inconsistent with what we do elsewhere, like we don't check that when you define an object field as int that a string will blow it up :)14:16
mriedemi've seen and i'm pretty used invalid schema before adding tests to validate it so it's a more complicated thing than type checking14:17
mriedem*pretty sure14:17
dansmithah dammit, I really wanted 2.8014:18
dansmithalso, I haven't had to write one of these in a long time, but it really sucks to have to do so much work if you miss your microversion14:18
mriedembauzas: join the party i guess, it's usually a party of 114:18
dansmithI wish we could do something better there14:19
mriedemi don't thinking the microversion stuff can be reasonably automated14:20
mriedem*think14:20
mriedemcoffee hasn't kicked into my brain yet14:20
*** priteau has joined #openstack-nova14:20
dansmithwell, it would take some changes in our workflow I think14:20
dansmithI know it's not trivial14:20
*** efried_pto is now known as efried14:29
efriedo/ nova14:29
efriedalex_xu, stephenfin, gibi: I copied the Denver proj update to a new google slide thingy. PM me whatever email address you have a google account for and I'll add you to editors.14:30
dansmithgmann: can you help me decipher mriedem's request here? or point me at what something new looks like that I can copy? https://review.opendev.org/#/c/687140/5/nova/tests/unit/test_policy.py@31214:35
*** tbachman has joined #openstack-nova14:37
gmanndansmith:  yeah, we can use the new naming standard for new policy. EX -https://github.com/openstack/nova/blob/master/nova/policies/server_topology.py#L1914:39
dansmithgmann: oh meaning just s/os-// ?14:39
*** dklyle has quit IRC14:40
mriedemdecipher :/14:40
mriedemi said "compute:aggregates:images" or "compute:aggregates:images:cache"14:40
*** dtantsur is now known as dtantsur|brb14:40
dansmithokay, I dunno what I was thinking last week, but reading it this morning he *did* tell me what to type14:40
*** dklyle has joined #openstack-nova14:40
gmann "compute:aggregates:images:cache" seems perfect14:40
dansmithmriedem: yeah I dunno, sorry I read that a couple times last week and it seemed like you were waffling on what you thought it should be and my eyes glazed over14:41
mriedemi was only waffling on making it specific to the cache action for future actions like purge14:41
dansmithI probably didn't see that the os- was removed, so it looked like what I had and I assumed I was missing something bigger14:42
bauzasmriedem: sure, will do14:42
fricklermriedem: did you see the thing I noticed last week? I'm assuming you might like that one, too ;) https://bugs.launchpad.net/tempest/+bug/184775914:42
openstackLaunchpad bug 1847759 in tempest "reboot_server_hard can trigger corrupted file system" [Undecided,New]14:42
dansmithhaving one operation in aggregates be different from the rest seems unhelpful to admins, but I'm more than happy to just do whatever as I don't know anything about this stuff14:42
mriedemdansmith: i believe gmann has an approved blueprint to re-write all of those policy rules so it's just less work when he does14:43
dansmithack14:43
mriedemfrickler: i didn't14:43
gmannnaming change for existing policy we dropped from policy improvement spec but for new one we can follow the new naming standard14:44
gmannat least not part of scope+new read-only defaults spec14:44
mriedemfrickler: i've definitely seen this in the gate jobs that have ssh failures "cp: write error: No space left on device"14:45
mriedemfrickler: http://status.openstack.org/elastic-recheck/#180801014:45
*** ociuhandu has joined #openstack-nova14:47
fricklermriedem: yes, the "no space left" is pretty generic, though, this one seems directly tied to hard reboots and the resulting corruption causing kernel panics14:47
gmannbut yes, i wanted to change all old name but that is something needs to be discussed if worth to do or not from operator perspective. this will be separate spec14:47
mriedemdansmith: so we haven't done this for train yet - https://review.opendev.org/#/c/642599/ - does that need to be in train?14:48
mriedemor is ok for just ussuri?14:48
mriedem*is it14:48
fricklermriedem: so I'd say it is not a duplicate, but a specific subset of it, tied to a single test scenario14:49
mriedemgmann: renaming all of the old policy rules seems like a ton of churn for not much gain14:49
mriedemfrickler: ack14:49
dansmithmriedem: it'd be nice for it to be in train eventually in case anyone sets their train level to train in an attempt to bump to current14:50
*** brault has quit IRC14:50
dansmithmriedem: really they should unpin or use auto, but you can imagine setting train to train meaning current making sense to people14:50
mriedemhttps://wiki.openstack.org/wiki/Nova/ReleaseChecklist says "Ideally add the RPC version alias just before opening the next release"14:50
mriedemi noticed that the rocky one was added in stein https://review.opendev.org/#/c/589972/14:50
dansmithmriedem: right, that wiki link means like "in rc2 when we know we're not going to be adding anything else to the rpc api"14:51
gmannmriedem: true. initially i added name change as part of other policy changes  but then we dropped that as it was changing every policies. i agree that doing it separately  is not so much gain.14:51
dansmithmriedem: I think after auto came on the scene we got a little lax about that14:51
dansmithmriedem: and certainly people using a numbered version that has nothing to do with "train", that word means not very much to them14:52
openstackgerritNicolas Belouin proposed openstack/nova master: Add support for Xen PVH VM Mode in libvirt driver  https://review.opendev.org/68782714:52
*** liuyulong has quit IRC14:54
mriedemok i'd push the patch but i'm rebasing this cross-cell series which will take awhile,14:54
mriedemif someone else wants to push it i'd +214:54
dansmithack I can do it14:56
dansmithwaiting for that conductor patch to land before rebasing mine14:56
openstackgerritDan Smith proposed openstack/nova master: Update compute rpc version alias for train  https://review.opendev.org/68841814:58
dansmitheff, conductor patch failed, might as well rebase now I guess14:58
openstackgerritBalazs Gibizer proposed openstack/nova master: Refactor rebuild_instance  https://review.opendev.org/68841915:00
dansmithhmm, I don't see that 2.80 patch in the queue15:00
*** beekneemech is now known as bnemec15:01
openstackgerritBalazs Gibizer proposed openstack/nova master: Allow evacuating server with port resource request  https://review.opendev.org/68838715:01
mriedemit's #3 in the gate but the #1 spot just failed15:02
mriedemwhich was yours15:02
efriedalex_xu, stephenfin, gibi: I updated the slides to have no Denver info in them (I think) and filled out a few easy ones including the microversion table.15:02
stephenfinmriedem: It's low priority (because 3rd party CI, at a minimum), but I'm going to need your help on the remove python2 patch at some point. I haven't a clue how to tell the two remaining legacy jobs to use python315:04
*** TxGirlGeek has joined #openstack-nova15:04
*** tbachman has quit IRC15:04
mriedemdevstack jobs?15:04
mriedemhttps://review.opendev.org/#/c/649097/15:05
mriedemdepends-on that?15:05
*** maciejjozefczyk has quit IRC15:05
*** ivve has quit IRC15:05
mriedemi'll be annoyed if we drop py2 and have to define USE_PYTHON3=True everywhere15:05
efriedmriedem: I haven't been following https://blueprints.launchpad.net/nova/+spec/add-user-id-field-to-the-migrations-table -- see patches +A this morning, is that the last of it? Can we mark complete (when merged)?15:09
gmannstephenfin: mriedem or you can use d-g var for legacy if not writing directly on localrc - DEVSTACK_GATE_USE_PYTHON3. But moving the devstack default to py3 and keep py2 setting in devstack on demand if anyone needed is something we should do first. I will review mriedem patch (not sure it does the same).15:11
*** udesale has joined #openstack-nova15:12
dansmithmriedem: yeah it wasn't showing up on dash for me at the time, but is now.. hadn't run anything so must've just been queued or something15:14
openstackgerritDan Smith proposed openstack/nova master: Add cache_images() to conductor  https://review.opendev.org/68713915:14
openstackgerritDan Smith proposed openstack/nova master: Add image caching API for aggregates  https://review.opendev.org/68714015:14
dansmithmriedem: conductor patch was just rebased to kick it out of the gate right away and get in line behind the 2.80 one ^15:15
mriedemefried: no, there is still a novaclient change15:15
gmannstephenfin: mriedem FYI. fully dropping py2 from devstack or other common tooling/lib/oslo etc will be the last once all projects drop py2. We discussed the plan in TC for that and i will send the same on ML also, further and final plan/discussion will be on 24th TC office hour.15:15
efriedmriedem: ack thx15:15
mriedemgmann: can't change the default on DEVSTACK_GATE_USE_PYTHON3 in d-g since it's branchless, unless we add branch logic in d-g for that like we did for using neutron by default back in i think newton or ocata15:16
mriedemthis https://github.com/openstack/devstack-gate/blob/master/devstack-vm-gate-wrap.sh#L19915:16
*** ttx has quit IRC15:18
gmannmriedem: yeah, legacy jobs and d-g need branch handling.15:18
openstackgerritStephen Finucane proposed openstack/nova master: Remove unused 'nova-dsvm-base' job  https://review.opendev.org/68838915:18
openstackgerritStephen Finucane proposed openstack/nova master: zuul: Make functional job inherit from openstack parents  https://review.opendev.org/68842515:18
mriedemone question is going to be grenade, since if grenade train side starts with py2 and upgrades to ussuri and starts using py3 packages, but https://review.opendev.org/#/c/649097/ should have hit that15:19
dansmithmriedem: is that what we'll do? or just go py3-py3 on both sides?15:19
dansmithI feel like staying on py3 on both sides is more in line with what we've done before with ubuntu upgrades, but not very realistic of couse15:19
mriedemsince https://review.opendev.org/#/c/649097/ isn't doing anything else i expect grenade starts with py2 and the new side is py315:20
mriedemyeah it's definitely not realisitc,15:20
mriedem*realistic15:20
mriedemwe - or someone that cares enough, like a distro - should make sure that people can upgrade from py2 in train to py3 in ussuri15:20
mriedemthis came up a long time ago of course when the tc was talking about py2 deprecation and the only thing i remember really coming up was maybe some oslo libraries, like messaging, maybe having some compat issues15:21
dansmithmriedem: why is that patch causing a switch?15:21
gmannthat should be tested as past of py2.7 jobs + py3 testing on train15:21
mriedemdansmith: because the old side of grenade in that run would run with py2 by default - since it's running from stable/n-115:21
dansmithmriedem: oh you mean the grenade testing of that patch in isolation, not once that patch is merged?15:21
gmannmean both version testing make sure train code can run on both py version so does upgrade is no issue15:21
mriedemdansmith: both15:22
mriedemgmann: not really15:22
mriedemsay you have multi-node grenade where one compute is left at n-1 (train),15:22
gmannmriedem: but we will keep only grenade-py3 in ussuri which will be py3->py315:22
mriedemi would expect that job to start with py2, upgrade controller services and one compute to py3 ussuri and leave the n-1 compute at py215:23
mriedemand everything, like rpc, should continue to work15:23
dansmithmriedem: oh, devstack has branched, d-g does not I see, yeah15:23
mriedemyup15:23
mriedemgmann: if we don't have something like what i described above with multinode grenade we're not really testing upgrades from py2 to py315:23
mriedeme.g. what happens if a py3 conductor sends serialized json rpc to a py2 compute?15:24
gmannhow that will be different than testing compute node for both version in separate jobs15:24
gmannohk15:24
dansmithmriedem: one *could* argue that we don't care to test that sort of thing15:25
mriedemsure15:25
mriedemthis isn't going to be my fight in ussuri15:25
dansmithmriedem: as you say, leave it to thedistros, but not specifically test or care about a platform change related problem15:25
mriedemif it blows up i'll laugh15:25
gmannthis came up in denver PTG (previous one) also15:25
mriedemgmann: lots of things come up15:26
mriedemidk if anyone (the tc?) documented test plans for this15:26
mriedemor statements about testing this kind of upgrade15:26
*** xek_ has quit IRC15:26
mriedemif there was previous agreement from the tc and/or qa or whatever that we as a community don't care about testing that scenario, then fine15:26
gmannhttp://lists.openstack.org/pipermail/openstack-dev/2018-September/134592.html15:27
mriedemyeah that's the thread i was thinking about when i said this came up earlier15:27
*** ttx has joined #openstack-nova15:29
openstackgerritStephen Finucane proposed openstack/nova master: Remove unused 'nova-dsvm-base' job  https://review.opendev.org/68838915:30
openstackgerritStephen Finucane proposed openstack/nova master: Stop testing Python 2  https://review.opendev.org/68795415:30
openstackgerritStephen Finucane proposed openstack/nova master: zuul: Make functional job inherit from openstack parents  https://review.opendev.org/68842515:30
gmannmriedem: i think there was agreement 'community don't care ' but documented anywhere15:30
mriedemthat's what doug seems to be saying http://lists.openstack.org/pipermail/openstack-dev/2018-September/134655.html15:31
mriedemso *shrug*15:31
gmann* not documented anywhere15:31
*** elod has quit IRC15:31
gmannI think we can add that in TC doc/resolution. i will add that part as part of 24th TC office hour discussion15:33
*** awalende has joined #openstack-nova15:37
*** elod has joined #openstack-nova15:42
*** damien_r has quit IRC15:42
*** awalende has quit IRC15:42
*** ttsiouts has quit IRC15:46
*** ttsiouts has joined #openstack-nova15:46
*** ccamacho has quit IRC15:47
openstackgerritChris Dent proposed openstack/nova master: single pass instance info fetch in host manager  https://review.opendev.org/62355815:48
*** ttsiouts has quit IRC15:51
melwittmriedem: I thought you might be interested in this, I wrote a regression test for the "run online_data_migrations twice fails" in the post test hook and a different fix on top https://review.opendev.org/68820515:58
mriedemthat would get around the mysql in functional test thing yeah15:58
mriedemget your power back?15:59
melwittyeah, restored friday evening16:03
*** tbachman has joined #openstack-nova16:03
*** openstackstatus has joined #openstack-nova16:03
*** ChanServ sets mode: +v openstackstatus16:03
melwittnice to have internet again16:03
mriedemheh, i bet16:03
melwittand yeah, I couldn't find another way to exercise the bug and explained briefly why in the commit message. lmk if you think the explanation could use more info, or if there are alternatives that I missed16:06
*** mlavalle has joined #openstack-nova16:06
*** rpittau is now known as rpittau|afk16:06
*** ociuhandu_ has joined #openstack-nova16:07
openstackgerritMerged openstack/nova master: Avoid using image with kernel in BDM large request func test  https://review.opendev.org/68813216:07
*** dtantsur|brb is now known as dtantsur16:08
efriedstephenfin: Does https://review.opendev.org/#/c/688249/ look right to you?16:08
stephenfinsure does16:10
*** ociuhandu has quit IRC16:10
efriedstephenfin: cool, I've got a few similar ones...16:11
efriedthanks16:11
melwittzzzeek: I would appreciate your review on this fix I proposed related to the questions I asked you on friday https://review.opendev.org/68820616:13
zzzeekmelwitt: yah, saw the link, will look now16:13
melwitt(and thanks again for helping me out with that)16:13
*** mkrai_ has joined #openstack-nova16:13
*** ociuhandu_ has quit IRC16:14
efriedhttps://review.opendev.org/#/c/688250/ https://review.opendev.org/#/c/688235/ https://review.opendev.org/#/c/688087/ if you care stephenfin16:15
*** cdent has joined #openstack-nova16:15
*** mkrai_ has quit IRC16:15
cdentmriedem: my to do list has just reminded me to prompt folk about https://review.opendev.org/#/c/623558/ which is jay's patch to improve instance info fetch in the host manager. A while back I added some unit tests and did the tweaks that eric requested16:17
openstackgerritMerged openstack/os-resource-classes master: Bump the openstackdocstheme extension to 1.20  https://review.opendev.org/68824916:18
*** igordc has joined #openstack-nova16:21
*** dave-mccowan has quit IRC16:23
zzzeekmelwitt: before I look silly on the review because I'm missing something, do i read correctly that security_group_ensure_default is called twice inside of instance_create() ?16:24
melwittzzzeek: that is correct. the first time, it intends to create it if it doesn't exist, the second time, it's really only doing it to read it back (was my conclusion)16:25
zzzeekmelwitt: what condition would casue the return value to have changed in the intervening 15 or so lines of code?16:25
melwittare you thinking another way would be to just use the returned object from the first time, later on?16:25
openstackgerritEric Fried proposed openstack/nova-specs master: Add 'Feature Liaison' spec process  https://review.opendev.org/68585716:25
openstackgerritEric Fried proposed openstack/nova-specs master: Add spec for vm scoped sriov numa affinity  https://review.opendev.org/68317416:25
efriedbauzas: ^16:26
zzzeekmelwitt: beacsue i dont know the concepts being worked with here, from what im seeing i dont immediately see why a. the top call should use the context, not independent transaction and b. what's wrong with default_gruop assigned at the top there16:26
*** mkrai_ has joined #openstack-nova16:26
openstackgerritGhanshyam Mann proposed openstack/nova-specs master: Re-propose policy-defaults-refresh spec for Ussuri  https://review.opendev.org/68605816:26
zzzeekmelwitt: then there's this _Get_Sec_group_models closure, which appears to be called only once, three lines later, no recursion, no passing it onwards, so why is it like that16:27
gmannmelwitt: ^^^ re-proposed policy spec for ussuri16:27
mriedemcdent: without digging into the code change yet, i dropped a few comments in the commit message16:27
mriedemsoft -1, i know it's an adopted patch16:27
*** mkrai_ has quit IRC16:27
zzzeekmelwitt: it makes it look like this used to be very different and was refactored, but not all the way16:27
cdentmriedem: thanks16:27
zzzeekmelwitt: if you agree thatim reading it right i can post my comments to this effect and folks can discuss16:27
*** eharney has quit IRC16:28
melwittzzzeek: indeed, I could only assume similar because I'm not _that_ familiar with the method and definitely not familiar with the history. I'm not quite parsing your question a) though, why the top call should not use a separate transcation? did you mean why does it?16:28
mriedemnote that we don't even need that call if using neutron https://review.opendev.org/#/c/653065/16:29
melwittzzzeek: and I agree that it looks partially refactored16:29
zzzeekmelwitt: yes, got my negatives reversed.16:29
zzzeekmelwitt: yes why use a separte trans there16:29
zzzeekmelwitt: when the separate trans is used, that means that you either want the thing you're doing to modify the DB even if your method fails, or you want the thing you're doing to modify the DB while your mehtod is still running (like logging a slow operation), or, otehrwise you're trying to read some state that occurs within the middle of your ongoing, slow transaction16:30
zzzeekmelwitt: this instance_create doesn't look like a slow function and it looks like the originator of the transaction that it commits.  but i dont know16:31
*** brault has joined #openstack-nova16:31
melwittzzzeek: ok, yeah, I went through thinking about it and I think what that comment is saying is that if we don't fork out a separate transaction, two racing server creates would try to insert the default group (the first time) and then one will succeed and the second would fail the COMMIT and then raise a DBError and fail the second server create [to the user]. using a separate transaction avoids that. now, I think an alternative way to16:31
melwitt have handled that would have been to try-except the DBError and call instance_create again in that case, but that's not what was done16:31
zzzeekmelwitt: only if this function is overall slow-ish is there a race created that wouldn't also exist in the security_group_ensure_default() call itself16:32
melwitt*fail the COMMIT because of the unique constraint on project_id16:33
zzzeeklike, those can race anyway right?16:33
zzzeekoh it has the retry on it16:33
zzzeekI see so its independent so that it can run and do a retry16:33
zzzeekOK but...it returns the value at least, so, you can use it at the top?  the return value that is, and not call it a second time16:34
*** udesale has quit IRC16:34
*** BjoernT_ is now known as BjoernT16:34
melwittyeah... thinking... on the surface I don't see why we couldn't. I was trying to think if there would be any implications for racing requests to instance_create, but there shouldn't be right..16:36
zzzeekmelwitt: I did a blame.  this code is oldish16:37
zzzeekPavel Kholkin in 2015 seems to be most of what I'm asking about, is that person still around16:37
zzzeeke.g. i can put up my qeustions and bring them onto a review if they still work for openstack16:38
melwittmriedem: yeah, I remember that patch. I'm not opposed to working around it but I landed on this because I was genuinely puzzled about how dupe groups were being created when the logic is clearly "get or create". once I found the root cause of how two identical groups were being created, I went ahead and proposed a way to address that directly16:38
zzzeekhaha "work for openstack"16:38
zzzeekmelwitt: is a "security group" kind of an infrequently created concept and the race here is the kind of thing that happens only during some kind of installation?  a "Security group" sounds like the kind of thing an administrator creates just once16:39
melwittzzzeek: yeah, the code is old and as mriedem mentioned, it doesn't come into play (default group not ever used) if running with neutron, and neutron is the only networking we support as of... a few releases ago. this code is dealing with legacy nova-network security group record16:40
melwittand I don't think Pavel Kholkin is still around working on openstack16:40
zzzeekmelwitt: OK ill comment to these effects16:41
*** xek_ has joined #openstack-nova16:42
*** xek_ has quit IRC16:42
melwittzzzeek: yeah, this code only creates a group once per project and the only time it becomes a problem is when it's done with an anonymous RequestContext which has project_id=NULL, which is happening because nova-manage is inserting a "dummy" instance record to support a marker/paging (don't ask, haha) and the unique constraint on project_id won't "work" for NULL values16:42
*** xek has joined #openstack-nova16:44
openstackgerritEric Fried proposed openstack/nova master: Stop using NoAuthMiddleware in tests  https://review.opendev.org/68741616:45
openstackgerritEric Fried proposed openstack/nova master: Repro bug 1845530: versioned discovery is authed  https://review.opendev.org/68518016:45
openstackbug 1845530 in OpenStack Compute (nova) "Versioned discovery endpoint should not require authentication" [Undecided,In progress] https://launchpad.net/bugs/1845530 - Assigned to Eric Fried (efried)16:45
openstackgerritEric Fried proposed openstack/nova master: Allow versioned discovery unauthenticated  https://review.opendev.org/68518116:45
efriedmriedem: ^16:45
*** mkrai_ has joined #openstack-nova16:46
openstackgerritDan Smith proposed openstack/nova master: Add image caching API for aggregates  https://review.opendev.org/68714016:48
*** xek has quit IRC16:49
*** mkrai_ has quit IRC16:49
*** brinzhang has quit IRC16:50
*** brinzhang has joined #openstack-nova16:50
*** cdent has quit IRC16:51
openstackgerritEric Fried proposed openstack/nova-specs master: Add 'Feature Liaison' spec process  https://review.opendev.org/68585716:52
openstackgerritEric Fried proposed openstack/nova-specs master: Add spec for vm scoped sriov numa affinity  https://review.opendev.org/68317416:52
efriedbauzas, gmann: Had to respin to fix the title on gibi's s/Core/Feature/ Liaison section ^16:52
efried(gibi thanks for adding that section preemptively :)16:53
gmannefried: +116:55
efriedthanks gmann16:55
*** derekh has quit IRC17:00
*** brinzhang_ has joined #openstack-nova17:00
*** munimeha1 has quit IRC17:02
*** brinzhang has quit IRC17:04
*** brinzhang has joined #openstack-nova17:05
*** brinzhang_ has quit IRC17:07
*** priteau has quit IRC17:11
*** BjoernT has quit IRC17:20
*** gmann is now known as gmann_afk17:25
*** tesseract has quit IRC17:25
openstackgerritMatt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method  https://review.opendev.org/63329317:26
openstackgerritMatt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask  https://review.opendev.org/62789017:26
openstackgerritMatt Riedemann proposed openstack/nova master: FUP for I66d8f06f19c5c631e33208580428aa843abb38d2  https://review.opendev.org/67895117:26
openstackgerritMatt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method  https://review.opendev.org/63483217:26
openstackgerritMatt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask  https://review.opendev.org/62789117:26
openstackgerritMatt Riedemann proposed openstack/nova master: Add finish_snapshot_based_resize_at_dest compute method  https://review.opendev.org/63508017:26
openstackgerritMatt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask  https://review.opendev.org/63564617:26
openstackgerritMatt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask  https://review.opendev.org/63566817:26
openstackgerritMatt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize()  https://review.opendev.org/63568417:26
openstackgerritMatt Riedemann proposed openstack/nova master: Filter duplicates from compute API get_migrations_sorted()  https://review.opendev.org/63622417:26
openstackgerritMatt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize  https://review.opendev.org/63625317:26
openstackgerritMatt Riedemann proposed openstack/nova master: Handle target host cross-cell cold migration in conductor  https://review.opendev.org/64259117:26
openstackgerritMatt Riedemann proposed openstack/nova master: Validate image/create during cross-cell resize functional testing  https://review.opendev.org/64259217:26
openstackgerritMatt Riedemann proposed openstack/nova master: Add zones wrinkle to TestMultiCellMigrate  https://review.opendev.org/64345017:26
openstackgerritMatt Riedemann proposed openstack/nova master: Add negative test for cross-cell finish_resize failing  https://review.opendev.org/64345117:26
openstackgerritMatt Riedemann proposed openstack/nova master: Refresh instance in MigrationTask.execute Exception handler  https://review.opendev.org/66901217:26
openstackgerritMatt Riedemann proposed openstack/nova master: Add negative test for prep_snapshot_based_resize_at_source failing  https://review.opendev.org/66901317:26
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source compute method  https://review.opendev.org/63705817:26
*** lpetrut has quit IRC17:28
*** ircuser-1 has joined #openstack-nova17:29
*** SonPham has joined #openstack-nova17:32
*** ricolin has quit IRC17:33
*** dtantsur is now known as dtantsur|afk17:36
*** maciejjozefczyk has joined #openstack-nova17:40
*** maciejjozefczyk has quit IRC17:49
openstackgerritSon Pham proposed openstack/nova-specs master: [WIP]Add blueprint snapshot-vms-with-state of-processes  https://review.opendev.org/68852217:51
*** ivve has joined #openstack-nova17:53
openstackgerritSon Pham proposed openstack/nova-specs master: [WIP]Add blueprint snapshot-vms-with-state-of-processes  https://review.opendev.org/68694818:00
*** brinzhang_ has joined #openstack-nova18:00
mriedembauzas: here are a set of queens changes that are good to go: https://review.opendev.org/#/c/676285/ https://review.opendev.org/#/c/675355/ https://review.opendev.org/#/c/682722/ https://review.opendev.org/680873 https://review.opendev.org/#/c/676500/18:01
*** brinzhang has quit IRC18:04
*** psachin has quit IRC18:05
*** SonPham has quit IRC18:10
dansmithmriedem: I guess you didn't squash that fixup patch into your second patch when you rebased just now eh/18:11
*** gbarros has quit IRC18:11
*** gbarros has joined #openstack-nova18:12
dansmithdoesn't look like anything is running yet, so you could do that without losing much still I think18:12
*** ralonsoh has quit IRC18:12
mriedemhttps://review.opendev.org/#/c/678951/ you mean?18:12
mriedemat some point the thing it was FUP'ing ( https://review.opendev.org/#/c/627890/ ) had +2s on it a couple of times so i figured it was easier to not squash for those +2s to come back as a diff check18:13
mriedemi can if you want, i'm sure at this point a diff isn't trivial for gibi or stephenfin anymore anyway18:14
dansmithmriedem: yeah, but rebased on master now so there's a ton of churn in the patches from there now18:14
dansmithyeah18:14
mriedemnone of this code is impacted much from master...18:14
mriedemwell, this conductor task code18:14
dansmithI think it makes sense to just do it now18:14
dansmithoh, fair I guess. I just eye-diffed the bottom one and there was a lot of churn, but true about this one18:15
mriedemhttps://review.opendev.org/#/c/627890/53..59 it's not really trivial anyway18:15
dansmithanyway, whatever, the reason for doing it was a +2 on there in August, so seems safe to squash now but whatever18:15
mriedemyeah i will18:15
mriedemi'll take most any excuse to take a break from reviewing these queens backports...18:16
openstackgerritMatt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask  https://review.opendev.org/62789018:22
openstackgerritMatt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method  https://review.opendev.org/63483218:22
openstackgerritMatt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask  https://review.opendev.org/62789118:22
openstackgerritMatt Riedemann proposed openstack/nova master: Add finish_snapshot_based_resize_at_dest compute method  https://review.opendev.org/63508018:22
openstackgerritMatt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask  https://review.opendev.org/63564618:22
openstackgerritMatt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask  https://review.opendev.org/63566818:22
openstackgerritMatt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize()  https://review.opendev.org/63568418:22
openstackgerritMatt Riedemann proposed openstack/nova master: Filter duplicates from compute API get_migrations_sorted()  https://review.opendev.org/63622418:22
openstackgerritMatt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize  https://review.opendev.org/63625318:22
openstackgerritMatt Riedemann proposed openstack/nova master: Handle target host cross-cell cold migration in conductor  https://review.opendev.org/64259118:22
openstackgerritMatt Riedemann proposed openstack/nova master: Validate image/create during cross-cell resize functional testing  https://review.opendev.org/64259218:22
openstackgerritMatt Riedemann proposed openstack/nova master: Add zones wrinkle to TestMultiCellMigrate  https://review.opendev.org/64345018:22
openstackgerritMatt Riedemann proposed openstack/nova master: Add negative test for cross-cell finish_resize failing  https://review.opendev.org/64345118:22
openstackgerritMatt Riedemann proposed openstack/nova master: Refresh instance in MigrationTask.execute Exception handler  https://review.opendev.org/66901218:22
openstackgerritMatt Riedemann proposed openstack/nova master: Add negative test for prep_snapshot_based_resize_at_source failing  https://review.opendev.org/66901318:22
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source compute method  https://review.opendev.org/63705818:22
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask  https://review.opendev.org/63707018:22
openstackgerritMatt Riedemann proposed openstack/nova master: Add confirm_snapshot_based_resize conductor RPC method  https://review.opendev.org/63707518:22
dansmithmriedem: so that filter patch that added 2.80,18:33
dansmithcan I rebased that on master real quick since it needs a recheck anyway?18:33
dansmithotherwise I can't run tests easily because it doesn't have the rpc changes18:33
dansmith(which are in master already)18:33
dansmithrebasing it will only take ten minutes away from its queue time and it's behind your massive set anyway18:34
mriedemsure18:34
mriedemi'll re-approve18:34
openstackgerritDan Smith proposed openstack/nova master: Filter migrations by user_id/project_id  https://review.opendev.org/67424318:35
openstackgerritDan Smith proposed openstack/nova master: Add cache_images() to conductor  https://review.opendev.org/68713918:35
openstackgerritDan Smith proposed openstack/nova master: Add image caching API for aggregates  https://review.opendev.org/68714018:35
dansmithdone18:35
openstackgerritMatt Riedemann proposed openstack/nova master: Fix legacy issues in filter migrations by user_id/project_id  https://review.opendev.org/68219818:49
*** factor has quit IRC18:50
*** gbarros has quit IRC18:53
openstackgerritDan Smith proposed openstack/nova master: Add image caching API for aggregates  https://review.opendev.org/68714018:54
*** brinzhang has joined #openstack-nova19:00
*** brinzhang_ has quit IRC19:03
openstackgerritmelanie witt proposed openstack/nova master: Remove redundant call to get/create default security group  https://review.opendev.org/68820619:07
*** bnemec has quit IRC19:22
*** Jeffrey4l has quit IRC19:23
*** brault has quit IRC19:24
*** Jeffrey4l has joined #openstack-nova19:24
dansmithmriedem: question for you here: https://review.opendev.org/#/c/627890/60/nova/conductor/tasks/cross_cell_migrate.py@49519:35
dansmithI didn't -1 in case you had a good reason or excuse, but figured it was worth pointing out if you hadn't thought about it19:36
*** gbarros has joined #openstack-nova19:39
*** awalende has joined #openstack-nova19:42
*** awalende has quit IRC19:42
*** awalende has joined #openstack-nova19:42
*** gbarros has quit IRC19:42
*** awalende has quit IRC19:43
*** awalende has joined #openstack-nova19:43
mriedemreplied19:49
mriedemi've got some negative tests for things failing higher in the stack but not the scenario you're asking about19:49
mriedemit wouldn't be hard to add though19:49
sean-k-mooneymriedem: while that would be good i think its also i higher bar of testing then we have of similarly hairy failure modes.19:53
sean-k-mooneyi have wondered in the past if we should have a delete --force and change delete to raise a error with --force when the vm is in task state that are potentially hard to clean up19:54
sean-k-mooneythe issue with that is force deleteion is somehting i proably woudl want only my admin to do sicne they are the ones that woudl have to clean it up but as a use if i dont want a thing any more i should be able to say that19:55
sean-k-mooneyespcially since im being billed for it in most cases19:55
*** bnemec has joined #openstack-nova19:56
*** dpawlik has joined #openstack-nova19:56
sean-k-mooneywhich reminds me we shoud get around to merging this so we dont leak vifs when we delete the vm while its spwaning https://review.opendev.org/#/c/663382/20:00
*** nweinber has quit IRC20:01
*** gmann_afk is now known as gmann20:01
*** gbarros has joined #openstack-nova20:05
sean-k-mooneydansmith: melwitt could one of ye review ^ efried since you looked at that ages ago if you get a chance that would be cool too20:06
efriedsean-k-mooney: are you +1 at this point?20:07
sean-k-mooneyim rereviewing it now to determind what matt changed20:08
sean-k-mooneybut it think it ws just the unit tests20:08
mriedemtrying to catch up, was on a call20:08
sean-k-mooneybut it was rebased so doing eye ball diff manually20:08
mriedemso (1) sean-k-mooney you're saying fault injection testing per dansmith's request is a higher bar of testing than we usually have, right? if so i agree but i also agree we should do that kind of testing while it's fresh in our minds so i don't have a problem adding it20:09
mriedemand (2) https://review.opendev.org/#/c/663382/ the last time i looked at that there were questions about how you can even get into that situation - did anyone (stephen?) sort that out?20:10
sean-k-mooneymriedem: yes im not saying we should not do it just we normally have not been that proactive20:10
sean-k-mooneyi just notice that mdbooth updated it and so maybe that is a ping to early20:11
dansmithmriedem: sorry didn't see you replied.. fault injection would be a tempest-level test, I'm more looking for a test where you make the save() delete the instance and then fail like save would have to make sure you call all the cleanups you expect20:11
mriedemdansmith: maybe fault injection is a bad word but that's essentially what i proposed in my reply, stub resize_claim on the dest to (through the api) delete the instance from the source, which will trigger a failure in conductor and rollback the instance from the target cell db and the test would assert the various cleanups20:12
sean-k-mooneymriedem: it looks like nothing substantive has change so i need to parse all the comments again to confim if there was still any outstanding questions20:12
dansmithmriedem: ack20:12
mriedemsean-k-mooney: the reply from stephen is the last time i looked at it https://review.opendev.org/#/c/663382/4/nova/compute/manager.py@250620:14
mriedem"open to ideas here"20:14
mriedemif there is a way to recreate that issue, then i'd say write a functional test to recreate it and then lay the fix on top to show it's fixed20:14
mriedemwhich goes back to our historically bad level of testing which is why we have requests for fault injection and more complicated test scenarios in my cross-cell resize series above20:15
mriedem*bad level of testing for non-happy path scenarios20:15
*** slaweq_ has joined #openstack-nova20:15
sean-k-mooneymriedem: right20:15
sean-k-mooneywhich is exeactly why this bug laid dorment for so long20:16
sean-k-mooneyi know how the issue happend in the customer env20:16
sean-k-mooneyit would be hard to recreate but maybe not imposible20:16
sean-k-mooneyill try an think about how to recreate in a functionla test although mdbooth might also be looking into that20:18
*** slaweq has quit IRC20:18
*** slaweq__ has joined #openstack-nova20:19
mriedemnote that it's not against the rules to have a functional test to recreate a bug which stubs something to inject the failure mode,20:20
mriedemwe do that and still do it with functional tests to make sure the api / service / db interaction is what we'd expect20:20
sean-k-mooneyya i was wondign if i could use a spy or mock to inject the fault at the correct time20:21
mriedeme.g. the test would do something like start creating an instance, stub something to then call the api to delete the instance, and then go back to spawning which would fail and make sure we deallocaten networks properly20:21
*** slaweq_ has quit IRC20:21
*** BjoernT has joined #openstack-nova20:24
*** pcaruana has quit IRC20:26
mriedemsean-k-mooney: i left a comment to that effect on the review20:27
*** benj has joined #openstack-nova20:28
*** benj is now known as Guest3991020:28
sean-k-mooneycool20:28
mriedemdansmith: so on https://review.opendev.org/#/c/627890/60/nova/conductor/tasks/cross_cell_migrate.py@495 - the functional tests start later in the series and in at least one or two cases gibi has asked a "what happens if x fails" question like you are here and i've added a test later in the series before he proceeds,20:29
mriedemare you ok with that here as well?20:29
*** dpawlik has quit IRC20:31
*** BjoernT_ has joined #openstack-nova20:32
*** BjoernT has quit IRC20:32
dansmithmriedem: with adding it later you mean?20:33
mriedemdansmith: no i mean i'd add it to the series and you'd hold off on +2ing that change until it's up and shows things are cool20:36
dansmiththat's what I mean20:36
dansmithsure20:36
mriedemalright20:36
mriedemthat'll be my tomrrow20:36
mriedem*tomorrow20:36
* sean-k-mooney realises i forgot to press y when creating the github links to the site where the race happens and now they are all pointing at different code :(20:41
*** dpawlik has joined #openstack-nova20:42
sean-k-mooneyi think its https://github.com/openstack/nova/blob/stable/queens/nova/compute/manager.py#L2164-L216720:44
sean-k-mooneyif we get the delete after we call _update_instance_after_spawn but before we call  instance.save(expected_task_state=task_states.SPAWNING)20:46
sean-k-mooneywe will raise the unexpected task state error.20:46
*** dpawlik has quit IRC20:46
sean-k-mooneyi could simulated tat by mocking _update_instance_after_spawn to raise it but it would be a bit of a hack20:46
sean-k-mooneyor maybe better20:47
sean-k-mooneyhave _update_instance_after_spawn inject the delete20:47
sean-k-mooneyeffefctly the race only happen in the interaval betwen reciving the libvirt event to notify that the spawn has completed and updating the db with that state20:49
*** slaweq has joined #openstack-nova20:50
*** slaweq__ has quit IRC20:51
openstackgerritMerged openstack/nova master: Deprecate [api]auth_strategy and noauth2  https://review.opendev.org/68737320:56
openstackgerritMerged openstack/nova master: Leave brackets on Ceph IP addresses for libguestfs  https://review.opendev.org/64940520:56
openstackgerritMerged openstack/nova stable/queens: Fix wrong assertions in unit tests  https://review.opendev.org/67739020:57
*** rcernin has joined #openstack-nova20:57
*** brinzhang_ has joined #openstack-nova21:00
*** brinzhang has quit IRC21:04
sean-k-mooneyno that is not the point were it happens... we handel the exception in that case.21:05
sean-k-mooneyactully no it is the right spot we reraise teh expection to do the cleanup else where and that is where we dont catch it21:08
mriedemso uh21:26
mriedemlet's just have a functional test that recreates the bug how about21:26
mriedemsince this spaghetti code is hard to reason about otherwise (as you've just demonstrated)21:26
sean-k-mooneyyes21:28
sean-k-mooneyi commented on the bug how to do it i think21:28
sean-k-mooneymriedem: mdbooth siad he was going to add a funcitonl test to one of stephen pathces this morining21:28
sean-k-mooneyif it was not this one ill look into adding it later in the week21:29
*** awalende has quit IRC21:30
*** awalende_ has joined #openstack-nova21:30
*** awalende_ has quit IRC21:35
*** awalende has joined #openstack-nova21:36
*** awalende has quit IRC21:40
*** brault has joined #openstack-nova21:46
*** brinzhang has joined #openstack-nova21:48
*** brinzhang_ has quit IRC21:52
*** jistr has quit IRC21:52
*** vesper11- has joined #openstack-nova21:53
*** vesper11 has quit IRC21:53
*** bnemec has quit IRC21:55
*** rcernin has quit IRC21:59
*** jistr has joined #openstack-nova22:01
*** jistr has quit IRC22:02
*** jistr has joined #openstack-nova22:03
*** trident has quit IRC22:04
*** trident has joined #openstack-nova22:07
*** slaweq has quit IRC22:14
*** mlavalle has quit IRC22:16
*** TxGirlGeek has quit IRC22:31
*** TxGirlGeek has joined #openstack-nova22:33
openstackgerritDustin Cowles proposed openstack/nova master: Provider Config File: YAML file loading and schema validation  https://review.opendev.org/67334122:48
openstackgerritDustin Cowles proposed openstack/nova master: Provider Config File: Function to further validate and retrieve configs  https://review.opendev.org/67602922:48
openstackgerritDustin Cowles proposed openstack/nova master: Provider Config File: Merge provider configs to provider tree  https://review.opendev.org/67652222:49
*** ivve has quit IRC22:49
*** slaweq has joined #openstack-nova22:52
*** slaweq has quit IRC22:56
*** rcernin has joined #openstack-nova23:07
*** artom has joined #openstack-nova23:11
*** tkajinam has joined #openstack-nova23:17
*** tkajinam_ has joined #openstack-nova23:17
*** tkajinam has quit IRC23:21
*** TxGirlGeek has quit IRC23:33
*** brault has quit IRC23:36
*** brault has joined #openstack-nova23:38
openstackgerritMerged openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method  https://review.opendev.org/63329323:40
*** mriedem has quit IRC23:48
openstackgerritsean mooney proposed openstack/nova master: block rebuild when numa toplogy changed  https://review.opendev.org/68795723:54

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!