Wednesday, 2019-04-17

openstackgerritMerged openstack/nova stable/rocky: Override the 'get' method in DriverBlockDevice class  https://review.openstack.org/64765100:01
*** tetsuro has joined #openstack-nova00:02
alex_xufrancoisp_: already add it my review list, will try to reach it00:06
*** gyee has quit IRC00:08
*** sapd1_x has joined #openstack-nova00:11
mriedemcfriesen: jackding: wee stein regression https://bugs.launchpad.net/nova/+bug/182502000:25
openstackLaunchpad bug 1825020 in OpenStack Compute (nova) "resize of instance, that was booted from volume, trigger image size check then fails resize" [Undecided,New]00:26
*** jding1_ has joined #openstack-nova00:29
*** sapd1_x has quit IRC00:30
*** jackding has quit IRC00:32
*** mriedem has quit IRC00:38
*** nicolasbock has quit IRC00:39
*** jding1__ has joined #openstack-nova00:45
*** jding1_ has quit IRC00:47
*** bryan_stephenson has quit IRC00:54
*** ricolin has joined #openstack-nova00:56
*** hamzy has joined #openstack-nova01:22
*** threestrands has joined #openstack-nova01:24
*** yedongcan has joined #openstack-nova01:24
melwittaspiers: something odd for your morrow http://eavesdrop.openstack.org/irclogs/%23openstack-placement/%23openstack-placement.2019-04-17.log.html#t2019-04-17T00:13:0601:24
melwitttl;dr placement rejecting a PUT of the COMPUTE_NET_ATTACH_INTERFACE trait from nova compute. I don't understand how that could be happening01:26
*** tetsuro has quit IRC01:27
*** whoami-rajat has joined #openstack-nova01:31
*** hongbin has joined #openstack-nova01:37
openstackgerritMerged openstack/nova stable/pike: Fix functional tests for USE_NEUTRON  https://review.openstack.org/64938501:54
openstackgerritMerged openstack/nova stable/pike: Add functional regression test for bug 1669054  https://review.openstack.org/64938601:54
openstackbug 1669054 in OpenStack Compute (nova) pike "RequestSpec.ignore_hosts from resize is reused in subsequent evacuate" [Medium,In progress] https://launchpad.net/bugs/1669054 - Assigned to Matt Riedemann (mriedem)01:54
*** ileixe has quit IRC01:55
*** ileixe has joined #openstack-nova01:58
*** Sundar has quit IRC01:58
openstackgerritMerged openstack/nova stable/pike: Do not persist RequestSpec.ignore_hosts  https://review.openstack.org/64938702:06
*** itlinux has joined #openstack-nova02:09
*** mriedem has joined #openstack-nova02:11
*** lbragstad has quit IRC02:30
*** cfriesen has quit IRC02:30
openstackgerritMatt Riedemann proposed openstack/nova master: Exclude fake marker instance when listing servers  https://review.openstack.org/65315802:32
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add post-test wrinkle to list deleted servers before archive  https://review.openstack.org/65313102:32
*** itlinux has quit IRC02:52
openstackgerritGhanshyam Mann proposed openstack/nova-specs master: Spec for API inconsistency cleanup  https://review.openstack.org/60396902:54
*** itlinux has joined #openstack-nova02:54
mriedemmelwitt: regarding the change in behavior for quota usage calculations pre and post pike with counting, you might want to report a bug just to capture the change in behavior just so we can at least document it if we want outside of mailing list threads and code review comments02:57
mriedemi was thinking something like if we fleshed out the quota docs reference page we could / should the big shift in design with counting in pike but could also list that as a known issue02:57
mriedemhttps://docs.openstack.org/nova/latest/user/quotas.html02:58
mriedemthere was also https://bugs.launchpad.net/nova/+bug/171670602:59
openstackLaunchpad bug 1716706 in OpenStack Compute (nova) "Should count instances in build requests when check quotas" [Medium,Triaged]02:59
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional regression recreate test for bug 1825020  https://review.openstack.org/65326803:03
openstackbug 1825020 in OpenStack Compute (nova) stein "resize of instance, that was booted from volume, trigger image size check then fails resize" [High,Triaged] https://launchpad.net/bugs/182502003:03
*** psachin has joined #openstack-nova03:12
*** ileixe has quit IRC03:25
melwittmriedem: ack03:28
eanderssonmriedem, nvm wasn't a nova bug03:29
eanderssonhttps://review.openstack.org/#/c/653271/03:29
eandersson* https://review.openstack.org/#/c/653233/03:29
*** ileixe has joined #openstack-nova03:31
mriedemeandersson: ack - you should report a bug for the latter03:33
mriedemor story or whatever03:33
eanderssonYep03:33
eanderssonI'll deal with that tomorrow03:33
*** hongbin has quit IRC03:38
*** itlinux has quit IRC03:43
openstackgerritMatt Riedemann proposed openstack/nova master: Pass root bdm to _validate_flavor_image_nostatus during resize  https://review.openstack.org/65327304:02
*** imacdonn has quit IRC04:07
*** imacdonn has joined #openstack-nova04:07
*** mriedem has quit IRC04:08
*** david-lyle has joined #openstack-nova04:13
*** manjeets_ has joined #openstack-nova04:14
*** dklyle has quit IRC04:16
*** manjeets has quit IRC04:16
openstackgerritmelanie witt proposed openstack/nova master: Add get_usages_counts_for_quota to SchedulerReportClient  https://review.openstack.org/65314504:27
openstackgerritmelanie witt proposed openstack/nova master: Count instances from mappings and cores/ram from placement  https://review.openstack.org/63807304:27
openstackgerritmelanie witt proposed openstack/nova master: Set [quota]count_usage_from_placement = True in nova-next  https://review.openstack.org/65314604:27
openstackgerritmelanie witt proposed openstack/nova master: Use instance mappings to count server group members  https://review.openstack.org/63832404:27
*** mdbooth has joined #openstack-nova04:30
*** ileixe has quit IRC04:31
*** mdbooth has quit IRC04:34
*** udesale has joined #openstack-nova04:37
*** ratailor has joined #openstack-nova04:51
*** ratailor has quit IRC04:51
*** ratailor has joined #openstack-nova04:53
*** ileixe has joined #openstack-nova05:02
*** vishakha has joined #openstack-nova05:09
*** ivve has joined #openstack-nova05:13
*** sridharg has joined #openstack-nova05:23
*** janki has joined #openstack-nova05:49
*** ileixe has quit IRC05:55
*** ileixe has joined #openstack-nova05:58
*** pcaruana has joined #openstack-nova06:11
*** belmoreira has joined #openstack-nova06:17
*** ralonsoh has joined #openstack-nova06:21
*** dpawlik has joined #openstack-nova06:31
*** udesale has quit IRC06:37
*** udesale has joined #openstack-nova06:38
*** udesale has quit IRC06:39
*** udesale has joined #openstack-nova06:44
*** udesale has quit IRC06:46
*** bhagyashris has joined #openstack-nova06:49
*** udesale has joined #openstack-nova06:56
*** ileixe has quit IRC06:57
*** ileixe has joined #openstack-nova07:02
*** tesseract has joined #openstack-nova07:03
*** belmoreira has quit IRC07:09
*** belmoreira has joined #openstack-nova07:10
*** rpittau|afk is now known as rpittau07:19
kashyapaspiers: Morning, just took a quick look.  I see the topic is: machine types as 'traits'.  Will respond today once I finish something07:22
*** tesseract has quit IRC07:29
*** tosky has joined #openstack-nova07:29
*** tesseract has joined #openstack-nova07:29
*** dtantsur|afk is now known as dtantsur07:33
*** tesseract has quit IRC07:33
*** tesseract has joined #openstack-nova07:34
*** tssurya has joined #openstack-nova07:43
*** awalende has joined #openstack-nova07:52
openstackgerritHamdy Khader proposed openstack/nova master: Do not perform port update in case of baremetal instance.  https://review.openstack.org/64934508:00
*** ttsiouts has joined #openstack-nova08:06
*** klindgren has quit IRC08:10
*** klindgren has joined #openstack-nova08:10
*** rcernin has quit IRC08:13
*** ttsiouts has quit IRC08:17
*** ttsiouts has joined #openstack-nova08:17
*** tkajinam has quit IRC08:17
*** ttsiouts has quit IRC08:22
*** rcernin has joined #openstack-nova08:27
*** ttsiouts has joined #openstack-nova08:28
*** vishakha has quit IRC08:33
openstackgerritLee Yarwood proposed openstack/nova-specs master: Re-propose stable device rescue for Train  https://review.openstack.org/65115108:33
*** luksky has joined #openstack-nova08:43
*** udesale has quit IRC08:51
*** ccamacho has joined #openstack-nova08:53
openstackgerritTetsuro Nakamura proposed openstack/nova master: Add in_tree field to RequestGroup object  https://review.openstack.org/64953409:09
openstackgerritTetsuro Nakamura proposed openstack/nova master: Add get_compute_nodes_by_host_or_node()  https://review.openstack.org/65087709:09
openstackgerritTetsuro Nakamura proposed openstack/nova master: Pass target host to RequestGroup.in_tree  https://review.openstack.org/65087809:09
openstackgerritTetsuro Nakamura proposed openstack/nova master: Query `in_tree` to placement  https://review.openstack.org/64953509:09
openstackgerritTetsuro Nakamura proposed openstack/nova master: WIP: FUP on use-placement-in-tree series  https://review.openstack.org/65336209:09
*** ttsiouts has quit IRC09:17
*** ttsiouts has joined #openstack-nova09:18
*** ttsiouts_ has joined #openstack-nova09:19
*** ttsiouts has quit IRC09:19
*** udesale has joined #openstack-nova09:20
*** mdbooth has joined #openstack-nova09:21
*** threestrands has quit IRC09:36
*** lpetrut has joined #openstack-nova09:40
*** mdbooth_ has joined #openstack-nova09:43
*** mdbooth_ has quit IRC09:46
*** mdbooth has quit IRC09:46
*** mdbooth has joined #openstack-nova09:49
*** bhagyashris has quit IRC09:49
stephenfinrechecking all the things09:49
*** luksky has quit IRC10:08
*** jchhatbar has joined #openstack-nova10:14
*** janki has quit IRC10:16
*** janki has joined #openstack-nova10:23
*** jchhatbar has quit IRC10:25
openstackgerritKashyap Chamarthy proposed openstack/nova-specs master: Add "Secure Boot support for KVM & QEMU guests" spec  https://review.openstack.org/50672010:26
kashyapsean-k-mooney: Hi, when you get a minute ^ I've wrote a summary response.  And addressed a few things.10:27
kashyapI spent a lot of time writing the summary response, please read it in full before responding.10:28
*** awalende has quit IRC10:32
*** awalende has joined #openstack-nova10:32
*** awalende has quit IRC10:36
*** ttsiouts_ has quit IRC10:37
*** ttsiouts has joined #openstack-nova10:38
*** nicolasbock has joined #openstack-nova10:38
*** yedongcan has left #openstack-nova10:39
*** luksky has joined #openstack-nova10:39
*** awalende has joined #openstack-nova10:42
*** ttsiouts has quit IRC10:42
*** tbachman has quit IRC10:45
kashyaplyarwood: gibi: If you have 15 more minutes today, appreciate another look at this: https://review.openstack.org/50672010:45
*** awalende has quit IRC10:56
*** awalende has joined #openstack-nova10:57
aspierskashyap: cool thanks11:01
kashyapaspiers: About to hit 'submit' with my comment11:02
aspiersawesome11:02
kashyapaspiers: Also if you have spare neurons, here's the UEFI SB spec: https://review.openstack.org/#/c/506720/11:02
kashyapI addressed as much as I can.  And my eyes are weary from reading my own text.  (Despite extensive editing)11:02
aspiersif anything I have a neuron deficit, but that's my problem ;-)11:02
aspiersHaha I know that feeling11:02
kashyapaspiers: No problem at all.  Then just say "no".  :-)11:04
aspiersI hate asking people for help and not offering some in return though :-/11:06
*** ttsiouts has joined #openstack-nova11:08
sean-k-mooneykashyap: ya most of your responce makes sense. ill re review with itopen in a secon tab11:12
kashyapNod.  I would like to avoid "scope creep".  "Takcle few things, and takle them well"11:13
sean-k-mooneywell i dont think you can tackel it well without the trait for secure boot support and if we add one then we have to add it for all hyperviors that support it or require operator to manully add it to the flavor instead of making it automatic11:16
sean-k-mooneyso the way to avoid scope creap is to documnet the reqrirement to add it to the flavor.11:16
kashyapLet's talk systematically on the change.  I need to go be AFK for lunch11:16
sean-k-mooneysure :)11:16
*** panda is now known as panda|lunch11:18
NewBrucesean-k-mooney: (mnaser) - have restared nova-conductor, issue still persists; cold migration RDO - OSA works fine, and OSA - OSA is ok11:19
NewBruce- have updated the launchpad11:19
*** jchhatbar has joined #openstack-nova11:20
*** ttsiouts has quit IRC11:20
*** ttsiouts has joined #openstack-nova11:21
sean-k-mooneyNewBruce: you are sure your python interperter was not built on friday the 13th and shipt on april 1st with some magical gremlens thrown in for good mesure11:21
NewBrucehe he he, seems that way, huh?11:21
NewBruceive got on my to try 1) disable binding-extended (if possible) 2) upgrade_levels = queens 3) priest + full a moon11:22
*** janki has quit IRC11:22
sean-k-mooneyim wondering if we litrally need to do a singel node osa deployment + add a singel node rdo compute to actlly be able to reporduce this11:22
NewBruceyeah, im thinking what we can do to reproduce11:23
sean-k-mooneyNewBruce: is you rdo deployment made with triplo or packstack?11:24
NewBruceNot that im aware of, it pre-dates my time here - but i think it was pretty ad-hoc11:24
*** ttsiouts has quit IRC11:25
sean-k-mooneyso its a manual install of RDO without a tool?11:25
NewBrucebut going back to basics and what we know; we can localise it to post_migrate and an attempt to update the port profiles, which it shouldnt be doing. right?11:25
sean-k-mooneyim wondering if we can use a gate job to repoduce11:25
openstackgerritJiri Suchomel proposed openstack/nova master: WIP: Initial prototype for simple RPC health checks  https://review.openstack.org/65340611:26
sean-k-mooneymnaser: it should be fairly simple to create an all in one OSA right, im wondering how hard it would be to also add a singel node rdo computej too maybe via packstack to repoducs NewBruce's issue11:27
NewBruceso, peppering the api.py with enough debug statements we’ve been able to localise where thats done - and we should be able to walk backwards right?11:27
*** ttsiouts has joined #openstack-nova11:27
NewBrucesean-k-mooney: did you see the full log traces i sent?11:28
sean-k-mooneyyes it is ending in post migrate so we should be able to walk backward.11:28
NewBrucegrabbed your email from you launchpad page - will drop you a mail shortly11:29
sean-k-mooneyNewBruce: no not yet. bit of a late start today so still catching up on things11:29
NewBrucequick question to the assembled masses - does anyone know if Enable cold migration with target host - Queen (https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/cold-migration-with-target-queens.html was ever implemented? ive not had any luck getting it to work11:33
sean-k-mooneywell its in the implemented section so it should have been yes11:34
sean-k-mooneyyou cant do it via osc11:35
NewBruceonly apl?11:35
sean-k-mooneynova migrate [--host <host>] [--poll] <server>11:36
NewBruceAha - gotcha; cheers11:36
*** cdent has joined #openstack-nova11:36
sean-k-mooneyyou have to add --host and the order is cahnged11:36
sean-k-mooneywe should proablly add that to osc too11:36
*** _erlon_ has joined #openstack-nova11:42
*** mdbooth_ has joined #openstack-nova11:42
*** ttsiouts has quit IRC11:43
*** ttsiouts has joined #openstack-nova11:44
*** boxiang has quit IRC11:44
*** boxiang has joined #openstack-nova11:45
*** mdbooth has quit IRC11:45
*** ttsiouts_ has joined #openstack-nova11:46
*** ttsiouts has quit IRC11:47
*** mdbooth_ has quit IRC11:47
*** ratailor has quit IRC11:48
*** ttsiouts_ has quit IRC11:51
*** awalende has quit IRC11:52
*** awalende has joined #openstack-nova11:53
*** awalende has quit IRC11:53
*** awalende has joined #openstack-nova11:54
*** awalende has quit IRC11:54
*** awalende has joined #openstack-nova11:54
*** awalende has quit IRC11:55
*** awalende has joined #openstack-nova11:56
*** awalende has joined #openstack-nova11:57
*** awalende has quit IRC11:58
*** cdent has quit IRC11:59
*** cdent has joined #openstack-nova12:04
*** dtantsur is now known as dtantsur|brb12:07
*** awalende has joined #openstack-nova12:08
*** pcaruana has quit IRC12:30
*** panda|lunch is now known as panda12:32
*** mdbooth has joined #openstack-nova12:41
*** artom has quit IRC12:42
*** pcaruana has joined #openstack-nova12:53
*** lbragstad has joined #openstack-nova12:56
*** vishakha has joined #openstack-nova12:57
*** mriedem has joined #openstack-nova12:57
*** mdbooth has quit IRC13:00
*** dtantsur|brb is now known as dtantsur13:07
*** mdbooth has joined #openstack-nova13:13
*** jding1__ has quit IRC13:16
mriedemthoughts on a potentially less invasive fix (see inline comments) on this stein regression https://review.openstack.org/#/c/653273/ are appreciated13:24
mriedemi wrote the fix late last night but aren't super happy with how grossly tightly coupled all of that code is13:25
*** mchlumsky has joined #openstack-nova13:26
*** awaugama has joined #openstack-nova13:27
sean-k-mooneymriedem: i was talking to NewBruce about the bug they are hitting and i tink i might know how to fix it but still not clear on why its happening. im starting on a patch now. should i add you to review list when i submit13:32
*** cdent has quit IRC13:36
*** cdent has joined #openstack-nova13:36
*** cdent has quit IRC13:40
sean-k-mooneylyarwood: mriedem when ye have time can one of ye +2 +W this https://review.openstack.org/#/c/644035/ so that we use the correct upper_constratins13:42
mriedemsean-k-mooney: done and yes13:49
sean-k-mooneythanks and cool will do13:51
*** artom has joined #openstack-nova13:53
mriedembauzas: want to hit this backport https://review.openstack.org/#/c/647911/ ?13:54
bauzassure13:54
mriedemall of them actually since it's a gate race issue https://review.openstack.org/#/q/If966eb1161c842ff49aa530e4482dbca87b61a3e+status:open13:54
bauzaslooking13:54
mriedemhttp://status.openstack.org/elastic-recheck/index.html#182033713:54
*** mlavalle has joined #openstack-nova14:02
*** awalende has quit IRC14:06
*** awalende has joined #openstack-nova14:06
*** cfriesen has joined #openstack-nova14:08
kashyapaspiers: Hiya...seems like I need the getDomainCapabilities() API call for Secure Boot14:08
kashyapaspiers: Wondering if you're planning to split it out: https://review.openstack.org/#/c/633855/11/nova/virt/libvirt/host.py@68014:08
aspiersI wasn't, as per previous conversation14:09
aspiersbut maybe that can be revisited14:09
kashyapaspiers: Yeah, let's revisit that conversation, plese14:09
aspiersCan you do me a favour and find the link to it? ;-)14:10
kashyapaspiers: Because, IMHO, it is not needed to tie that to SEV at all14:10
aspiersMaybe in that review14:10
aspiersI remember explicitly explaining the dependency somewhere14:10
aspiersOh, that *is* the link14:11
kashyapYeah :-)14:11
aspiersYeah, my comment on March 19th14:11
*** awalende_ has joined #openstack-nova14:11
kashyapaspiers: Oh, I'm blind.  Will read it14:11
aspiers:)14:11
*** awalende has quit IRC14:11
kashyapaspiers: The reason I ask is ... just now a libvirt dev was suggesting:14:11
kashyap    "I suggest Nova queries domcapabilities first, to see if 'efi' with Secure Boot is available"14:12
kashyap(When implementing SB support, i.e.)14:12
* kashyap bbiab14:12
*** awalende_ has quit IRC14:16
*** dansmith changes topic to "Current runways: https://etherpad.openstack.org/p/nova-runways-train -- This channel is for Nova development. For support of Nova deployments, please use #openstack."14:16
*** ChanServ sets mode: -o dansmith14:16
*** itlinux has joined #openstack-nova14:19
*** dpawlik has quit IRC14:19
mriedemso we have a RESIZE_CONFIRMING task_state enum but never use it while confirming a resize https://github.com/openstack/nova/blob/e25d59078e61fe9f925dbef53dfe88e575d34dab/nova/compute/api.py#L353514:21
mriedemthat seems...not good?14:21
mriedemunlike when reverting a resize we set the task_state in the API before casting to compute https://github.com/openstack/nova/blob/e25d59078e61fe9f925dbef53dfe88e575d34dab/nova/compute/api.py#L351514:22
*** itlinux has quit IRC14:22
openstackgerritVladyslav Drok proposed openstack/nova master: libvirt: Do not reraise DiskNotFound exceptions during resize  https://review.openstack.org/57141014:26
*** liuyulong has quit IRC14:27
mriedemvdrok: mdbooth: lyarwood: more comments in https://review.openstack.org/#/c/571410/914:28
*** liuyulong has joined #openstack-nova14:28
*** lpetrut has quit IRC14:28
*** cdent has joined #openstack-nova14:28
mriedemi'm inclined to take a more conservative incremental approach to fixing the VERIFY_RESIZE steady state issue14:29
mriedemwhich we could also backport more safely i think14:29
vdrokmeaning somewhat like it was done in ps7 https://review.openstack.org/#/c/571410/7/nova/virt/libvirt/driver.py14:31
mriedemexactly14:31
mriedemi'm not entirely sold on the completely ignore case like in bug 1824974 where the disk on the host is garbage and we just put our heads in the sand14:33
openstackbug 1824974 in OpenStack Compute (nova) "Nova Compute Manager (Resource update) fails if a disk is missing" [Low,In progress] https://launchpad.net/bugs/1824974 - Assigned to Vladyslav Drok (vdrok)14:33
*** markvoelker has joined #openstack-nova14:35
*** david-lyle is now known as dklyle14:35
mnaseras far as I know RDO ships upstream code14:39
mnaserNewBruce: I wonder if it would be a good exercise to change one of the compute nodes to log it's current highest negotiated rpc version14:40
mnaserand then see if for some reason it's negotiating a bad number14:40
artommnaser, I believe you're right, RDO is straight up upstream14:41
artomAny backports we do is OSP, not RSO14:41
artom*RDO14:41
*** tbachman has joined #openstack-nova14:42
mnaseryeah, so RDO is upstream, OSA is upstream14:45
*** luksky has quit IRC14:46
openstackgerritDan Smith proposed openstack/nova master: Make libvirt expose supported image types  https://review.openstack.org/65345414:47
sean-k-mooneymnaser: im currently updating unit test but i think http://paste.openstack.org/show/749431/ will fix NewBruce issue14:48
efriedmriedem: There's no blueprint or anything for sdk-in-nova yet because up to this point we've been experimenting to see how it would work and what we can get done. I do have it on the ptg etherpads in some form. But will also try to file a bp soon.14:51
sean-k-mooneyefried: well you can file a blueprint just as a tracker right14:51
efriedyeah, could do.14:51
openstackgerritVladyslav Drok proposed openstack/nova master: libvirt: Do not reraise DiskNotFound exceptions during resize  https://review.openstack.org/57141014:51
mnasersean-k-mooney: that's fair, I do feel like that might be a bit of a workaround I guess :-p14:52
efrieddustinc: How do you feel about leading that?14:52
sean-k-mooneyefried: it does not need to have a solution atached just "sdk is a thing we should use it"14:52
mriedem"x exists we should use it" isn't sufficient justification imo14:52
sean-k-mooneymnaser: well i have not found the root casue so yes but i think we likely should make this chagne on master and backport and then see if we can create a reporducer14:52
efriedsean-k-mooney: Part of what we've been working on is the scope too. "Plumb SDK" is one aspect; "stop using python-${service}client" is another.14:53
sean-k-mooneymriedem: maybe not to approve the blueprint but it enough to file it14:53
*** jchhatbar has quit IRC14:53
sean-k-mooneyefried: ah ok14:53
mriedemsure, blueprints are free14:53
*** jchhatbar has joined #openstack-nova14:54
mriedemfixing the bugs they introduce is not14:54
sean-k-mooneywas "stop using python-*-client" in services approved as a comunity goal by the way14:54
mriedemhttps://governance.openstack.org/tc/goals/14:55
efriedsean-k-mooney, mriedem: The reasoning would go something like: "We want to stop using python-${service}client => We could use ksa adapter stuff, which we've already set up, but then we would have to write do-a-thing wrappers around get/put/etc primitives => so let's use the sdk, which already has those wrappers"14:55
sean-k-mooneyactully i guess that is a dumb question that will be decided at the fourm/ptg in 2 weeks14:55
openstackgerritAdam Spiers proposed openstack/nova master: Add ids to sections of flavors guide to allow deep-linking  https://review.openstack.org/65345814:55
*** dustinc has quit IRC14:56
*** dustinc has joined #openstack-nova14:56
*** dustinc is now known as dustinc_away14:56
mriedemsean-k-mooney: is that even a proposed goal for train or are you just making it up?14:56
sean-k-mooneyefried: part of the scoping would be ensuring the sdk has the actual wrapper we need14:56
efriedmriedem: there was a proposed goal to stop using python-${service}client for CLIs.14:57
*** weshay is now known as weshay|rover14:57
mriedemyeah, that's quite a bit different14:57
efriedwhich is distantly related to this, but not really overlapping.14:57
efriedyeah.14:57
*** udesale has quit IRC14:57
sean-k-mooneyah that is proably what im mixing it up with14:57
sean-k-mooneyit was one of the tread i lost track of on the mailing list14:58
efriedmriedem: anyway, the ironic team (as an example) is strongly behind the idea of ripping ironicclient out of nova14:58
efriedso I poked at a couple of ways of doing that14:58
*** udesale has joined #openstack-nova14:59
*** udesale has quit IRC14:59
efriedand it's about time we did *something* to break our pattern of "set up a whole KSA adapter just to grab the endpoint out of it and send that into *client which uses it to create a whole nother Adapter to do its work"14:59
efriedwhich we do (in different ways!) for glance, ironic, neutron...14:59
*** udesale has joined #openstack-nova14:59
mriedemthe ksa stuff in nova still feels new to me, but i'm slow and old15:00
mriedemand was a big improvement over the per-service config and all that15:00
mriedemcinder excluded since we haven't fixed that one yet15:01
*** lpetrut has joined #openstack-nova15:01
gmannmriedem: updated this with review comments.  -  https://review.openstack.org/#/c/603969/15:01
efriedmriedem: It was an improvement in the sense that operators can now use same-looking conf values for talking to the services. But internally we didn't improve much: we're still doing the dance I mentioned above --^15:04
efriedAnd, wasn't the ksa cutover in Queens? That's a pretty long time ago. Maybe just seems that way for me as it's pretty much when I first got involved in nova proper.15:05
*** udesale has quit IRC15:06
mriedemqueens is still very new to me15:07
mriedemhell pike is as well15:08
mriedemif i'm still fixing bugs in ocata/pike etc it's new to me15:08
*** dakshina-ilangov has joined #openstack-nova15:10
*** cdent has quit IRC15:11
mriedemvdrok: just some small updates to https://review.openstack.org/#/c/571410/15:16
mriedemdansmith: can you hit this backport to keep things moving https://review.openstack.org/#/c/651639/215:24
mriedemand https://review.openstack.org/#/q/If966eb1161c842ff49aa530e4482dbca87b61a3e+status:open since that's a gate issue hitting stable backports15:24
mriedemmelwitt: can you hit these backports? https://review.openstack.org/#/q/topic:bug/1819963+(status:open+OR+status:merged)+branch:stable/rocky15:25
cfriesenmriedem: thanks for triaging that resize regression issue and writing the regression test.  looks like Jack and I dropped the ball there.15:25
mriedemcfriesen: i approved the change15:25
mriedemi probably approved all of these busted changes15:25
mriedemcfriesen: please see my comment-to-self on the fix and drop your opinion on decoupling the grossitude15:25
dansmithmriedem: ack15:25
cfriesenmriedem: for sure15:26
mriedemcfriesen: i'll also say that as much as i don't like these regressions, i'm happy there are people in your org somewhere doing real testing15:26
mriedemlyarwood: can you hit this backport? https://review.openstack.org/#/c/647623/15:27
openstackgerritVladyslav Drok proposed openstack/nova master: libvirt: Do not reraise DiskNotFound exceptions during resize  https://review.openstack.org/57141015:27
mriedemgmann: ack15:27
mriedemvdrok: +2 thanks15:28
mriedemwould be good to get another core on ^ since that's been around a long time15:28
mriedemand is very latent15:28
mriedemefried: ^15:28
lyarwoodmriedem: ack, I'll also try to get around to that bugfix again later, slightly busy downstream today.15:31
mriedemlyarwood: which bug fix?15:31
*** belmoreira has quit IRC15:32
lyarwoodmriedem: vdrok's above15:32
mriedemoh ok15:32
*** jchhatbar has quit IRC15:33
*** jchhatbar has joined #openstack-nova15:33
*** itlinux has joined #openstack-nova15:35
*** jchhatbar has quit IRC15:36
*** jchhatbar has joined #openstack-nova15:36
*** jchhatbar has quit IRC15:40
*** lpetrut has quit IRC15:42
*** cdent has joined #openstack-nova15:54
mriedemcfriesen: i'm going with that alternative - have it locally and it's a much simpler diff15:57
*** gyee has joined #openstack-nova15:58
openstackgerritMerged openstack/os-vif stable/stein: Update UPPER_CONSTRAINTS_FILE for stable/stein  https://review.openstack.org/64403516:03
cfriesenmriedem: so you're moving the last ~10 lines of _validate_flavor_image_nostatus() into another function?16:05
mriedemyup16:05
aspiersefried: run out of time for today, but I've almost finished updating the spec to talk about the new resource class rather than the existing HW_CPU_AMD_SEV trait. I'll submit another patchset tomorrow16:07
efriedaspiers: I'm not quite caught up, did you get other cores to ack that shift in direction?16:07
*** dims has quit IRC16:07
efriedmriedem: what am I looking at? https://review.openstack.org/#/c/571410/ ?16:07
*** lpetrut has joined #openstack-nova16:09
cfriesenmriedem: seems like it could work16:09
mriedemefried: hold on that one16:11
efriedack16:11
*** ccamacho has quit IRC16:16
openstackgerritMatt Riedemann proposed openstack/nova master: Fix volume-backed resize with a smaller disk flavor  https://review.openstack.org/65327316:18
openstackgerritMatt Riedemann proposed openstack/nova master: Update volume-backed comment in _validate_flavor_image_nostatus  https://review.openstack.org/65348316:18
openstackgerritMatt Riedemann proposed openstack/nova master: Remove FlavorNotFound dead code condition in API.resize  https://review.openstack.org/65348416:18
mriedemcfriesen: https://review.openstack.org/#/c/653273/16:18
*** rpittau is now known as rpittau|afk16:20
openstackgerritEric Fried proposed openstack/nova master: Add ids to sections of flavors guide to allow deep-linking  https://review.openstack.org/65345816:23
*** dtantsur is now known as dtantsur|afk16:24
*** david-lyle has joined #openstack-nova16:24
*** igordc has joined #openstack-nova16:25
*** ileixe has quit IRC16:25
*** manjeets__ has joined #openstack-nova16:26
*** dklyle has quit IRC16:27
*** manjeets_ has quit IRC16:27
cfriesenmriedem: couple comments in the review16:30
mriedemi'll fix those image status docstrings even though i blame you :)16:33
cfriesenmy bad entirely. :)16:34
openstackgerritMatt Riedemann proposed openstack/nova master: Fix volume-backed resize with a smaller disk flavor  https://review.openstack.org/65327316:34
openstackgerritMatt Riedemann proposed openstack/nova master: Update volume-backed comment in _validate_flavor_image_nostatus  https://review.openstack.org/65348316:34
openstackgerritMatt Riedemann proposed openstack/nova master: Remove FlavorNotFound dead code condition in API.resize  https://review.openstack.org/65348416:34
cfriesenmriedem: it's still a bit grody, but the underlying special-casing is complicated so I don't see a nice way to make it cleaner16:35
*** tesseract has quit IRC16:40
*** idlemind has joined #openstack-nova16:51
*** dims has joined #openstack-nova16:54
*** dims has quit IRC16:59
*** dims has joined #openstack-nova17:01
*** ricolin has quit IRC17:04
openstackgerritDan Smith proposed openstack/nova master: Make libvirt expose supported image types  https://review.openstack.org/65345417:08
*** luksky has joined #openstack-nova17:09
*** dakshina-ilangov has quit IRC17:26
*** igordc has quit IRC17:26
*** markvoelker has quit IRC17:31
*** markvoelker has joined #openstack-nova17:31
artomWould there be a point in adding multinode resize tests somewhere?17:32
artomAFAICT currently we're only testing in in the allinone tempest-full job17:32
artomWhich isn't super realistic...17:32
artomI want to try it regardless, but how I do it will depend. Either as a throwaway DNM hack, or I'll put in some minimal amount of effort17:33
*** markvoelker has quit IRC17:35
*** ralonsoh has quit IRC17:36
*** awaugama has quit IRC17:39
*** awaugama has joined #openstack-nova17:39
*** jangutter has quit IRC17:39
openstackgerritArtom Lifshitz proposed openstack/nova master: Revert "Wait for network-vif-plugged on resize revert"  https://review.openstack.org/63939617:40
openstackgerritArtom Lifshitz proposed openstack/nova master: Revert resize: wait for external events in compute manager  https://review.openstack.org/64488117:40
openstackgerritArtom Lifshitz proposed openstack/nova master: DNM: Run test_resize_server_revert in the live migration job  https://review.openstack.org/65349817:40
artomh4x17:40
*** igordc has joined #openstack-nova17:44
*** bryan_stephenson has joined #openstack-nova17:52
*** ivve has quit IRC17:52
*** manjeets__ is now known as manjeets17:53
*** david-lyle is now known as dklyle17:55
*** tssurya has quit IRC17:58
openstackgerritEric Fried proposed openstack/nova master: DNM: DEMO: Reinstate bug 1819963  https://review.openstack.org/65350318:01
openstackbug 1819963 in OpenStack Compute (nova) rocky "Reverting a resize does not update the instance.availability_zone value to the source az" [Medium,In progress] https://launchpad.net/bugs/1819963 - Assigned to Matt Riedemann (mriedem)18:01
openstackgerritEric Fried proposed openstack/nova master: DEMO: Functional recreate test for bug 1819963  https://review.openstack.org/65350418:01
openstackgerritEric Fried proposed openstack/nova master: DEMO: Update instance.availability_zone on revertResize  https://review.openstack.org/65350518:01
efrieddammit, spelling those tags wrong didn't prevent them from posting to lp18:03
efriedoh well, guess I might as well spell them correctly then.18:03
openstackgerritEric Fried proposed openstack/nova master: DEMO: Functional recreate test for bug 1819963  https://review.openstack.org/65350418:04
openstackbug 1819963 in OpenStack Compute (nova) rocky "Reverting a resize does not update the instance.availability_zone value to the source az" [Medium,In progress] https://launchpad.net/bugs/1819963 - Assigned to Matt Riedemann (mriedem)18:04
openstackgerritEric Fried proposed openstack/nova master: DEMO: Update instance.availability_zone on revertResize  https://review.openstack.org/65350518:04
openstackgerritsean mooney proposed openstack/nova master: [WIP] only update port binding if there is not an inactive binding  https://review.openstack.org/65350618:06
sean-k-mooneymriedem: i spent the last hour or more fighting with some of the unit test so im takeing a break form it but ^ i think will fix NewBruce issue18:12
NewBrucewill test it out shortly18:12
NewBruceThanks sean-k-mooney18:12
sean-k-mooneyok im going to go have dinner. ill check back later to see what the gate thinks.18:13
*** psachin has quit IRC18:13
*** boxiang has quit IRC18:20
*** zhubx has joined #openstack-nova18:20
openstackgerritMatt Riedemann proposed openstack/nova master: libvirt: set device address tag only if setting disk unit  https://review.openstack.org/61197418:21
*** cfriesen has quit IRC18:21
mriedemmelwitt: begin backport procedures18:21
efriedmriedem: I'm setting up a demo for the summit onboarding session. I want to show adding a regression test for a bug and then fixing the bug. I want to use https://bugs.launchpad.net/nova/+bug/181996318:34
openstackLaunchpad bug 1819963 in OpenStack Compute (nova) rocky "Reverting a resize does not update the instance.availability_zone value to the source az" [Medium,In progress] - Assigned to Matt Riedemann (mriedem)18:34
efriedI've got most of the setup done at this point, just have a question about the bug itself: Given that the API was masking that the instance AZ in the db doesn't match where the instance actually is, for whom is it actually a problem, and why?18:34
mriedemthe most obvious one is filtering instances by az18:38
mriedemGET /servers?availability_zone=foo18:38
mriedemwhen it's really in bar18:38
*** tbachman has quit IRC18:39
*** lpetrut has quit IRC18:41
*** lpetrut has joined #openstack-nova18:42
*** cfriesen has joined #openstack-nova18:43
*** sridharg has quit IRC18:48
openstackgerritmelanie witt proposed openstack/nova stable/stein: libvirt: set device address tag only if setting disk unit  https://review.openstack.org/65351018:52
*** tbachman has joined #openstack-nova18:55
*** vishakha has quit IRC18:56
mriedemgmann: done https://review.openstack.org/#/c/603969/18:56
openstackgerritmelanie witt proposed openstack/nova stable/stein: libvirt: set device address tag only if setting disk unit  https://review.openstack.org/65351018:57
openstackgerritmelanie witt proposed openstack/nova stable/rocky: libvirt: set device address tag only if setting disk unit  https://review.openstack.org/65351118:59
openstackgerritmelanie witt proposed openstack/nova stable/queens: libvirt: set device address tag only if setting disk unit  https://review.openstack.org/65351219:00
*** cdent has quit IRC19:04
openstackgerritmelanie witt proposed openstack/nova stable/pike: libvirt: set device address tag only if setting disk unit  https://review.openstack.org/65351419:05
*** tbachman has quit IRC19:06
openstackgerritEric Fried proposed openstack/nova master: DEMO: Functional recreate test for bug 1819963  https://review.openstack.org/65350419:08
openstackbug 1819963 in OpenStack Compute (nova) rocky "Reverting a resize does not update the instance.availability_zone value to the source az" [Medium,In progress] https://launchpad.net/bugs/1819963 - Assigned to Matt Riedemann (mriedem)19:08
openstackgerritEric Fried proposed openstack/nova master: DEMO: Update instance.availability_zone on revertResize  https://review.openstack.org/65350519:08
*** keekz is now known as nicholas19:08
efriedmriedem: Thanks. I've done a couple of things to improve that test case ^ - lmk if you think it's worth pulling those out into a real patch.19:08
efriedactually, the first thing - checking the db explicitly - would have only applied to the interim recreate patch. So it's just the part where we do that GET /servers?availability_zone=* query that would be added now.19:09
*** tbachman has joined #openstack-nova19:12
openstackgerritGhanshyam Mann proposed openstack/nova-specs master: Spec for API inconsistency cleanup  https://review.openstack.org/60396919:20
gmannmriedem: thanks. ^^ updated.19:21
*** artom has quit IRC19:22
*** tbachman has quit IRC19:26
*** tjgresha has quit IRC19:33
*** tjgresha_nope has joined #openstack-nova19:33
*** tjgresha_nope has quit IRC19:40
*** lpetrut has quit IRC19:40
*** tjgresha has joined #openstack-nova19:44
*** tjgresha has quit IRC19:51
openstackgerritMerged openstack/nova master: Remove '/os-cells' REST APIs  https://review.openstack.org/65129120:16
openstackgerritMerged openstack/nova master: Stop handling cells v1 in '/os-hypervisors' API  https://review.openstack.org/65129220:16
openstackgerritMerged openstack/nova stable/rocky: Fix incomplete instance data returned after build failure  https://review.openstack.org/64791120:29
openstackgerritMerged openstack/nova stable/queens: Update instance.availability_zone during live migration  https://review.openstack.org/64762320:29
*** francoisp_ has quit IRC20:31
*** tjgresha has joined #openstack-nova20:34
*** tjgresha has quit IRC20:39
*** francoisp has joined #openstack-nova20:42
*** pcaruana has quit IRC20:43
*** tjgresha has joined #openstack-nova20:54
efriedmriedem: Prior to "handle down cells", would the APIs just return failures?20:55
*** tjgresha has quit IRC20:56
mriedemefried: by default no, but it's configurable20:57
mriedemhttps://docs.openstack.org/nova/latest/admin/cells.html#handling-cell-failures20:57
openstackgerritsean mooney proposed openstack/nova master: [WIP] only update port binding if there is not an inactive binding  https://review.openstack.org/65350620:57
mriedemhttps://docs.openstack.org/nova/latest/configuration/config.html#api.list_records_by_skipping_down_cells20:57
efriedokay, so previously we would either (a) show information only for up cells, completely skipping down cells as if they weren't there; or (b) error the API call20:59
efriedand now we (c) show partial information from the down cells20:59
mriedem(c) if you're using microversion >= 2.6921:00
*** whoami-rajat has quit IRC21:01
efriedack, thx21:02
*** awaugama has quit IRC21:03
*** slaweq has quit IRC21:17
*** slaweq has joined #openstack-nova21:20
openstackgerritsean mooney proposed openstack/nova stable/rocky: [WIP] only update port binding if there is not an inactive binding  https://review.openstack.org/65360921:21
*** bryan_stephenson has quit IRC21:22
sean-k-mooney^ is for testing21:24
*** slaweq has quit IRC21:24
*** rchurch has joined #openstack-nova21:24
*** mchlumsky has quit IRC21:29
*** logan- has quit IRC21:34
*** logan- has joined #openstack-nova21:37
*** slaweq has joined #openstack-nova21:41
*** slaweq has quit IRC21:45
*** luksky has quit IRC21:48
*** takashin has joined #openstack-nova21:50
*** slaweq has joined #openstack-nova21:51
*** hongbin has joined #openstack-nova21:52
*** slaweq has quit IRC21:55
mriedemhere it comes22:01
openstackgerritMatt Riedemann proposed openstack/nova master: Fix ProviderUsageBaseTestCase._run_periodics for multi-cell  https://review.openstack.org/64117922:02
openstackgerritMatt Riedemann proposed openstack/nova master: Improve CinderFixtureNewAttachFlow  https://review.openstack.org/63938222:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional recreate test for bug 1818914  https://review.openstack.org/64152122:02
openstackbug 1818914 in OpenStack Compute (nova) "Hypervisor resource usage on source still shows old flavor usage after resize confirm until update_available_resource periodic runs" [Low,In progress] https://launchpad.net/bugs/1818914 - Assigned to Matt Riedemann (mriedem)22:02
openstackgerritMatt Riedemann proposed openstack/nova master: Remove unused context parameter from RT._get_instance_type  https://review.openstack.org/64179222:02
openstackgerritMatt Riedemann proposed openstack/nova master: Update usage in RT.drop_move_claim during confirm resize  https://review.openstack.org/64180622:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid  https://review.openstack.org/61401222:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method  https://review.openstack.org/61403622:02
openstackgerritMatt Riedemann proposed openstack/nova master: DNM: Add instance hard delete  https://review.openstack.org/65098422:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add Instance.hidden field  https://review.openstack.org/63112322:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add TargetDBSetupTask  https://review.openstack.org/62789222:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask  https://review.openstack.org/63158122:02
openstackgerritMatt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask  https://review.openstack.org/63385322:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method  https://review.openstack.org/62131322:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63329322:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask  https://review.openstack.org/62789022:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method  https://review.openstack.org/63483222:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add nova.compute.utils.delete_image  https://review.openstack.org/63760522:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask  https://review.openstack.org/62789122:02
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add RevertResizeTask  https://review.openstack.org/63804622:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize conductor RPC method  https://review.openstack.org/63804722:02
openstackgerritMatt Riedemann proposed openstack/nova master: Revert cross-cell resize from the API  https://review.openstack.org/63804822:02
openstackgerritMatt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server  https://review.openstack.org/63826822:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add archive_deleted_rows wrinkle to cross-cell functional test  https://review.openstack.org/65165022:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add CrossCellWeigher  https://review.openstack.org/61435322:02
openstackgerritMatt Riedemann proposed openstack/nova master: Add cross-cell resize policy rule and enable in API  https://review.openstack.org/63826922:02
mriedemtime to scram http://grafana.openstack.org/d/T6vSHcSik/zuul-status?orgId=1&fullscreen&panelId=322:03
mriedembtw, ^ is in a runway slot and the bottom 3 changes have a +2 on them, should be relatively easy if another core can help out22:04
*** mriedem has quit IRC22:10
*** zhubx has quit IRC22:18
*** mlavalle has quit IRC22:18
*** boxiang has joined #openstack-nova22:19
mnaserhmm22:29
mnaseris there a possible race condition around cell discover hosts?22:29
mnaserthis is really weird22:29
mnaserhttp://logs.openstack.org/32/653532/1/gate/openstack-ansible-deploy-aio_metal-ubuntu-bionic/a2f00da/logs/ara-report/result/2a432934-7b70-491a-9571-a40e85cd15c6/22:29
mnaser"Checking host mapping for compute host" repeated twice, but with the same uuid?22:30
mnaserthe heck22:31
mnaserhttp://logs.openstack.org/32/653532/1/gate/openstack-ansible-deploy-aio_metal-ubuntu-bionic/a2f00da/logs/host/nova/nova-compute.log.txt.gz#_2019-04-17_21_52_21_72722:31
mnaserhttp://logs.openstack.org/32/653532/1/gate/openstack-ansible-deploy-aio_metal-ubuntu-bionic/a2f00da/logs/host/nova/nova-compute.log.txt.gz#_2019-04-17_21_52_21_81922:31
mnaserin the same pid..22:32
melwittare you running discover_hosts in parallel during a deployment?22:33
*** lbragstad has quit IRC22:33
melwitthttps://bugs.launchpad.net/nova/+bug/182444522:34
openstackLaunchpad bug 1824445 in OpenStack Compute (nova) "nova-manage cellv2 discover_hosts traces when run in parallel" [Low,In progress] - Assigned to melanie witt (melwitt)22:35
*** lbragstad has joined #openstack-nova22:35
mnasermelwitt: nope, should be once only22:36
mnaserbut also is it normal that there is two log lines that say compute node created22:36
mnasermelwitt: we also delegate_to the nova-conductor host so we run it once22:37
melwittI've only seen this before when the command is run in parallel presumably on different hosts22:39
melwittthat's not normal, it shouldn't be trying to create the host mapping for the same host twice22:39
melwittI don't even get how that could happen non-parallel22:39
mnaseryeah..22:39
mnaserand the pid is the same too22:40
melwittI dunno, seems like dark magic22:41
melwitthm, ok the last log is ringing a bell22:42
melwittdid you delete a service via the API and then not restart the nova-compute that goes with it?22:42
melwittbecause nova-compute will only create a compute node record when it starts up, it will not recover if the compute node record gets deleted out from under it while it's running22:43
mnaserwe don't delete anything afaik :\22:46
mnaserduring the CI run at least22:46
melwittok. I don't see how that would be related to the discover_hosts thing anyway if it were happening22:46
mnaserhttps://github.com/openstack/openstack-ansible-os_nova/blob/d1aa5839b482c7412404ebb7bf050ee806fbf281/tasks/main.yml#L233-L24322:48
melwittmnaser: oh, sorry, I see now it's saying Checking first and then Creating, so that's probably normal22:48
mnaseroh oops22:48
melwittI thought it was the same message twice but it's not22:48
mnaseryou're right yes22:49
mnaser"Compute node record created" is just once22:49
melwittbut the duplicate, I've only seen that when one nova-manage is racing against another. do you have the discover_hosts scheduler periodic enabled?22:49
*** tbachman has joined #openstack-nova22:49
mnaserI think we enable that based on something22:49
mnaserone sec22:49
openstackgerritEric Fried proposed openstack/nova master: Introduces the openstacksdk to nova  https://review.openstack.org/64366422:49
melwittthat's another possibility, if you were unlucky enough to have that fire at the same time as another scheduler, or if you were to run 'nova-manage cellv2 discover_hosts' at the same time a scheduler periodic discover_hosts fires22:50
mnaserscheduler.discover_hosts_in_cells_interval = 6022:50
openstackgerritEric Fried proposed openstack/nova master: WIP/PoC: Use SDK instead of ironicclient for node.get  https://review.openstack.org/64289922:50
openstackgerritEric Fried proposed openstack/nova master: WIP/PoC: Use SDK instead of ironicclient for node.get  https://review.openstack.org/64289922:51
mnaserdoes discover_hosts run in the conductor I assume?22:51
melwittok, and is that set on all schedulers or just one? (recommendation is to enable it on only one) see https://review.openstack.org/65194722:51
melwittscheduler22:51
mnaserhttp://logs.openstack.org/32/653532/1/gate/openstack-ansible-deploy-aio_metal-ubuntu-bionic/a2f00da/logs/host/nova/nova-scheduler.log.txt.gz#_2019-04-17_21_52_03_77622:52
mnaseryeah its around the same time22:52
mnaserwe have it enabled on all hosts :X22:52
melwittok. yeah, I don't think we knew this was going to be a potential problem, or at least I didn't know22:53
mnaserI mean its the first time I see this22:53
mnaserbut at least we kno wwhy22:53
melwittthe trace isn't harmful, it just means discover_hosts collided with another one22:53
melwittso the patch I linked earlier aims to warn and give guidance instead of tracing22:53
mnaseryeah that would be good because the ansible obviously fails when it sees that22:54
mnaserso that would be good to know22:54
melwittcommit message explains why to do that too vs ignore DBDuplicateEntry22:54
mnaserneat22:54
melwittoh it does? how do yall do discover_hosts? keep running until you receive status code 0? that's what tripleo does22:54
*** tkajinam has joined #openstack-nova22:54
melwittthe status code is a little weird on that command where non-zero means "there's more work to do" and zero means "I'm done"22:55
melwittand I think that's the review feedback I got on that patch currently is to enhance the doc to explain that22:55
openstackgerritMerged openstack/nova master: Stop handling cells v1 in '/os-servers' API  https://review.openstack.org/65129322:57
openstackgerritMerged openstack/nova stable/rocky: Add functional recreate test for bug 1819963  https://review.openstack.org/64840922:57
openstackbug 1819963 in OpenStack Compute (nova) rocky "Reverting a resize does not update the instance.availability_zone value to the source az" [Medium,In progress] https://launchpad.net/bugs/1819963 - Assigned to Matt Riedemann (mriedem)22:57
openstackgerritMerged openstack/nova stable/rocky: Update instance.availability_zone on revertResize  https://review.openstack.org/64841022:57
mnasermelwitt: we run discover_hosts every time once only on one of the conductors22:57
mnaserI didn't know discover_hosts does work in batches22:58
melwittoh, so always expect to receive 022:58
melwittI don't think it does. tripleo retries everything22:58
mnaserhttps://github.com/openstack/openstack-ansible-os_nova/blob/d1aa5839b482c7412404ebb7bf050ee806fbf281/tasks/nova_db_post_setup.yml#L21-L2622:58
*** zigo has quit IRC22:58
mnaserya I try to avoid, uh, brute forcing stuff :p22:59
melwittyeah it doesn't batch. it's just that non-zero means "I couldn't do the work I needed to do"22:59
melwittok... so this will be a problem. we had been thinking to still return 1 if a collision happens i.e. not change the behavior23:00
mnaserright, we can adapt our tasks easily then23:00
melwittbut if you happen to run it while your scheduler does a discover_hosts, you will get a fail status code23:00
mnaseryeah, our case is not parallel but racing with scheduler23:00
melwittyeah23:01
melwittok, well if you want/need different behavior than is currently proposed, please add a comment on the patch review23:01
mnaserill add comments that we hit this23:02
melwittexample: specific non-zero code that means "collision detected"23:02
melwittok23:02
mnasermelwitt: let a comment, danke :)23:05
melwittkewl23:06
*** igordc has quit IRC23:21
*** itlinux has quit IRC23:26
*** tosky has quit IRC23:38
*** hongbin has quit IRC23:38
*** itlinux has joined #openstack-nova23:43
*** itlinux has quit IRC23:44
*** artom has joined #openstack-nova23:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!