Tuesday, 2019-02-12

*** jmlowe has quit IRC00:03
*** wolverineav has quit IRC00:04
*** wolverineav has joined #openstack-nova00:05
*** wolverineav has quit IRC00:10
*** wolverineav has joined #openstack-nova00:14
*** tetsuro has joined #openstack-nova00:25
*** TxGirlGeek has quit IRC00:25
*** macza has joined #openstack-nova00:29
*** tetsuro has quit IRC00:30
*** tetsuro has joined #openstack-nova00:32
*** macza has quit IRC00:34
*** tetsuro has quit IRC00:35
*** tetsuro_ has joined #openstack-nova00:35
*** wolverineav has quit IRC00:37
*** takashin has joined #openstack-nova00:38
*** wolverineav has joined #openstack-nova00:39
*** wolverineav has quit IRC00:43
*** wolverineav has joined #openstack-nova00:44
*** gyee has quit IRC00:54
*** zioproto has joined #openstack-nova00:59
*** fyx has joined #openstack-nova00:59
*** TheJulia has joined #openstack-nova00:59
*** mnasiadka has joined #openstack-nova00:59
*** coreycb has joined #openstack-nova00:59
*** icey has joined #openstack-nova00:59
*** mrhillsman has joined #openstack-nova00:59
*** sdake has joined #openstack-nova01:01
*** moshele has joined #openstack-nova01:07
*** macza has joined #openstack-nova01:08
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Start functional testing for cross-cell resize  https://review.openstack.org/63625301:09
*** wolverineav has quit IRC01:09
mriedemwoooowee01:09
mriedemdansmith: ^ well that's progress01:09
mriedemretrofitted my previous functional tests to the existing series, flushed a bunch of bugs, and have it tested up through VERIFY_RESIZE01:09
*** wolverineav has joined #openstack-nova01:09
*** macza has quit IRC01:13
*** tetsuro has joined #openstack-nova01:20
*** tetsuro_ has quit IRC01:20
*** tiendc has joined #openstack-nova01:22
*** bhagyashris has joined #openstack-nova01:22
*** mdbooth has joined #openstack-nova01:22
*** mriedem has quit IRC01:23
*** mdbooth_ has quit IRC01:24
*** moshele has quit IRC01:25
*** yedongcan has joined #openstack-nova01:25
*** markvoelker has quit IRC01:30
*** markvoelker has joined #openstack-nova01:31
*** tetsuro has quit IRC01:35
*** tetsuro_ has joined #openstack-nova01:35
*** markvoelker has quit IRC01:35
*** _fragatina has quit IRC01:37
*** sdake has quit IRC01:37
*** sdake has joined #openstack-nova01:41
*** sdake has quit IRC01:43
*** wolverineav has quit IRC01:49
*** Dinesh_Bhor has joined #openstack-nova01:49
*** sdake has joined #openstack-nova01:51
*** sdake has quit IRC01:51
*** wolverineav has joined #openstack-nova01:52
*** wolverineav has quit IRC01:56
*** tetsuro has joined #openstack-nova02:01
*** tetsuro_ has quit IRC02:01
openstackgerritMerged openstack/nova master: Plumbing for allowing the all-tenants filter with down cells  https://review.openstack.org/63514502:01
*** TxGirlGeek has joined #openstack-nova02:05
*** lbragstad has quit IRC02:06
*** macza has joined #openstack-nova02:22
*** macza has quit IRC02:27
*** macza has joined #openstack-nova02:27
*** hongbin has joined #openstack-nova02:29
*** macza has quit IRC02:31
*** markvoelker has joined #openstack-nova02:31
*** TxGirlGeek has quit IRC02:36
*** sapd1 has joined #openstack-nova02:47
*** markvoelker has quit IRC03:05
*** tetsuro has quit IRC03:22
*** tetsuro_ has joined #openstack-nova03:22
*** tetsuro has joined #openstack-nova03:26
*** tetsuro_ has quit IRC03:26
*** takashin has quit IRC03:28
*** sdake has joined #openstack-nova03:40
*** jmlowe has joined #openstack-nova03:43
openstackgerritArtom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods  https://review.openstack.org/63621003:48
openstackgerritArtom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination  https://review.openstack.org/63460503:48
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source  https://review.openstack.org/63482703:48
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim()  https://review.openstack.org/63566903:48
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources  https://review.openstack.org/63460603:48
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source  https://review.openstack.org/63482803:48
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source  https://review.openstack.org/63522903:48
openstackgerritArtom Lifshitz proposed openstack/nova master: Trivial: reorder hashes according to object_hashes.txt  https://review.openstack.org/63626103:48
artomActually I should probably put the trivial one at the very bottom...03:49
*** takashin has joined #openstack-nova03:49
openstackgerritArtom Lifshitz proposed openstack/nova master: Trivial: reorder hashes according to object_hashes.txt  https://review.openstack.org/63626103:50
openstackgerritArtom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods  https://review.openstack.org/63621003:50
openstackgerritArtom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination  https://review.openstack.org/63460503:50
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source  https://review.openstack.org/63482703:50
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim()  https://review.openstack.org/63566903:50
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources  https://review.openstack.org/63460603:50
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source  https://review.openstack.org/63482803:50
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source  https://review.openstack.org/63522903:50
*** TxGirlGeek has joined #openstack-nova03:58
*** dave-mccowan has quit IRC03:58
*** sapd1 has quit IRC03:59
*** sapd1 has joined #openstack-nova03:59
*** sdake has quit IRC03:59
*** markvoelker has joined #openstack-nova04:02
*** udesale has joined #openstack-nova04:02
*** tetsuro has quit IRC04:05
*** lbragstad has joined #openstack-nova04:09
*** gbarros has quit IRC04:28
*** janki has joined #openstack-nova04:30
*** markvoelker has quit IRC04:34
*** manjeets has quit IRC04:41
*** jmlowe has quit IRC04:50
*** ratailor has joined #openstack-nova05:08
*** hongbin has quit IRC05:30
*** markvoelker has joined #openstack-nova05:32
*** abhishekk has joined #openstack-nova05:33
*** udesale has quit IRC05:37
*** wolverineav has joined #openstack-nova05:55
*** TxGirlGeek has quit IRC05:56
*** markvoelker has quit IRC06:05
*** Dinesh_Bhor has quit IRC06:05
openstackgerritTakashi NATSUME proposed openstack/nova master: api-ref: Add descriptions for vol-backed snapshots  https://review.openstack.org/61508406:07
*** Dinesh_Bhor has joined #openstack-nova06:09
*** whoami-rajat has joined #openstack-nova06:19
*** sridharg has joined #openstack-nova06:20
*** wolverineav has quit IRC06:38
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix resetting non-persistent fields when saving obj  https://review.openstack.org/63627106:43
*** tiendc has quit IRC06:44
*** ccamacho has quit IRC06:49
*** cfriesen has quit IRC06:50
*** lbragstad has quit IRC06:50
*** Luzi has joined #openstack-nova06:53
*** abhishekk has quit IRC06:58
*** ralonsoh has joined #openstack-nova06:59
*** NewBruce has joined #openstack-nova07:01
*** markvoelker has joined #openstack-nova07:02
*** NewBruce has quit IRC07:03
*** udesale has joined #openstack-nova07:09
*** mdbooth_ has joined #openstack-nova07:11
*** moshele has joined #openstack-nova07:12
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix resetting non-persistent fields when saving obj  https://review.openstack.org/63627107:12
*** mdbooth has quit IRC07:14
*** abhishekk has joined #openstack-nova07:16
*** NewBruce has joined #openstack-nova07:23
*** markvoelker has quit IRC07:35
*** takashin has quit IRC07:35
*** takashin has joined #openstack-nova07:38
*** slaweq has joined #openstack-nova07:44
*** ccamacho has joined #openstack-nova07:58
*** ccamacho has quit IRC07:59
*** ccamacho has joined #openstack-nova07:59
*** awalende has joined #openstack-nova08:06
openstackgerritMerged openstack/nova master: doc: update the security groups admin doc  https://review.openstack.org/63473508:06
openstackgerritMerged openstack/nova master: doc: link admin/configuration from admin home page  https://review.openstack.org/63472808:07
*** yan0s has joined #openstack-nova08:08
*** tesseract has joined #openstack-nova08:09
*** awalende has quit IRC08:09
*** rpittau has joined #openstack-nova08:13
*** awalende has joined #openstack-nova08:20
*** xek_ has joined #openstack-nova08:28
*** tkajinam has quit IRC08:29
*** helenafm has joined #openstack-nova08:30
*** markvoelker has joined #openstack-nova08:32
openstackgerritMerged openstack/nova stable/queens: Fix InstanceNotFound during _destroy_evacuated_instances  https://review.openstack.org/62335508:42
*** takashin has left #openstack-nova08:52
*** whoami-rajat has quit IRC08:54
*** priteau has joined #openstack-nova08:59
*** tetsuro has joined #openstack-nova09:00
*** panda|off is now known as panda09:04
*** tetsuro has quit IRC09:05
*** markvoelker has quit IRC09:05
*** tetsuro has joined #openstack-nova09:05
*** jistr_ is now known as jistr09:09
*** tetsuro has quit IRC09:16
*** tssurya has joined #openstack-nova09:17
*** tetsuro has joined #openstack-nova09:17
*** cdent has joined #openstack-nova09:35
*** tetsuro has quit IRC09:39
openstackgerritBalazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding  https://review.openstack.org/56945909:46
openstackgerritBalazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request  https://review.openstack.org/57331709:46
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove port allocation during detach  https://review.openstack.org/62242109:46
openstackgerritBalazs Gibizer proposed openstack/nova master: Record requester in the InstancePCIRequest  https://review.openstack.org/62531009:46
openstackgerritBalazs Gibizer proposed openstack/nova master: Add pf_interface_name tag to passthrough_whitelist  https://review.openstack.org/62531109:46
openstackgerritBalazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF  https://review.openstack.org/62354309:46
*** NewBruce has quit IRC09:54
*** NewBruce has joined #openstack-nova09:59
*** stephenfin_ is now known as stephenfin10:01
*** markvoelker has joined #openstack-nova10:02
*** NewBruce has quit IRC10:03
lyarwoodbauzas: https://review.openstack.org/#/c/628726/ - would you mind hitting this stable backport today if you have a chance?10:05
bauzaslyarwood: yup, actively working on fixing some problems with the reshape change, but sure will look10:06
lyarwoodbauzas: ack thanks10:07
*** moshele has quit IRC10:08
*** belmoreira has joined #openstack-nova10:12
gibilyarwood: hi! If we are at stable backports, could you check https://review.openstack.org/#/c/635921/1 and https://review.openstack.org/#/c/623358/1 ?10:14
lyarwoodgibi: ack can do10:14
gibithanks!10:14
*** NewBruce has joined #openstack-nova10:19
bauzasgibi: around ?10:20
abhishekkhi all, any idea how can I print queries in logs?10:20
bauzasgibi: I'm puzzled with testing your functional tests again my new revision10:20
abhishekkI have tried to set default_log_levels = sqlalchemy=INFO but its not working10:20
gibibauzas: push your new revision up and I will check the functional test10:20
bauzasack10:20
bauzasit will fail your change tho10:21
openstackgerritLee Yarwood proposed openstack/nova master: compute: Reject migration requests when source is down  https://review.openstack.org/62348910:21
bauzasgibi: I'll just add a new change for correctly using the right mdev before tho10:21
gibibauzas: you mean it will fail the functional test? That is OK I will check that10:21
lyarwoodsean-k-mooney / stephenfin / mdbooth_ ^ I'd appreciate your thoughts on my bugfix above if you have time today10:21
lyarwoodoh and is anyone else having weird issues building releasenotes on f29?10:22
* stephenfin clicks10:23
stephenfinand tries building releasenotes to check10:23
* mdbooth_ is getting weirder issues on f2910:24
*** mdbooth_ is now known as mdbooth10:24
lyarwoodstephenfin: /src/openstack/nova/releasenotes/source/rocky.rst:3436:Duplicate explicit target name: "rest api version history". - is the error I'm getting locally10:24
*** bhagyashris has quit IRC10:25
*** derekh has joined #openstack-nova10:27
stephenfinlyarwood: That doesn't sound like something F29 specific. That would happen if someone had defined the same target in two release notes, which it looks like someone has10:27
stephenfinthough it's building just fine for me...10:28
lyarwoodstephenfin: kk there must be some cruft in my local repo somewhere10:28
stephenfin'git clean -xfd -e .tox' <-- I should have this aliased10:29
*** abhishekk has quit IRC10:30
*** Dinesh_Bhor has quit IRC10:32
*** markvoelker has quit IRC10:35
*** moshele has joined #openstack-nova10:39
*** purplerbot has joined #openstack-nova10:41
openstackgerritBoris Bobrov proposed openstack/nova master: Fix memtune parameters according to libvirt docs  https://review.openstack.org/63630110:42
*** NewBruce has quit IRC10:42
openstackgerritBoris Bobrov proposed openstack/nova master: Fix memtune parameters according to libvirt docs  https://review.openstack.org/63630110:43
*** sapd1 has quit IRC10:45
*** udesale has quit IRC10:49
*** NewBruce has joined #openstack-nova10:49
*** NewBruce has quit IRC10:54
*** NewBruce has joined #openstack-nova11:04
*** NewBruce has quit IRC11:06
*** erlon has joined #openstack-nova11:07
*** lpetrut has joined #openstack-nova11:15
*** NewBruce has joined #openstack-nova11:31
*** markvoelker has joined #openstack-nova11:32
*** tetsuro has joined #openstack-nova11:33
*** mvkr has quit IRC11:44
*** tetsuro has quit IRC11:46
*** NewBruce has quit IRC11:47
*** NewBruce has joined #openstack-nova11:50
*** erlon has quit IRC11:51
*** _fragatina has joined #openstack-nova11:56
*** _fragatina has quit IRC11:59
*** _fragatina has joined #openstack-nova12:00
openstackgerritZhenyu Zheng proposed openstack/nova master: Add method to allow reset fields for root bdm in BDM obj  https://review.openstack.org/61467212:00
*** markvoelker has quit IRC12:05
openstackgerritBoris Bobrov proposed openstack/nova master: Add configs for AMD SEV  https://review.openstack.org/63631812:10
*** kaisers has quit IRC12:24
*** mrch has joined #openstack-nova12:24
*** erlon has joined #openstack-nova12:29
*** tbachman has quit IRC12:29
*** mrch has quit IRC12:29
*** mrch has joined #openstack-nova12:30
*** kaisers has joined #openstack-nova12:31
*** rha has joined #openstack-nova12:32
*** panda is now known as panda|lunch12:32
*** ratailor has quit IRC12:34
*** coreycb has quit IRC12:36
*** coreycb has joined #openstack-nova12:37
*** lpetrut has quit IRC12:38
*** lpetrut has joined #openstack-nova12:39
*** mrch has quit IRC12:39
*** janki has quit IRC12:44
*** janki has joined #openstack-nova12:44
*** sapd1 has joined #openstack-nova12:44
openstackgerritJim Rollenhagen proposed openstack/nova master: Ironic: bump minimum API version to 1.38  https://review.openstack.org/63632612:45
jrollTheJulia: ^ sanity check me that this is cleanup we want to do, please12:45
*** mvkr has joined #openstack-nova12:46
*** _fragatina has quit IRC12:56
*** moshele has quit IRC13:00
*** markvoelker has joined #openstack-nova13:02
*** yan0s has quit IRC13:02
*** mriedem has joined #openstack-nova13:04
*** Dinesh_Bhor has joined #openstack-nova13:04
*** tbachman has joined #openstack-nova13:04
*** mvkr has quit IRC13:08
*** udesale has joined #openstack-nova13:10
*** panda|lunch is now known as panda13:13
mriedemtssurya: i think we might have some traces in the api logs that are saying a cell is down when it's really something else13:14
mriedemhttp://logs.openstack.org/67/620667/3/check/tempest-full/8fa0aa0/controller/logs/screen-n-api.txt.gz?level=TRACE#_Feb_12_00_25_30_61542313:14
mriedemalthough it looks like there are some MQ issues in those logs13:14
*** tbachman has quit IRC13:22
*** whoami-rajat has joined #openstack-nova13:22
*** tbachman has joined #openstack-nova13:23
*** agopi has joined #openstack-nova13:26
*** ociuhandu has joined #openstack-nova13:26
*** NewBruce has quit IRC13:29
*** NewBruce has joined #openstack-nova13:30
*** ade_lee_ has quit IRC13:30
*** sdake has joined #openstack-nova13:32
*** agopi has quit IRC13:34
*** markvoelker has quit IRC13:35
*** tbachman has quit IRC13:35
*** openstackgerrit has quit IRC13:37
lyarwoodmriedem: https://review.openstack.org/#/c/551302/ - has been rebased and should be ready to review again if you have anytime this week.13:37
mriedemlyarwood: yup in the queue13:38
mriedemdid you see the conversation in there about rollback also being busted?13:38
*** mvkr has joined #openstack-nova13:40
lyarwoodmriedem: so I think mdbooth was addressing that in a follow up here https://review.openstack.org/#/c/551349/213:40
lyarwoodmdbooth: ^ I can also rebase / refresh that if you'd like13:40
*** yedongcan has left #openstack-nova13:41
mdboothlyarwood mriedem: Yeah, although not having time to address that too is probably the biggest reason I haven't looked at this in ages.13:42
mdboothDefinitely a thing, though.13:42
*** openstackgerrit has joined #openstack-nova13:49
openstackgerritBalazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF  https://review.openstack.org/62354313:49
openstackgerritBoris Bobrov proposed openstack/nova master: Extract SEV-specific bits on host detection  https://review.openstack.org/63633413:57
*** mlavalle has joined #openstack-nova13:59
stephenfinsean-k-mooney: Thoughts? http://paste.openstack.org/show/B9unIL8e2KpeSMBGaINe/14:00
*** efoley has joined #openstack-nova14:01
stephenfinsean-k-mooney: I'm annoyed by the fact we have no central definition (for docs) or validation of flavour extra specs. Thinking it might be a good, easy spec for T (for the ones we're not deprecating with the cpu-resources spec14:02
*** agopi has joined #openstack-nova14:02
*** efoley has left #openstack-nova14:02
*** sdake has quit IRC14:03
bauzasstephenfin: we have documentation for flavor extra specs14:05
bauzasunless I misunderstand your concern14:05
sean-k-mooneystephenfin: we do it called the glance metadef registry14:06
bauzashttps://docs.openstack.org/nova/latest/user/flavors.html14:06
stephenfinbauzas: Yup, but it's not "in code" or verified against the code, unlike e.g. the nova.conf options14:06
bauzasahah14:06
bauzaswell14:07
bauzasAFAIK, we said meh14:07
gibibauzas: if you have time, can you push this two-patch backport? https://review.openstack.org/#/c/623358/114:07
sean-k-mooneystephenfin: https://github.com/openstack/glance/blob/master/etc/metadefs/compute-cpu-pinning.json14:07
bauzasgibi: looking14:07
sean-k-mooneythat is in code in a way14:07
gibibauzas: thanks14:07
bauzasgibi: oh that one14:07
bauzasgibi: silverlining, already reviewed it14:07
sean-k-mooneystephenfin: it is intended to be used for validating glance metadata entries and its used but horizon to generate teh ui with the dropdown vlaues and for validation14:08
sean-k-mooneyit also has defintions for the flavor extra specs too.14:09
mriedemtssurya: dansmith: i left comments in the down cell servers view builder change, the series is going to have to be rebased14:09
* mriedem goes to blow snow14:09
stephenfinsean-k-mooney: Aye, I knew about. If that was ever kept up-to-date though, I'm not sure if it still is. Barely any edits in years https://github.com/openstack/glance/tree/master/etc/metadefs14:10
*** mriedem is now known as mriedem_snow14:10
sean-k-mooneystephenfin: what might make sense it to extract it into a seperate repo that glance,nova and cinder can all import and share14:10
stephenfinI didn't know that it supported flavour extra specs though. That's good to know14:10
*** zul has joined #openstack-nova14:10
stephenfinsean-k-mooney: Aye, I think so. I'll have a look into it when I get a chance. Just been annoying me for some time now14:10
sean-k-mooneystephenfin: ya it was orginally created to keep the falvor extra specs and image metadata in sync14:11
sean-k-mooneyif we did extract it i dont see why we could not autogenerate the docs for it too14:11
*** NewBruce has quit IRC14:13
openstackgerritArtom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods  https://review.openstack.org/63621014:13
openstackgerritArtom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination  https://review.openstack.org/63460514:13
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source  https://review.openstack.org/63482714:13
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim()  https://review.openstack.org/63566914:13
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources  https://review.openstack.org/63460614:13
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source  https://review.openstack.org/63482814:13
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source  https://review.openstack.org/63522914:13
kashyapGosh, this serial / char devices code is making me go slowly mental...14:13
sean-k-mooneystephenfin: can we put it as a cross project topic with the glance and maybe cinder folks for the ptg14:13
stephenfinsean-k-mooney: Good idea, that14:14
sean-k-mooneystephenfin: it looks like it also cover nova host aggrage metadata too https://github.com/openstack/glance/blob/master/etc/metadefs/compute-aggr-num-instances.json14:15
bauzaslyarwood: I just agree with mriedem_snow on https://review.openstack.org/#/c/628726/414:15
bauzaslyarwood: we should make backports easy by providing a series14:15
*** liuyulong has joined #openstack-nova14:15
bauzasWDYT ?14:15
*** sapd1 has quit IRC14:15
lyarwoodbauzas: yeah ack sorry I totally forgot about that14:16
bauzassince you already identified the source of the conflicts, and you also want to backport the missing changes, I don't see any difficulty in making the backports intertwined14:16
bauzaslyarwood: ack, ping me when you're done then14:16
* bauzas goes back finding a creative way to not change a lot of things for a simple concern14:17
bauzasthat's what happens when you write poor interfaces14:17
*** Dinesh_Bhor has quit IRC14:18
*** NewBruce has joined #openstack-nova14:18
*** sdake has joined #openstack-nova14:20
NewBrucesean-k-mooney any chance i can get your thoughts on a bit of a bug that looks to be related to the port binding API changes?14:20
*** sdake has quit IRC14:21
sean-k-mooneyNewBruce: i saw you comments this moring14:25
NewBruceawesome, sorry for chasing14:25
NewBrucedone some more debugging now, and feels like i isolated the point it fails14:25
sean-k-mooneyno worries i just havent had a chance to reply was booking flights and stuff14:25
sean-k-mooneyso one general question14:26
NewBrucenp. hope your goin' somewhere warm14:26
*** priteau has quit IRC14:26
sean-k-mooneyPTG flights14:26
openstackgerritMerged openstack/nova stable/pike: Handle IndexError in _populate_neutron_binding_profile  https://review.openstack.org/63592114:26
NewBrucesure -14:26
sean-k-mooneyam you have a mixed cloud with OSA nodes and RDO nodes14:27
NewBrucecorrect14:27
sean-k-mooneyis the contol plan. e.g. your contolers all RDO or is it also mixed14:27
NewBrucecontrol plane is upgrade to Rocky/OSA14:27
NewBrucewe have multiple sites, with the same config which dont exhibit the problem14:28
sean-k-mooneyok so you have Rocky/OSA contol plane + OSA computes  + some rocky/RDO computes14:28
NewBrucecorrect14:28
NewBrucelooking in the setup today we have some RDO net nodes as well14:28
sean-k-mooneyand on the neutron side is it also a mix or all OSA for you neturon controler services14:29
NewBruceso they could do with being upgraded.... but in the debugging ive done today, it seems to bail out in the call to _setup_migration_port_profile14:29
dansmithmriedem_snow: okay I was thinking we should maybe try real hard to get that merged today14:29
NewBrucein a successful migration, the host_id passed to _setup_migration_port_profile is the target/dest compute; whereas in a failure the host_id passed to _setup_migration_port_profile is the source14:29
sean-k-mooneyhow the multiple portbinging code is ment to work is that the nova condocutor is first ment to check the neutron api to determin if neutron support it14:30
*** erlon has quit IRC14:30
openstackgerritLee Yarwood proposed openstack/nova stable/queens: Fail to live migration if instance has a NUMA topology  https://review.openstack.org/62959714:31
openstackgerritLee Yarwood proposed openstack/nova stable/queens: libvirt: Add workaround to cleanup instance dir when using rbd  https://review.openstack.org/62872614:31
*** ade_lee_ has joined #openstack-nova14:31
sean-k-mooneythe condoctor then check the version of both the souce and dest compute service and if they both support the multiple port binding workflow it is enabled14:31
NewBruceIf both compute nodes are queens or newer. In this case the new workflow14:31
NewBruce  will be used as described below.14:31
lyarwoodbauzas / stephenfin / mriedem_snow ^ reworked these into a series now on top of https://review.openstack.org/#/c/620111/, hopefully that's cleaner for everyone.14:32
*** markvoelker has joined #openstack-nova14:32
NewBruceso that check is based on the nova versions, yeah? which is fair enough, as when i monitor the ml2_port_bindings. i see the duplicate port entries in both successful and fail cases;14:32
NewBrucein the services table both source and dest are v3514:33
NewBrucefor nova-compute14:33
sean-k-mooneyNewBruce: thew new workflow should only take effect if the source and dest compute nodes are rocky14:33
NewBrucewhich they are14:33
*** lbragstad has joined #openstack-nova14:34
NewBrucesorry sean-k-mooney - gotta run for 10-15 will be back online14:34
sean-k-mooneysure no worries14:34
*** NewBruce has quit IRC14:34
zigoHi there!14:35
zigoI'm getting a weirdo bug inside Horizon which is related to Novaclient.14:36
openstackgerritVladyslav Drok proposed openstack/nova master: libvirt: Do not reraise DiskNotFound exceptions during resize  https://review.openstack.org/57141014:36
zigoWhen listing instances, Horizon first try to get the nova microversion.14:36
kaisersstephenfin: Hi! would you care to revisit https://review.openstack.org/#/c/522245/ ? Old driver bugfix change that has been sitting for some time and is not overly complex.14:36
zigoSo it tries to get https://api/compute/v2.114:36
*** sdake_ has joined #openstack-nova14:37
zigoBut nova sends as reply a redirect to https://api/compute/v2.1/ (note the lasting slash ...)14:37
stephenfinkaisers: Sure. I'd totally forgotten about that :) I'll take a look now14:37
kaisersstephenfin: greate, thnx14:37
zigoIt's probably a miss-configuration of haproxy, apache, or something else...14:37
zigoThough shouldn't nova-api reply with the microversion even without the lasting / ?14:37
zigobauzas: ^14:42
bauzasefried: gibi: I'm torn, given we only pass allocations down to the virt driver, how can the driver know whether the related RP is a root RP or a child ?14:42
bauzasit should then look at the ProviderTree, right?14:42
efriedbauzas: You can't, until we implement https://review.openstack.org/#/c/597601/14:43
efrieddoes spawn have access to the provider tree?14:43
gibibauzas: I think from the allocation alone you cannot figure out14:43
* bauzas whispers14:43
bauzasbecause then we have a problem with the reshape14:44
efriedThe reshape comes from update_provider_tree, which *does* have access to the provider tree.14:44
gibibauzas: do you really need that information? can you simply look at the allocation, select the RP that has VGPU resource allocated from14:44
bauzasgibi: how can I know this question ?14:44
bauzasgibi: efried: I mean, for the moment, we use a mdev when creating an instance by https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L627314:46
gibibauzas: you want to know which PGPU you are allocated the mdev from. You have the allocations dict. That dict has RPs with resources in it. The RP you need is the one that has VGPU resource allocated from14:46
bauzasgibi: efried so I'm changing the above to check with RP was allocated14:46
bauzasand then only either creating a new mdev or use an existing mdev from the pci device related to the RP14:47
stephenfinkaisers: Done. Feel free to ping me whenever I do forget stuff. I'll simply ignore you if I'm too busy :)14:47
bauzasgibi: I'm getting allocations, right?14:47
*** janki has quit IRC14:47
stephenfinlyarwood: Thanks for tackling that (y) It was on my TODO list14:47
* stephenfin closes one more tab14:48
bauzasgibi: so I guess I'm getting https://developer.openstack.org/api-ref/placement/?expanded=list-allocations-detail#response-example-1-2814:48
gibiyes14:48
bauzasso how can I get the RP ?14:48
bauzasI just have the RP UUID14:49
efriedthe rp is in there, but only its uuid, not its name.14:49
*** hongbin has joined #openstack-nova14:49
gibiok it is just the UUID14:49
gibiwhat else you would need? then name ?14:49
efriedI thought we made a convention for setting the RP UUID based on some aspect of the VGPU14:49
*** tbachman has joined #openstack-nova14:49
gibiwhat connects an RP and an mdev?14:49
efriedgibi: he needs to be able to associate the RP with the PGPU14:49
efriedwhich he can do from the name for sure, because we used some kind of naming convention.14:49
gibiefried: I see14:49
efriedbut with just the uuid, unless we named the rp somehow after the uuid of the pgpu, we're fooked.14:50
gibiefried: that is a problem as the provider summaries are not send down14:50
efriedIs the name in the provider summary?14:50
gibihm14:50
gibilooking14:50
efrieddoesn't look like it.14:50
TheJuliajroll: That looks awesome. Thanks!14:51
*** NewBruce has joined #openstack-nova14:51
*** awaugama has joined #openstack-nova14:51
*** udesale has quit IRC14:52
efriedbauzas: So unless we did in fact make some special way of assigning the RP UUID based on the identity of the pgpu, you're going to need to store that mapping locally somehow, which is ew.14:52
gibiefried: you are right14:52
bauzasefried: yeah it's a problem14:53
efriedyou can store an instance variable in the virt driver from update_provider_tree, with a big TODO on it to figure out a better way.14:53
bauzaswe only have a convention for the name, not the uuid14:54
efriedAnd I'm wrong, it has nothing to do with gibi's spec14:54
jrollTheJulia: cool, I'll un-wip. thanks14:54
efriedjroll: I'll bump my vote14:54
jrollefried: thanks14:54
bauzasefried: okay, I guess we have a big problem honestly14:54
bauzasbecause I don't want to do this like that14:55
bauzasif we want to select a node, then we need to know which one14:55
efriedum, maybe I'm wrong about being wrong. But it gives us something extra to think about in gibi's spec, which is that we have to have the information in the allocation itself, OR figure out some other way to get that info into spawn.14:55
gibiefried: yeah, it is not the spec you linked, but I had similar issue to map pci device to RP, there it is done by the RP name, and there is an extra call in the compute/manager to get the RP name14:55
efriedbut I think the libvirt problem is an extra one.14:55
efriedPerhaps we should consider amending the allocation structure to include the rp name.14:56
bauzasso, again, I'll say we're blocked14:56
bauzasunless the virt driver calls placement to know the RP tree14:56
efriedjaypipes: backscroll to :42:2114:56
cdenti'd be pretty opposed to include the name of the rp in the allocation unless we really can't come up with something14:56
efriedbauzas: Yes, that would be another "solution" that would be ew.14:56
cdentwhy _not_ pass the summaries?14:56
cdentor why not let the virt driver talk to placement?14:57
NewBrucesean-k-mooney : back14:57
cdentor any of several other options14:57
bauzasI'll upload my change anyway14:57
efriedcdent: not pass the summaries because a) way too much information, unless we pare it down to just the summaries in the allocation, and b) the name isn't in the summaries anyway :P14:57
*** awalende has quit IRC14:58
cdentb) right, but putting it in the summaries would make some form of sense: it's a provider _summary_ after all14:58
cdenthowever14:58
cdentusing meaningful names is a really super dangerous territory14:58
gibibauzas, efried: this is the PCI - RP mapping solution I come up with https://review.openstack.org/#/c/623543/23/nova/compute/manager.py It works as the pci claim can match devices by abitrary tag14:58
*** maciejjozefczyk has quit IRC14:58
efriedcdent: we need some way of mapping RP to "real thing".14:58
efriedI know: arbitrary metadata on resource_provider!14:58
cdentwhich isn't placements job14:58
*** awalende has joined #openstack-nova14:58
cdentmapping rp to real thing is the either nova or the virt drivers job14:59
efriedgibi: Right, so you're storing the mapping locally in memory.14:59
gibiefried: calculating the mapping by calling placement during boot14:59
gibiefried: nothing stored between boots14:59
efriedcdent: Which is what I suggested initially as the workaround. I though it was ew, but I'm kind of coming around to the idea.14:59
efriedyou don't need to store anything between boots; you construct the information in update_provider_tree in any case.15:00
efriedthat's where you would save the mapping.15:00
efriedyou can't spawn untile you've update_provider_treeD15:00
bauzasgibi: you're calling placement to get the RP name, right?15:00
gibibauzas: yes15:00
*** eharney has joined #openstack-nova15:00
efriedCould pass the provider_tree into spawn. That would let you look up the name.15:00
gibibauzas: https://review.openstack.org/#/c/623543/23/nova/compute/manager.py@213815:00
bauzasgibi: yeah that's what I saw15:01
bauzasit wasn't a question actually, more a clarification :)15:01
bauzasgibi: but here I'm working on the driver side15:01
bauzasI'm not sure I have the report client15:01
*** cfriesen has joined #openstack-nova15:01
efriedwould have to be a read-only/copy of the provider tree.15:01
gibibauzas: I know, you have to pass some extra info to the driver anyhow15:01
bauzasand I think we basically said no15:01
* jaypipes reads back...15:01
bauzasI mean, we agreed on not having the virt driver calling placement for various reasons15:02
efriedyeah15:02
openstackgerritBoris Bobrov proposed openstack/nova master: Add configs for AMD SEV  https://review.openstack.org/63631815:02
*** udesale has joined #openstack-nova15:02
*** sapd1 has joined #openstack-nova15:02
bauzasso, I'm stuck with finding a way to pass down the providertree structure down the wire15:02
*** awalende has quit IRC15:02
efriedbauzas: You have the provider tree in the compute manager.15:02
bauzasjust because I want to know which physical resource is there15:02
efriedno need to make calls to placement.15:03
kashyaptime git fetch gerrit15:03
bauzasefried: correct, I could amend https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L310615:03
*** moshele has joined #openstack-nova15:04
openstackgerritKashyap Chamarthy proposed openstack/nova master: [WIP] libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein"  https://review.openstack.org/63250715:04
efriedbauzas: Right. Add provider_tree. Get it the same way we get it before update_provider_tree (i.e. make a copy of it).15:04
bauzasthere could be race conditions, right?15:04
efriedNot that wouldn't cause blowups anyway.15:04
efriedE.g. if I remove the pgpu rp between allocating and spawning, I'm effed in any case.15:05
*** markvoelker has quit IRC15:05
gibiefried: but you cannot do that without removing the allocation as well, which breaks nova as that allocation is created by nova15:06
efriedgibi: Swhat I'm saying, I don't see how you could contrive a race condition that would make the provider_tree be out of sync with the allocations without already having broken something else in the process.15:07
gibiefried: I agree15:07
efriedI mean, some other unrelated provider could be out of sync I suppose. But I would hope that wouldn't matter.15:07
bauzasefried: to be honest, I don't care of allocations15:07
bauzasefried: I only care of the RP tree15:07
efriedwell, you need the allocation to determine the rp15:07
bauzasit's already there, and I trust it15:08
efriedAnd if you have more than one vgpu allocated, you need to know that.15:08
bauzasideally, it'd be waaaaay better and cleaner if I was getting the RP structure from placement as part of the message it returns me15:08
efriedI assume/hope you're getting that information from the allocation and not from somewhere else.15:08
bauzasbecause my cache could be stale15:08
efriedbauzas: But what do you mean by "structure"?15:09
efriedIt doesn't help you simply to know that the provider of the vgpu resource is a "child". You will eventually have to know *which* child.15:09
bauzasefried: the mdev allocation logic is made there https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L625115:09
bauzasefried: right15:09
*** hongbin has quit IRC15:09
bauzasefried: I want to ideally know the PCI device15:09
bauzasbut this mapping is under my responsibility15:09
efriedright15:10
bauzasso I need to know which RP is allocated15:10
bauzasand then map the PCI ID thanks to the convention15:10
efriedWell, you know which *RP* is allocated. But you don't know its name, only its UUID.15:10
bauzasright, that's what we're discussing since 15 mins15:10
bauzaswell, 20 even15:10
efriedRight, I think we've come full circle. If you had the provider_tree, you would be sorted. Otherwise you need some other way to maintain a mapping.15:11
efriedjaypipes: once you've caught up: Would you support adding a provider_tree arg to spawn()?15:11
bauzasI have one thought15:11
bauzaswe always push information down to the driver15:12
bauzascan we have the virt driver getting info from the service ?15:12
bauzasI mean a getter15:12
*** hongbin has joined #openstack-nova15:12
bauzasI don't like changing our interfaces (at least spawn()) for a very specific usecase15:12
bauzasand 99% of deployments won't care of VGPUs15:13
bauzasbut I'm not a virt driver specialist15:13
efriedyou mean introducing a protected static final ComputeDriver.get_provider_tree() method?15:14
bauzasI dunno, thinking out loud15:14
gibibauzas: for pci we have InstancePciRequest for NUMA we have InstanceNumaTopology for VGPU we have no object to hold this kind of information15:14
efriedhell, update_provider_tree could save off its provider tree as an instance var before it returns15:14
bauzassnap, I need to taxi my kids15:15
bauzasefried: that's called a cache15:15
efriedso it would be like the virt driver saving a mapping, except it doesn't need to do any special work to construct it - it's already there.15:15
bauzasefried: and we know it can be a problem15:15
bauzasI need to think about this15:15
bauzasanyway, I need to drop for a couple of hours15:16
bauzas\o15:16
jaypipesefried, bauzas, gibi, cdent: I have said for over a year that the virt driver will need to keep a mapping of the RP UUID to whatever name it uses to refer to that RP internally (mdev, PCI address, whatever the heck IBM does, etc)15:20
cdentyup15:20
cdentvirt driver's job15:21
openstackgerritKashyap Chamarthy proposed openstack/nova master: [WIP] libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein"  https://review.openstack.org/63250715:21
gibijaypipes: while I agree it is a bit more complicate if the RP is created outside of the virt driver like when  neutron and cyborg creates the RP but the virt driver consumes the RP15:22
gibiclearly for VGPU your suggestion works15:23
*** TxGirlGeek has joined #openstack-nova15:23
sean-k-mooneygibi: well in the cyboge/neutron case nova does not need to consomne the allocation15:24
sean-k-mooneyit just needs to pass it back to cyborg/neutorn so can they do the transaltion then15:25
gibisean-k-mooney: for sriov ports, the virt driver still responsible to select the PCI VF15:25
sean-k-mooneyyes this is true but currently they are not in placement15:26
gibisean-k-mooney: and the PCI VF should come from the same RP15:26
gibias the bandwidth15:26
sean-k-mooneythe same RP as the bandwith i am assuming15:26
sean-k-mooneyya15:26
gibiso transitively the virt driver needs to know which RP is used for the bandwidth15:26
*** liuyulong has quit IRC15:26
sean-k-mooneywell the bandwith woudl be an inventor on teh PF RP and we soudl similar have an inventory of VFs15:27
sean-k-mooneyso as long as the VF resust and bandwith request are in teh same requst group we shoudl be ok the issue is that would require 2 services to manage different inventoies in the same RP15:28
sean-k-mooneywhich we dont support today15:28
*** moshele has quit IRC15:28
gibisean-k-mooney: yeah, it is solved when VF inventory is moved to the placement15:28
sean-k-mooneyi personally had assumed that the resouce track would evolve to sotre the placement RP mappings15:29
efriedjaypipes: So that mapping has to be created in update_provider_tree (and could in fact *be* the provider tree created therein) and retrievable from spawn.15:29
*** gbarros has joined #openstack-nova15:30
efriedIf we state it that way ^, then I guess we can leave it up to each virt driver how it wants to create and store the mapping. Shoving the provider tree into an instance var seems like as easy a way as any.15:30
*** priteau has joined #openstack-nova15:31
*** sapd1 has quit IRC15:32
jaypipesefried: it could, yes. though there's nothing in what I said that *requires* it be that way. After all, the PCI device management code (gawd save us all), external Neutron agents (per gibi's series) and other things could easily keep a map of UUID -> internal name in whatever way they wanted. Shit, I believe I proposed at some point that a simple YAML file on disk storing the map of UUID -> internal device/provider name would be just fine15:33
openstackgerritRalf Haferkamp proposed openstack/nova master: compute: reset instance events on reload  https://review.openstack.org/42002615:34
*** NewBruce has quit IRC15:39
*** gbarros has quit IRC15:40
*** liuyulong has joined #openstack-nova15:41
*** Luzi has quit IRC15:41
*** macza has joined #openstack-nova15:42
sean-k-mooneyjaypipes: the other option which im not a huge fan of would be just to set the service specific internal name in the RP name field15:43
openstackgerritBalazs Gibizer proposed openstack/nova master: Support server create with ports having resource request  https://review.openstack.org/63636015:44
sean-k-mooneyjaypipes: that does not really work well in the general case and dose not work at all in the inter service case hence why im not a huge fan of that.15:44
*** maciejjozefczyk has joined #openstack-nova15:44
kashyapmdbooth: If you get some time, I'd really appreciate some libvirt driver and sensible eyes in general on this: https://review.openstack.org/#/c/632507/15:45
kashyapmdbooth: I know it not fully complete, but the serial / char dev code is tripping me up a bit.15:45
kashyap(Funnily enough, for me, the bug that is referenced in the code that's being deleted ... was a bug I filed 5 years ago)15:46
sean-k-mooneyin the sriov case if the RP name was the PF adress neutron and nova would agree on it at least15:46
sean-k-mooneybut really that is just "bit stealing" rather and  a convention rather then a dedicated mechanium15:46
*** moshele has joined #openstack-nova15:47
jaypipessean-k-mooney: that's basically what gibi's doing in https://review.openstack.org/#/c/623543/23/nova/compute/manager.py@213815:48
efriedsean-k-mooney: We already do that in the vgpu code too. It doesn't help, because the allocation doesn't include the rp name.15:48
sean-k-mooney i mean it will work as long as the service that need to share an RP agree on on how to chose the name15:48
jaypipesthe issue is that I see that as a fragile solution that depends on the name structure being codified and not changing ever. as soon as the device naming convention changes, everything would blow up.15:49
cdent++15:49
sean-k-mooneyefried: well you could lookup the rp by its uuid if you need the info form the allocation15:49
efriedshrug, be like saying if the mapping algorithm changes everything blows up15:49
efriedsean-k-mooney: This is where we started.15:49
efriedsean-k-mooney: Where do you look up the rp?15:49
sean-k-mooneyin placement15:49
efriedin placement? no, because we've said we don't want virt talking directly to placement15:49
efriedin the provider tree? Sure, but you have to get the provider tree somehow15:50
jaypipesproviders should always be *externally referenced* (i.e. any time two services need to exchange info about a provider) using a UUID. internally, within a service, the service should be responsible for caching an internal name -> UUID map.15:50
efriedin a mapping? Sure, but the mapping needs to be saved by the virt driver15:50
sean-k-mooneyif the virt dirver create the RP with a specific name in know it could take the uuid for the allcoation and look it up and get the name back15:50
efriedby talking to placement? No, see above. By looking in a mapping? Sure, see above.15:50
sean-k-mooneyefried: it only need to be saved for efficency.15:51
efriedsean-k-mooney: The proposed solution is for update_provider_tree to save a mapping of RP UUID to <something> and then for spawn to retrieve that information. The easiest thing would be for update_provider_tree simply to save the provider_tree as an instance var on the virt driver. Then spawn can just ask that guy for the provider with a given uuid and get the name. Since the name is already generated by convention, it can use it15:52
*** mriedem_snow is now known as mriedem15:52
*** erlon has joined #openstack-nova15:52
sean-k-mooneyefried: sure that makes sense15:53
efriedif we really don't like using the naming convention, upt could save some other arbitrarily-formatted dict as an instance var, to the same end.15:53
sean-k-mooneybut making that internal mapping to the uuid stable it the respociblity of the service consuming placment in that case not placement15:53
efriedyes, update_provider_tree is that thing in either case.15:54
efriedeven if spawn got to request the provider record from placement and use the naming convention, it was still upt's responsibility to name the provider appropriately.15:54
sean-k-mooneyya15:55
efriedbut I prefer the idea of upt caching^wahem, saving something as an instance var.15:55
sean-k-mooneyisnt this partly solved by https://review.openstack.org/#/c/612497/ also15:55
sean-k-mooneythe provider ymal file15:56
efriedsean-k-mooney: It could be, but I kind of hate the idea of making it the operator's responsibility15:56
efriedas long as there's reasonable default behavior if that file is missing (or composed poorly) it would be okay.15:56
sean-k-mooneywell there is no reason the virtdriver could not generate it on first boot if its not provided15:56
sean-k-mooneyand then operator could add to it15:57
efriedalso ew.15:57
sean-k-mooneythat said that might mess with som config managment stuff15:57
efriedNow what if I hot-(un)plug devices or otherwise change resources.15:57
efriedyeah15:57
efriednow I'm forced to hand-edit the file after that first time. Or remember to blow it away so it can be regenerated, but then I lose my customizations.15:57
sean-k-mooneyyou first update teh yaml file then send a sig hup to the compute agent then you remove the hardware15:58
efriedas opposed to just... removing the hardware and everything works.15:58
sean-k-mooneythat is not really a thing today15:59
efriedneither is generic numa affinity15:59
sean-k-mooneyi mean i "might" work in some cases15:59
efriedSeems like all of this is stuff we're striving for.15:59
*** mrch_ has joined #openstack-nova16:00
efriedThe whole reason update_provider_tree happens on a loop is so that it can, in the future, automatically adjust the provider tree when stuff like this happens.16:00
sean-k-mooneyi think https://etherpad.openstack.org/p/nova-ptg-train everyting from lin 7-30 is very related16:00
*** maciejjozefczyk has quit IRC16:00
*** erlon has quit IRC16:00
*** mrch_ has quit IRC16:01
*** mrch_ has joined #openstack-nova16:01
*** mrch_ has quit IRC16:01
*** mrch_ has joined #openstack-nova16:01
efriedIn fact, I think that it might "just work" after Sylvain's vgpu stuff lands to hot-(un)plug a GPU.16:01
efriedif you unplug one that has allocations against it, you can expect problems; but otherwise...16:01
efriedand even then, I think we've accounted for that by setting reserved=total.16:02
*** mrch_ has quit IRC16:02
*** NewBruce has joined #openstack-nova16:02
*** mrch_ has joined #openstack-nova16:02
sean-k-mooneywell there were edgecases like this https://review.openstack.org/#/c/626381/16:02
*** markvoelker has joined #openstack-nova16:03
sean-k-mooneyif the device you remove had allocation against it we cant remove it form placemetn until those allocation are also removed16:03
*** NewBruce has quit IRC16:03
sean-k-mooneyso the auto updating is greate but there are also dragons lurking there16:03
*** lpetrut has quit IRC16:03
*** NewBruce has joined #openstack-nova16:04
efriedright, like I said, that case is accounted for (at least I definitely remember this code in the xen vgpu patches) where, if a card goes away, but has allocations against it, we set reserved=total so no futher allocations can be made against it, until all the existing allocations go away, at which point we can delete the provider.16:05
sean-k-mooneyya so we have solution for this edgecase and likely other but its a little more involved then you would assume at first glance16:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Calculate RequestGroup resource provider mapping  https://review.openstack.org/61623916:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Fill the RequestGroup mapping during schedule  https://review.openstack.org/61952816:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Pass resource provider mapping to neutronv2 api  https://review.openstack.org/61624016:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Recalculate request group - RP mapping during re-schedule  https://review.openstack.org/61952916:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding  https://review.openstack.org/56945916:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request  https://review.openstack.org/57331716:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove port allocation during detach  https://review.openstack.org/62242116:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Record requester in the InstancePCIRequest  https://review.openstack.org/62531016:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Add pf_interface_name tag to passthrough_whitelist  https://review.openstack.org/62531116:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF  https://review.openstack.org/62354316:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Support server create with ports having resource request  https://review.openstack.org/63636016:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Support server create with ports having resource request  https://review.openstack.org/63636016:07
*** wwriverrat has quit IRC16:07
mriedemlyarwood: mdbooth: +2 on https://review.openstack.org/#/c/551302/16:09
mriedemdansmith: ^16:09
mdboothmriedem: Thanks!16:11
*** NewBruce has quit IRC16:13
*** wwriverrat has joined #openstack-nova16:14
*** dpawlik has joined #openstack-nova16:14
lyarwoodmriedem: cheers16:15
*** moshele has quit IRC16:15
openstackgerritBalazs Gibizer proposed openstack/nova master: Support server create with ports having resource request  https://review.openstack.org/63636016:16
efriedaspiers, mriedem: Were either of you planning to resurrect https://review.openstack.org/#/c/538498/ ?16:17
sean-k-mooneymriedem: when you have a sec can you let me know if the responces in https://review.openstack.org/#/c/624842/8//COMMIT_MSG make sense. if you are happy with them i can resping with that info added to the commit and your other comments addressed in an hour or so16:17
*** wwriverrat has quit IRC16:18
mriedemefried: i'm not working on it16:18
*** wwriverrat has joined #openstack-nova16:18
stephenfinefried: Comments on https://review.openstack.org/#/c/582392/16:19
efried...16:19
mriedemsean-k-mooney: i guess....16:19
mriedemsetting the vif mac to 0s works depending on the underlying driver16:20
mriedemis the summary i'm getting16:20
sean-k-mooneyyes16:20
mriedemi'm not sure how the minimum libvirt version plays into any of that16:20
*** mrch_ has quit IRC16:20
sean-k-mooneyoh with libvirt 3.2 it takes care of all of this16:20
mriedemset_vf_interface_vlan doesn't document it's expected exit codes at all either so i had no idea16:20
sean-k-mooneyso we can delete this code16:20
*** ccamacho has quit IRC16:23
aspiersefried: I started to rebase it and got confused by the conflicts through lack of familiarity with the code, so I installed git-irebase and started to incrementally rebase it one commit at a time, so that the conflicts are smaller and there's more chance of me understanding how to handle them correctly16:23
*** ccamacho has joined #openstack-nova16:24
efriedaspiers: tldr you're working on it?16:24
aspierswell, trying ;-)16:24
aspiersI reserve the right to give up through incompetence though16:24
sean-k-mooneymriedem: the behavior chagne 2 weeks ago in https://github.com/openstack/nova/commit/13e283cc82e7cab564850501f9781385fb7b3f68 ill have to respin anyway now.16:25
aspiersefried, mriedem: however I've just taken a brief pause to review the SEV patches my colleague just posted16:25
efriedaspiers: Just lmk if you do decide to give up on it, and I'll take over. I want it to get finished one way or another. (No particular hurry, just don't want it stalled indefinitely.)16:25
mriedemsean-k-mooney: behavior change?16:25
aspiersefried: sure thing16:26
efriedand thank you.16:26
aspiersyw16:26
mriedemefried: have you taken a gander at https://review.openstack.org/#/c/616239/ yet? that's gibi's "Calculate RequestGroup resource provider mapping" change which i figure you're the best person to review that16:27
sean-k-mooneymriedem: actully no never mind the   exit_code = [0, 2, 254] is now donw internally in another fucntion instead of pased in16:27
*** manjeets has joined #openstack-nova16:27
sean-k-mooneynever mind i dont need to respin due to a behavior change but i will to adress you comments16:27
efriedmriedem: It's on my list, but I admit my list is currently more of a... cloud. I can bump it.16:27
mriedemsame16:28
dansmithmriedem: lyarwood mdbooth: Tell me why we have to do the partial() and I'll remove my -1, otherwise that seems nuts to me16:28
mriedemb/c the virt driver calls the post method16:29
mriedemand doesn't have the source bdms16:29
mriedemit's a callback16:29
dansmithmriedem: right, but we can modify that interface16:30
mriedemand update all virt drivers to lookup and pass the bdms?16:30
dansmitheff yeah, otherwise you're just stacking partial()s on top of each other until the end of time16:30
openstackgerritEric Fried proposed openstack/nova master: Use tox 3.1.1 fixes  https://review.openstack.org/58239216:30
openstackgerritEric Fried proposed openstack/nova master: tox: Don't write byte code (maybe)  https://review.openstack.org/63637816:30
dansmithand nobody actually knows how/where all those things come from16:31
mriedemdansmith: actually it's already too late16:31
mriedemonce the virt driver calls the post method,16:31
mriedemthe bdms have been modified via pre_live_migration on the dest16:31
mriedemwhich is why he's looking them up before call pre_live_migration16:31
dansmithmriedem: too late for stacking the partials you mean right?16:31
efriedstephenfin: fixed, good suggestion.16:31
mriedemtoo late for the virt driver on the source host to lookup the bdms to pass to _post_live_migration because that happens *after* pre_live_migration has modified the bdms16:32
dansmithmriedem: all I'm saying is pass these things to the method you want to have pass them back, instead of wrapping16:32
dansmithmriedem: I'm not saying have the virt driver look them up16:32
*** mrch_ has joined #openstack-nova16:32
efriedmriedem: don't forget to send a courtesy note to the ML so OOT drivers aren't blindsided :*16:32
mriedemso pass through the driver.live_migration method16:32
dansmithmriedem: right16:33
mriedemi don't feel strongly either way, but this has to be backported and as noted that interface change will require all drivers to be updated lockstep16:33
*** tbachman has quit IRC16:33
* lyarwood reads back16:34
mriedemso if we were going to change the interface, i'd do it in a follow up which isn't backported16:34
dansmithmriedem: well, then backport this and fix it after if you want16:34
lyarwoodack, I can take that if mdbooth can't16:34
dansmithI would also define the partial  right before you make the call to the driver,16:34
mdboothdansmith: TBH, I actually like the partial better, and in this context I'd partial all the things.16:34
dansmithmdbooth: I have no doubt that you do16:35
*** markvoelker has quit IRC16:35
mriedemrollback still has to be fixed as well16:35
mriedemfor the same issue16:35
mdboothdansmith: The reason is that is simplifies the interface between compute and the driver by removing arguments which are purely local the the calling method.16:35
mdboothSo the callback method only passes relevant data, and everything else remains local.16:35
mdboothHowever, that isn't why I used one here. Here's it's just so I didn't have to change the interface.16:36
dansmithmdbooth: yeah, I get the obvious argument for it, I just think it hurts readability and understandability if you're just focused on that one method16:36
dansmithit'd be slightly better if you define the partial and all the extra things you're going to pass down right before you make the call to the driver,16:36
openstackgerritMerged openstack/nova stable/pike: Add functional regression test for bug 1794996  https://review.openstack.org/62335816:36
openstackbug 1794996 in OpenStack Compute (nova) pike "_destroy_evacuated_instances fails and kills n-cpu startup if lazy-loading flavor on a deleted instance" [High,In progress] https://launchpad.net/bugs/1794996 - Assigned to Matt Riedemann (mriedem)16:36
openstackgerritMerged openstack/nova stable/pike: Fix InstanceNotFound during _destroy_evacuated_instances  https://review.openstack.org/62335916:37
openstackgerritMerged openstack/nova master: Trivial: reorder hashes according to object_hashes.txt  https://review.openstack.org/63626116:37
dansmithbut if I were to add something else in a year, we could easily end up with a partial(partial(thing)) because someone didn't read 50 lines up16:37
*** udesale has quit IRC16:37
dansmithusing it to make a backport cleaner is exactly the hacky situation where I think the benefit outweighs the cost, but going forward in master, it's just a confusing hack, IMHO16:38
*** helenafm has quit IRC16:40
*** mrch_ has quit IRC16:40
*** jmlowe has joined #openstack-nova16:43
*** mvkr has quit IRC16:44
*** tbachman has joined #openstack-nova16:45
*** moshele has joined #openstack-nova16:51
*** ccamacho has quit IRC16:53
*** moshele has quit IRC16:54
*** TxGirlGeek has quit IRC16:57
*** sridharg has quit IRC16:58
*** tbachman has quit IRC16:58
openstackgerritStephen Finucane proposed openstack/nova master: Don't set bandwidth limits for vhostuser, hostdev interfaces  https://review.openstack.org/63517016:58
openstackgerritStephen Finucane proposed openstack/nova master: Remove get_config_vhostuser  https://review.openstack.org/56547116:58
openstackgerritStephen Finucane proposed openstack/nova master: Validate bandwidth configuration for other VIF types  https://review.openstack.org/63638316:58
openstackgerritStephen Finucane proposed openstack/nova master: Further de-dupe os-vif VIF tests  https://review.openstack.org/63638416:58
stephenfinjangutter, sean-k-mooney: Per our discussion on IRC last week ^16:58
lyarwooddansmith: so just to confirm, you'd be okay with https://review.openstack.org/#/c/551302/ as a backportable hack if the partial was moved down and we followed up with something different in master?16:59
dansmithlyarwood: yeah16:59
*** hongbin has quit IRC16:59
lyarwooddansmith: kk, I'll move that down now then, thanks17:00
*** sdake_ has quit IRC17:01
*** tbachman has joined #openstack-nova17:01
stephenfinmriedem, alex_xu: Any chance you could take a look at this again as some point this week? https://review.openstack.org/#/c/631038/17:03
*** sdake has joined #openstack-nova17:03
*** sapd1 has joined #openstack-nova17:04
openstackgerritLee Yarwood proposed openstack/nova master: Avoid redundant initialize_connection on source post live migration  https://review.openstack.org/55130217:04
*** prometheanfire has joined #openstack-nova17:05
lyarwooddansmith: ^ there we go, thanks again17:05
prometheanfiredoes nova require pci_passthough_whitelist to be in the [pci] section in rocky or can it remain in [DEFAULT]?17:06
*** hamzy has quit IRC17:06
*** rpittau has quit IRC17:08
*** tesseract has quit IRC17:09
*** liuyulong has quit IRC17:09
stephenfinprometheanfire: It will be read from DEFAULT but it should be moved as we'll eventually remove the aliases17:10
stephenfinprometheanfire: You'll get a warning. Also, the option in the '[pci]' group is simply called 'passthrough_whitelist', in case you'd missed that17:10
prometheanfireya17:11
prometheanfireI'm working on a os_nova backport to stable/rocky, stein is already updated17:11
sean-k-mooneyit was technically moved in queens i think17:11
prometheanfireya, docs show it in pike too17:11
prometheanfirehttps://docs.openstack.org/nova/pike/admin/pci-passthrough.html17:11
*** moshele has joined #openstack-nova17:11
sean-k-mooneyprometheanfire: oh it was move in ocata17:12
prometheanfireno one uses that :P17:12
sean-k-mooneyyou would be surprised how many people are on Newton still17:13
sean-k-mooneythere are still people on liberty but yes few people use ocata17:13
sean-k-mooneyits EOL upstream anyway17:13
*** sapd1 has quit IRC17:14
prometheanfireit was a joke :P17:14
sean-k-mooneybut anyway https://github.com/openstack/nova/blob/master/nova/conf/pci.py#L8417:14
sean-k-mooneywe still support both but we really shoudl drop support for [DEFAULT] at this point17:15
prometheanfireya17:15
sean-k-mooneyprometheanfire: if you are moving it yyou shoudl also move the pci alias too17:16
*** mvkr has joined #openstack-nova17:16
sean-k-mooneyhttps://github.com/openstack/nova/blob/master/nova/conf/pci.py#L2317:17
prometheanfirealias wasn't even defined/used until stein (I'm adding it in the backports)17:18
sean-k-mooneythey predate kolla by years17:19
sean-k-mooneyyou could use them via the config override mechanisium17:19
sean-k-mooneythe main issue was you could only have 1 alsi tuse to how the config_merger ansible plugin worked17:20
sean-k-mooneyit does not support MultiStrOpt fields17:20
sean-k-mooneyunlike the whitelist the alais does not support a json list format17:21
*** jmlowe has quit IRC17:21
*** _pewp_ has quit IRC17:27
*** _pewp_ has joined #openstack-nova17:28
prometheanfiredoes nova support out of tree drivers? (xenxl)17:30
*** markvoelker has joined #openstack-nova17:32
*** Shilpa has joined #openstack-nova17:32
melwitto/17:33
sean-k-mooneyprometheanfire: nova does nto support out of tree drivers in general no17:33
*** gryf has quit IRC17:34
*** ShilpaSD has quit IRC17:35
*** whoami-rajat has quit IRC17:36
*** moshele has quit IRC17:36
prometheanfiresean-k-mooney: ok, just curious17:36
*** spsurya has quit IRC17:40
*** idlemind has joined #openstack-nova17:42
stephenfinsean-k-mooney, adrianc: Finished SR-IOV series review and left some comments. Ping me if/when there's more to look at.17:44
* stephenfin heads home17:44
*** gbarros has joined #openstack-nova17:44
sean-k-mooneystephenfin: cool i will try to adress them later today17:44
*** sdake has quit IRC17:44
*** mriedem is now known as mriedem_afk17:45
sean-k-mooneystephenfin: it can wait till tomorow but we need to review https://review.openstack.org/#/c/636061/ tool the ironic and neutron spec were approved after m2 so apparently this is intending to target stien.17:47
*** sdake has joined #openstack-nova17:47
*** gyee has joined #openstack-nova17:47
sean-k-mooneystephenfin: so 1 we need to validate it makes sense and 2 merge it and releas a new os-vif version before teh 27th. i was hoping to do a new release on friday week anyway but i would have prefered to at least have a nova/os-vif blueprint for this earlier17:48
prometheanfiresean-k-mooney: they don't think they'd be able to support the driver fully upstream is the reason I asked17:48
sean-k-mooneyprometheanfire: so technically you can use out of tree drivers but they are not offically supported by nova (since they are out of tree) and nova may break them at any time17:49
sean-k-mooneyhistorically i dont know how volitile that interface has been17:50
openstackgerritSurya Seetharaman proposed openstack/nova master: Plumbing required in servers ViewBuilder to construct partial results  https://review.openstack.org/63514617:50
openstackgerritSurya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells  https://review.openstack.org/59165717:50
openstackgerritSurya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells Documentation  https://review.openstack.org/63514717:50
sean-k-mooneyprometheanfire: so there is noting preventign kolla from supproting deploying them but they may break17:50
sean-k-mooneyprometheanfire: being out of tree means that maintaining comptablity with nova fall entirly onto the xenxl team. but yes being intree requires ci and other testing requirement so its non trivial17:52
*** dpawlik has quit IRC17:53
prometheanfiresean-k-mooney: not asking for osa (or kolla), that comes as the cost of out of tree :D17:54
sean-k-mooneywell being out of tree has benifits too17:54
sean-k-mooneyif the hypervior you are intergrating with is evolving quickly you can add features and adapt your driver much faster out of tree17:55
sean-k-mooneyso it best to bring it in tree when it stable and you have hit the baseline of feature you want o support17:55
*** tbachman has quit IRC17:56
*** derekh has quit IRC17:56
prometheanfireyep17:56
*** dtantsur is now known as dtantsur|afk17:58
odyssey4meprometheanfire OSA has implementations of out of tree drivers for nova already - for powervm IIRC17:58
prometheanfireodyssey4me: good point I'll point him there17:59
odyssey4menova supports the use of out of tree drivers, but obviously the core team for nova does not 'support' those drivers - that'd be for the out of tree driver team17:59
odyssey4meOSA also has nova-lxd driver support17:59
*** artom has quit IRC18:04
*** markvoelker has quit IRC18:06
*** dpawlik has joined #openstack-nova18:06
*** agopi is now known as agopi|food18:06
*** panda is now known as panda|off18:06
*** wwriverrat has quit IRC18:09
*** tssurya has quit IRC18:09
openstackgerritPaul-Emile Element proposed openstack/nova master: Add emulated TPM support to Nova  https://review.openstack.org/63136318:09
*** wwriverrat has joined #openstack-nova18:10
*** dpawlik has quit IRC18:10
*** tbachman has joined #openstack-nova18:13
prometheanfirethe pci_whitelist can accept a value like passthrough_whitelist= "{ "vendor_id": "10de", "product_id": "15f7" }"   ?18:17
prometheanfireor does it need to look like { "vendor_id": "10de", "product_id": "15f7" }18:17
*** dpawlik has joined #openstack-nova18:23
sean-k-mooneythe later i think18:24
*** imacdonn has quit IRC18:24
sean-k-mooneyprometheanfire: there a a bunch of examples here https://github.com/openstack/nova/blob/master/nova/conf/pci.py#L128-L16218:25
*** imacdonn has joined #openstack-nova18:25
openstackgerritMatt Riedemann proposed openstack/nova master: Move finish_resize.(start|end) notifications to helper method  https://review.openstack.org/63507918:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid  https://review.openstack.org/61401218:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field  https://review.openstack.org/61403518:28
mriedem_afkdansmith: there will be some easy pickings at the bottom of this series ^18:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method  https://review.openstack.org/61403618:28
openstackgerritMatt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells  https://review.openstack.org/61403718:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add CrossCellWeigher  https://review.openstack.org/61435318:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add Instance.hidden field  https://review.openstack.org/63112318:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add TargetDBSetupTask  https://review.openstack.org/62789218:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask  https://review.openstack.org/63158118:28
openstackgerritMatt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask  https://review.openstack.org/63385318:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method  https://review.openstack.org/62131318:28
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add prep_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63329318:28
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtDestTask  https://review.openstack.org/62789018:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method  https://review.openstack.org/63483218:28
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtSourceTask  https://review.openstack.org/62789118:28
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add finish_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63508018:28
openstackgerritMatt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask  https://review.openstack.org/63564618:28
openstackgerritMatt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask  https://review.openstack.org/63566818:28
openstackgerritMerged openstack/nova master: Follow up (#2) for the bw resource provider series  https://review.openstack.org/63476718:30
*** agopi|food is now known as agopi18:30
*** eharney has quit IRC18:33
*** panda|off has quit IRC18:34
*** gryf has joined #openstack-nova18:35
edleafemelwitt: At Doug Hellman's suggestion, I went ahead and created https://review.openstack.org/#/c/636416/18:37
*** ralonsoh has quit IRC18:37
edleafemelwitt: I was going to wait until we discussed it at Thursday's team meeting18:37
*** gcerami_ has joined #openstack-nova18:37
melwittedleafe: ack, thanks18:39
*** ociuhandu_ has joined #openstack-nova18:43
*** hamzy has joined #openstack-nova18:43
*** sdake has quit IRC18:44
*** wolverineav has joined #openstack-nova18:45
*** ociuhandu has quit IRC18:47
*** priteau has quit IRC18:47
*** ociuhandu_ has quit IRC18:48
*** sdake has joined #openstack-nova18:51
*** _fragatina has joined #openstack-nova18:55
*** ociuhandu has joined #openstack-nova18:55
*** krasmussen has joined #openstack-nova18:58
*** slaweq has quit IRC18:59
*** ociuhandu has quit IRC18:59
*** ociuhandu has joined #openstack-nova19:00
*** wolverineav has quit IRC19:00
*** markvoelker has joined #openstack-nova19:03
*** wolverineav has joined #openstack-nova19:04
*** ociuhandu has quit IRC19:04
*** dtantsur|afk has quit IRC19:04
*** alex_xu has quit IRC19:04
*** stephenfin has quit IRC19:04
*** jroll has quit IRC19:04
*** yankcrime has quit IRC19:04
*** mugsie has quit IRC19:04
*** belmoreira has quit IRC19:04
*** zul has quit IRC19:04
*** tinwood has quit IRC19:04
*** dr_gogeta86 has quit IRC19:04
*** tomtom001 has quit IRC19:04
*** med_ has quit IRC19:04
*** tinwood has joined #openstack-nova19:04
*** mugsie has joined #openstack-nova19:05
*** dr_gogeta86 has joined #openstack-nova19:05
*** stephenfin has joined #openstack-nova19:06
*** jroll has joined #openstack-nova19:06
*** alex_xu has joined #openstack-nova19:06
*** ociuhandu has joined #openstack-nova19:06
*** dtantsur has joined #openstack-nova19:08
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein"  https://review.openstack.org/63250719:13
kashyapOnly 16 more tests to fix!  (Down from 34.)19:14
*** gcerami_ is now known as panda19:14
*** yankcrime has joined #openstack-nova19:14
kashyapmriedem_afk: ^ When you get a moment: I think it makes sense to get the above in in Stein (even at the end of the release).  IMHO, it just gets rid of the needless cruft19:15
kashyapThe unit test "noise" is largely mechanical (as I noted in the commit message): remove the needless extra serial device.19:16
kashyapI'll fix the rest of the 16 tests in the morning.19:17
*** wwriverrat has left #openstack-nova19:18
*** artom has joined #openstack-nova19:34
*** artom has quit IRC19:35
*** artom has joined #openstack-nova19:35
*** markvoelker has quit IRC19:36
*** sdake has quit IRC19:42
*** wolverineav has quit IRC19:43
*** wolverineav has joined #openstack-nova19:44
*** wolverineav has quit IRC19:46
*** wolverineav has joined #openstack-nova19:47
*** eharney has joined #openstack-nova19:48
*** wolverineav has quit IRC19:49
*** wolverineav has joined #openstack-nova19:50
*** wolverineav has quit IRC19:55
*** wolverineav has joined #openstack-nova19:57
*** mriedem_afk is now known as mriedem20:05
*** dpawlik has quit IRC20:08
*** moshele has joined #openstack-nova20:14
*** slaweq has joined #openstack-nova20:19
*** wolverineav has quit IRC20:20
*** wolverineav has joined #openstack-nova20:21
*** wolverineav has quit IRC20:26
*** moshele has quit IRC20:27
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein"  https://review.openstack.org/63250720:28
kashyapAlright, actually rest of the 16 are fixed, too.20:28
* kashyap --> hits the hay20:28
*** penick has joined #openstack-nova20:33
*** awaugama_ has joined #openstack-nova20:33
*** markvoelker has joined #openstack-nova20:33
*** moshele has joined #openstack-nova20:34
*** awaugama has quit IRC20:34
*** xek__ has joined #openstack-nova20:35
*** xek_ has quit IRC20:37
*** awaugama_ has quit IRC20:47
mriedemhuh i see the 12.0.0 liberty GA release notes aren't published https://docs.openstack.org/releasenotes/nova/liberty.html20:50
mriedemdansmith: you wanna help get the legacy-grenade-dsvm-neutron-multinode-live-migration job fixed? https://review.openstack.org/#/c/634962/ already +2d20:54
*** jmlowe has joined #openstack-nova20:55
dansmithDO I EVER20:55
mriedemnot sure why that job isn't voting, probably should be20:55
mriedemseems like live migrating back and forth across n and n-1 nodes is good to keep working..20:55
*** dpawlik has joined #openstack-nova20:59
*** wolverineav has joined #openstack-nova21:00
cfriesendansmith: how does https://git.openstack.org/cgit/openstack/nova/commit/?id=804215017f38300ac429e4401ce9c23f0c8ff3c6 avoid the traceback when it saves and re-raises the exception?21:00
dansmithcfriesen: I think the point of that was to log the important details where we had them, which are lost in the generic stack trace above21:02
mriedemcfriesen: i think the point is it's avoiding this https://review.openstack.org/#/c/546423/5/nova/virt/block_device.py@31321:02
dansmithah, maybe that too,21:03
dansmithbut the bug was that the generic error got logged in like compute manager orsomething and it wasn't clear that the problem was that it failed to detach a volume21:03
*** slaweq_ has joined #openstack-nova21:03
mriedemi'm +1 now on the down cell servers view builder change https://review.openstack.org/#/c/635146/21:05
mriedemwithholding +2 for now since it calls out 2.6921:05
*** dpawlik has quit IRC21:06
*** slaweq has quit IRC21:06
*** dtantsur has quit IRC21:06
*** tinwood has quit IRC21:06
*** mugsie has quit IRC21:06
*** dtantsur has joined #openstack-nova21:06
*** markvoelker has quit IRC21:06
artomCan I rant about unit testing? I'm ranting about unit testing. Unit testing is not "mock everything the function calls and assert it was called the way the current implementation you're testing would.21:06
*** mugsie has joined #openstack-nova21:06
*** tinwood has joined #openstack-nova21:06
edleafeartom: Most of our unit tests could be named "test_that_python_works"21:09
artomedleafe, hah, I'm laughing now, but maybe I should cry?21:09
artommdbooth would love that21:17
*** moshele has quit IRC21:18
*** moshele has joined #openstack-nova21:19
*** dpawlik has joined #openstack-nova21:20
*** dpawlik has quit IRC21:24
*** sdake has joined #openstack-nova21:26
*** wolverineav has quit IRC21:30
*** xek__ has quit IRC21:31
*** slaweq_ is now known as slaweq21:32
*** wolverineav has joined #openstack-nova21:34
aspiersmriedem, efried: I seem to have successfully(?) rebased https://review.openstack.org/538498 on top of the removal of get_traits()21:35
aspiersmriedem, efried: git-irebase has just brought it 250 commits closer to master, but it's just hit another conflict at master~788 so I've got a way to go still ...21:36
aspiersbut I think I actually sort of understand what's going on now, which is encouraging21:36
openstackgerritArtom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods  https://review.openstack.org/63621021:37
openstackgerritArtom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination  https://review.openstack.org/63460521:37
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source  https://review.openstack.org/63482721:37
*** agopi_ has joined #openstack-nova21:37
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim()  https://review.openstack.org/63566921:37
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources  https://review.openstack.org/63460621:37
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source  https://review.openstack.org/63482821:37
openstackgerritArtom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source  https://review.openstack.org/63522921:37
aspiersAh, that conflict was super easy to resolve. Onwards...21:38
*** agopi has quit IRC21:39
aspiersOK, I think I am officially declaring git-irebase as awesome21:40
*** agopi_ is now known as agopi21:41
*** wolverineav has quit IRC21:41
*** wolverineav has joined #openstack-nova21:41
*** _alastor_ has quit IRC21:41
*** artom has quit IRC21:42
efriedaspiers: what is git-irebase?21:44
aspiersincremental rebase21:44
aspiersone commit at a time21:44
aspiersso you are only ever resolving conflicts against a single commit21:44
efriedye gods, that sounds like the hard way to go, but good on ya.21:45
aspiersno, it's really easier21:45
aspiersthis change you pointed me at is over 1000 commits behind master21:45
aspiersit conflicts with multiple commits along the way21:45
efriedright, but only a handful of those would have conflicted.21:45
*** hamzy has quit IRC21:46
aspiersyes, but I would have had to understand *all* of those commits at the same time to resolve the conflict via a normal single-shot rebase21:46
aspierswith code I've never seen before, that's a nightmare21:46
aspierseven if I'd seen it I suspect it would have been tricky21:46
aspiersI'm now at master~510 and so far I've had to handle one major conflict (the get_traits() removal from ironic driver) and 3 minor conflicts21:47
aspiersalso it means I can run the tests after each resolve, which increases the chances of catching logic errosr21:49
aspiersjust hit another major conflict21:49
aspiersbut it's only a single commit, so now manageable :)21:49
*** sdake has quit IRC21:50
aspiersah it wasn't major, just a method signature change21:51
*** moshele has quit IRC21:52
*** moshele has joined #openstack-nova21:53
efriedaspiers: I looked at the all-in-one rebase, and you're right, it would be pretty hairy. Thanks for tackling it.21:54
aspiersyw! only 307 commits behind master now :)21:54
aspiersefried: do I understand that for adding an SEV capability to the libvirt driver, capabilities has to be an instance variable rather than a class variable (like for powervm) because the SEV capability is determined dynamically per host?21:55
*** mchlumsky has quit IRC21:56
efriedaspiers: I'm pretty sure that's what we figured out for the PowerVM image cache capability, which is based on which disk driver is set up. edmondsw, back me up here.21:57
aspiersyeah, I saw the image cache cap being set at runtime21:57
efriedI actually can't remember the exact reason we couldn't use the class variable.21:57
efriedor maybe we just thought it was ugly to set PowerVMDriver.capabilities['image_cache'] from the context of 'self'.21:58
aspierspresumably it varies from host to host - does the driver get instantiated once per host?21:58
efriedAt least once, yes.21:59
efriedand it's the "possibly more than once" part that might be what gets you into trouble using a class var. But that doesn't make a lot of sense.21:59
*** _alastor_ has joined #openstack-nova21:59
efriedLet me go find the review where that happened.21:59
*** prometheanfire has left #openstack-nova22:00
efriedaspiers: Ah, it may have just been for testing purposes: https://github.com/openstack/nova-powervm/commit/cc7564549a90a59f2ef27cca14862c33b9a07c2522:01
aspiersahhh ok22:02
aspiersrebase finished!!22:03
*** markvoelker has joined #openstack-nova22:03
efriedswheet!22:03
aspiersI think there were about 8 commits conflicting in total22:03
aspiersout of ~100022:03
efriedquick, upload the rebase-only so you have a checkpoint.22:03
aspiersto the same review?22:03
efriedabsolutely22:04
aspiersOK22:04
openstackgerritAdam Spiers proposed openstack/nova master: WIP: Convert driver supported capabilities to compute node provider traits  https://review.openstack.org/53849822:05
aspiersefried: there ya go22:06
aspierswho knows, Zuul might even verify it :)22:06
cdentwow, that's like time travel or something22:07
efriedaspiers: I'm a fan of doing rebase in one PS and updates in another. Doing an incremental review when rebase+changes are in one PS is suck.22:07
aspiersefried: totally agree22:07
efriedespecially when the rebase is massive.22:07
mriedemdansmith: comments in the down cell api microversion change https://review.openstack.org/#/c/591657/ - at this point i think i need to pull it into a devstack environment and kick the tires because I think it's being aggressive about processing filters when 'nova list' might send some parameters by default that would otherwise be ignored22:08
*** moshele has quit IRC22:08
dansmithmriedem: okay22:08
aspiersefried: unfortunately in this case diffing between PS7 and my new PS8 is also full of suck, because we're running an ancient Gerrit which can't distinguish between real changes and stuff dragged in by the rebase22:08
*** maciejjozefczyk has joined #openstack-nova22:09
aspiersnewer Gerrit handles this correctly22:09
efriedaspiers: customary to use -W instead of -1 to signify wip with changes on the way.22:09
efriedaspiers: Oh, really, there's a better gerrit where that's concerned? That would be pretty amazing.22:09
aspiersefried: I would have if I had W permissions ...22:09
efriedaspiers: duh, of course.22:09
aspiersXD22:09
aspiersyes, I remember seeing it demoed at the Gerrit User Summit in London22:09
aspiersI can maybe even find the youtube for you22:10
efriedAs it stands, I open two tabs, one with old and one with new, and flip-diff 'em.22:10
aspiersyeah, that's better22:10
aspiersefried: https://youtu.be/yWTvUyvP24M?t=963 you can even see my shiny head in the 2nd row ;-)22:11
aspiersalthough unfortunately we're still on 2.13, not even 2.14 yet let alone 2.15 ...22:12
*** maciejjozefczyk has quit IRC22:13
aspiersOK, time for a very late dinner...22:18
aspiersl8rs22:18
*** slaweq has quit IRC22:34
*** markvoelker has quit IRC22:36
*** cdent has quit IRC22:36
edmondswefried yep, you found it... sorry, was in a mtg22:42
edmondsw(in a lot of those these days...)22:42
*** agopi has quit IRC22:43
openstackgerritMatt Riedemann proposed openstack/nova master: Change nova-next tempest test regex  https://review.openstack.org/63645922:52
mriedem^ drops a bunch of tempest API tests that we don't care about for nova changes22:52
*** mrjk_ has joined #openstack-nova22:52
mriedemhopefully making that job a bit faster and less redundant22:52
*** wolverineav has quit IRC22:54
*** wolverineav has joined #openstack-nova22:55
*** wolverineav has quit IRC22:58
*** wolverineav has joined #openstack-nova22:59
openstackgerritMerged openstack/nova master: Fix legacy-grenade-dsvm-neutron-multinode-live-migration  https://review.openstack.org/63496223:08
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: API microversion 2.69: Handles Down Cells  https://review.openstack.org/57956323:10
*** whoami-rajat has joined #openstack-nova23:12
*** tkajinam has joined #openstack-nova23:16
*** artom has joined #openstack-nova23:18
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: API microversion 2.69: Handles Down Cells  https://review.openstack.org/57956323:21
*** ociuhandu has quit IRC23:30
*** ociuhandu has joined #openstack-nova23:30
*** agopi has joined #openstack-nova23:33
*** markvoelker has joined #openstack-nova23:33
mriedemdansmith: what would be a simple way to effectively disable/stop the nova_cell1 db without actually stopping mysqld? failing at finding a db stop command23:35
mriedemjaypipes: ^23:35
mriedemi guess i could eff up the db_connection in the cell1 mapping23:36
sean-k-mooneymriedem: what do you mean by stop?23:36
mriedemi want to make a specific db unavailable23:36
sean-k-mooneyas in not allow any new queries/transactins23:36
mriedemi guess23:36
mriedemi want to simulate that the db has crashed23:36
mriedemw/o stopping mysqld23:37
dansmithmriedem: hard to do on a single server and make it real23:37
mriedemsince i still need mysql for nova_cell0 and nova_api23:37
mriedemi'll just eff up the db connection then23:37
sean-k-mooneyright because you want to allow the other dbs to work23:37
dansmithmriedem: you could revoke privs on the db23:37
dansmithmriedem: or try dropping the tables altogether I guess, but tbh we don't really know what the failure modes will look like I'd guess, and most of the time in reality it would be a connection timeout,23:37
dansmithwhich will be hard to mimic with one mysql23:38
dansmithone thing you could do, is change the connection string to point to some ip black hole for you23:38
dansmiththat would get you the timeout on that connection23:38
dansmithdepends on if you're trying to make it happen at runtime, or just start it up unable to reach one of the dbs23:38
sean-k-mooneydansmith: you might be able to do it with ssh forwarding23:40
dansmithdo what/23:40
mriedemyeah i just changed cell1 db connection to mysql+pymysql://root:openstack1@127.0.0.2/nova_cell1?charset=utf823:41
mriedemwill see what happens23:41
sean-k-mooneye.g. expose the mysql server port locally on anoter port point nova at it and then kill the ssh tunnel e.g. using ssh -L ...23:41
dansmithmriedem: that might not be a black hole for you :)23:41
dansmithmriedem: I would change it to some 172. or 192. or 10. address you definitely can't reach23:41
*** macza has quit IRC23:42
mriedemoh yeah the connections are cached,23:42
sean-k-mooneyusing ssh that way would allow nova to start up normmally and then you can kill the connect when ever you like by killing the ssh tunnel23:42
mriedemrestarting n-api23:42
*** macza has joined #openstack-nova23:42
mriedembingo23:43
mriedemFeb 12 23:42:52 downcell devstack@n-api.service[13288]: WARNING oslo_db.sqlalchemy.engines [None req-8002ab53-453c-488e-ae15-04e2a74e7e6f None None] SQL connection failed. 10 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on u'192.0.0.1' ([Errno 101] ENETUNREACH)") (Background on this error at: http://sqlalche.me/e/e3q8)23:43
dansmithright on23:43
mriedemoh goody and it's just spewing in the api logs23:43
mriedemwell i guess 10 retries23:43
dansmithsean-k-mooney: he could just do that with some outbound nat to whatever black hole address he chooses23:43
*** sdake has joined #openstack-nova23:43
mriedemFeb 12 23:43:49 downcell devstack@n-api.service[13288]: WARNING nova.context [None req-f257afc4-18c9-4989-a500-a4a1f017a322 None None] Timed out waiting for response from cell 0641a06e-4efa-4c25-acac-b92d1f9082b7: CellTimeout: Timeout waiting for response from cell23:44
mriedemFeb 12 23:43:49 downcell devstack@n-api.service[13288]: WARNING nova.objects.service [None req-f257afc4-18c9-4989-a500-a4a1f017a322 None None] Cell 0641a06e-4efa-4c25-acac-b92d1f9082b7 did not respond when getting minimum service version23:44
sean-k-mooneydansmith: ya you can do it with iptables but that is more annoying23:44
mriedemhow about i just changed the db connectoin in the db to a bad ip :)23:44
sean-k-mooneywith ssh its basically "ssh 127.0.0.1 -L 53306:127.0.0.1:3306"23:44
mriedemmuch easier23:44
sean-k-mooneymriedem: it is but do you want to be able to kill the connect at an arbitray point or just not have it work23:45
mriedemhmm, nova list with 2.69 hanging...this is probably expected while it's trying to gather results from the down cell db23:46
*** zhanglong has joined #openstack-nova23:46
*** ade_lee_ has quit IRC23:47
mriedemi seem to be stuck in an infinite reconnect loop blocking the api response23:48
mriedemand i got a 50023:49
*** wolverineav has quit IRC23:49
mriedemoh heh,23:50
sean-k-mooneyproably due to a timeout of the connect waiting for the responce23:50
*** wolverineav has joined #openstack-nova23:50
mriedemdansmith: i think when nova client is trying to do version negotiation it's getting hung up on that23:50
mriedemnova --debug list isn't even getting to the GET /servers/detail request23:50
dansmithmriedem: hmm, why would we be striping cells at that point?23:51
mriedemthis is basically the api log http://paste.openstack.org/show/744983/23:51
dansmithwhen it fails we should get an exception that tells us where it is right?23:53
dansmithbut yeah, ten connect attempts at a slow timeout is going to suck23:53
dansmithand if we swallow, might need to print it out somewhere so you can see23:53
mriedemhttps://docs.openstack.org/nova/latest/configuration/config.html#database.max_retries23:54
mriedemhttps://docs.openstack.org/nova/latest/configuration/config.html#database.retry_interval23:54
mriedem10 attempts at 10 seconds between attempts23:54
mriedemor this.. https://docs.openstack.org/nova/latest/configuration/config.html#database.db_max_retries23:54
sean-k-mooneyits connecting to the cell deb to get to the service table to figure out the mimium service version23:54
mriedemthere are a lot of similar sounding options23:54
sean-k-mooneydo we not have that in the api db?23:55
mriedemsean-k-mooney: we shouldn't need that in GET /23:55
mriedemthe API version doesn't care what the compute services are23:55
mriedemor shouldn't23:55
sean-k-mooneydid you mess with the nova_cell0 connection23:55
sean-k-mooneyisint that wehere the nova api service record would be23:55
mriedemclient side error isn't helpful http://paste.openstack.org/show/744984/23:56
*** wolverineav has quit IRC23:56
mriedembut yeah this is what is in the server API logs23:56
mriedemFeb 12 23:54:50 downcell devstack@n-api.service[13288]: WARNING nova.context [None req-e9c1b49d-437b-47f9-a28e-7d5811908b58 None None] Timed out waiting for response from cell 0641a06e-4efa-4c25-acac-b92d1f9082b7: CellTimeout: Timeout waiting for response from cell23:56
mriedemFeb 12 23:54:50 downcell devstack@n-api.service[13288]: WARNING nova.objects.service [None req-e9c1b49d-437b-47f9-a28e-7d5811908b58 None None] Cell 0641a06e-4efa-4c25-acac-b92d1f9082b7 did not respond when getting minimum service version23:56
dansmithah23:57
dansmithservice version23:57
dansmithneeds that for microversion discovery I guess?23:57
mriedemthe api version shouldn't depend on the service version though23:57
mriedemthe service version is about rpc yeah?23:57
dansmithI thought we didn't offer some microversions if computes hadn't ugpraded yet?23:58
dansmithor do we just refuse to do the thing?23:58
sean-k-mooneymriedem: when you curl / it has the service version in the responce i think23:58
dansmithsean-k-mooney: no23:58
mriedemcurl -g -i -X GET http://199.204.45.32/compute/v2.1 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}121cea354034cc9d2b715383c002e81f067eeb1207dd273ff76587143e309ae5"23:58
mriedemthat's the request23:58
mriedemi don't see what in the version router request handler is trying to do something with service versions23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!