*** jmlowe has quit IRC | 00:03 | |
*** wolverineav has quit IRC | 00:04 | |
*** wolverineav has joined #openstack-nova | 00:05 | |
*** wolverineav has quit IRC | 00:10 | |
*** wolverineav has joined #openstack-nova | 00:14 | |
*** tetsuro has joined #openstack-nova | 00:25 | |
*** TxGirlGeek has quit IRC | 00:25 | |
*** macza has joined #openstack-nova | 00:29 | |
*** tetsuro has quit IRC | 00:30 | |
*** tetsuro has joined #openstack-nova | 00:32 | |
*** macza has quit IRC | 00:34 | |
*** tetsuro has quit IRC | 00:35 | |
*** tetsuro_ has joined #openstack-nova | 00:35 | |
*** wolverineav has quit IRC | 00:37 | |
*** takashin has joined #openstack-nova | 00:38 | |
*** wolverineav has joined #openstack-nova | 00:39 | |
*** wolverineav has quit IRC | 00:43 | |
*** wolverineav has joined #openstack-nova | 00:44 | |
*** gyee has quit IRC | 00:54 | |
*** zioproto has joined #openstack-nova | 00:59 | |
*** fyx has joined #openstack-nova | 00:59 | |
*** TheJulia has joined #openstack-nova | 00:59 | |
*** mnasiadka has joined #openstack-nova | 00:59 | |
*** coreycb has joined #openstack-nova | 00:59 | |
*** icey has joined #openstack-nova | 00:59 | |
*** mrhillsman has joined #openstack-nova | 00:59 | |
*** sdake has joined #openstack-nova | 01:01 | |
*** moshele has joined #openstack-nova | 01:07 | |
*** macza has joined #openstack-nova | 01:08 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Start functional testing for cross-cell resize https://review.openstack.org/636253 | 01:09 |
---|---|---|
*** wolverineav has quit IRC | 01:09 | |
mriedem | woooowee | 01:09 |
mriedem | dansmith: ^ well that's progress | 01:09 |
mriedem | retrofitted my previous functional tests to the existing series, flushed a bunch of bugs, and have it tested up through VERIFY_RESIZE | 01:09 |
*** wolverineav has joined #openstack-nova | 01:09 | |
*** macza has quit IRC | 01:13 | |
*** tetsuro has joined #openstack-nova | 01:20 | |
*** tetsuro_ has quit IRC | 01:20 | |
*** tiendc has joined #openstack-nova | 01:22 | |
*** bhagyashris has joined #openstack-nova | 01:22 | |
*** mdbooth has joined #openstack-nova | 01:22 | |
*** mriedem has quit IRC | 01:23 | |
*** mdbooth_ has quit IRC | 01:24 | |
*** moshele has quit IRC | 01:25 | |
*** yedongcan has joined #openstack-nova | 01:25 | |
*** markvoelker has quit IRC | 01:30 | |
*** markvoelker has joined #openstack-nova | 01:31 | |
*** tetsuro has quit IRC | 01:35 | |
*** tetsuro_ has joined #openstack-nova | 01:35 | |
*** markvoelker has quit IRC | 01:35 | |
*** _fragatina has quit IRC | 01:37 | |
*** sdake has quit IRC | 01:37 | |
*** sdake has joined #openstack-nova | 01:41 | |
*** sdake has quit IRC | 01:43 | |
*** wolverineav has quit IRC | 01:49 | |
*** Dinesh_Bhor has joined #openstack-nova | 01:49 | |
*** sdake has joined #openstack-nova | 01:51 | |
*** sdake has quit IRC | 01:51 | |
*** wolverineav has joined #openstack-nova | 01:52 | |
*** wolverineav has quit IRC | 01:56 | |
*** tetsuro has joined #openstack-nova | 02:01 | |
*** tetsuro_ has quit IRC | 02:01 | |
openstackgerrit | Merged openstack/nova master: Plumbing for allowing the all-tenants filter with down cells https://review.openstack.org/635145 | 02:01 |
*** TxGirlGeek has joined #openstack-nova | 02:05 | |
*** lbragstad has quit IRC | 02:06 | |
*** macza has joined #openstack-nova | 02:22 | |
*** macza has quit IRC | 02:27 | |
*** macza has joined #openstack-nova | 02:27 | |
*** hongbin has joined #openstack-nova | 02:29 | |
*** macza has quit IRC | 02:31 | |
*** markvoelker has joined #openstack-nova | 02:31 | |
*** TxGirlGeek has quit IRC | 02:36 | |
*** sapd1 has joined #openstack-nova | 02:47 | |
*** markvoelker has quit IRC | 03:05 | |
*** tetsuro has quit IRC | 03:22 | |
*** tetsuro_ has joined #openstack-nova | 03:22 | |
*** tetsuro has joined #openstack-nova | 03:26 | |
*** tetsuro_ has quit IRC | 03:26 | |
*** takashin has quit IRC | 03:28 | |
*** sdake has joined #openstack-nova | 03:40 | |
*** jmlowe has joined #openstack-nova | 03:43 | |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods https://review.openstack.org/636210 | 03:48 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination https://review.openstack.org/634605 | 03:48 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source https://review.openstack.org/634827 | 03:48 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim() https://review.openstack.org/635669 | 03:48 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources https://review.openstack.org/634606 | 03:48 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source https://review.openstack.org/634828 | 03:48 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source https://review.openstack.org/635229 | 03:48 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Trivial: reorder hashes according to object_hashes.txt https://review.openstack.org/636261 | 03:48 |
artom | Actually I should probably put the trivial one at the very bottom... | 03:49 |
*** takashin has joined #openstack-nova | 03:49 | |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Trivial: reorder hashes according to object_hashes.txt https://review.openstack.org/636261 | 03:50 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods https://review.openstack.org/636210 | 03:50 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination https://review.openstack.org/634605 | 03:50 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source https://review.openstack.org/634827 | 03:50 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim() https://review.openstack.org/635669 | 03:50 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources https://review.openstack.org/634606 | 03:50 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source https://review.openstack.org/634828 | 03:50 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source https://review.openstack.org/635229 | 03:50 |
*** TxGirlGeek has joined #openstack-nova | 03:58 | |
*** dave-mccowan has quit IRC | 03:58 | |
*** sapd1 has quit IRC | 03:59 | |
*** sapd1 has joined #openstack-nova | 03:59 | |
*** sdake has quit IRC | 03:59 | |
*** markvoelker has joined #openstack-nova | 04:02 | |
*** udesale has joined #openstack-nova | 04:02 | |
*** tetsuro has quit IRC | 04:05 | |
*** lbragstad has joined #openstack-nova | 04:09 | |
*** gbarros has quit IRC | 04:28 | |
*** janki has joined #openstack-nova | 04:30 | |
*** markvoelker has quit IRC | 04:34 | |
*** manjeets has quit IRC | 04:41 | |
*** jmlowe has quit IRC | 04:50 | |
*** ratailor has joined #openstack-nova | 05:08 | |
*** hongbin has quit IRC | 05:30 | |
*** markvoelker has joined #openstack-nova | 05:32 | |
*** abhishekk has joined #openstack-nova | 05:33 | |
*** udesale has quit IRC | 05:37 | |
*** wolverineav has joined #openstack-nova | 05:55 | |
*** TxGirlGeek has quit IRC | 05:56 | |
*** markvoelker has quit IRC | 06:05 | |
*** Dinesh_Bhor has quit IRC | 06:05 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: api-ref: Add descriptions for vol-backed snapshots https://review.openstack.org/615084 | 06:07 |
*** Dinesh_Bhor has joined #openstack-nova | 06:09 | |
*** whoami-rajat has joined #openstack-nova | 06:19 | |
*** sridharg has joined #openstack-nova | 06:20 | |
*** wolverineav has quit IRC | 06:38 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix resetting non-persistent fields when saving obj https://review.openstack.org/636271 | 06:43 |
*** tiendc has quit IRC | 06:44 | |
*** ccamacho has quit IRC | 06:49 | |
*** cfriesen has quit IRC | 06:50 | |
*** lbragstad has quit IRC | 06:50 | |
*** Luzi has joined #openstack-nova | 06:53 | |
*** abhishekk has quit IRC | 06:58 | |
*** ralonsoh has joined #openstack-nova | 06:59 | |
*** NewBruce has joined #openstack-nova | 07:01 | |
*** markvoelker has joined #openstack-nova | 07:02 | |
*** NewBruce has quit IRC | 07:03 | |
*** udesale has joined #openstack-nova | 07:09 | |
*** mdbooth_ has joined #openstack-nova | 07:11 | |
*** moshele has joined #openstack-nova | 07:12 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix resetting non-persistent fields when saving obj https://review.openstack.org/636271 | 07:12 |
*** mdbooth has quit IRC | 07:14 | |
*** abhishekk has joined #openstack-nova | 07:16 | |
*** NewBruce has joined #openstack-nova | 07:23 | |
*** markvoelker has quit IRC | 07:35 | |
*** takashin has quit IRC | 07:35 | |
*** takashin has joined #openstack-nova | 07:38 | |
*** slaweq has joined #openstack-nova | 07:44 | |
*** ccamacho has joined #openstack-nova | 07:58 | |
*** ccamacho has quit IRC | 07:59 | |
*** ccamacho has joined #openstack-nova | 07:59 | |
*** awalende has joined #openstack-nova | 08:06 | |
openstackgerrit | Merged openstack/nova master: doc: update the security groups admin doc https://review.openstack.org/634735 | 08:06 |
openstackgerrit | Merged openstack/nova master: doc: link admin/configuration from admin home page https://review.openstack.org/634728 | 08:07 |
*** yan0s has joined #openstack-nova | 08:08 | |
*** tesseract has joined #openstack-nova | 08:09 | |
*** awalende has quit IRC | 08:09 | |
*** rpittau has joined #openstack-nova | 08:13 | |
*** awalende has joined #openstack-nova | 08:20 | |
*** xek_ has joined #openstack-nova | 08:28 | |
*** tkajinam has quit IRC | 08:29 | |
*** helenafm has joined #openstack-nova | 08:30 | |
*** markvoelker has joined #openstack-nova | 08:32 | |
openstackgerrit | Merged openstack/nova stable/queens: Fix InstanceNotFound during _destroy_evacuated_instances https://review.openstack.org/623355 | 08:42 |
*** takashin has left #openstack-nova | 08:52 | |
*** whoami-rajat has quit IRC | 08:54 | |
*** priteau has joined #openstack-nova | 08:59 | |
*** tetsuro has joined #openstack-nova | 09:00 | |
*** panda|off is now known as panda | 09:04 | |
*** tetsuro has quit IRC | 09:05 | |
*** markvoelker has quit IRC | 09:05 | |
*** tetsuro has joined #openstack-nova | 09:05 | |
*** jistr_ is now known as jistr | 09:09 | |
*** tetsuro has quit IRC | 09:16 | |
*** tssurya has joined #openstack-nova | 09:17 | |
*** tetsuro has joined #openstack-nova | 09:17 | |
*** cdent has joined #openstack-nova | 09:35 | |
*** tetsuro has quit IRC | 09:39 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding https://review.openstack.org/569459 | 09:46 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317 | 09:46 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Remove port allocation during detach https://review.openstack.org/622421 | 09:46 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Record requester in the InstancePCIRequest https://review.openstack.org/625310 | 09:46 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add pf_interface_name tag to passthrough_whitelist https://review.openstack.org/625311 | 09:46 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF https://review.openstack.org/623543 | 09:46 |
*** NewBruce has quit IRC | 09:54 | |
*** NewBruce has joined #openstack-nova | 09:59 | |
*** stephenfin_ is now known as stephenfin | 10:01 | |
*** markvoelker has joined #openstack-nova | 10:02 | |
*** NewBruce has quit IRC | 10:03 | |
lyarwood | bauzas: https://review.openstack.org/#/c/628726/ - would you mind hitting this stable backport today if you have a chance? | 10:05 |
bauzas | lyarwood: yup, actively working on fixing some problems with the reshape change, but sure will look | 10:06 |
lyarwood | bauzas: ack thanks | 10:07 |
*** moshele has quit IRC | 10:08 | |
*** belmoreira has joined #openstack-nova | 10:12 | |
gibi | lyarwood: hi! If we are at stable backports, could you check https://review.openstack.org/#/c/635921/1 and https://review.openstack.org/#/c/623358/1 ? | 10:14 |
lyarwood | gibi: ack can do | 10:14 |
gibi | thanks! | 10:14 |
*** NewBruce has joined #openstack-nova | 10:19 | |
bauzas | gibi: around ? | 10:20 |
abhishekk | hi all, any idea how can I print queries in logs? | 10:20 |
bauzas | gibi: I'm puzzled with testing your functional tests again my new revision | 10:20 |
abhishekk | I have tried to set default_log_levels = sqlalchemy=INFO but its not working | 10:20 |
gibi | bauzas: push your new revision up and I will check the functional test | 10:20 |
bauzas | ack | 10:20 |
bauzas | it will fail your change tho | 10:21 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: compute: Reject migration requests when source is down https://review.openstack.org/623489 | 10:21 |
bauzas | gibi: I'll just add a new change for correctly using the right mdev before tho | 10:21 |
gibi | bauzas: you mean it will fail the functional test? That is OK I will check that | 10:21 |
lyarwood | sean-k-mooney / stephenfin / mdbooth_ ^ I'd appreciate your thoughts on my bugfix above if you have time today | 10:21 |
lyarwood | oh and is anyone else having weird issues building releasenotes on f29? | 10:22 |
* stephenfin clicks | 10:23 | |
stephenfin | and tries building releasenotes to check | 10:23 |
* mdbooth_ is getting weirder issues on f29 | 10:24 | |
*** mdbooth_ is now known as mdbooth | 10:24 | |
lyarwood | stephenfin: /src/openstack/nova/releasenotes/source/rocky.rst:3436:Duplicate explicit target name: "rest api version history". - is the error I'm getting locally | 10:24 |
*** bhagyashris has quit IRC | 10:25 | |
*** derekh has joined #openstack-nova | 10:27 | |
stephenfin | lyarwood: That doesn't sound like something F29 specific. That would happen if someone had defined the same target in two release notes, which it looks like someone has | 10:27 |
stephenfin | though it's building just fine for me... | 10:28 |
lyarwood | stephenfin: kk there must be some cruft in my local repo somewhere | 10:28 |
stephenfin | 'git clean -xfd -e .tox' <-- I should have this aliased | 10:29 |
*** abhishekk has quit IRC | 10:30 | |
*** Dinesh_Bhor has quit IRC | 10:32 | |
*** markvoelker has quit IRC | 10:35 | |
*** moshele has joined #openstack-nova | 10:39 | |
*** purplerbot has joined #openstack-nova | 10:41 | |
openstackgerrit | Boris Bobrov proposed openstack/nova master: Fix memtune parameters according to libvirt docs https://review.openstack.org/636301 | 10:42 |
*** NewBruce has quit IRC | 10:42 | |
openstackgerrit | Boris Bobrov proposed openstack/nova master: Fix memtune parameters according to libvirt docs https://review.openstack.org/636301 | 10:43 |
*** sapd1 has quit IRC | 10:45 | |
*** udesale has quit IRC | 10:49 | |
*** NewBruce has joined #openstack-nova | 10:49 | |
*** NewBruce has quit IRC | 10:54 | |
*** NewBruce has joined #openstack-nova | 11:04 | |
*** NewBruce has quit IRC | 11:06 | |
*** erlon has joined #openstack-nova | 11:07 | |
*** lpetrut has joined #openstack-nova | 11:15 | |
*** NewBruce has joined #openstack-nova | 11:31 | |
*** markvoelker has joined #openstack-nova | 11:32 | |
*** tetsuro has joined #openstack-nova | 11:33 | |
*** mvkr has quit IRC | 11:44 | |
*** tetsuro has quit IRC | 11:46 | |
*** NewBruce has quit IRC | 11:47 | |
*** NewBruce has joined #openstack-nova | 11:50 | |
*** erlon has quit IRC | 11:51 | |
*** _fragatina has joined #openstack-nova | 11:56 | |
*** _fragatina has quit IRC | 11:59 | |
*** _fragatina has joined #openstack-nova | 12:00 | |
openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Add method to allow reset fields for root bdm in BDM obj https://review.openstack.org/614672 | 12:00 |
*** markvoelker has quit IRC | 12:05 | |
openstackgerrit | Boris Bobrov proposed openstack/nova master: Add configs for AMD SEV https://review.openstack.org/636318 | 12:10 |
*** kaisers has quit IRC | 12:24 | |
*** mrch has joined #openstack-nova | 12:24 | |
*** erlon has joined #openstack-nova | 12:29 | |
*** tbachman has quit IRC | 12:29 | |
*** mrch has quit IRC | 12:29 | |
*** mrch has joined #openstack-nova | 12:30 | |
*** kaisers has joined #openstack-nova | 12:31 | |
*** rha has joined #openstack-nova | 12:32 | |
*** panda is now known as panda|lunch | 12:32 | |
*** ratailor has quit IRC | 12:34 | |
*** coreycb has quit IRC | 12:36 | |
*** coreycb has joined #openstack-nova | 12:37 | |
*** lpetrut has quit IRC | 12:38 | |
*** lpetrut has joined #openstack-nova | 12:39 | |
*** mrch has quit IRC | 12:39 | |
*** janki has quit IRC | 12:44 | |
*** janki has joined #openstack-nova | 12:44 | |
*** sapd1 has joined #openstack-nova | 12:44 | |
openstackgerrit | Jim Rollenhagen proposed openstack/nova master: Ironic: bump minimum API version to 1.38 https://review.openstack.org/636326 | 12:45 |
jroll | TheJulia: ^ sanity check me that this is cleanup we want to do, please | 12:45 |
*** mvkr has joined #openstack-nova | 12:46 | |
*** _fragatina has quit IRC | 12:56 | |
*** moshele has quit IRC | 13:00 | |
*** markvoelker has joined #openstack-nova | 13:02 | |
*** yan0s has quit IRC | 13:02 | |
*** mriedem has joined #openstack-nova | 13:04 | |
*** Dinesh_Bhor has joined #openstack-nova | 13:04 | |
*** tbachman has joined #openstack-nova | 13:04 | |
*** mvkr has quit IRC | 13:08 | |
*** udesale has joined #openstack-nova | 13:10 | |
*** panda|lunch is now known as panda | 13:13 | |
mriedem | tssurya: i think we might have some traces in the api logs that are saying a cell is down when it's really something else | 13:14 |
mriedem | http://logs.openstack.org/67/620667/3/check/tempest-full/8fa0aa0/controller/logs/screen-n-api.txt.gz?level=TRACE#_Feb_12_00_25_30_615423 | 13:14 |
mriedem | although it looks like there are some MQ issues in those logs | 13:14 |
*** tbachman has quit IRC | 13:22 | |
*** whoami-rajat has joined #openstack-nova | 13:22 | |
*** tbachman has joined #openstack-nova | 13:23 | |
*** agopi has joined #openstack-nova | 13:26 | |
*** ociuhandu has joined #openstack-nova | 13:26 | |
*** NewBruce has quit IRC | 13:29 | |
*** NewBruce has joined #openstack-nova | 13:30 | |
*** ade_lee_ has quit IRC | 13:30 | |
*** sdake has joined #openstack-nova | 13:32 | |
*** agopi has quit IRC | 13:34 | |
*** markvoelker has quit IRC | 13:35 | |
*** tbachman has quit IRC | 13:35 | |
*** openstackgerrit has quit IRC | 13:37 | |
lyarwood | mriedem: https://review.openstack.org/#/c/551302/ - has been rebased and should be ready to review again if you have anytime this week. | 13:37 |
mriedem | lyarwood: yup in the queue | 13:38 |
mriedem | did you see the conversation in there about rollback also being busted? | 13:38 |
*** mvkr has joined #openstack-nova | 13:40 | |
lyarwood | mriedem: so I think mdbooth was addressing that in a follow up here https://review.openstack.org/#/c/551349/2 | 13:40 |
lyarwood | mdbooth: ^ I can also rebase / refresh that if you'd like | 13:40 |
*** yedongcan has left #openstack-nova | 13:41 | |
mdbooth | lyarwood mriedem: Yeah, although not having time to address that too is probably the biggest reason I haven't looked at this in ages. | 13:42 |
mdbooth | Definitely a thing, though. | 13:42 |
*** openstackgerrit has joined #openstack-nova | 13:49 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF https://review.openstack.org/623543 | 13:49 |
openstackgerrit | Boris Bobrov proposed openstack/nova master: Extract SEV-specific bits on host detection https://review.openstack.org/636334 | 13:57 |
*** mlavalle has joined #openstack-nova | 13:59 | |
stephenfin | sean-k-mooney: Thoughts? http://paste.openstack.org/show/B9unIL8e2KpeSMBGaINe/ | 14:00 |
*** efoley has joined #openstack-nova | 14:01 | |
stephenfin | sean-k-mooney: I'm annoyed by the fact we have no central definition (for docs) or validation of flavour extra specs. Thinking it might be a good, easy spec for T (for the ones we're not deprecating with the cpu-resources spec | 14:02 |
*** agopi has joined #openstack-nova | 14:02 | |
*** efoley has left #openstack-nova | 14:02 | |
*** sdake has quit IRC | 14:03 | |
bauzas | stephenfin: we have documentation for flavor extra specs | 14:05 |
bauzas | unless I misunderstand your concern | 14:05 |
sean-k-mooney | stephenfin: we do it called the glance metadef registry | 14:06 |
bauzas | https://docs.openstack.org/nova/latest/user/flavors.html | 14:06 |
stephenfin | bauzas: Yup, but it's not "in code" or verified against the code, unlike e.g. the nova.conf options | 14:06 |
bauzas | ahah | 14:06 |
bauzas | well | 14:07 |
bauzas | AFAIK, we said meh | 14:07 |
gibi | bauzas: if you have time, can you push this two-patch backport? https://review.openstack.org/#/c/623358/1 | 14:07 |
sean-k-mooney | stephenfin: https://github.com/openstack/glance/blob/master/etc/metadefs/compute-cpu-pinning.json | 14:07 |
bauzas | gibi: looking | 14:07 |
sean-k-mooney | that is in code in a way | 14:07 |
gibi | bauzas: thanks | 14:07 |
bauzas | gibi: oh that one | 14:07 |
bauzas | gibi: silverlining, already reviewed it | 14:07 |
sean-k-mooney | stephenfin: it is intended to be used for validating glance metadata entries and its used but horizon to generate teh ui with the dropdown vlaues and for validation | 14:08 |
sean-k-mooney | it also has defintions for the flavor extra specs too. | 14:09 |
mriedem | tssurya: dansmith: i left comments in the down cell servers view builder change, the series is going to have to be rebased | 14:09 |
* mriedem goes to blow snow | 14:09 | |
stephenfin | sean-k-mooney: Aye, I knew about. If that was ever kept up-to-date though, I'm not sure if it still is. Barely any edits in years https://github.com/openstack/glance/tree/master/etc/metadefs | 14:10 |
*** mriedem is now known as mriedem_snow | 14:10 | |
sean-k-mooney | stephenfin: what might make sense it to extract it into a seperate repo that glance,nova and cinder can all import and share | 14:10 |
stephenfin | I didn't know that it supported flavour extra specs though. That's good to know | 14:10 |
*** zul has joined #openstack-nova | 14:10 | |
stephenfin | sean-k-mooney: Aye, I think so. I'll have a look into it when I get a chance. Just been annoying me for some time now | 14:10 |
sean-k-mooney | stephenfin: ya it was orginally created to keep the falvor extra specs and image metadata in sync | 14:11 |
sean-k-mooney | if we did extract it i dont see why we could not autogenerate the docs for it too | 14:11 |
*** NewBruce has quit IRC | 14:13 | |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods https://review.openstack.org/636210 | 14:13 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination https://review.openstack.org/634605 | 14:13 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source https://review.openstack.org/634827 | 14:13 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim() https://review.openstack.org/635669 | 14:13 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources https://review.openstack.org/634606 | 14:13 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source https://review.openstack.org/634828 | 14:13 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source https://review.openstack.org/635229 | 14:13 |
kashyap | Gosh, this serial / char devices code is making me go slowly mental... | 14:13 |
sean-k-mooney | stephenfin: can we put it as a cross project topic with the glance and maybe cinder folks for the ptg | 14:13 |
stephenfin | sean-k-mooney: Good idea, that | 14:14 |
sean-k-mooney | stephenfin: it looks like it also cover nova host aggrage metadata too https://github.com/openstack/glance/blob/master/etc/metadefs/compute-aggr-num-instances.json | 14:15 |
bauzas | lyarwood: I just agree with mriedem_snow on https://review.openstack.org/#/c/628726/4 | 14:15 |
bauzas | lyarwood: we should make backports easy by providing a series | 14:15 |
*** liuyulong has joined #openstack-nova | 14:15 | |
bauzas | WDYT ? | 14:15 |
*** sapd1 has quit IRC | 14:15 | |
lyarwood | bauzas: yeah ack sorry I totally forgot about that | 14:16 |
bauzas | since you already identified the source of the conflicts, and you also want to backport the missing changes, I don't see any difficulty in making the backports intertwined | 14:16 |
bauzas | lyarwood: ack, ping me when you're done then | 14:16 |
* bauzas goes back finding a creative way to not change a lot of things for a simple concern | 14:17 | |
bauzas | that's what happens when you write poor interfaces | 14:17 |
*** Dinesh_Bhor has quit IRC | 14:18 | |
*** NewBruce has joined #openstack-nova | 14:18 | |
*** sdake has joined #openstack-nova | 14:20 | |
NewBruce | sean-k-mooney any chance i can get your thoughts on a bit of a bug that looks to be related to the port binding API changes? | 14:20 |
*** sdake has quit IRC | 14:21 | |
sean-k-mooney | NewBruce: i saw you comments this moring | 14:25 |
NewBruce | awesome, sorry for chasing | 14:25 |
NewBruce | done some more debugging now, and feels like i isolated the point it fails | 14:25 |
sean-k-mooney | no worries i just havent had a chance to reply was booking flights and stuff | 14:25 |
sean-k-mooney | so one general question | 14:26 |
NewBruce | np. hope your goin' somewhere warm | 14:26 |
*** priteau has quit IRC | 14:26 | |
sean-k-mooney | PTG flights | 14:26 |
openstackgerrit | Merged openstack/nova stable/pike: Handle IndexError in _populate_neutron_binding_profile https://review.openstack.org/635921 | 14:26 |
NewBruce | sure - | 14:26 |
sean-k-mooney | am you have a mixed cloud with OSA nodes and RDO nodes | 14:27 |
NewBruce | correct | 14:27 |
sean-k-mooney | is the contol plan. e.g. your contolers all RDO or is it also mixed | 14:27 |
NewBruce | control plane is upgrade to Rocky/OSA | 14:27 |
NewBruce | we have multiple sites, with the same config which dont exhibit the problem | 14:28 |
sean-k-mooney | ok so you have Rocky/OSA contol plane + OSA computes + some rocky/RDO computes | 14:28 |
NewBruce | correct | 14:28 |
NewBruce | looking in the setup today we have some RDO net nodes as well | 14:28 |
sean-k-mooney | and on the neutron side is it also a mix or all OSA for you neturon controler services | 14:29 |
NewBruce | so they could do with being upgraded.... but in the debugging ive done today, it seems to bail out in the call to _setup_migration_port_profile | 14:29 |
dansmith | mriedem_snow: okay I was thinking we should maybe try real hard to get that merged today | 14:29 |
NewBruce | in a successful migration, the host_id passed to _setup_migration_port_profile is the target/dest compute; whereas in a failure the host_id passed to _setup_migration_port_profile is the source | 14:29 |
sean-k-mooney | how the multiple portbinging code is ment to work is that the nova condocutor is first ment to check the neutron api to determin if neutron support it | 14:30 |
*** erlon has quit IRC | 14:30 | |
openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: Fail to live migration if instance has a NUMA topology https://review.openstack.org/629597 | 14:31 |
openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: libvirt: Add workaround to cleanup instance dir when using rbd https://review.openstack.org/628726 | 14:31 |
*** ade_lee_ has joined #openstack-nova | 14:31 | |
sean-k-mooney | the condoctor then check the version of both the souce and dest compute service and if they both support the multiple port binding workflow it is enabled | 14:31 |
NewBruce | If both compute nodes are queens or newer. In this case the new workflow | 14:31 |
NewBruce | will be used as described below. | 14:31 |
lyarwood | bauzas / stephenfin / mriedem_snow ^ reworked these into a series now on top of https://review.openstack.org/#/c/620111/, hopefully that's cleaner for everyone. | 14:32 |
*** markvoelker has joined #openstack-nova | 14:32 | |
NewBruce | so that check is based on the nova versions, yeah? which is fair enough, as when i monitor the ml2_port_bindings. i see the duplicate port entries in both successful and fail cases; | 14:32 |
NewBruce | in the services table both source and dest are v35 | 14:33 |
NewBruce | for nova-compute | 14:33 |
sean-k-mooney | NewBruce: thew new workflow should only take effect if the source and dest compute nodes are rocky | 14:33 |
NewBruce | which they are | 14:33 |
*** lbragstad has joined #openstack-nova | 14:34 | |
NewBruce | sorry sean-k-mooney - gotta run for 10-15 will be back online | 14:34 |
sean-k-mooney | sure no worries | 14:34 |
*** NewBruce has quit IRC | 14:34 | |
zigo | Hi there! | 14:35 |
zigo | I'm getting a weirdo bug inside Horizon which is related to Novaclient. | 14:36 |
openstackgerrit | Vladyslav Drok proposed openstack/nova master: libvirt: Do not reraise DiskNotFound exceptions during resize https://review.openstack.org/571410 | 14:36 |
zigo | When listing instances, Horizon first try to get the nova microversion. | 14:36 |
kaisers | stephenfin: Hi! would you care to revisit https://review.openstack.org/#/c/522245/ ? Old driver bugfix change that has been sitting for some time and is not overly complex. | 14:36 |
zigo | So it tries to get https://api/compute/v2.1 | 14:36 |
*** sdake_ has joined #openstack-nova | 14:37 | |
zigo | But nova sends as reply a redirect to https://api/compute/v2.1/ (note the lasting slash ...) | 14:37 |
stephenfin | kaisers: Sure. I'd totally forgotten about that :) I'll take a look now | 14:37 |
kaisers | stephenfin: greate, thnx | 14:37 |
zigo | It's probably a miss-configuration of haproxy, apache, or something else... | 14:37 |
zigo | Though shouldn't nova-api reply with the microversion even without the lasting / ? | 14:37 |
zigo | bauzas: ^ | 14:42 |
bauzas | efried: gibi: I'm torn, given we only pass allocations down to the virt driver, how can the driver know whether the related RP is a root RP or a child ? | 14:42 |
bauzas | it should then look at the ProviderTree, right? | 14:42 |
efried | bauzas: You can't, until we implement https://review.openstack.org/#/c/597601/ | 14:43 |
efried | does spawn have access to the provider tree? | 14:43 |
gibi | bauzas: I think from the allocation alone you cannot figure out | 14:43 |
* bauzas whispers | 14:43 | |
bauzas | because then we have a problem with the reshape | 14:44 |
efried | The reshape comes from update_provider_tree, which *does* have access to the provider tree. | 14:44 |
gibi | bauzas: do you really need that information? can you simply look at the allocation, select the RP that has VGPU resource allocated from | 14:44 |
bauzas | gibi: how can I know this question ? | 14:44 |
bauzas | gibi: efried: I mean, for the moment, we use a mdev when creating an instance by https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6273 | 14:46 |
gibi | bauzas: you want to know which PGPU you are allocated the mdev from. You have the allocations dict. That dict has RPs with resources in it. The RP you need is the one that has VGPU resource allocated from | 14:46 |
bauzas | gibi: efried so I'm changing the above to check with RP was allocated | 14:46 |
bauzas | and then only either creating a new mdev or use an existing mdev from the pci device related to the RP | 14:47 |
stephenfin | kaisers: Done. Feel free to ping me whenever I do forget stuff. I'll simply ignore you if I'm too busy :) | 14:47 |
bauzas | gibi: I'm getting allocations, right? | 14:47 |
*** janki has quit IRC | 14:47 | |
stephenfin | lyarwood: Thanks for tackling that (y) It was on my TODO list | 14:47 |
* stephenfin closes one more tab | 14:48 | |
bauzas | gibi: so I guess I'm getting https://developer.openstack.org/api-ref/placement/?expanded=list-allocations-detail#response-example-1-28 | 14:48 |
gibi | yes | 14:48 |
bauzas | so how can I get the RP ? | 14:48 |
bauzas | I just have the RP UUID | 14:49 |
efried | the rp is in there, but only its uuid, not its name. | 14:49 |
*** hongbin has joined #openstack-nova | 14:49 | |
gibi | ok it is just the UUID | 14:49 |
gibi | what else you would need? then name ? | 14:49 |
efried | I thought we made a convention for setting the RP UUID based on some aspect of the VGPU | 14:49 |
*** tbachman has joined #openstack-nova | 14:49 | |
gibi | what connects an RP and an mdev? | 14:49 |
efried | gibi: he needs to be able to associate the RP with the PGPU | 14:49 |
efried | which he can do from the name for sure, because we used some kind of naming convention. | 14:49 |
gibi | efried: I see | 14:49 |
efried | but with just the uuid, unless we named the rp somehow after the uuid of the pgpu, we're fooked. | 14:50 |
gibi | efried: that is a problem as the provider summaries are not send down | 14:50 |
efried | Is the name in the provider summary? | 14:50 |
gibi | hm | 14:50 |
gibi | looking | 14:50 |
efried | doesn't look like it. | 14:50 |
TheJulia | jroll: That looks awesome. Thanks! | 14:51 |
*** NewBruce has joined #openstack-nova | 14:51 | |
*** awaugama has joined #openstack-nova | 14:51 | |
*** udesale has quit IRC | 14:52 | |
efried | bauzas: So unless we did in fact make some special way of assigning the RP UUID based on the identity of the pgpu, you're going to need to store that mapping locally somehow, which is ew. | 14:52 |
gibi | efried: you are right | 14:52 |
bauzas | efried: yeah it's a problem | 14:53 |
efried | you can store an instance variable in the virt driver from update_provider_tree, with a big TODO on it to figure out a better way. | 14:53 |
bauzas | we only have a convention for the name, not the uuid | 14:54 |
efried | And I'm wrong, it has nothing to do with gibi's spec | 14:54 |
jroll | TheJulia: cool, I'll un-wip. thanks | 14:54 |
efried | jroll: I'll bump my vote | 14:54 |
jroll | efried: thanks | 14:54 |
bauzas | efried: okay, I guess we have a big problem honestly | 14:54 |
bauzas | because I don't want to do this like that | 14:55 |
bauzas | if we want to select a node, then we need to know which one | 14:55 |
efried | um, maybe I'm wrong about being wrong. But it gives us something extra to think about in gibi's spec, which is that we have to have the information in the allocation itself, OR figure out some other way to get that info into spawn. | 14:55 |
gibi | efried: yeah, it is not the spec you linked, but I had similar issue to map pci device to RP, there it is done by the RP name, and there is an extra call in the compute/manager to get the RP name | 14:55 |
efried | but I think the libvirt problem is an extra one. | 14:55 |
efried | Perhaps we should consider amending the allocation structure to include the rp name. | 14:56 |
bauzas | so, again, I'll say we're blocked | 14:56 |
bauzas | unless the virt driver calls placement to know the RP tree | 14:56 |
efried | jaypipes: backscroll to :42:21 | 14:56 |
cdent | i'd be pretty opposed to include the name of the rp in the allocation unless we really can't come up with something | 14:56 |
efried | bauzas: Yes, that would be another "solution" that would be ew. | 14:56 |
cdent | why _not_ pass the summaries? | 14:56 |
cdent | or why not let the virt driver talk to placement? | 14:57 |
NewBruce | sean-k-mooney : back | 14:57 |
cdent | or any of several other options | 14:57 |
bauzas | I'll upload my change anyway | 14:57 |
efried | cdent: not pass the summaries because a) way too much information, unless we pare it down to just the summaries in the allocation, and b) the name isn't in the summaries anyway :P | 14:57 |
*** awalende has quit IRC | 14:58 | |
cdent | b) right, but putting it in the summaries would make some form of sense: it's a provider _summary_ after all | 14:58 |
cdent | however | 14:58 |
cdent | using meaningful names is a really super dangerous territory | 14:58 |
gibi | bauzas, efried: this is the PCI - RP mapping solution I come up with https://review.openstack.org/#/c/623543/23/nova/compute/manager.py It works as the pci claim can match devices by abitrary tag | 14:58 |
*** maciejjozefczyk has quit IRC | 14:58 | |
efried | cdent: we need some way of mapping RP to "real thing". | 14:58 |
efried | I know: arbitrary metadata on resource_provider! | 14:58 |
cdent | which isn't placements job | 14:58 |
*** awalende has joined #openstack-nova | 14:58 | |
cdent | mapping rp to real thing is the either nova or the virt drivers job | 14:59 |
efried | gibi: Right, so you're storing the mapping locally in memory. | 14:59 |
gibi | efried: calculating the mapping by calling placement during boot | 14:59 |
gibi | efried: nothing stored between boots | 14:59 |
efried | cdent: Which is what I suggested initially as the workaround. I though it was ew, but I'm kind of coming around to the idea. | 14:59 |
efried | you don't need to store anything between boots; you construct the information in update_provider_tree in any case. | 15:00 |
efried | that's where you would save the mapping. | 15:00 |
efried | you can't spawn untile you've update_provider_treeD | 15:00 |
bauzas | gibi: you're calling placement to get the RP name, right? | 15:00 |
gibi | bauzas: yes | 15:00 |
*** eharney has joined #openstack-nova | 15:00 | |
efried | Could pass the provider_tree into spawn. That would let you look up the name. | 15:00 |
gibi | bauzas: https://review.openstack.org/#/c/623543/23/nova/compute/manager.py@2138 | 15:00 |
bauzas | gibi: yeah that's what I saw | 15:01 |
bauzas | it wasn't a question actually, more a clarification :) | 15:01 |
bauzas | gibi: but here I'm working on the driver side | 15:01 |
bauzas | I'm not sure I have the report client | 15:01 |
*** cfriesen has joined #openstack-nova | 15:01 | |
efried | would have to be a read-only/copy of the provider tree. | 15:01 |
gibi | bauzas: I know, you have to pass some extra info to the driver anyhow | 15:01 |
bauzas | and I think we basically said no | 15:01 |
* jaypipes reads back... | 15:01 | |
bauzas | I mean, we agreed on not having the virt driver calling placement for various reasons | 15:02 |
efried | yeah | 15:02 |
openstackgerrit | Boris Bobrov proposed openstack/nova master: Add configs for AMD SEV https://review.openstack.org/636318 | 15:02 |
*** udesale has joined #openstack-nova | 15:02 | |
*** sapd1 has joined #openstack-nova | 15:02 | |
bauzas | so, I'm stuck with finding a way to pass down the providertree structure down the wire | 15:02 |
*** awalende has quit IRC | 15:02 | |
efried | bauzas: You have the provider tree in the compute manager. | 15:02 |
bauzas | just because I want to know which physical resource is there | 15:02 |
efried | no need to make calls to placement. | 15:03 |
kashyap | time git fetch gerrit | 15:03 |
bauzas | efried: correct, I could amend https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3106 | 15:03 |
*** moshele has joined #openstack-nova | 15:04 | |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: [WIP] libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein" https://review.openstack.org/632507 | 15:04 |
efried | bauzas: Right. Add provider_tree. Get it the same way we get it before update_provider_tree (i.e. make a copy of it). | 15:04 |
bauzas | there could be race conditions, right? | 15:04 |
efried | Not that wouldn't cause blowups anyway. | 15:04 |
efried | E.g. if I remove the pgpu rp between allocating and spawning, I'm effed in any case. | 15:05 |
*** markvoelker has quit IRC | 15:05 | |
gibi | efried: but you cannot do that without removing the allocation as well, which breaks nova as that allocation is created by nova | 15:06 |
efried | gibi: Swhat I'm saying, I don't see how you could contrive a race condition that would make the provider_tree be out of sync with the allocations without already having broken something else in the process. | 15:07 |
gibi | efried: I agree | 15:07 |
efried | I mean, some other unrelated provider could be out of sync I suppose. But I would hope that wouldn't matter. | 15:07 |
bauzas | efried: to be honest, I don't care of allocations | 15:07 |
bauzas | efried: I only care of the RP tree | 15:07 |
efried | well, you need the allocation to determine the rp | 15:07 |
bauzas | it's already there, and I trust it | 15:08 |
efried | And if you have more than one vgpu allocated, you need to know that. | 15:08 |
bauzas | ideally, it'd be waaaaay better and cleaner if I was getting the RP structure from placement as part of the message it returns me | 15:08 |
efried | I assume/hope you're getting that information from the allocation and not from somewhere else. | 15:08 |
bauzas | because my cache could be stale | 15:08 |
efried | bauzas: But what do you mean by "structure"? | 15:09 |
efried | It doesn't help you simply to know that the provider of the vgpu resource is a "child". You will eventually have to know *which* child. | 15:09 |
bauzas | efried: the mdev allocation logic is made there https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6251 | 15:09 |
bauzas | efried: right | 15:09 |
*** hongbin has quit IRC | 15:09 | |
bauzas | efried: I want to ideally know the PCI device | 15:09 |
bauzas | but this mapping is under my responsibility | 15:09 |
efried | right | 15:10 |
bauzas | so I need to know which RP is allocated | 15:10 |
bauzas | and then map the PCI ID thanks to the convention | 15:10 |
efried | Well, you know which *RP* is allocated. But you don't know its name, only its UUID. | 15:10 |
bauzas | right, that's what we're discussing since 15 mins | 15:10 |
bauzas | well, 20 even | 15:10 |
efried | Right, I think we've come full circle. If you had the provider_tree, you would be sorted. Otherwise you need some other way to maintain a mapping. | 15:11 |
efried | jaypipes: once you've caught up: Would you support adding a provider_tree arg to spawn()? | 15:11 |
bauzas | I have one thought | 15:11 |
bauzas | we always push information down to the driver | 15:12 |
bauzas | can we have the virt driver getting info from the service ? | 15:12 |
bauzas | I mean a getter | 15:12 |
*** hongbin has joined #openstack-nova | 15:12 | |
bauzas | I don't like changing our interfaces (at least spawn()) for a very specific usecase | 15:12 |
bauzas | and 99% of deployments won't care of VGPUs | 15:13 |
bauzas | but I'm not a virt driver specialist | 15:13 |
efried | you mean introducing a protected static final ComputeDriver.get_provider_tree() method? | 15:14 |
bauzas | I dunno, thinking out loud | 15:14 |
gibi | bauzas: for pci we have InstancePciRequest for NUMA we have InstanceNumaTopology for VGPU we have no object to hold this kind of information | 15:14 |
efried | hell, update_provider_tree could save off its provider tree as an instance var before it returns | 15:14 |
bauzas | snap, I need to taxi my kids | 15:15 |
bauzas | efried: that's called a cache | 15:15 |
efried | so it would be like the virt driver saving a mapping, except it doesn't need to do any special work to construct it - it's already there. | 15:15 |
bauzas | efried: and we know it can be a problem | 15:15 |
bauzas | I need to think about this | 15:15 |
bauzas | anyway, I need to drop for a couple of hours | 15:16 |
bauzas | \o | 15:16 |
jaypipes | efried, bauzas, gibi, cdent: I have said for over a year that the virt driver will need to keep a mapping of the RP UUID to whatever name it uses to refer to that RP internally (mdev, PCI address, whatever the heck IBM does, etc) | 15:20 |
cdent | yup | 15:20 |
cdent | virt driver's job | 15:21 |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: [WIP] libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein" https://review.openstack.org/632507 | 15:21 |
gibi | jaypipes: while I agree it is a bit more complicate if the RP is created outside of the virt driver like when neutron and cyborg creates the RP but the virt driver consumes the RP | 15:22 |
gibi | clearly for VGPU your suggestion works | 15:23 |
*** TxGirlGeek has joined #openstack-nova | 15:23 | |
sean-k-mooney | gibi: well in the cyboge/neutron case nova does not need to consomne the allocation | 15:24 |
sean-k-mooney | it just needs to pass it back to cyborg/neutorn so can they do the transaltion then | 15:25 |
gibi | sean-k-mooney: for sriov ports, the virt driver still responsible to select the PCI VF | 15:25 |
sean-k-mooney | yes this is true but currently they are not in placement | 15:26 |
gibi | sean-k-mooney: and the PCI VF should come from the same RP | 15:26 |
gibi | as the bandwidth | 15:26 |
sean-k-mooney | the same RP as the bandwith i am assuming | 15:26 |
sean-k-mooney | ya | 15:26 |
gibi | so transitively the virt driver needs to know which RP is used for the bandwidth | 15:26 |
*** liuyulong has quit IRC | 15:26 | |
sean-k-mooney | well the bandwith woudl be an inventor on teh PF RP and we soudl similar have an inventory of VFs | 15:27 |
sean-k-mooney | so as long as the VF resust and bandwith request are in teh same requst group we shoudl be ok the issue is that would require 2 services to manage different inventoies in the same RP | 15:28 |
sean-k-mooney | which we dont support today | 15:28 |
*** moshele has quit IRC | 15:28 | |
gibi | sean-k-mooney: yeah, it is solved when VF inventory is moved to the placement | 15:28 |
sean-k-mooney | i personally had assumed that the resouce track would evolve to sotre the placement RP mappings | 15:29 |
efried | jaypipes: So that mapping has to be created in update_provider_tree (and could in fact *be* the provider tree created therein) and retrievable from spawn. | 15:29 |
*** gbarros has joined #openstack-nova | 15:30 | |
efried | If we state it that way ^, then I guess we can leave it up to each virt driver how it wants to create and store the mapping. Shoving the provider tree into an instance var seems like as easy a way as any. | 15:30 |
*** priteau has joined #openstack-nova | 15:31 | |
*** sapd1 has quit IRC | 15:32 | |
jaypipes | efried: it could, yes. though there's nothing in what I said that *requires* it be that way. After all, the PCI device management code (gawd save us all), external Neutron agents (per gibi's series) and other things could easily keep a map of UUID -> internal name in whatever way they wanted. Shit, I believe I proposed at some point that a simple YAML file on disk storing the map of UUID -> internal device/provider name would be just fine | 15:33 |
openstackgerrit | Ralf Haferkamp proposed openstack/nova master: compute: reset instance events on reload https://review.openstack.org/420026 | 15:34 |
*** NewBruce has quit IRC | 15:39 | |
*** gbarros has quit IRC | 15:40 | |
*** liuyulong has joined #openstack-nova | 15:41 | |
*** Luzi has quit IRC | 15:41 | |
*** macza has joined #openstack-nova | 15:42 | |
sean-k-mooney | jaypipes: the other option which im not a huge fan of would be just to set the service specific internal name in the RP name field | 15:43 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support server create with ports having resource request https://review.openstack.org/636360 | 15:44 |
sean-k-mooney | jaypipes: that does not really work well in the general case and dose not work at all in the inter service case hence why im not a huge fan of that. | 15:44 |
*** maciejjozefczyk has joined #openstack-nova | 15:44 | |
kashyap | mdbooth: If you get some time, I'd really appreciate some libvirt driver and sensible eyes in general on this: https://review.openstack.org/#/c/632507/ | 15:45 |
kashyap | mdbooth: I know it not fully complete, but the serial / char dev code is tripping me up a bit. | 15:45 |
kashyap | (Funnily enough, for me, the bug that is referenced in the code that's being deleted ... was a bug I filed 5 years ago) | 15:46 |
sean-k-mooney | in the sriov case if the RP name was the PF adress neutron and nova would agree on it at least | 15:46 |
sean-k-mooney | but really that is just "bit stealing" rather and a convention rather then a dedicated mechanium | 15:46 |
*** moshele has joined #openstack-nova | 15:47 | |
jaypipes | sean-k-mooney: that's basically what gibi's doing in https://review.openstack.org/#/c/623543/23/nova/compute/manager.py@2138 | 15:48 |
efried | sean-k-mooney: We already do that in the vgpu code too. It doesn't help, because the allocation doesn't include the rp name. | 15:48 |
sean-k-mooney | i mean it will work as long as the service that need to share an RP agree on on how to chose the name | 15:48 |
jaypipes | the issue is that I see that as a fragile solution that depends on the name structure being codified and not changing ever. as soon as the device naming convention changes, everything would blow up. | 15:49 |
cdent | ++ | 15:49 |
sean-k-mooney | efried: well you could lookup the rp by its uuid if you need the info form the allocation | 15:49 |
efried | shrug, be like saying if the mapping algorithm changes everything blows up | 15:49 |
efried | sean-k-mooney: This is where we started. | 15:49 |
efried | sean-k-mooney: Where do you look up the rp? | 15:49 |
sean-k-mooney | in placement | 15:49 |
efried | in placement? no, because we've said we don't want virt talking directly to placement | 15:49 |
efried | in the provider tree? Sure, but you have to get the provider tree somehow | 15:50 |
jaypipes | providers should always be *externally referenced* (i.e. any time two services need to exchange info about a provider) using a UUID. internally, within a service, the service should be responsible for caching an internal name -> UUID map. | 15:50 |
efried | in a mapping? Sure, but the mapping needs to be saved by the virt driver | 15:50 |
sean-k-mooney | if the virt dirver create the RP with a specific name in know it could take the uuid for the allcoation and look it up and get the name back | 15:50 |
efried | by talking to placement? No, see above. By looking in a mapping? Sure, see above. | 15:50 |
sean-k-mooney | efried: it only need to be saved for efficency. | 15:51 |
efried | sean-k-mooney: The proposed solution is for update_provider_tree to save a mapping of RP UUID to <something> and then for spawn to retrieve that information. The easiest thing would be for update_provider_tree simply to save the provider_tree as an instance var on the virt driver. Then spawn can just ask that guy for the provider with a given uuid and get the name. Since the name is already generated by convention, it can use it | 15:52 |
*** mriedem_snow is now known as mriedem | 15:52 | |
*** erlon has joined #openstack-nova | 15:52 | |
sean-k-mooney | efried: sure that makes sense | 15:53 |
efried | if we really don't like using the naming convention, upt could save some other arbitrarily-formatted dict as an instance var, to the same end. | 15:53 |
sean-k-mooney | but making that internal mapping to the uuid stable it the respociblity of the service consuming placment in that case not placement | 15:53 |
efried | yes, update_provider_tree is that thing in either case. | 15:54 |
efried | even if spawn got to request the provider record from placement and use the naming convention, it was still upt's responsibility to name the provider appropriately. | 15:54 |
sean-k-mooney | ya | 15:55 |
efried | but I prefer the idea of upt caching^wahem, saving something as an instance var. | 15:55 |
sean-k-mooney | isnt this partly solved by https://review.openstack.org/#/c/612497/ also | 15:55 |
sean-k-mooney | the provider ymal file | 15:56 |
efried | sean-k-mooney: It could be, but I kind of hate the idea of making it the operator's responsibility | 15:56 |
efried | as long as there's reasonable default behavior if that file is missing (or composed poorly) it would be okay. | 15:56 |
sean-k-mooney | well there is no reason the virtdriver could not generate it on first boot if its not provided | 15:56 |
sean-k-mooney | and then operator could add to it | 15:57 |
efried | also ew. | 15:57 |
sean-k-mooney | that said that might mess with som config managment stuff | 15:57 |
efried | Now what if I hot-(un)plug devices or otherwise change resources. | 15:57 |
efried | yeah | 15:57 |
efried | now I'm forced to hand-edit the file after that first time. Or remember to blow it away so it can be regenerated, but then I lose my customizations. | 15:57 |
sean-k-mooney | you first update teh yaml file then send a sig hup to the compute agent then you remove the hardware | 15:58 |
efried | as opposed to just... removing the hardware and everything works. | 15:58 |
sean-k-mooney | that is not really a thing today | 15:59 |
efried | neither is generic numa affinity | 15:59 |
sean-k-mooney | i mean i "might" work in some cases | 15:59 |
efried | Seems like all of this is stuff we're striving for. | 15:59 |
*** mrch_ has joined #openstack-nova | 16:00 | |
efried | The whole reason update_provider_tree happens on a loop is so that it can, in the future, automatically adjust the provider tree when stuff like this happens. | 16:00 |
sean-k-mooney | i think https://etherpad.openstack.org/p/nova-ptg-train everyting from lin 7-30 is very related | 16:00 |
*** maciejjozefczyk has quit IRC | 16:00 | |
*** erlon has quit IRC | 16:00 | |
*** mrch_ has quit IRC | 16:01 | |
*** mrch_ has joined #openstack-nova | 16:01 | |
*** mrch_ has quit IRC | 16:01 | |
*** mrch_ has joined #openstack-nova | 16:01 | |
efried | In fact, I think that it might "just work" after Sylvain's vgpu stuff lands to hot-(un)plug a GPU. | 16:01 |
efried | if you unplug one that has allocations against it, you can expect problems; but otherwise... | 16:01 |
efried | and even then, I think we've accounted for that by setting reserved=total. | 16:02 |
*** mrch_ has quit IRC | 16:02 | |
*** NewBruce has joined #openstack-nova | 16:02 | |
*** mrch_ has joined #openstack-nova | 16:02 | |
sean-k-mooney | well there were edgecases like this https://review.openstack.org/#/c/626381/ | 16:02 |
*** markvoelker has joined #openstack-nova | 16:03 | |
sean-k-mooney | if the device you remove had allocation against it we cant remove it form placemetn until those allocation are also removed | 16:03 |
*** NewBruce has quit IRC | 16:03 | |
sean-k-mooney | so the auto updating is greate but there are also dragons lurking there | 16:03 |
*** lpetrut has quit IRC | 16:03 | |
*** NewBruce has joined #openstack-nova | 16:04 | |
efried | right, like I said, that case is accounted for (at least I definitely remember this code in the xen vgpu patches) where, if a card goes away, but has allocations against it, we set reserved=total so no futher allocations can be made against it, until all the existing allocations go away, at which point we can delete the provider. | 16:05 |
sean-k-mooney | ya so we have solution for this edgecase and likely other but its a little more involved then you would assume at first glance | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Calculate RequestGroup resource provider mapping https://review.openstack.org/616239 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Fill the RequestGroup mapping during schedule https://review.openstack.org/619528 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Pass resource provider mapping to neutronv2 api https://review.openstack.org/616240 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Recalculate request group - RP mapping during re-schedule https://review.openstack.org/619529 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding https://review.openstack.org/569459 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Remove port allocation during detach https://review.openstack.org/622421 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Record requester in the InstancePCIRequest https://review.openstack.org/625310 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add pf_interface_name tag to passthrough_whitelist https://review.openstack.org/625311 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF https://review.openstack.org/623543 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support server create with ports having resource request https://review.openstack.org/636360 | 16:06 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support server create with ports having resource request https://review.openstack.org/636360 | 16:07 |
*** wwriverrat has quit IRC | 16:07 | |
mriedem | lyarwood: mdbooth: +2 on https://review.openstack.org/#/c/551302/ | 16:09 |
mriedem | dansmith: ^ | 16:09 |
mdbooth | mriedem: Thanks! | 16:11 |
*** NewBruce has quit IRC | 16:13 | |
*** wwriverrat has joined #openstack-nova | 16:14 | |
*** dpawlik has joined #openstack-nova | 16:14 | |
lyarwood | mriedem: cheers | 16:15 |
*** moshele has quit IRC | 16:15 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support server create with ports having resource request https://review.openstack.org/636360 | 16:16 |
efried | aspiers, mriedem: Were either of you planning to resurrect https://review.openstack.org/#/c/538498/ ? | 16:17 |
sean-k-mooney | mriedem: when you have a sec can you let me know if the responces in https://review.openstack.org/#/c/624842/8//COMMIT_MSG make sense. if you are happy with them i can resping with that info added to the commit and your other comments addressed in an hour or so | 16:17 |
*** wwriverrat has quit IRC | 16:18 | |
mriedem | efried: i'm not working on it | 16:18 |
*** wwriverrat has joined #openstack-nova | 16:18 | |
stephenfin | efried: Comments on https://review.openstack.org/#/c/582392/ | 16:19 |
efried | ... | 16:19 |
mriedem | sean-k-mooney: i guess.... | 16:19 |
mriedem | setting the vif mac to 0s works depending on the underlying driver | 16:20 |
mriedem | is the summary i'm getting | 16:20 |
sean-k-mooney | yes | 16:20 |
mriedem | i'm not sure how the minimum libvirt version plays into any of that | 16:20 |
*** mrch_ has quit IRC | 16:20 | |
sean-k-mooney | oh with libvirt 3.2 it takes care of all of this | 16:20 |
mriedem | set_vf_interface_vlan doesn't document it's expected exit codes at all either so i had no idea | 16:20 |
sean-k-mooney | so we can delete this code | 16:20 |
*** ccamacho has quit IRC | 16:23 | |
aspiers | efried: I started to rebase it and got confused by the conflicts through lack of familiarity with the code, so I installed git-irebase and started to incrementally rebase it one commit at a time, so that the conflicts are smaller and there's more chance of me understanding how to handle them correctly | 16:23 |
*** ccamacho has joined #openstack-nova | 16:24 | |
efried | aspiers: tldr you're working on it? | 16:24 |
aspiers | well, trying ;-) | 16:24 |
aspiers | I reserve the right to give up through incompetence though | 16:24 |
sean-k-mooney | mriedem: the behavior chagne 2 weeks ago in https://github.com/openstack/nova/commit/13e283cc82e7cab564850501f9781385fb7b3f68 ill have to respin anyway now. | 16:25 |
aspiers | efried, mriedem: however I've just taken a brief pause to review the SEV patches my colleague just posted | 16:25 |
efried | aspiers: Just lmk if you do decide to give up on it, and I'll take over. I want it to get finished one way or another. (No particular hurry, just don't want it stalled indefinitely.) | 16:25 |
mriedem | sean-k-mooney: behavior change? | 16:25 |
aspiers | efried: sure thing | 16:26 |
efried | and thank you. | 16:26 |
aspiers | yw | 16:26 |
mriedem | efried: have you taken a gander at https://review.openstack.org/#/c/616239/ yet? that's gibi's "Calculate RequestGroup resource provider mapping" change which i figure you're the best person to review that | 16:27 |
sean-k-mooney | mriedem: actully no never mind the exit_code = [0, 2, 254] is now donw internally in another fucntion instead of pased in | 16:27 |
*** manjeets has joined #openstack-nova | 16:27 | |
sean-k-mooney | never mind i dont need to respin due to a behavior change but i will to adress you comments | 16:27 |
efried | mriedem: It's on my list, but I admit my list is currently more of a... cloud. I can bump it. | 16:27 |
mriedem | same | 16:28 |
dansmith | mriedem: lyarwood mdbooth: Tell me why we have to do the partial() and I'll remove my -1, otherwise that seems nuts to me | 16:28 |
mriedem | b/c the virt driver calls the post method | 16:29 |
mriedem | and doesn't have the source bdms | 16:29 |
mriedem | it's a callback | 16:29 |
dansmith | mriedem: right, but we can modify that interface | 16:30 |
mriedem | and update all virt drivers to lookup and pass the bdms? | 16:30 |
dansmith | eff yeah, otherwise you're just stacking partial()s on top of each other until the end of time | 16:30 |
openstackgerrit | Eric Fried proposed openstack/nova master: Use tox 3.1.1 fixes https://review.openstack.org/582392 | 16:30 |
openstackgerrit | Eric Fried proposed openstack/nova master: tox: Don't write byte code (maybe) https://review.openstack.org/636378 | 16:30 |
dansmith | and nobody actually knows how/where all those things come from | 16:31 |
mriedem | dansmith: actually it's already too late | 16:31 |
mriedem | once the virt driver calls the post method, | 16:31 |
mriedem | the bdms have been modified via pre_live_migration on the dest | 16:31 |
mriedem | which is why he's looking them up before call pre_live_migration | 16:31 |
dansmith | mriedem: too late for stacking the partials you mean right? | 16:31 |
efried | stephenfin: fixed, good suggestion. | 16:31 |
mriedem | too late for the virt driver on the source host to lookup the bdms to pass to _post_live_migration because that happens *after* pre_live_migration has modified the bdms | 16:32 |
dansmith | mriedem: all I'm saying is pass these things to the method you want to have pass them back, instead of wrapping | 16:32 |
dansmith | mriedem: I'm not saying have the virt driver look them up | 16:32 |
*** mrch_ has joined #openstack-nova | 16:32 | |
efried | mriedem: don't forget to send a courtesy note to the ML so OOT drivers aren't blindsided :* | 16:32 |
mriedem | so pass through the driver.live_migration method | 16:32 |
dansmith | mriedem: right | 16:33 |
mriedem | i don't feel strongly either way, but this has to be backported and as noted that interface change will require all drivers to be updated lockstep | 16:33 |
*** tbachman has quit IRC | 16:33 | |
* lyarwood reads back | 16:34 | |
mriedem | so if we were going to change the interface, i'd do it in a follow up which isn't backported | 16:34 |
dansmith | mriedem: well, then backport this and fix it after if you want | 16:34 |
lyarwood | ack, I can take that if mdbooth can't | 16:34 |
dansmith | I would also define the partial right before you make the call to the driver, | 16:34 |
mdbooth | dansmith: TBH, I actually like the partial better, and in this context I'd partial all the things. | 16:34 |
dansmith | mdbooth: I have no doubt that you do | 16:35 |
*** markvoelker has quit IRC | 16:35 | |
mriedem | rollback still has to be fixed as well | 16:35 |
mriedem | for the same issue | 16:35 |
mdbooth | dansmith: The reason is that is simplifies the interface between compute and the driver by removing arguments which are purely local the the calling method. | 16:35 |
mdbooth | So the callback method only passes relevant data, and everything else remains local. | 16:35 |
mdbooth | However, that isn't why I used one here. Here's it's just so I didn't have to change the interface. | 16:36 |
dansmith | mdbooth: yeah, I get the obvious argument for it, I just think it hurts readability and understandability if you're just focused on that one method | 16:36 |
dansmith | it'd be slightly better if you define the partial and all the extra things you're going to pass down right before you make the call to the driver, | 16:36 |
openstackgerrit | Merged openstack/nova stable/pike: Add functional regression test for bug 1794996 https://review.openstack.org/623358 | 16:36 |
openstack | bug 1794996 in OpenStack Compute (nova) pike "_destroy_evacuated_instances fails and kills n-cpu startup if lazy-loading flavor on a deleted instance" [High,In progress] https://launchpad.net/bugs/1794996 - Assigned to Matt Riedemann (mriedem) | 16:36 |
openstackgerrit | Merged openstack/nova stable/pike: Fix InstanceNotFound during _destroy_evacuated_instances https://review.openstack.org/623359 | 16:37 |
openstackgerrit | Merged openstack/nova master: Trivial: reorder hashes according to object_hashes.txt https://review.openstack.org/636261 | 16:37 |
dansmith | but if I were to add something else in a year, we could easily end up with a partial(partial(thing)) because someone didn't read 50 lines up | 16:37 |
*** udesale has quit IRC | 16:37 | |
dansmith | using it to make a backport cleaner is exactly the hacky situation where I think the benefit outweighs the cost, but going forward in master, it's just a confusing hack, IMHO | 16:38 |
*** helenafm has quit IRC | 16:40 | |
*** mrch_ has quit IRC | 16:40 | |
*** jmlowe has joined #openstack-nova | 16:43 | |
*** mvkr has quit IRC | 16:44 | |
*** tbachman has joined #openstack-nova | 16:45 | |
*** moshele has joined #openstack-nova | 16:51 | |
*** ccamacho has quit IRC | 16:53 | |
*** moshele has quit IRC | 16:54 | |
*** TxGirlGeek has quit IRC | 16:57 | |
*** sridharg has quit IRC | 16:58 | |
*** tbachman has quit IRC | 16:58 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Don't set bandwidth limits for vhostuser, hostdev interfaces https://review.openstack.org/635170 | 16:58 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove get_config_vhostuser https://review.openstack.org/565471 | 16:58 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Validate bandwidth configuration for other VIF types https://review.openstack.org/636383 | 16:58 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Further de-dupe os-vif VIF tests https://review.openstack.org/636384 | 16:58 |
stephenfin | jangutter, sean-k-mooney: Per our discussion on IRC last week ^ | 16:58 |
lyarwood | dansmith: so just to confirm, you'd be okay with https://review.openstack.org/#/c/551302/ as a backportable hack if the partial was moved down and we followed up with something different in master? | 16:59 |
dansmith | lyarwood: yeah | 16:59 |
*** hongbin has quit IRC | 16:59 | |
lyarwood | dansmith: kk, I'll move that down now then, thanks | 17:00 |
*** sdake_ has quit IRC | 17:01 | |
*** tbachman has joined #openstack-nova | 17:01 | |
stephenfin | mriedem, alex_xu: Any chance you could take a look at this again as some point this week? https://review.openstack.org/#/c/631038/ | 17:03 |
*** sdake has joined #openstack-nova | 17:03 | |
*** sapd1 has joined #openstack-nova | 17:04 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: Avoid redundant initialize_connection on source post live migration https://review.openstack.org/551302 | 17:04 |
*** prometheanfire has joined #openstack-nova | 17:05 | |
lyarwood | dansmith: ^ there we go, thanks again | 17:05 |
prometheanfire | does nova require pci_passthough_whitelist to be in the [pci] section in rocky or can it remain in [DEFAULT]? | 17:06 |
*** hamzy has quit IRC | 17:06 | |
*** rpittau has quit IRC | 17:08 | |
*** tesseract has quit IRC | 17:09 | |
*** liuyulong has quit IRC | 17:09 | |
stephenfin | prometheanfire: It will be read from DEFAULT but it should be moved as we'll eventually remove the aliases | 17:10 |
stephenfin | prometheanfire: You'll get a warning. Also, the option in the '[pci]' group is simply called 'passthrough_whitelist', in case you'd missed that | 17:10 |
prometheanfire | ya | 17:11 |
prometheanfire | I'm working on a os_nova backport to stable/rocky, stein is already updated | 17:11 |
sean-k-mooney | it was technically moved in queens i think | 17:11 |
prometheanfire | ya, docs show it in pike too | 17:11 |
prometheanfire | https://docs.openstack.org/nova/pike/admin/pci-passthrough.html | 17:11 |
*** moshele has joined #openstack-nova | 17:11 | |
sean-k-mooney | prometheanfire: oh it was move in ocata | 17:12 |
prometheanfire | no one uses that :P | 17:12 |
sean-k-mooney | you would be surprised how many people are on Newton still | 17:13 |
sean-k-mooney | there are still people on liberty but yes few people use ocata | 17:13 |
sean-k-mooney | its EOL upstream anyway | 17:13 |
*** sapd1 has quit IRC | 17:14 | |
prometheanfire | it was a joke :P | 17:14 |
sean-k-mooney | but anyway https://github.com/openstack/nova/blob/master/nova/conf/pci.py#L84 | 17:14 |
sean-k-mooney | we still support both but we really shoudl drop support for [DEFAULT] at this point | 17:15 |
prometheanfire | ya | 17:15 |
sean-k-mooney | prometheanfire: if you are moving it yyou shoudl also move the pci alias too | 17:16 |
*** mvkr has joined #openstack-nova | 17:16 | |
sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/conf/pci.py#L23 | 17:17 |
prometheanfire | alias wasn't even defined/used until stein (I'm adding it in the backports) | 17:18 |
sean-k-mooney | they predate kolla by years | 17:19 |
sean-k-mooney | you could use them via the config override mechanisium | 17:19 |
sean-k-mooney | the main issue was you could only have 1 alsi tuse to how the config_merger ansible plugin worked | 17:20 |
sean-k-mooney | it does not support MultiStrOpt fields | 17:20 |
sean-k-mooney | unlike the whitelist the alais does not support a json list format | 17:21 |
*** jmlowe has quit IRC | 17:21 | |
*** _pewp_ has quit IRC | 17:27 | |
*** _pewp_ has joined #openstack-nova | 17:28 | |
prometheanfire | does nova support out of tree drivers? (xenxl) | 17:30 |
*** markvoelker has joined #openstack-nova | 17:32 | |
*** Shilpa has joined #openstack-nova | 17:32 | |
melwitt | o/ | 17:33 |
sean-k-mooney | prometheanfire: nova does nto support out of tree drivers in general no | 17:33 |
*** gryf has quit IRC | 17:34 | |
*** ShilpaSD has quit IRC | 17:35 | |
*** whoami-rajat has quit IRC | 17:36 | |
*** moshele has quit IRC | 17:36 | |
prometheanfire | sean-k-mooney: ok, just curious | 17:36 |
*** spsurya has quit IRC | 17:40 | |
*** idlemind has joined #openstack-nova | 17:42 | |
stephenfin | sean-k-mooney, adrianc: Finished SR-IOV series review and left some comments. Ping me if/when there's more to look at. | 17:44 |
* stephenfin heads home | 17:44 | |
*** gbarros has joined #openstack-nova | 17:44 | |
sean-k-mooney | stephenfin: cool i will try to adress them later today | 17:44 |
*** sdake has quit IRC | 17:44 | |
*** mriedem is now known as mriedem_afk | 17:45 | |
sean-k-mooney | stephenfin: it can wait till tomorow but we need to review https://review.openstack.org/#/c/636061/ tool the ironic and neutron spec were approved after m2 so apparently this is intending to target stien. | 17:47 |
*** sdake has joined #openstack-nova | 17:47 | |
*** gyee has joined #openstack-nova | 17:47 | |
sean-k-mooney | stephenfin: so 1 we need to validate it makes sense and 2 merge it and releas a new os-vif version before teh 27th. i was hoping to do a new release on friday week anyway but i would have prefered to at least have a nova/os-vif blueprint for this earlier | 17:48 |
prometheanfire | sean-k-mooney: they don't think they'd be able to support the driver fully upstream is the reason I asked | 17:48 |
sean-k-mooney | prometheanfire: so technically you can use out of tree drivers but they are not offically supported by nova (since they are out of tree) and nova may break them at any time | 17:49 |
sean-k-mooney | historically i dont know how volitile that interface has been | 17:50 |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Plumbing required in servers ViewBuilder to construct partial results https://review.openstack.org/635146 | 17:50 |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells https://review.openstack.org/591657 | 17:50 |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells Documentation https://review.openstack.org/635147 | 17:50 |
sean-k-mooney | prometheanfire: so there is noting preventign kolla from supproting deploying them but they may break | 17:50 |
sean-k-mooney | prometheanfire: being out of tree means that maintaining comptablity with nova fall entirly onto the xenxl team. but yes being intree requires ci and other testing requirement so its non trivial | 17:52 |
*** dpawlik has quit IRC | 17:53 | |
prometheanfire | sean-k-mooney: not asking for osa (or kolla), that comes as the cost of out of tree :D | 17:54 |
sean-k-mooney | well being out of tree has benifits too | 17:54 |
sean-k-mooney | if the hypervior you are intergrating with is evolving quickly you can add features and adapt your driver much faster out of tree | 17:55 |
sean-k-mooney | so it best to bring it in tree when it stable and you have hit the baseline of feature you want o support | 17:55 |
*** tbachman has quit IRC | 17:56 | |
*** derekh has quit IRC | 17:56 | |
prometheanfire | yep | 17:56 |
*** dtantsur is now known as dtantsur|afk | 17:58 | |
odyssey4me | prometheanfire OSA has implementations of out of tree drivers for nova already - for powervm IIRC | 17:58 |
prometheanfire | odyssey4me: good point I'll point him there | 17:59 |
odyssey4me | nova supports the use of out of tree drivers, but obviously the core team for nova does not 'support' those drivers - that'd be for the out of tree driver team | 17:59 |
odyssey4me | OSA also has nova-lxd driver support | 17:59 |
*** artom has quit IRC | 18:04 | |
*** markvoelker has quit IRC | 18:06 | |
*** dpawlik has joined #openstack-nova | 18:06 | |
*** agopi is now known as agopi|food | 18:06 | |
*** panda is now known as panda|off | 18:06 | |
*** wwriverrat has quit IRC | 18:09 | |
*** tssurya has quit IRC | 18:09 | |
openstackgerrit | Paul-Emile Element proposed openstack/nova master: Add emulated TPM support to Nova https://review.openstack.org/631363 | 18:09 |
*** wwriverrat has joined #openstack-nova | 18:10 | |
*** dpawlik has quit IRC | 18:10 | |
*** tbachman has joined #openstack-nova | 18:13 | |
prometheanfire | the pci_whitelist can accept a value like passthrough_whitelist= "{ "vendor_id": "10de", "product_id": "15f7" }" ? | 18:17 |
prometheanfire | or does it need to look like { "vendor_id": "10de", "product_id": "15f7" } | 18:17 |
*** dpawlik has joined #openstack-nova | 18:23 | |
sean-k-mooney | the later i think | 18:24 |
*** imacdonn has quit IRC | 18:24 | |
sean-k-mooney | prometheanfire: there a a bunch of examples here https://github.com/openstack/nova/blob/master/nova/conf/pci.py#L128-L162 | 18:25 |
*** imacdonn has joined #openstack-nova | 18:25 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Move finish_resize.(start|end) notifications to helper method https://review.openstack.org/635079 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid https://review.openstack.org/614012 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.openstack.org/614035 | 18:28 |
mriedem_afk | dansmith: there will be some easy pickings at the bottom of this series ^ | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Instance.hidden field https://review.openstack.org/631123 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add TargetDBSetupTask https://review.openstack.org/627892 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask https://review.openstack.org/631581 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask https://review.openstack.org/633853 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method https://review.openstack.org/621313 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add prep_snapshot_based_resize_at_dest compute method https://review.openstack.org/633293 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtDestTask https://review.openstack.org/627890 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.openstack.org/634832 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtSourceTask https://review.openstack.org/627891 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add finish_snapshot_based_resize_at_dest compute method https://review.openstack.org/635080 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask https://review.openstack.org/635646 | 18:28 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.openstack.org/635668 | 18:28 |
openstackgerrit | Merged openstack/nova master: Follow up (#2) for the bw resource provider series https://review.openstack.org/634767 | 18:30 |
*** agopi|food is now known as agopi | 18:30 | |
*** eharney has quit IRC | 18:33 | |
*** panda|off has quit IRC | 18:34 | |
*** gryf has joined #openstack-nova | 18:35 | |
edleafe | melwitt: At Doug Hellman's suggestion, I went ahead and created https://review.openstack.org/#/c/636416/ | 18:37 |
*** ralonsoh has quit IRC | 18:37 | |
edleafe | melwitt: I was going to wait until we discussed it at Thursday's team meeting | 18:37 |
*** gcerami_ has joined #openstack-nova | 18:37 | |
melwitt | edleafe: ack, thanks | 18:39 |
*** ociuhandu_ has joined #openstack-nova | 18:43 | |
*** hamzy has joined #openstack-nova | 18:43 | |
*** sdake has quit IRC | 18:44 | |
*** wolverineav has joined #openstack-nova | 18:45 | |
*** ociuhandu has quit IRC | 18:47 | |
*** priteau has quit IRC | 18:47 | |
*** ociuhandu_ has quit IRC | 18:48 | |
*** sdake has joined #openstack-nova | 18:51 | |
*** _fragatina has joined #openstack-nova | 18:55 | |
*** ociuhandu has joined #openstack-nova | 18:55 | |
*** krasmussen has joined #openstack-nova | 18:58 | |
*** slaweq has quit IRC | 18:59 | |
*** ociuhandu has quit IRC | 18:59 | |
*** ociuhandu has joined #openstack-nova | 19:00 | |
*** wolverineav has quit IRC | 19:00 | |
*** markvoelker has joined #openstack-nova | 19:03 | |
*** wolverineav has joined #openstack-nova | 19:04 | |
*** ociuhandu has quit IRC | 19:04 | |
*** dtantsur|afk has quit IRC | 19:04 | |
*** alex_xu has quit IRC | 19:04 | |
*** stephenfin has quit IRC | 19:04 | |
*** jroll has quit IRC | 19:04 | |
*** yankcrime has quit IRC | 19:04 | |
*** mugsie has quit IRC | 19:04 | |
*** belmoreira has quit IRC | 19:04 | |
*** zul has quit IRC | 19:04 | |
*** tinwood has quit IRC | 19:04 | |
*** dr_gogeta86 has quit IRC | 19:04 | |
*** tomtom001 has quit IRC | 19:04 | |
*** med_ has quit IRC | 19:04 | |
*** tinwood has joined #openstack-nova | 19:04 | |
*** mugsie has joined #openstack-nova | 19:05 | |
*** dr_gogeta86 has joined #openstack-nova | 19:05 | |
*** stephenfin has joined #openstack-nova | 19:06 | |
*** jroll has joined #openstack-nova | 19:06 | |
*** alex_xu has joined #openstack-nova | 19:06 | |
*** ociuhandu has joined #openstack-nova | 19:06 | |
*** dtantsur has joined #openstack-nova | 19:08 | |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein" https://review.openstack.org/632507 | 19:13 |
kashyap | Only 16 more tests to fix! (Down from 34.) | 19:14 |
*** gcerami_ is now known as panda | 19:14 | |
*** yankcrime has joined #openstack-nova | 19:14 | |
kashyap | mriedem_afk: ^ When you get a moment: I think it makes sense to get the above in in Stein (even at the end of the release). IMHO, it just gets rid of the needless cruft | 19:15 |
kashyap | The unit test "noise" is largely mechanical (as I noted in the commit message): remove the needless extra serial device. | 19:16 |
kashyap | I'll fix the rest of the 16 tests in the morning. | 19:17 |
*** wwriverrat has left #openstack-nova | 19:18 | |
*** artom has joined #openstack-nova | 19:34 | |
*** artom has quit IRC | 19:35 | |
*** artom has joined #openstack-nova | 19:35 | |
*** markvoelker has quit IRC | 19:36 | |
*** sdake has quit IRC | 19:42 | |
*** wolverineav has quit IRC | 19:43 | |
*** wolverineav has joined #openstack-nova | 19:44 | |
*** wolverineav has quit IRC | 19:46 | |
*** wolverineav has joined #openstack-nova | 19:47 | |
*** eharney has joined #openstack-nova | 19:48 | |
*** wolverineav has quit IRC | 19:49 | |
*** wolverineav has joined #openstack-nova | 19:50 | |
*** wolverineav has quit IRC | 19:55 | |
*** wolverineav has joined #openstack-nova | 19:57 | |
*** mriedem_afk is now known as mriedem | 20:05 | |
*** dpawlik has quit IRC | 20:08 | |
*** moshele has joined #openstack-nova | 20:14 | |
*** slaweq has joined #openstack-nova | 20:19 | |
*** wolverineav has quit IRC | 20:20 | |
*** wolverineav has joined #openstack-nova | 20:21 | |
*** wolverineav has quit IRC | 20:26 | |
*** moshele has quit IRC | 20:27 | |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein" https://review.openstack.org/632507 | 20:28 |
kashyap | Alright, actually rest of the 16 are fixed, too. | 20:28 |
* kashyap --> hits the hay | 20:28 | |
*** penick has joined #openstack-nova | 20:33 | |
*** awaugama_ has joined #openstack-nova | 20:33 | |
*** markvoelker has joined #openstack-nova | 20:33 | |
*** moshele has joined #openstack-nova | 20:34 | |
*** awaugama has quit IRC | 20:34 | |
*** xek__ has joined #openstack-nova | 20:35 | |
*** xek_ has quit IRC | 20:37 | |
*** awaugama_ has quit IRC | 20:47 | |
mriedem | huh i see the 12.0.0 liberty GA release notes aren't published https://docs.openstack.org/releasenotes/nova/liberty.html | 20:50 |
mriedem | dansmith: you wanna help get the legacy-grenade-dsvm-neutron-multinode-live-migration job fixed? https://review.openstack.org/#/c/634962/ already +2d | 20:54 |
*** jmlowe has joined #openstack-nova | 20:55 | |
dansmith | DO I EVER | 20:55 |
mriedem | not sure why that job isn't voting, probably should be | 20:55 |
mriedem | seems like live migrating back and forth across n and n-1 nodes is good to keep working.. | 20:55 |
*** dpawlik has joined #openstack-nova | 20:59 | |
*** wolverineav has joined #openstack-nova | 21:00 | |
cfriesen | dansmith: how does https://git.openstack.org/cgit/openstack/nova/commit/?id=804215017f38300ac429e4401ce9c23f0c8ff3c6 avoid the traceback when it saves and re-raises the exception? | 21:00 |
dansmith | cfriesen: I think the point of that was to log the important details where we had them, which are lost in the generic stack trace above | 21:02 |
mriedem | cfriesen: i think the point is it's avoiding this https://review.openstack.org/#/c/546423/5/nova/virt/block_device.py@313 | 21:02 |
dansmith | ah, maybe that too, | 21:03 |
dansmith | but the bug was that the generic error got logged in like compute manager orsomething and it wasn't clear that the problem was that it failed to detach a volume | 21:03 |
*** slaweq_ has joined #openstack-nova | 21:03 | |
mriedem | i'm +1 now on the down cell servers view builder change https://review.openstack.org/#/c/635146/ | 21:05 |
mriedem | withholding +2 for now since it calls out 2.69 | 21:05 |
*** dpawlik has quit IRC | 21:06 | |
*** slaweq has quit IRC | 21:06 | |
*** dtantsur has quit IRC | 21:06 | |
*** tinwood has quit IRC | 21:06 | |
*** mugsie has quit IRC | 21:06 | |
*** dtantsur has joined #openstack-nova | 21:06 | |
*** markvoelker has quit IRC | 21:06 | |
artom | Can I rant about unit testing? I'm ranting about unit testing. Unit testing is not "mock everything the function calls and assert it was called the way the current implementation you're testing would. | 21:06 |
*** mugsie has joined #openstack-nova | 21:06 | |
*** tinwood has joined #openstack-nova | 21:06 | |
edleafe | artom: Most of our unit tests could be named "test_that_python_works" | 21:09 |
artom | edleafe, hah, I'm laughing now, but maybe I should cry? | 21:09 |
artom | mdbooth would love that | 21:17 |
*** moshele has quit IRC | 21:18 | |
*** moshele has joined #openstack-nova | 21:19 | |
*** dpawlik has joined #openstack-nova | 21:20 | |
*** dpawlik has quit IRC | 21:24 | |
*** sdake has joined #openstack-nova | 21:26 | |
*** wolverineav has quit IRC | 21:30 | |
*** xek__ has quit IRC | 21:31 | |
*** slaweq_ is now known as slaweq | 21:32 | |
*** wolverineav has joined #openstack-nova | 21:34 | |
aspiers | mriedem, efried: I seem to have successfully(?) rebased https://review.openstack.org/538498 on top of the removal of get_traits() | 21:35 |
aspiers | mriedem, efried: git-irebase has just brought it 250 commits closer to master, but it's just hit another conflict at master~788 so I've got a way to go still ... | 21:36 |
aspiers | but I think I actually sort of understand what's going on now, which is encouraging | 21:36 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods https://review.openstack.org/636210 | 21:37 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination https://review.openstack.org/634605 | 21:37 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source https://review.openstack.org/634827 | 21:37 |
*** agopi_ has joined #openstack-nova | 21:37 | |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim() https://review.openstack.org/635669 | 21:37 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources https://review.openstack.org/634606 | 21:37 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] LM: Make dest send NUMAMigrateData to the source https://review.openstack.org/634828 | 21:37 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] LM: update NUMA-related XML on the source https://review.openstack.org/635229 | 21:37 |
aspiers | Ah, that conflict was super easy to resolve. Onwards... | 21:38 |
*** agopi has quit IRC | 21:39 | |
aspiers | OK, I think I am officially declaring git-irebase as awesome | 21:40 |
*** agopi_ is now known as agopi | 21:41 | |
*** wolverineav has quit IRC | 21:41 | |
*** wolverineav has joined #openstack-nova | 21:41 | |
*** _alastor_ has quit IRC | 21:41 | |
*** artom has quit IRC | 21:42 | |
efried | aspiers: what is git-irebase? | 21:44 |
aspiers | incremental rebase | 21:44 |
aspiers | one commit at a time | 21:44 |
aspiers | so you are only ever resolving conflicts against a single commit | 21:44 |
efried | ye gods, that sounds like the hard way to go, but good on ya. | 21:45 |
aspiers | no, it's really easier | 21:45 |
aspiers | this change you pointed me at is over 1000 commits behind master | 21:45 |
aspiers | it conflicts with multiple commits along the way | 21:45 |
efried | right, but only a handful of those would have conflicted. | 21:45 |
*** hamzy has quit IRC | 21:46 | |
aspiers | yes, but I would have had to understand *all* of those commits at the same time to resolve the conflict via a normal single-shot rebase | 21:46 |
aspiers | with code I've never seen before, that's a nightmare | 21:46 |
aspiers | even if I'd seen it I suspect it would have been tricky | 21:46 |
aspiers | I'm now at master~510 and so far I've had to handle one major conflict (the get_traits() removal from ironic driver) and 3 minor conflicts | 21:47 |
aspiers | also it means I can run the tests after each resolve, which increases the chances of catching logic errosr | 21:49 |
aspiers | just hit another major conflict | 21:49 |
aspiers | but it's only a single commit, so now manageable :) | 21:49 |
*** sdake has quit IRC | 21:50 | |
aspiers | ah it wasn't major, just a method signature change | 21:51 |
*** moshele has quit IRC | 21:52 | |
*** moshele has joined #openstack-nova | 21:53 | |
efried | aspiers: I looked at the all-in-one rebase, and you're right, it would be pretty hairy. Thanks for tackling it. | 21:54 |
aspiers | yw! only 307 commits behind master now :) | 21:54 |
aspiers | efried: do I understand that for adding an SEV capability to the libvirt driver, capabilities has to be an instance variable rather than a class variable (like for powervm) because the SEV capability is determined dynamically per host? | 21:55 |
*** mchlumsky has quit IRC | 21:56 | |
efried | aspiers: I'm pretty sure that's what we figured out for the PowerVM image cache capability, which is based on which disk driver is set up. edmondsw, back me up here. | 21:57 |
aspiers | yeah, I saw the image cache cap being set at runtime | 21:57 |
efried | I actually can't remember the exact reason we couldn't use the class variable. | 21:57 |
efried | or maybe we just thought it was ugly to set PowerVMDriver.capabilities['image_cache'] from the context of 'self'. | 21:58 |
aspiers | presumably it varies from host to host - does the driver get instantiated once per host? | 21:58 |
efried | At least once, yes. | 21:59 |
efried | and it's the "possibly more than once" part that might be what gets you into trouble using a class var. But that doesn't make a lot of sense. | 21:59 |
*** _alastor_ has joined #openstack-nova | 21:59 | |
efried | Let me go find the review where that happened. | 21:59 |
*** prometheanfire has left #openstack-nova | 22:00 | |
efried | aspiers: Ah, it may have just been for testing purposes: https://github.com/openstack/nova-powervm/commit/cc7564549a90a59f2ef27cca14862c33b9a07c25 | 22:01 |
aspiers | ahhh ok | 22:02 |
aspiers | rebase finished!! | 22:03 |
*** markvoelker has joined #openstack-nova | 22:03 | |
efried | swheet! | 22:03 |
aspiers | I think there were about 8 commits conflicting in total | 22:03 |
aspiers | out of ~1000 | 22:03 |
efried | quick, upload the rebase-only so you have a checkpoint. | 22:03 |
aspiers | to the same review? | 22:03 |
efried | absolutely | 22:04 |
aspiers | OK | 22:04 |
openstackgerrit | Adam Spiers proposed openstack/nova master: WIP: Convert driver supported capabilities to compute node provider traits https://review.openstack.org/538498 | 22:05 |
aspiers | efried: there ya go | 22:06 |
aspiers | who knows, Zuul might even verify it :) | 22:06 |
cdent | wow, that's like time travel or something | 22:07 |
efried | aspiers: I'm a fan of doing rebase in one PS and updates in another. Doing an incremental review when rebase+changes are in one PS is suck. | 22:07 |
aspiers | efried: totally agree | 22:07 |
efried | especially when the rebase is massive. | 22:07 |
mriedem | dansmith: comments in the down cell api microversion change https://review.openstack.org/#/c/591657/ - at this point i think i need to pull it into a devstack environment and kick the tires because I think it's being aggressive about processing filters when 'nova list' might send some parameters by default that would otherwise be ignored | 22:08 |
*** moshele has quit IRC | 22:08 | |
dansmith | mriedem: okay | 22:08 |
aspiers | efried: unfortunately in this case diffing between PS7 and my new PS8 is also full of suck, because we're running an ancient Gerrit which can't distinguish between real changes and stuff dragged in by the rebase | 22:08 |
*** maciejjozefczyk has joined #openstack-nova | 22:09 | |
aspiers | newer Gerrit handles this correctly | 22:09 |
efried | aspiers: customary to use -W instead of -1 to signify wip with changes on the way. | 22:09 |
efried | aspiers: Oh, really, there's a better gerrit where that's concerned? That would be pretty amazing. | 22:09 |
aspiers | efried: I would have if I had W permissions ... | 22:09 |
efried | aspiers: duh, of course. | 22:09 |
aspiers | XD | 22:09 |
aspiers | yes, I remember seeing it demoed at the Gerrit User Summit in London | 22:09 |
aspiers | I can maybe even find the youtube for you | 22:10 |
efried | As it stands, I open two tabs, one with old and one with new, and flip-diff 'em. | 22:10 |
aspiers | yeah, that's better | 22:10 |
aspiers | efried: https://youtu.be/yWTvUyvP24M?t=963 you can even see my shiny head in the 2nd row ;-) | 22:11 |
aspiers | although unfortunately we're still on 2.13, not even 2.14 yet let alone 2.15 ... | 22:12 |
*** maciejjozefczyk has quit IRC | 22:13 | |
aspiers | OK, time for a very late dinner... | 22:18 |
aspiers | l8rs | 22:18 |
*** slaweq has quit IRC | 22:34 | |
*** markvoelker has quit IRC | 22:36 | |
*** cdent has quit IRC | 22:36 | |
edmondsw | efried yep, you found it... sorry, was in a mtg | 22:42 |
edmondsw | (in a lot of those these days...) | 22:42 |
*** agopi has quit IRC | 22:43 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Change nova-next tempest test regex https://review.openstack.org/636459 | 22:52 |
mriedem | ^ drops a bunch of tempest API tests that we don't care about for nova changes | 22:52 |
*** mrjk_ has joined #openstack-nova | 22:52 | |
mriedem | hopefully making that job a bit faster and less redundant | 22:52 |
*** wolverineav has quit IRC | 22:54 | |
*** wolverineav has joined #openstack-nova | 22:55 | |
*** wolverineav has quit IRC | 22:58 | |
*** wolverineav has joined #openstack-nova | 22:59 | |
openstackgerrit | Merged openstack/nova master: Fix legacy-grenade-dsvm-neutron-multinode-live-migration https://review.openstack.org/634962 | 23:08 |
openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: API microversion 2.69: Handles Down Cells https://review.openstack.org/579563 | 23:10 |
*** whoami-rajat has joined #openstack-nova | 23:12 | |
*** tkajinam has joined #openstack-nova | 23:16 | |
*** artom has joined #openstack-nova | 23:18 | |
openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: API microversion 2.69: Handles Down Cells https://review.openstack.org/579563 | 23:21 |
*** ociuhandu has quit IRC | 23:30 | |
*** ociuhandu has joined #openstack-nova | 23:30 | |
*** agopi has joined #openstack-nova | 23:33 | |
*** markvoelker has joined #openstack-nova | 23:33 | |
mriedem | dansmith: what would be a simple way to effectively disable/stop the nova_cell1 db without actually stopping mysqld? failing at finding a db stop command | 23:35 |
mriedem | jaypipes: ^ | 23:35 |
mriedem | i guess i could eff up the db_connection in the cell1 mapping | 23:36 |
sean-k-mooney | mriedem: what do you mean by stop? | 23:36 |
mriedem | i want to make a specific db unavailable | 23:36 |
sean-k-mooney | as in not allow any new queries/transactins | 23:36 |
mriedem | i guess | 23:36 |
mriedem | i want to simulate that the db has crashed | 23:36 |
mriedem | w/o stopping mysqld | 23:37 |
dansmith | mriedem: hard to do on a single server and make it real | 23:37 |
mriedem | since i still need mysql for nova_cell0 and nova_api | 23:37 |
mriedem | i'll just eff up the db connection then | 23:37 |
sean-k-mooney | right because you want to allow the other dbs to work | 23:37 |
dansmith | mriedem: you could revoke privs on the db | 23:37 |
dansmith | mriedem: or try dropping the tables altogether I guess, but tbh we don't really know what the failure modes will look like I'd guess, and most of the time in reality it would be a connection timeout, | 23:37 |
dansmith | which will be hard to mimic with one mysql | 23:38 |
dansmith | one thing you could do, is change the connection string to point to some ip black hole for you | 23:38 |
dansmith | that would get you the timeout on that connection | 23:38 |
dansmith | depends on if you're trying to make it happen at runtime, or just start it up unable to reach one of the dbs | 23:38 |
sean-k-mooney | dansmith: you might be able to do it with ssh forwarding | 23:40 |
dansmith | do what/ | 23:40 |
mriedem | yeah i just changed cell1 db connection to mysql+pymysql://root:openstack1@127.0.0.2/nova_cell1?charset=utf8 | 23:41 |
mriedem | will see what happens | 23:41 |
sean-k-mooney | e.g. expose the mysql server port locally on anoter port point nova at it and then kill the ssh tunnel e.g. using ssh -L ... | 23:41 |
dansmith | mriedem: that might not be a black hole for you :) | 23:41 |
dansmith | mriedem: I would change it to some 172. or 192. or 10. address you definitely can't reach | 23:41 |
*** macza has quit IRC | 23:42 | |
mriedem | oh yeah the connections are cached, | 23:42 |
sean-k-mooney | using ssh that way would allow nova to start up normmally and then you can kill the connect when ever you like by killing the ssh tunnel | 23:42 |
mriedem | restarting n-api | 23:42 |
*** macza has joined #openstack-nova | 23:42 | |
mriedem | bingo | 23:43 |
mriedem | Feb 12 23:42:52 downcell devstack@n-api.service[13288]: WARNING oslo_db.sqlalchemy.engines [None req-8002ab53-453c-488e-ae15-04e2a74e7e6f None None] SQL connection failed. 10 attempts left.: DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on u'192.0.0.1' ([Errno 101] ENETUNREACH)") (Background on this error at: http://sqlalche.me/e/e3q8) | 23:43 |
dansmith | right on | 23:43 |
mriedem | oh goody and it's just spewing in the api logs | 23:43 |
mriedem | well i guess 10 retries | 23:43 |
dansmith | sean-k-mooney: he could just do that with some outbound nat to whatever black hole address he chooses | 23:43 |
*** sdake has joined #openstack-nova | 23:43 | |
mriedem | Feb 12 23:43:49 downcell devstack@n-api.service[13288]: WARNING nova.context [None req-f257afc4-18c9-4989-a500-a4a1f017a322 None None] Timed out waiting for response from cell 0641a06e-4efa-4c25-acac-b92d1f9082b7: CellTimeout: Timeout waiting for response from cell | 23:44 |
mriedem | Feb 12 23:43:49 downcell devstack@n-api.service[13288]: WARNING nova.objects.service [None req-f257afc4-18c9-4989-a500-a4a1f017a322 None None] Cell 0641a06e-4efa-4c25-acac-b92d1f9082b7 did not respond when getting minimum service version | 23:44 |
sean-k-mooney | dansmith: ya you can do it with iptables but that is more annoying | 23:44 |
mriedem | how about i just changed the db connectoin in the db to a bad ip :) | 23:44 |
sean-k-mooney | with ssh its basically "ssh 127.0.0.1 -L 53306:127.0.0.1:3306" | 23:44 |
mriedem | much easier | 23:44 |
sean-k-mooney | mriedem: it is but do you want to be able to kill the connect at an arbitray point or just not have it work | 23:45 |
mriedem | hmm, nova list with 2.69 hanging...this is probably expected while it's trying to gather results from the down cell db | 23:46 |
*** zhanglong has joined #openstack-nova | 23:46 | |
*** ade_lee_ has quit IRC | 23:47 | |
mriedem | i seem to be stuck in an infinite reconnect loop blocking the api response | 23:48 |
mriedem | and i got a 500 | 23:49 |
*** wolverineav has quit IRC | 23:49 | |
mriedem | oh heh, | 23:50 |
sean-k-mooney | proably due to a timeout of the connect waiting for the responce | 23:50 |
*** wolverineav has joined #openstack-nova | 23:50 | |
mriedem | dansmith: i think when nova client is trying to do version negotiation it's getting hung up on that | 23:50 |
mriedem | nova --debug list isn't even getting to the GET /servers/detail request | 23:50 |
dansmith | mriedem: hmm, why would we be striping cells at that point? | 23:51 |
mriedem | this is basically the api log http://paste.openstack.org/show/744983/ | 23:51 |
dansmith | when it fails we should get an exception that tells us where it is right? | 23:53 |
dansmith | but yeah, ten connect attempts at a slow timeout is going to suck | 23:53 |
dansmith | and if we swallow, might need to print it out somewhere so you can see | 23:53 |
mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#database.max_retries | 23:54 |
mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#database.retry_interval | 23:54 |
mriedem | 10 attempts at 10 seconds between attempts | 23:54 |
mriedem | or this.. https://docs.openstack.org/nova/latest/configuration/config.html#database.db_max_retries | 23:54 |
sean-k-mooney | its connecting to the cell deb to get to the service table to figure out the mimium service version | 23:54 |
mriedem | there are a lot of similar sounding options | 23:54 |
sean-k-mooney | do we not have that in the api db? | 23:55 |
mriedem | sean-k-mooney: we shouldn't need that in GET / | 23:55 |
mriedem | the API version doesn't care what the compute services are | 23:55 |
mriedem | or shouldn't | 23:55 |
sean-k-mooney | did you mess with the nova_cell0 connection | 23:55 |
sean-k-mooney | isint that wehere the nova api service record would be | 23:55 |
mriedem | client side error isn't helpful http://paste.openstack.org/show/744984/ | 23:56 |
*** wolverineav has quit IRC | 23:56 | |
mriedem | but yeah this is what is in the server API logs | 23:56 |
mriedem | Feb 12 23:54:50 downcell devstack@n-api.service[13288]: WARNING nova.context [None req-e9c1b49d-437b-47f9-a28e-7d5811908b58 None None] Timed out waiting for response from cell 0641a06e-4efa-4c25-acac-b92d1f9082b7: CellTimeout: Timeout waiting for response from cell | 23:56 |
mriedem | Feb 12 23:54:50 downcell devstack@n-api.service[13288]: WARNING nova.objects.service [None req-e9c1b49d-437b-47f9-a28e-7d5811908b58 None None] Cell 0641a06e-4efa-4c25-acac-b92d1f9082b7 did not respond when getting minimum service version | 23:56 |
dansmith | ah | 23:57 |
dansmith | service version | 23:57 |
dansmith | needs that for microversion discovery I guess? | 23:57 |
mriedem | the api version shouldn't depend on the service version though | 23:57 |
mriedem | the service version is about rpc yeah? | 23:57 |
dansmith | I thought we didn't offer some microversions if computes hadn't ugpraded yet? | 23:58 |
dansmith | or do we just refuse to do the thing? | 23:58 |
sean-k-mooney | mriedem: when you curl / it has the service version in the responce i think | 23:58 |
dansmith | sean-k-mooney: no | 23:58 |
mriedem | curl -g -i -X GET http://199.204.45.32/compute/v2.1 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}121cea354034cc9d2b715383c002e81f067eeb1207dd273ff76587143e309ae5" | 23:58 |
mriedem | that's the request | 23:58 |
mriedem | i don't see what in the version router request handler is trying to do something with service versions | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!