| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add support to update status_message in update_audit_state https://review.opendev.org/c/openstack/watcher/+/1014342 | 08:42 |
|---|---|---|
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Version the parameters spec of strategies https://review.opendev.org/c/openstack/watcher/+/1014569 | 08:42 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add update_parameters() to BaseStrategy https://review.opendev.org/c/openstack/watcher/+/1014570 | 08:42 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Update goals and strategies in-place in the Syncer https://review.opendev.org/c/openstack/watcher/+/1014571 | 08:42 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher stable/2026.1: Fix duplicate workload stabilization migrations https://review.opendev.org/c/openstack/watcher/+/1015018 | 09:11 |
| dviroel | sean-k-mooney: amoralej__ thanks for the spec review - going through the comments and update a new version today - i will drop questions here if needed. | 11:24 |
| amoralej__ | sure | 11:25 |
| amoralej__ | good to discuss here and agree on the open questions | 11:25 |
| *** amoralej__ is now known as amoralej | 11:29 | |
| sean-k-mooney | dviroel: cool | 12:13 |
| dviroel | sean-k-mooney: so IIUC, pinning to default availability zone may return an error in a near future | 12:14 |
| dviroel | but there is no way to know that from nova api | 12:15 |
| dviroel | so we would need a config option for that | 12:15 |
| sean-k-mooney | so there kind of is | 12:16 |
| sean-k-mooney | but you have too syntasize it form 2 diffen tinputs | 12:16 |
| sean-k-mooney | you can list teh avialablity zones and you can list the compute service and check what zones they report | 12:17 |
| sean-k-mooney | if there is an addtional zone in the compute service view | 12:18 |
| sean-k-mooney | then that is the fake defautl zone | 12:18 |
| sean-k-mooney | but that kind of a pain and ineffeicent | 12:18 |
| sean-k-mooney | so horizon and other just mirror the config option that nova has for its name | 12:18 |
| sean-k-mooney | and set it to the saem default value `nova` | 12:18 |
| sean-k-mooney | well sort of i think that is what dan is propsoing in his bug fix | 12:20 |
| sean-k-mooney | https://docs.openstack.org/api-ref/compute/#id290 | 12:20 |
| sean-k-mooney | the current example respocne show the nova zone | 12:21 |
| sean-k-mooney | i think dan was considering not reportign that any more and only reporting the real AZs form that api | 12:21 |
| sean-k-mooney | but i need to actully read there patch to confirm that | 12:21 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/1014772 | 12:22 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/1014772/1/api-ref/source/os-availability-zone.inc | 12:24 |
| sean-k-mooney | agaisn i need to read the patch for the details but that srongly implying that the defautl nova zone would nolnger be in the az list | 12:24 |
| sean-k-mooney | which form a watcher point of view would mean that in the future if we have an audit that refences an az that is not in the list we coudl treat that as an error | 12:25 |
| sean-k-mooney | but it does mean we woudl not be able to manged vms in the default zone if we did that by zone | 12:25 |
| sean-k-mooney | i.e. we coudl not express move all vms form the default zone to prod-1 | 12:25 |
| sean-k-mooney | so we need to decied if watcher will only support real zones | 12:26 |
| sean-k-mooney | or the implicit default one as well | 12:26 |
| dviroel | watcher today build the zone list from compute service list API | 12:27 |
| dviroel | I believe that this will still continue listing the default one? | 12:27 |
| sean-k-mooney | it would yes i think | 12:28 |
| dviroel | I also think that we should support draining from the default one, and may reject the default as a destination | 12:29 |
| sean-k-mooney | that would be a reasonable appproch | 12:29 |
| dviroel | we could validate that in the API, if we include the config option | 12:29 |
| dviroel | "please operator, create a real AZ to drain your workloads to" :) | 12:31 |
| sean-k-mooney | the other thing is we may want to expess a request to move the vms to ANY az in the future | 12:31 |
| dviroel | right | 12:32 |
| sean-k-mooney | we woudl do that by disabling the souce az nodes, unpinnning the vm and then aksing for a migration without passing a host or az | 12:32 |
| sean-k-mooney | then the schduler woudl choose form the online hosts | 12:32 |
| dviroel | ack, interesting | 12:32 |
| * dviroel can't open https://bugs.launchpad.net/nova/+bug/2173771 - network issue on my side or launchpad is down? | 12:34 | |
| amoralej | i can't open it either | 12:36 |
| sean-k-mooney | ya lachpad is down | 12:37 |
| dviroel | so we have the options: 1) do not pin to default_availability_zone (skips the pin after migrating) and 2) deny draining to the default AZ (dest_az) | 12:43 |
| dviroel | the 1) can live in Migrate option, for any other strategy to use it | 12:44 |
| dviroel | we can have both in the end, using an additional config option in watcher | 12:44 |
| dviroel | [nova] default_availability_zone | 12:44 |
| sean-k-mooney | ya so im not conviced that dans propsoed bug fix is valid and im saying so now | 12:46 |
| amoralej | dviroel, so that case is to drain to default_availability_zone from a different az | 12:55 |
| amoralej | right? | 12:55 |
| amoralej | drain prod-az to nova i.e. | 12:56 |
| amoralej | i have a doubt, from Nova pov, if a non-default az is created, the default one can still have compute services? i thought the default one was only for the case where no other AZ exists | 12:57 |
| sean-k-mooney | amoralej: so in my spec that would work with dans bugfix that woudl be a 400 error | 13:02 |
| sean-k-mooney | i belive there would be no way to reust the host are are not part of a real az other hten by there hostname | 13:02 |
| dviroel | amoralej: yes, "drain prod-az to nova" would be invalid. Also pin to default zone can be at Action level | 13:04 |
| sean-k-mooney | amoralej: no the default az is what a compute service is in | 13:04 |
| sean-k-mooney | unless you add it to a az explcitly | 13:04 |
| sean-k-mooney | there is no host-aggreage assocatied with teh default az | 13:04 |
| sean-k-mooney | and no aggreate metadata as a result | 13:05 |
| sean-k-mooney | that also menas the default az is not modeled in placement | 13:05 |
| sean-k-mooney | its litrally the set of host that are not grouped by an admin explcitly inot an az. all newly added comptue are part of it when you scale out implictly untill you use teh rest api to assign it to a real az | 13:06 |
| sean-k-mooney | psi didnt bother with AZs really which is why in horizon if you booted a vm it booted into the "nova" az | 13:07 |
| dviroel | so, in the end we need: 1) do not pin to default_availability_zone (skips the pin after migrating) a | 13:56 |
| dviroel | right? | 13:56 |
| sean-k-mooney | i think that woudl be for the best | 14:30 |
| sean-k-mooney | in the spec i also suggested havaing a pining policy at the stragey level | 14:30 |
| sean-k-mooney | so i think we may want to allow an operator to say drain this az but done pin any vms | 14:30 |
| sean-k-mooney | they may also want to say drain the node and preserve the existing pinned or unpinned state or force the pinning of the moved vms | 14:31 |
| sean-k-mooney | i dont think we need to do that per vm | 14:31 |
| sean-k-mooney | but i think that high level policy shoudl apply per audit | 14:31 |
| dviroel | so currently we have the 'skip_az_pinned_instances' proposal. which skipps the pinned ones. All pinned instances would be repinned in case a dest_az != src_az | 14:58 |
| dviroel | you are also suggesting a new parameter to allow op to choose when pinned instances should or not be pinned to the dest_az | 14:59 |
| dviroel | 'repin_az_pinned_instances' ? if true, it pins instance after migration, otherwise it just unpin the one pinned. | 15:06 |
| dviroel | 'skip_az_pinned_instances' would default to True, as also repin_az_pinned_instances (true by default) - it will work only for the ones that are already pinned. | 15:20 |
| dviroel | skip_az_pinned_instances=False + skip_az_pinned_instances=False would just unpin instances. | 15:20 |
| amoralej | so, the use case for that would be. "As a cloud admin I want to drain all my VMs from AZ1, including the pinned ones, to AZ2 but do not pin the VMS which were pinned before" | 15:23 |
| amoralej | that's what you are suggesting? | 15:23 |
| dviroel | that's what I understood from sean's comment. But the spec does not cover that today. | 15:49 |
| dviroel | sean-k-mooney: is that it? | 15:50 |
| dviroel | I think that doesn't add too much to the implementation today | 15:50 |
| dviroel | just more scenarios to test | 15:50 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher-specs master: Update goals and strategies in-place to prevent Audit cancellation on upgrades https://review.opendev.org/c/openstack/watcher-specs/+/1012696 | 15:53 |
| dviroel | availability_zone_pinning_policy = [repin, unpin_only, auto] is another suggestion from sean. It makes easier to extend. But that was a suggestion for the Migrate action only? Or we would propose also as end user configuration? | 16:00 |
| dviroel | skip_az_pinned_instances + availability_zone_pinning_policy? | 16:00 |
| dviroel | note that 'availability_zone_pinning_policy' in audit parameter would be only a bypass to Migrate Action in the end, which will hold the pin/unpin logic | 16:01 |
| dviroel | this thing about skipping instances feels more like Audit Scope duty. If we add the exclude_az_pinned_instances into the scope, it could mark all instances as watcher_excluded in the ed. | 16:03 |
| dviroel | end | 16:03 |
| sean-k-mooney | sorry reading back | 16:18 |
| sean-k-mooney | dviroel: the policy woud apply to all moved instnaces | 16:20 |
| sean-k-mooney | basically in the review | 16:20 |
| sean-k-mooney | i suggested 3 policies, unpin pin and auto | 16:21 |
| sean-k-mooney | that spereate form if we shoudl include the instnace to be moved | 16:21 |
| sean-k-mooney | auto means pin if it was pinned and dont pin if not pinned before the move | 16:22 |
| dviroel | right, so user would have the two options: skip pinned instances (boolen) and define the pin/unpin policy. I understood that would apply to all instances within an audit. | 16:56 |
| sean-k-mooney | ya i mean we coudl make a differnet design chocice as these are all trade off in complexity | 17:05 |
| sean-k-mooney | but to me those are two diffent chocies adn i feel liek we shoudl expose both | 17:05 |
| sean-k-mooney | and to be conservitive i think we shoudl excplude instnace that are pinned by defualt to make it safe for public clouds | 17:06 |
| sean-k-mooney | where changign an az by accinded could be condier a breach of the cloud sla | 17:06 |
| sean-k-mooney | but we coudl also incldue them by defautl | 17:06 |
| sean-k-mooney | we jed nteeed to docuemnt this well | 17:07 |
| sean-k-mooney | my bigger concern is with project like octaiva that scdhule loadbalncer isntance to diffent AZs via there api policy or k8s cloud-provider-openstack | 17:08 |
| sean-k-mooney | where the az of a k8s worker node is used by the k8s schduler when doing pod placement | 17:08 |
| sean-k-mooney | those workload likely wont be happy fi there vms are moved across az | 17:08 |
| sean-k-mooney | but if they are dependign on the az internally they shoudl be pinning to the az | 17:09 |
| sean-k-mooney | if the vm is floating watcher and an admin is free to assuem it can be moved across azs as they see fit | 17:09 |
| dviroel | i think that the pinning policy makes sense, since there are different use cases for this | 17:30 |
| dviroel | we can be conservative and set the skip to True also | 17:30 |
| dviroel | by setting to False, users don't need to worry about the policy one. While setting to False, they will likely know the choices that they have around pin/unpinnin | 17:32 |
| dviroel | that said, we could rename it to "include_az_pinned_instances: false/true" with "false" being the default. | 17:34 |
| dviroel | so the default is to be off | 17:34 |
| dviroel | and user would need to opt for enabling it | 17:34 |
| opendevreview | Winicius Allan Bezerra da Silva proposed openstack/watcher-specs master: Add Preemptible Instances spec for 2027.1 cycle https://review.opendev.org/c/openstack/watcher-specs/+/1014252 | 17:54 |
| sean-k-mooney | yep | 19:30 |
| sean-k-mooney | i think i suggested include_az_pinned_instances in the spec to avodi the double negitive of skip false | 19:31 |
| dviroel | yeah, I am updating the spec, lets see if alfredo and joan also agree with these changes.. | 20:21 |
| opendevreview | Douglas Viroel proposed openstack/watcher-specs master: Add spec for availability zone drain https://review.opendev.org/c/openstack/watcher-specs/+/1007853 | 22:09 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!