| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add second instance of watcher-decision-engine in the compute node https://review.opendev.org/c/openstack/watcher/+/964546 | 10:01 |
|---|---|---|
| morenod | #startmeeting Watcher IRC Weekly Meeting - October 23, 2025 | 12:00 |
| opendevmeet | Meeting started Thu Oct 23 12:00:43 2025 UTC and is due to finish in 60 minutes. The chair is morenod. Information about MeetBot at http://wiki.debian.org/MeetBot. | 12:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 12:00 |
| opendevmeet | The meeting name has been set to 'watcher_irc_weekly_meeting___october_23__2025' | 12:00 |
| dviroel | o/ | 12:00 |
| morenod | who's around today? | 12:00 |
| jgilaber | o/ | 12:01 |
| chandankumar | o/ | 12:01 |
| rlandy | o/ | 12:01 |
| morenod | courtesy ping: dviroel amoralej jgilaber sean-k-mooney chandankumar morenod rlandy | 12:01 |
| amoralej__ | o/ | 12:01 |
| *** amoralej__ is now known as amoralej | 12:01 | |
| sean-k-mooney | o/ | 12:01 |
| morenod | #link https://etherpad.opendev.org/p/openstack-watcher-irc-meeting#L22 | 12:02 |
| morenod | #topic vPTG | 12:03 |
| dviroel | so, ptg is next week | 12:04 |
| dviroel | I booked our slots some weeks ago | 12:04 |
| dviroel | https://ptg.opendev.org/ptg.html | 12:04 |
| dviroel | I have been working on the ptg etherpad: | 12:04 |
| dviroel | #link https://etherpad.opendev.org/p/watcher-2026.1-ptg | 12:04 |
| dviroel | adding some expected session duration time | 12:05 |
| dviroel | and grouping related topics | 12:05 |
| dviroel | one idea is to start with a quick retro on the last cycle | 12:05 |
| dviroel | before going through the proposed topics | 12:06 |
| dviroel | on tuesday we have a evenlet session at 15:00 UTC in Austin Room | 12:06 |
| dviroel | and at 16:00 UTC, there will be a session about integration tests with horizon | 12:07 |
| dviroel | so we may finish our topics earlier or schedule to the following days if needed to join other sessions | 12:08 |
| dviroel | there will be some other interesting topics in nova too | 12:09 |
| sean-k-mooney | ack we can likely finsih our topic early/rejoin | 12:09 |
| dviroel | yep | 12:09 |
| dviroel | so based on that, or othe schedule changes, we may need to redistribute the topics during the ptg | 12:09 |
| dviroel | one question for this team | 12:10 |
| dviroel | will we keep the meetpad as meeting tool? | 12:10 |
| dviroel | i don't really remember what we used in previous ptg tbh | 12:10 |
| amoralej | i was about to ask that :) | 12:11 |
| amoralej | what did we use? i don't remember | 12:11 |
| jgilaber | I think it was meetpad? IIRC we used the links provided in the PTG page | 12:11 |
| amoralej | i think it was meetpad | 12:11 |
| dviroel | right | 12:11 |
| chandankumar | we used to click on room names and then we joined there from https://ptg.opendev.org/ptg.html | 12:11 |
| chandankumar | not sure it was meetpad or something else | 12:11 |
| dviroel | yeah, these links can be also changed using the bot | 12:12 |
| rlandy | it was meetpad | 12:12 |
| sean-k-mooney | it worked ok, if we need to we can fallback to gmeet or zoom but lets stick with jiti meet | 12:12 |
| sean-k-mooney | which si meetpad for now | 12:12 |
| dviroel | just in case it gets unstable, we can create a gmeet room | 12:12 |
| jgilaber | sounds good, I think it worked well last time though | 12:13 |
| dviroel | nice | 12:13 |
| dviroel | and as a reminder, since we have PTG next week | 12:13 |
| dviroel | we will not have our regular irc meeting on thursay | 12:14 |
| dviroel | i can make that clear in the etherpad | 12:14 |
| dviroel | anything else about this topic? | 12:15 |
| dviroel | ok, morenod you can go with the next one | 12:16 |
| morenod | lets move to next topic | 12:16 |
| morenod | #topic Zone migration, volume migration and cinder concepts/naming | 12:16 |
| jgilaber | quite a long topic | 12:17 |
| jgilaber | but I tried to summarize the discussion we had with dviroel and sean-k-mooney yesterday | 12:17 |
| jgilaber | about cinder configuration and volume migration support in watcher | 12:17 |
| jgilaber | Currently in CI we have a job that configures two cinder volumes, one in the controller and one in the compute, both with the same backend and pool name | 12:17 |
| jgilaber | we discussed whether we needed to change it to only one cinder-volume with two backends, but decided it was not needed | 12:18 |
| jgilaber | btw feel free to interrupt me at any point | 12:18 |
| jgilaber | as a result of the discussion, we might want to rename some parameter in zone migration | 12:19 |
| jgilaber | the use of pool might be a bit misleading, because terms like pool and backend and quite overloaded in cinder | 12:19 |
| jgilaber | src_pool should probably be src_host since we use the volumes host parameter to compare, using the full name host@backend#pool | 12:20 |
| jgilaber | dst_pool is not as clear, because according to the cinder api docs, the migration host should be passed as host@backend | 12:20 |
| amoralej | so, we will call "host" to the full host@backend#pool ? | 12:20 |
| sean-k-mooney | yep | 12:21 |
| sean-k-mooney | that what its called in the cidner volume migrate api | 12:21 |
| jgilaber | that is what is returned from a 'openstack volume show <volume>' as well | 12:21 |
| sean-k-mooney | host is the fully qualifed name | 12:21 |
| amoralej | for me that's more a pool :) as there may be multiple pools in the same host@backend | 12:21 |
| amoralej | said that, i agree it's missleading and it's worthy to agree on something | 12:22 |
| sean-k-mooney | pools are optional in cinder and the volume migrate operation was orgianlly created to move volume between entirly diffent backends | 12:22 |
| sean-k-mooney | i.e. form nfs ot ceph | 12:22 |
| sean-k-mooney | all 3 names host, backend and pool are overloaded making this hard | 12:22 |
| jgilaber | yes, that is the main problem | 12:23 |
| jgilaber | we already have a reported bug from the storage capacity strategy about that | 12:23 |
| jgilaber | assuming there is a 'pool_name' when it's not always there, it's driver dependent | 12:23 |
| chandankumar | we use src_node or node to define compute node , I am not sure src_host will create confusion to users, understand compute node, Can we add src_storage_host and des_storage_host? | 12:24 |
| chandankumar | just a suggestion | 12:24 |
| amoralej | so, we have 4 concepts, host host@backend host@backend#pool and on top of that the field volume_backend_name which is optional and irelevant for zome_migration | 12:25 |
| amoralej | right? | 12:25 |
| jgilaber | no, just host@backend or host@backend#pool | 12:25 |
| dviroel | we can get everything from host@backend#pool | 12:26 |
| amoralej | ok, let's forget host alone | 12:26 |
| dviroel | but the backend_name is relevant for zone_migration | 12:26 |
| dviroel | since there are extra_spec in volume type that should be considered | 12:27 |
| sean-k-mooney | a you mean if hte volume reference a backend directly | 12:27 |
| amoralej | yes, in the form of types configuration, but not in the zone_migration parameters, i meant | 12:27 |
| jgilaber | +1 to that, we keep the existing examples using the full name and strip the pool name when calling cinder's migrate | 12:27 |
| sean-k-mooney | yes i dont think that is the norm but we shoudl not break it | 12:27 |
| sean-k-mooney | we can however parse the name filed and split it based on @ and # to get the 3 part and procees the extra spec if present | 12:28 |
| amoralej | i have question about that, so if we use full qualified pool name as destination, it fails? | 12:28 |
| jgilaber | we do that, but in the volume migration action, not in zone migration | 12:28 |
| jgilaber | I tried in the cli, and it worked with both | 12:28 |
| sean-k-mooney | you should be able to pass either | 12:28 |
| sean-k-mooney | host@backend#pool or host@backend | 12:29 |
| amoralej | actually, in case of having multiple pools, i guess it would be good to use the #pool | 12:29 |
| sean-k-mooney | yes | 12:29 |
| jgilaber | ok, so it looks like asking the user to provide the full host name is the best option for both parameters | 12:31 |
| jgilaber | I'll write a spec for it and we can discuss there the exact parameter names taking into account chandankumar point earlier, which I think is valid | 12:32 |
| amoralej | i'd say so, unless we want to allow host@backend as destination for the case where no pools are configured | 12:32 |
| jgilaber | I don't think that is a problem, even in ceph or nfs cinder creates a pool anyway | 12:33 |
| dviroel | every backend has a pool in the end | 12:33 |
| jgilaber | it just not exposed when getting the pools through api/cli | 12:33 |
| dviroel | since creates one with the same name as the backend | 12:33 |
| dviroel | s/since/since cinder | 12:34 |
| amoralej | then, i'd go for the full name for both | 12:34 |
| sean-k-mooney | we shoudl allow host@backend i think since that a valid cinder input | 12:34 |
| sean-k-mooney | the question is should watcehr add the pool when calling cider to contol the final placmeent of the volume or pass it though directly | 12:34 |
| amoralej | wrt chandan point of name, note these parameters are inside a top level parameter storage_pools which makes it more clear to understand | 12:34 |
| amoralej | said that, adding some storage related term looks reasonable | 12:35 |
| sean-k-mooney | yep we can add storage to the name | 12:35 |
| sean-k-mooney | note we cant remove the old names | 12:35 |
| sean-k-mooney | at least not entirly | 12:36 |
| sean-k-mooney | adding or remving files like this shoudl invole a new api microversion | 12:36 |
| sean-k-mooney | as we are chahing the api schema | 12:36 |
| sean-k-mooney | we can dicuss the upgrade impact and how to minizie it in the spec btu the simiplest way is with the old microverion we sue the old name and with the new one the new name | 12:37 |
| sean-k-mooney | but intenrelaly the code need to handel both inputs | 12:37 |
| jgilaber | yes I'll need to look into this sean-k-mooney that can be a bit complicated | 12:38 |
| amoralej | strategies are managed are plugins, should they be part of the main watcher api microversion? | 12:38 |
| amoralej | s/are plugins/as plugins/ | 12:38 |
| jgilaber | the parameters are part of the audit objects right? | 12:39 |
| jgilaber | are they stored in the db? | 12:39 |
| amoralej | yes | 12:39 |
| jgilaber | that might have an impact as well | 12:39 |
| amoralej | but as a single db field | 12:39 |
| amoralej | in json, i think | 12:39 |
| jgilaber | oh so the full parameters dict? | 12:39 |
| sean-k-mooney | amoralej: its somehtign we need to decie but we do need to verion there inputs | 12:39 |
| amoralej | that's my understanding jgilaber | 12:40 |
| jgilaber | ack, that would make it easier | 12:40 |
| amoralej | i agree sean-k-mooney, there is other projects with similar problems? like driver parameters | 12:40 |
| sean-k-mooney | well esier and harder. as we have to condier data migrationof the json on load | 12:40 |
| sean-k-mooney | yes and no, nova has to a degree | 12:41 |
| sean-k-mooney | you can add your own filter and as a result schduler hints | 12:41 |
| sean-k-mooney | those are unvovesion but on the other hand the pearmter hav e ajosn schema | 12:42 |
| sean-k-mooney | and we ahve an api to retirve that | 12:42 |
| jgilaber | hmm what happends with continuous audits in a migration? are they expected to be resumed? | 12:42 |
| sean-k-mooney | so we shoudl be versioning them | 12:42 |
| sean-k-mooney | jgilaber: that why im being this up | 12:42 |
| sean-k-mooney | they need to continue to work with the old defiions | 12:42 |
| amoralej | yep, that's the hard case | 12:42 |
| jgilaber | ok, that can be tricky | 12:43 |
| amoralej | we may use something like aliases | 12:43 |
| sean-k-mooney | that why we can really only make additive changes | 12:43 |
| sean-k-mooney | unless we can do a data migration on load | 12:43 |
| sean-k-mooney | similar to what we did with the scores when we change the colume form a decimal to a float | 12:43 |
| jgilaber | yes, I remember that, that was not so bad | 12:44 |
| jgilaber | but this case looks more complicated | 12:44 |
| sean-k-mooney | right os this is why im saying we shoudl not remove the old parmater just deprecate them | 12:44 |
| amoralej | +1 | 12:45 |
| jgilaber | but anyway I think I have enough to get started on the spec and we can continue there | 12:45 |
| sean-k-mooney | encurage the usage of the new ones and in 12+ months itme we can consider if it worth removing the old ones | 12:45 |
| sean-k-mooney | the removal is the problematic part | 12:45 |
| amoralej | we could not remove | 12:46 |
| amoralej | anyway, can be discussed later i guess | 12:47 |
| dviroel | we can further discuss more in the spec | 12:47 |
| jgilaber | ok, so final point we discussed is which migrations should Watcher support | 12:47 |
| dviroel | or even on a ptg session if jgilaber wants | 12:47 |
| jgilaber | currently Watcher only supports migrations between hosts configured with the same backend name because of https://bugs.launchpad.net/watcher/+bug/2127485 | 12:47 |
| sean-k-mooney | i think we should supprot what cinder supprots | 12:48 |
| sean-k-mooney | so migration between any backend or pool | 12:48 |
| jgilaber | dviroel, I think we've clarified enough here, but we could cover if there is time | 12:48 |
| jgilaber | sean-k-mooney, I agree | 12:48 |
| dviroel | ack | 12:48 |
| jgilaber | right now the only limitation is the bug in https://review.opendev.org/c/openstack/watcher/+/963857 | 12:48 |
| dviroel | agree | 12:49 |
| jgilaber | I have a possible fix for the bug in https://review.opendev.org/c/openstack/watcher/+/963857 | 12:49 |
| jgilaber | with that I think we should have full support | 12:49 |
| jgilaber | so please review that when time permits | 12:50 |
| jgilaber | and I think that is all, sorry I took most of the meeting | 12:50 |
| morenod | there is no more topics on the agenda, so it is ok to focus on this | 12:50 |
| morenod | but... if nothing else about that, we move to reviews | 12:52 |
| jgilaber | if there is nothing else I would like to cover quickly this bug I opened last week https://bugs.launchpad.net/watcher/+bug/2128056 | 12:53 |
| jgilaber | this describe the third use case we considered for zone migration, where a retype+migration is required | 12:53 |
| jgilaber | I've opened two WIP patches with two possibilites | 12:54 |
| sean-k-mooney | im not sure that is valid | 12:54 |
| jgilaber | 1. modify the retype volume migration action | 12:54 |
| jgilaber | 2. add a new type of volume migration action | 12:54 |
| sean-k-mooney | mainly because i dont htink cidner allows you to both do a rety and specify a migration at the same time for a single volume | 12:54 |
| jgilaber | it's not at the same time, it two sequential operations | 12:54 |
| jgilaber | during the retype cinder might migrate the volume | 12:55 |
| sean-k-mooney | right but im not convied we shoudl supprot asign for both in the same auction plan | 12:55 |
| sean-k-mooney | doing multipel operation on the same volume in one action plan feels a bit suspect | 12:55 |
| jgilaber | that is a fair point, that's why I wanted to dicuss it | 12:56 |
| amoralej | in that case we should enforce in the parameter to set only one of dst_type or dst_pool (or whatever name) | 12:56 |
| sean-k-mooney | that was what i was thinking too | 12:57 |
| amoralej | it may make sense to ask for two audits for that case, 1st run a retype 2nd run a migrate | 12:57 |
| sean-k-mooney | we shoudl make this a 400 bad request | 12:57 |
| jgilaber | yep, we need more input validation for zone migration | 12:57 |
| dviroel | i agree that this is a more complex scenario and we should allow work with both at the same time | 12:58 |
| dviroel | s/should/shouldn't | 12:58 |
| jgilaber | so, we close the bug as invalid or won't fix and we fix this when adding validation | 12:59 |
| jgilaber | will we treat the missing validation as a bug fix or have a spec for that? I think this is an issue common to all strategies IIRC? | 12:59 |
| sean-k-mooney | the stragies do do jsonscema validation of the parmaters | 13:00 |
| sean-k-mooney | btu the current scema is very lax | 13:00 |
| amoralej | yes, we do schema validation, depends on each strategy to be more strict or less | 13:00 |
| sean-k-mooney | so yes its a problem for all the stratgies and we likely need to fix it for all fo them | 13:00 |
| jgilaber | but will jsonschema validation be enough for cases like this? | 13:01 |
| amoralej | also there is the question of validating not only schema, but values, what if we use a src_host which does ot exist? | 13:01 |
| jgilaber | exactly, we probably need some early checks in code | 13:01 |
| sean-k-mooney | we can do this via a combinaiton of schema and non schema checks | 13:02 |
| amoralej | i think strategies have a method for validation, not sure if it's called in the api | 13:02 |
| sean-k-mooney | we have exampels of both | 13:02 |
| sean-k-mooney | in some cases we can express this in the schema via one_of | 13:02 |
| sean-k-mooney | or other constucto but i can be simpelr to just do it in python | 13:02 |
| sean-k-mooney | its better to do basic syntacitc validation in the schema and semantic validation in the python code | 13:03 |
| jgilaber | ok, I'll look for existing examples then | 13:03 |
| dviroel | right, values can be valid or not depending when the audit runs | 13:04 |
| * dviroel time check | 13:05 | |
| jgilaber | ack I'll update the bug with the focus on input validation, thanks all for the discussion! | 13:06 |
| morenod | no more topics, so I think we can close the meeting here | 13:06 |
| dviroel | +1 | 13:07 |
| morenod | we already have a volunteer for next meeting, thanks dviroel | 13:07 |
| morenod | #endmeeting | 13:07 |
| opendevmeet | Meeting ended Thu Oct 23 13:07:40 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 13:07 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/watcher_irc_weekly_meeting___october_23__2025/2025/watcher_irc_weekly_meeting___october_23__2025.2025-10-23-12.00.html | 13:07 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/watcher_irc_weekly_meeting___october_23__2025/2025/watcher_irc_weekly_meeting___october_23__2025.2025-10-23-12.00.txt | 13:07 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/watcher_irc_weekly_meeting___october_23__2025/2025/watcher_irc_weekly_meeting___october_23__2025.2025-10-23-12.00.log.html | 13:07 |
| amoralej | thanks for chairing morenod ! | 13:08 |
| opendevreview | Takashi Kajinami proposed openstack/watcher master: Use config validation for prometheus options https://review.opendev.org/c/openstack/watcher/+/941812 | 13:49 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add second instance of watcher-decision-engine in the compute node https://review.opendev.org/c/openstack/watcher/+/964546 | 13:55 |
| opendevreview | Merged openstack/watcher master: Remove dead NovaHelper methods https://review.opendev.org/c/openstack/watcher/+/963249 | 14:45 |
| opendevreview | Merged openstack/watcher master: Use config validation for prometheus options https://review.opendev.org/c/openstack/watcher/+/941812 | 16:40 |
| opendevreview | Joan Gilabert proposed openstack/watcher master: Add tests for get_dst_pool_and_type method https://review.opendev.org/c/openstack/watcher/+/964717 | 17:21 |
| opendevreview | Joan Gilabert proposed openstack/watcher master: Fix zone migration dest pool and type selection https://review.opendev.org/c/openstack/watcher/+/964718 | 17:21 |
| opendevreview | Douglas Viroel proposed openstack/watcher master: Replace some eventlet usage from the code https://review.opendev.org/c/openstack/watcher/+/964727 | 19:43 |
| opendevreview | Merged openstack/watcher master: Do not install openstackclient from git repo https://review.opendev.org/c/openstack/watcher/+/945347 | 23:52 |
| opendevreview | Merged openstack/watcher master: Migrate bandit options to pyproject.toml https://review.opendev.org/c/openstack/watcher/+/962830 | 23:52 |
| opendevreview | Merged openstack/watcher master: Add ThreadPool statistics debug log https://review.opendev.org/c/openstack/watcher/+/960881 | 23:52 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!