| opendevreview | David proposed openstack/watcher master: [DNM] Testing nodeset with three nodes (two computes + 1 controller) https://review.opendev.org/c/openstack/watcher/+/967331 | 09:44 |
|---|---|---|
| jgilaber | hello! just a reminder that we'll have our irc meeting in ~ 20 minutes, feel free to add topics to the agenda https://etherpad.opendev.org/p/openstack-watcher-irc-meeting | 11:42 |
| jgilaber | watcher meeting strating in two minutes | 11:59 |
| jgilaber | #startmeeting watcher | 12:01 |
| opendevmeet | Meeting started Thu Dec 4 12:01:54 2025 UTC and is due to finish in 60 minutes. The chair is jgilaber. Information about MeetBot at http://wiki.debian.org/MeetBot. | 12:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 12:01 |
| opendevmeet | The meeting name has been set to 'watcher' | 12:01 |
| jgilaber | hi all o/, how is around today? | 12:02 |
| rlandy | o/ | 12:02 |
| amoralej | o/ | 12:02 |
| dviroel | o/ | 12:02 |
| jgilaber | courtesy ping: sean-k-mooney chandankumar morenod | 12:03 |
| morenod | o/ | 12:03 |
| sean-k-mooney | o/ | 12:03 |
| jgilaber | thanks for joining, let's start with today's agenda | 12:03 |
| jgilaber | #link https://etherpad.opendev.org/p/openstack-watcher-irc-meeting#L26 (Meeting agenda) | 12:03 |
| jgilaber | feel free to add your own topics to the agenda | 12:04 |
| jgilaber | #topic new blueprint for openstacksdk migration | 12:04 |
| jgilaber | this was carried from last week | 12:04 |
| jgilaber | I created a blueprint and a spec to move watcher to use openstacksdk as discussed during the PTG and other meetings | 12:05 |
| jgilaber | #link https://blueprints.launchpad.net/watcher/+spec/use-openstacksdk | 12:05 |
| jgilaber | #link https://review.opendev.org/c/openstack/watcher-specs/+/968023 | 12:05 |
| jgilaber | this is the first time I created a blueprint, so I'm not sure if it's missing anything | 12:05 |
| jgilaber | do we need to do anything with the blueprint? if not I'll just use the opportunity to ask for reviews on the spec | 12:08 |
| sean-k-mooney | jgilaber: that looks ok | 12:08 |
| sean-k-mooney | when we ahve a spec the bluepirnt is often just the first paragah or a cut down version of the problem description | 12:09 |
| jgilaber | ack, thanks, if there are no further questions/comments we can move on the next topic from dviroel | 12:10 |
| jgilaber | #topic spec proposal freeze/spec freeze | 12:10 |
| dviroel | just added this one | 12:10 |
| sean-k-mooney | the default would be milestone 2 | 12:10 |
| dviroel | so we can agree on a deadline to get our specs for this release merged | 12:10 |
| dviroel | which is Jan 05 - Jan 09 (m2) | 12:11 |
| sean-k-mooney | which is january 8th ish | 12:11 |
| dviroel | #link https://releases.openstack.org/gazpacho/schedule.html | 12:11 |
| sean-k-mooney | https://docs.openstack.org/watcher/latest/contributor/release-guide.html#milestone-2 | 12:11 |
| jgilaber | I think that sound reasonable, we have currently 3 open specs | 12:11 |
| dviroel | due to holidays we may want to get most of them merged asap | 12:11 |
| jgilaber | #link https://review.opendev.org/q/project:openstack/watcher-specs+status:open | 12:11 |
| dviroel | jgilaber: yeah, I plan to send another one in a couple of minutes | 12:12 |
| dviroel | do we want to have a proposal freeze too? | 12:12 |
| sean-k-mooney | yep sooner rahter then later. nova pulled theres in to today to see if that can help with apporving less and compelting more | 12:12 |
| sean-k-mooney | dviroel: no | 12:12 |
| sean-k-mooney | most teams dont | 12:12 |
| dviroel | to have some time to review them.. | 12:13 |
| sean-k-mooney | we coudl ahve one but i dont know that we have the volume to do it | 12:13 |
| amoralej | I may need to create a new one for instrumenting the decision engine with osprofiler | 12:13 |
| amoralej | in this cycle | 12:13 |
| amoralej | i guess I'll need a spec for that, right? | 12:14 |
| sean-k-mooney | what might be more intersting is do we want an early Q1 "mid cycle" or check in for what we want to complete this cycel | 12:14 |
| sean-k-mooney | amoralej: that might qualify for a specless blueprint | 12:14 |
| sean-k-mooney | provide tehre is no upgrade impact (its off by defualt) | 12:14 |
| amoralej | right | 12:14 |
| sean-k-mooney | as it not really user facing | 12:14 |
| amoralej | ok, I'll prepare the blueprint for next meeting hopefully | 12:15 |
| jgilaber | sean-k-mooney, that "mid cycle" would be like a sort of PTG session to go over the status? | 12:15 |
| sean-k-mooney | yep | 12:16 |
| jgilaber | I like the idea rather than trying to impose a hard freeze | 12:16 |
| sean-k-mooney | well im suggeeting it after the hard freeze | 12:16 |
| sean-k-mooney | we acutlly have more approved already then i think we can complete | 12:16 |
| dviroel | they are 2 different things I think | 12:17 |
| jgilaber | oh ok, I misunderstood then | 12:17 |
| sean-k-mooney | this is our action item list form the ptg https://etherpad.opendev.org/p/watcher-2026.1-ptg#L56 | 12:17 |
| sean-k-mooney | in theory we shoudl be tryign to compelte thos as well this cycle | 12:17 |
| sean-k-mooney | so one of the things i was thinking we woudl do is review how much fo that we got done and what to compelte | 12:18 |
| dviroel | ok, just wanted to be clear on deadlines for specs | 12:18 |
| dviroel | because there are holidays right before | 12:19 |
| sean-k-mooney | right so in our contibutor docs is docuemneted spec freeze as mileston 2 | 12:19 |
| sean-k-mooney | so unless w eexpclity opt out of that that woudl be the freze for every cycle | 12:19 |
| dviroel | and we may want to accept/merge specs asap | 12:19 |
| dviroel | i may be on pto the first week of jan, so I would review all open in 2025 :) | 12:20 |
| jgilaber | so we have roughly 2 full weeks before people start going on PTO | 12:20 |
| sean-k-mooney | effectivly yes | 12:20 |
| jgilaber | we should try to prioritize reviewing the existing specs, but it seems doable | 12:21 |
| sean-k-mooney | i will likely finish december 20th and return january 5th | 12:21 |
| sean-k-mooney | the exact dates are tbd | 12:22 |
| dviroel | jgilaber: the idea was to make deadlines clear to contributors and reviewers, so I don't have anything else in this topic | 12:23 |
| jgilaber | ack thanks for bringing that up dviroel! | 12:24 |
| jgilaber | if there are no more comments we can move to reviews | 12:24 |
| jgilaber | #topci reviews | 12:24 |
| jgilaber | first one | 12:24 |
| jgilaber | #link https://review.opendev.org/c/openstack/watcher/+/968025 | 12:24 |
| jgilaber | amoralej, do you have anything to cover for this patch? | 12:25 |
| sean-k-mooney | oh the action docs cool ill add that to my review list | 12:25 |
| amoralej | just to mention it as i expect it to be quick to review and is blocking other stuff on top | 12:25 |
| jgilaber | ack, thanks we can move to the second one then | 12:26 |
| sean-k-mooney | well you can rebase the other on top | 12:26 |
| jgilaber | #link https://review.opendev.org/c/openstack/watcher/+/963981 | 12:26 |
| sean-k-mooney | but ya i think we can land that soon | 12:26 |
| jgilaber | the second one has +2 from dviroel and me | 12:26 |
| sean-k-mooney | yep so dviroel shoudl ahve added +w unless they were askign for a change | 12:26 |
| amoralej | that one already has 2 +2, it'd be great of i can also get +w | 12:27 |
| jgilaber | I was looking if we were waiting for anything in particular | 12:27 |
| sean-k-mooney | i just added it | 12:27 |
| dviroel | I still reviewing 963252, have some comments to sed | 12:27 |
| dviroel | sean-k-mooney: i gave a chance for more people to review it, but yeah, we can +w | 12:27 |
| amoralej | ack | 12:28 |
| amoralej | so, that was it from me | 12:28 |
| jgilaber | thanks, we can move to bug triage then | 12:28 |
| jgilaber | #topic Bug Triage | 12:28 |
| jgilaber | #link https://bugs.launchpad.net/watcher/+bug/2129692 | 12:28 |
| opendevreview | David proposed openstack/watcher master: [DNM] Testing nodeset with three nodes (two computes + 1 controller) https://review.opendev.org/c/openstack/watcher/+/967331 | 12:29 |
| jgilaber | I opened this bug a while back for zone migration | 12:29 |
| jgilaber | I'm not sure if we've covered it before | 12:29 |
| jgilaber | but in short after we added support for the 'src_type' parameter, I realized the strategy was not taking it into account when selecting the destination for the migration | 12:30 |
| sean-k-mooney | tis not really a bug in the normal sense so at most i would triage it as wishlist | 12:30 |
| sean-k-mooney | well sorry | 12:30 |
| sean-k-mooney | im mixing linkgs | 12:30 |
| sean-k-mooney | that bug is not for davids ci change | 12:31 |
| jgilaber | nope, it's https://bugs.launchpad.net/watcher/+bug/2129692 | 12:31 |
| sean-k-mooney | so "Zone migration does not select volumes properly according to user input" | 12:31 |
| sean-k-mooney | i dont thinkn i have looked at that before | 12:31 |
| jgilaber | right | 12:31 |
| sean-k-mooney | this was somethign you found when doing the other fixes | 12:31 |
| jgilaber | I found it after we landed that series of patches actually | 12:32 |
| jgilaber | basically if a user wanted to migrate some volumes that are in the same pool but have different types to different destination depending in the type | 12:32 |
| sean-k-mooney | you have already started on 2 patchs, so i think we can consier it validi and you shoudl assign it to yourself | 12:33 |
| jgilaber | it would not work properly | 12:33 |
| sean-k-mooney | do you have a feelign on the prioty | 12:33 |
| jgilaber | right, I though I had assigned it already | 12:33 |
| jgilaber | I think it's somewhat between medium and high | 12:33 |
| sean-k-mooney | oh right this is about doing a retry and a migrate at the same time | 12:33 |
| sean-k-mooney | well i dont think that is supproted in cinder | 12:33 |
| sean-k-mooney | your can rety btu i dont think you can specify the strage backend when you do a retry can you? | 12:34 |
| jgilaber | no, this is about selecting the destination pool | 12:34 |
| sean-k-mooney | we dont supprot have differnet voluem go to diffent destionats in the same audit today | 12:34 |
| amoralej | so, the problem is that, if ths src_pool is repeated with different src_type values, it does not work | 12:35 |
| jgilaber | why not? | 12:35 |
| amoralej | right? | 12:36 |
| sean-k-mooney | doesnet the volume migrate part refer to all voluems? | 12:36 |
| jgilaber | yes, amoralej that is a good summary | 12:36 |
| jgilaber | in zone migration we evaluate each volume migration individually https://github.com/openstack/watcher/blob/e427fa68a379cbff4641c937b71d353d9e6387a0/watcher/decision_engine/strategy/strategies/zone_migration.py#L409 | 12:37 |
| amoralej | yep, looks like a bug | 12:37 |
| jgilaber | so for each volume to consider we get the dst_pool and dst_type and crate the appropiate action (retype or migrate) according to that plus the volume type | 12:38 |
| sean-k-mooney | https://github.com/openstack/watcher/blob/master/watcher/decision_engine/strategy/strategies/zone_migration.py#L89-L147 | 12:38 |
| sean-k-mooney | so if you look at the schem today | 12:38 |
| sean-k-mooney | are you saying you think we shoudl supprot haveing | 12:38 |
| sean-k-mooney | 2 entries with the same srouce oool | 12:39 |
| sean-k-mooney | and different soruce types and dest pools | 12:39 |
| jgilaber | yes, I think there could be use cases for that | 12:39 |
| sean-k-mooney | you can technically express that in the schdme but i didnt thinkw e had any documented exampels or test to supprot that | 12:39 |
| jgilaber | I put one in the bug description, not sure how realistic it is | 12:40 |
| sean-k-mooney | i mean i guess its reasonabel if you wanted to split or merge you high performace and mass storage voluem | 12:40 |
| amoralej | yep ^ that's a valid case | 12:40 |
| sean-k-mooney | i.e. start with a san that is use for both and hten split them | 12:40 |
| sean-k-mooney | or do the reverse | 12:41 |
| jgilaber | that right now passes the schema validation | 12:41 |
| jgilaber | but the strategy does not do what the user I think would expect | 12:41 |
| sean-k-mooney | yes but the input validation in the api is not great | 12:41 |
| sean-k-mooney | we coudl supprot this but it feel more like an incompelte feature | 12:42 |
| sean-k-mooney | i.e. they never finished implementing it | 12:42 |
| sean-k-mooney | did you check fi this was described in the spec for zone migration | 12:42 |
| jgilaber | yes, I feel the same | 12:42 |
| dviroel | so maybe we should work on a schema validation of what is supported? and then improve the strategy to support different combinations? | 12:42 |
| sean-k-mooney | well not everything shoudl be checked in the schema | 12:42 |
| jgilaber | I don't think we can validate this case in the schema | 12:42 |
| sean-k-mooney | the schmea is for syntax not semantics | 12:43 |
| sean-k-mooney | semantics are validate after the schma validation | 12:43 |
| dviroel | that would be validate in python then | 12:43 |
| sean-k-mooney | so i think we coud supprot this, im not sure we would backprot this however | 12:43 |
| sean-k-mooney | i.e. this feels like something that is valid to fix on master but not valid to include in stable branches | 12:44 |
| jgilaber | the way the strategy works I think it's kind of implicit that it should be supported, but it was not tested, like many other things | 12:44 |
| jgilaber | sounds good to me to make it master only | 12:44 |
| sean-k-mooney | if its not tested and there are no docs i default to implecit is not enough | 12:44 |
| sean-k-mooney | lets see how invaisve it end up being | 12:44 |
| sean-k-mooney | and we can think about the backportablity then based on that | 12:45 |
| amoralej | jgilaber, is https://review.opendev.org/c/openstack/watcher/+/964717 enough to make that use case work fine? | 12:45 |
| jgilaber | the fix is quite small, just 14 lines of code | 12:45 |
| sean-k-mooney | oh right you have 2 patches up | 12:45 |
| jgilaber | amoralej, no in that patch I only added tests to reproduce | 12:46 |
| amoralej | sorry, i mean in the serie | 12:46 |
| amoralej | https://review.opendev.org/c/openstack/watcher/+/964718/ | 12:46 |
| jgilaber | yes, the second patch in the series is the fix | 12:46 |
| jgilaber | I don't expect anything else would be needed | 12:46 |
| sean-k-mooney | oh well lets mark it as medium i guess, jgilaber can you assing it to your self and then we can deciend based on teh review how to proceed once its fixed on master | 12:46 |
| jgilaber | sure, mediums seems fine | 12:46 |
| jgilaber | thanks for the discussion! I think we're done with this one | 12:47 |
| jgilaber | if there no more questions/comments we can continue with the other bugs | 12:47 |
| sean-k-mooney | we have been bit by pelople abusing input validation bugs in the past to achive something that we never inteded to supprot creating persitent tech debt | 12:47 |
| sean-k-mooney | so im alwasy causious with enablign supprot for a fucntionality if it was not documented as intended as a result | 12:47 |
| jgilaber | I can see how that could become a problem | 12:48 |
| sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1915282 | 12:48 |
| sean-k-mooney | that was orginally a secuity bug that we could not fix becuase folks built a feature in neutron that depended on it | 12:49 |
| jgilaber | that sounds like quite a headache | 12:50 |
| jgilaber | if there no more comments, we can cover the second bug | 12:50 |
| jgilaber | #link https://bugs.launchpad.net/watcher/+bug/2133505 | 12:50 |
| jgilaber | this one is from dviroel | 12:50 |
| dviroel | ah | 12:51 |
| dviroel | I opened as a RFE | 12:51 |
| dviroel | to track the limitation on applier running in native thread mode | 12:52 |
| sean-k-mooney | ah so as a tracker to note we need to eventually fix that | 12:52 |
| dviroel | since we merged the support for native threads in applier already | 12:52 |
| dviroel | so yes, we need to fix that before removing eventlet code | 12:52 |
| sean-k-mooney | i dont reacall if we left a fixme or todo comment for that in the code but ya we should eventrually fix that | 12:52 |
| jgilaber | so we should mark that as high right? | 12:53 |
| sean-k-mooney | i would say maybe hight ya | 12:53 |
| sean-k-mooney | although the actul fix migh tnot be somethign we end up tracking as a bug depending on what ti becomes | 12:53 |
| dviroel | high should be fine, since it is really important to fix | 12:53 |
| jgilaber | so for now we mark as triaged+high, agreed? | 12:53 |
| sean-k-mooney | +1 | 12:53 |
| dviroel | yeah, it may touch many places in the code | 12:53 |
| dviroel | +1 | 12:53 |
| jgilaber | ok, done, thanks for tracking that dviroel | 12:54 |
| jgilaber | we have a couple more bugs | 12:54 |
| jgilaber | #link https://bugs.launchpad.net/watcher-tempest-plugin/+bug/2133181 | 12:54 |
| dviroel | the next one is a tempest plugin | 12:54 |
| dviroel | which sporadically appears in ci | 12:54 |
| dviroel | action plan is created as defailt as Recommended | 12:55 |
| dviroel | and then goes to the correct state | 12:55 |
| dviroel | i.e.: if there is no actions, it is moved to Succeeded | 12:55 |
| sean-k-mooney | well recommend its not invalid | 12:55 |
| sean-k-mooney | i assume we are using the auto apply | 12:55 |
| sean-k-mooney | the tempest test shoudl handel that and keep checkign if its recommend until it get to succeeded ro canceled | 12:56 |
| jgilaber | it seems that that is precisely what the fix does | 12:57 |
| jgilaber | #link https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/968750 | 12:57 |
| dviroel | there is a fix propose yes | 12:57 |
| sean-k-mooney | yep more or less | 12:57 |
| sean-k-mooney | its wiating for it to be in a terminal state | 12:57 |
| sean-k-mooney | whic avoid the intermidary value | 12:57 |
| sean-k-mooney | so that look correct to me | 12:58 |
| sean-k-mooney | although im unsure about the valueeror | 12:58 |
| dviroel | yeah, and after that there is the real check for the expected state | 12:58 |
| sean-k-mooney | how/when is that raised | 12:58 |
| dviroel | hum, that's a copy and paste from other places, but I can double check if is correct | 12:59 |
| sean-k-mooney | i hope we are not rasing ValueError to signal that call_until_ture timeed out | 12:59 |
| sean-k-mooney | well its ok to raise an exepcitn of we time out | 12:59 |
| sean-k-mooney | it just shoudl be StopIteration or simialr | 12:59 |
| sean-k-mooney | or we shoudl just return false | 13:00 |
| sean-k-mooney | in either case i dont thnk we need the try excpet | 13:00 |
| sean-k-mooney | if we rase any expction it will fail the test | 13:00 |
| sean-k-mooney | if we return false we will faill the assertTrue | 13:00 |
| sean-k-mooney | this give a slightly better errror message but that all | 13:01 |
| dviroel | yeah ack | 13:01 |
| * dviroel time check | 13:01 | |
| dviroel | so yeah, pls review this one | 13:01 |
| jgilaber | thanks dviroel | 13:01 |
| sean-k-mooney | we can probaly proceed as it is and look at this laster | 13:01 |
| dviroel | an chandan's one, that is the next on the list | 13:01 |
| jgilaber | we can continue the discussion on the patch | 13:02 |
| jgilaber | we have one final bug | 13:02 |
| jgilaber | #link https://bugs.launchpad.net/watcher-tempest-plugin/+bug/2133777 | 13:02 |
| jgilaber | that is also an issue seen in ci | 13:02 |
| jgilaber | which also has a fix open | 13:02 |
| jgilaber | #link https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/969366 | 13:02 |
| jgilaber | since we're out of time we can discuss them in the reviews | 13:02 |
| dviroel | yep | 13:03 |
| jgilaber | last topic | 13:03 |
| jgilaber | #topic Volunteers to chair next meeting | 13:03 |
| amoralej | i can take it | 13:03 |
| chandankumar | i can take it | 13:03 |
| jgilaber | amoralej is our lucky winner :) thanks! | 13:03 |
| * dviroel can take it too :) | 13:03 | |
| jgilaber | that's all for today, thanks everyone! | 13:03 |
| dviroel | thanks jgilaber o/ | 13:04 |
| jgilaber | #endmeeting | 13:04 |
| opendevmeet | Meeting ended Thu Dec 4 13:04:04 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 13:04 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/watcher/2025/watcher.2025-12-04-12.01.html | 13:04 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/watcher/2025/watcher.2025-12-04-12.01.txt | 13:04 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/watcher/2025/watcher.2025-12-04-12.01.log.html | 13:04 |
| morenod | thanks jgilaber++! | 13:04 |
| amoralej | thanks jgilaber++ | 13:04 |
| opendevreview | David proposed openstack/watcher master: [DNM] Testing nodeset with three nodes (two computes + 1 controller) https://review.opendev.org/c/openstack/watcher/+/967331 | 16:14 |
| opendevreview | Douglas Viroel proposed openstack/watcher-specs master: Add spec for Strategy Pipeline feature https://review.opendev.org/c/openstack/watcher-specs/+/969840 | 19:23 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!