Thursday, 2026-06-11

opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Playwright Django52 job  https://review.opendev.org/c/openstack/watcher-dashboard/+/99258202:56
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright-based E2E testing framework  https://review.opendev.org/c/openstack/watcher-dashboard/+/97035304:03
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test for skip action workflow  https://review.opendev.org/c/openstack/watcher-dashboard/+/97659404:09
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test for skip action workflow  https://review.opendev.org/c/openstack/watcher-dashboard/+/97659404:12
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Playwright Django52 job  https://review.opendev.org/c/openstack/watcher-dashboard/+/99258204:13
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright-based E2E testing framework  https://review.opendev.org/c/openstack/watcher-dashboard/+/97035305:35
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright-based E2E testing framework  https://review.opendev.org/c/openstack/watcher-dashboard/+/97035306:48
opendevreviewAlfredo Moralejo proposed openstack/watcher master: Remove Nova API calls from zone_migration strategy  https://review.opendev.org/c/openstack/watcher/+/98828407:45
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test for skip action workflow  https://review.opendev.org/c/openstack/watcher-dashboard/+/97659407:51
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test job for Django 5.2  https://review.opendev.org/c/openstack/watcher-dashboard/+/99258207:56
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test job for Django 5.2  https://review.opendev.org/c/openstack/watcher-dashboard/+/99258207:56
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright-based E2E testing framework  https://review.opendev.org/c/openstack/watcher-dashboard/+/97035308:01
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test for skip action workflow  https://review.opendev.org/c/openstack/watcher-dashboard/+/97659408:01
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test for skip action workflow  https://review.opendev.org/c/openstack/watcher-dashboard/+/97659410:02
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test for skip action workflow  https://review.opendev.org/c/openstack/watcher-dashboard/+/97659410:25
opendevreviewchandan kumar proposed openstack/watcher-dashboard master: Add Playwright integration test job for Django 5.2  https://review.opendev.org/c/openstack/watcher-dashboard/+/99258211:10
opendevreviewWinicius Allan Bezerra da Silva proposed openstack/watcher-specs master: [WIP] Add spec for Preemptible Instances feature  https://review.opendev.org/c/openstack/watcher-specs/+/98717111:17
dviroelhi all - watcher weekly meeting will start in 10min - feel free to add topic to today's agenda:11:50
dviroelhttps://etherpad.opendev.org/p/openstack-watcher-irc-meeting#L3211:50
dviroel#startmeeting watcher12:01
opendevmeetMeeting started Thu Jun 11 12:01:10 2026 UTC and is due to finish in 60 minutes.  The chair is dviroel. Information about MeetBot at http://wiki.debian.org/MeetBot.12:01
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.12:01
opendevmeetThe meeting name has been set to 'watcher'12:01
* dviroel 1 min late :) 12:01
dviroelo/ hi all12:01
winiciusallan[m]o/12:01
morenodo/12:01
jgilabero/12:02
chandankumaro/12:02
dviroelcourtesy ping: amoralej sean-k-mooney rlandy12:02
rlandyo/12:02
amoralejo/12:02
dviroelthanks for joining o/12:02
dviroellet's start with today's meeting agenda12:03
dviroel#link https://etherpad.opendev.org/p/openstack-watcher-irc-meeting#L32 (Meeting agenda)12:03
dviroelas usual, feel free to add your own topics to the agenda12:03
dviroelnote that there is a topic to place your changes that requires attention from reviewers12:03
dviroelwe also have a topic for bugs, if you want to discuss any12:03
dviroellets start with the first one...12:03
dviroel#topic Preemptible Instances Spec12:04
dviroel#link https://review.opendev.org/c/openstack/watcher-specs/+/98717112:04
dviroelwiniciusallan[m]: thanks for adding the topic12:04
dviroelwant to share latest changes made to the spec?12:04
sean-k-mooneyo/12:04
dviroelsome of us already started reviewing it12:04
dviroelbut it seems that winiciusallan[m] added more content recently12:05
winiciusallan[m]yeah, thank you doug12:05
winiciusallan[m]hi all, sorry for not attending to the last meeting12:05
winiciusallan[m]i was back and forthing trying to define a concrete strategy12:06
winiciusallan[m]i believe i got something now. thank you all for all the feedbacks12:06
winiciusallan[m]basically what I want to achieve is to guarantee a % of free allocated resources12:07
winiciusallan[m]to guarantee this, we preempt the so-called preemptible instnaces12:07
winiciusallan[m]I put more details in the spec, but the idea here is a provider can utilize their spare capacity with preemptible instances, but also control the on-demand pool by guaranteeing this amount of resource12:08
winiciusallan[m]this percentage will be set via input_parameter12:08
amoralejI was reviewing it right before the meeting, sorry i didn't reply yet12:08
winiciusallan[m]I'm proposing two ways of sorting and selecting candidates instances: by oldest or by utilization12:09
winiciusallan[m]also put more details in the spec, feedbacks are really appreciated12:09
winiciusallan[m]the policies to preempt instances didn't change, I just define better how this will be configured and a default value12:10
winiciusallan[m]I believe that's all from my side12:10
amoraleji have a global question, this would monitor resource allocations or usage? or both?12:10
winiciusallan[m]thats fine amoralej :)12:10
dviroelyeah, that's an important question12:11
winiciusallan[m]the optimization target would be regarding the allocated, but we will need to monitor the usage in case the operator would like to preempt based on the utilization12:11
winiciusallan[m]does it make sense?12:11
amoralejthat sentence is confusing to me, sorry :) 12:11
dviroelso the Goal is to free allocated resource, to give space for new instances? 12:12
winiciusallan[m]yeah, that's right12:12
sean-k-mooneywiniciusallan[m]: i think what amoralej  is highlight is we currently have diffent stagees for the same goal12:12
sean-k-mooneyand some use allcoation only12:12
sean-k-mooneyother use allcoation+metics form teh data socue about utilisation12:12
sean-k-mooneywiniciusallan[m]: so in the spec we shoudl descieb what you intend to use12:13
sean-k-mooneywill it be configuabe12:13
sean-k-mooneywill it be only the placment/flavor allcation ifno12:13
sean-k-mooneyor will you take cpu utilsitiaon metics from teh data souce12:13
sean-k-mooneythere is no wrong answe rhere realy12:13
amoralejlet me give you a case. A cluster is 50% of CPUs allocation but 95% of usage (because of overcommitment of 4). Should the strategy stop instances?12:13
sean-k-mooneyjsut we need to docuement it in the spec12:14
sean-k-mooneyexciletn question i would say no12:14
amoralejuhh :)12:14
sean-k-mooneybut the operaotr coudl pass that in vai the treshold12:14
amoraleji was thinking yes, tbh12:14
amoralejbut right, its up to debate12:14
sean-k-mooneywe dont want to premet if we are not out of capsity12:15
winiciusallan[m]amoralej: no, the strategy is based on the allocation12:15
sean-k-mooneybut it depend on the input parmaters12:15
winiciusallan[m]because we would like guarantee free capacity12:15
sean-k-mooneyif the treshold was 40%12:15
sean-k-mooneythen yes it woudl peremet instance because the allcoation is over 50%12:15
sean-k-mooney*over 40%12:15
winiciusallan[m]yeah, thank you sean12:15
amoralejyes, i was thinking in threshold 75 i.e.12:15
winiciusallan[m]in this case, no instance will be preempted12:16
dviroelI see that allocation vs real usage are 2 different use cases12:16
sean-k-mooneyyep12:16
sean-k-mooneythe real susage is gettign clsoer to service assuarnce12:16
dviroelif the goal is to free allocation, shouldn't the strategy also check preemtible instances allocation only?12:16
amoraleji agree, but both may be treated in the same strategy12:16
sean-k-mooneyi.e. nosiy neighbough12:16
sean-k-mooneythey could or they coudl eb diffent12:17
sean-k-mooneybut that is what we need to deciend in the spec12:17
sean-k-mooneyand we can alwasy add new ones or extend them lather12:17
amoralejwe may have two sets of threashold12:17
amoralejallocation_threshold and usage_threshold12:17
sean-k-mooneyso i woudl focus on allcoation first but yes 2 threshodl for allcoation and usage coudl be ok12:17
dviroel+ the resource to be checked right12:18
dviroelcpu, memory12:18
amoralejand disk ?12:18
amoraleji mean, local disk12:18
dviroelyep12:18
jgilaberif we do that, we'll need to pay attention to the UX12:18
jgilaberwith that amount of parameters it could be confusing for users12:19
winiciusallan[m]just to ensure that I'm in the same page. in the case of utilization, are you guys talking about guarantee a % of utiliaztion instead of allocation?12:19
jgilaberwe could end up like with zone migration12:19
winiciusallan[m]or when selecting candidate instances?12:19
amoralejanother question, we'd account the resources globally (per-cluster or per-scope in the audit) or per-host?, i assume globally12:19
winiciusallan[m]good question amoralej. we can use audit scope for this, but I'm assuming for now cluster-wide resources12:20
winiciusallan[m]but would be a good addition (maybe in the future) to be able to scope to host or aggregate12:20
dviroelhumm12:21
amoralejscope or cluster should be transparent for the audit, my question was more to ensure tha it's not per-host12:21
dviroeli think that it depends12:21
sean-k-mooneywiniciusallan[m]: so utilisation need to be per host12:21
sean-k-mooneyand vm on that host12:21
sean-k-mooneynot lgoabl12:21
sean-k-mooneyso if we had a usage treshold of 75% we woudl need to evulate the total workload utilissation on a host vs its capstiy12:21
sean-k-mooneyand then if we exceed that for host with permtable isntance we woudl need to condier premeting them12:22
dviroelglobally, we may end up freeing resources on different hosts, but still not have capacity for a new instance12:22
winiciusallan[m]yeah, it makes sense12:22
sean-k-mooneyyou could use your pipeline feature12:22
winiciusallan[m]so we would end up with fragmentation12:22
sean-k-mooneyso you would do a premept pass folod by vm consolidation12:22
sean-k-mooney*followed12:23
dviroelright12:23
dviroelmaybe allocation-based strategy could work globally12:23
winiciusallan[m]sean-k-mooney: this idea of usage threshold is more about in a noisy neighbour situation right?12:24
dviroelusage-based strategy I'm not sure. You may want a behavior like noisy neighbor for a usage-based strategy..12:24
winiciusallan[m]what I'm indeed thinking is allocation to ensure that VMs can be created12:24
dviroelthen it would be a per host analysis12:24
amoralejit's tricky12:25
dviroelso I think that allocation vs usage could even be 2 different strategies in the end?12:25
jgilaberI was thinking that same thing dviroel 12:25
amoralejbut they may interact poorly12:25
amoraleji.e. if a spot instance is going to be removed because of allocation limit, it's usage should take into account when calculating usage12:26
amoralejor the other way around12:26
sean-k-mooneydviroel: i was expecitng them to be seprate stragies as well yes12:27
amoraleji think spot instances should be treated in the same12:27
amoraleji mean, both use cases in the same12:27
dviroelbut the use case that winiciusallan[m] wants to cover is more related to allocation. A No valid Host error could be a trigger for this Audit. Is that it winiciusallan[m] ?12:27
sean-k-mooneyi think those are two diffent optimsation approch so combining them seam more complicte to start with12:28
sean-k-mooneyfor what it worht i woudl be fien if thies state out only based on allcaotion12:28
amoralejok, we can start with allocations only, but let's clearly explain it12:28
winiciusallan[m]dviroel: a NoValidHost should not be trigger i would say12:28
sean-k-mooneyand we defered usage based premetion till later unless winiciusallan[m] wants to do that now12:28
winiciusallan[m]but it can happens12:28
amoralejgiven that spot instances may affect performance on non-spot12:29
sean-k-mooneyamoralej: i woudl really not like to bring that into the scope fo this12:29
sean-k-mooneynot until we have the allcoation based premation doen first12:29
sean-k-mooneythe oginaly converation about usage12:29
sean-k-mooneywas if i have 2 vms that can be premetned12:29
sean-k-mooneyon a given host12:29
winiciusallan[m]amoralej: that is what was confusing me because you can follow a lot of heuristics to decide when preempt instances12:29
sean-k-mooneywhich one do i choose12:29
sean-k-mooneythe oldes or the least used12:30
winiciusallan[m]but I think it will be simpler go for the allocation case12:30
sean-k-mooneyso there are 2 seperat thigns shoudl we prement and if so what do we prement12:30
amoralejok, i'm fine with going only with the allocation case, just let's clearly document12:30
sean-k-mooneyusage and allcaotion are interesting for both12:30
winiciusallan[m]ack amoralej12:31
sean-k-mooneywiniciusallan[m]: what i woudl preopsoe for v1 (we can go into more detail in the spec) is this 12:31
sean-k-mooneyuse allcation as teh treshsold for shoudl we prement12:32
sean-k-mooneyand supprto 2-3 poslics for which premetabel isnts to prement12:32
sean-k-mooneyage size adn usage are the 3 for which to prement that i think make sense12:32
sean-k-mooneyafter we have decied based ont the threshold tha twe should preempt12:32
winiciusallan[m]ack12:33
sean-k-mooneywiniciusallan[m]: if  you have other ideas of course feel free to propsoe those12:33
winiciusallan[m]i've already put two of these12:33
winiciusallan[m]will work on adding one more12:33
sean-k-mooneyill try and make tiem for this spec before the next meeting12:33
winiciusallan[m]ack12:34
winiciusallan[m]thank you all for the feedback :)12:34
dviroelok, it seems that we are aligned with this12:35
dviroelwe then can focus on reviewing the spec in the following week12:35
dviroeland discuss more in gerrit or in next meetings12:35
dviroelany other question or concern about that?12:36
winiciusallan[m]nothing from my side12:36
winiciusallan[m]thanks for raising my topic12:36
dviroelas for the next topic that I added, we have a release schedule to follow12:36
dviroelthanks winiciusallan[m] for bringing the discussion here12:36
dviroel#topic Hibuscus Schedule12:37
dviroel#link https://static.openstack.org/project/releases.openstack.org/hibiscus/schedule.html12:37
dviroelwe are 3 weeks aways from milestone 212:37
dviroelwhich is our spec freeze deadline too12:37
sean-k-mooneywe are also jsut past the mid poing fo the cyle i belive12:38
dviroelsome teams have different deadlines, and have their spec freeze this week for instance12:38
sean-k-mooneyya nova is very very early this release12:38
dviroelyeah, nova and manila (R-16)12:39
dviroelwe are in the schedule with the specs open/merged12:39
dviroeljgilaber also added a reminder about release tasks, thanks for following up12:39
dviroelanything else related to the schedule?12:40
dviroelotehrwise we can move to Reviews12:40
jgilaberwhen the time comes we can think if we want to make a stable release12:40
jgilaberwe don't have a lot of changes right now12:40
dviroelack, we need to focus on merging some backports too12:41
jgilaberwe have a few for the 2025 branches and just one for 2026.112:41
sean-k-mooneymaybe do a stabel review day in the next week or two12:41
dviroelyeah, as we discussed at the ptg, that would help12:41
sean-k-mooneywhy not next thrsday12:42
sean-k-mooneywe can do soem revdiew and check in on the progress in the meeting12:42
dviroelit works for me12:42
jgilaber+112:43
amoralej+112:43
dviroel#action stable review day next thurday Jun 18th12:43
dviroelI may start on 17th :) 12:44
dviroelok, lets move to next topic12:44
dviroelrunning out of time12:44
dviroel#topic Reviews12:44
dviroel#link  https://review.opendev.org/q/topic:%22use-openstacksdk%22+project:openstack/watcher+status:open12:44
dviroeljgilaber: o/12:44
jgilaberI've pushed a few more patches to migrate the placement helper to use the sdk12:44
dviroel++12:44
jgilaberI remove some unused methods in the very first patch12:45
dviroeli love to delete unused code12:45
jgilaberand then I do the usual creating classes, swithcing + cleanup at the end12:45
jgilaberlike keystone, the placement helper has little functionality, so the patches are not too big12:45
dviroelack, the L patches seems to be more delete than add12:46
jgilaberthe second link is for an old patch to remove  the retry logic we had in the NovaHelper12:46
jgilaber#link https://review.opendev.org/c/openstack/watcher/+/97549812:46
* dviroel not this one :P 12:47
jgilaberI found another bug in the sdk regarding the retries code so I had to wait for another release12:47
jgilaberbut this week it came out and I could test everything and it seems to work12:47
jgilaberI've kept the old defaults with the new parameters12:47
jgilaber3 retries, 2 seconds delay12:47
jgilaberthe patch will fail in CI until the uppers-constraints in the requirement repo is updated for the sdk12:48
dviroelok so the latest fix is in openstacksdk 4.15 12:48
jgilaberthat's right12:48
jgilaberthat's all I wanted to say, we can address any feedback in gerrit12:49
dviroelack for the upper constraints, thanks for persuing and fixing this12:49
dviroeljgilaber++12:49
amoralejyep, thanks jgilaber, i'll try to review part of that at least in the incoming days12:49
dviroelany other questions folks?12:49
dviroelnext is12:49
dviroel"Replace the process-wide global lock @lockutils.synchronized("model_root") with an per instance RLock"12:50
dviroel#link https://review.opendev.org/c/openstack/watcher/+/987540 12:50
dviroel#link https://review.opendev.org/c/openstack/watcher/+/98829712:50
dviroelthe second link is the regression test12:50
dviroeljgilaber: sean-k-mooney chandankumar morenod ^ please review when you have some time12:50
dviroelthis fix should improve our strategies by replacing the current locking system12:51
amoralejit does, actually :)12:51
dviroelsince strategies do a copy of the model, per-instance lock makes more sense here12:52
dviroelamoralej: TY for testing the changes++12:52
amoralejin my tests with workload_stabilization, i.e. it reduces execution time to almost half12:52
dviroelthese ones fixes the locking model12:53
dviroelthere are 2other ones that improves workload_stabilization algorithym12:53
dviroelwhich we may discuss in gerrit on in another meeting12:53
amoralejyep, that can be a long discussion :) 12:54
dviroel#link https://review.opendev.org/c/openstack/watcher/+/98829712:54
dviroeland12:54
dviroel#link https://review.opendev.org/c/openstack/watcher/+/98853312:54
dviroelthanks amoralej and winiciusallan[m] for reviewing them :) 12:55
dviroelack, the next one is12:55
dviroel"Fix scalability for vm_workload_consolidation"12:55
dviroel#link https://review.opendev.org/c/openstack/watcher/+/99267612:55
dviroelthis is new right12:55
dviroelamoralej: 12:55
amoraleji just sent that today12:55
amoralejin my scale tests with simulator, that fixes the main scalability issues12:56
amoralejin the real scalability tests we did some months ago, we didn't get the strategy finishing after waiting more than one hour12:56
amoralejwe got the same result with the simulator12:56
amoralejand with the patch, it takes ~ 14 minutes in my environment12:57
amoralejactually, i'd expect to take less time in a real environment, but i think it's a good test12:57
amoralejso only that, keep that one in mind when you get some review time :) 12:58
* dviroel my chrome is freezing with some gerrit changes...12:58
dviroelack, thanks amoralej 12:58
dviroeli will take a look, I already have a question in mind, but I need to look your patch with details.12:59
dviroel#topic Bugs12:59
dviroelwe are out of time,but morenod bugs are there for a while12:59
dviroelmorenod: do you want to highlight one of them?12:59
morenodI updated this morning the descriptions, with more context and reproducing steps13:00
dviroelthanks for that13:00
amoralejyeah, much more clear now :) 13:00
dviroelit may help some ai agents reproducers :) 13:00
sean-k-mooneyso 13:01
sean-k-mooney#link https://bugs.launchpad.net/watcher/+bug/215480513:01
dviroelthey seems to have the same importance 13:01
sean-k-mooney#link https://bugs.launchpad.net/watcher/+bug/215480613:01
sean-k-mooney#link https://bugs.launchpad.net/watcher/+bug/215480713:01
sean-k-mooneycorrect13:01
dviroelyes13:02
sean-k-mooneyya so the dectipion look reasonable to me13:02
amoraleji think we discussed them to be medium ?13:02
sean-k-mooneyya i think medium is correct here13:02
jgilaber+1 medium seems reasonable13:03
sean-k-mooneyi guess the decision-engine tag13:03
sean-k-mooneywe dotn ahve scope so i think decision-engine  is the most fitting13:03
dviroelyeah13:03
jgilaber+113:04
dviroeli made the changes13:04
dviroelsomeone is planning to work on these?13:04
dviroelif yes, please assign it to you13:05
amoralejnot in next days at least13:05
dviroelso people know that someone is already working on it13:05
dviroelack13:05
sean-k-mooneyya probaly not in the next week or so13:05
sean-k-mooneybut if i do ill asgin it or check with folks first13:05
dviroelthanks folks13:06
dviroel#topic Volunteers to chair next meeting13:06
dviroelnext week we will circle back the stable branches reviews13:06
dviroeli can take the meeting again, unless someone wants to chair13:06
dviroelok, so we meet again next week13:07
dviroellet's wrap up for today13:07
dviroelsorry for extending the meeting13:07
dviroelthank you all for participating13:07
dviroel#endmeeting13:07
opendevmeetMeeting ended Thu Jun 11 13:07:46 2026 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:07
opendevmeetMinutes:        https://meetings.opendev.org/meetings/watcher/2026/watcher.2026-06-11-12.01.html13:07
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/watcher/2026/watcher.2026-06-11-12.01.txt13:07
opendevmeetLog:            https://meetings.opendev.org/meetings/watcher/2026/watcher.2026-06-11-12.01.log.html13:07
amoralejthanks dviroel for chairing!13:08
sean-k-mooneyby the way there si sort of 2 bugs in those bugs13:08
sean-k-mooney openstack server create --flavor m1.tiny --image cirros \13:08
sean-k-mooney    --availability-zone nova:compute-1 instance-113:08
sean-k-mooneywhen you adminsitvialy boot a hsot and force a host like that13:09
sean-k-mooneyyou are kind of pinning the vm to hte host13:09
sean-k-mooneyfi yo shelve and unshelve it for exampel we will try and put it back on the same host13:10
sean-k-mooneyso the sort of bug is watche has no awarness fo that in general13:10
sean-k-mooneyand that it shodl not try to mvoe vms that were forced to a specifc host in the first place13:11
dviroelack, to "force" a host during the test, it is easier to disable the other compute services, create the servers, and then enable the compute services again13:11
sean-k-mooneythe worklaod consodliation/blance strageis for example shoudl avoid movign them13:11
sean-k-mooneydviroel: this is also fine it does not realy invalidat ehte test in this specifc case13:12
sean-k-mooneybut just an aside that force host ectra is not somethign watcher really knows about13:12
sean-k-mooneywe will need to eventualy adress that13:12
dviroelyes13:12
sean-k-mooneyfor telcos this is genreally pretty imporant for normla clouds this is rarely used and kind of an anti pattern13:14
sean-k-mooneywhich is partly why we restict forcign a host to admin only in defautl policy13:14
opendevreviewMerged openstack/watcher master: Add regression test for map_instance reentrant deadlock  https://review.opendev.org/c/openstack/watcher/+/98829716:02

Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!