12:01:26 <rlandy> #startmeeting Watcher Meeting - 29 may 2025
12:01:26 <opendevmeet> Meeting started Thu May 29 12:01:26 2025 UTC and is due to finish in 60 minutes.  The chair is rlandy. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:01:26 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
12:01:26 <opendevmeet> The meeting name has been set to 'watcher_meeting___29_may_2025'
12:01:40 <rlandy> hi all .. who's around?
12:01:47 <dviroel> o/
12:01:48 <chandankumar> o/
12:02:04 <jgilaber> o/
12:02:16 <amoralej_> o/
12:02:28 <rlandy> Courtesy ping: sean-k-mooney
12:02:40 <sean-k-mooney> o/
12:02:46 <mtembo> o/
12:02:49 <rlandy> ok - let's get started
12:02:53 <sean-k-mooney> that working already :)
12:03:02 <rlandy> Topics for today are on:
12:03:06 <rlandy> #link https://etherpad.opendev.org/p/openstack-watcher-irc-meeting#L58
12:03:29 <rlandy> I start :)
12:03:39 <rlandy> #topic: (rlandy) Strategies marked [POC] "This is a proof of concept that is not meant to be used in production"
12:04:03 <rlandy> Two strategies that have active work on them are still marked POC in the doc:
12:04:17 <rlandy> #link https://docs.openstack.org/watcher/latest/strategies/host_maintenance.html
12:04:29 <rlandy> #link: https://docs.openstack.org/watcher/latest/strategies/workload_balance.html
12:04:50 <rlandy> examples of strategies that are NOT POC are included in the notes ...
12:05:01 <rlandy> #link https://docs.openstack.org/watcher/latest/strategies/zone_migration.html
12:05:04 <rlandy> not POC
12:05:06 <rlandy> so ...
12:05:30 <rlandy> do we want to keep that label POC  in the documentation if we have fully updated/tested/doc'ed a strategy?
12:06:08 <rlandy> the doc for both host maintenance and balance is being worked on for updates
12:06:21 <rlandy> and there are open bugs
12:06:42 <sean-k-mooney> thats a good question
12:07:04 <sean-k-mooney> so i want to move away form t he POC termonlogy in general and adopt experimental
12:07:41 <sean-k-mooney> so i woudl prefer to update the docs to do that and again we need to loop back to the idea of adding a supprot level matix in the docs too
12:07:49 <chandankumar> +1 to experimental
12:07:58 <sean-k-mooney> i think we can remvoe the code level comment and perhaps the doc comment
12:08:12 <sean-k-mooney> im not sure if we shoudl graduate it to fully supprot or not yet however
12:08:44 <chandankumar> How a strategy graduate from experimental/poc to supported in upstream?
12:08:49 <rlandy> what do we want to label zone migration as?
12:09:01 <sean-k-mooney> both are good questions.
12:09:08 <amoralej_> I'd prefer a different term that supported for upstream, but i agree on the idea
12:09:14 <sean-k-mooney> in most project we do not merge the code until we think its ready for production
12:09:29 <sean-k-mooney> so we would not have merge thse poc impleation at all
12:09:33 <sean-k-mooney> but since they exist
12:09:48 <sean-k-mooney> to me its wether we think its mature enogh to run in production
12:10:01 <sean-k-mooney> and are we happy that we can continue to test and maintain it going forward
12:10:10 <chandankumar> since we have fixed the strategies, now it is working with ample test coverage but we have not data on usage Do we still want to mark them experimental?
12:10:37 <sean-k-mooney> if we are happy with the level of testing and docs and the stragey is fucntioanl we can remvoe the experimental lable
12:10:52 <jgilaber> we discussed at PTG to have an overview/status of the project, should the strategies state be documented there?
12:11:28 <sean-k-mooney> yes we agreed to create a matrics in the docs that woudl docuemtn the testing and supprot level of all stragies
12:11:52 <sean-k-mooney> like this https://docs.openstack.org/nova/latest/user/support-matrix.html
12:12:16 <rlandy> so something like ...
12:12:27 <rlandy> on the top level strategies page:
12:12:57 <rlandy> strategy: test level experimental
12:13:14 <rlandy> and we remove all references of support level in other places
12:13:20 <rlandy> and we control it only there
12:13:26 <chandankumar> +1
12:13:31 <sean-k-mooney> yes i think so so here https://docs.openstack.org/watcher/latest/strategies/
12:13:56 <sean-k-mooney> when we move something ebtween levels i think we shoud create a relese note for that as well
12:14:05 <rlandy> as to which strategy gets what level, we can get feedback and agreement on the review
12:14:08 <rlandy> +1
12:14:16 <sean-k-mooney> i.e. when it goes form experiemtal to production ready we have a short relese not for that
12:14:35 <amoralej> +1
12:14:45 <sean-k-mooney> rlandy: yep we can do that in the review
12:14:45 <rlandy> ok - thank you all for the discussion
12:15:12 <rlandy> #action rlandy to put up a review with a table (removed from other places in the doc) for review
12:15:35 <rlandy> ie: of strategies and test/experimental level
12:15:39 <chandankumar> I have one more question
12:15:47 <rlandy> go ahead
12:15:54 <chandankumar> many strategies are not triggered via horizon UI
12:16:06 <rlandy> good point
12:16:07 <chandankumar> Do we want to document that also in each strategy doc?
12:16:11 <rlandy> those requiring params
12:16:13 <rlandy> yes!
12:16:28 <rlandy> in the table?
12:16:49 <jgilaber> +1, we can have an extra column with UI support
12:17:10 <chandankumar> +1 for extra column
12:17:11 <sean-k-mooney> we could
12:17:18 <chandankumar> Do we want to put it in main strategy page?
12:17:21 <mtembo> +1 extra column
12:17:25 <sean-k-mooney> the columns i was thinking of are as follws
12:18:07 <sean-k-mooney> name, testing level, supprot level(supproted, experimental, deprecation, removed)
12:18:22 <sean-k-mooney> we can add horizon supprot
12:18:37 <rlandy> removed?
12:18:38 <sean-k-mooney> or we can encode that in a similar table in teh watcher-dashbaord repo
12:18:41 <sean-k-mooney> and cross linked
12:19:00 <sean-k-mooney> rlandy: so sometiems we docuemnt the release in which a feature is remvoed
12:19:09 <sean-k-mooney> im not sure if we need that or not
12:19:22 <sean-k-mooney> but that more for upgrades
12:20:05 <sean-k-mooney> when you are upgrding to a new release you can review the matics to understand if all the stragies you use are still suprpoted. this would be captrued in an upgrades and deprecation release note as well
12:20:10 <sean-k-mooney> so its not required
12:20:29 <sean-k-mooney> we can defer the detail i think to the review
12:20:39 <rlandy> agreed - let's start with the review and continue the conversation there. thank you all for your input
12:20:55 <chandankumar> rlandy: thank you for bringing it!
12:21:01 <rlandy> anything else on this topic?
12:21:50 <rlandy> moving on ...
12:21:58 <rlandy> #topic (dviroel) Eventlet Removal
12:22:04 <dviroel> o/
12:22:06 <rlandy> dviroel, do you want to take this?
12:22:21 <dviroel> yep, first, wrt to the email sent last week, the call for MAAS maintainers, no answers so far
12:22:39 <rlandy> now what do we do?
12:22:41 <dviroel> so  I am planning to proceed with a patch to mark both MAAS and Ironic as experimental features, since they lack support, testing and docs
12:23:03 <dviroel> the question is
12:23:25 <dviroel> is enough just to update the docs (the plugins part)
12:23:43 <dviroel> or should we also consider adding some log messages?
12:24:48 <sean-k-mooney> we should likely add a log message and release note + a doc
12:25:07 <sean-k-mooney> the log message shoudl only log on startup IMO
12:25:16 <dviroel> ack, yeah, log once
12:25:22 <sean-k-mooney> neutron did that when they made linux bridge experimental
12:25:55 <dviroel> right, so I will proceed with a patch for that
12:26:04 <dviroel> tks for the inputs
12:26:07 <sean-k-mooney> just for consitency we likely shoudl havea support/testing matrics for datasouces and integrations as well
12:26:15 <sean-k-mooney> looking at https://docs.openstack.org/watcher/latest/admin/index.html
12:26:33 <sean-k-mooney> there is a high level page for datasouces https://docs.openstack.org/watcher/latest/datasources/index.html so that is simple
12:26:42 <sean-k-mooney> but im not seeing anythin for mass/ironic there
12:26:57 <sean-k-mooney> we might want to add an "services" section to the doc
12:27:13 <sean-k-mooney> wehre we list integration with nova cinder ironic mass ectra
12:27:31 <sean-k-mooney> i.e. the external services that watcher can interact with
12:28:19 <sean-k-mooney> there we can follow the same patteern as we jsut dicssed for rlandy topic
12:28:45 <dviroel> yep, i found mention to ironic only on change_node_power_state action doc
12:29:32 <sean-k-mooney> ya so the name of that is a littel hard "services" works but it could also be "integrations" or similar
12:29:53 <sean-k-mooney> maas makes it a littel werid because its not part of openstack
12:31:00 <sean-k-mooney> the complete lack of docs about ironic and mass is part of why they are experimental so we probly shoudl have a "docs level" column on these as well
12:31:28 <dviroel> yeah
12:32:07 <dviroel> any other ideas?
12:32:28 <rlandy> sounds right for now
12:32:30 <dviroel> i will take a look on docs and see what we can do
12:32:35 <dviroel> and we can chat more
12:32:41 <sean-k-mooney> dviroel: back to your question i think we can proceed with marking them as experimental and i would suggest startign a patch to do that and we can revisit next week.
12:32:42 <dviroel> and we can chat more
12:33:19 <rlandy> dviroel: thank you for raising this ... anything more on the topic?
12:33:29 <dviroel> next thing is about changes in BackgroundSchedulerService and Watcher Services
12:33:37 <dviroel> still in progress but, the good news is that the patch with the new threading backend for oslo.service merged recently
12:33:53 <dviroel> thread in ML:
12:33:54 <dviroel> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/33EPOBRCGEKYUVDPFUZJZ2HENGDXVRE4/
12:34:51 <dviroel> i have a open pacth for initial evenlet move, but I still need to updated it from comments
12:35:06 <dviroel> and that't what I have for this week
12:36:07 <rlandy> ok- thank you - any more comments on anything dviroel raised?
12:36:18 <dviroel> no, thanks
12:36:49 <rlandy> #action dviroel to look at doc additions
12:37:28 <rlandy> moving on ...
12:37:42 <rlandy> #topic (sean): just an fyi we are going to use https://etherpad.opendev.org/p/grian-ui-status to track/drive developemt of the grian-ui plugin
12:37:53 <rlandy> sean-k-mooney: ^^ would you like to take that?
12:38:12 <sean-k-mooney> am the short verson is as follows
12:38:47 <sean-k-mooney> durign the ptg we agreed with the telemery team to create a new horizon plugin for tenant metrics vizualisation
12:39:03 <sean-k-mooney> over the last 2-3 weeks we have created the repo and started bootstraping it
12:39:28 <sean-k-mooney> currently we are in the forming state choosing wehre we will host meeting, how we will track work ectra
12:39:56 <sean-k-mooney> we have decied to use that ether pad to drive that for now simialr to howe we are suign the etherpad for this meeting
12:40:19 <sean-k-mooney> we likely will use #openstack-telemetry to dicussion this topic going forward
12:40:49 <rlandy> +1 do you need any additional support/resources?
12:41:22 <sean-k-mooney> thats about it. for now no, we are hopign to have some intiall demos around m2 perhaps a littel later
12:41:58 <sean-k-mooney> we will likely starte with create 2 example staic dashbaord 1 an admin only dashboard
12:42:10 <sean-k-mooney> and a seocn that is tenatn facing to prove out both types
12:42:29 <sean-k-mooney> those will evlove over time
12:42:43 <sean-k-mooney> btu once we get to that initall pont we will likely ask for some user feedback
12:42:58 <sean-k-mooney> i think that is where the wider team can help
12:43:11 <sean-k-mooney> if you want to be more involed before that that fine too
12:43:11 <rlandy> thank you sean-k-mooney
12:43:49 <sean-k-mooney> anyway i think thats it unless there are any other questions?
12:44:27 <amoralej> let us known where you have something that we can install and test
12:44:46 <amoralej> i'd like to check the dashboards once ready
12:45:08 <chandankumar> Do we need anything from devstack-plugin-prometheus for grian-ui?
12:45:08 <sean-k-mooney> yep we will
12:45:19 <sean-k-mooney> chandankumar: not currently
12:45:25 <chandankumar> ok
12:45:29 <sean-k-mooney> i have a basic devstack job ready
12:45:41 <sean-k-mooney> it currently fails becasue we dont have a functial plugin yet
12:45:45 <sean-k-mooney> so the install fails as a result
12:46:00 <sean-k-mooney> but i have focused sofar on setign up the testing and jobs for the repo
12:46:15 <sean-k-mooney> so now that that is mostly done we can start on buildign out actual fucntionaliy
12:47:58 <rlandy> ok - anything more here?
12:48:08 <chandankumar> thank you sean-k-mooney for sharing the progress here!
12:48:47 <rlandy> moving on to give the other topics some time ...
12:48:52 <rlandy> #topic: (jgilaber) Updated zone migration strategy etherpad with what I think are the relevant use cases
12:49:05 <rlandy> #link https://etherpad.opendev.org/p/zone_migration_volume_migration_parameters
12:49:14 <rlandy> jgilaber: ^^ do you want to take this?
12:49:23 <jgilaber> yep
12:49:33 <jgilaber> I have added a section on the same etherpad that I brought last week
12:49:54 <jgilaber> with the few use cases for volume migration that I think the zone migration can support
12:50:00 <jgilaber> with the help of amoralej
12:50:18 <jgilaber> we came up with 3 scenarios
12:50:48 <jgilaber> 1 that should work currently, and 2 other that will require some changes to the volume migration action
12:51:13 <jgilaber> I also have a small sections with questions that I think we'll need to ask to cinder experts
12:51:41 <jgilaber> let me know if you see anything that you think is wrong or is not clear
12:52:30 <sean-k-mooney> what is the status of the vm migration part of this. is that still pending or did we merge the first two patches
12:52:49 <jgilaber> vm migration seems to work well
12:53:10 <jgilaber> we merged the path for the dst_node that I think is the only bug that affects that part
12:53:11 <sean-k-mooney> i think its good to continue working on the cinder part in followups but we can likely merge the intall partcial fix if we ahve nto already done so
12:53:20 <sean-k-mooney> ok good
12:53:30 <sean-k-mooney> i jsut didnt recall if that part was merged or not
12:54:05 <jgilaber> I've also changed the storage follow-up to that patch https://review.opendev.org/c/openstack/watcher/+/950149
12:54:26 <jgilaber> after the use cases review, I don't think it's great to have the dst_pool parameter mandatory
12:54:42 <sean-k-mooney> so the 3 types fo volume migratoin are retype, migration between host in a pool and migration between backend host in different pools correct?
12:55:09 <sean-k-mooney> jgilaber: yes i agree i talks to gmaan about this a litttle offline
12:55:25 <jgilaber> I think that's right sean-k-mooney
12:55:33 <sean-k-mooney> i think the way forward is to remvoe the required fields form the schema
12:55:57 <sean-k-mooney> and move the validation to the api where we can encode the logic of which combintions of optiosn are valid more cleanly
12:56:22 <sean-k-mooney> for example for retry neither the souce or dest pool shoudl eb requried as far as i am aware
12:56:55 <sean-k-mooney> *retype
12:56:59 <amoralej> wrt the cases, the one that should work currently, which is the retype (dst_pool empty) has some known bugs that we need to fix. There is a list at the end of the etherpad
12:57:23 <jgilaber> src_pool is required currently as it is the only way it has to filter which volumes to migrate
12:57:33 <jgilaber> that is a bug, because src_type is ignored
12:57:56 <sean-k-mooney> hum ok
12:58:40 <sean-k-mooney> that is perhaps unfortunate. it would be nice in the case wehre you are moving vms to be able to retype the volumes assocated with those vms and only those volumes
12:58:48 <jgilaber> btw there is also a list on known bugs at the bottom of the etherpad
12:59:21 <jgilaber> definitely we should fix that, we should be able to work with just the src_type
12:59:32 <sean-k-mooney> i.e. please mvoe the vms on host A in az 1 to AZ2 and retype there voluems form mass_storage_az1 to mass_storage_az2
13:00:45 <amoralej> sean-k-mooney, actually there is the opossite, migrate or retype the volumes in a pool and the vms with volumes attached to them
13:00:54 <amoralej> that exists already
13:01:06 <amoralej> (with certain bugs)
13:01:41 <sean-k-mooney> i htink oen of the bugs that is more impoant to fix is the lack of documenation
13:01:42 <amoralej> but not the "migrate the VMs and their attached volumes"
13:02:00 <sean-k-mooney> for such a complex stagy i think we will need to build out a set of usecase driven example
13:02:28 <amoralej> yep
13:02:35 <rlandy> (we are overtime but we can continue this discussion - will move the rest of the topics to next week)
13:02:42 <jgilaber> +1, that's what we set out to do in the etherpad
13:02:44 <sean-k-mooney> ack
13:02:58 <jgilaber> those use cases can then serve as the backbone of the documentation
13:03:09 <sean-k-mooney> ack
13:03:23 <sean-k-mooney> ill try and fined time to review that with more detail over the next week
13:03:38 <sean-k-mooney> thanks for writing this up
13:03:51 <jgilaber> thanks!
13:04:14 <rlandy> jgilaber: anything more on this topic?
13:04:21 <jgilaber> nop, that is all from me
13:04:28 <rlandy> ok - thank you
13:04:50 <rlandy> I will move the bug triage to next week or we can take that on line ...
13:04:55 <rlandy> pls note:
13:05:05 <rlandy> Request for review: https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/949557
13:05:18 <rlandy> chandankumar: ^^ any comments on that for the reviewers?
13:05:33 <chandankumar> It contains improvements from previous host maintenance tempest plugin patches
13:05:39 <chandankumar> Please have a look, thank you!
13:06:02 <rlandy> thank you chandankumar
13:06:13 <rlandy> moving on ...
13:06:24 <rlandy> volunteers to chair next week's meeting?
13:06:56 <chandankumar> count me in for next week
13:07:15 <rlandy> thank you chandankumar
13:07:38 <rlandy> thanks all for attending and the good discussions
13:07:51 <chandankumar> thank you rlandy !
13:08:14 <rlandy> pls take any topics we ran out of time for on to the channel or next week's agenda
13:08:20 <rlandy> #endmeeting