12:01:26 <rlandy> #startmeeting Watcher Meeting - 29 may 2025 12:01:26 <opendevmeet> Meeting started Thu May 29 12:01:26 2025 UTC and is due to finish in 60 minutes. The chair is rlandy. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:01:26 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 12:01:26 <opendevmeet> The meeting name has been set to 'watcher_meeting___29_may_2025' 12:01:40 <rlandy> hi all .. who's around? 12:01:47 <dviroel> o/ 12:01:48 <chandankumar> o/ 12:02:04 <jgilaber> o/ 12:02:16 <amoralej_> o/ 12:02:28 <rlandy> Courtesy ping: sean-k-mooney 12:02:40 <sean-k-mooney> o/ 12:02:46 <mtembo> o/ 12:02:49 <rlandy> ok - let's get started 12:02:53 <sean-k-mooney> that working already :) 12:03:02 <rlandy> Topics for today are on: 12:03:06 <rlandy> #link https://etherpad.opendev.org/p/openstack-watcher-irc-meeting#L58 12:03:29 <rlandy> I start :) 12:03:39 <rlandy> #topic: (rlandy) Strategies marked [POC] "This is a proof of concept that is not meant to be used in production" 12:04:03 <rlandy> Two strategies that have active work on them are still marked POC in the doc: 12:04:17 <rlandy> #link https://docs.openstack.org/watcher/latest/strategies/host_maintenance.html 12:04:29 <rlandy> #link: https://docs.openstack.org/watcher/latest/strategies/workload_balance.html 12:04:50 <rlandy> examples of strategies that are NOT POC are included in the notes ... 12:05:01 <rlandy> #link https://docs.openstack.org/watcher/latest/strategies/zone_migration.html 12:05:04 <rlandy> not POC 12:05:06 <rlandy> so ... 12:05:30 <rlandy> do we want to keep that label POC in the documentation if we have fully updated/tested/doc'ed a strategy? 12:06:08 <rlandy> the doc for both host maintenance and balance is being worked on for updates 12:06:21 <rlandy> and there are open bugs 12:06:42 <sean-k-mooney> thats a good question 12:07:04 <sean-k-mooney> so i want to move away form t he POC termonlogy in general and adopt experimental 12:07:41 <sean-k-mooney> so i woudl prefer to update the docs to do that and again we need to loop back to the idea of adding a supprot level matix in the docs too 12:07:49 <chandankumar> +1 to experimental 12:07:58 <sean-k-mooney> i think we can remvoe the code level comment and perhaps the doc comment 12:08:12 <sean-k-mooney> im not sure if we shoudl graduate it to fully supprot or not yet however 12:08:44 <chandankumar> How a strategy graduate from experimental/poc to supported in upstream? 12:08:49 <rlandy> what do we want to label zone migration as? 12:09:01 <sean-k-mooney> both are good questions. 12:09:08 <amoralej_> I'd prefer a different term that supported for upstream, but i agree on the idea 12:09:14 <sean-k-mooney> in most project we do not merge the code until we think its ready for production 12:09:29 <sean-k-mooney> so we would not have merge thse poc impleation at all 12:09:33 <sean-k-mooney> but since they exist 12:09:48 <sean-k-mooney> to me its wether we think its mature enogh to run in production 12:10:01 <sean-k-mooney> and are we happy that we can continue to test and maintain it going forward 12:10:10 <chandankumar> since we have fixed the strategies, now it is working with ample test coverage but we have not data on usage Do we still want to mark them experimental? 12:10:37 <sean-k-mooney> if we are happy with the level of testing and docs and the stragey is fucntioanl we can remvoe the experimental lable 12:10:52 <jgilaber> we discussed at PTG to have an overview/status of the project, should the strategies state be documented there? 12:11:28 <sean-k-mooney> yes we agreed to create a matrics in the docs that woudl docuemtn the testing and supprot level of all stragies 12:11:52 <sean-k-mooney> like this https://docs.openstack.org/nova/latest/user/support-matrix.html 12:12:16 <rlandy> so something like ... 12:12:27 <rlandy> on the top level strategies page: 12:12:57 <rlandy> strategy: test level experimental 12:13:14 <rlandy> and we remove all references of support level in other places 12:13:20 <rlandy> and we control it only there 12:13:26 <chandankumar> +1 12:13:31 <sean-k-mooney> yes i think so so here https://docs.openstack.org/watcher/latest/strategies/ 12:13:56 <sean-k-mooney> when we move something ebtween levels i think we shoud create a relese note for that as well 12:14:05 <rlandy> as to which strategy gets what level, we can get feedback and agreement on the review 12:14:08 <rlandy> +1 12:14:16 <sean-k-mooney> i.e. when it goes form experiemtal to production ready we have a short relese not for that 12:14:35 <amoralej> +1 12:14:45 <sean-k-mooney> rlandy: yep we can do that in the review 12:14:45 <rlandy> ok - thank you all for the discussion 12:15:12 <rlandy> #action rlandy to put up a review with a table (removed from other places in the doc) for review 12:15:35 <rlandy> ie: of strategies and test/experimental level 12:15:39 <chandankumar> I have one more question 12:15:47 <rlandy> go ahead 12:15:54 <chandankumar> many strategies are not triggered via horizon UI 12:16:06 <rlandy> good point 12:16:07 <chandankumar> Do we want to document that also in each strategy doc? 12:16:11 <rlandy> those requiring params 12:16:13 <rlandy> yes! 12:16:28 <rlandy> in the table? 12:16:49 <jgilaber> +1, we can have an extra column with UI support 12:17:10 <chandankumar> +1 for extra column 12:17:11 <sean-k-mooney> we could 12:17:18 <chandankumar> Do we want to put it in main strategy page? 12:17:21 <mtembo> +1 extra column 12:17:25 <sean-k-mooney> the columns i was thinking of are as follws 12:18:07 <sean-k-mooney> name, testing level, supprot level(supproted, experimental, deprecation, removed) 12:18:22 <sean-k-mooney> we can add horizon supprot 12:18:37 <rlandy> removed? 12:18:38 <sean-k-mooney> or we can encode that in a similar table in teh watcher-dashbaord repo 12:18:41 <sean-k-mooney> and cross linked 12:19:00 <sean-k-mooney> rlandy: so sometiems we docuemnt the release in which a feature is remvoed 12:19:09 <sean-k-mooney> im not sure if we need that or not 12:19:22 <sean-k-mooney> but that more for upgrades 12:20:05 <sean-k-mooney> when you are upgrding to a new release you can review the matics to understand if all the stragies you use are still suprpoted. this would be captrued in an upgrades and deprecation release note as well 12:20:10 <sean-k-mooney> so its not required 12:20:29 <sean-k-mooney> we can defer the detail i think to the review 12:20:39 <rlandy> agreed - let's start with the review and continue the conversation there. thank you all for your input 12:20:55 <chandankumar> rlandy: thank you for bringing it! 12:21:01 <rlandy> anything else on this topic? 12:21:50 <rlandy> moving on ... 12:21:58 <rlandy> #topic (dviroel) Eventlet Removal 12:22:04 <dviroel> o/ 12:22:06 <rlandy> dviroel, do you want to take this? 12:22:21 <dviroel> yep, first, wrt to the email sent last week, the call for MAAS maintainers, no answers so far 12:22:39 <rlandy> now what do we do? 12:22:41 <dviroel> so I am planning to proceed with a patch to mark both MAAS and Ironic as experimental features, since they lack support, testing and docs 12:23:03 <dviroel> the question is 12:23:25 <dviroel> is enough just to update the docs (the plugins part) 12:23:43 <dviroel> or should we also consider adding some log messages? 12:24:48 <sean-k-mooney> we should likely add a log message and release note + a doc 12:25:07 <sean-k-mooney> the log message shoudl only log on startup IMO 12:25:16 <dviroel> ack, yeah, log once 12:25:22 <sean-k-mooney> neutron did that when they made linux bridge experimental 12:25:55 <dviroel> right, so I will proceed with a patch for that 12:26:04 <dviroel> tks for the inputs 12:26:07 <sean-k-mooney> just for consitency we likely shoudl havea support/testing matrics for datasouces and integrations as well 12:26:15 <sean-k-mooney> looking at https://docs.openstack.org/watcher/latest/admin/index.html 12:26:33 <sean-k-mooney> there is a high level page for datasouces https://docs.openstack.org/watcher/latest/datasources/index.html so that is simple 12:26:42 <sean-k-mooney> but im not seeing anythin for mass/ironic there 12:26:57 <sean-k-mooney> we might want to add an "services" section to the doc 12:27:13 <sean-k-mooney> wehre we list integration with nova cinder ironic mass ectra 12:27:31 <sean-k-mooney> i.e. the external services that watcher can interact with 12:28:19 <sean-k-mooney> there we can follow the same patteern as we jsut dicssed for rlandy topic 12:28:45 <dviroel> yep, i found mention to ironic only on change_node_power_state action doc 12:29:32 <sean-k-mooney> ya so the name of that is a littel hard "services" works but it could also be "integrations" or similar 12:29:53 <sean-k-mooney> maas makes it a littel werid because its not part of openstack 12:31:00 <sean-k-mooney> the complete lack of docs about ironic and mass is part of why they are experimental so we probly shoudl have a "docs level" column on these as well 12:31:28 <dviroel> yeah 12:32:07 <dviroel> any other ideas? 12:32:28 <rlandy> sounds right for now 12:32:30 <dviroel> i will take a look on docs and see what we can do 12:32:35 <dviroel> and we can chat more 12:32:41 <sean-k-mooney> dviroel: back to your question i think we can proceed with marking them as experimental and i would suggest startign a patch to do that and we can revisit next week. 12:32:42 <dviroel> and we can chat more 12:33:19 <rlandy> dviroel: thank you for raising this ... anything more on the topic? 12:33:29 <dviroel> next thing is about changes in BackgroundSchedulerService and Watcher Services 12:33:37 <dviroel> still in progress but, the good news is that the patch with the new threading backend for oslo.service merged recently 12:33:53 <dviroel> thread in ML: 12:33:54 <dviroel> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/33EPOBRCGEKYUVDPFUZJZ2HENGDXVRE4/ 12:34:51 <dviroel> i have a open pacth for initial evenlet move, but I still need to updated it from comments 12:35:06 <dviroel> and that't what I have for this week 12:36:07 <rlandy> ok- thank you - any more comments on anything dviroel raised? 12:36:18 <dviroel> no, thanks 12:36:49 <rlandy> #action dviroel to look at doc additions 12:37:28 <rlandy> moving on ... 12:37:42 <rlandy> #topic (sean): just an fyi we are going to use https://etherpad.opendev.org/p/grian-ui-status to track/drive developemt of the grian-ui plugin 12:37:53 <rlandy> sean-k-mooney: ^^ would you like to take that? 12:38:12 <sean-k-mooney> am the short verson is as follows 12:38:47 <sean-k-mooney> durign the ptg we agreed with the telemery team to create a new horizon plugin for tenant metrics vizualisation 12:39:03 <sean-k-mooney> over the last 2-3 weeks we have created the repo and started bootstraping it 12:39:28 <sean-k-mooney> currently we are in the forming state choosing wehre we will host meeting, how we will track work ectra 12:39:56 <sean-k-mooney> we have decied to use that ether pad to drive that for now simialr to howe we are suign the etherpad for this meeting 12:40:19 <sean-k-mooney> we likely will use #openstack-telemetry to dicussion this topic going forward 12:40:49 <rlandy> +1 do you need any additional support/resources? 12:41:22 <sean-k-mooney> thats about it. for now no, we are hopign to have some intiall demos around m2 perhaps a littel later 12:41:58 <sean-k-mooney> we will likely starte with create 2 example staic dashbaord 1 an admin only dashboard 12:42:10 <sean-k-mooney> and a seocn that is tenatn facing to prove out both types 12:42:29 <sean-k-mooney> those will evlove over time 12:42:43 <sean-k-mooney> btu once we get to that initall pont we will likely ask for some user feedback 12:42:58 <sean-k-mooney> i think that is where the wider team can help 12:43:11 <sean-k-mooney> if you want to be more involed before that that fine too 12:43:11 <rlandy> thank you sean-k-mooney 12:43:49 <sean-k-mooney> anyway i think thats it unless there are any other questions? 12:44:27 <amoralej> let us known where you have something that we can install and test 12:44:46 <amoralej> i'd like to check the dashboards once ready 12:45:08 <chandankumar> Do we need anything from devstack-plugin-prometheus for grian-ui? 12:45:08 <sean-k-mooney> yep we will 12:45:19 <sean-k-mooney> chandankumar: not currently 12:45:25 <chandankumar> ok 12:45:29 <sean-k-mooney> i have a basic devstack job ready 12:45:41 <sean-k-mooney> it currently fails becasue we dont have a functial plugin yet 12:45:45 <sean-k-mooney> so the install fails as a result 12:46:00 <sean-k-mooney> but i have focused sofar on setign up the testing and jobs for the repo 12:46:15 <sean-k-mooney> so now that that is mostly done we can start on buildign out actual fucntionaliy 12:47:58 <rlandy> ok - anything more here? 12:48:08 <chandankumar> thank you sean-k-mooney for sharing the progress here! 12:48:47 <rlandy> moving on to give the other topics some time ... 12:48:52 <rlandy> #topic: (jgilaber) Updated zone migration strategy etherpad with what I think are the relevant use cases 12:49:05 <rlandy> #link https://etherpad.opendev.org/p/zone_migration_volume_migration_parameters 12:49:14 <rlandy> jgilaber: ^^ do you want to take this? 12:49:23 <jgilaber> yep 12:49:33 <jgilaber> I have added a section on the same etherpad that I brought last week 12:49:54 <jgilaber> with the few use cases for volume migration that I think the zone migration can support 12:50:00 <jgilaber> with the help of amoralej 12:50:18 <jgilaber> we came up with 3 scenarios 12:50:48 <jgilaber> 1 that should work currently, and 2 other that will require some changes to the volume migration action 12:51:13 <jgilaber> I also have a small sections with questions that I think we'll need to ask to cinder experts 12:51:41 <jgilaber> let me know if you see anything that you think is wrong or is not clear 12:52:30 <sean-k-mooney> what is the status of the vm migration part of this. is that still pending or did we merge the first two patches 12:52:49 <jgilaber> vm migration seems to work well 12:53:10 <jgilaber> we merged the path for the dst_node that I think is the only bug that affects that part 12:53:11 <sean-k-mooney> i think its good to continue working on the cinder part in followups but we can likely merge the intall partcial fix if we ahve nto already done so 12:53:20 <sean-k-mooney> ok good 12:53:30 <sean-k-mooney> i jsut didnt recall if that part was merged or not 12:54:05 <jgilaber> I've also changed the storage follow-up to that patch https://review.opendev.org/c/openstack/watcher/+/950149 12:54:26 <jgilaber> after the use cases review, I don't think it's great to have the dst_pool parameter mandatory 12:54:42 <sean-k-mooney> so the 3 types fo volume migratoin are retype, migration between host in a pool and migration between backend host in different pools correct? 12:55:09 <sean-k-mooney> jgilaber: yes i agree i talks to gmaan about this a litttle offline 12:55:25 <jgilaber> I think that's right sean-k-mooney 12:55:33 <sean-k-mooney> i think the way forward is to remvoe the required fields form the schema 12:55:57 <sean-k-mooney> and move the validation to the api where we can encode the logic of which combintions of optiosn are valid more cleanly 12:56:22 <sean-k-mooney> for example for retry neither the souce or dest pool shoudl eb requried as far as i am aware 12:56:55 <sean-k-mooney> *retype 12:56:59 <amoralej> wrt the cases, the one that should work currently, which is the retype (dst_pool empty) has some known bugs that we need to fix. There is a list at the end of the etherpad 12:57:23 <jgilaber> src_pool is required currently as it is the only way it has to filter which volumes to migrate 12:57:33 <jgilaber> that is a bug, because src_type is ignored 12:57:56 <sean-k-mooney> hum ok 12:58:40 <sean-k-mooney> that is perhaps unfortunate. it would be nice in the case wehre you are moving vms to be able to retype the volumes assocated with those vms and only those volumes 12:58:48 <jgilaber> btw there is also a list on known bugs at the bottom of the etherpad 12:59:21 <jgilaber> definitely we should fix that, we should be able to work with just the src_type 12:59:32 <sean-k-mooney> i.e. please mvoe the vms on host A in az 1 to AZ2 and retype there voluems form mass_storage_az1 to mass_storage_az2 13:00:45 <amoralej> sean-k-mooney, actually there is the opossite, migrate or retype the volumes in a pool and the vms with volumes attached to them 13:00:54 <amoralej> that exists already 13:01:06 <amoralej> (with certain bugs) 13:01:41 <sean-k-mooney> i htink oen of the bugs that is more impoant to fix is the lack of documenation 13:01:42 <amoralej> but not the "migrate the VMs and their attached volumes" 13:02:00 <sean-k-mooney> for such a complex stagy i think we will need to build out a set of usecase driven example 13:02:28 <amoralej> yep 13:02:35 <rlandy> (we are overtime but we can continue this discussion - will move the rest of the topics to next week) 13:02:42 <jgilaber> +1, that's what we set out to do in the etherpad 13:02:44 <sean-k-mooney> ack 13:02:58 <jgilaber> those use cases can then serve as the backbone of the documentation 13:03:09 <sean-k-mooney> ack 13:03:23 <sean-k-mooney> ill try and fined time to review that with more detail over the next week 13:03:38 <sean-k-mooney> thanks for writing this up 13:03:51 <jgilaber> thanks! 13:04:14 <rlandy> jgilaber: anything more on this topic? 13:04:21 <jgilaber> nop, that is all from me 13:04:28 <rlandy> ok - thank you 13:04:50 <rlandy> I will move the bug triage to next week or we can take that on line ... 13:04:55 <rlandy> pls note: 13:05:05 <rlandy> Request for review: https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/949557 13:05:18 <rlandy> chandankumar: ^^ any comments on that for the reviewers? 13:05:33 <chandankumar> It contains improvements from previous host maintenance tempest plugin patches 13:05:39 <chandankumar> Please have a look, thank you! 13:06:02 <rlandy> thank you chandankumar 13:06:13 <rlandy> moving on ... 13:06:24 <rlandy> volunteers to chair next week's meeting? 13:06:56 <chandankumar> count me in for next week 13:07:15 <rlandy> thank you chandankumar 13:07:38 <rlandy> thanks all for attending and the good discussions 13:07:51 <chandankumar> thank you rlandy ! 13:08:14 <rlandy> pls take any topics we ran out of time for on to the channel or next week's agenda 13:08:20 <rlandy> #endmeeting