12:00:29 <marios> #startmeeting watcher
12:00:29 <opendevmeet> Meeting started Thu Mar 20 12:00:29 2025 UTC and is due to finish in 60 minutes.  The chair is marios. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:00:29 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
12:00:29 <opendevmeet> The meeting name has been set to 'watcher'
12:00:52 <marios> hello o/ as usual agenda items are there https://etherpad.opendev.org/p/openstack-watcher-irc-meeting
12:01:04 <marios> we'll get going in a moment while people join
12:01:11 <marios> who is around today?
12:01:16 <chandankumar> o/
12:01:27 <amoralej> o/
12:01:41 <rlandy> o/
12:01:41 <marios> #info weekly meeting agenda at https://etherpad.opendev.org/p/openstack-watcher-irc-meeting
12:02:22 <marios> not a lot there so should be a brief one today. we'll take them as listed. feel free to add any topics that come to mind while we proceed.
12:02:56 <jgilaber_> o/
12:03:04 <marios> any comment before we begin?
12:03:19 <mtembo> o/
12:04:01 <marios> #topic     (rlandy): Anything additional for PTG? Joined seesion with telemetry requested for Wednesday
12:04:28 <marios> rlandy: do you want to introduce this? or something further to add besides ^^
12:04:28 <rlandy> thanks marios ...
12:04:41 <rlandy> so just what it says :)
12:05:02 <rlandy> #link https://etherpad.opendev.org/p/apr2025-ptg-watcher has our topics
12:05:24 <marios> thanks i was just looking for that to add a link
12:05:30 <rlandy> they all seem to have owners/chairs (thank you)
12:05:44 <rlandy> and then there are some requested joined sessions
12:05:48 <rlandy> one with nova
12:06:16 <rlandy> #link https://etherpad.opendev.org/p/nova-2025.2-ptg#L62
12:06:48 <rlandy> and one with horizon/telemetry (requested on wednesday - since that day was open for this group)
12:07:18 <rlandy> thanks for dviroel for organizing the watcher sessions
12:07:32 <rlandy> marios: that is all from my side - thank you
12:07:46 <marios> thank you rlandy
12:08:04 <marios> any other comments thoughts concerns or anything else on this topic from anyone?
12:09:43 <marios> ok moving on
12:09:48 <marios> #topic     Bug Triage
12:09:59 <marios> #link     https://bugs.launchpad.net/watcher/+bug/2103458
12:10:29 <marios> not sure who added this to the agenda there is no name. mtembo is that yours?
12:10:43 <marios> but is by amoralej
12:11:05 <marios> #info  workload_stabilization strategy does not report standard_deviation properly  https://bugs.launchpad.net/watcher/+bug/2103458
12:11:08 <amoralej> i created the ticket, right
12:11:14 <amoralej> i realized on that while testing
12:11:36 <amoralej> it's just related to the strategy reporting the efficiency
12:11:41 <amoralej> it sets values to 0.0
12:11:58 <amoralej> although internally seems to be calculating it fine
12:12:45 <marios> this sounds like an obvious yes valid/'triaged' bug... is it something you are working on currently amoralej ?
12:12:57 <amoralej> not really
12:13:00 <amoralej> just reported it
12:13:14 <amoralej> can be taken by anyone
12:13:42 <marios> so, triaged and high sound right?
12:13:48 <marios> remaining unassigned
12:14:22 <amoralej> could be high or medium, i'm not sure
12:14:39 <amoralej> and i didn't dig into the code itself yet
12:14:56 <rlandy> there are folks who are up for looking at the strategies in the upcoming weeks - so that can fall into the same bucket
12:15:21 <marios> any other thoughts or comment from anyone on this?
12:15:33 <marios> based on discussion thus far i'll set triaged and high. any objection?
12:16:12 <jgilaber_> +1 to that, if no one is working on it I'll probably take it early next week
12:16:13 <amoralej> wfm
12:17:21 <marios> ok moving on momentarily if there is nothing further. any other bugs we should look at (none in the etherpad)
12:17:35 <amoralej> i also created https://bugs.launchpad.net/watcher-tempest-plugin/+bug/2103640
12:17:54 <amoralej> i logged it as a RFE as it's coverage improvement on the tempest plugin
12:18:13 <marios> #info https://bugs.launchpad.net/watcher-tempest-plugin/+bug/2103640  [RFE] Watcher tempest plugin should provide a way to run tests with real load instead of fake metrics
12:19:07 <marios> as an rfe i am not sure we should/could triage this in the weekly irc meet
12:19:09 <chandankumar> I have moved the importance to wishlist
12:20:01 <marios> well i mean, its hard to argue with the bug subject... so we could set it to triaged. that doesn't speak to when/who would do that
12:20:17 <amoralej> yeah, it was just for awareness
12:20:48 <marios> ok and chandankumar also set triaged
12:21:08 <sean-k-mooney> technially there is already a way
12:21:12 <amoralej> wrt importance, for me it's higher that wishlist
12:21:22 <sean-k-mooney> you set the datasouce in the tempest plugin to none
12:21:33 <sean-k-mooney> but only a small subset of test will work
12:21:50 <sean-k-mooney> im not sure we should provide a generic way for all test to use real vs fake data
12:22:06 <sean-k-mooney> but we proably shoul add some specific test that dont inject data
12:22:20 <amoralej> so you mean, setting datasource to '' and doing the load "from outside" ?
12:22:22 <sean-k-mooney> but do use metrics just to validate that we can wrok without that
12:22:32 <sean-k-mooney> kind of
12:23:00 <sean-k-mooney> what im say is we do not have a way today to generate the load to trigger the rebalnace or whatever
12:23:09 <sean-k-mooney> but i dont think we should provide 2 ways of runing every test
12:23:10 <amoralej> so my proposal was to create a small set of different tests in which we don't create metrics but run load
12:23:18 <amoralej> i don't expect it to work for all scenarios
12:23:19 <sean-k-mooney> i.e with injected data and real data
12:23:28 <sean-k-mooney> ack
12:23:34 <sean-k-mooney> that alings with my thinking too
12:23:46 <sean-k-mooney> although small set could just be 1-2
12:24:10 <amoralej> for me the important is that we chose some strategy that fails if metrics are not found
12:24:26 <sean-k-mooney> yep
12:24:35 <amoralej> some metrics have fallback mechanism, i.e. getting memory size from nova if there is no memory metric
12:24:56 <amoralej> as what i know the workload_stabilization is good candidate
12:25:05 <sean-k-mooney> the problem is unless we have two jobs then we have to reviert disabling the cpu metric collection in node exporter
12:25:18 <amoralej> uh, good point
12:25:24 <sean-k-mooney> i dont think two jobs is a good use of ci resouces
12:25:33 <sean-k-mooney> so i woudl prefer to not do that
12:25:51 <sean-k-mooney> as far as i can tell we were not getting test failure when the cpu metric was collected
12:25:59 <sean-k-mooney> so i dont think reverting that will be a problem
12:26:38 <amoralej> is it possible to do two tempest run in a single job?
12:26:55 <sean-k-mooney> technially yes but its non trivial
12:27:16 <sean-k-mooney> doing service reconfiguration in a job is possibel but not advised
12:27:26 <amoralej> disable cpu metric colletion -> 1st tempest run with only fake metrics -> enable cpu metrics -> 2nd run with only real data one
12:27:36 <sean-k-mooney> it would also be a whitebox test at that point rather then blackbox
12:28:31 <sean-k-mooney> i think this type of testing is a pretty low priority
12:29:02 <sean-k-mooney> im not saying we should not do it just we should not take on a lot of technial debt to do it
12:29:12 <sean-k-mooney> so if we want to do two tempest runs we can
12:29:38 <amoralej> i agree that we should implement it in a proper way
12:29:39 <sean-k-mooney> with serivce reconfituation in between but normally that type of testing is done as a post playbook
12:30:23 <sean-k-mooney> so we would do a normal tempest run with fake data
12:30:30 <amoralej> one option may be to add the feature in the tempest plugin even if we don't use those jobs by default, and make specific job runs periodically or only in experimental pipeline
12:31:00 <sean-k-mooney> ya a weekly perodic would be ok
12:31:07 <sean-k-mooney> we could review its status in this meeting
12:31:10 <amoralej> +1
12:31:32 <sean-k-mooney> that pretty maintainable and low impact
12:31:56 <sean-k-mooney> and ya we can add it to experimental too to be able to trigger on specifc patches as needed
12:32:49 <marios> amoralej: it would be nice if you could capture some of these points into the bug as pointers for when (whoever) picks it up?
12:32:55 <amoralej> yes, i will
12:33:00 <amoralej> thanks
12:33:53 <marios> anything further for now on this one ?
12:34:35 <marios> are there any other bugs someone wanted to hilight today?
12:35:35 <marios> ok moving on momentarily unless there is anything else here
12:36:23 <chandankumar> I need reviews on https://review.opendev.org/q/topic:%22bug/2100741%22 and https://review.opendev.org/q/project:openstack/devstack-plugin-prometheus+status:open
12:37:00 <marios> ok chandankumar lets hilight those on the open floor in a moment ?
12:37:09 <chandankumar> marios: sure
12:37:13 <marios> #topic Volunteer to chair the next meeting
12:37:44 <marios> #info mtembo to chair irc meeting on Thursday 27th March 1200 UTC
12:37:46 <marios> thank you mtembo
12:37:47 <mtembo> I can take the next one
12:38:07 <marios> and that is the end of the agenda for today, so lets do open floor
12:38:11 <marios> #topic open floor
12:38:13 <marios> thank you chandankumar
12:38:30 <marios> #info review requests https://review.opendev.org/q/project:openstack/devstack-plugin-prometheus+status:open https://review.opendev.org/q/topic:%22bug/2100741%22
12:38:46 <marios> did you want to add some more info or anything else here chandankumar ^
12:39:06 <chandankumar> sure
12:39:31 <chandankumar> https://review.opendev.org/q/topic:%22bug/2100741%22 - contains all the reviews related to movement of functional tests to python-watcherclient
12:40:21 <chandankumar> Below is the review order
12:40:32 <chandankumar> [Replace watcherclient functional job with python-watcherclient-functional](https://review.opendev.org/c/openstack/watcher/+/943136)
12:40:45 <chandankumar> [Add deprecation warning about client_functional tests](https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/943133)
12:41:08 <chandankumar> sean-k-mooney: ^^ I might need help to here on how to highlight deprecation message when we run tests via tempest
12:41:25 <chandankumar> and last one on this [Drop watcherclient-tempest-functional job](https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/945009)
12:42:20 <chandankumar> On prometheus devstack plugin side
12:42:36 <chandankumar> [Use get_extra_file for downloading node_exporter/prometheus tarballs](https://review.opendev.org/c/openstack/devstack-plugin-prometheus/+/944622)
12:42:57 <chandankumar> [Add devstack-prometheus job to validate prometheus plugin](https://review.opendev.org/c/openstack/devstack-plugin-prometheus/+/944095) - to validate the plugin itself
12:43:39 <chandankumar> https://review.opendev.org/c/openstack/devstack-plugin-prometheus/+/944829 - Install podman prometheus exporter via devstack plugin
12:44:03 <chandankumar> that's it from my side, feel free to take a look!
12:44:11 <chandankumar> thank you marios!
12:44:30 <marios> thank you chandankumar holding to see if there are any comments or questions on any of that for now
12:45:48 <marios> any other items anyone wants to raise today?
12:46:29 <marios> i will close out the meeting in a moment if there is nothing further
12:46:29 <rlandy> do we want to skip the week of PTG?
12:47:03 <marios> good point. we could, though we also don't have sessions scheduled for that week. though given the ptg, indeed it is redundant probably? not sure
12:47:39 <marios> sorry, i meant we don't have sessions scheduled for that *day*
12:48:13 <marios> no strong opinions on this one rlandy perhaps revisit closer to (like next week irc meeting)
12:48:23 <marios> if there are no objections then probably canceling is fine
12:48:25 <rlandy> ack
12:49:00 <marios> any other topics today before we close out?
12:50:28 <opendevreview> Merged openstack/watcher master: Query by fqdn_label instead of instance for host metrics  https://review.opendev.org/c/openstack/watcher/+/944795
12:51:01 <marios> ok then, thank you all for attending today o/
12:51:04 <marios> #endmeeting