| opendevreview | chandan kumar proposed openstack/watcher-dashboard master: Add Start and END time fields for continuous audit https://review.opendev.org/c/openstack/watcher-dashboard/+/957232 | 05:01 |
|---|---|---|
| opendevreview | chandan kumar proposed openstack/watcher-dashboard master: Add Start and END time fields for continuous audit https://review.opendev.org/c/openstack/watcher-dashboard/+/957232 | 05:11 |
| opendevreview | chandan kumar proposed openstack/watcher-dashboard master: Add option to SKIP Actions https://review.opendev.org/c/openstack/watcher-dashboard/+/958209 | 08:05 |
| opendevreview | Takashi Kajinami proposed openstack/watcher master: Add basic common middlewares https://review.opendev.org/c/openstack/watcher/+/960044 | 13:19 |
| tkajinam | I happened to notice that watcher does not support CORS (due to lack of CORS middleware) and I wonder if ^^ can interest people | 13:20 |
| opendevreview | Takashi Kajinami proposed openstack/watcher master: Add basic common middlewares https://review.opendev.org/c/openstack/watcher/+/960044 | 13:31 |
| opendevreview | Takashi Kajinami proposed openstack/watcher master: Add basic common middlewares https://review.opendev.org/c/openstack/watcher/+/960044 | 14:32 |
| sean-k-mooney | tkajinam: Oh yes we shoudl fix that. | 14:59 |
| sean-k-mooney | tkajinam: we likely need to review our entire api tech stack in general and evauate how to eventually move on form using paste/paste deploy and wsme ectra | 15:00 |
| sean-k-mooney | longterm just adopting flask and or fastapi i think is the direction we shoudl go in but that is less urgent then the eventlet removal | 15:01 |
| sean-k-mooney | pecan is a littel mroe maintained the rest of the rest api stack that is currently in place and i just dont know enough about it to knwo if we shoudl keep it or not longterm | 15:02 |
| sean-k-mooney | there are some other middelware that we likely shoudl add like the requestid middlewar or request size but ill add a ptg topic for this | 15:04 |
| tkajinam | pecan is used by a few projects like aodh or octavia so getting rid of it is a bit long term thing. | 15:04 |
| sean-k-mooney | and ironic | 15:04 |
| tkajinam | yeah | 15:04 |
| tkajinam | though it looks much more maintained than paste/PasteDeploy in my view | 15:04 |
| sean-k-mooney | and cyborg | 15:04 |
| sean-k-mooney | yep so pecan is not the lib im concerned about | 15:05 |
| tkajinam | I think I the above change may add most of the common middlewares we use in multiple projects but yeah more concrete audit would be nice | 15:05 |
| sean-k-mooney | the ones that give me pase are paste/paste-deploy, WSME and WebOB | 15:05 |
| tkajinam | actually "audit" middleware is the one we may want to discuss. and also healthcheck middleware though I know it's controversial | 15:06 |
| sean-k-mooney | the oslo team maintance of the latter too is low but they are used in other places too | 15:06 |
| tkajinam | yeah | 15:06 |
| tkajinam | though I'm hoping that we could somehow get rid of WSME... it's often in "dead" state due to its governance being outside of oslo | 15:07 |
| sean-k-mooney | the healthcheck one i dont like because i dont think it provide any useful info over curling the microverion endpoint which requires no auth | 15:07 |
| tkajinam | sean-k-mooney, the one benefit of healthcheck middleware is the capability to "disable" the service using a flag file though I agree its benefit is "thin" | 15:07 |
| sean-k-mooney | tkajinam: ya so i think if we had identifed a replcement, like the eventlet remvoal we coudl collectively move to somethign else | 15:07 |
| sean-k-mooney | tkajinam: that now someitng i was aware of and not somethign i think we shoudl eb supproteing | 15:08 |
| tkajinam | a few projects like octavia implements their own healthcheck plugin to do some deep check like messagebus communication. | 15:08 |
| sean-k-mooney | tkajinam: how is that implemented/conroled exactly | 15:08 |
| sean-k-mooney | yes i had a workign implation for nova but we paused it to start removing eventlet | 15:09 |
| tkajinam | ok it's not messagebus check but db access check | 15:09 |
| sean-k-mooney | i had hopped to restart that effort eventually to add per-proces native healthchecks btu time | 15:09 |
| tkajinam | sean-k-mooney, https://github.com/openstack/octavia/blob/master/setup.cfg#L109-L110 | 15:09 |
| tkajinam | this would be a good entry point to look into it. I have to restore my old memory to explain it in detail, though | 15:10 |
| sean-k-mooney | in my nova serisee i had checks for db conntiosn and libvirt i belive. i had not fully worked out howt to make rabbit comnnectivet just work | 15:11 |
| sean-k-mooney | but that was on the list | 15:11 |
| sean-k-mooney | the impoant point was to make it work for each process not just the rest api | 15:11 |
| tkajinam | yeah. I remember the old discussion to expose some endpoint (probably over unix domain socket) for liveness probe | 15:11 |
| sean-k-mooney | yep we agreed to do it as http instead | 15:11 |
| sean-k-mooney | http over tcp rahter then http over unix socket | 15:12 |
| tkajinam | which is much more sophisticated than hack-scripts with ss -npa | grep foo | 15:12 |
| tkajinam | yeah we don't have to expose it over network | 15:12 |
| sean-k-mooney | i used the eventlet webserver in my initall version but cherrypy cheroot or the standardlib webserver are likely what we would use now | 15:12 |
| tkajinam | ironic selected cheroot and a few other followed it | 15:13 |
| tkajinam | so cheroot would be the base choice | 15:13 |
| sean-k-mooney | its nice to be able too for k8s or collectd but it was going to default to not being enabled. | 15:13 |
| sean-k-mooney | yep i say jayf's email that why im aware of it | 15:13 |
| sean-k-mooney | im tore betwen no external deps and just using that since others are | 15:13 |
| tkajinam | +100 | 15:14 |
| sean-k-mooney | i dont realy want to have to deal with ssl myself if if restart that work :) | 15:14 |
| sean-k-mooney | but its also not like this woudl need to be high performace so spanign a single tread in the background shoudl be enough | 15:14 |
| sean-k-mooney | anyway that liek item number 20 on my list of "this would be ncie to do" | 15:15 |
| tkajinam | someone should seriously consider the way to create AI agents to share these tasks :-P | 15:16 |
| sean-k-mooney | honestly, not somethign qute this complex, i have been using ai a bit to help do some of these tasks that i normlaly dont get too | 15:16 |
| sean-k-mooney | im looking at your core patch and i have 2 questions | 15:17 |
| sean-k-mooney | 1 how do we know this is working and prove that in ci (we can actully do that in a followup). | 15:18 |
| sean-k-mooney | 2, i have not looked yet but since we have paste-deploy i was expecting an past.ini change but i cant find that in watcher | 15:19 |
| sean-k-mooney | i guess that measn we are not supprotign configurable midelware today | 15:20 |
| tkajinam | 1. For CORS middleware we can probably import the existing tests from ironic, though as far as I know it's not quite common to validate the feature in individual projects | 15:20 |
| tkajinam | and I have no clear idea about the other middlewares, mainly http_proxy_to_wsgi | 15:20 |
| tkajinam | 2. watcher is not using paste/paste deploy. It uses fixed set of middlewares when setting up the app | 15:20 |
| tkajinam | the logic I'm touching in the change shows it | 15:21 |
| tkajinam | there are a few additional middleware like authtoken middleware loaded in different places | 15:21 |
| sean-k-mooney | ack, i think a good start would be for us to turn on the tls_proxy if we dont alreay have it adn confirm things liek the link in teh respocne get https correctly ectra | 15:21 |
| tkajinam | yeah. that sounds reasonable | 15:21 |
| opendevreview | chandan kumar proposed openstack/watcher-dashboard master: Add option to SKIP Actions https://review.opendev.org/c/openstack/watcher-dashboard/+/958209 | 15:22 |
| tkajinam | though I have to check if any API in watcher may return full URL, instead of partial paths) | 15:22 |
| sean-k-mooney | ya we dont need to do that now | 15:22 |
| sean-k-mooney | but given the recent mailing list dicussion it somethign i need to confirm in general | 15:23 |
| sean-k-mooney | tkajinam: https://docs.openstack.org/api-ref/resource-optimization/#response-example implies it does but ill check now | 15:23 |
| sean-k-mooney | again not really in scope fo your patch but while we are here checking this and fileign a bug/todo makes sense | 15:24 |
| tkajinam | I can file bugs. will let you know once I create ones | 15:26 |
| sean-k-mooney | dviroel: just an fyi we have a bug in watcherclient | 15:27 |
| sean-k-mooney | its not doign version negocation prerly and jsut tryign to use 1.5 | 15:27 |
| tkajinam | one thing we can probably discuss is whether we consider lack of CORS support is a "bug" | 15:27 |
| sean-k-mooney | it shoudl default to either 1.0 or the latest supproted by the clodu we are talking too | 15:27 |
| sean-k-mooney | tkajinam: its tecnially a featuer but we can reviwe and dicuss. its proably not sometihing we will merge for 2025.2 | 15:28 |
| tkajinam | ok | 15:28 |
| tkajinam | no rush | 15:28 |
| sean-k-mooney | but i coudl see it as a secuirty hardening but that shoudl be backported | 15:28 |
| sean-k-mooney | im just not sure we will have time to include it for RC1 and im hoping to avoid rc2 | 15:28 |
| dviroel | sean-k-mooney: hum, ack, and there is no workaround for that right? no configuration, just hardcoded? | 15:29 |
| opendevreview | chandan kumar proposed openstack/watcher-dashboard master: Fix translation context for watcher action states https://review.opendev.org/c/openstack/watcher-dashboard/+/959189 | 15:30 |
| sean-k-mooney | tkajinam: https://paste.opendev.org/show/bPBx4xyrrml8IDgrk3BP/ | 15:30 |
| sean-k-mooney | so yes we do return the full url | 15:30 |
| tkajinam | ah, ok | 15:30 |
| sean-k-mooney | so we shoudl also check that that will work using the forwared hearder once we enabel tls in ci | 15:31 |
| sean-k-mooney | dviroel: you can set it with --os-infra-optim-api-version 1.4 | 15:31 |
| sean-k-mooney | or in an envionment variable | 15:31 |
| dviroel | right, have been doing that to test | 15:32 |
| sean-k-mooney | so there is but we shoudl file a bug and fix it and backprot it in the next few weeks | 15:32 |
| sean-k-mooney | right now i bet it just using the hardcoded max version instad of checkign what the max version supproted by the server is and usign that | 15:33 |
| sean-k-mooney | dviroel: ill file a bug for that now and then file a second bug for not having tls testing enabeld in ci upstream | 15:34 |
| sean-k-mooney | tkajinam: we will proably need to include the oslo midellware for that too. | 15:34 |
| dviroel | ack sean-k-mooney - tks | 15:35 |
| sean-k-mooney | https://bugs.launchpad.net/python-watcherclient/+bug/2122343 | 15:41 |
| sean-k-mooney | dviroel: do you have an opipion on if we should treat a lack of CORS supprot in the wsgi application as a feature or a bug? | 15:42 |
| sean-k-mooney | if your using apache i think it can handel the CORS enforcement for you but that is not true for wsgi servers in general | 15:43 |
| sean-k-mooney | and since the middleware pipeline is not configurabel it limits the options to workaround this. we dont need to decied now but we proably shoudl dicuss this at the irc call | 15:44 |
| tkajinam | sean-k-mooney, yes and it may require additional knowledge about request patterns in watcher service | 15:44 |
| tkajinam | while we "bake" default options when we are adding CORS middleware, which may be more convienient for users | 15:44 |
| sean-k-mooney | tkajinam: to me its not a question o fif we shoudl do it really, we shoudl supprot it for no other reason then consitency with everythign else IMO but i wonder if we should backprot or not | 15:45 |
| tkajinam | yeah | 15:46 |
| tkajinam | that's what I was wondering, too | 15:46 |
| tkajinam | anyway I'll create a bug about CORS middleware then we can discuss details later according to it | 15:47 |
| tkajinam | I've been juggling a few things and haven't created it yet | 15:47 |
| sean-k-mooney | being entirly transparent now that we knwo that this gap exists we will proably need to backprot this internally so unless elod or other stable cores object to it we will propably want to do that upstream at least to epoxy | 15:48 |
| opendevreview | Takashi Kajinami proposed openstack/watcher master: Fix missing CORS middleware https://review.opendev.org/c/openstack/watcher/+/960044 | 15:58 |
| sean-k-mooney | tkajinam: thanks. we will still likely need to adjust the spec file but if we are using a lower min its more likely the thirdparty job will pass. | 16:01 |
| sean-k-mooney | that wont block the patch form moving by the way it just something we will have to account for when consdierign doing backports | 16:02 |
| sean-k-mooney | i dont belive any of the middlewares you are adding are particalarly new | 16:02 |
| tkajinam | sean-k-mooney, yes | 16:03 |
| tkajinam | I pulled it from octavia but seems they bumped the min for some reason | 16:03 |
| tkajinam | ironic has much lower lower-bound so I used it instead | 16:03 |
| sean-k-mooney | ack. | 16:04 |
| dviroel | sean-k-mooney: sorry, i was out for lunch... | 16:50 |
| dviroel | we could treat as a bug yeah, just need to check if is backportable, based on the changes required... | 16:51 |
| opendevreview | Takashi Kajinami proposed openstack/watcher master: Fix missing X-OpenStack-Request-ID header https://review.opendev.org/c/openstack/watcher/+/960154 | 17:07 |
| opendevreview | Takashi Kajinami proposed openstack/watcher master: Add capability to parse forward headers https://review.opendev.org/c/openstack/watcher/+/960157 | 17:20 |
| tkajinam | sean-k-mooney, I've reported a few bugs related to missing middlewares and proposed separate changes for each. I wrote these as "bug fixes" (just in case we want to discuss backports) but I'm fine converting these to features. | 17:21 |
| tkajinam | just fyi. | 17:21 |
| dviroel | sean-k-mooney: if you have time, pls check this 1 line patch: https://review.opendev.org/c/openstack/watcher/+/959354 - threading job is unstable due to the missing devstack configs (like disabling real metrics) - other option is copy these configs to the job... i think that is easier to just inherit from the non-threading job... | 17:36 |
| dviroel | will make CI more stable | 17:36 |
| opendevreview | Takashi Kajinami proposed openstack/watcher master: Drop PasteDeploy https://review.opendev.org/c/openstack/watcher/+/960165 | 17:50 |
| sean-k-mooney | dviroel: approved https://review.opendev.org/c/openstack/watcher/+/959354 | 18:20 |
| dviroel | ty | 18:30 |
| opendevreview | Douglas Viroel proposed openstack/watcher-specs master: Add 2026.1 spec dirs and move specs implemented in 2025.2 https://review.opendev.org/c/openstack/watcher-specs/+/960177 | 18:35 |
| opendevreview | Merged openstack/watcher master: Update prometheus-threading parent jop https://review.opendev.org/c/openstack/watcher/+/959354 | 18:35 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!