| *** bauzas_ is now known as bauzas | 00:08 | |
| *** bauzas_ is now known as bauzas | 00:21 | |
| *** bauzas_ is now known as bauzas | 01:01 | |
| *** bauzas_ is now known as bauzas | 01:26 | |
| *** bauzas_ is now known as bauzas | 01:34 | |
| *** __ministry is now known as Guest2936 | 01:43 | |
| *** bauzas_ is now known as bauzas | 02:24 | |
| *** bauzas_ is now known as bauzas | 02:40 | |
| *** bauzas_ is now known as bauzas | 02:48 | |
| *** bauzas_ is now known as bauzas | 02:56 | |
| *** bauzas_ is now known as bauzas | 04:05 | |
| *** __ministry is now known as Guest2944 | 04:14 | |
| *** bauzas_ is now known as bauzas | 04:26 | |
| *** bauzas_ is now known as bauzas | 04:58 | |
| *** bauzas_ is now known as bauzas | 06:20 | |
| bauzas | morning | 07:17 |
|---|---|---|
| *** noonedeadpunk_ is now known as noonedeadpunk | 07:29 | |
| opendevreview | zhou zhong proposed openstack/nova master: nova-manage: modify image properties in request_spec https://review.opendev.org/c/openstack/nova/+/924319 | 08:37 |
| opendevreview | zhou zhong proposed openstack/nova master: nova-manage: modify image properties in request_spec https://review.opendev.org/c/openstack/nova/+/924319 | 08:39 |
| opendevreview | zhou zhong proposed openstack/nova master: nova-manage: modify image properties in request_spec https://review.opendev.org/c/openstack/nova/+/924319 | 08:42 |
| opendevreview | Balazs Gibizer proposed openstack/os-traits master: Add support for hw_vif_model igb image property https://review.opendev.org/c/openstack/os-traits/+/928582 | 09:12 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928583 | 09:14 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584 | 09:14 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584 | 09:15 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928585 | 09:15 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928456 | 09:16 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584 | 09:16 |
| *** bauzas- is now known as bauzas | 09:54 | |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928456 | 10:05 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584 | 10:05 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Refactor obj_make_compatible to reduce complexity https://review.opendev.org/c/openstack/nova/+/928590 | 10:05 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Refactor obj_make_compatible to reduce complexity https://review.opendev.org/c/openstack/nova/+/928590 | 10:18 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928456 | 10:18 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584 | 10:18 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Refactor obj_make_compatible to reduce complexity https://review.opendev.org/c/openstack/nova/+/928590 | 11:36 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [ovo]Add igb value to hw_vif_model image property https://review.opendev.org/c/openstack/nova/+/928456 | 11:36 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [libvirt]Support hw_vif_model = igb https://review.opendev.org/c/openstack/nova/+/928584 | 11:36 |
| ratailor_ | Hi All, can I get reviews on my open patches, https://review.opendev.org/q/owner:ratailor@redhat.com+status:open | 12:46 |
| ratailor_ | some of those are open since long. | 12:46 |
| sean-k-mooney | have you put them on the review etherpad | 12:48 |
| sean-k-mooney | or pingged us about them recently | 12:48 |
| sean-k-mooney | they should all be listed here https://etherpad.opendev.org/p/nova-dalmatian-status#L60 | 12:49 |
| sean-k-mooney | we will be creating an epoxy version of that soon | 12:49 |
| sean-k-mooney | so maybe we should add them there | 12:50 |
| sean-k-mooney | bauzas: is there a nova-epoxy-status etherpad yet | 12:50 |
| sean-k-mooney | or nova-2025.1-status | 12:50 |
| sean-k-mooney | i would prefer 2025.1 personally like ubuntu i prefer the number over the name but either is fine | 12:51 |
| ratailor_ | sean-k-mooney, no yet, I will do it asap. Thanks! | 12:59 |
| bauzas | sean-k-mooney: not yet, I'm literrally catching up with all pre-RC1 PTL tasks | 13:01 |
| bauzas | which include rpcapin pin patch and other fancy prelude writing | 13:01 |
| sean-k-mooney | no worries | 13:01 |
| sean-k-mooney | the only thing you really need to decied is which one to use and then we can just create it | 13:02 |
| bauzas | I'm rushing over deadlines, since RC1 is planned on Thur | 13:02 |
| sean-k-mooney | ill create it for you | 13:02 |
| sean-k-mooney | which would you prefer name or number | 13:02 |
| bauzas | well, creating an etherpad isn't really a problem | 13:02 |
| sean-k-mooney | right its not so i can do it quickly | 13:02 |
| bauzas | I just discussed with the Foundation and they were amazed we were having some tracking process | 13:02 |
| sean-k-mooney | well we have alwasy had one | 13:03 |
| bauzas | but given we don't document it actually, this isn't really useful | 13:03 |
| sean-k-mooney | well we document that we shoudl use review priorty in gerrit | 13:03 |
| sean-k-mooney | which i still perfer | 13:03 |
| sean-k-mooney | but since you wanted to use ether pad we document that currently in the channel topic | 13:03 |
| sean-k-mooney | but not in our contubutors guide | 13:04 |
| ratailor_ | sean-k-mooney, also could you please move this to Triaged, its still showing in new. https://bugs.launchpad.net/nova/+bug/2058928 | 13:05 |
| sean-k-mooney | ok although you stared on it a while ago so should i put it in inprogress | 13:06 |
| ratailor_ | sean-k-mooney, yes, sure. Thanks! | 13:06 |
| sean-k-mooney | note the traige team is open so you can join it and do this yourself too | 13:06 |
| ratailor_ | sean-k-mooney, ack. sure. | 13:08 |
| sean-k-mooney | i set it to inprogress and low beacuse of how long this has been broken | 13:09 |
| bauzas | sean-k-mooney: sorry was afk for family matters | 13:19 |
| bauzas | sean-k-mooney: fwiw, after that discussion with the foundation folks, I now have an AI on me, which is to update our contrib docs (and how to reach them) | 13:20 |
| bauzas | like our wikipage is the first POC for nova but it's fucking old and stale | 13:20 |
| bauzas | so I'll do all of that stuff shortly after RC1 | 13:21 |
| sean-k-mooney | bauzas: ok if we really are not going to back to using gerrit for this | 13:21 |
| sean-k-mooney | then ill update our gerrit config to remvoe the review priorty flag | 13:21 |
| bauzas | about the etherpad, we usually use codenames for etherpads, but I'm not opposed to use release numbers if people agree | 13:21 |
| sean-k-mooney | bauzas: we proposed stoping using in the last two ptgs | 13:21 |
| sean-k-mooney | both for the ptg and this status one | 13:21 |
| sean-k-mooney | also for the launchpad series | 13:22 |
| sean-k-mooney | we made the change in lauchpad form caracal | 13:22 |
| sean-k-mooney | we didnt make the change for the ptg or status page however | 13:23 |
| sean-k-mooney | in the retos both dan and i have prevsiouly expressed our desire to stop using names | 13:23 |
| sean-k-mooney | for all our nova trackers/processes | 13:23 |
| sean-k-mooney | but its not a hill i want to die on | 13:24 |
| sean-k-mooney | my personal recolection (which might be wrong) is we agreed to use the number in caracal but that change was not implemnted | 13:26 |
| sean-k-mooney | we started disucssing htis as far back as antelope | 13:27 |
| bauzas | the written resolution on release names says that numbers are the official way to describe a release | 13:30 |
| sean-k-mooney | yep | 13:30 |
| bauzas | hence why we use them in both launchpad, stable branches and specs repo | 13:30 |
| bauzas | that said, the resolution says we can use codenames for internal use | 13:31 |
| bauzas | which is why we were naming the etherpad that way | 13:31 |
| sean-k-mooney | right but we as a nova team discussed this and at lease dan and i advocated for using the number for lanchpad as well | 13:31 |
| bauzas | now that I realize that this tracking etherpad is useful for new contributors, I think we can start using release numbers | 13:31 |
| sean-k-mooney | and i tought we agreed we shoudl do that | 13:31 |
| bauzas | launchpad *is* using numbers | 13:32 |
| sean-k-mooney | but the etherpad is not | 13:32 |
| bauzas | from 2023.1 caracal at least :) | 13:32 |
| bauzas | correct, that's what I just said, | 13:32 |
| sean-k-mooney | i was gong to bring this up as a low light in the rethro athat we had not chagned to the numeber for the epther pad for a second release in a row | 13:32 |
| bauzas | I'm just creating the new etherpad, that's quick | 13:33 |
| bauzas | https://etherpad.opendev.org/p/nova-2025.1-status | 13:35 |
| bauzas | sean-k-mooney: ralonsoh ^ | 13:35 |
| bauzas | I'm also creating a second etherpad for rc tracking | 13:35 |
| bauzas | (which will use the codename format since it's for dalmatian) | 13:35 |
| sean-k-mooney | i think you ment ratailor_ https://etherpad.opendev.org/p/nova-2025.1-status | 13:36 |
| bauzas | my bad, typed too fast and autocompletion hit the road | 13:36 |
| * bauzas goes back into datamining mode for RC prep | 13:37 | |
| opendevreview | Merged openstack/nova stable/2023.1: hardware: Correct log https://review.opendev.org/c/openstack/nova/+/928485 | 13:39 |
| *** ykarel_ is now known as ykarel | 14:02 | |
| ratailor_ | sean-k-mooney, bauzas Thanks! | 14:08 |
| ralonsoh | bauzas, thanks for th elink | 14:23 |
| opendevreview | Rajesh Tailor proposed openstack/nova master: Fix KeyError on assisted snapshot call https://review.opendev.org/c/openstack/nova/+/900783 | 14:31 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: doc: mark the maximum microversion for 2024.2 Dalmatian https://review.opendev.org/c/openstack/nova/+/928660 | 16:07 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: Update compute rpc alias for dalmatian https://review.opendev.org/c/openstack/nova/+/928661 | 16:10 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: Add service version for Dalmatian https://review.opendev.org/c/openstack/nova/+/928662 | 16:17 |
| noonedeadpunk | hey folks! is there any way to suppli a custom URI to the VNC console? As I'm obviously getting `404 File not found` when tried to define `https://domain.com/novnc/vnc_lite.html` | 16:19 |
| noonedeadpunk | I tried to find inspiration in devstack, but seems vnc there is just port-oriented? | 16:20 |
| noonedeadpunk | https://zuul.opendev.org/t/openstack/build/092ca27659d5466aa3c19120b58b5031/log/controller/logs/etc/nova/nova-cpu_conf.txt#155 | 16:20 |
| noonedeadpunk | so basically console is not under /compute while nova overall is | 16:22 |
| opendevreview | Sylvain Bauza proposed openstack/placement master: Update 2024.2 reqs to support os-traits 3.1.0 as min version We're now close to RC1, we need to ensure that 2024.2 Placement will support the new traits. https://review.opendev.org/c/openstack/placement/+/928663 | 16:28 |
| opendevreview | Sylvain Bauza proposed openstack/placement master: Update 2024.2 reqs to support os-traits 3.1.0 as min version https://review.opendev.org/c/openstack/placement/+/928663 | 16:28 |
| opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Move Dalmatian implemented specs https://review.opendev.org/c/openstack/nova-specs/+/928666 | 16:38 |
| opendevreview | ribaudr proposed openstack/nova master: Amend ShareMappingStatus due to asynchronous call https://review.opendev.org/c/openstack/nova/+/908864 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Amend DB model add a unique constraint. https://review.opendev.org/c/openstack/nova/+/912518 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (manila abstraction) https://review.opendev.org/c/openstack/nova/+/831194 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Use client token when talking to manila https://review.opendev.org/c/openstack/nova/+/925277 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (drivers and compute manager part) https://review.opendev.org/c/openstack/nova/+/833090 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Mounting the shares as part of the initialization process https://review.opendev.org/c/openstack/nova/+/880075 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Deletion of associated share mappings on instance deletion https://review.opendev.org/c/openstack/nova/+/881472 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add metadata for shares https://review.opendev.org/c/openstack/nova/+/850500 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add share_info parameter to reboot method for each driver (driver part) https://review.opendev.org/c/openstack/nova/+/854823 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Support rebooting an instance with shares (compute manager part) https://review.opendev.org/c/openstack/nova/+/854824 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add share_info parameter to resume method for each driver (driver part) https://review.opendev.org/c/openstack/nova/+/860284 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Support resuming an instance with shares (compute manager part) https://review.opendev.org/c/openstack/nova/+/860285 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add helper methods to rescue/unrescue shares https://review.opendev.org/c/openstack/nova/+/860286 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Support rescuing an instance with shares https://review.opendev.org/c/openstack/nova/+/860287 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Allow to mount manila share using Cephfs protocol https://review.opendev.org/c/openstack/nova/+/883862 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Check shares support (compute manager) https://review.opendev.org/c/openstack/nova/+/885751 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (API) https://review.opendev.org/c/openstack/nova/+/836830 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add helper methods to attach/detach shares https://review.opendev.org/c/openstack/nova/+/885753 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add instance.share_attach notification https://review.opendev.org/c/openstack/nova/+/850501 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add instance.share_detach notification https://review.opendev.org/c/openstack/nova/+/851028 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add shares to InstancePayload https://review.opendev.org/c/openstack/nova/+/851029 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add instance.share_attach_error notification https://review.opendev.org/c/openstack/nova/+/860282 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add instance.share_detach_error notification https://review.opendev.org/c/openstack/nova/+/860283 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Reports instance events to the DB regarding attaching and detaching a share https://review.opendev.org/c/openstack/nova/+/927088 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add libvirt test to ensure metadata are working. https://review.opendev.org/c/openstack/nova/+/852086 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Add virt/libvirt error test cases https://review.opendev.org/c/openstack/nova/+/852087 | 16:57 |
| opendevreview | ribaudr proposed openstack/nova master: Docs about Manila shares API usage https://review.opendev.org/c/openstack/nova/+/871642 | 16:57 |
| noonedeadpunk | it feels like the usecase of custom URI for consoleproxy is just not supported | 16:57 |
| noonedeadpunk | as I'd guess I'd expect to see some api-paste for nova-novncproxy but obviously there's none | 17:02 |
| sean-k-mooney | noonedeadpunk: you can customise it to a degree | 17:25 |
| sean-k-mooney | what are you tryign to do | 17:25 |
| noonedeadpunk | trying to have smth like `https://domain.com/novnc/vnc_lite.html` | 17:26 |
| noonedeadpunk | instead of `https://domain.com:6080/vnc_lite.html` | 17:26 |
| noonedeadpunk | and feels like the way forward would be to place novnc install folder somewhere under DEFAULT.web | 17:27 |
| noonedeadpunk | ie make a subtree or smth | 17:27 |
| sean-k-mooney | ya so you can do that | 17:27 |
| sean-k-mooney | we have conifg option for tha tlet me find it | 17:28 |
| sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#vnc.novncproxy_base_url | 17:28 |
| sean-k-mooney | we should really upgrade our default to default to vnc_lite.html at some point | 17:28 |
| noonedeadpunk | yeah, so ^ is not helpful | 17:28 |
| sean-k-mooney | how so | 17:29 |
| noonedeadpunk | as I have that and novncproxy ends up with 404 | 17:29 |
| noonedeadpunk | openstack console url show returns expected URL though | 17:29 |
| noonedeadpunk | ie https://domain.com/novnc/vnc_lite.html?path=%3Ftoken%3D880916e7-2e8a-4d0c-a814-9e16f6633194 | 17:30 |
| sean-k-mooney | right so to use that you need to aslo modify your reverse proxy | 17:30 |
| noonedeadpunk | but then it's just 404 from nova-novncproxy service itself | 17:30 |
| noonedeadpunk | with File not found | 17:30 |
| noonedeadpunk | I'd guess I would need to do rewrite | 17:30 |
| sean-k-mooney | yep that what i used to do with kolla ansible | 17:31 |
| noonedeadpunk | but I'm using haproxy which can't really do rewrites | 17:31 |
| sean-k-mooney | you could set https://docs.openstack.org/nova/latest/configuration/config.html#vnc.novncproxy_port to 80 or 443 | 17:31 |
| noonedeadpunk | it's probably easier when you have apache though | 17:31 |
| sean-k-mooney | so i used trafix to do it | 17:31 |
| sean-k-mooney | it was doing all my tlscert stuff with letencypt and doing the rewrite | 17:32 |
| noonedeadpunk | I don't think that port will anyhow influence URI after it? | 17:32 |
| noonedeadpunk | regardless on which port nova-novncproxy.service will listen | 17:32 |
| noonedeadpunk | as long as balancer passes traffic to it - it should work right? | 17:32 |
| noonedeadpunk | for nova-api I can do rewrites with uWSGI | 17:33 |
| noonedeadpunk | or, using api-paste for eventlet | 17:33 |
| sean-k-mooney | so if you want to have vnc in a sub folder | 17:33 |
| sean-k-mooney | you need to strip that in your revirse proxy | 17:33 |
| sean-k-mooney | api-paste is not really used for this | 17:33 |
| noonedeadpunk | oh, well, it kind of is> | 17:33 |
| noonedeadpunk | https://opendev.org/openstack/nova/src/branch/master/etc/nova/api-paste.ini#L20-L29 | 17:34 |
| sean-k-mooney | kind of but we dont have midelware enabled by default for rewrites | 17:34 |
| sean-k-mooney | its better to do that in the webserver/reverse_proxy | 17:34 |
| noonedeadpunk | ah, for this - novnc - yeah, it does not | 17:34 |
| noonedeadpunk | so in the webserver - yeah, that would be fine. But reverse proxies not designed for rewrites, imo | 17:35 |
| sean-k-mooney | ya so the novnc proxy is useing websockify and the eventlet webserver | 17:35 |
| sean-k-mooney | noonedeadpunk: tell that to kubernetes | 17:35 |
| sean-k-mooney | that what ingress is | 17:35 |
| noonedeadpunk | I don't think k8s is good bfor lb... | 17:35 |
| noonedeadpunk | it's nginx | 17:35 |
| noonedeadpunk | which is technically not a reverse proxy | 17:36 |
| sean-k-mooney | its the runciton its used for however | 17:36 |
| noonedeadpunk | it's a web server with hugely reduced feature list that acts as a reverse proxy | 17:36 |
| sean-k-mooney | i would consier it to be both a reverse proxy and a webserver | 17:36 |
| sean-k-mooney | yep but even haproxy can do this too | 17:36 |
| noonedeadpunk | can it? | 17:36 |
| sean-k-mooney | i belive so when used as a http loadbalbnce | 17:37 |
| noonedeadpunk | As it can do maps, sure, but it's not really rewrites | 17:37 |
| noonedeadpunk | so it can pass /console to correct backend | 17:37 |
| noonedeadpunk | but not alter uri. I guess | 17:37 |
| noonedeadpunk | not 100% sure | 17:37 |
| sean-k-mooney | ya. so that is how i would personaly do it. i generall prefer using subdomain insteda of subpaths | 17:38 |
| noonedeadpunk | I was actually hoping to see answer in devstack, but it's jsut using port :D | 17:38 |
| noonedeadpunk | oh, yes, sure, subdomain works nicely | 17:38 |
| sean-k-mooney | im currently in the process of rebuildign my home lab | 17:38 |
| noonedeadpunk | jsut decided to practise on subpaths as that's what devstack does | 17:38 |
| jrosser | does any of this help https://www.haproxy.com/documentation/haproxy-configuration-tutorials/http-rewrites/ | 17:38 |
| sean-k-mooney | but my home openstack did everything behind traefix exposing everything over port 443 | 17:39 |
| noonedeadpunk | oh | 17:39 |
| sean-k-mooney | openshifts defautl ingress controller(route contoler) is haproxy and it does supprot this to at least some degree | 17:40 |
| noonedeadpunk | jrosser: I guess I tried to have a look at https://www.haproxy.com/documentation/haproxy-configuration-tutorials/http-rewrites/#set-the-url-path but failed to implement so dropped this idea a while ago | 17:42 |
| noonedeadpunk | but maybe for consoles it's actually worth checking | 17:42 |
| sean-k-mooney | https://www.haproxy.com/documentation/haproxy-configuration-tutorials/http-rewrites/#replace-part-of-a-header-by-using-a-regular-expression | 17:44 |
| sean-k-mooney | you can proably use the regex replace value funtionality | 17:44 |
| sean-k-mooney | in your case set-path | 17:45 |
| sean-k-mooney | is proably simpler | 17:45 |
| noonedeadpunk | I've jsut tried to place novnc actually in subfolder, but then it's also failing on `wss://domain.com/?token=4741df38-cc54-4be6-b63f-1d7b7b1b336d` | 17:45 |
| noonedeadpunk | as somehow wss is not adjusted or trimms subpath /o\ | 17:46 |
| noonedeadpunk | and that I'm even not sure how to catch | 17:47 |
| noonedeadpunk | so eventually what creates wss should also pass a path | 17:57 |
| noonedeadpunk | or at least respect passed one | 17:57 |
| sean-k-mooney | well you would need to also rewrite it | 17:58 |
| noonedeadpunk | it's tough to rewrite just /?token given that / is serving horizon | 17:59 |
| noonedeadpunk | but yeah | 17:59 |
| noonedeadpunk | I'd say there should be code in place respecting passed URIs when returning wss | 18:00 |
| sean-k-mooney | unfortuetly i dont have a config to had for that. i tought i had that working in my previous home cloud. i.e. the one i had pre 2020 | 18:00 |
| sean-k-mooney | but my current one is forwarding 6080 and everything else was using subdomains | 18:01 |
| noonedeadpunk | yeah, I managed to get it working with subdomain jsut out of the box | 18:02 |
| noonedeadpunk | but then it's a bit tougher with let's encrypt as for wildcards I'd need dns-01 | 18:04 |
| sean-k-mooney | yep thats what i use now although before i didnt use wildcards i added a dns entry for each serice | 18:15 |
| sean-k-mooney | i use cloudfail as my dns provider mainly for there restapi adn the fact it supproted in a lot of things | 18:15 |
| sean-k-mooney | *cloudflare | 18:16 |
| noonedeadpunk | well. these are quite long-shot workarounds for smth which should be just possible to do quite simply. And I have pretty much everything working with subpaths except console... | 18:18 |
| sean-k-mooney | honestly none of this is inscope of nova | 18:20 |
| sean-k-mooney | noonedeadpunk: a webseocket is just a upgradded http connection | 18:20 |
| sean-k-mooney | my guess is haproxy does not offcially supprto them or apply the rewite to them | 18:20 |
| noonedeadpunk | yes, but it's https://opendev.org/openstack/nova/src/branch/master/nova/console/websocketproxy.py which replies with wss URL I assume? | 18:21 |
| noonedeadpunk | as when I open novnc under specific path, and it fully loads (I reached this point), now I assume nova provides wss url that is not under same path at all | 18:22 |
| noonedeadpunk | So I'd argue about scope here | 18:22 |
| sean-k-mooney | we ues a websocket between the proxy and the html5 applciation that part of novnc | 18:22 |
| noonedeadpunk | yeah, but it's created with no respect to where html5 is running I assume? | 18:23 |
| noonedeadpunk | as it always thinks it runs in `/` | 18:24 |
| sean-k-mooney | the websocket is created regardless yes | 18:24 |
| sean-k-mooney | but we do not supprot subpaths | 18:24 |
| sean-k-mooney | we only supprot runing it on a port | 18:24 |
| sean-k-mooney | anythign beyond that is out so scope | 18:24 |
| noonedeadpunk | so regardless of any rewrites, except trying to catch some GETs under `/` which is really weird - it won't work, yes | 18:25 |
| sean-k-mooney | i belive it can be made work if you rewirte teh websocet too | 18:26 |
| sean-k-mooney | https://www.haproxy.com/documentation/haproxy-configuration-tutorials/load-balancing/websocket/#configure-websockets ha porxy has some supprot for this | 18:26 |
| noonedeadpunk | but then what's the whole point of devstack to be serving services under subpaths.... | 18:26 |
| sean-k-mooney | to avoid using domain | 18:26 |
| noonedeadpunk | yeah, but jsut use ports as used to - would be way more simple to configure... | 18:27 |
| sean-k-mooney | that it we wanted to test that we can run all the rest apis over https on port 4434 | 18:27 |
| sean-k-mooney | *443 | 18:27 |
| sean-k-mooney | there is nothing stoping you using ports | 18:27 |
| sean-k-mooney | btu we did have a lot of operator want to run everything on 443 | 18:28 |
| noonedeadpunk | *all except consoles | 18:28 |
| sean-k-mooney | you can run the console on 443 | 18:28 |
| noonedeadpunk | (that is not tested in devstack) | 18:28 |
| noonedeadpunk | https://zuul.opendev.org/t/openstack/build/092ca27659d5466aa3c19120b58b5031/log/controller/logs/etc/nova/nova-cpu_conf.txt#152-158 | 18:28 |
| sean-k-mooney | sure but devstack does not test everything | 18:28 |
| noonedeadpunk | I can recall some tempest tests for consoles? | 18:28 |
| sean-k-mooney | there are some yes | 18:29 |
| sean-k-mooney | its not that devstack could not test this | 18:29 |
| sean-k-mooney | its that its not in scope to test every combination of how openstack is deployed | 18:29 |
| noonedeadpunk | yeah, fair | 18:30 |
| noonedeadpunk | that I agree with | 18:30 |
| noonedeadpunk | I guess I just not fully in agreement that serving consoles under subpath should be out of scope of nova-novncproxy service | 18:30 |
| sean-k-mooney | for our new installer we deploy novnc on a subdomain | 18:31 |
| sean-k-mooney | https://logserver.rdoproject.org/57/857/e3248dbf95dfb1878d852beef4475bfb68f4e811/github-check/nova-operator-tempest-multinode/616a00d/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/namespaces/openstack/routes/nova-novncproxy-cell1-public.yaml | 18:31 |
| noonedeadpunk | yeah, subdomain works really nicely.... | 18:32 |
| noonedeadpunk | ok, gotcha | 18:32 |
| sean-k-mooney | yep so again using subpath need a reversproxy that support both http and websockets | 18:32 |
| sean-k-mooney | as in it need to be able to rewirte both properly | 18:32 |
| sean-k-mooney | it might be posibel to do some other way | 18:33 |
| noonedeadpunk | the problem here of reverseproxy, that then you can't serve anything on root | 18:33 |
| noonedeadpunk | so if you want to have smth like skyline/horizon on root - you can't do that jsut because of vnc console | 18:33 |
| sean-k-mooney | right but normally you sever that on /dashboard | 18:33 |
| sean-k-mooney | and have a redirect for root to it | 18:33 |
| noonedeadpunk | but then redirecting is also fishy a bit, as you need to account for wss | 18:34 |
| sean-k-mooney | you shoudl be able to have the proxy on /vnc by the way and use / for horizon aslong as there isnt a collition | 18:34 |
| noonedeadpunk | which is just /?token= | 18:34 |
| noonedeadpunk | but probably possible to come up with some regex | 18:35 |
| sean-k-mooney | you shoudl be able to have the websoceket be wss://<domain>/novnc?token=... | 18:35 |
| noonedeadpunk | so I just got `https://domain.com/console/vnc_lite.html` loading but still wss was just wss://<domain>/?token= | 18:36 |
| noonedeadpunk | so it ended up with "something went wrong" | 18:37 |
| sean-k-mooney | right but the websocket is not beeing writhe properly | 18:37 |
| sean-k-mooney | you said the url looked correct form nova right | 18:38 |
| sean-k-mooney | do you have an exmaple of the consol url returned | 18:38 |
| noonedeadpunk | yes, but what does return wss URL? | 18:38 |
| sean-k-mooney | at least for the serial console oepnstack server console show returns it | 18:39 |
| noonedeadpunk | openstack console url show returns `https://domain.com/console/vnc_lite.html?path=%3Ftoken%3D76a1303f-c40c-4169-912d-19c05432dbc5` is what is returned | 18:39 |
| sean-k-mooney | but i think we get https back by defualt for novnc | 18:39 |
| noonedeadpunk | it loads, vnc console renders | 18:39 |
| sean-k-mooney | so with devstack the dash board load at "http://192.168.16.127/dashboard/auth/login/?next=/dashboard/" | 18:40 |
| noonedeadpunk | but then html5 tries to connect to wss on `wss://domain.com/?token=76a1303f-c40c-4169-912d-19c05432dbc5` | 18:40 |
| noonedeadpunk | ok, I gues I'm jsut confused where this wss path is coming from | 18:41 |
| noonedeadpunk | as indeed it's probably not from nova at all | 18:41 |
| sean-k-mooney | its part of the html content provided to the javascript app | 18:41 |
| sean-k-mooney | so in my case it returns http://192.168.16.127:6080/vnc_lite.html?path=%3Ftoken%3Dd8a68169-dbd6-4333-8b4a-f57bc3beb53c by edfault | 18:43 |
| sean-k-mooney | if i open that direclty that is what nova is serving and what ironic is embding in an iframe | 18:43 |
| noonedeadpunk | yup, true, and that's working as well (if clients don't have 6080 firewalled) | 18:43 |
| noonedeadpunk | byt heir isp | 18:44 |
| noonedeadpunk | ok, I'll try to do some rewrites indeed. | 18:44 |
| noonedeadpunk | and look one more time what does construct wss uri | 18:44 |
| sean-k-mooney | tehre are two path to how htis works. we use websoxkify to wrap th raw tcp connectoin to the vnc console create by qemu | 18:45 |
| sean-k-mooney | and we use novnc ot provide the webpage that conenct to that and provde the vnc clinet | 18:45 |
| noonedeadpunk | as htmlk5 console does have `const path = readQueryVariable('path', 'websockify');` | 18:45 |
| noonedeadpunk | and it does `url += '/' + path;` - that's why I kind of complained that smth not passing html5 all details for it to perform | 18:46 |
| sean-k-mooney | ok but that novnc that doing that | 18:47 |
| sean-k-mooney | not nova | 18:47 |
| sean-k-mooney | there might eb some header you can set like x_forwared_for or similar | 18:47 |
| sean-k-mooney | in my case the path query arge is empty | 18:48 |
| noonedeadpunk | I think it's just because path being used to pass token | 18:48 |
| sean-k-mooney | well technialy path=%3F | 18:48 |
| noonedeadpunk | ah | 18:48 |
| noonedeadpunk | true | 18:49 |
| noonedeadpunk | but it's what nova replies then | 18:49 |
| sean-k-mooney | you may be able to add it to novncproxy_base_url | 18:49 |
| sean-k-mooney | so instead of http://127.0.0.1:6080/vnc_auto.html | 18:50 |
| sean-k-mooney | http://domain/subpath/vnc_auto.html?path=/subpath | 18:50 |
| sean-k-mooney | can you try setting that in your broser and see if setting path=/subpath is enouch | 18:51 |
| noonedeadpunk | hm, I somehow don't think that will work? | 18:51 |
| noonedeadpunk | yeah, I'm trying now | 18:52 |
| noonedeadpunk | in terms won't work - regarding defining novncproxy_base_url that way | 18:52 |
| sean-k-mooney | well its being used by vnc_lite.html | 18:52 |
| sean-k-mooney | whic is part of novnc | 18:52 |
| sean-k-mooney | nova might not allow you pas qurey sting arge in baseurl | 18:52 |
| sean-k-mooney | but it could be added by the proxy | 18:53 |
| sean-k-mooney | *reverse_proxy when rewriteing | 18:53 |
| sean-k-mooney | we coudl maybe add a cofnig option for that if it works | 18:53 |
| sean-k-mooney | based on https://github.com/novnc/noVNC/blob/master/vnc_lite.html#L134-L153 | 18:54 |
| sean-k-mooney | i dont see why it woudl not | 18:54 |
| noonedeadpunk | yeah, adding manually shiould work | 18:54 |
| noonedeadpunk | what probably wont - adjustin novncproxy_base_url as you said | 18:55 |
| sean-k-mooney | ya we likely dont have logic to parse the query arges and append them | 18:55 |
| noonedeadpunk | I don't think that nova gonna respect already defined path and wont'd add default path=%3F | 18:55 |
| sean-k-mooney | but should be doable exterally | 18:55 |
| noonedeadpunk | yeah, this should be not hard to do indeed | 18:56 |
| noonedeadpunk | (I'd say would make sense for config option though) | 18:56 |
| noonedeadpunk | or indeed parse for the option | 18:56 |
| sean-k-mooney | the other thing you said is there isp blockes 6080 | 19:00 |
| sean-k-mooney | we do hwave supprot for using alternitive ports alredy | 19:00 |
| sean-k-mooney | so that might be an option in that case | 19:00 |
| sean-k-mooney | it would be a pretty minor feature to add novncproxy_extra_query_args | 19:01 |
| noonedeadpunk | ok, so `https://domain.com/console/vnc_lite.html?path=console%3Ftoken%3Dd7bf7a47-f081-4500-a961-05ddb7994e13` | 19:05 |
| noonedeadpunk | does work indeed | 19:05 |
| noonedeadpunk | so thanks a ton for your time | 19:07 |
| noonedeadpunk | I know I was annoying but quite happy now as you said - it should be trivial both to rewrite and implement a "native" way | 19:07 |
| sean-k-mooney | your bringing a pain point to our attention that not annoying | 19:08 |
| sean-k-mooney | i still think subdomains are preferable | 19:08 |
| sean-k-mooney | but if someone wants to add this minor feature im not agaisnt it | 19:08 |
| sean-k-mooney | i just rpaobly wont go implemtnet it in my spare time | 19:08 |
| sean-k-mooney | noonedeadpunk: you may have hit https://github.com/haproxy/haproxy/issues/829 by the way | 19:10 |
| sean-k-mooney | there is a set-pathq to set items in teh query string now | 19:12 |
| sean-k-mooney | i guess that migh now be https://www.haproxy.com/documentation/haproxy-configuration-tutorials/http-rewrites/#set-the-query-string | 19:14 |
| sean-k-mooney | so i think you can jsut add "http-request set-query path=/subpath" | 19:15 |
| sean-k-mooney | noonedeadpunk: im going to call it a day. if that works you could porpsoe a docs update if you were so inclidned or a devstack path | 19:16 |
| sean-k-mooney | *patch | 19:16 |
| noonedeadpunk | ++ thanks for the help as usual! | 19:20 |
| noonedeadpunk | yeah, so here it goes: https://opendev.org/openstack/nova/src/branch/master/nova/objects/console_auth_token.py#L78 | 19:30 |
| samcat116 | Hi all, whats a good place to start debugging the GET /servers api being slow? Running `openstack server list --all-projects --no-name-lookup` on a stack with about 1500 instances takes over 48s to return, with the GET https://example.com:8774/v2.1/servers/detail?deleted=False&all_tenants=True call taking over 30s itself. CPU usage across our 3 controllers is around 50% and I'm running with 10 api_workers, so don't think its a raw | 19:49 |
| samcat116 | horsepower issue | 19:49 |
| *** bauzas_ is now known as bauzas | 21:15 | |
| *** bauzas_ is now known as bauzas | 21:56 | |
| sean-k-mooney | samcat116: the server detail list need to hit neutron for each server to lookup secuirty groups, but openstack client is doing more then jsut /v2.1/servers/detail?deleted=False&all_tenants=Tr | 22:10 |
| sean-k-mooney | | ue | 22:10 |
| sean-k-mooney | that specific query however does require conenctin to all cell db and pulling a lot of info so its one of the more expensive api calls you can make to nova | 22:12 |
| sean-k-mooney | its not as expensieve as simple tenant usage but it s more expensive then just about any other read only call | 22:13 |
| samcat116 | Ok I guess I’ll look on the neutron side as we are pretty port and network heavy as well | 22:19 |
| samcat116 | I just don’t know why it would be that slow | 22:19 |
| sean-k-mooney | it might not be related to that but its on eof tthe peices of data we lookup form other services | 22:20 |
| sean-k-mooney | we cache alotof the info reatled to other services but not all | 22:20 |
| sean-k-mooney | tehre are some profiling tools liek os-profiler | 22:20 |
| sean-k-mooney | but im not famialr with who to properly use them | 22:21 |
| samcat116 | Ok. I do have memcache all setup so I would assume this could be heavily cached | 22:21 |
| sean-k-mooney | no | 22:22 |
| sean-k-mooney | i think we have almsot no caching for this | 22:22 |
| sean-k-mooney | we use memcache very sparingly | 22:22 |
| sean-k-mooney | its main use in nova is for the metadata api | 22:23 |
| sean-k-mooney | we generally dont cache api responces | 22:24 |
| sean-k-mooney | the other main use of memcafhe is for keytone auth | 22:25 |
| samcat116 | Good to know | 22:27 |
| samcat116 | How far off of an estimate could this be. Just trying to figure out if this is an order of magnitude slower than it should be or not | 22:30 |
| sean-k-mooney | that kind of depend on other factors like how many cells you have | 22:30 |
| sean-k-mooney | you not the first person to ask about this | 22:30 |
| sean-k-mooney | samcat116: can you try making the same query to /server instead of /server/detials | 22:31 |
| sean-k-mooney | the delta is /server just hits the api db and list all the server we know about without going to the cell dbs or calling other services | 22:31 |
| sean-k-mooney | that should be very fast as it basicaly a select instance_uuid from istance_mappings | 22:32 |
| sean-k-mooney | if /servers/ is fast and /servers/detail is slow then one thing to check is if any of the cells are slow to respond | 22:35 |
| sean-k-mooney | internally the api when repondign ot /servers/detail is calling into each cell db and geting the data form the instances table | 22:35 |
| *** bauzas_ is now known as bauzas | 22:37 | |
| sean-k-mooney | the security group info i thin need a call to neutron for every port | 22:38 |
| samcat116 | Ok I’ll try that. We only have one cell | 22:38 |
| sean-k-mooney | that should simplfy thing then as the api is not making parralel requests | 22:39 |
| sean-k-mooney | i think the security gorups are the only thing not cached in that responce. i.e. that is not just a direct db lookup | 22:41 |
| sean-k-mooney | this has been know for a long time https://bugs.launchpad.net/nova/+bug/1923560 and a fix was prospoed here https://review.opendev.org/c/openstack/nova/+/786348 | 22:42 |
| sean-k-mooney | i was personaly on the side of removing the security groups form the instance obejct entirly | 22:43 |
| sean-k-mooney | btu if we dont then we shoudl cache them | 22:43 |
| *** bauzas_ is now known as bauzas | 23:10 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!