| opendevreview | Amit Uniyal proposed openstack/nova master: Adds check, if admin has set compute service down https://review.opendev.org/c/openstack/nova/+/848886 | 04:58 |
|---|---|---|
| bauzas | good Friday Nova | 07:34 |
| Uggla | bauzas, hello did you upgrade to F36 ? | 08:04 |
| bauzas | yup | 08:04 |
| bauzas | had an issue with my network, but this was on me | 08:04 |
| bauzas | my perf issues with Wayland and any video stream eating all my CPU aren't fixed | 08:05 |
| bauzas | I have to try to see whether I could use hardware acceleration (I have an intel gpu) for those streams | 08:05 |
| bauzas | intel soc I mean | 08:06 |
| Uggla | bauzas, any diff if you use xorg ? | 08:06 |
| bauzas | Uggla: haven't tested yet | 08:07 |
| bauzas | but I need a reproducible test | 08:07 |
| bauzas | I'll use webrtc | 08:07 |
| bauzas | sar gives me a lot of information | 08:07 |
| bauzas | but this is clear this is just a CPU-bound task, that's it | 08:08 |
| bauzas | not a lot of context switching | 08:08 |
| bauzas | my sys % is high, about 20% | 08:08 |
| bauzas | and user % is about 60% | 08:08 |
| bauzas | so I straced | 08:08 |
| bauzas | and I see lots of polling | 08:08 |
| gibi | good morning | 08:08 |
| Uggla | bauzas, mine is also slow sometime when it is getting hot. I think the cooling system is a bit limited. | 08:09 |
| Uggla | gibi, o/ | 08:09 |
| bauzas | Uggla: good point, I should dig sar's power information | 08:09 |
| gibi | Uggla: give me 20 mins and I'm all yours if neede | 08:10 |
| gibi | d | 08:10 |
| bauzas | and yeah, in general, my fan is yelling | 08:10 |
| bauzas | I suspected that the CPU is stepping down | 08:10 |
| bauzas | hence the perf issues | 08:10 |
| bauzas | but that's a guess | 08:11 |
| Uggla | bauzas, in such case, I can not switch the power balance to performance, it sticks to balanced. | 08:11 |
| bauzas | Uggla: me too, can't modify it | 08:11 |
| bauzas | this is like I was running my laptop unplugged | 08:11 |
| bauzas | which shouldn't be the rule | 08:11 |
| bauzas | if my laptop is plugged, I shall expect max performance and CPU power not stepping down | 08:12 |
| bauzas | but yeah, haven't checked the temperature | 08:12 |
| bauzas | you make me realize a thing | 08:12 |
| bauzas | I think my fan was yelling because the CPU was intensively high | 08:13 |
| bauzas | thought* | 08:13 |
| bauzas | but that could be the contrary | 08:13 |
| bauzas | if the temperature rises, then the fan yells and then the CPU could step down | 08:13 |
| bauzas | all of this seems tied to Fedora power management tools | 08:14 |
| bauzas | but yeah, needs a reproducer to validate | 08:14 |
| bauzas | Uggla: either way, I was having performance trouble with F35 already | 08:14 |
| bauzas | so F36 isn't a regression | 08:15 |
| Uggla | bauzas, I think it is what happened in my case. And that's dramatically long when you run the openstack tests suite. | 08:15 |
| bauzas | point is, the TB3 system doesn't help | 08:16 |
| bauzas | previously with my other laptops, those were docked | 08:16 |
| bauzas | so the fan was having room for air | 08:16 |
| bauzas | now, with the tb3 plug, the laptop is just sit down | 08:16 |
| Uggla | clearly | 08:16 |
| bauzas | will continue to dig into it | 08:17 |
| bauzas | there are actions I could do | 08:17 |
| bauzas | and I'd like to verify | 08:17 |
| bauzas | maybe the cooling system is sufficient and this is only Fedora which is too much conservative in terms of power consumption | 08:18 |
| bauzas | having the fan burning isn't a problem | 08:18 |
| bauzas | that's the CPU stepping down which is an issue | 08:18 |
| Uggla | yep I think so as well. | 08:19 |
| gibi | I also have perf issues with google meet (on T14, debian sid, xorg, i3, chrome). During our calls sometimes chrome starts to hogg the CPU and the audio and video starts lagging, and clipping. I haven't figured out yet what triggers the CPU hogging | 08:19 |
| gibi | but as CPU get hot, thermal throttling kicks in and worsen the situation | 08:20 |
| Uggla | gibi, welcome to the club :) | 08:21 |
| gibi | then after 1 minute chrome stops hogging the CPU and after an additional minute google meet audio/video recovers | 08:21 |
| gibi | I suspect that in my case it is chrome that does something crazy | 08:21 |
| Uggla | bauzas, maybe we should have a look to firmware as well. | 08:23 |
| * Uggla have no idea if this laptop is updated. | 08:24 | |
| Uggla | *has | 08:24 |
| Uggla | gibi, have you a moment to look at my issue ? | 08:25 |
| gibi | yes, I'm ready now | 08:25 |
| gibi | lets test some google meet ;) | 08:25 |
| Uggla | yep | 08:25 |
| Uggla | :) | 08:25 |
| bauzas | ok, I tuned sar to report my CPU clock freq | 08:34 |
| bauzas | we'll see, I have to join auniyal_'s meeting now :) | 08:34 |
| bauzas | hah ! | 08:39 |
| bauzas | my clock freq turns down to 800MHz while the temperature on the devices is only about 50°C | 08:39 |
| gibi | that sounds bad | 08:40 |
| bauzas | when I playing gmeet I mean | 08:40 |
| gibi | mine can sustain 1.8GHz while hot | 08:40 |
| bauzas | sar reports the frequency constantly flipping between 866MHz and 1.5GHz | 08:41 |
| bauzas | every second | 08:41 |
| bauzas | yeah, found a direct correlation | 08:42 |
| bauzas | huzzah | 08:42 |
| bauzas | between the CPU clock freq and the CPU usage | 08:43 |
| bauzas | https://paste.opendev.org/show/bHWo1z8ElaEDqW2b95fZ/ | 08:47 |
| bauzas | [sbauza@sbauza ~]$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor | 08:51 |
| bauzas | powersave | 08:51 |
| bauzas | hmmmpfff | 08:51 |
| kashyap | bauzas: What laptop is yours? | 08:56 |
| bauzas | 11:02:30 CPU MHz | 09:04 |
| bauzas | 11:02:31 all 410,34 | 09:04 |
| bauzas | t14s | 09:04 |
| kashyap | I see | 09:05 |
| Uggla | bauzas, maybe you can run sudo fwupdmgr get-devices it will give you fw revision on your laptop. So we could compare with mine and gibi's. | 09:30 |
| bauzas | on a meeting with auniyal_ | 09:31 |
| bauzas | will reply later | 09:31 |
| bauzas | but I think this is a the CPU governor's fault | 09:31 |
| gibi | I have a ThinkPad T14s Gen 1 | 09:33 |
| gibi | hm, the powersave governor could be a problem | 09:33 |
| Uggla | bauzas from release note latest system relase note: • Updated includes thermal optimization. | 09:34 |
| Uggla | clearly mine is not at the latest version 0.1.15 vs 0.1.21 | 09:35 |
| Uggla | so bauzas has you have received this laptop first maybe maybe your fws are really "old". | 09:38 |
| Uggla | My current version: https://paste.openstack.org/show/b9cnjsnSNOqzjw52AA8L/ | 09:39 |
| gibi | this is mine https://paste.opendev.org/show/b48RVDFNHGRaeHUq2qMg/ | 09:47 |
| sean-k-mooney1 | bauzas: powersave keep the cpu at the minium frequency | 10:10 |
| *** sean-k-mooney1 is now known as sean-k-mooney | 10:10 | |
| sean-k-mooney | you should set it to the hardware pstate dirver or use ondemand | 10:10 |
| sean-k-mooney | also i recommend not using tuned | 10:10 |
| sean-k-mooney | i have had severl issue with it in the past so i dont trust it anymore | 10:11 |
| sean-k-mooney | by issue i mean putting one or both ports in a bond to sleep or disabling my mouse/keyboard randomly | 10:12 |
| bauzas | sean-k-mooney1: yup, was considering to use ondemand | 10:14 |
| bauzas | sean-k-mooney1: but I'd like a governor different between being plugged or not | 10:15 |
| bauzas | I thought Fedora was managing it but apparently not | 10:15 |
| sean-k-mooney | bauzas: i used to use udev rules to do that | 10:15 |
| bauzas | yup, that's what I thought I need to do | 10:15 |
| sean-k-mooney | ill see if i have them but i dont know if i have it on this laptop since i reinstalled | 10:16 |
| bauzas | but this sucks for users | 10:16 |
| bauzas | how many people know about udev rules ? | 10:16 |
| bauzas | and systemctl ? | 10:16 |
| sean-k-mooney | well gnome/cinanamon/kde often can | 10:16 |
| sean-k-mooney | its normally built into the destop envionment | 10:16 |
| bauzas | why can't we have a standard behaviour to be ondemand when plugged and powersave when unplugged ? | 10:16 |
| sean-k-mooney | with a nice gui | 10:16 |
| bauzas | sean-k-mooney: yup, and I saw it | 10:17 |
| bauzas | sean-k-mooney1: but now with F35 and F36, this is stuck to powersave apparently | 10:17 |
| bauzas | I still have the GUI but clicking makes no result | 10:17 |
| sean-k-mooney | have you tried removing tuned | 10:17 |
| sean-k-mooney | its not using tuned to implement this | 10:17 |
| bauzas | and I can't define a plugged/unplugged strategy like before | 10:17 |
| sean-k-mooney | so it could be fighting with it | 10:17 |
| bauzas | I haven't investigated it | 10:18 |
| sean-k-mooney | https://linux.die.net/man/1/gnome-power-manager | 10:18 |
| bauzas | I'm already glad I found the root cause... | 10:18 |
| sean-k-mooney | that is what shoudl be doing it on the gnome side | 10:18 |
| bauzas | but again, how many users are knowing how to collect power metrics with sar ? | 10:19 |
| bauzas | and even knowing what sar is ? | 10:19 |
| sean-k-mooney | well your not going to find me defending fedroa | 10:19 |
| sean-k-mooney | you know i stongly dislike it | 10:19 |
| bauzas | if you want MHO, Fedora is too much opiniated about what people should do | 10:19 |
| sean-k-mooney | just pointing out its probaly tuned that is the issue | 10:20 |
| bauzas | I'll look into it | 10:20 |
| bauzas | thanks | 10:20 |
| * bauzas goes cooking for his wife | 10:20 | |
| sean-k-mooney | i havenet used fedora since f33 or f32 but it did not used to have tuned installed so i wonder whne that got added | 10:21 |
| kashyap | sean-k-mooney: I'm on F36, and I don't have 'tuned' installed by default (mine is a 'Workstation' install) | 10:22 |
| sean-k-mooney | ack | 10:23 |
| sean-k-mooney | i wonder why bauzas has it then | 10:23 |
| bauzas | I haven't said I have tuned | 10:23 |
| sean-k-mooney | oh i tought you did | 10:24 |
| sean-k-mooney | ignore the noise then | 10:24 |
| bauzas | [sbauza@sbauza ~]$ rpm -qa | grep tuned | 10:24 |
| bauzas | [sbauza@sbauza ~]$ | 10:24 |
| bauzas | I'll continue on it this afternoon | 10:24 |
| bauzas | looks like an Hack&Hustle session | 10:25 |
| sean-k-mooney | ya maybe | 10:26 |
| sean-k-mooney | let us know if you figure it out either way | 10:26 |
| bauzas | my sar figures are explicit | 10:27 |
| bauzas | my temperature was still fine | 10:27 |
| bauzas | but I was struggling with CPU clock freq at 400Mhz | 10:27 |
| bauzas | and my audio was then choppy as the load was spiking up to 8.0 | 10:27 |
| bauzas | (8 CPUs, which is normal) | 10:27 |
| bauzas | when the CPU clock was about 1500MHz, then the audio was better (and the load down to 4.0) | 10:28 |
| bauzas | f**** governor | 10:28 |
| sean-k-mooney | wisely or not im using tlp to manage my power https://paste.opendev.org/show/b01fBUjtb7EA4fQDmvuM/ | 10:28 |
| sean-k-mooney | https://linrunner.de/tlp/index.html | 10:29 |
| bauzas | power-profiles-daemon.service loaded active running Power Profiles daemon | 10:30 |
| * bauzas needs to see what this daemon is doing | 10:30 | |
| sean-k-mooney | ya that is one of the things you have to remove if you are using tlp | 10:31 |
| sean-k-mooney | just notifced im actully using schedutil for my govoner | 10:31 |
| sean-k-mooney | if you are not using the intel pstate driver thats actully the better option vs ondemand | 10:31 |
| bauzas | Conflicts=tlp.service shutdown.target auto-cpufreq.service system76-power.service tuned.service | 10:31 |
| bauzas | huhuh | 10:31 |
| bauzas | I'll have to dig into it | 10:32 |
| bauzas | but... lunch | 10:32 |
| sean-k-mooney | yep its one of the ones i tried | 10:32 |
| sean-k-mooney | it didnt work well for me when i use the power-profiles-daemon | 10:32 |
| sean-k-mooney | i think my issue was mainly fan noise and it was not really goign down form max cpu | 10:33 |
| sean-k-mooney | like idling on ac was full non turbo/bost clock speeed | 10:34 |
| opendevreview | Merged openstack/python-novaclient master: Imported Translations from Zanata https://review.opendev.org/c/openstack/python-novaclient/+/848804 | 10:40 |
| *** tosky_ is now known as tosky | 11:11 | |
| bauzas | found the guilty https://fedoraproject.org/wiki/Changes/Power_Profiles_Daemon | 12:01 |
| bauzas | "The "performance" mode is only available on select systems and is implemented by different "drivers" based on the system or systems it targets. " | 12:02 |
| bauzas | that's maybe me but I thought we were having different power management profiles whether the laptop was docked or not, previously | 12:10 |
| bauzas | (in Gnome, I mean) | 12:10 |
| bauzas | https://forums.lenovo.com/t5/Other-Linux-Discussions/Lenovo-L14-Gen1-AMD-Lap-Mode-Throttling/m-p/5154197 | 12:20 |
| * bauzas facepalms | 12:20 | |
| bauzas | but my surface is steady... | 12:20 |
| sean-k-mooney | bauzas: so ya i would jsut remove that and go with one of the many other tools | 12:25 |
| bauzas | checked, I'm not in lap mode | 12:25 |
| bauzas | but there are a ton of issues with PPD with Thinkpads https://gitlab.freedesktop.org/hadess/power-profiles-daemon/-/issues | 12:25 |
| bauzas | and that's the issue I hit when changing the PPD profile to 'perf' https://gitlab.freedesktop.org/hadess/power-profiles-daemon/-/issues/72 | 12:26 |
| bauzas | sean-k-mooney: any power management tool you'd recommend that could set the power based on charging vs. discharging ? | 12:28 |
| bauzas | powersave seems reasonable to me if I'm running on battery | 12:28 |
| bauzas | but I don't want to clockdown if I'm plugged | 12:29 |
| opendevreview | Merged openstack/nova master: Retry attachment delete API call for 504 Gateway Timeout https://review.opendev.org/c/openstack/nova/+/845543 | 12:30 |
| sean-k-mooney | bauzas: i use tlp | 12:33 |
| sean-k-mooney | i dont actully use power save on battery but i configure it to limit the max frequency | 12:33 |
| sean-k-mooney | but you can certenly set the profile to power save too if you prefer | 12:33 |
| sean-k-mooney | https://paste.opendev.org/show/bPVClbF8MnuFjrhBE2z3/ | 12:34 |
| sean-k-mooney | that is my tlp config | 12:34 |
| sean-k-mooney | bauzas: so you could change line 6 two powersaver | 12:34 |
| sean-k-mooney | or liek i do on line 11 | 12:35 |
| sean-k-mooney | limite the cpu frequency | 12:35 |
| sean-k-mooney | bauzas: tlp also has a tlp-stats tool which is kind of nice | 12:36 |
| sean-k-mooney | you can see that i still have 90% capasity on my battery after 4 years | 12:37 |
| sean-k-mooney | https://paste.opendev.org/show/bNtQTQ0T2Tx4Bd2nFyLu/ | 12:37 |
| sean-k-mooney | partly becasue i only let it chagne to 80% | 12:37 |
| sean-k-mooney | so i went with tlp since i coudl use it to do more then save power it can also extend battry life by contoling change and i cloud have a little more contol over all | 12:38 |
| sean-k-mooney | https://termbin.com/ic06r is the full set of stats it has | 12:39 |
| sean-k-mooney | and you can twek many of the paramters if you care about it | 12:39 |
| opendevreview | Amit Uniyal proposed openstack/nova master: Adds check, if admin has set compute service down https://review.opendev.org/c/openstack/nova/+/848886 | 12:49 |
| opendevreview | Amit Uniyal proposed openstack/nova master: add regression test case for bug 1978983 https://review.opendev.org/c/openstack/nova/+/849104 | 12:49 |
| bauzas | ok, eventually fixed my issue, upgraded the firware | 12:50 |
| gibi | sean-k-mooney: I'm wondering that we reintroduced a an issue by reenabling greendns. See my fresh bugreport https://bugs.launchpad.net/nova/+bug/1981080 | 12:51 |
| bauzas | when the lid is closed, you can't upgrade your firmware, bizarre but ok | 12:51 |
| bauzas | so this one was PEBKAC | 12:51 |
| bauzas | I'll need to use to change the profile based on the docking info | 12:51 |
| bauzas | need to see* | 12:52 |
| sean-k-mooney | gibi: i was still getting that waring before that | 13:11 |
| gibi | hm, I don't get it if I turn off greendns | 13:11 |
| sean-k-mooney | we coudl perhaps disable greendns in our func tests but i dont think a revert would be correct | 13:12 |
| sean-k-mooney | we shoudl not be doing dns lookup in the func tests | 13:12 |
| gibi | this is not about func test, there it is probably cosmetic. But I wonder if we reintroduced https://bugs.launchpad.net/nova/+bug/1808951 | 13:13 |
| gibi | as the fix for that was to blacklist urllib3 | 13:13 |
| gibi | now we see that urllib3 is imported before monkey patching | 13:14 |
| gibi | so we might retrigger https://bugs.launchpad.net/nova/+bug/1808951 | 13:14 |
| sean-k-mooney | well we now use eventliet directly in the nova-api | 13:15 |
| sean-k-mooney | for scater gather so we cant skip monky patching in general there anymore | 13:15 |
| gibi | I don't follow you how that is related | 13:16 |
| sean-k-mooney | we know that disabling greendns can lead to the nova-api and other serivce had locking up on dns quieres | 13:16 |
| sean-k-mooney | so if we revert the other patch we woudl break things | 13:16 |
| gibi | OK, so if we disable greendns we break thing, but now that we enabled it we might break other things | 13:17 |
| sean-k-mooney | we might but i dont think we will | 13:17 |
| sean-k-mooney | have you found a code path where urllib3 is imported before monkeypatching? | 13:17 |
| gibi | yes, it is the dns code in eventlet that imports it | 13:18 |
| sean-k-mooney | so https://github.com/openstack/nova/blob/90c0c687a487601e009c72f60c88be92f6a55264/nova/monkey_patch.py#L30= | 13:18 |
| sean-k-mooney | imports urllib3? | 13:18 |
| gibi | indireclt yes | 13:18 |
| gibi | indirectly | 13:19 |
| sean-k-mooney | that sounds like an eventlet bug that we canot fix then | 13:19 |
| sean-k-mooney | since we cant monkeypatch until after we import eventlet | 13:19 |
| sean-k-mooney | so ya maybe we need to remove urllib3 form the problem list if eventlet have fixed the issue | 13:20 |
| sean-k-mooney | if not then we shoudl file an eventlet bug | 13:20 |
| sean-k-mooney | unless from nova import debugger | 13:20 |
| sean-k-mooney | is pulling it in instead fo eventlet | 13:20 |
| sean-k-mooney | not that i can see | 13:21 |
| sean-k-mooney | https://github.com/openstack/nova/blob/90c0c687a487601e009c72f60c88be92f6a55264/nova/debugger.py | 13:21 |
| gibi | hm I assuming that deployers had some workaround for our disable greendns as we had that disable for a long time | 13:25 |
| gibi | so we might fixed a known and worked around problem with but re-introduced a break somewhere else. Still I feel like reverting the greendns patch would be safer while we figure out is urllib3 is safe | 13:27 |
| gibi | anyhow updated the bug report linking to this discussion as I have now bandwidth to dig deeper into this now | 13:29 |
| * gibi moves back tracking down a locking issue in our CellDatabases fixture found by Uggla | 13:30 | |
| Uggla | gibi, sorry for that.... Uggla is a black cat that attract bug. | 13:32 |
| gibi | Uggla: no worries. I like these kind of challenges :) | 13:33 |
| *** dasm|off is now known as dasm | 13:56 | |
| sean-k-mooney | gibi: it was breaking our downstream customers | 14:13 |
| sean-k-mooney | so we cant assume that | 14:13 |
| sean-k-mooney | gibi: when i did the revert an renebaled greendns | 14:14 |
| sean-k-mooney | i noted you could sitll diablie it by setting the env var | 14:14 |
| sean-k-mooney | gibi: we were hardcodeing os.environ['EVENTLET_NO_GREENDNS'] = 'yes' | 14:14 |
| sean-k-mooney | before | 14:14 |
| sean-k-mooney | but you can just export that before you run nova-api | 14:15 |
| sean-k-mooney | or set it in the wsgi env if you need too | 14:15 |
| sean-k-mooney | but i dont think you should | 14:15 |
| gibi | sean-k-mooney: so if somebody get hit by https://bugs.launchpad.net/nova/+bug/1808951 then they can set EVENTLET_NO_GREENDNS=yes in there api service script to get back the old behavior | 14:18 |
| gibi | good point. I agree that is a viable workaround | 14:19 |
| gibi | I will not that in https://bugs.launchpad.net/nova/+bug/1808951 | 14:19 |
| sean-k-mooney | i just did in https://bugs.launchpad.net/nova/+bug/1981080/comments/2 | 14:19 |
| sean-k-mooney | but yes you could not that in the orginal bug | 14:20 |
| sean-k-mooney | gibi: do you happen to know what the orginal urllib3 bug is and if it was fixed | 14:20 |
| sean-k-mooney | i have read over https://bugs.launchpad.net/nova/+bug/1808951 | 14:20 |
| sean-k-mooney | and that references https://github.com/eventlet/eventlet/issues/371 | 14:21 |
| opendevreview | Sergii Golovatiuk proposed openstack/nova master: Replace "db archive" with "db archive_deleted_raws" https://review.opendev.org/c/openstack/nova/+/847963 | 14:21 |
| sean-k-mooney | that seams to have been fiex and broken | 14:21 |
| sean-k-mooney | based on differnt python releases | 14:21 |
| gibi | I think it is https://github.com/eventlet/eventlet/issues/371 as you noted | 14:22 |
| gibi | at least https://github.com/eventlet/eventlet/issues/371#issuecomment-1047336652 reports that it still exists | 14:22 |
| sean-k-mooney | https://github.com/eventlet/eventlet/issues/726 | 14:22 |
| sean-k-mooney | so they have fixed some version fo this | 14:23 |
| sean-k-mooney | gibi: did you actully se this cause issues | 14:23 |
| sean-k-mooney | or did you just notice the warning | 14:23 |
| sean-k-mooney | we use uwsgi in the devstack jobs | 14:23 |
| sean-k-mooney | and i have not seen them fail | 14:23 |
| sean-k-mooney | so just trying ot undersand if there is an actul functional issue | 14:24 |
| sean-k-mooney | or just a warnign message in our test | 14:24 |
| gibi | I only noticed the warning | 14:24 |
| gibi | I did not see actuall failures | 14:24 |
| sean-k-mooney | ok then i think we should just remove the warnign unless someoen reports an error | 14:25 |
| sean-k-mooney | since it appears to work properly | 14:25 |
| sean-k-mooney | given it has been runnign upstream and in production for down stream custoemr for a whiel now | 14:26 |
| sean-k-mooney | hum actully | 14:26 |
| opendevreview | Amit Uniyal proposed openstack/nova master: Adds check, if admin has set compute service down https://review.opendev.org/c/openstack/nova/+/848886 | 14:26 |
| sean-k-mooney | gibi: ok we hae no shipped it downstream yet | 14:27 |
| sean-k-mooney | my backport is still pending | 14:27 |
| sean-k-mooney | it has been runin in the gate though for a hile | 14:27 |
| sean-k-mooney | *while | 14:28 |
| gibi | at least we are not setting EVENTLET_NO_GREENDNS so we should see it failing in the gate | 14:30 |
| gibi | *not setting in devstack | 14:30 |
| sean-k-mooney | yep | 14:30 |
| sean-k-mooney | and since we dont i think it still correct | 14:31 |
| gibi | do we test with ssl in the gate? | 14:33 |
| sean-k-mooney | we use the tls_proxy | 14:38 |
| sean-k-mooney | so we use apache to do tls termination using a prviate ca in front of uwsgi | 14:38 |
| sean-k-mooney | i dont think uwsgi support tls directly or its a pain to configure which is why we do it that way | 14:39 |
| gibi | then we probably don't trigger the ssl code that caused the original problem | 14:40 |
| sean-k-mooney | maybe not sure if the api calls other service we would open an ssh clonenction | 14:41 |
| sean-k-mooney | so i would expect the keystone middelware to trigger it no? | 14:41 |
| sean-k-mooney | i dont thinkthe recursion was on the server side | 14:42 |
| gibi | hm, yeah it was in keystonemiddleware | 14:43 |
| Airamek | Hi! I know this isn't the place to ask for support, but #openstack is kinda dead. If anyone has a spare minute, could you look into this?(https://lists.openstack.org/pipermail/openstack-discuss/2022-July/029454.html). Thanks in advance! | 14:52 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Add extra info about limitation of CellDatabase fixture https://review.opendev.org/c/openstack/nova/+/849123 | 14:59 |
| gibi | Uggla: ^^ I added extra info the the lock escalation error so that next time we can track down the cause easier | 14:59 |
| opendevreview | sean mooney proposed openstack/nova master: add regression test case for bug 1978983 https://review.opendev.org/c/openstack/nova/+/849104 | 15:07 |
| opendevreview | sean mooney proposed openstack/nova master: Adds check, if admin has set compute service down https://review.opendev.org/c/openstack/nova/+/848886 | 15:07 |
| gibi | I'm done for the week. Have a nice weekend folks! | 15:53 |
| opendevreview | Alexey Stupnikov proposed openstack/nova master: Remove deleted projects from flavor access list https://review.opendev.org/c/openstack/nova/+/849131 | 16:06 |
| opendevreview | Rajesh Tailor proposed openstack/placement master: Fix typos https://review.opendev.org/c/openstack/placement/+/848636 | 16:17 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP: api: Drop generating a keypair and add special chars to naming https://review.opendev.org/c/openstack/nova/+/849133 | 17:03 |
| *** dasm is now known as dasm|off | 21:36 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!