| *** bauzas_ is now known as bauzas | 00:05 | |
| *** bauzas_ is now known as bauzas | 01:44 | |
| opendevreview | Merged openstack/nova master: Ignore support-matrix.css https://review.opendev.org/c/openstack/nova/+/927659 | 02:27 |
|---|---|---|
| *** bauzas_ is now known as bauzas | 02:33 | |
| *** bauzas_ is now known as bauzas | 02:53 | |
| *** bauzas_ is now known as bauzas | 03:05 | |
| *** bauzas_ is now known as bauzas | 04:19 | |
| opendevreview | Takashi Kajinami proposed openstack/nova master: Migrate MEM_ENCRYPTION_CONTEXT from root provider https://review.opendev.org/c/openstack/nova/+/921814 | 05:52 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: Detect AMD SEV-ES support https://review.opendev.org/c/openstack/nova/+/925685 | 05:52 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | 05:52 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | 05:53 |
| opendevreview | renliang proposed openstack/nova master: Update documentation link for openSUSE index. https://review.opendev.org/c/openstack/nova/+/927705 | 05:54 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | 06:03 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: Add hw_mem_encryption_model image property https://review.opendev.org/c/openstack/nova/+/927706 | 06:03 |
| *** bauzas_ is now known as bauzas | 06:38 | |
| *** bauzas_ is now known as bauzas | 07:23 | |
| opendevreview | Takashi Kajinami proposed openstack/nova master: Add hw_mem_encryption_model image property https://review.opendev.org/c/openstack/nova/+/927706 | 07:26 |
| opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | 07:26 |
| opendevreview | Fabian Wiesel proposed openstack/nova master: Vmware: Remove uuid parameter from get_vmdk_info call https://review.opendev.org/c/openstack/nova/+/910627 | 07:48 |
| *** bauzas_ is now known as bauzas | 08:00 | |
| *** bauzas_ is now known as bauzas | 08:14 | |
| *** bauzas_ is now known as bauzas | 08:39 | |
| opendevreview | Merged openstack/nova master: Handle neutron-client conflict https://review.opendev.org/c/openstack/nova/+/918048 | 09:28 |
| *** bauzas_ is now known as bauzas | 09:44 | |
| *** bauzas_ is now known as bauzas | 10:18 | |
| *** bauzas_ is now known as bauzas | 10:34 | |
| *** ykarel_ is now known as ykarel | 10:44 | |
| ykarel | is it known that https://review.opendev.org/c/openstack/nova/+/899753 broke 9-stream jobs atleast? | 10:44 |
| ykarel | fails as chown: cannot access '/opt/stack/nova/*.egg-info': No such file or directory | 10:45 |
| ykarel | sean-k-mooney, ^ | 10:46 |
| *** bauzas_ is now known as bauzas | 11:19 | |
| *** bauzas_ is now known as bauzas | 11:51 | |
| *** bauzas_ is now known as bauzas | 11:59 | |
| *** bauzas_ is now known as bauzas | 12:07 | |
| *** bauzas_ is now known as bauzas | 12:25 | |
| opendevreview | Rajesh Tailor proposed openstack/nova stable/2024.1: Handle neutron-client conflict https://review.opendev.org/c/openstack/nova/+/927732 | 12:43 |
| opendevreview | Merged openstack/nova master: Test live migration between hosts with differnet cpu_shared_sets https://review.opendev.org/c/openstack/nova/+/913744 | 12:48 |
| opendevreview | chandan kumar proposed openstack/nova master: [DNM] RDO third party experimental job check https://review.opendev.org/c/openstack/nova/+/927736 | 13:22 |
| ykarel | sean-k-mooney, for that egg-info issue the hack https://review.opendev.org/c/openstack/devstack/+/927730 worked fine | 13:25 |
| ykarel | also refreshed your patch for global venv https://review.opendev.org/c/openstack/devstack/+/898845 but looks need more work as jobs failing there | 13:26 |
| sean-k-mooney | ykarel: ya so it needs the wsgi stuff i think to be able to install in a venv | 14:57 |
| sean-k-mooney | ykarel: but ya you could ignore safe_chown or you could jsut check for "is_fedora == false" | 14:59 |
| sean-k-mooney | the wsgi stuff beign https://review.opendev.org/q/topic:%22remove-wsgi_scripts%22 | 14:59 |
| sean-k-mooney | gibi: it would be nice to have https://review.opendev.org/c/openstack/placement/+/919569 by the way for 2024.2 | 15:00 |
| sean-k-mooney | gibi: we have already merged the nova version so just placement left | 15:00 |
| *** bauzas_ is now known as bauzas | 15:10 | |
| *** bauzas_ is now known as bauzas | 15:22 | |
| opendevreview | Pavlo Shchelokovskyy proposed openstack/nova master: Actually schedule to default az when asked so https://review.opendev.org/c/openstack/nova/+/927744 | 15:26 |
| *** bauzas_ is now known as bauzas | 15:39 | |
| noonedeadpunk | hey folks! We've just updated one of our regions to 2024.1 and we do see a severe performance hit comparing to 2023.1. at least when trying to execute smth like `openstack server list --all-projects`. We've noticed, as we had to double timeout limit on the loadbalancer for nova-api backends | 15:44 |
| noonedeadpunk | from 60s to 3m as listing takes reliably 2m19s at the moment. | 15:45 |
| noonedeadpunk | do you happen to be aware of anything like that? | 15:45 |
| ykarel | sean-k-mooney, ack thx will check and update that hack patch tomorrow, need to drop now | 15:47 |
| *** ykarel is now known as ykarel|away | 15:47 | |
| sean-k-mooney | noonedeadpunk: not that im aware off, you should avoid that query but im not sure why would have regressed that | 16:01 |
| sean-k-mooney | hum | 16:01 |
| noonedeadpunk | we've noticed as ruinning it quite periodically for billing | 16:01 |
| sean-k-mooney | noonedeadpunk: there might be 2 things in effect | 16:02 |
| sean-k-mooney | https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-2024-1-caracal | 16:02 |
| sean-k-mooney | we added the pinned_avaiablity zone which need to read form the request spec | 16:02 |
| sean-k-mooney | noonedeadpunk: and around that time openstack client started using the latest microversion instaead of 2.1 | 16:02 |
| noonedeadpunk | well... I don't do --long | 16:03 |
| sean-k-mooney | noonedeadpunk: can you try setting --os-compute-api 2.1 | 16:03 |
| sean-k-mooney | if your osc is new enough to default to latest you got all the other thigns we added in the last 10 years too | 16:03 |
| noonedeadpunk | frankly - I was kind thinking more in a direction for compatability choices made for sqlalchemy 2 | 16:04 |
| sean-k-mooney | i dont think we are aware of any performace impacts for any of those | 16:04 |
| sean-k-mooney | certenly not doubling the responce time | 16:04 |
| * noonedeadpunk trying to see how long it would take with `--long` | 16:05 | |
| sean-k-mooney | noonedeadpunk: can you try pinning the microversion to 2.1 | 16:05 |
| noonedeadpunk | yeah, sec | 16:05 |
| noonedeadpunk | ok, so `--long` takes *exact* same time as not `--long` | 16:06 |
| noonedeadpunk | so I assume long just adds more response while all the same stuff is gathered | 16:06 |
| sean-k-mooney | long might be clinet side only parsing | 16:07 |
| *** bauzas_ is now known as bauzas | 16:07 | |
| sean-k-mooney | im not sure it changes the actually api request unless its changing between /servers/ and /servers/detail | 16:07 |
| noonedeadpunk | `--os-compute-api 2.1` doesn't make any difference to me | 16:08 |
| noonedeadpunk | yeah, seems just a client thing | 16:09 |
| sean-k-mooney | ok i guess we should capture this in a bug as a possibel performance reguression between 2023.1 and 2024.1 | 16:09 |
| noonedeadpunk | ok, I just wanted to check if you might be aware of that before digging deeper | 16:10 |
| sean-k-mooney | unfortunetly i suspect whatever it may be does not present on devstack or other small deployments | 16:10 |
| sean-k-mooney | i tought maybe it was https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id63 | 16:11 |
| sean-k-mooney | since if you have down cells that might change behaivor | 16:12 |
| noonedeadpunk | nah, it's a single cell and single AZ environment | 16:12 |
| noonedeadpunk | as we've started upgrade from the smallest thing we had | 16:13 |
| sean-k-mooney | ya so get /servers at the nova api level should be very quick | 16:13 |
| sean-k-mooney | https://docs.openstack.org/api-ref/compute/#id9 | 16:13 |
| sean-k-mooney | as it only retuns the minimal info in the api db | 16:14 |
| sean-k-mooney | but openstack server list | 16:14 |
| sean-k-mooney | is not the same thing as that | 16:14 |
| sean-k-mooney | i think it does more behind the scenes | 16:14 |
| sean-k-mooney | noonedeadpunk: around that time osc started replacing using python-novaclient with the sdk as well | 16:15 |
| noonedeadpunk | client also does pass bunch of filters: /v2.1/servers/detail?flavor=None&image=None&name=None&project_id=None&reservation_id=None&status=None&user_id=None&deleted=False&host=None&ip=None&ip6=None&changes-since=None&changes-before=None&all_tenants=True | 16:15 |
| noonedeadpunk | so I wonder if that might be client that I've upgraded | 16:15 |
| sean-k-mooney | you could try doing it with nova client or installing an older osc in a venv | 16:15 |
| sean-k-mooney | that woudl at least tell us if its server or client side | 16:16 |
| noonedeadpunk | or just do REST call | 16:16 |
| sean-k-mooney | yep or that | 16:16 |
| sean-k-mooney | you can do it with curl if you have a keyston token but i always have to look up the syntax | 16:16 |
| noonedeadpunk | so curl takes half of the time, as I assume osc does fetch more info to render uuids | 16:19 |
| noonedeadpunk | still 1m24s while haproxy used to be set to 60s timeout | 16:20 |
| noonedeadpunk | and request was `time curl -g -i --cacert "/etc/ssl/certs/ca-certificates.crt" -X GET "http://cloud.com:8774/v2.1/servers/detail?all_tenants=True" -H "Accept: application/json" -H "X-Auth-Token: ${token}" -H "X-OpenStack-Nova-API-Version: 2.92"` | 16:20 |
| sean-k-mooney | if you have the request id it would be good to check how long the request taks on the nova-api side | 16:26 |
| sean-k-mooney | /server/detail is not the same as /server | 16:27 |
| sean-k-mooney | so if you dont want all the addtional info avoid /server/detail | 16:27 |
| *** bauzas_ is now known as bauzas | 16:27 | |
| noonedeadpunk | well, I'm not sure about detail part, but assume it was preffered to limit amount of following requests. | 16:28 |
| noonedeadpunk | but also interesting to find | 16:28 |
| sean-k-mooney | well /detail is not really advisabel on large cloud if using --all-tenatns | 16:29 |
| sean-k-mooney | it can inovled calls to neutron or other sirvices for info in some cases | 16:29 |
| sean-k-mooney | we mostly try to cache some of that info but we dont cache all of it | 16:29 |
| sean-k-mooney | i am thinking of https://bugs.launchpad.net/nova/+bug/1923560 | 16:34 |
| sean-k-mooney | without https://review.opendev.org/c/openstack/nova/+/786348 we need to do a call to neutron for every server | 16:34 |
| sean-k-mooney | if you use /server/detail | 16:34 |
| opendevreview | sean mooney proposed openstack/nova master: enable numa live migration in the ceph job https://review.opendev.org/c/openstack/nova/+/913842 | 17:41 |
| opendevreview | sean mooney proposed openstack/nova master: enable numa live migration in the ceph job https://review.opendev.org/c/openstack/nova/+/913842 | 17:41 |
| *** bauzas_ is now known as bauzas | 17:57 | |
| *** bauzas_ is now known as bauzas | 18:05 | |
| *** bauzas_ is now known as bauzas | 18:13 | |
| *** bauzas_ is now known as bauzas | 18:30 | |
| *** priteau_ is now known as priteau | 19:12 | |
| *** bauzas_ is now known as bauzas | 19:27 | |
| *** bauzas_ is now known as bauzas | 20:53 | |
| *** bauzas_ is now known as bauzas | 21:33 | |
| *** bauzas_ is now known as bauzas | 21:41 | |
| *** bauzas_ is now known as bauzas | 21:50 | |
| *** bauzas_ is now known as bauzas | 22:51 | |
| opendevreview | sean mooney proposed openstack/nova master: [DNM] testing rdo experimental jobs https://review.opendev.org/c/openstack/nova/+/927762 | 22:53 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!