| manumoh | Hello all, I see the following error in my nova-api logs intermittently since last 24 hours . Nothing has been changed from service or config side. Any help in highly appreciated | 11:17 |
|---|---|---|
| manumoh | https://www.irccloud.com/pastebin/0nfGEBS7/connection_timeout_error | 11:18 |
| sean-k-mooney | that looks like your having network connectiviy issues betwhen nova and keystone | 11:20 |
| sean-k-mooney | assuming your not seeign issues in the keyston logs this is likely infra issue in your datacenter networking | 11:21 |
| sean-k-mooney | the call that is failling is to neutron to get the prots for the current tenant to check if there is enough quota | 11:22 |
| sean-k-mooney | so its posisbel that the keystaauth succeded and then then call to neutron is actully whats timign out | 11:22 |
| sean-k-mooney | i thinik thats actully more likely then it beign a keystone issue | 11:24 |
| manumoh | sean-k-mooney: ohhkk .. if its a neutron issue , then why the trace is giving timeout at /usr/lib/python3.9/site-packages/keystoneauth1/session.py | 11:29 |
| sean-k-mooney | the time out is nova calling neutron api via neutron client | 11:29 |
| sean-k-mooney | this is nova callign neutron https://cloud.test.com:9696/v2.0/ports?tenant_id=ad0ddb4fd2424410ae923d7f01746a83&fields=id is listing all port for that teant and only return the uuids so that we can count how many are in use to check if we can create teh vm without exceeding the port quota | 11:31 |
| manumoh | ok .. so if the neutron server fails to respond back for the port list api within 60 sec we will see the same error in logs | 11:33 |
| sean-k-mooney | yes | 11:34 |
| sean-k-mooney | this is part of the early validation logic in teh api so the user proably got a 500 | 11:34 |
| sean-k-mooney | if you have the request id you should be able to validate if neutron every recived the request and if it did why it took so long to reply | 11:35 |
| sean-k-mooney | FYI myself and other redhatter have a company "recharge day" today so we are all on PTO so im not sure how active the neutron channle is today | 11:36 |
| manumoh | yeah noticed it while creating an LB and the 500 errror in present in octavia logs | 11:36 |
| manumoh | @sean-k-mooney thanks for the reply .. i will check further | 11:36 |
| manumoh | I don't see the request ids in neutron logs | 11:42 |
| opendevreview | Merged openstack/nova stable/2024.1: Amend functional reproducer for bug 1899835 https://review.opendev.org/c/openstack/nova/+/950217 | 13:19 |
| opendevreview | Merged openstack/nova stable/2024.1: live migration: Avoid volume rollback mismatches https://review.opendev.org/c/openstack/nova/+/950218 | 14:41 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!