| opendevreview | yatin proposed openstack/neutron master: [DNM] Check functional timeouts https://review.opendev.org/c/openstack/neutron/+/950303 | 05:53 |
|---|---|---|
| opendevreview | yatin proposed openstack/neutron-tempest-plugin master: Use infra mirrors for OVN image prepare https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/949484 | 06:30 |
| opendevreview | yatin proposed openstack/neutron-tempest-plugin master: Use infra mirrors for advanced image prepare https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/949484 | 06:31 |
| opendevreview | yatin proposed openstack/neutron-tempest-plugin master: [DNM] Check infra mirror patch https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/951239 | 06:48 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron-tempest-plugin master: Use only "ubuntu-22.04-minimal" advance image https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/951149 | 06:51 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Deprecate `vlan_qinq` and `vlan_trasnaprent` config options https://review.opendev.org/c/openstack/neutron/+/951136 | 07:34 |
| opendevreview | yatin proposed openstack/neutron master: [DNM] Check functional timeouts https://review.opendev.org/c/openstack/neutron/+/950303 | 08:14 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron-tempest-plugin master: Use only "ubuntu-22.04-minimal" advance image https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/951149 | 08:32 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: DNM: Evaluating --concurrency vs --isolated https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951150 | 08:33 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: DNM: Evaluating --concurrency vs --isolated https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951150 | 08:56 |
| opendevreview | Merged openstack/neutron master: [FT] Match the correct event in ``test_subport_delete`` https://review.opendev.org/c/openstack/neutron/+/951119 | 09:09 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: DNM: Evaluating --concurrency vs --isolated https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951150 | 09:20 |
| ralonsoh | slaweq, hello! If you have some minutes: https://review.opendev.org/c/openstack/neutron/+/949771 | 10:05 |
| ralonsoh | I've also proposed the backports, because the n-t-p patch needs them too | 10:05 |
| ralonsoh | --> https://review.opendev.org/q/topic:%22bug/2109354%22 | 10:05 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron-tempest-plugin master: List networks with limit and filter by provider network attrs https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/949975 | 10:06 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: DNM: Evaluating --concurrency vs --isolated https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951150 | 10:14 |
| opendevreview | Merged openstack/neutron-tempest-plugin master: Use infra mirrors for advanced image prepare https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/949484 | 10:16 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Do not create the min-bw request if the network segment is deleted https://review.opendev.org/c/openstack/neutron/+/951262 | 10:47 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-removal] Remove the usage of eventlet in the L3 agent https://review.opendev.org/c/openstack/neutron/+/938404 | 10:49 |
| opendevreview | Lajos Katona proposed openstack/neutron master: [eventlet-removal] DHCP: move RPC init to service.start phase https://review.opendev.org/c/openstack/neutron/+/950498 | 10:50 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-removal] Remove the usage of eventlet from the OVN maintenance worker https://review.opendev.org/c/openstack/neutron/+/950854 | 10:51 |
| opendevreview | Renjing Xiao proposed x/whitebox-neutron-tempest-plugin master: Improve WSGI check for devstack https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/951264 | 10:56 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Ensure LB with HM rechecks member status after re-enable https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951274 | 12:08 |
| hilltoo | Hello, when the metadata agent attempts to grab a port for the linklocal metadata agent no port is returned. Should this log line be a debug instead of an error? Things seem to work and i'm not finding a doc to better explain what's happening behind the scenes here. But it appears this shouldn't be spitting out an error | 13:51 |
| hilltoo | https://github.com/openstack/neutron/blob/master/neutron/agent/metadata/agent.py#L178 | 13:51 |
| hilltoo | re; vm hits 169.254.169.254 and the 'no port found..' error is printed but the metadata request is successful | 13:51 |
| opendevreview | Merged openstack/ovn-octavia-provider master: [FT] Unlock the functional jobs https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951146 | 14:12 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Ensure LB with HM rechecks member status after re-enable https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951274 | 14:16 |
| haleyb | hilltoo: it could maybe be a warning, but it is a condition that should not really happen - metadata request from an IP and we can't find the port | 14:30 |
| ralonsoh | hilltoo, please send the metadata logs, in debug mode if possible | 14:35 |
| ralonsoh | and open a LP bug, at least to investigate that | 14:35 |
| ralonsoh | what version are you running? | 14:35 |
| opendevreview | Brian Haley proposed openstack/neutron master: Deprecate `vlan_qinq` and `vlan_trasnaprent` config options https://review.opendev.org/c/openstack/neutron/+/951136 | 14:41 |
| hilltoo | is there any docs/info i could read on the workflow here? The one's in the doc pages now seem out of date. I'm hoping to better understand the flow for a metadata request with ovn/ml2 and no dhcp agent. | 14:43 |
| ralonsoh | hilltoo, this is exactly the same: the http request is captured by the HAProxy, HAProxy sent that via unix socket to the Metadata proxy handler (in the OVN metadata agent) and that is sent to the Nova metadata server | 14:49 |
| ralonsoh | we use the HAProxy to "escape" from the namespace | 14:50 |
| opendevreview | Brian Haley proposed openstack/neutron master: Deprecate `vlan_qinq` and `vlan_transparent` config options https://review.opendev.org/c/openstack/neutron/+/951136 | 14:52 |
| hilltoo | so, there shouldn't be any complaints about not finding a port for 169.254.169.254 right? | 14:55 |
| ralonsoh | I would need to check the logs | 14:55 |
| hilltoo | will do, i have to get set back up and get debug enabled. Thanks for the info | 14:56 |
| ralonsoh | but this IP is the metadata IP, that shouldn't be assigned to any port | 14:56 |
| hilltoo | that's what i figured | 14:56 |
| haleyb | i don't think that's what it's complaining about, unless i read the code wrong. it should be complaining about some instance IP | 14:56 |
| hilltoo | yeah, in this case the log is basically 'error... "No port found in network None with IP address 169.254.169.254"' | 14:57 |
| hilltoo | i'll get debug going and put together a lp | 14:57 |
| haleyb | hilltoo: please file the bug then since i'm confused without the logs | 14:57 |
| haleyb | thanks | 14:57 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron-tempest-plugin master: Use only "ubuntu-22.04-minimal" advance image https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/951149 | 15:09 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [eventlet-removal] Remove the usage of eventlet in the L3 agent https://review.opendev.org/c/openstack/neutron/+/938404 | 15:11 |
| ralonsoh | haleyb, hi! Please check ^ if you have time | 15:11 |
| ralonsoh | I'm leaving now, I'll check it tomorrow | 15:11 |
| hilltoo | https://bugs.launchpad.net/neutron/+bug/2112130 Created that. The related debug output is minimal. I am still digging and will update the bug if I find anything else. Thanks for taking a look! | 15:37 |
| ykarel | hilltoo, can you also find what host is creating those metadata requests as doesn't looks originating from a nova vm | 15:45 |
| hilltoo | it appears it's coming from the host it's running on from the health probe? When i make a request from within the vm I get a 200 with the metadata returned. I only see this when the probe is running. Not sure how to determine where else that might be coming from | 15:48 |
| ykarel | based on X-Neutron-Router-Id: pod-health-probe-check-ignore-errors, looks like some healthcheck running from womewher | 15:48 |
| ykarel | s/womewher/somewhere | 15:49 |
| hilltoo | There's nothing else in the debug logs, not sure offhand how to figure out where that request might be coming from | 15:55 |
| hilltoo | so, it does appear to be coming from a container within the ovn-metadata pod https://github.com/openstack/openstack-helm/blob/master/neutron/templates/bin/_health-probe.py.tpl#L232 | 16:00 |
| hilltoo | and that's the neutron-ovn-metadata-agent container | 16:01 |
| haleyb | you might need to ask in the openstack-helm channel, i don't know anything about that code and/or if it needs to be updated based on any changes we've made to neutron ? | 16:14 |
| hilltoo | but that's just the request coming in right? Why would the agent itself error out there on an ip it knows about? https://github.com/openstack/neutron/blob/master/neutron/agent/metadata/agent.py#L178. That health probe code is 6 years old it seems. | 16:25 |
| hilltoo | i'll message in the helm chan and see what they say | 16:26 |
| haleyb | the IP is not an instance IP associated with a port, the 'X-Neutron-Router-ID': 'pod-health-probe-check-ignore-errors' part is most likely causing the applied filter to return [] for ports too | 16:58 |
| haleyb | so it's not really a valid request afaik | 16:58 |
| hilltoo | gotcha! I posted a message in openstack-helm to see if they have any thoughts. not too many folks there | 16:59 |
| haleyb | i wonder how many people have used that healthcheck over the past 6 years... | 17:01 |
| hilltoo | heh :) we're using it across our regions now with genestack | 17:06 |
| hilltoo | Maybe this is expected behavior from their side as it does seem to give a 404. From the commit notes: "Neutron Metadata agent: Sends a message to Unix Domain Socket opened by Metadata agent. Probe is success if agent returns with HTTP status 404." | 17:09 |
| hilltoo | Wonder if we want the agent to know about this type of request and not log error if it meets some defined rule(like looking at the header some how.. or just the linklocal ip itself) | 17:10 |
| hilltoo | looks like the code that throws the error log in the metadata agent was added lastyear https://github.com/openstack/neutron/commit/7b4e9f8c26188713c6885b3851f1c11b33d78881#diff-fdb54d359cab53447c9eddd78ed76a8bf5112088dec9a164a5f5685696e7f671L18-R142 | 17:16 |
| hilltoo | slightly more readable https://github.com/openstack/neutron/blame/master/neutron/agent/metadata/agent.py#L178 | 17:17 |
| haleyb | yeah, i know that guy (me) but it was LOG.error in the ovn code so i just copied over. It might be fine a LOG.info or warning looking at it again | 17:36 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!