Monday, 2021-05-24

« Sunday, 2021-05-23 Index Tuesday, 2021-05-25 »
*** ccamposr has joined #openstack-neutron00:54
*** gmann_afk is now known as gmann01:30
*** sapd1 has joined #openstack-neutron01:54
*** sapd1_x has quit IRC01:56
*** sapd1 has quit IRC02:03
*** sapd1 has joined #openstack-neutron02:03
openstackgerritliuyulong proposed openstack/neutron-lib master: Adds API extension for QoS rule type pps  https://review.opendev.org/c/openstack/neutron-lib/+/79090502:52
*** sapd1_x has joined #openstack-neutron03:17
*** TMM has quit IRC03:48
*** TMM has joined #openstack-neutron03:48
*** psahoo has joined #openstack-neutron04:12
*** ratailor has joined #openstack-neutron04:58
openstackgerritManu B proposed openstack/neutron-lib master: API changes to support router association to BGP speaker  https://review.opendev.org/c/openstack/neutron-lib/+/79277405:07
*** gcheresh has joined #openstack-neutron05:12
openstackgerritLajos Katona proposed openstack/neutron master: WIP: Add exception handling to assert_async_ping  https://review.opendev.org/c/openstack/neutron/+/79250705:24
*** ralonsoh has joined #openstack-neutron05:28
*** sapd1_x has quit IRC05:37
*** sapd1_x has joined #openstack-neutron06:01
*** sapd1_x has quit IRC06:25
*** slaweq has joined #openstack-neutron06:33
openstackgerritSlawek Kaplonski proposed openstack/neutron master: Revert "Set system_scope='all' in elevated context"  https://review.opendev.org/c/openstack/neutron/+/78807206:51
*** psahoo has quit IRC06:54
*** jlibosva has joined #openstack-neutron06:59
*** psahoo has joined #openstack-neutron07:08
*** sapd1_x has joined #openstack-neutron07:09
openstackgerritliuyulong proposed openstack/neutron master: Add test cases for concurrently Object delete  https://review.opendev.org/c/openstack/neutron/+/77770907:12
*** bengates has joined #openstack-neutron07:30
*** slaweq has quit IRC07:32
*** manpreet has joined #openstack-neutron07:34
*** slaweq has joined #openstack-neutron07:35
*** elvira has joined #openstack-neutron07:37
*** lucasagomes has joined #openstack-neutron07:58
openstackgerritRodolfo Alonso proposed openstack/neutron master: Allow the use of legacy routers within RPN segments  https://review.opendev.org/c/openstack/neutron/+/79117808:09
openstackgerritliuyulong proposed openstack/neutron-tempest-plugin master: Remove the case of multiple fip:port to one fixed_ip:port  https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/78747408:21
*** ociuhandu has joined #openstack-neutron08:23
*** stephenfin has quit IRC08:27
openstackgerritliuyulong proposed openstack/neutron master: [L3][Port forwarding] multiple floating_ip:port to same fixed_ip:port  https://review.opendev.org/c/openstack/neutron/+/78730408:29
openstackgerritliuyulong proposed openstack/neutron stable/stein: Clean port forwarding cache when router is DOWN  https://review.opendev.org/c/openstack/neutron/+/77977808:31
openstackgerritRodolfo Alonso proposed openstack/neutron master: [OVN][Placement] Add a SB Chassis event to track changes in BW config  https://review.opendev.org/c/openstack/neutron/+/77670108:36
openstackgerritRodolfo Alonso proposed openstack/neutron master: Remove ``ConfDriver`` code  https://review.opendev.org/c/openstack/neutron/+/79099908:37
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/wallaby: [DVR] Send allowed address pairs info to the L3 agents  https://review.opendev.org/c/openstack/neutron/+/79279008:51
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/victoria: [DVR] Send allowed address pairs info to the L3 agents  https://review.opendev.org/c/openstack/neutron/+/79275608:52
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/ussuri: [DVR] Send allowed address pairs info to the L3 agents  https://review.opendev.org/c/openstack/neutron/+/79275708:52
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/train: [DVR] Send allowed address pairs info to the L3 agents  https://review.opendev.org/c/openstack/neutron/+/79279109:12
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/stein: [DVR] Send allowed address pairs info to the L3 agents  https://review.opendev.org/c/openstack/neutron/+/79279209:15
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/rocky: [DVR] Send allowed address pairs info to the L3 agents  https://review.opendev.org/c/openstack/neutron/+/79275909:16
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/queens: [DVR] Send allowed address pairs info to the L3 agents  https://review.opendev.org/c/openstack/neutron/+/79276009:18
*** sapd1_y has joined #openstack-neutron09:19
*** sapd1_x has quit IRC09:19
*** sapd1 has quit IRC09:19
openstackgerritRodolfo Alonso proposed openstack/neutron master: Check router routes connectivity when GW port is updated  https://review.opendev.org/c/openstack/neutron/+/78992909:22
*** yankcrime has joined #openstack-neutron09:23
yankcrimehey neutron folks - any idea why i see a lot of "Linux bridge agent Agent has just been revived. Doing a full sync." in my logs, even though the agent never appears down?09:24
yankcrimein the meantime the agent is pegging a full cpu core and is very slow to process any required changes09:25
yankcrime(this is on ussuri)09:25
ralonsohcould be a rabbitmq problem, a bug in linux bridge agent, etc. Anything that triggers a full resync09:27
ralonsohyou should find the root cause, this is just the symptom09:27
ralonsohis there an error in the logs? set them in debug mdoe09:27
ralonsoh*mode09:27
yankcrimermq is fine with all other agents, including linuxbridge agents on other nodes09:28
yankcrimeand yeah, it's in debug and there's nothing either side09:28
yankcrimewell no error either side i mean09:28
ralonsohthat means some operation is stuck or slowing down the normal process and maybe (I'm just guessing) you hit the agent timeout09:29
ralonsohthat will trigger a resync09:29
ralonsohthe agent has a processing loop09:30
*** jpenag is now known as jpena09:30
ralonsohand sends heartbeats to the server09:30
yankcrimeare you talking about the rmq heartbeats?09:30
ralonsohyou need to find the operation that is not working fine09:30
ralonsohyes09:30
yankcrimeif it hits the rmq heartbeat timeout then it'd be disconnected, i assume anyway09:31
yankcrimeand i'd see messages to that effect09:31
yankcrimeis there another timeout setting i can apply to give the agent more time to complete its work?09:32
ralonsohreport_interval09:33
ralonsohbut I don't know what is failing in your env09:34
ralonsohand you said only one agent is in this state09:34
yankcrimeyup, it's just this one agent09:34
ralonsohso that should be something in this compute node09:34
*** ociuhandu has quit IRC09:34
ralonsoh(I guess this is a compute node)09:34
yankcrimeit's a network node09:34
yankcrimethere doesn't appear to be anything failing, it just sometimes takes a while before a port is updated and becomes active (for example)09:35
*** ociuhandu has joined #openstack-neutron09:35
ralonsohwithout logs, I just can speculate09:37
*** sapd1 has joined #openstack-neutron09:41
yankcrimei've just tried bumping those timeout values, then i'll grab some logs09:42
yankcrimeit's been bugging me for a few days now so any help on this is hugely appreciated09:42
*** ociuhandu has quit IRC09:43
openstackgerritDaniel Alvarez proposed openstack/neutron master: [ovn][metadata] Remove metadata readiness mechanism  https://review.opendev.org/c/openstack/neutron/+/79199709:47
*** jangutter_ has joined #openstack-neutron09:50
*** jangutter has quit IRC09:53
*** psahoo has quit IRC10:00
*** yamamoto has quit IRC10:02
openstackgerritJakub Libosvar proposed openstack/neutron master: [ovn][metadata] Remove metadata readiness mechanism  https://review.opendev.org/c/openstack/neutron/+/79199710:05
*** carloss has joined #openstack-neutron10:12
*** psahoo has joined #openstack-neutron10:13
*** sapd1 has quit IRC10:16
yankcrimeralonsoh: https://gist.github.com/yankcrime/2f6b9518a82ceeb5092cfd8c461918f710:17
ralonsohyankcrime, you have trunk ports disables in the server10:22
ralonsohwhy is this enabled in the agent?10:22
yankcrimeit's not explicitly enabled10:26
yankcrimeso it must be the default behaviour10:26
ralonsohyou have a problem with this extension: every time you lock a trunk, the agent spends more than 1 min10:27
ralonsohyou need to disable this extension10:28
*** sapd1_y has quit IRC10:29
*** sapd1 has joined #openstack-neutron10:30
yankcrimehow? it's not enabled10:30
yankcrimeservice_plugins = trunk isn't set anywhere10:30
*** sapd1_x has joined #openstack-neutron10:31
*** sapd1_x has quit IRC10:31
*** sapd1_x has joined #openstack-neutron10:32
yankcrimeaccording to https://docs.openstack.org/neutron/ussuri/admin/config-trunking.html10:34
yankcrimeservice_plugins = trunk isn't set either for neutron-server or any agents10:34
*** yamamoto has joined #openstack-neutron10:40
*** lpetrut has joined #openstack-neutron10:43
*** pmannidi has joined #openstack-neutron10:43
*** pmannidi has quit IRC10:43
*** ociuhandu has joined #openstack-neutron10:47
openstackgerritRodolfo Alonso proposed openstack/neutron master: Allow the use of legacy routers within RPN segments  https://review.opendev.org/c/openstack/neutron/+/79117810:52
*** ociuhandu has quit IRC11:00
*** ociuhandu has joined #openstack-neutron11:03
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/rocky: [ovs fw] Restrict IPv6 NA and DHCP(v6) IP and MAC source addresses  https://review.opendev.org/c/openstack/neutron/+/79146911:04
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/queens: [ovs fw] Restrict IPv6 NA and DHCP(v6) IP and MAC source addresses  https://review.opendev.org/c/openstack/neutron/+/79147011:04
*** ociuhandu has quit IRC11:15
*** sapd1 has quit IRC11:15
*** ccamposr has quit IRC11:21
*** ccamposr has joined #openstack-neutron11:21
openstackgerritSlawek Kaplonski proposed openstack/neutron stable/queens: [ovs fw] Restrict IPv6 NA and DHCP(v6) IP and MAC source addresses  https://review.opendev.org/c/openstack/neutron/+/79147011:24
*** elvira is now known as elvira-lunch11:27
*** ociuhandu has joined #openstack-neutron11:27
*** ociuhandu has quit IRC11:31
*** ociuhandu has joined #openstack-neutron11:34
*** jpena is now known as jpena|lunch11:34
*** ociuhandu has quit IRC11:38
*** ociuhandu has joined #openstack-neutron11:38
*** ociuhandu has quit IRC11:39
*** ociuhandu has joined #openstack-neutron11:41
*** sapd1 has joined #openstack-neutron11:42
yankcrimeralonsoh: could this be a bug in the linuxbridge agent where that trunk plugin is implicitly enabled?11:42
*** ociuhandu has quit IRC11:47
*** yumiriam has joined #openstack-neutron11:50
*** ociuhandu has joined #openstack-neutron11:59
*** ociuhandu has quit IRC12:02
*** ociuhandu has joined #openstack-neutron12:03
*** sapd1_x has quit IRC12:11
*** sapd1 has quit IRC12:12
openstackgerritLucas Alvares Gomes proposed openstack/networking-ovn stable/train: [OVN] Check for lock in check_for_mcast_flood_reports  https://review.opendev.org/c/openstack/networking-ovn/+/78958212:18
openstackgerritSzymon Wróblewski proposed openstack/neutron master: Fix ObjectChangeHandler thread usage  https://review.opendev.org/c/openstack/neutron/+/78851012:20
*** jpena|lunch is now known as jpena12:24
*** Carlotronics has joined #openstack-neutron12:29
*** elod has quit IRC12:56
*** elod has joined #openstack-neutron13:05
*** TMM has quit IRC13:06
*** TMM has joined #openstack-neutron13:06
*** sapd1 has joined #openstack-neutron13:11
openstackgerritDaniel Alvarez proposed openstack/neutron master: [ovn][metadata] Remove metadata readiness mechanism  https://review.opendev.org/c/openstack/neutron/+/79199713:15
*** yamamoto has quit IRC13:33
*** ociuhandu has quit IRC13:40
*** gregraka has joined #openstack-neutron13:42
*** gregraka has quit IRC13:44
*** gregraka has joined #openstack-neutron13:44
*** ratailor has quit IRC13:49
*** ociuhandu has joined #openstack-neutron13:54
*** ociuhandu has quit IRC13:59
*** ociuhandu has joined #openstack-neutron14:01
*** yamamoto has joined #openstack-neutron14:14
*** jangutter has joined #openstack-neutron14:16
*** jangutter has quit IRC14:16
*** jangutter_ has quit IRC14:16
*** jangutter has joined #openstack-neutron14:16
JohnnyRainbowHi, just a question regarding upgrade from neutron rocky to stein. Due to upgrade instruction I need to upgrade neutron-server first, before upgrading neutron agents. I have a HA configuration with 3VMs serving neutron functionalities. I was wondering if I can update neutron-01 VM first, but seems not, as upgraded agents are not able to connect due to amqp issue. So I'm wondering how I14:20
JohnnyRainbowcan simply perform upgrade of neutron-server on top of all three nodes, without agents upgrades, especially that operating system(ubuntu) is automatically trying to upgrade all correlated packages with neutron-server, so all agents...the problem is that between rocky and stein, python version is changed from 2 to 3...any advices how to do this the best? Thanks in advance!14:20
*** gregraka is now known as gregraka-afk14:24
*** ociuhandu has quit IRC14:25
JohnnyRainbowregarding mentioned RPC issues here is an example http://paste.openstack.org/show/805649/ how it looks when one agent is upgraded to stein where 2 of 3 neutron-servers are still rocky14:26
*** yamamoto has quit IRC14:29
*** ociuhandu has joined #openstack-neutron14:38
*** ociuhandu has quit IRC14:42
*** ociuhandu has joined #openstack-neutron14:43
*** jangutter_ has joined #openstack-neutron14:47
*** lpetrut has quit IRC14:50
*** jangutter has quit IRC14:51
*** gregraka-afk is now known as gregraka14:53
*** gregraka has quit IRC14:53
*** gregraka has joined #openstack-neutron14:54
*** jangutter has joined #openstack-neutron15:03
*** jangutter_ has quit IRC15:07
openstackgerritRodolfo Alonso proposed openstack/neutron master: Remove ``ConfDriver`` code  https://review.opendev.org/c/openstack/neutron/+/79099915:20
*** yamamoto has joined #openstack-neutron15:24
*** jangutter_ has joined #openstack-neutron15:33
*** cp- has quit IRC15:34
*** cp- has joined #openstack-neutron15:34
*** jangutter has quit IRC15:37
*** yamamoto has quit IRC15:41
*** bengates has quit IRC15:44
*** jlibosva has quit IRC15:52
*** lucasagomes has quit IRC16:02
*** elvira-lunch has quit IRC16:05
openstackgerritMerged openstack/neutron-lib master: Change minversion of tox to 3.18.0  https://review.opendev.org/c/openstack/neutron-lib/+/79199616:06
*** jangutter has joined #openstack-neutron16:09
*** ociuhandu has quit IRC16:12
*** jangutter_ has quit IRC16:13
fricklerJohnnyRainbow: you must upgrade all neutron-server instances first, then agents. if you are running agents on the same servers as the API, you'll likely need to disable the not-yet-upgraded API instances in your load-balancer16:17
*** psahoo has quit IRC16:21
*** psahoo has joined #openstack-neutron16:22
*** yumiriam has quit IRC16:22
*** psahoo has quit IRC16:39
openstackgerritMerged openstack/neutron stable/stein: Clean port forwarding cache when router is DOWN  https://review.opendev.org/c/openstack/neutron/+/77977817:00
openstackgerritMerged openstack/neutron master: Add test cases for concurrently Object delete  https://review.opendev.org/c/openstack/neutron/+/77770917:01
*** dsneddon has joined #openstack-neutron17:02
openstackgerritRodolfo Alonso proposed openstack/neutron master: Remove ``ConfDriver`` code  https://review.opendev.org/c/openstack/neutron/+/79099917:21
*** ralonsoh has quit IRC17:23
*** jpena is now known as jpena|off17:23
*** coreycb has joined #openstack-neutron17:25
openstackgerritsean mooney proposed openstack/os-vif stable/victoria: Resolve dependency issues  https://review.opendev.org/c/openstack/os-vif/+/79284018:30
*** sapd1_x has joined #openstack-neutron18:34
*** zijlboot has quit IRC18:35
*** sapd1 has quit IRC18:37
*** gcheresh has quit IRC19:35
openstackgerritMerged openstack/neutron stable/victoria: [OVN] Fix: Disabling snat after it was enabled  https://review.opendev.org/c/openstack/neutron/+/79253719:45
openstackgerritMerged openstack/neutron stable/ussuri: [OVN] Fix: Disabling snat after it was enabled  https://review.opendev.org/c/openstack/neutron/+/79253619:48
*** gcheresh has joined #openstack-neutron20:00
*** slaweq has quit IRC20:14
*** gcheresh has quit IRC20:25
*** gcheresh has joined #openstack-neutron21:01
*** gcheresh has quit IRC21:32
*** pmannidi has joined #openstack-neutron21:33
*** openstackgerrit has quit IRC22:05
*** yamamoto has joined #openstack-neutron22:36
*** snapiri has quit IRC23:14
*** snapiri has joined #openstack-neutron23:14
*** TMM has quit IRC23:19
*** TMM has joined #openstack-neutron23:21
*** openstack has joined #openstack-neutron23:50
*** ChanServ sets mode: +o openstack23:50
*** openstackgerrit has joined #openstack-neutron23:58
openstackgerritMerged openstack/neutron stable/wallaby: [OVN] Fix: Disabling snat after it was enabled  https://review.opendev.org/c/openstack/neutron/+/79253823:58
« Sunday, 2021-05-23 Index Tuesday, 2021-05-25 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!