Tuesday, 2019-10-22

*** markvoelker has joined #openstack-neutron00:03
*** macz has quit IRC00:05
*** armax has quit IRC00:08
*** openstackgerrit has joined #openstack-neutron00:09
openstackgerritManjeet Singh Bhatia proposed openstack/neutron master: Add SR-IOV ML2 driver support for VLAN trunking.  https://review.opendev.org/66546700:09
*** david-lyle is now known as dklyle00:09
*** frank_wang has joined #openstack-neutron00:12
*** markvoelker has quit IRC00:13
*** frankwang has quit IRC00:16
openstackgerritManjeet Singh Bhatia proposed openstack/neutron master: Add SR-IOV ML2 driver support for VLAN trunking.  https://review.opendev.org/66546700:19
openstackgerritMerged openstack/neutron stable/train: [nova][train] Docs: Remove deprecated RetryFilter  https://review.opendev.org/68964400:20
*** weifan has quit IRC00:30
*** ociuhandu has joined #openstack-neutron00:31
*** weifan has joined #openstack-neutron00:32
*** weifan has quit IRC00:33
*** weifan has joined #openstack-neutron00:34
*** ociuhandu has quit IRC00:35
*** slaweq has joined #openstack-neutron00:46
*** frankwang has joined #openstack-neutron00:47
*** frank_wang has quit IRC00:49
*** ileixe has joined #openstack-neutron00:49
*** frank_wang has joined #openstack-neutron00:51
ileixeamotoki: Hi there, I was lost my connection yesterday, and log tells you :)00:51
*** slaweq has quit IRC00:51
*** frankwang has quit IRC00:54
*** spatel has joined #openstack-neutron00:59
*** weifan has quit IRC01:05
*** frankwang has joined #openstack-neutron01:09
*** frank_wang has quit IRC01:11
*** yamamoto has joined #openstack-neutron01:25
*** nanzha has joined #openstack-neutron01:33
*** frank_wang has joined #openstack-neutron01:40
*** frankwang has quit IRC01:44
*** mhen has quit IRC01:47
*** mhen has joined #openstack-neutron01:47
*** slaweq has joined #openstack-neutron01:47
*** nanzha has quit IRC01:48
*** nanzha has joined #openstack-neutron01:49
*** slaweq has quit IRC01:53
*** nanzha has quit IRC01:54
openstackgerritDeepak proposed openstack/neutron master: Add SR-IOV ML2 driver support for VLAN trunking.  https://review.opendev.org/66546701:54
*** nanzha has joined #openstack-neutron01:55
*** weifan has joined #openstack-neutron02:05
*** markvoelker has joined #openstack-neutron02:14
*** nanzha has quit IRC02:18
*** markvoelker has quit IRC02:18
*** nanzha has joined #openstack-neutron02:19
*** yamamoto has quit IRC02:19
*** brault has quit IRC02:24
*** brault has joined #openstack-neutron02:25
*** frankwang has joined #openstack-neutron02:25
*** frank_wang has quit IRC02:27
*** weifan has quit IRC02:31
*** frank_wang has joined #openstack-neutron02:42
*** frankwang has quit IRC02:45
*** slaweq has joined #openstack-neutron02:48
*** spsurya has joined #openstack-neutron02:50
*** slaweq has quit IRC02:53
*** weifan has joined #openstack-neutron03:05
*** dave-mccowan has quit IRC03:05
openstackgerritzhanghao proposed openstack/neutron-lib master: Move fwaas_v2_log constants to neutron-lib  https://review.opendev.org/68962303:12
*** weifan has quit IRC03:20
*** rkukura has quit IRC03:34
*** slaweq has joined #openstack-neutron03:49
*** slaweq has quit IRC03:54
*** brault has quit IRC03:56
*** brault has joined #openstack-neutron03:56
*** spatel has quit IRC04:02
*** spatel has joined #openstack-neutron04:02
*** spatel has quit IRC04:03
*** frankwang has joined #openstack-neutron04:08
*** frank_wang has quit IRC04:10
*** frank_wang has joined #openstack-neutron04:37
*** frankwang has quit IRC04:41
*** nanzha has quit IRC04:43
*** nanzha has joined #openstack-neutron04:43
*** lajoskatona has joined #openstack-neutron04:45
*** slaweq has joined #openstack-neutron04:50
*** slaweq has quit IRC04:54
*** ileixe has quit IRC04:55
*** ileixe has joined #openstack-neutron04:57
*** do3meli has joined #openstack-neutron05:02
*** ratailor has joined #openstack-neutron05:09
*** frankwang has joined #openstack-neutron05:12
*** frank_wang has quit IRC05:14
*** ralonsoh has joined #openstack-neutron05:18
*** frank_wang has joined #openstack-neutron05:22
*** frankwang has quit IRC05:25
*** slaweq has joined #openstack-neutron05:28
*** gcheresh_ has joined #openstack-neutron05:28
*** janki has joined #openstack-neutron05:30
*** gcheresh_ has quit IRC05:39
*** weifan has joined #openstack-neutron05:39
*** weifan has quit IRC05:39
*** weifan has joined #openstack-neutron05:40
*** weifan has quit IRC05:44
openstackgerritNate Johnston proposed openstack/neutron-lib master: Allow for both regular and bulk resource extenders  https://review.opendev.org/68994505:53
*** slaweq has quit IRC05:59
*** slaweq has joined #openstack-neutron06:00
*** frankwang has joined #openstack-neutron06:01
*** frank_wang has quit IRC06:03
*** slaweq has quit IRC06:05
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: Temporary disable CI job neutron-functional-python27  https://review.opendev.org/68995706:05
*** nanzha has quit IRC06:15
*** nanzha has joined #openstack-neutron06:15
*** rcernin has quit IRC06:16
*** markvoelker has joined #openstack-neutron06:17
*** sapd1 has joined #openstack-neutron06:21
*** markvoelker has quit IRC06:21
*** frank_wang has joined #openstack-neutron06:21
*** frankwang has quit IRC06:24
*** igordc has joined #openstack-neutron06:34
*** ircuser-1 has quit IRC06:35
*** aedc has quit IRC06:40
*** frankwang has joined #openstack-neutron06:43
*** frank_wang has quit IRC06:45
*** pcaruana has joined #openstack-neutron06:47
*** gcheresh_ has joined #openstack-neutron06:47
*** igordc has quit IRC06:48
*** bobmel has joined #openstack-neutron06:49
*** trident has quit IRC06:53
*** njohnston_ has joined #openstack-neutron06:53
*** njohnston has quit IRC06:54
*** frank_wang has joined #openstack-neutron06:54
*** papasuder has joined #openstack-neutron06:54
*** frankwang has quit IRC06:57
*** trident has joined #openstack-neutron06:58
papasuderHello all, Paweł here. I would I like to clarify one thing related to OVS agent and extension manager. During troubleshooting issue with port removal of skipped devices, I found that there is a line added by slaweq'u to remove the within ext_manager: https://review.opendev.org/#/q/I3cf5c57c7f232deaa190ab6b0129e398fdabe59206:59
papasuderIt made me a small hope that port will be removed, but.. there is a shadow of the whole story - if there is NO extension configured, no information about NOT removed skipped devices will be printed.07:00
*** frank_wang has quit IRC07:00
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: Reset timeout exception in DietTestCase when retrying  https://review.opendev.org/68965807:00
*** frank_wang has joined #openstack-neutron07:00
papasuderhttps://github.com/openstack/neutron/blob/bf98b2a12a352e5bc7bbee1994d44b82e7dfe928/neutron/agent/l2/l2_agent_extensions_manager.py#L52-L60 there is no ELSE for the FOR loop and due to that no information about NOT removed port is printed.07:01
papasuderAnyway, what I checked, I cannot find suitable L2 extenstion which can handle that port deletion: https://github.com/openstack/neutron/blob/bf98b2a12a352e5bc7bbee1994d44b82e7dfe928/setup.cfg#L113-L11607:02
papasuderI am wondering if I understand it correctly. Case is to remove port from OVS integration bridge in case of "No such device" ofport = -107:02
ralonsohpapasuder, the goal of https://review.opendev.org/#/c/533318/1/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py if that: to remove the port if is not present in the integration brisge07:04
ralonsohfor https://github.com/openstack/neutron/blob/bf98b2a12a352e5bc7bbee1994d44b82e7dfe928/setup.cfg#L113-L11607:04
*** slaweq has joined #openstack-neutron07:04
ralonsohyou can propose a patch to log that this object was not treated07:04
ralonsohbut is the same for handle_port07:04
papasuderAll right, but is there any L2 extenstion which can be used for deleting port from OVS when ofport is -1?07:05
ralonsohofport -1 is an invalid port07:06
ralonsohthose ports are not considered07:07
papasuderBut those port are treated as skipped devices, so on are in this part of code: https://github.com/openstack/neutron/blob/bf98b2a12a352e5bc7bbee1994d44b82e7dfe928/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L1836-L184207:08
papasuderSo those ports are being processed with line: self.ext_manager.delete_port(...)07:08
*** frankwang has joined #openstack-neutron07:09
ralonsohif this happens, that means this port was added and in the same polling cycle, deleted07:10
ralonsohand now it's not present in the int br07:10
*** frank_wang has quit IRC07:11
*** cshen has joined #openstack-neutron07:11
*** frank_wang has joined #openstack-neutron07:12
*** sridharg has joined #openstack-neutron07:12
*** mjozefcz|lunch has joined #openstack-neutron07:12
*** ccamposr has joined #openstack-neutron07:14
*** frankwang has quit IRC07:15
papasuderralonsoh: yeap, that's what I have :)07:17
*** frankwang has joined #openstack-neutron07:21
*** frank_wang has quit IRC07:23
*** bobmel has quit IRC07:24
*** bobmel has joined #openstack-neutron07:25
*** jawad_axd has joined #openstack-neutron07:26
*** njohnston_ is now known as njohnston07:28
*** bobmel has quit IRC07:29
*** aedc has joined #openstack-neutron07:32
*** frank_wang has joined #openstack-neutron07:32
mjozefcz|lunchpapasuder, \o07:34
*** mjozefcz|lunch is now known as mjozefcz07:34
*** mjozefcz is now known as maciejjozefczyk07:34
*** jawad_axd is now known as gchristian07:35
*** frankwang has quit IRC07:36
*** jawad_axd has joined #openstack-neutron07:36
*** ajay33 has joined #openstack-neutron07:37
*** trident has quit IRC07:40
*** priteau has joined #openstack-neutron07:43
*** ratailor_ has joined #openstack-neutron07:43
*** trident has joined #openstack-neutron07:43
*** ivve has joined #openstack-neutron07:43
openstackgerritMaciej Józefczyk proposed openstack/networking-ovn master: Add Design documentation for L3 HA Rescheduling  https://review.opendev.org/65371807:45
*** janki has quit IRC07:46
*** ratailor has quit IRC07:46
*** jangutter has joined #openstack-neutron07:52
*** janki has joined #openstack-neutron07:52
openstackgerritMaciej Józefczyk proposed openstack/networking-ovn master: Bump coverage job to 77% and don't skip Octavia driver  https://review.opendev.org/69000208:07
*** jlibosva has joined #openstack-neutron08:18
*** tesseract has joined #openstack-neutron08:21
*** tesseract has quit IRC08:21
*** frankwang has joined #openstack-neutron08:24
*** frank_wang has quit IRC08:27
*** jawad_axd has quit IRC08:28
*** andyzon has joined #openstack-neutron08:29
ralonsohHi folks. py27 FT tests is giving some problems since this weekend08:32
ralonsohplease check https://review.opendev.org/#/c/689658/ and https://review.opendev.org/#/c/689957/08:32
*** sapd1 has quit IRC08:33
*** lucasagomes has joined #openstack-neutron08:37
*** papasuder has quit IRC08:43
*** tssurya has joined #openstack-neutron08:53
*** dtantsur|afk is now known as dtantsur08:56
*** papasuder has joined #openstack-neutron09:01
openstackgerritkangyufei proposed openstack/python-neutronclient master: Switch to Ussuri jobs  https://review.opendev.org/69001409:05
*** davidsha has joined #openstack-neutron09:17
*** TomStappaerts has quit IRC09:17
*** weifan has joined #openstack-neutron09:24
*** frank_wang has joined #openstack-neutron09:33
*** weifan has quit IRC09:36
*** frankwang has quit IRC09:37
*** papasuder has quit IRC09:42
*** nanzha has quit IRC09:51
*** nanzha has joined #openstack-neutron09:53
*** frankwang has joined #openstack-neutron10:07
*** janki has quit IRC10:07
*** frank_wang has quit IRC10:09
openstackgerritLucas Alvares Gomes proposed openstack/networking-ovn master: Add support for virtual port type  https://review.opendev.org/67622310:10
*** ociuhandu has joined #openstack-neutron10:10
ivvehello, i have a wierd problem with neutron 14.0.2 and wondering if anyone has any input on how i should troubleshoot this issue. whenever i migrate a machine from one host to another L2 between the instance and DHCP server is not established, once i restart the neutron openvswitch agent the vxlan mesh is bumped and it is reestablished.. the only neutron errors i keep getting from controller is "Exception during message handling: TooManyExternalNetworks:10:14
ivve More than one external network exists"10:14
ivveim running ovs, l3_ha and dvr10:16
ivveit is 100% consistent when moving instances between two hosts10:16
*** pcaruana has quit IRC10:16
*** papasuder has joined #openstack-neutron10:16
*** papasuder has left #openstack-neutron10:18
*** markvoelker has joined #openstack-neutron10:19
*** pcaruana has joined #openstack-neutron10:21
*** markvoelker has quit IRC10:24
slaweqivve: can You check if You didn't hit https://bugs.launchpad.net/neutron/+bug/1824571 ?10:33
openstackLaunchpad bug 1824571 in neutron "l3agent can't create router if there are multiple external networks" [Medium,Fix released] - Assigned to Miguel Lavalle (minsel)10:33
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: Temporary disable CI job neutron-functional-python27  https://review.opendev.org/68995710:36
*** tbachman has quit IRC10:44
*** do3meli has quit IRC11:04
*** frank_wang has joined #openstack-neutron11:05
*** frankwang has quit IRC11:08
*** yamamoto has joined #openstack-neutron11:13
*** yamamoto has quit IRC11:19
f0oWould this be the right place to talk about issues with os-vif ?11:19
*** tonyb has joined #openstack-neutron11:31
ivveslaweq: sorry was at lunch, im checking now11:36
openstackgerritGregoire Mahe proposed openstack/neutron master: Allow to parse keywords in dns labels  https://review.opendev.org/68634311:37
*** cloudkitten has joined #openstack-neutron11:37
cloudkittenis segments_db.py:get_networks_segments() the best way to query for a few segmentation_id ?11:38
*** do3meli has joined #openstack-neutron11:39
*** do3meli has quit IRC11:39
*** do3meli has joined #openstack-neutron11:40
*** ratailor__ has joined #openstack-neutron11:43
*** ratailor_ has quit IRC11:46
ivveslaweq: okay so im not sure how to answer you but here it goes: it do get the error in logs "Exception during message handling: TooManyExternalNetworks: More than one external network exists". HOWEVER. i have 14.0.2 installed on all controller and compute nodes...(!?)11:51
ivveso i do have multiple external networks in tenants11:51
ivveand i can reach the flips of each instance11:52
ivvedhcp works when the instance is deployed11:52
ivvethis only occurs once the instance is migrated11:52
ralonsohf0o, and #openstack-nova, because Nova is the project using this library11:53
*** ratailor__ has quit IRC11:53
ivveand if (after migration) i restart neutron-openvswitch-agent, the L2 mesh is updated and i can again communicate with dhcp11:53
ivveslaweq: so i guess my short answer is no11:54
ivveim on my way to submit a bug, just collecting information to paste11:54
ivvebasically just wanted to know if there is something known before i create it or if im doing something wrong11:55
*** beagles|afk is now known as beagles11:57
f0oralonsoh: ok good to know.. basically I'm running into an issue that only affects vlan-backed networks when using the patch introduced in https://bugs.launchpad.net/os-vif/+bug/1837252 (CVE-2019-15753 / OSSA-2019-004) Do you happen to know if this is neutron or nova's domain?11:58
openstackLaunchpad bug 1837252 in os-vif stein "[OSSA-2019-004] Ageing time of 0 disables linuxbridge MAC learning (CVE-2019-15753)" [High,Fix committed] - Assigned to sean mooney (sean-k-mooney)11:58
*** markvoelker has joined #openstack-neutron12:01
ralonsohf0o what's the problem you have?12:01
sean-k-mooneyf0o: the changes in the patch that adressed that issue are not effectected by the segmentaion type of the nuetorn network12:03
*** yamamoto has joined #openstack-neutron12:04
ralonsohthat's true12:04
sean-k-mooneye.g. that woudl effect vxlan or vlan networks equally12:04
f0oralonsoh: basically the ageing prevents instances from getting network connectivity when backed by vlan networks (vxlan works fine)12:04
ivveslaweq: also, that bug is fixed in the version i have deployed. 14.0.212:05
f0ofdb shows the instance MAC on the provider network instead of the tap interface, the bridge is thus not moving the packets into the tap interface (I got pcaps to show that)12:05
ralonsohf0o ?? In what situation? because as sean commented the segmentation type does not affect12:05
f0oflipping the ageing parameter back to 0 (disable mac learning) fixes it12:05
ralonsohthe change is https://review.opendev.org/#/c/672834/3/vif_plug_ovs/linux_net.py@13512:05
sean-k-mooneyf0o: i have a hard time beliving that give that we had aging enable in all previous releases12:05
f0oStein has it disabled but Train has it enabled12:05
f0oso Train is breaking for me12:06
sean-k-mooneyyes and every release before stien had again enabled12:06
sean-k-mooneythe change in stien was a copy paste error effectivly12:06
*** kevinz has joined #openstack-neutron12:06
f0othen I wonder why packets arent being forwarded to the tap interface when ageing is enabled12:06
f0obecause the mac seems to be learned from the wrong interface12:06
f0oand I cant seem to figure out why12:06
ralonsohf0o ok this is not a problem in the ageing config12:07
ralonsohbut a problem in the ARP table12:07
ralonsohand how it was populated12:07
*** maciejjozefczyk has quit IRC12:07
*** yamamoto has quit IRC12:07
sean-k-mooneyf0o: what interface was the mac learned on ?12:07
*** yamamoto has joined #openstack-neutron12:07
f0oseems reasonable - I only came across the ageing issue because I have a refence host in Stein that works fine and that stood out as the obvious one12:07
sean-k-mooneythe vlan interface?12:07
f0omac was learned on ens* instead of tap*12:08
sean-k-mooneyf0o: was the vm migrated??12:08
f0onope12:08
sean-k-mooneythat could happen if it was it should not happen if it a new vm12:08
sean-k-mooneye.g. it was spawned12:08
f0oI rebooted the host a few times even and the VMs are a few days old and rebooted fresh when the host comes up12:09
sean-k-mooneyhave you logged in to the vm via the console and tried to ping the dhcp server12:09
f0oyeah I cant acquire DHCP even. on tcpdump I see DHCP replies coming into the br* but not being forwarded to tap*12:09
*** spatel has joined #openstack-neutron12:09
sean-k-mooneythe way i would debug this is login via consol12:10
f0oif I set a static IP on the VM it works but with packetloss12:10
ralonsohare you using hybrid or direct ports?12:10
sean-k-mooneythen add the ip manuall then ping the dhcp agent12:10
sean-k-mooneythen start tracing the packet flow12:10
sean-k-mooneyralonsoh: its linux bridge12:10
sean-k-mooneyso that is not a thing12:11
f0oI should've mentioned before, I'm using linuxbridge and not OVS12:11
sean-k-mooneyin linux bridge world you have 1 linux bridge per neutron netwrok on the host and all taps are addded directly too it12:11
ivvei think i have the same issue you do. however it only occurs when i migrate a machine (im using OVS tho)12:11
sean-k-mooneyand then a vlan or tunnel port is added to the linux bridge12:11
ralonsohright12:11
f0osean-k-mooney: exactly, for whatever reason the MAC is learned from the vlan ens256.3002 interface instead of tap*12:12
sean-k-mooneyivve: when you migrate a vm it can have learned the mac on the upstream interface form its previous location12:12
ivveeven tho vxlan is up (vtep) mac isn't learned for the two dhcp servers and i can't reach them12:12
f0oso packets are not being forwarded correctly - However if I disable aging it just magically works. vxlan backed networks are not affected whatsoever12:12
sean-k-mooneybut when the vm starts it will send a rarp to advitise its new location12:12
f0oso I image it being some weird arp_announce magic that makes a loop12:13
sean-k-mooneyf0o: that should only happen if the vm was moved although12:13
f0ohrm12:13
ivvewhenever i restart neutron-openvswitch-agent on the host it becomes populated and things start working again12:13
ivve(dhcp that is)12:13
f0owhat would the solution be if it was migrated tho?12:13
sean-k-mooneyf0o: it would age out12:14
sean-k-mooneyand teh mac would be leand form the tap12:14
f0oso 5 minutes of network disruption?12:14
sean-k-mooneyno12:14
sean-k-mooneyit should see that the mac is not directly attached12:14
f0oso I guess my issue is that the MAC is not being learned from the tap then12:14
sean-k-mooneyas the tap will have the mac adress12:14
sean-k-mooneyand that should cause it to be removed form the ens... interface12:15
*** spatel has quit IRC12:15
sean-k-mooneyf0o: did you confirm it has the mac you expect?12:15
sean-k-mooneyivve: i think you problem is similar but unrelated12:15
sean-k-mooneyivve: are you useing dvr?12:15
*** yamamoto has quit IRC12:16
sean-k-mooneyivve: and if so are you using vlan networks?12:16
f0osean-k-mooney: the instance has fa:16:3e:0d:c0:42, the tap has fe:16:3e:0d:c0:42 (I did not override any base-mac settings)12:16
f0owould that be the issue?12:16
sean-k-mooneyno they are the same which is what we expect12:16
f0oon the other hand, the vxlan tap and interface also differs in the fa/fe initial12:16
sean-k-mooneyso the mac learning table shoudl know that the tap is the destiation for that mac12:17
*** tbachman has joined #openstack-neutron12:17
sean-k-mooneyoh sorry missed that12:17
ivvesean-k-mooney: using l3_ha dvr and OVS12:17
sean-k-mooneyivve: do you have vlan networks?12:17
ivveyes, many12:17
ivveand yes many in the same tenant12:17
sean-k-mooneyok did you also enable the l2_population driver12:17
ivve(external)12:17
slaweqivve: ok, please report new bug than12:17
openstackgerritGregoire Mahe proposed openstack/neutron master: Allow to parse keywords in dns labels  https://review.opendev.org/68634312:18
sean-k-mooneyivve: to user dvr with vlan on ovs you are requried to run the l2_population driver12:18
slaweqivve: sorry, I'm in training this week so I don't have too much time to work on it but I will try to check it12:18
sean-k-mooneymechdriver that is12:18
ivvehmm just askin but, if l2_pop was disabled, would i ever get anything to work?12:19
f0osean-k-mooney: when I grep `bridge fdb` for the MAC of a vlan-backed vm, I see: fa:16:3e:0d:c0:42 dev ens256.3002 master brqa50c5b7b-db12:19
sean-k-mooneyit shoudl work i belive12:19
sean-k-mooneyivve: ^12:19
f0osean-k-mooney: however, when I do the same for a vxlan backed vm, I see: fa:16:3e:83:2f:b5 dev tapb798e733-ad master brq3e90bc8f-0f12:19
sean-k-mooneybut there are some negitive sideffects12:19
ivve[ml2] \n type_drivers = flat,vlan,vxlan \n tenant_network_types = vxlan \n mechanism_drivers = openvswitch,l2population12:20
sean-k-mooneyivve: https://specs.openstack.org/openstack/neutron-specs/specs/kilo/neutron-ovs-dvr-vlan.html12:20
sean-k-mooneyivve: ok so its not that then12:20
ivveslaweq: no problem, as long as we don't migrate anything without forgetting to restart neutron-ovs-agent we should be fine :P12:21
ivveits just a hard bug to detect since you won't notice until lease expires12:22
*** maciejjozefczyk has joined #openstack-neutron12:22
sean-k-mooneyivve: hum the spec actuly does not mention l2 pop which is weired12:22
sean-k-mooneyf0o: ok so can you check 2 things for me12:23
sean-k-mooneyf0o: 1 what is the mac on the neutron port12:23
sean-k-mooneyf0o: 2 what is the mack in the libvirt xml12:23
ivveim gonna check a few things in ovs12:23
ivveim quite sure thats where the problem lies12:23
f0osean-k-mooney: Neutron: fa:16:3e:0d:c0:42 / libvirt: fa:16:3e:0d:c0:42 (libvirt interface output: http://paste.openstack.org/show/785468/)12:25
sean-k-mooneyf0o: ok so neutron and libvirt agree and in the vm? you see the fa:16:3e:0d:c0:42 adress too?12:26
f0oyep inside the VM I have the same MAC12:26
sean-k-mooneyso its only the mac that has teh fe address?12:26
ralonsohsean-k-mooney, I think when seg type is VLAN, the code is https://github.com/openstack/os-vif/blob/master/vif_plug_linux_bridge/linux_bridge.py#L103-L10612:27
sean-k-mooneyis there another interface on the host that has the fa address12:27
f0othe tap has the fe: address but inside the VM I have fa:12:27
ralonsohsean-k-mooney, but "should_provide_vlan" is something from NOva networks12:27
ivveoh and also, the config has: [agent]\n tunnel_types = vxlan \n l2_population = true \n arp_responder = true \n enable_distributed_routing = True12:27
f0oand bridge fdb maps the fa: to ens256.3002 instead of tapb2b8c5ff-8c12:27
sean-k-mooney ralonsoh right but we should not be modifing the mac in os-vif12:28
f0oI can see on the vxlan backed vm that the fa: is mapped to the tap* interface, so I assume this should be the same for the vlan backed vm12:28
ralonsohsean-k-mooney, yes, this is another probleem12:28
ralonsohI don't know where is this happening12:28
sean-k-mooneyralonsoh: by should not i ment i dont think we are12:28
sean-k-mooneye.g. i think something else modified it12:28
ivvehmm12:28
ivveis there a certain magic order neutron/ovs needs to be started?12:29
ivvei might have found it if it exists lol12:29
*** yamamoto has joined #openstack-neutron12:29
ivvelike neutron components overall.. vs ovs components12:29
sean-k-mooneyovs should be started first12:29
*** nanzha has quit IRC12:30
openstackgerritLucas Alvares Gomes proposed openstack/networking-ovn master: Add support for virtual port type  https://review.opendev.org/67622312:30
sean-k-mooneyalthough the neutorn agent will try to recoonect if it is started before ovs12:30
f0osean-k-mooney: the moment I send traffic over from the vm (such as DHCP requests), bridge fdb relearns the MAC but still puts it on the ens* interface and not on the tap* interface12:30
ivvei just restarted ovs-agent on controller nodes and after a migration one dhcp server is now reachable while other is not12:30
ivve(dhcp agent was recently restarted)12:30
ralonsohf0o who has the fe* address?12:31
sean-k-mooneyf0o: have you run tshark or tcpdump on the tap to verify it has teh fa source mac12:31
sean-k-mooneyralonsoh: the tap device12:31
f0oralonsoh: the tap device on the host side12:31
f0osean-k-mooney: do you mean from inside an instance or from the hypervisor?12:31
sean-k-mooneyspecifcly tapb2b8c5ff-8c12:31
sean-k-mooneyf0o: tshark/tcpdump on the hypervior with a packet sent form in the vm12:32
f0o12:32:28.969742 fa:16:3e:0d:c0:42 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:0d:c0:42, length 30012:32
f0o# tcpdump -neli tapb2b8c5ff-8c udp12:32
sean-k-mooneyok so its sent with the correct mac from insed the vm12:32
f0oif I tcpdump the br* I will get the reply12:33
sean-k-mooneyok so the anti spoof firewall rules are not droping it12:33
sean-k-mooneyactully they might be12:33
sean-k-mooneydo you see the reply on the tap12:33
sean-k-mooneyor just the bridge12:33
f0ojust the bridge12:33
f0oneutron-linuxbri-ib2b8c5ff-8 is the incomming chain that handles traffic for that VM afaik. I added a -j ACCEPT at the top of it to rule it out12:34
*** nanzha has joined #openstack-neutron12:34
sean-k-mooneyf0o: can you check the mac address on the vlan interface for me12:36
sean-k-mooneyis it the fa address by any chance?12:36
f0oens256.3002: fa:16:3e:ba:fa:33 / brqa50c5b7b-db: fa:16:3e:ba:fa:33 / tapb2b8c5ff-8c: fe:16:3e:0d:c0:4212:37
sean-k-mooneyok that is fine12:38
sean-k-mooneywell the tap is wrong12:38
sean-k-mooneybut the vlan and bridge have the correct mac12:38
f0oall tap* have fe instead of fa though12:38
sean-k-mooneyi was just checking that https://github.com/openstack/os-vif/blob/master/vif_plug_linux_bridge/linux_net.py#L72-L85 worked corerctly12:38
ralonsohos-vif is not defining the TAP mac12:39
sean-k-mooneyralonsoh: yep just finished going through the os-vif code to confirm that too12:40
sean-k-mooneyralonsoh: and libvirt has teh correct mac set and the vm sees the correct mac12:40
ralonsohsean-k-mooney, the point is the vlan interface should route the traffic into the tap device12:40
ralonsohthe vlan interface has the correct mac12:41
ralonsohand will filter the vlan traffic12:41
sean-k-mooneyralonsoh: yes if the mac was learned form the correct interface12:41
sean-k-mooneyf0o: can you provide your arp table for that bridge?12:41
f0obridge fdb? or ip neigh?12:41
sean-k-mooneyno i mean the output of arp command12:42
ralonsohbrctl showmacs <bridge>12:42
sean-k-mooneyi want to confirm that there is not a static arp entry12:42
sean-k-mooney"brctl showmacs <bridge>" would also be useful12:42
f0ohttp://paste.openstack.org/show/785470/12:43
f0oboth outputs12:43
f0ohttp://paste.openstack.org/show/785472/ brctl showstp12:43
f0oto be able to identify port 1/2 ;)12:43
sean-k-mooneyf0o: is this just happening on this 1 vm by the way or is it happening on others too?12:45
sean-k-mooneyyou dont have any staic arp entries which is good.12:46
sean-k-mooneyit looks like you have a mac adress conflict12:46
f0oit happens on any vm with vlan backed network on any hypervisor running Train (or well, the fix that re-enabled ageing)12:46
sean-k-mooneyok12:46
sean-k-mooneyi think the root cause is that the tap has the wrong mac12:46
*** dtantsur is now known as dtantsur|brb12:47
ralonsohagree12:47
sean-k-mooneyif it had the correct mac the "is local"12:47
ralonsoh(but I don't know how this mac ended there)12:47
f0osean-k-mooney: but how comes it works with vxlan backed networks?12:47
sean-k-mooneyfield would be ture and it woudl take prescendes12:47
sean-k-mooneyim guessing there is a bug i the linux bridge agent12:47
sean-k-mooneylibvirt has no awareness of the segmentaiton type12:48
f0oI just want to confirm that this line is correct: fa:16:3e:0d:c0:42 dev ens256.3002 master brqa50c5b7b-db12:48
sean-k-mooneynor does os-vif12:48
sean-k-mooneyno it is not12:48
f0obecause on all vms that arent vlan-backed but vxlan-backed it shows `dev tap*`12:48
f0oeventho the tap does not have fa: but also uses fe:12:48
sean-k-mooneyit should be on the tap12:48
sean-k-mooneyoh12:48
ralonsoheven in vxlan??12:49
f0oyep12:49
ralonsohok12:49
sean-k-mooneythe fe adress is on the tap for vxlan12:49
f0oI can paste an example sec12:49
sean-k-mooneyok i guess that might be a redherring then12:49
ralonsoh??12:49
sean-k-mooneyralonsoh: the fact that the tap has the fe address might not be the issue.12:50
*** spatel has joined #openstack-neutron12:50
sean-k-mooneyi still think its strange and would liek to know why that is the case but if its the same for vxlan it implies that something esle is happening12:51
f0osean-k-mooney: ralonsoh : http://paste.openstack.org/show/785473/12:51
ralonsohthe weird thing is this transformation from fa:**** to fe:*****12:51
f0othe obvious difference is that vlan backed shows on the provider interface while vxlan shows on the tap interface12:52
ralonsohnonono12:52
f0omaybe provider is the wrong word ehre12:52
ralonsohthis is ok12:52
*** frankwang has joined #openstack-neutron12:52
ralonsohthe vlan interfaces and the tap devices have the same mac12:52
ralonsohfe:16:3e:83:2f:b5 dev tapb798e733-ad master brq3e90bc8f-0f permanent12:52
ralonsohfe:16:3e:83:2f:b5 dev tapb798e733-ad vlan 1 master brq3e90bc8f-0f permanent12:52
ralonsohthis is correct12:52
f0oI was more referring to the first two lines12:53
*** nanzha has quit IRC12:53
f0ofa:16:3e:0d:c0:42 dev ens256.3002 master brqa50c5b7b-db - vs - fa:16:3e:83:2f:b5 dev tapb798e733-ad master brq3e90bc8f-0f12:53
f0oshouldnt the first also be pointing to a tap*?12:53
f0ospecifiaclly tapb2b8c5ff-8c12:53
*** frank_wang has quit IRC12:54
ralonsohit should, we don;t create named interfaces12:55
ralonsoh*create those names interfaces12:55
*** spatel has quit IRC12:55
sean-k-mooneyos-vif does12:55
f0oif I run `bridge fdb replace fa:16:3e:0d:c0:42 dev tapb2b8c5ff-8c master` it doesnt change anything sadly, the bridge-fdb looks correct with having tap* as the target for the mac but the VM still wont get the traffic12:55
sean-k-mooneyand neutron dose create the ens256.3002 interface12:55
ralonsohsean-k-mooney, not with this naming convention12:56
ralonsohsean-k-mooney, ok yes, we do L179 LB agent12:57
sean-k-mooneyralonsoh: os vif does https://github.com/openstack/os-vif/blob/master/vif_plug_linux_bridge/linux_net.py#L72-L7612:58
ralonsohand the LB agent with "name.vlan"12:58
sean-k-mooneyya12:58
ralonsohsean-k-mooney, ok ok12:59
sean-k-mooneyin both cases it will end up with ens256.3002 i think12:59
ralonsohand LB is creating this by default12:59
ralonsohbut we are not providing the MAC address there12:59
sean-k-mooneywe do it on both sides to ensure it is created12:59
ralonsohwhen creating the vlan interface12:59
sean-k-mooneyyes although we can set it on line 79 if we have one13:00
ralonsohsean-k-mooney, the point is when we are creating a vlan interface13:00
ralonsohif the MAC is not defined13:00
f0oneutron/tests/unit/objects/test_common_types.py and neutron_lib/tests/unit/objects/test_common_types.py seem to derive macs starting with fe:16:3e...13:01
ralonsohthe vlan interface will inherit the mac of the upper interface13:01
ralonsohright now13:01
ralonsoh6391: d2: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 100013:01
ralonsoh    link/ether fe:bd:48:40:96:26 brd ff:ff:ff:ff:ff:ff13:01
ralonsoh6393: vlan_d2@d2: <BROADCAST,NOARP,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 100013:01
ralonsoh    link/ether fe:bd:48:40:96:26 brd ff:ff:ff:ff:ff:ff13:01
sean-k-mooneyralonsoh: yes and the only place we invoke ensuer_vlan_bridge https://github.com/openstack/os-vif/blob/734d3bfa5d1324537f50cc95221703d4c19a734f/vif_plug_linux_bridge/linux_bridge.py#L10513:01
sean-k-mooneywe dont pass a mac13:01
*** nanzha has joined #openstack-neutron13:02
ralonsohand that's ok13:02
sean-k-mooneyyep13:02
ralonsohbecause it will create an interface inheriting the mac13:02
sean-k-mooneyit means thats dead code we can safely delete13:02
f0oralonsoh: the vlan interface has the underlying interface's mac13:02
sean-k-mooneye.g. its only there form nova networks13:02
ralonsohwhat's the brqa50c5b7b-db bridge mac??13:02
*** yamamoto has quit IRC13:02
*** nweinber__ has joined #openstack-neutron13:02
sean-k-mooneyf0o: yep that is what we expect13:02
sean-k-mooneyralonsoh: the bridge mac shoudl be the same as teh vlan mac13:03
ralonsohI know13:03
f0obrqa50c5b7b-db has the same mac as the vlan interface13:03
*** spatel has joined #openstack-neutron13:03
ralonsohok, those FDB rules are created wrongly by Neutron13:05
*** yamamoto has joined #openstack-neutron13:05
sean-k-mooneyralonsoh: did you fined the point in the code?13:06
sean-k-mooneyi was looking at https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py#L776-L78813:06
ralonsohsean-k-mooney, just a guess13:06
ralonsohbut the LB agent should be my initial searching point13:06
ralonsohyes, exactly13:06
f0oralonsoh: how should the fdb rules look like?13:08
ralonsohf0o, I'm not 100% sure13:08
ralonsohjust checking13:08
f0obecause I just replaced the one pointing to ens256.3002 to point to tapb2b8c5ff-8c instead and it didnt change anything13:08
f0ois there any sysctl setting that might be in the way? I dont have so much trust in Ubuntu anymore to be honest13:09
f0ohttp://paste.openstack.org/show/785474/ sysctl of net.ipv4.conf.$interfaces13:11
*** jlibosva has quit IRC13:12
openstackgerritMerged openstack/networking-ovn stable/rocky: Avoid port group creation race  https://review.opendev.org/68756813:13
*** jlibosva has joined #openstack-neutron13:13
ralonsohsean-k-mooney, https://www.redhat.com/archives/libvir-list/2012-June/msg01330.html13:16
ralonsoh"vnet0 is the backend of the guest NIC, and its MAC addr13:16
ralonsohis more or less irrelevant to functioning of the guest13:16
ralonsohitself, since traffic does not originate on this NIC.13:16
ralonsohThe only important thing is that this TAP device must13:16
ralonsohhave a high value MAC address, to avoid the bridge13:16
ralonsohdevice using the TAP device's MAC as its own. Hence13:16
ralonsohwhen creating the TAP Device  libvirt takes the guest13:16
ralonsohMAC addr and simply sets the top byte to 0xFE"13:16
ralonsohhttps://openstack.nimeyo.com/48447/openstack-dev-how-does-instances-tap-device-address-generate13:16
sean-k-mooneyright i was reading https://www.iana.org/assignments/ethernet-numbers/ethernet-numbers.xhtml13:17
sean-k-mooneyFE-00-00-00-00 to FE-FF-FF-FF-FF  IPv4 Addr Holders13:17
openstackgerritLucas Alvares Gomes proposed openstack/networking-ovn master: Add support for virtual port type  https://review.opendev.org/67622313:18
sean-k-mooneyso the 0xFE prefix is reserved for macs with ipv4 connectivity13:18
sean-k-mooney10-00-00-01-00 to FC-FF-FF-FF-FF is Unassigned13:18
sean-k-mooneyand we default to values in teh FA range in neutron13:18
*** dave-mccowan has joined #openstack-neutron13:25
*** sridharg has quit IRC13:29
*** dave-mccowan has quit IRC13:29
*** ajay33 has quit IRC13:38
*** jmlowe has quit IRC13:46
*** kevinz has quit IRC13:47
*** ociuhandu has quit IRC13:48
*** kevinz has joined #openstack-neutron13:48
sean-k-mooneyralonsoh: are you happy that f0o  issue is on the neutron side by the way if so ill leave it in neutrons capable hands13:49
ralonsohsean-k-mooney, but this FE* mac is being set by libvirt13:49
ralonsohwe are not creating/modifying this mac13:49
sean-k-mooneyyes13:49
sean-k-mooneyalthough since that is done for all vms13:50
*** trident has quit IRC13:50
sean-k-mooneythat is not the issue13:50
*** ociuhandu has joined #openstack-neutron13:50
sean-k-mooneythe question is what is different in the l2 agent for vxlan and vlan13:50
sean-k-mooneythat results in the different behavior13:50
sean-k-mooneygiven that libvirt, nova and os-vif do the same thing in both cases13:51
*** mlavalle has joined #openstack-neutron13:51
f0othis turned out to be quite a long thread all of a sudden :|13:52
*** trident has joined #openstack-neutron13:54
*** do3meli has quit IRC13:54
sean-k-mooneyf0o: well it appears to be a ligitmate issue in your env and we are not seeing it in the gate so its non triveial to resolve13:54
sean-k-mooneythe gate supostion is based on the fact the jobs dont explode13:55
*** ociuhandu has quit IRC13:58
ivveim getting nowhere with this issue. its so random and strange13:59
ivvehowever13:59
*** jmlowe has joined #openstack-neutron13:59
ivvesomething that bother me is that neutron 14.0.2 is still getting "too many external networks" error14:00
f0osean-k-mooney: I understand and agree. I'm unsure if neutron/nova is even to blame or perhaps it's just an ubuntu kernel oddity as well... Because in the end it's the kernel who forwards the packets within the bridge and the packets do arrive and outbound packets are being forwarded. So something must be off on that regard...14:01
ivvecould it be that l3_ha + dvr + ovs is still affected by https://bugs.launchpad.net/neutron/+bug/1824571 ?14:02
openstackLaunchpad bug 1824571 in neutron "l3agent can't create router if there are multiple external networks" [Medium,Fix released] - Assigned to Miguel Lavalle (minsel)14:02
ivveand somehow causes my problem with dhcp and migrations14:02
*** nweinber_ has joined #openstack-neutron14:04
*** cshen has quit IRC14:05
*** kevinz has quit IRC14:05
*** nweinber__ has quit IRC14:07
*** yamamoto has quit IRC14:07
*** do3meli has joined #openstack-neutron14:15
*** ociuhandu has joined #openstack-neutron14:17
*** andyzon has quit IRC14:19
*** andyzon has joined #openstack-neutron14:20
*** igordc has joined #openstack-neutron14:23
*** andyzon has quit IRC14:25
*** gibi is now known as gibi_off14:29
openstackgerritBrian Haley proposed openstack/neutron master: Update bug contacts  https://review.opendev.org/69009414:32
*** andyzon has joined #openstack-neutron14:33
*** slaweq has quit IRC14:33
*** maciejjozefczyk has quit IRC14:34
*** maciejjozefczyk has joined #openstack-neutron14:38
*** yamamoto has joined #openstack-neutron14:42
*** dtantsur|brb is now known as dtantsur14:42
*** dswebb has joined #openstack-neutron14:46
*** yamamoto has quit IRC14:46
ivvesean-k-mooney: do you have experience with larger environments (3control and 50+ computes) and ovs?14:47
ivvei think the issue i have i related with many requests, even tho i have the  error (multiple external networks) if i migrate 1 instance at a time it works fine. 10 at a time. not so much14:48
*** gcheresh_ has quit IRC14:49
sean-k-mooneythe largest clust i have personally run is about 20 nodes but 50 shoudl not be hitting scaling issues14:49
ivvethats my thought as well14:49
sean-k-mooneyi know people do hit issue when you get into the mid hundereds due to amqp bottelnecks14:50
ivvewell restarting l3 agent on the controllernodes do generate errors and it can take a very long time to get them fully up and running14:50
sean-k-mooneyalthough i also belive the neutron team was working on improvment there14:50
ivvebut the error with multiple external networks is worrying because i can only see 1 fip namespace on the computes now14:50
ivveinstead of 1 per flip14:51
sean-k-mooneyim not actully familar with that error but its possibel it still exists. i mainly work on nova these days so i dont follow neutorn too closely14:51
ivveah alright14:51
ivvethat error was the reason i stayed in  rocky for quite some time14:54
ivvebut tested it in a smaller env and didn't have that issue14:54
ivvenow i moved it all up and things start behaving this way14:54
*** lajoskatona has quit IRC14:55
*** pcaruana has quit IRC14:58
*** rkukura has joined #openstack-neutron14:58
*** jlibosva has quit IRC14:58
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: [WIP] [OVS] Handle added/removed ports in the same polling iteration  https://review.opendev.org/69009814:59
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: Reset timeout exception in DietTestCase when retrying  https://review.opendev.org/68965814:59
dswebbsean-k-mooney, this slide by the cern admins says much the same. https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619c416ff4bd19e1c087f27a43eea/www-assets-prod/presentation-media/Evolution-of-OpenStack-Networking-at-CERN3.pdf15:00
dswebbthough they had a pretty crazy neutron setup with lots of flat networks15:00
sean-k-mooney ya so ovns decaritvie distubted state model rathar then imperitive "hay this chagne go do somthing" apparch that we have in ml2/ovs  sacalse much better15:02
*** ratailor has joined #openstack-neutron15:03
*** ratailor has quit IRC15:03
*** hjensas has quit IRC15:04
sean-k-mooneyalthough some of there slides are questionable15:04
sean-k-mooneymain line the refence to dpdk when comapreing ovn/odl and ml2 ovs.15:04
sean-k-mooneyml2/ovs has the best support for dpdk out of all 315:05
sean-k-mooneyalthough odl and ovn have better routing support when usign dpdk15:05
sean-k-mooneythey have worse vm support15:06
*** slaweq has joined #openstack-neutron15:06
*** armax has joined #openstack-neutron15:06
*** mmethot has quit IRC15:09
*** mmethot has joined #openstack-neutron15:09
slaweqhaleyb: about neutron-lib, I wanted to raise this on PTG and ask if there is any new volunteer to take care of it15:10
slaweqhaleyb: but if You would update this doc now, You can simply put my name there15:11
slaweqhaleyb: actually I think I will need to do more updates to this doc now15:11
slaweqso I can take care of it if You want15:11
*** ivve has quit IRC15:13
haleybslaweq: i sent out a review, and had put "Neutron PTL" for now, https://review.opendev.org/69009415:13
slaweqhaleyb: thx15:13
*** otsukahy has joined #openstack-neutron15:13
*** otsukahy has quit IRC15:13
*** trident has quit IRC15:13
*** maciejjozefczyk has quit IRC15:13
*** otsukahy has joined #openstack-neutron15:14
ralonsohdavidsha, hmmm next meeting is during the PTG15:21
*** dswebb has quit IRC15:21
ralonsohwe'll see if we need to cancel it or we can reschedule15:21
davidsharalonsoh, ack, are you attending?15:21
ralonsohnope15:21
davidshasame, it's up to you.15:22
davidshaIs there going to be some kind of stream for it actually?15:23
*** trident has joined #openstack-neutron15:26
*** mattw4 has joined #openstack-neutron15:26
ralonsohsean-k-mooney, I'll do my best with the LB problem15:27
*** do3meli has quit IRC15:27
*** slaweq has quit IRC15:28
*** rubasov has quit IRC15:32
*** hjensas has joined #openstack-neutron15:38
*** ksambor has quit IRC15:38
*** slaweq has joined #openstack-neutron15:40
*** cloudkitten has quit IRC15:41
*** otsukahy has quit IRC15:43
*** otsukahy has joined #openstack-neutron15:44
*** slaweq has quit IRC15:45
*** otsukahy has quit IRC15:48
*** andyzon has quit IRC15:50
*** ircuser-1 has joined #openstack-neutron15:54
*** rubasov has joined #openstack-neutron15:57
*** lucasagomes has quit IRC15:57
*** otsukahy has joined #openstack-neutron16:03
*** otsukahy has quit IRC16:03
*** jmlowe has quit IRC16:06
*** otsukahy has joined #openstack-neutron16:09
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: Temporary disable CI job neutron-functional-python27  https://review.opendev.org/68995716:11
*** macz has joined #openstack-neutron16:12
*** rpittau is now known as rpittau|afk16:17
*** igordc has quit IRC16:20
f0osean-k-mooney ralonsoh - So about my issue, should I open an issue on launchpad or is there anything I should be doing?16:21
ralonsohyes please, open a bug in launchpad16:21
f0owill do :)16:22
*** ccamposr has quit IRC16:22
sean-k-mooneyf0o: yes we sould track it as a sperate bug rather then reuse the cve16:25
*** hjensas has quit IRC16:31
*** nanzha has quit IRC16:38
*** pcaruana has joined #openstack-neutron16:41
*** davidsha has quit IRC16:43
*** pcaruana has quit IRC16:52
*** markvoelker has quit IRC16:54
*** bobmel has joined #openstack-neutron17:02
*** dtantsur is now known as dtantsur|afk17:02
*** armax has quit IRC17:05
*** bobmel has quit IRC17:05
*** markvoelker has joined #openstack-neutron17:12
*** spatel has quit IRC17:15
*** tbachman has quit IRC17:22
*** priteau has quit IRC17:22
*** bobmel has joined #openstack-neutron17:24
*** otsukahy has quit IRC17:24
*** weifan has joined #openstack-neutron17:27
*** tbachman has joined #openstack-neutron17:28
*** ozzzo has quit IRC17:32
*** tssurya has quit IRC17:32
*** ociuhandu_ has joined #openstack-neutron17:32
*** mmethot_ has joined #openstack-neutron17:35
*** mmethot has quit IRC17:35
*** ociuhandu has quit IRC17:36
*** bobmel has quit IRC17:38
*** ociuhandu_ has quit IRC17:39
openstackgerritMerged openstack/networking-bagpipe stable/train: assorted fixes in networking_bagpipe.bagpipe_bgp module  https://review.opendev.org/68881117:42
*** ociuhandu has joined #openstack-neutron17:43
*** andyzon has joined #openstack-neutron17:46
*** ociuhandu has quit IRC17:47
*** armax has joined #openstack-neutron17:49
*** mhen has quit IRC17:51
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: [WIP] Use threads insted of greethreads in IP monitor  https://review.opendev.org/69014417:54
ralonsohdo we have another problem in the CI?17:55
ralonsohhttps://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d78/689658/3/check/neutron-tempest-iptables_hybrid-fedora/d78280a/job-output.txt17:55
ralonsoh2019-10-22 15:19:50.319322 | controller | Collecting tempest===22.1.0 (from -c /opt/stack/requirements/upper-constraints.txt (line 622))17:55
ralonsoh2019-10-22 15:19:50.340347 | controller |   Could not find a version that satisfies the requirement tempest===22.1.0 (from -c /opt/stack/requirements/upper-constraints.txt (line 622)) (from versions: 10.0.0, 11.0.0, 12.0.0, 12.1.0, 12.2.0, 13.0.0, 14.0.0, 15.0.0, 16.0.0, 16.1.0, 17.0.0, 17.1.0, 17.2.0, 18.0.0, 19.0.0, 20.0.0, 21.0.0, 22.0.0)17:55
ralonsoh2019-10-22 15:19:50.427785 | controller | No matching distribution found for tempest===22.1.0 (from -c /opt/stack/requirements/upper-constraints.txt (line 622))17:55
*** otsukahy has joined #openstack-neutron18:05
*** tbachman has quit IRC18:07
*** otsukahy has quit IRC18:09
*** njohnston has quit IRC18:10
*** njohnston has joined #openstack-neutron18:11
openstackgerritBrian Haley proposed openstack/neutron master: Update bug contacts  https://review.opendev.org/69009418:13
*** igordc has joined #openstack-neutron18:14
*** ivve has joined #openstack-neutron18:15
jrosserralonsoh: I asked about that tempest package issue in #infra earlier and it seems that the pypi cdn may be inconsistent18:16
ralonsohansssss18:16
ralonsohjrosser, thanks!!18:16
*** tbachman has joined #openstack-neutron18:18
*** weifan has quit IRC18:22
*** ralonsoh has quit IRC18:22
*** CeeMac has joined #openstack-neutron18:26
*** cshen has joined #openstack-neutron18:26
*** spsurya has quit IRC18:30
*** weifan has joined #openstack-neutron18:31
*** weifan has quit IRC18:39
*** cshen has quit IRC18:42
*** igordc has quit IRC18:47
*** igordc has joined #openstack-neutron18:56
*** weifan has joined #openstack-neutron18:59
*** andyzon has quit IRC19:00
*** weifan has quit IRC19:00
*** weifan has joined #openstack-neutron19:00
*** tbachman has quit IRC19:03
*** pcaruana has joined #openstack-neutron19:04
*** weifan has quit IRC19:06
*** jmlowe has joined #openstack-neutron19:09
*** pcaruana has quit IRC19:19
*** slaweq has joined #openstack-neutron19:20
*** yamamoto has joined #openstack-neutron19:26
*** henriqueof has joined #openstack-neutron19:27
henriqueofI've been using UniFi Network Controller as an instance on OpenStack for a while, after updating from Rocky to Stein my users can't get redirected to captive portal anymore, is there any significant change that can cause it?19:27
*** yamamoto has quit IRC19:31
*** weifan has joined #openstack-neutron19:31
*** otsukahy has joined #openstack-neutron19:32
openstackgerritBrian Haley proposed openstack/neutron master: Clean-up ssl packages from bindep.txt  https://review.opendev.org/69038819:32
*** hjensas has joined #openstack-neutron19:36
*** weifan has quit IRC19:38
*** maciejjozefczyk has joined #openstack-neutron19:40
*** weifan has joined #openstack-neutron19:45
*** slaweq has quit IRC19:46
*** slaweq has joined #openstack-neutron19:50
*** do3meli has joined #openstack-neutron19:54
*** do3meli has quit IRC19:56
*** nweinber_ has quit IRC20:00
openstackgerritThomas Morin proposed openstack/networking-bagpipe stable/train: bagpipe-bgp: cleanly ignore RTC route of unsupported type  https://review.opendev.org/69039520:04
openstackgerritThomas Morin proposed openstack/networking-bagpipe stable/stein: bagpipe-bgp: cleanly ignore RTC route of unsupported type  https://review.opendev.org/69039620:04
*** maciejjozefczyk has quit IRC20:04
*** otsukahy has quit IRC20:05
*** otsukahy has joined #openstack-neutron20:05
*** weifan has quit IRC20:07
*** weifan has joined #openstack-neutron20:09
*** weifan has quit IRC20:22
*** tbachman has joined #openstack-neutron20:24
*** weifan has joined #openstack-neutron20:24
*** weifan has quit IRC20:29
*** weifan has joined #openstack-neutron20:30
*** aedc has quit IRC20:32
*** do3meli has joined #openstack-neutron20:34
*** weifan has quit IRC20:34
*** weifan has joined #openstack-neutron20:41
*** otsukahy has quit IRC20:46
*** weifan has quit IRC20:46
*** weifan has joined #openstack-neutron20:49
*** cgoncalves has quit IRC20:49
*** colby_ has quit IRC20:50
*** weifan has quit IRC20:53
openstackgerritBrian Haley proposed openstack/neutron master: Conclude adoption of new enginefacade in Neutron  https://review.opendev.org/54550120:55
*** weifan has joined #openstack-neutron20:55
*** cgoncalves has joined #openstack-neutron20:55
*** aedc has joined #openstack-neutron20:56
*** aedc has quit IRC21:01
*** weifan has quit IRC21:01
*** aedc has joined #openstack-neutron21:03
*** weifan has joined #openstack-neutron21:03
*** weifan has quit IRC21:05
*** weifan has joined #openstack-neutron21:05
*** slaweq has quit IRC21:08
*** weifan has quit IRC21:10
*** weifan has joined #openstack-neutron21:13
*** weifan has quit IRC21:18
*** slaweq has joined #openstack-neutron21:19
*** weifan has joined #openstack-neutron21:23
*** slaweq has quit IRC21:23
NobodyCamGood Afternoon Neutron folks, I had a off the wall question: is it possible to change the provider for a network with deployed resources.21:26
*** mattw4 has quit IRC21:40
*** mattw4 has joined #openstack-neutron21:41
*** otsukahy has joined #openstack-neutron21:44
*** armax has quit IRC21:56
*** otsukahy has quit IRC22:01
*** otsukahy has joined #openstack-neutron22:04
*** otsukahy has quit IRC22:07
*** otsukahy has joined #openstack-neutron22:07
*** otsukahy has quit IRC22:14
*** otsukahy has joined #openstack-neutron22:15
*** weifan has quit IRC22:25
*** weifan has joined #openstack-neutron22:28
*** otsukahy has quit IRC22:29
*** weifan has quit IRC22:34
*** weifan has joined #openstack-neutron22:35
*** otsukahy has joined #openstack-neutron22:40
*** otsukahy has quit IRC22:44
*** ivve has quit IRC22:45
*** ociuhandu has joined #openstack-neutron22:46
*** ociuhandu has quit IRC22:51
*** weifan has quit IRC22:53
openstackgerritMerged openstack/neutron-specs master: Bump the openstackdocstheme extension to 1.20  https://review.opendev.org/68823322:53
*** tkajinam has joined #openstack-neutron22:53
*** weifan has joined #openstack-neutron22:54
*** armax has joined #openstack-neutron22:55
*** weifan has quit IRC23:05
*** weifan has joined #openstack-neutron23:18
*** weifan has quit IRC23:25
*** yamamoto has joined #openstack-neutron23:28
*** weifan has joined #openstack-neutron23:30
*** weifan has quit IRC23:34
*** weifan has joined #openstack-neutron23:35
*** weifan has quit IRC23:42
*** macz has quit IRC23:43
*** weifan has joined #openstack-neutron23:44
*** weifan has quit IRC23:48
*** weifan has joined #openstack-neutron23:51
*** weifan has quit IRC23:54
*** igordc has quit IRC23:55
*** mattw4 has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!