| opendevreview | OpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata https://review.opendev.org/c/openstack/openstack-ansible/+/954681 | 03:06 |
|---|---|---|
| opendevreview | Jonathan Rosser proposed openstack/ansible-role-pki master: Generate ca_bundle during cert creation for standalone backend https://review.opendev.org/c/openstack/ansible-role-pki/+/954628 | 06:02 |
| opendevreview | Jonathan Rosser proposed openstack/ansible-role-pki master: Allow certificates to be installed by specifying them by name https://review.opendev.org/c/openstack/ansible-role-pki/+/954239 | 06:02 |
| opendevreview | Merged openstack/openstack-ansible master: Imported Translations from Zanata https://review.opendev.org/c/openstack/openstack-ansible/+/954681 | 07:29 |
| f0o | Hi, I'm poking around stabilizing IPv6 for tenant networks and I'm struggling wrapping my head around how to distribute the tenant networks onto the Top-Of-Rack switches. I figured as much that I need to use OVN-BGP for this but I'm a bit confused what is going to run the BGPD. Is it going to be only the hypervisors? only the gateway-hosts? Both? Also what's the state of | 08:45 |
| f0o | `enable_distributed_ipv6` in [ovn] ? | 08:45 |
| f0o | Feels like IPv6 is still very much experimental in OpenStack/Neutron for anything that isnt a vlan/flat network | 08:45 |
| f0o | or at least the documentation is very obscure about it. Showing a million ways how things *could* work but no clear indication of what is the preferred/reference way | 08:46 |
| noonedeadpunk | f0o: so I think there are 2 options available (at least) | 08:49 |
| noonedeadpunk | if we are not going back to discussion of ovn-bgp-agent (which we had somer time ago), I'd guess it's worth to rely on old bgp dragent | 08:49 |
| noonedeadpunk | so it would be quite same setup as for ovs afaik | 08:49 |
| noonedeadpunk | I have not tried that personally with OVN, but I heard miultople times that it just works | 08:50 |
| noonedeadpunk | we're having IPv6, but with OVN driver we do have ovn-bgp-agent | 08:50 |
| f0o | my main fear is the placement of a FRR instance on our gateway hosts which would break those as they host FRR themselves for their own connectivity to the rest of the fabric | 08:51 |
| noonedeadpunk | so ipv6 and ipv4 work quite alike, except that for IPv6 we do have subnet pool and allocations | 08:51 |
| noonedeadpunk | so frr should be placed on each gateway node | 08:52 |
| f0o | I have IPv6 working entirely fine without BGP by just using a flat-network'ed ovn-router which then ties into every tenant's vxlan network. But this puts an imense stress on the gateway host becoming a singular bottleneck obviously | 08:52 |
| noonedeadpunk | if gateway != compute ofc | 08:52 |
| f0o | ok that's very good to know | 08:52 |
| f0o | then regardless what I do, I need to move the gateway hosts away from the Top-Of-Rack routers | 08:53 |
| noonedeadpunk | so in our scenario we set FRR on just cvouple of nodes which act as a gateway | 08:53 |
| noonedeadpunk | well | 08:53 |
| noonedeadpunk | maybe you can continue doing the same way with flat network... not 100% sure.... | 08:54 |
| noonedeadpunk | but regardless of bgp, what would you need to do, is create an address scope and allocation pool with IPv6 subnet in it | 08:55 |
| noonedeadpunk | then you create an external network or add a subnet from this subnet pool to existing one (it can be same as for ipv4) | 08:56 |
| noonedeadpunk | and client routers should allocate an own /64 or smth from this same subnet pool and add them to there internal geneve/vxlan network | 08:56 |
| noonedeadpunk | while connect their router to the public subnet as external one, so that router would get ipv6 as well | 08:57 |
| noonedeadpunk | traffic would still come through the gateway, but you kinda not limited to single router, and they are spread across multiple ones. | 08:57 |
| f0o | The way Out is not the problem here | 08:58 |
| f0o | but the way back in | 08:58 |
| noonedeadpunk | well, thus we announce the router IP and tenant subnet with ovn bgp agent | 08:59 |
| f0o | if I have /48 on vlan123; and set an allocation pool to distribute /64 to tenants. Then I create a Tenant Router which sits on that vlan123 network as well as on the tenant's vxlan432 which now is in charge of 2001:db8:1::/64, how would I know that it is charge of it and route the traffic to that specific tenant router? | 08:59 |
| f0o | so regardless how I spin this, it seems I will always need ovn-bgp-agent in some form or shape to make that revpath available to my network infrastructure | 09:00 |
| noonedeadpunk | but bgp dragent should be announcing that as well | 09:00 |
| noonedeadpunk | at least we had exact same setup in OVS with it | 09:00 |
| noonedeadpunk | (and I heard it does work in OVN) | 09:00 |
| f0o | and since the agent needs to sit on the gateway hosts, I will need to migrate those out of the current ToR infrastructure and place them on some new dedicated machines somewhere | 09:01 |
| f0o | or am I thinking this wrong? | 09:01 |
| noonedeadpunk | I think for IPv6 you need BGP in some way or form, yes | 09:02 |
| noonedeadpunk | though this could be second option if not ovn-bgp-agent: https://docs.openstack.org/neutron/latest/admin/config-bgp-dynamic-routing.html | 09:02 |
| noonedeadpunk | as you would need to re-do ipv4 as well for it | 09:02 |
| f0o | trying to find a place to upload a diagram to show where my brain-knot is :D | 09:04 |
| f0o | https://imgur.com/a/VWnJ0Cg | 09:04 |
| f0o | so if those two links with `?` become "Gateway Hosts" and "OVN-BGP-Agent" then I know a path forward | 09:05 |
| noonedeadpunk | so with ovn-bgp-agent, vlan ext net is really a virtual vlan, which is present only inside of OVN | 09:05 |
| noonedeadpunk | while you don't need to have it on switches | 09:06 |
| f0o | I do for distributed_fip tho | 09:06 |
| noonedeadpunk | instead, you need to figure you a peering vlan or jsut use default route for that | 09:06 |
| noonedeadpunk | then, ovn-bgp-agent create a vrf on ovn gateway node, where fakes the vlan as noop device. And then frr picks up addresses from VRF for announcement | 09:07 |
| noonedeadpunk | so if you do distributed fip, then I guess you'd need that on all computes actually | 09:07 |
| noonedeadpunk | but also you may apply that not for all vlan networks... | 09:08 |
| noonedeadpunk | you still can have just regular vlans, and announce on others... | 09:08 |
| noonedeadpunk | but yeah | 09:08 |
| noonedeadpunk | btw, fip is not really applicable for ipv6... not helpful if you wanna have dual stack network though | 09:08 |
| f0o | I know but the fip stuff already made me span the vlan to all computes | 09:09 |
| noonedeadpunk | right... | 09:09 |
| f0o | the dragent; that just needs to run on *some* host and sets foreign next-hop like a route-server right? | 09:09 |
| noonedeadpunk | well, I mean. if it's only about ipv6, and you don't wanna touch ipv4 - I'd suggest really give old dragent a try | 09:10 |
| noonedeadpunk | I think it runs on neutron-api, but not 100% sure tbh.... | 09:10 |
| noonedeadpunk | maybe indeed on gateways... | 09:10 |
| f0o | yeah I'm reading the docs right now and I'm intruiged if it doesnt become it's own nexthop | 09:10 |
| f0o | if it can just announce the prefix with the OVN router's interface as next-hop directly then that would be absolutely ideal | 09:10 |
| noonedeadpunk | well, ovn router does not have directly exposed interface, does it? | 09:11 |
| f0o | it does because the extnet is a vlan | 09:12 |
| noonedeadpunk | oh, ok, yes... | 09:12 |
| f0o | so I can place the extnet's v6 subnet as linknet | 09:12 |
| noonedeadpunk | I don't think it does that though tbh... | 09:12 |
| noonedeadpunk | but not sure | 09:12 |
| f0o | then it and the switches (or even ToR) has direct L2 access | 09:12 |
| f0o | so if I had some BGP speaker that just tells me "2001:db:1::/48 next-hop 2001:db:0::12ab | 09:12 |
| f0o | where 2001:db:0::12ab is the IP from the tenant's router external interface | 09:13 |
| f0o | although in reality the linknet might just be a ULA instead of a GUA | 09:14 |
| f0o | doesnt really matter wther ULA or GUA in the end. IPv6 is free anyway heh | 09:14 |
| f0o | gonna take a look at the dragent codebase and investigate what it annunces as next-hop and if I can just patch it to do what I have in mind | 09:15 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!