14:01:26 <slaweq> #startmeeting neutron_drivers
14:01:27 <opendevmeet> Meeting started Fri Jun  4 14:01:26 2021 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:28 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:30 <opendevmeet> The meeting name has been set to 'neutron_drivers'
14:01:35 <ralonsoh> hi
14:01:40 <slaweq> hi
14:01:40 <ivc_> o/
14:01:41 <obondarev> hi
14:01:45 <amotoki> hi
14:02:47 <slaweq> let's wait few minutes for mlavalle, haleyb, njohnston and yamamoto
14:02:58 <njohnston> o/
14:03:23 <haleyb> hi, sorry i'm late
14:04:41 <slaweq> ok, I think that mlavalle and yamamoto are not connected to irc for now
14:04:42 <slaweq> so we can probably start
14:04:48 <slaweq> #topic RFEs
14:04:55 <slaweq> we have 1 new RFE to discuss today
14:05:01 <slaweq> https://bugs.launchpad.net/neutron/+bug/1930200
14:05:03 <opendevmeet> Launchpad bug 1930200 in neutron "[RFE] Add support for Node-Local virtual IP" [Wishlist,New]
14:05:24 <slaweq> it is proposed by Ilya and (probably) obondarev, right?
14:05:31 <ivc_> yes
14:05:34 <obondarev> yep
14:06:00 <ivc_> Hi. My name is Ilya Chukhnakov. I’m the leader of Cloud Technologies Research Team at Advanced Software Technology Lab of Huawei.
14:06:09 <ivc_> Oleg Bondarev and Mamatisa Nurmatov are the other 2 members of this team working on OpenStack projects (and we have other members working on Kubernetes).
14:06:18 <ivc_> We are currently working on a feature for OpenStack Neutron that we internally named Node-Local IP. We would like to share it with the community.
14:06:27 <ivc_> The RFE is available here: https://bugs.launchpad.net/neutron/+bug/1930200
14:06:28 <opendevmeet> Launchpad bug 1930200 in neutron "[RFE] Add support for Node-Local virtual IP" [Wishlist,New]
14:06:37 <ivc_> We are currently working on a detailed specification for this feature. We plan to describe multiple options/variants for various parts of the feature as part of  the spec (from API, UX and backend perspective) so we can discuss these options during the review process and choose the best solution. Please let me know if you have any questions.
14:06:47 <ivc_> (done copy-pasting)
14:07:19 <obondarev> :D
14:07:25 <slaweq> thx ivc_ for the introduction
14:07:37 <slaweq> I see that there are some questions from amotoki in the launchpad
14:08:07 <amotoki> yeah, I added small comments just before the meeting as I thought it may be starting points.
14:09:41 <ralonsoh> sorry but I still don't see the usecase for this RFE
14:09:42 <ivc_> do you want me to answer those questions here?
14:09:54 <slaweq> ivc_ yes, please
14:09:54 <slaweq> :)
14:10:27 <ivc_> our current main use-case for this feature is a per-nod distributed caching service
14:11:08 <ivc_> for large clouds where centralized caches with multiple nodes might not be enough
14:11:39 <ivc_> one such example is caching Docker registry - a side-car proxy+cache that optimizes the delivery of container images to K8s worker nodes (VMs)
14:12:24 <slaweq> amotoki regarding Your question, I'm not sure if I understand it really, what routing You want to have there? Isn't that proposal related to the single L2 network?
14:12:27 <ivc_> without such per-node cache each worker node would have to connect to central registry (or centralized cache) and consume traffic
14:12:52 <ralonsoh> so you need to have a replica in each node
14:12:52 <ivc_> with per-node side-car cache we can save on data transfer and improve delivery speed
14:13:03 <ralonsoh> and each replica will have a local IP
14:13:17 <ralonsoh> and you want this IP to be the same in each node?
14:13:22 <ivc_> yes
14:13:24 <ralonsoh> of course, this IP no accesible
14:13:40 <obondarev> ralonsoh, maybe not on all nodes
14:13:52 <amotoki> slaweq: I am not sure the proposal is limited to a single L2 network or not.
14:13:56 <ivc_> in a sense it's similar to metadata-proxy.and 169.254.169.254 address
14:14:04 <haleyb> sounds a little like the metadata IP, which is a distributed thing in OVN
14:14:18 <ralonsoh> right
14:14:30 <ivc_> yes that's right
14:14:40 <ralonsoh> ivc_, what about amotoki's question?
14:15:10 <ivc_> ralonsoh amotoki regarding the 'default' route we have several options
14:15:34 <ivc_> I'll go into more detail in a spec. but can cover briefly
14:15:59 <ivc_> one option is to have such a default route (and maybe attach it to floating ip)
14:16:54 <ivc_> another option would be to later expand the feature and instead of it being strictly 'local' to the node make it more like topology-aware load-balancer
14:17:00 <lajoskatona1> Hi
14:17:34 <ivc_> but the topology-aware use case is out of the scope of the initial version (as it is too complicated) and we would like to start with just a local one
14:17:36 <slaweq> hi lajoskatona1 :)
14:18:07 <ralonsoh> so far, for me this is more like a floating "fixed_ip" with DVR
14:18:15 <ivc_> sort of
14:18:50 <obondarev> but without keepalived
14:18:50 <slaweq> I'm still not sure if I understand that correct
14:19:01 <slaweq> in the LP You wrote that Port-C-Src will not reach any destination
14:19:17 <slaweq> is that a must? or it not exactly, just kind of "side effect"?
14:19:48 <ivc_> it's not a "must" but imho the most reasonable 'default behavior'
14:20:09 <obondarev> as suggested by amotoki we can have a default route, and Port-C-Src could go to a remote resource with same IP
14:20:17 <obondarev> its optional
14:20:30 <njohnston> so is the key benefit of this proposal the predictable addressing, so that rather than engage in some kind of discovery you can always know that 127.1.0.1 is the sidecar CDN service?  Or is it making sure instance-to-sidecar traffic doesn't leave the local node?
14:20:47 <slaweq> ok
14:21:32 <ivc_> njohnston yes
14:21:56 <slaweq> so core of this RFE is to block traffic send to such Local-IP-address so it will not go outside br-int for sure
14:21:59 <slaweq> right
14:22:00 <slaweq> ?
14:22:06 <ivc_> for the 127.1.0.1, except we plan to use a different ip range :)
14:22:28 <amotoki> I agree 'default' behavior is an option. we can choose either. more important point is an IP address resolution inside a node.
14:22:36 <amotoki> (as slaweq mentioned above)
14:22:48 <obondarev> I would say key benefit is performance
14:23:08 <obondarev> blocking traffic from other nodes is more a side effect
14:23:20 <obondarev> key point is to have API resource for such address
14:23:22 <njohnston> ivc_: Sorry to quibble but you said yes to an “or” question. Does that mean the latter option, or does it mean both?
14:23:42 <obondarev> and guarantee local-node performance
14:24:11 <obondarev> njohnston, I think yes was for first one
14:24:23 <njohnston> thanks for the clarification
14:24:26 <obondarev> ""the predictable addressing, so that rather than engage in some kind of discovery you can always know that 127.1.0.1 is the sidecar CDN service
14:24:47 <ivc_> njohnston sorry, i tried to clarify later :) i meant the first one - the 127.1.0.1, but we do not want to use 127.0.0.0/8 network for such IPs
14:25:27 <ivc_> a better example would be 169.254.169.254 (metadata-proxy)
14:25:33 <obondarev> usability should be similar to current Floating IP flow
14:25:35 <njohnston> understood
14:25:53 <obondarev> meaning transparency
14:26:14 <ivc_> but ideally we'd like to be able to use address from regular neutron subnets too
14:26:32 <ivc_> and not limit it to hard-coded well-known network ranges
14:27:46 <amotoki> ivc_: is "well-known network ranges" a loopback network address like 127.0.0.0/8?
14:28:11 <amotoki> ^ is just for confirmation
14:29:09 <ivc_> amotoki yes, but to clarify - ideally we want to be able to use arbitrary addresses for Node-Local IP (e.g. such as 10.10.10.10)
14:29:51 <ivc_> and we want to be able to create multiple such IPs and assign it to different VM groups
14:29:57 <amotoki> ivc_: thanks. I believe what you would like to achieve. it is just to clarify the wording.
14:30:24 <amotoki> s/I believe/I believe I understand/
14:31:33 <njohnston> would this be a subnet of such IPs or would you do this on a per-IP basis? For example if not constrained to a specific subnet you had cdn.example.org on 10.10.10.10 and all hosts go directly but you could drop in a sidecar and transparently hijack traffic to that IP and redirect to a local cache.
14:31:47 <njohnston> but that could be really hard to debug when there are issues.
14:33:49 <ivc_> njohnston we'd like Node-Local IP to behave similar to current Floating IP - i.e. the creation of NodeLocalIP would be restricted to only allowed ranges
14:34:25 <ivc_> from the API perspective we expect it to behave similar to current Floating IPs
14:34:36 <njohnston> ok
14:35:47 <ivc_> in fact that's part of this RFE's core - to have a new API resource type that can be used to assign such IPs to groups of VMs
14:35:59 <amotoki> do an IP address of a VM and Node-Local IP belong to a same subnet?
14:36:06 <slaweq> what about MAC address, will it have some "common" MAC which will be configured on all vms which have this IP?
14:36:14 <slaweq> or will it work on different MAC on each vm?
14:36:24 <haleyb> and what about IPv6?
14:37:13 <obondarev> amotoki, not restricted to a single subnet
14:37:21 <obondarev> same*
14:38:10 <amotoki> obondarev: so, traffic is routable (like local traffic of DVR) right?
14:38:13 <haleyb> i'm also interested in the possible overlap with octavia, just don't have a firm grasp yet on how it would/should work with this
14:38:22 <ivc_> slaweq that decision is still WIP - there are several options, e.g. we can implement NodeLocalIP as L2 feature. in that case we think MAC can be randomly generated for each IP, but same MAC would be used for same IP on different nodes
14:38:36 <obondarev> amotoki, not exactly, traffic never goes out br-int
14:39:08 <ivc_> another option would be to make NodeLocalIP an L3 feature (but not routable) similar to current 169.254.169.254 address translation
14:39:16 * haleyb has to leave for appt, but will read transcript later
14:39:31 <obondarev> amotoki, it's more a form of NAT
14:39:55 <ivc_> we would like to describe both of these options in a spec and decide on the best option during the review
14:40:00 <haleyb> we don't do NAT with IPv6 right now...
14:40:25 <obondarev> haleyb, it's not exactly NAT, it's on flow level
14:40:50 <amotoki> obondarev: I wonder if it is L2 thing or L3 thing. NAT is a kind of L3 thing and traffic is L3-routed.
14:40:58 <amotoki> obondarev: that's confuses me .....
14:41:02 <obondarev> OVN's localport is similar thing
14:41:32 <obondarev> amotoki, no routing, L2
14:42:40 <ivc_> haleyb re:octavia that's very understandable. in a sense Node-Local IP can be seen as a limited load-balancer, except instead of balancing it always directs traffic to local single predefined VM
14:43:46 <amotoki> obondarev: I am not sure I understand it correctly. NodeLocal-IP can belong to a different subnet from a subnet which an IP of a VM belongs to... I am okay others already understand it correctly though.
14:43:47 <ivc_> haleyb we also think that this feature might in fact at some point benefit Octavia - it can simplify the creation of distributed loadbalancer with load-balance-at-source pattern
14:44:36 <ivc_> haleyb similar to how Kubernets' Istio project works
14:46:55 <ivc_> @all do you think that would be easier to discuss this feature if we share the spec draft for it?
14:47:18 <slaweq> ivc_ I think that we will need to discuss details in the spec's review
14:47:43 <slaweq> but in overall I'm ok with that RFE as an idea, so I'm +1 for approving it
14:48:01 <ralonsoh> +1 to the RFE
14:48:11 <njohnston> +1
14:48:26 <njohnston> I can definitely see beneficial use cases
14:48:29 <amotoki> I am okay with the idea itself (optimizing such thing as the neturon level)
14:48:45 <amotoki> even if it is a flow-level, it emmulates existing L2/L3 behavior (when we see it from a VM perspective), so I am confused with "it's a flow-lvel"....
14:49:03 <amotoki> but i believe it will be clarified in a discussion in the spec review.
14:49:18 <obondarev> amotoki, correct, I meant emulation
14:49:40 <obondarev> transparent to user/VM
14:50:02 <obondarev> amotoki, sorry for confusion
14:50:14 <slaweq> ok, so I'm going to mark the rfe as approved and we will wait for the spec to review
14:50:24 <ivc_> ok, thanks!
14:50:31 <slaweq> thx all for the discussion today
14:50:31 <obondarev> thanks everyone
14:50:33 <amotoki> obondarev: no problem. the wording is always difficult :)
14:50:43 <obondarev> amotoki, right :)
14:50:56 <slaweq> that was the only RFE for today
14:50:59 <slaweq> so we are (almost) done for today
14:51:18 <slaweq> one last thing - please remember that next week's meeting will be in the #openstack-neutron channel
14:51:35 <slaweq> have a great weekend and see You all online next week o/
14:51:39 <ralonsoh> bye!
14:51:41 <njohnston> o/
14:51:43 <amotoki> o/
14:51:49 <manub> bye
14:51:49 <obondarev> o/
14:51:50 <slaweq> #endmeeting