15:01:03 #startmeeting neutron_l3 15:01:04 Meeting started Thu Mar 27 15:01:03 2014 UTC and is due to finish in 60 minutes. The chair is carl_baldwin. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:08 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:11 The meeting name has been set to 'neutron_l3' 15:01:14 #topic Announcements 15:01:26 #link https://wiki.openstack.org/wiki/Meetings/Neutron-L3-Subteam 15:01:49 Hi , hello carl_baldwin , YorikSar , amuller , safchain 15:01:59 hiya 15:02:05 Hi 15:02:32 In general, I encourage you to add your action items. Please feel free. Look up meetbot for commands that you can use during the meeting. 15:03:08 Is there anything that we need to discuss related to the Icehouse release? 15:04:14 The only thing that I'm still working on is documentation of the sudo fix that we've identified. 15:04:38 I've been in contact with Edgar about that. 15:04:44 carl_baldwin, do you mean, the sudo performance fix? 15:04:51 ajo: yes 15:05:10 Some of our testing has shown good promise for it. 15:05:24 #link http://www.sudo.ws/repos/sudo/rev/e9dc28c7db60 15:05:50 ajo: Thanks for the link. 15:05:53 It's supposed to slow down as you have more network interfaces unless you disable the new option in this patch. 15:06:11 Correct. 15:06:16 ok, that's good, I haven't been able to test it yet 15:06:37 neither the iproute one 15:06:40 It is looking good from my perspective. 15:06:51 I haven't been able to get much testing on the iproute one. 15:07:03 #topic l3-high-availability 15:07:16 safchain: Any update? 15:08:30 carl_baldwin, nothing this week 15:09:03 hi 15:09:05 Okay. I'm getting closer to having some time to test but I haven't yet. 15:09:16 Swami: hi. Your timing is good. 15:09:23 #topic neutron-ovs-dvr 15:09:24 thanks 15:09:32 Swami: Any updates for the team? 15:09:40 Yes 15:09:50 DVR Progress update was provided to the subteam. 15:09:56 The Plugin extension is almost done 15:10:10 Agent work is done for most part of the East-West 15:10:28 We still have some issues with the ovs rules to resolve, we fixing it right now. 15:10:47 Design docs are out there and we are addressing all the comments from it. 15:10:53 Good progress. 15:11:01 That's all from me. 15:11:32 Swami: Thanks. I know we're all dying to see code. :) 15:11:50 =) 15:12:14 #topic bgp-dynamic-routing 15:12:19 carl: Yes I understand the problem, and we will upload it soon. 15:12:36 Swami: You know us developers. :) 15:13:00 safchain: Swami: I saw the patch that optionally removes the "normally learned" flows from br-tun 15:13:18 safchain: Thanks for the patch 15:14:08 nextone92: hi 15:14:14 Swami, thank for suggesting this improvement 15:14:15 Hi 15:14:27 The proposed changes to dynamic routing blueprint remove BGP functionality from tenant routers 15:14:44 And then add a data structure “GatewaySpec” to external network 15:15:05 this new, optionally specified, data structure will contain dynamic routing configuration 15:15:24 it may only be configured by the openstack administrator 15:15:27 I like the association with a network. 15:15:44 If that sounds good to everyone, I will update the document shortly :) 15:15:45 For one, it should remove all of my scale concerns. 15:16:31 I think it's clean and will only impact those who want additional routing services at the edge of neutron network 15:16:35 nextone92: what does this GatewaySpec datastructure hold. 15:17:14 Here's what I'm thinking: routing type: static/dynamic-bgp (more in the future) 15:17:29 and then information to accomplish the type of routing 15:17:45 for bgp it will be such information as remote peers, AS, and shared secrets 15:18:24 BTW, I took a crack at outlining the use cases as I promised to last week. It needs more detail but it is a start. 15:18:32 #link https://wiki.openstack.org/wiki/Neutron/DynamicRoutingUseCases 15:18:55 Oooh great! 15:19:00 The first use case is lifted directly from nextone92 's document. 15:19:05 I added the second. 15:19:27 nice carl_baldwin 15:19:29 The third is the result of my trying to wrap my head around the BGP/MPLS blueprint and use case. 15:19:59 Hopefully, this will aid in our discussions about the topic. 15:20:13 carl: I will go through the doc and provide my comments 15:20:38 carl_baldwin: Thanks for that 15:21:31 amuller: Feel free to ask questions that come up on IRC on on the ML. I will be adding more detail as questions come up. 15:21:39 One question I had is, will this be configurable by a tenant or will it be intelligent enough to add "static/versus dynamic" based on the service type that we choose. Or is it all controlled by the Admin at this point. 15:22:13 It would only be configurable by the administrator 15:22:19 In my personal view of the world, this will all be controlled by the admin. What do others think? 15:23:06 With any dynamic routing relationship, I think it requires some administrative knowledge of the relationship of the openstack cloud with the infrastructure around it. 15:24:20 Today for example VPN service is configured by a Tenant and not by administrator and if the VPN Service requires Dynamic routing capability then it should be done already by an admin. Is that what you are suggesting. 15:24:29 It might make sense on a provider network or something where the tenant is given some control or knowledge of the outside. 15:25:09 Swami: the difference with BGP/MPLS is that it doesn't rely on strong endpoint authentication or encryption for privacy. 15:25:37 Instead, it relies on trust between customer and provider and, now, the cloud provider who is a third provider. 15:25:57 Also, Swami, Carl - if the dynamic routing is attached to the external network, would it be difficult to setup dynamic routing just for the tenant and not the rest of the network? 15:26:23 Maybe if a tenant owns the external network wholly? 15:26:46 nextone92: I wrote a few words on that. I think if the external network is shared between tenants, it will be very difficult. 15:26:59 agreed, carl_baldwin 15:27:07 nextone92: Yes, agreed. 15:27:15 Yes, that is what I am not clear. Is it per tenant basis or global flag that enables the dynamic routing. Or is it based on a router context. 15:27:46 Swami: the association is shifting from the router context to the external network context. 15:28:07 carl: thanks for the clarification. 15:28:37 carl_baldwin, Swami - to clarify, it would be good to associate one or more external networks with the same gatewayspec 15:28:57 so a quagga instance could potentially serve more than one external network 15:29:47 Swami: yw, you're questions are very well-timed. These are the details we need to get out in the open. 15:30:32 carl: Yes we need to do it. 15:31:12 I think we're on track for a good discussion at the summit. nextone92 will you be updating your document? 15:31:35 Yes, I will take an action item to update the document 15:32:16 I will join briefly after a break. 15:32:27 Great. Feel free to use the action command to take action items. 15:33:28 #action carl_baldwin will announce use case wiki on ML and field questions. 15:34:17 #action nextone92 will update bhp-dynamic-routing document 15:34:39 #topic DNS lookup of instances 15:35:09 I wrote up a draft. It is pending approval to make public. So, soon. 15:35:23 #topic Agent Performance with Wrapper Overhead 15:35:27 carl 15:35:28 sorry, 15:35:40 ahh, nothing, you'll make it public soon 15:35:46 I was going to ask about the DNS lookup of instances 15:35:55 you mean, from outside tenant networks, right? 15:36:09 Feel free to ping me on IRC. 15:36:15 sure 15:36:36 YorikSar, 15:36:46 ajo: I've tagged you as the lead on this performance stuff though I know YorikSar has been very active. 15:36:46 hi 15:37:00 o/ 15:37:12 Yes, yorikstar is doing an awesome work 15:37:17 I've started working on Neutron changes to integrate rootwrap daemon 15:37:18 Mostly just because ajo brought it up on the ML originally. 15:37:31 https://review.openstack.org/82787 - I've already added carl_baldwin and ajo to that CR 15:37:43 YorikSar: I am part-way through a review of each of your two patches. 15:37:44 #link https://review.openstack.org/82787 15:37:58 I have draft comments but it may take me a couple more days. 15:38:24 It became rather intrusive but I hope it won't be opposed too strongly. 15:38:27 I encourage others to review. Especially those who have been active in the ML thread. 15:38:48 YorikSar, did you see the security issues Thierry found in https://review.openstack.org/#/c/81798/ ? 15:39:17 #link https://review.openstack.org/#/c/81798/7/oslo/rootwrap/daemon.py 15:39:21 I doubt I will be able to work on this more this week (well, maybe only weekend) so it'll most likely be on hold till Tuesday. 15:39:33 I appreciate any comments or thoughts you can provide. 15:39:44 ajo: Let me check... 15:39:56 it seems that basemanager doesn't process the "serializer" parameter 15:40:09 and it will rely on pickle always 15:40:12 Oh... I'll respond there. 15:40:18 at least from the doc perspective, I didn't check the actual code of BaseManager 15:40:27 BaseManager does support this, but it's undocumented. 15:40:35 YorikSar: heh 15:40:36 ahhhh, that explains a lot 15:40:51 #action carl_baldwin will communicate details about sudo patch to Edgar to be included in the release documentation. 15:41:02 ^ Just throwing that in there. :) 15:41:05 That's unfortunate but it shouldn't change in future releases. 15:41:29 YorikSar, is that true for py26 too? 15:41:56 I can try to push change to CPython to include this parameter into docs. 15:42:12 ajo: Em.. I'm not sure. Let me check py26 15:42:27 YorikSar: That'd be great to get some commitment on the feature. 15:42:37 I can see, the code is there for py27 15:42:49 ajo: Good call on checking py26. 15:42:52 ajo: Yes, it's there in py26 15:43:15 And py32 15:43:21 And py33 15:43:24 ahh, good YorikSar 15:43:36 and, is it documented for py3.3 ? 15:43:49 Gentoo is cool - you can have as many Python interpreters as you want installed in the system :) 15:43:54 hehe :) 15:44:21 Nope, it's not documented... 15:44:29 I wonder why 15:44:31 For 3.4 I mean. 15:44:44 I guess noone ever used this. 15:45:21 I hope we don't find bugs because of this, or that they change the implementation in the future unexpectedly 15:45:52 may be we should git-blame and ask the original authors of that feature. 15:45:56 We're switching off pickle just for security reasons and this might be never anticipated by authors of this :) 15:46:19 ajo: Since it's in 2.6, it might be svn-blame :) 15:46:27 :) 15:46:42 I'll ask on python-dev 15:46:54 YorikSar, I'll do it if you want 15:47:01 I was writting an #action line 15:47:03 to provide some help :) 15:47:18 Oh, no. It's not for python-dev, but for python@, I guess. 15:48:28 ajo: I'll do it myself. I want to get to the bottom of it :) 15:48:36 #action carl_baldwin will finish review of root wrap daemon patches. 15:48:37 #action ajo ask in python-dev / python@ mail list about the undocumented 'serializer' parameter of multiprocessing.BaseManager object 15:48:48 ahh :D 15:48:49 ':D 15:49:02 #action YorikSar ask in python-dev / python@ mail list about the undocumented 'serializer' parameter of multiprocessing.BaseManager object 15:49:34 #action ajo finish review on neutron-side root wrap daemon patches 15:49:59 Good progress. Anything else? 15:50:39 Nope 15:51:17 #topic General Discussion 15:51:37 On, maybe I have a question about rootwrap. 15:51:47 Do you think it worth creating a blueprint? 15:52:14 I think so. 15:52:25 I guess, on for oslo and one for Neutron (and one for each of Nova and Cinder later) 15:52:25 YorikSar, yes, probably, you just need a link to your etherpad. 15:52:35 Ok, will do 15:52:54 I was listening to some concerns, minutes ago, about the qrouter- namespace memory usage (due to neutron-ns-metadata-proxy) 15:52:58 Yeah, it shouldn't be too much work at this point. Just formalizing the blueprint's existence and linking. 15:53:29 what was the initial reason to use this + domain socket ? 15:54:08 ajo: I'm not sure I understand the question yet. 15:54:11 thinking about it again... there isn't a decent way to route that via iptables to the neutron-metadata-server directly 15:54:24 We use the neutron-ns-metadata-proxy in every qrouter namespace 15:54:31 ajo: because that proxy doesn't have access to the management network 15:54:33 (or qdhcp, when isolated_metadata is enableD) 15:54:44 I think those proxies will be replaced with one process with one thread per ns later. 15:55:04 haleyb, right, 15:55:11 YorikSar, is it possible to setns per thread? 15:55:14 i could not find a way to reduce it's memory footprint, it's all just based on what it imports 15:55:22 ajo: Yes. setns works per-thread. 15:55:48 hmm, that's interesting 15:56:04 ajo: I didn't verify it with fork calls, so if we ever want to add this to rootwrap it'll require some investigation. 15:56:05 so we could just start a thread + setns for every tenant network we're serving 15:56:07 I imagine those threads must be OS level threads. 15:56:19 carl_baldwin: Sure. 15:56:30 carl_baldwin, sure, 15:56:57 although python GIL doesn't provide much benefit of os threads... in this case, memory benefit... could do the work 15:56:58 I guess we can even spawn thread, do setns(), create socket, pass it to main thread and let helper thread die. 15:57:02 just clarifying. :) 15:57:23 but then there's the problem of scaling 15:57:25 ajo: Ah.. Those prejudices again... 15:58:22 if we don't get real threading speedup, and just memory speedup 15:58:32 we could end up being a bottleneck here 15:58:44 memory speedup -> memory usage reduction 15:59:05 can be ok if we fork once per "N" threads 15:59:06 I don't think there will be any measurable difference between thread per socket and greenthread per socket. 15:59:33 Since threads sleep on I/O anyway and GIL won't be held by them. 15:59:43 YorikSar: Can it still listen and accept connections on the address inside the ns if the worker thread dies? 16:00:08 carl_baldwin: I don't know. It should. Need more research here :) 16:00:15 I suppose it can't but just guessing 16:00:35 carl_baldwin: I guess I will be able to get to this task during J cycle 16:01:02 may be we should start the performance subteam ;) 16:01:06 We can discuss more on IRC, ML, or later meetings. For now, we're out of time. 16:01:14 sure 16:01:36 sure, thanks carl_baldwin , YorikSar , * 16:01:42 Great meeting. Thanks everyone. 16:01:53 #endmeeting