Friday, 2020-02-14

*** sfernand has quit IRC00:00
*** ijw has joined #openstack-meeting00:10
*** slaweq has joined #openstack-meeting00:11
*** yamamoto has joined #openstack-meeting00:15
*** slaweq has quit IRC00:16
*** ociuhandu has joined #openstack-meeting00:17
*** jmasud has quit IRC00:18
*** ociuhandu has quit IRC00:22
*** armax has joined #openstack-meeting00:35
*** diablo_rojo has quit IRC00:52
*** migawa is now known as migawa|AFK00:52
*** migawa|AFK is now known as migawa00:52
*** slaweq has joined #openstack-meeting01:11
*** slaweq has quit IRC01:16
*** yamamoto has quit IRC01:19
*** Lucas_Gray has quit IRC01:26
*** rfolco has quit IRC01:32
*** gyee has quit IRC01:36
*** vishalmanchanda has joined #openstack-meeting01:54
*** slaweq has joined #openstack-meeting02:11
*** slaweq has quit IRC02:16
*** jmasud has joined #openstack-meeting02:21
*** jawad_axd has joined #openstack-meeting02:22
*** jiaopengju1 has quit IRC02:22
*** jiaopengju1 has joined #openstack-meeting02:22
*** jawad_axd has quit IRC02:27
*** ijw has quit IRC02:41
*** jawad_axd has joined #openstack-meeting02:43
*** migawa is now known as migawa|AFK02:45
*** migawa|AFK is now known as migawa02:47
*** jawad_axd has quit IRC02:47
*** rosmaita has quit IRC02:48
*** jamesmcarthur has joined #openstack-meeting02:50
*** rosmaita has joined #openstack-meeting02:51
*** migawa is now known as migawa|AFK02:53
*** armax has quit IRC02:54
*** armax has joined #openstack-meeting03:07
*** jamesmcarthur has quit IRC03:08
*** slaweq has joined #openstack-meeting03:11
*** apetrich has quit IRC03:13
*** jamesmcarthur has joined #openstack-meeting03:15
*** slaweq has quit IRC03:16
*** masahito has joined #openstack-meeting03:19
*** masahito_ has joined #openstack-meeting03:22
*** masahito has quit IRC03:22
*** yamamoto has joined #openstack-meeting03:25
*** armax has quit IRC03:55
*** migawa|AFK is now known as migawa03:56
*** slaweq has joined #openstack-meeting04:11
*** jmasud has quit IRC04:13
*** jmasud has joined #openstack-meeting04:14
*** slaweq has quit IRC04:16
*** jamesmcarthur has quit IRC04:21
*** masahito_ has quit IRC04:32
*** masahito has joined #openstack-meeting04:40
*** masahito has quit IRC04:46
*** masahito has joined #openstack-meeting04:52
*** nicolasbock has quit IRC05:09
*** slaweq has joined #openstack-meeting05:11
*** slaweq has quit IRC05:16
*** jamesmcarthur has joined #openstack-meeting05:23
*** jamesmcarthur has quit IRC05:28
*** psachin has joined #openstack-meeting05:37
*** hyunsikyang has joined #openstack-meeting05:46
*** migawa is now known as migawa|AFK05:53
*** migawa|AFK is now known as migawa05:54
*** slaweq has joined #openstack-meeting06:11
*** slaweq has quit IRC06:16
*** hyunsikyang has quit IRC06:23
*** lajoskatona has joined #openstack-meeting06:24
*** lajoskatona has left #openstack-meeting06:24
*** masahito has quit IRC06:26
*** manuvakery has joined #openstack-meeting06:27
*** masahito has joined #openstack-meeting06:34
*** hyunsikyang has joined #openstack-meeting06:34
*** jhesketh has quit IRC06:48
*** jhesketh has joined #openstack-meeting06:50
*** slaweq has joined #openstack-meeting07:11
*** jhesketh has quit IRC07:15
*** slaweq has quit IRC07:16
*** tetsuro has joined #openstack-meeting07:22
*** jawad_axd has joined #openstack-meeting07:22
*** tetsuro has quit IRC07:23
*** jhesketh has joined #openstack-meeting07:25
*** bbowen_ has joined #openstack-meeting07:29
*** bbowen has quit IRC07:30
*** rsimai has quit IRC07:56
*** rsimai has joined #openstack-meeting07:57
*** ralonsoh has joined #openstack-meeting07:57
*** maciejjozefczyk has joined #openstack-meeting07:58
*** slaweq has joined #openstack-meeting08:00
*** rosmaita has quit IRC08:02
*** rosmaita has joined #openstack-meeting08:04
*** lpetrut has joined #openstack-meeting08:14
*** tesseract has joined #openstack-meeting08:21
*** slaweq has quit IRC08:33
*** slaweq has joined #openstack-meeting08:37
*** rcernin has quit IRC09:12
*** lucasagomes has joined #openstack-meeting09:17
*** apetrich has joined #openstack-meeting09:37
*** beagles has quit IRC09:54
*** yamamoto has quit IRC09:57
*** hyunsikyang has quit IRC09:58
*** rcernin has joined #openstack-meeting10:00
*** yamamoto has joined #openstack-meeting10:04
*** ociuhandu has joined #openstack-meeting10:12
*** rcernin has quit IRC10:17
*** psachin has quit IRC10:25
*** yamamoto has quit IRC10:28
*** vishalmanchanda has quit IRC10:28
*** e0ne has joined #openstack-meeting10:30
*** psachin has joined #openstack-meeting10:32
*** yamamoto has joined #openstack-meeting10:39
*** belmoreira has quit IRC10:46
*** ociuhandu has quit IRC10:54
*** ociuhandu has joined #openstack-meeting10:59
*** ociuhandu has quit IRC11:01
*** ociuhandu has joined #openstack-meeting11:01
*** psachin has quit IRC11:10
*** psachin has joined #openstack-meeting11:11
*** yamamoto has quit IRC11:21
*** yamamoto has joined #openstack-meeting11:54
*** yamamoto has quit IRC11:59
*** Lucas_Gray has joined #openstack-meeting12:06
*** ociuhandu has quit IRC12:07
*** ociuhandu has joined #openstack-meeting12:12
*** nicolasbock has joined #openstack-meeting12:13
*** psachin has quit IRC12:15
*** ociuhandu has quit IRC12:18
*** maciejjozefczyk has quit IRC12:21
*** Lucas_Gray has quit IRC12:26
*** Lucas_Gray has joined #openstack-meeting12:33
*** maciejjozefczyk has joined #openstack-meeting12:37
*** masahito has quit IRC12:42
*** tetsuro has joined #openstack-meeting12:42
*** yamamoto has joined #openstack-meeting12:43
*** raildo has joined #openstack-meeting12:44
*** yamamoto has quit IRC12:45
*** rfolco has joined #openstack-meeting12:47
*** ociuhandu has joined #openstack-meeting12:53
*** yamamoto has joined #openstack-meeting13:06
*** dmacpher has quit IRC13:09
*** ociuhandu has quit IRC13:11
*** rh-jelabarre has joined #openstack-meeting13:12
*** ysandeep is now known as ysandeep|brb13:16
*** ysandeep|brb is now known as ysandeep13:24
*** Lucas_Gray has quit IRC13:26
*** ociuhandu has joined #openstack-meeting13:31
*** Lucas_Gray has joined #openstack-meeting13:32
*** enriquetaso has joined #openstack-meeting13:35
*** psachin has joined #openstack-meeting13:36
*** ociuhandu has quit IRC13:45
*** stephen-ma has joined #openstack-meeting13:46
*** ociuhandu has joined #openstack-meeting13:49
*** Lucas_Gray has quit IRC13:51
*** ociuhandu has quit IRC13:55
*** lajoskatona has joined #openstack-meeting13:56
*** Lucas_Gray has joined #openstack-meeting13:56
*** ociuhandu has joined #openstack-meeting13:58
slaweq#startmeeting neutron_drivers14:00
openstackMeeting started Fri Feb 14 14:00:10 2020 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: neutron_drivers)"14:00
openstackThe meeting name has been set to 'neutron_drivers'14:00
slaweqlets wait few more minutes for amotoki haleyb and mlavalle14:01
slaweqhi haleyb14:01
slaweqso we have already quorum and I think we can start14:01
slaweqas TheJulia is here, lets start not as usual :)14:02
slaweq#topic On Demand Agenda14:02
*** openstack changes topic to "On Demand Agenda (Meeting topic: neutron_drivers)"14:02
slaweqTheJulia has added topic to
*** mlavalle has joined #openstack-meeting14:03
njohnstonhello TheJulia14:03
amotokihi1 sorry for late14:04
slaweqhi mlavalle and amotoki14:04
*** tetsuro has quit IRC14:04
TheJuliaSo bottom line is we're wondering if the mac address update can be made non-admin or covered by a specific policy because ironic is making the service more multitenanty and usable for non-admins, but we pass credentials through for port actions and are trying to avoid pulling a second admin session as the ironic service user to just update the mac address14:04
slaweqjust FYI, I started today with On demand agenda as TheJulia added some topic to it and I didn't want to hold her on the meeting for whole hour :)14:04
TheJuliaslaweq: much appreciated,.... for I have hours of meetings ahead of me :)14:04
njohnstonI wonder if this could be achieved with a policy.json modification defining a role tied to a specific service credential for Ironic14:05
slaweqnjohnston: I think it can:
slaweqit's defined there IIUC what is the need from Ironic14:05
slaweqand it seems for me that it can be done by admin or advsvc user14:06
njohnstonslaweq: yes that is exactly what I was thinking about14:06
amotokiare we discussing mac address update by all non-admin users or users with specific roles?14:07
TheJulianon-admin users of baremetal, which has me thinking we're going to have to do the thing we don't want to do which is pull a separate client/session to directly update the port mac as a separate action14:08
njohnstonthe thing is, Neutron has no way of distinguishing between the Ironic use case and the other use cases where non-admin access to this would be a bad idea14:08
amotokiwe prepared the advsvc role for such purpose. If it works it would be great.14:09
TheJuliaYeah, I suspect we could just have ironic learn how to do it separately, which would prevent potential security issues. I guess well need to look at that. Anyway thanks everyone!14:10
amotokione thing to note is that updating mac address should be limited to a private network.14:11
amotokiI mean "self-service network".14:11
njohnstonAre there some baremetal scenarios where it would not be a good idea to allow mac address updating?14:11
amotokiit should not be allowed in a shared network.14:12
amotokiin other words, the operation should be limited to a network owner IMHO.14:12
TheJulianjohnston: none, we must be able to update the mac address for pxe booting and addressing of physical ports.14:12
TheJuliain that case the ironic service account needs to perform the port update action for just the mac address14:14
TheJuliasince we know it and manage it14:15
njohnstonI am wondering if we could permit port update for all in policy.json and then later in the logic require specific privileges unless it's a baremetal port.14:16
njohnstonBut I don't know if there are baremetal-but-not-Ironic scenarios that could bite us with that14:17
slaweqnjohnston: I'm not sure if we should do such hard coded rules for some specific kinds of resources14:17
TheJuliathat is only informed via the ?binding profile? which I think is later on, also since users can request vifs on user created networks and ironic will request it be attached to that network14:17
*** TrevorV has joined #openstack-meeting14:18
njohnstonHaving a separate admin session has the virtue of simplicity, other methods for doing it get complex quickly it seems to me14:18
TheJuliaThanks, I'll let the contributor working on the multitenancy feature set know!14:18
*** Lucas_Gray has quit IRC14:19
*** ociuhandu has quit IRC14:19
slaweqok, so I think we are good with Your topic TheJulia, right? You will try to use advsvc role for this action.14:19
*** ociuhandu has joined #openstack-meeting14:19
TheJuliayup, thanks14:20
slaweqthx TheJulia :)14:21
slaweqso now we can move to our regular topic14:21
slaweq#topic RFEs14:21
*** openstack changes topic to "RFEs (Meeting topic: neutron_drivers)"14:21
slaweqwe have 2 RFEs for today14:21
slaweqfirst one:14:21
openstackLaunchpad bug 1860521 in neutron "L2 pop notifications are not reliable" [Undecided,New]14:21
slaweqI remember from when I was working in OVH that we had similar problems with L2pop mechanism and we added something like periodic sync of tunnels config on host14:24
njohnstonI am not sure what the impact to the message bus would be to change from fanout/cast to RPC calls14:25
slaweqnjohnston: to the message bus not much but for neutron-rpc workers which will send such messages and wait for reply, impact will be at least "noticeable" IMO14:27
njohnstonslaweq: Yeah, I was worried more about the RPC workers14:27
slaweqnjohnston: ahh, ok :)14:28
mlavalleit's a matter of whether the mesh of tunnels works vs the cost14:28
njohnstonDoes OVN use l2pop?  IIRC it doesn't but I haven't looked in that part of the code in a while.14:28
slaweqnjohnston: nope14:29
amotokiI am not sure right now which is better to switch it to RPC calls or to sync info periodically.14:29
amotokiif the number of nodes to be informed is small, it makes sense to switch it to RPC calls.14:29
mlavallebut I am sure the problem is more acute in large deployments14:30
njohnstonThere are costs and benefits each way - if the RPC call idea adds overhead then it could make the situation in large deployments worse.  With the periodic sync you have a period of time where things might not work correctly, before the next sync.14:31
mlavalleit's always a trade off14:31
*** jlibosva has joined #openstack-meeting14:31
njohnstonI personally favor the periodic sync as being in keeping with our "eventually consistent" way of doing things, but I have a bias towards large deployment thinking.14:31
slaweqactually looking at the code it will not be possible to switch always to call() from cast()14:32
mlavallebut if the mesh of tunnels gets to a point where it doesn't work, then it's time to consider the trade offs14:32
slaweqin some cases it uses fanout=True and then it can't call() be used14:32
ralonsohslaweq, why?14:33
ralonsohactually one problem we have with the MQ is that some calls are blocking14:33
slaweqcall() waits for return value so You can't send it to many hosts and wait for many replies14:35
slaweqthat's at least how I understand it14:35
ralonsohI know, but why do we need to use call instead of cast?14:35
ralonsohif the MQ is down, the server will stop working14:35
slaweqralonsoh: this was "Option 2" in the RFE14:35
ralonsoh(maybe this is out of topic, sorry)14:36
amotokiyeah, switching cast() into call() is not straight-forward. in case of fanout=True, we need to convert it into multiple call() and also need to check the status of individual call().14:36
amotokiusing call() allows us to check the result14:36
amotokibut perhaps it will bring another scaling issue in this case.14:36
stephen-maslaweq: when can an RFE can be brought up for discussion?14:36
slaweqamotoki: yes, but IMO that's not good idea as L2pop was IMO designed to address some scale problems and such change would make it totally not scallable14:37
amotokislaweq: exactly14:37
amotokiI just tried to explain what would happen.14:37
slaweqstephen-ma: are You asking about ? If yes, I keep it for the end of the meeting :)14:37
openstackLaunchpad bug 1861529 in neutron "[RFE] A port's network should be changable" [Wishlist,New]14:37
stephen-mayes that's the RFE14:38
*** jawad_axd has quit IRC14:38
slaweqamotoki: ralonsoh  so IMO to address issue described by Oleg, we should only consider "option 1 - periodic sync mechanism"14:38
amotokislaweq: agree. I personally prefer to the periodic sync.14:38
ralonsoh(I don't, that's why we have the MQ)14:39
ralonsohbut the problem is why the MQ is not reliable14:39
*** jawad_axd has joined #openstack-meeting14:39
ralonsohanyway, if making this update periodic can solve this problem, I'm ok14:39
slaweqralonsoh: problem here is that with cast() method neutron-server don't know if agent configured everything fine14:40
*** jawad_axd has quit IRC14:41
ralonsohI will comment in the bug14:41
ralonsohbut that should not be a server problem14:41
ralonsohif agent is down, server should keep working14:41
amotokiMQ is durable in some cases, but from my operator experience it is not easy to ensure MQ msgs are reliable, so it is nice if neutron (MQ users) provides some mechanism for reliability.14:41
ralonsohif the agent received the config and everything went fine, ok14:42
ralonsohif not, the agent should communicate to the server informing about the error14:42
njohnstonThe other alternative, just to play devil's advocate, is to build the reliability higher up in the stack14:42
mlavallewhich is synch of sorts, right?14:42
ralonsohnjohnston, the reliability is on the services: agent, server, etc14:43
njohnstonralonsoh: If the MQ never sent the message tot he agent then the agent has no idea it has something to complain about14:43
ralonsohthe agent should be smart enough to send a warning message to the server14:43
ralonsohnjohnston, I know14:43
ralonsohand if the MQ does not work, then will have a stopped Neutron server14:43
slaweqralonsoh: exactly how njohnston said, in case of cast() You will not have e.g. message timeout error on server side14:43
njohnstonralonsoh: Instead of depending on the call() method the agent sends a message saying "I processed this update", and neutron server counts these acks.14:44
ralonsoh(something very common in some bugs)14:44
njohnstonSimilarly to how you design a reliable service on top of UDP, you don't have the transmission mechanism ensure reliability, you build it into the application layer14:44
ralonsohexactly: this should be like a UDP call, and the client should inform.....14:44
ralonsohI was writing the same14:45
mlavalleand all of his is a synch mechanism, isn't it?14:45
ralonsohgoal: do NOT block the server14:45
njohnstonthat requires neutron to track what agent(s) should respond to this kind of request and keep an account for responses received14:45
ralonsohin a async way14:46
njohnstonmlavalle: the value here is that the approach is not periodic or timer-based14:46
*** manuvakery has quit IRC14:47
mlavallewhenever to asynch entities need to cooperate (server and agent for example) you need a way to fins synchronization points, periodic or otherwise14:47
*** psachin has quit IRC14:48
mlavallenature of distributed systems14:48
mlavalleI'd say the idea has merit and we should explore it further with a spce14:48
njohnstonMy main question: the work of syncing to the database for FDB updates and then keeping an account of responses received, is it worth the effort?  Compared to the simpler periodic sync mechanism.14:49
ralonsohIMO, you don't need to track the responses14:50
slaweqok, so to sumup what we discussed so far: we should continue discussion in the sync (periodic or not) mechanism in the spec, we don't want to switch from cast() to call(), right?14:50
ralonsohyou'll have error responses or nothing14:50
njohnstonIf you don't track the responses then you can't reissue the update when a response is not received14:50
ralonsoh1) if the agent is not working, the server will notice that, period checks14:50
ralonsoh2) if the agent message didn't work, the agent will reply with an error14:51
ralonsoh3) if the MQ is unreliable.... well, this IS a problem14:51
ralonsohbut not Neutron's problem14:51
njohnstonBut addressing 3 is the point of the RFE, it is Neutron's problem14:52
njohnstonIf AMQP drops the message then neither server nor client will know there was an error14:52
njohnstondrops a casted message to be specific14:52
slaweqI agree with njohnston here - we should do as much as we can to address such case on our side14:52
mlavalleit is Neutron's problem in the sense that is has at least cope with it14:52
slaweqso, I will sum up this discussion in RFE and ask for spec to continue discussion there, right?14:55
mlavalleslaweq: I would point out in the RFE that in light of today's discussion, we lean towards some sort of synch problem14:55
*** ociuhandu has quit IRC14:55
mlavalleand that we would like to explore it in a spec14:55
slaweqmlavalle: sure14:55
mlavalleI meant synch mechanism14:56
slaweqok, thx for discussion about this RFE - it was the good one today :)14:56
slaweqas we are almost on top of the hour, I don't want to start discussion about next rfe14:57
slaweqbut I want to ask all of You to check
openstackLaunchpad bug 1861529 in neutron "[RFE] A port's network should be changable" [Wishlist,New]14:58
slaweqand that's all for today from me14:58
slaweqthx for attending14:58
slaweqand have a great weekend14:58
*** openstack changes topic to "OpenStack Meetings ||"14:58
openstackMeeting ended Fri Feb 14 14:58:46 2020 UTC.  Information about MeetBot at . (v 0.1.4)14:58
openstackMinutes (text):
*** mlavalle has left #openstack-meeting14:58
*** stephen-ma has quit IRC14:59
*** gagehugo has left #openstack-meeting15:03
*** ociuhandu has joined #openstack-meeting15:05
*** rosmaita has left #openstack-meeting15:14
*** apetrich has quit IRC15:20
*** ociuhandu has quit IRC15:31
*** eharney has joined #openstack-meeting15:33
*** maciejjozefczyk has quit IRC15:42
*** armax has joined #openstack-meeting15:53
*** ociuhandu has joined #openstack-meeting16:06
*** TrevorV has quit IRC16:09
*** jlibosva has quit IRC16:10
*** jamesmcarthur has joined #openstack-meeting16:16
*** manuvakery has joined #openstack-meeting16:24
*** rsimai has quit IRC16:37
*** ijw has joined #openstack-meeting16:53
*** lucasagomes has quit IRC16:55
*** ijw has quit IRC16:57
*** ijw has joined #openstack-meeting16:58
*** diablo_rojo has joined #openstack-meeting17:00
*** e0ne has quit IRC17:01
*** yamamoto has quit IRC17:05
*** yamamoto has joined #openstack-meeting17:06
*** yamamoto has quit IRC17:06
*** yamamoto has joined #openstack-meeting17:06
*** lpetrut has quit IRC17:09
*** yamamoto has quit IRC17:11
*** ociuhandu has quit IRC17:16
*** yamamoto has joined #openstack-meeting17:26
*** tesseract has quit IRC17:29
*** gmann is now known as gmann_afk17:42
*** igordc has joined #openstack-meeting17:43
*** eharney has quit IRC17:45
*** jmasud has quit IRC17:45
*** jmasud_ has joined #openstack-meeting17:45
*** jamesmcarthur has quit IRC17:54
*** lajoskatona has quit IRC17:55
*** jamesmcarthur has joined #openstack-meeting17:58
*** eharney has joined #openstack-meeting17:58
*** jamesmcarthur has quit IRC18:07
*** jamesmcarthur has joined #openstack-meeting18:09
*** jamesmcarthur has quit IRC18:25
*** jamesmcarthur has joined #openstack-meeting18:26
*** manuvakery has quit IRC18:27
*** jamesmcarthur has quit IRC18:31
*** eharney has quit IRC18:33
*** jamesmcarthur has joined #openstack-meeting18:36
*** jamesmcarthur has quit IRC18:48
*** ociuhandu has joined #openstack-meeting19:01
*** ijw_ has joined #openstack-meeting19:02
*** cmurphy is now known as cmorpheus19:05
*** ijw has quit IRC19:05
*** ociuhandu has quit IRC19:06
*** ralonsoh has quit IRC19:07
*** e0ne has joined #openstack-meeting19:15
*** eharney has joined #openstack-meeting19:54
*** eharney has quit IRC20:10
*** gmann_afk is now known as gmann20:12
*** igordc has quit IRC20:15
*** e0ne has quit IRC20:41
*** ijw has joined #openstack-meeting20:43
*** Lucas_Gray has joined #openstack-meeting20:43
*** ijw_ has quit IRC20:46
*** eharney has joined #openstack-meeting21:15
*** raildo has quit IRC21:29
*** enriquetaso has quit IRC21:49
*** jmasud_ has quit IRC21:53
*** nicolasbock has quit IRC21:54
*** yamamoto has quit IRC22:01
*** rfolco has quit IRC22:02
*** slaweq has quit IRC22:15
*** rh-jelabarre has quit IRC22:31
*** yamamoto has joined #openstack-meeting22:37
*** ociuhandu has joined #openstack-meeting22:40
*** ociuhandu has quit IRC22:42
*** ociuhandu has joined #openstack-meeting22:43
*** yamamoto has quit IRC22:45
*** ociuhandu has quit IRC22:48
*** slaweq has joined #openstack-meeting23:11
*** slaweq has quit IRC23:15
*** armax has quit IRC23:33
*** armax has joined #openstack-meeting23:33
*** ociuhandu has joined #openstack-meeting23:55

Generated by 2.15.3 by Marius Gedminas - find it at!