14:05:10 <ajo> #startmeeting neutron_qos 14:05:11 <openstack> Meeting started Wed Aug 19 14:05:10 2015 UTC and is due to finish in 60 minutes. The chair is ajo. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:05:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:05:15 <openstack> The meeting name has been set to 'neutron_qos' 14:05:17 <ihrachyshka> do it, ping me when there is smth for me, I may be off sometimes 14:05:32 <ajo> ihrachyshka, let's start with an status update, may be you're better for that 14:05:38 <ihrachyshka> ok 14:05:47 <ihrachyshka> #topic where we are 14:05:55 <ihrachyshka> ajo, gimme a co-chain 14:05:58 <ihrachyshka> *chair 14:05:59 <ajo> #topic where we are 14:06:04 <ajo> ihrachyshka how do I do that? :) 14:06:12 <ajo> #help 14:06:18 <ihrachyshka> nah, whatever 14:06:21 <ihrachyshka> so where we are. 14:06:28 <ihrachyshka> we merged into master on Mon 14:06:37 <moshele> hi 14:06:56 <ihrachyshka> meaning, feature/qos is about to retire and hooks cleaned up from project-config: https://review.openstack.org/212475 14:06:57 <jschwarz> #addmeeting ihrachyshka 14:06:59 <jschwarz> ;-) 14:07:05 <jschwarz> sorry, #addchair ihrachyshka 14:07:12 <ajo> #addchair ihrachyshka 14:07:30 <ihrachyshka> now, during merge we disabled gating for qos because master lacked the hooks in project-config 14:07:30 <ajo> ihrachyshka++ :-) 14:07:34 <ajo> everybody++ 14:07:53 <ihrachyshka> so the critical thing is to enable the gating before we introduce a regression 14:07:58 <ajo> NOTE: disabled gating means: no api tests or tempest 14:08:08 <ihrachyshka> first, we tried to make it in devstack: https://review.openstack.org/212453 14:08:11 <ajo> functional, unit, and fullstack may be ok 14:08:21 <ihrachyshka> but folks requested a devstack plugin. so here we go: https://review.openstack.org/214249 14:08:49 <ihrachyshka> I need to enable the qos api tests in that same patch to check it worked as expected. (afaik it did not though) 14:09:02 <ihrachyshka> ajo, how do you know what will affect API? 14:09:25 <ihrachyshka> anyway... gating is the thing we need like yesterday. 14:09:36 <ihrachyshka> so apart from it... 14:09:51 <ihrachyshka> ajo started to report bugs in LP. ajo link to qos tag? 14:10:16 <ajo> ihrachyshka: what do you mean by "what will affect API?" 14:10:18 <ihrachyshka> now if anything buggy qos related in the product, please report a bug and mark with 'qos' tag for ease of searching 14:10:19 * ajo looks for the link 14:10:30 <ajo> #link https://bugs.launchpad.net/neutron/+bugs?field.tag=qos neutron qos bugs 14:10:41 <ihrachyshka> ajo, I mean, that you can't ever be sure changing some code that it won't affect api tests. 14:10:47 <ihrachyshka> thanks ajo 14:11:02 <ajo> ihrachyshka: that's true 14:11:03 <ihrachyshka> I saw people actively fix qos bugs and send reviews. 14:11:30 <ihrachyshka> I suggest all patches to have marked with bp/quantum-qos-api top or whatever relevant 14:11:32 <ajo> ihrachyshka: may be we should hold those until api-tests are re-enabled to avoid later head-aches? 14:11:39 <ihrachyshka> so that it shows up in reviewer dashboard for L 14:11:55 <ajo> sounds like a good idea 14:11:56 <ihrachyshka> ajo, I think we should hold merges before gating, yes. 14:12:11 <ajo> I will set my patches on-hold for that reason 14:12:19 <ihrachyshka> anyone apart from the owner can change topic, so please check your lists 14:12:46 <ihrachyshka> ajo, ok. I hope people will mind. if it's something clearly agent side, I think we are good to merge. 14:12:53 <ihrachyshka> so server side is problematic 14:13:16 <ihrachyshka> ok, what else... ah right. THANKS ALL FOR YOU GREAT WORK 14:13:32 <ihrachyshka> it's a pleasure to work with you folks :) 14:13:46 <ajo> ihrachyshka: makes sense 14:13:47 <ihrachyshka> so ajo I think you can now move to specific discussions 14:13:57 <ihrachyshka> ajo, while I clear some stuff in donwstream 14:13:58 <ajo> so server/db/api side -> let's hold until api-tests are back 14:14:03 <moshele> it was pleasure to work with you guys as well :) 14:14:14 <ajo> yeah :-) 14:14:24 <ajo> ok 14:14:44 <ajo> so, may be we could go over a couple of bugs, and then irenab and I were talking about nova integration with nova flavors 14:15:10 <ajo> #topic Bugs 14:15:34 <ajo> first, this bug, I think it's important: 14:15:40 <ajo> #link https://bugs.launchpad.net/neutron/+bug/1486039 14:15:41 <openstack> Launchpad bug 1486039 in neutron "Setting a policy to a network will limit the router/dhcp/net-device ports, that's not expected" [Undecided,New] 14:15:55 <ajo> basically 14:16:02 <ajo> if we set a rule to "private-net" 14:16:10 <ajo> policy with a bw limit rule, I mean 14:16:19 <ajo> and then you create the router, and the dhcp port is created... 14:16:30 <ajo> both the inner leg of the router will be BW limited, and the dhcp port will be bw limited too 14:16:35 <ajo> which is not expected 14:17:14 <ajo> therea are use cases, where the tenant or admin could want to explicitly limit some of the device ports (router legs, etc...) 14:17:28 <irenab> ajo: would filtering out network types device owners will be ok? 14:17:29 <ajo> so all the agreggated network connection to the outside world is limited, etc... 14:17:37 <jlibosva> ajo: if there is a usecase like kevinbenton pointed out - would it make sense to *not* limit router by default and if needed, they can update port with --qos-policy-id ? 14:17:37 <ajo> irenab: it depends on where we do it 14:17:48 <ajo> jlibosva, irenab , correct 14:17:52 <ajo> *but* 14:18:09 <ajo> (sec) 14:18:13 <ajo> I had a distracting monkey ;) 14:18:20 <ajo> -kid- :) 14:18:31 <ajo> the issue comes when we put other rules in play 14:18:32 <ajo> for example 14:18:47 <ajo> DSCP marking of a private network, could be desired for the inner router leg, or the dhcp leg 14:19:00 <ajo> as... that would also prioritize those packets in the switch 14:19:10 <ajo> (expected if you set a network to have a qos policy with dscp marking) 14:19:16 <ajo> so 14:19:22 <ajo> I think that a possible solution 14:19:44 <ajo> could be to provide a separate qos_profile_id in the get_info_for_devices with the network qos_profile_id 14:19:52 <irenab> sounds like we should have per rule type policy if to apply settings on network ports 14:19:54 <ajo> network_qos_profile_id ? ... 14:20:03 <ajo> and let the rule handler at low level decide 14:20:09 <ajo> or the plugin 14:20:10 <ajo> :) 14:20:31 <ajo> as the rules are network objects... 14:20:45 <ajo> may be we can add an attribute like "apply_to_network_devices" 14:21:00 <ajo> and then that's clear for all the stack up/down 14:21:46 <ajo> it's a funny bug :) 14:21:49 <ajo> lots of corner cases 14:21:50 <irenab> ajo: by attribute you don’t mean API wise? 14:22:04 <ajo> irenab: yes, internal implementation attribute in the neutron object 14:22:10 <ajo> QoSBandwidthLimitRule in this case 14:22:16 <ajo> under neutron.objects.qos 14:22:28 <ajo> not stored in db 14:22:34 <ajo> just a static attribute of the rule 14:23:18 <ajo> ok, let's not spend time on design stuff, as long as we are aware of the corner cases, and we agree on how to handle it.. 14:23:19 <irenab> ajo: So back to basic use case, once compute port is created without policy on network that has associated policy, the port will inherit it? 14:23:37 <ajo> yes 14:23:49 <ajo> that boils down to the other bug 14:23:49 <irenab> ajo: can you share the link to this bug? 14:24:01 <ajo> from the spec, we said: 14:24:12 <ajo> 1) if port has no qos policy, the policy from network is used 14:24:21 <ajo> 2) if port has an specific policy, the network policy is overriden 14:24:31 <ajo> now, the other: 14:24:42 <ajo> #link https://bugs.launchpad.net/neutron/+bug/1486028 network policy changes propagation 14:24:42 <openstack> Launchpad bug 1486028 in neutron "If you attach a QoS policy to a network, agents handling the port networks are not notified" [Undecided,New] 14:25:13 <ajo> if port updates are propagated, I'd also expect that to happen 14:25:40 <ajo> otherwise we'd be violating the eventual consistency with the model 14:25:52 <irenab> ajo: regarding this, I am not sure updatedpolicy should be applied on the existing bound ports 14:26:17 <ajo> irenab, I'm quite convinced that it should be the case, otherwise the current spec is missleading 14:26:38 <ajo> if the model says that a network with a policy will have the ports with no policy controlled by such network policy... 14:26:43 <irenab> ajo: spec update patch is not merged yet :-) 14:26:49 <ajo> the system will be left in an unconsistent state 14:27:00 <ajo> irenab: true, we need a +A ':) 14:28:35 <ajo> any other opinion on network policy updates to agent? 14:28:44 <ajo> ihrachyshka, jlibosva , jschwarz , moshele ? 14:29:13 <jlibosva> I like item 1) in the bug :) We just need to be careful not to update ports with already assigned policy 14:29:24 <jlibosva> in given network 14:29:33 <ajo> yes, that's mandatory :) 14:29:43 <ajo> otherwise the model is again unconsistent with the system :) 14:29:54 <jlibosva> yes, consistency++ 14:30:32 <jschwarz> no opinions here 14:30:35 <ajo> ok 14:30:38 <ajo> let's move on 14:31:21 <ajo> #link https://review.openstack.org/#/c/199112/ qos-spec update 14:31:24 <ajo> we need a +A there 14:31:29 <irenab> ajo: I just think no policy can be on intension as well… 14:31:55 <irenab> but lets move on 14:31:58 <ajo> irenab, then we will have to extend the model to block policies 14:32:06 <ajo> irenab, otherwise at next vm boot... the state will be changed 14:32:25 <ajo> irenab: like "disable port security" -> "disable qos policies" 14:32:33 <ajo> or something like that 14:32:45 <ajo> in that case, I'd advise the tenant not to set a policy on the net, and do it per-port 14:33:23 <ajo> I will ping mestery/armax about the +A 14:33:32 <irenab> ajo: I would prefer we provide ‘idiot proof’ support 14:33:59 <ajo> irenab, but, if you leave policy: None on the port, network_policy = A later 14:34:02 <ajo> and not update the port 14:34:03 <irenab> anyway, as long as its well documented, it ok 14:34:04 <ajo> then... 14:34:14 <ajo> somebody stops/starts the VM... 14:34:20 <ajo> or it's migrated 14:34:25 <ajo> the policy get's applied 14:34:31 <ajo> we should document it well, 14:34:42 <ajo> and if the no-policy case comes up... we find a way to do it 14:35:01 <irenab> ajo: probably you are correct about disable-qos-policy, so its feature for next version :-) 14:35:35 <ajo> :) 14:36:14 <ajo> I guess the other bugs don't need much discussion 14:36:28 <ajo> so probably we can move to the nova integration 14:36:47 <ajo> #topic nova integration via flavors 14:37:00 <ajo> irenab, the stage is yours :) 14:37:20 <irenab> we started etherpad with initial ideas here: https://etherpad.openstack.org/p/qos-nova-neutron 14:37:37 <ajo> #link https://etherpad.openstack.org/p/qos-nova-neutron initial ideas for neutron-nova integration on qos 14:38:13 <irenab> The use case is to associate qos policy with nova flavor, so that tenant can deploy VM and get qos configured without additional commands 14:38:47 <jlibosva> irenab: why it can't be part during port creation for nova instance? It calls neutron api to create port, right? 14:39:19 <irenab> according to some available documentation, there is already the following that supposed to work: 14:39:32 <irenab> nova-manage flavor set_key --name m1.small --key quota:vif_inbound_average --value 10240 14:40:09 <irenab> as we discussed with ajo, it maybe modified to nova-manage flavor set_key --name m1.small --key quota:vif_qos_policy --value <the-policy-id> 14:40:29 <ajo> yes, the other will work for nova network, or for neutron without qos-plugin 14:40:30 <irenab> I am not sure what quota means here 14:40:43 <ajo> yeah, the specific key must be decided by the nova team 14:40:46 <sfinucan> IIRC, that 'vif_qos_policy' only works with libvirt-kvm/qemu 14:40:57 <sfinucan> Would you be removing this feature? 14:41:06 <ajo> sfinucan, not removing, nova-net probably needs it 14:41:16 <ajo> the day nova-net is deprecated: AKA never :) 14:41:34 <irenab> sfinucan: ajo :so this one only for nova-network? 14:41:34 <ajo> vif_qos_policy is what we'd be adding 14:41:52 <sfinucan> ajo: OK, just checking :) 14:41:57 <ajo> sfinucan: I think it works with any tap port coming from a VM with KVM 14:42:04 <irenab> and as jlibosva mentioned it should be sent to neutron port create 14:42:08 <ajo> irenab^ , sorry 14:42:12 <ajo> yes 14:42:23 <ajo> I agree, if we attach a neutron qos policy id to the flavor 14:42:32 <jlibosva> ah, sorry. I asked before I read the etherpad carefully :) 14:42:32 <ajo> then , the ports could be created in such policy, *but* 14:42:41 <ajo> there's an issue 14:42:54 <ajo> I think port creation from nova happens with the tenant credentials 14:42:56 <irenab> ajo: previously you raised a valid point regarding ownership of the policy 14:43:03 <ajo> and qos policies, could be admin, and not shared 14:43:25 <ajo> (for example, not shared because you don't want tenants to attach ports to those specific policies, and raise the flavor limits..) 14:43:27 <ajo> example: 14:43:38 <ajo> flavor mini = 1CPU, 2GB, 10Mbps egress 14:43:47 <ajo> flavor cool = 2CPU, 4GB, 100Mbps egress 14:43:58 <ajo> user creates instance in flavor mini... 14:44:04 <ajo> then ... looks for the port and does 14:44:24 <ajo> neutron port-update <port-id> --qos-policy "100mbps-policy-id" 14:44:30 <ajo> cheating the cloud provider :) 14:44:38 <ajo> so, 14:44:59 <ajo> IMO, one way to prevent that, could be done via RBAC, but we need to explore that 14:45:13 <ajo> or, to let the admin leave the policies as not-shared 14:45:29 <ajo> ad nova does call neutron after port creation, to set the policy, with admin credentials 14:45:49 <ajo> ad->and 14:45:56 <ajo> not sure wether that's possible or not 14:46:04 <ajo> again, in the current state, tenant could go and do 14:46:12 <ajo> neutron port-update <port-id> --no-qos-policy 14:46:27 <irenab> ajo: so basically nova should query neutron for policy owner and either port create or port update should set it 14:46:30 <ajo> I guess we would need to add business logic to disallow port detachment, when it's a non-shared policy 14:46:55 <ajo> irenab: I mean, nova would call neutron with admin credentials, to set the policy 14:47:14 <irenab> ajo: and if the policy is owned by tenant? 14:47:33 <ajo> irenab: then admin can set it too :) 14:47:40 <ajo> admin can do anything 14:47:50 <ajo> well, anything that makes sense... 14:47:55 <ajo> hmmm 14:47:59 <ajo> this comes to another bug, 14:48:03 <ajo> I'm just seeing 14:48:08 <irenab> ajo: this if feature! 14:48:11 <ajo> lol 14:48:12 <ajo> :) 14:48:20 <ajo> feature: tenant-cheating-capability 14:48:20 <ajo> :D 14:48:39 <ajo> in the current state, imagine the CSP uses the network qos_policy_id to enforce a policy to tenants 14:48:56 <ajo> tenants could just go, and... neutron net-update <my-net> --no-qos-policy 14:48:59 <ajo> bye policy! ;) 14:49:47 <ajo> So I guess, we may generally want to prevent a non-shared-policy to be detached by a tenant who does not own such policy 14:50:21 <ajo> does it make sense? 14:50:40 <ajo> I will fill it in the tracker, we could throw it away later, 14:50:45 <ajo> let's go back to nova-neutron integration 14:50:58 <ajo> irenab: how do we do it? , probably we should arrange a meeting with nova guys 14:51:08 <ajo> or ask for a timeslot in one of their meetings? 14:52:03 <irenab> ajo: sounds great. Lets just put more content on the etherpad so we can share prior to discussion. I will try to do it asap 14:52:42 <irenab> ajo: what about nova scheduler guarante for qos? 14:53:20 <ajo> irenab: yes, we should start talking about other rules 14:53:32 <ajo> we had DSCP in the line, anybody from comcast? 14:53:43 <ajo> and we had BandwidthGuarantees 14:53:46 <ajo> probably to start with 14:54:15 <ajo> Bandwidth Guarantees may require cooperation from nova scheduler, when those are not "best effort" 14:54:47 <njohnston> Hi! Yes, would you like me to update that etherpad with some DSCP info? 14:54:52 <ajo> for example, to not plug (SUM(ports, port.min_limit)>interface limit) 14:55:11 <ajo> njohnston, etherpad is for nova-neutron integration only, 14:55:19 <ajo> njohnston, for DSCP, do we have a RFE already? 14:55:25 <ajo> (bug with RFE mark? 14:55:37 <irenab> ajo: this would fail too late, need ‘not schedule if (SUM(ports, port.min_limit)>interface limit)' 14:55:39 <ajo> I know vhoward submited a spec, but not sure about the RFE 14:55:58 <ajo> irenab: correct, we need nova to be aware, and schedule the instance to the right place 14:56:04 <ajo> or.. 14:56:07 <ajo> even the port 14:56:19 <ajo> imagine you have , in sriov 2 PF ports, with 10G each 14:56:33 <ajo> and you are requested to plug 4VFs with 5G min limit 14:56:41 <ajo> if you plug the 4VFs to the first PF... 14:56:44 <ajo> the limit won't be met 14:56:55 <ajo> I mean, the guarantee won't be met 14:57:07 <ajo> so it has to be node-aware and port aware... 14:57:19 <irenab> ajo: agree 14:57:21 <ajo> and I guess, nova has to query neutron and ask about the policy 14:57:27 <ajo> to gather info 14:57:40 <njohnston> ajo: I will check with vhoward and if it isn't there we'll get that in ASAP 14:58:48 <ajo> njohnston, if there's an RFE, please ask him to tag it with the qos tag too 14:59:01 <njohnston> will do 14:59:02 <irenab> ajo: lets see if we can chat with nova guys regarding scheduler 14:59:17 <ajo> njohnston: https://bugs.launchpad.net/neutron/+bug/1468353 14:59:17 <openstack> Launchpad bug 1468353 in neutron "QoS DSCP marking rule support" [Undecided,In progress] - Assigned to James Reeves (james-reeves5546) 14:59:17 <ajo> :-) 14:59:26 <irenab> looks like we need to brainstorm it first 15:00:31 <ajo> irenab: makes sense 15:00:46 <ajo> oops 15:00:50 <ajo> we're at the top of the hour 15:00:58 <ajo> let's release the channel, 15:01:06 <ajo> thanks everybody, keep rocking! ;) 15:01:09 <ajo> #endmeeting