14:03:30 <ajo> #startmeeting neutron_qos 14:03:31 <openstack> Meeting started Wed Jul 15 14:03:30 2015 UTC and is due to finish in 60 minutes. The chair is ajo. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:03:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:03:36 <openstack> The meeting name has been set to 'neutron_qos' 14:03:40 <ajo> #topic status update from the trenches 14:04:35 <ajo> So, we're advancing a bit slow, but it's nice to see more testing coming from all directions 14:04:39 <ajo> and more patched merged. 14:04:54 <ajo> Still, the important ones are taking more time than expected, 14:05:08 <ihrachyshka> yeah, I shout "service plugin!!" 14:05:18 <ihrachyshka> but hopefully, we'll get it today. I'll do everything for that 14:05:28 <ajo> and we just found that using the callbacks to extend introduced a DBDeadlock, we're partly lucky that it was triggered twice in a row 14:06:05 <ajo> digging into mixinless core-resource extension 14:06:19 <ajo> we also found that relying directly on callbacks is a messy mechanism 14:06:43 <moshele> ajo: this is the etherpad for qos sync https://etherpad.openstack.org/p/qos-sync I think echo one should add his tasks so we can keep track on everything 14:06:59 <ajo> and we must provide a better interface, similar to ml2 extension drivers, but more for the general neutron, I talked with mkolesni about it... not to be fully done for neutron now, that needs broader discussion, 14:07:19 <ajo> but he will do something in the middle, which can be consumed by the ML2 extension drivers (with an adaptor) and by other plugins 14:07:26 <irenab> ajo: can you clarify this one? 14:07:29 <ihrachyshka> moshele, ok, I'll update the etherpad. I didn't know about that one 14:07:42 <ajo> #link https://etherpad.openstack.org/p/qos-sync 14:07:46 <ajo> sure irenab , 14:07:50 <ajo> let me explain the thing 14:08:19 <moshele> ihrachyshka: thanks 14:08:22 <ajo> extending a core resource, doing it with louse coupling, requires "something" to handle a few things 14:08:40 <ajo> 1) extending the core resource attributes (what we already do with AFTER_READ) 14:09:02 <irenab> someting = plugin that handles the esource extenision, right 14:09:22 <irenab> resource 14:09:22 <ajo> 2) tracking core resource updates to keep track of such extended atrtibutes in DB 14:09:37 <ajo> in this case , the bindings to policies of networks and ports 14:09:41 <ajo> yes, in this case 14:09:45 <ajo> resource = networks / ports 14:09:59 <ajo> and something (right now) is the service plugin 14:10:08 <ajo> after looking at... 14:10:14 <ajo> the ml2 extension drivers, let me link to int 14:10:16 <ajo> int->it 14:11:09 <ajo> #link https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/driver_api.py#L893 14:11:15 <ajo> after looking to that, 14:11:18 <ajo> an example is this: 14:11:31 <ajo> https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/extensions/port_security.py 14:11:38 <ajo> they're basically doing that, but only within ml2 14:11:38 <gcossu> nice link 14:12:03 <irenab> ajo: what I am missing is what is not already handled by current implementation 14:12:11 <ajo> basically, they came to almost the same interface we're finally exposing in the service, via callbacks 14:12:26 <irenab> We skip ML2 extension, since resource is extended by qos plugin 14:13:03 <irenab> ajo: so what you want is to add Resource extension interface? 14:13:04 <irenab> to be implemented by QoS plugin? 14:13:11 <ajo> mkolesni is thinking about generalizing the resource extension, and decouple that from the plugin 14:13:22 <ajo> for now, he's going to provide that in the same file, 14:13:29 <ajo> and make it available to all plugins 14:13:41 <ajo> so you have a common class to call from your plugin 14:13:48 <ajo> to update_network/port create_network/port 14:13:53 <ajo> and get the bindings done/undone 14:14:04 <ajo> and also, put the dict extension in that class 14:14:26 <irenab> ajo: the rational to eanble ML2 to call QoS plugin via this interface? 14:14:49 <ajo> irenab: create a ml2 extension which will call this class 14:14:59 <ajo> and other plugins, can call this class 14:15:04 <ajo> at a later time, in neutron 14:15:12 <ajo> we will need to find the right way for plugins not needing to do such thing 14:15:16 <ajo> (manually) 14:15:37 <ajo> just say "I support this resource extension", yey... 14:15:37 <irenab> ajo: I think you jump into technical details before explaining what you want to solve :-) 14:16:15 <ajo> irenab, it's mike idea's actually, I was refusing to listen this morning, but he's so convincing... 14:16:20 <ajo> and I believe he's right 14:16:38 <moshele> I feel we need another code sprint :) 14:16:39 <ihrachyshka> ajo, why isn't he on the meeting? 14:16:42 <irenab> What is missing with current callback AFTER_READ? 14:16:47 <ihrachyshka> he should have convinced all of us! 14:16:52 <ajo> he just wants to move all the resource extension/extension tracking out of the class 14:16:58 <ajo> into a separate one 14:17:06 <ajo> I think that's neat, and ends in a cleaner plugin 14:17:17 <ajo> and it's something we can reuse neutron wide later 14:17:26 <irenab> ajo: the change that was done for ML2Plugin is absolutly no-go 14:17:28 <ajo> but one step at a time 14:17:35 <ajo> irenab: correct 14:17:43 <irenab> so I assumed we trying to resolve this 14:17:49 <ajo> correct 14:17:55 <ajo> this, and more :) 14:18:07 <ajo> as we solve this, we want to clean up the way we extend resources 14:18:10 <ihrachyshka> ajo, can we take more of this and less of more? 14:18:15 <irenab> lso to resolve this (without more) what is missing? 14:18:35 <ihrachyshka> irenab, I like your 'result first' approach 14:18:44 <irenab> ihrachyshka: :-) 14:18:58 <irenab> more of: lets keep focused 14:19:13 <ihrachyshka> I'm fine with good design as long as we stick to schedule (that we don't have, and I blame our Lord Miguel for not guiding us!) 14:19:15 <ajo> I agree, I couldn't convince mkolesni to do less of more, but I convinced him to do something very thin 14:19:57 <irenab> ajo: ca you share the plan, I still do not understand what was missing so it required to “polute” ML2 plugin 14:20:10 <ajo> irenab, you will see it in next patch. 14:20:24 <ihrachyshka> yes, please make communication explicit. you talk via phone, you have chats in redhat irc... 14:20:26 <ajo> basically it's putting the relevant parts of the ml2 extension driver available in qos_plugin 14:20:31 <ihrachyshka> and then it's not clear what's going on 14:20:37 <ajo> he's just going to move functions, not reinvent the world 14:20:38 <irenab> ajo: can you still explain the problem, will see the solution in the patch :-) 14:21:17 <ajo> the problem is, we planned to use callbacks for all this, 14:21:27 <ajo> and callbacks are not suitable for this as they are today 14:21:34 <ajo> only the AFTER_READ part is ok 14:21:41 <ajo> and still we will move that in a separate interface 14:21:52 <ajo> tracking the resources via AFTER_UPDATE AFTER_CREATE for ports 14:22:08 <ajo> does not work, because it's too tied to ml2 (those callbacks are only called from ml2) 14:22:11 <ajo> well 14:22:19 <ajo> some of them, some other don't exist 14:22:21 <ajo> since 14:22:39 <ajo> the plan was to use callbacks temporarily, until we had the definitive solution from armax to "resource extension" 14:22:55 <irenab> ajo: initially we discussed that qos extension has two parts : policy management and port/net association. 14:23:07 <ajo> correct 14:23:12 <ajo> is the port/net association 14:23:21 <ajo> and dissociation 14:23:24 <ajo> the pain point 14:23:38 <irenab> if we split this in two extensions and make ML2 extension for second, this will be the ML2 way 14:24:04 <ajo> irenab: I don't follow 14:24:20 <ajo> we want something that works for all , not ml2 only 14:25:11 <gsagie> so the call backs are called from the db layer? 14:25:14 <irenab> ajo: I agree, but this looks like complicating the flow 14:25:33 <ajo> it's harder to discuss than to implement 14:25:42 <ajo> basically it's the same we have scinded to a separate class 14:26:04 <ihrachyshka> irenab, ok, the problem seems to be clear; let's now give them one day to see the solution 14:26:04 <ajo> just a few methods moved out to another class, not inside the plugin itself all together 14:26:20 <ihrachyshka> ajo, will it be up for review tomorrow? 14:26:24 <ajo> yeah, I will push mkolesni to show us his work by tomorrow 14:26:33 <ihrachyshka> good, push is the right word 14:26:54 <irenab> ihrachyshka: +1, we can always fallback for ML2 Extension of net/port qos association to call into Qo plugin 14:27:13 <ihrachyshka> irenab, yeah, as long as we don't start to fallback one day before merge :) 14:27:26 <ajo> ihrachyshka, that's my point 14:27:29 <ihrachyshka> that's why we need to move quickly and get solution up tomorrow 14:27:34 <ajo> I don't want to change directions 14:27:38 <ihrachyshka> ajo, when do you leave for PTO? 14:27:54 <ajo> next week I'm out, but I'll dedicate 30m-1h daily 14:28:02 <ajo> for mail, and reviews 14:28:04 <irenab> ajo: sometimes better change than keep digging :-) 14:28:07 <ajo> and probably irc 14:28:34 <ihrachyshka> ajo, good. we'll try to limit pings 14:28:36 <ajo> irenab, I have limited the dig depth, if I left mkolesni alone he'd rewrite the whole callbacks :D :P :) 14:28:50 <ihrachyshka> ajo++ for staying online on PTO for emergencies 14:29:05 <ihrachyshka> ajo, yeah, those Java guys... 14:29:08 <ajo> irenab, if we find it getting too complicated for some other reasons, I agree, falling back to a simple ml2 extension would be good 14:29:27 <ajo> and then next cycle can be used for doing something more general 14:29:36 <gsagie> its important to see how a plugin adds this support 14:29:47 <irenab> ajo: fine, this mwill require spliiting qos extension into two, but generally simple 14:30:05 <ajo> irenab, splitting in two? 14:30:22 <ajo> gsagie, yes we will need to document that 14:30:29 <irenab> extensions/qos.py to split 14:30:42 <ajo> irenab: I don't follow :) 14:30:58 <irenab> ajo: we can take it offline in case will be needed 14:31:06 <ihrachyshka> can we move forward with the topic? I think we are stuck on that Mike's patch, and we have ETA and plan for it already. And yeah, the way we will rewrite stuff when in master. 14:31:22 <irenab> I just mentioned the code change that will be required to make ML2 extension fallback 14:31:47 <ihrachyshka> irenab, you can try to describe the idea in email maybe. I also don't follow. 14:32:02 <irenab> ihrachyshka: fine, will do it 14:32:12 <ihrachyshka> any other high prio patches? 14:32:28 <ihrachyshka> if not, I vote for rules inside policies one from me: https://review.openstack.org/200608 14:32:41 <ihrachyshka> ajo, eagarly waiting for review ;) 14:32:55 <irenab> ajo: want to update on cli apprach? 14:33:07 <ihrachyshka> ajo, once we will have +2 from you, it should be easier to get another one 14:33:23 <ajo> yes, that patch is important 14:33:25 <ajo> please review 14:33:34 * ajo <- please, review! 14:33:41 <ajo> about CLI 14:34:07 <ajo> we found a stopper for using a common qos-create-rule / qos-update-rule 14:34:18 <ajo> if we provide <type> after that, 14:34:34 <ajo> we have not found a sane way to provide different arguments per rule in neutron-client 14:34:35 <ajo> so 14:34:42 <ajo> the call is to have a separate create/update per rule 14:34:46 <ajo> so we may have 14:34:52 <ajo> qos-create-bandwidth-limit-rule 14:34:57 <ajo> qos-update-bandwidth-limit-rule 14:35:17 <ihrachyshka> ajo, is it just for help message sake? 14:35:24 <ajo> qos-{create, update}-<type>-rule for every rule type 14:35:28 <ajo> ihrachyshka, also for argument parsing 14:35:35 <ihrachyshka> ajo, because afaik passing random --options should still work 14:35:48 <ajo> we don't want to accept a ton of optional arguments, where some are not really optional for some rule types 14:35:59 <gcossu> ihrachyshka: I'll try to review https://review.openstack.org/200608 14:36:09 <ihrachyshka> gcossu, thanks 14:36:27 <ihrachyshka> for cli, I'm fine and don't mind. any new rule type will require a patch for neutronclient anyway. 14:36:34 <ajo> yes 14:36:36 <ajo> that's correct 14:36:44 <ihrachyshka> and if the approach simplifies things, I'm good 14:36:48 <ajo> and every rule type has a different URL path 14:37:03 <ajo> it will be weird, btw, that... 14:37:21 <ajo> qos-rule-delete, qos-rule-show, will need to specify rule type 14:37:36 <Vichoward> if it simplifies i'm okay with it also 14:37:46 <ihrachyshka> ajo, ouch, that's a bit weird indeed 14:37:49 <irenab> ajo: no, no need for rule type 14:37:53 <ihrachyshka> ideally, id is enough 14:38:04 <ajo> irenab, how then do you point it to the right api url? 14:38:23 <ajo> qos/policy/<policy-uid>/<rule-type>/<rule-uuid> ? 14:38:36 <irenab> ajo: hope vikram can clarify 14:38:54 <irenab> maybe you right, lets see 14:39:18 <ajo> I've seen that other services (lbaasv2) solves that by exposing "rules" at a lower level 14:39:28 <ajo> they have that in monitor or heartbeats or something like that 14:39:31 <ajo> so you can just do 14:39:44 <ajo> qos/rule/<uuid> for some operations 14:40:11 <ajo> but... 14:40:20 <ihrachyshka> yeah, that simplifies cli but makes API inconsistent a bit 14:40:25 <ajo> correct 14:40:34 <ajo> we can just do a first version, and refine over time 14:40:37 <ajo> we're experimental 14:40:39 <ajo> so it should be ok 14:40:52 <ajo> once done, we may make the odds exposed and discussed IMHO 14:41:21 <ihrachyshka> ok, let's do the simple form even if it requires --type 14:41:30 <ajo> if having a less beautiful API makes CLI people happier, I'm ok with that 14:41:37 <ajo> other option, could be ... being able to discover the rule type 14:41:44 <ajo> by looking at the policy 14:42:06 <ajo> so we , in two calls, read the policy, ... find what's the category of the rule, then... kill the rule 14:42:17 <ajo> or then show the rule 14:42:25 <ajo> and then we don't need to provide type 14:42:39 <ajo> we already do a lookup when somebody passes the policy name, instead of the policy id 14:42:51 <ajo> irenab, vikram , thoughts? 14:42:58 <ajo> d 14:43:07 <ajo> devil is in the details.. 14:43:21 <gcossu> :) 14:43:23 <ihrachyshka> let's have simple thing first. then go into UX 14:43:30 <ajo> +1 14:43:42 <irenab> for me it looks doable, but I have very limited knowledge on cli implementation 14:44:12 <ajo> I'd then just stick to providing the type for now, may be later we can make it optional by auto-discovert 14:44:16 <ajo> discovery 14:44:21 <ihrachyshka> there is another thing that is overdue: devref that we said before will be somehow ready for Wed: https://review.openstack.org/201536 and followups 14:44:33 <ihrachyshka> ajo, what's the plan here? 14:45:18 * ajo plans to duplicate himself :) 14:45:24 <ihrachyshka> I know it seems like it's not a big deal since it's not code, but I believe it's a must for merge-back, and the earlier we start to document stuff, the better 14:45:34 <ajo> yes 14:45:37 <ihrachyshka> ajo2 = copy.copy(ajo) 14:45:45 <ajo> it's a very expensive operation 14:45:49 <ihrachyshka> ajo, do you need any help with that? 14:45:58 <ajo> last two copies still taking too many resources, instead of producing 14:46:01 <ihrachyshka> ajo, you're not that huge 14:46:17 <ajo> ':D 14:46:22 <ihrachyshka> so, any help? 14:46:40 <ihrachyshka> I think I'm better in writing than coding, so 14:46:55 <ajo> if you can address current comments, and join to your other patch 14:47:03 <ajo> I think gsagie was doing something too 14:47:04 <ajo> I'd really thank you 14:47:13 <ajo> while I keep tackling the plugin towards merge... 14:47:16 <ihrachyshka> ok 14:47:23 <ajo> thanks -as usual- 14:47:46 <ihrachyshka> any more critical pieces? 14:47:54 <ajo> gsagie, did you have time to look into the devref of your pieces? 14:48:13 <ajo> ihrachyshka, I will respond to comments 14:48:22 <ajo> so you can update the parts I'm more aware correctly 14:48:51 * ajo looks again for the list of patches.. 14:49:09 <moshele> ajo: for full flow with ovs agent we need this 2 to merge https://review.openstack.org/#/c/201975/ https://review.openstack.org/#/c/201583/ 14:49:09 <gsagie> ajo: not yet 14:49:50 <ajo> ahm nice, I wil check your update moshele , thanks 14:49:55 <ihrachyshka> moshele, I see the 1st one -1's on jenkins 14:50:05 <moshele> ajo: my patch is failing becuase a bug in the ovs driver which gsagie fixed https://review.openstack.org/#/c/201583/ 14:50:20 <ihrachyshka> ah, there is order involved 14:50:31 <ajo> ah, ok 14:50:39 <ajo> let's make a comment on the other 14:50:55 <ihrachyshka> moshele, can you rebase on gsagie's? 14:51:07 <ihrachyshka> moshele, I've sent it to gate, but it will buy us several hours 14:51:26 <ajo> moshele, I can rebase it if you want 14:51:38 <moshele> ajo: yes please 14:51:43 <ajo> ack, I will 14:51:45 <ihrachyshka> ajo, make sure not to rebase the Gal's one :) 14:51:56 <ajo> yep 14:52:22 <ajo> ok, any more critical pieces ? 14:52:39 <ihrachyshka> ajo, we may also +A https://review.openstack.org/#/c/202119/ 14:52:48 <ihrachyshka> but that's not critical :) 14:53:05 <ihrachyshka> though I will be more happy to see less patches in the queue :) 14:53:20 <ajo> done 14:53:30 <ajo> ihrachyshka, should we wait for jenkins before +A ? 14:53:32 <ihrachyshka> ok, I think we can go back to trenches? 14:53:36 <ihrachyshka> ajo, I think no 14:53:42 <ajo> ok, I never got the logic of that 14:53:50 <ihrachyshka> ajo, at least that's what I see from what zuul does 14:53:52 <moshele> ajo: get_info rpc will return version object or dict in the current implementation (I know it should be object at the end)? 14:54:01 <ajo> ihrachyshka, may be it was necessary back in time 14:54:18 <ajo> moshele, it should be object in the end, but the current dict usage may work 14:54:37 <ajo> ihrachyshka, we made them dict-compatible for now, right? 14:55:22 <ihrachyshka> ajo, well... they are similar, but not identical to dict. you can call .to_dict() to get the real one 14:55:35 <ihrachyshka> though I'm not sure where we are in terms of serialization 14:55:53 <moshele> ajo: for know for the server-> agent flow to work we need dict 14:56:15 <moshele> know -.> flow 14:57:21 <moshele> ihrachyshka: I think we need to convert to dict in registry.get_info 14:57:38 <moshele> ihrachyshka: just for now to make it work 14:58:00 <ihrachyshka> moshele, I'm fine either way. we'll get objects later. 14:58:12 <moshele> ok 14:58:13 <ihrachyshka> do whatever you need to integrate agent side :) 14:58:22 <ajo> moshele, you can access the objects as dicts 14:58:24 <ajo> they will work 14:58:42 <ajo> just for sending over wire you may need to use the functions I pointed this morning 14:59:28 <ajo> ok https://review.openstack.org/#/c/201975/ rebased 14:59:30 <ajo> #endmeeting