15:01:01 <ihrachys> #startmeeting neutron_upgrades
15:01:02 <openstack> Meeting started Mon Sep 12 15:01:01 2016 UTC and is due to finish in 60 minutes.  The chair is ihrachys. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:05 <ihrachys> hi all
15:01:06 <asingh> o/
15:01:06 <openstack> The meeting name has been set to 'neutron_upgrades'
15:01:10 <sindhu> Hi
15:01:16 <ihrachys> #link https://wiki.openstack.org/wiki/Meetings/Neutron-Upgrades-Subteam Agenda
15:01:17 <dasanind_> Hi
15:01:27 <electrocucaracha> o/
15:01:37 <korzen> hello
15:01:52 <electrocucaracha> welcome back korzen
15:02:00 <ihrachys> rossella_: are you with us?
15:02:01 <ihrachys> :)
15:02:18 <ihrachys> #topic Announcements
15:02:40 <ihrachys> not much to announce; we'll have newton-rc1 released this week, after which master will be open for Ocata
15:03:10 <ihrachys> #topic Partial Multinode Grenade
15:03:17 <rossella_> ihrachys, yep
15:03:56 <ihrachys> grenade linuxbridge job is still in experimental, we are basically waiting for stable/newton branch creation, so that we don't need to follow a huge number of devstack backports to fix it
15:04:18 <ihrachys> since master will then gate grenade against stable/newton, we should have all the needed linuxbridge/devstack fixes to proceed
15:04:27 <ihrachys> abregman and jschwarz will take a look
15:05:20 <ihrachys> other than that, I believe jobs are behaving fine. there is more work to do in terms of gate setup definitions (moving services to secondary nodes, checking dvr tests), so if people have cycles, there is work to join.
15:05:27 <ihrachys> #topic Object implementation
15:05:30 <ihrachys> first Newton
15:05:36 <ihrachys> we basically track two things
15:05:52 <ihrachys> 1. expose db_obj on objects: https://review.openstack.org/#/c/348279/
15:06:36 <ihrachys> (plus its dependency https://review.openstack.org/#/c/367268/ )
15:06:58 <ihrachys> 2. not failing on unknown API filters: https://review.openstack.org/#/c/365659/ by electrocucaracha
15:07:13 <ihrachys> for the 1st one, it passes the gate, no negative comments so far.
15:07:42 <ihrachys> for the 2nd one, it's good, though small nits to consider + we now need a bug reported and targeted for RC1 for everything we land
15:07:57 <ihrachys> electrocucaracha: I hope you will get to report the bug and update the patch?
15:07:58 <electrocucaracha> yes, I was planning to create the bug today
15:08:41 <ihrachys> electrocucaracha: I believe next iteration should be ready to go in :)
15:08:47 <manjeets-> ihrachys, for everything you mean every patch ?
15:09:00 <ihrachys> manjeets-: just those that are tracked for Newton
15:09:09 <ihrachys> manjeets-: which are afaik just two those pieces
15:09:17 <manjeets-> ok
15:09:19 <ihrachys> manjeets-: for ocata, we are sticking to blueprint still
15:09:39 <ihrachys> ok, now for Ocata. as I said, next week master will be open
15:09:56 <korzen> I've seen that ocata is going to be very short
15:10:00 <korzen> ending in Feb
15:10:02 <ihrachys> I am hopeful we will be able to land most model relocation patches then.
15:10:09 <ihrachys> korzen: yeah, it's 4.5 months
15:10:28 <ihrachys> which may impact some of our plans.
15:10:36 <electrocucaracha> I included the progress in the spreadsheet
15:10:38 <manjeets-> hopefully we'll make all ovo implementation by then
15:10:39 <electrocucaracha> https://docs.google.com/spreadsheets/d/1FeeQlQITsZSj_wpOXiLbS36dirb_arX0XEWBdFVPMB8/edit#gid=1434170112
15:10:40 <korzen> I'm seeing that rc1 is  Jan 30
15:11:11 <ihrachys> manjeets-: electrocucaracha: anyone from your team already looking at reusing existing objects for database code?
15:11:25 <ihrachys> I mean, things like subnet or port or network or sec group
15:12:06 <manjeets-> you mean implementing those ?
15:12:07 <korzen> subnet is partially covered
15:12:26 <ihrachys> korzen: good, though I believe the patch is still not in shape to pass CI?
15:12:40 <ihrachys> manjeets-: more like using in plugin code what's already merged.
15:12:49 <electrocucaracha> well, I think that it's not reusing, they are creating new ones that depends on those objects or working on the integration
15:12:59 <korzen> ihrachys, it is passing but with hacks
15:13:13 <ihrachys> manjeets-: we have quite some OVO objects that are not hooked into plugin code, and I believe it's of no one benefit to keep it long that way. objects rot.
15:13:30 <korzen> #link https://review.openstack.org/#/c/321001/ get, update and delete converted to Subnet OVO usage
15:13:57 <korzen> #link https://review.openstack.org/#/c/351740/ create subnet converted to OVO
15:13:58 <ihrachys> electrocucaracha: I talk about integration
15:14:27 <manjeets-> yea and currently some have api are not passing context and are passing session directly
15:14:57 <ihrachys> manjeets-: right, those will land as needs arise.
15:15:11 <manjeets-> I thought we can build context on fly before calling object api without modifying the create update api
15:16:09 <ihrachys> manjeets-: ? context contains a lot of valuable info that you cannot just drop, like auth info.
15:16:27 <manjeets-> yes thats the reason we have to modify the api
15:17:02 <ihrachys> not sure it's API though. it's just internal functions. If that's indeed api somewhere, we may need to do something smarter, that would be more compatible.
15:19:05 <manjeets-> ihrachys, https://review.openstack.org/#/c/363206/
15:19:21 <manjeets-> example change
15:19:51 <ihrachys> manjeets-: but is the code used outside the neutron tree?
15:20:00 <asingh> ihrachys while integration, got CORE plugin not found error , https://review.openstack.org/#/c/359379/8/neutron/tests/unit/scheduler/test_dhcp_agent_scheduler.py does it look acceptable?
15:20:34 <manjeets-> ihrachys, I don't know about that at the moment will check if it is used outside tree or not
15:20:48 <ihrachys> manjeets-: looking at codesearch.openstack.org, I don't see any usage outside the tree, it seems more like internal classes.
15:20:57 <manjeets-> so then we are good
15:21:06 <manjeets-> we can change that to context
15:21:38 <ihrachys> asingh: yeah, we need it because objects rely on plugin instance. we may think about moving that into base test case class though because we already have it in quite some places.
15:21:53 <ihrachys> ok. next week we should be ready to restart active implementation. I hope folks will prepare patches in queue for that so that we can land them quicker.
15:22:48 <ihrachys> #topic Other patches on review
15:22:55 * ihrachys looks thru his review queue
15:23:03 <manjeets-> ihrachys, in newton they were delayed because of priority but assuming in ocata they will be higher priority  ??
15:23:30 <ihrachys> manjeets-: yeah, we actually had some discussions with PTL about raising priority, and there are some ideas on how to make it happen.
15:23:45 <ihrachys> manjeets-: it's of no one's benefit to linger foundational features for cycles
15:24:36 <ihrachys> ok, I checked my queue, I don't see any patches of interest for the team.
15:24:57 <ihrachys> let's switch to the last section and discuss anything you may have there
15:25:00 <ihrachys> #topic Open discussion
15:25:24 <ankur-gupta-f1> sindhu: had a question
15:25:37 <manjeets-> ihrachys, i'll rebase networkdns and portdns patch for implementation on top of your ovo port patch this week
15:25:53 <ihrachys> manjeets-: ok, thanks.
15:25:57 <asingh> ihrachys while integration of segment host host mapping, current implementation combines 4 queries , https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L667-#L679 .
15:25:57 <asingh> Final query looks like http://paste.openstack.org/show/572359/
15:26:36 <asingh> ihrachys It includes Subnet, SubnetServiceType and SegmentHostMapping OVO.
15:26:36 <asingh> How should i got about it?
15:27:04 <sindhu> ihrachys:  just a small confusion regarding whether to have the ovo creation and integration in same patch or different patch? (considering its use for push notification)
15:27:41 * electrocucaracha was thinking in the same question as sindhu
15:27:47 <manjeets-> ihrachys, for joins should be issue a get_object on each model and and make a list then filter out based on join conditions ?
15:28:16 <manjeets-> should we** //should be
15:28:42 <ihrachys> asingh: the way I see it, you pick an object that is of more relevance here (subnet?) and add a classmethod there that will do the job. we won't be able to get rid of model interrelationships completely, so the goal is at least to isolate model mess under neutron/objects/.
15:29:20 <ihrachys> sindhu: I would say, case by case. if the object is huge and integration is not obvious, consider split. but otherwise better not to.
15:29:30 <dasanind_> ihrachys: i have  a query with_lockmode update https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/type_vlan.py#L105. Do you have any suggestion how to go about it?
15:29:48 <ihrachys> manjeets-: the problem with using object API to mimic queries is that then you have multiple queries instead of a single one
15:30:00 <ihrachys> and multiple is bad for scaling
15:30:15 <ihrachys> so it's better to stick to query builder.
15:30:33 <manjeets-> yea that is less efficient
15:30:52 <ihrachys> electrocucaracha had some ideas about making synthetic fields work as advanced filters, though it's not obvious how to expose that kind of behaviour in a generic way.
15:32:24 <korzen> I like the idea of per class classmethods
15:32:46 <ihrachys> dasanind_: I admit I don't know much about the lock, but I would imagine we may need to keep it. now, the question is whether we need to expose that kind of behaviour through common objects API, or just keep it the way it is now, just in another place.
15:33:01 <korzen> maybe after few implementation we will have good idea how to implement it in generic way
15:33:46 <manjeets-> ihrachys, if that's the only place it is used there we may not need to expose it through object api
15:33:52 <dasanind_> ihrachys: yeah in this query I am not sure how to introduce the object. It's acquiring lock on the table and updating the table
15:33:55 <ihrachys> + to what korzen said. it's probably easier to start from actual use cases, then seek for commonality, then implement everything we may stumble on in a generic way right away
15:34:04 <electrocucaracha> korzen: I like that strategy for looking patters or things in common
15:34:31 <dasanind_> ihrachys: if i try to separate this I will run into integrity error
15:35:00 <ihrachys> dasanind_: it can be a huge class method on the object that does the same as now, for the time being.
15:35:48 <dasanind_> ihrachys: you mean keep the query same as now??
15:36:27 <electrocucaracha> I think that ihrachys is suggesting to extract those complex queries into class methods
15:36:51 <ihrachys> dasanind_: I would need to look into details. but yeah, that would be my first attempt. do you think we can remodel the query without losing db optimizations?
15:37:18 <dasanind_> ihrachys: will look into that and let you know
15:37:28 <ihrachys> electrocucaracha: right, and place all those dirty things under neutron/objects/ under respective object definitions.
15:37:33 <manjeets-> for now it'll be good electrocucaracha
15:37:59 <dasanind_> electrocucaracha: ok
15:38:19 <ihrachys> we will make sure we tag everything we hate about the code with TODOs :)
15:38:26 <ihrachys> when moving
15:38:27 <electrocucaracha> we can handle the db schema changes from the ovo later which is the main purpose
15:38:41 <dasanind_> ihrachys: sure
15:38:57 <ihrachys> then it's a matter of resources to get it into better shape. I don't want us to spend too much time now solving problems that honestly are not induced by our effort. :)
15:39:24 <electrocucaracha> ihrachys: regarding TODOs, we should consider some time during ocata for complete them
15:39:36 <ihrachys> electrocucaracha: yeah, I have quite some on my plate :)
15:40:16 <ihrachys> electrocucaracha: if it's tech debt, we should not add more; if it's just 'things to implement later', I am fine to land it that way.
15:41:16 <dasanind_> ihrachys: In this patch https://review.openstack.org/#/c/348279 we are exposing the database model for NeutronDbObject instances. Will it not impact the rolling upgrades?
15:42:00 <ihrachys> dasanind_: assuming plugins don't rely on any new nasty things, I don't see it affecting us much.
15:42:10 <ihrachys> dasanind: and btw we detach the model from session just in case
15:42:27 <ihrachys> dasanind: so plugins can't really do much with it except read from its attributes
15:42:40 <ihrachys> dasanind: we will need to switch our plugin API to objects though.
15:42:51 <ihrachys> dasanind: it's just that we are not ready to do it across the board.
15:43:21 <ihrachys> dasanind: and we have no plan implemented yet on how to evolve the current plugins API that relies on models.
15:44:09 <ihrachys> dasanind: so it's a matter of switching all the code to objects, then working on some evolving solution, then taking some more time to deprecate models for plugins API. it's not a single cycle job.
15:44:57 <dasanind_> ihrachys: makes sense.
15:45:24 <dasanind_> ihrachys: these plugin API's have the capability to update the db as well correct?
15:45:55 <ihrachys> dasanind: most current plugin code does pass an attached model, so yeah, in theory they can do it
15:46:31 <ihrachys> dasanind: though it's my belief, and some other people agree, that's unsafe and hence we should be able to break people that try to rely on it
15:47:55 <dasanind_> ihrachys: so with the introduction of OVO they will be able to use the plugin API to read the attributes but update will break
15:47:56 <electrocucaracha> so eventually, we have to create more class methods for plugins
15:48:38 <ihrachys> dasanind: yeah, kinda. I hope we won't need to attach the model. :)
15:48:50 <ihrachys> electrocucaracha: not sure I follow
15:48:58 <ihrachys> electrocucaracha: you mean for very specific queries? yes
15:49:07 <electrocucaracha> ihrachys: exactly
15:49:27 <dasanind_> ihrachys: thank you for the clarification
15:50:45 <ihrachys> ok, I guess it's time to end the meeting :) we'll meet again in a week when we will have the master branch open and unicorns everywhere.
15:50:48 <ihrachys> see ya
15:50:51 <ihrachys> #endmeeting